benmichael/gods-eye-view
1
0 Fork 0
mirror of https://github.com/bilawalsidhu/gods-eye-view.git synced 2026-11-04 02:14:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into fix/363-geocode-proxy

Browse Source 
# Conflicts:
#	CHANGELOG.md
This commit is contained in:
Niccolò Saggese committed 2026-09-29 19:16:15 +02:00
2 parents 771c9e5632 e7707d9a0f
commit 5083a74413
542 files changed
+96845 -3034
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.env.example
+10
View File
@@ -129,6 +129,13 @@ AISSTREAM_API_KEY=
# 0 disables the silence watch entirely.
# AISSTREAM_SILENCE_TIMEOUT_MS=120000
# Optional: local ADS-B decoder feeds for the Local ADS-B layer. SERVER-SIDE
# ONLY. Comma-separated band=url entries; band is 1090 or 978 (UAT). Each host
# must be loopback, a private (RFC1918) address, localhost or a *.local name;
# the path must end in aircraft.json. Invalid entries are logged and skipped.
# See docs/LOCAL-RECEIVERS.md.
# LOCAL_RECEIVER_FEEDS=1090=http://localhost:8080/data/aircraft.json,978=http://localhost:8978/data/aircraft.json
# Optional: TomTom live traffic flow (BYOK freemium — the free tier is 200,000
# Traffic Flow & Incidents tile requests per MONTH, not per day; no credit card
# required. Get a key at https://developer.tomtom.com). SERVER-SIDE ONLY:
@@ -176,3 +183,6 @@ AISSTREAM_API_KEY=
# CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs
# CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry
# CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured
# Optional: comma-separated Overpass URLs; use only instances you run or pay for.
OVERPASS_UPSTREAMS=
.github/workflows/ci.yml
+5
View File
@@ -29,6 +29,7 @@ jobs:
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
persist-credentials: false
fetch-depth: 0
- name: Use Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
@@ -48,6 +49,10 @@ jobs:
- name: Check package boundaries
run: npm run check:boundaries
- name: Check published layer-token allocations
if: github.event_name == 'pull_request'
run: npm run layer-token:check -- --base-ref origin/main
- name: Run unit tests
run: npm test
CHANGELOG.md
+374 -4
View File
@@ -1,5 +1,354 @@
# Changelog
- Public Overpass instances are no longer used by default. Street Traffic
roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen
on the layer row (TomTom / OSM / Hybrid) or with `?trafficRoads=`. With a
TomTom key the default is Hybrid: TomTom roads with live flow, plus
OpenFreeMap roads TomTom does not cover, simulated. Without a key every
choice draws OpenFreeMap roads. TomTom mode shows only roads with flow; OSM
mode matches TomTom flow onto OpenFreeMap roads for congestion colors,
speeds and closures, and unmatched roads stay simulated. Road failures name
OpenFreeMap and retain HTTP status or timeout reasons separately from TomTom.
Flow coverage counts
rendered dots on roads with flow; mapped military areas come from OpenFreeMap and
follow settled camera changes with cached tiles and retained site entities.
Superseded requests preserve the next view’s pending load.
Tile and API transports can be injected independently; the optional Overpass
ALPR adapter remains available.
OSM-derived layers share one attribution entry and a persistent inline map credit,
including routing, Warendorf cameras, datacenters, dams and the Nepal locator.
Military areas use bundled Overture/OSM names; wide views show bounded,
decluttered named points that follow camera motion while Contacts holds a subject,
using the shared world-overlay host for wide-point and polygon titles. Polygons appear while names load; cancelled views
cannot publish late names, and stalled name downloads can retry. Merged groups
keep their identity and parent name when a pan hides the largest member. Late
names reconcile ids across zooms while retaining selection and marker ownership. Wide
named points remain visible until matching close polygons take over. Installation
titles share datacenter/dam card arbitration and keep incumbents while panning.
Selecting an installation replaces its label with one card; deselection restores it.
Selected footprints receive a translucent fill draped on the active map surface.
ALPR cameras come from an hourly OpenStreetMap extract (US and Canada). Features
without a replacement say they are unavailable; area and footprint
annotations check once whether an Overpass instance is configured and skip
the query when none is. Area annotations outline countries, states and provinces
(Natural Earth, including UK constituent countries) and US counties (US Census Bureau) from bundled data, with no
lookup. County names are disambiguated across countries by aliases, qualifiers
and geography. `OVERPASS_UPSTREAMS` sets an
Overpass instance you run or pay for. The cockpit regional brief resolves
regions from bundled Natural Earth data instead of Nominatim. Traffic starts on enable and paints
locally grounded roads incrementally, shares concurrent tile requests and
reuses validated heights across pans. Camera moves and arriving tiles retain
existing traffic dots on their roads; budget changes fade in/out incrementally,
height refinements ease vertically, and live flow changes color and speed in
place. The traffic footprint follows the reticle square at oblique angles, with
cached detailed near tiles and coarse distant roads. Failed detail tiles retain
coarse roads and retry at most three times. Road-pass deadlines also cancel
queued tile paints and terrain callbacks;
off-screen height corrections cannot hold loading open, and visible corrections
stay pixel-limited at canvas and camera-plane boundaries. Completed road snapshots replace
streamed previews without duplicating roads. Unresolved previews no longer delay or suppress detailed street roads.
Road admission excludes private, parking, walking and unknown traffic categories;
OpenStreetMap service ways and tunnels no longer receive surface vehicles.
Detail tiles load alongside previews, arrival bypasses the movement debounce,
and geographically validated rendered depth avoids repeated mesh redraws.
Traffic releases surface-frame listeners on disable/destroy and reuses unchanged
surface identities without frame-by-frame allocations.
Grounded dots respect building occlusion instead of drawing through roofs,
and the traffic status chip reports completion without a delayed text animation.
Short road chunks and local surface elevation keep the budget on visible
streets in elevated cities. Congestion and closures respect travel
direction. ALPR distinguishes unsupported coverage from empty results and
asks for zoom-in before exceeding its tile budget. Installation footprints
retain visible fragments without joining separate parcels across zooms or
across quantization gaps between different mapped identities. Keyless
terrain tiles retry HTTP 429 and transient gateway failures with bounded,
shared backoff. Operator-configured Overpass area and footprint queries
retain relation member geometry so their outlines remain available.
Contacts filters mapped installations to a true 100 km surface radius around
the current tracked subject,
including in Cockpit, instead of "?" or nothing. Search arrivals frame against the resolved
ground and lift the eye above the rendered surface, so Camp Mabry and Denver
no longer land underground (Milan Khanal, #118). Arrival corrections yield to
new camera owners and are cancelled on layer/app teardown. ALPR floor preparation
and marker placement share terrain requests, cancelled when the layer is disabled.
Map-source changes reposition every ALPR marker without replacing entities. ALPR loads
whole-city views from z9-z12 tiles, drops native ground wedges and clamping,
keeps frame time with ALPR on within a few percent of ALPR off, seats nearby
badges on the rendered surface and says when no loaded camera is on screen.
- CCTV cameras whose bearing is a guess now say so. Packs mark bearings derived
from a hash of the camera id as `headingConfidence: 'low'`, but nothing read the
flag, so roughly 70% of a default catalog rendered like surveyed facings. The HUD
now reads `HDG n° (ESTIMATED)` and the coverage wireframe draws dashed; manual
calibrations and curated poses are never marked estimated (bassem chagra, #643).
- Report which upstream declined a Street Traffic road load. The layer row now
reads `Overpass rate-limited`, `Overpass timed out`, or
`Overpass refused the road query (HTTP 406)` instead of a general
"Road data temporarily unavailable", so a reader is not sent to check a
TomTom key when the public OpenStreetMap mirrors are the side that failed.
The proxy's own 502 (every mirror unreachable) and 503 (local limiter busy)
read `Overpass mirrors unreachable` and `Overpass temporarily unavailable`.
Failures the layer cannot classify keep the general line (daikaginza, #665).
- Bound the client terrain-height cache at 20 000 entries with least-recently-used eviction, so a long session no longer retains every coordinate it ever resolved. Consumer reads promote their entry and a batch still reports every point it resolved (Pedro Lobato, #594).
- Release CCTV media streams whose upstream falls silent after answering. The
15-second media deadline covered only the wait for response headers, so a
camera that replied and then stopped sending held both the proxy connection
and its upstream socket open for as long as the camera host allowed; the
declared `Content-Length` ceiling was the only body bound, and a chunked or
length-less body had none. A 30-second idle deadline now bounds the gap
between upstream chunks and releases a body that has gone silent. It is
rescheduled while the response is still waiting to drain, so a viewer on a
slow link is not mistaken for a dead camera; live feeds are unaffected
(Ethan Stoner, #688).
- Aircraft track backfill (`/api/opensky-track`, `/api/adsblol/trace`) now
answers 502 when the upstream body exceeds the 5 MB cap, instead of a 200
whose error body the client read as an empty track. The failure is cached
like other upstream errors, so retries inside the 60 s window do not spend
OpenSky credits (Raushankumar0720, #720, #722).
- Saving a key from Provider Settings works again on Macs where Nix or
Homebrew coreutils sit ahead of `/bin` on `PATH`. The credential hardener
now spawns Apple's `/bin/chmod -N` by absolute path; GNU `chmod` has no `-N`,
so the ACL strip failed closed and every save was refused (Arthur Bogaart, #694).
- Render on iPad and iPhone instead of stopping with "An error occurred while
rendering." Cesium's per-vertex model atmosphere binds shader `out`
parameters directly to varyings, which Apple's Metal/ANGLE backend cannot
link, so the program failed and the render loop was torn down. The stage is
now kept out of the pipeline on affected devices by clearing
`scene.fog.renderable`, which leaves `fog.enabled` — and the fog density that
drives 3D Tiles refinement — untouched. Detection is a WebGL2 link probe of
the same pattern, so a future driver fix restores the effect with no code
change, with iOS/iPadOS detection as a backstop. Sky atmosphere and the
ground-atmosphere fragment path route through locals and are unaffected;
affected devices lose distance fog on 3D tiles and on the globe basemaps.
`src/app/atmosphereCompat.test.mjs` pins the probe, the platform matrix and
the `renderable`-not-`enabled` choice (KnottyDyes, #705).
- Make share-link layer tokens durable allocations instead of ad hoc picks.
Existing one-character mappings are pinned permanently, while new layers use
the next free single-character digit, then two-character base-36 tokens.
Authors record each token in the permanent JSON ledger after running
`npm run layer-token:next -- <layer-id>` on a rebased branch.
`npm run layer-token:check -- --base-ref origin/main` guards published
assignments and allocation order in
pull-request CI. Existing v2 links keep their exact meaning.
Malformed enabled-layer lists now reject the whole layer payload instead of
restoring a partial list. A QA-only browser fixture exercises the future
two-character path after a fixture-only exhausted-digit scenario, without
reserving `00` in production.
- Region scopes in voice analyst queries ("in the Gulf of Mexico", "over
the Alps") work again in the dev server: the bundled Natural Earth and
neighborhood packs are fetched as JSON in the browser
(`src/data/bundledJson.js`). When a region is not in the bundled packs, the
geocode and admin-boundary fallback answers `region-timeout` after 3 s
instead of holding the reply.
- Transit and Directions rows repaint as soon as their data lands again:
`refreshLayerStats()` now lives on the layer lifecycle, not only on the
compatibility facade. `scripts/qa-radio.mjs` uses it instead of a private
panel method.
- On phones the title bar sits 16 px from the top so both Radio broadcast
waves stay on-screen.
- Keep Cyber right-rail panels mutually exclusive and Display, CCTV and Context
headers and frames fixed during content scrolling. Restore Radio's nested Context placement and compact
player. Add Cyber Sonar voice controls with settings and effect-state readback.
Keep keyboard-focus outlines visible inside Cyber's clipped map and cockpit
expand/collapse buttons, with a matching red hover border.
- Add the opt-in Cyber HUD layout with coordinated map and cockpit panel
styling. Display exposes Sonar on/off, rings, range, power, opacity and sector.
Native point, billboard and label highlighting uses GPU draw commands; there
is no scene-dimming effect selector. Unsupported shaders retain native contact
rendering, and leaving Cyber restores the standard shell and contact treatment.
- Add a **Recent Imagery** data layer (NASA GIBS · HLS + VIIRS, keyless).
Select a box (drag, the current view, or around a pin; up to 1,000 km a
side) and the right-rail panel lists the last 30 days of Sentinel-2 /
Landsat (30 m) and, when switched on, VIIRS daily overview imagery over it,
with thumbnails and scene cloud. Three modes: IMAGE shows one day, VS
BASEMAP swipes it against the map, A / B swipes two days; a SHOW or A / B
chip on each day card pins it, arrow keys preview the focused day while a
slot is empty, and no control moves when anything changes. While imagery is
shown on Google 3D the map switches to Esri and comes back when it is
cleared. Either image exports as a PNG; box, pins, mode and split travel in
share links; the NASA acknowledgement is in the credits and
`DATA_SOURCES.md`. The swipe is now shared with the Nepal scene
(`src/ui/imagerySplit.js`, `src/maps/imageryComparison.js`), and
`MapSourceController.subscribe()` reports every settled map switch.
## Unreleased — local receiver feeds
- The Local ADS-B layer also reads local 1090 MHz and 978 MHz UAT decoder
feeds: the `aircraft.json` that dump1090-fa, readsb, tar1090 or skyaware978
serves. Configure them server-side with `LOCAL_RECEIVER_FEEDS`
(`band=url`, comma-separated); there is no feed editing in the browser.
Hosts must be loopback, RFC1918, `localhost` or `*.local`, the scheme http or
https and the path must end in `aircraft.json`; any other entry is logged at
startup, reported `invalid` and never fetched.
- Add `GET /api/local-receivers/aircraft`. It reads every configured feed in
parallel (2 s timeout, redirects refused, 2 MB body cap, about 1 s of shared
cache) and reports each feed `live`, `stale` (its own `now` is over 10 s old),
`unreachable` or `invalid`. Unconfigured, it answers
`{ configured: false }` and fetches nothing. Upstream error text is never
returned.
- The layer merges browser-SDR and feed aircraft by ICAO, keeping the newest
position, polls the route every second only while it is enabled, and remembers
which bands and sources heard each aircraft in the last 60 s. The click card
names them (for example "Heard by your receiver · 978 MHz UAT · decoder
feed"); aircraft heard only on 978 MHz carry a thin ring. The row status
covers both inputs ("2 feeds live · 14 heard", "feed 978 unreachable"), and
the Local RTL-SDR card shows a read-only decoder-feed line.
- While any input is producing aircraft (the browser receiver streaming, or a
feed live), the Local ADS-B row stays ON and lists feeds that are not live as
a trailing note ("3 heard · USB 5.8 msg/s · feed 1090 stale") instead of
showing DEGRADED.
- Records carry `band` (`1090`/`978`) and `source` (`webusb`/`feed`).
- See `docs/LOCAL-RECEIVERS.md`.
## Unreleased — local RTL-SDR and Local ADS-B
- Add a Local RTL-SDR card to the Radio panel. It connects a USB RTL-SDR in
desktop Chrome or Edge through WebUSB and receives broadcast FM (tune, seek,
volume) or 1090 MHz ADS-B. Local FM and internet-radio playback never play
together: starting one stops the other.
- Add a gain control: AUTO or a manual R820T step, remembered per mode. ADS-B
defaults to 28.0 dB (the earlier 20.7 dB gave about 1 msg/s against about 9
at 28.0 dB on the same antenna; a stored choice still wins), FM to AUTO;
changes apply without reconnecting. In
ADS-B mode the card shows CRC-valid messages per second, aircraft heard,
aircraft positioned and the IQ level.
- Prefer the ADS-B/1090 MHz channel of a dual-channel receiver, remember an
explicitly chosen device per mode, and add CHANGE DEVICE to reopen the
WebUSB picker.
- Add the Local ADS-B layer (`local-adsb`, off by default, not part of share
links). Aircraft heard by the receiver draw in magenta beside public Flights;
a marker drops when its position is 60 s old and the aircraft is forgotten
after 60 s without a message. Clicking one opens a card; markers are not
camera-followed. Voice `set_layer_visibility` accepts `local-adsb`.
- Normalize local ADS-B into one record shape, with a pure adapter for
dump1090/readsb `aircraft.json` documents.
- Add `@jtarrio/webrtlsdr` and `@jtarrio/signals` (Apache-2.0); see
`THIRD_PARTY_NOTICES.md`.
## Unreleased — weather review
- On 3D Tiles, draw a 4096×2048 detail window around the view on each
observed-weather shell except global infrared, sampled by the shell's own
surface. It follows the view on camera move end, keeps its place while the
view stays near its centre, and hides until its image is ready after a move;
an older frame's detail stays over at most one newer frame. The image proxy
accepts a 2:1 `bbox` inside the product bounds, rounded to 0.25°.
Lightning's whole-extent image is now 4096×2048; each shell caches up to
128 MiB of decoded images.
- On 3D Tiles, show observed weather and the wind color field as raised,
translucent shells (5.0–6.6 km, lightning highest) with one full-extent image
per frame instead of draping onto tiles; they show at any camera height.
Globe hosts are unchanged. The weather image proxy serves every product
(radar and regional infrared up to 4096×2048, lightning and global infrared
up to 2048×1024) with a size parameter and a 16 MiB cap. Accept bounded tile
sizes in the proxy with separate immutable cache entries.
- Cache exact-time weather images and tiles for 24 hours. Retain up to 6 decoded
global mosaics per renderer and warm the next observation during playback;
tile prefetch is bounded to eight requests and cancels when suspended.
- Move weather times, coverage, legends and cyclone advisory details into keyed
right-rail cards; left rows keep status and configuration. Use one native
observed-history timeline with local preview and coalesced drag commits.
- Add reusable rail card and timeline components. Keep panel collapse, count,
hidden-empty behavior and first-appearance expansion.
- Show wind unit controls beside speed legends and inside inspection readings;
changing units preserves the sampled location and open reading.
- Name satellite imagery Satellite clouds, with Clouds only / Full image modes
and explicit regional coverage. Existing share parameters are unchanged.
- Share one observed history clock across radar, infrared and lightning. Step
through their union timeline with bounded nearest-at-or-before selection;
hide products without an eligible frame. Latest uses each product's newest
observation; playback waits for all frame loads to settle before advancing.
- Label wind as a forecast that does not follow observed history. Keep history
transient and product readouts synchronized.
- Keep Google 3D Tiles drawing while draped weather imagery loads and retain the
old observation until the replacement has rendered.
- Decode one bounded global infrared mosaic per frame on both map hosts, then
crop local tiles to avoid request-dependent brightness seams.
- Add Clouds only / Full image controls and share-link state. Filtered mode uses
a soft linear-brightness alpha ramp from 0.40 to 0.70 around the old 0.55 cut.
- Dock the weather summary in the right rail with standard panel collapse and drag chrome.
- Filter infrared brightness so cold cloud tops stand out; this is a display
filter, not a cloud mask.
- Retry throttled weather tiles up to three times per tile.
- Cull regional wind batches per frame, fade curves below 60 km, and stop idle
rendering when no curve is visible.
- Fade the wind color field below ~1,200 km camera height and hide it at 200 km.
On 3D Tiles, update alpha in 0.1 steps only at camera move end or installation;
globe imagery keeps its smooth per-frame fade.
- Hide cyclone markers, labels, tracks and cones beyond the horizon on every map source.
- Reserve stable weather status space and coalesce panel refreshes per frame.
- Add keyless NOAA/NHC cyclone advisory positions, coherent forecast tracks and
uncertainty cones, plus NOAA's observed 15-minute lightning density imagery.
Preserve source clocks, basin coverage and explicit pending/stale states.
- Make wind default to trails, preserve earlier share-link appearance, retain
geometry across scalar changes, and show a compact weather summary with a
location marker and selected-field emphasis for forecast inspection.
- Add keyless NOAA observed rain radar and infrared satellite layers to Weather,
with explicit coverage/freshness, recent observation playback and native Cesium tiles.
- Increase desktop wind density to 7,200 paths and improve temperature contrast
while retaining the 1,200-path narrow-screen budget and unchanged forecast values.
- Expand Wind into a surface-weather prototype: globe-draped speed shading,
optional same-run 2 m temperature and mean sea-level pressure, GFS/ECMWF model
selection, a numeric legend, wind units, Pause, and a dismissible map-center
reading. Keep wind visible when an optional field is unavailable; respect
reduced motion and stop animation while hidden or disabled. Bake bounded
forecast-following curves once per field and animate their phase on the GPU,
with a canvas fallback and globe view lighting owned only while Wind is enabled.
Display lift does not
change the 10 m forecast level; this adds no cloud volume, radar or forecast-time
playback. Native hardware GPU behavior remains unverified.
Add feed provenance to analyst/view answers and HUD context while retaining existing response fields and runner ownership (Matt Van Horn, #347).
Analyst records for loaded satellites, datacenters and dams, with explicit bounded count/rank coverage (Matt Van Horn, #351).
- New Fire Perimeters layer (Events group): live NIFC WFIGS interagency
wildfire incident perimeters as ground-clamped polygons with a
containment-colored fire line, refreshed every 5 minutes from the public
keyless feature service with truncation paging. Clicking a perimeter shows
an incident card (acreage, containment, cause, behavior, personnel, county,
cost, complex membership) and, when the incident has a state- and
recency-verified InciWeb page, a click-through link to it. Recency uses
incident page origin and update times because the publication API was retired.
The layer is reachable from the panel, voice control, share links (token `2`), and the
analyst query engine. WFIGS and InciWeb requests use a capped, cached
same-origin proxy with timeouts and a per-client limit. Unchanged refreshes
retain geometry; incident-link checks abort on disable or selection change,
and the row includes a containment legend.
- Remove the spurious scrollbars that appeared on both panel stacks at narrow
widths (720px and below) as soon as a panel was expanded. The stacks scroll
vertically there, and each panel's decorative glow, absolutely positioned
with a negative inset, became 18–20px of scrollable overflow on both axes: a
horizontal scrollbar band under the expanded CCTV, Context, Data Layers or
Scenes panel plus a vertical scrollbar that scrolled nothing but glow. The
narrow-screen rules now pin the glow to its panel box; the stacks still
scroll for genuinely tall content such as an expanded DISPLAY panel, and the
Context radio popover is not clipped. `src/ui/panelRails.test.mjs` pins the
rule against every 720px media block.
- Move Google geocoding behind the local server. Forward and reverse lookups now
go through `/api/google/geocode` and `/api/google/reverse-geocode` with the
server key, joining Places and Street View. Google's Geocoding web service
@@ -61,17 +410,14 @@
with shared playback/seek interpolation, easing and holds. Navigation and
manual input cancel authored motion; older scene files retain existing flights.
- Director validates bounded version-3 scene files before replacing a project,
preserves unreadable browser saves, migrates legacy bloom once and preserves
zero-pitch/low-altitude camera and scope/detection edits. Project normalization has a separate owner.
- Separate Director timing, seek calculations, playback clocks and registered
scene-pack presentation rules. Preserve authored content and controls; Stop
releases pending hold timers and stale ticks cannot affect replacement playback.
- Keep parked transit vehicles aligned to their world course during camera orbits, fall back to reported bearing, and keep vehicles with no course consistently screen-up.
- Separate Realtime connection, response/tool, Radio, input/audio, cost, viewport
@@ -165,6 +511,13 @@
- Split application scene, controls, catalog, tools and HTML into reusable components; configure application request services and sources without changing global fetch. Preserve standalone markup and voice behavior. Explicit annotation navigation may resolve a distant named target.
## Satellite pass prediction
- Bisect pass rise/set to ~0.2 s and fit peak elevation with a parabola.
- Mark passes visible from Earth-shadow and civil-twilight checks.
- Add `getNextSatellitePass(noradId, options)` for any loaded catalog satellite.
- `next_iss_pass` retains the next geometric pass and adds visibility metadata. `next_satellite_pass` adds bounded loaded-catalog name/NORAD lookup and optional visible-only filtering (Rehaan Delmotra, #451; maintainer adaptation).
## Voice component boundaries
- Separate voice controls, Realtime connection requests and the action runner.
@@ -256,8 +609,10 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
## [Unreleased]
- Add bounded Director feature actions with accessible controls, explicit camera/layer admission and cancellation; restore pack geometry on same-shot seek. Preserve existing scenes and content attribution.
- Add ECMWF IFS model selection to Wind (#464, thanks @beneduzi), with model-scoped forecast-step caches, cancellation of replaced requests, and separate issue/valid timestamps.
- Add bounded Director feature actions with accessible controls, explicit camera/layer admission and cancellation; restore pack geometry on same-shot seek. Preserve existing scenes and content attribution.
- Give application request services, terrain/floor caches and annotation lookup state explicit owners and cancellation; share them across controls, layers and voice.
@@ -352,6 +707,8 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
### Added
- Add MODIS NRT (Terra+Aqua, ~1 km) active fires to the FIRMS layer, sharing the
existing `FIRMS_MAP_KEY` and 30-minute cache.
- Two map-orientation controls sit beside Share in the top-center globe
actions. Tilt Map swings between a straight-down map and a 35-degree oblique
around the point under the centre of the view, keeping that point and the
@@ -386,6 +743,12 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
Visible animation, detection membership, history storage and proxy requests
are bounded. Share links carry Transit as token `j`.
- Add a keyless **Wind** layer from NOAA GFS 10 m wind (#459, thanks @beneduzi). The `/api/wind` proxy
byte-range fetches only the UGRD/VGRD GRIB2 messages from the public AWS bucket,
decodes them with ecCodes (WASM), and serves a compact Float32 U/V grid; the
client renders nullschool-style animated particles in a canvas overlay that
follows the Cesium camera and skips globe-occluded points. Forecast, not
observations. Requires Node ≥24 for the WASM decoder.
- Add Ontario 511 as a keyless CCTV source pack, including Kitchener-area
highway cameras, with server-registered still URLs and attribution.
- CCTV Mesh adds Finland: Fintraffic road weather cameras, keyless, nationwide, 300 by default. Each camera view of a station is placed separately; ambient stills refresh on the source's 10-minute cadence (the active camera keeps the usual 10-second refresh).
@@ -1048,3 +1411,10 @@ represent previously published GitHub Releases.
## [0.1.0] — 2026-02-09
- Initial project version.
### Live CCTV integration candidate
- Live HLS video shares one decoder between the camera panel and projection,
with a DelDOT HTTPS source pack. Credit: Daniel Slay (@Danielslay86), PR #489.
- Maintainer adjustments bound sessions and downloads, remove disk/subprocess
remuxing, reject redirects, and clean up playback on switching or disabling.
CONTRIBUTING.md
+33
View File
@@ -70,6 +70,39 @@ gate for the feature you changed.
> and a browser. Include the applicable local run in your PR's validation
> evidence.
## Share-link layer tokens
Every shareable layer owns an immutable token in `LAYER_STATE_REGISTRY` and
`src/data/layerStateTokenReservations.json`. Never change or reuse an existing
token: published v2 links are authored state, and the reservation remains even
if its layer is later removed.
New layers first use the remaining unreserved single-character digits (`0`
through `9`), then the two-character base-36 namespace (`00` through `zz`). On the
current published ledger, `1` and `2` are already owned, so the next choices
are `0`, `3` through `9`, then `00`. After rebasing onto the latest `main`, run
`npm run layer-token:next -- <layer-id>`, add that result to both the permanent
JSON reservation ledger and registry entry, and keep the exact registered-layer
count assertion up to date. The token is assigned against the merge-time base, not
claimed when a PR opens, so another merged layer may require rerunning the
command after a rebase. Never repurpose a published or retired token.
Run `npm run layer-token:check -- --base-ref origin/main` after the rebase;
this checks that every published assignment remains reserved and that new
assignments consume free digits before the two-character tokens, in order.
Pull-request CI runs the same check against `main`.
The helper reads the local ledger; it does not run during a rebase or edit files.
If another PR publishes your provisional token first, keep every published
registry entry and ledger reservation from the refreshed `main`. Temporarily
remove only your still-unpublished layer's registry entry and ledger row so the
local registry is valid again, then rerun `npm run layer-token:next -- <layer-id>`.
For example, if PR A publishes `0` while PR B still provisionally uses `0`,
PR B removes only its own provisional entry, reruns the helper, and receives
`3`. Put `3` on PR B's registry entry and ledger row, update its pinned count
and token tests, then rerun `npm run layer-token:check -- --base-ref origin/main`.
An invalid duplicate ledger can fail during module import before the helper
prints anything; never fix that by renaming or deleting PR A's published token.
## Good first contributions
The highest-leverage places to jump in:
DATA_SOURCES.md
+179 -38
View File
@@ -15,7 +15,12 @@ How to read this:
| Source | Used for | License / terms | Attribution |
| --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| **OpenStreetMap ALPR camera locations** (including DeFlock community mapping) | Optional mapped automatic license-plate-reader camera layer | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/); commercial use permitted with applicable attribution and database share-alike obligations | [© OpenStreetMap contributors](https://www.openstreetmap.org/copyright); [DeFlock](https://deflock.org) community mapping |
| **NOAA GFS (wind)** (`noaa-gfs-bdp-pds.s3.amazonaws.com`) | Global 10 m wind, optional 2 m temperature and mean sea-level pressure for Wind. Keyless; latest 6-hourly 0.25° cycle, byte-range GRIB2 reads, cached for an hour | U.S. public domain (NOAA); keyless via NOAA Open Data on AWS | "NOAA Global Forecast System (GFS)" (courtesy; not an endorsement) |
| **ECMWF IFS (wind)** (`data.ecmwf.int/forecasts`, ECMWF Open Data) | The alternative Wind model: 10 m wind, optional 2 m temperature and mean sea-level pressure. Keyless; latest 6-hourly 0.25° run, byte-range GRIB2 reads, cached for an hour | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) plus the [ECMWF Terms of Use](https://apps.ecmwf.int/datasets/licences/general/) | "This service is based on data and products of the European Centre for Medium-Range Weather Forecasts (ECMWF)", with the CC BY 4.0 link, the modification notice and ECMWF's liability disclaimer — shown in-app |
| **NOAA nowCOAST observed weather** (`nowcoast.noaa.gov/geoserver/observations/{weather_radar,satellite}/ows`, WMS 1.1.1) | Rain radar (MRMS base reflectivity, contiguous US, about 4-minute updates) and Satellite clouds (GOES-19/18 Band 14 regional infrared, about 5-minute updates; global infrared mosaic, hourly). Keyless; layer metadata refreshed every 2 minutes, exact-time images cached for up to 24 hours | [NOAA disclaimer](https://oceanservice.noaa.gov/disclaimer.html) | "NOAA nowCOAST · NWS/OAR MRMS radar; NESDIS GOES and global satellite partners" |
| **NOAA nowCOAST lightning density** (`nowcoast.noaa.gov/geoserver/observations/lightning_detection/ows`) | Lightning density: 15-minute accumulated strike density on an approximately 8 km grid, Americas and Pacific. Keyless; layer metadata refreshed every 10 minutes, exact-time images cached for up to 24 hours | NOAA Level-5 derived product from Vaisala NLDN/GLD360; the [product description](https://ocean.weather.gov/lightning/lightning_pdd.php) permits public distribution of this density product, not of raw Vaisala detections | "NOAA/NWS nowCOAST; derived from Vaisala NLDN/GLD360" |
| **NOAA NHC / CPHC advisories** (`www.nhc.noaa.gov/CurrentStorms.json` + the `mapservices.weather.noaa.gov` tropical weather summary MapServer) | Cyclone advisories: storm positions, forecast tracks, points and cones for the Atlantic and eastern/central North Pacific. Keyless through `/api/cyclones`; cached for 5 minutes | [NWS public-data terms](https://www.weather.gov/disclaimer); no endorsement implied | "NOAA/NWS NHC / CPHC" |
| **OpenStreetMap** | Map and place data (roads, military areas and names, ALPR, place search, outlines); via vector tiles, hourly ALPR extract, bundled data, Nominatim and optional self-configured Overpass | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | © OpenStreetMap contributors |
| **Google Map Tiles API** (Photorealistic 3D Tiles) + Places/Geocoding | The 3D globe, voice scene context, and on-demand nearby installation search | Google Maps Platform ToS (proprietary, your own key + billing) | "Google" / "Google Maps" logo — **shown in-app**, required |
| **OpenSky Network** | Primary worldwide live-flight snapshot | Non-commercial research/education license | Schäfer et al., _"Bringing Up OpenSky"_, IPSN 2014 + opensky-network.org |
| **adsb.lol point API** | Bounded live-flight fallback when OpenSky has no usable snapshot | ODbL 1.0 | adsb.lol contributors; `api.adsb.lol/v2/lat/{lat}/lon/{lon}/dist/{radius}` |
@@ -26,11 +31,13 @@ How to read this:
| **The Space Devs — Launch Library 2 v2.3** | Recent launch, payload, stage, and recovery metadata for Space Missions (30d) | [The Space Devs terms of use](https://github.com/TheSpaceDevs/Tutorials/blob/main/faqs/faq_TSD.md#terms-of-use): data may be used and shared in any form; avoid forwarding it without added value; attribution is encouraged (not mandatory). [Official API limits](https://ll.thespacedevs.com/docs/): 15 unauthenticated calls/hour; optional token | "Launch Library 2 — The Space Devs" (courtesy attribution) |
| **Esri World Imagery** (ArcGIS Online tile service) | The keyless satellite basemap — the default landing when no Google/ion credential is configured, and the "Esri Satellite" map stack | [Esri Master Agreement](https://www.esri.com/en-us/legal/terms/full-master-agreement): the public World Imagery service is usable in public-facing apps with attribution; no key is required for this classic endpoint, but Esri governs and can change access — an app at scale should review current ArcGIS Location Platform terms | "Powered by Esri — Source: Esri, Maxar, Earthstar Geographics, and the GIS User Community" (provider carries the service's own credit line) |
| **USGS** | Earthquakes | U.S. public domain | "Data courtesy of the U.S. Geological Survey" |
| **OpenStreetMap (Overpass API)** | Road geometry for traffic | ODbL 1.0 | "© OpenStreetMap contributors" |
| **TomTom Traffic API** (flow vector tiles) | Live congestion coloring for the traffic layer (optional, BYOK) | [TomTom for Developers terms](https://developer.tomtom.com) (proprietary, your own key; free tier currently 200K tile requests/month — see [current pricing](https://docs.tomtom.com/pricing/)) | "Traffic flow data © TomTom" — registered when live mode activates |
| **OpenStreetMap (Overpass API)** | Viewport-bounded mapped installation context for Global Context | ODbL 1.0 | "© OpenStreetMap contributors" (incomplete mapped context) |
| **Photon** (komoot) | Keyless place search — the fallback when no Google Maps key is configured, or when Google declines the Geocoding request | Service: free public instance, [fair use](https://photon.komoot.io) (no bulk/heavy use); underlying data: **ODbL 1.0** | "Photon (komoot)" + "© OpenStreetMap contributors" |
| **OpenStreetMap (Nominatim)** | Reverse-geocoded place label in the cockpit Local Info page, and last-resort forward place search via `/api/geocode` when Google and Photon do not answer | ODbL 1.0 + [Nominatim usage policy](https://operations.osmfoundation.org/policies/nominatim/) (maximum 1 request/second, results cached, identifying User-Agent) | "© OpenStreetMap contributors" |
| **NASA GIBS** (WMTS tiles + Worldview Snapshots) and **NASA CMR** (granule catalog) | Recent Imagery layer: HLS S30/L30 (Sentinel-2, Landsat 8/9) and VIIRS true-colour tiles draped on the map, per-day thumbnails and PNG exports; the HLS granule catalog (dates, cloud, footprints) for a selected box | U.S. public domain (NASA data); keyless, browser-direct (CORS `*`); no bulk use — one 30-day catalog query per box, tiles bounded to the box, six tile requests in flight | "We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS)." — [gibs.earthdata.nasa.gov](https://gibs.earthdata.nasa.gov), [HLS](https://lpdaac.usgs.gov/products/hlss30v002/) |
| **NIFC WFIGS** (Wildland Fire Interagency Geospatial Services, ArcGIS feature service) | Fire Perimeters layer: current interagency wildfire incident perimeters, containment, and incident facts | U.S. public domain (interagency wildland-fire data published through the [NIFC Open Data portal](https://data-nifc.opendata.arcgis.com/)); fetched keyless through `/api/fire-perimeters` (5-minute server cache) with `exceededTransferLimit` paging | "Wildfire perimeters: National Interagency Fire Center (WFIGS)" |
| **InciWeb** (inciweb.wildfire.gov) | Per-incident "InciWeb ↗" card links to official incident pages (narratives, evacuation/closure notices) | U.S. government public incident information. Links are matched best-effort by incident name and state, with incident page origin and update times checked on selection. Same-origin `/api/fire-perimeters/inciweb` routes cache the catalog for 1 hour and publication metadata for 30 minutes; linked pages remain InciWeb content | "Incident information: InciWeb (inciweb.wildfire.gov)" |
| **OpenFreeMap / OpenMapTiles** | Vector tiles | OpenStreetMap data under ODbL 1.0 | OpenFreeMap © OpenMapTiles |
| **Overture Maps Foundation** | Military area names pack | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | Overture Maps Foundation |
| **TomTom Traffic API** (flow vector tiles) | Traffic roads and live congestion (optional, BYOK) | [TomTom for Developers terms](https://developer.tomtom.com) (proprietary, your own key; free tier currently 200K tile requests/month — see [current pricing](https://docs.tomtom.com/pricing/)) | "Traffic flow data © TomTom" — registered when live mode activates |
| **Photon** (komoot) | Keyless place search — the fallback when no Google Maps key is configured, or when Google declines the Geocoding request | Service: free public instance, [fair use](https://photon.komoot.io) (no bulk/heavy use); underlying data: **ODbL 1.0** | "Photon (komoot)" |
| **Open-Meteo** | Current weather in the cockpit Local Info page and cockpit-local dynamic atmospheric effects | [CC BY 4.0 data licence and adjacent-link attribution requirement](https://open-meteo.com/en/licence) | Linked "Weather data by Open-Meteo.com" beside the displayed local data |
| **Google News RSS** | Primary locality-matched headlines in the cockpit Regional News page | [Google News Terms of Service](https://www.google.com/intl/en_us/terms_google_news.html) restrict use to personal, noncommercial use; linked articles remain third-party publisher content and retain publisher terms | "Google News RSS" plus each article's linked publisher/domain |
| **GDELT Project DOC 2.0** | Fail-soft fallback for location-matched cockpit headlines | [GDELT Terms of Use](https://www.gdeltproject.org/about.html#termsofuse): unrestricted academic/commercial/governmental dataset use, with citation and link required; linked articles retain publisher terms | "GDELT Project" plus each article's linked publisher/domain |
@@ -38,9 +45,10 @@ How to read this:
| **TxDOT ITS (its.txdot.gov)** | CCTV camera catalogs + frames, Texas districts | Public TxDOT traffic camera data | "Texas Department of Transportation" (courtesy) |
| **City of Tallinn (ristmikud.tallinn.ee)** | CCTV camera catalog + frames, Tallinn intersections | Public City of Tallinn traffic camera data | "City of Tallinn — ristmikud.tallinn.ee" (courtesy) |
| **Transpordiamet / Tarktee** | CCTV camera catalog + frames, Estonia road-weather cameras | Public Transpordiamet / Tarktee road camera data | "Transpordiamet / Tarktee — tarktee.transpordiamet.ee" (courtesy) |
| **Stadt Warendorf webcam** | Webcam frames, Marktplatz / Historisches Rathaus (`config/cctv_sources.warendorf.json`) | Public municipal webcam data | "Stadt Warendorf — webcam.warendorf.de" (courtesy); mount and heading derived from OSM geometry, "© OpenStreetMap contributors" (ODbL) |
| **Stadt Warendorf webcam** | Webcam frames, Marktplatz / Historisches Rathaus (`config/cctv_sources.warendorf.json`) | Public municipal webcam data | "Stadt Warendorf — webcam.warendorf.de" (courtesy) |
| **Live Traffic NSW (Transport for NSW)** | CCTV camera catalog + frames, New South Wales | [CC BY 4.0](https://opendata.transport.nsw.gov.au/) — attribution REQUIRED | "Live Traffic NSW — Transport for NSW" |
| **Caltrans (cwwp2.dot.ca.gov)** | CCTV camera catalogs + frames, California districts | Public Caltrans traffic camera data | "Caltrans — cwwp2.dot.ca.gov" (courtesy) |
| **DelDOT (video.deldot.gov)** | CCTV camera catalog (`tmc.deldot.gov/json/videocamera.json`, cached for 15 minutes) + live HTTPS HLS video, Delaware. Keyless | Public DelDOT traffic camera streams via the [DelDOT traffic map](https://deldot.gov/map/) | "DelDOT — Delaware Department of Transportation" (courtesy) |
| **TfL Open Data (JamCams)** | CCTV camera catalog + frames, London | [TfL Open Data terms](https://tfl.gov.uk/info-for/open-data-users/) — attribution REQUIRED | "Powered by TfL Open Data. Contains OS data © Crown copyright and database rights" |
| **Ontario 511** | CCTV camera catalog + frames, Ontario highways including Kitchener-area routes | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) | "Ontario 511" plus Open Government Licence - Ontario |
| **Fintraffic / Digitraffic (weathercams)** | CCTV camera catalog + frames, Finland | [CC BY 4.0](https://www.digitraffic.fi/en/terms-of-service/) — attribution REQUIRED | "Fintraffic / digitraffic.fi, license CC BY 4.0" |
@@ -56,7 +64,11 @@ How to read this:
| **HSL** (realtime.hsl.fi) | Transit layer live vehicles, Helsinki | HSL (Helsinki Region Transport) open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to HSL (Helsinki Region Transport) (courtesy text) |
| **Radio Browser** | Geolocated internet-radio station directory and station-level tags | Public-domain directory data under PDDL 1.0; individual broadcaster stream terms apply | "Radio Browser" plus a link to the selected broadcaster |
| **Re:Earth Terrain** (Mapterhorn) | Terrain (keyless globe stacks — OSM etc. — + `/api/terrain/heights` ellipsoidal-height lookups) | Terrain mesh: CC BY 4.0; geoid: EGM2008 (NGA, public domain) | "Terrain (keyless globe stacks): Re:Earth Terrain / Mapterhorn (CC BY 4.0) / EGM2008 (NGA)" |
| **OSRM on the FOSSGIS routing servers** (`routing.openstreetmap.de`) | Street-following routes for the Directions layer and voice route annotations, via `/api/route` | [FOSSGIS routing usage policy](https://routing.openstreetmap.de/about.html): "Display the required attribution and display a link to 'fix the map'", "Use a valid user agent and, if applicable, a correct referrer", "One request per second max", "No scraping, no heavy usage". The full policy is the German [FOSSGIS Nutzungsbedingungen](https://www.fossgis.de/arbeitsgruppen/osm-server/nutzungsbedingungen/); FOSSGIS also states that the server may be embedded in your own pages but "eine gewerbliche Nutzung ist nur mit Einschränkungen erlaubt" (commercial use only with restrictions). Route data derives from OpenStreetMap (ODbL 1.0) | "Routing: OSRM on the FOSSGIS servers" + "© OpenStreetMap contributors" + a "fix the map" link — shown in the Data attribution popover |
| **OSRM on the FOSSGIS routing servers** (`routing.openstreetmap.de`) | Street-following routes for the Directions layer and voice route annotations, via `/api/route` | [FOSSGIS routing usage policy](https://routing.openstreetmap.de/about.html): "Display the required attribution and display a link to 'fix the map'", "Use a valid user agent and, if applicable, a correct referrer", "One request per second max", "No scraping, no heavy usage". The full policy is the German [FOSSGIS Nutzungsbedingungen](https://www.fossgis.de/arbeitsgruppen/osm-server/nutzungsbedingungen/); FOSSGIS also states that the server may be embedded in your own pages but "eine gewerbliche Nutzung ist nur mit Einschränkungen erlaubt" (commercial use only with restrictions). Route data derives from OpenStreetMap (ODbL 1.0) | "Routing: OSRM on the FOSSGIS servers" + a "fix the map" link — shown in the Data attribution popover |
### Local receiver input (not fetched from a service)
The optional Local ADS-B layer and the broadcast-FM receiver use radio you receive yourself: a USB RTL-SDR opened in the browser through WebUSB, or the `aircraft.json` of a decoder you run on your own machine or network (`LOCAL_RECEIVER_FEEDS`). Radio samples, audio and the optional receiver location stay in the browser. Decoder feeds are read by your local GEV server and relayed only to your browser, with about a second of in-memory caching. When you select a local aircraft, its ICAO address (and an airline-style callsign) is looked up, and aircraft that qualify for a 3D model are looked up by ICAO address, through the same adsbdb proxy the public flight layers use; your server caches those answers on disk and treats them as fresh for 24 hours. Received radio data is not stored or redistributed. ADS-B is an unauthenticated public broadcast and may be incomplete, delayed, spoofed or decoded in error; it is not suitable for navigation or operational use.
### Notes on the live sources
@@ -67,6 +79,7 @@ How to read this:
- **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state.
- **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover.
- **Ontario 511 CCTV.** The camera list comes from the keyless `511on.ca/api/v2/get/cameras` endpoint; frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence - Ontario.
- **DelDOT live video cameras.** The keyless `tmc.deldot.gov/json/videocamera.json` catalog supplies active cameras and official `https://video.deldot.gov/live/…/playlist.m3u8` links. The local server transiently buffers bounded MPEG-TS segments in memory and deletes them on session release or idle expiry. No recordings are retained. `CCTV_DELDOT_ENABLED=0` disables the pack. Public availability is not a blanket redistribution license; deployment beyond personal local viewing needs confirmation of applicable DelDOT terms.
- **Fintraffic weathercams.** The camera list is one keyless GeoJSON request to `tie.digitraffic.fi/api/weathercam/v1/stations` (809 stations / 2,275 presets), sent with the `Digitraffic-User: gods-eye-view` header the service asks clients to identify themselves with and with gzip; frames are stills on `weathercam.digitraffic.fi`, refreshed on each station's 600 s collection interval, which is also the client's ambient-still refresh cadence for this pack (the active camera keeps the usual 10-second refresh). One preset (a fixed camera view) is one camera; stations not `GATHERING` and presets not `inCollection` are dropped. Attribution is required by the CC BY 4.0 licence and is registered in the Data attribution popover. `CCTV_FINTRAFFIC_ENABLED=0` is the kill switch.
- **DriveBC highway cameras.** The camera list comes from the keyless `https://www.drivebc.ca/api/webcams/` endpoint served by the DriveBC.ca site ([bcgov/DriveBC.ca](https://github.com/bcgov/DriveBC.ca)); frames come from `https://www.drivebc.ca/images/<id>.jpg`, built from the numeric camera id. DataBC's [DriveBC HighwayCams](https://catalogue.data.gov.bc.ca/dataset/bc-highwaycams) catalogue entry publishes the same cameras under the Open Government Licence – British Columbia, but its CSV still lists the retired `images.drivebc.ca` frame URLs, which now return a placeholder. The licence's attribution statement is required and is registered in the Data attribution popover. By default the 250 cameras nearest Vancouver and Victoria load; `CCTV_DRIVEBC_MAX_SOURCES` changes the cap and `CCTV_DRIVEBC_ENABLED=0` turns the pack off. Some cameras in the feed are supplied by partners (TransLink, the City of Vancouver, the City of Surrey, Parks Canada / Alberta Motor Association and others); the feed's per-camera credit is carried onto the camera record and shown beside the provider in the CCTV panel.
- **TxDOT ITS cameras.** The catalog comes from the keyless `its.txdot.gov/its/DistrictIts/GetCctvStatusListByDistrict` endpoint, one call per district. Frames come from the per-camera `GetCctvSnapshotByIcdId` endpoint, which answers JSON carrying a base64 JPEG rather than an image body; the proxy decodes it only for responses from the official TxDOT origin and validates the JPEG header before serving. Only cameras reporting `Device Online` are registered. Districts default to Austin and San Antonio (`CCTV_TXDOT_DISTRICTS`; 25 codes exist statewide); `CCTV_TXDOT_MAX_SOURCES` caps the pack and `CCTV_TXDOT_ENABLED=0` turns it off. Frames are fetched live at request time and are never stored or redistributed.
@@ -82,9 +95,9 @@ How to read this:
- **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line.
- **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable.
- **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit.
- **Global Context installation context.** `/api/military-installations` queries only an allow-listed subset of OSM `military=*` and `landuse=military` features inside a maximum 10° non-dateline viewport. It caches and may serve stale mapped context, but it is neither a global installation database nor evidence of capability, activity, or absence. User-requested Google Places results remain separately sourced candidates unless their returned types explicitly establish military classification; generic offices, museums, and similarly ambiguous matches are excluded from military proximity counts.
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon's public instance, then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` shares the cockpit's Nominatim queue at no more than one request per second, sends an identifying `User-Agent` and `Referer`, caches answers for five minutes, shares one upstream call between identical searches already in flight, bounds the queue so a burst is refused rather than held, and drops a queued search whose caller has given up. It asks for one result per search and is not used for systematic or bulk queries.
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and serializes Nominatim calls at no more than one request per second. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
- **Mapped installations.** OpenFreeMap military polygons use a bundled Overture/OSM name index; wide views show named points. Optional operator-configured Overpass and explicit Google Places searches keep their own source paths. Mapped data does not establish current activity or complete coverage.
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon's public instance, then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` uses the shared Nominatim forward-search queue at no more than one request per second, sends an identifying `User-Agent` and `Referer`, caches answers for five minutes, shares one upstream call between identical searches already in flight, bounds the queue so a burst is refused rather than held, and drops a queued search whose caller has given up. It asks for one result per search and is not used for systematic or bulk queries.
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and resolves physical regions from bundled public-domain Natural Earth polygons without reverse-geocoding requests. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
- **Dynamic weather presentation.** While cockpit mode is active, `/api/weather-effects` requests current Open-Meteo observations for the aircraft/camera location, rounds coordinates into 0.1° cache cells, caches results for five minutes, and may retain a stale observation for up to 30 minutes during a transient outage. WMO condition code selects the visual family; observed cloud cover, precipitation, visibility, wind speed, and wind direction bound its strength and motion. Missing or expired weather renders no synthetic atmospheric effect, and normal globe view never renders the weather overlay.
---
@@ -97,11 +110,14 @@ The [Bhote Koshi event pack](public/events/bhote-koshi-2026/README.md), under `p
| Dataset | Folder | License | Commercial use? | Attribution |
| --------------------------------------------------------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------- |
| **Datacenters** (~4.3K) | `datacenters/` | **ODbL 1.0** (OpenStreetMap extract) | ✅ (attribution + share-alike on data) | "© OpenStreetMap contributors" |
| **Dams** (704) | `dams/` | **ODbL 1.0** (OpenInfraMap / OSM extract) | ✅ (attribution + share-alike on data) | "© OpenStreetMap contributors" (+ Open Infrastructure Map) |
| **Datacenters** (4,351) | `datacenters/` | **ODbL 1.0** (OpenStreetMap extract) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) |
| **Dams** (704) | `dams/` | **ODbL 1.0** (OpenInfraMap / OSM extract) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + Open Infrastructure Map |
| **Military area names** (36,466) | `osm_military_names/` | **ODbL 1.0** (OpenStreetMap, via Overture Maps) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + Overture Maps Foundation |
| **TeleGeography Submarine Cable Map** (712 cables + 1,917 landing points) | `telegeography_submarine_cables/` | **CC BY-NC-SA 3.0** | ❌ **NonCommercial — remove for commercial use** | "© TeleGeography — submarinecablemap.com" |
| **Natural Earth physical regions** (1,046 land + 292 marine named polygons) | `natural_earth/` | **Public domain** | ✅ (no restrictions) | "Made with Natural Earth" (courtesy credit — not legally required) |
| **DataSF Analysis Neighborhoods** (41 SF neighborhood polygons) | `neighborhoods/` | **PDDL 1.0** (public domain) | ✅ (no restrictions) | "City & County of San Francisco — DataSF" (courtesy — not legally required) |
| **Natural Earth countries, states and provinces** (260 admin-0/map units + 4,587 admin-1 polygons) | `natural_earth/` | **Public domain** | ✅ (no restrictions) | "Made with Natural Earth" (courtesy credit — not legally required) |
| **US Census Bureau counties** (3,235 county polygons) | `us_census_counties/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
| **CCTV ground heights** (3,445 cameras) | `cctv_ground_heights/` | Precomputed camera placement heights, aligned to work with Google Photorealistic 3D Tiles (folder README) | — | — |
### ⚠️ TeleGeography is bundled but NonCommercial
@@ -114,33 +130,23 @@ The richer structured dataset is licensed separately/commercially by TeleGeograp
### ALPR camera mapping
The optional ALPR layer queries OpenStreetMap nodes tagged `man_made=surveillance`
and `surveillance:type=ALPR` (including case-insensitive semicolon multi-values)
through the shared Overpass proxy. Source/tag reference:
[OSM ALPR tagging](https://wiki.openstreetmap.org/wiki/Tag:surveillance:type%3DALPR).
No DeFlock software, branded assets or separate database is bundled. The layer
shows contributor-supplied locations and tags, not plate records, camera video,
current operating status, or exhaustive coverage. All manufacturers share the
ALPR category and marker color. Manufacturer, operator and camera type are shown
only when tagged, not inferred or independently verified. Every selected card
names OpenStreetMap in a smaller, muted source line beneath those details.
The optional ALPR layer reads OpenStreetMap `surveillance:type=ALPR` nodes
from a community-hosted hourly US/Canada extract, using detail vector tiles.
It shows mapped locations and available tags, not plate records, camera video,
current operating status or exhaustive coverage. OSM edit timestamps are not
presented as field-verification dates. Other regions show no ALPR data.
Queries are limited to a city-scale viewport (at most 3° per axis), at most
1,500 returned nodes, with outward-snapped query reuse and explicit stale/limited
coverage notices. A linked © OpenStreetMap credit appears for five
seconds when camera locations first display after enabling, then collapses;
Data attribution retains the full source/license entry, following the
[OSMF interactive-map guidance](https://osmfoundation.org/wiki/Licence/Attribution_Guidelines).
OSM data remains separately attributed under ODbL; querying
at runtime does not replace the license obligations. ODbL permits commercial
use and does not license the application code. If publicly using a derivative
database, follow its share-alike and access requirements; keep independent
datasets separate rather than treating the whole collection as MIT data.
Overpass hosting capacity and usage policies are separate from the data license.
Reads are viewport-bounded, with at most 16 tiles per country, four concurrent
reads per country, 1,500 displayed records and an hour-long decoded-tile cache.
Displayed OSM-derived layers retain a short inline OpenStreetMap credit, plus
OpenMapTiles while OpenFreeMap tiles are used; Data attribution retains the full credits. ODbL permits
commercial use with attribution and applicable database share-alike/access
obligations; it does not license the application code. Trimmed test-only tile
fixtures retain their provenance in `src/data/fixtures/README.md`.
### ODbL share-alike (datacenters, dams)
### Bundled OpenStreetMap data
The OSM-derived datasets are under the **Open Database License**. ODbL's share-alike applies to the **data / derived database, not this MIT-licensed code** — the two coexist (exactly how Open Infrastructure Map ships: MIT software + ODbL data). If you publicly distribute a _modified_ version of these databases, you must offer it under ODbL. Keep the "© OpenStreetMap contributors" notice (link: https://www.openstreetmap.org/copyright).
Bundled OSM-derived data is distributed under [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/), separately from the MIT code: [datacenters](src/data/local_data/datacenters/), [dams](src/data/local_data/dams/) (via Open Infrastructure Map), [military names](src/data/local_data/osm_military_names/), and [OSM test fixtures](src/data/fixtures/). Keep © OpenStreetMap contributors and offer modified databases under ODbL.
### NASA FIRMS acknowledgement
@@ -148,7 +154,7 @@ The OSM-derived datasets are under the **Open Database License**. ODbL's share-a
FIRMS active fires are **fetched live at runtime** (CC0 / U.S. public domain data): the
`/api/firms` server-side proxy merges the three VIIRS NRT sources (NOAA-20, NOAA-21,
Suomi-NPP) clamped to the trailing 24 h, cached 30 min to respect the shared MAP_KEY
Suomi-NPP) and MODIS NRT (Terra + Aqua), clamped to the trailing 24 h, cached 30 min to respect the shared MAP_KEY
transaction quota. Requires a free `FIRMS_MAP_KEY`
(https://firms.modaps.eosdis.nasa.gov/api/map_key/); the layer is empty without it.
The former bundled 2026-05-25 snapshot was removed 2026-07-16.
@@ -191,6 +197,141 @@ Douglas-Peucker simplification, 6-decimal rounding).
---
## Weather sources
### ECMWF IFS wind
Wind also offers ECMWF IFS 10 m forecasts from the keyless [ECMWF Open Data](https://www.ecmwf.int/en/forecasts/datasets/open-data) service. The proxy reads its JSON Lines inventory and byte-range fetches 10u/10v GRIB messages, optionally adding 2t (2 m temperature) or msl (mean sea-level pressure) from the same run and forecast time. It resamples these fields to the approximately 1° display grid. Data is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The [ECMWF Terms of Use](https://apps.ecmwf.int/datasets/licences/general/) also apply. In-app attribution identifies this service as based on ECMWF data and products, links the CC BY 4.0 licence, identifies resampling and animation as modifications, and retains the required ECMWF liability disclaimer. Both model issue time and the selected forecast valid time appear in the WEATHER panel. These are model forecasts, not observations.
### NOAA GFS wind
The optional **Wind** layer animates the global 10 m wind field from NOAA's
Global Forecast System (GFS). The `/api/wind` server-side proxy selects the
latest available 0.25° cycle, reads its `.idx` inventory, byte-range fetches only
the `UGRD`/`VGRD` 10 m GRIB2 messages, optionally adding `TMP` at 2 m or
`PRMSL` at mean sea level from the same run and forecast time. It decodes the
selected messages with ecCodes (WASM), resamples to a compact grid (1° by
default), and serves a manifest plus Float32 U/V and optional scalar values. It is keyless, cached
per cycle for an hour, and is a **forecast, not an observation**: the particles
show model flow, not measured wind. The renderer bakes bounded curves through
the sampled field and animates their phase on the GPU, with a canvas fallback.
Any display lift is a visual aid and does not change the forecast's 10 m level.
NOAA GFS data is U.S. public domain; the credit above is a courtesy and does not
imply endorsement.
For both models, optional temperature is normalized from K to °C and pressure
from Pa to hPa before delivery. A missing, malformed or timed-out companion field
does not discard valid wind: the manifest marks that field unavailable, and the
UI retains wind without inventing scalar values. The proxy caches by model and
requested field, with bounded current/previous grids and shared in-flight loads.
Wind speed shading is computed locally from U/V and needs no companion download.
The legend and map-center reading explain units and forecast validity; the
animation is a visual flow through one forecast, not advancing forecast time.
Optional globe relief uses existing terrain vertex normals for view lighting;
without those normals it shades global globe curvature only, not local terrain
relief. Neither mode represents measured sunlight or a new elevation source.
This Wind prototype adds no cloud-volume or radar data. Mapped.earth's public
bundles were studied for rendering ideas, but
no code or assets were reused and no application licence granting reuse was
found. Its presentation is not a weather-data source for this implementation.
### Observed weather: NOAA nowCOAST
The Weather section provides keyless, observed **MRMS radar reflectivity** for the
contiguous United States and **infrared satellite imagery**. Fixed upstream WMS
services: `https://nowcoast.noaa.gov/geoserver/observations/weather_radar/ows` and
`https://nowcoast.noaa.gov/geoserver/observations/satellite/ows`.
- `conus_base_reflectivity_mosaic`: approximately 1 km, usually 4-minute updates.
dBZ measures radar reflectivity, not rainfall rate, probability or future rain.
Coverage gaps do not mean no precipitation. The numeric legend follows NOAA's
`weather_radar_base_reflectivity` style.
- `goes_longwave_imagery`: GOES-19/18 Band 14, approximately 2 km, 5-minute updates,
regional North America. **Clouds only** softly dims pixels with a linear
brightness ramp from 0.40 to 0.70; **Full image** retains source alpha. This
is a display filter, not a cloud mask or measured cloud volume.
- `global_longwave_imagery_mosaic`: approximately 3 km, hourly, nominal 60°S–60°N
coverage and typically 2–3-hour latency. It is slower global context.
The UI displays the exact advertised observation time separately from acquisition;
latest means the newest available observation, not zero-delay real time. Up to 13
recent advertised frames can be replayed as history. No nowcast is synthesized.
Tiles use WMS 1.1.1 EPSG:4326 longitude/latitude bounds; Cesium's geographic 2×1
root grid is capped at level 6 for radar and regional GOES. Global infrared uses
one fixed 2048×1024 geographic image on both hosts: the source's request-dependent
contrast otherwise creates brightness seams between tiles. The browser decodes
and filters it once, then crops 256 px tiles bounded to the manifest extent at
maximum level 3. This broad context view has a coarser display resolution than
its 3 km source. No browser reprojection is used. Browser imagery owns a displayed
frame and a staging frame, retaining the outgoing layer for one render after
admission.
On Google 3D Tiles each observed product is instead a raised shell showing one
whole-extent image per frame (up to 4096×2048; global infrared 2048×1024). Every
product except global infrared also draws a 4096×2048 detail window around the
view, requested with a 2:1 `bbox` inside the product bounds, rounded to 0.25°.
The proxy has an 8-request concurrency budget and a 12-second deadline; tiles
(256, 512 or 1024 px) are capped at 1 MiB or 4 bytes per pixel plus 64 KiB,
whichever is larger, and whole-extent and detail images at 16 MiB. Its shared
cache holds up to 128 images and 16 MiB, each exact-time image for at most 24
hours.
Metadata refreshes every two minutes with explicitly stale last-good fallback.
No key, new dependency, image reprojection job, or full-disk image download is needed.
Credit: NOAA nowCOAST, NWS/OAR MRMS, NESDIS GOES and global satellite partners.
[NOAA disclaimer](https://oceanservice.noaa.gov/disclaimer.html).
Community context: [#85](https://github.com/bilawalsidhu/gods-eye-view/issues/85),
[#588 radar](https://github.com/bilawalsidhu/gods-eye-view/pull/588), and
[#457 clouds](https://github.com/bilawalsidhu/gods-eye-view/pull/457).
This implementation is original; those contributions have not been merged here.
### Weather: NHC/CPHC cyclone advisories
- **Sources:** [NHC current tropical cyclone status](https://www.nhc.noaa.gov/CurrentStorms.json) and
[NOAA tropical weather summary GIS](https://mapservices.weather.noaa.gov/tropical/rest/services/tropical/NHC_tropical_weather_summary/MapServer).
- **Rights/credit:** NOAA/NWS National Hurricane Center / Central Pacific Hurricane Center;
[NWS public-data terms](https://www.weather.gov/disclaimer). Fetch official products at runtime;
no bundled advisory archive and no NOAA endorsement implied.
- **Coverage/meaning:** Atlantic and eastern/central North Pacific, not global cyclone coverage.
Status position time and advisory issue time remain separate. Forecast track, points and cone
require matching advisory numbers across all GIS parts. Cone means uncertainty in the
forecast center track, not storm size or a complete hazard boundary.
- **Delivery:** keyless same-origin `/api/cyclones`, five-minute singleflight cache,
fixed upstream endpoints, bounded bodies/geometry/deadline, explicit unavailable/stale states.
Direct status fetching is unsuitable in browsers because NHC does not advertise CORS.
### Weather: observed lightning density
- **Source:** [NOAA nowCOAST lightning detection WMS](https://nowcoast.noaa.gov/geoserver/observations/lightning_detection/ows),
fixed `ldn_lightning_strike_density` layer and `lightning_density` style.
- **Rights/credit:** NOAA/NWS nowCOAST; derived from Vaisala NLDN/GLD360.
[Official product description and public distribution terms](https://ocean.weather.gov/lightning/lightning_pdd.php)
permit distribution of this NOAA Level-5 derived product, not raw Vaisala detections.
- **Meaning:** 15-minute accumulated density on an approximately 8 km grid;
source color scale is strikes per km² per minute ×10³. Not individual GLM optical flashes,
a live ground-strike counter, an all-clear indication, or global coverage.
- **Coverage:** 110°E across the Pacific/Americas to 0°, 25°S–80°N. Display exact advertised
observations; ten-minute metadata refresh, source-matched colors, bounded shared imagery cache.
### Community weather prior art
The weather design builds on Gustavo Beneduzi's retained GFS/ECMWF contribution
commits ([#459](https://github.com/bilawalsidhu/gods-eye-view/pull/459),
[#464](https://github.com/bilawalsidhu/gods-eye-view/pull/464)). Related community
proposals informed the observed-weather experience: Sandiv D's on-demand radar
and imagery controls ([#588](https://github.com/bilawalsidhu/gods-eye-view/pull/588));
Gustavo Beneduzi's GOES and GLM source work
([#457](https://github.com/bilawalsidhu/gods-eye-view/pull/457),
[#458](https://github.com/bilawalsidhu/gods-eye-view/pull/458)); and HadiMuhammed's
NOAA hazard provenance and freshness work
([#414](https://github.com/bilawalsidhu/gods-eye-view/pull/414)).
The fixed nowCOAST imagery and NHC advisory implementations are original maintainer
work; this acknowledgement does not represent merging those four PRs or equate
NOAA density imagery with the raw GLM product.
---
## In-app attribution
The required Google Maps / Cesium credit renders on the on-globe credit line (`#cesium-credits`, bottom-left) and must stay visible — including in clean-view and recording modes (the whole line, logo + "Google Maps" + the "Data attribution" link, stays on screen; only the GEV panels/HUD fade). The layer-specific credits (adsb.lol, adsbdb, TeleGeography, OSM datacenters/dams/roads, NASA FIRMS, CelesTrak, USGS, City of Austin, Fintraffic, GBFS, Radio Browser, OpenSky, AISStream) are registered into the expandable **"Data attribution"** popover on that credit line via `viewer.creditDisplay.addStaticCredit(new Cesium.Credit(html, /* showOnScreen */ false))` — see `src/data/dataCredits.js`. When you add a new data source, add its license and attribution to this file **and** append an entry to `DATA_CREDITS` in `src/data/dataCredits.js` so it surfaces in the app.
README.md
+60 -16
View File
@@ -2,7 +2,7 @@
# 🌐 God's Eye View
[![CI](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml)
[![CI](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml) [![Reached #1 on GitHub Trending](https://img.shields.io/badge/%231_GitHub_Trending-thank_you!-F0A63C?style=flat-square&logo=github)](https://x.com/bilawalsidhu/status/2093798887815348521)
### A spy-satellite simulator in your browser — then you realize the sources are public and the data is real.
@@ -18,8 +18,6 @@ _No place left behind._
▶️ **From the project behind the viral God's Eye View series** _(formerly WorldView)_ — [5M+ on YouTube](https://youtube.com/playlist?list=PL6qSg2I-7_koPbDnSMo0QeeHX_RknA2uv&si=nBGYMoHWQw41v93Q) · [25M+ across socials](https://www.google.com/search?q=god%27s+eye+view)
[![Reached #1 on GitHub Trending](https://img.shields.io/badge/%231_GitHub_Trending-thank_you!-F0A63C?style=flat-square&logo=github)](https://x.com/bilawalsidhu/status/2093798887815348521)
🏆 **Reached #1 on GitHub Trending, daily and weekly · August 2026**
**[#8 Product of the Day](https://www.producthunt.com/products/god-s-eye-view?launch=god-s-eye-view)** · Hunted by Chris Messina, creator of the hashtag
@@ -70,6 +68,14 @@ Start with the included data sources, then add your own. Each layer is a separat
- **🎥 Scene director:** Capture cinematic camera tours for clips and demos.
- **🔗 Share Links:** Camera, style, layers, and even one tracked target serialize into a URL — a live target is a handoff, not a bookmark.
- **🏠 Reset Globe:** One control — or one sentence — back to the full Earth.
- **🌦️ Weather:** Animate GFS or ECMWF forecast wind, replay observed radar, satellite clouds and lightning on one timeline, and follow NHC/CPHC cyclone tracks. No key needed.
- **📷 Mapped ALPR cameras:** License-plate-reader camera locations tagged in OpenStreetMap, one city at a time. Locations and tags only; no key needed.
- **🔭 Satellite passes:** Ask by voice when any loaded satellite next rises over you: rise, peak and set times, and whether you can see it.
- **🔎 Analyst answers:** Count, filter and rank satellites, datacenters and dams by voice, alongside flights, ships, fires and quakes; answers say when a feed is stale or on a fallback.
- **🧭 Tilt and North Up:** Switch between straight-down and a 35° oblique, or put north at the top, while a tracked target stays centered.
- **🔍 Keyless search:** Type coordinates or a bundled city or landmark to fly there with no network request; other names use Google when configured, then Photon and Nominatim.
- **📦 Shareable scenes:** Import, preview and share Director scenes as files or bundles, with camera anchors, authored moves and data packs.
- **🌊 Nepal flood scene:** Replay the Bhote Koshi flood: flood path, witness sources and before-and-after imagery (bundled data is non-commercial; see [DATA_SOURCES.md](DATA_SOURCES.md)).
---
@@ -90,9 +96,13 @@ cameras, radio, and launches are available without keys.
For photorealistic 3D, add a **Cesium ion token** for eligible personal,
non-commercial use, or a **Google Maps key** for the direct, metered route and
in-app place search. Provider terms and quotas apply. Add keys through the
Google place search. Provider terms and quotas apply. Add keys through the
app's **POWER UP** panel; [Keys & Costs](#-api-keys) explains the options.
> **Already installed?** Update to the latest version. Older versions query
> public OpenStreetMap Overpass servers, which now refuse them, so Traffic,
> Mapped Installations and ALPR stay empty until you update.
### Path 1 — One click, no terminal
1. Install or update [Pinokio](https://desktop.pinokio.co/) to **8.2 or later**.
@@ -153,8 +163,8 @@ reopens the same panel.
- **What to get first:** the free [Cesium ion](https://cesium.com/ion) token
(eligible personal, non-commercial use; current terms and quotas apply) for
photorealistic 3D and world terrain; a Google Maps key only for the
billing-enabled, metered route + place search; OpenAI when you want to talk
to the world. Full map, costs included, in [Keys & Costs](#-api-keys).
billing-enabled, metered route + Google place search; OpenAI when you want to
talk to the world. Full map, costs included, in [Keys & Costs](#-api-keys).
<details>
<summary>Older Pinokio versions and credential storage</summary>
@@ -188,7 +198,7 @@ Choose a first-run mission, or try these in order. The GIFs show Google Photorea
![Moving from a full airport overhead down to close taxiway inspection with 3D flight models](docs/media/start-here/airport-ground-traffic-google-3d.gif)
4. **Look through a public camera.** Turn on **CCTV** over Austin, London, California, or Finland. The feeds aren't webcam embeds — they project _into_ the 3D city. Cycle coverage to **VIEWSHED** and every camera draws its estimated coverage volume — where it reaches, and where it goes blind.
4. **Look through a public camera.** Turn on **Cameras** over Austin, London, California, Finland, or Delaware (live video). The feeds aren't webcam embeds — they project _into_ the 3D city. Cycle coverage to **VIEWSHED** and every camera draws its estimated coverage volume — where it reaches, and where it goes blind.
![Diving into an Austin intersection with a live public camera projected into the 3D scene](docs/media/03-austin-cctv.gif)
@@ -203,7 +213,7 @@ Choose a first-run mission, or try these in order. The GIFs show Google Photorea
7. **Talk to it** _(needs an OpenAI key)_: _"Take me to LAX and select the nearest airborne aircraft."_
8. **Come home.** Hit **Reset Globe** — or just say _"zoom out to a globe view."_
**Keyboard:** `1`–`7` visual styles · `H` HUD · `D` detection · `C` cockpit · `Esc` out.
**Keyboard:** `1`–`7` visual styles · `H` HUD · `D` detection · `C` cockpit · `` ` `` frame rate · `Esc` out. Trackpad pinch zooms the globe.
---
@@ -235,7 +245,7 @@ Click **GEV MIC**, grant the microphone, and just talk. This is more than a voic
- **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director.
- **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token.
Twenty-eight tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook:
Twenty-nine tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook:
**🎥 Direct it** — drone-operator camera verbs:
@@ -253,7 +263,7 @@ Twenty-eight tools, four jobs — the commands below come straight from the prod
**🔎 Interrogate it** — analyst queries against the live layers:
> 🗣️ _"How many flights are over Texas right now?"_ · _"Which ships are headed to Oakland?"_ · _"What is the biggest fire near Los Angeles?"_ · _"Is anything flying above forty thousand feet?"_ · _"When does the ISS pass over next?"_
> 🗣️ _"How many flights are over Texas right now?"_ · _"Which ships are headed to Oakland?"_ · _"What is the biggest fire near Los Angeles?"_ · _"Is anything flying above forty thousand feet?"_ · _"When does the ISS pass over next?"_ · _"How many datacenters are in view?"_
**🎛️ Operate it** — the whole console, hands-free:
@@ -271,7 +281,7 @@ _Ask for radio near anywhere and the globe starts broadcasting — every station
## 🛰️ What's on the Globe
Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
Nineteen layers and map sources. **Seventeen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
| Layer | What you get | Source | Auth |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------- |
@@ -282,7 +292,8 @@ Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer add
| 🛰️ **Satellites** | 838-object catalog, color-coded by class with a live legend — the **DENSE** chip drops in the whole Starlink shell | CelesTrak | 🟢 |
| 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 |
| 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds |
| 📹 **CCTV Mesh** | ~3,600 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · New South Wales (Live Traffic NSW) · Calgary. Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
| 📹 **CCTV Mesh** | ~3,600 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary. Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
| 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 |
| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 |
| 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland | Operator GTFS-Realtime feeds | 🟢 |
| 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 |
@@ -290,6 +301,13 @@ Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer add
| 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 |
| 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🟡 optional token raises the allowance) |
| 🎖️ **Mapped Installations** | Viewport-bounded military-site context from community mapping — incomplete by nature, and labeled that way | OpenStreetMap | 🟢 |
| 🌬️ **Wind** | Animated 10 m forecast wind with an optional speed, temperature or pressure color field and a reading at map center | NOAA GFS / ECMWF IFS | 🟢 |
| 🌦️ **Observed Weather** | Rain radar (contiguous US), Satellite clouds (GOES regional or global infrared, Clouds only or Full image) and Lightning density (Americas and Pacific) on one history timeline | NOAA nowCOAST | 🟢 |
| 🌀 **Cyclones** | Advisory positions, forecast tracks and uncertainty cones for the Atlantic and eastern/central North Pacific | NOAA NHC / CPHC | 🟢 |
**Weather, keyless.** **Wind** animates 10 m forecast flow from NOAA GFS or ECMWF IFS, with an optional color field and a reading at map center. **Rain radar**, **Satellite clouds** and **Lightning density** are observations on one history timeline in the **WEATHER** panel (step back, **Play**, **Latest**); **Cyclone advisories** draw NHC and CPHC positions, forecast tracks and cones. On Google 3D Tiles the observed layers float above the city as translucent shells, sharpened around your view.
In the app, **Data Layers** groups them as Movement, Cameras, Infrastructure, Events, Weather and Utilities.
**The basemap ladder — what each tier buys you:**
@@ -297,7 +315,7 @@ Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer add
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 🟢 Nothing | Esri World Imagery satellite basemap + keyless terrain, in 2D. OSM takes over automatically if Esri is unreachable; if terrain is unavailable the globe continues without it |
| 🟡 A free Cesium ion token | **Google Photorealistic 3D cities** and world terrain — eligible personal, non-commercial use; current ion terms and quotas apply |
| 🔴 A Google Maps key | The same 3D direct from Google, plus in-app place search — the billing-enabled, metered route |
| 🔴 A Google Maps key | The same 3D direct from Google, plus Google place search — the billing-enabled, metered route |
![A reconstructed Falcon 9 ascent climbing and curving into its projected orbit](docs/media/08-falcon9-replay.gif)
@@ -364,7 +382,8 @@ src/
├── hud.js # Intelligence HUD + AI scene summary
├── keySetup.js # POWER UP panel — in-app provider keys (dev server only)
├── mapStackController.js # Basemap switching — Google 3D / Esri / OSM / ion stacks
├── voice/ # OpenAI Realtime session + 28 voice tools
├── voice/ # OpenAI Realtime session + 29 voice tools
├── layers/ # Layer components — weather, wind, cyclones, transit, ALPR, …
├── data/ # One module per layer + orchestration + context store
│ ├── iconOrientation.js # Screen-projected headings + horizon cull
│ └── local_data/ # Bundled datasets (per-folder provenance)
@@ -384,6 +403,11 @@ each provider enables; none is required to start. See the
[setup instructions](#then-power-it-up--in-the-app-not-in-a-file) for storage
and configuration details.
No key needed for Wind, Rain radar, Satellite clouds, Lightning density,
Cyclone advisories, Mapped ALPR Cameras, Transit, Directions, Delaware live
video, and coordinate or bundled-place search. A layer row waiting on a key
names it.
### Choose the capabilities you want
Six keys. Four have a free tier, and the two 🔴 ones are metered:
@@ -441,15 +465,35 @@ OpenSky can run fully anonymous (`OPENSKY_AUTH_MODE=anon`), or import OAuth cred
</details>
<details>
<summary>Live video cameras (HLS)</summary>
CCTV sources with `"feedType": "hls"` and a registered HTTP(S) `.m3u8`
URL play through a lazily loaded hls.js decoder shared by the monitor plane
and panel. DelDOT uses its official HTTPS HLS catalog links. Disable that
pack with `CCTV_DELDOT_ENABLED=0`.
The server allows two concurrent sessions. Each retains at most 12 segments
and 24 MiB in memory; individual downloads are capped at 4 MiB with a ten
second deadline. There are no segment files or ffmpeg processes. Redirects,
off-origin references, encrypted playlists and non-MPEG-TS segments are refused.
Each decoder has its own client lease (at most eight per session), including
native HLS. Closing it releases only that lease; abandoned leases expire after
15 seconds without access. The last release stops upstream work. Failed live video
uses the existing still/Street View/synthetic fallback, which is not live video.
RTMP-only sources are not supported by this integration.
</details>
### 💸 What it actually costs
Honest numbers, roughly, as of mid-2026 — always check the provider pricing pages:
| | Cost reality |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, bundled datasets. |
| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, transit feeds, OSRM routing, NOAA and ECMWF weather, OpenStreetMap ALPR mapping, bundled datasets. |
| **🟡 The free-key tier** | **$0 with a signup.** AISStream, FIRMS, TomTom, OpenSky, plus Cesium ion for eligible personal/non-commercial use. Provider quotas and eligibility still apply. |
| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for GEV place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. |
| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for Google place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. |
| **🔴 OpenAI voice** | **The one that costs real money — so the app meters it for you.** Realtime audio runs a few cents per active minute; an evening of heavy use is single-digit dollars. A live session-spend readout sits next to the mic, with an STD/MINI model toggle, a $2 warning, and a **$5 hard cap that ends the session**. The voice context window is kept deliberately short too. |
Google's direct 3D route is surprisingly generous: the first 1,000 Photorealistic
SECURITY.md
+2
View File
@@ -62,9 +62,11 @@ browser-key fallback for existing single-key setups.
The data proxies under `server/providers/` are written so the browser cannot turn the server into an open relay:
- **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). Other proxies target fixed upstream hosts.
- **Live HLS is bounded and origin-pinned.** Registered HTTP(S) HLS sources use a Node puller that rejects redirects and off-origin playlist references, bounds playlists and segment bodies while streaming, and aborts timed-out or released sessions. At most two sessions retain 12 segments/24 MiB each, plus bounded download buffers. Stores are memory-only and idle sessions expire after 15 seconds. RTMP/ffmpeg execution is not enabled. Locally configured source URLs remain an operator trust boundary.
- **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio.
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
- **Transit fetches registered feeds only.** `/api/transit/vehicles/<id>` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator.
- **Local receiver feeds are operator-configured and local-only.** `/api/local-receivers/aircraft` reads only the `aircraft.json` URLs in the server's `LOCAL_RECEIVER_FEEDS`; the browser never supplies an address. Each host must pass the shared tap address rule (`src/data/tapAddress.js`: loopback, RFC1918, `localhost`, `*.local`; no link-local, IPv6 or other names), the scheme must be http(s), the path must end in `aircraft.json`, and credentials, queries and fragments are refused. Invalid entries are logged at startup and never fetched. Reads use a 2 s timeout, refuse redirects, cap bodies at 2 MB and share one read per second; responses name each feed by band only and never carry upstream error text.
- **Response-size caps and timeouts** on proxied responses.
- **Sanitized errors** — internal error details are not echoed back to clients.
- **Coalesced OAuth refresh** and cached successful responses only (OpenSky).
TESTING.md
+12
View File
@@ -210,6 +210,18 @@ While recording, call out anything in these areas — this is the feedback I mos
happened? Any command it misunderstood?
- **Anything that looks wrong, janky, or surprising** — screenshot it; that's the gold.
---
## 6. Live video CCTV (HTTPS HLS) — soak, no mic
> Requires the DelDOT pack (loads by default; `CCTV_DELDOT_ENABLED=0` disables it).
1. Turn on **CCTV**, fly to Dover, DE, and activate **DE 1 @ Main Toll Plaza**. ✅ Within ~30 s the monitor plane and the panel preview both show moving video,.
2. Leave it for **five minutes**. ✅ Continuous playback; at most a short hitch about once a minute (the agency restarts its stream on a timer). ❌ A freeze that does not recover, or the panel picture stopping while the plane keeps moving.
3. Toggle **PROJECTION** off. ✅ The panel keeps playing; the plane and the active camera's cone hide.
4. Switch to **DE 8 @ Saulsbury Rd**, then back. ✅ Each switch resumes within ~10 s with no stale frame from the previous camera.
5. Optional: DevTools → Network, filter `media/`. ✅ Playlist responses carry `X-CCTV-Source: hls-pull`; segments return `video/mp2t`.
## If something looks off
- **Grey globe / slow tiles:** wait a few seconds after a camera flight; photoreal streams in.
THIRD_PARTY_NOTICES.md
+21
View File
@@ -0,0 +1,21 @@
# Third-party software notices
The project depends on third-party npm packages whose own licenses apply. This
file records the packages added for the browser-local SDR feature; the complete
resolved dependency inventory remains in `package-lock.json`.
## Web RTL-SDR
- Package: `@jtarrio/webrtlsdr` 3.0.6
- Author: Jacobo Tarrio Barreiro; portions copyright Google Inc.
- Source: <https://github.com/jtarrio/webrtlsdr>
- License: Apache License 2.0
- License text: <https://www.apache.org/licenses/LICENSE-2.0>
## Signals
- Package: `@jtarrio/signals` 0.10.1
- Author: Jacobo Tarrio Barreiro
- Source: <https://github.com/jtarrio/signals>
- License: Apache License 2.0
- License text: <https://www.apache.org/licenses/LICENSE-2.0>
build/vite.js
+15
View File
@@ -9,10 +9,25 @@ export function createBrowserViteConfig({
cesiumToken,
host = 'localhost',
port = 4173,
command,
} = {}) {
return {
plugins: [cesium(), applicationHtmlPlugin(), ...plugins],
...(publicDir === undefined ? {} : { publicDir }),
// A production build must not clean the dependency cache a running dev
// server is still serving optimized module URLs from.
...(command === 'build' ? { cacheDir: 'node_modules/.vite-build' } : {}),
optimizeDeps: {
// First reached through the SDR worker or a dynamic import. Pre-bundle
// them at startup so first use cannot invalidate already-transformed
// URLs with Vite's "Outdated Optimize Dep" 504 response.
include: [
'@jtarrio/signals/demod/demodulator.js',
'@jtarrio/signals/demod/modes.js',
'@jtarrio/webrtlsdr/rtlsdr.js',
'egm96-universal',
],
},
server: {
host: host || 'localhost',
port: parseInt(port, 10) || 4173,
docs/APPLICATION.md
+16 -9
View File
@@ -105,14 +105,17 @@ checks when changing this wiring.
`src/search` owns forward/reverse geocoding, place search and route provider
interfaces. `createStandaloneApplication({ geospatial })` accepts `endpoints`
and `providers`; the defaults preserve Google/Photon search, Google place context
and the local OSRM route proxy. Location, annotations, HUD labels and voice use
the composed service. Provider-specific credentials belong in the selected
transport; server secrets never belong in browser configuration.
and `providers`. Default forward search tries decimal coordinates and bundled
place names (`presets`) first, without a network request, then Google when a key
is configured, Photon, and the local `/api/geocode` Nominatim route. The
defaults also keep Google place context and the local OSRM route proxy.
Location, annotations, HUD labels and voice use the composed service.
Provider-specific credentials belong in the selected transport; server secrets
never belong in browser configuration.
For a compatible protocol, configure `geocode`, `photon`, `reverse`, `textSearch`,
`nearby` or `route` endpoints. These are developer-selected configuration, not
URLs accepted from page queries or model arguments. A different protocol supplies
For a compatible protocol, configure `geocode`, `photon`, `nominatim`, `reverse`,
`textSearch`, `nearby` or `route` endpoints. These are developer-selected
configuration, not URLs accepted from page queries or model arguments. A different protocol supplies
an adapter function instead. The exported `createDefaultPlaceSearch` constructor
supports the same options for direct composition:
@@ -181,10 +184,14 @@ separate constructors. Pass these to `createApplication`; use its `signal` and
`defer` for ownership. The compatibility catalog is still page-scoped: construct
one application per page. Controls, actions and renderers use the same layer
instances. Configure sources with `application/sources` before registration or
state restoration, and release them after the consumers stop.
state restoration, and release them after the consumers stop. The catalog
constructor (`application/layers`) also requires `wind`, `weather` and
`cyclones` sources with `getSnapshot` for the Weather layers, and owns the
observed-history clock those layers share for the catalog's lifetime.
`application/services` accepts boundary, terrain, regional-context, weather and
summary services. `application/requests` supplies the existing HTTP protocols
summary services; its weather service supplies cockpit local conditions, not the
Weather layers. `application/requests` supplies the existing HTTP protocols
with configurable endpoints and a scoped transport. Changing an endpoint works
only for a compatible protocol; another protocol supplies a service adapter.
Cancellation discards late response bodies, and replacing a source invalidates
docs/CODE-BOUNDARIES.md
+91 -3
View File
@@ -17,6 +17,8 @@ until deliberately adopted.
Generated output, local configuration, browser evidence and bundled datasets are
excluded. The formatter validates every entry before writing any file.
New world text labels must use the world-overlay host; Cesium text labels are forbidden.
## Current component ownership
Package imports use `gods-eye-view`; `package.json` is the authoritative
@@ -277,6 +279,90 @@ share restoration and movement of controls between containers remain caller-owne
Existing helper imports from `cockpitMath.js` and `rightRailPolicy.js` remain
compatible through re-exports.
### Adding a right-rail readout panel
Add the shell in `src/ui/templates/context.html` inside `#right-context-rail`:
a `.panel-collapsible` with a stable `id` and matching `data-panel-id`, the
existing `.panel-header`, collapse button and a dedicated body. Register it
with the existing panel chrome/layout owners; do not allocate rail space in
the body renderer. Set `hidden` while empty so the allocator ignores it, and
expand through the existing collapse button only on first appearance per page
session, and only when the chrome marked the restored state as a default
(`data-collapsed-preference`), never over a stored or shared choice. Preserve
the user's later collapse choice, including body remounts. Mark the body
`data-rail-scroller` when it scrolls: the rail's measuring pass lifts its
`max-height` and puts the scroll offset back afterwards. Panel bodies only
scroll on an open-card change, with one write, never on a refresh.
Compose `createRailCards({ container, document, onParams, onOpen })` and
`createRailTimeline({ container, document, onCommit, onPreview, onStep,
onLatest, onPlay })`. Cards reconcile `{ id, title, badge, open, compact,
compactStatus, blocks }` in place. Their full-width native button headers expose
`aria-expanded` and `aria-controls`; articles expose `data-open`. The feature
adapter owns accordion state. Generic cards do not select data or enable layers.
`blocks` is an ordered array of loose, optional `{ id, type, ... }` descriptors,
rendered by `railCardBlocks.js`. There are no per-section disclosures:
- `summary`: `text`, a single summary line.
- `list`: `list: { ariaLabel, items }`, reconciled by `syncRowList` in `rowList.js`.
- `lines`: keyed `lines: [{ id, text, muted? }]` for details and status.
- `settings`: `settings: [{ id, label, chips }]`; each labelled row has a 56 px
label column and a wrapping `syncChipGroup` from `chipGroup.js`.
- `legend`: `legend: { colors, labels, units?, zeroIndex?, categorical? }` for a
ramp/scale (including a physical freezing anchor) or separate keyed swatches.
- `actions`: keyed `actions: [{ id, label, params?, href?, onClick?, disabled?,
hint? }]` in a footer, using buttons or safe new-tab links.
- `result`: `label`, keyed `lines`, and optional `clear: { params | onClick }`;
its accessible × button sits beside the title.
Every block and control has a stable id. Lists, settings, actions and results
can appear in any requested order, including a scene panel's rectangle action,
results list, then selected-scene details. Chip/list/action `params` dispatch
through `onParams(cardId, params)`; the feature adapter supplies
`setLayerParams(id, params, { origin: 'user' })`. Current descriptors and disabled
state are read at dispatch. Identical updates do not move nodes or write DOM;
focused controls survive feed refreshes.
Weather descriptors retain top-level `chips`, `list` and `legend` for non-DOM
consumers. They expose `summary.settings`, `summary.actions` and optional
`summary.result`; wind also retains its raw captured `summary.reading`.
`readout: true` layer rows render only the toggle and source/meta line; their
subscriptions still refresh the cards. Cards own configuration and readings.
Weather orders cyclones, wind, then a bordered Observed history group containing
the timeline and active radar, satellite and lightning cards. The group heading
and scope name its active products. The timeline stays visible whenever any
observed product is enabled; fewer than two ticks disables transport. Pass
`heading: false` to the timeline when the containing group owns the heading.
Exactly one active card is open. Header clicks and newly enabled ids choose it
(the last new entry wins a batch); refreshes and clock ticks retain it. First
appearance uses cyclones with storms, otherwise the first card. Disabling the
open layer falls back to the first remaining card. A WeakMap per document keeps
explicit choices through body remounts without adding share state. Card opening,
storm selection and layer enabling remain independent.
Optional card/badge/slider class names provide feature styling. Card titles use
`.data-name` typography; meta/time/badges use `.data-toggle-meta` typography;
setting/result labels use `.panel-title`; chips retain `.data-toggle-chip`.
Reserve one status line, even without a legend. Use app tokens. Wind's footer
reads the map center into a result after the action. The captured location stays
fixed until another read; model changes resample that location, units reformat
it, and × clears both reading and map marker.
The timeline shows endpoint times and receives ticks, index, mode, playing,
readout and disabled state. Its preview callback receives the tick and index and
may return readout text; commits receive the same values after a 150 ms coalesced
drag or final change. Feature adapters own data mapping and service subscriptions;
generic modules own DOM/listeners/timers and release them in `destroy()`.
Keep persistence with panel chrome: collapse uses
`godsEyeView.v6.panelCollapsed.<panelId>`, position uses
`godsEyeView.v8.panelPos.<panelId>`, and share state uses the registered panel id.
Do not rename existing
ids or reset persistence versions when adding a body. The WEATHER adapter is
`src/ui/weatherPanel.js`; its observed-history selection remains transient.
## Visual input
`gods-eye-view/ui/input` exports `bindApplicationShortcuts` and
@@ -584,8 +670,8 @@ An empty search array is a definitive miss. HTTP failures, malformed results
and oversized responses remain retryable. Attribution identifies OpenStreetMap /
Nominatim.
Overpass currently supplies traffic road geometry, ALPR camera records, military
installation footprints, and annotation geometry (administrative boundaries,
Vector tile sources supply traffic roads, ALPR records and keyless military
areas. Operator-configured Overpass supplies named installations and annotation geometry (administrative boundaries,
neighborhoods, streets, building/grounds outlines and monument candidates).
Nominatim forward/reverse lookup does not replace those queries. Layer source
interfaces select traffic, camera and installation ingestion separately;
@@ -627,7 +713,9 @@ query-language strings or decoding backend tags and relation members.
Points use `{lat, lon}`. Options carry `signal`. A successful array, including an
empty array, is definitive; `null` means a transient failure;
`{rateLimited: true, retryAfterMs}` retains a provider's retry delay. Candidates
`{rateLimited: true, retryAfterMs}` retains a provider's retry delay.
`{unavailable:true, code:'OVERPASS_NOT_CONFIGURED', retryable:false}` is a
capability miss that stops retries while preserving pins and offline geometry. Candidates
carry `id`, `names` (`primary`, `english`, `official`, `alternate`, `short`),
`coordinates` (objects with `lat` and `lon`), `building` and optional `heightM`,
`center` and `point`. Administrative candidates use `category: 'administrative'`
docs/CURRENT-STATE.md
+987 -110
View File
File diff suppressed because it is too large. Load diff
docs/INFRASTRUCTURE-LAYERS.md
+2 -1
View File
@@ -36,7 +36,8 @@ const layers = createInfrastructureLayers({
| `governorRequestRender` | `(reason)`; request a frame through the existing render scheduler |
Each returned layer supports `init(viewer)`, `enable(viewer)`, `disable(viewer)`,
`update(viewer)`, `destroy(viewer)`, `getStats()` and `getLodDiagnostics()`.
`update(viewer)`, `destroy(viewer)`, `getStats()`, `getLodDiagnostics()` and
`getAnalystRecords()`.
Use the same viewer for the instance's entire lifetime. Each viewer/context/overlay
host permits one live instance per layer ID; destroy an old instance before
replacing it. Separate applications may use identical IDs with separate hosts.
docs/KNOWN-ISSUES.md
+62 -3
View File
@@ -1,6 +1,6 @@
# KNOWN ISSUES
Updated: July 8, 2026
Updated: September 22, 2026
This file tracks active runtime issues only.
@@ -78,6 +78,56 @@ Status: Open (owner-accepted 2026-07-08, documented)
---
### Weather layers: coverage and meaning
Status: Open (source limits, by design)
- **Rain radar covers the contiguous United States only.** It shows MRMS
reflectivity in dBZ, not rainfall rate or a forecast; a gap in coverage does not
mean no precipitation.
- **Lightning density is a ground-network grid, not GLM flashes.** It is NOAA's
15-minute density product derived from Vaisala NLDN/GLD360 on an approximately
8 km grid, for the Americas and Pacific only (110°E across the dateline to 0°,
25°S–80°N). It is not a live strike counter or an all-clear.
- **Cyclone advisories cover NOAA's basins only:** the Atlantic and the
eastern/central North Pacific (NHC and CPHC). Storms elsewhere do not appear.
The cone is forecast-center uncertainty, not storm size.
- **Satellite clouds:** global infrared is hourly with typically 2–3 hours of
source latency. Clouds only is a brightness filter, not a cloud mask.
- **Wind is a forecast.** The animation flows through one forecast on an
approximately 1° grid and does not advance forecast time or follow the
observed-history timeline.
- **History is not shared.** Share links carry the weather layers and their
settings but open at Latest.
- **3D Tiles detail window during history playback:** when a frame's detail image
is slow to arrive, the window may fall back to the coarser full-extent image
until it does.
- Native hardware GPU behavior of the wind and weather renderers is not yet
verified.
---
### Satellite passes and analyst answers
Status: Open (by design)
- Pass predictions search the next 24 hours for satellites in the loaded catalog.
Visibility is a geometric estimate (satellite sunlit, observer's Sun at or
below −6°); it ignores weather, satellite brightness and orbital-element age.
- Analyst counts and ranks cover a bounded set of loaded records (by default
2,000 per layer for satellites, datacenters and dams), so omitted records can
change a nearest or count answer. Satellite distance is ground distance.
---
### Live camera video (HLS)
Status: Open (limits)
- At most two live-video sessions run at once. RTMP-only sources and encrypted,
fMP4 or byte-range playlists are not supported.
- When live video fails, the camera falls back to its labeled still, Street View
or placeholder frame, which is not live video.
---
## Closed / Intentional (for clarity)
### Proxy SSRF and error-surface hardening gaps
@@ -118,5 +168,14 @@ Context:
converted to **live NASA FIRMS data** on 2026-07-16: the `/api/firms` proxy merges
three VIIRS NRT sources (trailing 24 h, 30 min cache, serve-stale-on-failure) and the
bundled snapshot was deleted. Requires a free server-side `FIRMS_MAP_KEY`; without it
the layer shows a KEY REQUIRED state.
- Weather radar is still held out of OSS v1 after QA found the previous overlay did not provide reliable visible value.
the layer row reads "Needs FIRMS_MAP_KEY — add it in Provider Settings".
---
### Weather radar held out of the open-source release
Status: Closed — observed rain radar shipped (September 2026)
Context:
- Rain radar, Satellite clouds, Lightning density, Wind and Cyclone advisories
are in the Weather group of Data Layers. Their current limits are listed under
"Weather layers: coverage and meaning" above.
docs/LOCAL-RECEIVERS.md
+82
View File
@@ -0,0 +1,82 @@
# Local receivers
The Local ADS-B layer draws aircraft heard by your own receivers. It has two
inputs, and it can use both at once.
| Input | What it is | Bands |
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| Browser SDR | A USB RTL-SDR opened by the browser through WebUSB (Radio panel, Local RTL-SDR card). Decoding runs in the page. | 1090 MHz |
| Decoder feed | The `aircraft.json` served by a decoder you run: dump1090-fa, readsb or tar1090 (1090 MHz), or dump978-fa + skyaware978 (978 MHz). | 1090 MHz, 978 MHz UAT |
Aircraft from both inputs are merged by ICAO address. The newest position
wins, and the click card names every band and source that heard the aircraft
in the last 60 s. Aircraft heard only on 978 MHz UAT draw with a thin ring.
Local aircraft usually lead their public twins. The public Flights layer
renders positions about one poll interval (~30 s) behind so it can interpolate
smoothly between fixes, while local receiver aircraft are shown as heard. An
aircraft in both layers therefore draws twice, the magenta local marker ahead
of the public one; both markers are drawn on purpose.
## Which to use
- **One plain RTL-SDR dongle, desktop Chrome or Edge:** use the browser SDR.
Nothing else to install.
- **A dual-band board, a Raspberry Pi feeder, or a dongle already used by a
decoder:** use a decoder feed. A USB dongle can be opened by only one program
at a time, so a dongle that dump1090 or dump978 owns cannot also be opened by
the browser.
## Configuring feeds
Feeds are set in the server environment only (`.env`), never in the browser:
```sh
LOCAL_RECEIVER_FEEDS=1090=http://localhost:8080/data/aircraft.json,978=http://localhost:8978/data/aircraft.json
```
Each entry is `band=url`, where `band` is `1090` or `978`. The URL must:
- use `http` or `https`;
- point at loopback, a private (RFC1918) address, `localhost` or a `*.local`
name. Public addresses, link-local addresses (including `169.254.169.254`),
IPv6 and other host names are refused;
- have a path ending in `aircraft.json`, with no credentials, query or
fragment.
An entry that breaks a rule is logged when the server starts, shown as
`invalid` and never fetched. Restart the server after changing the value.
The server reads every feed about once a second while the Local ADS-B layer is
on. A feed is `live`, `stale` (its own `now` is more than 10 s old, usually a
stopped decoder), `unreachable` or `invalid`. The Layers panel row and the
Local RTL-SDR card show these states.
## Example: dump1090-fa and dump978-fa on macOS or Linux
Write each decoder's JSON to a directory, then serve the directory on
localhost. Each command keeps running, so start each in its own terminal:
```sh
# 1090 MHz
dump1090-fa --device-index 0 --write-json /tmp/adsb1090
python3 -m http.server 8080 --bind 127.0.0.1 --directory /tmp/adsb1090
# 978 MHz UAT: dump978-fa decodes, skyaware978 writes aircraft.json
dump978-fa --sdr driver=rtlsdr,rtl=1 --raw-port 30978
skyaware978 --connect localhost:30978 --json-dir /tmp/adsb978
python3 -m http.server 8978 --bind 127.0.0.1 --directory /tmp/adsb978
```
```sh
LOCAL_RECEIVER_FEEDS=1090=http://localhost:8080/aircraft.json,978=http://localhost:8978/aircraft.json
```
A board with two identical-serial channels, such as the Nooelec FlyCatcher,
cannot be told apart by serial number. Select each channel by index:
`--device-index <n>` for dump1090-fa and `--sdr driver=rtlsdr,rtl=<n>` for
dump978-fa. `rtl_test` lists the indexes.
A tar1090 or readsb install on a Raspberry Pi already serves
`http://<pi-address>/tar1090/data/aircraft.json`. Use the Pi's private address
or its `.local` name.
package-lock.json
Generated
+50
View File
@@ -9,9 +9,13 @@
"version": "0.1.1",
"license": "MIT",
"dependencies": {
"@jtarrio/signals": "^0.10.0",
"@jtarrio/webrtlsdr": "^3.0.6",
"@mapbox/vector-tile": "^3.0.0",
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"hls.js": "^1.7.3",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
"satellite.js": "^6.0.2"
@@ -1059,6 +1063,22 @@
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@jtarrio/signals": {
"version": "0.10.1",
"resolved": "https://registry.npmjs.org/@jtarrio/signals/-/signals-0.10.1.tgz",
"integrity": "sha512-X6tJOugEATYH9KJy6j2feEBVCUmRv1U9SSbg9ITQZCx/Qruw2fE2zsS1HKH3yOsxrZBE+eQTGM+d0roX4xuDdw==",
"license": "Apache-2.0"
},
"node_modules/@jtarrio/webrtlsdr": {
"version": "3.0.6",
"resolved": "https://registry.npmjs.org/@jtarrio/webrtlsdr/-/webrtlsdr-3.0.6.tgz",
"integrity": "sha512-+P8dsbXQ/6Mwru/vV613yHH68nHFyUJ9zPK68hE/Hh2F3jwyirUAjjBtIAfN/MlJbFioMJ3NS6dMGrRj9deFvw==",
"license": "Apache-2.0",
"dependencies": {
"@jtarrio/signals": "^0.10.0",
"@types/w3c-web-usb": "^1.0.10"
}
},
"node_modules/@mapbox/point-geometry": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@mapbox/point-geometry/-/point-geometry-1.1.0.tgz",
@@ -1076,6 +1096,24 @@
"pbf": "^5.0.0"
}
},
"node_modules/@meri-imperiumi/eccodes-wasm": {
"version": "2.48.2",
"resolved": "https://registry.npmjs.org/@meri-imperiumi/eccodes-wasm/-/eccodes-wasm-2.48.2.tgz",
"integrity": "sha512-Z7rD3xInomPG7FS97ORD1UGKAdgeeRP9ofa9k4m8d+k2sDfDQeAjTzvs9UOndyaZovaSbEQeIoNX8gTmQNhiSA==",
"cpu": [
"x64",
"arm64"
],
"license": "Apache-2.0",
"os": [
"darwin",
"linux",
"win32"
],
"engines": {
"node": ">=24.0.0"
}
},
"node_modules/@puppeteer/browsers": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/@puppeteer/browsers/-/browsers-3.2.2.tgz",
@@ -1518,6 +1556,12 @@
"license": "MIT",
"optional": true
},
"node_modules/@types/w3c-web-usb": {
"version": "1.0.14",
"resolved": "https://registry.npmjs.org/@types/w3c-web-usb/-/w3c-web-usb-1.0.14.tgz",
"integrity": "sha512-Qu3Nn6JFuF4+sHKYl+IcX9vYiI40ogleXzFFSxoE1W94rG98o/kXs8uJ0QSfFzuwBCZWlGfUGpPkgwuuX4PchA==",
"license": "MIT"
},
"node_modules/@zip.js/zip.js": {
"version": "2.8.18",
"resolved": "https://registry.npmjs.org/@zip.js/zip.js/-/zip.js-2.8.18.tgz",
@@ -1944,6 +1988,12 @@
"integrity": "sha512-bzh50DW9kTPM00T8y4o8vQg89Di9oLJVLW/KaOGIXJWP/iqCN6WKYkbNOF04vFLJhwcpYUh9ydh/+5vpOqV4YQ==",
"license": "MIT"
},
"node_modules/hls.js": {
"version": "1.7.3",
"resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.7.3.tgz",
"integrity": "sha512-MsPlx6yVW4Qv4C7mEVou4/gk/5cN2dVxOTnrNjPSmyH2f0Ln+/UkG10Ax1PE6OvrmTVPKgQYozTtBaPOuCzq7A==",
"license": "Apache-2.0"
},
"node_modules/http-errors": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz",
package.json
+10 -1
View File
@@ -2,7 +2,7 @@
"name": "gods-eye-view",
"private": true,
"version": "0.1.1",
"description": "A real-time intelligence console for planet Earth \u2014 photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
"description": "A real-time intelligence console for planet Earth — photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
"type": "module",
"license": "MIT",
"author": "Bilawal Sidhu",
@@ -35,6 +35,8 @@
"doctor": "node scripts/setup-doctor.mjs",
"format": "node scripts/format.mjs --write",
"format:check": "node scripts/format.mjs --check",
"layer-token:next": "node scripts/next-layer-state-token.mjs",
"layer-token:check": "node scripts/check-layer-state-tokens.mjs",
"dev": "vite",
"dev:secure": "./scripts/dev-secure.sh",
"opensky:import": "./scripts/opensky-import-client.sh",
@@ -47,9 +49,13 @@
"qa:transit": "node scripts/qa-transit.mjs"
},
"dependencies": {
"@jtarrio/signals": "^0.10.0",
"@jtarrio/webrtlsdr": "^3.0.6",
"@mapbox/vector-tile": "^3.0.0",
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"hls.js": "^1.7.3",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
"satellite.js": "^6.0.2"
@@ -150,6 +156,8 @@
"./aircraft": "./src/layers/aircraft/index.js",
"./layers/vessels": "./src/layers/vessels/index.js",
"./layers/earthquakes": "./src/layers/earthquakes/index.js",
"./layers/recent-imagery": "./src/layers/recentImagery/index.js",
"./layers/perimeters": "./src/layers/perimeters/index.js",
"./sources/firms-csv": "./src/data/firmsCsv.js",
"./layers/firms": "./src/layers/firms/index.js",
"./layers/satellites": "./src/layers/satellites/index.js",
@@ -221,6 +229,7 @@
"./layers/bikeshare/source": "./src/layers/bikeshare/source.js",
"./layers/cctv/source": "./src/layers/cctv/source.js",
"./layers/earthquakes/source": "./src/layers/earthquakes/source.js",
"./layers/perimeters/source": "./src/layers/perimeters/source.js",
"./layers/firms/source": "./src/layers/firms/source.js",
"./layers/installations/source": "./src/layers/installations/source.js",
"./layers/launches/source": "./src/layers/launches/source.js",
scripts/build-admin-packs.mjs
+759
View File
@@ -0,0 +1,759 @@
#!/usr/bin/env node
/**
* Build the bundled administrative-boundary packs read by
* `src/data/adminBoundaries.js`:
*
* src/data/local_data/natural_earth/states_provinces.json
* Natural Earth 10m admin-1 states and provinces (public domain).
* src/data/local_data/us_census_counties/counties.json
* US Census Bureau cartographic boundary counties (public domain).
*
* Both sources are pinned by URL and SHA-256, so a rerun reproduces the packs
* byte for byte. Downloads are cached outside the repository (default: the
* system temp directory) and are never committed.
*
* Usage:
* node scripts/build-admin-packs.mjs [--cache <dir>] [--only countries|admin1|counties]
*
* The provenance READMEs beside each pack record these parameters.
*/
import { createHash } from 'node:crypto';
import { mkdir, readFile, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { inflateRawSync } from 'node:zlib';
import { normalizeAdminName } from '../src/data/adminBoundaries.js';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const NE_COMMIT = 'ca96624a56bd078437bca8184e78163e5039ad19';
const NE_BASE = `https://raw.githubusercontent.com/nvkelso/natural-earth-vector/${NE_COMMIT}/geojson`;
export const SOURCES = Object.freeze({
countries: {
url: `${NE_BASE}/ne_10m_admin_0_countries.geojson`,
sha256: '239eec57ac17f100a11e2536cffc56752c318b50ae765b0918ff7aab4ce8f255',
},
mapUnits: {
url: `${NE_BASE}/ne_10m_admin_0_map_units.geojson`,
sha256: '57da82be755f4afccd8f3b14251bb2752f5df1395f47d2d86f817470c4a48862',
},
admin1: {
url: `${NE_BASE}/ne_10m_admin_1_states_provinces.geojson`,
sha256: '22d0e3ad85eb3e27f17cabf8ba2d50e554fbc27a87796ff891d958185da62fb5',
},
places: {
url: `${NE_BASE}/ne_10m_populated_places_simple.geojson`,
sha256: 'fd3fa867a320cbd5c5b6bb5bc550afeec2939fb2cef688e508007282a55ac42f',
},
counties: {
url: 'https://www2.census.gov/geo/tiger/GENZ2025/shp/cb_2025_us_county_5m.zip',
sha256: 'faec522080681e79be5be435c981009a77891206ff8a7f1d142f3bf5da9ebd74',
},
});
/**
* Pack parameters. The Douglas-Peucker tolerance scales with the unit's size
* (`toleranceFactor` × √area, clamped to `[minToleranceDeg, maxToleranceDeg]`)
* because a unit is viewed at a zoom that fits it: Texas keeps ~2 km detail,
* Bavaria ~1 km, Travis County ~150 m. Parts smaller than `minPartKm2` or than
* `minPartShare` of the largest part are dropped (the largest part always
* stays); holes under `minHoleKm2` are dropped.
*/
export const PARAMS = Object.freeze({
countries: {
toleranceFactor: 0.004,
minToleranceDeg: 0.005,
maxToleranceDeg: 0.02,
decimals: 3,
minPartKm2: 20,
minPartShare: 0.00003,
minHoleKm2: 20,
},
admin1: {
toleranceFactor: 0.004,
minToleranceDeg: 0.005,
maxToleranceDeg: 0.02,
decimals: 3,
minPartKm2: 20,
minPartShare: 0.0003,
minHoleKm2: 20,
},
counties: {
toleranceFactor: 0.003,
minToleranceDeg: 0.0005,
maxToleranceDeg: 0.01,
decimals: 4,
minPartKm2: 1,
minPartShare: 0.001,
minHoleKm2: 1,
},
// A bare admin-1 name is marked ambiguous — the resolver then lets a
// geocoder decide — when it is also a country, the name of a city of at
// least `insideMinPopulation` inside the unit's box ("New York", "Madrid"),
// or the name of a city of at least `elsewhereMinPopulation` outside it
// ("Santa Fe", "Victoria"). A unit no larger than `cityStateMaxKm2` whose
// own city lies inside is the city itself (Berlin, Vienna, Paris): it is
// marked `cityState` and may be named alone however small.
ambiguity: {
insideMinPopulation: 200_000,
elsewhereMinPopulation: 50_000,
cityStateMaxKm2: 1_000,
},
});
const args = process.argv.slice(2);
const cacheDir = path.resolve(
args.includes('--cache')
? args[args.indexOf('--cache') + 1]
: path.join(os.tmpdir(), 'gev-admin-packs'),
);
const only = args.includes('--only') ? args[args.indexOf('--only') + 1] : null;
async function fetchPinned(name) {
const { url, sha256 } = SOURCES[name];
const file = path.join(cacheDir, path.basename(new URL(url).pathname));
let bytes = await readFile(file).catch(() => null);
if (!bytes) {
console.log(`Downloading ${url}`);
const response = await fetch(url);
if (!response.ok) throw new Error(`HTTP ${response.status} for ${url}`);
bytes = Buffer.from(await response.arrayBuffer());
await mkdir(cacheDir, { recursive: true });
await writeFile(file, bytes);
}
const digest = createHash('sha256').update(bytes).digest('hex');
if (digest !== sha256)
throw new Error(
`${name}: SHA-256 ${digest} does not match the pinned ${sha256}`,
);
return { bytes, digest };
}
// ── geometry ────────────────────────────────────────────────────────────
const EARTH_RADIUS_KM = 6371;
const toRad = (d) => (d * Math.PI) / 180;
/** Spherical ring area (km²); matches src/data/naturalEarthRegions.js. */
function ringAreaKm2(ring) {
const n = ring.length;
if (n < 3) return 0;
let sum = 0;
for (let i = 0; i < n; i++) {
const [lon1, lat1] = ring[i];
const [lon2, lat2] = ring[(i + 1) % n];
sum +=
toRad(lon2 - lon1) * (2 + Math.sin(toRad(lat1)) + Math.sin(toRad(lat2)));
}
return Math.abs((sum * EARTH_RADIUS_KM * EARTH_RADIUS_KM) / 2);
}
/** Signed planar (shoelace) area: positive counter-clockwise in lon/lat. */
function signedArea(ring) {
let sum = 0;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
sum += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
return sum / 2;
}
function segDist(p, a, b) {
let [x, y] = p;
const [x1, y1] = a;
const [x2, y2] = b;
const dx = x2 - x1;
const dy = y2 - y1;
if (dx !== 0 || dy !== 0) {
const t = ((x - x1) * dx + (y - y1) * dy) / (dx * dx + dy * dy);
if (t > 1) {
x -= x2;
y -= y2;
return Math.hypot(x, y);
}
if (t > 0) {
x -= x1 + dx * t;
y -= y1 + dy * t;
return Math.hypot(x, y);
}
}
return Math.hypot(x - x1, y - y1);
}
/** Iterative Douglas-Peucker over an open point list. */
function douglasPeucker(points, tolerance) {
if (points.length <= 2) return points.slice();
const keep = new Uint8Array(points.length);
keep[0] = keep[points.length - 1] = 1;
const stack = [[0, points.length - 1]];
while (stack.length) {
const [first, last] = stack.pop();
let maxDist = 0;
let index = -1;
for (let i = first + 1; i < last; i++) {
const d = segDist(points[i], points[first], points[last]);
if (d > maxDist) {
maxDist = d;
index = i;
}
}
if (maxDist > tolerance && index !== -1) {
keep[index] = 1;
stack.push([first, index], [index, last]);
}
}
return points.filter((_, i) => keep[i]);
}
/**
* Simplify a closed ring to an OPEN rounded ring. The ring is split at its
* farthest vertex from the start so Douglas-Peucker has two anchored halves
* (a closed ring has no chord to measure against). Returns null when fewer
* than four distinct vertices survive.
*/
function simplifyRing(ring, tolerance, decimals) {
let open = ring.slice();
const first = open[0];
const last = open[open.length - 1];
if (open.length > 1 && first[0] === last[0] && first[1] === last[1])
open = open.slice(0, -1);
if (open.length < 4) return null;
let far = 1;
let farDist = -1;
for (let i = 1; i < open.length; i++) {
const d = Math.hypot(open[i][0] - first[0], open[i][1] - first[1]);
if (d > farDist) {
farDist = d;
far = i;
}
}
const a = douglasPeucker(open.slice(0, far + 1), tolerance);
const b = douglasPeucker([...open.slice(far), first], tolerance);
const simplified = [...a, ...b.slice(1, -1)];
const factor = 10 ** decimals;
const out = [];
for (const [lon, lat] of simplified) {
const p = [
Math.round(lon * factor) / factor,
Math.round(lat * factor) / factor,
];
const prev = out[out.length - 1];
if (prev && prev[0] === p[0] && prev[1] === p[1]) continue;
out.push(p);
}
while (
out.length > 1 &&
out[0][0] === out[out.length - 1][0] &&
out[0][1] === out[out.length - 1][1]
)
out.pop();
return out.length >= 4 ? out : null;
}
/**
* Simplify polygons ([outer, ...holes][]): drop parts and holes under the
* area floors (the largest part always stays), orient outers
* counter-clockwise and holes clockwise, sort parts by area descending.
*/
function simplifyPolygons(polygons, params, tolerance, decimals) {
const parts = [];
for (const poly of polygons) {
const outer = simplifyRing(poly[0], tolerance, decimals);
if (!outer) continue;
const area = ringAreaKm2(outer);
if (signedArea(outer) < 0) outer.reverse();
const holes = [];
for (const hole of poly.slice(1)) {
const ring = simplifyRing(hole, tolerance, decimals);
if (!ring || ringAreaKm2(ring) < params.minHoleKm2) continue;
if (signedArea(ring) > 0) ring.reverse();
holes.push(ring);
}
parts.push({ rings: [outer, ...holes], area });
}
parts.sort((x, y) => y.area - x.area);
const floor = Math.max(
params.minPartKm2,
(parts[0]?.area || 0) * params.minPartShare,
);
return parts
.filter((part, index) => index === 0 || part.area >= floor)
.map((part) => part.rings);
}
/**
* Simplify one unit at its size-scaled tolerance. A unit too small to
* survive the pack precision (a Maldives atoll) is kept at one more decimal
* and a fifth of the minimum tolerance, or failing that unsimplified at two
* more decimals; the caller records `d`.
*/
function simplifyUnit(polygons, params) {
const sourceArea = polygons.reduce(
(sum, poly) => sum + ringAreaKm2(poly[0]),
0,
);
const tolerance = Math.min(
params.maxToleranceDeg,
Math.max(
params.minToleranceDeg,
(params.toleranceFactor * Math.sqrt(sourceArea)) / 111.32,
),
);
const coarse = simplifyPolygons(polygons, params, tolerance, params.decimals);
if (coarse.length) return { polygons: coarse, decimals: params.decimals };
const fine = simplifyPolygons(
polygons,
params,
params.minToleranceDeg / 5,
params.decimals + 1,
);
if (fine.length) return { polygons: fine, decimals: params.decimals + 1 };
// Smaller still (Vatican City): the source ring as drawn.
const exact = simplifyPolygons(polygons, params, 0, params.decimals + 2);
return { polygons: exact, decimals: params.decimals + 2 };
}
/**
* Encode a rounded open ring as integers in units of 10^-decimals degrees:
* the first vertex absolute, every later vertex as a delta from the previous
* one — [lon0, lat0, dLon1, dLat1, …]. Decoded by `decodeRing` in
* src/data/adminBoundaries.js.
*/
function encodeRing(ring, decimals) {
const factor = 10 ** decimals;
const out = [];
let px = 0;
let py = 0;
for (const [lon, lat] of ring) {
const x = Math.round(lon * factor);
const y = Math.round(lat * factor);
out.push(x - px, y - py);
px = x;
py = y;
}
return out;
}
function encodeUnit({ polygons, decimals }) {
return polygons.map((poly) => poly.map((ring) => encodeRing(ring, decimals)));
}
function geojsonPolygons(geometry) {
if (geometry?.type === 'Polygon') return [geometry.coordinates];
if (geometry?.type === 'MultiPolygon') return geometry.coordinates;
return [];
}
function bboxOf(polygons) {
let w = Infinity;
let s = Infinity;
let e = -Infinity;
let n = -Infinity;
for (const poly of polygons)
for (const [lon, lat] of poly[0]) {
if (lon < w) w = lon;
if (lon > e) e = lon;
if (lat < s) s = lat;
if (lat > n) n = lat;
}
return [w, s, e, n];
}
// ── zip / shapefile / dbf ───────────────────────────────────────────────
/** Read named entries from a zip archive (stored or deflated). */
function unzip(buffer) {
let eocd = buffer.length - 22;
while (eocd >= 0 && buffer.readUInt32LE(eocd) !== 0x06054b50) eocd--;
if (eocd < 0) throw new Error('zip: end of central directory not found');
const count = buffer.readUInt16LE(eocd + 10);
let offset = buffer.readUInt32LE(eocd + 16);
const files = new Map();
for (let i = 0; i < count; i++) {
if (buffer.readUInt32LE(offset) !== 0x02014b50)
throw new Error('zip: bad central directory entry');
const method = buffer.readUInt16LE(offset + 10);
const compressed = buffer.readUInt32LE(offset + 20);
const nameLength = buffer.readUInt16LE(offset + 28);
const extraLength = buffer.readUInt16LE(offset + 30);
const commentLength = buffer.readUInt16LE(offset + 32);
const local = buffer.readUInt32LE(offset + 42);
const name = buffer.toString('utf8', offset + 46, offset + 46 + nameLength);
const dataStart =
local +
30 +
buffer.readUInt16LE(local + 26) +
buffer.readUInt16LE(local + 28);
const data = buffer.subarray(dataStart, dataStart + compressed);
if (method === 0) files.set(name, data);
else if (method === 8) files.set(name, inflateRawSync(data));
else throw new Error(`zip: unsupported method ${method} for ${name}`);
offset += 46 + nameLength + extraLength + commentLength;
}
return files;
}
/** Polygon records (shape type 5) as arrays of closed [lon, lat] rings. */
function readShp(buffer) {
const shapes = [];
let offset = 100;
while (offset < buffer.length) {
const length = buffer.readInt32BE(offset + 4) * 2;
const content = offset + 8;
const type = buffer.readInt32LE(content);
const rings = [];
if (type === 5) {
const numParts = buffer.readInt32LE(content + 36);
const numPoints = buffer.readInt32LE(content + 40);
const partsAt = content + 44;
const pointsAt = partsAt + numParts * 4;
for (let p = 0; p < numParts; p++) {
const start = buffer.readInt32LE(partsAt + p * 4);
const end =
p + 1 < numParts
? buffer.readInt32LE(partsAt + (p + 1) * 4)
: numPoints;
const ring = [];
for (let i = start; i < end; i++)
ring.push([
buffer.readDoubleLE(pointsAt + i * 16),
buffer.readDoubleLE(pointsAt + i * 16 + 8),
]);
rings.push(ring);
}
} else if (type !== 0) {
throw new Error(`shp: unsupported shape type ${type}`);
}
shapes.push(rings);
offset = content + length;
}
return shapes;
}
/** dBASE records as plain objects (character and numeric fields). */
function readDbf(buffer) {
const count = buffer.readUInt32LE(4);
const headerLength = buffer.readUInt16LE(8);
const recordLength = buffer.readUInt16LE(10);
const fields = [];
for (let at = 32; buffer[at] !== 0x0d; at += 32) {
fields.push({
name: buffer.toString('latin1', at, at + 11).replace(/\0.*$/, ''),
length: buffer[at + 16],
});
}
const records = [];
for (let r = 0; r < count; r++) {
let at = headerLength + r * recordLength + 1;
const record = {};
for (const field of fields) {
record[field.name] = buffer
.toString('utf8', at, at + field.length)
.trim();
at += field.length;
}
records.push(record);
}
return records;
}
/** Group shapefile rings into polygons: clockwise outers, holes assigned by containment. */
function shpPolygons(rings) {
const outers = [];
const holes = [];
for (const ring of rings) {
if (ring.length < 4) continue;
(signedArea(ring) < 0 ? outers : holes).push(ring);
}
const polygons = outers.map((outer) => [outer]);
for (const hole of holes) {
const [x, y] = hole[0];
const owner = polygons.find(([outer]) => pointInRing(outer, x, y));
if (owner) owner.push(hole);
else polygons.push([hole.slice().reverse()]);
}
return polygons;
}
function pointInRing(ring, x, y) {
let inside = false;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++) {
const [xi, yi] = ring[i];
const [xj, yj] = ring[j];
if (yi > y !== yj > y && x < ((xj - xi) * (y - yi)) / (yj - yi) + xi)
inside = !inside;
}
return inside;
}
// ── packs ────────────────────────────────────────────────────────────────
const LATIN = /^[\p{Script=Latin}\p{N}\s.,'’()\-/&]+$/u;
function uniqueNames(values, exclude) {
const seen = new Set(exclude.map(normalizeAdminName));
const out = [];
for (const value of values) {
const text = String(value || '').trim();
const key = normalizeAdminName(text);
if (!text || !key || seen.has(key) || !LATIN.test(text)) continue;
seen.add(key);
out.push(text);
}
return out;
}
async function buildCountries() {
const source = await fetchPinned('countries');
const mapUnits = await fetchPinned('mapUnits');
const params = PARAMS.countries;
const features = [];
for (const { properties: p, geometry } of JSON.parse(source.bytes).features) {
const unit = simplifyUnit(geojsonPolygons(geometry), params);
if (!unit.polygons.length) continue;
const name = p.ADMIN || p.NAME_EN || p.NAME;
features.push({
name,
alt: uniqueNames(
[p.NAME, p.NAME_LONG, p.NAME_EN, p.FORMAL_EN, p.BRK_NAME].filter(
Boolean,
),
[name],
),
country: name,
iso: p.ADM0_A3,
iso2: p.ISO_A2_EH || p.ISO_A2,
type: 'Country',
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
}
for (const { properties: p, geometry } of JSON.parse(mapUnits.bytes)
.features) {
if (p.ADM0_A3 !== 'GBR') continue;
const unit = simplifyUnit(geojsonPolygons(geometry), params);
features.push({
name: p.GEOUNIT,
country: p.ADMIN,
iso: p.GU_A3,
iso2: 'GB',
type: 'Constituent country',
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
}
features.sort((a, b) => a.name.localeCompare(b.name));
return {
meta: {
title: 'Natural Earth admin-0 countries',
mapUnits: SOURCES.mapUnits,
source: 'ne_10m_admin_0_countries',
url: SOURCES.countries.url,
sha256: source.digest,
commit: NE_COMMIT,
license:
'Public domain (https://www.naturalearthdata.com/about/terms-of-use/)',
decimals: params.decimals,
curation: params,
script: 'scripts/build-admin-packs.mjs',
},
features,
};
}
async function buildAdmin1() {
const params = PARAMS.admin1;
const source = await fetchPinned('admin1');
const places = await fetchPinned('places');
const collection = JSON.parse(source.bytes.toString('utf8'));
const { ambiguity } = PARAMS;
const cities = JSON.parse(places.bytes.toString('utf8'))
.features.map((f) => f.properties)
.filter((p) => p.pop_max >= ambiguity.elsewhereMinPopulation)
.map((p) => ({
keys: new Set([p.name, p.nameascii].map(normalizeAdminName)),
lon: p.longitude,
lat: p.latitude,
population: p.pop_max,
country: p.adm0name,
}));
const countryNames = new Set();
for (const { properties: p } of collection.features)
for (const name of [p.admin, p.geonunit])
if (name) countryNames.add(normalizeAdminName(name));
const features = [];
for (const { properties: p, geometry } of collection.features) {
const name = String(p.name || '').trim();
if (!name) continue;
const unit = simplifyUnit(geojsonPolygons(geometry), params);
if (!unit.polygons.length) continue;
const nameEn =
p.name_en && normalizeAdminName(p.name_en) !== normalizeAdminName(name)
? String(p.name_en).trim()
: null;
const alt = uniqueNames(
String(p.name_alt || '').split('|'),
[name, nameEn].filter(Boolean),
);
const bbox = bboxOf(unit.polygons);
const areaKm2 = unit.polygons.reduce(
(sum, poly) => sum + ringAreaKm2(poly[0]),
0,
);
const keys = new Set(
[name, nameEn, ...alt].filter(Boolean).map(normalizeAdminName),
);
const amb = [];
const ambAbroad = [];
let cityState = false;
const inBox = (c) =>
c.lon >= bbox[0] &&
c.lon <= bbox[2] &&
c.lat >= bbox[1] &&
c.lat <= bbox[3];
for (const key of keys) {
const namesakes = cities.filter((c) => c.keys.has(key));
const ownCity = namesakes.some(
(c) => inBox(c) && c.population >= ambiguity.insideMinPopulation,
);
const cityInside = ownCity && areaKm2 > ambiguity.cityStateMaxKm2;
if (ownCity && !cityInside) cityState = true;
const elsewhere = namesakes.filter((c) => !inBox(c));
if (
countryNames.has(key) ||
cityInside ||
elsewhere.some((c) => c.country === p.admin)
)
amb.push(key);
// Only abroad: "Victoria, Australia" settles it; "Victoria" alone not.
else if (elsewhere.length) ambAbroad.push(key);
}
const postal = String(p.postal || '').trim();
features.push({
name,
...(nameEn ? { nameEn } : {}),
...(alt.length ? { alt } : {}),
...(/^[A-Z]{2}$/.test(postal) ? { postal } : {}),
type: p.type_en || p.type || null,
...(Number.isFinite(p.min_label) ? { rank: p.min_label } : {}),
country: p.admin,
iso2: p.iso_a2 && p.iso_a2 !== '-99' ? p.iso_a2 : null,
iso: p.iso_3166_2 && !p.iso_3166_2.includes('~') ? p.iso_3166_2 : null,
...(amb.length ? { amb: amb.sort() } : {}),
...(ambAbroad.length ? { ambAbroad: ambAbroad.sort() } : {}),
...(cityState ? { cityState } : {}),
...(Number.isFinite(p.longitude) && Number.isFinite(p.latitude)
? {
label: [
Math.round(p.longitude * 1000) / 1000,
Math.round(p.latitude * 1000) / 1000,
],
}
: {}),
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
}
features.sort(
(a, b) =>
a.country.localeCompare(b.country) ||
a.name.localeCompare(b.name) ||
(a.iso || '').localeCompare(b.iso || ''),
);
return {
meta: {
title: 'Natural Earth admin-1 states and provinces',
source: 'ne_10m_admin_1_states_provinces',
url: SOURCES.admin1.url,
sha256: source.digest,
ambiguitySource: SOURCES.places.url,
ambiguitySha256: places.digest,
commit: NE_COMMIT,
license:
'Public domain (https://www.naturalearthdata.com/about/terms-of-use/)',
schema:
'features[]: name, nameEn?, alt?[], postal?, type, rank? (min_label: the zoom at which Natural Earth labels the unit; lower is more prominent), country, iso2, iso (ISO 3166-2), amb?[] (normalized names that are also a country or a city — alone or with the country named), ambAbroad?[] (names shared only with a city in another country — ambiguous alone, settled by naming the country), cityState? (the unit is its own city), label? [lon, lat] label point, d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
decimals: params.decimals,
curation: { ...params, ambiguity },
script: 'scripts/build-admin-packs.mjs',
},
features,
};
}
async function buildCounties() {
const params = PARAMS.counties;
const source = await fetchPinned('counties');
const files = unzip(source.bytes);
const base = path.basename(new URL(SOURCES.counties.url).pathname, '.zip');
const shapes = readShp(files.get(`${base}.shp`));
const records = readDbf(files.get(`${base}.dbf`));
if (shapes.length !== records.length)
throw new Error('counties: shp/dbf record counts differ');
const features = [];
records.forEach((record, index) => {
const unit = simplifyUnit(shpPolygons(shapes[index]), params);
if (!unit.polygons.length) return;
const lsad = record.NAMELSAD.startsWith(record.NAME)
? record.NAMELSAD.slice(record.NAME.length).trim()
: '';
features.push({
name: record.NAME,
full: record.NAMELSAD,
...(lsad ? { lsad } : {}),
state: record.STATE_NAME,
st: record.STUSPS,
geoid: record.GEOID,
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
});
features.sort((a, b) => a.geoid.localeCompare(b.geoid));
return {
meta: {
title: 'US Census Bureau cartographic boundary counties',
source: base,
url: SOURCES.counties.url,
sha256: source.digest,
license: 'Public domain (U.S. Government work, 17 U.S.C. § 105)',
schema:
'features[]: name, full (NAMELSAD), lsad? (County, Parish, Borough, …), state, st (USPS), geoid, d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
decimals: params.decimals,
curation: params,
script: 'scripts/build-admin-packs.mjs',
},
features,
};
}
async function writePack(relative, pack) {
const file = path.join(ROOT, relative);
await mkdir(path.dirname(file), { recursive: true });
const text = JSON.stringify(pack) + '\n';
await writeFile(file, text);
const parts = pack.features.reduce((n, f) => n + f.polygons.length, 0);
const vertices = pack.features.reduce(
(n, f) => n + f.polygons.flat().reduce((m, ring) => m + ring.length / 2, 0),
0,
);
console.log(
`${relative}: ${pack.features.length} features, ${parts} parts, ${vertices} vertices, ${Buffer.byteLength(text)} bytes`,
);
}
if (!only || only === 'countries')
await writePack(
'src/data/local_data/natural_earth/countries.json',
await buildCountries(),
);
if (!only || only === 'admin1')
await writePack(
'src/data/local_data/natural_earth/states_provinces.json',
await buildAdmin1(),
);
if (!only || only === 'counties')
await writePack(
'src/data/local_data/us_census_counties/counties.json',
await buildCounties(),
);
scripts/build-military-names.py
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Maintainer-only Overture military-name pack build (DuckDB 1.5.5)."""
import argparse
import gzip
import hashlib
import json
import math
from pathlib import Path
import re
ROOT = Path(__file__).resolve().parent.parent
RELEASE = '2026-09-23.1'
CLASSES = ['military_land', 'airfield', 'naval_base', 'range', 'barracks', 'base', 'training_area']
PACK = ROOT / 'src/data/local_data/osm_military_names'
def assemble(rows):
records = []
snapshots = set()
for row in rows:
source = next(s for s in row['sources'] if s['dataset'] == 'OpenStreetMap')
match = re.fullmatch(r'([nwr])(\d+)@\d+', source['record_id'])
if not match:
raise ValueError('Invalid typed OSM identity')
ident = match[1] + match[2]
bbox = [row['bbox'][k] for k in ['xmin', 'ymin', 'xmax', 'ymax']]
lon, lat, area = row['lon'], row['lat'], row['area']
if not all(math.isfinite(v) for v in [lon, lat, area, *bbox]):
raise ValueError('Invalid geometry measurement')
if not bbox[0] - 1e-7 <= lon <= bbox[2] + 1e-7 or not bbox[1] - 1e-7 <= lat <= bbox[3] + 1e-7:
raise ValueError('Label outside source bounds')
kind = row['military'] if row['military'] in CLASSES else 'military_land'
records.append([ident, row['name'], round(lon, 5), round(lat, 5),
*[round(v, 4) for v in bbox], CLASSES.index(kind), max(1, round(abs(area)))])
snapshots.add(source['version'][:10])
records.sort(key=lambda r: (r[0][0], int(r[0][1:])))
if len(set(r[0] for r in records)) != len(records):
raise ValueError('Duplicate OSM identity')
return {'release': RELEASE, 'snapshots': sorted(snapshots), 'classes': CLASSES, 'records': records}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--cache', type=Path, required=True)
parser.add_argument('--extensions', type=Path)
parser.add_argument('--verify', action='store_true')
args = parser.parse_args()
args.cache.mkdir(parents=True, exist_ok=True)
cached = args.cache / ('military-names-' + RELEASE + '.json')
if not cached.exists():
import duckdb
if duckdb.__version__ != '1.5.5':
raise RuntimeError('Use duckdb==1.5.5')
config = {'threads': '8', 'memory_limit': '3GB', 'temp_directory': str(args.cache / 'temp')}
if args.extensions:
config['extension_directory'] = str(args.extensions)
connection = duckdb.connect(config=config)
if not args.extensions:
connection.execute('INSTALL httpfs; INSTALL spatial;')
connection.execute('LOAD httpfs; LOAD spatial; SET geometry_always_xy=true; SET enable_progress_bar=false;')
files = json.loads((ROOT / 'scripts/military-names-files.json').read_text())
if not all('/' + RELEASE + '/' in url for url in files):
raise ValueError('Unexpected source release')
cursor = connection.execute((ROOT / 'scripts/military-names.sql').read_text(), [files])
fields = [d[0] for d in cursor.description]
rows = [dict(zip(fields, row)) for row in cursor.fetchall()]
cached.write_text(json.dumps(rows, ensure_ascii=False, separators=(',', ':')))
pack = assemble(json.loads(cached.read_text()))
raw = json.dumps(pack, ensure_ascii=False, separators=(',', ':')).encode()
digest = hashlib.sha256(raw).hexdigest()
if args.verify:
expected = (PACK / 'names.sha256').read_text().split()[0]
if digest != expected:
raise ValueError(f'Output SHA-256 mismatch: {digest}')
else:
PACK.mkdir(parents=True, exist_ok=True)
(PACK / 'names.json').write_bytes(raw)
(PACK / 'names.sha256').write_text(digest + ' names.json\n')
print(json.dumps({'records': len(pack['records']), 'rawBytes': len(raw),
'gzipBytes': len(gzip.compress(raw, compresslevel=9, mtime=0)), 'sha256': digest}))
if __name__ == '__main__':
main()
scripts/check-layer-state-tokens.mjs
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env node
import { execFileSync } from 'node:child_process';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import {
LAYER_STATE_TOKEN_RESERVATIONS,
parseLayerStateTokenReservations,
validateLayerStateAllocations,
validateLayerStateRegistry,
} from '../src/data/layerState.js';
// The pre-ledger main revision had exactly the 28 pinned v2 registry entries.
// Accept only that source blob as the one-time bootstrap; any other published
// source without a ledger must fail closed rather than risk losing a token.
const PRE_LEDGER_LAYER_STATE_BLOB = 'e559a4c40191512dcefb83134958a3bfd4a98a69';
// This snapshot belongs to the verified published blob above, not to the PR's
// editable legacy map. Otherwise a candidate could redefine both its ledger
// and its comparison baseline before the first ledger reaches main.
export const PRE_LEDGER_LAYER_STATE_TOKENS = Object.freeze({
'ais-live-vessels': 'a',
'alpr-cameras': 'p',
'bhote-koshi-2026': 'h',
'bhote-koshi-locator': 'z',
bikeshare: 'b',
cctv: 'c',
directions: 'n',
earthquakes: 'e',
'fire-perimeters': '2',
flights: 'f',
'local-dams': 'q',
'local-datacenters': 'd',
'local-firms': 'w',
military: 'm',
'military-awareness': 'g',
'military-installations': 'i',
radio: 'r',
'recent-imagery': '1',
'rocket-launches': 'x',
satellites: 's',
'telegeography-submarine-cables': 'u',
traffic: 't',
transit: 'j',
'weather-cyclones': 'y',
'weather-lightning': 'l',
'weather-radar': 'v',
'weather-satellite': 'o',
wind: 'k',
});
const LEDGER_PATH = 'src/data/layerStateTokenReservations.json';
const SOURCE_PATH = 'src/data/layerState.js';
function git(args, cwd) {
return execFileSync('git', args, {
cwd,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
}).trim();
}
/** Read every permanent assignment from a published Git revision. */
export function readPublishedLayerStateReservations(
baseRef,
cwd = process.cwd(),
) {
const commit = git(
['rev-parse', '--verify', '--end-of-options', `${baseRef}^{commit}`],
cwd,
);
const ledgerSpec = `${commit}:${LEDGER_PATH}`;
let hasLedger = true;
try {
git(['cat-file', '-e', ledgerSpec], cwd);
} catch {
hasLedger = false;
}
if (!hasLedger) {
const sourceBlob = git(['rev-parse', `${commit}:${SOURCE_PATH}`], cwd);
if (sourceBlob !== PRE_LEDGER_LAYER_STATE_BLOB) {
throw new Error(
'Published base has no recognizable layer-state token ledger',
);
}
return PRE_LEDGER_LAYER_STATE_TOKENS;
}
let rows;
try {
rows = JSON.parse(git(['show', ledgerSpec], cwd));
} catch (error) {
throw new Error(
`Cannot read published layer-state token ledger: ${error.message}`,
);
}
return parseLayerStateTokenReservations(rows);
}
export function checkLayerStateTokens(baseRef, cwd = process.cwd()) {
const base = readPublishedLayerStateReservations(baseRef, cwd);
validateLayerStateRegistry();
validateLayerStateAllocations(base, LAYER_STATE_TOKEN_RESERVATIONS);
return {
published: Object.keys(base).length,
added:
Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length -
Object.keys(base).length,
};
}
const invokedPath = process.argv[1]
? pathToFileURL(path.resolve(process.argv[1])).href
: '';
if (import.meta.url === invokedPath) {
const args = process.argv.slice(2);
const baseRef =
args[0] === '--base-ref' && args.length === 2 ? args[1] : null;
if (!baseRef || baseRef.startsWith('-')) {
console.error('Usage: npm run layer-token:check -- --base-ref origin/main');
process.exitCode = 1;
} else {
try {
const result = checkLayerStateTokens(baseRef);
console.log(
`Layer tokens valid against ${baseRef}: ${result.published} published, ` +
`${result.added} new.`,
);
} catch (error) {
console.error(`Layer token check failed: ${error.message}`);
process.exitCode = 1;
}
}
}
scripts/check-package-boundaries.mjs
+10 -1
View File
@@ -89,6 +89,10 @@ export async function checkPackageBoundaries(root) {
const seen = new Set();
await build({
root,
// Keep this gate out of a running dev server's dependency cache
// (node_modules/.vite): a build there re-optimizes dependencies and the
// dev server's workers then fail with 504 "Outdated Optimize Dep".
cacheDir: path.join(root, 'node_modules/.vite-boundaries'),
configFile: false,
envFile: false,
publicDir: false,
@@ -115,7 +119,12 @@ export async function checkPackageBoundaries(root) {
assetsInlineLimit: 0,
rollupOptions: {
input,
external: group.external,
// A declared dependency also owns its package subpath exports
// (for example `@jtarrio/webrtlsdr/rtlsdr.js`).
external: (id) =>
group.external.some(
(name) => id === name || id.startsWith(`${name}/`),
),
// Unused imports must still obey ownership; tree shaking is not a boundary.
treeshake: false,
preserveEntrySignatures: 'strict',
scripts/format-scope.json
+120 -1
View File
@@ -4,6 +4,11 @@
"scripts/format-scope.json",
"scripts/package-boundaries.json",
"scripts/format.mjs",
"scripts/next-layer-state-token.mjs",
"scripts/check-layer-state-tokens.mjs",
"scripts/qa-layer-token-twochar.mjs",
"src/data/layerStateTokenReservations.json",
"src/data/layerStateTokenLedger.test.mjs",
"scripts/check-package-boundaries.mjs",
"scripts/precompute-cctv-heights.mjs",
"src/tooling/precomputeCctvHeights.test.mjs",
@@ -21,16 +26,23 @@
"src/annotations/drawMode.test.mjs",
"src/annotations/drawTool.js",
"src/annotations/drawTool.test.mjs",
"src/annotations/adminOutline.test.mjs",
"src/annotations/regionRing.test.mjs",
"src/data/adminBoundaries.test.mjs",
"scripts/build-admin-packs.mjs",
"scripts/qa-admin-outlines.mjs",
"src/data/inputOwnership.test.mjs",
"docs/CODE-BOUNDARIES.md",
"src/main.js",
"docs/APPLICATION.md",
"scripts/qa-application.mjs",
"scripts/qa-terrain-429.mjs",
"src/app/application.js",
"src/app/application.test.mjs",
"src/app/mapLifecycle.test.mjs",
"src/app/startupCamera.test.mjs",
"src/app/viewer.js",
"src/app/atmosphereCompat.test.mjs",
"src/standalone/application.js",
"src/standalone/controls.js",
"src/standalone/data.js",
@@ -99,6 +111,8 @@
"src/tooling/previewServing.test.mjs",
"server/providers/common/request.js",
"server/providers/common/source-root.js",
"src/data/tapAddress.test.mjs",
"src/data/tapAddress.js",
"server/providers/local.js",
"server/providers/military-installations.js",
"server/providers/military-installations/cache.js",
@@ -142,6 +156,8 @@
"src/keylessGeocoder.js",
"src/standalone/placeSearch.js",
"src/ui/panelDisclosure.js",
"src/ui/imagerySplit.js",
"src/ui/imagerySplit.test.mjs",
"src/ui/surfaceKeyboard.js",
"src/ui/surfaceKeyboard.test.mjs",
"src/ui/panelRails.js",
@@ -334,6 +350,16 @@
"src/layers/earthquakes/model.js",
"src/layers/earthquakes/source.js",
"src/layers/earthquakes/ownership.test.mjs",
"src/layers/perimeters/index.js",
"src/layers/perimeters/records.js",
"src/layers/perimeters/source.js",
"src/layers/perimeters/cards.js",
"src/layers/perimeters/inciweb.js",
"src/layers/perimeters/records.test.mjs",
"src/layers/perimeters/source.test.mjs",
"src/layers/perimeters/cards.test.mjs",
"src/layers/perimeters/inciweb.test.mjs",
"src/layers/perimeters/ownership.test.mjs",
"src/data/firmsHeatmap.js",
"src/data/fireAnchors.js",
"src/layers/firms/anchors.js",
@@ -383,6 +409,10 @@
"src/layers/launches/model.js",
"src/layers/launches/selection.js",
"src/layers/launches/camera.js",
"src/data/localGeojson.analyst.test.mjs",
"src/data/satellites.analyst.test.mjs",
"src/data/layerSnapshot.test.mjs",
"src/data/satellitePass.test.mjs",
"src/data/satellites.js",
"src/data/rocketLaunches.js",
"src/layers/satellites/source.test.mjs",
@@ -544,17 +574,21 @@
"src/data/telegeographySubmarineCables.test.mjs",
"src/data/naturalEarthRegions.js",
"src/data/neighborhoodPolygons.js",
"src/data/bundledJson.js",
"src/layers/submarineCables/bundledSource.test.mjs",
"src/maps/availability.js",
"src/maps/catalog.js",
"src/maps/controller.js",
"src/maps/controller.test.mjs",
"src/maps/imageryComparison.js",
"src/maps/credits.js",
"src/maps/defaultSources.js",
"src/maps/google3d.js",
"src/maps/imagery.js",
"src/maps/registry.js",
"src/maps/terrain.js",
"src/maps/terrainRetry.js",
"src/maps/terrainRetry.test.mjs",
"src/mapStackController.js",
"src/mapStartup.js",
"src/mapStackChips.test.mjs",
@@ -596,6 +630,7 @@
"src/app/layers/flights.js",
"src/app/layers/radio.js",
"src/app/layers/earthquakes.js",
"src/app/layers/perimeters.js",
"src/app/layers/cctv.js",
"src/app/layers/firms.js",
"src/app/layers/satellites.js",
@@ -611,6 +646,7 @@
"src/data/surfaceServices.js",
"src/annotations/resolver.js",
"src/services/terrainHeights.js",
"src/services/terrainHeights.test.mjs",
"src/services/groundFloor.js",
"src/services/meshFloorSampler.js",
"src/services/groundSnap.js",
@@ -730,6 +766,27 @@
"scripts/format-runtime.json",
"src/tooling/transitService.test.mjs",
"src/sources/overpassFeatures.test.mjs",
"src/data/cctvHlsStream.test.mjs",
"src/layers/cctv/videoPlayback.test.mjs",
"src/ui/cctvVideo.test.mjs",
"src/ui/styles/weather.css",
"src/layers/weather/index.js",
"src/layers/weather/clock.js",
"src/layers/weather/clock.test.mjs",
"src/layers/weather/rendering.js",
"src/layers/weather/shellRendering.js",
"src/layers/weather/shellRendering.test.mjs",
"src/layers/weather/infraredAlpha.js",
"src/layers/weather/infraredAlpha.test.mjs",
"src/layers/weather/infraredImage.js",
"src/layers/weather/infraredImage.test.mjs",
"src/layers/weather/source.js",
"src/layers/weather/weather.test.mjs",
"src/layers/weather/rasterTiles.test.mjs",
"src/layers/wind/rendering.test.mjs",
"src/layers/weather/source.test.mjs",
"src/data/weatherProxy.test.mjs",
"server/providers/weather.js",
"src/director/clock.test.mjs",
"src/director/timeline.test.mjs",
"src/scenes/packs/registry.test.mjs",
@@ -745,5 +802,67 @@
"scripts/qa-director-interactions.mjs",
"src/director/sharing/sharing.test.mjs",
"scripts/qa-director-sharing.mjs",
"src/data/aisStreamSentinels.test.mjs"
"src/ui/weatherPanel.js",
"src/ui/weatherPanel.test.mjs",
"src/ui/railCards.js",
"src/ui/railCards.test.mjs",
"src/ui/railTimeline.js",
"src/ui/railTimeline.test.mjs",
"src/layers/weather/imageryOrder.js",
"src/layers/weather/imageryOrder.test.mjs",
"server/providers/cyclones.js",
"server/providers/firePerimeters.js",
"src/data/cyclonesProxy.test.mjs",
"src/data/firePerimetersProxy.test.mjs",
"src/layers/cyclones/index.js",
"src/layers/cyclones/rendering.js",
"src/layers/cyclones/labels.js",
"src/layers/cyclones/source.js",
"src/layers/cyclones/rendering.test.mjs",
"src/layers/cyclones/labels.test.mjs",
"src/layers/cyclones/index.test.mjs",
"src/layers/cyclones/source.test.mjs",
"src/data/aisStreamSentinels.test.mjs",
"scripts/qa-weather-perf.mjs",
"scripts/qa-weather-journey.mjs",
"scripts/qa-weather-swap.mjs",
"scripts/qa-weather-teardown.mjs",
"src/layers/weather/imageryHost.test.mjs",
"src/sdr/adsbDecoder.test.mjs",
"src/sdr/controller.test.mjs",
"src/sdr/gain.test.mjs",
"src/sdr/spectrum.test.mjs",
"src/sdr/usbDevices.test.mjs",
"src/sources/adsbRecords.test.mjs",
"src/layers/localAdsb/index.test.mjs",
"src/ui/localSdrControls.test.mjs",
"src/ui/localSdrMarkup.test.mjs",
"src/ui/localSdrPresentation.test.mjs",
"src/data/localReceiversProxy.test.mjs",
"src/layers/localAdsb/feeds.test.mjs",
"src/ui/railScrollers.test.mjs",
"src/layers/localAdsb/motion.test.mjs",
"src/data/modelEligibility.test.mjs",
"src/layers/recentImagery/model.test.mjs",
"src/layers/recentImagery/catalog.test.mjs",
"src/maps/imageryHost.test.mjs",
"src/layers/recentImagery/rendering.test.mjs",
"src/layers/recentImagery/thumbnails.test.mjs",
"src/layers/recentImagery/index.test.mjs",
"src/ui/imageryBoxTool.test.mjs",
"src/ui/recentImagery.test.mjs",
"src/layers/recentImagery/testDoubles.mjs",
"scripts/qa-recent-imagery.mjs",
"scripts/qa-overpass-offload.mjs",
"src/overpassOffload.test.mjs",
"src/sources/openFreeMap.test.mjs",
"src/sources/vectorTiles.test.mjs",
"src/sources/militaryTileGeometry.test.mjs",
"src/layers/traffic/surface.test.mjs",
"src/layers/traffic/roadModes.test.mjs",
"scripts/qa-traffic-motion.mjs",
"src/layers/traffic/motion.test.mjs",
"scripts/qa-journey-recorder.mjs",
"scripts/qa-alpr-journey.mjs",
"scripts/qa-context-arrivals.mjs"
]
scripts/military-names-files.json
+66
View File
@@ -0,0 +1,66 @@
[
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00000-18dcb239-f29e-5e57-b36f-4bd46c12fb86-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00001-4a0ec645-f2e2-5fd1-aeaf-a2210290fe76-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00002-0bfb9ec4-5947-583a-9b20-5b39a6635e7d-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00003-0aa711cc-b475-5914-9670-7295640f184f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00004-9a1af7ec-89dd-5aa1-a419-d6325719c330-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00005-e85508c1-3f80-5883-8539-35302da710f3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00006-184886ec-67b4-5a65-852f-f26f757aaeea-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00007-adbd870a-4116-50a2-9554-57064af19a18-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00008-6362555a-8b0c-5589-a6fe-b71ddca82083-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00009-19bb41f3-6c3b-579a-adae-8f69030c37c7-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00010-0e372a9c-a60d-5b06-89e2-866989d612dd-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00011-2ae04990-7b00-5def-bf72-12b2fdc13b48-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00012-dcfe8859-26a4-544b-ad61-dfbc7e7f272a-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00013-a369ed98-5e3b-5bda-94c9-460f06e8746b-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00014-d03d3cde-6ae3-5938-abe9-b2018f7ab0e3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00015-57a736a1-7d58-5851-877a-2fca7de484ba-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00016-141b8ad5-f6d4-5f5e-8353-450a53188e27-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00017-14133e77-297b-57f3-b028-c5133bb56578-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00018-65485e6e-8939-51ab-8566-47dd3887b6c3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00019-64fb52b8-5ba4-5f00-860f-4fe71e1c9f3a-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00020-0feea924-6adb-5118-9a2b-e0a0d3ef6963-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00021-a9115b0c-a202-5468-bfae-0a1e6f953be3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00022-440ee3b6-f6ae-5bf9-b5d1-cc81fcb0a430-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00023-53a4d462-62b9-589c-9650-f7c9652354e7-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00024-54089af7-32f4-5ad8-8970-cea7350cbb20-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00025-59cc63a7-e7da-58e2-8d06-1df15f3a0636-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00026-ba0e8b28-756c-5b5e-aa01-0939a15e45c8-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00027-8237ebb3-2fcb-50d7-884b-81c8e81bda1e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00028-6fb209d3-9fbe-5155-9983-896b1bb9e84e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00029-e86227ec-f5f7-51c6-b537-a10334796cc4-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00030-a2779b0d-003a-5f5d-812c-02eb4a5db71f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00031-554320d6-a62d-5be7-80b0-d32fb67d87ca-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00032-a49d2756-469a-571f-86ef-be60690cfaea-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00033-0b36146b-6963-5133-8a67-ced0bbb37d6a-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00034-b676aef2-1e90-5966-8531-40f42c5d01d6-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00035-c9b04728-fc59-50f2-bfa8-5f8692ea731e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00036-22a0fbd7-fd24-5832-ac6c-85e82b56ced6-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00037-d157ddb7-8962-515b-9ec3-d54333fcf836-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00038-39d77c44-55f2-5792-83e1-cac6416d0630-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00039-f6871cf2-753b-53f1-aacf-645ea035826d-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00040-3d142e7a-4189-5e50-9f68-0f77e9a51047-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00041-9e6d1cdd-f130-5089-9d0f-703626552100-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00042-4cefe8fe-4966-5e1e-9cf5-19fd90384db8-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00043-65360671-af63-58dd-9784-bceb07abe219-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00044-b423bff1-dd2b-52c4-90f6-cd1605e4d83f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00045-1f4d97be-99fc-50de-b9cb-f3624c9a1817-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00046-32a9756a-e3a0-5980-a8d9-88dd475f25b9-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00047-cf80d668-e9f1-5f69-8e21-0a73f1732e5f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00048-8d932283-4f3f-533d-895b-684faaa9cc34-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00049-996168f3-0c3a-5b58-8444-168c7d859a04-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00050-f9c3c526-3ea5-5dab-a46d-05c623c3e905-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00051-72b4d98d-4b3b-5b3b-a565-62c206d4fa29-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00052-7385fa90-2fa9-53c8-9ed4-96d5703f3a37-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00053-96804709-94fa-56d2-bafd-69b2cbce3881-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00054-520c3c50-82f8-5f3d-97b5-fa8f5f4e65e7-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00055-8742f073-69cd-579e-bf00-b16b6e6a7669-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00056-58083e97-7020-5522-ac19-d8a5b4dcbc98-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00057-bf201c73-1fb3-5dbc-8cec-d344d999820e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00058-9e1561c3-ba15-528b-8db3-62ce04b1570b-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00059-dd8cd7ca-410f-54f8-9b50-66beb004d640-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00060-203c7177-ec4a-55c2-8003-d828b144a6d6-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00061-0da76969-7fb1-56b9-bb83-4e40f3acea8d-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00062-13344b87-db6c-5415-a48e-afd95a928a85-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00063-a61ab643-210f-518d-b6b8-894f72e5c094-c000.zstd.parquet"
]
scripts/military-names.sql
+11
View File
@@ -0,0 +1,11 @@
SELECT names.primary AS name, bbox, sources,
source_tags['military'] AS military,
ST_X(ST_PointOnSurface(geometry)) AS lon,
ST_Y(ST_PointOnSurface(geometry)) AS lat,
ST_Area_Spheroid(geometry) AS area
FROM read_parquet(?)
WHERE subtype = 'military'
AND nullif(trim(names.primary), '') IS NOT NULL
AND (source_tags['landuse'] = 'military'
OR source_tags['military'] IN
('airfield', 'naval_base', 'range', 'barracks', 'base', 'training_area'))
scripts/next-layer-state-token.mjs
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env node
import {
LAYER_STATE_TOKEN_RESERVATIONS,
nextLayerStateToken,
} from '../src/data/layerState.js';
const layerId = String(process.argv[2] || '').trim();
if (!/^[a-z0-9-]+$/.test(layerId)) {
console.error('Usage: npm run layer-token:next -- <layer-id>');
process.exitCode = 1;
} else if (Object.hasOwn(LAYER_STATE_TOKEN_RESERVATIONS, layerId)) {
console.error(
`${layerId} already owns token ${LAYER_STATE_TOKEN_RESERVATIONS[layerId]}`,
);
process.exitCode = 1;
} else {
console.log(`${layerId}: ${nextLayerStateToken()}`);
console.log('Order: free digits 0-9, then two-character base-36 00-zz.');
console.log('Re-run after rebasing onto the latest main before merge.');
}
scripts/package-boundaries.json
+409 -31
View File
File diff suppressed because it is too large. Load diff
scripts/qa-admin-outlines.mjs
+655
View File
@@ -0,0 +1,655 @@
#!/usr/bin/env node
/**
* qa-admin-outlines.mjs — bundled state/province/county outlines, driven like a
* user would drive them.
*
* Run: node scripts/qa-admin-outlines.mjs http://localhost:4173 [--headful] [--no-video]
*
* Journey: frame Texas at state scale and outline "Texas"; pan and zoom toward
* Austin in smooth 2–3 s moves; outline "Travis County, Texas"; pan north and
* outline the neighbouring "Williamson County, Texas"; zoom out to see both;
* then outline "Bavaria" (framed, as a voice request for a distant place is),
* and finally Brandenburg (Berlin is a hole) and Hawaii (eight islands).
*
* Asserts for every outline: it resolves from the bundled packs (no geocode,
* boundary or Overpass request), is drawn as a draped outline within the
* budget, and holds still — no rebuilt primitives and no new entities while
* the camera is still. The whole journey must log zero console/page errors and
* send zero requests to any Overpass host or nominatim.openstreetmap.org.
*
* Evidence in qa-shots/ (gitignored): admin-outlines-<step>.png screenshots,
* admin-outlines-journey.mp4 (CDP screencast at ~24 fps, frames re-timed from
* their capture timestamps), and admin-outlines-result.json with per-step
* timings, frame-time p50/p95 and still-camera frame-difference statistics.
* Exits nonzero on any failed assertion.
*/
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import fs from 'node:fs/promises';
import path from 'node:path';
import puppeteer from 'puppeteer';
import sharp from 'sharp';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-admin-outlines.mjs <dev-server-url> [--headful] [--no-video]',
);
process.exit(0);
}
const url = args.find((arg) => !arg.startsWith('--'));
if (!url || !['http:', 'https:'].includes(new URL(url).protocol))
throw new Error('Supply a running dev server URL; see --help');
const recordVideo = !args.includes('--no-video');
const shots = path.resolve('qa-shots');
const framesDir = path.join(shots, 'admin-outlines-frames');
await fs.mkdir(shots, { recursive: true });
await fs.rm(framesDir, { recursive: true, force: true });
await fs.mkdir(framesDir, { recursive: true });
const OUTLINE_BUDGET_MS = 1500;
const STILL_MS = 3000;
const FRAME_INTERVAL_S = 1 / 24;
const result = {
url,
steps: [],
forbiddenRequests: [],
lookups: [],
packRequests: [],
consoleErrors: [],
pageErrors: [],
screenshots: [],
video: null,
};
const browser = await puppeteer.launch({
headless: !args.includes('--headful'),
protocolTimeout: 300_000,
args: [
'--no-sandbox',
'--use-gl=angle',
'--disable-dev-shm-usage',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
let page;
const frames = [];
let lastKept = -Infinity;
let recording = false;
let marker = 'startup';
try {
page = await browser.newPage();
await page.evaluateOnNewDocument(() => {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
});
await page.setViewport({ width: 1280, height: 800 });
const client = await page.createCDPSession();
await client.send('Network.enable');
client.on('Network.requestWillBeSent', ({ request }) => {
const target = new URL(request.url);
const where = `${request.method} ${target.origin}${target.pathname}`;
if (
target.hostname.includes('overpass') ||
target.hostname === 'nominatim.openstreetmap.org'
)
result.forbiddenRequests.push({ step: marker, request: where });
// Lookups an outline could make: a forward geocode, a place search, an
// Overpass boundary query. The HUD's reverse geocode of the camera centre
// (latlng=) is not one of them.
const local = target.origin === new URL(url).origin;
const reverse = target.searchParams.has('latlng');
const lookup =
!reverse &&
((local &&
/^\/api\/(overpass|geocode|google\/text-search|nominatim|photon)/.test(
target.pathname,
)) ||
(target.hostname === 'maps.googleapis.com' &&
target.pathname.includes('/geocode/')) ||
target.hostname === 'photon.komoot.io');
if (lookup) result.lookups.push({ step: marker, request: where });
if (/\/(states_provinces|countries|counties)\.json$/.test(target.pathname))
result.packRequests.push({ step: marker, pack: target.pathname });
});
page.on('pageerror', (error) =>
result.pageErrors.push({ step: marker, message: error.message }),
);
page.on('console', (message) => {
if (message.type() === 'error')
result.consoleErrors.push({ step: marker, message: message.text() });
});
client.on('Page.screencastFrame', async ({ data, metadata, sessionId }) => {
client.send('Page.screencastFrameAck', { sessionId }).catch(() => {});
if (!recording) return;
const t = metadata.timestamp;
if (t - lastKept < FRAME_INTERVAL_S * 0.9) return;
lastKept = t;
const file = path.join(
framesDir,
`${String(frames.length).padStart(6, '0')}.jpg`,
);
frames.push({ file, t, step: marker });
await fs.writeFile(file, Buffer.from(data, 'base64'));
});
console.log('Loading viewer...');
const navigationUrl = new URL(url);
navigationUrl.searchParams.set('welcome', '0');
await page.goto(navigationUrl.href, {
waitUntil: 'domcontentloaded',
timeout: 60_000,
});
await page.waitForFunction(
() => window.__godsEyeView?.viewer && window.__godsEyeView?.annotations,
{ timeout: 90_000, polling: 500 },
);
await page.waitForFunction(
() =>
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 90_000, polling: 250 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.waitForFunction(
() => !document.querySelector('#first-run-launcher:not([hidden])'),
{ timeout: 10_000 },
);
result.renderer = await page.evaluate(() => {
const gl = window.__godsEyeView.viewer.scene.context._gl;
const debug = gl.getExtension('WEBGL_debug_renderer_info');
return debug
? gl.getParameter(debug.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER);
});
console.log('Renderer:', result.renderer);
// In-page helpers: camera moves, frame timing and annotation state.
await page.evaluate(() => {
const { viewer } = window.__godsEyeView;
const Cesium = window.__CESIUM__;
window.__qaOutlines = {
fly(view, duration) {
return new Promise((resolve) => {
viewer.camera.cancelFlight();
viewer.camera.flyTo({
destination: Cesium.Cartesian3.fromDegrees(
view.lon,
view.lat,
view.height,
),
orientation: {
heading: Cesium.Math.toRadians(view.heading ?? 0),
pitch: Cesium.Math.toRadians(view.pitch ?? -70),
roll: 0,
},
duration,
easingFunction: Cesium.EasingFunction.QUADRATIC_IN_OUT,
complete: resolve,
cancel: resolve,
});
});
},
frameTimes(ms) {
return new Promise((resolve) => {
const deltas = [];
let last = performance.now();
const end = last + ms;
const step = (now) => {
deltas.push(now - last);
last = now;
if (now < end) requestAnimationFrame(step);
else resolve(deltas.slice(1));
};
requestAnimationFrame(step);
});
},
annotationSource() {
for (let i = 0; i < viewer.dataSources.length; i++) {
const source = viewer.dataSources.get(i);
if (source.name === 'gev-annotations') return source;
}
return null;
},
groundPrimitives() {
const list = [];
const walk = (collection) => {
for (let i = 0; i < collection.length; i++) {
const item = collection.get(i);
if (item?.length !== undefined && typeof item.get === 'function')
walk(item);
else list.push(item);
}
};
walk(viewer.scene.groundPrimitives);
return list;
},
snapshot() {
const source = this.annotationSource();
const entities = source ? source.entities.values : [];
const primitives = this.groundPrimitives();
window.__qaPrimitiveIds ??= new WeakMap();
let next = window.__qaPrimitiveNext ?? 1;
const ids = primitives.map((primitive) => {
if (!window.__qaPrimitiveIds.has(primitive))
window.__qaPrimitiveIds.set(primitive, next++);
return window.__qaPrimitiveIds.get(primitive);
});
window.__qaPrimitiveNext = next;
return {
entities: entities.length,
polygons: entities.filter((e) => e.polygon).length,
polylines: entities.filter((e) => e.polyline).length,
draped:
entities.every(
(e) =>
!e.polygon ||
e.polygon.classificationType?.getValue() ===
Cesium.ClassificationType.BOTH,
) &&
entities.every(
(e) => !e.polyline || e.polyline.clampToGround?.getValue(),
),
primitiveIds: ids,
ready: viewer.dataSourceDisplay.ready,
};
},
};
});
const settle = () =>
page
.waitForFunction(
() => {
const { scene } = window.__godsEyeView.viewer;
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (p?.show && typeof p.tilesLoaded === 'boolean' && !p.tilesLoaded)
return false;
}
return true;
},
{ timeout: 60_000, polling: 500 },
)
.catch(() => {});
async function shot(name) {
await page.evaluate(
() =>
new Promise((resolve) =>
requestAnimationFrame(() => requestAnimationFrame(resolve)),
),
);
const filename = `admin-outlines-${name}.png`;
await page.screenshot({ path: path.join(shots, filename) });
result.screenshots.push(filename);
}
async function move(name, view, duration = 2.5) {
marker = `move:${name}`;
const [, times] = await Promise.all([
page.evaluate((v, d) => window.__qaOutlines.fly(v, d), view, duration),
page.evaluate((ms) => window.__qaOutlines.frameTimes(ms), duration * 900),
]);
result.steps.push({ step: `move:${name}`, frameTimes: stats(times) });
}
/** Annotate like the voice tool does, then watch the mark while still. */
async function outline(name, target, { flyTo = false } = {}) {
marker = `outline:${name}`;
const lookupsBefore = result.lookups.length;
const errorsBefore = result.consoleErrors.length + result.pageErrors.length;
const timing = await page.evaluate(
async (target, flyTo) => {
const started = performance.now();
const response = await window.__godsEyeView.annotations.annotate(
[{ type: 'area', target, label: target, footprint: true }],
{ flyTo },
);
const resolvedMs = performance.now() - started;
// "Drawn" = the first rendered frame after the entity visualizers have
// built the mark's primitives (each tick updates them before render).
const { viewer } = window.__godsEyeView;
await new Promise((resolve) => {
const off = viewer.scene.postRender.addEventListener(() => {
if (!viewer.dataSourceDisplay.ready) return;
off();
resolve();
});
});
const drawnMs = performance.now() - started;
const mark = window.__godsEyeView.annotations
.list()
.find((m) => m.id === response.ids?.[0]);
return {
response: response.results?.[0] ?? null,
resolvedMs,
drawnMs,
parts: mark?.polygons?.length ?? (mark?.ring ? 1 : 0),
holes: (mark?.polygons ?? []).reduce(
(n, part) => n + part.length - 1,
0,
),
ringVertices: mark?.ring?.length ?? 0,
pending: Boolean(mark?.pendingOutline),
};
},
target,
flyTo,
);
if (flyTo) await new Promise((r) => setTimeout(r, 2200));
await settle();
// Hold still: the drawn mark must not rebuild or grow.
marker = `still:${name}`;
const before = await page.evaluate(() => window.__qaOutlines.snapshot());
const stillStart = frames.length;
const times = await page.evaluate(
(ms) => window.__qaOutlines.frameTimes(ms),
STILL_MS,
);
const after = await page.evaluate(() => window.__qaOutlines.snapshot());
const step = {
step: `outline:${name}`,
target,
...timing,
entities: after.entities,
polygons: after.polygons,
polylines: after.polylines,
draped: after.draped,
stableWhileStill:
before.entities === after.entities &&
JSON.stringify(before.primitiveIds) ===
JSON.stringify(after.primitiveIds),
stillFrameTimes: stats(times),
stillFrames: [stillStart, frames.length],
lookups: result.lookups.slice(lookupsBefore),
errors:
result.consoleErrors.length + result.pageErrors.length - errorsBefore,
};
result.steps.push(step);
console.log(
`${name}: resolved ${timing.resolvedMs.toFixed(0)} ms, drawn ${timing.drawnMs.toFixed(0)} ms, ` +
`${timing.parts} part(s), ${timing.holes} hole(s), via ${timing.response?.resolvedVia}; ` +
`still p50/p95 ${step.stillFrameTimes.p50}/${step.stillFrameTimes.p95} ms; ` +
`stable=${step.stableWhileStill}`,
);
assert.equal(timing.response?.ok, true, `${name}: annotation drawn`);
assert.equal(timing.response.outline, true, `${name}: outline, not a pin`);
assert.equal(timing.response.resolvedVia, 'bundled', `${name}: bundled`);
assert.equal(timing.pending, false, `${name}: nothing pending`);
assert.ok(
timing.drawnMs < OUTLINE_BUDGET_MS,
`${name}: drawn in ${timing.drawnMs} ms`,
);
assert.equal(step.draped, true, `${name}: draped on terrain/tiles`);
assert.equal(step.stableWhileStill, true, `${name}: no redraw when still`);
assert.deepEqual(step.lookups, [], `${name}: no network lookups`);
assert.equal(step.errors, 0, `${name}: no console errors`);
return step;
}
// Start from a clean board and the default layers, as a user would.
await page.evaluate(() => window.__godsEyeView.annotations.clear?.());
await page.evaluate(() =>
window.__qaOutlines.fly(
{ lat: 25.5, lon: -99.6, height: 2_600_000, pitch: -80 },
0,
),
);
await settle();
if (recordVideo) {
await client.send('Page.startScreencast', {
format: 'jpeg',
quality: 72,
maxWidth: 1280,
maxHeight: 800,
everyNthFrame: 1,
});
recording = true;
}
await outline('texas', 'Texas');
await shot('texas');
await move('pan-west', {
lat: 25.2,
lon: -101.5,
height: 2_300_000,
pitch: -75,
});
await move('zoom-central-texas', {
lat: 28.8,
lon: -98.2,
height: 700_000,
pitch: -65,
});
await move('zoom-austin', {
lat: 29.75,
lon: -97.8,
height: 120_000,
pitch: -55,
});
await settle();
await outline('travis', 'Travis County, Texas');
await shot('travis');
await move('pan-williamson', {
lat: 30.05,
lon: -97.65,
height: 110_000,
pitch: -55,
});
await settle();
await outline('williamson', 'Williamson County, Texas');
// Close over the shared Travis/Williamson line near Pflugerville: the
// outline must lie on the photoreal mesh, not float above or sink into it.
const border = await page.evaluate(() => {
const travis = window.__godsEyeView.annotations
.list()
.find((m) => m.label === 'Travis County, Texas');
let best = null;
for (const [lon, lat] of travis.ring) {
const d = (lat - 30.46) ** 2 + (lon + 97.62) ** 2;
if (!best || d < best.d) best = { lon, lat, d };
}
return best;
});
await move(
'drape-close',
{ lat: border.lat - 0.035, lon: border.lon, height: 2500, pitch: -40 },
3,
);
await settle();
await page.evaluate((ms) => new Promise((r) => setTimeout(r, ms)), 2500);
await shot('drape-close');
await move('zoom-out-both', {
lat: 29.55,
lon: -97.7,
height: 260_000,
pitch: -60,
});
await settle();
await page.evaluate((ms) => new Promise((r) => setTimeout(r, ms)), 1500);
await shot('both-counties');
await outline('bavaria', 'Bavaria', { flyTo: true });
await shot('bavaria');
await move(
'orbit-bavaria',
{ lat: 46.6, lon: 11.9, height: 650_000, heading: 20, pitch: -55 },
3,
);
await settle();
await shot('bavaria-orbit');
// A hole (Berlin inside Brandenburg) and islands (Hawaii) draw as such.
const brandenburg = await outline('brandenburg', 'Brandenburg', {
flyTo: true,
});
assert.ok(brandenburg.holes >= 1, 'Brandenburg keeps the Berlin hole');
await shot('brandenburg');
const hawaii = await outline('hawaii', 'Hawaii', { flyTo: true });
assert.ok(hawaii.parts >= 7, 'Hawaii keeps its islands');
await shot('hawaii');
marker = 'end';
for (const [name, target, lat, lon, height] of [
['scotland', 'Scotland', 57, -4, 800000],
['england', 'England', 52.5, -1.5, 800000],
['wales', 'Wales', 52.3, -3.7, 420000],
['northern-ireland', 'Northern Ireland', 54.7, -6.7, 300000],
['switzerland', 'Switzerland', 46.8, 8.2, 500000],
['iran', 'Iran', 32, 54, 2200000],
['france', 'France', 46.6, 2.2, 1600000],
['japan', 'Japan', 37, 138, 2200000],
['georgia-state', 'Georgia', 32.6, -83.4, 650000],
['georgia-country', 'Georgia', 42, 43.6, 700000],
['georgia-qualified', 'the country of Georgia', 42, 43.6, 700000],
]) {
await page.evaluate(() => window.__godsEyeView.annotations.clear());
await move(name, { lat, lon, height, pitch: -90 }, 1);
await outline(name, target);
await shot(name);
const rendered = await page.evaluate(() => window.__qaOutlines.snapshot());
assert.ok(rendered.polygons > 0, `${target} polygon renders`);
}
if (recordVideo) {
recording = false;
await client.send('Page.stopScreencast');
}
result.totals = {
forbiddenRequests: result.forbiddenRequests.length,
consoleErrors: result.consoleErrors.length,
pageErrors: result.pageErrors.length,
};
assert.deepEqual(
result.forbiddenRequests,
[],
'no Overpass or nominatim.openstreetmap.org requests',
);
// The packs load on the first outline that needs them, never at start-up.
assert.deepEqual(
result.packRequests.map((r) => [r.step, path.basename(r.pack)]),
[
['outline:texas', 'states_provinces.json'],
['outline:travis', 'counties.json'],
['outline:scotland', 'countries.json'],
],
'packs load lazily, once each',
);
assert.deepEqual(result.pageErrors, [], 'no page errors');
assert.deepEqual(result.consoleErrors, [], 'no console errors');
} catch (error) {
result.failure = error.message;
console.error(error);
process.exitCode = 1;
if (page)
await page
.screenshot({ path: path.join(shots, 'admin-outlines-failure.png') })
.catch(() => {});
} finally {
await browser.close();
}
if (recordVideo && frames.length > 1) {
// Frame-to-frame change while the camera is still: steady pulses read as
// small, even values; a flicker (a part vanishing and returning) as a spike.
for (const step of result.steps.filter((s) => s.stillFrames)) {
const [start, end] = step.stillFrames;
const diffs = [];
let previous = null;
for (const frame of frames.slice(start, end)) {
const pixels = await sharp(frame.file)
.resize(160, 100, { fit: 'fill' })
.greyscale()
.raw()
.toBuffer();
if (previous) {
let sum = 0;
let max = 0;
for (let i = 0; i < pixels.length; i++) {
const d = Math.abs(pixels[i] - previous[i]);
sum += d;
if (d > max) max = d;
}
diffs.push(sum / pixels.length);
}
previous = pixels;
}
step.stillFrameDiff = {
frames: end - start,
meanAbsDiff: stats(diffs, 3),
};
}
const list = [];
for (let i = 0; i < frames.length; i++) {
const duration =
i + 1 < frames.length ? frames[i + 1].t - frames[i].t : FRAME_INTERVAL_S;
list.push(`file '${frames[i].file}'`, `duration ${duration.toFixed(4)}`);
}
list.push(`file '${frames.at(-1).file}'`);
const listFile = path.join(framesDir, 'frames.txt');
await fs.writeFile(listFile, list.join('\n') + '\n');
const video = path.join(shots, 'admin-outlines-journey.mp4');
execFileSync('ffmpeg', [
'-y',
'-loglevel',
'error',
'-f',
'concat',
'-safe',
'0',
'-i',
listFile,
'-vf',
'fps=24,format=yuv420p',
'-c:v',
'libx264',
'-crf',
'23',
video,
]);
const span = frames.at(-1).t - frames[0].t;
result.video = {
file: path.relative(process.cwd(), video),
frames: frames.length,
seconds: Number(span.toFixed(1)),
capturedFps: Number(((frames.length - 1) / span).toFixed(1)),
};
console.log('Video:', result.video);
await fs.rm(framesDir, { recursive: true, force: true });
}
await fs.writeFile(
path.join(shots, 'admin-outlines-result.json'),
JSON.stringify(result, null, 2),
);
console.table(
result.steps.map((s) => ({
step: s.step,
resolvedMs: s.resolvedMs?.toFixed(0),
drawnMs: s.drawnMs?.toFixed(0),
parts: s.parts,
stable: s.stableWhileStill,
p50: (s.stillFrameTimes || s.frameTimes)?.p50,
p95: (s.stillFrameTimes || s.frameTimes)?.p95,
stillDiffP95: s.stillFrameDiff?.meanAbsDiff.p95,
stillDiffMax: s.stillFrameDiff?.meanAbsDiff.max,
})),
);
console.log(process.exitCode ? 'FAILED' : 'PASSED');
function stats(values, digits = 1) {
if (!values.length) return { n: 0 };
const sorted = [...values].sort((a, b) => a - b);
const at = (q) =>
Number(
sorted[
Math.min(sorted.length - 1, Math.floor(q * sorted.length))
].toFixed(digits),
);
return { n: values.length, p50: at(0.5), p95: at(0.95), max: at(1) };
}
scripts/qa-alpr-journey.mjs
+588
View File
@@ -0,0 +1,588 @@
#!/usr/bin/env node
/**
* qa-alpr-journey.mjs — ALPR cameras on a realistic Austin camera journey.
*
* Run: node scripts/qa-alpr-journey.mjs http://localhost:4173 [--headful]
* [--passes off,on,on] [--out qa-shots/alpr-journey] [--no-assert]
*
* Drives the same eased camera path several times: a neighbourhood by Camp
* Mabry (~2 km across), two 600 m pans, a small zoom out and back, downtown
* (~6 km), the whole city (~30 km) and back to the neighbourhood. The first
* pass runs with Mapped ALPR Cameras off (it also warms the photoreal tiles),
* then ALPR is switched on mid-journey from its layer row and the path repeats
* cold and warm.
*
* Per stop it records: time from the camera stopping to cameras shown (or an
* explicit "zoom in" state), the camera count and row status, rAF interval and
* Cesium render CPU time (mean / p50 / p95) while moving and while holding, and
* marker stability while the camera is still (camera-set changes after the stop
* has settled). Asserts, unless --no-assert: neighbourhood and downtown show
* cameras within 2 s on the warm pass; the whole-city stop shows cameras or a
* zoom-in prompt and never ends a hold still loading; at most one visible
* marker update per stop and none after 3 s while still; warm-pass frame p50/p95 within 10% of the ALPR-off pass (plus a
* 2 ms floor for timer noise); zero console errors.
*
* Rendered-state checks read the recorded frames, not entity state: ALPR-cyan
* pixels per frame during each hold, minus the ALPR-off pass at the same stop,
* give when markers actually reached the screen. Cameras projected on screen
* must be painted, no later than 700 ms after the layer state settled (a
* missing render request fails here), within 2 s of the stop on the warm pass;
* a view whose loaded cameras are all off screen must say "None on screen".
* At the end of each hold the screen is compared with the same scene after
* forcing four frames; any marker difference is a missing render request.
* --self-test-drop-renders disables explicit render requests during the ALPR
* passes to prove those checks fail.
*
* Writes <out>/result.json, <out>/clip.mp4 (CDP screencast with real frame
* timing) and per-stop screenshots. Needs ffmpeg on PATH.
*/
import fs from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
import sharp from 'sharp';
import {
createScreencast,
frameStats,
installFrameProbe,
moveCamera,
pageNow,
readFrames,
} from './qa-journey-recorder.mjs';
const argv = process.argv.slice(2);
if (argv.includes('--help') || argv.includes('-h')) {
console.log(
'Usage: node scripts/qa-alpr-journey.mjs <dev-server-url> [--headful] [--passes off,on,on] [--out dir] [--no-assert] [--self-test-drop-renders]',
);
process.exit(0);
}
const option = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] ? argv[i + 1] : fallback;
};
const appUrl =
argv.find((arg) => /^https?:/.test(arg)) || 'http://localhost:4173';
const outDir = path.resolve(option('--out', 'qa-shots/alpr-journey'));
const passes = option('--passes', 'off,on,on').split(',');
const assertOn = !argv.includes('--no-assert');
const dropRenders = argv.includes('--self-test-drop-renders');
fs.mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const cyanCount = async (file) => {
const { data, info } = await sharp(file)
.resize({ width: 640 })
.raw()
.toBuffer({ resolveWithObject: true });
let n = 0;
for (let i = 0; i < data.length; i += info.channels) {
const r = data[i];
const g = data[i + 1];
const b = data[i + 2];
if (r < 140 && g > 160 && b > 205 && b - r > 80) n += 1;
}
return n;
};
/** Cyan pixels (at 640 px width) that mean at least one badge is painted. */
const MIN_MARKER_PIXELS = 12;
// Camp Mabry neighbourhood (Tarrytown / Bryker Woods), downtown, whole city.
const N = {
lat: 30.2985,
lon: -97.7655,
height: 1500,
heading: 20,
pitch: -55,
};
const east = (view, m) => ({
...view,
lon: view.lon + m / (111_320 * Math.cos((view.lat * Math.PI) / 180)),
});
const north = (view, m) => ({ ...view, lat: view.lat + m / 111_320 });
const STOPS = [
{ name: 'neighbourhood', view: N, move: 0, hold: 6, scale: 'neighbourhood' },
{
name: 'pan-east-600m',
view: east(N, 600),
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'pan-north-600m',
view: north(east(N, 600), 600),
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'zoom-out-3km',
view: { ...north(east(N, 600), 600), height: 3000 },
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'zoom-back-in',
view: north(east(N, 600), 600),
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'downtown-6km',
view: { lat: 30.262, lon: -97.7431, height: 4500, heading: 0, pitch: -60 },
move: 3,
hold: 7,
scale: 'downtown',
},
{
name: 'city-30km',
view: { lat: 30.2, lon: -97.75, height: 24000, heading: 0, pitch: -65 },
move: 3,
hold: 8,
scale: 'city',
},
{
name: 'back-to-neighbourhood',
view: N,
move: 3,
hold: 6,
scale: 'neighbourhood',
},
];
const browser = await puppeteer.launch({
headless: !argv.includes('--headful'),
protocolTimeout: 300_000,
args: [
'--no-sandbox',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--disable-backgrounding-occluded-windows',
],
});
const result = {
url: appUrl,
startedAt: new Date().toISOString(),
passes: [],
consoleErrors: [],
failures: [],
};
const fail = (message) => {
result.failures.push(message);
console.log(`[FAIL] ${message}`);
};
let page;
try {
page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
page.on('console', (message) => {
if (message.type() === 'error') result.consoleErrors.push(message.text());
});
page.on('pageerror', (error) => result.consoleErrors.push(error.message));
const url = new URL(appUrl);
url.searchParams.set('welcome', '0');
await page.goto(url.href, { waitUntil: 'domcontentloaded', timeout: 90_000 });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120_000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await installFrameProbe(page);
// ALPR watcher: row state every 100 ms and the rendered camera set (ids and
// visibility) as a cheap hash, so structural churn while still is visible.
await page.evaluate(() => {
const samples = [];
window.__gevAlprSamples = samples;
setInterval(() => {
const entry = window.__godsEyeView.dataManager.layers.get('alpr-cameras');
const stats = entry?.module?.getStats?.() || {};
// Layers that publish a render revision are measured by it; otherwise
// hash the rendered entity ids and their visibility.
let hash = 0;
let shown = 0;
if (Number.isFinite(stats.renderRevision)) {
hash = stats.renderRevision;
shown = stats.shown ?? stats.count ?? 0;
} else {
const entities =
entry?.module && entry.enabled
? window.__godsEyeView.viewer.dataSources.getByName(
'alpr-cameras',
)[0]?.entities.values || []
: [];
for (const entity of entities) {
if (entity.show === false) continue;
shown += 1;
const id = String(entity.id);
for (let i = 0; i < id.length; i++)
hash = (hash * 31 + id.charCodeAt(i)) | 0;
}
}
samples.push({
t: performance.now(),
enabled: Boolean(entry?.enabled),
count: stats.count ?? 0,
shown,
hash,
loading: Boolean(stats.loading),
status: stats.status || null,
label: stats.loadingLabel || '',
onScreen: stats.onScreen ?? null,
});
if (samples.length > 20_000) samples.splice(0, 5_000);
}, 100);
});
const recorder = await createScreencast(page, outDir);
await recorder.start();
async function waitTiles(maxMs = 20_000) {
await page
.waitForFunction(
() => {
const { scene } = window.__godsEyeView.viewer;
if (scene.globe.show) return scene.globe.tilesLoaded;
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (p.show && p.tilesLoaded === false) return false;
}
return true;
},
{ timeout: maxMs, polling: 250 },
)
.catch(() => {});
}
for (const [passIndex, mode] of passes.entries()) {
const alpr = mode === 'on';
const passName = `${passIndex + 1}-${mode}${alpr && passes.slice(0, passIndex).includes('on') ? '-warm' : alpr ? '-cold' : ''}`;
console.log(`Pass ${passName}`);
await moveCamera(page, STOPS[0].view, 0);
await waitTiles();
if (
alpr !==
(await page.evaluate(() =>
window.__godsEyeView.dataManager.isEnabled('alpr-cameras'),
))
) {
// The user's switch: the layer row, not a private API.
await page.evaluate(
(enabled) =>
window.__godsEyeView.dataManager.setEnabled('alpr-cameras', enabled, {
origin: 'user',
}),
alpr,
);
}
if (alpr && dropRenders)
// Self-test: drop explicit render requests so markers reach the screen
// only with the next camera move. The rendered-pixel checks must fail.
await page.evaluate(() => {
window.__godsEyeView.viewer.scene.requestRender = () => {};
});
const pass = { name: passName, alpr, stops: [] };
for (const stop of STOPS) {
const moveStart = await pageNow(page);
if (stop.move) await moveCamera(page, stop.view, stop.move);
else await moveCamera(page, stop.view, 0);
const holdStart = await pageNow(page);
const holdStartEpoch = await page.evaluate(
() => performance.timeOrigin + performance.now(),
);
await sleep(stop.hold * 1000);
const holdEnd = await pageNow(page);
// Render truth: what the screen shows at the end of the hold versus the
// same scene after forcing a few frames. A difference means the layer
// changed state without asking for a render.
let staleFrame = null;
if (alpr) {
const shown = await cyanCount(await page.screenshot({ type: 'png' }));
await page.evaluate(async () => {
const { scene } = window.__godsEyeView.viewer;
const mode = scene.requestRenderMode;
scene.requestRenderMode = false;
for (let i = 0; i < 4; i++)
await new Promise((resolve) => requestAnimationFrame(resolve));
scene.requestRenderMode = mode;
});
const forced = await cyanCount(await page.screenshot({ type: 'png' }));
staleFrame = {
shown,
forced,
stale: Math.abs(forced - shown) > Math.max(8, forced * 0.15),
};
}
const moving = frameStats(await readFrames(page, moveStart, holdStart));
const holding = frameStats(await readFrames(page, holdStart, holdEnd));
const samples = await page.evaluate(
(a, b) => window.__gevAlprSamples.filter((s) => s.t >= a && s.t < b),
holdStart,
holdEnd,
);
// Time to cameras: the last visible change (or the end of loading) after
// the camera stopped. More than one visible change, or any change after
// 3 s, is flicker: a still camera must see one clean update at most.
let visualChanges = 0;
const changeLog = [];
let lastChangeAt = null;
let loadingEndAt = null;
let wasLoading = false;
let previous = null;
for (const s of samples) {
if (
previous &&
(s.hash !== previous.hash || s.shown !== previous.shown)
) {
visualChanges += 1;
lastChangeAt = s.t;
changeLog.push({
atMs: Math.round(s.t - holdStart),
shown: s.shown,
loading: s.loading,
});
}
if (s.loading) wasLoading = true;
else if (wasLoading && loadingEndAt === null) loadingEndAt = s.t;
previous = s;
}
const first = samples[0] || {};
const settledAt = Math.max(
lastChangeAt ?? holdStart,
loadingEndAt ?? (first.loading ? holdEnd : holdStart),
);
const end = samples.at(-1) || {};
const row = {
stop: stop.name,
scale: stop.scale,
holdEpochMs: [holdStartEpoch, holdStartEpoch + (holdEnd - holdStart)],
onScreen: end.onScreen ?? null,
staleFrame,
timeToCamerasMs: alpr ? Math.round(settledAt - holdStart) : null,
cameras: end.count ?? 0,
shown: end.shown ?? 0,
status: end.status,
label: end.label,
loadingAtHoldEnd: Boolean(end.loading),
visualChanges,
changeLog,
lateChanges: samples.filter(
(sample, i) =>
i > 0 &&
sample.t > holdStart + 3000 &&
(sample.hash !== samples[i - 1].hash ||
sample.shown !== samples[i - 1].shown),
).length,
moving,
holding,
};
pass.stops.push(row);
console.log(
` ${stop.name.padEnd(22)} cams ${String(row.cameras).padStart(5)} t ${String(row.timeToCamerasMs ?? '-').padStart(5)} ms frame p50/p95 ${holding.frameP50}/${holding.frameP95} render p50/p95 ${holding.renderP50}/${holding.renderP95} changes ${visualChanges}/${row.lateChanges} ${row.status || ''} ${row.label || ''}`,
);
if (passIndex === passes.length - 1) {
await page.screenshot({
path: path.join(outDir, `${passName}-${stop.name}.png`),
});
}
}
result.passes.push(pass);
}
const clip = await recorder.stop();
result.clip = clip;
// Clip-relative hold windows, so a reviewer can jump to each stop.
const clipStart = recorder.frames[0]?.timestamp * 1000;
for (const pass of result.passes)
for (const stop of pass.stops)
stop.clipSeconds = stop.holdEpochMs.map(
(ms) => Math.round((ms - clipStart) / 100) / 10,
);
console.log('clip', clip);
// Rendered-state check from the recorded frames themselves: count ALPR-cyan
// pixels per frame during each hold, subtract the ALPR-off pass at the same
// stop, and find when the markers actually became visible on screen. Entity
// state can be ready while nothing is painted (a missing render request),
// which this catches and the in-page samples cannot.
const framesIn = ([a, b]) =>
recorder.frames.filter(
(frame) => frame.timestamp * 1000 >= a && frame.timestamp * 1000 < b,
);
const median = (list) =>
list.length ? [...list].sort((x, y) => x - y)[list.length >> 1] : 0;
const offPass = result.passes.find((p) => !p.alpr);
for (const pass of result.passes) {
for (const [index, stop] of pass.stops.entries()) {
const frames = framesIn(stop.holdEpochMs);
const counts = [];
for (const frame of frames)
counts.push({
atMs: Math.round(frame.timestamp * 1000 - stop.holdEpochMs[0]),
cyan: await cyanCount(frame.file),
});
stop.pixelTimeline = counts;
if (!pass.alpr) {
stop.cyanBaseline = median(counts.map((c) => c.cyan));
continue;
}
const baseline = offPass?.stops[index]?.cyanBaseline ?? 0;
const tail = counts.filter(
(c) => c.atMs >= stop.holdEpochMs[1] - stop.holdEpochMs[0] - 1000,
);
const final = median(tail.map((c) => c.cyan)) - baseline;
stop.renderedMarkerPixels = final;
let visibleAt = null;
if (final >= MIN_MARKER_PIXELS) {
for (let i = counts.length - 1; i >= 0; i--) {
if (counts[i].cyan - baseline >= final * 0.8)
visibleAt = counts[i].atMs;
else break;
}
}
stop.renderedVisibleAtMs = visibleAt;
console.log(
` ${pass.name} ${stop.stop.padEnd(22)} on screen ${stop.onScreen ?? '-'} marker px ${final} visible at ${visibleAt ?? '-'} ms (state ${stop.timeToCamerasMs} ms) end-of-hold px ${stop.staleFrame?.shown} / forced ${stop.staleFrame?.forced}`,
);
}
}
// Aggregate and assert.
const aggregate = (pass, phase) => {
const frames = pass.stops.map((s) => s[phase]);
const weighted = (key) => {
const total = frames.reduce((n, f) => n + (f.frames || 0), 0);
return (
Math.round(
(frames.reduce((n, f) => n + (f[key] || 0) * (f.frames || 0), 0) /
Math.max(1, total)) *
10,
) / 10
);
};
return {
frameP50: weighted('frameP50'),
frameP95: weighted('frameP95'),
renderP50: weighted('renderP50'),
renderP95: weighted('renderP95'),
longFrames: frames.reduce((n, f) => n + f.longFrames, 0),
};
};
result.summary = result.passes.map((pass) => ({
pass: pass.name,
moving: aggregate(pass, 'moving'),
holding: aggregate(pass, 'holding'),
}));
console.table(
result.summary.map((s) => ({
pass: s.pass,
'move p50/p95': `${s.moving.frameP50}/${s.moving.frameP95}`,
'hold p50/p95': `${s.holding.frameP50}/${s.holding.frameP95}`,
'render hold p50/p95': `${s.holding.renderP50}/${s.holding.renderP95}`,
'long frames': s.moving.longFrames + s.holding.longFrames,
})),
);
if (assertOn) {
const off = result.summary.find((s) => s.pass.includes('off'));
const warm = result.summary.find((s) => s.pass.endsWith('warm'));
const warmPass = result.passes.find((p) => p.name.endsWith('warm'));
const onPasses = result.passes.filter((p) => p.alpr);
if (off && warm) {
for (const phase of ['moving', 'holding'])
for (const key of ['frameP50', 'frameP95'])
if (warm[phase][key] > off[phase][key] * 1.1 + 2)
fail(
`${phase} ${key} ${warm[phase][key]} ms with ALPR vs ${off[phase][key]} ms without`,
);
}
for (const stop of warmPass?.stops || []) {
if (
stop.scale !== 'city' &&
!(
stop.timeToCamerasMs !== null &&
stop.timeToCamerasMs <= 2000 &&
stop.cameras > 0
)
)
fail(
`${stop.stop}: cameras after ${stop.timeToCamerasMs} ms (${stop.cameras}) on the warm pass`,
);
}
for (const pass of onPasses)
for (const stop of pass.stops) {
if (stop.staleFrame?.stale)
fail(
`${pass.name} ${stop.stop}: screen showed ${stop.staleFrame.shown} marker px, ${stop.staleFrame.forced} after forcing a render`,
);
// Cameras on screen must be painted, and painted when state says so.
if (stop.onScreen > 0 && stop.renderedMarkerPixels < MIN_MARKER_PIXELS)
fail(
`${pass.name} ${stop.stop}: ${stop.onScreen} cameras on screen but no markers rendered`,
);
if (
stop.renderedVisibleAtMs !== null &&
stop.renderedVisibleAtMs > stop.timeToCamerasMs + 700
)
fail(
`${pass.name} ${stop.stop}: markers rendered ${stop.renderedVisibleAtMs} ms after the stop, ${stop.timeToCamerasMs} ms after state`,
);
if (
pass.name.endsWith('warm') &&
stop.scale !== 'city' &&
stop.renderedVisibleAtMs !== null &&
stop.renderedVisibleAtMs > 2000
)
fail(
`${pass.name} ${stop.stop}: markers visible only after ${stop.renderedVisibleAtMs} ms`,
);
if (
stop.cameras > 0 &&
stop.onScreen === 0 &&
!/none on screen/i.test(stop.label)
)
fail(
`${pass.name} ${stop.stop}: no camera on screen and no explicit state`,
);
if (stop.loadingAtHoldEnd)
fail(`${pass.name} ${stop.stop}: still loading when the hold ended`);
if (
stop.scale === 'city' &&
!(stop.cameras > 0 || /zoom in/i.test(stop.label))
)
fail(
`${pass.name} ${stop.stop}: neither cameras nor a zoom-in prompt`,
);
if (stop.visualChanges > 1 || stop.lateChanges > 0)
fail(
`${pass.name} ${stop.stop}: ${stop.visualChanges} marker changes (${stop.lateChanges} after 3 s) while the camera was still`,
);
}
const errors = result.consoleErrors.filter(
(text) => !/favicon/i.test(text),
);
if (errors.length)
fail(`console errors: ${errors.slice(0, 3).join(' | ')}`);
}
} catch (error) {
fail(error.stack || error.message);
} finally {
fs.writeFileSync(
path.join(outDir, 'result.json'),
JSON.stringify(result, null, 1),
);
await browser.close();
}
console.log(
`${result.failures.length ? 'FAIL' : 'PASS'} — ${path.join(outDir, 'result.json')}`,
);
process.exit(result.failures.length ? 1 : 0);
scripts/qa-cockpit-utility.mjs
+92
View File
@@ -1943,6 +1943,98 @@ try {
}),
);
// Cyber relocates its collapsed briefing header into the expanded utility
// lane. Exercise actual height changes without reload and measure both cards.
await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
manager._setHudVariant('cyber');
manager.hud.setMode('on');
manager._setCockpitDisclosure('display', false);
manager._setCockpitDisclosure('radio', true);
});
const cyberRadioOrder = await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
manager.cockpitView.syncSignalLayout();
const display = document
.getElementById('cockpit-display-toggle-btn')
.closest('.cockpit-utility-control');
const radio = document
.getElementById('cockpit-radio-toggle-btn')
.closest('.cockpit-utility-control');
const displayRect = display.getBoundingClientRect();
const radioRect = radio.getBoundingClientRect();
return {
display: displayRect.toJSON(),
radio: radioRect.toJSON(),
displayExpanded: display.classList.contains('is-expanded'),
radioExpanded: radio.classList.contains('is-expanded'),
gap: radioRect.top - displayRect.bottom,
};
});
check(
'Cyber expanding Radio preserves the authored Display-then-Radio order',
!cyberRadioOrder.displayExpanded
&& cyberRadioOrder.radioExpanded
&& cyberRadioOrder.display.height > 0
&& cyberRadioOrder.radio.height > 0
&& cyberRadioOrder.gap >= 7,
JSON.stringify(cyberRadioOrder),
);
await page.screenshot({ path: path.join(shotsDir, 'cyber-radio-order.png') });
await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
manager._setCockpitDisclosure('radio', false);
manager._setCockpitDisclosure('display', true);
});
const cyberStyleStates = await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
const states = [];
for (const style of ['normal', 'surveillance', 'thermal', 'retro']) {
document.querySelector(`.style-btn[data-style="${style}"]`).click();
states.push({
requestedStyle: style,
activeStyle: manager.activeStyle,
hud: manager.hud.getVariant(),
theme: document.documentElement.dataset.uiTheme,
selector: document.getElementById('hud-layout-select').value,
savedHud: manager.getVisualState().hud.variant,
});
}
return states;
});
check('Cockpit visual-style controls retain Cyber HUD, selector, shell and saved layout',
cyberStyleStates.every(state => state.activeStyle === state.requestedStyle
&& state.hud === 'cyber' && state.theme === 'cyber'
&& state.selector === 'cyber' && state.savedHud === 'cyber'),
JSON.stringify(cyberStyleStates));
for (const height of [987, 720, 640, 900]) {
await page.setViewport({ width: 1440, height, deviceScaleFactor: 1 });
await new Promise((resolve) => setTimeout(resolve, 400));
const geometry = await page.evaluate(() => {
const signal = document.getElementById('cockpit-signal-stream');
const utility = document.getElementById('cockpit-utility-controls');
const expanded = utility.querySelector('.cockpit-utility-control.is-expanded');
const s = signal.getBoundingClientRect();
const u = expanded.getBoundingClientRect();
const toggle = document.getElementById('cockpit-signal-toggle');
const t = toggle.getBoundingClientRect();
const hit = document.elementFromPoint(t.x + t.width / 2, t.y + t.height / 2);
return { height: innerHeight, theme: document.documentElement.dataset.uiTheme,
collapsed: signal.dataset.collapsed, signal: s.toJSON(), utility: u.toJSON(),
reachable: Boolean(hit && toggle.contains(hit)),
contained: u.bottom <= s.top - 7 && s.bottom < innerHeight,
};
});
check(`Cyber collapsed Live Signals remains visible and reachable after resize to ${height}px`,
geometry.theme === 'cyber' && geometry.collapsed === 'true'
&& geometry.signal.width > 0 && geometry.signal.height > 0
&& geometry.reachable && geometry.contained, JSON.stringify(geometry));
if (height === 640 || height === 900) {
await page.screenshot({ path: path.join(shotsDir, `cyber-resize-${height}.png`) });
}
}
await page.evaluate(() => window.__godsEyeView.styleManager._setHudVariant('operator'));
const signalTransition = await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
const cockpit = manager.cockpitView;
scripts/qa-context-arrivals.mjs
+590
View File
@@ -0,0 +1,590 @@
#!/usr/bin/env node
/**
* qa-context-arrivals.mjs — search arrivals above the surface, and Contacts
* listing mapped installations around a moving tracked aircraft.
*
* Run: node scripts/qa-context-arrivals.mjs http://localhost:4173 [--headful]
* [--out qa-shots/context-arrivals] [--only arrivals|contacts]
*
* Arrivals: types "Camp Mabry" and "39.7392, -104.9903" (downtown Denver)
* into the location search, waits for the flight and the photoreal tiles,
* orbits a little like a user looking around, and asserts the camera sits at
* least 60 m above the rendered surface under it and under the view centre.
*
* Contacts: serves two synthetic aircraft through the page's own
* /api/opensky response (one flying north over Camp Mabry at 2,500 m, one
* near Fort Cavazos) and an empty military feed, starts from a wide Texas
* view, and presses CONTACTS as a user would. Asserts the Mapped
* installations row lists at least one site with a distance within 15 s,
* never falls back to "?" while the aircraft moves (30 s), keeps its site
* entities (no rebuilds while the view is still), still lists sites in
* Cockpit, and that SEARCH NEARBY SITES reports what it found. Then re-enters
* Contacts on the Fort Cavazos aircraft and asserts a site within 30 km.
* Requires zero console errors.
*
* Writes <out>/result.json, screenshots and <out>/clip.mp4 (CDP screencast
* with real frame timing; needs ffmpeg).
*/
import fs from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
import { createScreencast, moveCamera } from './qa-journey-recorder.mjs';
const argv = process.argv.slice(2);
if (argv.includes('--help') || argv.includes('-h')) {
console.log(
'Usage: node scripts/qa-context-arrivals.mjs <dev-server-url> [--headful] [--out dir] [--only arrivals|contacts]',
);
process.exit(0);
}
const option = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] ? argv[i + 1] : fallback;
};
const appUrl =
argv.find((arg) => /^https?:/.test(arg)) || 'http://localhost:4173';
const outDir = path.resolve(option('--out', 'qa-shots/context-arrivals'));
const only = option('--only', null);
fs.mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const result = {
url: appUrl,
arrivals: [],
contacts: {},
consoleErrors: [],
failures: [],
};
const check = (name, ok, detail = '') => {
console.log(
`[${ok ? 'PASS' : 'FAIL'}] ${name}${detail ? ` — ${detail}` : ''}`,
);
if (!ok) result.failures.push(`${name}${detail ? ` — ${detail}` : ''}`);
};
const browser = await puppeteer.launch({
headless: !argv.includes('--headful'),
protocolTimeout: 300_000,
args: [
'--no-sandbox',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--disable-backgrounding-occluded-windows',
],
});
async function openPage({ aircraft = false } = {}) {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
page.on('console', (message) => {
if (message.type() === 'error') result.consoleErrors.push(message.text());
});
page.on('pageerror', (error) => result.consoleErrors.push(error.message));
page.on('response', (response) => {
if (response.status() < 400) return;
const url = new URL(response.url());
// Track/enrichment lookups for the synthetic ICAOs have no upstream record.
const synthetic = /aaa05\d/i.test(url.search + url.pathname);
(synthetic
? (result.expectedMisses ??= [])
: (result.httpErrors ??= [])
).push(`${response.status()} ${url.pathname}${url.search}`);
});
await page.evaluateOnNewDocument((aircraft) => {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
if (!aircraft) return;
// Synthetic traffic through the app's own feed endpoints: one aircraft
// flying north over Camp Mabry, one loitering near Fort Cavazos.
const started = Date.now();
const realFetch = window.fetch.bind(window);
window.fetch = (input, init) => {
const raw =
typeof input === 'string' || input instanceof URL
? String(input)
: input?.url;
let url;
try {
url = new URL(raw, location.href);
} catch {
return realFetch(input, init);
}
if (url.origin === location.origin && url.pathname === '/api/opensky') {
const now = Math.floor(Date.now() / 1000);
const seconds = (Date.now() - started) / 1000;
const lat = 30.29 + (seconds * 120) / 111_320;
const json = {
time: now,
states: [
[
'aaa051',
'QA051',
'Synthetic',
now,
now,
-97.765,
lat,
2500,
false,
120,
0,
0,
null,
2500,
null,
false,
0,
],
[
'aaa052',
'QA052',
'Synthetic',
now,
now,
-97.72,
31.12,
3000,
false,
90,
270,
0,
null,
3000,
null,
false,
0,
],
],
};
return Promise.resolve(Response.json(json));
}
if (url.origin === location.origin && url.pathname === '/api/adsblol/mil')
return Promise.resolve(
Response.json({ ac: [], now: Date.now(), total: 0 }),
);
return realFetch(input, init);
};
}, aircraft);
const url = new URL(appUrl);
url.searchParams.set('welcome', '0');
await page.goto(url.href, { waitUntil: 'domcontentloaded', timeout: 90_000 });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120_000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
return page;
}
async function waitTiles(page, maxMs = 30_000) {
await page
.waitForFunction(
() => {
const { scene } = window.__godsEyeView.viewer;
if (scene.globe.show) return scene.globe.tilesLoaded;
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (p.show && p.tilesLoaded === false) return false;
}
return true;
},
{ timeout: maxMs, polling: 250 },
)
.catch(() => {});
}
/** Camera height above the rendered surface under it and under the view centre. */
function clearance(page) {
return page.evaluate(() => {
const { viewer } = window.__godsEyeView;
const { scene, camera } = viewer;
const carto = camera.positionCartographic.clone();
const sample = (c) => {
try {
const h = scene.sampleHeight(c);
return Number.isFinite(h) ? h : null;
} catch {
return null;
}
};
const under = sample(carto);
let centre = null;
const canvas = scene.canvas;
const hit = scene.pickPosition?.({
x: canvas.clientWidth / 2,
y: canvas.clientHeight / 2,
});
if (hit) centre = scene.globe.ellipsoid.cartesianToCartographic(hit);
const centreSurface = centre ? sample(centre) : null;
const surface = Math.max(under ?? -Infinity, centreSurface ?? -Infinity);
return {
lat: (carto.latitude * 180) / Math.PI,
lon: (carto.longitude * 180) / Math.PI,
cameraM: Math.round(carto.height),
surfaceUnderM: under === null ? null : Math.round(under),
surfaceCentreM: centreSurface === null ? null : Math.round(centreSurface),
clearanceM: Number.isFinite(surface)
? Math.round(carto.height - surface)
: null,
pitchDeg: Math.round((camera.pitch * 180) / Math.PI),
};
});
}
async function arrivals() {
const page = await openPage();
const dir = path.join(outDir, 'arrivals');
fs.mkdirSync(dir, { recursive: true });
const recorder = await createScreencast(page, dir);
await recorder.start();
for (const query of ['Camp Mabry', '39.7392, -104.9903']) {
await page.evaluate(() =>
document.getElementById('search-toggle')?.click(),
);
await page.focus('#location-search');
await page.evaluate(() => {
document.getElementById('location-search').value = '';
});
await page.type('#location-search', query, { delay: 40 });
await page.keyboard.press('Enter');
// Wait for the flight to land: the camera still for 1.5 s.
const landed = Date.now();
let last = null;
let stillSince = null;
while (Date.now() - landed < 30_000) {
const now = await page.evaluate(() => {
const c = window.__godsEyeView.viewer.camera.positionWC;
return [c.x, c.y, c.z];
});
const moved =
!last ||
Math.hypot(now[0] - last[0], now[1] - last[1], now[2] - last[2]) > 0.5;
last = now;
if (moved) stillSince = null;
else stillSince ??= Date.now();
if (
Date.now() - landed > 4000 &&
stillSince &&
Date.now() - stillSince > 1500
)
break;
await sleep(250);
}
await waitTiles(page);
await sleep(2500); // the ground guard measures after tiles stream in
const arrived = await clearance(page);
const slug = query.replace(/\W+/g, '-').replace(/^-|-$/g, '').toLowerCase();
await page.screenshot({ path: path.join(dir, `arrival-${slug}.png`) });
// Look around: a gentle orbit, then check the eye still clears the ground.
const view = await page.evaluate(() => {
const c = window.__godsEyeView.viewer.camera;
return {
lat: (c.positionCartographic.latitude * 180) / Math.PI,
lon: (c.positionCartographic.longitude * 180) / Math.PI,
height: c.positionCartographic.height,
heading: (c.heading * 180) / Math.PI,
pitch: (c.pitch * 180) / Math.PI,
};
});
await moveCamera(page, { ...view, heading: view.heading + 25 }, 2.5);
await sleep(1500);
const lookedAround = await clearance(page);
result.arrivals.push({ query, arrived, lookedAround });
check(
`${query}: arrival clears the rendered surface`,
arrived.clearanceM !== null && arrived.clearanceM >= 60,
JSON.stringify(arrived),
);
}
result.arrivalClip = await recorder.stop();
await page.close();
}
async function contacts() {
const page = await openPage({ aircraft: true });
const dir = path.join(outDir, 'contacts');
fs.mkdirSync(dir, { recursive: true });
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('flights', true, {
origin: 'user',
}),
);
// A wide Texas view: installation viewports are unbounded here.
await moveCamera(
page,
{ lat: 30.35, lon: -97.76, height: 900_000, pitch: -90 },
0,
);
await sleep(3000);
// In-page probe: row text and installation entity churn every 250 ms.
await page.evaluate(() => {
const samples = [];
window.__gevContextSamples = samples;
let created = 0;
const seen = new WeakSet();
setInterval(() => {
const row = [
...document.querySelectorAll(
'#military-awareness-panel .military-awareness-row',
),
].find(
(candidate) =>
candidate.querySelector('strong')?.textContent?.trim() ===
'Mapped installations',
);
const ds = window.__godsEyeView.viewer.dataSources.getByName(
'military-installations',
)[0];
for (const entity of ds?.entities.values || [])
if (!seen.has(entity)) {
seen.add(entity);
created += 1;
}
samples.push({
t: performance.now(),
count: row?.querySelector('b')?.textContent?.trim() ?? null,
text: row?.innerText?.replace(/\s+/g, ' ').trim() ?? null,
installations: [
...(row?.querySelectorAll('button[data-awareness-id]') || []),
].map((button) => ({
id: button.dataset.awarenessId,
name: button.childNodes[0]?.textContent?.trim(),
distance: button.querySelector('span')?.textContent?.trim(),
})),
entities: ds?.entities.values.length ?? 0,
created,
tracked:
window.__godsEyeView.viewer.trackedEntity?.gevTrackedId ?? null,
cockpit: document.body.classList.contains('cockpit-mode'),
});
if (samples.length > 5000) samples.splice(0, 1000);
}, 250);
});
const recorder = await createScreencast(page, dir);
await recorder.start();
const t0 = await page.evaluate(() => performance.now());
await page.evaluate(() =>
document.getElementById('global-context-flights-btn').click(),
);
await page
.waitForFunction(
() =>
window.__godsEyeView.viewer.trackedEntity?.gevTrackedId ===
'flights:aaa051',
{ timeout: 60_000 },
)
.catch(() => {});
const readSamples = (from) =>
page.evaluate(
(a) => window.__gevContextSamples.filter((s) => s.t >= a),
from,
);
const numeric = (s) => /^\d+/.test(String(s.count ?? ''));
// Up to 15 s for the first answer, then watch 30 s of flight.
const deadline = Date.now() + 15_000;
let first = null;
while (Date.now() < deadline && !first) {
first = (await readSamples(t0)).find(
(s) =>
numeric(s) &&
Number.parseInt(s.count, 10) >= 1 &&
/\d+(\.\d+)? k?m\b/.test(s.text),
);
if (!first) await sleep(500);
}
check(
'Contacts lists a mapped installation with a distance',
Boolean(first),
first
? `${Math.round(first.t - t0)} ms: ${first.text}`
: 'row never answered',
);
await page.screenshot({ path: path.join(dir, 'contacts-camp-mabry.png') });
await sleep(30_000);
const flight = (await readSamples(first?.t ?? t0)).filter(
(s) => s.tracked === 'flights:aaa051',
);
const reverted = flight.filter((s) => !numeric(s));
const createdDuring = flight.length
? flight.at(-1).created - flight[0].created
: 0;
result.contacts.mabry = {
firstAnswerMs: first ? Math.round(first.t - t0) : null,
firstText: first?.text ?? null,
lastText: flight.at(-1)?.text ?? null,
samples: flight.length,
revertedSamples: reverted.length,
entities: flight.at(-1)?.entities ?? 0,
entitiesCreatedWhileTracking: createdDuring,
};
check(
'the row never falls back to "?" while the aircraft moves',
reverted.length === 0,
`${reverted.length}/${flight.length}`,
);
check(
'site entities are not rebuilt while tracking',
createdDuring === 0,
`${createdDuring} created`,
);
// Cockpit: the same context must hold with a horizon view.
const cockpit = await page.evaluate(() => {
const view = window.__godsEyeView.styleManager.cockpitView;
view.syncEntry();
return view.enter();
});
await sleep(12_000);
const inCockpit = (
await readSamples((await page.evaluate(() => performance.now())) - 8000)
).filter((s) => s.cockpit);
const scope = await page.evaluate(
() =>
document.getElementById('cockpit-context-subject')?.textContent || null,
);
await page.screenshot({ path: path.join(dir, 'cockpit-camp-mabry.png') });
result.contacts.cockpit = {
entered: cockpit,
samples: inCockpit.length,
last: inCockpit.at(-1)?.text ?? null,
scope,
};
check(
'Cockpit scope names the installation window',
/INSTALLATIONS WITHIN \d+ KM/.test(scope || ''),
scope || 'no scope',
);
check(
'Cockpit keeps mapped installations listed',
cockpit !== false &&
inCockpit.length > 0 &&
inCockpit.every((s) => numeric(s) && Number.parseInt(s.count, 10) >= 1),
inCockpit.at(-1)?.text || 'no cockpit samples',
);
await page.evaluate(() =>
window.__godsEyeView.styleManager.cockpitView.exit?.(),
);
await sleep(2000);
// SEARCH NEARBY SITES, as the user presses it.
await page.evaluate(() => {
document.getElementById('toast').textContent = '';
document.getElementById('installations-search-btn').click();
});
await page
.waitForFunction(
() => document.getElementById('toast').textContent.trim().length > 0,
{ timeout: 20_000 },
)
.catch(() => {});
const toast = await page.evaluate(() =>
document.getElementById('toast').textContent.trim(),
);
await sleep(1500);
const afterSearch = (
await readSamples((await page.evaluate(() => performance.now())) - 1000)
).at(-1);
result.contacts.search = { toast, row: afterSearch?.text ?? null };
check(
'SEARCH NEARBY SITES reports what it found and the row still lists sites',
/mapped sites? within \d+ km/i.test(toast) && numeric(afterSearch || {}),
`toast "${toast}" · ${afterSearch?.text}`,
);
// Fort Cavazos: leave Contacts, track the other aircraft, re-enter.
await page.evaluate(() =>
document.getElementById('global-context-flights-btn').click(),
);
await sleep(2000);
await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('flights')
.module.trackById('aaa052'),
);
await sleep(3000);
const t1 = await page.evaluate(() => performance.now());
await page.evaluate(() =>
document.getElementById('global-context-flights-btn').click(),
);
let cavazos = null;
const cavazosDeadline = Date.now() + 20_000;
while (Date.now() < cavazosDeadline) {
const latest = (await readSamples(t1))
.filter((s) => s.tracked === 'flights:aaa052' && numeric(s))
.at(-1);
const site = latest?.installations.find(
(record) => record.name === 'Fort Hood',
);
const nearest = site?.distance?.match(/^(\d+(?:\.\d+)?) (k?m)$/);
const km = nearest
? Number(nearest[1]) / (nearest[2] === 'm' ? 1000 : 1)
: null;
if (km !== null && km <= 30) {
cavazos = {
text: latest.text,
nearestKm: km,
site,
afterMs: Math.round(latest.t - t1),
};
break;
}
await sleep(500);
}
await sleep(2000);
await page.screenshot({ path: path.join(dir, 'contacts-fort-cavazos.png') });
result.contacts.cavazos = cavazos;
check(
'Fort Cavazos contact lists Fort Hood within 30 km',
Boolean(cavazos),
JSON.stringify(cavazos),
);
result.contactsClip = await recorder.stop();
await page.close();
}
try {
if (only !== 'contacts') await arrivals();
if (only !== 'arrivals') await contacts();
// A 404 for a synthetic aircraft's track logs "Failed to load resource";
// count those only when every failed response was such an expected miss.
const resourceErrors = result.consoleErrors.filter((text) =>
/Failed to load resource/.test(text),
).length;
const errors = result.consoleErrors.filter(
(text) =>
!/favicon/i.test(text) &&
!(
/Failed to load resource/.test(text) &&
!(result.httpErrors || []).length &&
resourceErrors <= (result.expectedMisses || []).length
),
);
check(
'zero console errors',
errors.length === 0,
errors.slice(0, 3).join(' | '),
);
check(
'zero failed app responses',
!(result.httpErrors || []).length,
(result.httpErrors || []).slice(0, 3).join(' | '),
);
} catch (error) {
check('harness completed', false, error.stack || error.message);
} finally {
fs.writeFileSync(
path.join(outDir, 'result.json'),
JSON.stringify(result, null, 1),
);
await browser.close();
}
console.log(
`${result.failures.length ? 'FAIL' : 'PASS'} — ${path.join(outDir, 'result.json')}`,
);
process.exit(result.failures.length ? 1 : 0);
scripts/qa-final-polish-credits.mjs
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env node
/** Persistent inline attribution through actual OSM layer display lifetimes. */
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import puppeteer from 'puppeteer';
import { moveCamera } from './qa-journey-recorder.mjs';
const url = process.argv.find((a) => /^https?:/.test(a));
if (!url) throw new Error('Supply the running server URL');
const dir = process.env.QA_POLISH_DIR || 'qa-shots/final-polish';
await fs.mkdir(dir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const result = { states: [], errors: [] };
const browser = await puppeteer.launch({
headless: true,
args: ['--use-angle=metal'],
});
try {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
await page.goto(`${url}/?welcome=0`, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.styleManager.setDetection({ enabled: false });
});
await moveCamera(
page,
{ lon: -97.765, lat: 30.3125, height: 3500, pitch: -80 },
0,
);
const toggle = (id, value) =>
page.evaluate(
(id, value) => window.__godsEyeView.dataManager.setEnabled(id, value),
id,
value,
);
const text = () => page.$eval('#cesium-credits', (e) => e.innerText);
const shot = async (name, osm, tiles) => {
await sleep(300);
const state = await page.$eval('#cesium-credits', (e) => {
const nodes = [
...e.querySelectorAll('a,img,.cesium-credit-expand-link'),
].filter((n) =>
n.checkVisibility({ visibilityProperty: true, opacityProperty: true }),
);
return {
text: e.innerText,
rects: nodes
.map((n) => {
const b = n.getBoundingClientRect();
return {
text: n.textContent || n.alt || 'image',
x: b.x,
y: b.y,
width: b.width,
height: b.height,
reachable: n.contains(
document.elementFromPoint(
b.x + b.width / 2,
b.y + b.height / 2,
),
),
};
})
.filter((b) => b.width > 0 && b.height > 0),
};
});
assert.equal(state.text.includes('© OpenStreetMap'), osm);
assert.equal(state.text.includes('© OpenMapTiles'), tiles);
assert.equal(
(state.text.match(/© OpenStreetMap/g) || []).length,
osm ? 1 : 0,
);
assert.ok(state.rects.length >= 3);
const centers = state.rects.map((b) => b.y + b.height / 2);
assert.ok(
Math.max(...centers) - Math.min(...centers) < 12,
`${name}: credits stay on one desktop row`,
);
assert.ok(
state.rects.every(
(b) => b.x >= 0 && b.x + b.width <= page.viewport().width,
),
'no credit clipped',
);
assert.ok(
state.rects.every((b) => b.reachable),
'all credits and the attribution link remain clear of controls',
);
await page.screenshot({ path: `${dir}/after-${name}.png` });
await page.screenshot({
path: `${dir}/after-${name}-row.png`,
clip: { x: 16, y: 750, width: 1150, height: 150 },
});
result.states.push({ name, ...state });
};
await sleep(1600);
await shot('nothing', false, false);
await toggle('alpr-cameras', true);
await page.waitForFunction(
() =>
document
.querySelector('#cesium-credits')
.innerText.includes('© OpenStreetMap'),
{ timeout: 60000 },
);
await shot('alpr', true, false);
await sleep(6000);
await shot('alpr-persistent', true, false);
await toggle('traffic', true);
await page.waitForFunction(
() =>
document
.querySelector('#cesium-credits')
.innerText.includes('© OpenMapTiles'),
{ timeout: 60000 },
);
await shot('alpr-and-traffic', true, true);
await toggle('alpr-cameras', false);
await shot('traffic', true, true);
for (const width of [1280, 1920]) {
await page.setViewport({ width, height: 900 });
await sleep(500);
await shot(`traffic-${width}`, true, true);
}
await page.setViewport({ width: 1440, height: 900 });
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(true),
);
await shot('traffic-clean-ui', true, true);
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(false),
);
await toggle('traffic', false);
await shot('nothing-restored', false, false);
assert.deepEqual(result.errors, []);
console.log(JSON.stringify(result, null, 2));
} finally {
await fs.writeFile(`${dir}/credits.json`, JSON.stringify(result, null, 2));
await browser.close();
}
scripts/qa-installation-navigation.mjs
+412
View File
@@ -0,0 +1,412 @@
#!/usr/bin/env node
/** Mapped installations: isolated camera navigation, cached revisits and Contacts release. */
import fs from 'node:fs';
import puppeteer from 'puppeteer';
import { moveCamera } from './qa-journey-recorder.mjs';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-installation-navigation.mjs <url> [--baseline]',
);
process.exit(0);
}
const baseline = args.includes('--baseline');
const result = { views: [], errors: [], forbidden: [], failures: [] };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const browser = await puppeteer.launch({
headless: true,
args: ['--use-angle=metal', '--disable-background-timer-throttling'],
protocolTimeout: 120000,
});
try {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
const tiles = [];
page.on('request', (r) => {
const u = new URL(r.url());
if (
u.hostname === 'tiles.openfreemap.org' &&
/\/\d+\/\d+\/\d+(\.pbf)?$/.test(u.pathname)
)
tiles.push(u.pathname);
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
result.forbidden.push(u.hostname);
});
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
await page.goto(
args.find((a) => /^https?:/.test(a)),
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
window.installationEvents = { changed: 0, end: 0 };
window.installationEntityRefs = new Map();
g.viewer.camera.changed.addEventListener(
() => window.installationEvents.changed++,
);
g.viewer.camera.moveEnd.addEventListener(
() => window.installationEvents.end++,
);
});
let low = 60000,
high = 400000;
for (let i = 0; i < 9; i++) {
const height = (low + high) / 2;
await moveCamera(page, { lat: 32.72, lon: -117.16, height, pitch: -90 }, 0);
const fits = await page.evaluate(async () => {
const { installationTileZoom } =
await import('/src/layers/installations/source.js');
const r = window.__godsEyeView.viewer.camera.computeViewRectangle();
const d = 180 / Math.PI;
return (
r &&
installationTileZoom({
west: r.west * d,
east: r.east * d,
south: r.south * d,
north: r.north * d,
}) !== null
);
});
if (fits) low = height;
else high = height;
}
const widestHeight = Math.floor(low);
result.widestHeight = widestHeight;
result.firstTooWideHeight = Math.ceil(high);
let enabled = false;
const places = [
['san-diego', 32.72, -117.16],
['pendleton', 33.33, -117.4],
['miramar', 32.87, -117.14],
['san-diego-return', 32.72, -117.16],
];
for (const height of [
5000,
20000,
60000,
150000,
widestHeight,
400000,
5000,
]) {
for (const [name, lat, lon] of height >= 150000
? places.slice(0, 1)
: places) {
const before = tiles.length;
const previous = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().lastUpdate,
);
await moveCamera(
page,
{ lat, lon, height, pitch: -90 },
enabled ? 0.6 : 0,
);
const stop = Date.now();
if (!enabled) {
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'military-installations',
true,
),
);
enabled = true;
}
await page
.waitForFunction(
(prev, wide) => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return (
!s.loading &&
(s.lastUpdate > prev || (wide && s.status === 'zoom-in'))
);
},
{ timeout: 6000 },
previous,
height === 400000,
)
.catch(() => {});
const snapshot = await page.evaluate(() => {
const g = window.__godsEyeView;
const ds = g.viewer.dataSources.getByName('military-installations')[0];
const points = ds.entities.values.filter((e) => e.point);
const stable = points.every(
(e) =>
!window.installationEntityRefs.has(e.id) ||
window.installationEntityRefs.get(e.id) === e,
);
window.installationEntityRefs = new Map(points.map((e) => [e.id, e]));
return {
...g.dataManager.layers
.get('military-installations')
.module.getStats(),
stable,
ids: points.map((e) => e.id),
events: { ...window.installationEvents },
};
});
const row = {
name,
height,
ms: Date.now() - stop,
tiles: tiles.length - before,
updated: snapshot.lastUpdate > previous,
...snapshot,
};
if (
(!row.stable && result.views.at(-1)?.height === height) ||
new Set(row.ids).size !== row.ids.length
)
result.failures.push(
`${name}/${height}: unstable or duplicate site entities`,
);
if (name.endsWith('return')) {
const first = result.views.find(
(v) => v.name === 'san-diego' && v.height === height,
);
if (
first &&
JSON.stringify([...first.ids].sort()) !==
JSON.stringify([...row.ids].sort())
)
result.failures.push(
`${name}/${height}: changed site ids on revisit`,
);
}
result.views.push(row);
console.log(JSON.stringify(row));
if (!row.updated && row.status !== 'zoom-in')
result.failures.push(`${name}/${height}: no reload`);
if (row.ms > 1500 && row.status !== 'zoom-in')
result.failures.push(`${name}/${height}: ${row.ms} ms`);
if (name.endsWith('return') && row.tiles)
result.failures.push(`${name}/${height}: revisit fetched ${row.tiles}`);
}
}
for (const gesture of [
'wheel',
'keyboard',
'contacts-close',
'contacts-clear',
]) {
let previous = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().lastUpdate,
);
if (gesture === 'wheel') {
await page.mouse.move(640, 400);
await page.mouse.wheel({ deltaY: -350 });
} else if (gesture === 'keyboard') {
await page.evaluate(() =>
document.getElementById('search-toggle').click(),
);
await page.focus('#location-search');
await page.evaluate(() => {
document.getElementById('location-search').value = '';
});
await page.type('#location-search', '33.33, -117.40');
await page.keyboard.press('Enter');
await sleep(4500);
} else {
await moveCamera(
page,
{ lat: 32.87, lon: -117.14, height: 5000, pitch: -90 },
0.6,
);
await page.waitForFunction(
() =>
window.__godsEyeView.viewer.dataSources
.getByName('military-installations')[0]
.entities.values.some((e) => e.point),
{ timeout: 5000 },
);
await page.evaluate(() => {
const button = document.getElementById('global-context-flights-btn');
if (button.getAttribute('aria-selected') !== 'true') button.click();
});
await page.waitForFunction(
() => {
const g = window.__godsEyeView;
return (
g.dataManager.isEffectivelyEnabled('military-awareness') &&
!g.dataManager.layers.get('military-awareness').module.getParams()
.passive
);
},
{ timeout: 15000 },
);
await sleep(1000);
await page.evaluate(async () => {
const g = window.__godsEyeView;
const ds = g.viewer.dataSources.getByName('military-installations')[0];
const id = ds.entities.values.find((e) => e.point).id;
window.installationSubjectId = id;
await g.dataManager.layers
.get('military-awareness')
.module.focusTarget('military-installations', id);
});
await page.waitForFunction(
() => {
const a =
window.__godsEyeView.dataManager.layers.get(
'military-awareness',
).module;
const subject = a.getContextSnapshot()?.subject;
return (
subject?.layerId === 'military-installations' &&
subject.id === window.installationSubjectId &&
!a._getAwarenessNavigationStateForTest().pendingSelectionKey
);
},
{ timeout: 15000 },
);
await page.waitForFunction(
() => !window.__godsEyeView.viewer.camera._currentFlight,
{ timeout: 10000 },
);
await page.waitForFunction(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().coverage.kind === 'subject',
{ timeout: 10000 },
);
await page.waitForFunction(
() =>
document
.getElementById('global-context-flights-btn')
.getAttribute('aria-busy') === 'false',
{ timeout: 30000 },
);
const anchored = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats(),
);
if (anchored.coverage.kind !== 'subject')
result.failures.push(`${gesture}: Contacts did not acquire subject`);
result.contextClear = await page.evaluate(async (clear) => {
if (clear) {
const { clearSelectedEntityContextForLayer, getContextStore } =
await import('/src/data/contextStore.js');
const store = getContextStore();
const before = {
selectedId: store.selectedEntityId,
record: store.entities.get(store.selectedEntityId)?.id,
subject: window.__godsEyeView.dataManager.layers
.get('military-awareness')
.module.getContextSnapshot()?.subject,
};
clearSelectedEntityContextForLayer('military-installations');
return before;
} else document.getElementById('global-context-flights-btn').click();
}, gesture === 'contacts-clear');
if (gesture === 'contacts-clear') {
await page.waitForFunction(
() =>
!window.__godsEyeView.dataManager.layers
.get('military-awareness')
.module.getContextSnapshot(),
{ timeout: 5000 },
);
} else {
await page.waitForFunction(
() => {
const button = document.getElementById(
'global-context-flights-btn',
);
return (
button.getAttribute('aria-selected') === 'false' &&
button.getAttribute('aria-busy') === 'false'
);
},
{ timeout: 30000 },
);
}
await page.waitForFunction(
() => !window.__godsEyeView.viewer.camera._currentFlight,
{ timeout: 10000 },
);
await sleep(500);
previous = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().lastUpdate,
);
await moveCamera(
page,
{ lat: 33.33, lon: -117.4, height: 5000, pitch: -90 },
0.6,
);
}
await page
.waitForFunction(
(prev) => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return (
!s.loading && s.lastUpdate > prev && s.coverage.kind === 'viewport'
);
},
{ timeout: 6000 },
previous,
)
.catch(() => result.failures.push(`${gesture}: no viewport reload`));
const row = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats(),
);
if (gesture.startsWith('contacts') && row.count !== 1)
result.failures.push(
`${gesture}: destination count ${row.count}, expected 1`,
);
result.views.push({ name: gesture, ...row });
console.log(JSON.stringify({ name: gesture, ...row }));
}
await page.screenshot({ path: 'qa-shots/installations-navigation.png' });
} finally {
fs.mkdirSync('qa-shots', { recursive: true });
fs.writeFileSync(
`qa-shots/installations-navigation-${baseline ? 'before' : 'after'}.json`,
JSON.stringify(result, null, 2),
);
await browser.close();
}
if (
!baseline &&
(result.failures.length || result.errors.length || result.forbidden.length)
)
process.exitCode = 1;
scripts/qa-installation-polish.mjs
+404
View File
@@ -0,0 +1,404 @@
/** Shared browser assertions for installation label persistence and selection. */
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import { moveCamera, createScreencast } from './qa-journey-recorder.mjs';
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
export async function checkInstallationPan(page, dir) {
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('local-datacenters', true),
);
await sleep(3000);
await page.evaluate(async () => {
const { getOverlayPaintRect } =
await import('/src/overlays/worldOverlay.js');
const g = window.__godsEyeView;
const installations = g.dataManager.layers.get(
'military-installations',
).module;
const centerIds = [];
for (let i = 0; i < g.viewer.dataSources.length; i++)
for (const e of g.viewer.dataSources.get(i).entities.values)
if (e.__localLayerId === 'local-datacenters') centerIds.push(e.id);
window.__polishPanFrames = [];
window.__stopPolishPan = g.viewer.scene.postRender.addEventListener(() => {
const sites = [],
centers = [];
installations.visitNamedMarkers((id) => {
if (getOverlayPaintRect('military-installations', id)) sites.push(id);
});
for (const id of centerIds)
if (getOverlayPaintRect('local-datacenters', id)) centers.push(id);
window.__polishPanFrames.push({ t: performance.now(), sites, centers });
});
});
await moveCamera(page, { lon: -99, lat: 39, height: 6500000, pitch: -90 }, 8);
const frames = await page.evaluate(() => {
window.__stopPolishPan();
return window.__polishPanFrames;
});
const count = (key) => {
let appeared = 0,
disappeared = 0,
reappeared = 0;
const seen = new Set(frames[0][key]);
for (let i = 1; i < frames.length; i++) {
const previous = new Set(frames[i - 1][key]),
next = new Set(frames[i][key]);
frames[i][`${key}Appeared`] = [...next].filter((id) => !previous.has(id));
frames[i][`${key}Disappeared`] = [...previous].filter(
(id) => !next.has(id),
);
for (const id of frames[i][`${key}Appeared`]) {
appeared++;
if (seen.has(id)) reappeared++;
seen.add(id);
}
disappeared += frames[i][`${key}Disappeared`].length;
}
const mean =
frames.reduce((sum, f) => sum + f[key].length, 0) / frames.length;
return {
appeared,
disappeared,
reappeared,
mean,
eventsPerLabel: (appeared + disappeared) / mean,
};
};
const result = {
frames: frames.length,
installations: count('sites'),
datacenters: count('centers'),
};
await fs.writeFile(
`${dir}/slow-pan-frames.json`,
JSON.stringify(frames, null, 2),
);
await fs.writeFile(`${dir}/slow-pan.json`, JSON.stringify(result, null, 2));
assert.ok(frames.length > 100);
assert.ok(
result.installations.mean >= 5 && result.datacenters.mean >= 5,
'both real layers contribute visible labels on the same pan',
);
assert.ok(
frames.every((f) => f.sites.length <= 24),
'shared-domain budget cannot exceed the installation title cap',
);
assert.ok(
result.installations.eventsPerLabel <=
result.datacenters.eventsPerLabel + 0.1,
'installation label transitions per label match Data Centers on the same slow pan',
);
assert.ok(
result.installations.reappeared <= result.datacenters.reappeared,
'installation labels do not flicker off and back on more than Data Centers',
);
await page.screenshot({ path: `${dir}/slow-pan-side-by-side.png` });
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('local-datacenters', false),
);
return result;
}
export async function checkInstallationSelection(
page,
dir,
{ video = true } = {},
) {
const result = [];
const recorder = video
? await createScreencast(page, `${dir}/selection-orbit`, { fps: 24 })
: null;
await recorder?.start();
try {
for (const site of [
{
key: 'mabry',
name: 'Camp Mabry',
lon: -97.765,
lat: 30.3125,
height: 3500,
},
{
key: 'naval',
name: 'Naval Base San Diego',
lon: -117.119,
lat: 32.684,
height: 6500,
},
]) {
await moveCamera(page, { ...site, pitch: -80 }, 1.5);
await page.waitForFunction(
() => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return !s.loading && s.count > 0;
},
{ timeout: 60000 },
);
await sleep(1500);
const picked = await page.evaluate(async (site) => {
const C = await import('/node_modules/cesium/Build/Cesium/index.js');
const g = window.__godsEyeView,
layer = g.dataManager.layers.get('military-installations').module;
const record = layer
.getNearby(C.Cartesian3.fromDegrees(site.lon, site.lat), 100000, 1000)
.find((r) => r.name === site.name && r.footprints?.length);
if (!record) throw new Error(`No polygon for ${site.name}`);
let point;
layer.visitNamedMarkers((id, p) => {
if (id === record.id) point = p.position;
});
const screen = C.SceneTransforms.worldToWindowCoordinates(
g.viewer.scene,
point,
);
window.__polishSelectedId = record.id;
window.__polishTitlePaints = [];
const canvas = document.getElementById('world-overlay-canvas'),
ctx = canvas.getContext('2d');
const original = ctx.fillText;
const matches = (text) =>
String(text).toLowerCase() === site.name.toLowerCase();
ctx.fillText = function (text, ...args) {
if (matches(text))
window.__polishTitlePaints.push({ t: performance.now(), text });
return original.call(this, text, ...args);
};
window.__restorePolishPaint = () => {
ctx.fillText = original;
};
window.__polishSelectionFrames = [];
const { getSelectedEntityContext } =
await import('/src/data/contextStore.js');
const { getOverlayPaintRect, getWorldOverlayDiagnostics } =
await import('/src/overlays/worldOverlay.js');
const knownFills = new Set();
const fillEntities = () => {
const list = [];
for (let i = 0; i < g.viewer.dataSources.length; i++)
for (const e of g.viewer.dataSources.get(i).entities.values)
if (e.id.startsWith(`${record.id}:selected-fill:`)) list.push(e);
for (const e of list) knownFills.add(e);
return list;
};
// Geometry-instance ids prove the ground primitive actually built, not just an entity declaration.
const primitives = () => {
let count = 0;
const walk = (collection) => {
if (collection?._collectionsArray) {
for (const child of collection._collectionsArray) walk(child);
return;
}
for (let i = 0; i < (collection?.length || 0); i++) {
const p = collection.get(i);
if (
p.ready &&
typeof p.getGeometryInstanceAttributes === 'function'
) {
for (const e of knownFills) {
try {
const a = p.getGeometryInstanceAttributes(e);
if (a && (!a.show || a.show[0])) count++;
} catch {}
}
}
if (typeof p.get === 'function' || p._collectionsArray) walk(p);
}
};
walk(g.viewer.scene.groundPrimitives);
return count;
};
window.__polishFillProbe = () => ({
entities: fillEntities().length,
primitives: primitives(),
classification: fillEntities().map((e) =>
e.polygon.classificationType.getValue(),
),
selected: getSelectedEntityContext()?.id || null,
});
window.__stopPolishSelection =
g.viewer.scene.postRender.addEventListener(() => {
window.__polishSelectionFrames.push({
paints: window.__polishTitlePaints.splice(0).length,
own: Boolean(
getOverlayPaintRect('military-installations', record.id),
),
tracked: Boolean(
getOverlayPaintRect('tracked', `installations:${record.id}`),
),
selected: getSelectedEntityContext()?.id === record.id,
total:
getWorldOverlayDiagnostics().paintedBySource[
'military-installations'
] || 0,
});
});
return {
id: record.id,
x: screen.x,
y: screen.y,
parts: record.footprints.length,
};
}, site);
await page.screenshot({
path: `${dir}/after-${site.key}-unselected.png`,
});
await page.mouse.click(picked.x, picked.y);
await page
.waitForFunction(
() => {
const p = window.__polishFillProbe();
return p.selected === window.__polishSelectedId && p.primitives > 0;
},
{ timeout: 20000 },
)
.catch(async (error) => {
await page.screenshot({ path: `${dir}/selection-failure.png` });
const diagnostic = await page.evaluate(() => {
const g = window.__godsEyeView;
const describe = (p) => ({
type: p.constructor.name,
ready: p.ready,
length: p.length,
keys: Object.keys(p).filter((k) =>
/primitive|instance|collection/i.test(k),
),
ids: p._instanceIds?.map((id) => id.id || String(id)),
children:
typeof p.get === 'function'
? Array.from({ length: p.length }, (_, i) =>
describe(p.get(i)),
)
: p._collectionsArray?.map(describe),
});
return {
probe: window.__polishFillProbe(),
stats: g.dataManager.layers
.get('military-installations')
.module.getStats(),
primitives: describe(g.viewer.scene.groundPrimitives),
frames: window.__polishSelectionFrames.slice(-5),
};
});
await fs.writeFile(
`${dir}/selection-failure.json`,
JSON.stringify({ picked, ...diagnostic }, null, 2),
);
throw error;
});
await sleep(500);
const selected = await page.evaluate(() => window.__polishFillProbe());
await page.screenshot({ path: `${dir}/after-selection-${site.key}.png` });
// Close, oblique angles expose surface clipping and floating fills.
for (const heading of [30, 120]) {
await page.evaluate(
async (site, heading) => {
const C =
await import('/node_modules/cesium/Build/Cesium/index.js');
const g = window.__godsEyeView;
const target = C.Cartesian3.fromDegrees(
site.lon,
site.lat,
site.key === 'mabry' ? 180 : 10,
);
g.viewer.camera.lookAt(
target,
new C.HeadingPitchRange(
C.Math.toRadians(heading),
C.Math.toRadians(-30),
800,
),
);
g.viewer.camera.lookAtTransform(C.Matrix4.IDENTITY);
},
site,
heading,
);
await sleep(1700);
assert.ok(
(await page.evaluate(() => window.__polishFillProbe())).selected,
'orbit must preserve installation selection',
);
await page.screenshot({
path: `${dir}/after-${site.key}-orbit-${heading}.png`,
});
}
await moveCamera(page, { ...site, pitch: -80 }, 1);
await sleep(900);
const empty = await page.evaluate(() => {
const g = window.__godsEyeView;
for (let y = 250; y < 650; y += 75)
for (let x = 400; x < 850; x += 75) {
const pick = g.viewer.scene.pick({ x, y });
if (!pick?.id?.installationId && !pick?.id?.gevTrackedId)
return { x, y };
}
throw new Error('No empty map point');
});
await page.mouse.click(empty.x, empty.y);
await page.waitForFunction(
() =>
window.__polishFillProbe().selected === null &&
window.__polishFillProbe().entities === 0,
);
await sleep(600);
const deselected = await page.evaluate(() => {
window.__stopPolishSelection();
window.__restorePolishPaint();
return {
...window.__polishFillProbe(),
frames: window.__polishSelectionFrames,
};
});
assert.equal(selected.entities, picked.parts);
assert.equal(selected.primitives, picked.parts);
assert.ok(
selected.classification.every((c) => c === 1),
'Google 3D tile classification',
);
assert.equal(deselected.entities, 0);
assert.equal(deselected.primitives, 0);
const selectedFrames = deselected.frames.filter(
(f) => f.selected && f.own,
);
assert.ok(selectedFrames.length > 10);
assert.ok(
selectedFrames.every((f) => !f.tracked && f.paints === 1),
'selected record has exactly one painted overlay element',
);
assert.ok(
deselected.frames
.slice(-5)
.every((f) => !f.selected && f.own && !f.tracked && f.paints === 1),
'deselect restores the one ambient entry',
);
await page.screenshot({
path: `${dir}/after-${site.key}-deselected.png`,
});
result.push({ site, picked, selected, deselected });
}
} finally {
await fs.writeFile(
`${dir}/selection-checks.json`,
JSON.stringify(result, null, 2),
);
if (recorder)
await fs.writeFile(
`${dir}/selection-video.json`,
JSON.stringify(await recorder.stop(), null, 2),
);
}
return result.map(({ site, picked, selected, deselected }) => ({
site: site.name,
id: picked.id,
selected,
deselected: {
entities: deselected.entities,
primitives: deselected.primitives,
},
frames: deselected.frames.length,
}));
}
scripts/qa-journey-recorder.mjs
+223
View File
@@ -0,0 +1,223 @@
/**
* qa-journey-recorder.mjs — shared support for journey QA gates (not a gate).
*
* - `createScreencast(page, dir)` records CDP `Page.startScreencast` frames
* (every compositor frame) with their real timestamps and builds
* `<dir>/clip.mp4` through ffmpeg's concat demuxer with per-frame
* durations, so held frames last as long as they did on screen.
* - `installFrameProbe(page)` measures every animation frame in the page:
* the requestAnimationFrame interval and, for frames Cesium rendered, the
* CPU time between `scene.preUpdate` and `scene.postRender`.
* - `frameStats(frames)` reduces a probe window to count / mean / p50 / p95.
* - `moveCamera(page, view, seconds)` eases the camera like a drag or wheel
* zoom rather than teleporting it.
*/
import fs from 'node:fs';
import path from 'node:path';
import { execFileSync } from 'node:child_process';
/** Percentile of a numeric list (nearest rank); NaN for an empty list. */
export function percentile(values, p) {
if (!values.length) return Number.NaN;
const sorted = [...values].sort((a, b) => a - b);
return sorted[
Math.min(sorted.length - 1, Math.ceil((p / 100) * sorted.length) - 1)
];
}
/** Summarize probe frames: rAF interval and Cesium render CPU time (ms). */
export function frameStats(frames) {
const intervals = frames.map((f) => f.dt).filter(Number.isFinite);
const renders = frames.map((f) => f.renderMs).filter(Number.isFinite);
const round = (v) => (Number.isFinite(v) ? Math.round(v * 10) / 10 : null);
const mean = (list) =>
list.length ? list.reduce((a, b) => a + b, 0) / list.length : Number.NaN;
return {
frames: intervals.length,
frameMean: round(mean(intervals)),
frameP50: round(percentile(intervals, 50)),
frameP95: round(percentile(intervals, 95)),
longFrames: intervals.filter((v) => v > 50).length,
renderedFrames: renders.length,
renderMean: round(mean(renders)),
renderP50: round(percentile(renders, 50)),
renderP95: round(percentile(renders, 95)),
};
}
/** Install the in-page frame probe once per document; returns nothing. */
export async function installFrameProbe(page) {
await page.evaluate(() => {
if (window.__gevFrameProbe) return;
const { viewer } = window.__godsEyeView;
const probe = { frames: [], started: 0 };
window.__gevFrameProbe = probe;
let last = performance.now();
let renderStart = null;
let pendingRender = null;
viewer.scene.preUpdate.addEventListener(() => {
renderStart = performance.now();
});
viewer.scene.postRender.addEventListener(() => {
if (renderStart !== null) pendingRender = performance.now() - renderStart;
renderStart = null;
});
const tick = (now) => {
probe.frames.push({ t: now, dt: now - last, renderMs: pendingRender });
if (probe.frames.length > 40_000) probe.frames.splice(0, 10_000);
pendingRender = null;
last = now;
requestAnimationFrame(tick);
};
requestAnimationFrame(tick);
});
}
/** Read probe frames with t in [t0, t1) (page performance.now() times). */
export async function readFrames(page, t0, t1 = Infinity) {
return page.evaluate(
(a, b) =>
window.__gevFrameProbe.frames.filter(
(f) => f.t >= a && f.t < (b ?? Infinity),
),
t0,
Number.isFinite(t1) ? t1 : null,
);
}
/** Page clock in ms (performance.now()). */
export function pageNow(page) {
return page.evaluate(() => performance.now());
}
/**
* Ease the camera to a view over `seconds`, resolving when it lands.
* @param {import('puppeteer').Page} page Page.
* @param {{lat:number, lon:number, height:number, heading?:number, pitch?:number}} view
* Destination (degrees, metres).
* @param {number} seconds Flight duration; 0 jumps.
*/
export async function moveCamera(page, view, seconds) {
await page.evaluate(
(v, duration) =>
new Promise((resolve) => {
const { viewer } = window.__godsEyeView;
const C = viewer.camera;
C.cancelFlight();
C.flyTo({
destination: viewer.scene.globe.ellipsoid.cartographicToCartesian({
latitude: (v.lat * Math.PI) / 180,
longitude: (v.lon * Math.PI) / 180,
height: v.height,
}),
orientation: {
heading: ((v.heading ?? 0) * Math.PI) / 180,
pitch: ((v.pitch ?? -45) * Math.PI) / 180,
roll: 0,
},
duration,
// Straight, eased motion like a drag or wheel zoom, not an arc.
maximumHeight: Math.max(v.height, C.positionCartographic.height),
complete: resolve,
cancel: resolve,
});
}),
view,
seconds,
);
}
/**
* Record the page with CDP screencast frames and real per-frame timing.
* @param {import('puppeteer').Page} page Page to record.
* @param {string} dir Output directory (frames + clip.mp4).
* @param {{maxWidth?: number, maxHeight?: number, quality?: number, fps?: number|null}} [options]
*/
export async function createScreencast(page, dir, options = {}) {
const {
maxWidth = 1280,
maxHeight = 800,
quality = 70,
fps = null,
} = options;
const framesDir = path.join(dir, 'frames');
fs.rmSync(framesDir, { recursive: true, force: true });
fs.mkdirSync(framesDir, { recursive: true });
const cdp = await page.createCDPSession();
const frames = [];
let recording = false;
cdp.on('Page.screencastFrame', async (frame) => {
const file = path.join(
framesDir,
`f${String(frames.length).padStart(5, '0')}.jpg`,
);
if (recording) {
fs.writeFileSync(file, Buffer.from(frame.data, 'base64'));
frames.push({ file, timestamp: frame.metadata.timestamp });
}
await cdp
.send('Page.screencastFrameAck', { sessionId: frame.sessionId })
.catch(() => {});
});
return {
frames,
async start() {
recording = true;
await cdp.send('Page.startScreencast', {
format: 'jpeg',
quality,
maxWidth,
maxHeight,
everyNthFrame: 1,
});
},
/** Stop, encode with real timing, and return clip facts. */
async stop() {
recording = false;
await cdp.send('Page.stopScreencast').catch(() => {});
await cdp.detach().catch(() => {});
if (frames.length < 2) return { clip: null, frames: frames.length };
const lines = [];
for (let i = 0; i < frames.length; i++) {
const next = frames[i + 1]?.timestamp;
const duration = next
? Math.min(2, Math.max(0.001, next - frames[i].timestamp))
: 0.05;
lines.push(`file '${path.resolve(frames[i].file)}'`);
lines.push(`duration ${duration.toFixed(4)}`);
}
// The concat demuxer ignores the last duration unless the file repeats.
lines.push(`file '${path.resolve(frames.at(-1).file)}'`);
const list = path.join(dir, 'frames.txt');
fs.writeFileSync(list, `${lines.join('\n')}\n`);
const clip = path.join(dir, 'clip.mp4');
execFileSync('ffmpeg', [
'-y',
'-loglevel',
'error',
'-f',
'concat',
'-safe',
'0',
'-i',
list,
'-fps_mode',
fps ? 'cfr' : 'vfr',
'-c:v',
'libx264',
'-pix_fmt',
'yuv420p',
'-vf',
`scale=trunc(iw/2)*2:trunc(ih/2)*2${fps ? `,fps=${fps}` : ''}`,
clip,
]);
const seconds = frames.at(-1).timestamp - frames[0].timestamp;
return {
clip,
frames: frames.length,
seconds: Math.round(seconds * 10) / 10,
fps: Math.round((frames.length / Math.max(seconds, 0.001)) * 10) / 10,
};
},
};
}
scripts/qa-layer-panel.mjs
+15 -11
View File
@@ -23,13 +23,17 @@ try {
`${process.env.QA_BASE_URL || 'http://localhost:4173'}/?welcome=0`,
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() => window.__godsEyeView?.dataManager?._layerPanel,
{ timeout: 60000 },
);
await page.waitForFunction(() => window.__godsEyeView?.dataManager, {
timeout: 60000,
});
const results = await page.evaluate(async () => {
const manager = window.__godsEyeView.dataManager;
const container = manager._toggleContainer;
const entry = document.querySelector(
'script[type="module"][src*="/src/main.js"]',
);
const { application } = await import(entry.src);
const { presentation } = application.getComponents().data;
const container = presentation.panel._toggleContainer;
const id = 'qa-panel-lifecycle';
let listener = null;
let enabled = 0;
@@ -54,7 +58,7 @@ try {
});
const result = [];
try {
manager.buildTogglePanel(container);
presentation.mount(container);
const row = () => container.querySelector(`[data-layer-id="${id}"]`);
const first = row().querySelector('.data-toggle-btn');
result.push([
@@ -66,7 +70,7 @@ try {
'row subscription is installed',
typeof listener === 'function',
]);
manager.buildTogglePanel(container);
presentation.mount(container);
first.click();
await Promise.resolve();
result.push([
@@ -97,7 +101,7 @@ try {
]);
await manager.setEnabled(id, true);
const final = row().querySelector('.data-toggle-btn');
manager._layerPanel.destroy();
presentation._panel.destroy();
final.click();
await Promise.resolve();
result.push([
@@ -109,10 +113,10 @@ try {
manager.isEnabled(id),
]);
} finally {
manager._layerPanel?.destroy();
manager._layerPanel = null;
presentation._panel?.destroy();
presentation._panel = null;
await window.__gevQaUnregisterLayer(manager, id);
manager.buildTogglePanel(container);
presentation.mount(container);
}
result.push([
'ordinary layer rows return after reassembly',
scripts/qa-layer-token-twochar.mjs
+337
View File
@@ -0,0 +1,337 @@
#!/usr/bin/env node
/**
* End-to-end share/reload proof for the first two-character token. The QA
* layer and an exhausted-digit reservation scenario are injected into
* dev-server responses in this isolated browser only. Neither belongs in a
* production bundle or ledger.
*
* Run against a local Vite server: node scripts/qa-layer-token-twochar.mjs
*/
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import puppeteer from 'puppeteer';
import {
LAYER_STATE_TOKEN_RESERVATIONS,
nextLayerStateToken,
parseLayerStateTokenReservations,
validateLayerStateAllocations,
} from '../src/data/layerState.js';
const base = new URL(process.env.QA_BASE_URL || 'http://127.0.0.1:4173');
if (!['127.0.0.1', 'localhost'].includes(base.hostname))
throw new Error('The QA fixture requires a loopback-only dev server');
const origin = base.origin;
const fixtureId = 'qa-twochar-fixture';
const injections = { codec: 0, catalog: 0 };
const errors = [];
const reservations = JSON.parse(
await readFile(
new URL('../src/data/layerStateTokenReservations.json', import.meta.url),
'utf8',
),
);
assert.equal(
reservations.some(([, token]) => token === '00'),
false,
'00 must remain unallocated in the production ledger for this QA fixture',
);
const productionTokens = new Set(reservations.map(([, token]) => token));
const priorDigitRows = [...'0123456789']
.filter((digit) => !productionTokens.has(digit))
.map((digit) => [`qa-prior-digit-${digit}`, digit]);
const priorState = parseLayerStateTokenReservations([
...reservations,
...priorDigitRows,
]);
assert.equal(nextLayerStateToken(priorState), '00');
assert.equal(
validateLayerStateAllocations(
LAYER_STATE_TOKEN_RESERVATIONS,
parseLayerStateTokenReservations([
...reservations,
...priorDigitRows,
[fixtureId, '00'],
]),
),
true,
);
console.log(
'PASS: isolated fixture exhausts prior digits before allocating 00',
);
for (const source of [
'../src/data/layerState.js',
'../src/app/constructCatalog.js',
]) {
assert.equal(
(await readFile(new URL(source, import.meta.url), 'utf8')).includes(
fixtureId,
),
false,
`${fixtureId} must not exist in production source`,
);
}
function replaceOnce(source, from, to, label) {
const first = source.indexOf(from);
if (first < 0 || source.indexOf(from, first + from.length) >= 0)
throw new Error(`QA fixture anchor changed: ${label}`);
return source.replace(from, to);
}
function injectCodec(source) {
const fixtureRows = JSON.stringify([...priorDigitRows, [fixtureId, '00']]);
let result = replaceOnce(
source,
'parseLayerStateTokenReservations(reservationRows)',
`parseLayerStateTokenReservations([...reservationRows, ...${fixtureRows}])`,
'reservation',
);
result = replaceOnce(
result,
']);\n\nexport const REGISTERED_LAYER_IDS',
` Object.freeze({ id: '${fixtureId}', token: '00', disposition: 'enabled-only' }),\n]);\n\nexport const REGISTERED_LAYER_IDS`,
'registry',
);
injections.codec += 1;
return result;
}
function injectCatalog(source) {
const result = replaceOnce(
source,
' createBhoteKoshiEventLayer(),',
` {
id: '${fixtureId}',
name: 'Synthetic two-character token',
icon: '◆',
source: 'QA-only fixture',
async init() { return true; },
async enable() { return true; },
async update() { return true; },
async disable() { return true; },
async destroy() { return true; },
getStats() { return { count: 0 }; },
},
createBhoteKoshiEventLayer(),`,
'catalog',
);
injections.catalog += 1;
return result;
}
const browser = await puppeteer.launch({
headless: true,
args: [
'--no-sandbox',
'--disable-dev-shm-usage',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
],
});
async function newPage() {
const context = await browser.createBrowserContext();
const page = await context.newPage();
page.on('pageerror', (error) => errors.push(error.message));
await page.setViewport({ width: 1280, height: 800 });
await page.setCacheEnabled(false);
await page.evaluateOnNewDocument(() => {
window.__qaRestoreEvents = [];
window.addEventListener('gev:initial-share-restore-settled', (event) => {
window.__qaRestoreEvents.push(event.detail);
});
});
await page.setRequestInterception(true);
page.on('request', async (request) => {
try {
const url = new URL(request.url());
if (url.origin !== origin) return await request.abort();
const transform =
url.pathname === '/src/data/layerState.js'
? injectCodec
: url.pathname === '/src/app/constructCatalog.js'
? injectCatalog
: null;
if (!transform) return await request.continue();
const response = await fetch(request.url());
assert.equal(response.status, 200, `dev module ${url.pathname}`);
await request.respond({
status: 200,
contentType: 'application/javascript',
body: transform(await response.text()),
});
} catch (error) {
errors.push(String(error?.stack || error));
await request.abort().catch(() => {});
}
});
return { context, page };
}
async function load(page, url) {
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60_000 });
await page.waitForFunction(
(id) => window.__godsEyeView?.dataManager?.layers?.has(id),
{ timeout: 60_000 },
fixtureId,
);
}
async function loadedState(page) {
return page.evaluate((id) => {
const manager = window.__godsEyeView.dataManager;
return {
enabled: manager.isEnabled(id),
flights: manager.isEnabled('flights'),
localStorage: localStorage.getItem('gev:layer-state:v2'),
restoreEvents: window.__qaRestoreEvents,
hash: location.hash,
};
}, fixtureId);
}
async function waitForRestore(page) {
await page.waitForFunction(() => window.__qaRestoreEvents.length > 0, {
timeout: 60_000,
});
return loadedState(page);
}
function withLayers(link, layers) {
const url = new URL(link);
const params = new URLSearchParams(url.hash.slice(1));
params.set('l', layers);
url.hash = params.toString();
return url.href;
}
const contexts = [];
try {
const sender = await newPage();
contexts.push(sender.context);
await load(sender.page, `${origin}/?welcome=0`);
const toggled = await sender.page.evaluate(
(id) =>
window.__godsEyeView.dataManager.setEnabled(id, true, { origin: 'user' }),
fixtureId,
);
assert.equal(toggled, true, 'synthetic layer enables through manager');
await sender.page.waitForFunction(
() => new URLSearchParams(location.hash.slice(1)).get('l') === '00',
{ timeout: 15_000 },
);
const sharedUrl = sender.page.url();
const authored = new URLSearchParams(new URL(sharedUrl).hash.slice(1));
assert.equal(authored.get('v'), '2');
assert.equal(authored.get('l'), '00');
assert.ok(authored.has('lat') && authored.has('lon'));
console.log('PASS: real user-origin enable encoded 00 in a full share URL');
const recipient = await newPage();
contexts.push(recipient.context);
await load(recipient.page, sharedUrl);
const restored = await waitForRestore(recipient.page);
assert.equal(restored.enabled, true);
assert.equal(restored.localStorage, null);
assert.equal(restored.restoreEvents.at(-1)?.status, 'settled');
assert.ok(
restored.restoreEvents
.at(-1)
?.layers?.some(
(layer) =>
layer.layerId === fixtureId &&
layer.targetEnabled === true &&
layer.succeeded === true,
),
'share coordinator restored the synthetic layer',
);
console.log(
'PASS: cold recipient reload restored the 00 layer without writing local preferences',
);
await recipient.page.reload({
waitUntil: 'domcontentloaded',
timeout: 60_000,
});
await recipient.page.waitForFunction(
(id) => window.__godsEyeView?.dataManager?.layers?.has(id),
{ timeout: 60_000 },
fixtureId,
);
const reloaded = await waitForRestore(recipient.page);
assert.equal(reloaded.enabled, true);
assert.equal(reloaded.localStorage, null);
assert.equal(new URLSearchParams(reloaded.hash.slice(1)).get('l'), '00');
console.log('PASS: a full recipient reload restores the same 00 layer state');
const disabled = await recipient.page.evaluate(
(id) =>
window.__godsEyeView.dataManager.setEnabled(id, false, {
origin: 'user',
}),
fixtureId,
);
assert.equal(disabled, true);
await recipient.page.waitForFunction(
() => new URLSearchParams(location.hash.slice(1)).get('l') === '',
{ timeout: 15_000 },
);
assert.equal((await loadedState(recipient.page)).enabled, false);
console.log('PASS: explicit OFF removes 00 from the authored share URL');
const legacy = await newPage();
contexts.push(legacy.context);
await load(legacy.page, withLayers(sharedUrl, 'f'));
const legacyState = await waitForRestore(legacy.page);
assert.equal(legacyState.flights, true);
assert.equal(legacyState.enabled, false);
assert.equal(legacyState.restoreEvents.at(-1)?.status, 'settled');
console.log('PASS: a legacy one-character token still restores');
const mixed = await newPage();
contexts.push(mixed.context);
await load(mixed.page, withLayers(sharedUrl, 'f.00'));
const mixedState = await waitForRestore(mixed.page);
assert.equal(mixedState.flights, true);
assert.equal(mixedState.enabled, true);
console.log('PASS: mixed legacy and two-character tokens restore together');
const malformed = await newPage();
contexts.push(malformed.context);
await load(malformed.page, withLayers(sharedUrl, 'f..00'));
const malformedState = await waitForRestore(malformed.page);
assert.equal(malformedState.flights, false);
assert.equal(malformedState.enabled, false);
assert.equal(malformedState.localStorage, null);
console.log(
'PASS: malformed layer list is rejected without partial restoration',
);
const repeated = await newPage();
contexts.push(repeated.context);
const repeatedUrl = new URL(withLayers(sharedUrl, 'f'));
const repeatedParams = new URLSearchParams(repeatedUrl.hash.slice(1));
repeatedParams.append('l', '00');
repeatedUrl.hash = repeatedParams.toString();
await load(repeated.page, repeatedUrl.href);
const repeatedState = await waitForRestore(repeated.page);
assert.equal(repeatedState.flights, false);
assert.equal(repeatedState.enabled, false);
assert.equal(repeatedState.localStorage, null);
console.log('PASS: repeated layer fields reject the entire share payload');
assert.ok(injections.codec >= contexts.length);
assert.ok(injections.catalog >= contexts.length);
assert.deepEqual(errors, []);
console.log(
'PASS: QA-only codec and catalog injections reached every isolated session',
);
console.log(
'Synthetic end-to-end evidence only; production 00 allocation remains pending.',
);
console.log('RESULT: 10 passed, 0 failed, 0 skipped');
} finally {
await Promise.all(contexts.map((context) => context.close().catch(() => {})));
await browser.close();
}
scripts/qa-military-names.mjs
+485
View File
@@ -0,0 +1,485 @@
#!/usr/bin/env node
/** Named military areas: continent performance, close identities and a recorded zoom. */
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import puppeteer from 'puppeteer';
import {
checkInstallationPan,
checkInstallationSelection,
} from './qa-installation-polish.mjs';
import {
createScreencast,
installFrameProbe,
readFrames,
frameStats,
pageNow,
moveCamera,
} from './qa-journey-recorder.mjs';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log('Usage: node scripts/qa-military-names.mjs <url> [--no-video]');
process.exit(0);
}
const url = args.find((a) => /^https?:/.test(a));
if (!url) throw new Error('Supply the running server URL');
const dir = process.env.QA_NAMES_DIR || 'qa-shots/military-names';
await fs.mkdir(dir, { recursive: true });
const result = {
views: [],
sites: [],
errors: [],
forbidden: [],
packs: [],
video: null,
};
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const browser = await puppeteer.launch({
headless: true,
protocolTimeout: 180000,
args: [
'--use-angle=metal',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
let recorder;
try {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
page.on('request', (request) => {
const u = new URL(request.url());
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
result.forbidden.push(u.origin);
if (u.pathname.endsWith('/osm_military_names/names.json'))
result.packs.push(u.pathname);
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
await page.goto(url + (url.includes('?') ? '&' : '?') + 'welcome=0', {
waitUntil: 'domcontentloaded',
});
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
result.page = await page.evaluate(() => ({
url: location.href,
title: document.title,
canvas: Boolean(document.querySelector('#cesiumContainer canvas')),
errorOverlay: Boolean(document.querySelector('vite-error-overlay')),
}));
assert.equal(new URL(result.page.url).origin, new URL(url).origin);
assert.ok(
result.page.title && result.page.canvas && !result.page.errorOverlay,
);
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.styleManager.setDetection({ enabled: false });
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
});
assert.equal(result.packs.length, 0, 'names pack stays unloaded at startup');
result.renderer = await page.evaluate(() => {
const gl = window.__godsEyeView.viewer.scene.context._gl;
const ext = gl.getExtension('WEBGL_debug_renderer_info');
return ext
? gl.getParameter(ext.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER);
});
assert.doesNotMatch(result.renderer, /swiftshader|software/i);
await installFrameProbe(page);
const stats = () =>
page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats(),
);
const settle = async () => {
await sleep(500);
await page.waitForFunction(
() => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return !s.loading && s.count > 0 && s.status === 'ready';
},
{ timeout: 60000 },
);
await sleep(1200);
};
const us = { lon: -100, lat: 39, height: 6500000, pitch: -90 };
await moveCamera(page, us, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('military-installations', true),
);
await settle();
result.credit = await page.evaluate(() => {
const el = document.querySelector('#cesium-credits');
return {
text: el?.innerText || '',
visible: Boolean(el && getComputedStyle(el).display !== 'none'),
};
});
assert.match(result.credit.text, /© OpenStreetMap/);
assert.doesNotMatch(
result.credit.text,
/© OpenMapTiles/,
'wide bundled names use no vector tiles',
);
assert.ok(result.credit.visible);
await page.screenshot({ path: `${dir}/credit.png` });
for (const [name, view] of [
['us', us],
['europe', { lon: 12, lat: 50, height: 4500000, pitch: -90 }],
]) {
await moveCamera(page, view, 2);
await settle();
const before = await stats();
const overlaps = await page.evaluate(async () => {
const { getOverlayPaintRect } =
await import('/src/overlays/worldOverlay.js');
const layer = window.__godsEyeView.dataManager.layers.get(
'military-installations',
).module;
const rects = [];
layer.visitNamedMarkers((id) => {
const rect = getOverlayPaintRect('military-installations', id);
if (rect) rects.push(rect);
});
let overlaps = 0;
for (let a = 0; a < rects.length; a++)
for (let b = a + 1; b < rects.length; b++) {
const x = rects[a],
y = rects[b];
if (
x.x < y.x + y.w &&
y.x < x.x + x.w &&
x.y < y.y + y.h &&
y.y < x.y + x.h
)
overlaps++;
}
return overlaps;
});
// Incumbents retain their placements during camera motion, as for Data Centers.
assert.ok(
before.namedInView > 0 &&
before.namedMarkers > 0 &&
before.pointsOnScreen > 0,
);
assert.ok(
before.labelsOnScreen > 0 && before.labelsOnScreen <= before.labelCap,
);
await page.evaluate(async () => {
// Use the governor's hold so an asynchronous idle transition cannot
// overwrite a raw scene flag during the sustained-render measurement.
const { holdContinuousRender } = await import('/src/renderGovernor.js');
holdContinuousRender('qa-military-names-fps');
});
const t0 = await pageNow(page);
await sleep(3500);
const frames = await readFrames(page, t0);
await page.evaluate(async () => {
const { releaseContinuousRender } =
await import('/src/renderGovernor.js');
releaseContinuousRender('qa-military-names-fps');
});
const metrics = frameStats(frames);
const seconds = frames.reduce((sum, frame) => sum + frame.dt, 0) / 1000;
const fps = metrics.renderedFrames / seconds;
const rafFps = metrics.frames / seconds;
result.views.push({ name, ...before, overlaps, fps, rafFps, ...metrics });
console.log(name, JSON.stringify(result.views.at(-1)));
await page.screenshot({ path: `${dir}/${name}.png` });
assert.ok(fps >= 55, `${name} fps ${fps.toFixed(1)} < 55`);
}
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(true),
);
await sleep(300);
result.cleanUi = await page.evaluate(async () => {
const host = await import('/src/overlays/worldOverlay.js');
const root = document.getElementById('world-overlay-root');
return {
active: document.body.classList.contains('ui-clean-view'),
visible: root.checkVisibility({
visibilityProperty: true,
opacityProperty: true,
}),
labels:
host.getWorldOverlayDiagnostics().paintedBySource[
'military-installations'
] || 0,
};
});
assert.ok(
result.cleanUi.active &&
result.cleanUi.visible &&
result.cleanUi.labels > 0,
);
await page.screenshot({ path: `${dir}/clean-ui.png` });
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(false),
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'military-installations',
false,
),
);
result.disabled = await page.evaluate(async () => {
const host = await import('/src/overlays/worldOverlay.js');
return host.getWorldOverlayDiagnostics().entriesBySource[
'military-installations'
];
});
assert.equal(result.disabled, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('military-installations', true),
);
await moveCamera(page, us, 1);
await settle();
result.slowPan = await checkInstallationPan(page, dir);
await moveCamera(page, us, 1);
await settle();
if (!args.includes('--no-video')) {
recorder = await createScreencast(page, `${dir}/us-to-san-diego`, {
fps: 24,
});
await recorder.start();
}
await page.evaluate(async () => {
const C = await import('/node_modules/cesium/Build/Cesium/index.js');
const { getOverlayPaintRect } =
await import('/src/overlays/worldOverlay.js');
const g = window.__godsEyeView;
const layer = g.dataManager.layers.get('military-installations').module;
const overlay = document.createElement('div');
overlay.id = 'installation-label-probe';
overlay.style.cssText =
'position:fixed;top:60px;left:360px;z-index:99999;background:#07121cdd;color:#fff;padding:8px 12px;font:13px monospace;pointer-events:none';
document.body.append(overlay);
const screen = new C.Cartesian2();
window.__installationFrames = [];
window.__stopInstallationFrames =
g.viewer.scene.postRender.addEventListener(() => {
const stats = layer.getStats();
const markers = [],
labels = [],
rects = [];
layer.visitNamedMarkers((id, point) => {
const rect = getOverlayPaintRect('military-installations', id);
if (rect) rects.push(rect);
const p = C.SceneTransforms.worldToWindowCoordinates(
g.viewer.scene,
point.position,
screen,
);
if (
point.show &&
p &&
p.x >= 0 &&
p.y >= 0 &&
p.x < 1280 &&
p.y < 800
) {
markers.push(id);
if (rect) labels.push(id);
}
});
let overlaps = 0;
for (let a = 0; a < rects.length; a++)
for (let b = a + 1; b < rects.length; b++) {
const x = rects[a],
y = rects[b];
if (
x.x < y.x + y.w &&
y.x < x.x + x.w &&
x.y < y.y + y.h &&
y.y < x.y + x.h
)
overlaps++;
}
const altitude = g.viewer.camera.positionCartographic.height;
window.__installationFrames.push({
t: performance.now(),
altitude,
markers,
labels,
paintedLabels: stats.labelsOnScreen,
overlaps,
loading: stats.loading,
wide: stats.wide,
});
overlay.textContent = `${(altitude / 1000).toFixed(1)} km | markers ${markers.length} | labels ${labels.length} / 24`;
});
});
const clipStart = await pageNow(page);
await sleep(1200);
for (const view of [
{ lon: -116, lat: 35, height: 1500000, pitch: -85 },
{ lon: -117.2, lat: 33, height: 200000, pitch: -80 },
{ lon: -117.1, lat: 32.86, height: 18000, pitch: -80 },
]) {
await moveCamera(page, view, 3);
await sleep(800);
}
await settle();
await sleep(1500);
result.clipFrames = await readFrames(page, clipStart);
result.labelFrames = await page.evaluate(() => {
window.__stopInstallationFrames();
document.getElementById('installation-label-probe').remove();
return window.__installationFrames;
});
await fs.writeFile(
`${dir}/label-frames.json`,
JSON.stringify(result.labelFrames, null, 2),
);
assert.ok(result.labelFrames.length > 100);
assert.ok(
result.labelFrames.every(
(frame) =>
frame.markers.length > 0 &&
frame.labels.length > 0 &&
frame.paintedLabels > 0 &&
frame.paintedLabels <= 24,
),
'zoom never loses all visible named markers or labels',
);
if (recorder) {
result.video = await recorder.stop();
recorder = null;
}
result.selection = await checkInstallationSelection(page, dir, {
video: !args.includes('--no-video'),
});
const sites = [
['mabry', 'Camp Mabry', -97.765, 30.3125, 3500],
['pendleton', 'Marine Corps Base Camp Pendleton', -117.4, 33.33, 25000],
['miramar', 'Marine Corps Air Station Miramar', -117.14, 32.87, 18000],
['ramstein', 'Ramstein Air Base', 7.6, 49.44, 7000],
['lakenheath', 'RAF Lakenheath', 0.56, 52.41, 7000],
['hood', 'Fort Hood', -97.78, 31.135, 30000],
];
for (const [key, expected, lon, lat, height] of sites) {
await moveCamera(page, { lon, lat, height, pitch: -80 }, 1.5);
await settle();
const snapshot = await page.evaluate(
(lat, lon) => {
const g = window.__godsEyeView;
const layer = g.dataManager.layers.get('military-installations').module;
const position = g.viewer.scene.globe.ellipsoid.cartographicToCartesian(
{
longitude: (lon * Math.PI) / 180,
latitude: (lat * Math.PI) / 180,
height: 0,
},
);
return {
stats: layer.getStats(),
records: layer.getNearby(position, 100000, 1000).map((r) => ({
id: r.id,
name: r.name,
memberNames: r.memberNames || [],
hasPolygon: Boolean(r.footprints?.length),
})),
};
},
lat,
lon,
);
result.sites.push({ key, expected, ...snapshot });
assert.equal(
new Set(snapshot.records.map((r) => r.id)).size,
snapshot.records.length,
);
assert.ok(
snapshot.stats.labelsOnScreen > 0 &&
snapshot.stats.labelsOnScreen <= snapshot.stats.labelCap,
);
assert.ok(
snapshot.records.some((r) => r.name === expected && r.hasPolygon),
`${key}: missing named polygon ${expected}; got ${snapshot.records.map((r) => r.name).join(', ')}`,
);
await page.screenshot({ path: `${dir}/${key}.png` });
console.log(key, expected);
}
await page.evaluate(() => {
if (
document
.getElementById('global-context-panel')
.classList.contains('collapsed')
)
document
.querySelector('[data-collapse-target="global-context-panel"]')
.click();
});
await page.waitForSelector('#global-context-flights-btn', { visible: true });
await page.evaluate(() => {
const button = document.getElementById('global-context-flights-btn');
if (button.getAttribute('aria-selected') !== 'true') button.click();
});
await page.waitForFunction(
() => {
const state = window.__godsEyeView.styleManager.getContextModeState();
return state.mode === 'flights' && !state.changing;
},
{ timeout: 60000 },
);
result.card = await page.evaluate(() => {
const g = window.__godsEyeView;
return g.dataManager.layers
.get('military-installations')
.module.focusById('osm:military:r13529728');
});
assert.equal(result.card, true);
await page.waitForFunction(
() =>
document
.getElementById('military-awareness-panel')
?.textContent.includes('Named areas') &&
document
.getElementById('military-awareness-panel')
.checkVisibility({ visibilityProperty: true, opacityProperty: true }),
{ timeout: 15000 },
);
result.cardText = await page.$eval(
'#military-awareness-panel',
(el) => el.textContent,
);
assert.match(result.cardText, /Fort Hood/);
assert.match(result.cardText, /TA-25/);
await sleep(1800);
await page.$eval(
'#military-awareness-panel .military-awareness-names',
(el) => {
el.open = true;
},
);
await page.screenshot({ path: `${dir}/member-names-card.png` });
assert.equal(result.packs.length, 1, 'single lazy asset fetch');
assert.deepEqual(result.forbidden, []);
assert.deepEqual(result.errors, []);
} finally {
if (recorder)
result.video = await recorder.stop().catch((e) => ({ error: e.message }));
await fs.writeFile(`${dir}/result.json`, JSON.stringify(result, null, 2));
await browser.close();
}
scripts/qa-overpass-offload.mjs
+1502
View File
File diff suppressed because it is too large. Load diff
scripts/qa-radio.mjs
+9 -3
View File
@@ -374,7 +374,7 @@ async function main() {
enabled: true,
uncertain: true,
});
gev.dataManager._refreshTogglePanel();
gev.dataManager.refreshLayerStats();
const staticResult = module.setTuningStatic(true);
const filterBefore = module.getUIState().filter;
const filterControl = document.getElementById('radio-filter');
@@ -447,7 +447,7 @@ async function main() {
enabled: true,
uncertain: false,
});
gev.dataManager._refreshTogglePanel();
gev.dataManager.refreshLayerStats();
module.cancelTuning();
camera.flyTo = originalFlyTo;
return result;
@@ -4048,7 +4048,13 @@ async function main() {
const { getOverlayPaintRect, getWorldOverlayDiagnostics } = await import('/src/overlays/worldOverlay.js');
const expectedEntryId = `selected:${selected.id}`;
let painted = null;
for (let attempt = 0; attempt < 20 && !painted?.rect; attempt += 1) {
// Radio's horizon pass reveals the selected entity on its own timer
// after the camera jump, so a stationary view waits for both to settle.
for (
let attempt = 0;
attempt < 20 && !(painted?.rect && selectedEntity?.show === true);
attempt += 1
) {
painted = await new Promise((resolve) => {
let removePostRender = null;
const timeout = setTimeout(() => {
scripts/qa-recent-imagery.mjs
+1480
View File
File diff suppressed because it is too large. Load diff
scripts/qa-terrain-429.mjs
+362
View File
@@ -0,0 +1,362 @@
#!/usr/bin/env node
/**
* Keyless terrain throttling probe.
*
* On the keyless map source the browser fetches Re:Earth quantized-mesh tiles
* (`terrain.reearth.land/cesium-mesh/ellipsoid/{z}/{x}/{y}.terrain`) directly.
* A zoom-out burst asks for a few dozen shallow tiles at once and the upstream
* answers HTTP 429 for most of them. This harness drives that burst, counts the
* throttled responses, keeps the first 429's headers as evidence, and then
* checks whether every throttled tile eventually loaded — the contract the
* client-side retry in `src/maps/terrain.js` provides.
*
* Run against a keyless dev server (no Google or ion credentials):
* env -u GOOGLE_MAPS_API_KEY -u CESIUM_ION_TOKEN npm run dev -- --port 4180
* QA_BASE_URL=http://localhost:4180 node scripts/qa-terrain-429.mjs
*
* Flags: `--mode zoomOut|wheel` (default zoomOut), `--notches N` (default 12),
* `--settle MS` (default 25000), `--rounds N` (default 1), `--inject N`
* (answer the first N tile requests of each burst with a synthetic 429),
* `--headful`.
* Exits 1 when a throttled tile never recovered or a terrain tile is left in
* Cesium's FAILED state.
*/
import fs from 'node:fs';
import puppeteer from 'puppeteer';
const appUrl = process.env.QA_BASE_URL || 'http://localhost:4173';
const argv = process.argv.slice(2);
const flag = (name, fallback) => {
const index = argv.indexOf(`--${name}`);
return index >= 0 && argv[index + 1] !== undefined
? argv[index + 1]
: fallback;
};
const mode = flag('mode', 'zoomOut');
const notches = Number(flag('notches', 12));
const settleMs = Number(flag('settle', 25000));
const rounds = Number(flag('rounds', 1));
// The upstream throttle is load-dependent and does not fire on demand. With
// `--inject N`, the first N tile requests of each burst are answered 429 by an
// XMLHttpRequest shim inside the page so the recovery contract can be proven
// deterministically; real 429s are still counted and kept as evidence. (CDP
// request interception is not used: it stalls Cesium's request scheduler.)
const inject = Number(flag('inject', 0));
const headful = argv.includes('--headful');
const TERRAIN_HOST = 'terrain.reearth.land';
const TILE_PATTERN = /\/cesium-mesh\/ellipsoid\/\d+\/\d+\/\d+\.terrain/;
const executablePath =
process.env.PUPPETEER_EXECUTABLE_PATH ||
(await puppeteer.executablePath().catch(() => null));
if (!executablePath || !fs.existsSync(executablePath)) {
throw new Error('Puppeteer Chrome for Testing is unavailable');
}
const browser = await puppeteer.launch({
headless: headful ? false : 'new',
executablePath,
args: [
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
'--no-sandbox',
],
});
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
/** @type {Map<string, {statuses: number[], failures: number}>} per tile URL. */
const tiles = new Map();
const evidence = [];
let layerJsonSeen = false;
let recording = false;
let requestsInWindow = 0;
let throttledInWindow = 0;
const tileEntry = (url) => {
if (!tiles.has(url)) tiles.set(url, { statuses: [], failures: 0 });
return tiles.get(url);
};
if (inject > 0) {
// Cesium fetches terrain through XMLHttpRequest and reads status, response
// and getAllResponseHeaders() in its onload handler; shadowing those on the
// instance is enough to make it see a throttled reply.
await page.evaluateOnNewDocument((pattern) => {
const tilePattern = new RegExp(pattern);
const shim = { remaining: 0, injected: [] };
window.__qaTerrainThrottle = shim;
const open = XMLHttpRequest.prototype.open;
const send = XMLHttpRequest.prototype.send;
XMLHttpRequest.prototype.open = function (method, url, ...rest) {
this.__qaUrl = String(url);
return open.call(this, method, url, ...rest);
};
XMLHttpRequest.prototype.send = function (...args) {
if (!(shim.remaining > 0 && tilePattern.test(this.__qaUrl || ''))) {
return send.apply(this, args);
}
shim.remaining -= 1;
shim.injected.push(this.__qaUrl);
const xhr = this;
setTimeout(() => {
for (const [key, value] of [
['status', 429],
['readyState', 4],
['response', null],
]) {
Object.defineProperty(xhr, key, { value, configurable: true });
}
xhr.getAllResponseHeaders = () => 'retry-after: 1\r\n';
xhr.onload?.(new Event('load'));
}, 30);
return undefined;
};
}, TILE_PATTERN.source);
}
page.on('response', async (response) => {
const url = response.url();
if (!url.includes(TERRAIN_HOST)) return;
if (url.endsWith('layer.json')) layerJsonSeen = true;
if (!TILE_PATTERN.test(url)) return;
const status = response.status();
tileEntry(url).statuses.push(status);
if (recording) requestsInWindow += 1;
if (status !== 429) return;
if (recording) throttledInWindow += 1;
if (evidence.length < 3) {
let body = '';
try {
body = (await response.text()).slice(0, 160);
} catch {
/* body unavailable */
}
evidence.push({ url, status, headers: response.headers(), body });
}
});
page.on('requestfailed', (request) => {
const url = request.url();
if (!TILE_PATTERN.test(url)) return;
tileEntry(url).failures += 1;
if (recording) requestsInWindow += 1;
});
const consoleErrors = [];
page.on('pageerror', (error) => consoleErrors.push(error.message));
page.on('console', (message) => {
if (message.type() === 'error') consoleErrors.push(message.text());
});
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const waitFor = async (predicate, timeoutMs, label) => {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (await predicate()) return;
await sleep(250);
}
throw new Error(`timed out waiting for ${label}`);
};
const setCamera = (lon, lat, height) =>
page.evaluate(
({ lon, lat, height }) => {
const viewer = window.__godsEyeView.viewer;
// The app's own startup flight would otherwise keep steering the camera
// through the burst and turn the zoom-out into a zoom-in.
viewer.camera.cancelFlight();
const cartographic =
viewer.camera.positionCartographic.constructor.fromDegrees(
lon,
lat,
height,
);
viewer.camera.setView({
destination:
viewer.scene.globe.ellipsoid.cartographicToCartesian(cartographic),
orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 },
});
window.__godsEyeView.requestRender?.();
},
{ lon, lat, height },
);
const cameraHeight = () =>
page.evaluate(
() => window.__godsEyeView.viewer.camera.positionCartographic.height,
);
const globeState = () =>
page.evaluate(() => {
const viewer = window.__godsEyeView.viewer;
const surface = viewer.scene.globe._surface;
let failed = 0;
let loaded = 0;
const visit = (tile) => {
if (!tile) return;
const state = tile.data?.terrainState;
if (state === 0) failed += 1;
else if (state === 6) loaded += 1;
for (const child of tile._children || []) visit(child);
};
for (const tile of surface._levelZeroTiles || []) visit(tile);
return {
failed,
loaded,
tilesLoaded: viewer.scene.globe.tilesLoaded,
provider: viewer.terrainProvider?.constructor?.name,
};
});
const burst = async () => {
if (mode === 'zoomOut') {
for (let i = 0; i < notches; i += 1) {
await page.evaluate(() => {
const viewer = window.__godsEyeView.viewer;
viewer.camera.zoomOut(viewer.camera.positionCartographic.height * 0.6);
window.__godsEyeView.requestRender?.();
});
await sleep(40);
}
return;
}
await page.mouse.move(640, 400);
for (let i = 0; i < notches; i += 1) {
await page.mouse.wheel({ deltaY: 200 });
await page.evaluate(() => window.__godsEyeView.requestRender?.());
await sleep(40);
}
};
try {
await page.goto(appUrl, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForFunction(() => Boolean(window.__godsEyeView?.viewer), {
timeout: 60000,
});
await waitFor(() => layerJsonSeen, 60000, 'the Re:Earth layer.json fetch');
await waitFor(
async () => (await globeState()).provider === 'CesiumTerrainProvider',
30000,
'the keyless terrain provider',
);
const summaries = [];
for (let round = 1; round <= rounds; round += 1) {
// Re-apply the view until it sticks: a still-running startup flight can
// overwrite the first setView on the very next frame.
for (let attempt = 0; attempt < 5; attempt += 1) {
await setCamera(-97.7431, 30.2672, 40000);
await sleep(1500);
const height = await cameraHeight();
if (Math.abs(height - 40000) < 1000) break;
}
await waitFor(
async () => (await globeState()).tilesLoaded,
30000,
'the 40 km view to settle',
).catch(() => {});
await sleep(2000);
const settledHeight = await cameraHeight();
if (Math.abs(settledHeight - 40000) > 1000) {
throw new Error(
`camera did not hold the 40 km start view (at ${Math.round(settledHeight)} m)`,
);
}
const before = await globeState();
tiles.clear();
requestsInWindow = 0;
throttledInWindow = 0;
if (inject > 0) {
await page.evaluate((count) => {
window.__qaTerrainThrottle.remaining = count;
window.__qaTerrainThrottle.injected = [];
}, inject);
}
recording = true;
const startHeight = await cameraHeight();
await burst();
const endHeight = await cameraHeight();
await sleep(settleMs);
recording = false;
// Injected throttles never reach the network, so fold them in from the
// page: a later real 200 for the same tile is the retry succeeding.
const injected =
inject > 0
? await page.evaluate(() => window.__qaTerrainThrottle.injected)
: [];
for (const url of injected) tileEntry(url).statuses.unshift(429);
const throttled = [...tiles.entries()].filter(([, entry]) =>
entry.statuses.includes(429),
);
const recovered = throttled.filter(([, entry]) =>
entry.statuses.includes(200),
);
const unrecovered = throttled.filter(
([, entry]) => !entry.statuses.includes(200),
);
const after = await globeState();
summaries.push({
round,
mode,
notches,
startHeightKm: Math.round(startHeight / 1000),
endHeightKm: Math.round(endHeight / 1000),
tileResponses: requestsInWindow,
throttledResponses: throttledInWindow,
injectedThrottles: injected.length,
throttledTiles: throttled.length,
recoveredTiles: recovered.length,
unrecoveredTiles: unrecovered.length,
failedTilesBefore: before.failed,
failedTilesAfter: after.failed,
loadedTilesAfter: after.loaded,
});
console.log(JSON.stringify(summaries.at(-1)));
if (unrecovered.length) {
console.log(
' unrecovered: ' +
unrecovered
.slice(0, 8)
.map(
([url, entry]) =>
`${new URL(url).pathname} [${entry.statuses.join(',')}]`,
)
.join(' · '),
);
}
}
if (evidence.length) {
console.log('first 429 evidence:');
for (const item of evidence) {
const headers = Object.fromEntries(
Object.entries(item.headers).filter(([name]) =>
/^(cf-|retry-after|server|content-type|date|x-ratelimit|ratelimit|access-control)/i.test(
name,
),
),
);
console.log(
` ${new URL(item.url).pathname} ${JSON.stringify(headers)} body=${JSON.stringify(item.body)}`,
);
}
}
const failures = summaries.filter(
(summary) => summary.unrecoveredTiles > 0 || summary.failedTilesAfter > 0,
);
const terrainErrors = consoleErrors.filter((text) =>
/terrain tile/i.test(text),
);
if (terrainErrors.length) {
console.log(`terrain console errors: ${terrainErrors.length}`);
console.log(` ${terrainErrors[0].slice(0, 200)}`);
}
console.log(
failures.length
? `FAIL — ${failures.length}/${summaries.length} round(s) left throttled terrain tiles unloaded`
: `PASS — every throttled terrain tile recovered in ${summaries.length} round(s)`,
);
process.exitCode = failures.length ? 1 : 0;
} finally {
await browser.close();
}
scripts/qa-traffic-motion.mjs
+533
View File
@@ -0,0 +1,533 @@
#!/usr/bin/env node
/**
* Traffic motion acceptance: real-GPU Puppeteer, stable identities, stationary
* camera holds and timestamped CDP JPEGs encoded with ffmpeg. No tile mocks.
* Run: node scripts/qa-traffic-motion.mjs http://localhost:4186
* --baseline records the unmodified pre-fix implementation (object identities
* via WeakMap, never collection indices); it reports failures without exiting 1.
* --mode=tomtom|osm|hybrid limits a diagnostic run. --tag=name keeps captures.
* --journey follows a neighborhood-to-city route with layer toggles, then
* checks Capitol/greenbelt/parking/corridor views against a 1.5 s limit.
*/
import fs from 'node:fs';
import path from 'node:path';
import { execFileSync } from 'node:child_process';
import puppeteer from 'puppeteer';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-traffic-motion.mjs <url> [--baseline] [--mode=tomtom|osm|hybrid] [--tag=name] [--journey]',
);
process.exit(0);
}
const url = args.find((a) => /^https?:/.test(a));
if (!url) throw new Error('A running dev server URL is required');
const baseline = args.includes('--baseline');
const journey = args.includes('--journey');
const option = (key) =>
args.find((a) => a.startsWith(`--${key}=`))?.split('=')[1];
const modes = option('mode') ? [option('mode')] : ['tomtom', 'osm', 'hybrid'];
if (modes.some((m) => !['tomtom', 'osm', 'hybrid'].includes(m)))
throw new Error('Invalid road mode');
const root = path.resolve(
'qa-shots/motion',
option('tag') || (baseline ? 'before' : '.'),
);
fs.mkdirSync(root, { recursive: true });
const revision = execFileSync('git', ['rev-parse', 'HEAD'], {
encoding: 'utf8',
}).trim();
const browser = await puppeteer.launch({
headless: true,
protocolTimeout: 300000,
args: [
'--use-angle=metal',
'--window-size=1280,800',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
const results = [];
const delay = (ms) => new Promise((r) => setTimeout(r, ms));
try {
for (const mode of modes) {
const dir = path.join(root, mode);
fs.mkdirSync(dir, { recursive: true });
const page = await browser.newPage();
const errors = [],
forbidden = [],
logs = [],
tileRequests = [];
page.on('pageerror', (e) => errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') errors.push(m.text());
if (m.text().startsWith('[Data:Traffic]')) logs.push(m.text());
});
page.on('request', (r) => {
const u = new URL(r.url());
if (
u.hostname === 'tiles.openfreemap.org' &&
/\/\d+\/\d+\/\d+(?:\.pbf)?$/.test(u.pathname)
)
tileRequests.push(u.pathname);
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
forbidden.push(u.hostname);
});
await page.setViewport({ width: 1280, height: 800 });
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
const u = new URL(url);
u.searchParams.set('trafficRoads', mode);
u.searchParams.set('trafficDebug', '1');
await page.goto(u.href, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(() => {
const g = window.__godsEyeView;
g.styleManager.setDetection({ enabled: false });
for (const [id] of g.dataManager.layers)
g.dataManager.setEnabled(id, false);
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
});
const identity = await page.evaluate(() => ({
title: document.title,
text: document.body.innerText.length,
overlay: Boolean(document.querySelector('vite-error-overlay')),
}));
await page.evaluate((baseline) => {
const { viewer, dataManager } = window.__godsEyeView;
const layer = dataManager.layers.get('traffic').module;
if (!baseline && typeof layer.visitMotionDots !== 'function')
throw new Error('Traffic must expose stable dot identities');
const scene = viewer.scene;
const weakIds = new WeakMap();
let nextId = 0,
rebuilds = 0;
const collections = [];
function discoverCollections() {
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (!Array.isArray(p._pointPrimitives) || collections.includes(p))
continue;
collections.push(p);
const removeAll = p.removeAll;
p.removeAll = function () {
if (this.length) rebuilds++;
return removeAll.call(this);
};
}
}
let previous = new Map(),
previousN = 0,
lastRebuilds = 0,
lastTime = 0;
const rows = [];
window.__motion = {
rows,
active: false,
moveEnds: [],
label: 'camera moving',
holdStart: null,
};
const clock = document.createElement('div');
clock.style.cssText =
'position:fixed;bottom:4px;left:38%;padding:3px 8px;background:#000c;color:#fff;font:12px monospace;z-index:999999;pointer-events:none';
document.body.append(clock);
viewer.camera.moveEnd.addEventListener(() =>
window.__motion.moveEnds.push(performance.now()),
);
scene.postRender.addEventListener(() => {
if (!window.__motion.active) return;
discoverCollections();
const now = performance.now(),
points = new Map();
clock.textContent =
`QA: ${window.__motion.label}` +
(window.__motion.holdStart === null
? ''
: ` | camera stopped +${((now - window.__motion.holdStart) / 1000).toFixed(1)}s`);
const visit = (id, roadId, point) => {
if (!point.show) return;
const s = scene.cartesianToCanvasCoordinates(point.position);
if (s && s.x >= 0 && s.y >= 0 && s.x <= 1280 && s.y <= 800)
points.set(id, { x: s.x, y: s.y, roadId });
};
let n = 0,
offRoad = 0;
if (!baseline && layer.visitMotionDots) {
layer.visitMotionDots((id, roadId, point, road) => {
if (point.show) {
const p = road?.roadProperties;
const allowed = road?.directFlow
? [
'Motorway',
'International road',
'Major road',
'Secondary road',
'Connecting road',
'Major local road',
'Local road',
'Minor local road',
].includes(p?.roadType)
: [
'motorway',
'trunk',
'primary',
'secondary',
'tertiary',
'minor',
].includes(p?.class) &&
!p?.service &&
!p?.subclass &&
!['no', 'private'].includes(p?.access) &&
p?.brunnel !== 'tunnel';
if (!allowed) offRoad++;
}
if (point.show) n++;
visit(id, roadId, point);
});
} else {
for (const coll of collections)
if (coll.show)
for (let i = 0; i < coll.length; i++) {
const p = coll.get(i);
if (!weakIds.has(p)) weakIds.set(p, ++nextId);
if (p.show) n++;
visit(weakIds.get(p), null, p);
}
}
let tracked = 0,
jumps = 0,
reassigned = 0,
births = 0,
deaths = 0;
for (const [id, p] of points) {
const prev = previous.get(id);
if (!prev) {
births++;
continue;
}
tracked++;
if (Math.hypot(p.x - prev.x, p.y - prev.y) > 30) jumps++;
if (p.roadId !== prev.roadId) reassigned++;
}
for (const id of previous.keys()) if (!points.has(id)) deaths++;
rows.push({
t: now,
loading: layer.getStats().loading,
syncing:
document
.getElementById('traffic-sync-chip')
?.classList.contains('visible') &&
document.getElementById('traffic-sync-progress')?.textContent ===
'...',
surfacePending: layer.getStats().surfacePending,
offRoad,
dt: now - lastTime,
n,
visible: points.size,
tracked,
jumps,
jumpFrac: tracked ? jumps / tracked : 0,
reassigned,
births,
deaths,
sizeChanged: n !== previousN,
rebuilds: rebuilds - lastRebuilds,
});
previous = points;
previousN = n;
lastRebuilds = rebuilds;
lastTime = now;
});
}, baseline);
const fly = (lat, lon, height, pitch, duration, heading = 20) =>
page.evaluate(
async (v) => {
const { viewer } = window.__godsEyeView;
window.__motion.label = 'camera moving';
window.__motion.holdStart = null;
viewer.camera.cancelFlight();
await new Promise((resolve) =>
viewer.camera.flyTo({
destination: viewer.scene.globe.ellipsoid.cartographicToCartesian(
{
latitude: (v.lat * Math.PI) / 180,
longitude: (v.lon * Math.PI) / 180,
height: v.height,
},
),
orientation: {
heading: (v.heading * Math.PI) / 180,
pitch: (v.pitch * Math.PI) / 180,
roll: 0,
},
duration: v.duration,
complete: resolve,
cancel: resolve,
}),
);
},
{ lat, lon, height, pitch, duration, heading },
);
const cdp = await page.createCDPSession();
let fi = 0;
const frames = [];
cdp.on('Page.screencastFrame', (f) => {
const name = `f${String(fi++).padStart(5, '0')}.jpg`;
fs.writeFileSync(path.join(dir, name), Buffer.from(f.data, 'base64'));
frames.push({ name, timestamp: f.metadata.timestamp });
cdp
.send('Page.screencastFrameAck', { sessionId: f.sessionId })
.catch(() => {});
});
await fly(
...(journey
? [30.25, -97.7497, 700, -40, 0, 10]
: [30.2685, -97.7425, 450, -35, 0]),
);
await cdp.send('Page.startScreencast', {
format: 'jpeg',
quality: 65,
maxWidth: 1280,
maxHeight: 800,
everyNthFrame: journey ? 1 : 3,
});
await page.evaluate(() => {
window.__motion.active = true;
});
const holds = [];
const hold = async (name, warm = false, holdMs = 10000, empty = false) => {
const start = await page.evaluate((label) => {
const start = performance.now();
window.__motion.label = label;
window.__motion.holdStart = start;
return start;
}, name);
await delay(holdMs);
const end = await page.evaluate(() => performance.now());
holds.push({
name,
warm,
start,
end,
empty,
settleMs: journey ? 1500 : warm ? 1000 : 3000,
});
await page.screenshot({ path: path.join(dir, `${name}.png`) });
console.log(`${mode}: captured ${name}`);
};
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
// Match motion-reference: begin the camera-motion scenario from a
// settled street view. Initial mesh acquisition is recorded in the clip
// and first-dot diagnostics, but is not a stationary motion hold.
if (journey) {
await delay(4000);
await hold('south-congress', true, 6000);
const step = async (name, view, ms = 6000, empty = false) => {
await fly(...view);
await hold(name, false, ms, empty);
};
await step('pan-north-400m', [30.2536, -97.7497, 700, -40, 2, 10], 4000);
await step('pan-east-500m', [30.2536, -97.7445, 700, -40, 2, 10], 4000);
await step('zoom-out-2.5km', [30.2536, -97.7445, 2500, -45, 2.5, 10]);
await step(
'downtown-2.5km',
[30.2672, -97.7431, 2500, -45, 2.5, 10],
5000,
);
await step('downtown-500m', [30.2672, -97.7431, 500, -35, 2.5, 20]);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('alpr-cameras', true),
);
await hold('enable-alpr', true, 6000);
await step('pan-west-600m', [30.2672, -97.7494, 500, -35, 2, 20], 5000);
await step('city-12km', [30.28, -97.75, 12000, -60, 3, 0], 8000, true);
await fly(30.3125, -97.765, 1000, -45, 3, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'military-installations',
true,
),
);
await hold('camp-mabry-1km', false, 8000);
await step('camp-mabry-pan', [30.3125, -97.76, 1000, -45, 2, 30], 5000);
await step('greenbelt', [30.312, -97.775, 1000, -55, 3, 0], 6000);
await step('capitol', [30.2747, -97.7404, 500, -70, 3, 0], 6000);
await step('commercial-parking', [30.307, -97.735, 600, -60, 3, 0], 6000);
await step('corridor', [30.347, -97.799, 1000, -45, 3, 0], 6000);
} else {
await delay(12000);
await page.waitForFunction(
() => window.__motion.rows.some((r) => r.n > 0),
{ timeout: 1000 },
);
await hold('street-warm', true);
await fly(30.2685, -97.7425, 3000, -45, 1.5);
await hold('zoom-out');
await fly(30.2685, -97.72, 3000, -45, 1.5);
await hold('pan-east');
await fly(30.2685, -97.725, 600, -35, 1.5);
await hold('zoom-in');
await fly(-23.5505, -46.6333, 1250, -35, 1.5);
await hold('sao-paulo');
await fly(30.2685, -97.7425, 450, -35, 1.5);
await hold('austin-revisit', true);
}
await page.evaluate(() => {
window.__motion.active = false;
});
await cdp.send('Page.stopScreencast');
const rows = await page.evaluate(() => window.__motion.rows);
const surfacePhases = await page.evaluate(() =>
performance
.getEntriesByType('measure')
.filter(
(e) => e.name.startsWith('roads:') || e.name.startsWith('traffic:'),
)
.map((e) => ({
name: e.name,
start: e.startTime,
duration: e.duration,
detail: e.detail,
})),
);
const stats = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers.get('traffic').module.getStats(),
);
const phases = holds.map((h) => {
const all = rows.filter((r) => r.t >= h.start && r.t < h.end);
const settled = all.filter((r) => r.t >= h.start + h.settleMs);
const n = settled.map((r) => r.n);
const lastBusy = all.findLastIndex(
(r) => r.loading || r.syncing || r.sizeChanged || r.jumpFrac > 0.02,
);
// First observed settled frame after the last population/loading event.
const steadyMs =
lastBusy < 0
? 0
: Math.round((all[lastBusy + 1]?.t ?? h.end) - h.start);
const phase = {
mode,
phase: h.name,
steadyMs,
lateLoadingFrames: settled.filter((r) => r.loading).length,
offRoadDots: Math.max(0, ...all.map((r) => r.offRoad)),
settleMs: h.settleMs,
frames: settled.length,
rebuilds: all.reduce((s, r) => s + r.rebuilds, 0),
lateRebuilds: settled.reduce((s, r) => s + r.rebuilds, 0),
jumpFrames: settled.filter((r) => r.jumpFrac > 0.02).length,
reassigned: settled.reduce((s, r) => s + r.reassigned, 0),
sizeChanges: settled.filter((r) => r.sizeChanged).length,
minDots: Math.min(...n),
maxDots: Math.max(...n),
maxFrameMs: Math.round(Math.max(...settled.map((r) => r.dt))),
};
phase.passed =
phase.frames >= 20 &&
phase.rebuilds <= 1 &&
phase.lateRebuilds === 0 &&
phase.jumpFrames === 0 &&
phase.reassigned === 0 &&
phase.sizeChanges === 0 &&
(h.empty ? phase.maxDots === 0 : phase.minDots > 0) &&
(!journey ||
(phase.steadyMs <= 1500 &&
phase.offRoadDots === 0 &&
all.at(-1)?.surfacePending === 0));
return phase;
});
fs.writeFileSync(
path.join(dir, 'frames.ffconcat'),
'ffconcat version 1.0\n' +
frames
.map(
(f, i) =>
`file '${f.name}'\nduration ${Math.max(0.001, Math.min(2, (frames[i + 1]?.timestamp || f.timestamp + 0.1) - f.timestamp))}\n`,
)
.join(''),
);
execFileSync('ffmpeg', [
'-y',
'-loglevel',
'error',
'-safe',
'0',
'-f',
'concat',
'-i',
path.join(dir, 'frames.ffconcat'),
'-fps_mode',
'vfr',
'-c:v',
'libx264',
'-pix_fmt',
'yuv420p',
'-vf',
journey ? 'scale=1280:-2' : 'scale=960:-2',
path.join(dir, 'clip.mp4'),
]);
const result = {
mode,
revision,
baseline,
identity,
url: u.href,
phases,
errors,
forbidden,
tileRequests,
duplicateTiles: tileRequests.filter(
(p, i) => tileRequests.indexOf(p) !== i,
),
stats,
surfacePhases,
logs,
holds,
rows,
moveEnds: await page.evaluate(() => window.__motion.moveEnds),
clip: path.join(dir, 'clip.mp4'),
frames: frames.length,
};
result.passed =
phases.every((p) => p.passed) &&
!errors.length &&
!forbidden.length &&
result.duplicateTiles.length === 0 &&
!identity.overlay &&
identity.text > 100;
fs.writeFileSync(
path.join(dir, 'metrics.json'),
JSON.stringify(result, null, 2),
);
results.push(result);
console.table(phases);
await page.close();
}
} finally {
await browser.close();
fs.writeFileSync(
path.join(root, 'results.json'),
JSON.stringify(
results.map(({ rows, ...r }) => r),
null,
2,
),
);
}
if (!baseline && results.some((r) => !r.passed)) process.exitCode = 1;
scripts/qa-traffic-oblique.mjs
+315
View File
@@ -0,0 +1,315 @@
#!/usr/bin/env node
/** Reticle coverage on oblique arrivals: source requests, toggle oracle and zoom revisits. */
import fs from 'node:fs';
import puppeteer from 'puppeteer';
import {
moveCamera,
createScreencast,
installFrameProbe,
} from './qa-journey-recorder.mjs';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-traffic-oblique.mjs <url> [--baseline] [--mode=hybrid|osm|tomtom]',
);
process.exit(0);
}
const baseline = args.includes('--baseline');
const modeArg = args.find((a) => a.startsWith('--mode='))?.split('=')[1];
const modes = modeArg ? [modeArg] : ['tomtom', 'osm', 'hybrid'];
const out = `qa-shots/oblique-${baseline ? 'before' : 'after'}`;
fs.mkdirSync(out, { recursive: true });
const result = { views: [], errors: [], forbidden: [], failures: [] };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const browser = await puppeteer.launch({
headless: true,
protocolTimeout: 180000,
args: [
'--use-angle=metal',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
try {
for (const mode of modes) {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
const counts = { ofm: 0, tomtom: 0 };
page.on('request', (r) => {
const u = new URL(r.url());
if (
u.hostname === 'tiles.openfreemap.org' &&
/\/\d+\/\d+\/\d+(\.pbf)?$/.test(u.pathname)
)
counts.ofm++;
if (u.pathname.startsWith('/api/tomtom/flow/')) counts.tomtom++;
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
result.forbidden.push(u.hostname);
});
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
const url = new URL(args.find((a) => /^https?:/.test(a)));
url.searchParams.set('trafficRoads', mode);
await page.goto(url.href, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.styleManager.setDetection({ enabled: false });
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
});
const snap = () =>
page.evaluate(() => {
const { viewer, dataManager } = window.__godsEyeView;
let upper = 0,
lower = 0,
near = 0;
const camera = viewer.camera.positionWC;
dataManager.layers
.get('traffic')
.module.visitMotionDots((id, key, p) => {
if (!p.show) return;
const s = viewer.scene.cartesianToCanvasCoordinates(p.position);
if (s && s.x >= 240 && s.x <= 1040 && s.y >= 0 && s.y <= 800) {
if (s.y < 400) upper++;
else lower++;
}
});
return {
upper,
lower,
...dataManager.layers.get('traffic').module.getStats(),
};
});
const settle = async () => {
await sleep(700);
await page
.waitForFunction(
() => {
const s = window.__godsEyeView.dataManager.layers
.get('traffic')
.module.getStats();
return !s.loading;
},
{ timeout: 15000 },
)
.catch(() => {});
await sleep(1200);
return snap();
};
for (const [name, lat, lon, height] of [
['dubai', 24.964, 55.037, 276],
['austin', 30.2672, -97.7431, 300],
['london', 51.507, -0.128, 300],
]) {
const before = { ...counts };
await moveCamera(page, { lat, lon, height: 3000, pitch: -45 }, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
await moveCamera(page, { lat, lon, height, pitch: -15 }, 1.4);
const settled = await settle();
const load = {
ofm: counts.ofm - before.ofm,
tomtom: counts.tomtom - before.tomtom,
};
await page.screenshot({ path: `${out}/${name}-${mode}.png` });
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', false),
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
const toggled = await settle();
const oracle =
Math.abs(settled.upper - toggled.upper) / Math.max(1, toggled.upper);
const preZoom = { ...counts };
await moveCamera(page, { lat, lon, height: 3000, pitch: -15 }, 0.7);
await settle();
const atWide = { ...counts };
await moveCamera(page, { lat, lon, height, pitch: -15 }, 0.7);
await settle();
const revisit = {
ofm: counts.ofm - atWide.ofm,
tomtom: counts.tomtom - atWide.tomtom,
};
const row = {
mode,
name,
load,
settled,
toggled,
oracle,
revisit,
zoomOut: {
ofm: atWide.ofm - preZoom.ofm,
tomtom: atWide.tomtom - preZoom.tomtom,
},
};
result.views.push(row);
console.log(JSON.stringify(row));
if (oracle > 0.15)
result.failures.push(`${name}/${mode}: upper oracle ${oracle}`);
if (revisit.ofm || revisit.tomtom)
result.failures.push(`${name}/${mode}: revisit requests`);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', false),
);
}
const locationView = { lat: 24.973, lon: 55.037 };
await moveCamera(
page,
{ lat: 24.964, lon: 55.037, height: 3000, pitch: -45 },
0,
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
const dir = `${out}/dubai-${mode}-journey`;
const recording =
mode === 'hybrid' ? await createScreencast(page, dir, { fps: 24 }) : null;
await installFrameProbe(page);
await page.evaluate(() => {
const { viewer, dataManager } = window.__godsEyeView;
window.__obliqueFrames = [];
window.__obliqueMoving = true;
viewer.scene.postRender.addEventListener(() => {
const s = dataManager.layers.get('traffic').module.getStats();
let upper = 0,
lower = 0,
near = 0;
const camera = viewer.camera.positionWC;
dataManager.layers
.get('traffic')
.module.visitMotionDots((id, key, p) => {
if (!p.show) return;
const q = viewer.scene.cartesianToCanvasCoordinates(p.position);
if (q && q.x >= 240 && q.x <= 1040 && q.y >= 0 && q.y <= 800) {
if (q.y < 400) upper++;
else lower++;
const dx = p.position.x - camera.x,
dy = p.position.y - camera.y,
dz = p.position.z - camera.z;
if (dx * dx + dy * dy + dz * dz <= 1500 * 1500) near++;
}
});
window.__obliqueFrames.push({
t: performance.now(),
moving: window.__obliqueMoving,
count: s.count,
loading: s.loading,
upper,
lower,
near,
publishes: s.motion.publishes,
rebuilds: s.motion.rebuilds,
});
});
});
await recording?.start();
await page.evaluate(async (v) => {
const { flyToLandmark } = await import('/src/locations.js');
await new Promise((resolve) =>
flyToLandmark(window.__godsEyeView.viewer, v.lat, v.lon, {
range: 1000,
pitch: -15,
heading: 0,
duration: 1.6,
buildingHeight: 0,
onComplete: resolve,
onCancel: resolve,
}),
);
window.__obliqueMoving = false;
}, locationView);
const arrived = await settle();
await page.evaluate(() => {
window.__obliqueMoving = true;
});
await moveCamera(
page,
{ lat: 24.964, lon: 55.042, height: 300, pitch: -15 },
1.2,
);
await page.evaluate(() => {
window.__obliqueMoving = false;
});
const panned = await settle();
await sleep(1800);
const frames = await page.evaluate(() => window.__obliqueFrames);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', false),
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
const fresh = await settle();
const oracle =
Math.abs(panned.upper - fresh.upper) / Math.max(1, fresh.upper);
const stops = frames.flatMap((f, i) =>
i && !f.moving && frames[i - 1].moving ? [i] : [],
);
const nearSettles = stops.map((start, index) => {
let end = start + 1;
while (end < frames.length && !frames[end].moving) end++;
const phase = frames.slice(start, end),
target = phase.at(-1).near;
const reached = phase.find((f) => f.near >= target * 0.85);
const ms = reached ? Math.round(reached.t - phase[0].t) : Infinity;
if (target && ms > 1100)
result.failures.push(
`dubai-location-pan/${mode}/${index}: near settle ${ms} ms`,
);
return {
phase: index === 0 ? 'arrival' : 'pan',
ms,
target,
first: phase[0].near,
};
});
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(`${dir}/metrics.json`, JSON.stringify(frames));
const clip = await recording?.stop();
result.views.push({
mode,
name: 'dubai-location-pan',
nearSettles,
arrived,
settled: panned,
toggled: fresh,
oracle,
clip,
});
if (oracle > 0.15)
result.failures.push(
`dubai-location-pan/${mode}: upper oracle ${oracle}`,
);
await page.close();
}
} finally {
fs.writeFileSync(`${out}/result.json`, JSON.stringify(result, null, 2));
await browser.close();
}
if (
!baseline &&
(result.failures.length || result.errors.length || result.forbidden.length)
)
process.exitCode = 1;
scripts/qa-voice-routing.mjs
+16 -1
View File
@@ -625,12 +625,27 @@ async function runBehaviorLayer() {
// (4b) analyst engine end-to-end: count flights over Texas (region ring
// via NE-pack/admin machinery), then a follow-up over the same set.
// A cold boundary lookup can exceed the resolver's budget; it answers
// region-timeout and keeps loading, so asking again must succeed.
let texasTimeouts = 0;
r = await run('analyst_query', { layers: ['flights'], scope: { kind: 'region', name: 'Texas' }, limit: 5 });
while (r?.code === 'region-timeout' && texasTimeouts < 8) {
texasTimeouts += 1;
await settle(5000);
r = await run('analyst_query', { layers: ['flights'], scope: { kind: 'region', name: 'Texas' }, limit: 5 });
}
report(r?.ok === true && Number.isFinite(r?.count) && r.count > 0 && String(r?.coverage?.scope || '').includes('Texas'),
'behavior: analyst counts flights over Texas', `count=${r?.count} scope=${r?.coverage?.scope} err=${r?.error || ''}`);
'behavior: analyst counts flights over Texas', `count=${r?.count} scope=${r?.coverage?.scope} timeouts=${texasTimeouts} err=${r?.error || ''}`);
r = await run('analyst_query', { followUp: true, filters: [{ field: 'onGround', op: 'eq', value: false }], sortBy: 'altitudeM', limit: 3 });
report(r?.ok === true && r?.coverage?.followUp === true,
'behavior: analyst follow-up re-filters the remembered set', `count=${r?.count} followUp=${r?.coverage?.followUp}`);
// A marine region resolves from the bundled Natural Earth pack in the page,
// without the slower geocode fallback.
const gulfStarted = Date.now();
r = await run('analyst_query', { layers: ['flights'], scope: { kind: 'region', name: 'Gulf of Mexico' }, limit: 5 });
const gulfMs = Date.now() - gulfStarted;
report(r?.ok === true && String(r?.coverage?.scope || '').includes('Gulf of Mexico') && gulfMs < 3000,
'behavior: analyst resolves the Gulf of Mexico from the bundled pack', `scope=${r?.coverage?.scope} ms=${gulfMs} err=${r?.error || ''}`);
// (5) zoom_to_globe is ABSOLUTE full-earth (>12,000 km band)
r = await run('zoom_to_globe', {});
scripts/qa-weather-journey.mjs
+423
View File
@@ -0,0 +1,423 @@
#!/usr/bin/env node
/**
* qa-weather-journey — drive the Weather layers the way a person would and
* keep the evidence: numbered screenshots plus per-step layer diagnostics.
*
* Starts at the Earth view, opens Data Layers, switches Wind on from its real
* row button, cycles the wind fields/models, switches radar, satellite clouds,
* lightning and cyclones on, follows "View storm" and "View US radar", zooms
* into a regional view, a city view and a street view, replays history, and
* ends with real wheel/drag input. Every step records the enabled layers'
* diagnostics (imagery tiles loaded, shown observation time, wind geometry)
* and a 2 s `scene.postRender` count so a stalled renderer cannot pass as a
* healthy screenshot.
*
* Usage:
* node scripts/qa-weather-journey.mjs [--url http://localhost:4173]
* [--label google] [--headless] [--out qa-shots/weather-journey]
*
* Requires a running dev server. Screenshots and steps.json land in
* <out>/<label>/.
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg('--label', 'journey');
const outDir = path.resolve(arg('--out', 'qa-shots/weather-journey'), label);
mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const WEATHER = ['weather-radar', 'weather-satellite', 'weather-lightning'];
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
),
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--window-size=1400,1000',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const steps = [];
let index = 0;
try {
const page = await browser.newPage();
await page.setViewport({ width: 1400, height: 920, deviceScaleFactor: 2 });
page.on('pageerror', (e) => console.log(' [pageerror]', e.message));
// Non-2xx responses, counted per path so a burst of throttled tiles reads
// as one line instead of one console error per tile.
const failures = new Map();
page.on('response', (r) => {
const status = r.status();
if (status < 400) return;
const key = `${status} ${new URL(r.url()).pathname}`;
failures.set(key, (failures.get(key) || 0) + 1);
});
const reportFailures = () => {
if (!failures.size) return;
console.log(
' [http failures] ' +
[...failures].map(([k, n]) => `${k} ×${n}`).join(' · '),
);
failures.clear();
};
page.on('console', (m) => {
if (m.type() === 'error')
console.log(' [console.error]', m.text().slice(0, 200));
});
// The Data Layers panel remembers its collapsed state; start it open.
await page.evaluateOnNewDocument(() => {
try {
localStorage.setItem('godsEyeView.v6.panelCollapsed.data-panel', '0');
} catch {
/* storage may be unavailable */
}
});
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(12_000);
const diagnostics = () =>
page.evaluate(
({ WEATHER }) => {
const gev = window.__godsEyeView;
const dm = gev.dataManager;
const out = {
basemap: gev.tileset ? 'google-3d' : 'keyless-globe',
globeShown: gev.viewer.scene.globe.show,
heightM: Math.round(gev.viewer.camera.positionCartographic.height),
summaryHeight: Math.round(
document.querySelector('.weather-summary')?.getBoundingClientRect()
.height ?? 0,
),
};
for (const id of ['wind', ...WEATHER, 'weather-cyclones']) {
const entry = dm.layers.get(id);
if (!entry?.enabled) continue;
const d = entry.module?.getDiagnostics?.() || {};
const s = entry.module?.getStats?.() || {};
out[id] =
id === 'wind'
? {
mode: d.renderMode,
ready: d.gpu?.ready,
paths: d.gpu?.pathCount,
visibleCells: d.gpu?.visibleCells,
heightFade: d.gpu?.heightFade,
overlay: d.overlay,
imagery: d.imageryActive,
error: s.error,
}
: id === 'weather-cyclones'
? { count: s.count, error: s.error }
: {
time: d.time,
playing: d.playing,
loading: d.loading,
loadedTiles: d.loadedTiles,
frameLoadMs: d.frameLoadMs && Math.round(d.frameLoadMs),
error: d.error,
};
}
return out;
},
{ WEATHER },
);
const renders = (ms = 2000) =>
page.evaluate(
(ms) =>
new Promise((resolve) => {
const scene = window.__godsEyeView.viewer.scene;
let n = 0;
const off = scene.postRender.addEventListener(() => {
n += 1;
});
setTimeout(() => {
off();
resolve(n);
}, ms);
}),
ms,
);
const shot = async (name, note = '') => {
index += 1;
const file = `${String(index).padStart(2, '0')}-${name}.png`;
const d = await diagnostics();
const r = await renders(2000);
await page.screenshot({ path: path.join(outDir, file) });
reportFailures();
steps.push({ index, name, note, file, renders2s: r, ...d });
console.log(` ${file} renders/2s=${r} ${note} ${JSON.stringify(d)}`);
};
const clickRow = async (id) => {
const ok = await page.evaluate((id) => {
const btn = document.querySelector(
`[data-layer-id="${id}"] .data-toggle-btn`,
);
if (!btn) return false;
btn.scrollIntoView({ block: 'center' });
btn.click();
return true;
}, id);
if (!ok) console.log(` !! no row button for ${id}`);
};
const clickChip = async (id, chip) => {
const ok = await page.evaluate(
({ id, chip }) => {
const btn = document.querySelector(
`[data-layer-id="${id}"] [data-chip-id="${chip}"]`,
);
if (!btn || btn.disabled) return false;
btn.scrollIntoView({ block: 'center' });
btn.click();
return true;
},
{ id, chip },
);
if (!ok) console.log(` !! chip ${id}/${chip} missing or disabled`);
return ok;
};
const waitReady = (ids, timeout = 90_000) =>
page
.waitForFunction(
(ids) => {
const dm = window.__godsEyeView.dataManager;
for (const id of ids) {
const entry = dm.layers.get(id);
if (!entry?.enabled) return false;
const d = entry.module?.getDiagnostics?.() || {};
if (id === 'wind') {
if (d.renderMode === 'gpu-streamlines' && !d.gpu?.ready)
return false;
} else if (id !== 'weather-cyclones') {
if (d.loading || (!d.time && !d.error)) return false;
}
}
return true;
},
{ timeout, polling: 250 },
ids,
)
.catch(() =>
console.log(` !! ${ids.join(',')} not ready within ${timeout} ms`),
);
const fly = async (
lon,
lat,
height,
pitch = -90,
heading = 0,
duration = 2,
) => {
await page.evaluate(
({ lon, lat, height, pitch, heading, duration }) => {
const v = window.__godsEyeView.viewer;
const d2r = Math.PI / 180;
v.camera.cancelFlight();
v.camera.flyTo({
destination: v.scene.ellipsoid.cartographicToCartesian({
longitude: lon * d2r,
latitude: lat * d2r,
height,
}),
orientation: { heading: heading * d2r, pitch: pitch * d2r, roll: 0 },
duration,
});
},
{ lon, lat, height, pitch, heading, duration },
);
await sleep(duration * 1000 + 3000);
};
console.log(`\nqa-weather-journey · ${label} · ${url}`);
await page.evaluate(async () => {
const gev = window.__godsEyeView;
for (const [id, entry] of gev.dataManager.layers)
if (entry.enabled)
await gev.dataManager.setEnabled(id, false, { origin: 'user' });
document.querySelector('#data-panel')?.classList.remove('collapsed');
});
// 1. Earth view, clean.
await fly(-97.7431, 30.2672, 17_368_000);
await shot('globe-clean', 'Earth view, no layers');
// 2. Wind from its row, then each field, then ECMWF.
await clickRow('wind');
await waitReady(['wind']);
await sleep(1500);
await shot('globe-wind-trails', 'Wind on (trails, no field)');
await clickChip('wind', 'overlay-speed');
await sleep(2500);
await shot('globe-wind-speed', 'Wind speed field');
await clickChip('wind', 'overlay-temperature');
await sleep(2500);
await waitReady(['wind']);
await shot('globe-wind-temperature', 'Wind + 2 m temperature');
await clickChip('wind', 'overlay-pressure');
await sleep(2500);
await waitReady(['wind']);
await shot('globe-wind-pressure', 'Wind + MSL pressure');
await clickChip('wind', 'model-ifs');
await waitReady(['wind']);
await sleep(2500);
await shot('globe-wind-ecmwf-pressure', 'ECMWF IFS model');
await clickChip('wind', 'model-gfs');
await clickChip('wind', 'overlay-speed');
await waitReady(['wind']);
await sleep(1500);
// 3. Observed products.
for (const id of WEATHER) await clickRow(id);
await clickRow('weather-cyclones');
await waitReady([...WEATHER]);
await sleep(2500);
await shot(
'globe-all-weather',
'Wind speed + radar + regional IR + lightning + cyclones',
);
await clickChip('weather-satellite', 'clouds');
await waitReady(['weather-satellite']);
await sleep(2500);
await shot('globe-global-infrared', 'Satellite: global infrared mosaic');
await clickChip('weather-satellite', 'clouds-regional');
await waitReady(['weather-satellite']);
// 4. Follow a storm if there is one (advisories arrive a few seconds after enable).
await page
.waitForFunction(
() =>
!!document.querySelector(
'[data-layer-id="weather-cyclones"] .data-toggle-chip:not([data-chip-id="focus"]):not([data-chip-id="advisory"])',
),
{ timeout: 25_000, polling: 500 },
)
.catch(() => {});
const storm = await page.evaluate(() => {
const first = document.querySelector(
'[data-layer-id="weather-cyclones"] .data-toggle-chip:not([data-chip-id="focus"]):not([data-chip-id="advisory"])',
);
if (!first) return null;
first.click();
return first.textContent.trim();
});
if (storm) {
await sleep(1500);
if (await clickChip('weather-cyclones', 'focus')) {
await sleep(6000);
await shot('storm-view', `View storm: ${storm}`);
await page.evaluate(() =>
window.__godsEyeView.viewer.camera.zoomIn(400_000),
);
await sleep(4000);
await shot('storm-closer', 'Zoomed toward the storm');
}
} else console.log(' (no active cyclone advisories to follow)');
// 5. US radar, then regional → city → street over the same spot.
await clickChip('weather-radar', 'coverage');
await sleep(6000);
await shot('us-radar', 'View US radar');
await fly(-94.6, 37.5, 600_000);
await shot('regional-600km', 'Regional 600 km');
await fly(-94.6, 37.5, 120_000);
await shot('regional-120km', 'Regional 120 km');
await fly(-97.7431, 30.2672, 12_000, -60, 20);
await shot('city-12km', 'Austin 12 km oblique');
await fly(-97.7431, 30.2672, 1_200, -45, 20);
await shot('city-1200m', 'Austin 1.2 km oblique (the perf route pose)');
await fly(-97.7431, 30.2672, 350, -30, 60);
await shot('street-350m', 'Austin 350 m street level');
// 6. History playback at the city pose (panel behavior while frames swap).
await clickChip('weather-radar', 'play');
const heights = [];
for (let i = 0; i < 4; i++) {
await sleep(2200);
heights.push(
await page.evaluate(() =>
Math.round(
document.querySelector('.weather-summary')?.getBoundingClientRect()
.height ?? 0,
),
),
);
}
await shot(
'city-history-playing',
`History playing; summary heights ${heights.join('/')}`,
);
await clickChip('weather-radar', 'play');
await clickChip('weather-radar', 'latest');
// 7. Real input: wheel-zoom out, drag, wheel-zoom in.
await fly(-97.7431, 30.2672, 40_000, -55, 0, 1.5);
const canvas = await page.$(
'canvas.cesium-widget-canvas, .cesium-widget canvas',
);
const box = canvas
? await canvas.boundingBox()
: { x: 700, y: 460, width: 0, height: 0 };
const cx = box.x + (box.width || 0) / 2;
const cy = box.y + (box.height || 0) / 2;
await page.mouse.move(cx, cy);
for (let i = 0; i < 12; i++) {
await page.mouse.wheel({ deltaY: 240 });
await sleep(120);
}
await sleep(3000);
await shot('wheel-out', 'Wheel zoom out ×12');
await page.mouse.move(cx - 200, cy);
await page.mouse.down();
for (let i = 1; i <= 20; i++) {
await page.mouse.move(cx - 200 + i * 20, cy + i * 4, { steps: 2 });
await sleep(30);
}
await page.mouse.up();
await sleep(2500);
await shot('drag-pan', 'Drag pan');
for (let i = 0; i < 10; i++) {
await page.mouse.wheel({ deltaY: -240 });
await sleep(120);
}
await sleep(4000);
await shot('wheel-in', 'Wheel zoom in ×10');
// 8. Back to Earth with everything on.
await fly(-97.7431, 30.2672, 17_368_000, -90, 0, 2.5);
await shot('globe-return', 'Return to Earth view');
writeFileSync(
path.join(outDir, 'steps.json'),
JSON.stringify({ label, url, steps }, null, 2),
);
console.log(`\nWrote ${steps.length} steps to ${outDir}`);
} finally {
await browser.close();
}
scripts/qa-weather-perf.mjs
+568
View File
@@ -0,0 +1,568 @@
#!/usr/bin/env node
/**
* qa-weather-perf — matched-route throughput and idle-work measurement for the
* Weather layers (Wind, Rain radar, Satellite clouds, Lightning density).
*
* This is a measurement harness, not a pass/fail gate. It reproduces one fixed
* camera route per weather state so two builds (or two basemaps) can be
* compared on the same numbers:
*
* park 10 km over Austin → 3 s flight to 1.2 km (heading 20°, pitch −45°)
* → 6 s close idle → 8 s orbit at 1.8 km range → 3 s flight back to the
* 17,368 km globe view → 6 s globe idle.
*
* Route windows hold render demand constant (`scene.requestRender()` on every
* animation frame) so the idle-render governor cannot masquerade as low FPS.
* FPS counts `scene.postRender` fires, and every window asserts the canvas
* size did not change, because a stopped renderer with a free-running rAF
* reads as a clean 60 fps otherwise.
*
* The separate idle test restores the normal governor, parks the camera at
* the 1.2 km pose with Wind off, and counts scene redraws, DOM mutations
* inside `.weather-summary`, and `.weather-summary` height changes over
* 8-second windows for: weather off, history paused, history playing.
*
* Usage:
* node scripts/qa-weather-perf.mjs [--url http://localhost:4173]
* [--label google-baseline] [--states off,wind,...] [--repeat 1]
* [--headless] [--no-route] [--no-idle] [--out qa-shots/weather-perf]
*
* States: off · history-paused · history-playing · wind · wind+history-paused
* · wind+history-playing. "history" is radar + regional clouds + lightning
* together; "wind" is animated GFS wind with the Speed field selected.
* Requires a running dev server. Screenshots and results.json land in
* <out>/<label>/.
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
// The first-launch chooser would otherwise sit over the globe; `?welcome=0`
// is the app's own suppression flag (src/firstRunExperience.js).
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg(
'--label',
new Date().toISOString().slice(0, 19).replace(/[:T]/g, '-'),
);
const ALL_STATES = [
'off',
'history-paused',
'history-playing',
'wind',
'wind+history-paused',
'wind+history-playing',
];
const states = arg('--states', ALL_STATES.join(','))
.split(',')
.map((s) => s.trim())
.filter(Boolean);
for (const s of states)
if (!ALL_STATES.includes(s)) {
console.error(`unknown state "${s}"; choose from ${ALL_STATES.join(', ')}`);
process.exit(2);
}
const repeat = Math.max(1, Number(arg('--repeat', '1')) || 1);
// `--scale 2` renders at 2× resolutionScale (4× the pixels) so a GPU that idles
// at 60 fps natively still exposes per-frame cost differences.
const scale = Number(arg('--scale', '1')) || 1;
const outDir = path.resolve(arg('--out', 'qa-shots/weather-perf'), label);
mkdirSync(outDir, { recursive: true });
const WEATHER = ['weather-radar', 'weather-satellite', 'weather-lightning'];
const AUSTIN = { lon: -97.7431, lat: 30.2672 };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const chromePath =
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
);
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath: chromePath,
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--window-size=1200,960',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const results = [];
let reportFailures = null;
const record = (row) => {
results.push(row);
reportFailures?.();
const {
phase,
state,
run,
window: win,
fps,
p95,
longTasks,
mutations,
heightChanges,
canvasStable,
} = row;
console.log(
` ${phase.padEnd(5)} ${state.padEnd(22)} r${run} ${win.padEnd(12)} ` +
`fps=${String(fps).padStart(5)} p95=${String(p95).padStart(6)}ms ` +
`long=${String(longTasks).padStart(2)} dom=${String(mutations).padStart(4)} ` +
`h∆=${String(heightChanges).padStart(2)}${canvasStable ? '' : ' CANVAS CHANGED'}`,
);
};
try {
const page = await browser.newPage();
await page.setViewport({ width: 1200, height: 886, deviceScaleFactor: 2 });
page.on('pageerror', (e) => console.log(' [pageerror]', e.message));
// Non-2xx responses, counted per path so a burst of throttled tiles reads
// as one line instead of one console error per tile.
const failures = new Map();
page.on('response', (r) => {
const status = r.status();
if (status < 400) return;
const key = `${status} ${new URL(r.url()).pathname}`;
failures.set(key, (failures.get(key) || 0) + 1);
});
reportFailures = () => {
if (!failures.size) return;
console.log(
' [http failures] ' +
[...failures].map(([k, n]) => `${k} ×${n}`).join(' · '),
);
failures.clear();
};
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(15_000);
if (scale !== 1)
await page.evaluate((scale) => {
window.__godsEyeView.viewer.resolutionScale = scale;
window.__godsEyeView.viewer.scene.requestRender();
}, scale);
const env = await page.evaluate(() => {
const gev = window.__godsEyeView;
const scene = gev.viewer.scene;
return {
basemap: gev.tileset ? 'google-3d' : 'keyless-globe',
globeShown: scene.globe.show,
canvas: `${scene.canvas.width}x${scene.canvas.height}`,
dpr: devicePixelRatio,
ua:
navigator.userAgent.match(/Chrome\/[\d.]+/)?.[0] || navigator.userAgent,
msaa: scene.msaaSamples,
resolutionScale: gev.viewer.resolutionScale,
};
});
console.log(`\nqa-weather-perf · ${label}\n url=${url}`);
console.log(` ${JSON.stringify(env)}`);
// Park deterministically, disable every layer, and instrument once.
await page.evaluate(async () => {
const gev = window.__godsEyeView;
const v = gev.viewer;
v.camera.cancelFlight();
for (const [id, entry] of gev.dataManager.layers)
if (entry.enabled) {
try {
await gev.dataManager.setEnabled(id, false, { origin: 'user' });
} catch {
/* counted through the layer, not here */
}
}
const P = (window.__gevWeatherPerf = {
renders: 0,
longTasks: 0,
longTaskMs: 0,
mutations: 0,
});
v.scene.postRender.addEventListener(() => {
P.renders += 1;
});
try {
new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
P.longTasks += 1;
P.longTaskMs += entry.duration;
}
}).observe({ type: 'longtask', buffered: false });
} catch {
/* longtask unsupported → stays 0 */
}
const summary = document.querySelector('.weather-summary');
if (summary)
new MutationObserver((records) => {
P.mutations += records.length;
}).observe(summary, {
subtree: true,
childList: true,
attributes: true,
characterData: true,
});
});
const pose = (which) =>
page.evaluate(
({ which, AUSTIN }) => {
const C = window.Cesium || null;
const v = window.__godsEyeView.viewer;
v.camera.cancelFlight();
v.camera.lookAtTransform?.(
(C || v.camera.constructor).Matrix4?.IDENTITY ??
v.scene.camera.transform,
);
const ell = v.scene.ellipsoid || v.scene.globe.ellipsoid;
const d2r = Math.PI / 180;
const destination = ell.cartographicToCartesian({
longitude: AUSTIN.lon * d2r,
latitude: AUSTIN.lat * d2r,
height:
which === 'overhead'
? 10_000
: which === 'close'
? 1_200
: 17_368_000,
});
v.camera.setView({
destination,
orientation:
which === 'close'
? { heading: 20 * d2r, pitch: -45 * d2r, roll: 0 }
: { heading: 0, pitch: -90 * d2r, roll: 0 },
});
},
{ which, AUSTIN },
);
/** Start a 3 s flight without awaiting it; the caller measures alongside. */
const fly = (which) =>
page.evaluate(
({ which, AUSTIN }) => {
const v = window.__godsEyeView.viewer;
const ell = v.scene.ellipsoid || v.scene.globe.ellipsoid;
const d2r = Math.PI / 180;
v.camera.flyTo({
destination: ell.cartographicToCartesian({
longitude: AUSTIN.lon * d2r,
latitude: AUSTIN.lat * d2r,
height: which === 'close' ? 1_200 : 17_368_000,
}),
orientation:
which === 'close'
? { heading: 20 * d2r, pitch: -45 * d2r, roll: 0 }
: { heading: 0, pitch: -90 * d2r, roll: 0 },
duration: 3,
});
},
{ which, AUSTIN },
);
/** Start an 8 s orbit at 1.8 km range around the Austin anchor (not awaited). */
const orbit = () =>
page.evaluate(
({ AUSTIN }) => {
const v = window.__godsEyeView.viewer;
const camera = v.camera;
const ell = v.scene.ellipsoid || v.scene.globe.ellipsoid;
const d2r = Math.PI / 180;
const center = ell.cartographicToCartesian({
longitude: AUSTIN.lon * d2r,
latitude: AUSTIN.lat * d2r,
height: 0,
});
const HPR = camera.constructor?.HeadingPitchRange
? camera.constructor.HeadingPitchRange
: null;
const t0 = performance.now();
let heading = 20 * d2r;
const step = (now) => {
const elapsed = (now - t0) / 1000;
heading = 20 * d2r + (elapsed / 8) * Math.PI * 2;
const hpr = HPR
? new HPR(heading, -45 * d2r, 1800)
: { heading, pitch: -45 * d2r, range: 1800 };
camera.lookAt(center, hpr);
if (elapsed < 8) requestAnimationFrame(step);
else camera.lookAtTransform(camera.transform.constructor.IDENTITY);
};
requestAnimationFrame(step);
},
{ AUSTIN },
);
/** Measure one window; `hold` keeps render demand constant. */
const measure = (ms, hold) =>
page.evaluate(
({ ms, hold }) =>
new Promise((resolve) => {
const gev = window.__godsEyeView;
const scene = gev.viewer.scene;
const P = window.__gevWeatherPerf;
const canvas = scene.canvas;
const w0 = canvas.width;
const h0 = canvas.height;
const r0 = P.renders;
const l0 = P.longTasks;
const lm0 = P.longTaskMs;
const m0 = P.mutations;
const summary = document.querySelector('.weather-summary');
const intervals = [];
const heights = [];
let lastH = -1;
let rafs = 0;
let last = performance.now();
const t0 = last;
const finish = (now) => {
const seconds = (now - t0) / 1000;
intervals.sort((a, b) => a - b);
const q = (p) =>
intervals[
Math.min(intervals.length - 1, Math.floor(p * intervals.length))
] ?? 0;
const hs = heights.map(([, h]) => h);
resolve({
seconds: Number(seconds.toFixed(2)),
renders: P.renders - r0,
fps: Number(((P.renders - r0) / seconds).toFixed(1)),
rafHz: Number((rafs / seconds).toFixed(1)),
p50: Number(q(0.5).toFixed(1)),
p95: Number(q(0.95).toFixed(1)),
max: Number((intervals.at(-1) ?? 0).toFixed(1)),
longTasks: P.longTasks - l0,
longTaskMs: Math.round(P.longTaskMs - lm0),
mutations: P.mutations - m0,
heightChanges: Math.max(0, heights.length - 1),
heightMin: hs.length ? Math.min(...hs) : null,
heightMax: hs.length ? Math.max(...hs) : null,
canvasStable: canvas.width === w0 && canvas.height === h0,
canvas: `${w0}x${h0}`,
});
};
const tick = (now) => {
rafs += 1;
intervals.push(now - last);
last = now;
if (hold) scene.requestRender();
if (summary && !summary.hidden) {
const h =
Math.round(summary.getBoundingClientRect().height * 2) / 2;
if (h !== lastH) {
heights.push([Math.round(now - t0), h]);
lastH = h;
}
}
if (now - t0 < ms) requestAnimationFrame(tick);
else finish(now);
};
requestAnimationFrame(tick);
}),
{ ms, hold },
);
const diagnostics = () =>
page.evaluate(
({ WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
const out = {};
for (const id of ['wind', ...WEATHER]) {
const entry = dm.layers.get(id);
if (!entry?.enabled) continue;
const d = entry.module?.getDiagnostics?.() || {};
out[id] =
id === 'wind'
? {
mode: d.renderMode,
ready: d.gpu?.ready,
paths: d.gpu?.pathCount,
vertices: d.gpu?.vertexCount,
buildMs: d.gpu?.buildMs && Math.round(d.gpu.buildMs),
cells: d.gpu?.cellCount,
visibleCells: d.gpu?.visibleCells,
visibleVertices: d.gpu?.visibleVertexCount,
heightFade: d.gpu?.heightFade,
}
: {
time: d.time,
playing: d.playing,
loading: d.loading,
loadedTiles: d.loadedTiles,
imageryCount: d.imageryCount,
frameLoadMs: d.frameLoadMs && Math.round(d.frameLoadMs),
error: d.error,
suspended: d.suspended ?? d.globeHidden ?? null,
};
}
return out;
},
{ WEATHER },
);
/** Enable the layers a state names, wait for first imagery/geometry, then set playback. */
const applyState = async (state) => {
const wantWind = state.includes('wind');
const wantHistory = state.includes('history');
const wantPlaying = state.includes('playing');
const started = Date.now();
await page.evaluate(
async ({ wantWind, wantHistory, WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
const set = async (id, on) => {
const entry = dm.layers.get(id);
if (!entry || Boolean(entry.enabled) === on) return;
await dm.setEnabled(id, on, { origin: 'user' });
};
await set('wind', wantWind);
if (wantWind)
dm.setLayerParams('wind', { overlay: 'speed' }, { origin: 'user' });
for (const id of WEATHER) await set(id, wantHistory);
},
{ wantWind, wantHistory, WEATHER },
);
let ready = true;
await page
.waitForFunction(
({ wantWind, wantHistory, WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
if (wantWind) {
const d = dm.layers.get('wind')?.module?.getDiagnostics?.();
if (!d) return false;
if (d.renderMode === 'gpu-streamlines' && !d.gpu?.ready)
return false;
}
if (wantHistory)
for (const id of WEATHER) {
const d = dm.layers.get(id)?.module?.getDiagnostics?.();
if (!d || d.loading || (!d.time && !d.error)) return false;
}
return true;
},
{ timeout: 90_000, polling: 250 },
{ wantWind, wantHistory, WEATHER },
)
.catch(() => {
ready = false;
});
const readyMs = Date.now() - started;
await page.evaluate(
({ wantPlaying, WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
for (const id of WEATHER) {
const module = dm.layers.get(id)?.module;
if (!module || !dm.layers.get(id)?.enabled) continue;
const playing = Boolean(module.getDiagnostics?.().playing);
if (playing !== wantPlaying)
dm.setLayerParams(id, { play: true }, { origin: 'user' });
}
},
{ wantPlaying, WEATHER },
);
await sleep(2_500);
return { ready, readyMs, layers: await diagnostics() };
};
const shot = (name) =>
page.screenshot({ path: path.join(outDir, `${name}.png`) });
// ── Matched route ────────────────────────────────────────────────────────
if (!flag('--no-route')) {
console.log('\nRoute windows (continuous-render hold):');
for (let run = 1; run <= repeat; run++) {
const order = run % 2 === 1 ? states : [...states].reverse();
for (const state of order) {
await pose('overhead');
const setup = await applyState(state);
console.log(
` setup ${state}: ready=${setup.ready} in ${setup.readyMs}ms ${JSON.stringify(setup.layers)}`,
);
const base = { phase: 'route', state, run, setup };
await fly('close');
record({ ...base, window: 'zoom-in', ...(await measure(6_000, true)) });
record({
...base,
window: 'close-idle',
...(await measure(6_000, true)),
});
await shot(`${state}-r${run}-close`);
await orbit();
record({ ...base, window: 'orbit', ...(await measure(8_000, true)) });
await fly('globe');
record({
...base,
window: 'zoom-out',
...(await measure(6_000, true)),
});
record({
...base,
window: 'globe-idle',
...(await measure(6_000, true)),
});
await shot(`${state}-r${run}-globe`);
console.log(` after ${state}: ${JSON.stringify(await diagnostics())}`);
}
}
}
// ── Idle test (normal governor) ──────────────────────────────────────────
if (!flag('--no-idle')) {
console.log('\nIdle windows (normal governor, fixed 1.2 km pose):');
for (const state of ['off', 'history-paused', 'history-playing', 'wind']) {
await pose('close');
const setup = await applyState(state);
await sleep(4_000);
for (let run = 1; run <= 2; run++)
record({
phase: 'idle',
state,
run,
window: 'fixed-8s',
setup,
...(await measure(8_000, false)),
});
await shot(`idle-${state}`);
}
await applyState('off');
}
writeFileSync(
path.join(outDir, 'results.json'),
JSON.stringify({ label, url, env, results }, null, 2),
);
const lines = [
`# qa-weather-perf · ${label}`,
'',
`url: ${url} · ${env.basemap} · canvas ${env.canvas} @ DPR ${env.dpr} · ${env.ua}`,
'',
'| phase | state | run | window | fps | p50 ms | p95 ms | max ms | long tasks | long ms | dom mut | height ∆ | height min–max |',
'|---|---|---|---|---|---|---|---|---|---|---|---|---|',
...results.map(
(r) =>
`| ${r.phase} | ${r.state} | ${r.run} | ${r.window} | ${r.fps}${r.canvasStable ? '' : ' ⚠'} | ${r.p50} | ${r.p95} | ${r.max} | ${r.longTasks} | ${r.longTaskMs} | ${r.mutations} | ${r.heightChanges} | ${r.heightMin ?? '—'}–${r.heightMax ?? '—'} |`,
),
];
writeFileSync(path.join(outDir, 'summary.md'), lines.join('\n') + '\n');
console.log(`\nWrote ${path.join(outDir, 'results.json')} and summary.md`);
} finally {
await browser.close();
}
scripts/qa-weather-swap.mjs
+203
View File
@@ -0,0 +1,203 @@
/**
* Weather history-swap probe: does the scene keep drawing while a draped
* weather frame is replaced? Runs against a live dev server, turns on one
* observed product at the Earth view, steps Earlier and plays history while
* screenshotting, then measures the black fraction inside the globe disc of
* every capture. Writes <out>/<label>/summary.json.
*
* node scripts/qa-weather-swap.mjs --url http://localhost:4173 --label google
* [--layer weather-satellite] [--product clouds] [--headless]
*/
import {
existsSync,
mkdirSync,
readdirSync,
readFileSync,
writeFileSync,
} from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg('--label', 'swap');
const layerId = arg('--layer', 'weather-satellite');
const product = arg(
'--product',
layerId === 'weather-satellite' ? 'clouds' : null,
);
const outDir = path.resolve(arg('--out', 'qa-shots/weather-swap'), label);
mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
// Globe disc in CSS pixels for the 1400×920 viewport at the Earth view.
const DISC = { cx: 700, cy: 460, r: 300 };
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
),
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--window-size=1400,1000',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const frames = [];
try {
const page = await browser.newPage();
await page.setViewport({ width: 1400, height: 920, deviceScaleFactor: 1 });
page.on('pageerror', (e) => console.log(' [pageerror]', e.message));
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(12_000);
await page.evaluate(
async ({ layerId, product }) => {
const gev = window.__godsEyeView;
const dm = gev.dataManager;
for (const [id, e] of dm.layers)
if (e.enabled) await dm.setEnabled(id, false, { origin: 'user' });
const v = gev.viewer;
const d = Math.PI / 180;
v.camera.setView({
destination: v.scene.ellipsoid.cartographicToCartesian({
longitude: -80 * d,
latitude: 25 * d,
height: 17_368_000,
}),
orientation: { heading: 0, pitch: -90 * d, roll: 0 },
});
await dm.setEnabled(layerId, true, { origin: 'user' });
if (product) dm.setLayerParams(layerId, { product }, { origin: 'user' });
},
{ layerId, product },
);
await page
.waitForFunction(
({ layerId, product }) => {
const d = window.__godsEyeView.dataManager.layers
.get(layerId)
?.module?.getDiagnostics?.();
return d && !d.loading && d.time && (!product || d.product === product);
},
{ timeout: 90_000 },
{ layerId, product },
)
.catch(() => console.log(' !! first frame not ready'));
await sleep(3000);
const diag = () =>
page.evaluate((layerId) => {
const d = window.__godsEyeView.dataManager.layers
.get(layerId)
.module.getDiagnostics();
return {
time: d.time,
loading: d.loading,
pending: d.pendingTiles,
loaded: d.loadedTiles,
imagery: d.imageryCount,
mosaic: d.mosaic ?? null,
};
}, layerId);
const capture = async (name) => {
const file = `${name}.png`;
await page.screenshot({ path: path.join(outDir, file) });
frames.push({ file, ...(await diag()) });
};
await capture('00-latest');
const step = (params) =>
page.evaluate(
({ layerId, params }) =>
window.__godsEyeView.dataManager.setLayerParams(layerId, params, {
origin: 'user',
}),
{ layerId, params },
);
await step({ step: -1 });
for (let i = 1; i <= 15; i++) {
await sleep(400);
await capture(`earlier-${String(i).padStart(2, '0')}`);
}
await step({ play: true });
for (let i = 1; i <= 20; i++) {
await sleep(500);
if (i % 2 === 0) await capture(`play-${String(i).padStart(2, '0')}`);
}
await step({ play: true });
// Black fraction inside the globe disc, measured in a blank page.
const meter = await browser.newPage();
for (const frame of frames) {
const b64 = readFileSync(path.join(outDir, frame.file)).toString('base64');
const black = await meter.evaluate(
async ({ b64, DISC }) => {
const img = new Image();
img.src = `data:image/png;base64,${b64}`;
await img.decode();
const c = document.createElement('canvas');
c.width = img.naturalWidth;
c.height = img.naturalHeight;
const ctx = c.getContext('2d');
ctx.drawImage(img, 0, 0);
const s = img.naturalWidth / 1400;
const d = ctx.getImageData(0, 0, c.width, c.height).data;
const rr = (DISC.r * s) ** 2;
let n = 0;
let dark = 0;
for (let y = 0; y < c.height; y += 2)
for (let x = 0; x < c.width; x += 2) {
if ((x - DISC.cx * s) ** 2 + (y - DISC.cy * s) ** 2 > rr) continue;
n++;
const i = (y * c.width + x) * 4;
if (d[i] < 18 && d[i + 1] < 18 && d[i + 2] < 18) dark++;
}
return dark / n;
},
{ b64, DISC },
);
frame.black = Number(black.toFixed(4));
console.log(
`${frame.file}\t${(black * 100).toFixed(1)}%\t${frame.time}\tpending=${frame.pending}`,
);
}
const blackFrames = frames.filter((f) => f.black > 0.15).length;
const distinctTimes = new Set(frames.map((f) => f.time).filter(Boolean)).size;
const summary = {
url: rawUrl,
label,
layerId,
product,
frames: frames.length,
blackFrames,
maxBlack: Math.max(...frames.map((f) => f.black)),
distinctTimes,
capturedAt: new Date().toISOString(),
};
writeFileSync(
path.join(outDir, 'summary.json'),
JSON.stringify({ summary, frames }, null, 2),
);
console.log(
`frames=${frames.length} black(>15%)=${blackFrames} max=${(summary.maxBlack * 100).toFixed(1)}% distinctTimes=${distinctTimes}`,
);
} finally {
await browser.close();
}
scripts/qa-weather-teardown.mjs
+341
View File
@@ -0,0 +1,341 @@
/**
* Weather teardown and cross-layer regression probe.
*
* Against a running dev server: records a resource baseline, turns every
* weather layer on and exercises it (history playback, a wind reading, a
* storm selection), turns them all off, and asserts the scene is back to the
* baseline (imagery layers on the globe and the tileset, primitives, ground
* primitives, data sources, entities, camera/scene listeners, the weather
* panel, the wind reading marker). Then it enables a set of non-weather
* layers one at a time and samples the render rate, long tasks and console
* errors for each, and repeats the same non-weather sequence in a control
* page that never had weather on. Writes <out>/<label>/results.json.
*
* node scripts/qa-weather-teardown.mjs --url http://localhost:4173 --label google
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg('--label', 'teardown');
const outDir = path.resolve(arg('--out', 'qa-shots/weather-teardown'), label);
mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const WEATHER = [
'wind',
'weather-radar',
'weather-satellite',
'weather-lightning',
'weather-cyclones',
];
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
),
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--window-size=1400,1000',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const boot = async () => {
const page = await browser.newPage();
await page.setViewport({ width: 1400, height: 920, deviceScaleFactor: 1 });
const errors = [];
page.on('pageerror', (e) =>
errors.push(`pageerror: ${e.message}`.slice(0, 200)),
);
page.on('console', (m) => {
if (m.type() === 'error') errors.push(`console: ${m.text()}`.slice(0, 200));
});
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(10_000);
await page.evaluate(async () => {
const gev = window.__godsEyeView;
const dm = gev.dataManager;
for (const [id, e] of dm.layers)
if (e.enabled) await dm.setEnabled(id, false, { origin: 'user' });
const v = gev.viewer;
const d = Math.PI / 180;
v.camera.setView({
destination: v.scene.ellipsoid.cartographicToCartesian({
longitude: -95 * d,
latitude: 32 * d,
height: 4_500_000,
}),
orientation: { heading: 0, pitch: -90 * d, roll: 0 },
});
});
await sleep(3000);
return { page, errors };
};
const snapshot = (page) =>
page.evaluate(() => {
const gev = window.__godsEyeView;
const v = gev.viewer;
const s = v.scene;
const t = gev.tileset;
const dm = gev.dataManager;
const weather = {};
for (const id of [
'weather-radar',
'weather-satellite',
'weather-lightning',
]) {
const d = dm.layers.get(id)?.module?.getDiagnostics?.();
if (d)
weather[id] = {
imagery: d.imageryCount,
mosaics: d.cache?.mosaics ?? 0,
prefetching: !!d.cache?.prefetching,
};
}
const wind = dm.layers.get('wind')?.module?.getDiagnostics?.() || {};
return {
globeImagery: v.imageryLayers.length,
tilesetImagery: t?.imageryLayers?.length ?? null,
primitives: s.primitives.length,
groundPrimitives: s.groundPrimitives.length,
dataSources: v.dataSources.length,
entities: v.entities.values.length,
listeners: {
postRender: s.postRender.numberOfListeners,
preRender: s.preRender.numberOfListeners,
postUpdate: s.postUpdate.numberOfListeners,
moveEnd: v.camera.moveEnd.numberOfListeners,
moveStart: v.camera.moveStart.numberOfListeners,
changed: v.camera.changed.numberOfListeners,
tick: v.clock.onTick.numberOfListeners,
},
dom: {
weatherPanelHidden:
document.getElementById('weather-panel')?.hidden ?? null,
weatherCards: document.querySelectorAll('.weather-card').length,
windReading: !!document.querySelector(
'.gev-wind-reading:not([hidden])',
),
},
weather,
windCells: wind.gpu?.pathCount ?? wind.paths ?? null,
enabled: [...dm.layers].filter(([, e]) => e.enabled).map(([id]) => id),
};
});
const sample = async (page, seconds) => {
await page.evaluate(() => {
window.__qaRenders = 0;
window.__qaLong = { count: 0, ms: 0 };
const s = window.__godsEyeView.viewer.scene;
window.__qaOff = s.postRender.addEventListener(() => window.__qaRenders++);
window.__qaObs = new PerformanceObserver((list) => {
for (const e of list.getEntries()) {
window.__qaLong.count++;
window.__qaLong.ms += e.duration;
}
});
window.__qaObs.observe({ entryTypes: ['longtask'] });
window.__qaRaf = () => {
s.requestRender();
window.__qaRafId = requestAnimationFrame(window.__qaRaf);
};
window.__qaRaf();
});
await sleep(seconds * 1000);
return page.evaluate((seconds) => {
cancelAnimationFrame(window.__qaRafId);
window.__qaOff();
window.__qaObs.disconnect();
return {
fps: +(window.__qaRenders / seconds).toFixed(1),
longTasks: window.__qaLong.count,
longMs: Math.round(window.__qaLong.ms),
};
}, seconds);
};
const setLayer = (page, id, on) =>
page.evaluate(
async ({ id, on }) => {
const dm = window.__godsEyeView.dataManager;
const e = dm.layers.get(id);
if (e && Boolean(e.enabled) !== on)
await dm.setEnabled(id, on, { origin: 'user' });
return !!e;
},
{ id, on },
);
const diff = (a, b) => {
const out = {};
const walk = (x, y, prefix) => {
for (const key of Object.keys(x)) {
const p = prefix ? `${prefix}.${key}` : key;
if (x[key] && typeof x[key] === 'object' && !Array.isArray(x[key]))
walk(x[key], y?.[key] || {}, p);
else if (JSON.stringify(x[key]) !== JSON.stringify(y?.[key]))
out[p] = [x[key], y?.[key]];
}
};
walk(a, b, '');
return out;
};
const results = { url: rawUrl, label, startedAt: new Date().toISOString() };
try {
// ── Run A: weather on → exercised → off → baseline check → other layers ──
const a = await boot();
const others = await a.page.evaluate(() =>
[...window.__godsEyeView.dataManager.layers.keys()].filter((id) =>
/^flights$|^satellites$|vessel|firms|earthquake|cables|^military$/.test(
id,
),
),
);
console.log('other layers under test:', others.join(', '));
const baseline = await snapshot(a.page);
console.log('baseline', JSON.stringify(baseline));
for (const id of WEATHER) await setLayer(a.page, id, true);
await a.page
.waitForFunction(
() =>
['weather-radar', 'weather-satellite', 'weather-lightning'].every(
(id) => {
const d = window.__godsEyeView.dataManager.layers
.get(id)
?.module?.getDiagnostics?.();
return d && !d.loading && d.time;
},
),
{ timeout: 120_000 },
)
.catch(() => console.log(' !! weather not ready'));
await sleep(2000);
// exercise: history playback, wind reading, storm selection, product switch
await a.page.evaluate(() => {
const dm = window.__godsEyeView.dataManager;
dm.setLayerParams('weather-radar', { play: true }, { origin: 'user' });
dm.setLayerParams('wind', { overlay: 'speed' }, { origin: 'user' });
dm.setLayerParams('wind', { inspect: true }, { origin: 'user' });
dm.setLayerParams(
'weather-satellite',
{ product: 'clouds' },
{ origin: 'user' },
);
});
await sleep(8000);
await a.page.evaluate(() => {
const dm = window.__godsEyeView.dataManager;
const storms =
dm.layers.get('weather-cyclones')?.module?.getRowControls?.().list
?.items || [];
if (storms[1])
dm.setLayerParams('weather-cyclones', storms[1].params, {
origin: 'user',
});
dm.setLayerParams('weather-radar', { play: true }, { origin: 'user' });
});
await sleep(4000);
const weatherOn = await snapshot(a.page);
console.log('weather on', JSON.stringify(weatherOn));
results.weatherOnSample = await sample(a.page, 5);
for (const id of WEATHER) await setLayer(a.page, id, false);
await sleep(4000);
const afterOff = await snapshot(a.page);
const leak = diff(
{ ...baseline, enabled: undefined, weather: undefined },
{ ...afterOff, enabled: undefined, weather: undefined },
);
results.baseline = baseline;
results.afterOff = afterOff;
results.leak = leak;
console.log('after off', JSON.stringify(afterOff));
console.log(
Object.keys(leak).length
? `!! DIFFERENCES after teardown: ${JSON.stringify(leak)}`
: 'teardown: scene back to baseline',
);
results.afterWeather = [];
for (const id of others) {
const ok = await setLayer(a.page, id, true);
await sleep(9000);
const m = await sample(a.page, 5);
const snap = await snapshot(a.page);
results.afterWeather.push({
id,
ok,
...m,
primitives: snap.primitives,
dataSources: snap.dataSources,
});
console.log(
`after weather + ${id}: fps=${m.fps} long=${m.longTasks}/${m.longMs}ms`,
);
}
results.errorsA = a.errors;
await a.page.screenshot({ path: path.join(outDir, 'a-others-on.png') });
await a.page.close();
// ── Run B: control (same others, never had weather) ──
const b = await boot();
results.control = [];
for (const id of others) {
const ok = await setLayer(b.page, id, true);
await sleep(9000);
const m = await sample(b.page, 5);
const snap = await snapshot(b.page);
results.control.push({
id,
ok,
...m,
primitives: snap.primitives,
dataSources: snap.dataSources,
});
console.log(
`control + ${id}: fps=${m.fps} long=${m.longTasks}/${m.longMs}ms`,
);
}
results.errorsB = b.errors;
await b.page.screenshot({ path: path.join(outDir, 'b-control.png') });
await b.page.close();
console.log(
'\nerrors (weather run):',
results.errorsA.length ? results.errorsA : 'none',
);
console.log(
'errors (control run):',
results.errorsB.length ? results.errorsB : 'none',
);
writeFileSync(
path.join(outDir, 'results.json'),
JSON.stringify(results, null, 2),
);
console.log(`wrote ${path.join(outDir, 'results.json')}`);
} finally {
await browser.close();
}
scripts/qa-wind-canvas.mjs
+44
View File
@@ -0,0 +1,44 @@
/** Real-canvas wind regression. Run against this candidate's dev server. */
import assert from 'node:assert/strict';
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch({ executablePath: process.env.PUPPETEER_EXECUTABLE_PATH || await puppeteer.executablePath(), headless: false, args: ['--no-sandbox', '--disable-dev-shm-usage'] });
try {
const page = await browser.newPage();
await page.goto(`${process.env.QA_BASE_URL || 'http://localhost:4173'}/`, { waitUntil: 'domcontentloaded' });
const result = await page.evaluate(async () => {
const { createWindRendering } = await import('/src/layers/wind/rendering.js');
const raf = window.requestAnimationFrame; const caf = window.cancelAnimationFrame;
const callbacks = new Map(); let sequence = 0;
window.requestAnimationFrame = callback => { callbacks.set(++sequence, callback); return sequence; };
window.cancelAnimationFrame = id => callbacks.delete(id);
const container = document.createElement('div');
container.style.cssText = 'position:fixed;left:0;top:0;width:800px;height:600px;z-index:9999;background:#102030';
document.body.appendChild(container);
const sceneCanvas = { clientWidth: 800, clientHeight: 600 };
const viewer = { scene: { canvas: sceneCanvas, camera: {} }, isDestroyed: () => false };
const cesium = { Cartesian3: { fromDegrees: (lon,lat) => ({ lon,lat }) }, SceneTransforms: { worldToWindowCoordinates: (scene, point) => ({ x: 400 + point.lon, y: 300 + point.lat }) } };
const rendering = createWindRendering({ cesium, container, getViewer: () => viewer });
try {
rendering.attach(); rendering.start();
const idleCallbacks = callbacks.size;
rendering.setField({ grid: { nx:1,ny:1,lo1:0,la1:90,dx:360,dy:180 }, u: new Float32Array([10]), v: new Float32Array([0]) });
const canvas = container.querySelector('canvas'); const context = canvas.getContext('2d');
const tick = time => { const [id, callback] = callbacks.entries().next().value; callbacks.delete(id); callback(time); };
tick(16); // First camera signature clears the surface.
context.fillStyle = '#ff0000'; context.fillRect(10,10,5,5);
tick(32);
const alpha = context.getImageData(12,12,1,1).data[3];
const count = rendering.getParticleCount();
sceneCanvas.clientWidth = 320; sceneCanvas.clientHeight = 240;
tick(48);
const resizedCount = rendering.getParticleCount();
rendering.stop(); rendering.clear();
return { idleCallbacks, alpha, count, resizedCount, pending: callbacks.size, afterClear: context.getImageData(12,12,1,1).data[3] };
} finally { rendering.destroy(); container.remove(); window.requestAnimationFrame = raf; window.cancelAnimationFrame = caf; }
});
assert.equal(result.idleCallbacks, 0);
assert.ok(result.alpha > 200, 'unchanged canvas dimensions preserve existing trail pixels');
assert.ok(result.resizedCount < result.count);
assert.equal(result.pending, 0); assert.equal(result.afterClear, 0);
console.log(JSON.stringify(result));
} finally { await browser.close(); }
scripts/track-regression.mjs
+44
View File
@@ -88,6 +88,7 @@
* --source-base <path> Browser module prefix (default /src)
* --url <url> App URL (default http://localhost:4173)
* --headful Show the browser (debugging)
* --offline-imagery Use bundled texture for known Esri/OSM tile images only
* --keep-open Leave the browser open after the run (debugging)
*/
@@ -110,6 +111,7 @@ const SOURCE_BASE = getOpt('--source-base', '/src').replace(/\/$/, '');
const APP_URL = getOpt('--url', 'http://localhost:4173');
const APP_ORIGIN = new URL(APP_URL).origin;
const HEADFUL = getFlag('--headful');
const OFFLINE_IMAGERY = getFlag('--offline-imagery');
const KEEP_OPEN = getFlag('--keep-open');
const CHROME_EXECUTABLE_CANDIDATES = [
@@ -201,6 +203,15 @@ async function main() {
console.log(` App URL : ${APP_URL}`);
console.log(` Mode : ${HEADFUL ? 'headful' : 'headless'}\n`);
// Tracking does not assert imagery geography. This optional fixture leaves
// provider metadata, terrain, models and every tracking assertion untouched.
// Read before launching so a missing bundled asset fails explicitly.
const offlineImageryTile = OFFLINE_IMAGERY
? fs.readFileSync(new URL('../node_modules/cesium/Build/Cesium/Assets/Textures/NaturalEarthII/0/0/0.jpg', import.meta.url))
: null;
let offlineImageryRequests = 0;
if (OFFLINE_IMAGERY) console.log(' Imagery : bundled NaturalEarthII fixture (tile images only)');
// Verify the dev server is up before launching a browser.
try {
const res = await fetch(APP_URL, { method: 'GET' });
@@ -256,6 +267,25 @@ async function main() {
await page.setRequestInterception(true);
page.on('request', (request) => {
const url = new URL(request.url());
// Reuse the one existing interceptor; do not install competing handlers.
// Exact HTTPS origins and image paths only: metadata, feature queries,
// other providers, terrain and local/model resources continue normally.
const knownImageryTile = request.method() === 'GET' && (
(url.origin === 'https://services.arcgisonline.com'
&& /^\/ArcGIS\/rest\/services\/World_Imagery\/MapServer\/tile\/\d+\/\d+\/\d+$/.test(url.pathname))
|| (url.origin === 'https://tile.openstreetmap.org'
&& /^\/\d+\/\d+\/\d+\.png$/.test(url.pathname))
);
if (OFFLINE_IMAGERY && knownImageryTile) {
offlineImageryRequests++;
request.respond({
status: 200,
contentType: 'image/jpeg',
headers: { 'Access-Control-Allow-Origin': '*' },
body: offlineImageryTile,
});
return;
}
if (url.origin === APP_ORIGIN && url.pathname === '/api/openai/hud-summary') {
request.respond({
status: 200,
@@ -365,6 +395,19 @@ async function main() {
lastMessageAt: null,
}));
}
// Contacts also enables mapped installations. This tracking harness
// does not test installation acquisition; a valid empty viewport keeps
// public Overpass outages out of aircraft/satellite tracking results.
// Preserve the real response shape and all console-error assertions.
if (isAppRequest && url.pathname === '/api/military-installations') {
return Promise.resolve(jsonResponse({
elements: [],
saturated: false,
elementCap: 700,
retrievedAt: new Date().toISOString(),
status: 'ready',
}));
}
// CelesTrak TLE groups. Served from a fixed two-satellite catalog so
// the satellite assertions are deterministic and the run never depends
// on an upstream that rate-limits (it was answering 403 the night this
@@ -4082,6 +4125,7 @@ async function main() {
finishAndExit();
} finally {
if (OFFLINE_IMAGERY) console.log(` Bundled imagery tiles served: ${offlineImageryRequests}`);
if (!KEEP_OPEN) {
await browser.close();
} else {
server/providers/aircraft/tracks.js
+6 -2
View File
@@ -45,7 +45,9 @@ export function trackBackfillProxies() {
RESPONSE_CAP_BYTES,
);
let body;
let status = upstream.status;
if (tooLarge) {
status = 502;
body = JSON.stringify({ error: 'Upstream track response too large' });
} else if (!upstream.ok) {
// Sanitize upstream error surface; status code is signal enough
@@ -53,8 +55,10 @@ export function trackBackfillProxies() {
} else {
body = text;
}
cachePut(key, { at: Date.now(), status: upstream.status, body });
res.statusCode = upstream.status;
// Cache the 502 like any upstream error: /tracks/all costs OpenSky
// credits, so a retry inside the window must not re-download the body.
cachePut(key, { at: Date.now(), status, body });
res.statusCode = status;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'no-store');
res.end(body);
server/providers/cctv.js
+121 -3
View File
@@ -17,6 +17,7 @@ import {
CCTV_MAX_SOURCES_CEILING,
} from './cctv/constants.js';
import { sanitizeCctvRangeHeader } from './cctv/range.js';
import { createHlsPuller } from './cctv/stream.js';
import { googleServerApiKey } from './places/google-key.js';
export { CCTV_FRAME_FETCH_TIMEOUT_MS, fetchCctvImageFromUpstream };
/**
@@ -40,6 +41,8 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
* CCTV_MAX_SOURCES ceiling so health/status observability is never evicted
* for any catalog the proxy can actually serve. */
const HEALTH_MAX_ENTRIES = CCTV_MAX_SOURCES_CEILING;
/** Live HLS strategies (see ./cctv/stream.js). Shared across dev and preview. */
const puller = createHlsPuller();
/** Update the health entry for a camera, evicting the oldest entry if at capacity. */
const setHealth = (cameraId, patch) => {
@@ -127,6 +130,9 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
};
const installMiddleware = (server) => {
server.httpServer?.on('close', () => {
puller.shutdown();
});
server.middlewares.use('/api/cctv', async (req, res) => {
try {
const sources = await getCctvSources();
@@ -194,12 +200,124 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
}
if (url.pathname.startsWith('/media/')) {
const cameraId =
decodeURIComponent(url.pathname.replace('/media/', '').trim()) ||
'camera';
const match = /^\/media\/([^/]+)(?:\/(seg_(\d+)\.ts))?$/.exec(
url.pathname,
);
if (!match) {
res.writeHead(404);
res.end();
return;
}
const cameraId = decodeURIComponent(match[1]);
const source = sourceById.get(cameraId);
const mediaUrl = source?.url || '';
const feedType = normalizeFeedType(source?.feedType || 'image');
const leaseId = url.searchParams.get('lease');
if (feedType === 'hls' && !/^[a-f0-9-]{36}$/i.test(leaseId || '')) {
res.writeHead(400);
res.end();
return;
}
if (req.method === 'DELETE') {
if (leaseId) puller.release(cameraId, leaseId);
res.writeHead(204);
res.end();
return;
}
if (req.method !== 'GET') {
res.writeHead(405);
res.end();
return;
}
if (feedType === 'hls') {
if (
!/^https?:\/\//i.test(mediaUrl) ||
!/\.m3u8(?:\?|$)/i.test(mediaUrl)
) {
res.writeHead(503, { 'Content-Type': 'application/json' });
res.end(
JSON.stringify({
error:
'This stream requires an unsupported transport; use the frame fallback',
}),
);
return;
}
if (match[2]) {
const body = puller.getSegment(
cameraId,
url.searchParams.get('session'),
Number(match[3]),
leaseId,
);
res.writeHead(body ? 200 : 404, {
'Content-Type': 'video/mp2t',
'Cache-Control': 'no-store',
});
res.end(body || undefined);
return;
}
const downstream = watchDownstreamClose(res);
let entry;
const cancelPending = () => {
if (entry) puller.release(cameraId, leaseId);
};
try {
entry = await puller.ensure(cameraId, mediaUrl, leaseId);
if (downstream.closed) {
cancelPending();
return;
}
downstream.signal.addEventListener('abort', cancelPending, {
once: true,
});
if (!(await puller.waitReady(entry, downstream.signal)))
throw new Error('Stream unavailable');
const playlist = await puller.buildPlaylist(
entry,
cameraId,
leaseId,
);
if (downstream.closed) return;
if (!playlist) throw new Error('Stream unavailable');
setHealth(cameraId, {
status: 'ok',
sourceKind: 'live',
label: source?.provider || 'Configured source',
message: 'Live HLS connected',
});
res.writeHead(200, {
'Content-Type': 'application/vnd.apple.mpegurl',
'Cache-Control': 'no-store',
'X-CCTV-Source': 'hls-pull',
'X-CCTV-Session': entry.token,
});
res.end(playlist);
} catch {
setHealth(cameraId, {
status: 'degraded',
sourceKind: 'fallback',
label: source?.provider || 'Configured source',
message: 'Live HLS unavailable',
});
if (!downstream.closed) {
res.writeHead(503, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
'Retry-After': '2',
});
res.end(JSON.stringify({ error: 'Live stream unavailable' }));
}
} finally {
downstream.signal.removeEventListener('abort', cancelPending);
}
return;
}
if (match[2] || req.method !== 'GET') {
res.writeHead(404);
res.end();
return;
}
if (!mediaUrl || !/^https?:\/\//i.test(mediaUrl)) {
setHealth(cameraId, {
server/providers/cctv/catalog.js
+6
View File
@@ -17,6 +17,7 @@ import {
loadWarendorfSourcesFromCatalog,
loadNswSourcesFromOpenData,
loadCalgarySourcesFromOpenData,
loadDelDOTSourcesFromOpenData,
} from './sources.js';
/** Env kill switch: unset or anything but "0" means enabled. */
@@ -86,6 +87,11 @@ const LIVE_PACKS = [
enabled: () => envEnabled('CCTV_CALGARY_ENABLED'),
load: loadCalgarySourcesFromOpenData,
},
{
name: 'deldot',
enabled: () => envEnabled('CCTV_DELDOT_ENABLED'),
load: loadDelDOTSourcesFromOpenData,
},
];
/**
* Load CCTV sources from a local JSON file (CCTV_SOURCES_FILE env or default).
server/providers/cctv/constants.js
+13
View File
@@ -236,6 +236,14 @@ export const CALGARY_DOWNTOWN = { lat: 51.0461, lon: -114.0626 };
* body cannot be buffered without limit. */
export const CALGARY_MAX_CATALOG_BYTES = 4 * 1024 * 1024;
/** DelDOT CCTV: one keyless statewide JSON catalog; live video via RTMP-over-HTTP (rtmpt:80). */
export const DELDOT_CCTV_URL = 'https://tmc.deldot.gov/json/videocamera.json';
export const DEFAULT_DELDOT_MAX_SOURCES = 300;
export const DELDOT_ANCHORS = [
{ lat: 39.7459, lon: -75.5466 }, // Wilmington (New Castle)
{ lat: 39.1582, lon: -75.5244 }, // Dover (Kent)
{ lat: 38.6906, lon: -75.3877 }, // Georgetown (Sussex)
];
/** Camera CATALOGS change rarely; 15 min keeps multi-megabyte upstream list refetches (Austin rows.json + 4 Caltrans districts + TfL + Ontario 511) infrequent. Frames are fetched per-request and are unaffected. */
export const CCTV_SOURCE_CACHE_MS = 15 * 60 * 1000;
/** Per-provider catalog-fetch timeout. Bounds the worst-case refresh so one
@@ -253,5 +261,10 @@ export const CCTV_FRAME_MAX_BODY_BYTES = 16 * 1024 * 1024;
/** Deadline for upstream response headers; live bodies keep streaming afterward. */
export const CCTV_MEDIA_FETCH_TIMEOUT_MS = 15 * 1000;
/** Silence a live body may carry before the relay gives up on it. Twice the
* header deadline, because a camera that is merely slow between frames is far
* more common than one that has died mid-stream, and a viewer would rather
* wait than be dropped. */
export const CCTV_MEDIA_IDLE_TIMEOUT_MS = 30 * 1000;
/** Declared size ceiling for fixed media responses. */
export const CCTV_MEDIA_MAX_BODY_BYTES = 64 * 1024 * 1024;
server/providers/cctv/media.js
+45 -1
View File
@@ -4,6 +4,7 @@ import {
CCTV_FRAME_FETCH_TIMEOUT_MS,
CCTV_FRAME_MAX_BODY_BYTES,
CCTV_MEDIA_FETCH_TIMEOUT_MS,
CCTV_MEDIA_IDLE_TIMEOUT_MS,
CCTV_MEDIA_MAX_BODY_BYTES,
NSW_IMAGE_ORIGIN,
NSW_IMAGE_USER_AGENT,
@@ -103,11 +104,17 @@ export function toReadable(body) {
* @param {Response} upstream - fetch() Response object.
* @param {object} [opts]
* @param {string} [opts.sourceHeader='upstream'] - Value for X-CCTV-Source header.
* @param {number} [opts.idleTimeoutMs=CCTV_MEDIA_IDLE_TIMEOUT_MS] - Silence
* allowed between upstream chunks before the stream is released. Injectable
* only to keep the deadline unit-testable.
*/
export async function proxyMediaResponse(
res,
upstream,
{ sourceHeader = 'upstream' } = {},
{
sourceHeader = 'upstream',
idleTimeoutMs = CCTV_MEDIA_IDLE_TIMEOUT_MS,
} = {},
) {
const contentType =
upstream.headers.get('content-type') || 'application/octet-stream';
@@ -153,7 +160,20 @@ export async function proxyMediaResponse(
return;
}
// The header deadline only covers the wait for a response line. Past that an
// upstream can hold the connection open and send nothing at all, and the
// relay would wait on it for as long as the camera host cared to. The
// deadline below measures the gap between upstream chunks rather than the
// life of the stream, so a feed that keeps delivering keeps its connection.
let idleTimer = null;
const clearIdleDeadline = () => {
if (!idleTimer) return;
clearTimeout(idleTimer);
idleTimer = null;
};
stream.on('error', () => {
clearIdleDeadline();
if (!res.writableEnded) res.end();
});
@@ -164,6 +184,7 @@ export async function proxyMediaResponse(
const releaseUpstream = () => {
if (released) return;
released = true;
clearIdleDeadline();
stream.unpipe(res);
// Destroying the Node stream cancels the web body it wraps; the direct
// cancel covers a body that was never wrapped, and rejects harmlessly when
@@ -176,14 +197,37 @@ export async function proxyMediaResponse(
/* already closed */
}
};
const armIdleDeadline = () => {
clearIdleDeadline();
idleTimer = setTimeout(onIdleDeadline, idleTimeoutMs);
idleTimer.unref?.();
};
const onIdleDeadline = () => {
// A viewer who cannot keep up pauses the pipe, and no upstream bytes arrive
// while it is paused. That is a slow client rather than a dead camera, so
// it gets the deadline again instead of a teardown.
if (res.writableNeedDrain) {
armIdleDeadline();
return;
}
releaseUpstream();
if (!res.writableEnded) res.end();
};
res.once('close', () => {
clearIdleDeadline();
if (!res.writableEnded) releaseUpstream();
});
res.once('error', releaseUpstream);
stream.once('end', () => {
clearIdleDeadline();
released = true;
});
armIdleDeadline();
stream.pipe(res);
// Attached after the pipe because a data listener resumes the stream, and
// flowing before the destination is attached would spill chunks nobody
// forwards. Only bytes from upstream renew the deadline.
stream.on('data', armIdleDeadline);
}
/**
server/providers/cctv/sources.js
+107
View File
@@ -56,6 +56,9 @@ import {
CALGARY_DOWNTOWN,
CALGARY_MAX_CATALOG_BYTES,
CCTV_SOURCE_FETCH_TIMEOUT_MS,
DELDOT_CCTV_URL,
DEFAULT_DELDOT_MAX_SOURCES,
DELDOT_ANCHORS,
} from './constants.js';
import {
toFiniteNumber,
@@ -1594,3 +1597,107 @@ export async function loadCalgarySourcesFromOpenData() {
return [];
}
}
export async function loadDelDOTSourcesFromOpenData() {
try {
const resp = await fetch(DELDOT_CCTV_URL, {
headers: { Accept: 'application/json' },
signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
redirect: 'error',
});
if (!resp.ok) {
console.warn('[CCTV] DelDOT source download failed:', resp.status);
return [];
}
const payload = await readResponseJsonCapped(resp, 2 * 1024 * 1024);
const rows = Array.isArray(payload?.videoCameras)
? payload.videoCameras
: [];
if (!rows.length) return [];
const cameras = [];
for (const row of rows) {
if (String(row?.status).toLowerCase() !== 'active') continue;
const lat = toFiniteNumber(row?.lat);
const lon = toFiniteNumber(row?.lon);
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
// Delaware bounding box — a bad upstream coord can't place a camera out of state.
if (lat < 38.4 || lat > 39.9 || lon < -75.85 || lon > -75.0) continue;
// Prefer the catalog's HTTPS HLS transport; no subprocess is required.
let stream;
try {
stream = new URL(String(row?.urls?.m3u8s || ''));
} catch {
continue;
}
if (
stream.origin !== 'https://video.deldot.gov' ||
stream.username ||
stream.password ||
!/^\/live\/[A-Za-z0-9_.-]+\/playlist\.m3u8$/.test(stream.pathname)
)
continue;
const streamUrl = stream.href;
const id = String(row?.id || '').trim();
if (!/^[A-Za-z0-9_-]{1,80}$/.test(id)) continue;
const cameraId = `deldot-${id.toLowerCase()}`;
const title = String(row?.title || '').trim();
// Title carries an unambiguous travel token for ~1 in 5 cameras
// ("US 13 SB @ ..."). directionToHeading (allowBare=false) matches only
// NB/SB/EB/WB, never a bare "W"/"N" street prefix — "W NORTH ST" yields
// no heading and falls back.
const heading = directionToHeading(title);
const hasHeading = Number.isFinite(heading);
cameras.push({
id: cameraId,
name: title || `DelDOT ${id}`,
city: String(row?.county ? `${row.county} County` : 'Delaware'),
cityId: `deldot-${String(row?.county || 'de')
.toLowerCase()
.replace(/\s+/g, '-')}`,
provider: 'DelDOT',
lat,
lon,
headingDeg: hasHeading ? heading : fallbackHeadingFromId(cameraId),
headingConfidence: hasHeading ? 'high' : 'low',
// Fabricated RAW PRIOR poses (same personalities as Austin/Caltrans);
// the client ground-snap + manual calibration own the truth.
pitchDeg: hasHeading ? -24 : -18,
fovDeg: hasHeading ? 56 : 44,
rangeM: hasHeading ? 210 : 145,
mountHeightM: hasHeading ? 10 : 8,
groundElevationM: 10, // Estimated prior; client ground resolution owns placement.
feedType: 'hls',
url: streamUrl,
snapshotUrl: '',
sourceKind: 'deldot-open-data',
license: 'Public DelDOT traffic camera',
});
}
const unique = Array.from(
new Map(cameras.map((camera) => [camera.id, camera])).values(),
);
const maxRaw = Number(
process.env.CCTV_DELDOT_MAX_SOURCES || DEFAULT_DELDOT_MAX_SOURCES,
);
const maxCount = Number.isFinite(maxRaw)
? Math.max(8, Math.min(400, Math.floor(maxRaw)))
: DEFAULT_DELDOT_MAX_SOURCES;
const prioritized = prioritizeSources(unique, maxCount, DELDOT_ANCHORS);
console.log(
`[CCTV] Loaded DelDOT camera sources: ${unique.length} Active (using nearest ${prioritized.length})`,
);
return prioritized;
} catch (error) {
console.warn(
'[CCTV] DelDOT source download error:',
error?.message || error,
);
return [];
}
}
server/providers/cctv/stream.js
+348
View File
@@ -0,0 +1,348 @@
/** Bounded, demand-owned HLS sessions. No subprocess, disk cache or idle sweep. */
import { randomUUID } from 'node:crypto';
export const HLS_LIMITS = Object.freeze({
sessions: 2,
leasesPerSession: 8,
playlistBytes: 256 * 1024,
segmentBytes: 4 * 1024 * 1024,
sessionBytes: 24 * 1024 * 1024,
segments: 12,
pollMs: 2000,
idleMs: 15000,
timeoutMs: 10000,
readyMs: 12000,
});
/** Refuse redirects; count actual streamed bytes, including chunked responses. */
export async function fetchHlsBytes(
url,
{
fetchImpl = fetch,
signal,
maxBytes,
timeoutMs = HLS_LIMITS.timeoutMs,
} = {},
) {
const controller = new AbortController();
const abort = () => controller.abort();
if (signal?.aborted) controller.abort();
else signal?.addEventListener('abort', abort, { once: true });
const timer = setTimeout(abort, timeoutMs);
let reader;
let response;
try {
controller.signal.throwIfAborted();
response = await fetchImpl(url, {
redirect: 'error',
signal: controller.signal,
headers: { 'User-Agent': 'gods-eye-view-cctv-proxy/1.0' },
});
if (
!response.ok ||
response.redirected ||
(response.url && new URL(response.url).href !== new URL(url).href)
)
throw new Error('HLS upstream refused');
if (Number(response.headers.get('content-length')) > maxBytes)
throw new Error('HLS response too large');
reader = response.body?.getReader?.();
if (!reader) throw new Error('HLS response is not streamable');
const chunks = [];
let length = 0;
for (;;) {
controller.signal.throwIfAborted();
const { done, value } = await reader.read();
controller.signal.throwIfAborted();
if (done) break;
length += value.byteLength;
if (length > maxBytes) throw new Error('HLS response too large');
chunks.push(Buffer.from(value));
}
return Buffer.concat(chunks, length);
} finally {
controller.abort();
if (reader) await reader.cancel().catch(() => {});
else await response?.body?.cancel().catch(() => {});
reader?.releaseLock();
clearTimeout(timer);
signal?.removeEventListener('abort', abort);
}
}
export function sameOriginHlsUrl(value, base) {
const url = new URL(value, base);
if (
!['https:', 'http:'].includes(url.protocol) ||
url.origin !== new URL(base).origin ||
url.username ||
url.password
)
throw new Error('HLS reference escapes registered origin');
return url.href;
}
/** Only clear MPEG-TS media playlists: encrypted, fMP4 and byte ranges fail closed. */
export function parseHlsMedia(text, base, limit = HLS_LIMITS.segments) {
if (
!text.startsWith('#EXTM3U') ||
/#EXT-X-(?:KEY|MAP|BYTERANGE|I-FRAMES-ONLY)/.test(text)
)
throw new Error('Unsupported HLS playlist');
let seq = 0;
let duration = null;
let discontinuity = false;
const segments = [];
for (const raw of text.split('\n')) {
const line = raw.trim();
if (line.startsWith('#EXT-X-MEDIA-SEQUENCE:')) {
seq = Number(line.slice(22));
if (!Number.isSafeInteger(seq) || seq < 0)
throw new Error('Invalid media sequence');
} else if (line === '#EXT-X-DISCONTINUITY') {
discontinuity = true;
} else if (line.startsWith('#EXTINF:')) {
duration = Number.parseFloat(line.slice(8));
if (!Number.isFinite(duration) || duration <= 0 || duration > 60)
throw new Error('Invalid segment duration');
} else if (line && !line.startsWith('#')) {
if (duration === null || !Number.isSafeInteger(seq))
throw new Error('Invalid HLS segment');
const uri = sameOriginHlsUrl(line, base);
if (!new URL(uri).pathname.endsWith('.ts'))
throw new Error('Only MPEG-TS segments are supported');
segments.push({ seq: seq++, duration, uri, discontinuity });
discontinuity = false;
duration = null;
}
}
return segments.slice(-limit);
}
export function createHlsPuller({
fetchImpl = fetch,
limits = HLS_LIMITS,
} = {}) {
const active = new Map();
let closed = false;
const stop = (cameraId, token) => {
const entry = active.get(cameraId);
if (!entry || (token && token !== entry.token)) return;
active.delete(cameraId);
entry.stopping = true;
entry.controller.abort();
clearTimeout(entry.timer);
for (const lease of entry.leases.values()) clearTimeout(lease.timer);
entry.leases.clear();
entry.segments.clear();
entry.upstream.clear();
entry.bytes = 0;
};
const release = (cameraId, leaseId) => {
const entry = active.get(cameraId);
const lease = entry?.leases.get(leaseId);
if (!lease) return;
clearTimeout(lease.timer);
entry.leases.delete(leaseId);
if (!entry.leases.size) stop(cameraId, entry.token);
};
const touch = (entry, leaseId, create = false) => {
let lease = entry.leases.get(leaseId);
if (!lease) {
if (!create || entry.leases.size >= limits.leasesPerSession)
throw new Error('HLS lease capacity reached');
lease = {};
entry.leases.set(leaseId, lease);
}
clearTimeout(lease.timer);
lease.timer = setTimeout(
() => release(entry.cameraId, leaseId),
limits.idleMs,
);
lease.timer.unref?.();
};
const read = (entry, url, maxBytes) =>
fetchHlsBytes(url, {
fetchImpl,
signal: entry.controller.signal,
maxBytes,
timeoutMs: limits.timeoutMs,
});
const poll = async (entry) => {
try {
let base = entry.chunklistUrl || entry.url;
let text = (await read(entry, base, limits.playlistBytes)).toString(
'utf8',
);
if (!text.includes('#EXTINF:')) {
const variant = text
.split('\n')
.map((line) => line.trim())
.find((line) => line && !line.startsWith('#'));
if (!variant || !text.includes('#EXT-X-STREAM-INF:'))
throw new Error('Missing HLS variant');
base = sameOriginHlsUrl(variant, entry.url);
text = (await read(entry, base, limits.playlistBytes)).toString('utf8');
}
entry.chunklistUrl = base;
const segments = parseHlsMedia(text, base, limits.segments);
const newest = segments.at(-1)?.seq ?? -1;
const restarted =
newest < entry.upstreamNewest ||
segments.some((segment) => {
const prior = entry.upstream.get(segment.seq);
return prior && prior !== segment.uri;
});
if (restarted) {
entry.upstream.clear();
entry.pendingDiscontinuity = true;
}
entry.upstreamNewest = newest;
for (const segment of segments) {
if (entry.stopping) return;
if (entry.upstream.has(segment.seq)) continue;
const body = await read(entry, segment.uri, limits.segmentBytes);
if (entry.stopping) return;
// Evict BEFORE retaining the incoming segment. One bounded download may
// exist in addition to the cache; no unbounded disk output is possible.
while (
entry.segments.size &&
(entry.bytes + body.length > limits.sessionBytes ||
entry.segments.size >= limits.segments)
) {
const oldest = entry.segments.keys().next().value;
const removed = entry.segments.get(oldest);
entry.bytes -= removed.body.length;
if (removed.discontinuity) entry.discontinuitiesRemoved++;
entry.segments.delete(oldest);
}
if (body.length > limits.sessionBytes)
throw new Error('Segment exceeds session budget');
const seq = entry.nextSeq++;
entry.segments.set(seq, {
...segment,
seq,
body,
discontinuity: segment.discontinuity || entry.pendingDiscontinuity,
});
entry.pendingDiscontinuity = false;
entry.upstream.set(segment.seq, segment.uri);
while (entry.upstream.size > limits.segments * 2)
entry.upstream.delete(entry.upstream.keys().next().value);
entry.bytes += body.length;
}
entry.failures = 0;
} catch {
entry.chunklistUrl = null;
entry.failures++;
if (entry.failures >= 3) {
stop(entry.cameraId, entry.token);
return;
}
} finally {
if (!entry.stopping) {
entry.timer = setTimeout(() => {
entry.polling = poll(entry);
}, limits.pollMs);
entry.timer.unref?.();
}
}
};
const ensure = async (cameraId, url, leaseId = 'legacy') => {
if (closed) throw new Error('HLS service closed');
sameOriginHlsUrl(url, url);
let entry = active.get(cameraId);
if (entry && entry.url !== url) {
stop(cameraId);
entry = null;
}
if (entry) {
touch(entry, leaseId, true);
return entry;
}
// Reserve synchronously, before any await: concurrent creation is singleflight.
if (active.size >= limits.sessions)
throw new Error('HLS session capacity reached');
entry = {
cameraId,
url,
token: randomUUID(),
controller: new AbortController(),
segments: new Map(),
bytes: 0,
stopping: false,
failures: 0,
nextSeq: 0,
upstream: new Map(),
leases: new Map(),
upstreamNewest: -1,
pendingDiscontinuity: false,
discontinuitiesRemoved: 0,
};
active.set(cameraId, entry);
touch(entry, leaseId, true);
entry.polling = poll(entry);
return entry;
};
const waitReady = async (entry, signal) => {
const until = Date.now() + limits.readyMs;
while (!entry.stopping && !signal?.aborted && Date.now() < until) {
if (entry.segments.size >= 2) return true;
await new Promise((resolve) => setTimeout(resolve, 50));
}
return false;
};
const buildPlaylist = async (entry, cameraId, leaseId = 'legacy') => {
if (entry.stopping || entry.segments.size < 2) return null;
touch(entry, leaseId);
const segments = [...entry.segments.values()].sort((a, b) => a.seq - b.seq);
const lines = [
'#EXTM3U',
'#EXT-X-VERSION:3',
`#EXT-X-TARGETDURATION:${Math.ceil(Math.max(...segments.map((s) => s.duration)))}`,
`#EXT-X-MEDIA-SEQUENCE:${segments[0].seq}`,
`#EXT-X-DISCONTINUITY-SEQUENCE:${entry.discontinuitiesRemoved}`,
];
let previous;
for (const segment of segments) {
if (
segment.discontinuity ||
(previous !== undefined && segment.seq !== previous + 1)
)
lines.push('#EXT-X-DISCONTINUITY');
lines.push(
`#EXTINF:${segment.duration.toFixed(3)},`,
`/api/cctv/media/${encodeURIComponent(cameraId)}/seg_${segment.seq}.ts?session=${entry.token}&lease=${encodeURIComponent(leaseId)}`,
);
previous = segment.seq;
}
return lines.join('\n') + '\n';
};
const getSegment = (cameraId, token, seq, leaseId = 'legacy') => {
const entry = active.get(cameraId);
if (!entry || entry.token !== token || !entry.leases.has(leaseId))
return null;
const body = entry.segments.get(seq)?.body;
if (body) touch(entry, leaseId);
return body || null;
};
const shutdown = async () => {
closed = true;
const entries = [...active.values()];
for (const entry of entries) stop(entry.cameraId);
await Promise.allSettled(entries.map((entry) => entry.polling));
};
return {
ensure,
waitReady,
buildPlaylist,
getSegment,
stop,
release,
shutdown,
stats: () => ({
sessions: active.size,
bytes: [...active.values()].reduce((total, e) => total + e.bytes, 0),
}),
};
}
server/providers/cyclones.js
+416
View File
@@ -0,0 +1,416 @@
import { readResponseTextCapped } from './common/http.js';
const STATUS_URL = 'https://www.nhc.noaa.gov/CurrentStorms.json';
const GIS =
'https://mapservices.weather.noaa.gov/tropical/rest/services/tropical/NHC_tropical_weather_summary/MapServer';
const HOUR = 3600_000;
const COVERAGE =
'Atlantic and eastern/central North Pacific; not worldwide cyclone coverage.';
const LAYERS = [
{ id: 5, cap: 512 * 1024, count: 500, kind: 'points' },
{ id: 6, cap: 512 * 1024, count: 32, kind: 'track' },
{ id: 7, cap: 2 * 1024 * 1024, count: 32, kind: 'cone' },
];
function invalid() {
return new Error('invalid_cyclone_data');
}
function text(value, max = 80) {
if (
typeof value !== 'string' ||
!value.trim() ||
value.length > max ||
/[\u0000-\u001f<>]/.test(value)
)
throw invalid();
return value.trim();
}
function advisory(value) {
if (typeof value !== 'string' || !/^\d{1,3}[A-Z]?$/i.test(value))
throw invalid();
return value.replace(/^0+(?=\d)/, '').toUpperCase();
}
function number(value, min, max) {
if (
value === null ||
value === undefined ||
value === '' ||
!['string', 'number'].includes(typeof value)
)
return null;
const n = Number(value);
return Number.isFinite(n) && n >= min && n <= max ? n : null;
}
function iso(value, now) {
if (
typeof value !== 'string' ||
!/^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(?:\.\d{3})?Z$/.test(value)
)
throw invalid();
const n = Date.parse(value);
if (
!Number.isFinite(n) ||
n > now + 5 * 60_000 ||
now - n > 12 * HOUR ||
new Date(n).toISOString().replace('.000Z', 'Z') !==
value.replace('.000Z', 'Z')
)
throw invalid();
return new Date(n).toISOString();
}
function position(coordinates) {
if (
!Array.isArray(coordinates) ||
coordinates.length !== 2 ||
!coordinates.every((n) => typeof n === 'number' && Number.isFinite(n)) ||
Math.abs(coordinates[0]) > 180 ||
Math.abs(coordinates[1]) > 90
)
throw invalid();
return { longitude: coordinates[0], latitude: coordinates[1] };
}
function officialLink(value) {
if (typeof value !== 'string' || value.length > 256) return null;
try {
const url = new URL(value);
return url.origin === 'https://www.nhc.noaa.gov' &&
!url.username &&
!url.password &&
!url.search &&
!url.hash &&
/^\/text\/[A-Z0-9]+\.shtml$/.test(url.pathname)
? url.href
: null;
} catch {
return null;
}
}
export function parseCycloneStatus(payload, now = Date.now()) {
if (!Array.isArray(payload?.activeStorms) || payload.activeStorms.length > 32)
throw invalid();
const seen = new Set();
return payload.activeStorms.map((raw) => {
if (
typeof raw?.id !== 'string' ||
!/^(?:al|ep|cp)\d{6}$/.test(raw.id) ||
seen.has(raw.id)
)
throw invalid();
seen.add(raw.id);
const forecast = raw.forecastAdvisory;
const issuedAt = iso(forecast?.issuance, now);
const advisoryNumber = advisory(forecast?.advNum);
const positionAt = iso(raw.lastUpdate, now);
return {
id: raw.id,
name: text(raw.name),
classification: text(raw.classification, 16),
basin: raw.id.slice(0, 2).toUpperCase(),
position: position([raw.longitudeNumeric, raw.latitudeNumeric]),
positionAt,
advisoryNumber,
issuedAt,
windKt: number(raw.intensity, 0, 300),
pressureHpa: number(raw.pressure, 800, 1100),
movement: {
directionDegrees: number(raw.movementDir, 0, 360),
speedKt: number(raw.movementSpeed, 0, 200),
},
advisoryUrl: officialLink(forecast.url),
outlookUrl: raw.id.startsWith('al')
? 'https://www.nhc.noaa.gov/gtwo.php?basin=atlc&fdays=7'
: raw.id.startsWith('cp')
? 'https://www.nhc.noaa.gov/gtwo.php?basin=cpac&fdays=7'
: 'https://www.nhc.noaa.gov/gtwo.php?basin=epac&fdays=7',
geometryStatus: 'pending',
geometryAdvisoryNumber: null,
forecastPoints: [],
track: null,
cone: null,
};
});
}
function geometry(raw, kind, budget) {
const types =
kind === 'points'
? ['Point']
: kind === 'track'
? ['LineString', 'MultiLineString']
: ['Polygon', 'MultiPolygon'];
if (!types.includes(raw?.type)) throw invalid();
const point = (value) => {
position(value);
if (++budget.count > 25_000) throw invalid();
return [...value];
};
const line = (value, ring = false) => {
if (
!Array.isArray(value) ||
value.length < (ring ? 4 : 2) ||
value.length > 10_000
)
throw invalid();
const result = value.map(point);
if (
ring &&
(result[0][0] !== result.at(-1)[0] || result[0][1] !== result.at(-1)[1])
)
throw invalid();
return result;
};
const list = (value, read) => {
if (!Array.isArray(value) || !value.length || value.length > 128)
throw invalid();
return value.map(read);
};
const polygon = (value) => list(value, (ring) => line(ring, true));
const coordinates =
raw.type === 'Point'
? point(raw.coordinates)
: raw.type === 'LineString'
? line(raw.coordinates)
: raw.type === 'MultiLineString'
? list(raw.coordinates, (part) => line(part))
: raw.type === 'Polygon'
? polygon(raw.coordinates)
: list(raw.coordinates, polygon);
return { type: raw.type, coordinates };
}
/** Never relabel earlier GIS geometry with the newer status advisory. */
export function attachCycloneGeometry(storms, collections) {
const budget = { count: 0 };
const parsed = LAYERS.map((spec, index) => {
const payload = collections[index];
if (
payload?.type !== 'FeatureCollection' ||
payload.exceededTransferLimit ||
!Array.isArray(payload.features) ||
payload.features.length > spec.count
)
throw invalid();
return payload.features.map((feature) => {
if (feature?.type !== 'Feature') throw invalid();
const p = feature.properties;
const source =
typeof p?.idp_source === 'string' &&
p.idp_source.match(
/^((?:al|ep|cp)\d{6})-(\d{1,3}[a-z]?)_5day_(pts|lin|pgn)$/i,
);
if (!source || source[3].toLowerCase() !== ['pts', 'lin', 'pgn'][index])
throw invalid();
const adv = advisory(p.advisnum);
if (adv !== advisory(source[2])) throw invalid();
return {
id: source[1].toLowerCase(),
advisoryNumber: adv,
geometry: geometry(feature.geometry, spec.kind, budget),
tauHours: number(p.tau, 0, 168),
windKt: number(p.maxwind, 0, 300),
gustKt: number(p.gust, 0, 350),
};
});
});
return storms.map((storm) => {
const groups = parsed.map((items) =>
items.filter(
(item) =>
item.id === storm.id && item.advisoryNumber === storm.advisoryNumber,
),
);
if (groups[1].length > 1 || groups[2].length > 1) throw invalid();
if (!groups[0].length || groups[1].length !== 1 || groups[2].length !== 1)
return storm;
const taus = new Set();
const forecastPoints = groups[0]
.map((item) => {
if (item.tauHours === null || taus.has(item.tauHours)) throw invalid();
taus.add(item.tauHours);
return {
position: position(item.geometry.coordinates),
tauHours: item.tauHours,
windKt: item.windKt,
gustKt: item.gustKt,
};
})
.sort((a, b) => a.tauHours - b.tauHours);
return {
...storm,
geometryStatus: 'current',
geometryAdvisoryNumber: storm.advisoryNumber,
forecastPoints,
track: groups[1][0].geometry,
cone: groups[2][0].geometry,
};
});
}
/** Fixed official endpoints; one shared bounded refresh, no user destinations. */
export function cycloneProxy({
fetchImpl = fetch,
now = () => Date.now(),
timeoutMs = 12_000,
} = {}) {
let cache = null;
let operation = null;
let attemptedAt = -Infinity;
async function upstream(url, cap, signal) {
signal.throwIfAborted();
const response = await fetchImpl(url, {
signal,
redirect: 'error',
headers: {
Accept: 'application/geo+json,application/json',
'User-Agent': 'Gods Eye View (public NOAA weather context)',
},
});
if (!response.ok) {
await response.body?.cancel();
throw new Error('cyclone_upstream_unavailable');
}
const result = JSON.parse(
await readResponseTextCapped(response, cap, signal),
);
signal.throwIfAborted();
return result;
}
async function refresh(signal) {
let storms = parseCycloneStatus(
await upstream(STATUS_URL, 128 * 1024, signal),
now(),
);
if (storms.length) {
try {
const results = await Promise.allSettled(
LAYERS.map(async (spec) => {
const url = new URL(`${GIS}/${spec.id}/query`);
url.search = new URLSearchParams({
where: '1=1',
outFields:
spec.kind === 'points'
? 'idp_source,advisnum,tau,maxwind,gust'
: 'idp_source,advisnum',
outSR: '4326',
resultRecordCount: String(spec.count),
geometryPrecision: '4',
f: 'geojson',
}).toString();
return upstream(url.href, spec.cap, signal);
}),
);
if (results.some((result) => result.status === 'rejected'))
throw invalid();
storms = attachCycloneGeometry(
storms,
results.map((result) => result.value),
);
} catch {
storms = storms.map((storm) => ({
...storm,
geometryStatus: 'unavailable',
}));
}
}
signal.throwIfAborted();
cache = { storms, fetchedAt: now() };
return cache;
}
async function acquire(signal) {
signal.throwIfAborted();
if (cache && now() - cache.fetchedAt < 300_000) return cache;
if (operation?.controller.signal.aborted) operation = null;
if (!operation) {
if (now() - attemptedAt < 60_000) throw new Error('cyclone_retry_later');
attemptedAt = now();
const controller = new AbortController();
const owned = { controller, waiters: 0 };
const timer = setTimeout(() => controller.abort(), timeoutMs);
owned.promise = refresh(controller.signal).finally(() => {
clearTimeout(timer);
if (operation === owned) operation = null;
});
operation = owned;
}
const owned = operation;
if (owned.waiters >= 32)
throw Object.assign(new Error('cyclone_busy'), { status: 429 });
owned.waiters++;
let abort;
const cancelled = new Promise((_, reject) => {
abort = () => reject(signal.reason ?? new Error('cancelled'));
signal.addEventListener('abort', abort, { once: true });
});
try {
return await Promise.race([owned.promise, cancelled]);
} finally {
signal.removeEventListener('abort', abort);
if (--owned.waiters === 0 && operation === owned) {
owned.controller.abort();
if (signal.aborted) attemptedAt = -Infinity;
}
}
}
function describe(value, stale = false) {
return {
schemaVersion: 1,
source: 'NOAA NHC / CPHC',
attribution:
'NOAA/NWS National Hurricane Center / Central Pacific Hurricane Center',
coverage: COVERAGE,
fetchedAt: value?.fetchedAt ?? null,
stale: stale || !value,
unavailable: !value,
reason: !value
? 'Cyclone data unavailable'
: stale
? 'Cached cyclone advisory; upstream unavailable'
: null,
storms: value?.storms ?? [],
};
}
async function handler(req, res) {
const controller = new AbortController();
const close = () => controller.abort();
res.once?.('close', close);
const json = (status, value) => {
if (controller.signal.aborted) return;
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(status === 429 ? { 'Retry-After': '2' } : {}),
});
res.end(JSON.stringify(value));
};
try {
if (req.method !== 'GET')
return json(405, { error: 'method_not_allowed' });
if (req.url !== '/' && req.url !== '')
return json(400, { error: 'invalid_cyclone_query' });
try {
json(200, describe(await acquire(controller.signal)));
} catch (error) {
if (error.status === 429) return json(429, { error: 'cyclone_busy' });
const usable =
cache &&
now() - cache.fetchedAt <= 12 * HOUR &&
cache.storms.every(
(storm) => now() - Date.parse(storm.issuedAt) <= 12 * HOUR,
);
json(200, describe(usable ? cache : null, true));
}
} finally {
res.removeListener?.('close', close);
}
}
return {
name: 'cyclones',
configureServer({ middlewares }) {
middlewares.use('/api/cyclones', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/cyclones', handler);
},
};
}
server/providers/firePerimeters.js
+232
View File
@@ -0,0 +1,232 @@
import { normalizeFirePerimeterSnapshot } from '../../src/layers/perimeters/records.js';
import {
readResponseJsonCapped,
readResponseTextCapped,
coalesceProxyRequest,
} from './common/http.js';
import { makeRateLimiter, clientKey } from './common/rate-limit.js';
// NIFC WFIGS current interagency fire perimeters (public, keyless).
// maxAllowableOffset trades ~100 m of boundary fidelity for a payload small
// enough to refresh continuously (~1 MB for a typical fire season).
const API_URL =
'https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/' +
'WFIGS_Interagency_Perimeters_Current/FeatureServer/0/query?' +
new URLSearchParams({
where: '1=1',
outFields: [
'poly_IncidentName',
'attr_UniqueFireIdentifier',
'attr_IncidentSize',
'attr_PercentContained',
'attr_POOState',
'attr_IncidentTypeCategory',
'attr_FireDiscoveryDateTime',
'poly_DateCurrent',
'attr_FireCause',
'attr_FireBehaviorGeneral',
'attr_TotalIncidentPersonnel',
'attr_POOCounty',
'attr_EstimatedCostToDate',
'attr_IncidentComplexityLevel',
'attr_CpxName',
].join(','),
maxAllowableOffset: '0.001',
outSR: '4326',
f: 'geojson',
}).toString();
// The service caps a single response at its maxRecordCount (2000); a peak
// season can exceed that, so follow exceededTransferLimit with offset pages.
const MAX_PAGES = 5;
const exceededTransferLimit = (payload) =>
payload?.exceededTransferLimit === true ||
payload?.properties?.exceededTransferLimit === true;
const MIB = 1024 * 1024;
/** Fixed-origin, bounded WFIGS and InciWeb routes for dev and preview. */
export function firePerimetersProxy({
fetchImpl = (...args) => globalThis.fetch(...args),
now = () => Date.now(),
} = {}) {
const cache = new Map();
const publications = new Map();
const inFlight = new Map();
const allow = makeRateLimiter({ windowMs: 60_000, max: 60, globalMax: 1200 });
async function upstream(url, cap, timeout, options = {}) {
const signal = AbortSignal.timeout(timeout);
const response = await fetchImpl(url, {
...options,
signal,
redirect: 'error',
});
if (!response.ok) {
await response.body?.cancel();
throw new Error('upstream_unavailable');
}
return readResponseJsonCapped(response, cap, signal);
}
async function fetchPerimeters() {
const features = [];
for (let page = 0; page < MAX_PAGES; page++) {
const url =
page === 0 ? API_URL : `${API_URL}&resultOffset=${features.length}`;
const payload = await upstream(url, 16 * MIB, 30_000);
if (!Array.isArray(payload?.features))
throw new Error('invalid_snapshot');
features.push(...payload.features);
if (!exceededTransferLimit(payload) || !payload.features.length) break;
}
const rows = normalizeFirePerimeterSnapshot({ features });
return { fetchedAt: now(), rows };
}
async function fetchIndex() {
const rows = await upstream(
'https://inciweb.wildfire.gov/api/single-publication/',
4 * MIB,
20_000,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title: '' }),
},
);
if (!Array.isArray(rows)) throw new Error('invalid_index');
return rows;
}
// InciWeb's publication API was retired; incident pages carry origin and update times.
async function fetchPublication(id) {
const signal = AbortSignal.timeout(15_000);
let response = await fetchImpl(`https://inciweb.wildfire.gov/node/${id}`, {
signal,
redirect: 'manual',
});
if ([301, 302, 303, 307, 308].includes(response.status)) {
await response.body?.cancel();
const location = response.headers.get('location');
if (!location || /\.\.|[?%\\]/.test(location))
throw new Error('invalid_redirect');
const url = new URL(location, 'https://inciweb.wildfire.gov/');
if (
url.hostname !== 'inciweb.wildfire.gov' ||
!/^\/[a-z0-9-]+(?:\/[a-z0-9-]+)?$/.test(url.pathname) ||
!['http:', 'https:'].includes(url.protocol) ||
url.port ||
url.username ||
url.password
)
throw new Error('invalid_redirect');
url.protocol = 'https:';
url.search = '';
url.hash = '';
response = await fetchImpl(url.href, { signal, redirect: 'error' });
}
if (response.status !== 200) {
await response.body?.cancel();
throw new Error('upstream_unavailable');
}
const html = await readResponseTextCapped(response, 2 * MIB, signal);
const changed = Date.parse(
html.match(/<meta\s+property="og:updated_time"\s+content="([^"]+)"/)?.[1],
);
const created = Date.parse(
html.match(
/Date of Origin<\/th>\s*<td[^>]*>\s*<time\s+datetime="([^"]+)"/,
)?.[1],
);
const createdMs = Number.isFinite(created) ? created : null;
const changedMs = Number.isFinite(changed) ? changed : null;
if (createdMs === null && changedMs === null)
throw new Error('invalid_incident_page');
return { createdMs, changedMs };
}
async function acquire(key, ttl, load, store) {
const previous = store.get(key);
if (previous && now() - previous.savedAt < ttl)
return { value: previous.value, stale: false };
// Bound distinct concurrent publication requests as well as retained entries.
if (!inFlight.has(key) && inFlight.size >= 256)
throw Object.assign(new Error('busy'), { status: 429 });
try {
const { promise } = coalesceProxyRequest(inFlight, key, async () => {
const value = await load();
store.delete(key);
store.set(key, { value, savedAt: now() });
if (store === publications && store.size > 256)
store.delete(store.keys().next().value);
return value;
});
return { value: await promise, stale: false };
} catch (error) {
if (previous) return { value: previous.value, stale: true };
throw error;
}
}
async function handler(req, res) {
const json = (status, value, stale = false) => {
if (res.destroyed) return;
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(status === 405 ? { Allow: 'GET' } : {}),
...(status === 429 ? { 'Retry-After': '60' } : {}),
...(stale ? { 'X-Data-Stale': 'true' } : {}),
});
res.end(JSON.stringify(value));
};
if (req.method !== 'GET') return json(405, { error: 'method_not_allowed' });
const path = (req.url || '/').split('?')[0];
let key,
ttl,
load,
store = cache;
if (path === '/' || path === '') {
key = 'perimeters';
ttl = 300_000;
load = fetchPerimeters;
} else if (path === '/inciweb/index') {
key = 'index';
ttl = 3_600_000;
load = fetchIndex;
} else if (path.startsWith('/inciweb/publication/')) {
const id = path.slice('/inciweb/publication/'.length);
if (!/^\d{1,9}$/.test(id))
return json(400, { error: 'invalid_publication_id' });
key = `publication:${id}`;
ttl = 1_800_000;
load = () => fetchPublication(id);
store = publications;
} else return json(404, { error: 'unknown_route' });
if (!allow(clientKey(req))) return json(429, { error: 'rate_limited' });
try {
const { value, stale } = await acquire(key, ttl, load, store);
json(
200,
key === 'perimeters' && stale ? { ...value, stale: true } : value,
stale,
);
} catch (error) {
json(error.status === 429 ? 429 : 502, {
error: 'fire_perimeters_unavailable',
});
}
}
return {
name: 'fire-perimeters',
configureServer({ middlewares }) {
middlewares.use('/api/fire-perimeters', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/fire-perimeters', handler);
},
};
}
server/providers/firms.js
+11 -5
View File
@@ -8,10 +8,11 @@ import { filterTrailing24h, parseFirmsCsv } from '../../src/data/firmsCsv.js';
* Upstream: https://firms.modaps.eosdis.nasa.gov/api/area/csv/{KEY}/{SOURCE}/world/2
*
* Merges three VIIRS NRT sources (NOAA-20, NOAA-21, Suomi-NPP — independent
* satellites, no cross-source dedup) fetched sequentially with `days=2`
* (`days=1` means "current UTC day", nearly empty just after 00:00Z) and
* clamps to the trailing 24 h via src/data/firmsCsv.js. FIRMS quota is
* 5,000 transactions / 10 min per MAP_KEY, so the cache is the point:
* satellites, no cross-source dedup) plus MODIS_NRT (combined Terra+Aqua,
* ~1 km, numeric 0-100 confidence), fetched sequentially (quota courtesy)
* with `days=2` (`days=1` means "current UTC day", nearly empty just after
* 00:00Z) and clamps to the trailing 24 h via src/data/firmsCsv.js. FIRMS
* quota is 5,000 transactions / 10 min per MAP_KEY, so the cache is the point:
* TTL 30 min, single-flight refresh, serve-stale-on-failure, and a
* fresh-enough disk cache (.gev-cache/firms.json) prevents ANY upstream
* fetch across dev-server restarts. Pattern mirrors celestrakProxy.
@@ -28,7 +29,12 @@ import { filterTrailing24h, parseFirmsCsv } from '../../src/data/firmsCsv.js';
export function firmsProxy() {
const TTL_MS = 30 * 60_000;
const STATUS_TTL_MS = 5 * 60_000;
const SOURCES = ['VIIRS_NOAA20_NRT', 'VIIRS_NOAA21_NRT', 'VIIRS_SNPP_NRT'];
const SOURCES = [
'VIIRS_NOAA20_NRT',
'VIIRS_NOAA21_NRT',
'VIIRS_SNPP_NRT',
'MODIS_NRT',
];
const CACHE_DIR = path.join(process.cwd(), '.gev-cache');
const CACHE_PATH = path.join(CACHE_DIR, 'firms.json');
server/providers/local-receivers.js
+521
View File
@@ -0,0 +1,521 @@
import { lookup as dnsLookup } from 'node:dns/promises';
import http from 'node:http';
import https from 'node:https';
import { Readable } from 'node:stream';
import { coalesceProxyRequest, readResponseTextCapped } from './common/http.js';
import { isLocalIpv4, parseTapAddress } from '../../src/data/tapAddress.js';
import { normalizeDump1090Aircraft } from '../../src/sources/adsbRecords.js';
/**
* Local decoder feeds for the Local ADS-B layer.
*
* A user who runs dump1090 / readsb / tar1090 (1090 MHz) or dump978-fa +
* skyaware978 (978 MHz UAT) exposes an `aircraft.json` over HTTP. This route
* reads those documents and returns them as Local ADS-B records.
*
* The feed list comes only from the operator's `LOCAL_RECEIVER_FEEDS`
* environment value; the browser never supplies an address. Every host must
* pass the shared receiver-tap address rule (`src/data/tapAddress.js`:
* loopback, RFC1918, `localhost`, `*.local`), the scheme must be http(s) and
* the path must end in `aircraft.json`. Invalid entries are logged at startup,
* reported as `invalid` and never fetched.
*
* A name (`localhost`, `*.local`) is resolved before every read; each address
* it resolves to must itself be loopback or RFC1918, and the connection is
* pinned to exactly those validated addresses, so a later re-resolution
* cannot redirect it anywhere else.
*/
export const LOCAL_RECEIVERS_ROUTE = '/api/local-receivers/aircraft';
export const LOCAL_RECEIVER_FEEDS_ENV = 'LOCAL_RECEIVER_FEEDS';
export const LOCAL_RECEIVER_BANDS = Object.freeze(['1090', '978']);
export const LOCAL_RECEIVER_TIMEOUT_MS = 2_000;
export const LOCAL_RECEIVER_MAX_BODY_BYTES = 2 * 1024 * 1024;
export const LOCAL_RECEIVER_CACHE_MS = 1_000;
/** A feed whose own `now` is older than this is reported stale. */
export const LOCAL_RECEIVER_STALE_MS = 10_000;
const MAX_FEEDS = 8;
const LOG_PREFIX = '[local-receivers]';
const BAND_LABELS = Object.freeze({ 1090: '1090 MHz', 978: '978 MHz UAT' });
/** Statuses whose `Response` must be constructed without a body. */
const NULL_BODY_STATUSES = new Set([101, 103, 204, 205, 304]);
function defaultPort(protocol) {
return protocol === 'https:' ? 443 : 80;
}
/**
* Validate one `band=url` entry.
* @param {string} entry Raw entry text.
* @returns {{band:string, url:string}|{band:string|null, reason:string}}
*/
function parseEntry(entry) {
const separator = entry.indexOf('=');
if (separator <= 0) return { band: null, reason: 'expected band=url' };
const band = entry.slice(0, separator).trim();
const rawUrl = entry.slice(separator + 1).trim();
if (!LOCAL_RECEIVER_BANDS.includes(band))
return { band: null, reason: 'band must be 1090 or 978' };
let url;
try {
url = new URL(rawUrl);
} catch {
return { band, reason: 'not a valid URL' };
}
if (url.protocol !== 'http:' && url.protocol !== 'https:')
return { band, reason: 'scheme must be http or https' };
if (url.username || url.password)
return { band, reason: 'credentials are not allowed in the URL' };
if (url.search || url.hash || rawUrl.includes('?') || rawUrl.includes('#'))
return { band, reason: 'query and fragment are not allowed' };
const port = url.port ? Number(url.port) : defaultPort(url.protocol);
const address = parseTapAddress(`${url.hostname}:${port}`);
if (!address)
return {
band,
reason: 'host must be loopback, RFC1918, localhost or *.local',
};
if (!/(?:^|\/)aircraft\.json$/.test(url.pathname))
return { band, reason: 'path must end in aircraft.json' };
return {
band,
url: `${url.protocol}//${address.host}:${address.port}${url.pathname}`,
};
}
/**
* Parse `LOCAL_RECEIVER_FEEDS` (`band=url[,band=url…]`).
*
* Labels name the band only (plus an ordinal when a band repeats), so feed
* addresses never reach the browser.
* @param {string|undefined} value Environment value.
* @returns {{configured:boolean, feeds:object[]}} Each feed is
* `{ id, band, label, url }` when valid or `{ id, band, label, reason }`
* when invalid (never fetched).
*/
export function parseLocalReceiverFeeds(value) {
const entries = String(value ?? '')
.split(',')
.map((entry) => entry.trim())
.filter(Boolean);
const feeds = [];
const perBand = new Map();
entries.forEach((entry, index) => {
const parsed =
index < MAX_FEEDS
? parseEntry(entry)
: { band: null, reason: `at most ${MAX_FEEDS} feeds are read` };
const band = parsed.band;
const ordinal = (perBand.get(band) || 0) + 1;
perBand.set(band, ordinal);
const base = band ? BAND_LABELS[band] : `entry ${index + 1}`;
const label = band && ordinal > 1 ? `${base} #${ordinal}` : base;
feeds.push({
id: `feed-${index + 1}`,
band,
label,
...(parsed.url ? { url: parsed.url } : { reason: parsed.reason }),
});
});
// Ordinals are only needed when a band actually repeats.
for (const feed of feeds) {
if (feed.band && perBand.get(feed.band) > 1 && !feed.label.includes('#'))
feed.label = `${feed.label} #1`;
}
return { configured: feeds.length > 0, feeds };
}
class FeedError extends Error {
constructor(code) {
super(code);
this.code = code;
}
}
/**
* Whether a resolved address satisfies the receiver-tap rule: loopback or
* RFC1918 IPv4, the IPv6 loopback, or an IPv4-mapped form of the former.
* @param {string} address Resolved IP address.
* @returns {boolean}
*/
export function isLocalReceiverAddress(address) {
const value = String(address || '').toLowerCase();
if (value === '::1') return true;
const mapped = /^::ffff:(\d{1,3}(?:\.\d{1,3}){3})$/.exec(value);
return isLocalIpv4(mapped ? mapped[1] : value);
}
/**
* Resolve a feed host name and require every address to be local.
* @param {string} hostname `localhost` or a `*.local` name.
* @param {typeof dnsLookup} lookupImpl `dns.promises.lookup`-compatible.
* @returns {Promise<Array<{address:string, family:number}>>}
*/
export async function resolveLocalReceiverAddresses(hostname, lookupImpl) {
let resolved;
try {
resolved = await lookupImpl(hostname, { all: true, verbatim: true });
} catch {
throw new FeedError('UNRESOLVED');
}
const addresses = (Array.isArray(resolved) ? resolved : [resolved])
.map((row) => ({
address: String(row?.address || ''),
family:
Number(row?.family) || (String(row?.address).includes(':') ? 6 : 4),
}))
.filter((row) => row.address);
if (!addresses.length) throw new FeedError('UNRESOLVED');
if (addresses.some((row) => !isLocalReceiverAddress(row.address)))
throw new FeedError('FORBIDDEN_ADDRESS');
return addresses;
}
/**
* Settle with `promise`, or reject with an AbortError as soon as `signal`
* aborts; a later settlement of `promise` is ignored.
*/
function untilAborted(promise, signal) {
return new Promise((resolve, reject) => {
const onAbort = () => {
const error = new Error('aborted');
error.name = 'AbortError';
reject(error);
};
// Handlers are attached first so a late rejection is never unhandled.
promise.then(
(value) => {
signal.removeEventListener('abort', onAbort);
resolve(value);
},
(error) => {
signal.removeEventListener('abort', onAbort);
reject(error);
},
);
if (signal.aborted) onAbort();
else signal.addEventListener('abort', onAbort, { once: true });
});
}
/** A `net`-style lookup that only ever answers the validated addresses. */
function pinnedLookup(addresses) {
return (_hostname, options, callback) => {
const done = typeof options === 'function' ? options : callback;
if (options?.all)
done(
null,
addresses.map((row) => ({ ...row })),
);
else done(null, addresses[0].address, addresses[0].family);
};
}
/**
* Default transport. With `options.lookup` (a pinned feed name) the request
* is made with node:http(s) so the socket connects only to the validated
* addresses (TLS still verifies the name); otherwise it is a plain fetch of
* an IP-literal URL. Redirects are never followed either way.
* @param {string} url
* @param {object} options
* @returns {Promise<Response>}
*/
function fetchLocalReceiverFeed(url, options) {
if (typeof options?.lookup !== 'function') return fetch(url, options);
const client = new URL(url).protocol === 'https:' ? https : http;
return new Promise((resolve, reject) => {
const request = client.request(
url,
{
method: 'GET',
headers: options.headers,
signal: options.signal,
lookup: options.lookup,
// A fresh socket per read: a pooled one could outlive the pin.
agent: false,
},
(response) => {
// Nothing thrown here may escape: this callback runs outside the
// promise, so an exception would be uncaught and crash the server.
try {
const status = response.statusCode;
// A `Response` only represents final statuses 200–599.
if (!Number.isInteger(status) || status < 200 || status > 599)
throw new FeedError('HTTP_STATUS');
const headers = new Headers();
for (const [name, value] of Object.entries(response.headers)) {
if (Array.isArray(value))
value.forEach((item) => headers.append(name, item));
else if (value !== undefined) headers.set(name, String(value));
}
const nullBody = NULL_BODY_STATUSES.has(status);
const upstream = new Response(
nullBody ? null : Readable.toWeb(response),
{ status, statusText: response.statusMessage || '', headers },
);
if (nullBody) response.destroy();
resolve(upstream);
} catch (error) {
response.destroy();
reject(error);
}
},
);
request.on('error', reject);
request.end();
});
}
/**
* Fetch one validated feed document without following redirects and with the
* body cap enforced while it streams.
* @param {string} url Validated feed URL.
* @param {object} options
* @returns {Promise<object>} Parsed JSON document.
*/
export async function fetchLocalReceiverDocument(
url,
{
fetchImpl = fetchLocalReceiverFeed,
lookupImpl = dnsLookup,
timeoutMs = LOCAL_RECEIVER_TIMEOUT_MS,
maxBytes = LOCAL_RECEIVER_MAX_BODY_BYTES,
} = {},
) {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
let upstream;
try {
const { hostname } = new URL(url);
// IP literals were validated when the feed list was parsed; a name is
// validated by what it resolves to now, and the socket is pinned there.
// Resolution runs against the same deadline as the read: a stalled
// lookup fails this feed instead of holding the whole snapshot.
const lookup = isLocalIpv4(hostname)
? undefined
: pinnedLookup(
await untilAborted(
resolveLocalReceiverAddresses(hostname, lookupImpl),
controller.signal,
),
);
upstream = await fetchImpl(url, {
method: 'GET',
headers: { Accept: 'application/json' },
redirect: 'manual',
signal: controller.signal,
...(lookup ? { lookup } : {}),
});
if (upstream.status >= 300 && upstream.status < 400)
throw new FeedError('REDIRECT_REFUSED');
// Opaque redirect responses from `redirect: 'manual'` report status 0.
if (upstream.type === 'opaqueredirect')
throw new FeedError('REDIRECT_REFUSED');
if (!upstream.ok) throw new FeedError('HTTP_STATUS');
const text = await readResponseTextCapped(
upstream,
maxBytes,
controller.signal,
);
let json;
try {
json = JSON.parse(text);
} catch {
throw new FeedError('BAD_JSON');
}
if (!json || typeof json !== 'object' || !Array.isArray(json.aircraft))
throw new FeedError('BAD_DOCUMENT');
return json;
} catch (error) {
controller.abort();
if (upstream?.body && !upstream.body.locked)
void upstream.body.cancel().catch(() => {});
if (error instanceof FeedError) throw error;
if (error?.code === 'RESPONSE_TOO_LARGE')
throw new FeedError('RESPONSE_TOO_LARGE');
if (error?.name === 'AbortError' || controller.signal.aborted)
throw new FeedError('TIMEOUT');
throw new FeedError('UNREACHABLE');
} finally {
clearTimeout(timeoutId);
}
}
/**
* Read one feed and describe it.
* @returns {Promise<{status:object, records:object[]}>}
*/
async function readFeed(
feed,
{ fetchImpl, lookupImpl, now, timeoutMs, maxBytes, log },
) {
const summary = { band: feed.band, label: feed.label };
if (!feed.url)
return {
status: { ...summary, status: 'invalid', aircraft: 0, ageMs: null },
records: [],
};
let json;
try {
json = await fetchLocalReceiverDocument(feed.url, {
fetchImpl,
lookupImpl,
timeoutMs,
maxBytes,
});
} catch (error) {
log(feed, 'unreachable', error.code);
return {
status: { ...summary, status: 'unreachable', aircraft: 0, ageMs: null },
records: [],
};
}
const receivedAt = now();
const feedNowS = Number(json.now);
// The feed's own clock decides staleness. A clock ahead of ours reads as
// age 0; a document without `now` is treated as current.
const ageMs = Number.isFinite(feedNowS)
? Math.max(0, Math.round(receivedAt - feedNowS * 1000))
: 0;
const status = ageMs > LOCAL_RECEIVER_STALE_MS ? 'stale' : 'live';
log(feed, status);
// Anchor record ages at the document time so a stale feed ages out
// naturally, without trusting a receiver clock that runs ahead of ours.
const records = normalizeDump1090Aircraft(json, receivedAt - ageMs, {
band: feed.band,
});
return {
status: { ...summary, status, aircraft: records.length, ageMs },
records,
};
}
/**
* Build the route handler. Exposed for tests; the plugin wraps it.
* @param {object} [options]
* @param {string} [options.feedsValue] `LOCAL_RECEIVER_FEEDS` value.
* @param {typeof fetch} [options.fetchImpl] Receives `options.lookup` when
* the feed host is a name; the connection must resolve only through it.
* @param {Function} [options.lookupImpl] `dns.promises.lookup`-compatible.
* @param {() => number} [options.now]
* @param {{warn:Function, info?:Function}} [options.logger]
* @returns {{handle:Function, config:object}}
*/
export function createLocalReceiversHandler({
feedsValue = process.env[LOCAL_RECEIVER_FEEDS_ENV],
fetchImpl = fetchLocalReceiverFeed,
lookupImpl = dnsLookup,
now = Date.now,
logger = console,
timeoutMs = LOCAL_RECEIVER_TIMEOUT_MS,
maxBytes = LOCAL_RECEIVER_MAX_BODY_BYTES,
cacheMs = LOCAL_RECEIVER_CACHE_MS,
} = {}) {
const config = parseLocalReceiverFeeds(feedsValue);
for (const feed of config.feeds) {
if (feed.reason)
logger.warn(
`${LOG_PREFIX} ${LOCAL_RECEIVER_FEEDS_ENV} ${feed.id} (${feed.label}) rejected: ${feed.reason}. It will not be fetched.`,
);
}
const lastStatus = new Map();
// Log a feed only when its status changes, so a feed that is down does not
// write a line every second.
const log = (feed, status, code) => {
if (lastStatus.get(feed.id) === status) return;
lastStatus.set(feed.id, status);
if (status === 'live') logger.info?.(`${LOG_PREFIX} ${feed.label} live`);
else
logger.warn(
`${LOG_PREFIX} ${feed.label} ${status}${code ? ` (${code})` : ''}`,
);
};
const inFlight = new Map();
let cached = null;
async function snapshot() {
const results = await Promise.all(
config.feeds.map((feed) =>
readFeed(feed, {
fetchImpl,
lookupImpl,
now,
timeoutMs,
maxBytes,
log,
}),
),
);
return {
configured: true,
generatedAt: now(),
feeds: results.map((result) => result.status),
records: results.flatMap((result) => result.records),
};
}
async function payload() {
if (!config.configured)
return { configured: false, feeds: [], records: [] };
if (cached && now() - cached.at < cacheMs) return cached.payload;
const { promise } = coalesceProxyRequest(inFlight, 'all', snapshot);
const result = await promise;
cached = { at: now(), payload: result };
return result;
}
async function handle(req, res) {
if (req.method !== 'GET' && req.method !== 'HEAD') {
res.writeHead(405, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
return;
}
try {
const body = await payload();
res.writeHead(200, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify(body));
} catch {
res.writeHead(502, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify({ error: 'Local receiver feeds unavailable' }));
}
}
return { handle, config };
}
/**
* Vite plugin mounting `GET /api/local-receivers/aircraft`.
*
* The environment is read when the server starts, so construction alone
* neither parses configuration nor fetches anything.
* @param {object} [options] Passed to {@link createLocalReceiversHandler}.
* @returns {import('vite').Plugin}
*/
export function localReceiversProxy(options = {}) {
const install = (server) => {
const { handle } = createLocalReceiversHandler(options);
server.middlewares.use(LOCAL_RECEIVERS_ROUTE, (req, res, next) => {
const path = new URL(req.url || '/', 'http://localhost').pathname;
if (path !== '/' && path !== '') {
if (typeof next === 'function') return next();
return undefined;
}
return handle(req, res);
});
};
return {
name: 'local-receivers-proxy',
configureServer: install,
configurePreviewServer: install,
};
}
server/providers/local.js
+10
View File
@@ -13,6 +13,7 @@ import { cctvProxy } from './cctv.js';
import { defaultSourceRoot } from './common/source-root.js';
import { radioBrowserProxy } from './radio.js';
import { gbfsProxy } from './gbfs.js';
import { localReceiversProxy } from './local-receivers.js';
import { transitProxy } from './transit.js';
import { adsbLolProxy } from './aircraft/adsb-lol.js';
import { aisLiveProxy } from './vessels/ais-live.js';
@@ -20,6 +21,10 @@ import { trackBackfillProxies } from './aircraft/tracks.js';
import { openAiRealtimeProxy } from './openai.js';
import { googlePlacesContextProxy } from './places.js';
import { keySetupEndpoint } from '../standalone/key-setup.js';
import { weatherProxy } from './weather.js';
import { firePerimetersProxy } from './firePerimeters.js';
import { cycloneProxy } from './cyclones.js';
import { windProxy } from './wind.js';
/** Construct the local provider plugins in their established order. */
function localProviderPlugins() {
@@ -39,12 +44,17 @@ function localProviderPlugins() {
cctvProxy({ sourceRoot: defaultSourceRoot }),
radioBrowserProxy(),
gbfsProxy(),
localReceiversProxy(),
transitProxy(),
adsbLolProxy(),
aisLiveProxy(),
trackBackfillProxies(),
openAiRealtimeProxy(),
googlePlacesContextProxy(),
windProxy(),
weatherProxy(),
cycloneProxy(),
firePerimetersProxy(),
keySetupEndpoint(),
];
}
server/providers/military-installations.js
+40 -9
View File
@@ -5,7 +5,11 @@ import {
MILITARY_INSTALLATION_DISK_TTL_MS,
MILITARY_INSTALLATION_STALE_MS,
} from './military-installations/constants.js';
import { fetchOverpassPayload } from './overpass/transport.js';
import {
fetchOverpassPayload,
overpassNotConfigured,
} from './overpass/transport.js';
import { resolveOverpassUpstreams } from './overpass/constants.js';
import {
_militaryInstallationCache,
trimMilitaryInstallationCache,
@@ -45,6 +49,7 @@ function militaryInstallationsProxy() {
throw Object.assign(
new Error('Mapped installation upstream unavailable'),
{
retryAfterMs: upstream.retryAfterMs,
installationReason: upstream.rateLimited
? 'rate_limited'
: upstream.status === 504
@@ -83,14 +88,6 @@ function militaryInstallationsProxy() {
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
return;
}
if (!_militaryInstallationsRateLimiter(clientKey(req))) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Retry-After': '5',
});
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
return;
}
const url = new URL(req.url, 'http://localhost');
const requested = validMilitaryInstallationBox(url.searchParams);
if (!requested) {
@@ -117,6 +114,36 @@ function militaryInstallationsProxy() {
: militaryInstallationCacheKey(box);
const now = Date.now();
const cached = _militaryInstallationCache.get(key);
if (!resolveOverpassUpstreams().length) {
const stale =
cached || (await readMilitaryInstallationDisk(key, Infinity));
const disabled = overpassNotConfigured();
res.writeHead(200, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
'X-Military-Installations': stale ? 'STALE' : 'DISABLED',
});
res.end(
stale
? JSON.stringify({
...stale.payload,
status: 'stale',
retrievedAt:
stale.payload.retrievedAt ||
new Date(stale.cachedAt).toISOString(),
})
: disabled.body,
);
return;
}
if (!_militaryInstallationsRateLimiter(clientKey(req))) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Retry-After': '5',
});
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
return;
}
const preflight = await resolveMilitaryInstallationTier({
cacheKey: key,
memoryCache: _militaryInstallationCache,
@@ -176,6 +203,9 @@ function militaryInstallationsProxy() {
return;
}
res.writeHead(503, {
...(error?.retryAfterMs
? { 'Retry-After': String(Math.ceil(error.retryAfterMs / 1000)) }
: {}),
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
@@ -183,6 +213,7 @@ function militaryInstallationsProxy() {
JSON.stringify({
error: 'Mapped installation context is temporarily unavailable',
reason: militaryInstallationFailureReason(error),
retryAfterMs: error?.retryAfterMs,
}),
);
}
server/providers/openai/hud-summary.js
+5 -8
View File
@@ -1,4 +1,7 @@
import { keylessHudSummaryResponse } from '../../../src/hudSummaryResponse.js';
import {
HUD_SUMMARY_INSTRUCTIONS,
keylessHudSummaryResponse,
} from '../../../src/hudSummaryResponse.js';
import { enforceOptInRateLimit, openAiRateLimiter } from './rate-limit.js';
import { readRequestBody } from '../common/request.js';
import { OPENAI_HUD_SUMMARY_MODEL_DEFAULT } from './constants.js';
@@ -61,13 +64,7 @@ async function handleHudSummary(req, res) {
model:
process.env.OPENAI_HUD_SUMMARY_MODEL ||
OPENAI_HUD_SUMMARY_MODEL_DEFAULT,
instructions: [
"Write one concise intelligence-HUD summary for God's Eye View.",
'Use only the supplied place, street, nearby-place, and enabled-layer text labels.',
'Prefer the clearest named place and include a relevant enabled layer only when useful.',
'Do not infer from coordinates or invent a place.',
'Output exactly five words with no title, punctuation, markdown, or introductory phrase.',
].join(' '),
instructions: HUD_SUMMARY_INSTRUCTIONS,
input: JSON.stringify(context),
reasoning: { effort: 'minimal' },
max_output_tokens: 100,
server/providers/openai/instructions.js
+4 -2
View File
@@ -18,7 +18,8 @@ function realtimeInstructions(
'For camera-motion requests — "orbit around this", "pan left", "tilt up", "stop moving" — call move_camera. For "fly the route" over a drawn route, call fly_route. Confirm with the RESULTING state ("Orbiting slowly", "Flying the route").',
'analyst_query ANSWERS questions; it never moves the camera or starts tracking. For requests to FOLLOW or TRACK a specific aircraft/ship, call track_entity (get_entity_context first when the target is ambiguous), never analyst_query as the final or only action. For "follow/track the nearest aircraft", first call analyst_query with the aircraft layer(s), sortBy=distance, and limit=1, then call track_entity with the returned aircraft identity in the same turn. The lookup alone does not fulfill a follow/track command.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data: report the returned feed source briefly, never call it an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about layer data — how many / which / fastest / highest / biggest / nearest flights, ships, fires, or earthquakes ("how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Narrate the count plus two or three notable examples by name, and reflect the result\'s coverage note honestly: the answer covers data loaded by enabled layers, not the whole world. If the needed layer is disabled, say so and offer to enable it. For follow-ups about the same set ("which of THOSE is closest?"), call analyst_query with followUp=true and only the new filter/sort.',
'For ANALYTICAL questions about layer data — how many / which / fastest / highest / biggest / nearest flights, ships, fires, earthquakes, satellites, datacenters, or dams ("how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Narrate the count plus two or three notable examples by name, and reflect the result\'s coverage note honestly: the answer covers data loaded by enabled layers, not the whole world. If the needed layer is disabled, say so and offer to enable it. For follow-ups about the same set ("which of THOSE is closest?"), call analyst_query with followUp=true and only the new filter/sort.',
'FEED STATE CONTRACT — analyst_query and get_current_view_state carry feedProvenance from the same layerFeedState the Data Layers chips use (nominal/loading/degraded/stale/fallback/unavailable, or off). NEVER narrate a count as live or current when feedProvenance.overall is not nominal. Say the feedState in the same breath as the count ("12 flights in view, STALE"). If unavailable, do not invent a count; say the feed is unavailable. Fallback is valid data — name the fallback source, never call it an enable failure. Confirm only what the tool result returned.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — those are the exact numbers on the user\'s panel. set_context_mode, analyst_query, and get_current_view_state carry it after Contacts settles. For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft; do not answer from a pre-Contacts analyst query. analyst_query\'s own count measures currently-loaded records and is usually lower; never give it as the window count. CENTER PRECEDENCE for a nearby/how-many ask, in order: an explicit place in the question ("over Texas", "near Austin") always wins and ignores Contacts state; else the CONTACTS SUBJECT when Contacts is active and has one — a selected datacenter, dam, fire, or cable does NOT silently become the center; else an entity the user explicitly names ("around this datacenter"); else the current view, said aloud ("nothing is selected, so this is the current view"). With Contacts active but NO subject yet, use the view and say so; never read an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two different numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge ("a few", "less than a dozen"): if a tool returns 46, say 46. (5) When it matters, add once: counts cover loaded data, and the flights layer loads where you look.',
'While Cockpit is active, navigate with control_cockpit (next/previous, optionally targetLayer or aircraftClass). track_entity and fly_to_location are REFUSED by design while Cockpit owns the camera — that refusal is correct, not an error to retry. To go somewhere else, exit Cockpit first. control_cockpit enter establishes Contacts itself, so do not call set_context_mode before or after it.',
'When the target layer is unknown, OMIT layerId in track_entity so it searches all enabled layers. Passing the wrong layerId ("flights" for a military contact) returns "Nothing matched" even though the contact is loaded.',
@@ -48,7 +49,8 @@ function realtimeInstructions(
'NAMED VIEWS are shorthand for tool calls you already have — there is no "mode" tool for them. Treat ONLY these as the shorthand: "infrastructure mode" / "the infrastructure view" / "show me global infrastructure" means three set_layer_visibility calls (local-datacenters, local-dams, telegeography-submarine-cables) plus zoom_to_globe; "environmental mode" / "earth watch" / "active events", said as the name of a view, means set_layer_visibility for local-firms and earthquakes plus zoom_to_globe. Anything vaguer is NOT this shorthand — an open-ended question about the world or the news is an ordinary question: answer it, or use analyst_query over the layers already on. Never switch a whole view on to answer a question nobody asked to see. When you do run one, make every call before speaking, then give one confirmation naming the resulting state; if the fires layer comes back unavailable because no FIRMS key is configured, say so plainly — the earthquakes still loaded. "Live contacts" and "space missions" are NOT this pattern: they stay set_context_mode{mode:"contacts"} and set_context_mode{mode:"space-missions"}.',
'For visual filter requests, call set_visual_style with one of the allowed style IDs.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any mention of "satellite" or "satellites" ALWAYS means the satellites DATA LAYER via set_layer_visibility, never a basemap. "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'HUD requests ("hud on/off", "switch to operator/minimal/tactical layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
'HUD requests ("hud on/off", "switch to operator/minimal/tactical/cyber layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
'Cyber sonar requests use set_cyber_sonar: enabled, rings, rangePct, intensityPct (Power), opacityPct, sectorDeg. For settings questions read get_current_view_state.controls.hud.sonar. Sonar is not detection. Do not silently switch themes, reveal the HUD or enable sonar when changing a slider. If the user explicitly requests Cyber plus sonar settings, call set_hud first, wait for success, then set_cyber_sonar. Confirm returned values; mapSweepActive and contactSweepActive distinguish saved settings from active map effects, including in Cockpit or reduced motion. Do not claim visible contacts solely because enabled is true.',
'Bloom/sharpen requests use set_post_processing. Scene requests ("play orbital watch", "stop the scene", "what scenes are there") use control_scene. CCTV camera requests ("next camera", "nearest camera", "select the Congress camera", "show coverage") use control_cctv — the CCTV layer must be enabled first.',
'Radio playback requests use control_radio. "Turn on/start the radio" means action=play; action=enable only reveals Radio markers and must be reserved for explicit "show/enable the Radio layer/markers" requests. After a prepared playback result, briefly confirm any other completed actions and say "Turning on the radio"—never claim it is already playing. The client keeps Radio muted until playback is verified, then closes voice before restoring Radio volume. Examples: "play news near Austin" → select category=news locationId=austin; "play US news" → select category=news country=US; "Radio volume 30" → volume; pause/resume/stop/next/previous use the matching action. Radio selection never moves the camera.',
'"Track/follow <something specific>" (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.',
server/providers/openai/toolDescriptions.js
+31 -3
View File
@@ -82,7 +82,7 @@ export const ACTION_DESCRIPTIONS = {
properties: {
layerId: {
description:
'Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras.',
'Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.',
$position: 1,
},
},
@@ -182,6 +182,29 @@ export const ACTION_DESCRIPTIONS = {
},
},
},
set_cyber_sonar: {
description:
'Adjust Cyber-only sonar controls. Requires the Cyber HUD layout; never switches layout or HUD visibility. Omitted fields stay unchanged. Returns actual settings and separate map/contact sweep activity. A saved setting does not mean the effect is active.',
$position: 1,
parameters: {
properties: {
enabled: { description: 'Explicitly turn sonar on or off.' },
rings: { description: 'Number of decorative sonar rings, 3–12.' },
rangePct: {
description:
'Visual ring range, 60–120 percent; not geographic distance.',
},
intensityPct: {
description: 'Sonar Power slider, 0–100 percent. Zero is valid.',
},
opacityPct: {
description:
'Contact opacity floor between passes, 35–100 percent; labels have a derived floor. Not whole-scene dimming.',
},
sectorDeg: { description: 'Sonar sweep sector width, 8–60 degrees.' },
},
},
},
set_detection: {
description:
'Control the detection overlay: on/off, density-derived Sparse/Balanced/Dense profile, and Elastic/Weighted layer allocation.',
@@ -505,7 +528,7 @@ export const ACTION_DESCRIPTIONS = {
},
analyst_query: {
description:
'Answer questions ABOUT the data currently loaded on the map — counts, lists, superlatives, and attribute filters over live layers (flights, military, ships, fires, earthquakes). Examples: "how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet", "fastest thing in view". Queries ONLY client-side data from ENABLED layers — if the needed layer is off, say so and offer to enable it. For a follow-up about the previous answer\'s set ("which of those is closest?"), set followUp=true and send only the new filters/sort.',
'Answer questions ABOUT the data currently loaded on the map — counts, lists, superlatives, and attribute filters over live layers (flights, military, ships, fires, earthquakes, satellites, datacenters, dams). Examples: "how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet", "fastest thing in view". For satellites and infrastructure, counts and ranks cover only bounded examined loaded records; omitted records can change nearest/count. Queries ONLY client-side data from ENABLED layers — if the needed layer is off, say so and offer to enable it. For a follow-up about the previous answer\'s set ("which of those is closest?"), set followUp=true and send only the new filters/sort.',
$position: 1,
parameters: {
properties: {
@@ -549,7 +572,7 @@ export const ACTION_DESCRIPTIONS = {
},
next_iss_pass: {
description:
"When the user asks when the ISS / the space station will next fly over: returns the next visible ISS pass for the current camera location (or an explicit lat/lon) — rise time (ISO + minutes from now), rise compass direction, peak elevation, and duration. Requires the satellites layer to have loaded its catalog at least once this session; if it hasn't, tell the user to enable the satellites layer and try again.",
"When the user asks when the ISS / the space station will next fly over: returns the next geometric ISS pass with estimated visibility for the current camera location (or an explicit lat/lon) — rise time (ISO + minutes from now), rise compass direction, peak elevation, and duration. Requires the satellites layer to have loaded its catalog at least once this session; if it hasn't, tell the user to enable the satellites layer and try again.",
$position: 1,
parameters: {
properties: {
@@ -571,4 +594,9 @@ export const ACTION_DESCRIPTIONS = {
},
},
},
next_satellite_pass: {
description:
'Predict the next pass in 24 hours for one satellite in the loaded catalog, identified by exact NORAD ID or name. Ambiguous names return candidates: ask for a choice rather than selecting one. Defaults to geometric passes; visibleOnly requires estimated illumination and a dark observer sky, which does not guarantee naked-eye visibility. Uses camera location unless coordinates are supplied.',
$position: 1,
},
};
server/providers/overpass.js
+37 -6
View File
@@ -3,6 +3,7 @@ import { readRequestBodyCapped } from './common/request.js';
import {
OVERPASS_MAX_BODY_BYTES,
OVERPASS_MAX_CONCURRENT,
resolveOverpassUpstreams,
} from './overpass/constants.js';
import { sanitizeOverpassBody } from './overpass/query.js';
import {
@@ -17,6 +18,7 @@ import {
import {
overpassPayloadIsData,
fetchOverpassPayload,
overpassNotConfigured,
} from './overpass/transport.js';
import { installRouteMiddleware } from './places/routes.js';
@@ -41,9 +43,16 @@ const _overpassRateLimiter = makeRateLimiter({
function sendOverpassResponse(res, payload, cacheStatus = 'MISS') {
res.writeHead(payload.status, {
'Content-Type': payload.contentType || 'application/json',
'Cache-Control': 'public, max-age=15',
'Cache-Control': overpassPayloadIsData(payload)
? 'public, max-age=15'
: 'no-store',
'X-Overpass-Cache': cacheStatus,
'X-Overpass-Upstream': payload.endpoint || 'unknown',
...(Number.isFinite(payload.cachedAt)
? { 'X-Overpass-Cached-At': new Date(payload.cachedAt).toISOString() }
: {}),
...(payload.retryAfterMs
? { 'Retry-After': String(Math.ceil(payload.retryAfterMs / 1000)) }
: {}),
});
res.end(payload.body || '');
}
@@ -52,7 +61,7 @@ function sendOverpassResponse(res, payload, cacheStatus = 'MISS') {
* Vite plugin: Overpass API proxy with response caching and request coalescing.
*
* Accepts POST requests at /api/overpass, normalizes the query body for
* cache keying, and fans out to multiple Overpass mirrors with per-upstream
* cache keying, and uses only operator-configured endpoints with per-upstream
* timeout and rate-limit detection. Successful responses are cached for
* OVERPASS_CACHE_MS. Concurrent identical queries share a single upstream
* request via the in-flight map.
@@ -66,6 +75,19 @@ function overpassProxy({ routing = {} } = {}) {
// (a body-read failure would otherwise hit an out-of-scope reference).
let cacheKey = null;
try {
// Capability probe: lets clients skip queries when nothing is configured.
if (req.method === 'GET' && req.url?.split('?')[0] === '/status') {
res.writeHead(200, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(
JSON.stringify({
configured: resolveOverpassUpstreams().length > 0,
}),
);
return;
}
if (req.method !== 'POST') {
res.writeHead(405, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
@@ -104,12 +126,21 @@ function overpassProxy({ routing = {} } = {}) {
// Normalize whitespace so semantically identical Overpass QL queries share cache entries
cacheKey = safeBody.replace(/\s+/g, ' ').trim();
if (!resolveOverpassUpstreams().length) {
const cached = await readStaleOverpass(cacheKey);
sendOverpassResponse(
res,
cached || overpassNotConfigured(),
cached ? 'STALE' : 'DISABLED',
);
return;
}
const preflight = await resolveOverpassPreflight({
cacheKey,
memoryCache: _overpassCache,
inFlight: _overpassInFlight,
// Fresh-enough disk entries survive restarts and skip the public
// mirrors; boundary-class queries keep their month-long TTL.
// Fresh-enough disk entries survive restarts and skip upstream
// requests; boundary-class queries keep their month-long TTL.
readDisk: () =>
readOverpassDisk(cacheKey, overpassDiskTtlMs(cacheKey)),
allowUpstream: () => _overpassRateLimiter(clientKey(req)),
@@ -194,7 +225,7 @@ function overpassProxy({ routing = {} } = {}) {
sendOverpassResponse(res, stale, 'STALE');
return;
}
console.error('[Overpass Proxy]', e.message);
console.error('[Overpass Proxy] request failed');
res.writeHead(502, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'Overpass proxy error' }));
}
server/providers/overpass/constants.js
+32 -22
View File
@@ -3,30 +3,39 @@ import path from 'node:path';
// ---------------------------------------------------------------------------
// Overpass API proxy constants and cache state
// ---------------------------------------------------------------------------
/**
* User-Agent sent to every Overpass mirror.
*
* The OSM API usage policy asks for a "Valid User-Agent identifying application
* and version"; a generic proxy label is not one. A mirror is free to refuse a
* client it cannot identify, and `src/overpassProxy.test.mjs` pins what that
* costs: a refusal is never data, so the query falls through to whatever
* mirrors are left. Keep this honest and stable — if it is ever refused, the
* answer is less query volume, not a new name.
*/
/** Stable application identity for operator-configured Overpass instances. */
const OVERPASS_USER_AGENT =
'gods-eye-view/0.1 (+https://github.com/bilawalsidhu/gods-eye-view)';
/** Ordered list of Overpass API mirrors; tried sequentially on failure/rate-limit. */
const OVERPASS_UPSTREAMS = [
'https://overpass-api.de/api/interpreter',
'https://overpass.kumi.systems/api/interpreter',
'https://lz4.overpass-api.de/api/interpreter',
// Community full-planet instance (privateforge nonprofit) — added 2026-07-30
// when all three mirrors above refused this IP (likely a dev-traffic rate
// ban; refused connections fail in ms, so healthy mirrors above still win).
// Verified: planet coverage (Texas query), CORS *, ~5-20 s cold latency.
'https://overpass.private.coffee/api/interpreter',
];
/** Parse only operator-supplied HTTP(S) endpoints; private instances are allowed. */
function parseOverpassUpstreams(raw) {
const endpoints = [];
for (const token of String(raw || '').split(',')) {
try {
const url = new URL(token.trim());
if (
!['http:', 'https:'].includes(url.protocol) ||
!url.hostname ||
url.hash
)
continue;
if (!endpoints.includes(url.href)) endpoints.push(url.href);
} catch {
// Invalid configuration never becomes an upstream or appears in logs.
}
}
return endpoints.slice(0, 8);
}
let upstreamMemo = { raw: null, endpoints: [] };
/** Resolve after environment loading. Public Overpass instances are not used by default. */
function resolveOverpassUpstreams() {
const raw = process.env.OVERPASS_UPSTREAMS || '';
if (upstreamMemo.raw !== raw)
upstreamMemo = { raw, endpoints: parseOverpassUpstreams(raw) };
return [...upstreamMemo.endpoints];
}
/**
* TTL for FRESH cached Overpass responses (ms). Road geometry is static for
@@ -142,7 +151,8 @@ export {
OVERPASS_SIMPLIFY_MIN_POINTS,
OVERPASS_SIMPLIFY_TOLERANCE_DEG,
OVERPASS_MAX_RESPONSE_BYTES,
OVERPASS_UPSTREAMS,
parseOverpassUpstreams,
resolveOverpassUpstreams,
OVERPASS_USER_AGENT,
OVERPASS_TIMEOUT_MS,
};
server/providers/overpass/transport.js
+110 -62
View File
@@ -1,6 +1,6 @@
import {
OVERPASS_MAX_RESPONSE_BYTES,
OVERPASS_UPSTREAMS,
resolveOverpassUpstreams,
OVERPASS_USER_AGENT,
OVERPASS_TIMEOUT_MS,
} from './constants.js';
@@ -60,99 +60,147 @@ function overpassPayloadIsData(payload) {
);
}
const cooldowns = new Map();
/** A stable, non-retryable capability response shared by every Overpass route. */
export function overpassNotConfigured() {
return {
status: 503,
contentType: 'application/json',
body: JSON.stringify({
error: 'Detailed OpenStreetMap queries are not configured',
code: 'OVERPASS_NOT_CONFIGURED',
retryable: false,
}),
};
}
/** Honor Retry-After dates/seconds; absent values use bounded exponential backoff. */
function retryDelay(value, failures, now) {
const seconds = Number(value);
const explicit =
value && Number.isFinite(seconds)
? seconds * 1000
: Date.parse(value) - now;
return Number.isFinite(explicit)
? Math.max(1000, explicit)
: Math.min(300_000, 30_000 * 2 ** Math.min(failures, 4));
}
function refusal(status, retryAfterMs) {
return {
status,
contentType: 'application/json',
rateLimited: status === 429 || status === 406,
retryAfterMs,
body: JSON.stringify({
error: 'Configured Overpass upstream unavailable',
code: 'OVERPASS_UNAVAILABLE',
retryable: true,
retryAfterMs,
}),
};
}
/**
* Try each mirror once, retaining response-size and per-mirror timeout caps.
* Refusals and body-level failures rotate; total failure returns the last
* rate-limit payload, otherwise the first refusal, or throws a network error.
* @param {string} body URL-encoded Overpass QL query body.
* @param {number} [maxResponseBytes] Endpoint-specific response cap.
* @param {object} [options] Server-only endpoint and I/O overrides for tests.
* @returns {Promise<{status:number,body:string,contentType:string,endpoint:string,rateLimited:boolean}>}
* Query only the configured chain with capped reads, timeouts and per-endpoint cooldowns.
* Explicit endpoint/I/O overrides are server-only test seams. Empty data is valid.
*/
async function fetchOverpassPayload(
body,
maxResponseBytes = OVERPASS_MAX_RESPONSE_BYTES,
{
endpoints = OVERPASS_UPSTREAMS,
endpoints = resolveOverpassUpstreams(),
fetchImpl = fetch,
readBody = readResponseTextCapped,
simplify = simplifyOverpassPayloadBody,
now = Date.now,
} = {},
) {
let lastError = null;
let lastRateLimitPayload = null;
let lastRefusalPayload = null;
if (!endpoints.length) return overpassNotConfigured();
let failure = refusal(502, 30_000);
for (const endpoint of endpoints) {
const previous = cooldowns.get(endpoint);
if (previous?.until > now()) {
failure = refusal(previous.status, previous.until - now());
continue;
}
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), OVERPASS_TIMEOUT_MS);
try {
const upstream = await fetchImpl(endpoint, {
const requestUrl = new URL(endpoint);
const authorization =
requestUrl.username || requestUrl.password
? 'Basic ' +
Buffer.from(
`${decodeURIComponent(requestUrl.username)}:${decodeURIComponent(requestUrl.password)}`,
).toString('base64')
: null;
requestUrl.username = '';
requestUrl.password = '';
const upstream = await fetchImpl(requestUrl.href, {
method: 'POST',
redirect: 'error',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'User-Agent': OVERPASS_USER_AGENT,
...(authorization ? { Authorization: authorization } : {}),
},
body,
signal: controller.signal,
});
const responseBody = await readBody(upstream, maxResponseBytes);
const contentType =
upstream.headers.get('content-type') || 'application/json';
const status = upstream.status;
const rateLimited =
status === 429 || overpassLooksRateLimited(responseBody);
const runtimeError = overpassLooksRuntimeError(responseBody);
const payload = {
status,
body: responseBody,
contentType,
endpoint,
rateLimited,
runtimeError,
};
if (rateLimited) {
lastRateLimitPayload = payload;
continue;
}
// A 200 body carrying a runtime error / timeout is a transient upstream
// failure — skip to the next mirror rather than returning or caching it.
if (runtimeError) {
lastError = new Error(`Overpass runtime error (${endpoint})`);
continue;
}
// Anything but 2xx is this mirror declining, not an answer. Only 5xx used
// to rotate, so a 4xx ended the fan-out and was returned — and cached —
// as data: a mirror refusing this client answers 406 while the others
// answer 200 to the very same request, so every Overpass-backed layer
// failed on an error page with healthy mirrors untried. The first
// refusal is kept so a genuinely bad query still reports what upstream
// said, but only after every mirror has had the chance to answer it.
if (status < 200 || status >= 300) {
if (!lastRefusalPayload) lastRefusalPayload = payload;
lastError = new Error(
`Overpass upstream returned ${status} (${endpoint})`,
upstream.status === 429 ||
upstream.status === 406 ||
overpassLooksRateLimited(responseBody);
if (
rateLimited ||
upstream.status < 200 ||
upstream.status >= 300 ||
overpassLooksRuntimeError(responseBody)
) {
const failures = (previous?.failures || 0) + 1;
const delay = retryDelay(
upstream.headers.get('retry-after'),
failures - 1,
now(),
);
const status = rateLimited
? upstream.status === 406
? 406
: 429
: 502;
cooldowns.set(endpoint, { until: now() + delay, failures, status });
while (cooldowns.size > 64)
cooldowns.delete(cooldowns.keys().next().value);
failure = refusal(status, delay);
continue;
}
// Success: decimate giant boundary geometry before it reaches the cache,
// the disk, or the client (what makes the 32 MB read cap safe to hold).
payload.body = simplify(payload.body);
return payload;
} catch (error) {
lastError = error;
const parsed = JSON.parse(responseBody);
if (!Array.isArray(parsed?.elements) || parsed.remark)
throw new Error('Malformed Overpass response');
cooldowns.delete(endpoint);
return {
status: upstream.status,
body: simplify(responseBody),
contentType: 'application/json',
// Never retain a secret-bearing endpoint in cache or response metadata.
endpoint: 'configured',
rateLimited: false,
};
} catch {
const failures = (previous?.failures || 0) + 1;
const delay = retryDelay(null, failures - 1, now());
cooldowns.set(endpoint, { until: now() + delay, failures, status: 502 });
while (cooldowns.size > 64)
cooldowns.delete(cooldowns.keys().next().value);
failure = refusal(502, delay);
} finally {
clearTimeout(timeoutId);
}
}
if (lastRateLimitPayload) return lastRateLimitPayload;
if (lastRefusalPayload) return lastRefusalPayload;
throw lastError || new Error('All Overpass upstreams failed');
return failure;
}
export { overpassPayloadIsData, fetchOverpassPayload };
server/providers/regional/place.js
+7 -30
View File
@@ -1,7 +1,7 @@
import { makeRateLimiter, clientKey } from '../common/rate-limit.js';
import { coalesceProxyRequest } from '../common/http.js';
import { fetchRegionalJson } from './http.js';
import { normalizeRegionalPlace } from '../../../src/data/regionalModel.js';
import { naturalRegionAtPoint } from '../../../src/data/naturalEarthRegions.js';
import {
nominatimToGeocodeResult,
nominatimViewboxFromBounds,
@@ -51,8 +51,8 @@ const NOMINATIM_SEARCH_MAX_QUERY = 200;
// The pacer is deliberately module state while everything else here is
// per-instance. One request per second is a budget for the whole application,
// not for each provider object: two instances each pacing themselves would
// send two requests a second between them. Reverse lookups and searches
// therefore queue together.
// send two requests a second between them. Only last-resort forward searches
// use this queue.
let _nominatimQueue = Promise.resolve();
let _nominatimLastRequestAt = 0;
@@ -77,9 +77,7 @@ function abandonedError() {
* Run one piece of upstream work, never closer than the policy spacing to the
* last one.
*
* `bounded` applies the queue-depth and staleness limits. The cockpit's reverse
* lookups are a slow background trickle and stay unbounded; the search route,
* which a person can fire as fast as they can type, does not.
* `bounded` applies queue-depth and staleness limits to interactive forward searches.
*
* @param {() => Promise<unknown>} work
* @param {{bounded?: boolean, signal?: AbortSignal}} [options]
@@ -111,30 +109,9 @@ function enqueueNominatim(work, { bounded = false, signal } = {}) {
return task;
}
/** Construct the serialized Nominatim reverse adapter with a trusted endpoint. */
export function createRegionalPlaceProvider({
endpoint = 'https://nominatim.openstreetmap.org/reverse',
requestJson = fetchRegionalJson,
} = {}) {
function fetchRegionalPlace(point) {
return enqueueNominatim(async () => {
const params = new URLSearchParams({
format: 'jsonv2',
lat: point.latitude.toFixed(5),
lon: point.longitude.toFixed(5),
zoom: '10',
addressdetails: '1',
'accept-language': 'en',
});
const payload = await requestJson(`${endpoint}?${params}`, {
headers: NOMINATIM_HEADERS,
redirect: 'error',
});
return normalizeRegionalPlace(payload);
});
}
return fetchRegionalPlace;
/** Construct the offline regional context provider from bundled Natural Earth polygons. */
export function createRegionalPlaceProvider() {
return (point) => naturalRegionAtPoint(point.latitude, point.longitude);
}
export const fetchRegionalPlace = createRegionalPlaceProvider();
server/providers/weather.js
+610
View File
@@ -0,0 +1,610 @@
import { readResponseTextCapped } from './common/http.js';
import { readWindBody as readBytesCapped } from '../../src/sources/windBody.js';
const BASE = 'https://nowcoast.noaa.gov/geoserver/observations/';
const HOUR = 3600_000;
const PRODUCTS = Object.freeze({
lightning: Object.freeze({
service: 'lightning_detection',
layer: 'ldn_lightning_strike_density',
style: 'lightning_density',
title: 'Lightning density · 15 min',
coverage:
'Pacific and Americas: 110°E across the dateline to 0°, 25°S–80°N; not global coverage.',
description:
'Observed 15-minute lightning strike density on an approximately 8 km grid, scaled as strikes/km²/min ×10³. Ground-network density, not individual GLM flashes.',
attribution: 'NOAA/NWS nowCOAST; derived from Vaisala NLDN/GLD360',
metadataTtlMs: 600_000,
image: Object.freeze({ width: 4096, height: 2048 }),
}),
radar: Object.freeze({
service: 'weather_radar',
layer: 'conus_base_reflectivity_mosaic',
style: 'weather_radar_base_reflectivity',
title: 'CONUS radar reflectivity',
coverage:
'Contiguous United States; gaps do not establish absence of precipitation.',
description:
'Observed MRMS radar base reflectivity (dBZ), approximately 1 km and 4-minute updates; not a rainfall forecast.',
image: Object.freeze({ width: 4096, height: 2048 }),
}),
clouds: Object.freeze({
service: 'satellite',
layer: 'global_longwave_imagery_mosaic',
style: 'reflectance',
title: 'Global satellite infrared',
coverage:
'Global mosaic with incomplete polar coverage; nominal coverage 60°S–60°N.',
description:
'Longwave infrared cloud and land/sea temperature patterns, approximately 3 km; hourly updates with 2–3 hour source latency. Not a cloud-only mask.',
image: Object.freeze({ width: 2048, height: 1024 }),
}),
'clouds-regional': Object.freeze({
service: 'satellite',
layer: 'goes_longwave_imagery',
style: 'goes-lir',
title: 'GOES regional satellite infrared',
coverage:
'GOES East/West regional North American coverage; not a global image.',
description:
'GOES-19/18 longwave infrared Band 14 cloud and surface temperature patterns, approximately 2 km and 5-minute updates. Not a cloud-only mask.',
image: Object.freeze({ width: 4096, height: 2048 }),
}),
});
// Whole-extent images: 2:1 sizes up to each product's largest (the default).
const IMAGE_SIZES = Object.freeze(['1024x512', '2048x1024', '4096x2048']);
// Detail windows: any product up to 4096×2048 (the default).
const DETAIL_IMAGE = Object.freeze({ width: 4096, height: 2048 });
const BBOX_STEP = 0.25;
const MAX_IMAGE_BYTES = 16 * 1024 * 1024;
function failure(code, status = 503) {
return Object.assign(new Error(code), { code, status });
}
/** Canonicalize only explicit UTC observations; never expand time intervals. */
function observationTime(value) {
if (
typeof value !== 'string' ||
!/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{3})?Z$/.test(value)
)
return null;
const date = new Date(value);
if (!Number.isFinite(date.getTime())) return null;
const canonical = date.toISOString();
return canonical.replace('.000Z', 'Z') === value.replace('.000Z', 'Z')
? canonical
: null;
}
/** Read the named leaf from bounded capabilities without resolving XML entities. */
export function parseWeatherCapabilities(xml, product, nowMs = Date.now()) {
const spec = PRODUCTS[product];
if (
!spec ||
typeof xml !== 'string' ||
xml.length > 512 * 1024 ||
/<!DOCTYPE|<!ENTITY/i.test(xml)
)
throw failure('invalid_weather_metadata');
const stack = [];
let leaf = null;
let tags = 0;
for (const match of xml.matchAll(/<\/?(?:[\w.-]+:)?Layer\b[^>]*>/g)) {
if (++tags > 2048) throw failure('invalid_weather_metadata');
if (!match[0].startsWith('</')) {
if (stack.length) stack.at(-1).nested = true;
if (stack.length >= 16 || match[0].endsWith('/>'))
throw failure('invalid_weather_metadata');
stack.push({ start: match.index + match[0].length, nested: false });
} else {
const opened = stack.pop();
if (!opened) throw failure('invalid_weather_metadata');
if (opened.nested) continue;
const candidate = xml.slice(opened.start, match.index);
const name = candidate
.match(/<(?:[\w.-]+:)?Name\s*>([^<]+)<\/(?:[\w.-]+:)?Name>/)?.[1]
?.trim();
if (name === spec.layer) {
if (leaf !== null) throw failure('invalid_weather_metadata');
leaf = candidate;
}
}
}
if (stack.length || !leaf) throw failure('invalid_weather_metadata');
const box = leaf.match(
/<(?:[\w.-]+:)?EX_GeographicBoundingBox\s*>([\s\S]*?)<\/(?:[\w.-]+:)?EX_GeographicBoundingBox>/,
)?.[1];
const bounds = {};
for (const [key, tag] of Object.entries({
west: 'westBoundLongitude',
south: 'southBoundLatitude',
east: 'eastBoundLongitude',
north: 'northBoundLatitude',
})) {
const value = box
?.match(
new RegExp(`<(?:[\\w.-]+:)?${tag}\\s*>([^<]+)</(?:[\\w.-]+:)?${tag}>`),
)?.[1]
?.trim();
bounds[key] = value ? Number(value) : NaN;
}
if (
!Object.values(bounds).every(Number.isFinite) ||
bounds.west < -180 ||
bounds.east > 180 ||
bounds.south < -90 ||
bounds.north > 90 ||
bounds.west >= bounds.east ||
bounds.south >= bounds.north
)
throw failure('invalid_weather_metadata');
const dimensions = [
...leaf.matchAll(
/<(?:[\w.-]+:)?Dimension\b([^>]*)>([^<]*)<\/(?:[\w.-]+:)?Dimension>/g,
),
].filter((entry) => /\bname\s*=\s*["']time["']/.test(entry[1]));
if (
dimensions.length !== 1 ||
!/\bunits\s*=\s*["']ISO8601["']/.test(dimensions[0][1])
)
throw failure('invalid_weather_metadata');
const raw = dimensions[0][2].trim().split(',');
if (!raw.length || raw.length > 512)
throw failure('invalid_weather_metadata');
const times = raw.map((value) => observationTime(value.trim()));
if (times.some((value) => !value || Date.parse(value) > nowMs + 5 * 60_000))
throw failure('invalid_weather_metadata');
const defaultTime = observationTime(
dimensions[0][1].match(/\bdefault\s*=\s*["']([^"']+)["']/)?.[1],
);
if (!defaultTime || !times.includes(defaultTime))
throw failure('invalid_weather_metadata');
const recent = [...new Set(times)]
.filter((value) => nowMs - Date.parse(value) <= 24 * HOUR)
.sort()
.slice(-26);
if (!recent.length) throw failure('weather_observations_expired');
return { bounds, times: recent.slice(-13), allowedTimes: recent };
}
/** A detail window `west,south,east,north` in degrees, rounded to 0.25° so cache
* keys repeat, with a 2:1 aspect within 1 %; null when absent. Containment in
* the product bounds is checked once the metadata is known. */
export function weatherImageBbox(value) {
if (value === null) return null;
const parts = value.split(',');
if (
parts.length !== 4 ||
parts.some((part) => !/^-?\d{1,3}(?:\.\d{1,6})?$/.test(part))
)
throw failure('invalid_weather_bbox', 400);
// `+ 0` turns a rounded -0 into 0 so equal windows share one key.
const [west, south, east, north] = parts.map(
(part) => Math.round(Number(part) / BBOX_STEP) * BBOX_STEP + 0,
);
if (
west < -180 ||
east > 180 ||
south < -90 ||
north > 90 ||
west >= east ||
south >= north ||
Math.abs((east - west) / (north - south) / 2 - 1) > 0.01
)
throw failure('invalid_weather_bbox', 400);
return [west, south, east, north];
}
/** GeographicTilingScheme's two longitude tiles and one latitude tile at level zero. */
export function weatherTileBounds(z, x, y) {
if (
![z, x, y].every(Number.isInteger) ||
z < 0 ||
z > 6 ||
x < 0 ||
y < 0 ||
x >= 2 ** (z + 1) ||
y >= 2 ** z
)
throw failure('invalid_weather_tile', 400);
const span = 180 / 2 ** z;
return [
-180 + x * span,
90 - (y + 1) * span,
-180 + (x + 1) * span,
90 - y * span,
];
}
function validatePng(bytes, width, height) {
const buffer = Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength);
if (
buffer.length < 33 ||
!buffer
.subarray(0, 8)
.equals(Buffer.from([137, 80, 78, 71, 13, 10, 26, 10])) ||
buffer.readUInt32BE(8) !== 13 ||
buffer.toString('ascii', 12, 16) !== 'IHDR' ||
buffer.readUInt32BE(16) !== width ||
buffer.readUInt32BE(20) !== height
)
throw failure('invalid_weather_image');
return buffer;
}
/** Fixed NOAA observed-weather metadata and WMS tiles; no client-supplied destinations. */
export function weatherProxy({
fetchImpl = fetch,
now = () => Date.now(),
timeoutMs = 12_000,
} = {}) {
const metadata = new Map();
const attempts = new Map();
const operations = new Map();
const pending = [];
const tiles = new Map();
const tileFailures = new Map();
let tileBytes = 0;
let active = 0;
function pump() {
while (active < 8 && pending.length) {
const operation = pending.shift();
if (operation.controller.signal.aborted) continue;
active++;
operation.started = true;
Promise.resolve()
.then(() => operation.work(operation.controller.signal))
.then(operation.resolve, operation.reject)
.finally(() => {
active--;
pump();
});
}
}
async function shared(key, work, clientSignal) {
clientSignal.throwIfAborted();
let operation = operations.get(key);
if (operation?.controller.signal.aborted) {
operations.delete(key);
operation = null;
}
if (!operation) {
if (operations.size >= 120 || pending.length >= 96)
throw failure('weather_busy', 429);
const controller = new AbortController();
operation = { controller, work, waiters: 0, started: false };
operation.promise = new Promise((resolve, reject) => {
operation.resolve = resolve;
operation.reject = reject;
});
const cancel = () => {
const index = pending.indexOf(operation);
if (index >= 0) pending.splice(index, 1);
operation.reject(failure('weather_request_cancelled'));
};
controller.signal.addEventListener('abort', cancel, { once: true });
const timer = setTimeout(() => controller.abort(), timeoutMs);
operation.promise = operation.promise.finally(() => {
clearTimeout(timer);
controller.signal.removeEventListener('abort', cancel);
if (operations.get(key) === operation) operations.delete(key);
});
operations.set(key, operation);
pending.push(operation);
pump();
}
operation.waiters++;
const abort = () => {
if (--operation.waiters === 0) operation.controller.abort();
};
clientSignal.addEventListener('abort', abort, { once: true });
try {
const value = await operation.promise;
clientSignal.throwIfAborted();
return value;
} finally {
clientSignal.removeEventListener('abort', abort);
if (!clientSignal.aborted) operation.waiters--;
}
}
async function upstream(url, signal, image = null) {
signal.throwIfAborted();
const response = await fetchImpl(url, {
signal,
redirect: 'error',
headers: { Accept: image ? 'image/png' : 'application/xml,text/xml' },
});
if (
!response.ok ||
(image &&
!/^image\/png(?:;|$)/i.test(response.headers.get('content-type') || ''))
) {
await response.body?.cancel();
throw failure('weather_upstream_unavailable');
}
const value = image
? validatePng(
await readBytesCapped(response, image.maxBytes, signal),
image.width,
image.height,
)
: await readResponseTextCapped(response, 512 * 1024, signal);
signal.throwIfAborted();
return value;
}
async function getMetadata(product, signal) {
signal.throwIfAborted();
const old = metadata.get(product);
const spec = PRODUCTS[product];
if (old && now() - old.fetchedAt < (spec.metadataTtlMs ?? 120_000))
return { ...old, stale: false };
try {
if (
now() - (attempts.get(product) ?? -Infinity) < 30_000 &&
!operations.has(`metadata:${spec.service}`)
)
throw failure('weather_upstream_unavailable');
attempts.set(product, now());
const url = `${BASE}${spec.service}/ows?service=WMS&version=1.3.0&request=GetCapabilities`;
const xml = await shared(
`metadata:${spec.service}`,
(activeSignal) => upstream(url, activeSignal),
signal,
);
const value = {
...parseWeatherCapabilities(xml, product, now()),
fetchedAt: now(),
};
metadata.set(product, value);
return { ...value, stale: false };
} catch (error) {
if (signal.aborted) attempts.delete(product);
signal.throwIfAborted();
if (error.status === 429) throw error;
if (old && now() - old.fetchedAt <= HOUR) return { ...old, stale: true };
throw failure('weather_metadata_unavailable');
}
}
function rememberTile(key, bytes) {
if (tiles.has(key)) tileBytes -= tiles.get(key).bytes.length;
tiles.delete(key);
while (tiles.size >= 128 || tileBytes + bytes.length > 16 * 1024 * 1024) {
const oldest = tiles.keys().next().value;
tileBytes -= tiles.get(oldest).bytes.length;
tiles.delete(oldest);
}
tiles.set(key, { bytes, at: now() });
tileBytes += bytes.length;
}
function describe(product, value) {
const spec = PRODUCTS[product];
const time = value?.times.at(-1) ?? null;
return {
schemaVersion: 1,
product,
title: spec.title,
coverage: spec.coverage,
description: spec.description,
source: 'NOAA nowCOAST',
attribution: spec.attribution ?? 'NOAA/NWS/NESDIS nowCOAST',
bounds: value?.bounds ?? null,
times: value?.times ?? [],
latest: time,
time,
observedAt: time,
fetchedAt: value?.fetchedAt ?? null,
stale: value?.stale ?? true,
unavailable: !value,
reason: !value
? 'Weather imagery unavailable'
: value.stale
? 'Cached weather metadata; upstream unavailable'
: null,
tileSize: 256,
maxLevel: 6,
tilingScheme: 'geographic',
tileTemplate: time
? `/api/weather/tile?product=${product}&time=${encodeURIComponent(time)}&z={z}&x={x}&y={y}`
: null,
imageUrl: time
? `/api/weather/image?product=${product}&time=${encodeURIComponent(time)}`
: null,
imageSize: { ...spec.image },
};
}
function json(res, status, body) {
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(status === 429 ? { 'Retry-After': '2' } : {}),
});
res.end(JSON.stringify(body));
}
async function handler(req, res) {
const controller = new AbortController();
const close = () => controller.abort();
res.once?.('close', close);
try {
const url = new URL(req.url, 'http://localhost');
if (req.method !== 'GET')
return json(res, 405, { error: 'method_not_allowed' });
if (!['/manifest', '/tile', '/image'].includes(url.pathname))
return json(res, 404, { error: 'not_found' });
const allowed =
url.pathname === '/manifest'
? ['product']
: url.pathname === '/image'
? ['product', 'time', 'size', 'bbox']
: ['product', 'time', 'z', 'x', 'y', 'size'];
if (
req.url.length > 512 ||
[...url.searchParams.keys()].some(
(key) =>
!allowed.includes(key) || url.searchParams.getAll(key).length !== 1,
)
)
return json(res, 400, { error: 'invalid_weather_query' });
const product = url.searchParams.get('product');
if (!Object.hasOwn(PRODUCTS, product))
return json(res, 400, { error: 'unknown_weather_product' });
const wholeImage = url.pathname === '/image';
const detailBox = wholeImage
? weatherImageBbox(url.searchParams.get('bbox'))
: null;
const largest = detailBox ? DETAIL_IMAGE : PRODUCTS[product].image;
const size =
url.searchParams.get('size') ??
(wholeImage ? `${largest.width}x${largest.height}` : '256');
if (
wholeImage
? !IMAGE_SIZES.includes(size) ||
Number.parseInt(size, 10) > largest.width
: !['256', '512', '1024'].includes(size)
)
throw failure(
wholeImage
? 'invalid_weather_image_size'
: 'invalid_weather_tile_size',
400,
);
if (url.pathname === '/manifest') {
try {
const value = await getMetadata(product, controller.signal);
controller.signal.throwIfAborted();
return json(res, 200, describe(product, value));
} catch (error) {
if (controller.signal.aborted) return;
if (error.status === 429) throw error;
return json(res, 200, describe(product, null));
}
}
const coords = wholeImage
? null
: ['z', 'x', 'y'].map((key) => url.searchParams.get(key));
if (coords?.some((value) => !/^(?:0|[1-9]\d{0,2})$/.test(value ?? '')))
throw failure('invalid_weather_tile', 400);
const tileBounds = coords
? weatherTileBounds(...coords.map(Number))
: null;
const time = url.searchParams.get('time');
if (!time || observationTime(time) !== time)
throw failure('invalid_weather_time', 400);
const value = await getMetadata(product, controller.signal);
controller.signal.throwIfAborted();
if (
!value.allowedTimes.includes(time) ||
now() - Date.parse(time) > 24 * HOUR
)
throw failure('unknown_weather_time', 400);
if (
detailBox &&
(detailBox[0] < value.bounds.west ||
detailBox[1] < value.bounds.south ||
detailBox[2] > value.bounds.east ||
detailBox[3] > value.bounds.north)
)
throw failure('invalid_weather_bbox', 400);
// A single advertised extent keeps NOAA's global reflectance contrast
// consistent across the image; independently normalized tiles create seams.
const bbox =
detailBox ??
(wholeImage
? [
value.bounds.west,
value.bounds.south,
value.bounds.east,
value.bounds.north,
]
: tileBounds);
const [width, height] = wholeImage
? size.split('x').map(Number)
: [Number(size), Number(size)];
const imageShape = {
width,
height,
maxBytes: wholeImage
? MAX_IMAGE_BYTES
: Math.max(1024 * 1024, width ** 2 * 4 + 65_536),
};
// NOAA WMS capabilities advertise no MaxWidth/MaxHeight. One GetMap
// returns a whole 4096×2048 extent or window, or a 1024 px tile, without
// composition. Images are keyed by product, time, size and bbox.
const key = `${product}:${time}:${wholeImage ? `image:${size}:${bbox.join(',')}` : `tile:${size}:${coords.join('/')}`}`;
let cached = tiles.get(key);
if (cached && now() - cached.at <= 24 * HOUR) {
tiles.delete(key);
tiles.set(key, cached);
} else {
// Never ask nearestValue=1 WMS to silently replace an expired frame.
if (value.stale) throw failure('weather_metadata_stale');
if (now() - (tileFailures.get(key) ?? -Infinity) < 30_000)
throw failure('weather_upstream_unavailable');
const spec = PRODUCTS[product];
const upstreamUrl = new URL(`${BASE}${spec.service}/ows`);
upstreamUrl.search = new URLSearchParams({
service: 'WMS',
version: '1.1.1',
request: 'GetMap',
layers: spec.layer,
styles: spec.style,
srs: 'EPSG:4326',
bbox: bbox.join(','),
width: String(imageShape.width),
height: String(imageShape.height),
format: 'image/png',
transparent: 'true',
time,
}).toString();
try {
const bytes = await shared(
`image:${key}`,
(signal) => upstream(upstreamUrl.href, signal, imageShape),
controller.signal,
);
rememberTile(key, bytes);
cached = { bytes };
} catch (error) {
if (!controller.signal.aborted && error.status !== 429) {
tileFailures.delete(key);
tileFailures.set(key, now());
while (tileFailures.size > 128)
tileFailures.delete(tileFailures.keys().next().value);
}
throw error;
}
}
res.writeHead(200, {
'Content-Type': 'image/png',
'Cache-Control': 'public, max-age=86400, immutable',
'X-Content-Type-Options': 'nosniff',
});
res.end(cached.bytes);
} catch (error) {
if (!controller.signal.aborted)
json(
res,
error.status === 400 || error.status === 429 ? error.status : 503,
{
error:
error.status === 400 || error.status === 429
? error.code
: 'weather_upstream_unavailable',
},
);
} finally {
res.removeListener?.('close', close);
}
}
return {
name: 'weather',
configureServer({ middlewares }) {
middlewares.use('/api/weather', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/weather', handler);
},
};
}
server/providers/wind.js
+227
View File
@@ -0,0 +1,227 @@
import { fetchGfsWind } from './wind/gfs.js';
import { fetchIfsWind } from './wind/ifs.js';
import { decodeWindGribMessage } from './wind/decode.js';
import { weatherScalarMetadata, weatherScalarError } from './wind/grid.js';
/** Serve bounded, single-flight, per-model/overlay forecast snapshots from fixed providers. */
export function windProxy({
fetchImpl = fetch,
now = () => Date.now(),
decodeImpl = decodeWindGribMessage,
targetDx = 1,
ttlMs = 3600_000,
timeoutMs = 40_000,
models = { gfs: fetchGfsWind, ifs: fetchIfsWind },
} = {}) {
// Request validation limits these maps to 2 models × 3 overlays.
const caches = new Map();
const loadings = new Map();
const grids = new Map();
const attempts = new Map();
const unavailable = (model, overlay) => ({
manifest: {
model,
schemaVersion: 1,
...(overlay === 'none' ? {} : { overlay }),
unavailable: true,
stale: true,
reason: 'Wind upstream unavailable',
},
});
const sendJson = (res, value, status = 200) => {
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify(value));
};
function refresh(model, overlay, key) {
const controller = new AbortController();
const operation = { controller, waiters: 0, promise: null };
loadings.set(key, operation);
attempts.set(key, now());
operation.promise = (async () => {
const timer = setTimeout(() => controller.abort(), timeoutMs);
try {
const value = await models[model]({
fetchImpl,
now,
decodeImpl,
targetDx,
overlay,
signal: controller.signal,
});
controller.signal.throwIfAborted();
const { grid, cycle } = value;
const count = grid.nx * grid.ny;
const expectedScalar = weatherScalarMetadata(overlay);
const missingScalar =
expectedScalar &&
value.scalar === undefined &&
value.scalarError === weatherScalarError(overlay);
const scalar = missingScalar ? null : expectedScalar;
if (
!Number.isInteger(grid.nx) ||
!Number.isInteger(grid.ny) ||
grid.nx < 1 ||
grid.ny < 1 ||
count > 1_000_000 ||
![grid.lo1, grid.la1, grid.dx, grid.dy].every(Number.isFinite) ||
grid.dx <= 0 ||
grid.dy <= 0 ||
(scalar
? ['kind', 'units', 'level'].some(
(field) => value.scalar?.[field] !== scalar[field],
)
: value.scalar !== undefined || grid.scalar !== undefined) ||
(!expectedScalar && value.scalarError !== undefined) ||
(scalar && value.scalarError !== undefined)
)
throw new Error('Invalid wind grid');
for (const values of [
grid.u,
grid.v,
...(scalar ? [grid.scalar] : []),
]) {
if (
!(values instanceof Float32Array) ||
values.length !== count ||
!values.every(Number.isFinite)
)
throw new Error('Invalid wind grid');
}
const suffix =
overlay === 'none'
? ''
: `-${overlay}${missingScalar ? '-wind-only' : ''}`;
const query = `model=${model}${overlay === 'none' ? '' : `&overlay=${overlay}`}`;
const id = `${model}-${cycle.date}-${cycle.hour}-f${cycle.forecastHour || 0}-${targetDx}${suffix}`;
const manifest = {
schemaVersion: 1,
model,
cycle,
...(expectedScalar ? { overlay } : {}),
...(scalar ? { scalar } : {}),
...(missingScalar
? { scalarError: weatherScalarError(overlay) }
: {}),
fetchedAt: now(),
level: value.level,
units: value.units,
grid: {
nx: grid.nx,
ny: grid.ny,
lo1: grid.lo1,
la1: grid.la1,
dx: grid.dx,
dy: grid.dy,
},
stale: false,
unavailable: false,
reason: null,
gridUrl: `/api/wind/grid/${id}.bin?${query}`,
};
const state = { id, grid, manifest, fetchedAt: now() };
caches.set(key, state);
// Retain the previous issued grid as well to cover a manifest/grid rollover.
const history = grids.get(key) || new Map();
history.set(id, state);
while (history.size > 2) history.delete(history.keys().next().value);
grids.set(key, history);
return state;
} catch {
if (controller.signal.aborted && operation.waiters === 0) {
attempts.delete(key);
return unavailable(model, overlay);
}
const old = caches.get(key);
if (old) {
old.manifest = {
...old.manifest,
stale: true,
reason: 'Wind upstream unavailable',
};
return old;
}
return unavailable(model, overlay);
} finally {
clearTimeout(timer);
controller.abort(); // Cancel a sibling request if the other component failed.
if (loadings.get(key) === operation) loadings.delete(key);
}
})();
return operation;
}
const handler = async (req, res) => {
const url = new URL(req.url, 'http://localhost');
const model = url.searchParams.get('model') || 'gfs';
const overlay = url.searchParams.get('overlay') || 'none';
const key = `${model}:${overlay}`;
if (req.method !== 'GET')
return sendJson(res, { error: 'method_not_allowed' }, 405);
if (!['gfs', 'ifs'].includes(model) || !Object.hasOwn(models, model))
return sendJson(res, { error: 'unknown_model' }, 400);
if (!['none', 'temperature', 'pressure'].includes(overlay))
return sendJson(res, { error: 'unknown_overlay' }, 400);
if (url.pathname.startsWith('/grid/')) {
const id = url.pathname.slice(6).replace(/\.bin$/, '');
const state =
url.pathname === `/grid/${id}.bin` && grids.get(key)?.get(id);
if (!state) return sendJson(res, { error: 'unknown_grid' }, 404);
const components = [
state.grid.u,
state.grid.v,
...(state.grid.scalar ? [state.grid.scalar] : []),
];
const bytes = Buffer.concat(
components.map((values) =>
Buffer.from(values.buffer, values.byteOffset, values.byteLength),
),
);
res.writeHead(200, {
'Content-Type': 'application/octet-stream',
'Cache-Control': 'public, max-age=3600, immutable',
});
return res.end(bytes);
}
if (!['/', '/manifest', '/status'].includes(url.pathname))
return sendJson(res, { error: 'not_found' }, 404);
let state = caches.get(key);
if (!state || now() - state.fetchedAt >= ttlMs) {
let operation = loadings.get(key);
if (!operation && now() - (attempts.get(key) ?? -Infinity) >= 60_000)
operation = refresh(model, overlay, key);
if (operation) {
let disconnected = false;
operation.waiters += 1;
const close = () => {
disconnected = true;
if (--operation.waiters === 0) operation.controller.abort();
};
res.once?.('close', close);
try {
state = await operation.promise;
} finally {
res.removeListener?.('close', close);
if (!disconnected) operation.waiters -= 1;
}
if (disconnected) return;
}
}
const manifest = state?.manifest || unavailable(model, overlay).manifest;
if (url.pathname === '/status') {
const { gridUrl, ...status } = manifest;
return sendJson(res, status);
}
return sendJson(res, manifest);
};
return {
name: 'wind',
configureServer({ middlewares }) {
middlewares.use('/api/wind', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/wind', handler);
},
};
}
server/providers/wind/catalog.js
+34
View File
@@ -0,0 +1,34 @@
export const GFS_BUCKET = 'noaa-gfs-bdp-pds';
/** Build the public GFS object key. */
export function gfsObjectKey({ date, hour, forecastHour = 0 }) {
const hh = String(hour).padStart(2, '0');
return `gfs.${date}/${hh}/atmos/gfs.t${hh}z.pgrb2.0p25.f${String(forecastHour).padStart(3, '0')}`;
}
/** Select the latest GFS cycle that should be available. */
export function selectLatestGfsCycle(
nowMs,
{ availabilityLagMs = 5 * 3600_000 } = {},
) {
const d = new Date(nowMs - availabilityLagMs);
const date = new Date(
Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()),
);
const hour = Math.floor(d.getUTCHours() / 6) * 6;
date.setUTCHours(hour);
return { date: date.toISOString().slice(0, 10).replaceAll('-', ''), hour };
}
/**
* Round a time offset to the nearest published GFS forecast step so the field
* shown is the one valid closest to now, not always the analysis (f000).
* GFS 0.25° is hourly through f120, then every 3 h through f384.
* @param {number} hours - Hours elapsed since the cycle run time.
* @returns {number} A valid forecast step.
*/
export function nearestGfsStep(hours) {
const value = Math.max(0, Number.isFinite(hours) ? hours : 0);
const step = value <= 120 ? Math.round(value) : Math.round(value / 3) * 3;
return Math.max(0, Math.min(384, step));
}
server/providers/wind/decode.js
+51
View File
@@ -0,0 +1,51 @@
/** Lazy shared ecCodes WASM instance, created only for an active wind request. */
let eccodesPromise = null;
function loadEccodes() {
eccodesPromise ??= import('@meri-imperiumi/eccodes-wasm')
.then((module) => module.createEccodes())
.catch((error) => {
eccodesPromise = null;
throw error;
});
return eccodesPromise;
}
/** Decode one bounded message in WASM memory without mounting the host filesystem. */
export async function decodeWindGribMessage(buffer, { eccodesModule } = {}) {
if (buffer.byteLength > 8 * 1024 * 1024)
throw new Error('Wind GRIB exceeds byte budget');
const eccodes = eccodesModule ?? (await loadEccodes());
// Everything after this await is synchronous. A shared virtual file is safe
// across U/V and model requests and its storage is released after each decode.
const file = '/gev-wind.grib2';
let handle = null;
try {
eccodes.writeFile(file, buffer);
handle = eccodes.openGrib(file);
const ni = handle.getLong('Ni');
const nj = handle.getLong('Nj');
if (
!Number.isInteger(ni) ||
!Number.isInteger(nj) ||
ni < 1 ||
nj < 1 ||
ni * nj > 2_000_000
)
throw new Error('Invalid wind GRIB geometry');
return {
ni,
nj,
lo1: handle.getDouble('longitudeOfFirstGridPointInDegrees'),
la1: handle.getDouble('latitudeOfFirstGridPointInDegrees'),
di: handle.getDouble('iDirectionIncrementInDegrees'),
dj: handle.getDouble('jDirectionIncrementInDegrees'),
values: handle.getDoubleArray('values'),
shortName: handle.getString('shortName'),
level: handle.getLong('level'),
units: handle.getString('units'),
};
} finally {
handle?.delete();
eccodes.writeFile(file, new Uint8Array(0));
}
}
server/providers/wind/gfs.js
+221
View File
@@ -0,0 +1,221 @@
import { readWindBody } from '../../../src/sources/windBody.js';
/**
* Parse a GFS `.idx` inventory into ordered message records.
*
* Each line is `msgNumber:byteOffset:d=YYYYMMDDHH:VAR:LEVEL:...:`. Malformed
* lines are skipped; a payload with no valid lines is rejected so callers never
* treat an HTML error page as an empty inventory.
*
* @param {string} text - Raw `.idx` body.
* @returns {{ messages: Array<{index: number, offset: number, variable: string, level: string}> }}
*/
export function parseGfsIdx(text) {
const messages = [];
for (const line of String(text).split(/\r?\n/)) {
if (!line) continue;
const parts = line.split(':');
if (
parts.length < 6 ||
!/^\d+$/.test(parts[0]) ||
!/^\d+$/.test(parts[1]) ||
!parts[3] ||
!parts[4]
)
continue;
messages.push({
index: Number(parts[0]),
offset: Number(parts[1]),
variable: parts[3],
level: parts[4],
});
}
if (!messages.length) throw new Error('malformed GFS index');
return { messages };
}
/**
* Byte ranges for the two 10 m wind components. A message runs from its own
* offset to one byte before the next message in file order.
*
* @param {{messages: Array<object>}} parsed - {@link parseGfsIdx} result.
* @param {{level?: string}} [options]
* @returns {{u: {start: number, end: number}, v: {start: number, end: number}}}
*/
export function windMessageRanges(
parsed,
{ level = '10 m above ground', overlay = 'none' } = {},
) {
weatherScalarMetadata(overlay);
const range = (variable, fieldLevel = level) => {
const index = parsed.messages.findIndex(
(message) =>
message.variable === variable && message.level === fieldLevel,
);
const message = parsed.messages[index];
const next = parsed.messages[index + 1];
if (!message || !next)
throw new Error(`missing ${variable} at ${fieldLevel}`);
return { start: message.offset, end: next.offset - 1 };
};
return {
u: range('UGRD'),
v: range('VGRD'),
...(overlay === 'temperature'
? { scalar: range('TMP', '2 m above ground') }
: {}),
...(overlay === 'pressure'
? { scalar: range('PRMSL', 'mean sea level') }
: {}),
};
}
/** Fetch a bounded text response as a Buffer. */
export async function fetchText({ url, fetchImpl = fetch, signal }) {
const response = await fetchImpl(url, { signal, redirect: 'error' });
if (!response.ok) {
await response.body?.cancel();
throw new Error('Wind upstream unavailable');
}
return Buffer.from(await readWindBody(response, 2 * 1024 * 1024, signal));
}
/**
* Fetch one bounded byte range. Rejects oversized ranges, non-206 responses
* (except a whole-object 200 starting at zero), and short bodies.
*/
export async function fetchRange({
url,
start,
end,
fetchImpl = fetch,
signal,
maxBytes = 8 * 1024 * 1024,
}) {
if (
!Number.isSafeInteger(start) ||
!Number.isSafeInteger(end) ||
start < 0 ||
end < start ||
end - start + 1 > maxBytes
)
throw new Error('range too large');
const response = await fetchImpl(url, {
signal,
redirect: 'error',
headers: { Range: `bytes=${start}-${end}` },
});
const validStatus =
response.status === 206 || (response.status === 200 && start === 0);
if (!validStatus || !response.ok) {
await response.body?.cancel();
throw new Error('Wind upstream unavailable');
}
const buffer = Buffer.from(
await readWindBody(response, end - start + 1, signal),
);
if (buffer.length !== end - start + 1)
throw new Error('invalid range length');
return buffer;
}
/** A bounded optional request may fail without cancelling valid wind siblings. */
export async function loadOptionalScalar({
range,
url,
fetchImpl,
decodeImpl,
signal,
timeoutMs = 12_000,
}) {
if (!range) return undefined;
const controller = new AbortController();
const abort = () => controller.abort(signal.reason);
signal?.addEventListener('abort', abort, { once: true });
const timer = setTimeout(() => controller.abort(), timeoutMs);
try {
signal?.throwIfAborted();
const bytes = await fetchRange({
url,
...range,
fetchImpl,
signal: controller.signal,
});
const value = await decodeImpl(bytes);
controller.signal.throwIfAborted();
return value;
} catch {
return undefined;
} finally {
clearTimeout(timer);
controller.abort();
signal?.removeEventListener('abort', abort);
}
}
import {
GFS_BUCKET,
gfsObjectKey,
selectLatestGfsCycle,
nearestGfsStep,
} from './catalog.js';
import { decodeWindGribMessage } from './decode.js';
import { resampleWeatherSnapshot, weatherScalarMetadata } from './grid.js';
/** Fetch and decode the GFS 10 m wind field valid closest to now. */
export async function fetchGfsWind({
fetchImpl = fetch,
now = () => Date.now(),
targetDx = 1,
decodeImpl = decodeWindGribMessage,
signal,
overlay = 'none',
} = {}) {
weatherScalarMetadata(overlay);
const nowMs = now();
const cycle = selectLatestGfsCycle(nowMs);
const runMs = Date.UTC(
Number(cycle.date.slice(0, 4)),
Number(cycle.date.slice(4, 6)) - 1,
Number(cycle.date.slice(6, 8)),
cycle.hour,
);
// Pick the forecast step whose valid time is closest to now, so the layer
// shows the freshest field the cycle offers instead of the analysis.
const forecastHour = nearestGfsStep((nowMs - runMs) / 3600_000);
const base = `https://${GFS_BUCKET}.s3.amazonaws.com/${gfsObjectKey({ ...cycle, forecastHour })}`;
const index = await fetchText({ url: `${base}.idx`, fetchImpl, signal });
const inventory = parseGfsIdx(index.toString());
const ranges = windMessageRanges(inventory);
let scalarRange;
if (overlay !== 'none') {
try {
scalarRange = windMessageRanges(inventory, { overlay }).scalar;
} catch {
/* A missing optional field does not discard valid wind. */
}
}
// All selected messages come from this one issue/valid-time object.
const load = async (range) =>
decodeImpl(await fetchRange({ url: base, ...range, fetchImpl, signal }));
const [u, v, scalar] = await Promise.all([
load(ranges.u),
load(ranges.v),
loadOptionalScalar({
range: scalarRange,
url: base,
fetchImpl,
decodeImpl,
signal,
}),
]);
signal?.throwIfAborted();
const fields = resampleWeatherSnapshot({ u, v, scalar, overlay, targetDx });
const runIso = new Date(runMs).toISOString();
const validIso = new Date(runMs + forecastHour * 3600_000).toISOString();
return {
cycle: { ...cycle, forecastHour, runIso, validIso },
level: '10 m above ground',
units: 'm/s',
...fields,
};
}
server/providers/wind/grid.js
+172
View File
@@ -0,0 +1,172 @@
/**
* Resample a GRIB wind grid into a compact equirectangular grid.
*
* Source grids are north-to-south (`la1` at the top, `dj` increasing toward the
* south) and start at `lo1`. Longitude wraps across the seam; latitude clamps.
*
* @param {{u: ArrayLike<number>, v: ArrayLike<number>, ni: number, nj: number,
* lo1: number, la1: number, di: number, dj: number, dx?: number, dy?: number}} input
* @returns {{nx: number, ny: number, lo1: number, la1: number, dx: number,
* dy: number, u: Float32Array, v: Float32Array}}
*/
export function resampleWindGrid({
u,
v,
scalar,
scalarScale = 1,
scalarOffset = 0,
ni,
nj,
lo1,
la1,
di,
dj,
dx = 1,
dy = 1,
}) {
if (
!Number.isInteger(ni) ||
!Number.isInteger(nj) ||
ni < 1 ||
nj < 1 ||
ni * nj > 2_000_000 ||
u?.length !== ni * nj ||
v?.length !== ni * nj ||
(scalar !== undefined && scalar?.length !== ni * nj) ||
![lo1, la1, di, dj, dx, dy, scalarScale, scalarOffset].every(
Number.isFinite,
) ||
di <= 0 ||
dj <= 0 ||
dx < 0.25 ||
dy < 0.25 ||
dx > 180 ||
dy > 180
)
throw new Error('Invalid wind grid geometry');
for (const values of [u, v, ...(scalar === undefined ? [] : [scalar])]) {
if (!Array.prototype.every.call(values, Number.isFinite))
throw new Error('Invalid wind grid values');
}
const nx = Math.round(360 / dx);
const ny = Math.round(180 / dy) + 1;
if (nx * ny > 1_000_000)
throw new Error('Resampled wind grid exceeds point budget');
const outU = new Float32Array(nx * ny);
const outV = new Float32Array(nx * ny);
const outScalar = scalar === undefined ? null : new Float32Array(nx * ny);
const sample = (source, lon, lat) => {
const x = ((lon - lo1) / di + ni) % ni;
const y = Math.max(0, Math.min(nj - 1, (la1 - lat) / dj));
const x0 = Math.floor(x);
const x1 = (x0 + 1) % ni;
const y0 = Math.floor(y);
const y1 = Math.min(nj - 1, y0 + 1);
const fx = x - x0;
const fy = y - y0;
const top = source[y0 * ni + x0] * (1 - fx) + source[y0 * ni + x1] * fx;
const bottom = source[y1 * ni + x0] * (1 - fx) + source[y1 * ni + x1] * fx;
return top * (1 - fy) + bottom * fy;
};
for (let y = 0; y < ny; y += 1) {
for (let x = 0; x < nx; x += 1) {
const index = y * nx + x;
const lon = x * dx;
const lat = 90 - y * dy;
outU[index] = sample(u, lon, lat);
outV[index] = sample(v, lon, lat);
if (outScalar)
outScalar[index] =
sample(scalar, lon, lat) * scalarScale + scalarOffset;
}
}
for (const values of [outU, outV, ...(outScalar ? [outScalar] : [])]) {
if (!values.every(Number.isFinite))
throw new Error('Invalid resampled wind values');
}
return {
nx,
ny,
lo1: 0,
la1: 90,
dx,
dy,
u: outU,
v: outV,
...(outScalar ? { scalar: outScalar } : {}),
};
}
/** Metadata for the two optional same-cycle weather fields. */
export function weatherScalarMetadata(overlay = 'none') {
if (overlay === 'none') return null;
if (overlay === 'temperature')
return { kind: overlay, units: '°C', level: '2 m above ground' };
if (overlay === 'pressure')
return { kind: overlay, units: 'hPa', level: 'mean sea level' };
throw new Error('Unknown weather overlay');
}
/** Reject shifted/mismatched components before sharing U's grid geometry. */
export function resampleWeatherFields({
u,
v,
scalar,
overlay = 'none',
targetDx = 1,
}) {
const metadata = weatherScalarMetadata(overlay);
const fields = [u, v, ...(metadata ? [scalar] : [])];
if (
!u ||
fields.some(
(field) =>
!field ||
['ni', 'nj', 'lo1', 'la1', 'di', 'dj'].some(
(key) => field[key] !== u[key],
),
)
)
throw new Error('Mismatched weather grid geometry');
if (metadata && scalar.units !== (overlay === 'temperature' ? 'K' : 'Pa'))
throw new Error('Unexpected weather scalar units');
const grid = resampleWindGrid({
u: u.values,
v: v.values,
ni: u.ni,
nj: u.nj,
lo1: u.lo1,
la1: u.la1,
di: u.di,
dj: u.dj,
dx: targetDx,
dy: targetDx,
...(metadata
? {
scalar: scalar.values,
scalarScale: overlay === 'pressure' ? 0.01 : 1,
scalarOffset: overlay === 'temperature' ? -273.15 : 0,
}
: {}),
});
return { grid, ...(metadata ? { scalar: metadata } : {}) };
}
/** Preserve usable same-cycle wind when only its optional companion fails. */
export function resampleWeatherSnapshot(options) {
try {
return resampleWeatherFields(options);
} catch (error) {
if ((options.overlay ?? 'none') === 'none') throw error;
// Revalidate wind alone: a wind defect must never become a partial success.
const wind = resampleWeatherFields({ ...options, overlay: 'none' });
return { ...wind, scalarError: weatherScalarError(options.overlay) };
}
}
export function weatherScalarError(overlay) {
weatherScalarMetadata(overlay);
return overlay === 'temperature'
? 'Temperature field unavailable'
: 'Mean sea level pressure field unavailable';
}
server/providers/wind/ifs.js
+166
View File
@@ -0,0 +1,166 @@
import { fetchText, fetchRange, loadOptionalScalar } from './gfs.js';
import { decodeWindGribMessage } from './decode.js';
import { resampleWeatherSnapshot, weatherScalarMetadata } from './grid.js';
/** Keyless ECMWF Open Data root for real-time IFS forecasts. */
export const IFS_BASE = 'https://data.ecmwf.int/forecasts';
/**
* Select the latest IFS cycle that should be available.
* @param {number} nowMs - Reference epoch milliseconds.
* @param {{availabilityLagMs?: number}} [options]
* @returns {{date: string, hour: number}}
*/
export function selectLatestIfsCycle(
nowMs,
{ availabilityLagMs = 6 * 3600_000 } = {},
) {
const shifted = new Date(nowMs - availabilityLagMs);
return {
date: shifted.toISOString().slice(0, 10).replaceAll('-', ''),
hour: Math.floor(shifted.getUTCHours() / 6) * 6,
};
}
/**
* Build the IFS GRIB2 and `.index` URLs for one cycle and step.
* @param {{date: string, hour: number, step?: number}} cycle
* @returns {{grib: string, index: string}}
*/
export function ifsObjectUrls({ date, hour, step = 0 }) {
const hh = String(hour).padStart(2, '0');
const stem = `${IFS_BASE}/${date}/${hh}z/ifs/0p25/oper/${date}${hh}0000-${step}h-oper-fc`;
// The inventory sits beside the GRIB2 as `<stem>.index`, NOT `<grib2>.index`.
return { grib: `${stem}.grib2`, index: `${stem}.index` };
}
/**
* Parse the IFS JSON Lines inventory into `{param, offset, length}` records.
* Malformed lines are skipped; a payload with no valid lines is rejected.
* @param {string} text - Raw `.index` body.
* @returns {Array<{param: string, offset: number, length: number}>}
*/
export function parseIfsIndex(text) {
const entries = [];
for (const line of String(text).split(/\r?\n/)) {
if (!line) continue;
let value;
try {
value = JSON.parse(line);
} catch {
continue; // non-JSON line in the index
}
if (
value?.param &&
Number.isFinite(value._offset) &&
Number.isFinite(value._length)
)
entries.push({
param: value.param,
offset: value._offset,
length: value._length,
});
}
if (!entries.length) throw new Error('malformed IFS index');
return entries;
}
/**
* Byte ranges for the IFS 10 m wind components.
* @param {Array<{param: string, offset: number, length: number}>} entries
* @returns {{u: {start: number, end: number}, v: {start: number, end: number}}}
*/
export function ifsWindRanges(entries, { overlay = 'none' } = {}) {
weatherScalarMetadata(overlay);
const range = (param) => {
const item = entries.find((entry) => entry.param === param);
if (!item) throw new Error(`missing ${param}`);
return { start: item.offset, end: item.offset + item.length - 1 };
};
return {
u: range('10u'),
v: range('10v'),
...(overlay === 'temperature' ? { scalar: range('2t') } : {}),
...(overlay === 'pressure' ? { scalar: range('msl') } : {}),
};
}
/**
* Round a time offset to the nearest published IFS forecast step. IFS 0.25°
* oper is 3-hourly through f144, then 6-hourly to f360 for 00/12z.
* @param {number} hours - Hours elapsed since the cycle run time.
* @param {number} cycleHour - The cycle hour (0, 6, 12 or 18).
* @returns {number} A valid forecast step.
*/
export function nearestIfsStep(hours, cycleHour) {
const max = cycleHour === 6 || cycleHour === 18 ? 144 : 360;
const value = Math.max(0, Number.isFinite(hours) ? hours : 0);
const step =
value <= 144 ? Math.round(value / 3) * 3 : Math.round(value / 6) * 6;
return Math.max(0, Math.min(max, step));
}
/**
* Fetch and decode the IFS 10 m wind field valid closest to now.
* @param {{fetchImpl?: Function, now?: Function, targetDx?: number,
* decodeImpl?: Function}} [options]
* @returns {Promise<{cycle: object, level: string, units: string, grid: object}>}
*/
export async function fetchIfsWind({
fetchImpl = fetch,
now = () => Date.now(),
targetDx = 1,
decodeImpl = decodeWindGribMessage,
signal,
overlay = 'none',
} = {}) {
weatherScalarMetadata(overlay);
const nowMs = now();
const cycle = selectLatestIfsCycle(nowMs);
const runMs = Date.UTC(
Number(cycle.date.slice(0, 4)),
Number(cycle.date.slice(4, 6)) - 1,
Number(cycle.date.slice(6, 8)),
cycle.hour,
);
// Show the field valid closest to now rather than always the analysis.
const forecastHour = nearestIfsStep((nowMs - runMs) / 3600_000, cycle.hour);
const urls = ifsObjectUrls({ ...cycle, step: forecastHour });
const index = await fetchText({ url: urls.index, fetchImpl, signal });
const inventory = parseIfsIndex(index.toString());
const ranges = ifsWindRanges(inventory);
let scalarRange;
if (overlay !== 'none') {
try {
scalarRange = ifsWindRanges(inventory, { overlay }).scalar;
} catch {
/* A missing optional field does not discard valid wind. */
}
}
// All selected messages come from this one issue/valid-time object.
const load = async (range) =>
decodeImpl(
await fetchRange({ url: urls.grib, ...range, fetchImpl, signal }),
);
const [u, v, scalar] = await Promise.all([
load(ranges.u),
load(ranges.v),
loadOptionalScalar({
range: scalarRange,
url: urls.grib,
fetchImpl,
decodeImpl,
signal,
}),
]);
signal?.throwIfAborted();
const fields = resampleWeatherSnapshot({ u, v, scalar, overlay, targetDx });
const runIso = new Date(runMs).toISOString();
const validIso = new Date(runMs + forecastHour * 3600_000).toISOString();
return {
cycle: { ...cycle, forecastHour, runIso, validIso },
level: '10 m above ground',
units: 'm/s',
...fields,
};
}
server/standalone/key-setup-hardening.mjs
+5 -1
View File
@@ -123,7 +123,11 @@ export function hardenCredentialFile(
if (platform !== 'win32') {
try {
if (platform === 'darwin') {
const aclRemoval = spawn('chmod', ['-N', filepath], {
// `-N` (strip the ACL) is Apple-only. Spawn /bin/chmod by absolute
// path, never by name: a Nix/Homebrew coreutils profile puts GNU chmod
// first on PATH, and GNU chmod rejects `-N` — which was treated as a
// hardening failure and refused every save on such machines.
const aclRemoval = spawn('/bin/chmod', ['-N', filepath], {
stdio: 'ignore',
});
if (!commandCompletedSuccessfully(aclRemoval)) return false;
server/standalone/vite.config.js
+2 -1
View File
@@ -7,7 +7,7 @@ import { apiNotFoundPlugin } from './api-not-found.js';
const root = fileURLToPath(new URL('../../', import.meta.url));
/** Load this checkout's configuration and attach its local provider middleware. */
export default defineConfig(({ mode }) => {
export default defineConfig(({ command, mode }) => {
const loaded = loadEnv(mode, root, '');
for (const [key, value] of Object.entries(loaded)) {
if (process.env[key] === undefined) process.env[key] = value;
@@ -18,5 +18,6 @@ export default defineConfig(({ mode }) => {
cesiumToken: process.env.CESIUM_ION_TOKEN,
host: process.env.HOST,
port: process.env.PORT,
command,
});
});
src/annotations/adminOutline.test.mjs
+236
View File
@@ -0,0 +1,236 @@
// Area annotations draw bundled state/province/county outlines with no
// network lookup, and multi-part outlines reach the renderer intact.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { FEATURE_SOURCE_METHODS } from '../sources/featureSource.js';
import { createAnnotationResolver } from './resolver.js';
import { createAnnotationEngine } from './annotationEngine.js';
import { _resetRenderGovernorForTest } from '../renderGovernor.js';
function viewerAt(lat, lon, height = 20_000) {
return {
camera: {
positionCartographic: {
latitude: (lat * Math.PI) / 180,
longitude: (lon * Math.PI) / 180,
height,
},
},
};
}
/** A resolver whose boundary lookups and geocoder record every call. */
function harness(place = null) {
const lookups = [];
const featureSource = {};
for (const method of FEATURE_SOURCE_METHODS)
featureSource[method] = async () => {
lookups.push(method);
return null;
};
const geocodes = [];
const placeSearch = {
async geocode(query) {
geocodes.push(query);
return { place };
},
async textSearch() {
geocodes.push('text-search');
return [];
},
};
const { resolveAnnotationTarget } = createAnnotationResolver({
featureSource,
});
const resolve = (target, viewer, extra = {}) =>
resolveAnnotationTarget({
placeSearch,
viewer,
target,
footprint: true,
deferFootprint: true,
...extra,
});
return { resolve, lookups, geocodes };
}
const AUSTIN = viewerAt(30.2672, -97.7431, 3000);
test('"Texas", "Travis County, Texas" and "Bavaria" outline offline', async () => {
const { resolve, lookups, geocodes } = harness();
for (const [target, label] of [
['Texas', 'Texas'],
['Travis County, Texas', 'Travis County'],
// Far from the camera, but named outright: no proximity gate applies.
['Bavaria', 'Bavaria'],
]) {
const resolved = await resolve(target, AUSTIN);
assert.ok(resolved, target);
assert.equal(resolved.source, 'bundled', target);
assert.equal(resolved.label, label);
assert.equal(resolved.footprintKind, 'area');
assert.equal(resolved.synthesized, false);
assert.equal(resolved.resolveOutline, undefined, 'drawn at once');
assert.ok(resolved.ring.length >= 9, `${target} ring`);
assert.deepEqual(resolved.ring[0], resolved.ring.at(-1), 'ring closed');
assert.ok(resolved.polygons.length >= 1, `${target} parts`);
assert.equal(resolved.polygons[0][0].length + 1, resolved.ring.length);
assert.ok(resolved.viewport?.low && resolved.viewport?.high);
}
assert.deepEqual(geocodes, [], 'no geocoding');
assert.deepEqual(lookups, [], 'no boundary lookups');
});
test('duplicate county names go to the one near the camera', async () => {
const { resolve } = harness();
const nearPortland = await resolve(
'Washington County',
viewerAt(45.52, -122.68),
);
assert.ok(nearPortland.lat > 45 && nearPortland.lon < -122, 'Oregon');
const nearAustin = await resolve('Washington County', AUSTIN);
assert.ok(nearAustin.lat > 29.9 && nearAustin.lat < 30.5, 'Texas');
assert.ok(nearAustin.lon > -96.8 && nearAustin.lon < -96, 'Texas');
});
test('multi-part outlines carry every part and hole', async () => {
const { resolve } = harness();
const hawaii = await resolve('Hawaii', AUSTIN);
assert.ok(hawaii.polygons.length >= 7, 'islands');
const brandenburg = await resolve('Brandenburg', AUSTIN);
assert.ok(
brandenburg.polygons.some((part) => part.length === 2),
'Berlin hole',
);
});
test('Georgia resolves from camera geography without a geocoder', async () => {
const { resolve, lookups, geocodes } = harness();
for (const [query, viewer, latitude] of [
['Georgia', viewerAt(33.75, -84.39, 400_000), 32],
['the country of Georgia', AUSTIN, 42],
['Georgia', viewerAt(41.7, 44.8), 42],
]) {
const resolved = await resolve(query, viewer);
assert.equal(resolved.source, 'bundled');
assert.ok(Math.abs(resolved.lat - latitude) < 2);
assert.ok(resolved.ring.length > 20);
}
assert.deepEqual(lookups, []);
assert.deepEqual(geocodes, []);
});
test('country asks and geocoded country scopes use the bundled outline', async () => {
const direct = harness();
for (const name of ['Switzerland', 'Iran', 'France', 'Japan']) {
const resolved = await direct.resolve(name, AUSTIN);
assert.equal(resolved.source, 'bundled');
assert.ok(resolved.polygons.length);
if (name === 'Japan') assert.ok(resolved.polygons.length > 3);
}
assert.deepEqual(direct.lookups, []);
assert.deepEqual(direct.geocodes, []);
const geocoded = harness({
lat: 46.95,
lng: 7.45,
label: 'Switzerland',
name: 'Switzerland',
types: ['country', 'political'],
});
const resolved = await geocoded.resolve(
'Swiss country outline',
viewerAt(46.95, 7.45),
);
assert.equal(resolved.source, 'geocode');
const outline = await resolved.resolveOutline();
assert.ok(outline.polygons.length);
assert.deepEqual(geocoded.lookups, []);
});
test('non-admin asks and entity facts keep the existing path', async () => {
const { resolve, geocodes } = harness();
await resolve('Texas Capitol', AUSTIN);
await resolve('Texas', AUSTIN, { entityKind: 'building' });
await resolve('Zilker Park', AUSTIN);
assert.equal(
geocodes.filter((q) => q !== 'text-search').length,
3,
'each ask geocoded',
);
// Points and explicit coordinates never use the pack.
const point = await resolve('Texas', AUSTIN, { footprint: false });
assert.notEqual(point?.source, 'bundled');
});
test('the engine keeps every part on the mark, drawn at once or upgraded', async (t) => {
globalThis.requestAnimationFrame ??= () => 1;
globalThis.cancelAnimationFrame ??= () => {};
_resetRenderGovernorForTest();
t.after(() => _resetRenderGovernorForTest());
const polygons = [
[
[
[0, 0],
[2, 0],
[2, 2],
[0, 2],
],
[
[0.5, 0.5],
[0.5, 1],
[1, 1],
[1, 0.5],
],
],
[
[
[3, 3],
[4, 3],
[4, 4],
],
],
];
const ring = [...polygons[0][0], polygons[0][0][0]];
const seen = [];
const renderer = {
add: (anno) => seen.push(['add', anno.polygons]),
update: (anno) => seen.push(['update', anno.polygons]),
remove() {},
sync() {},
destroy() {},
};
const engine = createAnnotationEngine({
viewer: {},
renderer,
resolveTarget: async ({ target }) =>
target === 'now'
? { lon: 1, lat: 1, height: 0, ring, polygons, footprintKind: 'area' }
: {
lon: 1,
lat: 1,
height: 0,
ring: null,
resolveOutline: async () => ({
ring,
polygons,
footprintKind: 'area',
lat: 1,
lon: 1,
height: 0,
}),
},
});
await engine.annotate([{ type: 'area', target: 'now' }]);
await engine.annotate([{ type: 'area', target: 'later', label: 'later' }]);
for (let i = 0; i < 20; i += 1) await Promise.resolve();
const marks = engine.list();
assert.equal(marks[0].polygons, polygons);
assert.ok(
marks.some((m) => m.label === 'later' && m.polygons === polygons),
'upgrade carries the parts',
);
assert.ok(
seen.some(([kind, parts]) => kind === 'update' && parts === polygons),
);
engine.destroy();
});
src/annotations/annotationEngine.js
+32 -4
View File
@@ -1,3 +1,4 @@
import { isUnavailableCapability } from '../sources/capability.js';
import { defaultGeospatial } from '../search/defaults.js';
import * as Cesium from 'cesium';
import {
@@ -81,6 +82,7 @@ export async function resolveOutlineWithRetry(
} catch {
fp = null; // hard failure — definitive, keep the honest point
}
if (isUnavailableCapability(fp)) return fp;
if (isRateLimitedOutcome(fp)) {
// A throttle gets one deliberately spaced retry. If that retry is throttled
// too, stop this mark's outline task instead of replaying the batch storm.
@@ -333,6 +335,7 @@ export function createAnnotationEngine({
// refreshes it — usually instantly from the footprint cache.
if (anno.pendingOutline && !anno.ring && dup.ring) {
anno.ring = dup.ring;
anno.polygons = dup.polygons || null;
anno.footprintKind = dup.footprintKind || null;
anno.buildingHeight = dup.buildingHeight || null;
anno.synthesized = Boolean(dup.synthesized);
@@ -673,8 +676,11 @@ export function createAnnotationEngine({
if (myGen !== generation || controller.signal.aborted) return; // board superseded
if (!annotations.has(anno.id)) return; // mark replaced/removed while resolving
anno.pendingOutline = false;
if (fp) {
anno.outlineUnavailable = isUnavailableCapability(fp);
if (anno.outlineUnavailable) renderer.update(anno);
if (fp && !anno.outlineUnavailable) {
anno.ring = fp.ring;
anno.polygons = fp.polygons || null;
anno.footprintKind = fp.footprintKind || null;
anno.buildingHeight = fp.buildingHeight || null;
anno.synthesized = Boolean(fp.synthesized);
@@ -725,7 +731,14 @@ export function createAnnotationEngine({
id: anno.id,
label: anno.label || null,
target: anno.targetKey || null,
status: fp ? 'resolved' : 'failed',
status: anno.outlineUnavailable
? 'unavailable'
: fp
? 'resolved'
: 'failed',
...(anno.outlineUnavailable
? { message: 'Detailed outline unavailable' }
: {}),
...(fp && anno.synthesized ? { approximate: true } : {}),
});
} finally {
@@ -753,6 +766,9 @@ export function createAnnotationEngine({
// layer narrate without waiting out a slow Overpass — and without calling the
// missing outline a failure.
...(anno.pendingOutline ? { outlinePending: true } : {}),
...(anno.outlineUnavailable
? { outlineUnavailable: true, message: 'Detailed outline unavailable' }
: {}),
// approximate=true means the area was SYNTHESIZED (a buffered blob around a label
// point), not a real OSM boundary — so the voice layer can be honest.
...(anno.synthesized ? { approximate: true } : {}),
@@ -919,6 +935,9 @@ export function createAnnotationEngine({
anchor: { lon: resolved.lon, lat: resolved.lat, height: resolved.height },
to: null,
ring: resolved.ring || null,
// Every part with its holes ([outer, ...holes][]), when the outline has
// more than the main ring (Hawaii's islands, Berlin inside Brandenburg).
polygons: resolved.polygons || null,
footprintKind: resolved.footprintKind || null, // 'building' | 'area'
buildingHeight: resolved.buildingHeight || null, // meters, for extruded volume
synthesized: Boolean(resolved.synthesized), // approximate buffered area → dashed render
@@ -926,6 +945,7 @@ export function createAnnotationEngine({
// the upgrade task fills ring/kind in place when it lands. Transient render state
// (not persisted).
pendingOutline: typeof resolved.resolveOutline === 'function',
outlineUnavailable: Boolean(resolved.outlineUnavailable),
// Which THING + SHAPE was asked for — the dedup identity while geometry is still
// pending (see findDuplicate). targetKey is the normalized place name with trailing
// locality qualifiers stripped ("California, United States" ≡ "California"; null for
@@ -1047,7 +1067,11 @@ export function createAnnotationEngine({
// Places viewport box when we have one, so a big compound isn't framed at
// building scale while its outline is traced. Never re-fly when the ring lands.
const range = anno.ring
? ringRange(anno.ring)
? ringRange(
anno.polygons?.length
? anno.polygons.flatMap((poly) => poly[0])
: anno.ring,
)
: viewportRange(anno.viewport) || 600;
viewer.camera.flyToBoundingSphere(
new Cesium.BoundingSphere(
@@ -1388,7 +1412,11 @@ function ringRange(ring) {
let maxLat = -Infinity;
let minLon = Infinity;
let maxLon = -Infinity;
for (const [lon, lat] of ring) {
// Unwrap around the first vertex so parts cut at the antimeridian (the
// Aleutians) measure as neighbours, not as a globe-wide span.
const ref = ring[0]?.[0] ?? 0;
for (const [rawLon, lat] of ring) {
const lon = rawLon - 360 * Math.round((rawLon - ref) / 360);
if (lat < minLat) minLat = lat;
if (lat > maxLat) maxLat = lat;
if (lon < minLon) minLon = lon;
src/annotations/annotationEngine.test.mjs
+3 -1
View File
@@ -254,7 +254,9 @@ test('retry: HTTP 429 Retry-After 5s gets one ladder-spaced retry; a second 429
t.mock.timers.enable({ apis: ['setTimeout', 'Date'], now: 10_000 });
globalThis.window = { setTimeout: globalThis.setTimeout, clearTimeout: globalThis.clearTimeout };
const requestTimes = [];
globalThis.fetch = async () => {
globalThis.fetch = async (url) => {
// The one-time capability probe is not an outline request.
if (String(url).endsWith('/status')) return httpFailure(404);
requestTimes.push(Date.now());
return httpFailure(429, '5');
};
src/annotations/hybridAnnotationRenderer.js
+5 -1
View File
@@ -77,7 +77,11 @@ export function createHybridAnnotationRenderer(viewer) {
add(anno);
return;
}
if (anno.type !== 'area' || !anno.ring || anno.ring.length < 3) return;
if (anno.type !== 'area' || !anno.ring || anno.ring.length < 3) {
if (anno.outlineUnavailable && entry.screenProxy)
screen.update(entry.screenProxy);
return;
}
if (!entry.worldProxy) {
entry.worldProxy = liveProxy(anno, { label: null });
src/annotations/regionRing.test.mjs
+153
View File
@@ -0,0 +1,153 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { FEATURE_SOURCE_METHODS } from '../sources/featureSource.js';
import { createAnnotationResolver } from './resolver.js';
function featureSource(overrides = {}) {
const source = {};
for (const method of FEATURE_SOURCE_METHODS)
source[method] = async () => null;
return Object.assign(source, overrides);
}
// A state-typed place outside every bundled boundary, so the admin-boundary
// feature lookup (not the bundled packs) answers.
const offPackGeocode = {
async geocode() {
return {
place: {
lat: 0.5,
lng: -150.5,
label: 'Atlantis',
name: 'Atlantis',
types: ['administrative_area_level_1', 'political'],
},
};
},
};
function geocoderFor(place) {
const calls = [];
return {
calls,
async geocode(query) {
calls.push(query);
return { place };
},
};
}
test('region ring: Natural Earth names resolve without a geocoder', async () => {
const { resolveRegionRingForQuery } = createAnnotationResolver({
featureSource: featureSource(),
});
for (const name of ['Gulf of Mexico', 'the Alps']) {
const region = await resolveRegionRingForQuery(name);
assert.ok(region?.ring?.length >= 3, `${name} resolves to a ring`);
assert.equal(region.error, undefined);
}
});
test('region ring: a slow admin-boundary lookup returns region-timeout within the budget', async () => {
let adminSignal;
let release;
const { resolveRegionRingForQuery } = createAnnotationResolver({
featureSource: featureSource({
getAdministrativeAreas(_point, { signal }) {
adminSignal = signal;
return new Promise((resolve) => {
release = resolve;
});
},
}),
});
const started = Date.now();
const region = await resolveRegionRingForQuery(
'Atlantis',
undefined,
offPackGeocode,
{ budgetMs: 50 },
);
assert.deepEqual(region, {
name: 'Atlantis',
ring: null,
error: 'region-timeout',
});
assert.ok(
Date.now() - started < 1000,
'returns at the budget, not the lookup',
);
// The lookup is left running so it can fill the boundary cache.
assert.equal(adminSignal?.aborted ?? false, false);
release(null);
});
test('region ring: an unbounded budget waits for the lookup', async () => {
const { resolveRegionRingForQuery } = createAnnotationResolver({
featureSource: featureSource({
getAdministrativeAreas: () =>
new Promise((resolve) => setTimeout(() => resolve(null), 30)),
}),
});
const region = await resolveRegionRingForQuery(
'Atlantis',
undefined,
offPackGeocode,
{ budgetMs: Infinity },
);
assert.equal(region, null);
});
test('region ring: bundled states and counties resolve without a geocoder', async () => {
const lookups = [];
const { resolveRegionRingForQuery } = createAnnotationResolver({
featureSource: featureSource({
getAdministrativeAreas: async () => {
lookups.push('admin');
return null;
},
}),
});
const geocoder = geocoderFor(null);
for (const [query, name] of [
['Texas', 'Texas'],
['Bavaria', 'Bavaria'],
['Travis County, Texas', 'Travis County'],
]) {
const region = await resolveRegionRingForQuery(query, undefined, geocoder);
assert.equal(region?.name, name, query);
assert.ok(region.ring.length >= 8, `${query} has a real ring`);
}
assert.deepEqual(geocoder.calls, [], 'no geocoding');
assert.deepEqual(lookups, [], 'no boundary lookups');
});
test('region ring: a geocoder-typed state resolves from the bundled pack', async () => {
const lookups = [];
const { resolveRegionRingForQuery } = createAnnotationResolver({
featureSource: featureSource({
getAdministrativeAreas: async () => {
lookups.push('admin');
return null;
},
}),
});
// "Georgia" alone is also a country, so the words don't settle it; the
// geocoder's state type and point inside the US state do.
const geocoder = geocoderFor({
lat: 32.16,
lng: -82.9,
label: 'Georgia, USA',
name: 'Georgia',
types: ['administrative_area_level_1', 'political'],
});
const region = await resolveRegionRingForQuery(
'Georgia',
undefined,
geocoder,
);
assert.equal(region?.name, 'Georgia');
assert.ok(region.ring.length >= 8);
assert.deepEqual(geocoder.calls, ['Georgia']);
assert.deepEqual(lookups, [], 'no boundary lookups');
});
src/annotations/resolver.js
+186 -14
View File
@@ -1,3 +1,4 @@
import { isUnavailableCapability } from '../sources/capability.js';
import { requireFeatureSource } from '../sources/featureSource.js';
import { createOverpassFeatureSource } from '../sources/overpassFeatures.js';
import {
@@ -18,13 +19,26 @@ import {
lookupNaturalRegionOutline,
findNaturalRegion,
} from '../data/naturalEarthRegions.js';
import { findAdminArea, findAdminAreaAt } from '../data/adminBoundaries.js';
import {
registerDynamicCredit,
NATURAL_EARTH_CREDIT,
US_CENSUS_CREDIT,
} from '../data/dataCredits.js';
import { unavailablePlaceSearch } from '../search/placeSearch.js';
import { isPickedWorldPosition } from '../data/scenePick.js';
/** Ms the analyst region lookup waits on geocode + admin boundary. */
export const REGION_FALLBACK_BUDGET_MS = 3_000;
/** Entity facts that rule out a state/county reading of the ask. */
const NON_ADMIN_ENTITY_KINDS = new Set([
'building',
'compound',
'street',
'point_feature',
]);
/** Own annotation lookup caches and ranking of supplied feature candidates. */
export function createAnnotationResolver({
boundarySource,
@@ -173,6 +187,23 @@ export function createAnnotationResolver({
// ("the Texas Capitol" → the state), so geocode types must never grant it.
const bypassNearViewGuards = Boolean(adminScopeFromAsk(target, entityKind));
// Bundled administrative outlines resolve settled names without a lookup.
// Georgia uses its qualifier or camera; city/state homonyms defer to geocoding.
if (
footprint &&
(!Number.isFinite(lat) || !Number.isFinite(lon)) &&
trace.query &&
!NON_ADMIN_ENTITY_KINDS.has(entityKind)
) {
const center =
pickWorldFromScreen(viewer, 0.5, 0.5) || viewportProximity(viewer);
const admin = await findAdminArea(trace.query, { near: center }).catch(
() => null,
);
signal?.throwIfAborted();
if (admin) return bundledAdminTarget(viewer, admin, trace.query);
}
if (!Number.isFinite(lat) || !Number.isFinite(lon)) {
const query = String(target || '').trim();
if (query) {
@@ -471,8 +502,32 @@ export function createAnnotationResolver({
// GROUNDS-like ask must not short-circuit here: the model phrases "the Capitol
// grounds" as around_the_thing, but the grounds ARE the thing — the real enclosing
// polygon (below) beats a 400 m disc (field test 8's "spherical round one").
const availability = await fetchEnclosingArea(
lat,
lon,
signal,
matchName,
);
if (
isUnavailableCapability(availability) ||
isRateLimitedOutcome(availability)
)
return availability;
if (availability === undefined) return undefined;
fp = synthesizeBufferedArea(lat, lon, AROUND_LANDMARK_RADIUS_M);
} else if (isAdmin) {
// A country, state or county the geocoder typed: the bundled unit that carries the
// name AND contains the geocoded point (offline; the pack's ambiguity
// marks don't apply — the geocoder already chose "Georgia" the state).
if (scope === 'state' || scope === 'county' || scope === 'country') {
const admin = await findAdminAreaAt(
[target, matchName],
lat,
lon,
scope,
).catch(() => null);
if (admin) return bundledAdminOutline(viewer, admin);
}
// Pure admin: only an admin boundary is correct — never fall back to a
// building/landuse (a city is never a single building).
fp = await fetchAdminArea(lat, lon, matchName, scope, signal);
@@ -498,6 +553,7 @@ export function createAnnotationResolver({
// A neighborhood whose canonical name carries a city suffix ("Presidio of San
// Francisco") can match the CITY admin; treat an oversized admin as a DEFINITIVE
// no-neighborhood-polygon so it still falls through to the named-landuse path.
if (isUnavailableCapability(adminFp)) return adminFp;
if (adminFp && exceedsScopeArea(adminFp, scope)) adminFp = null;
if (adminFp) {
fp = adminFp;
@@ -512,6 +568,7 @@ export function createAnnotationResolver({
signal,
'strict',
);
if (isUnavailableCapability(footFp)) return footFp;
if (footFp) {
fp = footFp;
} else if (footFp === null) {
@@ -528,6 +585,7 @@ export function createAnnotationResolver({
signal,
'loose',
);
if (isUnavailableCapability(looseFp)) return looseFp;
if (looseFp && looseFp.kind !== 'building') fp = looseFp;
else if (looseFp === null)
fp = synthesizeBufferedArea(lat, lon, NEIGHBORHOOD_RADIUS_M);
@@ -577,6 +635,7 @@ export function createAnnotationResolver({
// synthesized disc only when OSM DEFINITIVELY has none.
if (groundsLike && (fp === null || fp?.kind === 'building')) {
const area = await fetchEnclosingArea(lat, lon, signal, matchName);
if (isUnavailableCapability(area)) return area;
if (area) {
fp = area; // real grounds polygon (synthesized:false) — beats both the dome and a disc
// It's a grounds/compound feature (already capped at SCOPE_AREA_CAP_M2.compound inside
@@ -616,7 +675,7 @@ export function createAnnotationResolver({
) {
fp = null;
}
if (isRateLimitedOutcome(fp)) return fp;
if (isUnavailableCapability(fp) || isRateLimitedOutcome(fp)) return fp;
if (fp === undefined) return undefined; // TRANSIENT — a backoff retry may still find it
if (!fp || !Array.isArray(fp.ring) || fp.ring.length < 3) return null;
const centroid = ringCentroid(fp.ring);
@@ -643,10 +702,12 @@ export function createAnnotationResolver({
let ring = null;
let footprintKind = null; // 'building' | 'area'
let buildingHeight = null; // meters, only for buildings
let outlineUnavailable = false;
let synthesized = false; // true = buffered/approximate area, render dashed/feathered
if (footprint && !deferFootprint) {
const fp = await resolveOutline();
if (fp && !isRateLimitedOutcome(fp)) {
outlineUnavailable = isUnavailableCapability(fp);
if (fp && !outlineUnavailable && !isRateLimitedOutcome(fp)) {
ring = fp.ring;
footprintKind = fp.footprintKind;
buildingHeight = fp.buildingHeight;
@@ -677,6 +738,7 @@ export function createAnnotationResolver({
label,
source,
synthesized,
outlineUnavailable,
viewport: placeViewport,
...(footprint && deferFootprint ? { resolveOutline } : {}),
};
@@ -717,6 +779,65 @@ export function createAnnotationResolver({
const GROUNDS_RADIUS_MIN_M = 150; // viewport-derived grounds disc is clamped to this band so a tiny
const GROUNDS_RADIUS_MAX_M = 1200; // place can't shrink to a dot, nor a city-wide viewport balloon
/** Credit the pack a bundled boundary came from. */
function creditAdminSource(viewer, admin) {
registerDynamicCredit(
viewer,
admin.source === 'us-census' ? US_CENSUS_CREDIT : NATURAL_EARTH_CREDIT,
);
}
/**
* Outline patch (resolveOutline's contract) for a bundled administrative unit.
* `ring` is the main part, closed; `polygons` carries every part with its
* holes (Hawaii's islands, Berlin inside Brandenburg) for renderers that
* draw them. Like the Natural Earth rung, it bypasses the scope caps and the
* centroid drift bound: the unit IS the asked scope.
*/
function bundledAdminOutline(viewer, admin) {
creditAdminSource(viewer, admin);
const { lat, lon } = admin.label;
return {
ring: closeRing([...admin.ring]), // copy: closeRing mutates, the pack is shared
polygons: admin.polygons,
footprintKind: 'area',
buildingHeight: null,
synthesized: false,
adminArea: admin.name,
lat,
lon,
height: sampleGroundHeight(viewer, lon, lat),
};
}
/** A complete resolved target for a bundled administrative unit (no outline pending). */
function bundledAdminTarget(viewer, admin, query) {
const outline = bundledAdminOutline(viewer, admin);
const [west, south, east, north] = admin.bbox;
console.log(
`[Resolver] "${query}": bundled ${admin.kind} "${admin.name}"` +
`${admin.region ? `, ${admin.region}` : ''} (${admin.source}, ` +
`${admin.polygons.length} part(s), ${admin.candidates} candidate(s)) → FINAL source=bundled`,
);
return {
lon: outline.lon,
lat: outline.lat,
height: outline.height,
ring: outline.ring,
polygons: outline.polygons,
footprintKind: 'area',
buildingHeight: null,
label: admin.name,
source: 'bundled',
synthesized: false,
outlineUnavailable: false,
viewport: {
low: { latitude: south, longitude: west },
high: { latitude: north, longitude: east },
},
};
}
/**
* Synthesize an approximate circular AREA by buffering a label point. Returns a ring
* marked `synthesized:true` so the renderers draw it "approximate" (dashed/feathered),
@@ -1001,7 +1122,8 @@ export function createAnnotationResolver({
{ lat, lon },
{ signal: signal },
);
if (isRateLimitedOutcome(candidates)) return candidates;
if (isUnavailableCapability(candidates) || isRateLimitedOutcome(candidates))
return candidates;
if (!candidates) return undefined; // transient upstream failure (vs null = definitive miss)
const queryWords = normalizedWords(query);
@@ -1064,7 +1186,8 @@ export function createAnnotationResolver({
// without first trying the place= polygon (and a named landuse via the caller).
if (scope === 'neighborhood' && (!best || bestCoverage < 0.8)) {
const place = await fetchPlaceArea(lat, lon, query, signal);
if (isRateLimitedOutcome(place)) return place;
if (isUnavailableCapability(place) || isRateLimitedOutcome(place))
return place;
if (place) {
cacheWrite(footprintCache, cacheKey, place);
return place;
@@ -1094,7 +1217,8 @@ export function createAnnotationResolver({
const relEls = await featureSource.getAreaGeometry(cand.el.id, {
signal: signal,
});
if (isRateLimitedOutcome(relEls)) return relEls;
if (isUnavailableCapability(relEls) || isRateLimitedOutcome(relEls))
return relEls;
if (relEls === null) {
transient = true;
break;
@@ -1123,7 +1247,8 @@ export function createAnnotationResolver({
if (exceedsScopeArea(fp, scope)) {
if (scope === 'neighborhood') {
const place = await fetchPlaceArea(lat, lon, query, signal);
if (isRateLimitedOutcome(place)) return place;
if (isUnavailableCapability(place) || isRateLimitedOutcome(place))
return place;
if (place) {
cacheWrite(footprintCache, cacheKey, place);
return place;
@@ -1146,7 +1271,8 @@ export function createAnnotationResolver({
// caller relies on that to gate named-landuse fallback / synthesis.
if (scope === 'neighborhood') {
const place = await fetchPlaceArea(lat, lon, query, signal);
if (isRateLimitedOutcome(place)) return place;
if (isUnavailableCapability(place) || isRateLimitedOutcome(place))
return place;
if (place) {
cacheWrite(footprintCache, cacheKey, place);
return place;
@@ -1168,7 +1294,7 @@ export function createAnnotationResolver({
{ lat, lon },
{ signal: signal },
);
if (isRateLimitedOutcome(els)) return els;
if (isUnavailableCapability(els) || isRateLimitedOutcome(els)) return els;
if (els === null) return undefined; // transient upstream failure (vs [] = no match)
let bestRing = null;
let bestScore = 0;
@@ -1217,7 +1343,8 @@ export function createAnnotationResolver({
{ lat, lon },
{ signal: signal },
);
if (isRateLimitedOutcome(areaEls)) return areaEls;
if (isUnavailableCapability(areaEls) || isRateLimitedOutcome(areaEls))
return areaEls;
if (areaEls) {
let bestRing = null;
let bestScore = 0;
@@ -1250,7 +1377,8 @@ export function createAnnotationResolver({
{ lat, lon },
{ signal: signal },
);
if (isRateLimitedOutcome(wayEls)) return wayEls;
if (isUnavailableCapability(wayEls) || isRateLimitedOutcome(wayEls))
return wayEls;
if (wayEls) {
const segments = [];
for (const el of wayEls) {
@@ -1323,7 +1451,8 @@ export function createAnnotationResolver({
{ lat, lon },
{ signal: signal },
);
if (isRateLimitedOutcome(elements)) return elements;
if (isUnavailableCapability(elements) || isRateLimitedOutcome(elements))
return elements;
if (elements === null || signal?.aborted) return undefined;
const fp = selectFootprint(elements, lat, lon, query, mode);
if (fp) {
@@ -1371,7 +1500,8 @@ export function createAnnotationResolver({
{ lat, lon },
{ signal: signal },
);
if (isRateLimitedOutcome(elements)) return elements;
if (isUnavailableCapability(elements) || isRateLimitedOutcome(elements))
return elements;
if (elements === null) return undefined; // transient upstream failure (vs [] = definitive no-match)
const queryWords = normalizedWords(query);
@@ -1494,7 +1624,11 @@ export function createAnnotationResolver({
pending = featureSource
.getMonuments({ lat, lon }, { signal: undefined })
.then((elements) => {
if (elements === null || isRateLimitedOutcome(elements))
if (
elements === null ||
isUnavailableCapability(elements) ||
isRateLimitedOutcome(elements)
)
return null; // transient — NEVER cached (a poisoned bucket
// would silently disable the snap for the whole session, field test 7 §1)
const feats = [];
@@ -1877,14 +2011,22 @@ export function createAnnotationResolver({
* resolve to a region-like boundary — the analyst engine reports that
* honestly rather than silently scoping to nothing.
*
* The geocode + admin-boundary rung is capped at `budgetMs`. Past it the
* call returns `{name, ring: null, error: 'region-timeout'}` so a voice
* answer is not held for tens of seconds; the lookup keeps running and
* fills the boundary cache, so asking again shortly is fast.
*
* @param {string} name e.g. "Texas", "the Alps", "France", "Gulf of Mexico"
* @param {AbortSignal} [signal]
* @returns {Promise<{name:string, ring:Array<[number,number]>}|null>}
* @param {object} [placeSearch]
* @param {{budgetMs?: number}} [options] `Infinity` waits for the lookup.
* @returns {Promise<{name:string, ring:Array<[number,number]>}|{name:string, ring:null, error:'region-timeout'}|null>}
*/
async function resolveRegionRingForQuery(
name,
signal,
placeSearch = unavailablePlaceSearch,
{ budgetMs = REGION_FALLBACK_BUDGET_MS } = {},
) {
signal = lifetime
? signal
@@ -1901,12 +2043,42 @@ export function createAnnotationResolver({
const ring = [...ne.polygons].sort((a, b) => b.length - a.length)[0];
if (ring?.length >= 3) return { name: ne.name, ring };
}
// Bundled states/provinces/counties by name (offline); the main part only,
// like the Natural Earth rung above.
const admin = await findAdminArea(q).catch(() => null);
if (admin) return { name: admin.name, ring: [...admin.ring] };
const lookup = resolveAdminRegionRing(q, signal, placeSearch);
if (!Number.isFinite(budgetMs)) return lookup;
let timer;
const timeout = new Promise((resolve) => {
timer = setTimeout(
() => resolve({ name: q, ring: null, error: 'region-timeout' }),
budgetMs,
);
});
try {
return await Promise.race([lookup, timeout]);
} finally {
clearTimeout(timer);
}
}
async function resolveAdminRegionRing(q, signal, placeSearch) {
const geo = await geocodePlace(q, null, signal, placeSearch).catch(
() => null,
);
if (!geo) return null;
const scope = scopeFromTypes(geo.types);
if (!['country', 'state', 'county', 'city'].includes(scope)) return null;
if (scope === 'state' || scope === 'county' || scope === 'country') {
const admin = await findAdminAreaAt(
[q, geo.primaryName],
geo.lat,
geo.lon,
scope,
).catch(() => null);
if (admin) return { name: q, ring: [...admin.ring] };
}
const fp = await fetchAdminArea(geo.lat, geo.lon, q, scope, signal).catch(
() => null,
);
src/annotations/screenAnnotationRenderer.js
+28 -4
View File
@@ -161,7 +161,12 @@ export function createScreenAnnotationRenderer(
filter: 'url(#gev-sketch)',
});
group.appendChild(parts.poly);
parts.label = makeCallout(anno.label, c);
parts.label = makeCallout(
anno.outlineUnavailable
? `${anno.label || ''} · Detailed outline unavailable`
: anno.label,
c,
);
if (parts.label) group.appendChild(parts.label.node);
} else if (anno.type === 'arrow' && anno.to) {
parts.path = svgEl('path', {
@@ -174,7 +179,12 @@ export function createScreenAnnotationRenderer(
});
ensureArrowMarker(defs, anno.color || 'primary', c);
group.appendChild(parts.path);
parts.label = makeCallout(anno.label, c);
parts.label = makeCallout(
anno.outlineUnavailable
? `${anno.label || ''} · Detailed outline unavailable`
: anno.label,
c,
);
if (parts.label) group.appendChild(parts.label.node);
} else if (
anno.type === 'route' &&
@@ -203,7 +213,12 @@ export function createScreenAnnotationRenderer(
group.appendChild(dot);
return dot;
});
parts.label = makeCallout(anno.label, c);
parts.label = makeCallout(
anno.outlineUnavailable
? `${anno.label || ''} · Detailed outline unavailable`
: anno.label,
c,
);
if (parts.label) group.appendChild(parts.label.node);
} else {
// pin / highlight / label — pulsing target rings + a marker dot + callout
@@ -242,7 +257,12 @@ export function createScreenAnnotationRenderer(
'stroke-opacity': '0.7',
});
group.appendChild(parts.leader);
parts.label = makeCallout(anno.label, c);
parts.label = makeCallout(
anno.outlineUnavailable
? `${anno.label || ''} · Detailed outline unavailable`
: anno.label,
c,
);
if (parts.label) group.appendChild(parts.label.node);
}
@@ -301,6 +321,10 @@ export function createScreenAnnotationRenderer(
return;
}
rec.anno = anno;
if (anno.outlineUnavailable && rec.parts.label) {
rec.parts.label.text.textContent = `${anno.label || ''} · Detailed outline unavailable`;
sizeCallout(rec.parts.label);
}
if (empty) {
for (const child of Array.from(rec.group.children)) child.remove();
src/annotations/worldAnnotationRenderer.js
+92 -95
View File
@@ -1,4 +1,5 @@
import * as Cesium from 'cesium';
import { showOsmCredit, hideOsmCredit } from '../data/dataCredits.js';
/**
* World-space annotation renderer (Direction A).
@@ -10,7 +11,7 @@ import * as Cesium from 'cesium';
* IMPORTANT — drawing on Google Photorealistic 3D Tiles:
* The Cesium globe is hidden, so there is no terrain to clamp to. Ground
* geometry (areas, rings, connectors) is draped onto the photoreal tiles with
* `classificationType: CESIUM_3D_TILE`; points and labels clamp to the tile
* `classificationType: CESIUM_3D_TILE`; points clamp to the tile
* surface with `heightReference: CLAMP_TO_GROUND` (which requires the tileset
* to have `enableCollision = true`, set in initAnnotations). This keeps marks
* sitting ON the world instead of buried at sea level.
@@ -49,6 +50,7 @@ const CLAMP = Cesium.HeightReference.CLAMP_TO_GROUND;
export function createWorldAnnotationRenderer(viewer) {
const dataSource = new Cesium.CustomDataSource('gev-annotations');
viewer.dataSources.add(dataSource);
const osmAnnotations = new Set();
// Register the GevRouteFlow fabric once so Cesium's Material.fromType() can build the
// material the route pipeline renders. The animated `time` uniform is read straight
@@ -159,51 +161,74 @@ export function createWorldAnnotationRenderer(viewer) {
},
}),
);
if (anno.label) entities.push(labelMarker(anno, base, { point: false }));
} else if (anno.ring && anno.ring.length >= 3) {
// Larger area (district / compound / park) → flat fill draped on the tiles
// + a glowing outline. Draping is right here: you can't extrude a whole
// neighbourhood, and the GIS overlay shows the boundary clearly.
const fillPositions = Cesium.Cartesian3.fromDegreesArray(
anno.ring.flat(),
);
entities.push(
dataSource.entities.add({
polygon: {
hierarchy: new Cesium.PolygonHierarchy(fillPositions),
// Synthesized (approximate) areas get a fainter fill so they don't read as solid.
material: new Cesium.ColorMaterialProperty(
liveColor(anno, base, {
alpha: anno.synthesized ? 0.1 : 0.2,
pulse: true,
}),
),
classificationType: CLASSIFY,
},
// Larger area (district / compound / park / state) → flat fill draped on
// the tiles + a glowing outline. Draping is right here: you can't extrude
// a whole neighbourhood, and the GIS overlay shows the boundary clearly.
// A multi-part outline (`anno.polygons`: Hawaii's islands, Berlin as a
// hole in Brandenburg) draws every part with its holes. All parts share
// ONE fill and ONE line material, so Cesium batches the whole mark into a
// ground primitive per kind instead of one per island.
const parts =
Array.isArray(anno.polygons) && anno.polygons.length
? anno.polygons
: [[anno.ring]];
const fillMaterial = new Cesium.ColorMaterialProperty(
// Synthesized (approximate) areas get a fainter fill so they don't read as solid.
liveColor(anno, base, {
alpha: anno.synthesized ? 0.1 : 0.2,
pulse: true,
}),
);
entities.push(
dataSource.entities.add({
polyline: {
positions: fillPositions,
width: 6,
// Synthesized → DASHED outline (signals "approximate, not an authoritative
// boundary", research §8.4/§8.6); real footprints → solid glow.
material: anno.synthesized
? new Cesium.PolylineDashMaterialProperty({
color: liveColor(anno, base, { alpha: 0.95 }),
dashLength: 24,
})
: new Cesium.PolylineGlowMaterialProperty({
glowPower: 0.35,
color: liveColor(anno, base, { alpha: 1 }),
}),
clampToGround: true,
classificationType: CLASSIFY,
},
}),
);
if (anno.label) entities.push(labelMarker(anno, base, { point: false }));
// Synthesized → DASHED outline (signals "approximate, not an authoritative
// boundary", research §8.4/§8.6); real footprints → solid glow.
const lineMaterial = anno.synthesized
? new Cesium.PolylineDashMaterialProperty({
color: liveColor(anno, base, { alpha: 0.95 }),
dashLength: 24,
})
: new Cesium.PolylineGlowMaterialProperty({
glowPower: 0.35,
color: liveColor(anno, base, { alpha: 1 }),
});
for (const [outer, ...holes] of parts) {
if (!Array.isArray(outer) || outer.length < 3) continue;
const rings = [outer, ...holes.filter((h) => h?.length >= 3)];
entities.push(
dataSource.entities.add({
polygon: {
hierarchy: new Cesium.PolygonHierarchy(
Cesium.Cartesian3.fromDegreesArray(outer.flat()),
rings
.slice(1)
.map(
(hole) =>
new Cesium.PolygonHierarchy(
Cesium.Cartesian3.fromDegreesArray(hole.flat()),
),
),
),
material: fillMaterial,
classificationType: CLASSIFY,
},
}),
);
for (const ring of rings) {
entities.push(
dataSource.entities.add({
polyline: {
positions: Cesium.Cartesian3.fromDegreesArray(
closedRing(ring).flat(),
),
width: 6,
material: lineMaterial,
clampToGround: true,
classificationType: CLASSIFY,
},
}),
);
}
}
} else if (
anno.type === 'route' &&
Array.isArray(anno.path) &&
@@ -228,7 +253,8 @@ export function createWorldAnnotationRenderer(viewer) {
},
}),
);
if (anno.label) entities.push(labelMarker(anno, base, { point: false }));
osmAnnotations.add(anno);
showOsmCredit(viewer, anno);
} else if (anno.type === 'arrow' && anno.to) {
// Connector draped across the ground from origin to destination.
const positions = [
@@ -248,18 +274,6 @@ export function createWorldAnnotationRenderer(viewer) {
},
}),
);
if (anno.label) {
const mid = {
lon: (anno.anchor.lon + anno.to.lon) / 2,
lat: (anno.anchor.lat + anno.to.lat) / 2,
};
entities.push(
dataSource.entities.add({
position: Cesium.Cartesian3.fromDegrees(mid.lon, mid.lat),
label: labelGraphic(anno, base),
}),
);
}
} else {
// pin / highlight / label — a camera-proportional target ring + a marker.
if (anno.type !== 'label') {
@@ -284,54 +298,28 @@ export function createWorldAnnotationRenderer(viewer) {
}),
);
}
entities.push(labelMarker(anno, base, { point: true }));
entities.push(pointMarker(anno, base));
}
}
// A clamped point + (optional) label that sits on the tile surface.
function labelMarker(anno, base, { point }) {
// Captions belong to the hybrid renderer's screen-space callouts.
function pointMarker(anno, base) {
return dataSource.entities.add({
position: Cesium.Cartesian3.fromDegrees(anno.anchor.lon, anno.anchor.lat),
point: point
? {
pixelSize: anno.type === 'label' ? 8 : 14,
color: liveColor(anno, base, { alpha: 1 }),
outlineColor: liveColor(anno, Cesium.Color.WHITE, { alpha: 0.95 }),
outlineWidth: 3,
heightReference: CLAMP,
disableDepthTestDistance: Number.POSITIVE_INFINITY,
}
: undefined,
label: anno.label ? labelGraphic(anno, base) : undefined,
point: {
pixelSize: anno.type === 'label' ? 8 : 14,
color: liveColor(anno, base, { alpha: 1 }),
outlineColor: liveColor(anno, Cesium.Color.WHITE, { alpha: 0.95 }),
outlineWidth: 3,
heightReference: CLAMP,
disableDepthTestDistance: Number.POSITIVE_INFINITY,
},
});
}
function labelGraphic(anno, base) {
return {
text: anno.label,
font: '600 14px "Inter", system-ui, sans-serif',
fillColor: liveColor(anno, Cesium.Color.WHITE, { alpha: 1 }),
outlineColor: liveColor(anno, Cesium.Color.BLACK, { alpha: 0.85 }),
outlineWidth: 3,
style: Cesium.LabelStyle.FILL_AND_OUTLINE,
verticalOrigin: Cesium.VerticalOrigin.BOTTOM,
pixelOffset: new Cesium.Cartesian2(0, -16),
showBackground: true,
backgroundColor: new Cesium.CallbackProperty(
() =>
Cesium.Color.fromCssColorString('#0b1622').withAlpha(
0.72 * (anno.alpha ?? 1),
),
false,
),
backgroundPadding: new Cesium.Cartesian2(8, 5),
heightReference: CLAMP,
disableDepthTestDistance: Number.POSITIVE_INFINITY,
scaleByDistance: new Cesium.NearFarScalar(500, 1.05, 14000, 0.55),
};
}
function remove(anno) {
hideOsmCredit(viewer, anno);
osmAnnotations.delete(anno);
if (!anno?._entities) return;
for (const entity of anno._entities) {
try {
@@ -348,6 +336,8 @@ export function createWorldAnnotationRenderer(viewer) {
}
function destroy() {
for (const anno of osmAnnotations) hideOsmCredit(viewer, anno);
osmAnnotations.clear();
try {
viewer.dataSources.remove(dataSource, true);
} catch {
@@ -452,6 +442,13 @@ FlowMaterialProperty.prototype.equals = function equals(other) {
return this === other;
};
/** The ring with its first vertex repeated at the end, so an outline closes. */
function closedRing(ring) {
const first = ring[0];
const last = ring[ring.length - 1];
return first[0] === last[0] && first[1] === last[1] ? ring : [...ring, first];
}
/** Evenly down-sample a [[lon,lat],...] ring to at most n points (keeps shape). */
function decimateRing(ring, n) {
if (ring.length <= n) return ring;
src/app/atmosphereCompat.js
+189
View File
@@ -0,0 +1,189 @@
/**
* Workaround for Apple's Metal/ANGLE shader translator rejecting Cesium's
* per-vertex model atmosphere.
*
* Cesium's `AtmosphereStageVS` calls a function whose `out` parameters are
* bound DIRECTLY to varyings:
*
* czm_computeGroundAtmosphereScattering(
* v_positionWC, lightDirection,
* v_atmosphereRayleighColor, v_atmosphereMieColor, v_atmosphereOpacity);
*
* ANGLE's Metal backend wraps each `out` argument in an `ANGLE_out(...)`
* helper that takes a `thread T &`. Vertex-output struct members live in
* `__metal_generic`, which cannot bind to `thread`, so the program fails to
* LINK (not compile) and Cesium tears the render loop down:
*
* RuntimeError: Program failed to link. ... MSL compilation error:
* reference to type 'thread float3' could not bind to an lvalue of type
* '__metal_generic float3'
*
* This is the only shader in Cesium that binds varyings to `out` params --
* SkyAtmosphere and the ground-atmosphere fragment path both route through
* locals first, so they are unaffected and stay on.
*
* Cesium attaches that stage in `ModelSceneGraph.configurePipeline` under a
* single condition:
*
* const fogRenderable = frameState.fog.enabled && frameState.fog.renderable;
* if (fogRenderable) modelPipelineStages.push(AtmospherePipelineStage);
*
* so clearing `scene.fog.renderable` keeps the broken stage out of the
* pipeline entirely -- the program is never built, so it cannot fail to link.
*
* We clear `renderable` rather than `enabled` deliberately:
* - `Fog.update` still computes fog density when only `renderable` is off,
* preserving the distance-based screen-space-error scaling that governs
* 3D Tiles refinement. Clearing `enabled` returns early and forfeits it.
* - `visualSettings.js` saves and restores `scene.fog.enabled` around IR
* boost. Parking our fix on `enabled` would let that restore resurrect
* the broken shader mid-session; `renderable` is orthogonal to it.
*
* @module app/atmosphereCompat
*/
/** Minimal vertex shader reproducing the exact out-param-to-varying bind. */
const PROBE_VERTEX_SHADER = `#version 300 es
precision highp float;
in vec3 a_position;
out vec3 v_probe;
void writeOut(out vec3 value) { value = vec3(1.0); }
void main() {
writeOut(v_probe);
gl_Position = vec4(a_position, 1.0);
}`;
const PROBE_FRAGMENT_SHADER = `#version 300 es
precision highp float;
in vec3 v_probe;
out vec4 fragColor;
void main() { fragColor = vec4(v_probe, 1.0); }`;
/**
* True when the user agent is iOS or iPadOS.
*
* iPadOS 13+ defaults to desktop-mode browsing and reports `MacIntel` with no
* iPad token, so it is identified by touch points instead -- real Macs report
* `maxTouchPoints` of 0, including Apple Silicon ones that also claim
* `MacIntel`. Brave/Chrome/Firefox on iOS are all WebKit underneath and share
* the same Metal backend, so matching the platform rather than the browser is
* what we want.
*/
export function isAppleMobilePlatform(navigatorLike = globalThis.navigator) {
if (!navigatorLike) return false;
const ua = String(navigatorLike.userAgent || '');
if (/\b(iPad|iPhone|iPod)\b/.test(ua)) return true;
// Desktop-mode iPadOS presents as a Mac. What separates it from a real Mac
// is touch: Macs report 0 touch points, including Apple Silicon ones that
// also claim `MacIntel`. `navigator.platform` is deprecated and may be
// empty, so the UA's Macintosh token is accepted as an equal signal rather
// than relying on platform alone.
const platform = String(navigatorLike.platform || '');
const macLike =
platform === 'MacIntel' ||
platform === 'MacARM' ||
/\bMacintosh\b/.test(ua);
const touchPoints = Number(navigatorLike.maxTouchPoints) || 0;
return macLike && touchPoints > 1;
}
/**
* Link the probe program and report whether the driver rejects the pattern.
*
* Returns `false` when WebGL2 is unavailable so that a missing context is
* never mistaken for a broken driver -- Cesium has its own handling for that,
* and the caller still falls back to platform detection.
*/
export function probeOutParamVaryingLinkFailure(gl) {
if (!gl) return false;
const vs = gl.createShader(gl.VERTEX_SHADER);
const fs = gl.createShader(gl.FRAGMENT_SHADER);
const program = gl.createProgram();
if (!vs || !fs || !program) return false;
try {
gl.shaderSource(vs, PROBE_VERTEX_SHADER);
gl.compileShader(vs);
gl.shaderSource(fs, PROBE_FRAGMENT_SHADER);
gl.compileShader(fs);
// A compile failure here is not the bug we are probing for: the driver
// rejects this pattern at LINK time, after both stages compile cleanly.
if (
!gl.getShaderParameter(vs, gl.COMPILE_STATUS) ||
!gl.getShaderParameter(fs, gl.COMPILE_STATUS)
)
return false;
gl.attachShader(program, vs);
gl.attachShader(program, fs);
gl.linkProgram(program);
return !gl.getProgramParameter(program, gl.LINK_STATUS);
} catch {
return false;
} finally {
gl.deleteShader(vs);
gl.deleteShader(fs);
gl.deleteProgram(program);
}
}
/** Build a throwaway WebGL2 context for the probe, or null if unavailable. */
function defaultProbeContext() {
try {
const canvas =
typeof OffscreenCanvas === 'function'
? new OffscreenCanvas(1, 1)
: globalThis.document?.createElement('canvas');
return canvas?.getContext('webgl2') ?? null;
} catch {
return null;
}
}
/**
* Decide whether the per-vertex model atmosphere must be kept out of the
* pipeline on this device.
*
* The probe is authoritative when it fires, so a future driver fix silently
* restores the effect without a code change. Platform detection is the
* backstop for the case where no context can be created -- on Apple mobile a
* link failure is fatal and unrecoverable, so we would rather lose fog than
* gamble the whole render loop on the probe having run.
*/
export function shouldDisableModelAtmosphere({
navigatorLike = globalThis.navigator,
createProbeContext = defaultProbeContext,
} = {}) {
const gl = createProbeContext();
if (gl) {
let linkFails;
try {
linkFails = probeOutParamVaryingLinkFailure(gl);
} finally {
// Browsers cap live WebGL contexts (Safari especially); release the
// throwaway probe context now rather than waiting for GC.
try {
gl.getExtension?.('WEBGL_lose_context')?.loseContext();
} catch {
// A context that cannot be released is left to GC.
}
}
if (linkFails) return true;
// Probe ran and the driver accepted the pattern. Apple mobile is still
// quarantined: the probe program is far simpler than Cesium's, and a
// false negative there costs the entire scene.
return isAppleMobilePlatform(navigatorLike);
}
return isAppleMobilePlatform(navigatorLike);
}
/**
* Keep Cesium's broken model-atmosphere stage out of the pipeline.
*
* Returns true when the workaround was applied. Safe to call on any scene;
* scenes without a fog object are left untouched.
*/
export function applyModelAtmosphereWorkaround(scene, options = {}) {
if (!scene?.fog) return false;
if (!shouldDisableModelAtmosphere(options)) return false;
scene.fog.renderable = false;
return true;
}
src/app/atmosphereCompat.test.mjs
+221
View File
@@ -0,0 +1,221 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
isAppleMobilePlatform,
probeOutParamVaryingLinkFailure,
shouldDisableModelAtmosphere,
applyModelAtmosphereWorkaround,
} from './atmosphereCompat.js';
const IPAD_OS12 = {
userAgent:
'Mozilla/5.0 (iPad; CPU OS 12_5 like Mac OS X) AppleWebKit/605.1.15',
platform: 'iPad',
maxTouchPoints: 5,
};
// iPadOS 13+ desktop mode: no iPad token, reports as a Mac.
const IPAD_DESKTOP_MODE = {
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
platform: 'MacIntel',
maxTouchPoints: 5,
};
const MAC_DESKTOP = {
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
platform: 'MacIntel',
maxTouchPoints: 0,
};
const LINUX_DESKTOP = {
userAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/141',
platform: 'Linux x86_64',
maxTouchPoints: 0,
};
/** Minimal GL double. `linkOk: false` mimics the Metal translator's refusal. */
function fakeGl({ linkOk = true, compileOk = true, throwOn = null } = {}) {
const deleted = { shaders: 0, programs: 0 };
return {
deleted,
VERTEX_SHADER: 1,
FRAGMENT_SHADER: 2,
COMPILE_STATUS: 3,
LINK_STATUS: 4,
createShader: () => ({}),
createProgram: () => ({}),
shaderSource() {
if (throwOn === 'shaderSource') throw new Error('context lost');
},
compileShader() {},
attachShader() {},
linkProgram() {},
getShaderParameter: () => compileOk,
getProgramParameter: () => linkOk,
deleteShader: () => (deleted.shaders += 1),
deleteProgram: () => (deleted.programs += 1),
};
}
test('identifies iPadOS in both legacy and desktop-mode guises', () => {
assert.equal(isAppleMobilePlatform(IPAD_OS12), true);
assert.equal(isAppleMobilePlatform(IPAD_DESKTOP_MODE), true);
});
test('does not mistake a real Mac or a Linux desktop for Apple mobile', () => {
// Regression guard: the desktop-mode check keys on touch points, and a Mac
// reports the same MacIntel platform string with zero of them.
assert.equal(isAppleMobilePlatform(MAC_DESKTOP), false);
assert.equal(isAppleMobilePlatform(LINUX_DESKTOP), false);
});
test('treats a missing navigator as not-Apple rather than throwing', () => {
assert.equal(isAppleMobilePlatform(undefined), false);
assert.equal(isAppleMobilePlatform(null), false);
});
test('detects desktop-mode iPadOS without navigator.platform', () => {
// `navigator.platform` is deprecated and can be absent or stripped; the UA's
// Macintosh token plus touch points has to carry the detection on its own.
assert.equal(
isAppleMobilePlatform({
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
maxTouchPoints: 5,
}),
true,
);
// Same UA, no touch: a real Mac, which must stay untouched.
assert.equal(
isAppleMobilePlatform({
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
maxTouchPoints: 0,
}),
false,
);
});
test('probe reports failure only when the program fails to link', () => {
assert.equal(
probeOutParamVaryingLinkFailure(fakeGl({ linkOk: false })),
true,
);
assert.equal(
probeOutParamVaryingLinkFailure(fakeGl({ linkOk: true })),
false,
);
});
test('a compile failure is not reported as the link bug', () => {
// The Metal defect surfaces at link time; both stages compile cleanly.
// Treating a compile error as the bug would disable fog for the wrong reason.
assert.equal(
probeOutParamVaryingLinkFailure(
fakeGl({ compileOk: false, linkOk: false }),
),
false,
);
});
test('probe releases its GL objects on both paths', () => {
const ok = fakeGl({ linkOk: true });
probeOutParamVaryingLinkFailure(ok);
assert.equal(ok.deleted.shaders, 2);
assert.equal(ok.deleted.programs, 1);
const thrown = fakeGl({ throwOn: 'shaderSource' });
probeOutParamVaryingLinkFailure(thrown);
assert.equal(thrown.deleted.shaders, 2, 'shaders freed even when GL throws');
assert.equal(thrown.deleted.programs, 1);
});
test('probe treats a null context as healthy, not broken', () => {
assert.equal(probeOutParamVaryingLinkFailure(null), false);
});
test('a failing probe disables the stage on any platform', () => {
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: LINUX_DESKTOP,
createProbeContext: () => fakeGl({ linkOk: false }),
}),
true,
);
});
test('a passing probe still quarantines Apple mobile but frees desktops', () => {
const createProbeContext = () => fakeGl({ linkOk: true });
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: IPAD_DESKTOP_MODE,
createProbeContext,
}),
true,
);
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: LINUX_DESKTOP,
createProbeContext,
}),
false,
);
});
test('falls back to platform detection when no context can be made', () => {
const createProbeContext = () => null;
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: IPAD_OS12,
createProbeContext,
}),
true,
);
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: MAC_DESKTOP,
createProbeContext,
}),
false,
);
});
test('applies to the scene by clearing renderable, never enabled', () => {
// `enabled` must survive: Fog.update short-circuits when it is false, which
// forfeits the density used for 3D Tiles screen-space-error scaling, and
// visualSettings' IR boost saves/restores that same flag.
const scene = { fog: { enabled: true, renderable: true } };
const applied = applyModelAtmosphereWorkaround(scene, {
navigatorLike: IPAD_DESKTOP_MODE,
createProbeContext: () => null,
});
assert.equal(applied, true);
assert.equal(scene.fog.renderable, false);
assert.equal(scene.fog.enabled, true);
});
test('leaves unaffected devices completely untouched', () => {
const scene = { fog: { enabled: true, renderable: true } };
const applied = applyModelAtmosphereWorkaround(scene, {
navigatorLike: LINUX_DESKTOP,
createProbeContext: () => fakeGl({ linkOk: true }),
});
assert.equal(applied, false);
assert.equal(scene.fog.renderable, true);
});
test('tolerates a scene with no fog object', () => {
assert.equal(applyModelAtmosphereWorkaround({}, {}), false);
assert.equal(applyModelAtmosphereWorkaround(null, {}), false);
});
test('releases the throwaway probe context after probing', () => {
let lost = 0;
const gl = fakeGl({ linkOk: true });
gl.getExtension = (name) =>
name === 'WEBGL_lose_context' ? { loseContext: () => (lost += 1) } : null;
shouldDisableModelAtmosphere({
navigatorLike: LINUX_DESKTOP,
createProbeContext: () => gl,
});
assert.equal(lost, 1, 'probe context must not linger until GC');
});
src/app/catalog.js
+1
View File
@@ -11,6 +11,7 @@ const CONTROL_LAYER_IDS = Object.freeze({
militaryAwarenessLayer: 'military-awareness',
militaryInstallationsLayer: 'military-installations',
rocketLaunchesLayer: 'rocket-launches',
localAdsbLayer: 'local-adsb',
});
/** Capture the ordered application instances and their serialization metadata. */
src/app/catalog.test.mjs
+2
View File
@@ -93,6 +93,7 @@ test('controls bind catalog instances rather than similarly named defaults', ()
'military-awareness',
'military-installations',
'rocket-launches',
'local-adsb',
];
const layers = ids.map((id) => ({ id }));
const catalog = createLayerCatalog(layers, metadata(layers));
@@ -100,6 +101,7 @@ test('controls bind catalog instances rather than similarly named defaults', ()
assert.equal(services.flightsLayer, layers[1]);
assert.equal(services.transitLayer, layers[7]);
assert.equal(services.aisLiveVesselsLayer, layers[8]);
assert.equal(services.localAdsbLayer, layers[12]);
assert.equal(new Set(Object.values(services)).size, layers.length);
assert.throws(
() => catalogControlServices(createLayerCatalog([], [])),
src/app/constructCatalog.js
+62 -3
View File
@@ -1,3 +1,7 @@
import { createWeatherClock } from '../layers/weather/clock.js';
import { createWeatherLayer } from '../layers/weather/index.js';
import { createCyclonesLayer } from '../layers/cyclones/index.js';
import { createWindLayer } from '../layers/wind/index.js';
import { createLayerCatalog } from './catalog.js';
import { LAYER_STATE_REGISTRY } from '../data/layerState.js';
import { createMilitaryRegistry } from '../layers/aircraft/classification.js';
@@ -9,14 +13,17 @@ import { createApplicationRadio } from './layers/radio.js';
import { createApplicationTraffic } from './layers/traffic.js';
import { createApplicationBikeshare } from './layers/bikeshare.js';
import { createApplicationDirections } from './layers/directions.js';
import { createApplicationRecentImagery } from './layers/recentImagery.js';
import { createApplicationTransit } from './layers/transit.js';
import { createApplicationInstallations } from './layers/militaryInstallations.js';
import { createApplicationSatellites } from './layers/satellites.js';
import { createApplicationLaunches } from './layers/rocketLaunches.js';
import { createApplicationAlpr } from './layers/alprCameras.js';
import { createApplicationLocalAdsb } from './layers/localAdsb.js';
import { createApplicationAwareness } from './layers/militaryAwareness.js';
import { createApplicationFirms } from './layers/firms.js';
import { createApplicationEarthquakes } from './layers/earthquakes.js';
import { createApplicationFirePerimeters } from './layers/perimeters.js';
import { createApplicationCables } from './layers/submarineCables.js';
import { createInfrastructureLayers } from '../data/infrastructure.js';
import { localGeoJsonServices } from './localGeojsonServices.js';
@@ -42,10 +49,28 @@ const SOURCE_METHODS = Object.freeze({
launches: ['getLaunches', 'getActiveTle'],
alpr: ['fetch'],
firms: ['getSnapshot'],
wind: ['getSnapshot'],
weather: ['getSnapshot'],
cyclones: ['getSnapshot'],
earthquakes: ['getSnapshot'],
'fire-perimeters': ['getSnapshot'],
cables: ['fetch'],
});
/**
* Hardware-local layers are registered like any other but never enter share
* links or stored layer state: another browser cannot have this receiver.
*/
export const LOCAL_ONLY_LAYER_METADATA = Object.freeze([
Object.freeze({ id: 'local-adsb', disposition: 'local-only' }),
]);
/** Serialization metadata for every layer the application catalog constructs. */
export const APPLICATION_LAYER_METADATA = Object.freeze([
...LAYER_STATE_REGISTRY,
...LOCAL_ONLY_LAYER_METADATA,
]);
/** Construct the current catalog without choosing any source provider.
* Scene engines remain page-owned; layers and classification have this app's lifetime.
* The manager owns layer destruction, while abort releases classification even if startup fails.
@@ -54,7 +79,7 @@ export function createApplicationCatalog({
surface,
sources,
signal,
metadata = LAYER_STATE_REGISTRY,
metadata = APPLICATION_LAYER_METADATA,
vesselOptions,
resolveAsset,
nepalBoundaryResolver,
@@ -72,9 +97,11 @@ export function createApplicationCatalog({
throw new TypeError(`Invalid catalog source: ${name}`);
}
const militaryRegistry = createMilitaryRegistry();
const weatherClock = createWeatherClock();
const dispose = () => {
signal.removeEventListener('abort', dispose);
militaryRegistry.dispose();
weatherClock.destroy();
};
signal.addEventListener('abort', dispose, { once: true });
try {
@@ -110,16 +137,26 @@ export function createApplicationCatalog({
}),
flights,
military,
createApplicationLocalAdsb({
surface,
enrichment: sources.flights,
displayParams: () => flights.getParams(),
...(resolveAsset ? { resolveAsset } : {}),
}),
createApplicationEarthquakes({ source: sources.earthquakes }),
createApplicationFirePerimeters({
source: sources['fire-perimeters'],
}),
createApplicationAlpr({ surface, source: sources.alpr }),
satellites,
createApplicationLaunches({ source: sources.launches, satellites }),
createApplicationTraffic({ source: sources.traffic }),
createApplicationTraffic({ source: sources.traffic, surface }),
createApplicationCctv({ surface, source: sources.cctv }),
createApplicationRadio({ surface, source: sources.radio }),
createApplicationTransit({ surface, source: sources.transit }),
createApplicationBikeshare({ source: sources.bikeshare }),
createApplicationDirections(),
createApplicationRecentImagery(),
vessels,
installations,
createApplicationAwareness({
@@ -128,6 +165,23 @@ export function createApplicationCatalog({
vessels,
installations,
}),
createWindLayer({ feed: sources.wind, clock: weatherClock }),
createWeatherLayer({
feed: sources.weather,
id: 'weather-radar',
clock: weatherClock,
}),
createWeatherLayer({
feed: sources.weather,
id: 'weather-satellite',
clock: weatherClock,
}),
createWeatherLayer({
feed: sources.weather,
id: 'weather-lightning',
clock: weatherClock,
}),
createCyclonesLayer({ feed: sources.cyclones }),
...createInfrastructureLayers(localGeoJsonServices),
createApplicationCables({ source: sources.cables }),
createApplicationFirms({
@@ -141,7 +195,12 @@ export function createApplicationCatalog({
],
metadata,
);
return Object.freeze({ ...catalog, militaryRegistry, surface });
return Object.freeze({
...catalog,
militaryRegistry,
surface,
weatherClock,
});
} catch (error) {
dispose();
throw error;
src/app/constructCatalog.test.mjs
+24 -1
View File
@@ -39,7 +39,26 @@ test('catalogs construct distinct layers and classification from their supplied
signal: b.signal,
surface: fixtureSurface(b.signal),
});
assert.equal(first.layers.length, 21);
assert.equal(first.layers.length, 29);
assert.ok(first.get('local-adsb'), 'Local ADS-B is registered');
assert.deepEqual(
first.metadata.find(({ id }) => id === 'local-adsb'),
{ id: 'local-adsb', disposition: 'local-only' },
'the hardware-local layer is never serialized into links',
);
assert.notEqual(first.weatherClock, second.weatherClock);
await first.weatherClock.setTarget('2026-09-21T12:00:00.000Z');
assert.match(
first.get('wind').getRowControls().info,
/Forecast · does not follow history/,
);
assert.equal(second.get('wind').getRowControls().summary.status, null);
for (const id of ['weather-radar', 'weather-satellite', 'weather-lightning'])
assert.equal(
first.get(id).getDiagnostics().clock.target,
'2026-09-21T12:00:00.000Z',
);
assert.ok(first.get('fire-perimeters'));
assert.ok(first.get('transit'));
const order = first.layers.map(({ id }) => id);
assert.deepEqual(
@@ -82,6 +101,10 @@ test('catalogs construct distinct layers and classification from their supplied
assert.equal(first.militaryRegistry.isMilitaryIcao('def456'), false);
assert.equal(second.militaryRegistry.isMilitaryIcao('def456'), true);
a.abort();
assert.equal(
await first.weatherClock.setTarget('2026-09-21T13:00:00.000Z'),
false,
);
assert.equal(callsA[0].aborted, true);
assert.equal(first.militaryRegistry.isMilitaryIcao('abc123'), false);
assert.equal(callsB[0].aborted, false);
src/app/data.js
+5 -1
View File
@@ -1,5 +1,6 @@
import { LayerLifecycle } from '../data/lifecycle.js';
import { LayerPresentation } from './layerPresentation.js';
import { createCyberSonarScene } from '../cyberSonarScene.js';
/** Register the application layer catalog before allowing state restoration. */
export function createApplicationData({
scene: { viewer, mapStackController },
@@ -20,7 +21,9 @@ export function createApplicationData({
`Data layers could not be destroyed: ${[...dataManager.layers.keys()].join(', ')}`,
);
});
const presentation = new LayerPresentation(dataManager);
const presentation = new LayerPresentation(dataManager, {
weatherClock: catalog?.weatherClock,
});
defer(() => presentation.destroy());
onData?.(dataManager);
if (!catalog?.layers || !catalog?.metadata)
@@ -53,6 +56,7 @@ export function createApplicationData({
}
presentation.mount(document.getElementById('data-toggles'));
styleManager.attachDataManager(dataManager);
defer(createCyberSonarScene(viewer, dataManager));
return { dataManager, catalog, presentation };
}
src/app/layerPresentation.js
+9
View File
@@ -7,11 +7,13 @@ export class LayerPresentation {
constructor(
manager,
{
weatherClock,
requestRender = governorRequestRender,
invalidateDetection = markDetectionSourcesChanged,
} = {},
) {
this.manager = manager;
this.weatherClock = weatherClock;
this._panel = null;
this.pendingVisible = false;
this._unsubscribe = manager.subscribeActivity((change) => {
@@ -35,6 +37,7 @@ export class LayerPresentation {
get panel() {
if (!this._panel)
this._panel = new LayerPanel({
weatherClock: this.weatherClock,
getLayers: () => this.manager.getAll(),
isEnabled: (id) => this.manager.isEnabled(id),
setEnabled: (id, enabled, options) =>
@@ -67,6 +70,12 @@ export class LayerPresentation {
mount(container) {
this.panel.mount(container);
}
/** Hand the Recent Imagery readout factory to the panel (see LayerPanel). */
attachRecentImagery(factory) {
// Detaching never builds a panel that teardown already released.
if (factory) this.panel.attachRecentImagery(factory);
else this._panel?.attachRecentImagery(null);
}
refresh() {
this._panel?._refreshTogglePanel();
}
src/app/layers/alprCameras.js
+2
View File
@@ -1,3 +1,4 @@
import * as credits from '../../data/dataCredits.js';
import { createAlprCamerasLayer } from '../../layers/alpr/index.js';
import * as render from '../../renderGovernor.js';
import * as context from '../../data/contextStore.js';
@@ -12,6 +13,7 @@ export function createApplicationAlpr({ surface, source }) {
return createAlprCamerasLayer({
source,
services: {
credits,
render,
context,
picking,
src/app/layers/cctv.js
+2
View File
@@ -1,4 +1,5 @@
import { createCctvLayer } from '../../layers/cctv/index.js';
import * as credits from '../../data/dataCredits.js';
import * as sprites from '../../data/spriteOrder.js';
import * as activation from '../../cctvFocusRequest.js';
import * as overlays from '../../overlays/worldOverlay.js';
@@ -13,6 +14,7 @@ export function createApplicationCctv({ surface, source }) {
return createCctvLayer({
source,
services: {
credits,
sprites,
activation,
overlays,
src/app/layers/localAdsb.js
+48
View File
@@ -0,0 +1,48 @@
import { createLocalAdsbLayer } from '../../layers/localAdsb/index.js';
import { createLocalReceiverFeeds } from '../../layers/localAdsb/feeds.js';
import { SdrController } from '../../sdr/controller.js';
import * as render from '../../renderGovernor.js';
import * as context from '../../data/contextStore.js';
import * as picking from '../../data/pickRegistry.js';
import * as trails from '../../data/trailRenderer.js';
import * as geoid from '../../data/geoid.js';
import { markDetectionSourcesChanged } from '../../data/detection.js';
import { refreshTrackedReadout } from '../../data/trackedReadout.js';
/**
* Construct the Local ADS-B layer, the browser RTL-SDR session it shares with
* the Radio panel, and the decoder-feed session. Construction opens no device
* and makes no request; the SDR starts only from an explicit Connect and the
* feeds are polled only while the layer is enabled.
*
* `displayParams` reads the public Flights layer's DISPLAY-rail 3D preference
* (`{ models3d, models3dMode }`) so local aircraft follow the same toggle;
* `enrichment` is the Flights source whose cached adsbdb proxy they share.
*/
export function createApplicationLocalAdsb({
receiver = new SdrController(),
feeds = createLocalReceiverFeeds(),
surface = null,
enrichment = null,
displayParams = null,
resolveAsset = (url) =>
`${import.meta.env?.BASE_URL || '/'}${url.replace(/^\//, '')}`,
} = {}) {
return createLocalAdsbLayer({
receiver,
feeds,
resolveAsset,
services: {
render,
context,
picking,
trails,
geoid,
groundSnap: surface?.groundSnap || null,
enrichment,
display: displayParams ? { getParams: displayParams } : null,
detection: { markSourcesChanged: markDetectionSourcesChanged },
overlays: { refreshReadout: refreshTrackedReadout },
},
});
}
src/app/layers/militaryInstallations.js
+24 -1
View File
@@ -1,3 +1,5 @@
import { overlayHost } from './overlayHost.js';
import * as credits from '../../data/dataCredits.js';
import { createInstallationsLayer } from '../../layers/installations/index.js';
import * as render from '../../renderGovernor.js';
import * as context from '../../data/contextStore.js';
@@ -8,6 +10,27 @@ export function createApplicationInstallations({ surface, source }) {
const { groundFloor: ground, anchors } = surface;
return createInstallationsLayer({
source,
services: { render, context, ground, anchors, picking },
overlayHost,
services: {
credits,
render,
context,
ground,
anchors,
picking,
maps: {
subscribeMapStack(callback) {
globalThis.window?.addEventListener?.(
'gev:map-stack-changed',
callback,
);
return () =>
globalThis.window?.removeEventListener?.(
'gev:map-stack-changed',
callback,
);
},
},
},
});
}
Loaded 100 of 542 files, more files were not shown because too many files have changed in this diff. Show more