diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f5ee5c10..cd6ff006b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,9 @@ the five borough cores load and `CCTV_NYC_MAX_SOURCES` changes the cap. The catalog-wide ceiling is unchanged; extra cameras share it round-robin with the other packs. +- Street Level counts unique tiles before ranking so overlapping dateline halves + in polar views do not waste slots in the 16-tile overview budget. + - Directions no longer shows a truncation notice for a route with exactly 200 turns, and a roundabout at the 200th turn keeps its exit distance. @@ -130,6 +133,16 @@ ## [Unreleased] +- Flights and Military Flights highlight emergency transponder broadcasts in + fleet billboards, 3D models and Cockpit contacts. Colors update when a status + starts or clears. Emergency squawks outrank ADS-B priority statuses; GEV + reports minimum fuel and lifeguard without red tint. Cockpit names the code + or ADS-B status, and analyst/voice results retain broadcast provenance and + the latest known contact time. One omitted ADS-B status poll during a source + switch may retain the previous status if the squawk is unchanged; an explicit + clear takes effect immediately. Broadcasts are not confirmed incidents. + Contributed by [Leo Pechnicki](https://github.com/leopechnicki) (#856). + - Street Level: KartaView is the third provider, keyless. Light its chip (purple; off by default, one provider at a time) and zoom in below about 6 km: the lines are the streets KartaView has photos of (z13–14 tiles from @@ -213,6 +226,26 @@ largest cities) and `CCTV_VEGVESEN_VIDEO=0` keeps stills only. Attributed under NLOD 2.0. +- Import your own GeoJSON, KML, KMZ and GPX files from DISPLAY or by dropping + them onto the app. Show, hide, fly to or remove each file for the session. + GeoJSON/GPX draw on the ground; KML retains its styles and explicit heights. + Large repeated-style exports use shared local pins, while recorded track + vertices and segment gaps stay intact. Imports respect newer camera gestures, + leave the shared clock alone, and report malformed or oversized files. + File bytes stay on the device; rendered overlays may appear in shared map + screenshots. Thanks to [Chase Mills](https://github.com/kapturner) (#925). + +- GNSS Integrity (Events, voice-addressable, share token `3`) draws navigation + accuracy scores from a keyless regional adsb.lol sample; aircraft-level analyst + queries are unavailable for these aggregate cells. Cells do not intercept + map clicks. Pan/zoom refreshes automatically; feed failures retain the display + and explain the next retry. Aircraft-reported quality includes MLAT-positioned + contacts after GPS loss, excludes TIS-B/MLAT-derived quality, and locates GPS-loss + evidence at the last good GNSS position when available. Scores discount one + bad aircraft per cell and retain 30 minutes of visited-view evidence; they + do not confirm jamming. See [method and limits](DATA_SOURCES.md#gnss-integrity). + Contributed by [Leo Pechnicki](https://github.com/leopechnicki) (#855). + - Live HLS cameras play on a page opened over plain HTTP from another machine (`HOST=0.0.0.0`). The client lease id came from `crypto.randomUUID`, which browsers only expose in secure contexts, so starting a live feed threw diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 63f83e4c2..cde8582f9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -79,8 +79,8 @@ if its layer is later removed. New layers first use the remaining unreserved single-character digits (`0` through `9`), then the two-character base-36 namespace (`00` through `zz`). On the -current published ledger, `0`, `1` and `2` are already owned, so the next -choices are `3` through `9`, then `00`. After rebasing onto the latest `main`, run +current published ledger, `0`, `1`, `2` and `3` are already owned, so the next +choices are `4` through `9`, then `00`. After rebasing onto the latest `main`, run `npm run layer-token:next -- `, add that result to both the permanent JSON reservation ledger and registry entry, and keep the exact registered-layer count assertion up to date. The token is assigned against the merge-time base, not @@ -98,7 +98,7 @@ remove only your still-unpublished layer's registry entry and ledger row so the local registry is valid again, then rerun `npm run layer-token:next -- `. For example, if PR A publishes `0` while PR B still provisionally uses `0`, PR B removes only its own provisional entry, reruns the helper, and receives -`3`. Put `3` on PR B's registry entry and ledger row, update its pinned count +`4`. Put `4` on PR B's registry entry and ledger row, update its pinned count and token tests, then rerun `npm run layer-token:check -- --base-ref origin/main`. An invalid duplicate ledger can fail during module import before the helper prints anything; never fix that by renaming or deleting PR A's published token. diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 15bcfbf2a..d0897a04f 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -26,7 +26,7 @@ How to read this: | **OpenStreetMap** | Map and place data (roads, military areas and names, ALPR, place search, outlines); via vector tiles, hourly ALPR extract, bundled data, Nominatim and optional self-configured Overpass | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | © OpenStreetMap contributors | | **Google Map Tiles API** (Photorealistic 3D Tiles) + Places/Geocoding | The 3D globe, voice scene context, and on-demand nearby installation search | Google Maps Platform ToS (proprietary, your own key + billing) | "Google" / "Google Maps" logo — **shown in-app**, required | | **OpenSky Network** | Primary worldwide live-flight snapshot | Non-commercial research/education license | Schäfer et al., _"Bringing Up OpenSky"_, IPSN 2014 + opensky-network.org | -| **adsb.lol point API** | Bounded live-flight fallback when OpenSky has no usable snapshot | ODbL 1.0 | adsb.lol contributors; `api.adsb.lol/v2/lat/{lat}/lon/{lon}/dist/{radius}` | +| **adsb.lol point API** | Bounded live-flight fallback when OpenSky has no usable snapshot; GNSS Integrity layer (aircraft-reported NIC/NACp and GPS-loss integrity facts, via `/api/gnss-integrity`, 60 s server cache) | ODbL 1.0 | adsb.lol contributors; `api.adsb.lol/v2/lat/{lat}/lon/{lon}/dist/{radius}` | | **adsb.lol** | Military flights + aircraft traces | ODbL 1.0 | "adsb.lol" (ODbL) | | **adsbdb** (`api.adsbdb.com`) | Aircraft type, model name and registration by ICAO hex for enriched flights; airline and origin/destination airports for the tracked flight's callsign | No licence or rate limit is published for the service. Its own credits: aircraft data from PlaneBase, ICAO-to-N-number conversion by Guillaume Michel, and flight route data that "is the work of David Taylor, Edinburgh and Jim Mason, Glasgow, and may not be copied, published, or incorporated into other databases without the explicit permission of David J Taylor, Edinburgh" | "adsbdb — api.adsbdb.com" (courtesy); route data credited to David Taylor, Edinburgh, and Jim Mason, Glasgow | | **AISStream.io** | Live vessels (AIS) | Free, beta, no formal ToS; AIS is a public broadcast | "AISStream.io" (courtesy) | @@ -80,6 +80,41 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei - **Google Maps Platform.** You supply your own API key and are bound by [Google's ToS](https://cloud.google.com/maps-platform/terms). Google Maps Content (tiles, geocodes, places) **may not be cached, stored, rehosted, or committed** — this app only ever uses it live, which is the compliant pattern. The "Google" attribution is displayed on the globe and must stay visible. Restrict your key (see [SECURITY.md](SECURITY.md)). - **OpenSky Network.** Its license is **non-commercial**, and operational use of the REST API in a live product can require a prior written agreement with OpenSky — even for non-profit/government use. If you deploy this commercially, contact OpenSky for your own terms. The flights layer is a toggle and runs anonymously by default. - **adsb.lol flight fallback.** When OpenSky is unavailable and no last-good OpenSky response exists, the server requests a cached, capped 250 nm adsb.lol point snapshot around the current camera subpoint. This is regional observed context, not worldwide completeness; provenance is exposed in the Flights stats/context row. Military ICAOs remain reconciled through the existing dedicated military registry rather than duplicated. + + +- **GNSS integrity (adsb.lol).** A separate keyless point request samples a 250 nm + circle around the screen centre, snapped once to whole degrees. This does not + reuse the Flights fallback cache. Cache lifetime is 60 s; 429 backoff is capped + at one hour and other upstream failures cool down for 15 s. Future access may + require feeder keys under the [API terms](https://github.com/adsblol/api). + Airborne decoded ADS-B v1/v2 aircraft with a known 0–60 s position age qualify, + including MLAT-positioned aircraft still broadcasting their own quality. + TIS-B/MLAT-derived NIC and NACp are ignored. GPS-loss evidence uses + `gpsOkLat/gpsOkLon` when present, otherwise the current known position. + The proxy strips identity metadata and tags its response with classifier + `gev-nic-nacp-v2`; rows carry only integrity facts. + + The method has three separately attributed parts: + 1. **GEV classifier (unvalidated):** GPS-loss flag, aircraft-reported NACp < 8, + or aircraft-reported NIC < 7 except NIC 0 with NACp ≥ 10 (unknown containment). + The numerical minima come from US 14 CFR 91.227(c), not a published jamming + classifier. Readsb's [JSON contract](https://github.com/wiedehopf/readsb/blob/dev/README-json.md) + describes the quality field origins and GPS-loss flag. + 2. **GPSJAM aggregation:** score = `100 × max(0, bad − 1) / total`, with percent + bands < 2, 2–10, > 10. One bad aircraft is discounted. This is not the raw + share of bad aircraft. See the [GPSJAM FAQ](https://gpsjam.org/faq). + 3. **GEV window:** 30 minutes over visited views, at least three aircraft per + 0.5° cell. These bands were developed for daily totals; with fewer than ten + observations a cell can skip the middle band. Source timestamps date evidence + even through cache replays, and degraded evidence expires independently. + + Medium/high cells report navigation-integrity anomalies. Interference is one + possible explanation, not a detected or located emitter. Avionics faults and + manoeuvring military trainers can also lower reported accuracy. Blank areas + can be outside coverage or lack enough observations. Attribution: adsb.lol + contributors, ODbL 1.0; GPSJAM for aggregation only. + + - **adsbdb enrichment.** `/api/adsbdb` resolves an ICAO hex to aircraft type, model name and registration, and the tracked flight's callsign to an airline and an origin/destination pair. Fan-out is deliberately small because adsbdb is a free community API that publishes no quota: at most 4 requests in flight, dispatches dripped to roughly 5/s, and each key asked for at most once per session, with route lookups reserved for the tracked aircraft. The dev-server proxy caches each key for 24 hours in `.gev-cache/adsbdb.json`, negative answers included, so fresh entries survive server restarts. That gitignored cache remains local to the deployment and is not bundled with this repo. adsbdb publishes no general service license; its README says the route data may not be copied, published, or incorporated into other databases without the explicit permission of David J Taylor, Edinburgh. Deployers are responsible for obtaining any permission their use requires. Enrichment is best-effort and fail-silent — a flight that gets no answer keeps what the flight feed already gave it. - **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state. - **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover. diff --git a/README.md b/README.md index 50c5a4003..95c8a24e0 100644 --- a/README.md +++ b/README.md @@ -303,7 +303,9 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------- | | 🗺️ **Map Stack** | Esri satellite imagery, Google Photorealistic 3D, OSM, plus additional ion-hosted stacks | Esri / Google / Ion / OSM | 🟢 Esri satellite + OSM · 🟡 ion-hosted Google 3D + world terrain · 🔴 direct Google + place search | | ✈️ **Live Flights** | 11,000+ live aircraft + route history | OpenSky + adsb.lol | 🟢 (🟡 optional for more polling credits) | -| 🎖️ **Military Flights** | ADS-B military traffic in amber | adsb.lol | 🟢 | +| 🎖️ **Military Flights** | ADS-B military traffic in amber, emergency broadcasts in red | adsb.lol | 🟢 | +| 📡 **GNSS Integrity** | Regional navigation accuracy scores; anomalies do not confirm interference | adsb.lol, GPSJAM aggregation | 🟢 | +| 🔥 **Fire Perimeters** | Incident boundaries and reported containment | WFIGS / NIFC | 🟢 | | 🚢 **Live Vessels** | Thousands of ships worldwide | AISStream | 🟡 | | 🛰️ **Satellites** | 838-object catalog, color-coded by class with a live legend — the **DENSE** chip drops in the whole Starlink shell | CelesTrak | 🟢 | | 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 | diff --git a/docs/CODE-BOUNDARIES.md b/docs/CODE-BOUNDARIES.md index a0a2325ad..19a5ee372 100644 --- a/docs/CODE-BOUNDARIES.md +++ b/docs/CODE-BOUNDARIES.md @@ -65,11 +65,10 @@ check, not a JavaScript sandbox. Common voice controls cannot depend on a Realtime protocol implementation. Negative fixtures cover indirect helpers, self-package imports, symlinks and unreachable files. -Source factories have dedicated exports for ALPR, bikeshare, CCTV, earthquakes, -FIRMS, installations, launches, radio, satellites and traffic. They preserve the -same factory implementations without loading layer rendering. ALPR/earthquake -record normalization and CCTV source endpoint policy have plain owners separate -from geometry/cards. Source exports do not start acquisition at import time. +Source-only factories use dedicated `layers//source` exports where +available, preserving the same factory implementation without loading rendering. +Record normalization and endpoint policies have plain owners separate from +geometry/cards. Source exports do not start acquisition at import time. Other layer `ingestion.js` files may still coordinate Cesium resources; the portable contract applies to the explicitly reviewed graphs above. Their layer @@ -630,9 +629,9 @@ explicit operations. Both exports exclude Cesium, DOM and application assembly. The snapshot renderer applies record changes; rendering owns a weak map of geometry and billboard resources used by cards, picking and trails. -`gods-eye-view/sources/reference` constructs fresh earthquake and bundled cable +`gods-eye-view/sources/reference` constructs fresh earthquake, GNSS integrity and bundled cable source instances independently of standalone setup. Individual sources remain -available through `layers/earthquakes/source` and `layers/submarine-cables/source`. +available through their `layers//source` exports, including `layers/gnss/source`. The latter retains the bundled dataset’s attribution and licensing requirements. ## Geocoding and feature-query providers diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 93e5214a5..e8ae3bc86 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -75,6 +75,8 @@ Windows/Claude Desktop Cowork confirmation remains outstanding (#927). Tool argument string lengths follow JSON Schema Unicode code-point counts, so supplementary characters count once rather than as two UTF-16 units. +Query timestamps outside the representable Date range are reported as null, preserving the rest of the result row. + `gods-eye-view/tools` defines queries that answer questions from the app's data, and `gods-eye-view/tools/mcp` exposes a composed catalog over the Model Context Protocol. `npm run mcp` serves Core's tools over stdio to a local MCP client, @@ -396,6 +398,10 @@ and geographic placement remain distinct; Stop/replacement releases resources. Import acquires nothing. See [data packs](DIRECTOR-DATA-PACKS.md). Existing scene content, assets and credit remain unchanged. +The asset directory source rejects with the caller's abort reason if +cancellation arrives during any read, including the final one, or while the +reader is being released. + Director version-4 documents add named geographic anchors and optional explicit camera moves with easing, duration and holds. Playback and seek share one coordinate sampler; navigation/manual input revokes authored motion and pending @@ -819,6 +825,30 @@ use the supplied asset resolver, including the preload path. Enrichment receives an abort signal and cannot update a later lifecycle after destruction. Existing camera, terrain floor, trail, selection and measured model-size policies remain. +## Transponder emergency highlighting + +`src/data/aircraftEmergency.js` derives one status per poll from the Mode A +squawk (7500 unlawful interference, 7600 radio failure, 7700 general emergency) +and the readsb ADS-B emergency/priority field where present. Higher severity +wins, with ADS-B taking a tie. GEV treats `minfuel` and `lifeguard` as priorities; +`general`, `unlawful`, `nordo` and `downed` are emergency statuses. + +Civil and military fleet billboards, 3D models and Cockpit dots turn red for +emergencies and repaint when the broadcast changes. Priorities retain normal +colors; tracked map colors keep their existing selection treatment. The Cockpit +meta line names the squawk or ADS-B status (only emergencies are red), replacing +COURSE ALIGNED while a status is present. + +Analyst and tracked voice results describe broadcasts, not confirmed incidents. +Fields include `squawk`, `emergency`, `emergencyKind`, `emergencySource` and +`emergencyBroadcast`. `lastContactAtMs` is the last known transponder contact, +not the arrival time of this status; missing contact times retain known contact. +Civil fallback preserves status alongside unchanged OpenSky vectors. One poll +omitting ADS-B status may retain the previous status when the squawk is unchanged; +explicit clears and changed squawks take effect immediately. The next omitted +poll expires that retention. `scripts/qa-emergency-squawk.mjs` checks billboard +and model transitions, Cockpit HUD, clearing and tracking release. + ## Browser live-source observations Flights, Military Flights and AIS Vessels obtain snapshots and optional history @@ -1198,11 +1228,11 @@ entries (`DEFAULT_MAX_CACHE_ENTRIES`, overridable per instance through visits to one place share a key but does not bound how many distinct places a session visits, and the cache previously had no eviction, so a long session retained every coordinate it ever resolved. Eviction is least-recently-used: -a consumer read promotes its entry, so the cell being watched is not dropped for -having been resolved early. The bound sits far above a city-scale session, so +a consumer read, including a warm hit in a batch lookup, promotes its entry, so +the cell being watched is not dropped for having been resolved early. The bound sits far above a city-scale session, so normal use never evicts and issues no additional proxy requests. A batch's -results are assembled from what that call resolved, so eviction during a large -batch cannot report a just-resolved point as unresolved. Re:Earth and +results are assembled from what that call resolved or found warm, so eviction +during a large batch cannot report one of its points as unresolved. Re:Earth and geoid-fallback entry semantics, the fallback cooldown and the abort-time flush are unchanged. @@ -1429,6 +1459,8 @@ accessible name remains "Don't show this again". Control behavior is unchanged. ## FIRMS source status +FIRMS ingestion and adaptation require explicit finite coordinates within WGS84 ranges; genuine zero coordinates remain valid and admitted detection indices stay sequential. + The FIRMS proxy records source success after appending its rows. If aggregation throws, that source reports failure without a contradictory success entry. The existing per-record append continues to support large feeds; sequential @@ -3110,8 +3142,34 @@ its criteria cannot be silently ignored. | FIRMS Active Fires ▲ | NASA FIRMS live (VIIRS ×3 NRT + MODIS NRT, trailing 24h) | `src/data/firmsHeatmap.js` | `/api/firms` (`FIRMS_MAP_KEY`) | 10 min (proxy TTL 30 min) | | Street Level 📷 | Street-level imagery; Mapillary is the first provider (vector tiles, Graph API, MapillaryJS) | `src/layers/streetLevel/` via `src/app/layers/streetLevel.js` | `/api/mapillary/status`, `/api/mapillary/tiles/coverage/{z}/{x}/{y}` (`MAPILLARY_CLIENT_TOKEN`) | camera-driven (320 ms debounce, ≤9 tiles, cached 24 h) | | Wind 🌬 | NOAA GFS 10 m wind (keyless, 0.25°→1° grid; animated particles) | `src/data/wind.js` | `/api/wind` | 1 h (forecast cycle) | +| GNSS Integrity 📡 (`gnss-integrity`) | adsb.lol point snapshot, low navigation accuracy binned per 0.5° cell (GEV NIC/NACp threshold + gpsjam.org formula and bands, keyless, ODbL) | `src/layers/gnss/` via `src/app/layers/gnss.js` | `/api/gnss-integrity` | camera-driven (600 ms debounce, minimum 2.5 s between reads) + 60s poll (server cache 60 s; 30 min rolling window) | | Fire Perimeters 🔥 | NIFC WFIGS current interagency perimeters (keyless, paged past the 2000-record cap); InciWeb catalog + incident page origin and update time checks for verified incident-page links | `src/layers/perimeters/` via `src/app/layers/perimeters.js` | `/api/fire-perimeters` + `/api/fire-perimeters/inciweb/*` | 5 min (server caches: catalog 1 h; publication 30 min) | +GNSS Integrity samples a 250 nm circle around the Earth point under the screen +centre. Camera reads debounce for 600 ms with a monotonic 2.5 s minimum gap; +nearby views share whole-degree cache keys, and a minute poll refreshes data. +Counts and bands cover visited views over 30 minutes, including regions now off +screen. Non-pickable ground primitives preserve Street Level and perimeter +selection. The legend describes a low-accuracy score, not the raw aircraft share; +`getStats().levels` exposes nominal, medium and high counts separately. + +Transient source failures keep activation enabled and show unavailable/degraded +health with the next automatic retry. Upstream 429 deadlines are capped at one +hour, with the longest concurrent deadline retained; 5xx, timeout and malformed +responses impose a 15 s server cooldown. Cache hits and shared reads do not spend +upstream admission. Cache age and cooldown use a monotonic clock; observation +age comes from the upstream snapshot and is preserved in the row and voice stats. +Snapshots older than 30 minutes or without a source timestamp are refused. +Disable/destroy cancels pending requests, timers, subscriptions and primitives. + +The GEV classifier remains unvalidated; anomalies do not confirm interference. +See [method, provenance and limits](../DATA_SOURCES.md#gnss-integrity), including +MLAT-positioned GPS loss, derived-field exclusions and the NIC 0 exception. +`scripts/qa-gnss.mjs` covers live freshness, bands, built primitives, camera/globe +anchors, recovery, keyboard and mobile states; emergency repainting is covered +by `scripts/qa-emergency-squawk.mjs`. Both accept `--timeout-ms` for slow boot. + + Fire Perimeters uses capped, timed server reads with stale-on-error caching and a per-client limit. Unchanged snapshots retain geometry; link checks cancel on disable or selection change, and the row legend shows reported containment. Street Level is a collapsible right-rail panel (`#street-level-panel`, layer @@ -3132,6 +3190,10 @@ sequence lines below 60 km; the proxy strips the unused `image` layer from z14 tiles (12 MB → ~80 KB) and shares one upstream fetch between concurrent requests for a tile. +Mapillary coverage ranks unique tile addresses before applying the 16-tile +overview budget. Overlapping dateline halves in a polar view therefore do not +waste overview slots on the same tile; loading already skips duplicate keys. + On Google 3D (`photoreal`) at street zoom (in below 1,400 m above ground, out above 1,800 m) the surface mode switches from `draped` to `terrain`: draped lines land on roofs and tree tops, so `groundCast.js` instead places lines, @@ -4447,6 +4509,18 @@ are omitted rather than framing the wrong part of the globe. - Pure half: `src/annotations/drawMode.js` (session, minimum vertices, double-click de-dup at 0.5 m, vertex ceiling, degenerate-geometry refusal, ring closing, dateline-safe length/area/centroid, hint text). Regression surface: `src/annotations/drawMode.test.mjs` and `src/data/inputOwnership.test.mjs` (pure), `src/annotations/drawToolBehaviour.test.mjs` (the tool driven against DOM and viewer doubles — leases, borrowed input actions, teardown, the Enter key, preview heights), `src/annotations/drawTool.test.mjs` (structural: where files live, which ids the markup carries, what the docs may claim), and the rendered harness `scripts/qa-draw-tool.mjs`. - The annotation tree is a declared ownership group: `./annotations` → `src/annotations/index.js` in `package.json`, with its module graph listed in `scripts/package-boundaries.json`, so an annotation module reaching outside that graph is a gate failure rather than a surprise in a layer's bundle. `drawMode.js`, `drawTool.js` and their tests are in `scripts/format-scope.json`. +### Import Your Own Files (October 2026) + +- DISPLAY ▸ **Import** reads GeoJSON (`.geojson`/`.json`), KML, KMZ and GPX from the picker or a file drop anywhere in the app. Each file gets show/hide, Fly-to and Remove controls. GeoJSON and GPX use the row's color; KML/KMZ retain their styles and show a hatched swatch. The file bytes remain on the device for the page session. Rendered overlays can appear in map screenshots sent through voice or other existing screenshot tools. +- **Ground and height.** GeoJSON XYZ coordinates become ground footprints. KML ground lines/rings are tessellated; static recorded tracks preserve their original vertices, segment gaps and explicit altitude modes. KML with absolute/relative altitude keeps that meaning. Ground pins resolve in batches of 256 against loaded terrain and, in the nearby camera footprint on photoreal maps, `clampToHeightMostDetailed`, then draw in one owned native BillboardCollection per file, with static positions and no per-frame point Entity visualizer or clamping callbacks. Point metadata remains in `dataSource.importedEntities` for framing/style inspection; non-point geometry stays in the normal source collection. Pin collections mount only after source admission, follow file show/hide, and are destroyed on removal. Settled camera/map-source changes and completed terrain loads refresh visible files through the same serial queue; distant points retain their latest available height. No detailed worldwide terrain sweep is started. Nearby mesh sampling can add requests/time; configured photoreal providers require separate validation. The depth test is disabled for ground pins, matching Draw's marker visibility policy. +- **Navigation.** Row Fly-to uses `runImmediateNavigation('file', …)`; Cockpit can refuse it and existing follow/track ownership is released first. Automatic Fly-to is stamped when a batch is submitted and reasserted before flying. Pointer/wheel gestures and newer navigation supersede it, including time spent in the queue. Framing uses every finite visible point/path/polygon-hole/wall vertex plus rotated overlay bounds, with viewport-aware range and a nadir view. The target uses available terrain/cached ground height and the ground guard on arrival. Empty footprints never fall back to deferred `viewer.flyTo`. +- **Clock.** Imports expose no source clock and no entity availability. KML times/view metadata are removed before loading; recorded tracks become static lines. GPX segments remain separate and GPX times are ignored. The shared clock stays unchanged. +- **Input handling.** Coordinates and GeoJSON geometry types are checked before Cesium allocation. Only short plain names reach GeoJSON property bags; file IDs, custom CRS and styling properties are removed. KML feature/geometry IDs are removed while Style/StyleMap references remain. Local style references are trimmed, incomplete style pairs removed, hidden-folder flags normalized to the import's show control, and unsupported prototype-named elements removed. UTF-16 BOMs and common KML namespace export defects are handled. Empty drawable content is refused. A loader failure or timeout does not hold the next file indefinitely; teardown aborts pending waits. +- **References and images.** File-authored remote links/icons/styles, network links, models and screen/photo overlays are removed. Packed KMZ and embedded PNG/JPEG/GIF/WebP/BMP images remain local; decoded dimensions are checked before image decoding against a total 16-megapixel and 8,192-pixel-side budget per file. Blocked IconStyle images fall back to local pins; authored empty icons remain empty. Colored overlays without an Icon survive; overlays with a removed external image are dropped. Metadata, native labels and descriptions are removed, and generated pins share one lossless image URL per canvas. +- **Bounds.** File size ≤50 MiB; inflated KMZ ≤120 MiB; ≤20,000 entities (including multipart children and Folder containers); ≤250,000 positions; ≤2,000 normalized style definitions and distinct image links; ≤12 files per session. Exact repeated styles share definitions without changing inheritance or inline overrides. Files of 20–30 MB fit the byte budget, subject to geometry/image limits. Refusals name a reason and batches continue; with multiple failures the hint shows the first reason. +- **Lifecycle and keyboard.** `createApplicationTools` wires `initGeoFileImport` with viewer, navigation, ground services and map-source subscription. Destroy aborts pending work, unsubscribes, removes data sources, revokes blob URLs, removes document/canvas listeners and releases `window.__gevGeoImport`. List rebuilds retain keyboard focus; removing a row hands focus to the next row, else the previous row, else Import. +- **Validation.** `node scripts/qa-geo-import.mjs` covers all four formats and real painted show/hide. Run `node scripts/qa-geo-import-regressions.mjs` against both dev and preview for XYZ/track heights, failure recovery, namespace/BOM support, portrait framing and panel drops. `node scripts/qa-geo-import-large.mjs [--file /absolute/path/sample.kml] [--expect-local-pins]` checks a 27.6 MB repeated-style file, native style/image equivalence, large-file painted output and frame times. It records the WebGL renderer; software-rendered runs do not establish GPU performance. Unit tests live beside the data/controller/framing modules. + ### Recent Imagery (September 2026) Thumbnail decoding requires explicit `Data-Present: true` evidence; false remains empty, while absent or malformed evidence reports an error. Injected `getThumbnail` sources must return an OK Response with readable `Data-Present: true` to admit image bytes, or `false` for confirmed empty coverage. Cross-origin sources must expose that header to browser JavaScript. @@ -4465,7 +4539,7 @@ Thumbnail decoding requires explicit `Data-Present: true` evidence; false remain - **The TRACKED contact's 2D↔3D handoff is DEFAULT behaviour (2026-08-19), driven by camera distance alone.** It does NOT consult the DISPLAY-rail `3D` toggle, which continues to own the FLEET (the un-instanced draw-call budget stays the operator's decision). Policy lives in `src/data/trackedModelRegime.js` and is shared by both layers: enter below `TRACKED_MODEL_ENTER_ALT_M` = 150,000 m and hand back to the billboard only above `TRACKED_MODEL_EXIT_ALT_M` = 172,500 m. **The swap distance was set by playtesting on 2026-08-20:** a first pass at 1,000,000 m switched too early; 2D reads correctly at ~600 km and the handoff belongs at ~150 km. **Consequence, recorded on purpose:** the tracked contact now enters 3D NEARER than the FLEET does (`MODEL_ALT_CEIL_M` = 800,000 m, unchanged), so with the DISPLAY-rail `3D` toggle on, 150–800 km draws surrounding contacts as models while the selected one is still a glyph. Nothing double-draws (the fleet pass skips the tracked icao) and aligning the two is a fleet-side decision, deliberately out of scope. **The two thresholds are asymmetric on purpose:** a single threshold makes a tracked orbit sitting ON the boundary strobe billboard↔model as the camera's altitude wobbles across it. Do not collapse them. The latch is scoped per selection, so a new target re-evaluates against the ENTER ceiling rather than inheriting the previous target's exit band. Exactly ONE model is involved; it loads on demand when the regime opens, is HIDDEN (not released) on regime exit so re-entry has no load gap, and is released by the existing teardown on deselect/re-track/destroy. Cockpit and TR-3B suppression are unchanged. **Two invariants around it:** (a) the hysteresis latch AND the load-failure latch are per-selection state cleared by `_resetTrackedSelectionState()` in the tracking lifecycle (deselect / re-track / cross-layer / init / destroy) — the predicate's icao-change guard is defence only, since it needs a drawn frame while nothing is selected and the render governor's idle mode does not promise one; (b) on-demand loading is bounded at 3 attempts per selection with a 1.5 s backoff and one console warning naming the asset — the driver runs every `scene.preUpdate`, so an unbounded catch means a missing GLB spins load→reject at frame rate. The billboard stays the visual throughout a failed load. - **Grounded 3D handoff is terrain-validity gated (2026-08-23).** A ready civilian or military glTF does not own the visual until `groundSnap` (`src/data/groundSnap.js`) can answer with a MEASURED photoreal-surface height. On success the layer writes `height + the model's measured belly offset` before revealing the model. Cache movement is measured on the WGS84 surface, not across altitude, so a stationary contact keeps its snap through poll-time vertical-datum changes. Model existence or GPU readiness alone never suppresses the billboard floor, and ordinary zoom/style/deselect transitions preserve a valid snap cache. **Two states, and the difference is the whole design.** COLD — nothing has ever resolved for this icao (tiles still streaming on first sight, sample failure, backoff after a first miss): there is no evidence of where the ground is, `heightFor` returns null, the model stays hidden and the depth-test-free 2D billboard remains opaque and floored. WARM — a snap resolved and then a >`MOVE_INVALIDATE_M` (50 m) taxi move stopped it answering directly: the measurement is DEMOTED to a bounded last-known rather than deleted, and it keeps answering while the resample is outstanding, so a taxiing aircraft does not pop 3D→2D→3D across a 2–30 s retry backoff. The bound is `HELD_SNAP_MAX_DRIFT_M` = 250 m from the spot the value was measured at, past which the hold is dropped rather than stretched and the contact is COLD again. It is spatial with no timer beside it (ground under a contact that has not moved does not change; what invalidates the value is the contact MOVING) and deliberately a quarter of the billboard chain's `HELD_FLOOR_MAX_DRIFT_KM` — that hold only ever RAISES a sprite, while a held snap IS the model's placement, so its error shows in both directions. A fresh sample releases the hold, and so does a ground flip (which already calls `forget`). **A loading model is HIDDEN, never zero-scaled:** admission sets `show = false` (Cesium's default is `true`, and an unplaced primitive would claim the visual at the identity matrix), and ownership is `ready && show` — Cesium 1.138's `Model.update` has no `show` guard, so hiding a primitive costs its load nothing. -- Commercial and military aircraft use the same high-level FLEET model regime: 2D billboards when zoomed out, optional glTF models when closer, controlled by the DISPLAY rail `3D` toggle and `Proximity` / `All` modes. Since 2026-08-16 (Hangar fleet) models are PER-CLASS: real CC-BY GLBs for light/bizjet/turboprop/widebody/helicopter/uav (`CLASS_MODEL_REAL` in `src/data/aircraftClass.js` — meters-baked, scale 1, per-model belly/radius; provenance in `public/models/README.md`), the shared `airplane.glb` for the remaining civilian classes, and the military layer maps weight classes (real GLBs / 747 heavies / `jet.glb` fastjets, per-model heading offsets, always flat amber). Textured civilians carry a HEAVY tint, not a light one: `MODEL_COLOR_BLEND_AMOUNT` is `0.94` in BOTH layers under Cesium's `ColorBlendMode.MIX`, so the class colour supplies 94% of the surface and the asset's own texture ~6%. The visual direction is clean light silhouettes with only a weak diffuse contribution from the approved textures, so liveries deliberately do NOT read. IR boost raises the blend to a full `1.0`. Under NVG/FLIR (map preset or Cockpit vision) models render unlit flat-white at full alpha and scene fog is disabled (fog otherwise blacks out distant models with the globe hidden); state restores on exit. +- Commercial and military aircraft use the same high-level FLEET model regime: 2D billboards when zoomed out, optional glTF models when closer, controlled by the DISPLAY rail `3D` toggle and `Proximity` / `All` modes. Since 2026-08-16 (Hangar fleet) models are PER-CLASS: real CC-BY GLBs for light/bizjet/turboprop/widebody/helicopter/uav (`CLASS_MODEL_REAL` in `src/data/aircraftClass.js` — meters-baked, scale 1, per-model belly/radius; provenance in `public/models/README.md`), the shared `airplane.glb` for the remaining civilian classes, and the military layer maps weight classes (real GLBs / 747 heavies / `jet.glb` fastjets, per-model heading offsets, flat amber or emergency red). Textured civilians carry a HEAVY tint, not a light one: `MODEL_COLOR_BLEND_AMOUNT` is `0.94` in BOTH layers under Cesium's `ColorBlendMode.MIX`, so the class colour supplies 94% of the surface and the asset's own texture ~6%. The visual direction is clean light silhouettes with only a weak diffuse contribution from the approved textures, so liveries deliberately do NOT read. IR boost raises the blend to a full `1.0`. Under NVG/FLIR (map preset or Cockpit vision) models render unlit flat-white at full alpha and scene fog is disabled (fog otherwise blacks out distant models with the globe hidden); state restores on exit. - The DISPLAY-rail `3D` toggle is the user-facing activation path for both aircraft layers. Their small approved GLBs build their render resources without Cesium's frame-spread job queue, preventing continuous Photorealistic 3D Tiles streaming from starving model readiness; model caps, tracking, camera, and fallback billboards are unchanged. - **The `3D` toggle DEFAULTS ON in `proximity` on a first run.** Proximity is itself the budget — models appear only below `MODEL_ALT_CEIL_M` and only for the nearest `MODEL_MAX` in view — so the default costs nothing at globe scale, and `all` remains a deliberate opt-in. A fresh boot runs NO layer-state restoration (`LayerStateCoordinator.start()` returns early with neither a share payload nor stored state), so four independent initializers decide what a first-run operator sees and must agree: `booleanOption('models3d', 'e', true)` in `src/data/layerState.js`, `_models3dEnabled = true` in BOTH flight layers, `this._models3dEnabled = true` in `src/ui.js`, and the `active` / `visible` classes on `#models3d-toggle` / `#models3d-mode-row` in `index.html`. All four are pinned together in `src/data/layerState.test.mjs`. Explicit state still wins: because the codec omits default-valued options, `models3d: false` is now what travels in a link (`lo=…f.e.0`) and restores OFF at both aircraft layers. **Consequence for returning users:** a stored `gev:layer-state:v2` blob is a FULL options snapshot, so a session that wrote one before this change carries `models3d:false` and keeps 3D off until the operator flips it (or clears the key) — the durable snapshot is treated as the recipient's own state, by design. - **Consequence of the flip on the recorded tracked/fleet inversion:** the 150–800 km band where surrounding contacts draw as models while the SELECTED one is still a glyph is now what an operator sees WITHOUT arming anything. The inversion itself is unchanged and still deliberate (see `src/data/trackedModelRegime.js`); only its reachability changed. @@ -5209,6 +5283,10 @@ falls back to OSM on construction failure or two active tile failures, and credits follow the source actually displayed. Terrain remains lazy while the photoreal globe is hidden. +Map-stack subscribers hear each settled activation once. A subscriber removed +during a delivery, or every subscriber once the controller is destroyed, gets +no further callback from it, and a destroyed controller accepts no new ones. + Map credentials are passed to each source constructor rather than changing SDK-wide Google or ion defaults. Bing Aerial and Bing Labels resolve their ion imagery endpoint with the supplied diff --git a/index.html b/index.html index 2e630c4a8..78d6e34ec 100644 --- a/index.html +++ b/index.html @@ -13,7 +13,7 @@ with the sources: src/materialSymbolsSubset.test.mjs fails when a source names a glyph this list is missing, because a missing glyph does not draw a box — the ligature never forms and the element renders the literal word. --> - + diff --git a/package.json b/package.json index 4b2829313..d5c900cbd 100644 --- a/package.json +++ b/package.json @@ -165,6 +165,7 @@ "./layers/recent-imagery": "./src/layers/recentImagery/index.js", "./layers/recent-imagery/source": "./src/layers/recentImagery/source.js", "./layers/perimeters": "./src/layers/perimeters/index.js", + "./layers/gnss": "./src/layers/gnss/index.js", "./sources/firms-csv": "./src/data/firmsCsv.js", "./layers/firms": "./src/layers/firms/index.js", "./layers/satellites": "./src/layers/satellites/index.js", @@ -238,6 +239,7 @@ "./layers/cctv/source": "./src/layers/cctv/source.js", "./layers/earthquakes/source": "./src/layers/earthquakes/source.js", "./layers/perimeters/source": "./src/layers/perimeters/source.js", + "./layers/gnss/source": "./src/layers/gnss/source.js", "./layers/firms/source": "./src/layers/firms/source.js", "./layers/installations/source": "./src/layers/installations/source.js", "./layers/launches/source": "./src/layers/launches/source.js", diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 296880561..34e12d6c2 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -18,6 +18,10 @@ "src/tooling/panelStorageDocs.test.mjs", "src/tooling/packageBoundaries.test.mjs", "src/data/infrastructure.js", + "src/data/aircraftEmergency.js", + "src/data/aircraftEmergency.test.mjs", + "src/data/aircraftEmergencyAnalyst.test.mjs", + "scripts/qa-emergency-squawk.mjs", "src/data/infrastructure.test.mjs", "src/data/localGeojsonCore.js", "src/data/localGeojsonLod.js", @@ -26,6 +30,8 @@ "src/annotations/drawMode.test.mjs", "src/annotations/drawTool.js", "src/annotations/drawTool.test.mjs", + "src/data/geoFileImport.test.mjs", + "src/ui/geoFileImport.test.mjs", "src/annotations/adminOutline.test.mjs", "src/annotations/regionRing.test.mjs", "src/data/adminBoundaries.test.mjs", @@ -199,6 +205,9 @@ "src/styles/snow.js", "scripts/qa-visual-effects.mjs", "scripts/qa-draw-tool.mjs", + "scripts/qa-geo-import.mjs", + "scripts/qa-geo-import-regressions.mjs", + "scripts/qa-geo-import-large.mjs", "src/ui/mapSource.js", "src/ui/mapSourceControls.js", "src/ui/mapSourceControls.test.mjs", @@ -827,6 +836,16 @@ "server/providers/firePerimeters.js", "src/data/cyclonesProxy.test.mjs", "src/data/firePerimetersProxy.test.mjs", + "server/providers/gnssIntegrity.js", + "src/data/gnssIntegrityProxy.test.mjs", + "src/app/layers/gnss.js", + "src/layers/gnss/index.js", + "src/layers/gnss/records.js", + "src/layers/gnss/source.js", + "src/layers/gnss/records.test.mjs", + "src/layers/gnss/source.test.mjs", + "src/layers/gnss/ownership.test.mjs", + "scripts/qa-gnss.mjs", "src/layers/cyclones/index.js", "src/layers/cyclones/rendering.js", "src/layers/cyclones/labels.js", @@ -1001,5 +1020,7 @@ "src/tools/globePanel.test.mjs", "src/costEndpointThrottle.test.mjs", "src/sources/gbfsStations.test.mjs", - "scripts/qa-gbfs-names.mjs" + "scripts/qa-gbfs-names.mjs", + "src/ui/flyToDataSource.js", + "src/ui/flyToDataSource.test.mjs" ] diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index ba1f0c36a..6ef5f7fde 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -781,6 +781,7 @@ "live-observations": { "exports": ["./sources/live"], "modules": [ + "src/data/aircraftEmergency.js", "src/sources/live/aircraft.js", "src/sources/live/contract.js", "src/sources/live/index.js", @@ -794,6 +795,7 @@ "modules": [ "src/cyberSonar.js", "src/data/aircraftClass.js", + "src/data/aircraftEmergency.js", "src/data/aircraftIcons.js", "src/data/aircraftMeta.js", "src/data/aircraftNearbyPolicy.js", @@ -837,6 +839,7 @@ "modules": [ "src/cyberSonar.js", "src/data/aircraftClass.js", + "src/data/aircraftEmergency.js", "src/data/aircraftIcons.js", "src/data/aircraftMeta.js", "src/data/aircraftNearbyPolicy.js", @@ -1502,6 +1505,7 @@ "src/contextModePolicy.js", "src/creditKeyboard.js", "src/data/aircraftClass.js", + "src/data/aircraftEmergency.js", "src/data/aircraftIcons.js", "src/data/aircraftMeta.js", "src/data/aircraftNearbyPolicy.js", @@ -2014,6 +2018,9 @@ "src/data/inputOwnership.js", "src/annotations/drawMode.js", "src/annotations/drawTool.js", + "src/data/geoFileImport.js", + "src/ui/flyToDataSource.js", + "src/ui/geoFileImport.js", "src/data/routeSteps.js", "src/layers/directions/index.js", "src/layers/directions/source.js", @@ -2037,6 +2044,10 @@ "src/layers/perimeters/records.js", "src/layers/perimeters/cards.js", "src/layers/perimeters/inciweb.js", + "src/app/layers/gnss.js", + "src/layers/gnss/index.js", + "src/layers/gnss/source.js", + "src/layers/gnss/records.js", "src/layers/cctv/sourcePolicy.js", "src/sources/reference.js", "src/search/nominatim.js", @@ -2225,7 +2236,10 @@ "src/tools/functions.js", "src/voice/actionSchemas.js", "src/view/index.js", - "src/app/embed.js" + "src/app/embed.js", + "src/data/cameraFraming.js", + "src/data/geoImportPins.js", + "src/data/geoImportImages.js" ], "external": [ "hls.js", @@ -2292,6 +2306,7 @@ "src/celestialRing.js", "src/cockpitMath.js", "src/data/aircraftClass.js", + "src/data/aircraftEmergency.js", "src/data/aircraftIcons.js", "src/data/aircraftMeta.js", "src/data/aircraftNearbyPolicy.js", @@ -2629,6 +2644,10 @@ "src/layers/perimeters/records.js", "src/layers/perimeters/cards.js", "src/layers/perimeters/inciweb.js", + "src/app/layers/gnss.js", + "src/layers/gnss/index.js", + "src/layers/gnss/source.js", + "src/layers/gnss/records.js", "src/layers/cctv/sourcePolicy.js", "src/search/nominatim.js", "src/nominatimGeocode.js", @@ -2760,7 +2779,8 @@ "src/data/local_data/osm_military_names/names.json", "src/layers/installations/namedMarkers.js", "src/data/infrastructureOverlayEntry.js", - "src/sources/infrastructureData.js" + "src/sources/infrastructureData.js", + "src/data/cameraFraming.js" ], "external": [ "hls.js", @@ -2825,7 +2845,8 @@ "src/sources/overpassFeatures.js", "src/sources/featureSource.js", "src/sources/capability.js", - "src/data/cameraArrival.js" + "src/data/cameraArrival.js", + "src/data/cameraFraming.js" ], "external": [ "cesium", @@ -2921,7 +2942,8 @@ "src/layers/flights/recordPolicy.js", "src/data/aircraftMeta.js", "src/data/renderAltitude.js", - "src/data/aircraftClass.js" + "src/data/aircraftClass.js", + "src/data/aircraftEmergency.js" ], "external": [] }, @@ -2940,7 +2962,8 @@ "src/layers/military/recordPolicy.js", "src/data/aircraftMeta.js", "src/data/renderAltitude.js", - "src/data/aircraftClass.js" + "src/data/aircraftClass.js", + "src/data/aircraftEmergency.js" ], "external": [] }, @@ -3137,7 +3160,8 @@ "src/sources/httpBody.js", "src/layers/perimeters/source.js", "src/layers/perimeters/records.js", - "src/layers/submarineCables/bundledSource.js" + "src/layers/submarineCables/bundledSource.js", + "src/layers/gnss/source.js" ], "external": [] }, @@ -3348,6 +3372,21 @@ ], "external": ["cesium"] }, + "gnss": { + "exports": ["./layers/gnss"], + "modules": [ + "src/layers/gnss/index.js", + "src/sources/httpBody.js", + "src/layers/gnss/source.js", + "src/layers/gnss/records.js" + ], + "external": ["cesium"] + }, + "gnss-source": { + "exports": ["./layers/gnss/source"], + "modules": ["src/sources/httpBody.js", "src/layers/gnss/source.js"], + "external": [] + }, "perimeters-source": { "exports": ["./layers/perimeters/source"], "modules": [ @@ -3475,6 +3514,7 @@ "src/sources/featureGeometry.js", "src/sources/live/standalone.js", "src/sources/live/contract.js", + "src/data/aircraftEmergency.js", "src/sources/live/aircraft.js", "src/sources/live/vessels.js", "src/tools/places.js", diff --git a/scripts/qa-emergency-squawk.mjs b/scripts/qa-emergency-squawk.mjs new file mode 100644 index 000000000..f877ea659 --- /dev/null +++ b/scripts/qa-emergency-squawk.mjs @@ -0,0 +1,554 @@ +#!/usr/bin/env node +/** + * qa-emergency-squawk.mjs — headless proof for transponder emergency highlighting. + * + * Drives the REAL app in headless Chromium against a dev server (keyless). + * `/api/flights` and `/api/military` are stubbed in the page so the codes + * are deterministic; everything downstream (normalizers, records, rendering, + * Cockpit HUD) is production code. + * + * (i) TINT — a civil 7600 and a military ADS-B `general` emergency are + * drawn red; a normal civil flight stays white, a normal military + * flight stays amber, and a military `minfuel` priority stays amber. + * POLL/MODELS verify existing untracked billboards and 3D models + * repaint when emergency status starts and clears. + * (ii) HUD — tracking the military emergency and entering Cockpit names + * the broadcast code on the aircraft-meta line, in red. + * (iii) CLEAR — the next poll drops the code: the HUD line returns to its + * normal wording (the status is not sticky). + * (iv) RELEASE — the code returns and tracking stops: the restored fleet + * billboard is red, not the plain military amber. + * + * Screenshots saved to qa-shots/emergency-*.png (gitignored). + * + * Run: node scripts/qa-emergency-squawk.mjs --url http://localhost:4173 + * Optional: --timeout-ms 180000 for a slow boot. + * Exits non-zero on any FAIL. Does not commit anything. + */ + +import puppeteer from 'puppeteer'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const REPO_ROOT = path.resolve(__dirname, '..'); +const SHOTS_DIR = path.join(REPO_ROOT, 'qa-shots'); + +const argv = process.argv.slice(2); +const getOpt = (name, dflt) => { + const i = argv.indexOf(name); + return i >= 0 && argv[i + 1] ? argv[i + 1] : dflt; +}; +const APP_URL = getOpt('--url', 'http://localhost:4173'); +const HEADFUL = argv.includes('--headful'); +const TIMEOUT_MS = Number(getOpt('--timeout-ms', '60000')); +const CENTER = { lon: 4.0, lat: 53.5 }; + +const results = []; +function record(name, ok, detail) { + results.push({ name, ok, detail }); + const tag = ok ? '\x1b[32mPASS\x1b[0m' : '\x1b[31mFAIL\x1b[0m'; + console.log(` [${tag}] ${name}${detail ? ` — ${detail}` : ''}`); +} +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** OpenSky state vector: squawk sits at index 14. */ +const civilRow = (icao24, callsign, lon, lat, squawk) => [ + icao24, + callsign, + 'Netherlands', + null, + null, + lon, + lat, + 10000, + false, + 230, + 90, + 0, + null, + 10050, + squawk, + false, + 0, +]; + +/** readsb row as adsb.lol /v2/mil returns it. */ +const milRow = (hex, flight, lon, lat, squawk, emergency) => ({ + hex, + flight, + lon, + lat, + alt_baro: 24000, + alt_geom: 24200, + gs: 380, + track: 270, + t: 'C17', + r: `QA-${hex}`, + squawk, + emergency, + seen: 1, + seen_pos: 1, +}); + +/** Page-side route stubs; `__squawkQaPhase` switches the military emergency off. */ +function installStubs() { + const realFetch = window.fetch.bind(window); + window.__squawkQaPhase = 'emergency'; + const json = (body) => + Promise.resolve( + new Response(JSON.stringify(body), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }), + ); + window.fetch = (input, init) => { + const url = typeof input === 'string' ? input : input?.url || ''; + const now = Date.now(); + const { civil, mil } = window.__squawkQaFixtures; + if ( + url.includes('/api/flights/track') || + url.includes('/api/military/track') + ) + return json({ path: [] }); + if (url.includes('/api/flights')) { + const t = Math.floor(now / 1000); + return json({ + time: t, + states: civil.map((row) => { + const copy = [...row]; + copy[3] = t - 1; + copy[4] = t - 1; + return copy; + }), + }); + } + if (url.includes('/api/military')) { + const cleared = window.__squawkQaPhase === 'cleared'; + return json({ + msg: 'No error', + now, + ac: mil.map((row) => + cleared && row.hex === 'ae7700' + ? { ...row, squawk: '4521', emergency: 'none' } + : row, + ), + }); + } + return realFetch(input, init); + }; +} + +/** Colour of every shown fleet billboard whose id is one of `ids`. */ +function billboardColors(ids) { + const { viewer } = window.__godsEyeView; + const found = {}; + const seen = new Set(); + const visit = (primitive) => { + if (!primitive || seen.has(primitive)) return; + seen.add(primitive); + if ( + typeof primitive.length !== 'number' || + typeof primitive.get !== 'function' + ) + return; + for (let i = 0; i < primitive.length; i++) { + const item = primitive.get(i); + if (item?.color && ids.includes(item.id) && item.show !== false) { + found[item.id] = { + r: Math.round(item.color.red * 255), + g: Math.round(item.color.green * 255), + b: Math.round(item.color.blue * 255), + }; + } else visit(item); + } + }; + visit(viewer.scene.primitives); + return found; +} + +const isRed = (c) => Boolean(c) && c.r > 200 && c.g < 110 && c.b < 110; +const isWhite = (c) => Boolean(c) && c.r > 220 && c.g > 220 && c.b > 220; +const isAmber = (c) => + Boolean(c) && c.r > 180 && c.g > 110 && c.b < 140 && !isRed(c); + +async function shoot(page, name) { + await page + .waitForFunction( + () => window.__godsEyeView.viewer.scene.globe.tilesLoaded, + { timeout: 30000 }, + ) + .catch(() => {}); + await sleep(600); + await page.screenshot({ + path: path.join(SHOTS_DIR, `emergency-${name}.png`), + }); +} + +async function main() { + console.log('\nEmergency squawk proof (qa-emergency-squawk)'); + console.log(` App URL : ${APP_URL}\n`); + try { + const res = await fetch(APP_URL); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + } catch (e) { + console.error( + `\x1b[31mDev server not reachable at ${APP_URL} (${e.message}).\x1b[0m`, + ); + process.exit(2); + } + fs.mkdirSync(SHOTS_DIR, { recursive: true }); + const executablePath = + process.env.PUPPETEER_EXECUTABLE_PATH || + (await puppeteer.executablePath().catch(() => null)) || + undefined; + const browser = await puppeteer.launch({ + headless: HEADFUL ? false : 'new', + protocolTimeout: 180000, + ...(executablePath && fs.existsSync(executablePath) + ? { executablePath } + : {}), + args: [ + '--use-gl=angle', + '--use-angle=swiftshader', + '--disable-dev-shm-usage', + '--disable-background-timer-throttling', + '--disable-renderer-backgrounding', + '--window-size=1440,900', + ], + }); + + let exitCode = 0; + const errors = []; + try { + const page = await browser.newPage(); + await page.setViewport({ width: 1440, height: 900 }); + page.on('pageerror', (e) => errors.push(e.message)); + const { lon, lat } = CENTER; + await page.evaluateOnNewDocument( + (fixtures) => { + window.__squawkQaFixtures = fixtures; + }, + { + civil: [ + civilRow('4847a1', 'KLM7600 ', lon - 0.6, lat + 0.25, '7600'), + civilRow('4847a2', 'KLM1234 ', lon - 0.6, lat - 0.25, '2000'), + ], + mil: [ + milRow('ae7700', 'RCH7700', lon + 0.6, lat + 0.25, '1200', 'general'), + milRow('ae0f01', 'RCH0F01', lon + 0.6, lat - 0.25, '1234', 'minfuel'), + milRow('ae0001', 'RCH0001', lon, lat - 0.5, '4520', 'none'), + ], + }, + ); + // Stub in-page rather than with request interception: an intercepted page + // stalls Cesium's workers (see qa-gnss.mjs). + await page.evaluateOnNewDocument(installStubs); + await page.goto(`${APP_URL}/?welcome=0`, { + waitUntil: 'domcontentloaded', + timeout: TIMEOUT_MS, + }); + await page.waitForFunction( + () => window.__godsEyeView?.viewer && window.__godsEyeView?.dataManager, + { timeout: TIMEOUT_MS }, + ); + await sleep(5000); + await page.keyboard.press('Escape'); + await page.evaluate( + (lo, la) => { + const gev = window.__godsEyeView; + const d2r = Math.PI / 180; + try { + gev.viewer.camera.cancelFlight(); + } catch { + /* no flight active */ + } + gev.viewer.camera.setView({ + destination: gev.viewer.scene.globe.ellipsoid.cartographicToCartesian( + { + longitude: lo * d2r, + latitude: la * d2r, + height: 260000, + }, + ), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + gev.viewer.scene.requestRender?.(); + }, + lon, + lat, + ); + + // ── (i) TINT ────────────────────────────────────────────────────────── + console.log('(i) TINT — emergency codes through the real layers...'); + await page.evaluate(async () => { + const dm = window.__godsEyeView.dataManager; + await dm.setEnabled('flights', true, { origin: 'user' }); + await dm.setEnabled('military', true, { origin: 'user' }); + }); + const ids = ['4847a1', '4847a2', 'ae7700', 'ae0f01', 'ae0001']; + const probe = `(${billboardColors})(${JSON.stringify(ids)})`; + await page + .waitForFunction(`Object.keys(${probe}).length === ${ids.length}`, { + timeout: TIMEOUT_MS, + polling: 500, + }) + .catch(() => {}); + await sleep(1500); + const colors = await page.evaluate(probe); + const show = (id) => JSON.stringify(colors[id] ?? null); + record( + 'TINT: civil squawk 7600 is red', + isRed(colors['4847a1']), + show('4847a1'), + ); + record( + 'TINT: normal civil flight stays white', + isWhite(colors['4847a2']), + show('4847a2'), + ); + record( + 'TINT: military ADS-B general emergency is red', + isRed(colors.ae7700), + show('ae7700'), + ); + record( + 'TINT: military minimum-fuel priority stays amber', + isAmber(colors.ae0f01), + show('ae0f01'), + ); + record( + 'TINT: normal military flight stays amber', + isAmber(colors.ae0001), + show('ae0001'), + ); + await shoot(page, 'globe-tint'); + + // Existing untracked billboards and models must repaint in both directions. + await page.evaluate(async () => { + window.__squawkQaPhase = 'cleared'; + await window.__godsEyeView.dataManager.refreshLayer('military'); + }); + record( + 'POLL: existing military billboard clears to amber', + isAmber((await page.evaluate(probe)).ae7700), + ); + await page.evaluate(async () => { + window.__squawkQaPhase = 'emergency'; + await window.__godsEyeView.dataManager.refreshLayer('military'); + }); + record( + 'POLL: existing military billboard turns red again', + isRed((await page.evaluate(probe)).ae7700), + ); + const modelProbe = () => { + const found = {}; + const visit = (items) => { + for (let i = 0; i < items.length; i++) { + const item = items.get(i); + if (item.ready && item.color && typeof item.id === 'string') + found[item.id] = { + r: Math.round(item.color.red * 255), + g: Math.round(item.color.green * 255), + b: Math.round(item.color.blue * 255), + }; + else if (item.get) visit(item); + } + }; + visit(window.__godsEyeView.viewer.scene.primitives); + return found; + }; + await page.waitForFunction( + `(${modelProbe})()['4847a1'] && (${modelProbe})().ae7700`, + { timeout: TIMEOUT_MS, polling: 250 }, + ); + const beforeModels = await page.evaluate(modelProbe); + record( + 'MODELS: civil and military emergency models are red', + isRed(beforeModels['4847a1']) && isRed(beforeModels.ae7700), + JSON.stringify(beforeModels), + ); + await page.evaluate(async () => { + window.__squawkQaFixtures.civil[0][14] = '2000'; + window.__squawkQaFixtures.civil[1][14] = '7600'; + window.__squawkQaPhase = 'cleared'; + const dm = window.__godsEyeView.dataManager; + await dm.refreshLayer('flights'); + await dm.refreshLayer('military'); + }); + await page.waitForFunction( + `(${modelProbe})()['4847a1']?.g > 220 && (${modelProbe})()['4847a2']?.g < 110 && (${modelProbe})().ae7700?.g > 110`, + { timeout: TIMEOUT_MS, polling: 250 }, + ); + const afterModels = await page.evaluate(modelProbe); + record( + 'MODELS: existing civil models start and clear, military clears', + isWhite(afterModels['4847a1']) && + isRed(afterModels['4847a2']) && + isAmber(afterModels.ae7700), + JSON.stringify(afterModels), + ); + await page.evaluate(async () => { + window.__squawkQaFixtures.civil[0][14] = '7600'; + window.__squawkQaFixtures.civil[1][14] = '2000'; + window.__squawkQaFixtures.mil[0].squawk = '7700'; + window.__squawkQaPhase = 'emergency'; + const dm = window.__godsEyeView.dataManager; + await dm.refreshLayer('flights'); + await dm.refreshLayer('military'); + }); + // ── (ii) HUD ────────────────────────────────────────────────────────── + console.log('\n(ii) HUD — track the emergency and enter Cockpit...'); + const tracked = await page.evaluate(() => + Boolean( + window.__godsEyeView.dataManager.layers + .get('military') + .module.trackById('ae7700', { origin: 'user' }), + ), + ); + record( + 'HUD: emergency aircraft can be tracked', + tracked, + `tracked=${tracked}`, + ); + // Cockpit entry is gated on the Contacts (flights) context mode. + await page.evaluate(() => + window.__godsEyeView.styleManager._contextControls._selectContextMode( + 'flights', + ), + ); + await page + .waitForFunction( + () => { + const entry = document.getElementById('cockpit-entry'); + return entry && !entry.hidden && !entry.disabled; + }, + { timeout: 20000 }, + ) + .catch(() => {}); + await page + .$eval('#cockpit-entry', (entry) => entry.click()) + .catch(() => {}); + const hudReady = await page + .waitForFunction( + () => + document.body.classList.contains('cockpit-mode') && + /SQUAWK 7700/.test( + document.getElementById('cockpit-aircraft-meta')?.textContent || '', + ), + { timeout: 30000 }, + ) + .then( + () => true, + () => false, + ); + const hud = await page.evaluate(() => { + const meta = document.getElementById('cockpit-aircraft-meta'); + return { + cockpit: document.body.classList.contains('cockpit-mode'), + text: meta?.textContent || '', + emergencyClass: meta?.classList.contains('emergency') || false, + color: meta ? getComputedStyle(meta).color : null, + }; + }); + record( + 'HUD: meta line names the broadcast code', + hudReady && hud.text.includes('SQUAWK 7700 · GENERAL EMERGENCY'), + JSON.stringify(hud), + ); + record( + 'HUD: meta line is styled as an emergency', + hud.emergencyClass && /255,\s*68,\s*68/.test(hud.color || ''), + `class=${hud.emergencyClass} color=${hud.color}`, + ); + const trackedBroadcast = await page.evaluate( + () => + window.__godsEyeView.dataManager.layers + .get('military') + .module.getTrackedInfo().emergencyBroadcast, + ); + record( + 'TRACKED: voice-facing status quotes the broadcast', + trackedBroadcast === 'broadcasting squawk 7700 (general emergency)', + trackedBroadcast, + ); + await shoot(page, 'cockpit-hud'); + + // ── (iii) CLEAR ─────────────────────────────────────────────────────── + console.log('\n(iii) CLEAR — the next poll drops the code...'); + await page.evaluate(async () => { + window.__squawkQaPhase = 'cleared'; + await window.__godsEyeView.dataManager.refreshLayer('military'); + }); + const cleared = await page + .waitForFunction( + () => + !document + .getElementById('cockpit-aircraft-meta') + ?.classList.contains('emergency'), + { timeout: 20000 }, + ) + .then( + () => true, + () => false, + ); + const after = await page.evaluate( + () => document.getElementById('cockpit-aircraft-meta')?.textContent || '', + ); + record( + 'CLEAR: HUD returns to its normal wording', + cleared && !after.includes('SQUAWK') && after.includes('COURSE ALIGNED'), + JSON.stringify(after), + ); + await shoot(page, 'cockpit-cleared'); + + // ── (iv) RELEASE ────────────────────────────────────────────────────── + console.log('\n(iv) RELEASE — the code returns, then tracking stops...'); + await page.evaluate(async () => { + window.__squawkQaPhase = 'emergency'; + const dm = window.__godsEyeView.dataManager; + await dm.refreshLayer('military'); + dm.layers.get('military').module.stopTracking({ origin: 'user' }); + }); + const releaseProbe = `(${billboardColors})(["ae7700"])`; + const released = await page + .waitForFunction(`Boolean((${releaseProbe}).ae7700)`, { + timeout: 20000, + polling: 250, + }) + .then( + () => true, + () => false, + ); + const releasedColor = (await page.evaluate(releaseProbe)).ae7700; + record( + 'RELEASE: untracked emergency billboard is red', + released && isRed(releasedColor), + JSON.stringify(releasedColor ?? null), + ); + record( + 'no uncaught browser errors', + errors.length === 0, + errors.slice(0, 3).join(' | '), + ); + } catch (err) { + console.error('\x1b[31mHarness error:\x1b[0m', err); + exitCode = 3; + } finally { + await browser.close(); + } + + const pass = results.filter((r) => r.ok).length; + const fail = results.length - pass; + console.log('\n' + '─'.repeat(60)); + console.log(` RESULT: ${pass} passed, ${fail} failed`); + console.log(` Shots : ${SHOTS_DIR}/emergency-*.png`); + console.log('─'.repeat(60) + '\n'); + process.exit(exitCode || (fail > 0 ? 1 : 0)); +} + +main().catch((e) => { + console.error(e); + process.exit(3); +}); diff --git a/scripts/qa-geo-import-large.mjs b/scripts/qa-geo-import-large.mjs new file mode 100644 index 000000000..679ea14a1 --- /dev/null +++ b/scripts/qa-geo-import-large.mjs @@ -0,0 +1,446 @@ +#!/usr/bin/env node +/** Large KML acceptance: style equivalence, 27.6 MB import and optional real file. + * QA_BASE_URL defaults to http://localhost:4173. Optional --file /path/file.kml. + * For a USGS fixture with blocked icons, add --expect-local-pins to check paint. + */ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import puppeteer from 'puppeteer'; +import { hookRenderErrors, readRenderErrors } from './qa-browserEvidence.mjs'; + +const base = process.env.QA_BASE_URL || 'http://localhost:4173'; +const supplied = process.argv.includes('--file') + ? path.resolve(process.argv[process.argv.indexOf('--file') + 1]) + : null; +const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'gev-large-kml-')); +const file = path.join(directory, 'metadata-rich-27mb.kml'); +const style = + ''; +const notes = 'A local observation with supporting detail. '.repeat(120); +const placemarks = Array.from({ length: 5000 }, (_, i) => { + const lon = -122.6 + (i % 100) * 0.002; + const lat = 37.65 + Math.floor(i / 100) * 0.003; + return `Point ${i}#base${style}${notes}${lon},${lat},0`; +}); +const wrap = (body) => + `${body}`; +const large = wrap(placemarks.join('')); +fs.writeFileSync(file, large); +const bytes = fs.statSync(file).size; +if (bytes < 20 * 1024 * 1024 || bytes > 30 * 1024 * 1024) + throw new Error(`Fixture must be 20–30 MiB, got ${bytes} bytes`); +const small = wrap( + Array.from({ length: 6 }, (_, i) => { + const kind = i % 3; + const geometry = + kind === 0 + ? '-122.49,37.77,0' + : kind === 1 + ? '-122.50,37.77,0 -122.49,37.77,0' + : '-122.50,37.77,0 -122.49,37.77,0 -122.49,37.78,0 -122.50,37.77,0'; + const override = + kind === 0 + ? style + : kind === 1 + ? '' + : ''; + return `Small ${i}#base${override}${geometry}`; + }).join('') + + '' + + '' + + 'normal' + + ['named-a', 'named-b', 'paired'] + .map( + (id) => + `${id}#${id}-122.49,37.77,0`, + ) + .join(''), +); +const browser = await puppeteer.launch({ + headless: true, + args: + process.platform === 'darwin' + ? ['--use-gl=angle', '--use-angle=metal'] + : ['--no-sandbox', '--use-gl=angle', '--use-angle=swiftshader'], +}); +const page = await browser.newPage(); +await hookRenderErrors(page); +const errors = []; +const fileRequests = []; +page.on('pageerror', (e) => errors.push(e.message)); +await page.setRequestInterception(true); +page.on('request', (request) => { + if (new URL(request.url()).hostname === 'earthquake.usgs.gov') { + fileRequests.push(request.url()); + void request.abort(); + } else void request.continue(); +}); +let failures = 0; +const check = (name, ok, detail) => { + console.log(`[${ok ? 'PASS' : 'FAIL'}] ${name}`, detail ?? ''); + if (!ok) failures++; +}; +try { + await page.setViewport({ width: 1440, height: 900 }); + await page.goto(`${base}/?welcome=0`, { waitUntil: 'domcontentloaded' }); + await page.waitForFunction( + () => + window.__gevGeoImport && + document.getElementById('loading-screen')?.classList.contains('hidden'), + { timeout: 60000 }, + ); + await page.evaluate(async () => { + const { viewer, dataManager } = window.__godsEyeView; + const C = window.__CESIUM__; + for (const id of dataManager.layers.keys()) + await dataManager.setEnabled(id, false); + viewer.clock.shouldAnimate = false; + viewer.imageryLayers.removeAll(); + viewer.terrainProvider = new C.EllipsoidTerrainProvider(); + viewer.camera.cancelFlight(); + viewer.camera.setView({ + destination: C.Cartesian3.fromDegrees(-122.5, 37.72, 50000), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + window.__qaStyleSnapshot = (source) => { + const now = viewer.clock.currentTime; + const images = new Map(); + const imageValue = (p) => { + const value = p?.getValue(now); + if (value?.tagName !== 'CANVAS') return value; + if (!images.has(value)) images.set(value, value.toDataURL()); + return images.get(value); + }; + const color = (p) => p?.getValue(now)?.toCssColorString(); + const positions = (p) => p?.getValue(now)?.map((v) => [v.x, v.y, v.z]); + const renderedPins = new Map(); + const collection = source.importedBillboards; + if (collection) + for (let i = 0; i < collection.length; i++) { + const pin = collection.get(i); + renderedPins.set(pin.id, pin); + } + const actual = (e, key, fallback) => { + const pin = renderedPins.get(e); + return pin ? pin[key] : (e.billboard?.[key]?.getValue(now) ?? fallback); + }; + return (source.importedEntities || source.entities.values) + .map((e) => ({ + name: e.name, + point: renderedPins.get(e)?.position || e.position?.getValue(now), + scale: e.billboard ? actual(e, 'scale', 1) : undefined, + rotation: e.billboard ? actual(e, 'rotation', 0) : undefined, + image: renderedPins.get(e)?.image || imageValue(e.billboard?.image), + color: e.billboard + ? actual(e, 'color', C.Color.WHITE).toCssColorString() + : undefined, + width: e.polyline?.width?.getValue(now), + lineColor: color(e.polyline?.material?.color), + line: positions(e.polyline?.positions), + fill: e.polygon?.fill?.getValue(now), + outline: e.polygon?.outline?.getValue(now), + polygonColor: color(e.polygon?.material?.color), + polygon: e.polygon?.hierarchy + ?.getValue(now) + ?.positions?.map((v) => [v.x, v.y, v.z]), + })) + .sort((a, b) => a.name.localeCompare(b.name)); + }; + window.__qaUnsharedKml = async (text) => { + const doc = new DOMParser().parseFromString(text, 'application/xml'); + for (const el of [...doc.getElementsByTagName('*')]) + if ( + ['description', 'ExtendedData', 'BalloonStyle'].includes(el.localName) + ) + el.remove(); + const source = await C.KmlDataSource.load(doc, { + camera: viewer.scene.camera, + canvas: viewer.scene.canvas, + clampToGround: true, + }); + for (const e of source.importedEntities || source.entities.values) + e.label = undefined; + return source; + }; + }); + const equivalent = await page.evaluate(async (text) => { + const { viewer } = window.__godsEyeView; + const control = await window.__qaUnsharedKml(text); + const outcomes = await window.__gevGeoImport.importFiles([ + new File([text], 'style-equivalence.kml'), + ]); + const candidate = viewer.dataSources.get(viewer.dataSources.length - 1); + const result = { + ok: outcomes[0]?.ok, + same: + JSON.stringify(window.__qaStyleSnapshot(control)) === + JSON.stringify(window.__qaStyleSnapshot(candidate)), + entities: (candidate.importedEntities || candidate.entities.values) + .length, + }; + await window.__gevGeoImport.remove(outcomes[0]?.id); + return result; + }, small); + check( + 'shared inline and named styles preserve point, line and polygon overrides/inheritance', + equivalent.ok && equivalent.same && equivalent.entities === 9, + JSON.stringify(equivalent), + ); + + // The control bypasses the former style-count refusal solely in this test, + // to compare native unshared rendering against normalized rendering. + await page.evaluate(async (text) => { + const source = await window.__qaUnsharedKml(text); + window.__qaUnsharedSnapshot = JSON.stringify( + window.__qaStyleSnapshot(source), + ); + window.__qaControl = source; + await window.__godsEyeView.viewer.dataSources.add(source); + }, large); + const measure = async () => + page.evaluate(async () => { + const C = window.__CESIUM__, + { viewer } = window.__godsEyeView; + const scene = viewer.scene; + viewer.camera.cancelFlight(); + viewer.camera.setView({ + destination: C.Cartesian3.fromDegrees(-122.5, 37.72, 50000), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + const started = performance.now(), + samples = []; + let last, + frames = 0; + const remove = scene.postRender.addEventListener(() => { + const t = performance.now(); + if (t - started < 2000) return; + frames++; + if (last !== undefined) samples.push(t - last); + last = t; + }); + await new Promise((resolve) => { + let previous = performance.now(); + function tick(t) { + viewer.camera.rotate( + C.Cartesian3.UNIT_Z, + Math.min(t - previous, 100) * 0.0000002, + ); + previous = t; + scene.requestRender(); + if (t - started < 7000) requestAnimationFrame(tick); + else resolve(); + } + requestAnimationFrame(tick); + }); + remove(); + samples.sort((a, b) => a - b); + const gl = scene.context._gl, + debug = gl.getExtension('WEBGL_debug_renderer_info'); + return { + fps: frames / 5, + p50Ms: samples[Math.floor(samples.length * 0.5)], + p95Ms: samples[Math.floor(samples.length * 0.95)], + renderer: debug ? gl.getParameter(debug.UNMASKED_RENDERER_WEBGL) : null, + }; + }); + console.log('UNSHARED CONTROL', JSON.stringify(await measure())); + await page.evaluate(() => + window.__godsEyeView.viewer.dataSources.remove(window.__qaControl, true), + ); + + const loadPicked = async (filename, expected) => { + const before = await page.evaluate( + () => window.__godsEyeView.viewer.dataSources.length, + ); + const priorIds = await page.evaluate(() => + window.__gevGeoImport.list().map((f) => f.id), + ); + const started = performance.now(); + await (await page.$('#geo-import-input')).uploadFile(filename); + const name = path.basename(filename); + const loaded = await page + .waitForFunction( + ({ name, priorIds }) => + window.__gevGeoImport.list().some((f) => !priorIds.includes(f.id)) || + document + .getElementById('geo-import-hint') + .textContent.startsWith(`${name}:`), + { timeout: 120000 }, + { name, priorIds }, + ) + .then( + () => true, + () => false, + ); + const state = await page.evaluate(() => ({ + files: window.__gevGeoImport.list(), + hint: document.getElementById('geo-import-hint').textContent, + sources: window.__godsEyeView.viewer.dataSources.length, + })); + const item = state.files.find((f) => !priorIds.includes(f.id)); + check( + `picked ${name} loads`, + loaded && + (expected === null + ? item?.entityCount > 0 + : item?.entityCount === expected) && + state.sources === before + 1, + JSON.stringify({ + bytes: fs.statSync(filename).size, + loadMs: Math.round(performance.now() - started), + ...state, + }), + ); + return item; + }; + const item = await loadPicked(file, 5000); + if (item) { + const same = await page.evaluate(() => { + const viewer = window.__godsEyeView.viewer; + const source = viewer.dataSources.get(viewer.dataSources.length - 1); + return ( + JSON.stringify(window.__qaStyleSnapshot(source)) === + window.__qaUnsharedSnapshot + ); + }); + check('all 5,000 normalized graphics match their unshared control', same); + console.log('NORMALIZED CANDIDATE', JSON.stringify(await measure())); + await page.evaluate((id) => window.__gevGeoImport.remove(id), item.id); + } + if (supplied) { + const clockBefore = await page.evaluate(() => { + const v = window.__godsEyeView.viewer, + C = window.__CESIUM__; + return [ + C.JulianDate.toIso8601(v.clock.startTime), + C.JulianDate.toIso8601(v.clock.stopTime), + C.JulianDate.toIso8601(v.clock.currentTime), + v.clock.multiplier, + ]; + }); + const suppliedItem = await loadPicked(supplied, null); + const clockAfter = await page.evaluate(() => { + const v = window.__godsEyeView.viewer, + C = window.__CESIUM__; + return [ + C.JulianDate.toIso8601(v.clock.startTime), + C.JulianDate.toIso8601(v.clock.stopTime), + C.JulianDate.toIso8601(v.clock.currentTime), + v.clock.multiplier, + ]; + }); + check( + 'real dated KML leaves the shared clock unchanged', + JSON.stringify(clockBefore) === JSON.stringify(clockAfter), + ); + if (suppliedItem && process.argv.includes('--expect-local-pins')) { + const markers = await page.evaluate(() => { + const { viewer } = window.__godsEyeView, + C = window.__CESIUM__; + const source = viewer.dataSources.get(viewer.dataSources.length - 1); + const points = ( + source.importedEntities || source.entities.values + ).filter((e) => e.position && e.billboard); + const images = points.filter((e) => + e.billboard.image?.getValue(viewer.clock.currentTime), + ); + if (images.length) { + const at = C.Cartographic.fromCartesian( + images[0].position.getValue(viewer.clock.currentTime), + ); + viewer.camera.cancelFlight(); + viewer.camera.setView({ + destination: C.Cartesian3.fromRadians( + at.longitude, + at.latitude, + 100000, + ), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + } + return { points: points.length, images: images.length }; + }); + check( + 'supplied point markers have drawable images', + markers.points > 0 && markers.images === markers.points, + JSON.stringify(markers), + ); + if (markers.points) { + const yellowPixels = async () => { + const shot = await page.screenshot({ + clip: { x: 620, y: 350, width: 200, height: 200 }, + type: 'png', + encoding: 'base64', + }); + return page.evaluate(async (base64) => { + const bitmap = await createImageBitmap( + await (await fetch(`data:image/png;base64,${base64}`)).blob(), + ); + const canvas = new OffscreenCanvas(bitmap.width, bitmap.height), + context = canvas.getContext('2d'); + context.drawImage(bitmap, 0, 0); + const data = context.getImageData( + 0, + 0, + bitmap.width, + bitmap.height, + ).data; + let count = 0; + for (let i = 0; i < data.length; i += 4) + if (data[i] > 150 && data[i + 1] > 150 && data[i + 2] < 100) + count++; + return count; + }, shot); + }; + const setVisible = (visible) => + page.evaluate( + ({ id, visible }) => window.__gevGeoImport.setVisible(id, visible), + { id: suppliedItem.id, visible }, + ); + await setVisible(false); + await new Promise((resolve) => setTimeout(resolve, 1000)); + const hidden = await yellowPixels(); + await setVisible(true); + let shown = hidden; + const deadline = performance.now() + 10000; + while (performance.now() < deadline && shown <= hidden + 10) { + await new Promise((resolve) => setTimeout(resolve, 500)); + shown = await yellowPixels(); + } + check( + 'real USGS markers paint when shown and disappear when hidden', + shown > hidden + 10, + JSON.stringify({ hidden, shown }), + ); + await page.screenshot({ + path: path.join(directory, 'real-kml-visible.png'), + }); + } + } + if (suppliedItem) + await page.evaluate( + (id) => window.__gevGeoImport.remove(id), + suppliedItem.id, + ); + } + check( + 'file-directed USGS icon/metadata requests never leave the page', + fileRequests.length === 0, + fileRequests.length, + ); + errors.push(...(await readRenderErrors(page))); + check( + 'no uncaught page or render errors', + errors.length === 0, + JSON.stringify(errors), + ); + check( + 'all imported sources are removed', + await page.evaluate(() => window.__gevGeoImport.list().length === 0), + ); + console.log('Fixture retained at', file); +} finally { + await browser.close(); +} +process.exitCode = failures ? 1 : 0; diff --git a/scripts/qa-geo-import-regressions.mjs b/scripts/qa-geo-import-regressions.mjs new file mode 100644 index 000000000..ba5a040a2 --- /dev/null +++ b/scripts/qa-geo-import-regressions.mjs @@ -0,0 +1,564 @@ +#!/usr/bin/env node +/** Import regressions through the real UI controller and Cesium loaders. + * Run against dev and preview: QA_BASE_URL=http://localhost:4173 node scripts/qa-geo-import-regressions.mjs + */ +import puppeteer from 'puppeteer'; +import * as Cesium from 'cesium'; +import { hookRenderErrors, readRenderErrors } from './qa-browserEvidence.mjs'; + +const browser = await puppeteer.launch({ headless: true }); +const page = await browser.newPage(); +const meshBytes = Buffer.alloc(60); +[ + -2000, 2000, -2000, 2000, 2000, -2000, -2000, 2000, 2000, 2000, 2000, 2000, +].forEach((value, i) => meshBytes.writeFloatLE(value, i * 4)); +[0, 1, 2, 2, 1, 3].forEach((value, i) => + meshBytes.writeUInt16LE(value, 48 + i * 2), +); +const fixture = { + '/__qa_import_surface/tileset.json': { + asset: { version: '1.1' }, + geometricError: 0, + root: { + transform: Cesium.Matrix4.toArray( + Cesium.Transforms.eastNorthUpToFixedFrame( + Cesium.Cartesian3.fromDegrees(-122.5, 37.7), + ), + ), + boundingVolume: { box: [0, 0, 2000, 2000, 0, 0, 0, 2000, 0, 0, 0, 100] }, + geometricError: 0, + content: { uri: 'surface.gltf' }, + }, + }, + '/__qa_import_surface/surface.gltf': { + asset: { version: '2.0' }, + scene: 0, + scenes: [{ nodes: [0] }], + nodes: [{ mesh: 0 }], + meshes: [ + { + primitives: [{ attributes: { POSITION: 0 }, indices: 1, material: 0 }], + }, + ], + materials: [{ doubleSided: true }], + buffers: [ + { + byteLength: 60, + uri: `data:application/octet-stream;base64,${meshBytes.toString('base64')}`, + }, + ], + bufferViews: [ + { buffer: 0, byteOffset: 0, byteLength: 48, target: 34962 }, + { buffer: 0, byteOffset: 48, byteLength: 12, target: 34963 }, + ], + accessors: [ + { + bufferView: 0, + componentType: 5126, + count: 4, + type: 'VEC3', + min: [-2000, 2000, -2000], + max: [2000, 2000, 2000], + }, + { bufferView: 1, componentType: 5123, count: 6, type: 'SCALAR' }, + ], + }, +}; +await page.setRequestInterception(true); +page.on('request', (request) => { + const body = fixture[new URL(request.url()).pathname]; + if (body) + void request.respond({ + status: 200, + contentType: 'application/json', + body: JSON.stringify(body), + }); + else void request.continue(); +}); +const errors = [], + requests = []; +page.on('pageerror', (e) => errors.push(e.message)); +page.on('console', (m) => { + if (/Ignoring clampToGround/.test(m.text())) errors.push(m.text()); +}); +await hookRenderErrors(page); +let failed = 0; +try { + await page.setViewport({ width: 390, height: 844 }); + await page.goto( + `${process.env.QA_BASE_URL || 'http://localhost:4173'}/?welcome=0`, + { waitUntil: 'domcontentloaded' }, + ); + await page.waitForFunction( + () => + window.__gevGeoImport && + document.getElementById('loading-screen')?.classList.contains('hidden'), + { timeout: 90000 }, + ); + await page.evaluate(async () => { + const { viewer, dataManager } = window.__godsEyeView, + C = window.__CESIUM__; + for (const id of dataManager.layers.keys()) + await dataManager.setEnabled(id, false); + viewer.imageryLayers.removeAll(); + viewer.terrainProvider = new C.EllipsoidTerrainProvider(); + viewer.clock.shouldAnimate = false; + viewer.camera.cancelFlight(); + viewer.camera.setView({ + destination: C.Cartesian3.fromDegrees(-122.5, 37.7, 100000), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + }); + page.on('request', (r) => { + if (!/^(data:|blob:)/.test(r.url())) requests.push(r.url()); + }); + const results = await page.evaluate(async () => { + const { viewer } = window.__godsEyeView, + C = window.__CESIUM__, + tool = window.__gevGeoImport; + const results = [], + now = C.JulianDate.clone(viewer.clock.currentTime); + const check = (name, pass, detail = '') => + results.push({ name, pass: Boolean(pass), detail }); + const wrap = (body, attrs = '') => + `${body}`; + const point = '-122.5,37.7,0'; + const load = async (value, name) => { + const content = + typeof value === 'object' && !ArrayBuffer.isView(value) + ? JSON.stringify(value) + : value; + const [outcome] = await tool.importFiles([new File([content], name)]); + return { + outcome, + source: outcome.ok + ? viewer.dataSources.get(viewer.dataSources.length - 1) + : null, + }; + }; + const remove = (r) => { + if (r.outcome.ok) tool.remove(r.outcome.id); + }; + let r = await load( + { + type: 'FeatureCollection', + features: [ + { + type: 'Feature', + id: 'kartaview-owned', + properties: { + name: 'XYZ', + addProperty: 1, + 'marker-symbol': 'rail', + }, + geometry: { type: 'Point', coordinates: [-122.5, 37.7, 100] }, + }, + { + type: 'Feature', + geometry: { + type: 'Polygon', + coordinates: [ + [ + [-122.5, 37.7, 10], + [-122.4, 37.7, 10], + [-122.4, 37.8, 10], + [-122.5, 37.7, 10], + ], + ], + }, + }, + ], + }, + 'xyz.geojson', + ); + const pin = (r.source?.importedEntities || r.source?.entities.values).find( + (e) => e.billboard, + ), + polygon = (r.source?.importedEntities || r.source?.entities.values).find( + (e) => e.polygon, + ); + check( + 'XYZ ground footprint and isolated entity IDs', + r.outcome.ok && + pin.id !== 'kartaview-owned' && + Math.abs( + C.Cartographic.fromCartesian(pin.position.getValue(now)).height, + ) < 0.01 && + polygon.polygon.perPositionHeight?.getValue(now) !== true, + ); + check( + 'ground pins have no persistent clamping callbacks', + pin?.billboard.heightReference?.getValue(now) === + C.HeightReference.NONE && + pin.billboard.disableDepthTestDistance.getValue(now) === Infinity, + ); + remove(r); + r = await load( + wrap( + 'ground-122.5 37.7 0-122.4 37.8 0' + + 'absoluteabsolute-122.5 37.7 10000-122.4 37.8 10000' + + 'line-122.5,37.7 -122.4,37.8', + ), + 'tracks.kml', + ); + const find = (name) => + (r.source?.importedEntities || r.source?.entities.values).find( + (e) => e.name === name, + ); + check( + 'ground tracks and untessellated authored lines clamp', + find('ground')?.polyline.clampToGround?.getValue(now) && + find('line')?.polyline.clampToGround?.getValue(now), + ); + check( + 'explicit track altitude and original vertices survive', + find('absolute')?.polyline.positions.getValue(now).length === 2 && + find('absolute') + .polyline.positions.getValue(now) + .every( + (p) => + Math.abs(C.Cartographic.fromCartesian(p).height - 10000) < 0.01, + ), + ); + remove(r); + r = await load( + wrap( + ` #s ${point}`, + ), + 'local-style.kml', + ); + check('whitespace around a local style remains local', r.outcome.ok); + remove(r); + r = await load( + wrap( + `normalhttps://gev-qa-trap.invalid/style${point}`, + ), + 'incomplete-style.kml', + ); + check('removing external style does not hang the queue', r.outcome.ok); + remove(r); + r = await load( + wrap( + `0${point}`, + ), + 'hidden-folder.kml', + ); + check( + 'import show controls own previously hidden KML content', + r.outcome.ok && + (r.source.importedEntities || r.source.entities.values).every( + (e) => e.isShowing, + ), + ); + remove(r); + r = await load( + wrap( + 'ff00ff0037.837.7-122.4-122.5', + ), + 'colored.kml', + ); + check( + 'a colored overlay without an icon is drawable', + r.outcome.ok && + (r.source.importedEntities || r.source.entities.values).some( + (e) => e.rectangle, + ), + ); + remove(r); + const xml = wrap(`${point}`); + const utf16 = new Uint8Array(2 + xml.length * 2); + utf16.set([255, 254]); + for (let i = 0; i < xml.length; i++) { + utf16[2 + i * 2] = xml.charCodeAt(i) & 255; + utf16[3 + i * 2] = xml.charCodeAt(i) >> 8; + } + r = await load(utf16, 'utf16.kml'); + check('UTF-16 BOM is decoded', r.outcome.ok); + remove(r); + r = await load( + wrap( + `${point}`, + 'xmlns:gx="http://www.google.com/kml/ext/2.2" xsi:schemaLocation="test"', + ), + 'namespaces.kml', + ); + check( + 'ordinary duplicate and missing export namespaces are repaired', + r.outcome.ok, + ); + remove(r); + for (const [name, value] of [ + ['null geometry', { type: 'Feature', geometry: null }], + ['invalid coordinates', { type: 'Point', coordinates: ['bad', 37] }], + [ + 'missing feature type', + { + type: 'FeatureCollection', + features: [{ geometry: { type: 'Point', coordinates: [0, 0] } }], + }, + ], + [ + 'nested unsupported geometry', + { type: 'Feature', geometry: { type: 'Topology', arcs: [] } }, + ], + ]) { + r = await load(value, 'refused.geojson'); + check(`${name} is refused without stopping rendering`, !r.outcome.ok); + remove(r); + } + r = await load( + wrap( + 'NaN,37,0', + ), + 'invalid.kml', + ); + check('non-finite KML coordinates are refused', !r.outcome.ok); + remove(r); + r = await load( + { + type: 'LineString', + coordinates: [ + [-122.6, 37.7], + [-122.4, 37.7], + ], + }, + 'portrait.geojson', + ); + await new Promise((resolve) => setTimeout(resolve, 2300)); + const positions = (r.source?.importedEntities || + r.source?.entities.values)[0].polyline.positions.getValue(now); + const screen = positions?.map((p) => + C.SceneTransforms.worldToWindowCoordinates(viewer.scene, p), + ); + check( + 'portrait flight includes both route endpoints', + screen?.every( + (p) => p && p.x > 0 && p.x < innerWidth && p.y > 0 && p.y < innerHeight, + ), + screen?.map((p) => p && [p.x, p.y]), + ); + remove(r); + r = await load( + { + type: 'MultiPoint', + coordinates: [ + [-74, 40.7], + [139.7, 35.7], + ], + }, + 'intercontinental.geojson', + ); + await new Promise((resolve) => setTimeout(resolve, 2300)); + const intercontinental = ( + r.source?.importedEntities || r.source?.entities.values + ) + .filter((e) => e.position) + .map((e) => e.position.getValue(now)); + const occluder = new C.EllipsoidalOccluder( + C.Ellipsoid.WGS84, + viewer.camera.positionWC, + ); + check( + 'intercontinental points remain in frame and in front of the horizon', + intercontinental?.every((p) => { + const s = C.SceneTransforms.worldToWindowCoordinates(viewer.scene, p); + return ( + occluder.isPointVisible(p) && + s && + s.x > 0 && + s.x < innerWidth && + s.y > 0 && + s.y < innerHeight + ); + }), + ); + remove(r); + r = await load( + wrap( + 'absolute14.9,52.37,3000 5.12,52.09,30005.12,52.09', + ), + 'wall.kml', + ); + await new Promise((resolve) => setTimeout(resolve, 2300)); + check( + 'a wall contributes both endpoints to the flight', + r.outcome.ok && + (r.source.importedEntities || r.source.entities.values) + .find((e) => e.wall) + ?.wall.positions.getValue(now) + .every((p) => { + const s = C.SceneTransforms.worldToWindowCoordinates( + viewer.scene, + p, + ); + return ( + s && s.x > 0 && s.x < innerWidth && s.y > 0 && s.y < innerHeight + ); + }), + ); + remove(r); + r = await load( + { + type: 'MultiPoint', + coordinates: [ + [0, 0], + [180, 0], + ], + }, + 'antipodal.geojson', + ); + await new Promise((resolve) => setTimeout(resolve, 1800)); + check( + 'symmetric global extents keep a finite camera', + r.outcome.ok && + Number.isFinite(viewer.camera.positionCartographic.height), + ); + remove(r); + viewer.camera.cancelFlight(); + viewer.camera.setView({ + destination: C.Cartesian3.fromDegrees(-122.5, 37.7, 100000), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + const mesh = await C.Cesium3DTileset.fromUrl( + `${location.origin}/__qa_import_surface/tileset.json`, + ); + viewer.scene.primitives.add(mesh); + // This fixture installs a mesh outside the app's map-source lifecycle. + // Let the new camera/tiles reach a rendered frame before testing sampling. + await new Promise((resolve) => { + let frames = 0; + const remove = viewer.scene.postRender.addEventListener(() => { + if (++frames >= 2) { + remove(); + resolve(); + } else viewer.scene.requestRender(); + }); + viewer.scene.requestRender(); + }); + const fixtureDeadline = performance.now() + 5000; + while (!mesh.tilesLoaded && performance.now() < fixtureDeadline) { + viewer.scene.requestRender(); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + const globeShown = viewer.scene.globe.show; + viewer.scene.globe.show = false; + + r = await load( + { type: 'Point', coordinates: [-122.5, 37.7, 0] }, + 'mesh-point.geojson', + ); + const meshPoint = (r.source?.importedEntities || r.source?.entities.values) + .find((e) => e.billboard) + ?.position.getValue(now); + const meshHeight = + meshPoint && C.Cartographic.fromCartesian(meshPoint).height; + check( + 'native 3D Tiles ground sampling keeps pins on the rendered mesh', + r.outcome.ok && Math.abs(meshHeight - 2000) < 2, + { + meshHeight, + tilesLoaded: mesh.tilesLoaded, + globeShown: viewer.scene.globe.show, + }, + ); + tool.flyTo(r.outcome.id); + await new Promise((resolve) => setTimeout(resolve, 2300)); + const meshScreen = + meshPoint && + C.SceneTransforms.worldToWindowCoordinates(viewer.scene, meshPoint); + check( + 'a high-surface arrival frames the elevated pin', + meshScreen && + Math.abs(meshScreen.x - innerWidth / 2) < 20 && + Math.abs(meshScreen.y - innerHeight / 2) < 20, + { + screen: meshScreen && [meshScreen.x, meshScreen.y], + camera: { + height: viewer.camera.positionCartographic.height, + pitch: viewer.camera.pitch, + }, + meshHeight, + }, + ); + const groundSource = r.source; + const elevated = await load( + wrap( + 'absolute-122.5,37.7,5000', + ), + 'elevated-pin.kml', + ); + await new Promise((resolve) => setTimeout(resolve, 2600)); + check( + 'a higher imported pin never becomes the sampled ground', + elevated.outcome.ok && + Math.abs( + C.Cartographic.fromCartesian( + groundSource.importedBillboards.get(0).position, + ).height - 2000, + ) < 2, + ); + remove(elevated); + remove(r); + viewer.scene.primitives.remove(mesh); + viewer.scene.globe.show = globeShown; + const before = tool.list().length; + const transfer = new DataTransfer(); + transfer.items.add( + new File( + [JSON.stringify({ type: 'Point', coordinates: [-122.5, 37.7] })], + 'panel-drop.geojson', + ), + ); + const event = new DragEvent('drop', { + bubbles: true, + cancelable: true, + dataTransfer: transfer, + }); + document.getElementById('geo-import-button').dispatchEvent(event); + for (let i = 0; i < 100 && tool.list().length === before; i++) + await new Promise((resolve) => setTimeout(resolve, 20)); + check( + 'a panel drop imports and prevents the browser default', + event.defaultPrevented && tool.list().length === before + 1, + ); + for (const file of tool.list()) tool.remove(file.id); + check( + 'the shared clock stays exactly unchanged', + C.JulianDate.equals(now, viewer.clock.currentTime), + ); + check('no file sources remain', tool.list().length === 0); + return results; + }); + for (const r of results) { + console.log( + `[${r.pass ? 'PASS' : 'FAIL'}] ${r.name}`, + JSON.stringify(r.detail), + ); + if (!r.pass) failed++; + } + const renderErrors = await readRenderErrors(page); + if (errors.length || renderErrors.length) { + failed++; + console.log( + '[FAIL] browser/render errors', + JSON.stringify({ errors, renderErrors }), + ); + } + // Terrain belongs to the base map; imported files may sample that provider. + const unexpected = requests.filter( + (url) => + new URL(url).pathname !== '/logo.svg' && + !new URL(url).pathname.startsWith('/__qa_import_surface/') && + !/\/api\/|\.terrain(?:\?|$)|reearth|cesium|arcgis|openstreetmap|\.pbf(?:\?|$)/i.test( + url, + ), + ); + console.log( + `[${unexpected.length ? 'FAIL' : 'PASS'}] no unexpected file-directed requests`, + JSON.stringify(unexpected), + ); + if (unexpected.length) failed++; +} finally { + await browser.close(); +} +process.exitCode = failed ? 1 : 0; diff --git a/scripts/qa-geo-import.mjs b/scripts/qa-geo-import.mjs new file mode 100644 index 000000000..074eb8fbd --- /dev/null +++ b/scripts/qa-geo-import.mjs @@ -0,0 +1,594 @@ +#!/usr/bin/env node +/** + * Rendered acceptance for DISPLAY ▸ Import: real files through the real file + * input and a real drop, parsed by Cesium in the page. + * + * Proves: GeoJSON, KML, KMZ (with a packed icon and ground overlay) and GPX + * each become one row and one data source; the imported shape actually paints + * on the globe in its row's color (a GeoJSON `stroke`/`fill` does not override + * it) and stops painting when hidden; no import makes the page request + * anything named in it (network link, remote icon, remote style, palette + * icon, balloon or extended-data HTML, maki marker) or anything outside the + * page for it; no entity keeps a description, label or properties; dated KML + * and GPX leave the app's clock alone and still draw now; a GPX track keeps + * its two segments apart; keyboard focus survives hide/show; a dropped file + * imports like a picked one; a file that is not geodata is refused with a + * reason and leaves nothing behind; Remove gives every data source back; no + * page errors. + * + * Usage: dev server up (`npm run dev`), then `node scripts/qa-geo-import.mjs`. + * QA_BASE_URL (default http://localhost:4173), QA_SHOT_DIR, QA_HEADFUL=1. + */ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import puppeteer from 'puppeteer'; +import { hookRenderErrors, readRenderErrors } from './qa-browserEvidence.mjs'; + +const BASE_URL = process.env.QA_BASE_URL || 'http://localhost:4173'; +const SHOT_DIR = process.env.QA_SHOT_DIR || 'qa-shots/geo-import'; +const HEADFUL = process.env.QA_HEADFUL === '1'; +// Golden Gate Park: a big, flat, green rectangle the test polygon can cover. +const VIEW = { lon: -122.4862, lat: 37.7694, height: 4200, pitch: -88 }; +const TRAP_HOST = 'gev-qa-trap.invalid'; + +/* ── fixtures ──────────────────────────────────────────────────────────── */ + +const ring = [ + [-122.511, 37.7645], + [-122.4545, 37.7645], + [-122.4545, 37.7745], + [-122.511, 37.7745], + [-122.511, 37.7645], +]; +const geojson = JSON.stringify({ + type: 'FeatureCollection', + features: [ + { + type: 'Feature', + // A file's own simplestyle and HTML must not survive the import. + properties: { + name: 'QA park', + stroke: '#ff0000', + fill: '#ff0000', + 'fill-opacity': 1, + description: ``, + }, + geometry: { type: 'Polygon', coordinates: [ring.map((p) => [...p, 0])] }, + }, + { + type: 'Feature', + properties: { name: 'QA pin', 'marker-symbol': 'rail' }, + geometry: { type: 'Point', coordinates: [-122.4862, 37.7694, 0] }, + }, + ], +}); +const kml = (iconHref) => ` + + + + + https://${TRAP_HOST}/link.kml + QA KML pin#s + ]]> + 2019-01-01T00:00:00Z2019-01-02T00:00:00Z + ]]> + -122.47,37.77,0 + QA palette pin#palette + -122.48,37.771,0 + QA moving + 2019-01-01T00:00:00Z2019-01-01T01:00:00Z + -122.49 37.767 0-122.475 37.772 0 + QA split tracks + -122.50 37.767 0-122.49 37.768 0 + -122.47 37.771 0-122.46 37.772 0 + + QA remote stylehttps://${TRAP_HOST}/styles.kml#x + -122.50,37.766 -122.46,37.772 + https://${TRAP_HOST}/overlay.png + 37.7737.76-122.46-122.47 + +`; +const gpx = ` + + QA waypoint + QA track<img src="https://${TRAP_HOST}/gpx.png"> + + + + +`; +// 1×1 PNG. +const PNG = Buffer.from( + 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFBQIAX8jx0gAAAABJRU5ErkJggg==', + 'base64', +); +/** Stored (uncompressed) zip — enough for a KMZ fixture. */ +function storedZip(files) { + const parts = []; + const central = []; + let offset = 0; + for (const [name, data] of files) { + const n = Buffer.from(name); + const local = Buffer.alloc(30); + local.writeUInt32LE(0x04034b50, 0); + local.writeUInt32LE(data.length, 18); + local.writeUInt32LE(data.length, 22); + local.writeUInt16LE(n.length, 26); + const c = Buffer.alloc(46); + c.writeUInt32LE(0x02014b50, 0); + c.writeUInt32LE(data.length, 20); + c.writeUInt32LE(data.length, 24); + c.writeUInt16LE(n.length, 28); + c.writeUInt32LE(offset, 42); + parts.push(local, n, data); + central.push(c, n); + offset += 30 + n.length + data.length; + } + const dir = Buffer.concat(central); + const end = Buffer.alloc(22); + end.writeUInt32LE(0x06054b50, 0); + end.writeUInt16LE(files.length, 8); + end.writeUInt16LE(files.length, 10); + end.writeUInt32LE(dir.length, 12); + end.writeUInt32LE(offset, 16); + return Buffer.concat([...parts, dir, end]); +} +const kmzDoc = kml('files/pin.png').replace( + `https://${TRAP_HOST}/overlay.png`, + 'files/overlay.png', +); + +const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gev-qa-geo-import-')); +const fixture = (name, data) => { + const file = path.join(dir, name); + fs.writeFileSync(file, data); + return file; +}; +const FILES = [ + fixture('park.geojson', geojson), + fixture('remote-bits.kml', kml(`https://${TRAP_HOST}/icon.png`)), + fixture( + 'packed.kmz', + storedZip([ + ['doc.kml', Buffer.from(kmzDoc)], + ['files/pin.png', PNG], + ['files/overlay.png', PNG], + ]), + ), + fixture('walk.gpx', gpx), +]; + +/* ── harness ───────────────────────────────────────────────────────────── */ + +const browser = await puppeteer.launch({ + headless: !HEADFUL, + args: [ + '--no-sandbox', + '--disable-background-timer-throttling', + '--disable-renderer-backgrounding', + ...(process.platform === 'darwin' + ? ['--use-angle=metal', '--enable-gpu'] + : ['--use-gl=angle', '--use-angle=swiftshader']), + ], +}); +const page = await browser.newPage(); +await hookRenderErrors(page); +const errors = []; +const trapped = []; +page.on('pageerror', (error) => errors.push(`pageerror: ${error.message}`)); +page.on('console', (message) => { + if (/Ignoring clampToGround/.test(message.text())) + errors.push(message.text()); +}); +// Anything a file names, and the icon assets Cesium would load on a file's +// behalf (KML palette icons from maps.google.com, GeoJSON maki markers). +const FILE_FETCH = [TRAP_HOST, 'maps.google.com/mapfiles', '/maki/']; +page.on('request', (request) => { + if (FILE_FETCH.some((needle) => request.url().includes(needle))) + trapped.push(request.url()); +}); +let failures = 0; +function check(name, passed, detail = '') { + console.log( + `[${passed ? 'PASS' : 'FAIL'}] ${name}${detail ? ` — ${detail}` : ''}`, + ); + if (!passed) failures++; +} +fs.mkdirSync(SHOT_DIR, { recursive: true }); +const shot = (name) => + page.screenshot({ + path: path.join(SHOT_DIR, `${name}.jpg`), + type: 'jpeg', + quality: 82, + }); +const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); +const CLIP = { x: 380, y: 240, width: 480, height: 320 }; + +/** Pixels in the clip close to the first import color (#39d0ff). */ +async function cyanPixels() { + await page.evaluate(() => window.__godsEyeView.viewer.scene.requestRender()); + await wait(600); + const base64 = await page.screenshot({ + clip: CLIP, + type: 'png', + encoding: 'base64', + }); + return page.evaluate(async (b64) => { + const bitmap = await createImageBitmap( + await (await fetch(`data:image/png;base64,${b64}`)).blob(), + ); + const canvas = new OffscreenCanvas(bitmap.width, bitmap.height); + const context = canvas.getContext('2d'); + context.drawImage(bitmap, 0, 0); + const { data } = context.getImageData(0, 0, bitmap.width, bitmap.height); + let n = 0; + for (let i = 0; i < data.length; i += 4) { + const [r, g, b] = [data[i], data[i + 1], data[i + 2]]; + if (b > 45 && g > 30 && r < 100 && b - r > 30) n += 1; + } + return n; + }, base64); +} +const state = () => + page.evaluate(() => ({ + files: window.__gevGeoImport.list(), + dataSources: window.__godsEyeView.viewer.dataSources.length, + rows: document.querySelectorAll('#geo-import-list .geo-import-row').length, + hint: document.getElementById('geo-import-hint')?.textContent || '', + })); + +try { + await page.setViewport({ width: 1280, height: 860 }); + await page.goto(`${BASE_URL}/?welcome=0`, { waitUntil: 'domcontentloaded' }); + await page.waitForFunction( + () => + window.__gevGeoImport && + window.__godsEyeView?.viewer && + document.getElementById('loading-screen')?.classList.contains('hidden'), + { timeout: 90_000 }, + ); + await page.evaluate(async (view) => { + const viewer = window.__godsEyeView.viewer; + for (const id of window.__godsEyeView.dataManager.layers.keys()) + await window.__godsEyeView.dataManager.setEnabled(id, false); + viewer.imageryLayers.removeAll(); + viewer.terrainProvider = new window.__CESIUM__.EllipsoidTerrainProvider(); + viewer.scene.globe.baseColor = window.__CESIUM__.Color.BLACK; + viewer.scene.globe.enableLighting = false; + viewer.camera.cancelFlight?.(); + viewer.camera.setView({ + destination: window.__CESIUM__.Cartesian3.fromDegrees( + view.lon, + view.lat, + view.height, + ), + orientation: { heading: 0, pitch: (view.pitch * Math.PI) / 180, roll: 0 }, + }); + viewer.scene.requestRender(); + }, VIEW); + await wait(4000); + await page.evaluate(() => { + const rail = document.getElementById('pp-toggles'); + if (rail?.classList.contains('collapsed')) + document + .querySelector('.panel-collapse-btn[data-collapse-target="pp-toggles"]') + ?.click(); + }); + await page.waitForFunction( + () => + document.getElementById('geo-import-button')?.getBoundingClientRect() + .width > 0, + { timeout: 10_000 }, + ); + const before = await state(); + check( + 'Import is on the DISPLAY rail with an empty, hidden list', + before.rows === 0 && + !(await page.evaluate(() => + document + .getElementById('geo-import-list') + .classList.contains('visible'), + )), + ); + const cyanBefore = await cyanPixels(); + await page.evaluate(() => { + window.__godsEyeView.viewer.clock.shouldAnimate = false; + }); + const clock = () => + page.evaluate(() => { + const c = window.__godsEyeView.viewer.clock; + const iso = (t) => window.__CESIUM__.JulianDate.toIso8601(t, 0); + return { + start: iso(c.startTime), + stop: iso(c.stopTime), + range: c.clockRange, + step: c.clockStep, + multiplier: c.multiplier, + animate: c.shouldAnimate, + // Wall-clock drift aside, the current time must not jump to 2018/2019. + currentTime: window.__CESIUM__.JulianDate.toIso8601(c.currentTime), + }; + }); + const clockBefore = await clock(); + const sourcesBefore = before.dataSources; + + // Picked through the real . + const input = await page.$('#geo-import-input'); + await input.uploadFile(...FILES); + const imported = await page + .waitForFunction(() => window.__gevGeoImport.list().length === 4, { + timeout: 60_000, + }) + .then( + () => true, + () => false, + ); + const after = await state(); + check( + 'four picked files become four rows and four data sources', + imported && + after.rows === 4 && + after.dataSources === before.dataSources + 4, + JSON.stringify( + after.files.map((f) => `${f.name}:${f.format}:${f.entityCount}`), + ), + ); + check( + 'formats are recognized', + after.files.map((f) => f.format).join() === 'geojson,kml,kmz,gpx', + ); + check( + 'the KML and KMZ had their outside references removed', + after.files[1].removed > 0 && after.files[2].removed > 0, + `removed ${after.files[1].removed}/${after.files[2].removed}`, + ); + check( + 'the hint says so', + /Added 4 files\..*left out/.test(after.hint), + after.hint, + ); + const clockAfter = await clock(); + check( + "dated KML and GPX leave the app's clock alone", + JSON.stringify(clockAfter) === JSON.stringify(clockBefore) && + (await page.evaluate( + () => + window.__godsEyeView.viewer.clockTrackedDataSource?.clock === + undefined, + )), + `${JSON.stringify(clockBefore)} → ${JSON.stringify(clockAfter)}`, + ); + const inside = await page.evaluate((from) => { + const viewer = window.__godsEyeView.viewer; + const now = viewer.clock.currentTime; + const sources = []; + for (let i = from; i < viewer.dataSources.length; i++) + sources.push(viewer.dataSources.get(i)); + const entities = sources.flatMap( + (ds) => ds.importedEntities || ds.entities.values, + ); + const named = (name) => entities.find((e) => e.name === name); + const gpx = sources[3].importedEntities || sources[3].entities.values; + const kmz = sources[2].importedEntities || sources[2].entities.values; + return { + sources: sources.length, + clocks: sources.filter((ds) => ds.clock !== undefined).length, + described: entities.filter((e) => e.description !== undefined).length, + labelled: entities.filter((e) => e.label !== undefined).length, + withProperties: entities.filter((e) => e.properties !== undefined).length, + withKml: entities.filter((e) => e.kml !== undefined).length, + hidden: entities.filter((e) => !e.isAvailable(now)).length, + datedPinNow: Boolean(named('QA KML pin')?.isAvailable(now)), + fallbackPins: ['QA KML pin', 'QA palette pin'].filter((name) => + named(name)?.billboard?.image?.getValue(now), + ).length, + movingIsLine: Boolean( + named('QA moving')?.polyline && !named('QA moving')?.position, + ), + movingVertices: + named('QA moving')?.polyline?.positions?.getValue(now)?.length, + splitTracks: (sources[1].importedEntities || sources[1].entities.values) + .filter((e) => e.parent?.name === 'QA split tracks' && e.polyline) + .map((e) => e.polyline.positions.getValue(now).length), + gpxLines: gpx.filter((e) => e.polyline).length, + gpxPoints: gpx.filter((e) => e.billboard || e.point).length, + kmzLocalImage: kmz.some((e) => + String(e.billboard?.image?.getValue(now) || '').startsWith('blob:'), + ), + kmzOverlay: kmz.some((e) => e.rectangle), + groundedLines: sources + .slice(1, 3) + .every((ds) => + (ds.importedEntities || ds.entities.values) + .filter((e) => e.polyline) + .every((e) => e.polyline.clampToGround?.getValue(now)), + ), + xyzPolygon: + (sources[0].importedEntities || sources[0].entities.values) + .find((e) => e.polygon) + ?.polygon.perPositionHeight?.getValue(now) !== true, + }; + }, sourcesBefore); + check( + 'no imported entity keeps a description, label, properties or KML data', + inside.sources === 4 && + inside.described + + inside.labelled + + inside.withProperties + + inside.withKml === + 0, + JSON.stringify(inside), + ); + check( + 'no imported source has a clock, and dated features draw now', + inside.clocks === 0 && + inside.hidden === 0 && + inside.datedPinNow && + inside.movingIsLine && + inside.movingVertices === 2, + JSON.stringify(inside), + ); + check( + 'blocked KML icon sources use drawable local fallback pins', + inside.fallbackPins === 2, + JSON.stringify({ fallbackPins: inside.fallbackPins }), + ); + check( + 'KML MultiTrack keeps its segments and original vertex count', + inside.splitTracks.join() === '2,2', + JSON.stringify(inside.splitTracks), + ); + check( + 'the GPX track keeps its two segments apart', + inside.gpxLines === 2 && inside.gpxPoints === 1, + JSON.stringify(inside), + ); + check( + 'KMZ retains its own packed image and overlay', + inside.kmzLocalImage && inside.kmzOverlay, + ); + check( + 'XYZ polygons and KML/KMZ lines use ground geometry', + inside.xyzPolygon && inside.groundedLines, + ); + await page.evaluate((from) => { + for (const entity of window.__godsEyeView.viewer.dataSources.get(from) + .entities.values) + if (entity.billboard) entity.show = false; + }, sourcesBefore); + await page.evaluate((view) => { + const viewer = window.__godsEyeView.viewer; + viewer.camera.cancelFlight?.(); + viewer.camera.setView({ + destination: window.__CESIUM__.Cartesian3.fromDegrees( + view.lon, + view.lat, + view.height, + ), + orientation: { heading: 0, pitch: (view.pitch * Math.PI) / 180, roll: 0 }, + }); + }, VIEW); + await wait(2500); + const cyanShown = await cyanPixels(); + await shot('imported'); + check( + 'the imported polygon paints on the globe', + cyanShown > cyanBefore * 4 && cyanShown > cyanBefore + 5000, + `${cyanBefore} → ${cyanShown}`, + ); + + const parkId = after.files[0].id; + await page.click( + `#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`, + ); + const cyanHidden = await cyanPixels(); + check( + 'hiding it stops it painting', + cyanHidden < cyanShown / 3, + `${cyanShown} → ${cyanHidden}`, + ); + await page.click( + `#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`, + ); + check( + 'showing it again paints it again', + (await cyanPixels()) > cyanHidden * 4, + ); + + // Keyboard: focus stays on the same control when the list is rebuilt. + const toggleSelector = `#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`; + await page.focus(toggleSelector); + await page.keyboard.press('Enter'); + const focusAfterHide = await page.evaluate( + (sel) => document.activeElement === document.querySelector(sel), + toggleSelector, + ); + await page.keyboard.press('Enter'); + const focusAfterShow = await page.evaluate( + (sel) => + document.activeElement === document.querySelector(sel) && + document.activeElement.getAttribute('aria-pressed') === 'true', + toggleSelector, + ); + check( + 'keyboard focus stays on the toggle through hide and show', + focusAfterHide && focusAfterShow, + ); + + // Dropped onto the globe, as a person drags from their file manager. + await page.evaluate((text) => { + const container = window.__godsEyeView.viewer.container; + const transfer = new DataTransfer(); + transfer.items.add( + new File([text], 'dropped.geojson', { type: 'application/geo+json' }), + ); + for (const type of ['dragenter', 'dragover', 'drop']) + container.dispatchEvent( + new DragEvent(type, { + dataTransfer: transfer, + bubbles: true, + cancelable: true, + }), + ); + }, geojson); + const dropped = await page + .waitForFunction(() => window.__gevGeoImport.list().length === 5, { + timeout: 20_000, + }) + .then( + () => true, + () => false, + ); + check('a file dropped on the globe imports', dropped); + + // Not geodata: refused with a reason, nothing added. + await page.evaluate(() => { + const transfer = new DataTransfer(); + transfer.items.add( + new File(['just some notes'], 'notes.txt', { type: 'text/plain' }), + ); + window.__godsEyeView.viewer.container.dispatchEvent( + new DragEvent('drop', { + dataTransfer: transfer, + bubbles: true, + cancelable: true, + }), + ); + }); + await wait(800); + const refused = await state(); + check( + 'a file that is not geodata is refused with a reason', + refused.files.length === 5 && + /notes\.txt: Only GeoJSON, KML, KMZ and GPX/.test(refused.hint), + refused.hint, + ); + + // Remove every file through its row button. + for (const { id } of refused.files) + await page.click( + `#geo-import-list [data-import-id="${id}"] button[data-action="remove"]`, + ); + const cleared = await state(); + check( + 'Remove gives every data source back', + cleared.rows === 0 && cleared.dataSources === before.dataSources, + `${cleared.dataSources} vs ${before.dataSources}`, + ); + await wait(1500); + check( + 'nothing named inside a file, and no icon asset for one, was ever requested', + trapped.length === 0, + trapped.join(' '), + ); + errors.push(...(await readRenderErrors(page))); + check('no page or rendering errors', errors.length === 0, errors.join(' | ')); +} catch (error) { + check('harness ran to completion', false, error?.stack || String(error)); +} finally { + await browser.close(); + fs.rmSync(dir, { recursive: true, force: true }); +} +console.log(failures ? `\n${failures} check(s) failed` : '\nall checks passed'); +process.exit(failures ? 1 : 0); diff --git a/scripts/qa-gnss.mjs b/scripts/qa-gnss.mjs new file mode 100644 index 000000000..41933141e --- /dev/null +++ b/scripts/qa-gnss.mjs @@ -0,0 +1,988 @@ +#!/usr/bin/env node +/** + * qa-gnss.mjs — headless proof for the GNSS Integrity layer + * (id `gnss-integrity`): low navigation accuracy reported by ADS-B. + * + * Drives the REAL app in headless Chromium against a dev server (keyless). + * The live section reads adsb.lol through `/api/gnss-integrity` (60 s server + * cache, one upstream read per whole-degree anchor); the other sections + * stub that route in the page so they are deterministic. + * + * (i) LIVE — over Poland and the southern Baltic the layer settles with no error, + * one geometry per cell and legend band counts that sum to the cell + * count; cells are required only when the returned rows, binned + * with the layer's own rules, reach the per-cell minimum (live + * traffic varies by hour). Requires a non-stale response no older than 60 s. + * (ii) BANDS (synthetic) — a 10-aircraft cell with 5 degraded is high, + * a 30-aircraft cell with 2 degraded is medium, a 3-aircraft cell + * with 1 degraded is low (first bad discounted), a clean cell at sea + * is low, a 2-aircraft cell is withheld; `stale: true` reaches + * getStats. Every band is drawn and low is shown over land and sea. + * The row reads as low navigation accuracy, and stats carry the + * machine-readable provenance (GEV classifier, gpsjam bands, window). + * PICK verifies actual perimeter and empty-ground clicks under GNSS. + * CAMERA/GLOBE verify automatic sampling and the screen-centre anchor. + * (iii) FAILURE — the route stubbed to 502: stats.error is set and the + * cells already on the globe are kept. + * (iv) DISABLE — owned ground primitives are hidden. + * (v) KEYBOARD — the row's toggle button, focused, turns the layer on + * with Enter and off with Space, keeps focus and names its state. + * (vi) VIEWPORT — at a 390x844 phone viewport the open row and its legend + * fit the panel width with no horizontal overflow. + * (vii) AGE (synthetic) — a fresh page whose stubbed proxy replays the same + * stale snapshot observed 31 minutes ago draws nothing and reports no + * error; observed 29 minutes ago it draws its cells. The replay keeps + * its observation time instead of restarting the 30-minute window. + * + * Screenshots wait until every cell's ground geometry is built and show the + * open Data Layers row with its legend. Visual proof saved to + * qa-shots/gnss-*.png (gitignored). + * + * Run: node scripts/qa-gnss.mjs --url http://localhost:4173 + * (also against a built preview: npm run build && npm run preview, + * then pass the preview URL) + * Optional: --timeout-ms 180000 for a slow boot. + * Exits non-zero on any FAIL. Does not commit anything. + */ + +import puppeteer from 'puppeteer'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { + accumulateGnssObservations, + binGnssCells, +} from '../src/layers/gnss/records.js'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const REPO_ROOT = path.resolve(__dirname, '..'); +const SHOTS_DIR = path.join(REPO_ROOT, 'qa-shots'); + +const argv = process.argv.slice(2); +const getOpt = (name, dflt) => { + const i = argv.indexOf(name); + return i >= 0 && argv[i + 1] ? argv[i + 1] : dflt; +}; +const APP_URL = getOpt('--url', 'http://localhost:4173'); +const HEADFUL = argv.includes('--headful'); +const TIMEOUT_MS = Number(getOpt('--timeout-ms', '60000')); +const LAYER_ID = 'gnss-integrity'; + +const results = []; +function record(name, ok, detail) { + results.push({ name, ok, detail }); + const tag = ok ? '\x1b[32mPASS\x1b[0m' : '\x1b[31mFAIL\x1b[0m'; + console.log(` [${tag}] ${name}${detail ? ` — ${detail}` : ''}`); +} +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** Boot the app, dismiss the intro and place the camera straight down. */ +async function boot(page, { lon, lat, height }) { + await page.goto(`${APP_URL}/?welcome=0`, { + waitUntil: 'domcontentloaded', + timeout: TIMEOUT_MS, + }); + await page.waitForFunction( + () => window.__godsEyeView?.viewer && window.__godsEyeView?.dataManager, + { timeout: TIMEOUT_MS }, + ); + await sleep(5000); + await page.keyboard.press('Escape'); + await page.evaluate( + (lo, la, h) => { + const gev = window.__godsEyeView; + const d2r = Math.PI / 180; + // The app's intro flyTo animation clobbers a setView issued mid-flight. + try { + gev.viewer.camera.cancelFlight(); + } catch { + /* no flight active */ + } + gev.viewer.camera.setView({ + destination: gev.viewer.scene.globe.ellipsoid.cartographicToCartesian({ + longitude: lo * d2r, + latitude: la * d2r, + height: h, + }), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + gev.viewer.scene.requestRender?.(); + }, + lon, + lat, + height, + ); +} + +/** Enable (or refresh) the layer and poll until it settles. */ +async function enableAndSettle(page, { refresh = false } = {}) { + return page.evaluate( + async (id, again) => { + const dm = window.__godsEyeView.dataManager; + const layer = dm.layers.get(id).module; + if (again) await dm.refreshLayer(id); + else await dm.setEnabled(id, true); + const stats = layer.getStats(); + const rowControls = layer.getRowControls().legend; + const legend = rowControls.map(({ label, count }) => ({ label, count })); + const blurb = rowControls[0]?.blurb ?? ''; + const rowName = + document.querySelector(`[data-layer-id="${id}"] .data-name`) + ?.textContent ?? ''; + const all = window.__godsEyeView.viewer.scene.groundPrimitives; + const ds = Array.from({ length: all.length }, (_, i) => all.get(i)).find( + (item) => item.id === id, + ); + return { + stats, + legend, + blurb, + rowName, + name: layer.name, + entities: ds?.get(0)?.geometryInstances.length ?? 0, + shown: ds?.show ?? null, + }; + }, + LAYER_ID, + refresh, + ); +} + +/** Open the Data Layers panel and bring the GNSS row into view. */ +async function showLayerRow(page) { + await page.evaluate((id) => { + const panel = document.getElementById('data-panel'); + if (panel.classList.contains('collapsed')) + panel.querySelector('[data-collapse-target="data-panel"]').click(); + document + .querySelector(`[data-layer-id="${id}"]`) + ?.scrollIntoView({ block: 'center' }); + }, LAYER_ID); + await sleep(800); +} + +/** Actual card pixels on the shared canvas, available in dev and production. */ +function paintedOverlayPixels() { + const canvas = document.getElementById('world-overlay-canvas'); + if (!canvas) return -1; + const pixels = canvas + .getContext('2d') + .getImageData(0, 0, canvas.width, canvas.height).data; + let count = 0; + for (let i = 3; i < pixels.length; i += 4) if (pixels[i] > 0) count++; + return count; +} + +async function shoot(page, name) { + await page + .waitForFunction( + () => window.__godsEyeView.viewer.scene.globe.tilesLoaded, + { timeout: TIMEOUT_MS }, + ) + .catch(() => {}); + await showLayerRow(page); + await page.screenshot({ path: path.join(SHOTS_DIR, `gnss-${name}.png`) }); +} + +/** Bounding-sphere state of every GNSS cell: DONE = 0, PENDING = 1, FAILED = 2. */ +function cellStates(id) { + const { viewer } = window.__godsEyeView; + const all = viewer.scene.groundPrimitives; + const ds = Array.from({ length: all.length }, (_, i) => all.get(i)).find( + (item) => item.id === id, + ); + if (!ds?.length) return []; + return [ds.get(0).ready ? 0 : 1]; +} + +/** Wait until every GNSS cell's ground rectangle has finished building. */ +async function waitForBuiltCells(page, timeoutMs = 60000) { + const built = await page + .waitForFunction( + `(${cellStates})(${JSON.stringify(LAYER_ID)}).every((state) => state === 0)`, + { timeout: timeoutMs, polling: 250 }, + ) + .then( + () => true, + () => false, + ); + const states = await page.evaluate( + `(${cellStates})(${JSON.stringify(LAYER_ID)})`, + ); + return { built, detail: `states=${JSON.stringify(states)}` }; +} + +const fixtureRow = (hex, lat, lon, degraded) => ({ + hex, + lat, + lon, + nic: degraded ? 0 : 8, + nacp: degraded ? 0 : 10, + gpsLost: false, + degraded, +}); + +/** + * Synthetic 0.5° cells: high at sea (5/10 bad), medium on land (2/30 bad), + * low on land (1/3 bad), low at sea (0/4 bad), withheld (2 aircraft). + */ +function fixturePayload() { + const rows = []; + const hex = (prefix, i) => `${prefix}${i.toString(16).padStart(2, '0')}`; + for (let i = 0; i < 10; i++) + rows.push(fixtureRow(hex('a000', i), 55.1 + i * 0.01, 19.1, i < 5)); + for (let i = 0; i < 30; i++) + rows.push(fixtureRow(hex('d000', i), 54.1 + i * 0.01, 19.6, i < 2)); + for (let i = 0; i < 3; i++) + rows.push(fixtureRow(hex('b000', i), 54.1 + i * 0.01, 18.1, i === 0)); + for (let i = 0; i < 4; i++) + rows.push(fixtureRow(hex('e000', i), 55.1 + i * 0.01, 18.1, false)); + for (let i = 0; i < 2; i++) + rows.push(fixtureRow(hex('c000', i), 53.1 + i * 0.01, 17.1, true)); + return { + fetchedAt: Date.now(), + anchor: { lat: 55, lon: 19 }, + radiusNm: 250, + rows, + stale: true, + }; +} + +/** Answer /api/gnss-integrity in-page from `fixturePayload()`. */ +async function stubRoute(page, { ageMs = 0, initialFailures = 0, stale } = {}) { + await page.evaluateOnNewDocument( + (payload, age, failures, staleOverride) => { + window.__gnssQaAgeMs = age; + window.__gnssQaRequests = []; + window.__gnssQaFailures = failures; + const realFetch = window.fetch.bind(window); + window.__gnssQaMode = 'fixture'; + window.fetch = (input, init) => { + const url = typeof input === 'string' ? input : input?.url || ''; + if (!url.includes('/api/gnss-integrity')) return realFetch(input, init); + window.__gnssQaRequests.push(url); + if (window.__gnssQaFailures > 0) { + window.__gnssQaFailures--; + return Promise.resolve( + Response.json( + { error: 'rate_limited' }, + { + status: 429, + headers: { 'Retry-After': '5' }, + }, + ), + ); + } + const ok = window.__gnssQaMode === 'fixture'; + const body = ok + ? { + ...payload, + fetchedAt: Date.now() - (window.__gnssQaAgeMs || 0), + ageMs: window.__gnssQaAgeMs || 0, + stale: staleOverride ?? payload.stale, + } + : { error: 'gnss_integrity_unavailable' }; + return Promise.resolve( + new Response(JSON.stringify(body), { + status: ok ? 200 : 502, + headers: { 'Content-Type': 'application/json' }, + }), + ); + }; + }, + fixturePayload(), + ageMs, + initialFailures, + stale, + ); +} + +/** Toggle button of the GNSS row, as the panel draws it. */ +function toggleState(id) { + const button = document.querySelector( + `[data-layer-id="${id}"] .data-toggle-btn`, + ); + const layer = window.__godsEyeView.dataManager.layers.get(id); + return { + focused: document.activeElement === button, + label: button?.getAttribute('aria-label') ?? '', + enabled: Boolean(layer?.enabled), + count: layer?.module.getStats().count ?? -1, + }; +} + +/** Press a key on the focused GNSS toggle and wait for the lifecycle to settle. */ +async function pressToggle(page, key, wantEnabled) { + await page.evaluate((id) => { + document.querySelector(`[data-layer-id="${id}"] .data-toggle-btn`)?.focus(); + }, LAYER_ID); + await page.keyboard.press(key); + await page + .waitForFunction( + (id, want) => { + const layer = window.__godsEyeView.dataManager.layers.get(id); + return ( + Boolean(layer?.enabled) === want && + (!want || layer.module.getStats().count > 0) + ); + }, + { timeout: 30000, polling: 250 }, + LAYER_ID, + wantEnabled, + ) + .catch(() => {}); + return page.evaluate(`(${toggleState})(${JSON.stringify(LAYER_ID)})`); +} + +async function main() { + console.log('\nGNSS Integrity proof (qa-gnss)'); + console.log(` App URL : ${APP_URL}\n`); + try { + const res = await fetch(APP_URL); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + } catch (e) { + console.error( + `\x1b[31mDev server not reachable at ${APP_URL} (${e.message}).\x1b[0m`, + ); + process.exit(2); + } + fs.mkdirSync(SHOTS_DIR, { recursive: true }); + const executablePath = + process.env.PUPPETEER_EXECUTABLE_PATH || + (await puppeteer.executablePath().catch(() => null)) || + undefined; + const browser = await puppeteer.launch({ + headless: HEADFUL ? false : 'new', + ...(executablePath && fs.existsSync(executablePath) + ? { executablePath } + : {}), + args: [ + '--use-gl=angle', + '--use-angle=swiftshader', + '--disable-dev-shm-usage', + '--disable-background-timer-throttling', + '--disable-renderer-backgrounding', + '--window-size=1440,900', + ], + }); + + let exitCode = 0; + const errors = []; + try { + // ── (i) LIVE ────────────────────────────────────────────────────────── + console.log( + '(i) LIVE — adsb.lol integrity snapshot over Poland and the southern Baltic...', + ); + const live = await browser.newPage(); + await live.setViewport({ width: 1440, height: 900 }); + live.on('pageerror', (e) => errors.push(e.message)); + const liveBodies = []; + live.on('response', (res) => { + if (res.url().includes('/api/gnss-integrity') && res.ok()) + liveBodies.push(res.json()); + }); + await boot(live, { lon: 19.6, lat: 52.6, height: 1_100_000 }); + const l = await enableAndSettle(live); + const legendTotal = l.legend.reduce((sum, { count }) => sum + count, 0); + // Bin what adsb.lol actually returned with the layer's own rules: live + // traffic varies by hour, so cells are required only when some cell + // really reached the minimum aircraft count. + const liveSnapshots = (await Promise.allSettled(liveBodies)) + .filter((r) => r.status === 'fulfilled') + .map((r) => r.value); + record( + 'LIVE: a successful, dated integrity response was received', + liveSnapshots.length > 0 && + liveSnapshots.every( + (body) => + Array.isArray(body.rows) && + Number.isFinite(body.ageMs) && + body.ageMs >= 0 && + body.ageMs <= 60000 && + body.stale !== true && + body.classifier === 'gev-nic-nacp-v2', + ), + `successful responses=${liveSnapshots.length}`, + ); + const liveRows = liveSnapshots.reduce( + (sum, body) => sum + (body.rows?.length || 0), + 0, + ); + const expectedStore = new Map(); + for (const body of liveSnapshots) { + const at = Date.now(); + accumulateGnssObservations(expectedStore, body.rows, at, { + observedAt: at - body.ageMs, + }); + } + const expectedCells = binGnssCells(expectedStore).length; + record( + 'LIVE: no error, one geometry per cell', + !l.stats.error && l.entities === l.stats.count, + `cells=${l.stats.count} entities=${l.entities} error=${JSON.stringify(l.stats.error)}`, + ); + if (expectedCells > 0) { + record( + 'LIVE: a snapshot with a qualifying cell yields cells', + l.stats.count > 0, + `rows=${liveRows} expected=${expectedCells} cells=${l.stats.count}`, + ); + } else { + console.log( + ` [info] ${liveRows} live rows, no cell reaches the minimum — cell count not asserted`, + ); + } + record( + 'LIVE: legend band counts sum to the cell count', + legendTotal === l.stats.count, + JSON.stringify(l.legend), + ); + const liveBuilt = await waitForBuiltCells(live); + record( + 'LIVE: every cell rectangle is built on a parked camera', + liveBuilt.built, + liveBuilt.detail, + ); + await shoot(live, 'live-poland-baltic'); + await live.close(); + + // ── (ii) BANDS ──────────────────────────────────────────────────────── + console.log( + '\n(ii) BANDS — synthetic integrity rows through the real layer...', + ); + const page = await browser.newPage(); + await page.setViewport({ width: 1440, height: 900 }); + page.on('pageerror', (e) => errors.push(e.message)); + // Stub the route in-page rather than with request interception: an + // intercepted page stalls Cesium's geometry workers, so ground rectangles + // never finish building and the screenshot would not show real behaviour. + await stubRoute(page); + await boot(page, { lon: 18.6, lat: 54.6, height: 450_000 }); + const f = await enableAndSettle(page); + const band = (prefix) => + f.legend.find(({ label }) => label.startsWith(prefix))?.count; + record( + 'BANDS: 4 cells shown, the 2-aircraft cell withheld', + f.stats.count === 4 && f.entities === 4, + `cells=${f.stats.count} entities=${f.entities}`, + ); + record( + 'BANDS: 5/10 is high, 2/30 is medium, 1/3 and 0/4 are low', + band('Low-accuracy score >') === 1 && + band('Low-accuracy score 2') === 1 && + band('Low-accuracy score <') === 2, + JSON.stringify(f.legend), + ); + const prov = f.stats.provenance; + record( + 'PROVENANCE: GEV classifier, gpsjam bands and visited-view window are machine-readable', + prov?.classifier?.id === 'gev-nic-nacp-v2' && + prov.classifier.definedBy === 'GEV' && + prov.classifier.validated === false && + prov.aggregation?.source === 'https://gpsjam.org/faq' && + JSON.stringify(prov.aggregation.bands) === '[2,10]' && + prov.window?.minutes === 30 && + prov.window.scope === 'visited-view' && + prov.interpretation?.validated === false, + JSON.stringify(prov?.classifier), + ); + record( + 'WORDING: the row leads with low navigation accuracy, interference only as the hedged reading', + f.name === 'GNSS Integrity' && + f.rowName.includes('GNSS Integrity') && + !/interference/i.test(f.rowName) && + f.legend.every(({ label }) => label.startsWith('Low-accuracy score')) && + /one aircraft is discounted/.test(f.blurb) && + /not a detection/.test(f.blurb), + `name=${JSON.stringify(f.name)}`, + ); + record( + 'BANDS: stale snapshot is reported', + f.stats.stale === true, + `stale=${f.stats.stale}`, + ); + const bandsBuilt = await waitForBuiltCells(page); + record( + 'BANDS: every cell rectangle is built on a parked camera', + bandsBuilt.built, + bandsBuilt.detail, + ); + await shoot(page, 'synthetic-bands'); + + // GNSS fills must not own map input, including over a real perimeter. + await page.evaluate(async () => { + const original = window.fetch.bind(window); + window.fetch = (input, init) => { + const url = typeof input === 'string' ? input : input.url; + if (url.includes('/api/fire-perimeters/inciweb')) + return Promise.resolve(Response.json({ rows: [] })); + if (url.includes('/api/fire-perimeters')) + return Promise.resolve( + Response.json({ + rows: [ + { + stableId: 'gnss-pick-qa', + name: 'GNSS PICK QA FIRE', + containedPct: 0, + acres: 100, + polygons: [ + [ + [ + [18.247, 54.247], + [18.253, 54.247], + [18.253, 54.253], + [18.247, 54.253], + [18.247, 54.247], + ], + ], + ], + }, + ], + }), + ); + return original(input, init); + }; + const { viewer, dataManager } = window.__godsEyeView; + viewer.camera.cancelFlight(); + viewer.camera.setView({ + destination: viewer.scene.globe.ellipsoid.cartographicToCartesian({ + longitude: (18.25 * Math.PI) / 180, + latitude: (54.25 * Math.PI) / 180, + height: 2500, + }), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + await dataManager.setEnabled('fire-perimeters', true); + viewer.scene.requestRender(); + }); + await page.waitForFunction( + () => { + const v = window.__godsEyeView.viewer, + p = v.scene.pick({ + x: v.scene.canvas.clientWidth / 2, + y: v.scene.canvas.clientHeight / 2, + }); + return String(p?.id?.id ?? p?.id ?? '').startsWith( + 'fire-perimeter:gnss-pick-qa:', + ); + }, + { timeout: TIMEOUT_MS, polling: 250 }, + ); + const pickProbe = await page.evaluate(() => { + const v = window.__godsEyeView.viewer, + c = v.scene.canvas; + const centre = { x: c.clientWidth / 2, y: c.clientHeight / 2 }; + const empty = { x: centre.x + 300, y: centre.y }; + const picked = v.scene.pick(empty), + box = c.getBoundingClientRect(); + return { + centre: { x: centre.x + box.left, y: centre.y + box.top }, + empty: { x: empty.x + box.left, y: empty.y + box.top }, + surface: !picked || picked.id === undefined, + height: v.camera.positionCartographic.height, + }; + }); + record('PICK: a perimeter remains selectable under GNSS', true); + record( + 'PICK: street ground remains a surface pick under GNSS', + pickProbe.surface, + ); + const beforeCardPixels = await page.evaluate(paintedOverlayPixels); + await page.mouse.click(pickProbe.centre.x, pickProbe.centre.y); + await page.waitForFunction( + `(${paintedOverlayPixels})() > ${beforeCardPixels + 100}`, + { timeout: 10000, polling: 250 }, + ); + const selectedCardPixels = await page.evaluate(paintedOverlayPixels); + record( + 'PICK: actual perimeter click paints its incident card', + selectedCardPixels > beforeCardPixels + 100, + `${beforeCardPixels} → ${selectedCardPixels} pixels`, + ); + await page.screenshot({ + path: path.join(SHOTS_DIR, 'gnss-perimeter-selected.png'), + }); + await page.mouse.click(pickProbe.empty.x, pickProbe.empty.y); + await page.waitForFunction( + `(${paintedOverlayPixels})() <= ${beforeCardPixels}`, + { timeout: 10000, polling: 250 }, + ); + const clearedCardPixels = await page.evaluate(paintedOverlayPixels); + record( + 'PICK: empty ground clears the perimeter card', + clearedCardPixels <= beforeCardPixels, + `${selectedCardPixels} → ${clearedCardPixels} pixels`, + ); + await page.mouse.click(pickProbe.empty.x, pickProbe.empty.y, { + clickCount: 2, + }); + await sleep(1000); + record( + 'PICK: empty double-click does not zoom to a GNSS cell', + await page.evaluate( + (height) => + Math.abs( + window.__godsEyeView.viewer.camera.positionCartographic.height - + height, + ) < 200, + pickProbe.height, + ), + ); + await page.evaluate(async () => { + await window.__godsEyeView.dataManager.setEnabled( + 'fire-perimeters', + false, + ); + }); + console.log('\nCAMERA — automatic regional refresh without toggling...'); + const beforeMove = await page.evaluate( + () => window.__gnssQaRequests.length, + ); + await page.evaluate(() => { + const { viewer } = window.__godsEyeView; + viewer.camera.setView({ + destination: viewer.scene.globe.ellipsoid.cartographicToCartesian({ + longitude: (139.7 * Math.PI) / 180, + latitude: (35.7 * Math.PI) / 180, + height: 1_100_000, + }), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + viewer.scene.requestRender(); + }); + await page.waitForFunction( + () => { + const url = window.__gnssQaRequests.at(-1); + return ( + url && + Math.abs( + Number(new URL(url, location.origin).searchParams.get('lon')) - + 139.7, + ) < 1 + ); + }, + { timeout: 15000 }, + ); + await sleep(3500); + const moved = await page.evaluate( + (id) => ({ + requests: window.__gnssQaRequests.length, + enabled: window.__godsEyeView.dataManager.isEnabled(id), + coverage: window.__godsEyeView.dataManager.layers + .get(id) + .module.getStats().coverage, + info: document.querySelector( + `[data-layer-id="${id}"] .data-toggle-controls-info`, + )?.textContent, + }), + LAYER_ID, + ); + record( + 'CAMERA: moving to Tokyo automatically refreshes the new region and updates the row', + moved.enabled && + moved.requests > beforeMove && + moved.requests <= beforeMove + 2 && + moved.coverage?.longitude === 140 && + /Regional sample.*250 nm radius/.test(moved.info), + JSON.stringify(moved), + ); + await shoot(page, 'camera-refresh'); + await page.evaluate(() => { + const { viewer } = window.__godsEyeView; + viewer.camera.setView({ + destination: viewer.scene.globe.ellipsoid.cartographicToCartesian({ + longitude: (12.4 * Math.PI) / 180, + latitude: (41.9 * Math.PI) / 180, + height: 16_000_000, + }), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + viewer.scene.requestRender(); + }); + await page.waitForFunction( + () => { + const url = window.__gnssQaRequests.at(-1); + const params = url && new URL(url, location.origin).searchParams; + return ( + params && + Math.abs(Number(params.get('lon')) - 12.4) < 1 && + Math.abs(Number(params.get('lat')) - 41.9) < 1 + ); + }, + { timeout: 15000 }, + ); + const world = await page.evaluate((id) => { + const { viewer, dataManager } = window.__godsEyeView; + const bounds = viewer.camera.computeViewRectangle( + viewer.scene.globe.ellipsoid, + ); + return { + enabled: dataManager.isEnabled(id), + height: viewer.camera.positionCartographic.height, + boundsMidLongitude: + bounds && ((bounds.west + bounds.east) * 90) / Math.PI, + coverage: dataManager.layers.get(id).module.getStats().coverage, + }; + }, LAYER_ID); + record( + 'GLOBE: zoomed-out sampling stays centred on Rome and exposes its regional radius', + world.enabled && + world.height > 15_000_000 && + world.coverage?.longitude === 12 && + world.coverage?.latitude === 42 && + world.coverage?.radiusNm === 250, + JSON.stringify(world), + ); + await shoot(page, 'globe-regional'); + await page.evaluate(() => { + const { viewer } = window.__godsEyeView; + viewer.camera.setView({ + destination: viewer.scene.globe.ellipsoid.cartographicToCartesian({ + longitude: (18.6 * Math.PI) / 180, + latitude: (54.6 * Math.PI) / 180, + height: 450_000, + }), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + viewer.scene.requestRender(); + }); + await page.waitForFunction( + () => { + const url = window.__gnssQaRequests.at(-1); + return ( + url && + Math.abs( + Number(new URL(url, location.origin).searchParams.get('lon')) - + 18.6, + ) < 1 + ); + }, + { timeout: 15000 }, + ); + await sleep(3500); + + // ── (iii) FAILURE ───────────────────────────────────────────────────── + console.log('\n(iii) FAILURE — /api/gnss-integrity stubbed to 502...'); + await page.evaluate(() => { + window.__gnssQaMode = 'error'; + }); + const e = await enableAndSettle(page, { refresh: true }); + record( + 'FAILURE: error surfaced, cells kept', + Boolean(e.stats.error) && e.stats.count === 4 && e.entities === 4, + `error=${JSON.stringify(e.stats.error)} cells=${e.stats.count}`, + ); + await shoot(page, 'failure'); + + // ── (iv) DISABLE ────────────────────────────────────────────────────── + console.log('\n(iv) DISABLE...'); + const shown = await page.evaluate(async (id) => { + await window.__godsEyeView.dataManager.setEnabled(id, false); + const all = window.__godsEyeView.viewer.scene.groundPrimitives; + return Array.from({ length: all.length }, (_, i) => all.get(i)).find( + (item) => item.id === id, + )?.show; + }, LAYER_ID); + record('DISABLE: data source hidden', shown === false, `show=${shown}`); + await shoot(page, 'disabled'); + + // ── (v) KEYBOARD ────────────────────────────────────────────────────── + console.log('\n(v) KEYBOARD — the row toggle, from the keyboard...'); + await page.evaluate(() => { + window.__gnssQaMode = 'fixture'; + }); + await showLayerRow(page); + const on = await pressToggle(page, 'Enter', true); + record( + 'KEYBOARD: Enter on the focused toggle enables the layer and keeps focus', + on.enabled && + on.focused && + on.count === 4 && + /^GNSS Integrity: /.test(on.label) && + !/OFF$/.test(on.label), + JSON.stringify(on), + ); + const off = await pressToggle(page, ' ', false); + record( + 'KEYBOARD: Space on the focused toggle disables it and the label says OFF', + !off.enabled && off.focused && /^GNSS Integrity: OFF$/.test(off.label), + JSON.stringify(off), + ); + + // ── (vi) VIEWPORT ───────────────────────────────────────────────────── + console.log( + '\n(vi) VIEWPORT — the open row on a 390x844 phone viewport...', + ); + await page.close(); + // A fresh page: switching an open page to isMobile reloads it, which + // would measure a half-booted app instead of the open row. + const phone = await browser.newPage(); + await phone.setViewport({ width: 390, height: 844, isMobile: true }); + phone.on('pageerror', (e) => errors.push(e.message)); + await stubRoute(phone); + await boot(phone, { lon: 18.6, lat: 54.6, height: 450_000 }); + await showLayerRow(phone); + await pressToggle(phone, 'Enter', true); + await showLayerRow(phone); + const fit = await phone.evaluate((id) => { + const row = document.querySelector(`[data-layer-id="${id}"]`); + const legend = [...row.querySelectorAll('.data-toggle-legend-item')]; + const rect = row.getBoundingClientRect(); + return { + width: window.innerWidth, + rowLeft: Math.round(rect.left), + rowRight: Math.round(rect.right), + overflow: row.scrollWidth - row.clientWidth, + legendItems: legend.length, + coverageInfo: row.querySelector('.data-toggle-controls-info') + ?.textContent, + legendInside: legend.every((item) => { + const r = item.getBoundingClientRect(); + return ( + r.width > 0 && r.left >= rect.left - 1 && r.right <= rect.right + 1 + ); + }), + }; + }, LAYER_ID); + record( + 'VIEWPORT: row and legend fit a 390 px viewport without horizontal overflow', + fit.rowLeft >= 0 && + fit.rowRight <= fit.width && + fit.overflow <= 0 && + fit.legendItems === 3 && + /Regional sample.*250 nm radius/.test(fit.coverageInfo) && + fit.legendInside, + JSON.stringify(fit), + ); + await phone.screenshot({ path: path.join(SHOTS_DIR, 'gnss-phone.png') }); + await phone.close(); + + console.log( + '\nSTARTUP — initial rate limit stays enabled and recovers automatically...', + ); + const retry = await browser.newPage(); + await retry.setViewport({ width: 1440, height: 900 }); + retry.on('pageerror', (e) => errors.push(e.message)); + // Pages share the disposable test context's storage. Start this one cold, + // so a previous fixture cannot auto-enable GNSS before the START test. + await retry.evaluateOnNewDocument(() => localStorage.clear()); + await stubRoute(retry, { initialFailures: 1, stale: false }); + await boot(retry, { lon: 18.6, lat: 54.6, height: 450_000 }); + await showLayerRow(retry); + await retry.evaluate((id) => { + window.__gnssQaFailedStarts = 0; + window.__godsEyeView.dataManager.subscribe((change) => { + if (change.layerId === id && change.type === 'visibility-failed') + window.__gnssQaFailedStarts++; + }); + }, LAYER_ID); + const initial = await enableAndSettle(retry); + const limited = await retry.evaluate((id) => { + const dm = window.__godsEyeView.dataManager; + const row = document.querySelector(`[data-layer-id="${id}"]`); + return { + enabled: dm.isEnabled(id), + failedStarts: window.__gnssQaFailedStarts, + label: row.querySelector('.data-toggle-btn').textContent, + info: row.querySelector('.data-toggle-controls-info').textContent, + }; + }, LAYER_ID); + record( + 'STARTUP: first 429 keeps the layer enabled and shows automatic retry timing', + limited.enabled && + limited.failedStarts === 0 && + limited.label === 'UNAVAILABLE' && + initial.stats.count === 0 && + initial.stats.retryInSec > 0 && + /rate limited.*Retrying automatically at/.test(limited.info), + JSON.stringify(limited), + ); + await retry.screenshot({ + path: path.join(SHOTS_DIR, 'gnss-startup-retrying.png'), + }); + await retry.waitForFunction( + (id) => { + const layer = window.__godsEyeView.dataManager.layers.get(id); + return ( + layer.enabled && + layer.module.getStats().error === null && + layer.module.getStats().count === 4 + ); + }, + { timeout: 20000 }, + LAYER_ID, + ); + const recovered = await retry.evaluate( + (id) => ({ + requests: window.__gnssQaRequests.length, + enabled: window.__godsEyeView.dataManager.isEnabled(id), + label: document.querySelector( + `[data-layer-id="${id}"] .data-toggle-btn`, + ).textContent, + info: document.querySelector( + `[data-layer-id="${id}"] .data-toggle-controls-info`, + ).textContent, + }), + LAYER_ID, + ); + record( + 'STARTUP: one automatic retry recovers to ON without toggling and clears the warning', + recovered.enabled && + recovered.requests === 2 && + recovered.label === 'ON' && + !/Retrying/.test(recovered.info), + JSON.stringify(recovered), + ); + await shoot(retry, 'startup-recovered'); + await retry.close(); + + // ── (vii) AGE ───────────────────────────────────────────────────────── + console.log( + '\n(vii) AGE — a replayed stale snapshot keeps its observation time...', + ); + const aged = await browser.newPage(); + await aged.setViewport({ width: 1440, height: 900 }); + aged.on('pageerror', (e) => errors.push(e.message)); + await stubRoute(aged, { ageMs: 31 * 60_000 }); + await boot(aged, { lon: 18.6, lat: 54.6, height: 450_000 }); + const old = await enableAndSettle(aged, {}); + record( + 'AGE: a snapshot observed 31 min ago adds no cells and is not an error', + old.stats.count === 0 && old.entities === 0 && !old.stats.error, + `cells=${old.stats.count} entities=${old.entities} error=${JSON.stringify(old.stats.error)}`, + ); + await aged.evaluate(() => { + window.__gnssQaAgeMs = 29 * 60_000; + }); + const recent = await enableAndSettle(aged, { refresh: true }); + record( + 'AGE: the same snapshot observed 29 min ago is still inside the window', + recent.stats.count === 4 && recent.entities === 4, + `cells=${recent.stats.count} entities=${recent.entities}`, + ); + await aged.close(); + record( + 'no uncaught browser errors', + errors.length === 0, + errors.slice(0, 3).join(' | '), + ); + } catch (err) { + console.error('\x1b[31mHarness error:\x1b[0m', err); + exitCode = 3; + } finally { + await browser.close(); + } + + const pass = results.filter((r) => r.ok).length; + const fail = results.length - pass; + console.log('\n' + '─'.repeat(60)); + console.log(` RESULT: ${pass} passed, ${fail} failed`); + console.log(` Shots : ${SHOTS_DIR}/gnss-*.png`); + console.log('─'.repeat(60) + '\n'); + process.exit(exitCode || (fail > 0 ? 1 : 0)); +} + +main().catch((e) => { + console.error(e); + process.exit(3); +}); diff --git a/server/providers/gnssIntegrity.js b/server/providers/gnssIntegrity.js new file mode 100644 index 000000000..adf28cea5 --- /dev/null +++ b/server/providers/gnssIntegrity.js @@ -0,0 +1,238 @@ +import { + GNSS_CLASSIFIER, + GNSS_WINDOW_MS, + normalizeGnssAircraft, +} from '../../src/layers/gnss/records.js'; +import { adsbLolFallbackAnchor } from './aircraft/opensky.js'; +import { coalesceProxyRequest, readResponseJsonCapped } from './common/http.js'; +import { makeRateLimiter, clientKey } from './common/rate-limit.js'; + +// adsb.lol regional snapshot (keyless, ODbL). Only the integrity fields the +// GNSS layer needs leave this proxy, so the browser never sees the full feed. +// This is a separate read from the flights fallback on purpose: that point +// cache keeps only the fields the flights layer renders (no nic / nac_p), is +// keyed on 0.25° anchors and lives for 12 s, while integrity cells need +// whole-degree anchors held for a minute. +const RADIUS_NM = 250; +const CACHE_MS = 60_000; +const CACHE_MAX = 64; +/** Default 429 cooldown when upstream sends no usable Retry-After. */ +const COOLDOWN_MS = 60_000; +/** Bound sane upstream deadlines to one hour; never shorten a deadline within that range. */ +const COOLDOWN_MIN_MS = 5_000; +const COOLDOWN_MAX_MS = 60 * 60_000; +const FAILURE_COOLDOWN_MS = 15_000; +const MAX_IN_FLIGHT = 8; +const MAX_RESPONSE_BYTES = 8 * 1024 * 1024; +const TIMEOUT_MS = 10_000; + +/** Snap a view anchor to a whole degree so nearby views share one upstream read. */ +export function gnssIntegrityAnchor(url) { + const anchor = adsbLolFallbackAnchor({ url }); + if (!anchor) return null; + return { + lat: Math.round(anchor.latitude), + lon: Math.round(anchor.longitude), + }; +} + +/** Cooldown (ms) an upstream 429 earns, honouring Retry-After when sane. */ +export function gnssRetryCooldownMs(raw, nowMs) { + if (raw) { + const seconds = /^\d+(?:\.\d+)?$/.test(raw.trim()) ? Number(raw) : NaN; + const ms = seconds * 1000; + if (Number.isFinite(seconds) && seconds > 0) + return Math.min(COOLDOWN_MAX_MS, Math.max(COOLDOWN_MIN_MS, ms)); + const at = + Number.isNaN(seconds) && /[a-z]/i.test(raw) ? Date.parse(raw) : NaN; + if (Number.isFinite(at) && at > nowMs) + return Math.min(COOLDOWN_MAX_MS, Math.max(COOLDOWN_MIN_MS, at - nowMs)); + } + return COOLDOWN_MS; +} + +/** Fixed-origin, bounded adsb.lol integrity route for dev and preview. */ +export function gnssIntegrityProxy({ + fetchImpl = (...args) => globalThis.fetch(...args), + now = () => Date.now(), + monotonicNow = () => performance.now(), + warn = (...args) => console.warn(...args), +} = {}) { + const cache = new Map(); + const inFlight = new Map(); + const allow = makeRateLimiter({ windowMs: 60_000, max: 30, globalMax: 600 }); + let cooldownUntil = 0; + let cooldownStatus = 429; + + async function load({ lat, lon }) { + if (monotonicNow() < cooldownUntil) + throw Object.assign(new Error('cooling_down'), { + status: cooldownStatus, + }); + const signal = AbortSignal.timeout(TIMEOUT_MS); + try { + const response = await fetchImpl( + `https://api.adsb.lol/v2/lat/${lat}/lon/${lon}/dist/${RADIUS_NM}`, + { + signal, + redirect: 'error', + headers: { + Accept: 'application/json', + 'User-Agent': 'gods-eye-view-gnss-integrity/1.0', + }, + }, + ); + if (!response.ok) { + await response.body?.cancel(); + if (response.status === 429) { + const at = monotonicNow(); + const raw = response.headers.get('retry-after'); + const delay = gnssRetryCooldownMs(raw, now()); + if (delay === COOLDOWN_MAX_MS) + warn('[gnss-integrity] upstream cooldown capped at one hour'); + cooldownStatus = 429; + cooldownUntil = Math.max(cooldownUntil, at + delay); + throw Object.assign(new Error('upstream_rate_limited'), { + status: 429, + }); + } + throw new Error('upstream_unavailable'); + } + const payload = await readResponseJsonCapped( + response, + MAX_RESPONSE_BYTES, + signal, + ); + const rows = normalizeGnssAircraft(payload); + if (!rows) throw new Error('invalid_snapshot'); + const sourceNow = payload.now; + if (!Number.isFinite(sourceNow) || sourceNow <= 0) + throw new Error('undated_snapshot'); + const fetchedAt = Math.min( + now(), + sourceNow > 10_000_000_000 ? sourceNow : sourceNow * 1000, + ); + if (now() - fetchedAt > GNSS_WINDOW_MS) + throw new Error('expired_snapshot'); + return { + fetchedAt, + anchor: { lat, lon }, + radiusNm: RADIUS_NM, + // `degraded` on each row is this classifier's verdict, not gpsjam's. + classifier: GNSS_CLASSIFIER.id, + rows, + }; + } catch (error) { + if (error.status !== 429) { + const until = monotonicNow() + FAILURE_COOLDOWN_MS; + if (until > cooldownUntil) { + cooldownUntil = until; + cooldownStatus = 502; + } + } + throw error; + } + } + + function age(entry) { + if (!entry) return Infinity; + const elapsed = monotonicNow() - entry.savedAt; + // A backwards injected clock cannot rejuvenate a snapshot. + return elapsed < 0 ? Infinity : entry.ageAtReceipt + elapsed; + } + + async function acquire(anchor, peer) { + const key = `${anchor.lat},${anchor.lon}`; + const previous = cache.get(key); + if (age(previous) < CACHE_MS) + return { value: previous.value, ageMs: age(previous), stale: false }; + try { + if (!inFlight.has(key) && inFlight.size >= MAX_IN_FLIGHT) + throw Object.assign(new Error('busy'), { status: 429, local: true }); + const { promise } = coalesceProxyRequest(inFlight, key, async () => { + if (monotonicNow() >= cooldownUntil && !allow(peer)) + throw Object.assign(new Error('rate_limited'), { + status: 429, + local: true, + }); + const value = await load(anchor); + const entry = { + value, + savedAt: monotonicNow(), + ageAtReceipt: Math.max(0, now() - value.fetchedAt), + }; + cache.delete(key); + cache.set(key, entry); + while (cache.size > CACHE_MAX) cache.delete(cache.keys().next().value); + return entry; + }); + const entry = await promise; + const ageMs = age(entry); + return { value: entry.value, ageMs, stale: ageMs >= CACHE_MS }; + } catch (error) { + if (age(previous) <= GNSS_WINDOW_MS) + return { value: previous.value, ageMs: age(previous), stale: true }; + throw error; + } + } + + async function handler(req, res) { + const json = (status, value, stale = false, retryAfterS = 60) => { + if (res.destroyed) return; + res.writeHead(status, { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + ...(status === 405 ? { Allow: 'GET' } : {}), + ...([429, 502].includes(status) + ? { 'Retry-After': String(retryAfterS) } + : {}), + ...(stale ? { 'X-Data-Stale': 'true' } : {}), + }); + res.end(JSON.stringify(value)); + }; + if (req.method !== 'GET') return json(405, { error: 'method_not_allowed' }); + const path = (req.url || '/').split('?')[0]; + if (path !== '/' && path !== '') + return json(404, { error: 'unknown_route' }); + const anchor = gnssIntegrityAnchor(req.url); + if (!anchor) return json(400, { error: 'invalid_anchor' }); + try { + const { value, stale, ageMs } = await acquire(anchor, clientKey(req)); + // ageMs is measured on this server's clock, so the browser can date + // the rows without comparing its clock to ours. + json(200, { ...value, ageMs, ...(stale ? { stale: true } : {}) }, stale); + } catch (error) { + if (error.status === 429) { + // Tell the browser how long the shared upstream cooldown has left. + const retryAfterS = Math.max( + 1, + error.local ? 60 : Math.ceil((cooldownUntil - monotonicNow()) / 1000), + ); + return json( + 429, + { + error: error.local ? error.message : 'gnss_integrity_rate_limited', + }, + false, + retryAfterS, + ); + } + json( + 502, + { error: 'gnss_integrity_unavailable' }, + false, + Math.max(1, Math.ceil((cooldownUntil - monotonicNow()) / 1000)), + ); + } + } + + return { + name: 'gnss-integrity', + configureServer({ middlewares }) { + middlewares.use('/api/gnss-integrity', handler); + }, + configurePreviewServer({ middlewares }) { + middlewares.use('/api/gnss-integrity', handler); + }, + }; +} diff --git a/server/providers/local.js b/server/providers/local.js index 9acbade31..9fb6ae9c0 100644 --- a/server/providers/local.js +++ b/server/providers/local.js @@ -24,6 +24,7 @@ import { mapillaryProxy } from './mapillary.js'; import { keySetupEndpoint } from '../standalone/key-setup.js'; import { weatherProxy } from './weather.js'; import { firePerimetersProxy } from './firePerimeters.js'; +import { gnssIntegrityProxy } from './gnssIntegrity.js'; import { cycloneProxy } from './cyclones.js'; import { windProxy } from './wind.js'; @@ -60,6 +61,7 @@ function localProviderPlugins({ realtime } = {}) { cycloneProxy(), mapillaryProxy(), firePerimetersProxy(), + gnssIntegrityProxy(), keySetupEndpoint(), ]; } diff --git a/server/providers/openai/toolDescriptions.js b/server/providers/openai/toolDescriptions.js index 450c9da81..18de877db 100644 --- a/server/providers/openai/toolDescriptions.js +++ b/server/providers/openai/toolDescriptions.js @@ -179,7 +179,7 @@ export const ACTION_DESCRIPTIONS = { }, get_current_view_state: { description: - 'Read the current camera, style, Context, Cockpit, HUD, detection, map stack, post-processing, scene-playback, tracked-entity, and layer state before choosing another action.', + 'Read the current camera, style, Context, Cockpit, HUD, detection, map stack, post-processing, scene-playback, tracked-entity, and layer state before choosing another action. Tracked aircraft emergency status is a crowd-sourced transponder broadcast, not a confirmed incident: quote emergencyBroadcast and never infer a hijack or an incident ending from a code change.', $position: 1, }, set_hud: { @@ -543,7 +543,7 @@ export const ACTION_DESCRIPTIONS = { }, analyst_query: { description: - 'Counts, lists, superlatives and attribute filters over records loaded by ENABLED layers — "how many flights over Texas", "biggest fire near LA", "ships headed to Oakland", "next rocket launch". A requested list or ranking requires this tool even after set_context_mode returned a Contacts count; that count alone has no requested items. complete:false means the count is a floor ("at least") and rankings cover the records examined; partial names unanswered layers. Items carry lat/lon for a follow-up fly_to_location. ok:false codes: LAYER_OFF (offer to enable), NOT_READY (loading), FOLLOW_UP_MISMATCH (ask again without followUp), UNKNOWN_FIELD/UNKNOWN_SCOPE/BAD_VALUE (retry with the allowed values).', + 'Counts, lists, superlatives and attribute filters over records loaded by ENABLED layers — "how many flights over Texas", "biggest fire near LA", "ships headed to Oakland", "next rocket launch". A requested list or ranking requires this tool even after set_context_mode returned a Contacts count; that count alone has no requested items. complete:false means the count is a floor ("at least") and rankings cover the records examined; partial names unanswered layers. Items carry lat/lon for a follow-up fly_to_location. ok:false codes: LAYER_OFF (offer to enable), NOT_READY (loading), FOLLOW_UP_MISMATCH (ask again without followUp), UNKNOWN_FIELD/UNKNOWN_SCOPE/BAD_VALUE (retry with the allowed values). Aircraft emergency fields describe broadcasts, not confirmed incidents: narrate emergencyBroadcast with emergencySource. emergencyKind values are general, unlawful, nordo, downed, minfuel and lifeguard; GEV treats the last two as priorities (emergency:false). lastContactAtMs says when the transponder was last heard, not when the status arrived.', $position: 1, parameters: { properties: { diff --git a/src/app/constructCatalog.js b/src/app/constructCatalog.js index 80315fe85..184b51f17 100644 --- a/src/app/constructCatalog.js +++ b/src/app/constructCatalog.js @@ -25,6 +25,7 @@ import { createApplicationAwareness } from './layers/militaryAwareness.js'; import { createApplicationFirms } from './layers/firms.js'; import { createApplicationEarthquakes } from './layers/earthquakes.js'; import { createApplicationFirePerimeters } from './layers/perimeters.js'; +import { createApplicationGnssIntegrity } from './layers/gnss.js'; import { createApplicationCables } from './layers/submarineCables.js'; import { createApplicationStreetLevel } from './layers/streetLevel.js'; import { createInfrastructureLayers } from '../data/infrastructure.js'; @@ -120,6 +121,9 @@ export function createApplicationCatalog({ createApplicationFirePerimeters({ source: sources['fire-perimeters'], }), + createApplicationGnssIntegrity({ + source: sources['gnss-integrity'], + }), createApplicationAlpr({ surface, source: sources.alpr }), createApplicationStreetLevel({ surface, diff --git a/src/app/constructCatalog.test.mjs b/src/app/constructCatalog.test.mjs index f42e82c7c..f8ebb1e73 100644 --- a/src/app/constructCatalog.test.mjs +++ b/src/app/constructCatalog.test.mjs @@ -40,7 +40,7 @@ test('catalogs construct distinct layers and classification from their supplied signal: b.signal, surface: fixtureSurface(b.signal), }); - assert.equal(first.layers.length, 30); + assert.equal(first.layers.length, 31); assert.ok(first.get('local-adsb'), 'Local ADS-B is registered'); assert.deepEqual( first.metadata.find(({ id }) => id === 'local-adsb'), @@ -60,6 +60,7 @@ test('catalogs construct distinct layers and classification from their supplied '2026-09-21T12:00:00.000Z', ); assert.ok(first.get('fire-perimeters')); + assert.ok(first.get('gnss-integrity')); assert.ok(first.get('transit')); const order = first.layers.map(({ id }) => id); assert.deepEqual( @@ -153,7 +154,7 @@ test('Mapillary can be omitted or replaced without changing catalog membership', surface: fixtureSurface(lifetime.signal), }; const catalog = createApplicationCatalog(options); - assert.equal(catalog.layers.length, 30); + assert.equal(catalog.layers.length, 31); assert.deepEqual(catalog.get('street-level').providerIds, []); assert.equal( catalogControlServices(catalog).streetLevelLayer, diff --git a/src/app/layers/gnss.js b/src/app/layers/gnss.js new file mode 100644 index 000000000..46983953b --- /dev/null +++ b/src/app/layers/gnss.js @@ -0,0 +1,6 @@ +import { createGnssIntegrityLayer } from '../../layers/gnss/index.js'; + +/** Wire the adsb.lol GNSS navigation-integrity layer into the application catalog. */ +export function createApplicationGnssIntegrity(options) { + return createGnssIntegrityLayer(options); +} diff --git a/src/app/sourceComposition.js b/src/app/sourceComposition.js index 14c864ac5..0b516f5b4 100644 --- a/src/app/sourceComposition.js +++ b/src/app/sourceComposition.js @@ -46,6 +46,7 @@ export const CATALOG_SOURCE_CONTRACTS = Object.freeze({ cyclones: contract(['weather-cyclones'], ['getSnapshot']), earthquakes: contract(['earthquakes'], ['getSnapshot']), 'fire-perimeters': contract(['fire-perimeters'], ['getSnapshot']), + 'gnss-integrity': contract(['gnss-integrity'], ['getSnapshot']), cables: contract(['telegeography-submarine-cables'], ['fetch']), transit: contract(['transit'], ['requestSnapshot', 'getHistory']), }); diff --git a/src/app/sourceComposition.test.mjs b/src/app/sourceComposition.test.mjs index 44a7c8fa0..cf8646e25 100644 --- a/src/app/sourceComposition.test.mjs +++ b/src/app/sourceComposition.test.mjs @@ -37,7 +37,7 @@ test('every acquisition source can be omitted without losing catalog membership' test('all acquisition sources may be absent, and enable fails before initializing or fetching', async (t) => { const actual = catalog({}, t); - assert.equal(actual.layers.length, 30); + assert.equal(actual.layers.length, 31); const manager = new LayerLifecycle( {}, { getSourceAvailability: actual.getSourceAvailability }, diff --git a/src/app/tools.js b/src/app/tools.js index 25d763e2d..7b3ea83c9 100644 --- a/src/app/tools.js +++ b/src/app/tools.js @@ -1,6 +1,7 @@ import { SceneDirector } from '../scenes/director.js'; import { initAnnotations } from '../annotations/index.js'; import { initDrawTool } from '../annotations/drawTool.js'; +import { initGeoFileImport } from '../ui/geoFileImport.js'; import { initImageryBoxTool } from '../ui/imageryBoxTool.js'; import { createRecentImageryPanel } from '../ui/recentImagery.js'; import { initGevVoiceCommands } from '../voice/gevRealtime.js'; @@ -56,6 +57,16 @@ export function createApplicationTools({ // lifetime rather than to whoever last pressed the button. const drawTool = initDrawTool({ viewer, annotations }); defer(() => drawTool?.destroy()); + // DISPLAY ▸ Import: the person's own GeoJSON/KML/KMZ/GPX files, each its + // own data source. They live for the page session, so the application + // lifetime removes them and the drop listeners on the viewer container. + const geoFileImport = initGeoFileImport({ + viewer, + navigation: styleManager, + ground: operations.surface?.groundFloor, + subscribeMapSource: (listener) => mapStackController?.subscribe?.(listener), + }); + defer(() => geoFileImport?.destroy()); // DATA ▸ Recent Imagery: the box tool claims the pointer like Draw and the // panel lives on the right rail, so both belong to the application // lifetime. The tileset lets the layer drape while the globe is hidden. diff --git a/src/cockpitMath.js b/src/cockpitMath.js index a0bd83676..74e6d1451 100644 --- a/src/cockpitMath.js +++ b/src/cockpitMath.js @@ -83,6 +83,26 @@ export function cockpitAltitudeDisplayFt(altitudeM, onGround) { return Number.isFinite(altitudeM) ? altitudeM * 3.28084 : null; } +/** + * HUD wording for a tracked aircraft's transponder emergency, or null. + * States what was broadcast, never a conclusion about the flight: the Mode A + * code when it names the status, else the ADS-B field it came from (an + * unrelated squawk is dropped). + * @param {{label?: string, squawk?: string|null, source?: string, + * severity?: string}|null|undefined} emergency + * @returns {string|null} + */ +export function formatCockpitEmergency(emergency) { + const label = String(emergency?.label || '') + .trim() + .toUpperCase(); + if (!label) return null; + if (emergency.source === 'squawk' && emergency.squawk) + return `SQUAWK ${emergency.squawk} · ${label}`; + const field = emergency.severity === 'priority' ? 'PRIORITY' : 'EMERGENCY'; + return `ADS-B ${field} · ${label}`; +} + /** Format the cockpit Context scope without overstating installation coverage. */ export function formatCockpitContextScope( subjectLabel, diff --git a/src/cockpitMath.test.mjs b/src/cockpitMath.test.mjs index ef1e8382b..2189ca35e 100644 --- a/src/cockpitMath.test.mjs +++ b/src/cockpitMath.test.mjs @@ -13,6 +13,7 @@ import { compassDivisions, formatAltitudeRulerTick, formatCockpitContextScope, + formatCockpitEmergency, formatCompassDivision, formatSpeedRulerTick, normalizeHeading, @@ -364,3 +365,21 @@ test('tracked-identity resolution falls back to layer precedence', () => { assert.equal(resolveTrackedAircraftInfo({ trackedId: 'flights:aaa077' }), null); assert.equal(resolveTrackedAircraftInfo(), null); }); + +test('cockpit emergency wording states the broadcast code, not a conclusion', () => { + assert.equal( + formatCockpitEmergency({ label: 'General emergency', squawk: '7700', source: 'squawk', severity: 'emergency' }), + 'SQUAWK 7700 · GENERAL EMERGENCY', + ); + assert.equal( + formatCockpitEmergency({ label: 'Downed aircraft', squawk: '1200', source: 'ads-b', severity: 'emergency' }), + 'ADS-B EMERGENCY · DOWNED AIRCRAFT', + 'an unrelated squawk is not quoted beside an ADS-B status', + ); + assert.equal( + formatCockpitEmergency({ label: 'Minimum fuel', squawk: null, source: 'ads-b', severity: 'priority' }), + 'ADS-B PRIORITY · MINIMUM FUEL', + ); + assert.equal(formatCockpitEmergency(null), null); + assert.equal(formatCockpitEmergency({ label: '' }), null); +}); diff --git a/src/data/adsbLolFallback.js b/src/data/adsbLolFallback.js index 7cd887dc6..40dd2bbc0 100644 --- a/src/data/adsbLolFallback.js +++ b/src/data/adsbLolFallback.js @@ -43,11 +43,18 @@ export function normalizeAdsbLolAircraftState(aircraft, nowSeconds) { .toLowerCase(); const latitude = finiteNumber(aircraft?.lat); const longitude = finiteNumber(aircraft?.lon); - if (!hex || latitude === null || longitude === null) return null; + if ( + !hex || + latitude === null || + longitude === null || + Math.abs(latitude) > 90 || + Math.abs(longitude) > 180 + ) + return null; const seenPosition = Math.max( 0, - finiteNumber(aircraft?.seen_pos) ?? finiteNumber(aircraft?.seen) ?? 0, + finiteNumber(aircraft?.seen_pos) ?? finiteNumber(aircraft?.seen) ?? NaN, ); const seen = Math.max(0, finiteNumber(aircraft?.seen) ?? seenPosition); const onGround = aircraft?.alt_baro === 'ground'; @@ -62,8 +69,10 @@ export function normalizeAdsbLolAircraftState(aircraft, nowSeconds) { hex, String(aircraft?.flight || aircraft?.r || '').trim() || null, null, - Math.max(0, nowSeconds - seenPosition), - Math.max(0, nowSeconds - seen), + Number.isFinite(seenPosition) + ? Math.max(0, nowSeconds - seenPosition) + : null, + Number.isFinite(seen) ? Math.max(0, nowSeconds - seen) : null, longitude, latitude, barometricFeet === null ? null : barometricFeet * FOOT_TO_M, @@ -94,8 +103,28 @@ export function normalizeAdsbLolPointResponse(payload) { : Math.floor( responseNow > 10_000_000_000 ? responseNow / 1000 : responseNow, ); - const states = (Array.isArray(payload?.ac) ? payload.ac : []) - .map((aircraft) => normalizeAdsbLolAircraftState(aircraft, nowSeconds)) - .filter(Boolean); - return { time: nowSeconds, states }; + const aircraft = Array.isArray(payload?.ac) ? payload.ac : []; + const states = []; + // Keep the OpenSky vectors unchanged. The regional fallback's extra readsb + // status travels alongside them, keyed only to admitted aircraft. + const admitted = new Set(); + const statuses = []; + for (const row of aircraft) { + const state = normalizeAdsbLolAircraftState(row, nowSeconds); + if (!state) continue; + states.push(state); + // The live contract admits the first valid vector for each ICAO. Its + // status must come from that same row, never an invalid or later duplicate. + const id = state[0]; + if (admitted.has(id)) continue; + admitted.add(id); + if (typeof row.emergency === 'string') + statuses.push([id, row.emergency.trim().toLowerCase()]); + } + const emergencyByIcao = Object.fromEntries(statuses); + return { + time: nowSeconds, + states, + ...(Object.keys(emergencyByIcao).length ? { emergencyByIcao } : {}), + }; } diff --git a/src/data/aircraftEmergency.js b/src/data/aircraftEmergency.js new file mode 100644 index 000000000..78bd4308f --- /dev/null +++ b/src/data/aircraftEmergency.js @@ -0,0 +1,93 @@ +/** + * Transponder emergency status for one aircraft, from the Mode A squawk and + * the ADS-B emergency/priority field. + * + * Emergency squawks are ICAO-standard worldwide: 7500 unlawful interference, + * 7600 radio failure, 7700 general emergency. readsb (adsb.lol) also reports + * `emergency`, "a superset of the 7x00 squawks": none, general, lifeguard, + * minfuel, nordo, unlawful, downed, reserved. OpenSky state vectors carry the + * squawk only. + * + * The status is what the transponder broadcast in THIS poll, never a finding: + * crews pass through 7x00 codes while dialling. Highlighting follows the latest + * broadcast, which does not confirm an incident or its resolution. + */ + +/** @constant {string} Tint for aircraft broadcasting an emergency (not a priority). */ +export const EMERGENCY_TINT_CSS = '#FF4444'; + +const SQUAWK_KINDS = { 7500: 'unlawful', 7600: 'nordo', 7700: 'general' }; + +const KINDS = { + general: { label: 'General emergency', severity: 'emergency' }, + unlawful: { label: 'Unlawful interference', severity: 'emergency' }, + nordo: { label: 'Radio failure', severity: 'emergency' }, + downed: { label: 'Downed aircraft', severity: 'emergency' }, + minfuel: { label: 'Minimum fuel', severity: 'priority' }, + lifeguard: { label: 'Lifeguard (medical priority)', severity: 'priority' }, +}; + +/** + * A Mode A code as its four octal digits, else null. + * @param {unknown} value + * @returns {string|null} + */ +export function normalizeSquawk(value) { + const code = String(value ?? '').trim(); + return /^[0-7]{4}$/.test(code) ? code : null; +} + +/** + * Emergency status from one observation. Higher severity wins; an ADS-B + * emergency wins a tie. A + * 7x00 squawk counts even when the ADS-B field says `none`. + * @param {{squawk?: unknown, emergency?: unknown}} [observation] + * @returns {{kind: string, label: string, severity: 'emergency'|'priority', + * squawk: string|null, source: 'ads-b'|'squawk'}|null} `source` is + * 'squawk' when the Mode A code itself names the status (so it can be + * quoted), else 'ads-b'. + */ +export function aircraftEmergency({ squawk, emergency } = {}) { + const code = normalizeSquawk(squawk); + const reported = String(emergency ?? '') + .trim() + .toLowerCase(); + const fromAdsb = Object.hasOwn(KINDS, reported); + const squawkKind = SQUAWK_KINDS[code] ?? null; + const kind = + fromAdsb && (KINDS[reported].severity === 'emergency' || !squawkKind) + ? reported + : squawkKind; + if (!kind) return null; + return { + kind, + ...KINDS[kind], + squawk: code, + source: SQUAWK_KINDS[code] === kind ? 'squawk' : 'ads-b', + }; +} + +/** + * How a summary may phrase the status without flattening it into an incident: + * quote the code when the Mode A squawk itself names the status, otherwise + * attribute it to the ADS-B emergency field. + * @param {{kind?: string, squawk?: string|null, source?: string}|null|undefined} status + * @returns {string|null} e.g. 'broadcasting squawk 7700 (general emergency)' + * or 'ADS-B reports general emergency'; null when there is no status. + */ +export function emergencyBroadcastPhrase(status) { + if (!status || !Object.hasOwn(KINDS, status.kind)) return null; + const label = KINDS[status.kind].label.toLowerCase(); + return status.source === 'squawk' && status.squawk + ? `broadcasting squawk ${status.squawk} (${label})` + : `ADS-B reports ${label}`; +} + +/** + * True when the aircraft is broadcasting an emergency (not a priority status). + * @param {{severity?: string}|null|undefined} status + * @returns {boolean} + */ +export function isEmergency(status) { + return status?.severity === 'emergency'; +} diff --git a/src/data/aircraftEmergency.test.mjs b/src/data/aircraftEmergency.test.mjs new file mode 100644 index 000000000..6646aa6bd --- /dev/null +++ b/src/data/aircraftEmergency.test.mjs @@ -0,0 +1,95 @@ +// src/data/aircraftEmergency.test.mjs +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { + aircraftEmergency, + emergencyBroadcastPhrase, + isEmergency, + normalizeSquawk, +} from './aircraftEmergency.js'; + +test('the three ICAO emergency squawks map to their kinds', () => { + assert.equal(aircraftEmergency({ squawk: '7500' }).kind, 'unlawful'); + assert.equal(aircraftEmergency({ squawk: '7600' }).kind, 'nordo'); + const general = aircraftEmergency({ squawk: ' 7700 ' }); + assert.deepEqual(general, { + kind: 'general', + label: 'General emergency', + severity: 'emergency', + squawk: '7700', + source: 'squawk', + }); +}); + +test('ordinary codes, blanks and non-octal values are not emergencies', () => { + for (const squawk of ['1200', '7000', '2000', '', null, undefined, '7800']) + assert.equal(aircraftEmergency({ squawk }), null, String(squawk)); + assert.equal(aircraftEmergency(), null); + assert.equal(normalizeSquawk('7800'), null); + assert.equal(normalizeSquawk(7700), '7700'); + assert.equal(normalizeSquawk('770'), null); +}); + +test('the ADS-B emergency field wins, and `none` defers to the squawk', () => { + const minfuel = aircraftEmergency({ squawk: '4521', emergency: 'MinFuel' }); + assert.equal(minfuel.kind, 'minfuel'); + assert.equal(minfuel.severity, 'priority'); + assert.equal(minfuel.source, 'ads-b'); + assert.equal(minfuel.squawk, '4521'); + assert.equal( + aircraftEmergency({ squawk: '7700', emergency: 'none' }).source, + 'squawk', + ); + assert.equal( + aircraftEmergency({ squawk: '7700', emergency: 'general' }).source, + 'squawk', + 'a code that names the status stays quotable', + ); + assert.equal( + aircraftEmergency({ squawk: '1200', emergency: 'general' }).source, + 'ads-b', + ); + assert.equal(aircraftEmergency({ emergency: 'downed' }).kind, 'downed'); + assert.equal(aircraftEmergency({ emergency: 'none' }), null); + assert.equal(aircraftEmergency({ emergency: 'reserved' }), null); +}); + +test('priority statuses are reported but are not emergencies', () => { + assert.equal(isEmergency(aircraftEmergency({ squawk: '7700' })), true); + assert.equal( + isEmergency(aircraftEmergency({ emergency: 'lifeguard' })), + false, + ); + assert.equal(isEmergency(aircraftEmergency({ emergency: 'minfuel' })), false); + assert.equal(isEmergency(null), false); +}); + +test('the broadcast phrase quotes a naming code, else attributes ADS-B', () => { + assert.equal( + emergencyBroadcastPhrase(aircraftEmergency({ squawk: '7700' })), + 'broadcasting squawk 7700 (general emergency)', + ); + assert.equal( + emergencyBroadcastPhrase( + aircraftEmergency({ squawk: '1200', emergency: 'general' }), + ), + 'ADS-B reports general emergency', + ); + assert.equal( + emergencyBroadcastPhrase(aircraftEmergency({ emergency: 'lifeguard' })), + 'ADS-B reports lifeguard (medical priority)', + ); + assert.equal(emergencyBroadcastPhrase(null), null); + assert.equal(emergencyBroadcastPhrase({ kind: 'reserved' }), null); +}); + +test('an emergency squawk outranks an ADS-B priority', () => { + assert.equal( + aircraftEmergency({ squawk: '7700', emergency: 'lifeguard' }).kind, + 'general', + ); + assert.equal( + aircraftEmergency({ squawk: '7500', emergency: 'minfuel' }).kind, + 'unlawful', + ); +}); diff --git a/src/data/aircraftEmergencyAnalyst.test.mjs b/src/data/aircraftEmergencyAnalyst.test.mjs new file mode 100644 index 000000000..27deb2b46 --- /dev/null +++ b/src/data/aircraftEmergencyAnalyst.test.mjs @@ -0,0 +1,524 @@ +import * as Cesium from 'cesium'; +import { TRACKED_ICON_COLOR } from '../layers/military/policy.js'; +import { createRendering as civilRendering } from '../layers/flights/rendering.js'; +import { createRendering as militaryRendering } from '../layers/military/rendering.js'; +import { createMilitarySnapshotRenderer } from '../layers/military/snapshotRenderer.js'; +import { aircraftEmergency } from './aircraftEmergency.js'; +// Source-to-voice provenance, status retention/clearing, contact-time contracts, +// and civil/military billboard, Cockpit and model tint regressions. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { openSkySnapshot, readsbSnapshot } from '../sources/live/index.js'; +import { FlightRecords } from '../layers/flights/records.js'; +import { MilitaryFlightRecords } from '../layers/military/records.js'; +import { mapAnalystRecord as mapFlight } from './flights.js'; +import { mapAnalystRecord as mapMilitary } from './militaryFlights.js'; +import { createAnalystEngine } from './analystEngine.js'; +import { normalizeAdsbLolPointResponse } from './adsbLolFallback.js'; +import { createGevActionRunner } from '../voice/gevActions.js'; + +const NOW_MS = 1800000000000; + +test('fallback status belongs to the first valid admitted aircraft row', () => { + const valid = { hex: 'a1b2c3', lat: 30, lon: -97, squawk: '1200' }; + for (const status of [undefined, 'general', 'none']) { + const payload = normalizeAdsbLolPointResponse({ + now: NOW_MS, + ac: [ + { ...valid, lat: 100, emergency: 'general' }, + { ...valid, emergency: status }, + { ...valid, lat: null, emergency: 'general' }, + { ...valid, emergency: status === 'general' ? 'none' : 'general' }, + ], + }); + const snapshot = openSkySnapshot(payload, { now: NOW_MS }); + assert.equal(snapshot.records.length, 1); + assert.equal(snapshot.records[0].emergency, status); + assert.equal(snapshot.records[0].latitude, 30); + } +}); + +function civilStore() { + return new FlightRecords({ + geoidHeight: () => 0, + cachedGroundFloor: () => null, + floorAltitudeM: (value) => value, + approxDistanceKm: () => 0, + }); +} +const receiveContext = { + viewerLatDeg: null, + viewerLonDeg: null, + trackedId: null, + floorWarmPoints: [], +}; + +test('real civil fallback preserves ADS-B emergency and priority, then clears', () => { + const store = civilStore(); + const aircraft = { + hex: 'a1b2c3', + flight: 'TEST1', + lat: 30, + lon: -97, + alt_baro: 10000, + gs: 150, + track: 90, + squawk: '1200', + seen: 3, + seen_pos: 1, + }; + for (const [status, expectedKind, flagged] of [ + ['general', 'general', true], + ['minfuel', 'minfuel', false], + ['none', null, false], + ]) { + const payload = normalizeAdsbLolPointResponse({ + now: NOW_MS, + ac: [{ ...aircraft, emergency: status }], + }); + assert.equal( + payload.states[0].length, + 18, + 'fallback vectors retain the OpenSky shape', + ); + const [observation] = openSkySnapshot(payload, { now: NOW_MS }).records; + const result = mapFlight( + aircraft.hex, + store.receive(observation, receiveContext).meta, + ); + assert.equal(result.emergency, flagged); + assert.equal(result.emergencyKind, expectedKind); + assert.equal(result.emergencySource, expectedKind ? 'ads-b' : null); + assert.equal(result.lastContactAtMs, NOW_MS - 3000); + } +}); + +test('changed status without contact time retains general contact, including voice projection', async () => { + const store = civilStore(); + const [observation] = readsbSnapshot( + { + ac: [ + { + hex: 'a1b2c3', + flight: 'TEST1', + lat: 30, + lon: -97, + alt_baro: 10000, + gs: 150, + track: 90, + squawk: '1200', + emergency: 'general', + seen: 3, + seen_pos: 1, + }, + ], + }, + { observedAtMs: NOW_MS, now: NOW_MS }, + ).records; + store.receive(observation, receiveContext); + const meta = store.receive( + { ...observation, contactTimeMs: null, squawk: '7600', emergency: 'none' }, + receiveContext, + ).meta; + const record = mapFlight(observation.id, meta); + assert.equal(record.emergencyKind, 'nordo'); + assert.equal(record.lastContactAtMs, NOW_MS - 3000); + assert.equal(Object.hasOwn(record, 'observedAtMs'), false); + globalThis.window = globalThis.window || { + clearTimeout, + setTimeout, + requestIdleCallback: null, + }; + const flights = { + id: 'flights', + source: 'fixture', + getStats: () => ({ count: 1 }), + getAnalystRecords: () => [record], + }; + const runner = createGevActionRunner({ + viewer: { + clock: { onTick: { addEventListener: () => () => {} } }, + scene: { canvas: { addEventListener() {}, removeEventListener() {} } }, + camera: { + moveEnd: { addEventListener() {} }, + positionCartographic: { + height: 300000, + latitude: 0.52, + longitude: -1.71, + }, + }, + }, + styleManager: { + getContextModeState: () => ({ mode: null, active: false }), + getCockpitState: () => ({ active: false }), + }, + dataManager: { + layers: new Map([['flights', { module: flights }]]), + isEnabled: (id) => id === 'flights', + getAll: () => [ + { id: 'flights', name: 'Flights', enabled: true, stats: { count: 1 } }, + ], + }, + }); + try { + const result = await runner('analyst_query', { + layers: ['flights'], + scope: { kind: 'anywhere' }, + filters: [{ field: 'emergency', op: 'eq', value: true }], + }); + assert.equal(result.ok, true); + assert.equal(result.items[0].emergency, true); + assert.equal(result.items[0].lastContactAtMs, NOW_MS - 3000); + assert.equal(result.items[0].emergencySource, 'squawk'); + assert.equal( + result.items[0].emergencyBroadcast, + 'broadcasting squawk 7600 (radio failure)', + ); + assert.equal( + Object.hasOwn(result.items[0], 'time'), + false, + 'general contact is not a generic incident timestamp', + ); + } finally { + runner.dispose?.(); + } +}); + +// OpenSky /states/all: [icao24, callsign, origin_country, time_position, +// last_contact, lon, lat, baro_alt, on_ground, velocity, true_track, +// vertical_rate, sensors, geo_alt, squawk, spi, position_source, category]. +const OPENSKY = { + time: NOW_MS / 1000, + states: [ + [ + 'a1b2c3', + 'SWA696 ', + 'United States', + NOW_MS / 1000 - 6, + NOW_MS / 1000 - 3, + -97.67, + 30.19, + 3000, + false, + 150, + 180, + -5, + null, + 3040, + '7700', + false, + 0, + 4, + ], + ], +}; + +// adsb.lol /v2/mil: readsb aircraft with `seen`/`seen_pos` ages in seconds. +const ADSB_LOL = { + ac: [ + { + hex: 'ae01ce', + flight: 'RCH123 ', + t: 'C17', + lat: 31.1, + lon: -97.4, + alt_baro: 18000, + gs: 300, + track: 90, + squawk: '1200', + emergency: 'general', + seen: 1.5, + seen_pos: 2, + }, + ], +}; + +function civilAnalystRecord() { + const [row] = openSkySnapshot(OPENSKY, { now: NOW_MS }).records; + const store = new FlightRecords({ + geoidHeight: () => 0, + cachedGroundFloor: () => null, + floorAltitudeM: (altitude) => altitude, + approxDistanceKm: () => 0, + }); + const { meta } = store.receive(row, { + viewerLatDeg: null, + viewerLonDeg: null, + trackedId: null, + floorWarmPoints: [], + }); + return mapFlight(row.id, meta); +} + +function militaryAnalystRecord() { + const [row] = readsbSnapshot(ADSB_LOL, { + observedAtMs: NOW_MS, + now: NOW_MS, + }).records; + const store = new MilitaryFlightRecords({ + geoidHeight: () => 0, + cachedGroundFloor: () => null, + floorAltitudeM: (altitude) => altitude, + }); + const { meta } = store.receive(row, { + observedAtMs: NOW_MS, + floorWarmPoints: [], + modelOwnsVisual: false, + }); + return mapMilitary(row.id, meta); +} + +test('OpenSky 7700 and adsb.lol emergency=general differ only in provenance', () => { + const civil = civilAnalystRecord(); + const military = militaryAnalystRecord(); + + assert.equal(civil.emergency, true); + assert.equal(civil.emergencyKind, 'general'); + assert.equal(civil.squawk, '7700'); + assert.equal(civil.emergencySource, 'squawk'); + assert.equal( + civil.emergencyBroadcast, + 'broadcasting squawk 7700 (general emergency)', + ); + assert.equal(civil.lastContactAtMs, NOW_MS - 3000, 'OpenSky last_contact'); + + assert.equal(military.emergency, true); + assert.equal(military.emergencyKind, 'general'); + assert.equal(military.squawk, '1200', 'the code is not an emergency code'); + assert.equal(military.emergencySource, 'ads-b'); + assert.equal(military.emergencyBroadcast, 'ADS-B reports general emergency'); + assert.equal(military.lastContactAtMs, NOW_MS - 1500, 'snapshot time - seen'); +}); + +test('the analyst can filter emergencies by source', async () => { + const records = { + flights: [civilAnalystRecord()], + military: [militaryAnalystRecord()], + }; + const engine = createAnalystEngine({ + getRecords: (key) => records[key] || [], + resolveRegionRing: async () => null, + getViewContext: () => ({ lat: 30.5, lon: -97.5, viewRadiusKm: 500 }), + }); + const query = (value) => + engine.query({ + layers: ['flights', 'military'], + scope: { kind: 'anywhere' }, + filters: [ + { field: 'emergency', op: 'eq', value: true }, + { field: 'emergencySource', op: 'eq', value }, + ], + }); + const bySquawk = await query('squawk'); + assert.deepEqual( + bySquawk.items.map((item) => [item.icao24, item.emergencyBroadcast]), + [['a1b2c3', 'broadcasting squawk 7700 (general emergency)']], + ); + const byAdsb = await query('ads-b'); + assert.deepEqual( + byAdsb.items.map((item) => [item.icao24, item.emergencyBroadcast]), + [['ae01ce', 'ADS-B reports general emergency']], + ); +}); + +test('military status changes retain only the latest known general contact', () => { + const store = new MilitaryFlightRecords({ + geoidHeight: () => 0, + cachedGroundFloor: () => null, + floorAltitudeM: (v) => v, + }); + const [observation] = readsbSnapshot(ADSB_LOL, { + observedAtMs: NOW_MS, + now: NOW_MS, + }).records; + const context = { + observedAtMs: NOW_MS, + floorWarmPoints: [], + modelOwnsVisual: false, + }; + const unknown = mapMilitary( + observation.id, + store.receive({ ...observation, contactTimeMs: null }, context).meta, + ); + assert.equal(unknown.lastContactAtMs, null); + store.receive(observation, context); + const changed = mapMilitary( + observation.id, + store.receive( + { + ...observation, + contactTimeMs: null, + squawk: '7600', + emergency: 'none', + }, + context, + ).meta, + ); + assert.equal(changed.emergencyKind, 'nordo'); + assert.equal(changed.lastContactAtMs, NOW_MS - 1500); +}); + +test('fallback missing contact age retains known contact through the real converter', () => { + const store = civilStore(); + const row = { + hex: 'a1b2c3', + lat: 30, + lon: -97, + squawk: '1200', + emergency: 'general', + seen: 3, + seen_pos: 1, + }; + const receive = (row) => + store.receive( + openSkySnapshot( + normalizeAdsbLolPointResponse({ now: NOW_MS, ac: [row] }), + { now: NOW_MS }, + ).records[0], + receiveContext, + ).meta; + const first = receive(row); + const second = receive({ ...row, seen: undefined, seen_pos: undefined }); + assert.equal(second.lastContactEpochMs, first.lastContactEpochMs); +}); + +test('one omitted ADS-B status poll is retained; explicit none and changed squawk clear immediately', () => { + const store = civilStore(); + const row = { + hex: 'a1b2c3', + lat: 30, + lon: -97, + squawk: '1200', + emergency: 'general', + seen: 3, + }; + const receive = (row) => + store.receive( + openSkySnapshot( + normalizeAdsbLolPointResponse({ now: NOW_MS, ac: [row] }), + { now: NOW_MS }, + ).records[0], + receiveContext, + ).meta; + assert.equal(receive(row).emergency.kind, 'general'); + assert.equal( + receive({ ...row, emergency: undefined }).emergency?.kind, + 'general', + ); + assert.equal(receive({ ...row, emergency: undefined }).emergency, null); + receive(row); + assert.equal(receive({ ...row, emergency: 'none' }).emergency, null); + receive(row); + assert.equal( + receive({ ...row, emergency: undefined, squawk: '2000' }).emergency, + null, + ); +}); + +function renderHarness(factory) { + const flightState = { records: { data: new Map() }, _trackedIcao: null }; + const rendering = factory({ + flightState, + services: { + militaryRegistry: { isMilitaryIcao: (id) => id === 'mil' }, + aircraftPresentation: {}, + camera: {}, + focus: {}, + recession: {}, + }, + parts: {}, + }); + return { flightState, rendering }; +} + +test('tint priorities cover normal, priority, emergency, tracking and Cockpit dots', () => { + for (const factory of [civilRendering, militaryRendering]) { + const { flightState, rendering } = renderHarness(factory); + const normal = rendering._fleetBillboardColor('civil', {}); + assert.equal( + rendering._fleetBillboardColor('civil', { + emergency: aircraftEmergency({ emergency: 'minfuel' }), + }), + normal, + ); + const red = rendering._fleetBillboardColor('civil', { + emergency: aircraftEmergency({ squawk: '7700' }), + }); + assert.ok(red.red > 0.9 && red.green < 0.5); + flightState.records.data.set('civil', { + emergency: aircraftEmergency({ squawk: '7700' }), + }); + assert.equal(rendering._modelColor('civil'), red); + if (rendering._cockpitContactColor) + assert.equal(rendering._cockpitContactColor('civil'), red); + flightState._trackedIcao = 'civil'; + assert.equal( + rendering._modelColor('civil'), + factory === civilRendering ? Cesium.Color.CYAN : TRACKED_ICON_COLOR, + ); + flightState._trackedIcao = null; + flightState.records.data.set('civil', { emergency: null }); + assert.equal(rendering._modelColor('civil'), normal); + } +}); + +test('GERMAN: an existing untracked military billboard repaints each poll without alpha loss', () => { + const { flightState, rendering } = renderHarness(militaryRendering); + const bb = { + color: rendering._fleetBillboardColor('abc123').withAlpha(0.65), + }; + Object.assign(flightState, { + _billboards: new Map([['abc123', bb]]), + _models: new Map(), + _positionHistory: new Map(), + _cullPositions: new Map(), + _groundSnap: { forget() {} }, + _cockpitContactMode: false, + }); + const records = new MilitaryFlightRecords({ + geoidHeight: () => 0, + cachedGroundFloor: () => null, + floorAltitudeM: (value) => value, + }); + flightState.records = records; + const apply = createMilitarySnapshotRenderer({ + flightState, + records, + groundFloor: { warmGroundFloor() {}, cachedGroundFloor: () => null }, + meshFloor: { sampleMeshFloorCells() {} }, + militaryRegistry: { registerMilitaryIcaos() {} }, + rendering: { + ...rendering, + _syncModelToClass() {}, + _applyFleetBillboardPresentation() {}, + }, + motion: {}, + tracking: {}, + queries: {}, + }); + for (const [emergency, red] of [ + ['general', true], + ['none', false], + ['general', true], + ]) { + const snapshot = readsbSnapshot( + { + ac: [ + { + hex: 'abc123', + lat: 30, + lon: -97, + alt_baro: 10000, + squawk: '1200', + emergency, + seen_pos: 1, + }, + ], + }, + { observedAtMs: NOW_MS }, + ); + apply(snapshot, {}); + assert.equal(bb.color.green < 0.5, red); + assert.equal(bb.color.alpha, 0.65); + } +}); diff --git a/src/data/aircraftRecession.js b/src/data/aircraftRecession.js index b260c5f1b..04c41e604 100644 --- a/src/data/aircraftRecession.js +++ b/src/data/aircraftRecession.js @@ -242,7 +242,7 @@ export function applyAircraftBillboardTreatment({ } /** - * Apply the already-composed aircraft alpha to an ambient glTF model without + * Apply layer-owned hue and composed aircraft alpha to an ambient glTF model without * disturbing Cesium's existing MIX/colorBlendAmount presentation semantics. * @param {object} input * @param {object} input.model @@ -261,7 +261,10 @@ export function applyAircraftModelTreatment({ if (!model || !baseColor || !Number.isFinite(alpha)) return 0; if ( Number.isFinite(model.color?.alpha) && - Math.abs(model.color.alpha - alpha) <= tuning.writeEpsilon + Math.abs(model.color.alpha - alpha) <= tuning.writeEpsilon && + model.color.red === baseColor.red && + model.color.green === baseColor.green && + model.color.blue === baseColor.blue ) return 0; model.color = baseColor.withAlpha(alpha); diff --git a/src/data/aircraftRecession.test.mjs b/src/data/aircraftRecession.test.mjs index 29bce05c5..23b8ceffb 100644 --- a/src/data/aircraftRecession.test.mjs +++ b/src/data/aircraftRecession.test.mjs @@ -142,3 +142,15 @@ test('ambient model presentation receives composed alpha without changing blend assert.equal(model.colorBlendAmount, 0.9); assert.equal(applyAircraftModelTreatment({ model, baseColor: color, alpha: 0.2, params }), 0); }); + + +test('model hue changes at constant alpha, then clears without redundant writes', () => { + const color = (red,green,blue) => ({red,green,blue,withAlpha(alpha){return {red,green,blue,alpha};}}); + const model = {color:color(1,1,1).withAlpha(0.65)}; + assert.equal(applyAircraftModelTreatment({model,baseColor:color(1,0.2,0.2),alpha:0.65}),1); + assert.equal(model.color.green,0.2); + assert.equal(model.color.alpha,0.65); + assert.equal(applyAircraftModelTreatment({model,baseColor:color(1,0.2,0.2),alpha:0.65}),0); + assert.equal(applyAircraftModelTreatment({model,baseColor:color(1,1,1),alpha:0.65}),1); + assert.equal(model.color.green,1); +}); diff --git a/src/data/cameraFraming.js b/src/data/cameraFraming.js new file mode 100644 index 000000000..a2d38b5ec --- /dev/null +++ b/src/data/cameraFraming.js @@ -0,0 +1,15 @@ +import * as Cesium from 'cesium'; + +/** Camera range with breathing room in both viewport dimensions. */ +export function rangeForBoundingSphere(viewer, radius) { + const frustum = viewer.camera?.frustum; + const verticalFov = Number(frustum?.fov) || Cesium.Math.toRadians(60); + const aspectRatio = Math.max(0.5, Number(frustum?.aspectRatio) || 1); + const horizontalFov = 2 * Math.atan(Math.tan(verticalFov / 2) * aspectRatio); + const limitingFov = Math.min(verticalFov, horizontalFov); + // Target 57% sphere occupancy so perspective still leaves at least 40% + // measured roof-to-base breathing room in ordinary oblique building views. + const occupiedViewportFraction = 0.57; + const desiredAngularRadius = (limitingFov * occupiedViewportFraction) / 2; + return (radius / Math.sin(desiredAngularRadius)) * 1.05; +} diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index e68caddb4..9f0b8b2d1 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -111,7 +111,7 @@ export const DATA_CREDITS = [ { key: 'adsblol', html: - 'Military flights, aircraft traces & bounded regional flight fallback: ' + + 'Military flights, aircraft traces, bounded regional flight fallback & GNSS navigation integrity: ' + 'adsb.lol ' + '(ODbL 1.0)', }, diff --git a/src/data/firms-coordinate-evidence.test.mjs b/src/data/firms-coordinate-evidence.test.mjs new file mode 100644 index 000000000..d60a7ee37 --- /dev/null +++ b/src/data/firms-coordinate-evidence.test.mjs @@ -0,0 +1,69 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { parseFirmsCsv } from './firmsCsv.js'; +import { adaptFirmsRecords } from './firmsAdapt.js'; +test('missing positions do not become fire detections at zero coordinates', () => { + const csv = + 'latitude,longitude,acq_date,acq_time,confidence,frp\n,1,2026-07-16,1200,n,1\n1, ,2026-07-16,1200,n,1\n95,1,2026-07-16,1200,n,1\n1,181,2026-07-16,1200,n,1\n0,0,2026-07-16,1200,n,1\n-90,-180,2026-07-16,1200,n,1'; + assert.deepEqual( + parseFirmsCsv(csv).map((r) => [r.lat, r.lon]), + [ + [0, 0], + [-90, -180], + ], + ); + const records = [ + { lat: null, lon: 1 }, + { lat: '', lon: 1 }, + { lat: false, lon: 1 }, + { lat: 95, lon: 1 }, + { lat: 1, lon: 181 }, + { lat: '0', lon: 0 }, + { lat: -90, lon: -180 }, + ]; + assert.deepEqual( + adaptFirmsRecords(records).map((r) => [r.index, r.lat, r.lon]), + [ + [0, 0, 0], + [1, -90, -180], + ], + ); +}); + +test('both axes keep their exact bounds and drop values just past them', () => { + const row = (lat, lon) => `${lat},${lon},2026-07-16,1200,n,1`; + const csv = [ + 'latitude,longitude,acq_date,acq_time,confidence,frp', + row(90, 180), + row(-90, -180), + row(90.000001, 0), + row(-90.000001, 0), + row(0, 180.000001), + row(0, -180.000001), + row(' ', 5), + row(5, '\t'), + ].join('\n'); + assert.deepEqual( + parseFirmsCsv(csv).map((r) => [r.lat, r.lon]), + [ + [90, 180], + [-90, -180], + ], + ); + const kept = adaptFirmsRecords([ + { lat: 90, lon: 180 }, + { lat: '-90', lon: '-180' }, + { lat: -90.000001, lon: 0 }, + { lat: 0, lon: -180.000001 }, + { lat: 5, lon: ' ' }, + { lat: 5, lon: undefined }, + { lat: [5], lon: 5 }, + ]); + assert.deepEqual( + kept.map((r) => [r.lat, r.lon]), + [ + [90, 180], + [-90, -180], + ], + ); +}); diff --git a/src/data/firmsAdapt.js b/src/data/firmsAdapt.js index 2fc909bce..312838e80 100644 --- a/src/data/firmsAdapt.js +++ b/src/data/firmsAdapt.js @@ -7,8 +7,8 @@ */ /** - * Map proxy fire records into internal fire records. Records with - * non-finite coordinates are skipped; `index` is the post-skip position + * Map proxy fire records into internal fire records. Records with missing, + * non-numeric or out-of-range (WGS84) coordinates are skipped; `index` is the post-skip position * (it keys pick ids and context-store ids, so it must stay sequential). * `contextEntity`/`position` start null and are lazily filled by the layer. * @param {?Array} records - /api/firms `fires` array. @@ -21,7 +21,15 @@ export function adaptFirmsRecords(records) { for (const record of records) { const lat = Number(record?.lat); const lon = Number(record?.lon); - if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue; + if ( + !hasCoordinateValue(record?.lat) || + !hasCoordinateValue(record?.lon) || + !Number.isFinite(lat) || + !Number.isFinite(lon) || + Math.abs(lat) > 90 || + Math.abs(lon) > 180 + ) + continue; fires.push({ index: fires.length, lat, @@ -110,3 +118,11 @@ function finiteNumber(value) { const number = Number(value); return Number.isFinite(number) ? number : 0; } + +// Same rule as parseFirmsCsv: a blank or non-numeric value is not a zero. +function hasCoordinateValue(value) { + return ( + typeof value === 'number' || + (typeof value === 'string' && value.trim() !== '') + ); +} diff --git a/src/data/firmsCsv.js b/src/data/firmsCsv.js index 7f9019e18..aafeac50b 100644 --- a/src/data/firmsCsv.js +++ b/src/data/firmsCsv.js @@ -105,7 +105,17 @@ export function parseFirmsCsv(text) { if (parts.length < header.length) continue; // malformed row — skip const lat = Number(parts[iLat]); const lon = Number(parts[iLon]); - if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue; + // Blank or out-of-range positions are dropped, never read as 0 (see + // hasCoordinateValue in firmsAdapt.js for the proxy-record side). + if ( + !parts[iLat].trim() || + !parts[iLon].trim() || + !Number.isFinite(lat) || + !Number.isFinite(lon) || + Math.abs(lat) > 90 || + Math.abs(lon) > 180 + ) + continue; records.push({ lat, diff --git a/src/data/flights.test.mjs b/src/data/flights.test.mjs index b09dc7014..c325ea744 100644 --- a/src/data/flights.test.mjs +++ b/src/data/flights.test.mjs @@ -79,6 +79,12 @@ test('flights analyst record: full record maps every contract field', () => { aircraftClass: 'airliner', originCountry: 'United States', operator: 'Southwest Airlines', + squawk: null, + emergency: false, + emergencyKind: null, + emergencySource: null, + emergencyBroadcast: null, + lastContactAtMs: null, routeOrigin: 'AUS', routeDestination: 'LAX', }); diff --git a/src/data/geoFileImport.js b/src/data/geoFileImport.js new file mode 100644 index 000000000..0ceaf9ef5 --- /dev/null +++ b/src/data/geoFileImport.js @@ -0,0 +1,1390 @@ +/** Session-local geodata loading with bounded geometry and local-only file references. */ +import * as Cesium from 'cesium'; +import { createImportImageBudget } from './geoImportImages.js'; +import { createImportedPins } from './geoImportPins.js'; + +/** Largest file accepted, in bytes. */ +export const MAX_IMPORT_BYTES = 50 * 1024 * 1024; +/** Largest total a KMZ may inflate to — a zip bomb stops here. */ +export const MAX_KMZ_INFLATED_BYTES = 120 * 1024 * 1024; +/** Most files on the globe at once. */ +export const MAX_IMPORTED_FILES = 12; +/** Most entities one file may create before it is refused. */ +export const MAX_IMPORT_ENTITIES = 20_000; +/** Most vertices (points of every line, ring and marker) in one file. */ +export const MAX_IMPORT_POSITIONS = 250_000; +/** Most normalized style definitions and local image links in one KML/KMZ. */ +export const MAX_KML_LINKS = 2_000; +/** Longest plain-text name kept from a file's features. */ +export const MAX_IMPORT_NAME_LENGTH = 120; +/** What the file picker offers. */ +export const GEO_FILE_ACCEPT = '.geojson,.json,.kml,.kmz,.gpx'; + +const GEOJSON_TYPES = new Set([ + 'FeatureCollection', + 'Feature', + 'Point', + 'MultiPoint', + 'LineString', + 'MultiLineString', + 'Polygon', + 'MultiPolygon', + 'GeometryCollection', +]); +const KMZ_IMAGE_TYPES = Object.freeze({ + png: 'image/png', + jpg: 'image/jpeg', + jpeg: 'image/jpeg', + gif: 'image/gif', + webp: 'image/webp', + bmp: 'image/bmp', +}); +// External content and metadata the session-local import does not use. +const KML_REMOVED_ELEMENTS = new Set([ + 'NetworkLink', + 'NetworkLinkControl', + 'ScreenOverlay', + 'PhotoOverlay', + 'Model', + 'Tour', + 'description', + 'BalloonStyle', + 'Snippet', + 'ExtendedData', + 'TimeSpan', + 'TimeStamp', + 'LookAt', + 'Camera', + 'visibility', + 'viewRefreshMode', + 'viewFormat', + 'httpQuery', +]); + +/** An error whose message is meant for the person who chose the file. */ +export class GeoImportError extends Error { + constructor(message) { + super(message); + this.name = 'GeoImportError'; + } +} + +/** + * The format of a file, from its name first and its first bytes second. + * @param {string} name File name. + * @param {Uint8Array} [head] The first bytes of the file. + * @returns {'geojson'|'kml'|'kmz'|'gpx'|null} + */ +export function detectGeoFileFormat(name, head) { + const ext = /\.([a-z0-9]+)$/i.exec(String(name || ''))?.[1]?.toLowerCase(); + if (ext === 'geojson') return 'geojson'; + if (ext === 'kml' || ext === 'kmz' || ext === 'gpx') return ext; + if (!head?.length) return ext === 'json' ? 'geojson' : null; + if ( + head[0] === 0x50 && + head[1] === 0x4b && + head[2] === 0x03 && + head[3] === 0x04 + ) + return 'kmz'; + const text = new TextDecoder() + .decode(head.subarray(0, 1024)) + .replace(/^/, '') + .trimStart(); + if (text.startsWith('{')) return 'geojson'; + if (text.startsWith('<')) { + if (/]/i.test(text)) return 'kml'; + if (/]/i.test(text)) return 'gpx'; + } + return ext === 'json' ? 'geojson' : null; +} + +/** + * A short, printable name for the list: the file name without control + * characters, cut to 60 characters. + * @param {string} name + * @returns {string} + */ +export function importDisplayName(name) { + const clean = String(name || '') + .replace(/[\u0000-\u001f\u007f]/g, '') + .trim(); + if (!clean) return 'Imported file'; + return clean.length > 60 ? `${clean.slice(0, 57)}…` : clean; +} + +/** + * Parse and check GeoJSON text. Throws `GeoImportError` for anything that is + * not a GeoJSON object. + * @param {string} text + * @returns {object} + */ +export function parseGeoJsonText(text) { + let value; + try { + value = JSON.parse(String(text).replace(/^/, '')); + } catch { + throw new GeoImportError('That file is not valid JSON.'); + } + if (!value || typeof value !== 'object' || !GEOJSON_TYPES.has(value.type)) + throw new GeoImportError('That JSON file is not GeoJSON.'); + if (value.type === 'FeatureCollection' && !Array.isArray(value.features)) + throw new GeoImportError('That FeatureCollection has no features list.'); + return value; +} + +/** + * Count features and vertices in a parsed GeoJSON object without recursion + * limits or array spreading, stopping as soon as a limit is passed. + * @param {object} geojson + * @param {{maxFeatures?: number, maxPositions?: number, maxEntities?: number}} [limits] + * @returns {{features: number, positions: number, entities: number, + * over: ''|'features'|'positions'|'entities'}} + */ +export function countGeoJsonGeometry( + geojson, + { + maxFeatures = MAX_IMPORT_ENTITIES, + maxPositions = MAX_IMPORT_POSITIONS, + maxEntities = MAX_IMPORT_ENTITIES, + } = {}, +) { + let features = ['Feature', 'FeatureCollection'].includes(geojson?.type) + ? 0 + : 1; + let positions = 0; + let entities = 0; + const result = (over = '') => ({ features, positions, entities, over }); + const stack = [geojson]; + while (stack.length) { + const node = stack.pop(); + if (!node || typeof node !== 'object') continue; + if (node.type === 'FeatureCollection') { + const list = Array.isArray(node.features) ? node.features : []; + for (let i = 0; i < list.length; i++) { + if (list[i]?.type !== 'Feature') + throw new GeoImportError( + 'Every GeoJSON feature must have type Feature.', + ); + stack.push(list[i]); + } + continue; + } + if (node.type === 'Feature') { + features += 1; + if (features > maxFeatures) return result('features'); + if (node.geometry == null) { + entities += 1; + if (entities > maxEntities) return result('entities'); + } + stack.push(node.geometry); + continue; + } + if (node.type === 'GeometryCollection') { + const list = node.geometries; + if (!Array.isArray(list)) + throw new GeoImportError('That GeoJSON has invalid geometries.'); + for (let i = 0; i < list.length; i++) { + if (!list[i] || ['Feature', 'FeatureCollection'].includes(list[i].type)) + throw new GeoImportError( + 'That GeoJSON contains an unsupported geometry.', + ); + stack.push(list[i]); + } + continue; + } + const depth = { + Point: 0, + MultiPoint: 1, + LineString: 1, + MultiLineString: 2, + Polygon: 2, + MultiPolygon: 3, + }[node.type]; + if (depth !== undefined) { + // Multipart geometries create one Cesium entity per component, even + // when the document wraps them in just one Feature. + entities += node.type.startsWith('Multi') + ? Array.isArray(node.coordinates) + ? node.coordinates.length + : 0 + : 1; + if (entities > maxEntities) return result('entities'); + // Walk the coordinate arrays iteratively: a position is an array whose + // first item is a number. + const coords = [[node.coordinates, depth]]; + while (coords.length) { + const [c, remaining] = coords.pop(); + if (!Array.isArray(c)) + throw new GeoImportError('That GeoJSON has invalid coordinates.'); + if (remaining === 0) { + if ( + c.length < 2 || + c.length > 3 || + !c.every(Number.isFinite) || + Math.abs(c[0]) > 180 || + Math.abs(c[1]) > 90 || + (c.length === 3 && Math.abs(c[2]) > 1_000_000) + ) + throw new GeoImportError('That GeoJSON has invalid coordinates.'); + positions += 1; + if (positions > maxPositions) return result('positions'); + } else + for (let i = 0; i < c.length; i++) coords.push([c[i], remaining - 1]); + } + } else + throw new GeoImportError( + 'That GeoJSON contains an unsupported geometry.', + ); + } + return result(); +} + +/** The refusal for a file over a size limit, in the person's terms. */ +function overLimitError({ over, features }) { + if (over === 'entities') + return new GeoImportError( + `That file creates more than ${MAX_IMPORT_ENTITIES.toLocaleString()} map objects (the limit).`, + ); + if (over === 'links') + return new GeoImportError( + `That file has more than ${MAX_KML_LINKS.toLocaleString()} style definitions and image links after cleanup (the limit).`, + ); + if (over === 'features') + return new GeoImportError( + `That file has more than ${MAX_IMPORT_ENTITIES.toLocaleString()} features (the limit).`, + ); + return new GeoImportError( + `That file has more than ${MAX_IMPORT_POSITIONS.toLocaleString()} points (the limit)${features ? ` across ${features.toLocaleString()} features` : ''}.`, + ); +} + +// The import uses WGS84 ground footprints and a short name only. Remove +// loader-sensitive metadata before Cesium allocates properties or entity IDs. +function prepareGeoJson(geojson) { + const stack = [geojson]; + while (stack.length) { + const node = stack.pop(); + if (!node) continue; + delete node.crs; + delete node.id; + if (node.type === 'FeatureCollection') { + for (const feature of node.features) stack.push(feature); + } else if (node.type === 'Feature') { + const name = importFeatureName(node.properties?.name); + node.properties = name ? { name } : {}; + stack.push(node.geometry); + } else if (node.type === 'GeometryCollection') { + for (const geometry of node.geometries) stack.push(geometry); + } else { + const coords = [node.coordinates]; + while (coords.length) { + const value = coords.pop(); + if (typeof value[0] === 'number') value.length = 2; + else for (const child of value) coords.push(child); + } + } + } +} + +/** Inflate raw-deflate bytes with the platform's stream, stopping at `maxBytes`. */ +async function inflateRaw(bytes, maxBytes) { + const stream = new Blob([bytes]) + .stream() + .pipeThrough(new DecompressionStream('deflate-raw')); + const reader = stream.getReader(); + const chunks = []; + let total = 0; + try { + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + total += value.length; + if (total > maxBytes) + throw new GeoImportError( + 'That KMZ unpacks to more than the import limit.', + ); + chunks.push(value); + } + } catch (error) { + await reader.cancel().catch(() => {}); + if (error instanceof GeoImportError) throw error; + throw new GeoImportError('That KMZ is damaged and could not be unpacked.'); + } + const out = new Uint8Array(total); + let at = 0; + for (const chunk of chunks) { + out.set(chunk, at); + at += chunk.length; + } + return out; +} + +/** + * Read the entries of a zip archive (stored or deflated; no encryption, no + * ZIP64). Directory entries are skipped. + * @param {Uint8Array} bytes The whole archive. + * @param {{maxInflatedBytes?: number}} [options] + * @returns {Promise>} Entry path → contents. + */ +export async function readZipEntries( + bytes, + { maxInflatedBytes = MAX_KMZ_INFLATED_BYTES } = {}, +) { + const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); + const fail = (message = 'That KMZ is not a readable zip archive.') => { + throw new GeoImportError(message); + }; + // End of central directory: 22 bytes plus a comment of up to 64 KiB. + let eocd = -1; + for ( + let i = bytes.length - 22; + i >= Math.max(0, bytes.length - 22 - 0xffff); + i-- + ) { + if (view.getUint32(i, true) === 0x06054b50) { + eocd = i; + break; + } + } + if (eocd < 0) fail(); + const count = view.getUint16(eocd + 10, true); + const dirSize = view.getUint32(eocd + 12, true); + let at = view.getUint32(eocd + 16, true); + if (count === 0xffff || at === 0xffffffff || at + dirSize > eocd) + fail('That KMZ uses a zip format this import does not read.'); + const entries = new Map(); + const names = new TextDecoder(); + let inflated = 0; + for (let n = 0; n < count; n++) { + if (at + 46 > bytes.length || view.getUint32(at, true) !== 0x02014b50) + fail(); + const flags = view.getUint16(at + 8, true); + const method = view.getUint16(at + 10, true); + const compressed = view.getUint32(at + 20, true); + const size = view.getUint32(at + 24, true); + const nameLength = view.getUint16(at + 28, true); + const extraLength = view.getUint16(at + 30, true); + const commentLength = view.getUint16(at + 32, true); + const local = view.getUint32(at + 42, true); + const name = names.decode(bytes.subarray(at + 46, at + 46 + nameLength)); + at += 46 + nameLength + extraLength + commentLength; + if (name.endsWith('/')) continue; + if (flags & 0x1) fail('That KMZ is encrypted.'); + if (compressed === 0xffffffff || size === 0xffffffff) + fail('That KMZ uses a zip format this import does not read.'); + if (local + 30 > bytes.length || view.getUint32(local, true) !== 0x04034b50) + fail(); + const start = + local + + 30 + + view.getUint16(local + 26, true) + + view.getUint16(local + 28, true); + if (start + compressed > bytes.length) fail(); + const data = bytes.subarray(start, start + compressed); + let contents; + if (method === 0) contents = data; + else if (method === 8) + contents = await inflateRaw(data, maxInflatedBytes - inflated); + else fail('That KMZ uses a compression method this import does not read.'); + inflated += contents.length; + if (inflated > maxInflatedBytes) + fail('That KMZ unpacks to more than the import limit.'); + entries.set(name, contents); + } + return entries; +} + +/** + * The KML document inside a KMZ: `doc.kml` when present, else the first + * `.kml` entry nearest the archive root. + * @param {Map} entries + * @returns {string|null} Entry path. + */ +export function kmzMainDocument(entries) { + if (entries.has('doc.kml')) return 'doc.kml'; + const kml = [...entries.keys()] + .filter((name) => /\.kml$/i.test(name)) + .sort( + (a, b) => a.split('/').length - b.split('/').length || a.localeCompare(b), + ); + return kml[0] ?? null; +} + +/** + * Resolve a KMZ-relative href to an entry path, or null. + * @param {string} href As written in the KML. + * @param {string} base The KML entry's path. + * @param {Map} entries + * @returns {string|null} + */ +export function resolveKmzPath(href, base, entries) { + const raw = String(href || '').trim(); + if (!raw || /^[a-z][a-z0-9+.-]*:/i.test(raw) || raw.startsWith('//')) + return null; + let decoded; + try { + decoded = decodeURIComponent(raw.split(/[?#]/)[0]); + } catch { + return null; + } + const parts = base.split('/').slice(0, -1); + for (const part of decoded.replace(/\\/g, '/').split('/')) { + if (!part || part === '.') continue; + if (part === '..') parts.pop(); + else parts.push(part); + } + const path = parts.join('/'); + return entries.has(path) ? path : null; +} + +/** MIME type for a KMZ image entry, or null when it is not an image we serve. */ +export function kmzImageType(path) { + const ext = /\.([a-z0-9]+)$/i.exec(path)?.[1]?.toLowerCase(); + return KMZ_IMAGE_TYPES[ext] ?? null; +} + +const elementsNamed = (doc, localName) => + [...doc.getElementsByTagName('*')].filter((el) => el.localName === localName); + +/** + * A `resolveLocal` for `sanitizeKmlDocument`: the blob URL of an image packed + * in the KMZ, or null. Each entry gets one blob and one URL however many + * hrefs name it, so repeated icons don't copy (and decode) the image again. + * @param {Map} entries + * @param {string} main Path of the KMZ's KML document. + * @param {(url: string) => void} onUrl Called once per new URL (to revoke later). + * @param {(blob: Blob) => string} [createUrl] + * @returns {(href: string) => string|null} + */ +export function kmzImageResolver( + entries, + main, + onUrl, + createUrl = (blob) => URL.createObjectURL(blob), +) { + const urls = new Map(); + return (href) => { + const path = resolveKmzPath(href, main, entries); + const type = path && kmzImageType(path); + if (!type) return null; + let url = urls.get(path); + if (!url) { + url = createUrl(new Blob([entries.get(path)], { type })); + urls.set(path, url); + onUrl(url); + } + return url; + }; +} + +/** + * Strip everything from a parsed KML document that would make the page reach + * outside the file. `resolveLocal(href)` returns a URL for an image packed in + * the same KMZ (or null); every other href is removed, so icons fall back to + * Cesium's default pin and a ground overlay without a local image is dropped. + * @param {Document} doc + * @param {(href: string) => string|null} [resolveLocal] + * @returns {{removed: number}} How many elements were taken out. + */ +export function sanitizeKmlDocument(doc, resolveLocal = () => null) { + let removed = 0; + const drop = (el) => { + if (el.parentNode) { + el.parentNode.removeChild(el); + removed += 1; + } + }; + // One snapshot and pass, rather than rescanning the whole XML tree for + // every discarded element type. Strip unused metadata before the loader + // constructs descriptions, availability intervals and camera objects. + for (const el of [...doc.getElementsByTagName('*')]) + if ( + KML_REMOVED_ELEMENTS.has(el.localName) || + el.localName in Object.prototype + ) + drop(el); + else if (!['Style', 'StyleMap'].includes(el.localName)) + el.removeAttribute('id'); + for (const href of elementsNamed(doc, 'href')) { + const value = (href.textContent || '').trim(); + if (!value) continue; + const local = resolveLocal(value); + if (local) href.textContent = local; + else if (/^data:image\/(png|jpeg|gif|webp|bmp);/i.test(value)) continue; + else { + const icon = href.parentNode; + const overlay = icon?.parentNode; + if (overlay?.localName === 'GroundOverlay') { + drop(overlay); + continue; + } + // An empty Icon explicitly hides the billboard in Cesium. Remove the + // blocked icon definition so the loader supplies its local default pin. + // Authored empty Icons (with no blocked href) still mean no marker. + drop( + icon?.localName === 'Icon' && icon.parentNode?.localName === 'IconStyle' + ? icon + : href, + ); + } + } + for (const styleUrl of elementsNamed(doc, 'styleUrl')) { + const value = (styleUrl.textContent || '').trim(); + if (!value.startsWith('#')) drop(styleUrl); + else styleUrl.textContent = value; + } + for (const pair of elementsNamed(doc, 'Pair')) + if ( + !childText(pair, 'key') || + (!childrenNamed(pair, 'styleUrl').length && + !childrenNamed(pair, 'Style').length) + ) + drop(pair); + // Draw recorded KML paths as their original static vertices. Letting the + // loader build time-dynamic tracks would expand each into thousands of + // interpolated points later and exceed the checked geometry budget. + for (const track of elementsNamed(doc, 'Track')) { + const line = doc.createElementNS( + 'http://www.opengis.net/kml/2.2', + 'LineString', + ); + const coordinates = doc.createElementNS( + 'http://www.opengis.net/kml/2.2', + 'coordinates', + ); + const recordedCoordinates = childrenNamed(track, 'coord').map((coord) => + coord.textContent.trim().split(/\s+/).join(','), + ); + coordinates.textContent = recordedCoordinates.join(' '); + line.appendChild(coordinates); + for (const mode of childrenNamed(track, 'altitudeMode')) + line.appendChild(mode.cloneNode(true)); + track.parentNode?.replaceChild(line, track); + } + for (const tracks of elementsNamed(doc, 'MultiTrack')) { + const geometry = doc.createElementNS( + 'http://www.opengis.net/kml/2.2', + 'MultiGeometry', + ); + for (const line of childrenNamed(tracks, 'LineString')) + geometry.appendChild(line); + tracks.parentNode?.replaceChild(geometry, tracks); + } + for (const line of [ + ...elementsNamed(doc, 'LineString'), + ...elementsNamed(doc, 'LinearRing'), + ]) { + const mode = childText(line, 'altitudeMode'); + if ( + !mode || + mode === 'clampToGround' || + mode === 'clampToSeaFloor' || + !childrenNamed(line, 'tessellate').length + ) { + let tessellate = childrenNamed(line, 'tessellate')[0]; + if (!tessellate) { + tessellate = doc.createElementNS( + 'http://www.opengis.net/kml/2.2', + 'tessellate', + ); + line.appendChild(tessellate); + } + tessellate.textContent = '1'; + } + } + return { removed }; +} + +/** Child elements of `el` with this local name (any namespace). */ +const childrenNamed = (el, localName) => { + const out = []; + const kids = el?.childNodes || []; + for (let i = 0; i < kids.length; i++) + if (kids[i].nodeType === 1 && kids[i].localName === localName) + out.push(kids[i]); + return out; +}; +const childText = (el, localName) => + (childrenNamed(el, localName)[0]?.textContent || '').trim(); + +/** + * Share identical styles before the loader builds their graphics. Inline + * overrides stay inline StyleMaps, so an existing styleUrl still supplies + * properties the override leaves unset. Named references are redirected. + * Random-color styles remain unchanged; duplicate IDs + * keep the loader's original precedence instead of being rewritten. + * @param {Document} doc Sanitized KML. + * @param {{serializeStyle?: Function, maxStyles?: number}} [options] + * @returns {number} Number of original style blocks replaced. + */ +export function deduplicateKmlStyles( + doc, + { serializeStyle, maxStyles = MAX_KML_LINKS } = {}, +) { + const serializer = serializeStyle ? null : new XMLSerializer(); + const serialize = + serializeStyle || ((el) => serializer.serializeToString(el)); + const groups = new Map(); + const usedIds = new Set(); + const duplicateIds = new Set(); + for (const el of doc.getElementsByTagName('*')) { + const id = el.getAttribute('id'); + if (id !== null) { + if (usedIds.has(id)) duplicateIds.add(id); + usedIds.add(id); + } + } + for (const style of elementsNamed(doc, 'Style')) { + if ( + elementsNamed(style, 'colorMode').some( + (el) => el.textContent.trim() === 'random', + ) + ) + continue; + const template = style.cloneNode(true); + template.removeAttribute('id'); + const key = serialize(template); + let group = groups.get(key); + if (!group) { + group = { template, nodes: [] }; + groups.set(key, group); + if (groups.size > maxStyles) throw overLimitError({ over: 'links' }); + } + group.nodes.push(style); + } + const redirects = new Map(); + let replaced = 0; + let serial = 0; + let container; + for (const { template, nodes } of groups.values()) { + if ( + nodes.length < 2 || + nodes.some((el) => duplicateIds.has(el.getAttribute('id'))) + ) + continue; + const ns = template.namespaceURI || 'http://www.opengis.net/kml/2.2'; + if (!container) { + const root = doc.documentElement; + container = + root.localName === 'Document' + ? root + : childrenNamed(root, 'Document')[0]; + if (!container) { + container = doc.createElementNS(ns, 'Document'); + while (root.firstChild) container.appendChild(root.firstChild); + root.appendChild(container); + } + } + let id; + do { + id = `gev-import-style-${++serial}`; + } while (usedIds.has(id)); + usedIds.add(id); + template.setAttribute('id', id); + container.appendChild(template); + for (const style of nodes) { + const originalId = style.getAttribute('id'); + if (originalId !== null) redirects.set(originalId, id); + let reference; + if (style.parentNode.localName === 'Pair') { + reference = doc.createElementNS(ns, 'styleUrl'); + reference.textContent = `#${id}`; + } else { + reference = doc.createElementNS(ns, 'StyleMap'); + for (const state of ['normal', 'highlight']) { + const pair = doc.createElementNS(ns, 'Pair'); + const key = doc.createElementNS(ns, 'key'); + key.textContent = state; + const url = doc.createElementNS(ns, 'styleUrl'); + url.textContent = `#${id}`; + pair.appendChild(key); + pair.appendChild(url); + reference.appendChild(pair); + } + } + style.parentNode.replaceChild(reference, style); + replaced += 1; + } + } + for (const url of elementsNamed(doc, 'styleUrl')) { + const value = url.textContent.trim(); + const id = value.startsWith('#') && redirects.get(value.slice(1)); + if (id) url.textContent = `#${id}`; + } + return replaced; +} + +/** + * Count placemarks and vertices in a parsed KML document before Cesium reads + * it: every `` tuple and every `gx:coord`, plus normalized + * style definitions and distinct image `` values (`links`). + * Inline StyleMaps are references, not additional shared style definitions. + * @param {Document} doc + * @returns {{features: number, entities: number, positions: number, links: number, + * over: ''|'features'|'entities'|'positions'|'links'}} + */ +export function countKmlGeometry( + doc, + { + maxFeatures = MAX_IMPORT_ENTITIES, + maxEntities = MAX_IMPORT_ENTITIES, + maxPositions = MAX_IMPORT_POSITIONS, + maxLinks = MAX_KML_LINKS, + } = {}, +) { + let features = 0; + let entities = 0; + let positions = 0; + let links = 0; + const imageLinks = new Set(); + const result = (over) => ({ features, entities, positions, links, over }); + const geometryNames = new Set([ + 'Point', + 'LineString', + 'LinearRing', + 'Polygon', + 'Track', + 'MultiTrack', + 'MultiGeometry', + 'Model', + ]); + for (const el of doc.getElementsByTagName('*')) { + const name = el.localName; + if ( + ['north', 'south', 'east', 'west', 'rotation', 'altitude'].includes(name) + ) { + const value = Number(el.textContent); + const limit = ['north', 'south'].includes(name) + ? 90 + : name === 'altitude' + ? 1_000_000 + : 360; + if ( + !el.textContent.trim() || + !Number.isFinite(value) || + Math.abs(value) > limit + ) + throw new GeoImportError('That KML has invalid geometry bounds.'); + } + // Cesium creates entities for folders, a parent for every placemark, and + // a child for each geometry in a MultiGeometry, including nested containers. + if ( + name === 'Placemark' || + name === 'Folder' || + name === 'GroundOverlay' || + (geometryNames.has(name) && + ['MultiGeometry', 'MultiTrack'].includes(el.parentNode?.localName)) + ) { + entities += 1; + if (entities > maxEntities) return result('entities'); + } + if ( + name === 'Style' || + (name === 'StyleMap' && el.getAttribute('id') !== null) + ) { + links += 1; + if (links > maxLinks) return result('links'); + } else if (name === 'href') { + const href = el.textContent.trim(); + if (!imageLinks.has(href)) { + imageLinks.add(href); + links += 1; + if (links > maxLinks) return result('links'); + } + } else if (name === 'Placemark') { + features += 1; + if (features > maxFeatures) return result('features'); + } else if (name === 'coordinates') { + const text = el.textContent || ''; + for (const match of text.matchAll(/\S+/g)) { + checkKmlPosition(match[0].split(',')); + positions += 1; + if (positions > maxPositions) return result('positions'); + } + } else if (name === 'coord') { + checkKmlPosition(el.textContent.trim().split(/\s+/)); + positions += 1; + if (positions > maxPositions) return result('positions'); + } + } + return result(''); +} + +function checkKmlPosition(tuple) { + const numbers = tuple.map(Number); + if ( + tuple.length < 2 || + tuple.length > 3 || + tuple.some((v) => !v.trim()) || + !numbers.every(Number.isFinite) || + Math.abs(numbers[0]) > 180 || + Math.abs(numbers[1]) > 90 || + (numbers.length === 3 && Math.abs(numbers[2]) > 1_000_000) + ) + throw new GeoImportError('That KML has invalid coordinates.'); +} + +/** + * Read a parsed GPX document into GeoJSON: waypoints become points, each + * route a line and each track one MultiLineString with a line per `` + * (segment breaks are gaps in the recording and stay gaps). Only names are + * kept, and no times, so a dated track draws without a clock. Throws + * `GeoImportError` past the size limits. + * @param {Document} doc + * @returns {object} A FeatureCollection. + */ +export function gpxToGeoJson( + doc, + { + maxFeatures = MAX_IMPORT_ENTITIES, + maxPositions = MAX_IMPORT_POSITIONS, + } = {}, +) { + const root = doc.documentElement; + if (root?.localName !== 'gpx') + throw new GeoImportError('That GPX file has no document.'); + const features = []; + let positions = 0; + const point = (el) => { + const rawLat = el.getAttribute('lat'); + const rawLon = el.getAttribute('lon'); + if (rawLat == null || rawLon == null || !rawLat.trim() || !rawLon.trim()) + return null; + const lat = Number(rawLat); + const lon = Number(rawLon); + if ( + !Number.isFinite(lat) || + !Number.isFinite(lon) || + Math.abs(lat) > 90 || + Math.abs(lon) > 180 + ) + return null; + positions += 1; + if (positions > maxPositions) + throw overLimitError({ + over: 'positions', + features: features.length, + positions, + }); + return [lon, lat]; + }; + const add = (name, geometry) => { + if (features.length >= maxFeatures) + throw overLimitError({ + over: 'features', + features: features.length + 1, + positions, + }); + features.push({ + type: 'Feature', + properties: name ? { name } : {}, + geometry, + }); + }; + for (const wpt of childrenNamed(root, 'wpt')) { + const at = point(wpt); + if (at) add(childText(wpt, 'name'), { type: 'Point', coordinates: at }); + } + for (const rte of childrenNamed(root, 'rte')) { + const line = []; + for (const pt of childrenNamed(rte, 'rtept')) { + const at = point(pt); + if (at) line.push(at); + } + if (line.length >= 2) + add(childText(rte, 'name'), { type: 'LineString', coordinates: line }); + } + for (const trk of childrenNamed(root, 'trk')) { + const lines = []; + for (const seg of childrenNamed(trk, 'trkseg')) { + const line = []; + for (const pt of childrenNamed(seg, 'trkpt')) { + const at = point(pt); + if (at) line.push(at); + } + if (line.length >= 2) lines.push(line); + } + if (lines.length) + add(childText(trk, 'name'), { + type: 'MultiLineString', + coordinates: lines, + }); + } + return { type: 'FeatureCollection', features }; +} + +/** A feature name as short plain text (no control characters), or undefined. */ +export function importFeatureName(value) { + const text = String(value ?? '') + .replace(/[\u0000-\u001f\u007f]/g, ' ') + .replace(/\s+/g, ' ') + .trim(); + if (!text) return undefined; + return text.length > MAX_IMPORT_NAME_LENGTH + ? `${text.slice(0, MAX_IMPORT_NAME_LENGTH - 1)}…` + : text; +} + +/** + * Make every entity of an imported data source inert: no description, + * balloon, extended data or properties, no native label, and a name that is + * short plain text. Generated pin canvases share a data URL so Cesium uploads + * each image once, rather than allocating a texture entry for every entity. + * Returns how many entities were touched. + * @param {{entities: {values: object[]}}} dataSource + * @returns {number} + */ +export function scrubImportedEntities(dataSource) { + const collection = dataSource?.entities; + const list = collection?.values || []; + const images = new Map(); + collection?.suspendEvents?.(); + try { + for (let i = 0; i < list.length; i++) { + const entity = list[i]; + let name; + try { + name = importFeatureName( + typeof entity.name === 'string' ? entity.name : undefined, + ); + } catch { + name = undefined; + } + entity.name = name; + entity.description = undefined; + entity.label = undefined; + entity.properties = undefined; + if (entity.kml && typeof entity.kml === 'object') entity.kml = undefined; + if ( + entity.billboard?.heightReference?.getValue?.() === + Cesium.HeightReference.CLAMP_TO_GROUND + ) + entity.billboard.disableDepthTestDistance = Number.POSITIVE_INFINITY; + const property = entity.billboard?.image; + if (property?.isConstant === false) continue; + const image = property?.getValue?.(); + if (image?.tagName === 'CANVAS') { + let url = images.get(image); + if (!url) { + url = image.toDataURL(); + images.set(image, url); + } + entity.billboard.image = url; + } + } + } finally { + collection?.resumeEvents?.(); + } + return list.length; +} + +/** + * Take time out of an imported data source so adding it cannot move the app's + * shared clock: the data source reports no clock (the viewer adopts the clock + * of every added source that has one), every entity is available for all + * time. KML tracks have already become their original static vertices in + * `sanitizeKmlDocument`, so no sampled-position fallback is needed here. + * @param {object} dataSource + * @returns {object} The same data source. + */ +export function detachImportedTime(dataSource) { + Object.defineProperty(dataSource, 'clock', { + configurable: true, + enumerable: false, + get: () => undefined, + set: () => {}, + }); + const list = dataSource?.entities?.values || []; + for (let i = 0; i < list.length; i++) { + const entity = list[i]; + entity.availability = undefined; + } + return dataSource; +} + +function parseXml(text, what) { + // Match the namespace repairs Cesium applies to ordinary exported KML. + if (what !== 'GPX') + text = String(text).replace(/]*>/i, (root) => { + const seen = new Set(); + root = root.replace( + /\s+xmlns:([\w.-]+)\s*=\s*(["'])[^"']*\2/g, + (declaration, prefix) => { + if (seen.has(prefix)) return ''; + seen.add(prefix); + return declaration; + }, + ); + if (!seen.has('xsi') && /[<\s]xsi:/.test(text)) + root = root.replace( + />$/, + ' xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">', + ); + return root; + }); + const doc = new DOMParser().parseFromString( + String(text).replace(/^/, ''), + 'application/xml', + ); + if (doc.getElementsByTagName('parsererror').length) + throw new GeoImportError(`That ${what} file is not valid XML.`); + return doc; +} + +function decodeXml(bytes) { + const encoding = + bytes[0] === 0xff && bytes[1] === 0xfe + ? 'utf-16le' + : bytes[0] === 0xfe && bytes[1] === 0xff + ? 'utf-16be' + : 'utf-8'; + return new TextDecoder(encoding).decode(bytes); +} + +function checkLocalImages(doc, entries = new Map(), main = '') { + const charge = createImportImageBudget(); + const checked = new Set(); + for (const href of elementsNamed(doc, 'href')) { + const value = href.textContent.trim(); + if (checked.has(value)) continue; + checked.add(value); + const path = resolveKmzPath(value, main, entries); + try { + if (path && kmzImageType(path)) charge(path, entries.get(path)); + else if (/^data:image\/(png|jpeg|gif|webp|bmp);/i.test(value)) { + const comma = value.indexOf(','); + const header = value.slice(0, comma); + const body = value.slice(comma + 1); + const raw = /;base64$/i.test(header) + ? atob(body) + : decodeURIComponent(body); + charge( + value, + Uint8Array.from(raw, (c) => c.charCodeAt(0)), + ); + } + } catch { + throw new GeoImportError( + 'That file has an unreadable image or exceeds the 16-megapixel image limit.', + ); + } + } +} + +// Cesium also processes KML in deferred tasks; an exception there does not +// reject its load promise. Keep a failed file from blocking the serial queue. +async function loadKml(doc, viewer, signal) { + let removeError; + try { + return await boundedImport( + Promise.race([ + Cesium.KmlDataSource.load(doc, { + camera: viewer.scene.camera, + canvas: viewer.scene.canvas, + clampToGround: true, + }), + new Promise((_, reject) => { + const failed = () => { + reject(new GeoImportError('That KML could not be read.')); + }; + globalThis.addEventListener?.('error', failed); + removeError = () => globalThis.removeEventListener?.('error', failed); + }), + ]), + signal, + ); + } finally { + removeError?.(); + } +} + +// Resolve a static file's ground pins in batches before they are visualized. +// Per-billboard CLAMP_TO_GROUND subscriptions scale poorly on large imports. +const groundPins = new WeakMap(); +async function settleGroundPins(dataSource, viewer, signal) { + let pins = groundPins.get(dataSource); + if (!pins) { + pins = []; + for (const entity of dataSource.entities.values) { + const reference = entity.billboard?.heightReference?.getValue?.(); + if ( + reference === Cesium.HeightReference.CLAMP_TO_GROUND || + reference === Cesium.HeightReference.RELATIVE_TO_GROUND + ) { + const position = entity.position?.getValue?.(Cesium.JulianDate.now()); + if (!position) continue; + const cartographic = Cesium.Cartographic.fromCartesian(position); + if (!cartographic) + throw new GeoImportError('That file has invalid point coordinates.'); + pins.push({ + entity, + original: position, + cartographic, + offset: + reference === Cesium.HeightReference.RELATIVE_TO_GROUND + ? cartographic.height + : 0, + }); + entity.billboard.heightReference = Cesium.HeightReference.NONE; + entity.billboard.disableDepthTestDistance = Number.POSITIVE_INFINITY; + } + } + groundPins.set(dataSource, pins); + } + for (let at = 0; at < pins.length; at += 256) { + signal?.throwIfAborted(); + const batch = pins.slice(at, at + 256); + const positions = batch.map( + ({ cartographic: c }) => + new Cesium.Cartographic(c.longitude, c.latitude, 0), + ); + const mesh = + viewer.scene?.globe?.show === false && + viewer.scene?.clampToHeightSupported; + // A coarse rendered ellipsoid mesh can report a height kilometres below + // zero. Its provider's exact height is zero even while a 3D map hides it; + // a buried starting point can make the downward mesh ray miss the surface. + const ellipsoid = + viewer.terrainProvider instanceof Cesium.EllipsoidTerrainProvider; + for (let i = 0; i < positions.length; i++) { + const sampled = ellipsoid + ? 0 + : viewer.scene?.globe?.getHeight?.(positions[i]); + positions[i].height = mesh + ? (batch[i].groundHeight ?? + Math.max(0, Number.isFinite(sampled) ? sampled : 0)) + : Number.isFinite(sampled) + ? sampled + : batch[i].groundHeight || 0; + } + const points = positions.map((c) => + Cesium.Cartesian3.fromRadians(c.longitude, c.latitude, c.height), + ); + const height = viewer.camera?.positionCartographic?.height; + if (mesh && Number.isFinite(height) && height < 250_000) { + const rectangle = viewer.camera.computeViewRectangle?.(); + const indices = []; + for (let i = 0; i < positions.length; i++) + if (rectangle && Cesium.Rectangle.contains(rectangle, positions[i])) + indices.push(i); + if (indices.length) { + const resolved = await boundedImport( + viewer.scene.clampToHeightMostDetailed( + indices.map((i) => { + const c = positions[i]; + return Cesium.Cartesian3.fromRadians( + c.longitude, + c.latitude, + Math.max(0, c.height), + ); + }), + ), + signal, + ); + for (let i = 0; i < indices.length; i++) + if (resolved[i]) points[indices[i]] = resolved[i]; + } + } + signal?.throwIfAborted(); + for (let i = 0; i < batch.length; i++) { + const item = batch[i]; + const c = points[i] && Cesium.Cartographic.fromCartesian(points[i]); + if (!c) + throw new GeoImportError( + 'Ground heights are not available for that file. Try again when the map has loaded.', + ); + item.groundHeight = c.height; + c.height += item.offset; + item.entity.position = + Math.abs(c.height - item.cartographic.height) < 1e-6 + ? item.original + : Cesium.Cartesian3.fromRadians(c.longitude, c.latitude, c.height); + } + if (at + batch.length < pins.length) + await new Promise((resolve) => setTimeout(resolve, 0)); + } +} + +async function boundedImport(promise, signal) { + let timer, abort; + try { + signal?.throwIfAborted(); + return await Promise.race([ + promise, + new Promise((_, reject) => { + abort = () => reject(signal.reason); + signal?.addEventListener('abort', abort, { once: true }); + timer = setTimeout( + () => reject(new GeoImportError('That file took too long to read.')), + 60_000, + ); + }), + ]); + } finally { + clearTimeout(timer); + signal?.removeEventListener('abort', abort); + } +} + +/** + * Load a ground footprint, retaining explicit KML altitude modes. Call mount after + * adding the source; revoke owns pin and blob cleanup. + * @param {File|Blob & {name?: string}} file + * @param {{viewer: Cesium.Viewer, color?: string}} options `color` is a CSS color. + * @returns {Promise<{dataSource: Cesium.DataSource, name: string, + * format: string, entityCount: number, removed: number, mount: Function, revoke: Function}>} + */ +export async function loadGeoFile( + file, + { viewer, color = '#39d0ff', signal } = {}, +) { + signal = signal + ? AbortSignal.any([signal, AbortSignal.timeout(60_000)]) + : AbortSignal.timeout(60_000); + signal?.throwIfAborted(); + if (!file || !Number.isFinite(file.size)) + throw new GeoImportError('Nothing to import.'); + if (file.size > MAX_IMPORT_BYTES) + throw new GeoImportError( + `That file is larger than ${MAX_IMPORT_BYTES / 1024 / 1024} MiB.`, + ); + let bytes = new Uint8Array(await boundedImport(file.arrayBuffer(), signal)); + const format = detectGeoFileFormat(file.name, bytes.subarray(0, 1024)); + if (!format) + throw new GeoImportError( + 'Only GeoJSON, KML, KMZ and GPX files can be imported.', + ); + const stroke = Cesium.Color.fromCssColorString(color); + const urls = []; + let pins; + const revoke = () => { + pins?.destroy(); + urls.splice(0).forEach((url) => URL.revokeObjectURL(url)); + }; + const name = importDisplayName(file.name); + let dataSource; + let removed = 0; + try { + if (format === 'geojson' || format === 'gpx') { + let geojson; + if (format === 'gpx') { + geojson = gpxToGeoJson(parseXml(decodeXml(bytes), 'GPX')); + } else { + geojson = parseGeoJsonText(new TextDecoder().decode(bytes)); + } + const size = countGeoJsonGeometry(geojson); + if (size.over) throw overLimitError(size); + if (!size.positions) + throw new GeoImportError('That file has nothing to draw.'); + prepareGeoJson(geojson); + bytes = null; + dataSource = await boundedImport( + Cesium.GeoJsonDataSource.load(geojson, { + clampToGround: true, + stroke, + fill: stroke.withAlpha(0.25), + strokeWidth: 3, + markerColor: stroke, + // No HTML description table is built from the file's properties. + describe: () => undefined, + }), + signal, + ); + } else { + let text; + let entries; + let main; + let resolveLocal = () => null; + if (format === 'kmz') { + entries = await readZipEntries(bytes); + main = kmzMainDocument(entries); + if (!main) throw new GeoImportError('That KMZ holds no KML document.'); + text = decodeXml(entries.get(main)); + resolveLocal = kmzImageResolver(entries, main, (url) => urls.push(url)); + } else { + text = decodeXml(bytes); + } + const doc = parseXml(text, format.toUpperCase()); + text = null; + bytes = null; + // Bound geometry before transformations, then count the styles and + // image links that survive sanitizing and sharing repeated styles. + const size = countKmlGeometry(doc, { maxLinks: Infinity }); + if (size.over) throw overLimitError(size); + checkLocalImages(doc, entries, main); + removed = sanitizeKmlDocument(doc, resolveLocal).removed; + entries?.clear(); + deduplicateKmlStyles(doc); + const normalizedSize = countKmlGeometry(doc); + if (normalizedSize.over) throw overLimitError(normalizedSize); + dataSource = await loadKml(doc, viewer, signal); + } + } catch (error) { + revoke(); + if (error instanceof GeoImportError) throw error; + throw new GeoImportError( + `That ${format.toUpperCase()} file could not be read.`, + ); + } + const entityCount = dataSource.entities.values.length; + if (entityCount > MAX_IMPORT_ENTITIES) { + revoke(); + throw new GeoImportError( + `That file has ${entityCount.toLocaleString()} map objects; the limit is ${MAX_IMPORT_ENTITIES.toLocaleString()}.`, + ); + } + const now = Cesium.JulianDate.now(); + const drawable = dataSource.entities.values.some( + (e) => + e.billboard?.image?.getValue(now) || + e.point || + e.rectangle || + e.polyline?.positions?.getValue(now)?.length >= 2 || + e.wall?.positions?.getValue(now)?.length >= 2 || + e.polygon?.hierarchy?.getValue(now)?.positions?.length >= 3, + ); + if (!drawable) { + revoke(); + throw new GeoImportError('That file has nothing to draw.'); + } + scrubImportedEntities(dataSource); + try { + await settleGroundPins(dataSource, viewer, signal); + } catch (error) { + revoke(); + throw error instanceof GeoImportError + ? error + : new GeoImportError( + 'Ground heights could not be read. Try that file again.', + ); + } + // Keep the clock invariant explicit even though unused KML dates have + // already been stripped and all supported geometries are static. + detachImportedTime(dataSource); + try { + pins = createImportedPins(dataSource, viewer); + } catch { + revoke(); + throw new GeoImportError('That file has invalid point graphics.'); + } + // The display name travels beside the data source (the file name, not + // anything the document says about itself). + return { + dataSource, + name, + format, + entityCount, + removed, + revoke, + mount: () => pins.mount(), + refreshGround: (refreshSignal) => + settleGroundPins( + dataSource, + viewer, + refreshSignal + ? AbortSignal.any([refreshSignal, AbortSignal.timeout(15_000)]) + : AbortSignal.timeout(15_000), + ).then(() => pins.sync()), + }; +} diff --git a/src/data/geoFileImport.test.mjs b/src/data/geoFileImport.test.mjs new file mode 100644 index 000000000..af02ec222 --- /dev/null +++ b/src/data/geoFileImport.test.mjs @@ -0,0 +1,1173 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { deflateRawSync } from 'node:zlib'; +import * as Cesium from 'cesium'; +import { + GeoImportError, + loadGeoFile, + MAX_IMPORT_POSITIONS, + MAX_IMPORT_ENTITIES, + countGeoJsonGeometry, + countKmlGeometry, + deduplicateKmlStyles, + detachImportedTime, + gpxToGeoJson, + importFeatureName, + scrubImportedEntities, + sanitizeKmlDocument, + detectGeoFileFormat, + importDisplayName, + kmzImageResolver, + kmzImageType, + kmzMainDocument, + parseGeoJsonText, + readZipEntries, + resolveKmzPath, +} from './geoFileImport.js'; +import { createImportImageBudget, importImageSize } from './geoImportImages.js'; + +test('real loader grounds XYZ GeoJSON and strips loader-sensitive metadata before allocation', async () => { + const previous = Cesium.PinBuilder.prototype.fromColor; + Cesium.PinBuilder.prototype.fromColor = () => ({ + toDataURL: () => 'data:image/png;base64,local', + tagName: 'CANVAS', + }); + try { + const object = { + type: 'FeatureCollection', + crs: { type: 'name', properties: { name: 'unsupported' } }, + features: [ + { + type: 'Feature', + id: 'kartaview-owned', + properties: { name: 'Test', addProperty: 1, 'marker-symbol': 'rail' }, + geometry: { type: 'Point', coordinates: [1, 2, 999] }, + }, + { + type: 'Feature', + geometry: { + type: 'Polygon', + coordinates: [ + [ + [1, 2, 0], + [2, 2, 0], + [2, 3, 0], + [1, 2, 0], + ], + ], + }, + }, + ], + }; + const loaded = await loadGeoFile( + new File([JSON.stringify(object)], 'xyz.geojson'), + { + viewer: { + scene: { + frameState: { mode: Cesium.SceneMode.SCENE3D }, + primitives: new Cesium.PrimitiveCollection(), + globe: { getHeight: () => 1234 }, + }, + }, + }, + ); + const [point, polygon] = + loaded.dataSource.importedEntities || loaded.dataSource.entities.values; + assert.notEqual(point.id, 'kartaview-owned'); + assert.equal(point.name, 'Test'); + assert.equal(point.properties, undefined); + assert.equal( + point.billboard.heightReference.getValue(), + Cesium.HeightReference.NONE, + ); + assert.equal(point.billboard.disableDepthTestDistance.getValue(), Infinity); + assert.ok( + Math.abs( + Cesium.Cartographic.fromCartesian( + point.position.getValue(Cesium.JulianDate.now()), + ).height - 1234, + ) < 1e-6, + ); + assert.notEqual(polygon.polygon.perPositionHeight?.getValue(), true); + assert.equal(loaded.dataSource.clock, undefined); + assert.equal( + loaded.dataSource.entities.values.length, + 1, + 'only polygon geometry remains in the per-frame entity collection', + ); + assert.equal(loaded.dataSource.importedBillboards.length, 1); + const update = Cesium.BillboardCollection.prototype.update; + let passes = 0; + Cesium.BillboardCollection.prototype.update = () => passes++; + try { + const collection = loaded.dataSource.importedBillboards; + collection.update({ passes: { pick: true, offscreen: true } }); + assert.equal(passes, 0, 'height picks ignore imported markers'); + collection.update({ passes: { pick: true, offscreen: false } }); + collection.update({ passes: { render: true, offscreen: true } }); + assert.equal(passes, 2, 'ordinary picks and screenshots include markers'); + } finally { + Cesium.BillboardCollection.prototype.update = update; + } + + loaded.mount(); + loaded.dataSource.show = false; + assert.equal(loaded.dataSource.importedBillboards.show, false); + loaded.dataSource.show = true; + await loaded.refreshGround(); + assert.equal( + point.billboard.heightReference.getValue(), + Cesium.HeightReference.NONE, + ); + loaded.revoke(); + assert.ok(loaded.dataSource.importedBillboards.isDestroyed()); + } finally { + Cesium.PinBuilder.prototype.fromColor = previous; + } +}); + +test('real loader refuses empty files and malformed nested geometry before allocating entities', async () => { + for (const object of [ + { type: 'Feature', geometry: null }, + { + type: 'FeatureCollection', + features: [{ geometry: { type: 'Point', coordinates: [1, 2] } }], + }, + { type: 'Feature', geometry: { type: 'Topology', arcs: [] } }, + ...[ + [1, '2'], + [Infinity, 0], + [0, 91], + [0, 0, 1e100], + ].map((coordinates) => ({ type: 'Point', coordinates })), + ]) + await assert.rejects( + loadGeoFile(new File([JSON.stringify(object)], 'bad.geojson'), { + viewer: {}, + }), + GeoImportError, + ); +}); + +test('KML tracks retain their explicit altitude and ground lines are tessellated', () => { + const track = xmlEl('Track', {}, [ + xmlEl('altitudeMode', {}, 'absolute'), + xmlEl('coord', {}, '1 2 10000'), + xmlEl('coord', {}, '2 3 10000'), + ]); + const line = xmlEl('LineString', {}, [xmlEl('coordinates', {}, '1,2 2,3')]); + const ground = xmlEl('Track', {}, [ + xmlEl('coord', {}, '1 2 0'), + xmlEl('coord', {}, '2 3 0'), + ]); + const root = xmlEl('kml', {}, [track, line, ground]); + sanitizeKmlDocument(xmlDoc(root)); + assert.equal( + root.childNodes[0].childNodes.find((n) => n.localName === 'altitudeMode') + .textContent, + 'absolute', + ); + assert.equal( + root.childNodes[1].childNodes.find((n) => n.localName === 'tessellate') + .textContent, + '1', + ); + assert.equal( + root.childNodes[2].childNodes.find((n) => n.localName === 'tessellate') + .textContent, + '1', + ); +}); + +test('sanitizer normalizes styles, removes incomplete pairs and keeps colored overlays', () => { + const local = xmlEl('styleUrl', {}, ' #s '); + const pair = xmlEl('Pair', {}, [ + xmlEl('key', {}, 'normal'), + xmlEl('styleUrl', {}, 'https://example.invalid/s'), + ]); + const overlay = xmlEl('GroundOverlay', {}, [xmlEl('color', {}, 'ff00ff00')]); + const hidden = xmlEl('visibility', {}, '0'); + const id = xmlEl('Placemark', { id: 'kartaview-123' }); + const root = xmlEl('kml', {}, [ + local, + xmlEl('StyleMap', { id: 's' }, [pair]), + overlay, + hidden, + id, + xmlEl('toString'), + ]); + sanitizeKmlDocument(xmlDoc(root)); + assert.equal(local.textContent, '#s'); + assert.equal(pair.parentNode, null); + assert.ok(root.childNodes.includes(overlay)); + assert.equal(hidden.parentNode, null); + assert.equal(id.getAttribute('id'), null); + assert.ok(!root.childNodes.some((n) => n.localName === 'toString')); +}); + +test('image budget rejects large decoded rasters before decoding and shares repeated images', () => { + const png = (w, h) => { + const b = new Uint8Array(24); + const v = new DataView(b.buffer); + v.setUint32(0, 0x89504e47); + b.set([73, 72, 68, 82], 12); + v.setUint32(16, w); + v.setUint32(20, h); + return b; + }; + assert.deepEqual(importImageSize(png(4096, 4096)), [4096, 4096]); + const charge = createImportImageBudget(); + charge('shared', png(4096, 4096)); + charge('shared', png(4096, 4096)); + assert.throws(() => charge('other', png(1, 1)), /limit/); + assert.throws( + () => createImportImageBudget()('huge', png(16384, 16384)), + /limit/, + ); +}); + +test('pending file reads can be cancelled without waiting for the loader timeout', async () => { + const controller = new AbortController(); + const file = { + name: 'waiting.geojson', + size: 1, + arrayBuffer: () => new Promise(() => {}), + }; + const promise = loadGeoFile(file, { viewer: {}, signal: controller.signal }); + controller.abort(new Error('cancelled by teardown')); + await assert.rejects(promise, /cancelled by teardown/); +}); + +test('a hidden coarse globe never seeds a mesh-clamp ray below the surface', async () => { + const previous = Cesium.PinBuilder.prototype.fromColor; + Cesium.PinBuilder.prototype.fromColor = () => ({ + tagName: 'CANVAS', + toDataURL: () => 'data:image/png;base64,local', + }); + const starts = []; + try { + for (const terrainProvider of [new Cesium.EllipsoidTerrainProvider(), {}]) { + const viewer = { + terrainProvider, + camera: { + positionCartographic: { height: 100000 }, + computeViewRectangle: () => Cesium.Rectangle.MAX_VALUE, + }, + scene: { + frameState: { mode: Cesium.SceneMode.SCENE3D }, + globe: { show: false, getHeight: () => -12311 }, + clampToHeightSupported: true, + clampToHeightMostDetailed: async (points) => + points.map((p) => { + const c = Cesium.Cartographic.fromCartesian(p); + starts.push(c.height); + return c.height < -9000 + ? undefined + : Cesium.Cartesian3.fromRadians(c.longitude, c.latitude, 2000); + }), + }, + }; + const loaded = await loadGeoFile( + new File( + [JSON.stringify({ type: 'Point', coordinates: [1, 2] })], + 'surface.geojson', + ), + { viewer }, + ); + assert.ok( + Math.abs(starts.at(-1)) < 1e-6, + `ray started at ${starts.at(-1)}`, + ); + assert.ok( + Math.abs( + Cesium.Cartographic.fromCartesian( + loaded.dataSource.importedBillboards.get(0).position, + ).height - 2000, + ) < 1e-6, + ); + const sampled = starts.length; + viewer.camera.computeViewRectangle = () => undefined; + await loaded.refreshGround(); + assert.equal( + starts.length, + sampled, + 'distant points do not start a detailed mesh sweep', + ); + assert.ok( + Math.abs( + Cesium.Cartographic.fromCartesian( + loaded.dataSource.importedBillboards.get(0).position, + ).height - 2000, + ) < 1e-6, + 'the last valid mesh height survives outside the camera footprint', + ); + const point = loaded.dataSource.importedEntities[0]; + const other = new Cesium.Entity({ id: 'unrelated-aircraft' }); + viewer.trackedEntity = starts.length === 1 ? other : point; + viewer.selectedEntity = point; + loaded.revoke(); + assert.equal(viewer.selectedEntity, undefined); + assert.equal( + viewer.trackedEntity, + starts.length === 1 ? other : undefined, + ); + } + } finally { + Cesium.PinBuilder.prototype.fromColor = previous; + } +}); + +const bytes = (text) => new TextEncoder().encode(text); + +/** A minimal zip writer: stored (0) or deflated (8) entries, optional flags. */ +function zip(files) { + const locals = []; + const centrals = []; + let offset = 0; + for (const { name, data, method = 8, flags = 0, size } of files) { + const nameBytes = bytes(name); + const body = method === 8 ? new Uint8Array(deflateRawSync(data)) : data; + const local = new Uint8Array(30 + nameBytes.length); + const lv = new DataView(local.buffer); + lv.setUint32(0, 0x04034b50, true); + lv.setUint16(6, flags, true); + lv.setUint16(8, method, true); + lv.setUint32(18, body.length, true); + lv.setUint32(22, size ?? data.length, true); + lv.setUint16(26, nameBytes.length, true); + local.set(nameBytes, 30); + const central = new Uint8Array(46 + nameBytes.length); + const cv = new DataView(central.buffer); + cv.setUint32(0, 0x02014b50, true); + cv.setUint16(8, flags, true); + cv.setUint16(10, method, true); + cv.setUint32(20, body.length, true); + cv.setUint32(24, size ?? data.length, true); + cv.setUint16(28, nameBytes.length, true); + cv.setUint32(42, offset, true); + central.set(nameBytes, 46); + locals.push(local, body); + centrals.push(central); + offset += local.length + body.length; + } + const dirSize = centrals.reduce((n, c) => n + c.length, 0); + const eocd = new Uint8Array(22); + const ev = new DataView(eocd.buffer); + ev.setUint32(0, 0x06054b50, true); + ev.setUint16(8, files.length, true); + ev.setUint16(10, files.length, true); + ev.setUint32(12, dirSize, true); + ev.setUint32(16, offset, true); + const parts = [...locals, ...centrals, eocd]; + const out = new Uint8Array(parts.reduce((n, p) => n + p.length, 0)); + let at = 0; + for (const p of parts) { + out.set(p, at); + at += p.length; + } + return out; +} + +test('format comes from the extension, then from the first bytes', () => { + assert.equal(detectGeoFileFormat('trail.GPX'), 'gpx'); + assert.equal(detectGeoFileFormat('farm.kml'), 'kml'); + assert.equal(detectGeoFileFormat('farm.kmz'), 'kmz'); + assert.equal(detectGeoFileFormat('parcels.geojson'), 'geojson'); + assert.equal(detectGeoFileFormat('parcels.json'), 'geojson'); + assert.equal( + detectGeoFileFormat('export', bytes(' {"type":"Feature"}')), + 'geojson', + ); + assert.equal( + detectGeoFileFormat( + 'export', + bytes(''), + ), + 'kml', + ); + assert.equal( + detectGeoFileFormat( + 'export', + bytes('\n'), + ), + 'gpx', + ); + assert.equal( + detectGeoFileFormat('export', new Uint8Array([0x50, 0x4b, 0x03, 0x04, 0])), + 'kmz', + ); + assert.equal(detectGeoFileFormat('notes.txt', bytes('hello')), null); + assert.equal( + detectGeoFileFormat('image.png', new Uint8Array([0x89, 0x50, 0x4e, 0x47])), + null, + ); +}); + +test('display names are printable and bounded', () => { + assert.equal(importDisplayName(' hike\u0000.gpx '), 'hike.gpx'); + assert.equal(importDisplayName(''), 'Imported file'); + const long = importDisplayName(`${'x'.repeat(80)}.kml`); + assert.equal(long.length, 58); + assert.ok(long.endsWith('…')); +}); + +test('GeoJSON text must be a GeoJSON object', () => { + assert.equal( + parseGeoJsonText('{"type":"FeatureCollection","features":[]}').type, + 'FeatureCollection', + ); + assert.equal( + parseGeoJsonText('{"type":"Point","coordinates":[0,0]}').type, + 'Point', + ); + assert.throws(() => parseGeoJsonText('{nope'), GeoImportError); + assert.throws(() => parseGeoJsonText('{"name":"not geo"}'), /not GeoJSON/); + assert.throws(() => parseGeoJsonText('[1,2]'), /not GeoJSON/); + assert.throws( + () => parseGeoJsonText('{"type":"FeatureCollection"}'), + /no features list/, + ); +}); + +test('a KMZ is read with stored and deflated entries, directories skipped', async () => { + const kml = bytes(''); + const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 1, 2, 3]); + const archive = zip([ + { name: 'files/', data: new Uint8Array(0), method: 0 }, + { name: 'doc.kml', data: kml }, + { name: 'files/icon.png', data: png, method: 0 }, + ]); + const entries = await readZipEntries(archive); + assert.deepEqual([...entries.keys()], ['doc.kml', 'files/icon.png']); + assert.deepEqual(entries.get('doc.kml'), kml); + assert.deepEqual(entries.get('files/icon.png'), png); +}); + +test('a KMZ that inflates past the limit is refused', async () => { + const archive = zip([{ name: 'doc.kml', data: new Uint8Array(200_000) }]); + await assert.rejects( + readZipEntries(archive, { maxInflatedBytes: 100_000 }), + /more than the import limit/, + ); +}); + +test('encrypted, damaged and non-zip archives are refused with a reason', async () => { + await assert.rejects( + readZipEntries( + zip([{ name: 'doc.kml', data: bytes(''), flags: 1, method: 0 }]), + ), + /encrypted/, + ); + await assert.rejects( + readZipEntries(bytes('not a zip at all, just text')), + /not a readable zip/, + ); + const truncated = zip([ + { name: 'doc.kml', data: bytes(''), method: 0 }, + ]); + const view = new DataView(truncated.buffer); + view.setUint32(truncated.length - 6, 9999, true); // directory offset past the end + await assert.rejects(readZipEntries(truncated), GeoImportError); + await assert.rejects( + readZipEntries( + zip([{ name: 'doc.kml', data: bytes(''), method: 12 }]), + ), + /compression method/, + ); +}); + +test('the main KML is doc.kml, else the shallowest .kml', () => { + const e = (...names) => new Map(names.map((n) => [n, new Uint8Array(0)])); + assert.equal(kmzMainDocument(e('a/b.kml', 'doc.kml')), 'doc.kml'); + assert.equal( + kmzMainDocument(e('deep/x/z.kml', 'top.kml', 'images/a.png')), + 'top.kml', + ); + assert.equal(kmzMainDocument(e('images/a.png')), null); +}); + +test('KMZ hrefs resolve only to entries inside the archive', () => { + const entries = new Map([ + ['doc.kml', new Uint8Array(0)], + ['files/icon one.png', new Uint8Array(0)], + ['overlay.jpg', new Uint8Array(0)], + ]); + assert.equal( + resolveKmzPath('files/icon%20one.png', 'doc.kml', entries), + 'files/icon one.png', + ); + assert.equal( + resolveKmzPath('./overlay.jpg', 'doc.kml', entries), + 'overlay.jpg', + ); + assert.equal( + resolveKmzPath('../overlay.jpg', 'files/sub.kml', entries), + 'overlay.jpg', + ); + assert.equal( + resolveKmzPath('https://example.com/overlay.jpg', 'doc.kml', entries), + null, + ); + assert.equal( + resolveKmzPath('//example.com/overlay.jpg', 'doc.kml', entries), + null, + ); + assert.equal(resolveKmzPath('file:///etc/passwd', 'doc.kml', entries), null); + assert.equal(resolveKmzPath('missing.png', 'doc.kml', entries), null); + assert.equal(resolveKmzPath('%E0%A4%A', 'doc.kml', entries), null); + assert.equal(kmzImageType('files/a.PNG'), 'image/png'); + assert.equal(kmzImageType('files/a.svg'), null); +}); + +/* ── a tiny element tree with the DOM surface the readers use ─────────── */ +function xmlEl(localName, attrs = {}, kids = []) { + const el = { + nodeType: 1, + localName, + namespaceURI: 'http://www.opengis.net/kml/2.2', + childNodes: [], + getAttribute: (n) => (n in attrs ? String(attrs[n]) : null), + get attributes() { + return Object.entries(attrs); + }, + setAttribute: (n, value) => { + attrs[n] = value; + }, + removeAttribute: (n) => { + delete attrs[n]; + }, + cloneNode() { + return xmlEl( + localName, + { ...attrs }, + text ?? el.childNodes.map((child) => child.cloneNode(true)), + ); + }, + getElementsByTagName(name) { + return xmlDoc(this).getElementsByTagName(name).slice(1); + }, + appendChild(child) { + child.parentNode?.removeChild(child); + child.parentNode = this; + this.childNodes.push(child); + }, + replaceChild(next, previous) { + const index = this.childNodes.indexOf(previous); + assert.ok(index >= 0); + next.parentNode?.removeChild(next); + this.childNodes[index] = next; + next.parentNode = this; + previous.parentNode = null; + }, + removeChild(child) { + this.childNodes.splice(this.childNodes.indexOf(child), 1); + child.parentNode = null; + }, + }; + for (const k of typeof kids === 'string' ? [] : kids) { + k.parentNode = el; + el.childNodes.push(k); + } + let text = typeof kids === 'string' ? kids : null; + Object.defineProperty(el, 'firstChild', { + get: () => el.childNodes[0] || null, + }); + Object.defineProperty(el, 'textContent', { + get: () => text ?? el.childNodes.map((c) => c.textContent).join(''), + set: (value) => { + text = value; + el.childNodes = []; + }, + }); + return el; +} +const xmlDoc = (root) => ({ + documentElement: root, + createElementNS: (_ns, name) => xmlEl(name), + getElementsByTagName(name) { + assert.equal(name, '*'); + const out = []; + const walk = (el) => { + out.push(el); + el.childNodes.forEach(walk); + }; + walk(root); + return out; + }, +}); +const serializeStyle = (el) => + JSON.stringify({ + name: el.localName, + attributes: el.attributes, + text: el.childNodes.length ? null : el.textContent, + children: el.childNodes.map(serializeStyle), + }); + +test('repeated inline styles are shared without replacing their inherited styleUrl', () => { + const style = () => + xmlEl('Style', {}, [xmlEl('IconStyle', {}, [xmlEl('scale', {}, '0.6')])]); + const placemarks = Array.from({ length: 2100 }, () => + xmlEl('Placemark', {}, [xmlEl('styleUrl', {}, '#base'), style()]), + ); + const doc = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Document', {}, [ + xmlEl('Style', { id: 'gev-import-style-1' }, [xmlEl('LineStyle')]), + ...placemarks, + ]), + ]), + ); + assert.equal(countKmlGeometry(doc).over, 'links'); + assert.equal(deduplicateKmlStyles(doc, { serializeStyle }), 2100); + assert.equal(countKmlGeometry(doc).over, ''); + const styles = doc + .getElementsByTagName('*') + .filter((el) => el.localName === 'Style'); + assert.equal(styles.length, 2); + assert.equal(styles[1].getAttribute('id'), 'gev-import-style-2'); + for (const placemark of placemarks) { + assert.equal(placemark.childNodes[0].textContent, '#base'); + assert.equal(placemark.childNodes[1].localName, 'StyleMap'); + assert.deepEqual( + placemark.childNodes[1].childNodes.map((pair) => + pair.childNodes.map((el) => el.textContent), + ), + [ + ['normal', '#gev-import-style-2'], + ['highlight', '#gev-import-style-2'], + ], + ); + } +}); + +test('identical named styles redirect existing references and Pair styles remain valid', () => { + const style = (id) => + xmlEl('Style', id ? { id } : {}, [ + xmlEl('LineStyle', {}, [xmlEl('width', {}, '3')]), + ]); + const reference = xmlEl('styleUrl', {}, '#b'); + const pair = xmlEl('Pair', {}, [xmlEl('key', {}, 'normal'), style()]); + const doc = xmlDoc( + xmlEl('kml', {}, [ + style('a'), + style('b'), + xmlEl('StyleMap', { id: 'map' }, [pair]), + xmlEl('Placemark', {}, [reference]), + ]), + ); + assert.equal(deduplicateKmlStyles(doc, { serializeStyle }), 3); + assert.equal(doc.documentElement.firstChild.localName, 'Document'); + assert.equal(reference.textContent, '#gev-import-style-1'); + assert.equal(pair.childNodes[1].localName, 'styleUrl'); + assert.equal(pair.childNodes[1].textContent, '#gev-import-style-1'); + assert.equal( + doc.getElementsByTagName('*').filter((el) => el.localName === 'Style') + .length, + 1, + ); +}); + +test('random colors and ambiguous style IDs retain independent evaluations and precedence', () => { + const random = () => + xmlEl('Style', {}, [ + xmlEl('IconStyle', {}, [xmlEl('colorMode', {}, 'random')]), + ]); + const named = (width) => + xmlEl('Style', { id: 'collision' }, [ + xmlEl('LineStyle', {}, [xmlEl('width', {}, width)]), + ]); + const original = [random(), random(), named('1'), named('1'), named('2')]; + const doc = xmlDoc(xmlEl('kml', {}, original)); + assert.equal(deduplicateKmlStyles(doc, { serializeStyle }), 0); + assert.deepEqual(doc.documentElement.childNodes, original); +}); + +test('distinct style definitions and named StyleMaps still have a pre-loader budget', () => { + const doc = xmlDoc( + xmlEl( + 'kml', + {}, + ['1', '2', '3'].map((width) => + xmlEl('Style', {}, [ + xmlEl('LineStyle', {}, [xmlEl('width', {}, width)]), + ]), + ), + ), + ); + assert.throws( + () => deduplicateKmlStyles(doc, { serializeStyle, maxStyles: 2 }), + GeoImportError, + ); + const maps = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('StyleMap', { id: 'a' }), + xmlEl('StyleMap', { id: 'b' }), + ]), + ); + assert.equal(countKmlGeometry(maps, { maxLinks: 1 }).over, 'links'); +}); + +test('KML extended metadata is removed before the document reaches the loader', () => { + const point = xmlEl('Point', {}, [xmlEl('coordinates', {}, '1,2,0')]); + const metadata = xmlEl('ExtendedData', {}, [ + xmlEl('Data', { name: 'note' }, [xmlEl('value', {}, 'ordinary metadata')]), + ]); + const placemark = xmlEl('Placemark', {}, [metadata, point]); + const doc = xmlDoc(xmlEl('kml', {}, [placemark])); + assert.equal(sanitizeKmlDocument(doc).removed, 1); + assert.deepEqual(placemark.childNodes, [point]); + assert.equal(metadata.parentNode, null); +}); +test('blocked icon sources fall back to local pins without changing intentional empty icons', () => { + const blocked = xmlEl('Icon', {}, [ + xmlEl('href', {}, 'https://example.invalid/icon.png'), + ]); + const blank = xmlEl('Icon'); + const embedded = xmlEl('Icon', {}, [ + xmlEl('href', {}, 'data:image/png;base64,AAAA'), + ]); + const packed = xmlEl('Icon', {}, [xmlEl('href', {}, 'packed.png')]); + const iconStyle = xmlEl('IconStyle', {}, [ + xmlEl('scale', {}, '0.3'), + blocked, + ]); + const doc = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Style', {}, [iconStyle]), + ...[blank, embedded, packed].map((icon) => + xmlEl('Style', {}, [xmlEl('IconStyle', {}, [icon])]), + ), + ]), + ); + assert.equal( + sanitizeKmlDocument(doc, (href) => + href === 'packed.png' ? 'blob:local-image' : null, + ).removed, + 1, + ); + assert.equal(blocked.parentNode, null); + assert.equal(iconStyle.childNodes[0].textContent, '0.3'); + assert.equal(blank.parentNode.localName, 'IconStyle'); + assert.equal( + embedded.childNodes[0].textContent, + 'data:image/png;base64,AAAA', + ); + assert.equal(packed.childNodes[0].textContent, 'blob:local-image'); +}); + +const trkpt = (lat, lon, time) => + xmlEl('trkpt', { lat, lon }, time ? [xmlEl('time', {}, time)] : []); + +test('GPX keeps each track segment as its own line, names as text, no times', () => { + const doc = xmlDoc( + xmlEl('gpx', {}, [ + xmlEl('wpt', { lat: 37.77, lon: -122.49 }, [xmlEl('name', {}, 'Camp')]), + xmlEl('wpt', { lat: 99, lon: 0 }), // out of range: skipped + xmlEl('rte', {}, [ + xmlEl('name', {}, 'Route'), + xmlEl('rtept', { lat: 1, lon: 2 }), + xmlEl('rtept', { lat: 3, lon: 4 }), + ]), + xmlEl('trk', {}, [ + xmlEl('name', {}, 'Morning walk'), + xmlEl('trkseg', {}, [ + trkpt(10, 20, '2024-05-01T10:00:00Z'), + trkpt(10.1, 20.1, '2024-05-01T10:01:00Z'), + ]), + // A pause in the recording: a new segment, not a straight jump. + xmlEl('trkseg', {}, [ + trkpt(11, 21, '2024-05-01T11:00:00Z'), + trkpt(11.1, 21.1, '2024-05-01T11:01:00Z'), + trkpt(11.2, 21.2, '2024-05-01T11:02:00Z'), + ]), + xmlEl('trkseg', {}, [trkpt(12, 22)]), // one point: not a line + ]), + ]), + ); + const fc = gpxToGeoJson(doc); + assert.deepEqual( + fc.features.map((f) => [f.properties.name, f.geometry.type]), + [ + ['Camp', 'Point'], + ['Route', 'LineString'], + ['Morning walk', 'MultiLineString'], + ], + ); + assert.deepEqual(fc.features[2].geometry.coordinates, [ + [ + [20, 10], + [20.1, 10.1], + ], + [ + [21, 11], + [21.1, 11.1], + [21.2, 11.2], + ], + ]); + assert.doesNotMatch(JSON.stringify(fc), /2024|time/); + assert.throws(() => gpxToGeoJson(xmlDoc(xmlEl('kml'))), GeoImportError); +}); + +test('GPX past the point limit is refused while reading, before Cesium', () => { + const seg = xmlEl( + 'trkseg', + {}, + Array.from({ length: 30 }, (_, i) => trkpt(1 + i / 100, 2)), + ); + const doc = xmlDoc(xmlEl('gpx', {}, [xmlEl('trk', {}, [seg])])); + assert.throws( + () => gpxToGeoJson(doc, { maxPositions: 20 }), + (e) => e instanceof GeoImportError && /more than .* points/.test(e.message), + ); + assert.throws( + () => + gpxToGeoJson( + xmlDoc( + xmlEl('gpx', {}, [ + xmlEl('wpt', { lat: 1, lon: 1 }), + xmlEl('wpt', { lat: 2, lon: 2 }), + ]), + ), + { maxFeatures: 1 }, + ), + /more than .* features/, + ); +}); + +test('GeoJSON features and vertices are counted before loading, without recursion', () => { + const line = Array.from({ length: 50 }, (_, i) => [i / 10, 1]); + const fc = { + type: 'FeatureCollection', + features: [ + { type: 'Feature', geometry: { type: 'LineString', coordinates: line } }, + { + type: 'Feature', + geometry: { + type: 'GeometryCollection', + geometries: [ + { type: 'Point', coordinates: [1, 2] }, + { + type: 'Polygon', + coordinates: [ + [ + [0, 0], + [1, 0], + [1, 1], + [0, 0], + ], + ], + }, + ], + }, + }, + { type: 'Feature', geometry: null }, + ], + }; + assert.deepEqual(countGeoJsonGeometry(fc), { + features: 3, + positions: 55, + entities: 4, + over: '', + }); + assert.equal( + countGeoJsonGeometry(fc, { maxPositions: 54 }).over, + 'positions', + ); + assert.equal(countGeoJsonGeometry(fc, { maxFeatures: 2 }).over, 'features'); + assert.deepEqual( + countGeoJsonGeometry({ type: 'Point', coordinates: [1, 2] }), + { + features: 1, + positions: 1, + entities: 1, + over: '', + }, + ); + // A very long line is counted iteratively and stops at the limit. + const huge = { + type: 'LineString', + coordinates: Array.from({ length: MAX_IMPORT_POSITIONS + 5 }, () => [0, 0]), + }; + assert.equal(countGeoJsonGeometry(huge).over, 'positions'); +}); + +test('multipart GeoJSON is refused before loading when it expands past the entity cap', () => { + const parts = Array.from({ length: MAX_IMPORT_ENTITIES + 1 }, () => [1, 2]); + for (const geometry of [ + { type: 'MultiPoint', coordinates: parts }, + { + type: 'GeometryCollection', + geometries: parts.map((coordinates) => ({ type: 'Point', coordinates })), + }, + { type: 'MultiLineString', coordinates: parts.map((p) => [p, [2, 3]]) }, + ]) { + const size = countGeoJsonGeometry({ type: 'Feature', geometry }); + assert.equal(size.over, 'entities'); + } +}); + +test('GPX points with missing or blank coordinates are skipped rather than placed at zero', () => { + const doc = xmlDoc( + xmlEl('gpx', {}, [ + xmlEl('wpt', { lon: 2 }), + xmlEl('wpt', { lat: 1 }), + xmlEl('wpt', { lat: '', lon: 2 }), + xmlEl('wpt', { lat: 1, lon: ' ' }), + xmlEl('wpt', { lat: 0, lon: 0 }), + ]), + ); + assert.equal(gpxToGeoJson(doc).features.length, 1); + assert.deepEqual(gpxToGeoJson(doc).features[0].geometry.coordinates, [0, 0]); +}); + +test('KML placemarks and coordinate tuples are counted before loading', () => { + const coords = (n) => + xmlEl( + 'coordinates', + {}, + Array.from({ length: n }, (_, i) => `${i},1,0`).join('\n '), + ); + const doc = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Placemark', {}, [xmlEl('LineString', {}, [coords(40)])]), + xmlEl('Placemark', {}, [ + xmlEl('Track', {}, [ + xmlEl('coord', {}, '1 2 0'), + xmlEl('coord', {}, '1 3 0'), + ]), + ]), + ]), + ); + assert.deepEqual(countKmlGeometry(doc), { + features: 2, + entities: 2, + positions: 42, + links: 0, + over: '', + }); + assert.equal(countKmlGeometry(doc, { maxPositions: 41 }).over, 'positions'); + assert.equal(countKmlGeometry(doc, { maxFeatures: 1 }).over, 'features'); +}); + +test('KML vertex counting recognizes the Unicode whitespace Cesium accepts', () => { + for (const separator of ['\u00a0', '\u2003', '\ufeff']) { + const doc = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Placemark', {}, [ + xmlEl('LineString', {}, [ + xmlEl('coordinates', {}, ['1,2', '3,4', '5,6'].join(separator)), + ]), + ]), + ]), + ); + assert.equal(countKmlGeometry(doc).positions, 3); + assert.equal(countKmlGeometry(doc, { maxPositions: 2 }).over, 'positions'); + } +}); + +test('KML multipart geometries count every Cesium entity before loading', () => { + const doc = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Placemark', {}, [ + xmlEl('MultiGeometry', {}, [ + xmlEl('Point'), + xmlEl('MultiGeometry', {}, [xmlEl('Point'), xmlEl('LineString')]), + xmlEl('MultiTrack', {}, [xmlEl('Track'), xmlEl('Track')]), + ]), + ]), + xmlEl('GroundOverlay'), + ]), + ); + assert.equal(countKmlGeometry(doc).entities, 9); + assert.equal(countKmlGeometry(doc, { maxEntities: 8 }).over, 'entities'); + const oversized = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Placemark', {}, [ + xmlEl( + 'MultiGeometry', + {}, + Array.from({ length: MAX_IMPORT_ENTITIES }, () => xmlEl('Point')), + ), + ]), + ]), + ); + assert.equal(countKmlGeometry(oversized).over, 'entities'); +}); + +test('KML folders contribute to the pre-loader entity budget', () => { + const doc = xmlDoc( + xmlEl('kml', {}, [ + xmlEl('Folder', {}, [xmlEl('Folder', {}, [xmlEl('Placemark')])]), + ]), + ); + assert.equal(countKmlGeometry(doc).entities, 3); + assert.equal(countKmlGeometry(doc, { maxEntities: 2 }).over, 'entities'); +}); + +test('KML style definitions and distinct icon links have their own limit', () => { + const style = () => + xmlEl('Style', {}, [ + xmlEl('IconStyle', {}, [xmlEl('Icon', {}, [xmlEl('href', {}, 'a.png')])]), + ]); + const doc = xmlDoc(xmlEl('kml', {}, [style(), style(), style()])); + assert.equal(countKmlGeometry(doc).links, 4); + assert.equal(countKmlGeometry(doc, { maxLinks: 3 }).over, 'links'); + const distinct = xmlDoc( + xmlEl( + 'kml', + {}, + ['a.png', 'b.png', 'c.png'].map((href) => xmlEl('href', {}, href)), + ), + ); + assert.equal(countKmlGeometry(distinct, { maxLinks: 2 }).over, 'links'); +}); + +test('a KMZ image gets one blob URL however many hrefs name it', () => { + const entries = new Map([ + ['doc.kml', new Uint8Array(0)], + ['a.png', new Uint8Array(8)], + ['b.png', new Uint8Array(8)], + ]); + const made = []; + const kept = []; + const resolve = kmzImageResolver( + entries, + 'doc.kml', + (url) => kept.push(url), + (blob) => { + made.push(blob); + return `blob:test/${made.length}`; + }, + ); + for (let i = 0; i < 1000; i++) assert.equal(resolve('a.png'), 'blob:test/1'); + assert.equal(resolve('./b.png'), 'blob:test/2'); + assert.equal(resolve('a.png'), 'blob:test/1'); + assert.equal(resolve('https://example.com/a.png'), null); + assert.equal(resolve('doc.kml'), null); + assert.equal(made.length, 2); + assert.deepEqual(kept, ['blob:test/1', 'blob:test/2']); +}); + +test('imported entities keep only a short plain-text name: no description, label or properties', () => { + const ds = new Cesium.CustomDataSource('t'); + const e = ds.entities.add({ + name: 'Trail\u0007 head\n' + 'x'.repeat(300), + description: '', + label: { text: 'Trail head' }, + properties: { note: '' }, + position: Cesium.Cartesian3.fromDegrees(1, 2), + }); + e.kml = { extendedData: { note: { value: '