From ffb332472d00e21aceb6d24e4f3ae97c8dbcff78 Mon Sep 17 00:00:00 2001 From: Matt Van Horn Date: Mon, 12 Oct 2026 19:35:29 +0000 Subject: [PATCH 01/83] feat(layers): add FAA drone restrictions airspace layer Show US prohibited and restricted special-use airspace from the keyless FAA SUA FeatureServer as an off-by-default Infrastructure layer, with share-link token 3 and an informational-only disclaimer. --- CHANGELOG.md | 8 + DATA_SOURCES.md | 2 + README.md | 3 +- docs/CURRENT-STATE.md | 25 +- package.json | 2 + scripts/format-scope.json | 11 + scripts/package-boundaries.json | 32 ++ server/providers/droneAirspace.js | 135 ++++++++ server/providers/local.js | 2 + src/app/constructCatalog.js | 4 + src/app/constructCatalog.test.mjs | 5 +- src/app/layers/droneAirspace.js | 22 ++ src/app/sourceComposition.js | 1 + src/app/sourceComposition.test.mjs | 2 +- src/data/dataCredits.js | 7 + src/data/droneAirspaceProxy.test.mjs | 146 +++++++++ src/data/layerState.js | 5 + src/data/layerState.test.mjs | 34 +- src/data/layerStateTokenLedger.test.mjs | 62 ++-- src/data/layerStateTokenReservations.json | 1 + src/firstRunExperience.test.mjs | 5 +- src/layers/droneAirspace/cards.js | 93 ++++++ src/layers/droneAirspace/cards.test.mjs | 74 +++++ src/layers/droneAirspace/index.js | 326 ++++++++++++++++++++ src/layers/droneAirspace/ownership.test.mjs | 161 ++++++++++ src/layers/droneAirspace/records.js | 97 ++++++ src/layers/droneAirspace/records.test.mjs | 95 ++++++ src/layers/droneAirspace/source.js | 23 ++ src/layers/droneAirspace/source.test.mjs | 105 +++++++ src/sources/reference.js | 2 + src/sources/reference.test.mjs | 3 + src/tooling/previewServing.test.mjs | 1 + src/ui/layerPanel.js | 1 + src/voice/layerManifest.js | 24 ++ src/voice/layerManifest.test.mjs | 128 ++++++-- 35 files changed, 1573 insertions(+), 74 deletions(-) create mode 100644 server/providers/droneAirspace.js create mode 100644 src/app/layers/droneAirspace.js create mode 100644 src/data/droneAirspaceProxy.test.mjs create mode 100644 src/layers/droneAirspace/cards.js create mode 100644 src/layers/droneAirspace/cards.test.mjs create mode 100644 src/layers/droneAirspace/index.js create mode 100644 src/layers/droneAirspace/ownership.test.mjs create mode 100644 src/layers/droneAirspace/records.js create mode 100644 src/layers/droneAirspace/records.test.mjs create mode 100644 src/layers/droneAirspace/source.js create mode 100644 src/layers/droneAirspace/source.test.mjs diff --git a/CHANGELOG.md b/CHANGELOG.md index b24ac95ed..90402cad5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +- Add a Drone Restrictions layer for FAA prohibited (P) and restricted (R) + special-use airspace. The keyless `/api/drone-airspace` proxy pages the + public SUA FeatureServer, keeps a one-hour snapshot, and serves stale data + when a refresh fails. Zones draw as ground polygons (red prohibited, orange + restricted), selection shows name, altitudes, times of use and agency, and + the row legend states the layer is informational only. Off by default; + share-link token `3`. + - Treat explicitly current, complete empty vessel coverage as a successful update, clearing obsolete contacts while retaining stale or incomplete snapshots. diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 0b8eac358..2aa957e2b 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -35,6 +35,7 @@ How to read this: | **NASA GIBS** (WMTS tiles + Worldview Snapshots) and **NASA CMR** (granule catalog) | Recent Imagery layer: HLS S30/L30 (Sentinel-2, Landsat 8/9) and VIIRS true-colour tiles draped on the map, per-day thumbnails and PNG exports; the HLS granule catalog (dates, cloud, footprints) for a selected box | U.S. public domain (NASA data); keyless, browser-direct (CORS `*`); no bulk use — one 30-day catalog query per box, tiles bounded to the box, six tile requests in flight | "We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS)." — [gibs.earthdata.nasa.gov](https://gibs.earthdata.nasa.gov), [HLS](https://lpdaac.usgs.gov/products/hlss30v002/) | | **NIFC WFIGS** (Wildland Fire Interagency Geospatial Services, ArcGIS feature service) | Fire Perimeters layer: current interagency wildfire incident perimeters, containment, and incident facts | U.S. public domain (interagency wildland-fire data published through the [NIFC Open Data portal](https://data-nifc.opendata.arcgis.com/)); fetched keyless through `/api/fire-perimeters` (5-minute server cache) with `exceededTransferLimit` paging | "Wildfire perimeters: National Interagency Fire Center (WFIGS)" | | **InciWeb** (inciweb.wildfire.gov) | Per-incident "InciWeb ↗" card links to official incident pages (narratives, evacuation/closure notices) | U.S. government public incident information. Links are matched best-effort by incident name and state, with incident page origin and update times checked on selection. Same-origin `/api/fire-perimeters/inciweb` routes cache the catalog for 1 hour and publication metadata for 30 minutes; linked pages remain InciWeb content | "Incident information: InciWeb (inciweb.wildfire.gov)" | +| **FAA Special Use Airspace** (`services6.arcgis.com` FeatureServer `Special_Use_Airspace`) | Drone Restrictions layer: US prohibited (P) and restricted (R) special-use airspace polygons, altitude limits, times of use, and controlling agency | U.S. public domain (FAA aeronautical data published through the [FAA AIS Open Data portal](https://adds-faa.opendata.arcgis.com/)); fetched keyless through `/api/drone-airspace` (1-hour server cache). Informational only — not a substitute for official UAS flight-planning tools | "Drone restrictions: FAA Special Use Airspace" | | **OpenFreeMap / OpenMapTiles** | Vector tiles | OpenStreetMap data under ODbL 1.0 | OpenFreeMap © OpenMapTiles | | **Overture Maps Foundation** | Military area names pack | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | Overture Maps Foundation | | **TomTom Traffic API** (flow vector tiles) | Traffic roads and live congestion (optional, BYOK) | [TomTom for Developers terms](https://docs.tomtom.com/legal/terms-and-conditions) (proprietary, your own key; free tier currently 200K tile requests/month — see [current pricing](https://docs.tomtom.com/pricing/)) | "Traffic flow data © TomTom" — registered when live mode activates | @@ -98,6 +99,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei - **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line. - **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable. - **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit. +- **FAA Special Use Airspace.** Keyless. `/api/drone-airspace` queries the public FAA AIS FeatureServer for prohibited (P) and restricted (R) zones only — not MOAs, TFRs or other SUA types — with simplified geometry, a body cap, paging, a one-hour cache and stale-on-error. The layer is US coverage, informational only, and is not a substitute for B4UFLY, LAANC or a preflight briefing. Data is fetched at request time and never stored or redistributed. US government public domain; courtesy attribution is registered in the Data attribution popover. - **Mapped installations.** OpenFreeMap military polygons use a bundled Overture/OSM name index; wide views show named points. Optional operator-configured Overpass and explicit Google Places searches keep their own source paths. Mapped data does not establish current activity or complete coverage. - **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon's public instance, then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` uses the shared Nominatim forward-search queue at no more than one request per second, sends an identifying `User-Agent` and `Referer`, caches answers for five minutes, shares one upstream call between identical searches already in flight, bounds the queue so a burst is refused rather than held, and drops a queued search whose caller has given up. It asks for one result per search and is not used for systematic or bulk queries. - **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and resolves physical regions from bundled public-domain Natural Earth polygons without reverse-geocoding requests. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit. diff --git a/README.md b/README.md index 5d97dfb2f..6b8043e5a 100644 --- a/README.md +++ b/README.md @@ -289,7 +289,7 @@ _Ask for radio near anywhere and the globe starts broadcasting — every station ## 🛰️ What's on the Globe -Twenty layers and map sources. **Seventeen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.) +Twenty-one layers and map sources. **Eighteen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.) | Layer | What you get | Source | Auth | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------- | @@ -310,6 +310,7 @@ Twenty layers and map sources. **Seventeen have a keyless path.** Some offer add | 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 | | 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🟡 optional token raises the allowance) | | 🎖️ **Mapped Installations** | Viewport-bounded military-site context from community mapping — incomplete by nature, and labeled that way | OpenStreetMap | 🟢 | +| 🚫 **Drone Restrictions** | FAA prohibited (P) and restricted (R) special-use airspace as ground polygons, with name, altitudes and times of use on select — informational only, not for flight planning | FAA SUA | 🟢 | | 🌬️ **Wind** | Animated 10 m forecast wind with an optional speed, temperature or pressure color field and a reading at map center | NOAA GFS / ECMWF IFS | 🟢 | | 🌦️ **Observed Weather** | Rain radar (contiguous US), Satellite clouds (GOES regional or global infrared, Clouds only or Full image) and Lightning density (Americas and Pacific) on one history timeline | NOAA nowCOAST | 🟢 | | 🌀 **Cyclones** | Advisory positions, forecast tracks and uncertainty cones for the Atlantic and eastern/central North Pacific | NOAA NHC / CPHC | 🟢 | diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 26d56e66b..2fc40751a 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1,5 +1,19 @@ # God's Eye View Current State +## Drone Restrictions — FAA special-use airspace + +The Infrastructure group includes **Drone Restrictions** (`drone-airspace`, +share-link token `3`), off by default. `/api/drone-airspace` pages the public +FAA Special Use Airspace FeatureServer for `TYPE_CODE` P (prohibited) and R +(restricted), simplifies geometry, caches for an hour, and serves the last +good snapshot as stale when a refresh fails. The globe draws ground-clamped +outlines with a translucent fill: red for prohibited, orange for restricted. +Selecting a zone shows name, altitudes, times of use and controlling agency, +plus a notice that the layer is informational only and not a substitute for +official UAS flight-planning tools. Pick ownership is unregistered so sibling +layer cards still work over a zone. Voice can toggle the layer and query the +loaded records. See the live-layers table below. + Vessel sources can report healthy empty coverage using a current, complete snapshot with zero raw rows, a healthy transport and a positive last-message time. This clears obsolete contacts and settles startup without manufacturing @@ -335,7 +349,7 @@ Consult the linked official advisory for safety decisions. Voice and HUD snapshots reuse the existing feedState classifier. Analyst follow-ups retain their original data provenance; current-view results append provenance without replacing legacy fields. HUD context and deterministic telemetry include non-nominal feed state. -Analyst records come on demand from each layer's `getAnalystRecords`: flights, military, local ADS-B, vessels, satellites, launches, earthquakes, FIRMS (VIIRS and MODIS, with a `sensor` field), fire perimeters, cyclones, transit, bikeshare, ALPR cameras, mapped installations, datacenters and dams. `analyst_query` counts the whole loaded set (up to 250,000 records per layer; past that `coverage.records` reports `returned`, `total` and `truncated: true` and the result carries `complete: false`). Ranking keeps a top-k and scans in 20,000-row slices that yield to the page and stop when superseded. It refuses unknown fields, operators, units that do not convert (`altitudeM(ft)` converts), wrong-typed values, invalid centres, unknown scope kinds and sort fields with `code` and `allowed`, refuses a follow-up with no previous answer (`NO_RESULT_CONTEXT`) or naming other layers than that answer (`FOLLOW_UP_MISMATCH`), and tells `LAYER_OFF`, `NOT_READY` and `FEED_UNAVAILABLE` apart from a real zero; when only some layers can answer the result is `partial` with `unanswered` ids. The Contacts-window path applies the same filters, ranking and item shape to one immutable panel snapshot. Each aircraft layer retains at most 20,000 rows; reaching that bound sets `complete: false` and the spoken/card count becomes “At least”, never an exact claim. A superseded query never replaces follow-up memory, while sibling analyst calls from one assistant response do not cancel one another. Items carry `lat`/`lon`; the feed window (earthquakes and FIRMS: last 24 h), the precise scope and any caveat sit in `display`, which is on-screen detail rather than something the model recites. "In view" is still a radius around the view centre, not the visible footprint. Satellite distance is ground distance. +Analyst records come on demand from each layer's `getAnalystRecords`: flights, military, local ADS-B, vessels, satellites, launches, earthquakes, FIRMS (VIIRS and MODIS, with a `sensor` field), fire perimeters, drone restrictions, cyclones, transit, bikeshare, ALPR cameras, mapped installations, datacenters and dams. `analyst_query` counts the whole loaded set (up to 250,000 records per layer; past that `coverage.records` reports `returned`, `total` and `truncated: true` and the result carries `complete: false`). Ranking keeps a top-k and scans in 20,000-row slices that yield to the page and stop when superseded. It refuses unknown fields, operators, units that do not convert (`altitudeM(ft)` converts), wrong-typed values, invalid centres, unknown scope kinds and sort fields with `code` and `allowed`, refuses a follow-up with no previous answer (`NO_RESULT_CONTEXT`) or naming other layers than that answer (`FOLLOW_UP_MISMATCH`), and tells `LAYER_OFF`, `NOT_READY` and `FEED_UNAVAILABLE` apart from a real zero; when only some layers can answer the result is `partial` with `unanswered` ids. The Contacts-window path applies the same filters, ranking and item shape to one immutable panel snapshot. Each aircraft layer retains at most 20,000 rows; reaching that bound sets `complete: false` and the spoken/card count becomes “At least”, never an exact claim. A superseded query never replaces follow-up memory, while sibling analyst calls from one assistant response do not cancel one another. Items carry `lat`/`lon`; the feed window (earthquakes and FIRMS: last 24 h), the precise scope and any caveat sit in `display`, which is on-screen detail rather than something the model recites. "In view" is still a radius around the view centre, not the visible footprint. Satellite distance is ground distance. Keyless terrain tiles retry when Re:Earth throttles them. The browser fetches `terrain.reearth.land/cesium-mesh/ellipsoid/{z}/{x}/{y}.terrain` directly; no @@ -3070,9 +3084,18 @@ its criteria cannot be silently ignored. | Street Level 📷 | Street-level imagery; Mapillary is the first provider (vector tiles, Graph API, MapillaryJS) | `src/layers/streetLevel/` via `src/app/layers/streetLevel.js` | `/api/mapillary/status`, `/api/mapillary/tiles/coverage/{z}/{x}/{y}` (`MAPILLARY_CLIENT_TOKEN`) | camera-driven (320 ms debounce, ≤9 tiles, cached 24 h) | | Wind 🌬 | NOAA GFS 10 m wind (keyless, 0.25°→1° grid; animated particles) | `src/data/wind.js` | `/api/wind` | 1 h (forecast cycle) | | Fire Perimeters 🔥 | NIFC WFIGS current interagency perimeters (keyless, paged past the 2000-record cap); InciWeb catalog + incident page origin and update time checks for verified incident-page links | `src/layers/perimeters/` via `src/app/layers/perimeters.js` | `/api/fire-perimeters` + `/api/fire-perimeters/inciweb/*` | 5 min (server caches: catalog 1 h; publication 30 min) | +| Drone Restrictions 🚫 | FAA Special Use Airspace prohibited (P) and restricted (R) zones (keyless FeatureServer, simplified geometry) | `src/layers/droneAirspace/` via `src/app/layers/droneAirspace.js` | `/api/drone-airspace` | 1 h (server cache 1 h; stale-on-error) | Fire Perimeters uses capped, timed server reads with stale-on-error caching and a per-client limit. Unchanged snapshots retain geometry; link checks cancel on disable or selection change, and the row legend shows reported containment. +Drone Restrictions is off by default (share-link token `3`) and sits in +Infrastructure. The proxy pages `TYPE_CODE IN ('P','R')` from the FAA SUA +FeatureServer, caps the body, and reuses a one-hour snapshot, serving the last +good payload as stale when a refresh fails. Client-supplied URLs are rejected. +Polygons keep holes; pick ownership is deliberately unregistered so sibling +cards still work over a zone. The row legend and selected card state that the +layer is informational only and not a substitute for B4UFLY / LAANC. + Street Level is a collapsible right-rail panel (`#street-level-panel`, layer token `0`, option owner `street-level`, panel `ui` token `t`) that starts collapsed. It opens itself only when the user, voice or a tool switches the diff --git a/package.json b/package.json index d9bf6cf8a..ca8a972ce 100644 --- a/package.json +++ b/package.json @@ -165,6 +165,7 @@ "./layers/recent-imagery": "./src/layers/recentImagery/index.js", "./layers/recent-imagery/source": "./src/layers/recentImagery/source.js", "./layers/perimeters": "./src/layers/perimeters/index.js", + "./layers/drone-airspace": "./src/layers/droneAirspace/index.js", "./sources/firms-csv": "./src/data/firmsCsv.js", "./layers/firms": "./src/layers/firms/index.js", "./layers/satellites": "./src/layers/satellites/index.js", @@ -238,6 +239,7 @@ "./layers/cctv/source": "./src/layers/cctv/source.js", "./layers/earthquakes/source": "./src/layers/earthquakes/source.js", "./layers/perimeters/source": "./src/layers/perimeters/source.js", + "./layers/drone-airspace/source": "./src/layers/droneAirspace/source.js", "./layers/firms/source": "./src/layers/firms/source.js", "./layers/installations/source": "./src/layers/installations/source.js", "./layers/launches/source": "./src/layers/launches/source.js", diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 6b5c9b544..0d72c1096 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -371,6 +371,14 @@ "src/layers/perimeters/cards.test.mjs", "src/layers/perimeters/inciweb.test.mjs", "src/layers/perimeters/ownership.test.mjs", + "src/layers/droneAirspace/index.js", + "src/layers/droneAirspace/records.js", + "src/layers/droneAirspace/source.js", + "src/layers/droneAirspace/cards.js", + "src/layers/droneAirspace/records.test.mjs", + "src/layers/droneAirspace/source.test.mjs", + "src/layers/droneAirspace/cards.test.mjs", + "src/layers/droneAirspace/ownership.test.mjs", "src/data/firmsHeatmap.js", "src/data/fireAnchors.js", "src/layers/firms/anchors.js", @@ -643,6 +651,7 @@ "src/app/layers/radio.js", "src/app/layers/earthquakes.js", "src/app/layers/perimeters.js", + "src/app/layers/droneAirspace.js", "src/app/layers/cctv.js", "src/app/layers/firms.js", "src/app/layers/satellites.js", @@ -824,8 +833,10 @@ "src/layers/weather/imageryOrder.test.mjs", "server/providers/cyclones.js", "server/providers/firePerimeters.js", + "server/providers/droneAirspace.js", "src/data/cyclonesProxy.test.mjs", "src/data/firePerimetersProxy.test.mjs", + "src/data/droneAirspaceProxy.test.mjs", "src/layers/cyclones/index.js", "src/layers/cyclones/rendering.js", "src/layers/cyclones/labels.js", diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index 497be5b08..ed1fb5c4d 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -2023,6 +2023,11 @@ "src/layers/perimeters/records.js", "src/layers/perimeters/cards.js", "src/layers/perimeters/inciweb.js", + "src/app/layers/droneAirspace.js", + "src/layers/droneAirspace/index.js", + "src/layers/droneAirspace/source.js", + "src/layers/droneAirspace/records.js", + "src/layers/droneAirspace/cards.js", "src/layers/cctv/sourcePolicy.js", "src/sources/reference.js", "src/search/nominatim.js", @@ -2602,6 +2607,11 @@ "src/layers/perimeters/records.js", "src/layers/perimeters/cards.js", "src/layers/perimeters/inciweb.js", + "src/app/layers/droneAirspace.js", + "src/layers/droneAirspace/index.js", + "src/layers/droneAirspace/source.js", + "src/layers/droneAirspace/records.js", + "src/layers/droneAirspace/cards.js", "src/layers/cctv/sourcePolicy.js", "src/search/nominatim.js", "src/nominatimGeocode.js", @@ -3097,6 +3107,8 @@ "src/sources/httpBody.js", "src/layers/perimeters/source.js", "src/layers/perimeters/records.js", + "src/layers/droneAirspace/source.js", + "src/layers/droneAirspace/records.js", "src/layers/submarineCables/bundledSource.js" ], "external": [] @@ -3317,6 +3329,26 @@ ], "external": [] }, + "drone-airspace": { + "exports": ["./layers/drone-airspace"], + "modules": [ + "src/layers/droneAirspace/index.js", + "src/sources/httpBody.js", + "src/layers/droneAirspace/source.js", + "src/layers/droneAirspace/records.js", + "src/layers/droneAirspace/cards.js" + ], + "external": ["cesium"] + }, + "drone-airspace-source": { + "exports": ["./layers/drone-airspace/source"], + "modules": [ + "src/layers/droneAirspace/records.js", + "src/sources/httpBody.js", + "src/layers/droneAirspace/source.js" + ], + "external": [] + }, "tools": { "exports": ["./tools"], "modules": [ diff --git a/server/providers/droneAirspace.js b/server/providers/droneAirspace.js new file mode 100644 index 000000000..3a1905fc9 --- /dev/null +++ b/server/providers/droneAirspace.js @@ -0,0 +1,135 @@ +import { normalizeDroneAirspaceSnapshot } from '../../src/layers/droneAirspace/records.js'; +import { readResponseJsonCapped, coalesceProxyRequest } from './common/http.js'; +import { makeRateLimiter, clientKey } from './common/rate-limit.js'; + +// FAA Special Use Airspace (public, keyless). TYPE_CODE P (prohibited) and +// R (restricted) only — the small, always-in-force slice of the national +// SUA table. maxAllowableOffset trades ~100 m of boundary fidelity for a +// payload small enough to refresh (~330 KB for the current P+R set). +const API_URL = + 'https://services6.arcgis.com/ssFJjBXIUyZDrSYZ/arcgis/rest/services/' + + 'Special_Use_Airspace/FeatureServer/0/query?' + + new URLSearchParams({ + where: "TYPE_CODE IN ('P','R')", + outFields: [ + 'NAME', + 'TYPE_CODE', + 'UPPER_DESC', + 'LOWER_DESC', + 'TIMESOFUSE', + 'CONT_AGENT', + 'CITY', + 'STATE', + 'OBJECTID', + ].join(','), + maxAllowableOffset: '0.001', + outSR: '4326', + f: 'geojson', + }).toString(); + +const MAX_PAGES = 5; +const MIB = 1024 * 1024; +const SNAPSHOT_TTL_MS = 3_600_000; + +const exceededTransferLimit = (payload) => + payload?.exceededTransferLimit === true || + payload?.properties?.exceededTransferLimit === true; + +/** Fixed-origin, bounded FAA SUA routes for dev and preview. */ +export function droneAirspaceProxy({ + fetchImpl = (...args) => globalThis.fetch(...args), + now = () => Date.now(), +} = {}) { + const cache = new Map(); + const inFlight = new Map(); + const allow = makeRateLimiter({ windowMs: 60_000, max: 60, globalMax: 1200 }); + + async function upstream(url, cap, timeout) { + const signal = AbortSignal.timeout(timeout); + const response = await fetchImpl(url, { + signal, + redirect: 'error', + }); + if (!response.ok) { + await response.body?.cancel(); + throw new Error('upstream_unavailable'); + } + return readResponseJsonCapped(response, cap, signal); + } + + async function fetchZones() { + const features = []; + for (let page = 0; page < MAX_PAGES; page++) { + const url = + page === 0 ? API_URL : `${API_URL}&resultOffset=${features.length}`; + const payload = await upstream(url, 8 * MIB, 30_000); + if (!Array.isArray(payload?.features)) + throw new Error('invalid_snapshot'); + features.push(...payload.features); + if (!exceededTransferLimit(payload) || !payload.features.length) break; + } + const rows = normalizeDroneAirspaceSnapshot({ features }); + if (!rows) throw new Error('invalid_snapshot'); + return { fetchedAt: now(), rows }; + } + + async function acquire(key, ttl, load) { + const previous = cache.get(key); + if (previous && now() - previous.savedAt < ttl) + return { value: previous.value, stale: false }; + if (!inFlight.has(key) && inFlight.size >= 256) + throw Object.assign(new Error('busy'), { status: 429 }); + try { + const { promise } = coalesceProxyRequest(inFlight, key, async () => { + const value = await load(); + cache.set(key, { value, savedAt: now() }); + return value; + }); + return { value: await promise, stale: false }; + } catch (error) { + if (previous) return { value: previous.value, stale: true }; + throw error; + } + } + + async function handler(req, res) { + const json = (status, value, stale = false) => { + if (res.destroyed) return; + res.writeHead(status, { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + ...(status === 405 ? { Allow: 'GET' } : {}), + ...(status === 429 ? { 'Retry-After': '60' } : {}), + ...(stale ? { 'X-Data-Stale': 'true' } : {}), + }); + res.end(JSON.stringify(value)); + }; + if (req.method !== 'GET') return json(405, { error: 'method_not_allowed' }); + const path = (req.url || '/').split('?')[0]; + if (path !== '/' && path !== '') + return json(404, { error: 'unknown_route' }); + if (!allow(clientKey(req))) return json(429, { error: 'rate_limited' }); + try { + const { value, stale } = await acquire( + 'zones', + SNAPSHOT_TTL_MS, + fetchZones, + ); + json(200, stale ? { ...value, stale: true } : value, stale); + } catch (error) { + json(error.status === 429 ? 429 : 502, { + error: 'drone_airspace_unavailable', + }); + } + } + + return { + name: 'drone-airspace', + configureServer({ middlewares }) { + middlewares.use('/api/drone-airspace', handler); + }, + configurePreviewServer({ middlewares }) { + middlewares.use('/api/drone-airspace', handler); + }, + }; +} diff --git a/server/providers/local.js b/server/providers/local.js index 9acbade31..e976455ed 100644 --- a/server/providers/local.js +++ b/server/providers/local.js @@ -24,6 +24,7 @@ import { mapillaryProxy } from './mapillary.js'; import { keySetupEndpoint } from '../standalone/key-setup.js'; import { weatherProxy } from './weather.js'; import { firePerimetersProxy } from './firePerimeters.js'; +import { droneAirspaceProxy } from './droneAirspace.js'; import { cycloneProxy } from './cyclones.js'; import { windProxy } from './wind.js'; @@ -60,6 +61,7 @@ function localProviderPlugins({ realtime } = {}) { cycloneProxy(), mapillaryProxy(), firePerimetersProxy(), + droneAirspaceProxy(), keySetupEndpoint(), ]; } diff --git a/src/app/constructCatalog.js b/src/app/constructCatalog.js index 80315fe85..fe36dd76a 100644 --- a/src/app/constructCatalog.js +++ b/src/app/constructCatalog.js @@ -25,6 +25,7 @@ import { createApplicationAwareness } from './layers/militaryAwareness.js'; import { createApplicationFirms } from './layers/firms.js'; import { createApplicationEarthquakes } from './layers/earthquakes.js'; import { createApplicationFirePerimeters } from './layers/perimeters.js'; +import { createApplicationDroneAirspace } from './layers/droneAirspace.js'; import { createApplicationCables } from './layers/submarineCables.js'; import { createApplicationStreetLevel } from './layers/streetLevel.js'; import { createInfrastructureLayers } from '../data/infrastructure.js'; @@ -120,6 +121,9 @@ export function createApplicationCatalog({ createApplicationFirePerimeters({ source: sources['fire-perimeters'], }), + createApplicationDroneAirspace({ + source: sources['drone-airspace'], + }), createApplicationAlpr({ surface, source: sources.alpr }), createApplicationStreetLevel({ surface, diff --git a/src/app/constructCatalog.test.mjs b/src/app/constructCatalog.test.mjs index f42e82c7c..e298580ad 100644 --- a/src/app/constructCatalog.test.mjs +++ b/src/app/constructCatalog.test.mjs @@ -40,7 +40,7 @@ test('catalogs construct distinct layers and classification from their supplied signal: b.signal, surface: fixtureSurface(b.signal), }); - assert.equal(first.layers.length, 30); + assert.equal(first.layers.length, 31); assert.ok(first.get('local-adsb'), 'Local ADS-B is registered'); assert.deepEqual( first.metadata.find(({ id }) => id === 'local-adsb'), @@ -60,6 +60,7 @@ test('catalogs construct distinct layers and classification from their supplied '2026-09-21T12:00:00.000Z', ); assert.ok(first.get('fire-perimeters')); + assert.ok(first.get('drone-airspace')); assert.ok(first.get('transit')); const order = first.layers.map(({ id }) => id); assert.deepEqual( @@ -153,7 +154,7 @@ test('Mapillary can be omitted or replaced without changing catalog membership', surface: fixtureSurface(lifetime.signal), }; const catalog = createApplicationCatalog(options); - assert.equal(catalog.layers.length, 30); + assert.equal(catalog.layers.length, 31); assert.deepEqual(catalog.get('street-level').providerIds, []); assert.equal( catalogControlServices(catalog).streetLevelLayer, diff --git a/src/app/layers/droneAirspace.js b/src/app/layers/droneAirspace.js new file mode 100644 index 000000000..f3472fec4 --- /dev/null +++ b/src/app/layers/droneAirspace.js @@ -0,0 +1,22 @@ +import * as Cesium from 'cesium'; +import { createDroneAirspaceLayer } from '../../layers/droneAirspace/index.js'; +import * as picking from '../../data/pickRegistry.js'; +import * as overlays from '../../overlays/worldOverlay.js'; +import { isPointerFree } from '../../data/inputOwnership.js'; + +/** Wire FAA special-use airspace into the application catalog. */ +export function createApplicationDroneAirspace(options) { + return createDroneAirspaceLayer({ + overlayHost: { + setEntries: overlays.setOverlayEntries, + setVisible: overlays.setOverlaySourceVisible, + clearSource: overlays.clearOverlaySource, + hitTest: overlays.hitTestWorldOverlay, + }, + screenSpaceEventHandlerFactory: (viewer) => + new Cesium.ScreenSpaceEventHandler(viewer.scene.canvas), + picking, + pointer: { isPointerFree }, + ...options, + }); +} diff --git a/src/app/sourceComposition.js b/src/app/sourceComposition.js index 14c864ac5..c9f79e407 100644 --- a/src/app/sourceComposition.js +++ b/src/app/sourceComposition.js @@ -46,6 +46,7 @@ export const CATALOG_SOURCE_CONTRACTS = Object.freeze({ cyclones: contract(['weather-cyclones'], ['getSnapshot']), earthquakes: contract(['earthquakes'], ['getSnapshot']), 'fire-perimeters': contract(['fire-perimeters'], ['getSnapshot']), + 'drone-airspace': contract(['drone-airspace'], ['getSnapshot']), cables: contract(['telegeography-submarine-cables'], ['fetch']), transit: contract(['transit'], ['requestSnapshot', 'getHistory']), }); diff --git a/src/app/sourceComposition.test.mjs b/src/app/sourceComposition.test.mjs index 44a7c8fa0..cf8646e25 100644 --- a/src/app/sourceComposition.test.mjs +++ b/src/app/sourceComposition.test.mjs @@ -37,7 +37,7 @@ test('every acquisition source can be omitted without losing catalog membership' test('all acquisition sources may be absent, and enable fails before initializing or fetching', async (t) => { const actual = catalog({}, t); - assert.equal(actual.layers.length, 30); + assert.equal(actual.layers.length, 31); const manager = new LayerLifecycle( {}, { getSourceAvailability: actual.getSourceAvailability }, diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index 2967f0a2b..2e4a8148e 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -160,6 +160,13 @@ export const DATA_CREDITS = [ ' · Incident information: ' + 'InciWeb', }, + { + key: 'faa-sua', + html: + 'Drone restrictions: ' + + 'FAA Special Use Airspace' + + ' (prohibited and restricted zones; informational only)', + }, OSM_CREDIT, OPENMAPTILES_CREDIT, { diff --git a/src/data/droneAirspaceProxy.test.mjs b/src/data/droneAirspaceProxy.test.mjs new file mode 100644 index 000000000..52eb0dcc8 --- /dev/null +++ b/src/data/droneAirspaceProxy.test.mjs @@ -0,0 +1,146 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { droneAirspaceProxy } from '../../server/providers/droneAirspace.js'; + +const feature = (id = 310, type = 'P') => ({ + properties: { + NAME: `${type}-${id}`, + TYPE_CODE: type, + OBJECTID: id, + UPPER_DESC: 'TNI', + }, + geometry: { + type: 'Polygon', + coordinates: [ + [ + [-77, 38], + [-76, 38], + [-76, 39], + [-77, 38], + ], + ], + }, +}); + +function install(options = {}, hook = 'configureServer') { + let handler; + const plugin = droneAirspaceProxy(options); + assert.equal(typeof plugin.configureServer, 'function'); + assert.equal(typeof plugin.configurePreviewServer, 'function'); + plugin[hook]({ + middlewares: { + use(path, callback) { + assert.equal(path, '/api/drone-airspace'); + handler = callback; + }, + }, + }); + return async (url = '/', method = 'GET', peer = 'local') => { + const res = { + writeHead(status, headers) { + this.status = status; + this.headers = headers; + }, + end(body) { + this.body = JSON.parse(body); + }, + }; + await handler({ url, method, socket: { remoteAddress: peer } }, res); + return res; + }; +} + +for (const hook of ['configureServer', 'configurePreviewServer']) { + test(`${hook}: paging stops at five pages and normalizes the combined snapshot`, async () => { + const calls = []; + const request = install( + { + now: () => 1234, + fetchImpl: async (url, options) => { + calls.push(new URL(url)); + assert.ok(options.signal instanceof AbortSignal); + assert.equal(options.redirect, 'error'); + return Response.json({ + features: [feature(calls.length)], + exceededTransferLimit: true, + }); + }, + }, + hook, + ); + const res = await request('/?url=https://invalid.example&resultOffset=999'); + assert.equal(res.status, 200); + assert.equal(res.body.fetchedAt, 1234); + assert.equal(res.body.rows.length, 5); + assert.equal(calls.length, 5); + assert.equal(calls[0].searchParams.get('where'), "TYPE_CODE IN ('P','R')"); + assert.equal(calls[0].searchParams.get('f'), 'geojson'); + assert.equal(calls[0].searchParams.has('resultOffset'), false); + assert.equal(calls[1].searchParams.get('resultOffset'), '1'); + }); +} + +test('a cached snapshot is reused until TTL, then refreshed', async () => { + let now = 1000; + let fetches = 0; + const request = install({ + now: () => now, + fetchImpl: async () => { + fetches += 1; + return Response.json({ features: [feature(fetches)] }); + }, + }); + const first = await request(); + now += 3_599_000; + const cached = await request(); + now += 2000; + const refreshed = await request(); + assert.equal(first.body.rows[0].stableId, '1'); + assert.equal(cached.body.rows[0].stableId, '1'); + assert.equal(refreshed.body.rows[0].stableId, '2'); + assert.equal(fetches, 2); +}); + +test('a failed refresh serves the last good snapshot as stale', async () => { + let now = 1000; + let fail = false; + const request = install({ + now: () => now, + fetchImpl: async () => { + if (fail) return new Response('nope', { status: 503 }); + return Response.json({ features: [feature(7)] }); + }, + }); + const first = await request(); + fail = true; + now += 3_600_001; + const stale = await request(); + assert.equal(first.status, 200); + assert.equal(stale.status, 200); + assert.equal(stale.headers['X-Data-Stale'], 'true'); + assert.equal(stale.body.stale, true); + assert.equal(stale.body.rows[0].stableId, '7'); +}); + +test('method, unknown routes and rate limits are rejected', async () => { + const request = install({ + fetchImpl: async () => Response.json({ features: [feature()] }), + }); + assert.equal((await request('/', 'POST')).status, 405); + assert.equal((await request('/nope')).status, 404); + const limited = install({ + fetchImpl: async () => Response.json({ features: [feature()] }), + }); + let last; + for (let i = 0; i < 61; i++) last = await limited('/', 'GET', '10.0.0.9'); + assert.equal(last.status, 429); +}); + +test('a cold upstream failure is 502, not an empty snapshot', async () => { + const request = install({ + fetchImpl: async () => new Response('nope', { status: 503 }), + }); + const res = await request(); + assert.equal(res.status, 502); + assert.equal(res.body.error, 'drone_airspace_unavailable'); +}); diff --git a/src/data/layerState.js b/src/data/layerState.js index 373eac20d..bc5c660ca 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -595,6 +595,11 @@ export const LAYER_STATE_REGISTRY = Object.freeze([ optionOwner: 'cctv', }), Object.freeze({ id: 'directions', token: 'n', disposition: 'enabled-only' }), + Object.freeze({ + id: 'drone-airspace', + token: '3', + disposition: 'enabled-only', + }), Object.freeze({ id: 'earthquakes', token: 'e', disposition: 'enabled-only' }), Object.freeze({ id: 'fire-perimeters', diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index 08120a604..0fbdcf000 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -202,8 +202,8 @@ function encode(state) { test('production registry is exact, canonical, and rejects incomplete contracts', async () => { assert.equal(validateLayerStateRegistry(), true); - assert.equal(REGISTERED_LAYER_IDS.length, 29); - assert.equal(new Set(REGISTERED_LAYER_IDS).size, 29); + assert.equal(REGISTERED_LAYER_IDS.length, 30); + assert.equal(new Set(REGISTERED_LAYER_IDS).size, 30); assert.ok(REGISTERED_LAYER_IDS.includes('transit')); assert.deepEqual(REGISTERED_LAYER_IDS, [...REGISTERED_LAYER_IDS].sort()); assert.deepEqual(LEGACY_LAYER_STATE_TOKENS, { @@ -242,7 +242,7 @@ test('production registry is exact, canonical, and rejects incomplete contracts' for (const [id, token] of Object.entries(LEGACY_LAYER_STATE_TOKENS)) { assert.equal(LAYER_STATE_TOKEN_RESERVATIONS[id], token); } - assert.equal(nextLayerStateToken(), '3'); + assert.equal(nextLayerStateToken(), '4'); assert.equal( nextLayerStateToken({ ...LAYER_STATE_TOKEN_RESERVATIONS, @@ -307,8 +307,8 @@ test('production registry is exact, canonical, and rejects incomplete contracts' assert.equal( validateLayerStateAllocations(LAYER_STATE_TOKEN_RESERVATIONS, { ...LAYER_STATE_TOKEN_RESERVATIONS, - future: '3', - next: '4', + future: '4', + next: '5', }), true, ); @@ -318,7 +318,7 @@ test('production registry is exact, canonical, and rejects incomplete contracts' ...LAYER_STATE_TOKEN_RESERVATIONS, future: '00', }), - /next free token 3/, + /next free token 4/, ); const beforeLastDigit = { ...digitsExhausted }; delete beforeLastDigit['prior-9']; @@ -341,10 +341,10 @@ test('production registry is exact, canonical, and rejects incomplete contracts' assert.throws( () => validateLayerStateAllocations( - { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '3' }, - { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '3', competing: '3' }, + { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '4' }, + { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '4', competing: '4' }, ), - /next free token 4/, + /next free token 5/, ); assert.throws( () => validateLayerStateAllocations({ future: '00' }, { future: '01' }), @@ -2639,6 +2639,22 @@ test('the recent-imagery split is share-link only: never stored locally, and a s ); }); +test('drone restrictions uses digit 3 without colliding with fire perimeters', () => { + const decoded = decodeLayerStateParams(new URLSearchParams('v=2&l=3.2')); + assert.deepEqual(decoded.enabledLayerIds, [ + 'drone-airspace', + 'fire-perimeters', + ]); + assert.equal( + LAYER_STATE_REGISTRY.find(({ id }) => id === 'drone-airspace').token, + '3', + ); + assert.deepEqual( + decodeLayerStateParams(new URLSearchParams(encode(decoded))), + decoded, + ); +}); + test('fire perimeters uses digit 2 without colliding with wind or recent imagery', () => { const decoded = decodeLayerStateParams(new URLSearchParams('v=2&l=2.k.1')); assert.deepEqual(decoded.enabledLayerIds, [ diff --git a/src/data/layerStateTokenLedger.test.mjs b/src/data/layerStateTokenLedger.test.mjs index 5a02d85cf..56b2e1582 100644 --- a/src/data/layerStateTokenLedger.test.mjs +++ b/src/data/layerStateTokenLedger.test.mjs @@ -128,10 +128,14 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed { ...parseLayerStateTokenReservations(reservationRows) }, { ...LAYER_STATE_TOKEN_RESERVATIONS }, ); - assert.equal(Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length, 29); + assert.equal(Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length, 30); assert.deepEqual( { ...LAYER_STATE_TOKEN_RESERVATIONS }, - { ...LEGACY_LAYER_STATE_TOKENS, 'street-level': '0' }, + { + ...LEGACY_LAYER_STATE_TOKENS, + 'street-level': '0', + 'drone-airspace': '3', + }, ); assert.throws( () => parseLayerStateTokenReservations(reservationRows.slice(1)), @@ -151,9 +155,9 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed /Duplicate layer-state token reservation/, ); assert.equal( - parseLayerStateTokenReservations([...reservationRows, ['future', '3']]) + parseLayerStateTokenReservations([...reservationRows, ['future', '4']]) .future, - '3', + '4', ); assert.throws( () => @@ -171,14 +175,14 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed test('published retired digits and competing PRs advance the merge-time allocation', () => { withPublishedBase( - { rows: [...reservationRows, ['retired-layer', '3']] }, + { rows: [...reservationRows, ['retired-layer', '4']] }, (cwd) => { const published = readPublishedLayerStateReservations('HEAD', cwd); - assert.equal(nextLayerStateToken(published), '4'); + assert.equal(nextLayerStateToken(published), '5'); assert.equal( validateLayerStateAllocations(published, { ...published, - newcomer: '4', + newcomer: '5', }), true, ); @@ -186,15 +190,15 @@ test('published retired digits and competing PRs advance the merge-time allocati () => validateLayerStateAllocations(published, { ...published, - competing: '3', + competing: '4', }), - /next free token 4/, + /next free token 5/, ); assert.throws( () => validateLayerStateAllocations(published, { ...LAYER_STATE_TOKEN_RESERVATIONS, - newcomer: '3', + newcomer: '4', }), /changed or removed: retired-layer/, ); @@ -202,8 +206,8 @@ test('published retired digits and competing PRs advance the merge-time allocati ); }); -test('PR B manually replaces provisional 3 with 4 after PR A publishes 3', () => { - withPublishedBase({ rows: [...reservationRows, ['pr-a', '3']] }, (cwd) => { +test('PR B manually replaces provisional 4 with 5 after PR A publishes 4', () => { + withPublishedBase({ rows: [...reservationRows, ['pr-a', '4']] }, (cwd) => { mkdirSync(path.join(cwd, 'scripts')); const fixtureChecker = path.join( cwd, @@ -224,8 +228,8 @@ test('PR B manually replaces provisional 3 with 4 after PR A publishes 3', () => writeCodecFixture( cwd, - [...reservationRows, ['pr-b', '3']], - [{ id: 'pr-b', token: '3' }], + [...reservationRows, ['pr-b', '4']], + [{ id: 'pr-b', token: '4' }], ); const beforeRebase = runCheck(); assert.equal(beforeRebase.status, 1); @@ -233,10 +237,10 @@ test('PR B manually replaces provisional 3 with 4 after PR A publishes 3', () => writeCodecFixture( cwd, - [...reservationRows, ['pr-a', '3'], ['pr-b', '3']], + [...reservationRows, ['pr-a', '4'], ['pr-b', '4']], [ - { id: 'pr-a', token: '3' }, - { id: 'pr-b', token: '3' }, + { id: 'pr-a', token: '4' }, + { id: 'pr-b', token: '4' }, ], ); const staleRebase = runCheck(); @@ -244,16 +248,16 @@ test('PR B manually replaces provisional 3 with 4 after PR A publishes 3', () => assert.match(staleRebase.stderr, /Duplicate layer-state token reservation/); const published = readPublishedLayerStateReservations('HEAD', cwd); - assert.equal(nextLayerStateToken(published), '4'); - const validBaselineRows = [...reservationRows, ['pr-a', '3']]; - writeCodecFixture(cwd, validBaselineRows, [{ id: 'pr-a', token: '3' }]); + assert.equal(nextLayerStateToken(published), '5'); + const validBaselineRows = [...reservationRows, ['pr-a', '4']]; + writeCodecFixture(cwd, validBaselineRows, [{ id: 'pr-a', token: '4' }]); const helper = spawnSync( process.execPath, [realpathSync(fixtureNext), 'pr-b'], { cwd, encoding: 'utf8' }, ); assert.equal(helper.status, 0, helper.stderr); - assert.match(helper.stdout, /pr-b: 4/); + assert.match(helper.stdout, /pr-b: 5/); assert.deepEqual( JSON.parse(readFileSync(path.join(cwd, LEDGER_PATH), 'utf8')), validBaselineRows, @@ -261,20 +265,20 @@ test('PR B manually replaces provisional 3 with 4 after PR A publishes 3', () => ); writeCodecFixture( cwd, - [...reservationRows, ['pr-a', '3'], ['pr-b', '4']], + [...reservationRows, ['pr-a', '4'], ['pr-b', '5']], [ - { id: 'pr-a', token: '3' }, - { id: 'pr-b', token: '4' }, + { id: 'pr-a', token: '4' }, + { id: 'pr-b', token: '5' }, ], ); const corrected = runCheck(); assert.equal(corrected.status, 0, corrected.stderr); - assert.match(corrected.stdout, /30 published, 1 new/); + assert.match(corrected.stdout, /31 published, 1 new/); }); }); test('allocation batches cross the last digit and base-36 pair boundaries in order', () => { - const digitRows = [...'3456789'].map((digit) => [`retired-${digit}`, digit]); + const digitRows = [...'456789'].map((digit) => [`retired-${digit}`, digit]); const allDigits = parseLayerStateTokenReservations([ ...reservationRows, ...digitRows, @@ -326,7 +330,7 @@ test('allocation batches cross the last digit and base-36 pair boundaries in ord test('an isolated valid two-character fixture round-trips an l field beyond the old 64-character cap', async () => { const cwd = mkdtempSync(path.join(tmpdir(), 'gev-layer-token-width-')); try { - const priorDigits = [...'3456789'].map((digit) => [ + const priorDigits = [...'456789'].map((digit) => [ `qa-prior-digit-${digit}`, digit, ]); @@ -363,7 +367,7 @@ test('an isolated valid two-character fixture round-trips an l field beyond the const params = new URLSearchParams([['v', '2']]); codec.encodeLayerStateParams(params, state); assert.ok(params.get('l').length > 64, 'fixture must cross the old cap'); - assert.equal(params.get('l').length, 69); + assert.equal(params.get('l').length, 71); const restored = codec.decodeLayerStateParams(params); assert.deepEqual(restored?.enabledLayerIds, expectedLayerIds); assert.deepEqual(restored?.options, state.options); @@ -381,7 +385,7 @@ test('checker reads a complete future base ledger and rejects retired-token reus }, (cwd) => { const published = readPublishedLayerStateReservations('HEAD', cwd); - assert.equal(Object.keys(published).length, 30); + assert.equal(Object.keys(published).length, 31); assert.equal(published['retired-layer'], '00'); assert.throws( () => diff --git a/src/data/layerStateTokenReservations.json b/src/data/layerStateTokenReservations.json index eb49eed27..6024f5248 100644 --- a/src/data/layerStateTokenReservations.json +++ b/src/data/layerStateTokenReservations.json @@ -6,6 +6,7 @@ ["bikeshare", "b"], ["cctv", "c"], ["directions", "n"], + ["drone-airspace", "3"], ["earthquakes", "e"], ["fire-perimeters", "2"], ["flights", "f"], diff --git a/src/firstRunExperience.test.mjs b/src/firstRunExperience.test.mjs index 49711163f..cd8efce77 100644 --- a/src/firstRunExperience.test.mjs +++ b/src/firstRunExperience.test.mjs @@ -674,10 +674,11 @@ test('the voice TOOL SCHEMA matches the pinned contract — the mission mapping // schema, which may not carry top-level anyOf), and Contacts requested-radius // list descriptions; the missions still ride existing tools. // Street Level adds toggle enum values and the generated alias hint. - assert.equal(block.length, 29802, 'serialized tool schema length drifted'); + // Drone Restrictions adds toggle/query enum values and spoken aliases. + assert.equal(block.length, 29988, 'serialized tool schema length drifted'); assert.equal( crypto.createHash('sha256').update(block).digest('hex'), - 'b316ca2e67848eb565b6847ac044a19d1465d8ef8252a5dfbd7d947a2a36d5ed', + 'e3605286e8b91c07f29b687cb3cf0e9400fef92225fbc7efeaa35451274df571', 'the first-run missions must ride EXISTING tools: no schema edit, no cache bust', ); const instructions = fs.readFileSync(new URL('../server/providers/openai/instructions.js', import.meta.url), 'utf8'); diff --git a/src/layers/droneAirspace/cards.js b/src/layers/droneAirspace/cards.js new file mode 100644 index 000000000..eb6cc6433 --- /dev/null +++ b/src/layers/droneAirspace/cards.js @@ -0,0 +1,93 @@ +/** Card model for one selected special-use airspace zone. Pure — no Cesium types. */ + +export const DRONE_AIRSPACE_OVERLAY_SOURCE_ID = 'drone-airspace'; + +/** Fill/line CSS color for a zone by TYPE_CODE. */ +export function droneAirspaceAccent(typeCode) { + return typeCode === 'P' ? '#ff3b30' : '#ff7a00'; +} + +/** Shoelace area (degree², sign dropped) — relative sizes only. */ +function ringArea(ring) { + let sum = 0; + for (let i = 0; i < ring.length - 1; i++) { + const [x1, y1] = ring[i]; + const [x2, y2] = ring[i + 1]; + sum += x1 * y2 - x2 * y1; + } + return Math.abs(sum / 2); +} + +/** + * Anchor point for the zone card: the centroid of the largest polygon's + * outer ring. Degrees, so callers own the Cartesian conversion. + * @param {Array>>} polygons + * @returns {{lon: number, lat: number}} + */ +export function droneAirspaceAnchorDegrees(polygons) { + let best = null; + let bestArea = -1; + for (const rings of polygons) { + const outer = rings[0]; + const area = ringArea(outer); + if (area > bestArea) { + bestArea = area; + best = outer; + } + } + let lon = 0; + let lat = 0; + const count = best.length - 1; + for (let i = 0; i < count; i++) { + lon += best[i][0]; + lat += best[i][1]; + } + return { lon: lon / count, lat: lat / count }; +} + +function altitudeLine(row) { + const lower = row.lowerDesc; + const upper = row.upperDesc; + if (lower && upper) return `${lower} – ${upper}`; + if (upper) return `up to ${upper}`; + if (lower) return `from ${lower}`; + return null; +} + +function placeLine(row) { + if (row.city && row.state) return `${row.city}, ${row.state}`; + return row.city || row.state || null; +} + +/** + * Build the overlay-host entry for one selected zone. The caller supplies + * `position` (Cartesian) separately — this model stays JSON-safe for tests. + * @param {Object} row - Normalized zone row (see records.js). + * @returns {Object} World-overlay entry without `position`. + */ +export function buildDroneAirspaceCard(row) { + const facts = [row.type || row.typeCode].filter(Boolean); + const altitude = altitudeLine(row); + if (altitude) facts.push(altitude); + if (row.timesOfUse) facts.push(row.timesOfUse); + + const details = [facts.join(' · ')]; + const place = placeLine(row); + if (place) details.push(place); + if (row.agency) details.push(row.agency); + details.push('Informational only · not for flight planning'); + + const title = `${(row.type || 'SUA').toUpperCase()} · ${row.name || 'Unnamed zone'}`; + return { + id: `drone-airspace-card:${row.stableId}`, + selected: true, + interactive: false, + title, + details, + accent: droneAirspaceAccent(row.typeCode), + priority: Number.MAX_SAFE_INTEGER, + gapPx: 15, + verticalOnly: true, + placement: 'above', + }; +} diff --git a/src/layers/droneAirspace/cards.test.mjs b/src/layers/droneAirspace/cards.test.mjs new file mode 100644 index 000000000..e8d58c37c --- /dev/null +++ b/src/layers/droneAirspace/cards.test.mjs @@ -0,0 +1,74 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + droneAirspaceAnchorDegrees, + droneAirspaceAccent, + buildDroneAirspaceCard, +} from './cards.js'; + +const square = [ + [-77.2, 38.8], + [-77.0, 38.8], + [-77.0, 39.0], + [-77.2, 39.0], + [-77.2, 38.8], +]; +const tiny = [ + [-76.01, 38.0], + [-76.0, 38.0], + [-76.0, 38.01], + [-76.01, 38.0], +]; + +test('the card anchors at the centroid of the largest polygon', () => { + const anchor = droneAirspaceAnchorDegrees([[tiny], [square]]); + assert.ok(Math.abs(anchor.lon - -77.1) < 1e-9); + assert.ok(Math.abs(anchor.lat - 38.9) < 1e-9); +}); + +test('a prohibited zone card names the type, altitudes and the disclaimer', () => { + const card = buildDroneAirspaceCard({ + stableId: '310', + name: 'P-56', + typeCode: 'P', + type: 'Prohibited', + lowerDesc: 'SFC', + upperDesc: 'TNI', + timesOfUse: 'CONTINUOUS', + agency: 'WASHINGTON CENTER', + city: 'WASHINGTON', + state: 'DC', + }); + assert.equal(card.id, 'drone-airspace-card:310'); + assert.equal(card.title, 'PROHIBITED · P-56'); + assert.deepEqual(card.details, [ + 'Prohibited · SFC – TNI · CONTINUOUS', + 'WASHINGTON, DC', + 'WASHINGTON CENTER', + 'Informational only · not for flight planning', + ]); + assert.equal(card.accent, droneAirspaceAccent('P')); + assert.equal(card.interactive, false); + assert.equal(card.selected, true); +}); + +test('missing attributes degrade to available facts instead of placeholders', () => { + const card = buildDroneAirspaceCard({ + stableId: 'x', + name: null, + typeCode: 'R', + type: 'Restricted', + lowerDesc: null, + upperDesc: null, + timesOfUse: null, + agency: null, + city: null, + state: null, + }); + assert.equal(card.title, 'RESTRICTED · Unnamed zone'); + assert.deepEqual(card.details, [ + 'Restricted', + 'Informational only · not for flight planning', + ]); + assert.equal(card.accent, droneAirspaceAccent('R')); +}); diff --git a/src/layers/droneAirspace/index.js b/src/layers/droneAirspace/index.js new file mode 100644 index 000000000..d7b4a40c8 --- /dev/null +++ b/src/layers/droneAirspace/index.js @@ -0,0 +1,326 @@ +import * as Cesium from 'cesium'; +import { + DRONE_AIRSPACE_OVERLAY_SOURCE_ID, + droneAirspaceAnchorDegrees, + buildDroneAirspaceCard, + droneAirspaceAccent, +} from './cards.js'; +export { normalizeDroneAirspaceSnapshot } from './records.js'; +export { createDroneAirspaceSource } from './source.js'; +export * from './cards.js'; + +/** Fill/line color for a zone by TYPE_CODE. */ +export function droneAirspaceColor(typeCode) { + return Cesium.Color.fromCssColorString(droneAirspaceAccent(typeCode)); +} + +const ringPositions = (ring) => + ring.map(([lon, lat]) => Cesium.Cartesian3.fromDegrees(lon, lat)); + +const PICK_PREFIX = 'drone-airspace:'; +const CARD_HOST_OPTIONS = Object.freeze({ + cohortLimit: 1, + collisionCapacity: 1, + moving: false, +}); + +/** Own one drone-restriction display, its refresh lifecycle, and click selection. */ +export function createDroneAirspaceLayer({ + source, + overlayHost = null, + screenSpaceEventHandlerFactory = null, + picking = null, + pointer = null, +} = {}) { + if (typeof source?.getSnapshot !== 'function') + throw new TypeError('Drone airspace requires a snapshot source'); + let _viewer = null; + let _request = null; + let _snapshotSignature = null; + let _dataSource = null; + let _count = 0; + let _lastUpdate = null; + let _lastError = null; + let _enabled = false; + let _clickHandler = null; + let _selectedId = null; + let _selectedCardId = null; + /** @type {Map} */ + const _rowById = new Map(); + + const canSelect = () => + overlayHost && screenSpaceEventHandlerFactory && picking; + + function publishSelectedCard() { + if (!canSelect()) return; + const row = _selectedId ? _rowById.get(_selectedId) : null; + if (!row) { + _selectedId = null; + _selectedCardId = null; + overlayHost.setEntries( + DRONE_AIRSPACE_OVERLAY_SOURCE_ID, + [], + CARD_HOST_OPTIONS, + ); + return; + } + const card = { + ...buildDroneAirspaceCard(row), + position: Cesium.Cartesian3.fromDegrees(row.anchor.lon, row.anchor.lat), + }; + _selectedCardId = card.id; + overlayHost.setEntries( + DRONE_AIRSPACE_OVERLAY_SOURCE_ID, + [card], + CARD_HOST_OPTIONS, + ); + } + + /** Resolve a scene pick to one of this layer's zone ids, or null. */ + function pickedZoneId(picked) { + const pickId = picking.resolvePickId(picked); + if (typeof pickId !== 'string' || !pickId.startsWith(PICK_PREFIX)) + return null; + const zoneId = pickId.slice(PICK_PREFIX.length).split(':')[0]; + return _rowById.has(zoneId) ? zoneId : null; + } + + function installClickHandler() { + if (!canSelect() || _clickHandler || !_viewer) return; + // Deliberately NOT registered in the pick-ownership registry: sibling + // layers yield to owned picks before their own overlay-card hit-tests, + // so claiming these large ground polygons would make FIRMS/vessel/CCTV + // cards inert anywhere over a zone. Unowned, our picks read as + // "empty space" to siblings — the same contract as fire perimeters. + _clickHandler = screenSpaceEventHandlerFactory(_viewer); + _clickHandler.setInputAction((click) => { + if (pointer && !pointer.isPointerFree()) return; + const cardHit = overlayHost.hitTest?.( + click.position?.x, + click.position?.y, + { sourceId: DRONE_AIRSPACE_OVERLAY_SOURCE_ID }, + ); + if (cardHit && cardHit.entryId === _selectedCardId) return; + const picked = _viewer.scene.pick(click.position); + const zoneId = picked ? pickedZoneId(picked) : null; + if (zoneId) { + _selectedId = zoneId; + publishSelectedCard(); + return; + } + if (picked) { + const pickId = picking.resolvePickId(picked); + if (pickId && picking.isOwnedByOtherLayer(layer.id, pickId)) return; + } + if (_selectedId) { + _selectedId = null; + publishSelectedCard(); + } + }, Cesium.ScreenSpaceEventType.LEFT_CLICK); + } + + function removeClickHandler() { + if (_clickHandler) { + _clickHandler.destroy(); + _clickHandler = null; + } + } + + function clearSelection() { + _selectedId = null; + _selectedCardId = null; + if (overlayHost) { + overlayHost.clearSource(DRONE_AIRSPACE_OVERLAY_SOURCE_ID); + overlayHost.setVisible?.(DRONE_AIRSPACE_OVERLAY_SOURCE_ID, false); + } + } + + const layer = { + id: 'drone-airspace', + name: 'Drone Restrictions', + icon: '🚫', + source: 'FAA SUA', + updateInterval: 3600000, + + init(viewer) { + if (_viewer) + throw new Error('Drone airspace layer is already initialized'); + _viewer = viewer; + _dataSource = new Cesium.CustomDataSource('drone-airspace'); + _dataSource.show = false; + viewer.dataSources.add(_dataSource); + _count = 0; + _lastUpdate = null; + _lastError = null; + _enabled = false; + console.log('[Data:DroneAirspace] Initialized'); + }, + + enable() { + _enabled = true; + if (_dataSource) _dataSource.show = true; + overlayHost?.setVisible?.(DRONE_AIRSPACE_OVERLAY_SOURCE_ID, true); + installClickHandler(); + }, + + disable() { + _request?.abort(); + _request = null; + _enabled = false; + if (_dataSource) _dataSource.show = false; + removeClickHandler(); + clearSelection(); + }, + + async update() { + if (!_enabled || !_dataSource) return false; + _request?.abort(); + const request = new AbortController(); + _request = request; + try { + const rows = await source.getSnapshot({ signal: request.signal }); + if (request.signal.aborted || _request !== request || !_enabled) + return false; + + const signature = JSON.stringify( + rows + .map(({ polygons, ...facts }) => facts) + .sort((a, b) => a.stableId.localeCompare(b.stableId)), + ); + if (signature === _snapshotSignature) { + if (_selectedId) publishSelectedCard(); + _lastUpdate = Date.now(); + _lastError = null; + return true; + } + const nextEntities = []; + _rowById.clear(); + for (const row of rows) { + const color = droneAirspaceColor(row.typeCode); + for (const [index, rings] of row.polygons.entries()) { + const [outer, ...holes] = rings; + const outerPositions = ringPositions(outer); + nextEntities.push( + new Cesium.Entity({ + id: `drone-airspace:${row.stableId}:${index}`, + polygon: { + hierarchy: new Cesium.PolygonHierarchy( + outerPositions, + holes.map( + (hole) => + new Cesium.PolygonHierarchy(ringPositions(hole)), + ), + ), + material: new Cesium.ColorMaterialProperty( + color.withAlpha(0.25), + ), + }, + polyline: { + positions: outerPositions, + clampToGround: true, + width: 2, + material: new Cesium.ColorMaterialProperty( + color.withAlpha(0.9), + ), + }, + }), + ); + } + const { polygons, ...facts } = row; + _rowById.set(row.stableId, { + ...facts, + anchor: droneAirspaceAnchorDegrees(polygons), + }); + } + + _dataSource.entities.removeAll(); + for (const entity of nextEntities) _dataSource.entities.add(entity); + _snapshotSignature = signature; + if (_selectedId) publishSelectedCard(); + _count = rows.length; + _lastUpdate = Date.now(); + _lastError = null; + console.log( + `[Data:DroneAirspace] Updated: ${_count} zones, ${nextEntities.length} polygons`, + ); + return true; + } catch (e) { + if (request.signal.aborted || _request !== request || !_enabled) + return false; + console.warn('[Data:DroneAirspace] Fetch error:', e); + _lastError = e?.message || 'Drone airspace source unavailable'; + return false; + } finally { + if (_request === request) _request = null; + } + }, + + destroy(viewer = _viewer) { + _request?.abort(); + _request = null; + removeClickHandler(); + clearSelection(); + _rowById.clear(); + _snapshotSignature = null; + _viewer = null; + _enabled = false; + if (_dataSource) { + viewer.dataSources.remove(_dataSource, true); + _dataSource = null; + } + _count = 0; + _lastUpdate = null; + _lastError = null; + }, + + getAnalystRecords(maxCount = 2000) { + if (!_dataSource || !_dataSource.show) return []; + const limit = Number.isFinite(maxCount) + ? Math.max(1, Math.floor(maxCount)) + : 2000; + const result = []; + for (const row of _rowById.values()) { + if (result.length >= limit) break; + const { anchor, stableId, ...facts } = row; + result.push({ + id: stableId, + ...facts, + lat: anchor.lat, + lon: anchor.lon, + }); + } + return result; + }, + + getRowControls() { + const bands = [ + { label: 'Prohibited (P)', color: droneAirspaceAccent('P') }, + { label: 'Restricted (R)', color: droneAirspaceAccent('R') }, + ]; + return { + chips: [], + legend: bands.map((band, index) => ({ + ...band, + count: [..._rowById.values()].filter( + (row) => droneAirspaceAccent(row.typeCode) === band.color, + ).length, + ...(index === 0 + ? { + blurb: + 'FAA special-use airspace. Informational only — not a substitute for official UAS flight-planning tools (B4UFLY / LAANC).', + } + : {}), + })), + }; + }, + + getStats() { + return { + count: _count, + lastUpdate: _lastUpdate, + error: _lastError, + }; + }, + }; + return layer; +} diff --git a/src/layers/droneAirspace/ownership.test.mjs b/src/layers/droneAirspace/ownership.test.mjs new file mode 100644 index 000000000..ae10184f2 --- /dev/null +++ b/src/layers/droneAirspace/ownership.test.mjs @@ -0,0 +1,161 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createDroneAirspaceLayer } from './index.js'; + +function harness(source, { pick = () => null, cardHit = () => null } = {}) { + const sources = []; + const overlay = { entries: new Map(), visible: null }; + const clicks = { handler: null, destroyed: 0 }; + const owners = new Map(); + const viewer = { + scene: { pick }, + dataSources: { + add(value) { + sources.push(value); + }, + remove(value) { + sources.splice(sources.indexOf(value), 1); + }, + }, + }; + const layer = createDroneAirspaceLayer({ + source, + overlayHost: { + setEntries(sourceId, entries) { + overlay.entries.set(sourceId, entries); + }, + setVisible(sourceId, visible) { + overlay.visible = visible; + }, + clearSource(sourceId) { + overlay.entries.delete(sourceId); + }, + hitTest: (x, y, options) => cardHit(x, y, options), + }, + screenSpaceEventHandlerFactory: () => ({ + setInputAction(callback) { + clicks.handler = callback; + }, + destroy() { + clicks.destroyed += 1; + clicks.handler = null; + }, + }), + picking: { + resolvePickId: (picked) => picked?.id ?? null, + isOwnedByOtherLayer: (layerId, pickedId) => + String(pickedId).startsWith('other-layer:'), + registerPickOwner: (layerId, predicate) => owners.set(layerId, predicate), + unregisterPickOwner: (layerId) => owners.delete(layerId), + }, + pointer: { isPointerFree: () => true }, + }); + layer.init(viewer); + layer.enable(viewer); + return { layer, viewer, sources, overlay, clicks, owners }; +} + +const ring = [ + [-77.04, 38.89], + [-77.03, 38.89], + [-77.03, 38.9], + [-77.04, 38.89], +]; +const row = { + stableId: '310', + name: 'P-56', + typeCode: 'P', + type: 'Prohibited', + lowerDesc: 'SFC', + upperDesc: 'TNI', + timesOfUse: 'CONTINUOUS', + agency: null, + city: 'WASHINGTON', + state: 'DC', + polygons: [[ring]], +}; + +test('each zone polygon renders as one filled entity with its outline', async () => { + const h = harness({ + getSnapshot: async () => [ + row, + { ...row, stableId: '400', typeCode: 'R', polygons: [[ring], [ring]] }, + ], + }); + assert.equal(await h.layer.update(h.viewer), true); + const entities = h.sources[0].entities.values; + assert.equal(entities.length, 3); + assert.equal(entities[0].id, 'drone-airspace:310:0'); + assert.ok(entities[0].polygon, 'zone fill missing'); + assert.ok(entities[0].polyline, 'zone outline missing'); + assert.equal(h.layer.getStats().count, 2); +}); + +test('clicking a zone publishes its card; empty space clears it', async () => { + let pickResult = null; + const h = harness( + { getSnapshot: async () => [row] }, + { pick: () => pickResult }, + ); + await h.layer.update(h.viewer); + assert.equal(typeof h.clicks.handler, 'function', 'click handler missing'); + assert.equal(h.owners.size, 0); + + pickResult = { id: 'drone-airspace:310:0' }; + h.clicks.handler({ position: { x: 10, y: 10 } }); + const entries = h.overlay.entries.get('drone-airspace'); + assert.equal(entries.length, 1); + assert.equal(entries[0].id, 'drone-airspace-card:310'); + assert.equal(entries[0].title, 'PROHIBITED · P-56'); + assert.ok(entries[0].position, 'card must carry a world anchor'); + + pickResult = { id: 'other-layer:aircraft-1' }; + h.clicks.handler({ position: { x: 10, y: 10 } }); + assert.equal( + h.overlay.entries.get('drone-airspace')?.length, + 1, + 'sibling-layer picks must not clear the selection', + ); + + pickResult = null; + h.clicks.handler({ position: { x: 10, y: 10 } }); + assert.equal(h.overlay.entries.get('drone-airspace')?.length ?? 0, 0); +}); + +test('late refresh cannot publish after disable or destroy', async () => { + for (const action of ['disable', 'destroy']) { + let resolve, signal; + const h = harness({ + getSnapshot(options) { + signal = options.signal; + return new Promise((done) => { + resolve = done; + }); + }, + }); + const pending = h.layer.update(h.viewer); + h.layer[action](h.viewer); + assert.equal(signal.aborted, true); + if (action === 'disable') h.layer.enable(h.viewer); + resolve([row]); + assert.equal(await pending, false); + assert.equal(h.layer.getStats().count, 0); + h.layer.destroy(h.viewer); + } +}); + +test('the row legend names prohibited and restricted with the disclaimer', async () => { + const h = harness({ + getSnapshot: async () => [ + row, + { ...row, stableId: '400', typeCode: 'R', type: 'Restricted' }, + ], + }); + await h.layer.update(h.viewer); + const legend = h.layer.getRowControls().legend; + assert.equal(legend[0].label, 'Prohibited (P)'); + assert.equal(legend[1].label, 'Restricted (R)'); + assert.equal(legend[0].count, 1); + assert.equal(legend[1].count, 1); + assert.match(legend[0].blurb, /not a substitute/); +}); diff --git a/src/layers/droneAirspace/records.js b/src/layers/droneAirspace/records.js new file mode 100644 index 000000000..c1d5f73e2 --- /dev/null +++ b/src/layers/droneAirspace/records.js @@ -0,0 +1,97 @@ +/** + * Normalize an FAA Special Use Airspace feed, skipping individually invalid + * features. Only a payload that is not a feature collection at all rejects + * the whole snapshot: a single degenerate polygon must never blank the layer. + */ + +const TYPE_LABELS = Object.freeze({ + P: 'Prohibited', + R: 'Restricted', +}); + +const finiteOrNull = (value) => (Number.isFinite(value) ? value : null); +const textOrNull = (value) => { + if (typeof value !== 'string') return null; + const text = value.trim(); + return text ? text : null; +}; + +function validRing(ring) { + if (!Array.isArray(ring) || ring.length < 4) return false; + for (const position of ring) { + if (!Array.isArray(position) || position.length < 2) return false; + const [lon, lat] = position; + if (!Number.isFinite(lon) || Math.abs(lon) > 180) return false; + if (!Number.isFinite(lat) || Math.abs(lat) > 90) return false; + } + return true; +} + +/** Normalize Polygon/MultiPolygon geometry to an array of polygons (each an + * array of rings). Returns null on malformed geometry, [] when empty. */ +function normalizePolygons(geometry) { + if (!geometry || typeof geometry !== 'object') return null; + let polygons; + if (geometry.type === 'Polygon') polygons = [geometry.coordinates]; + else if (geometry.type === 'MultiPolygon') polygons = geometry.coordinates; + else return null; + if (!Array.isArray(polygons)) return null; + const result = []; + for (const rings of polygons) { + if (!Array.isArray(rings)) return null; + if (!rings.length) continue; + if (!rings.every(validRing)) return null; + result.push(rings); + } + return result; +} + +/** Human label for a TYPE_CODE, or null when the code is not in the pack. */ +export function droneAirspaceTypeLabel(typeCode) { + return TYPE_LABELS[typeCode] || null; +} + +export function normalizeDroneAirspaceSnapshot(geojson) { + if (!Array.isArray(geojson?.features)) return null; + const rows = []; + const ids = new Set(); + for (const feature of geojson.features) { + const properties = feature?.properties; + if ( + !properties || + typeof properties !== 'object' || + Array.isArray(properties) + ) + continue; + const typeCode = textOrNull(properties.TYPE_CODE); + const type = droneAirspaceTypeLabel(typeCode); + if (!type) continue; + const polygons = normalizePolygons(feature.geometry); + if (polygons === null || !polygons.length) continue; + const objectId = finiteOrNull(Number(properties.OBJECTID)); + const featureId = + feature.id == null || feature.id === '' ? null : String(feature.id); + const stableId = + objectId != null + ? String(objectId) + : featureId && !ids.has(featureId) + ? featureId + : null; + if (stableId == null || ids.has(stableId)) continue; + ids.add(stableId); + rows.push({ + stableId, + name: textOrNull(properties.NAME), + typeCode, + type, + lowerDesc: textOrNull(properties.LOWER_DESC), + upperDesc: textOrNull(properties.UPPER_DESC), + timesOfUse: textOrNull(properties.TIMESOFUSE), + agency: textOrNull(properties.CONT_AGENT), + city: textOrNull(properties.CITY), + state: textOrNull(properties.STATE), + polygons, + }); + } + return rows; +} diff --git a/src/layers/droneAirspace/records.test.mjs b/src/layers/droneAirspace/records.test.mjs new file mode 100644 index 000000000..4c8c1a215 --- /dev/null +++ b/src/layers/droneAirspace/records.test.mjs @@ -0,0 +1,95 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + droneAirspaceTypeLabel, + normalizeDroneAirspaceSnapshot, +} from './records.js'; + +const ring = [ + [-77.04, 38.89], + [-77.03, 38.89], + [-77.03, 38.9], + [-77.04, 38.89], +]; + +function feature(overrides = {}) { + return { + id: 310, + geometry: { type: 'Polygon', coordinates: [ring] }, + properties: { + NAME: 'P-56', + TYPE_CODE: 'P', + UPPER_DESC: 'TNI', + LOWER_DESC: 'SFC', + TIMESOFUSE: 'CONTINUOUS', + CONT_AGENT: 'WASHINGTON CENTER', + CITY: 'WASHINGTON', + STATE: 'DC', + OBJECTID: 310, + ...overrides.properties, + }, + ...overrides, + }; +} + +test('a valid snapshot yields one row per prohibited or restricted zone', () => { + const rows = normalizeDroneAirspaceSnapshot({ + features: [ + feature(), + feature({ + id: 400, + properties: { + NAME: 'R-2508', + TYPE_CODE: 'R', + OBJECTID: 400, + }, + }), + ], + }); + assert.equal(rows.length, 2); + assert.deepEqual(rows[0], { + stableId: '310', + name: 'P-56', + typeCode: 'P', + type: 'Prohibited', + lowerDesc: 'SFC', + upperDesc: 'TNI', + timesOfUse: 'CONTINUOUS', + agency: 'WASHINGTON CENTER', + city: 'WASHINGTON', + state: 'DC', + polygons: [[ring]], + }); + assert.equal(rows[1].type, 'Restricted'); + assert.equal(rows[1].typeCode, 'R'); +}); + +test('MOA, warning and advisory codes are dropped, as are bad geometries', () => { + const rows = normalizeDroneAirspaceSnapshot({ + features: [ + feature({ properties: { TYPE_CODE: 'MOA', OBJECTID: 1 } }), + feature({ properties: { TYPE_CODE: 'W', OBJECTID: 2 } }), + feature({ properties: { TYPE_CODE: 'A', OBJECTID: 3 } }), + feature({ geometry: null, properties: { OBJECTID: 4 } }), + feature({ + geometry: { type: 'Point', coordinates: [-77, 38] }, + properties: { OBJECTID: 5 }, + }), + feature(), + ], + }); + assert.equal(rows.length, 1); + assert.equal(rows[0].name, 'P-56'); +}); + +test('duplicate ids and a missing collection reject or skip as documented', () => { + const dup = normalizeDroneAirspaceSnapshot({ + features: [feature(), feature({ id: 311 })], + }); + assert.equal(dup.length, 1); + assert.equal(normalizeDroneAirspaceSnapshot({}), null); + assert.equal(normalizeDroneAirspaceSnapshot(null), null); + assert.equal(droneAirspaceTypeLabel('P'), 'Prohibited'); + assert.equal(droneAirspaceTypeLabel('R'), 'Restricted'); + assert.equal(droneAirspaceTypeLabel('MOA'), null); +}); diff --git a/src/layers/droneAirspace/source.js b/src/layers/droneAirspace/source.js new file mode 100644 index 000000000..17e67b3ae --- /dev/null +++ b/src/layers/droneAirspace/source.js @@ -0,0 +1,23 @@ +import { readResponseJsonCapped } from '../../sources/httpBody.js'; + +/** Request a normalized snapshot through the bounded, same-origin FAA proxy. */ +export function createDroneAirspaceSource({ + fetchImpl = (...args) => globalThis.fetch(...args), +} = {}) { + return { + async getSnapshot({ signal } = {}) { + signal?.throwIfAborted(); + const response = await fetchImpl('/api/drone-airspace', { signal }); + if (!response.ok) throw new Error(`FAA SUA HTTP ${response.status}`); + const payload = await readResponseJsonCapped( + response, + 80 * 1024 * 1024, + signal, + ); + signal?.throwIfAborted(); + if (!Array.isArray(payload?.rows)) + throw new Error('Malformed drone airspace snapshot'); + return payload.rows; + }, + }; +} diff --git a/src/layers/droneAirspace/source.test.mjs b/src/layers/droneAirspace/source.test.mjs new file mode 100644 index 000000000..160fb1789 --- /dev/null +++ b/src/layers/droneAirspace/source.test.mjs @@ -0,0 +1,105 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createDroneAirspaceSource } from './source.js'; +import { normalizeDroneAirspaceSnapshot } from './records.js'; +import { droneAirspaceProxy } from '../../../server/providers/droneAirspace.js'; + +const ring = [ + [-77.04, 38.89], + [-77.03, 38.89], + [-77.03, 38.9], + [-77.04, 38.89], +]; +const validPayload = { + features: [ + { + id: 310, + geometry: { type: 'Polygon', coordinates: [ring] }, + properties: { + NAME: 'P-56', + TYPE_CODE: 'P', + OBJECTID: 310, + }, + }, + ], +}; + +test('a successful response yields normalized zone rows', async () => { + let requested; + const source = createDroneAirspaceSource({ + fetchImpl: async (url) => { + requested = String(url); + return Response.json({ + rows: normalizeDroneAirspaceSnapshot(validPayload), + }); + }, + }); + const rows = await source.getSnapshot(); + assert.equal(rows.length, 1); + assert.equal(rows[0].stableId, '310'); + assert.equal(requested, '/api/drone-airspace'); +}); + +test('a truncated response pages until the feed is complete', async () => { + const pageRing = (id) => ({ + id, + geometry: { type: 'Polygon', coordinates: [ring] }, + properties: { NAME: `P-${id}`, TYPE_CODE: 'P', OBJECTID: id }, + }); + const requests = []; + let handler; + const plugin = droneAirspaceProxy({ + fetchImpl: async (url) => { + requests.push(new URL(String(url)).searchParams.get('resultOffset')); + const page = requests.length; + return Response.json({ + features: [pageRing(page)], + properties: { exceededTransferLimit: page < 3 }, + }); + }, + }); + plugin.configureServer({ + middlewares: { + use: (_path, callback) => { + handler = callback; + }, + }, + }); + let payload; + await handler( + { url: '/', method: 'GET' }, + { + writeHead(status) { + assert.equal(status, 200); + }, + end(body) { + payload = JSON.parse(body); + }, + }, + ); + const rows = payload.rows; + assert.deepEqual( + rows.map((row) => row.stableId), + ['1', '2', '3'], + ); + assert.deepEqual(requests, [null, '1', '2']); +}); + +test('an upstream failure surfaces its HTTP status', async () => { + const source = createDroneAirspaceSource({ + fetchImpl: async () => ({ ok: false, status: 503 }), + }); + await assert.rejects(source.getSnapshot(), /FAA SUA HTTP 503/); +}); + +test('a malformed successful response is never accepted as an empty snapshot', async () => { + for (const payload of [{}, { rows: null }, { rows: {} }]) { + const source = createDroneAirspaceSource({ + fetchImpl: async () => Response.json(payload), + }); + await assert.rejects( + source.getSnapshot(), + /Malformed drone airspace snapshot/, + ); + } +}); diff --git a/src/sources/reference.js b/src/sources/reference.js index 4bdf7e9ac..302e15303 100644 --- a/src/sources/reference.js +++ b/src/sources/reference.js @@ -1,5 +1,6 @@ import { createUsgsEarthquakeSource } from '../layers/earthquakes/source.js'; import { createWfigsPerimeterSource } from '../layers/perimeters/source.js'; +import { createDroneAirspaceSource } from '../layers/droneAirspace/source.js'; import { createBundledCableSource } from '../layers/submarineCables/bundledSource.js'; /** Construct the existing reference feeds independently of application setup. */ @@ -7,6 +8,7 @@ export function createReferenceSources() { return { earthquakes: createUsgsEarthquakeSource(), 'fire-perimeters': createWfigsPerimeterSource(), + 'drone-airspace': createDroneAirspaceSource(), cables: createBundledCableSource(), }; } diff --git a/src/sources/reference.test.mjs b/src/sources/reference.test.mjs index 902a72c83..735b5c72e 100644 --- a/src/sources/reference.test.mjs +++ b/src/sources/reference.test.mjs @@ -15,13 +15,16 @@ test('reference factories retain compatibility without starting acquisition or s assert.deepEqual(Object.keys(first), [ 'earthquakes', 'fire-perimeters', + 'drone-airspace', 'cables', ]); assert.notEqual(first.earthquakes, second.earthquakes); assert.notEqual(first['fire-perimeters'], second['fire-perimeters']); + assert.notEqual(first['drone-airspace'], second['drone-airspace']); assert.notEqual(first.cables, second.cables); assert.equal(typeof first.earthquakes.getSnapshot, 'function'); assert.equal(typeof first['fire-perimeters'].getSnapshot, 'function'); + assert.equal(typeof first['drone-airspace'].getSnapshot, 'function'); assert.equal(typeof first.cables.fetch, 'function'); assert.equal(requests, 0); }); diff --git a/src/tooling/previewServing.test.mjs b/src/tooling/previewServing.test.mjs index d6eb80e20..b88b4e1f8 100644 --- a/src/tooling/previewServing.test.mjs +++ b/src/tooling/previewServing.test.mjs @@ -10,6 +10,7 @@ import { makeFixtureRoot } from './fixtureRoot.mjs'; test('data providers have both hooks; credential editing stays development-only', () => { const plugins = localProviderPlugins(); assert.ok(plugins.some(({ name }) => name === 'fire-perimeters')); + assert.ok(plugins.some(({ name }) => name === 'drone-airspace')); for (const plugin of plugins) { if (plugin.name === 'gev-key-setup') { assert.equal(plugin.configurePreviewServer, undefined); diff --git a/src/ui/layerPanel.js b/src/ui/layerPanel.js index e4c240f23..ecbf42cca 100644 --- a/src/ui/layerPanel.js +++ b/src/ui/layerPanel.js @@ -39,6 +39,7 @@ const PANEL_GROUPS = [ ids: [ 'alpr-cameras', 'military-installations', + 'drone-airspace', 'local-datacenters', 'telegeography-submarine-cables', 'local-dams', diff --git a/src/voice/layerManifest.js b/src/voice/layerManifest.js index 1e348ee41..facc90e32 100644 --- a/src/voice/layerManifest.js +++ b/src/voice/layerManifest.js @@ -327,6 +327,30 @@ const layers = [ context: true, query: { fields: { name: ['t'], class: ['t'], kind: ['t'] } }, }, + { + id: 'drone-airspace', + aliases: [ + 'drone restrictions', + 'drone airspace', + 'no-fly zones', + 'no fly zones', + 'drones', + ], + context: false, + query: { + fields: { + name: ['t'], + typeCode: ['t'], + type: ['t'], + lowerDesc: ['t'], + upperDesc: ['t'], + timesOfUse: ['t'], + agency: ['t'], + city: ['t'], + state: ['t'], + }, + }, + }, { id: 'local-datacenters', aliases: ['datacenters', 'data centers', 'data centres'], diff --git a/src/voice/layerManifest.test.mjs b/src/voice/layerManifest.test.mjs index d0222b934..98e3889bf 100644 --- a/src/voice/layerManifest.test.mjs +++ b/src/voice/layerManifest.test.mjs @@ -20,14 +20,28 @@ test('every catalog layer is in the voice manifest or explicitly voice-off', () const missing = catalogIds.filter( (id) => !manifestIds.includes(id) && !Object.hasOwn(VOICE_OFF_LAYERS, id), ); - assert.deepEqual(missing, [], `add a voice manifest entry for: ${missing.join(', ')}`); + assert.deepEqual( + missing, + [], + `add a voice manifest entry for: ${missing.join(', ')}`, + ); for (const id of Object.keys(VOICE_OFF_LAYERS)) { - assert.ok(catalogIds.includes(id), `${id} is voice-off but not in the catalog`); - assert.ok(!manifestIds.includes(id), `${id} is both voice-off and in the manifest`); + assert.ok( + catalogIds.includes(id), + `${id} is voice-off but not in the catalog`, + ); + assert.ok( + !manifestIds.includes(id), + `${id} is both voice-off and in the manifest`, + ); } const unknown = manifestIds.filter((id) => !catalogIds.includes(id)); assert.deepEqual(unknown, [], 'the manifest names only catalog layers'); - assert.equal(new Set(manifestIds).size, manifestIds.length, 'one entry per layer'); + assert.equal( + new Set(manifestIds).size, + manifestIds.length, + 'one entry per layer', + ); }); test('each layer declares a query or the reason it has none', () => { @@ -41,7 +55,8 @@ test('each layer declares a query or the reason it has none', () => { }); test('tool enums are generated from the manifest', () => { - const toggle = tool('set_layer_visibility').parameters.properties.layerId.enum; + const toggle = tool('set_layer_visibility').parameters.properties.layerId + .enum; assert.deepEqual(toggle, [...VOICE_TOGGLE_LAYER_IDS]); for (const id of [ 'transit', @@ -57,9 +72,13 @@ test('tool enums are generated from the manifest', () => { 'rocket-launches', 'bikeshare', 'local-adsb', + 'drone-airspace', ]) assert.ok(toggle.includes(id), `${id} is voice-toggleable`); - assert.ok(!toggle.includes('military-awareness'), 'Global Context is reached through set_context_mode'); + assert.ok( + !toggle.includes('military-awareness'), + 'Global Context is reached through set_context_mode', + ); assert.deepEqual( tool('show_data_layers_menu').parameters.properties.layerId.enum, [...VOICE_TOGGLE_LAYER_IDS], @@ -68,47 +87,98 @@ test('tool enums are generated from the manifest', () => { tool('get_entity_context').parameters.properties.layerId.enum, [...VOICE_CONTEXT_LAYER_IDS], ); - assert.ok(!VOICE_CONTEXT_LAYER_IDS.includes('telegeography-submarine-cables'), 'cables register no context records'); + assert.ok( + !VOICE_CONTEXT_LAYER_IDS.includes('telegeography-submarine-cables'), + 'cables register no context records', + ); assert.deepEqual( tool('analyst_query').parameters.properties.layers.items.enum, [...VOICE_QUERY_LAYER_IDS], ); assert.deepEqual(Object.keys(ANALYST_LAYERS), [...VOICE_QUERY_LAYER_IDS]); const panels = tool('set_panel_open').parameters.properties.panelId.enum; - assert.ok(panels.includes('weather-panel') && panels.includes('recent-imagery-panel')); + assert.ok( + panels.includes('weather-panel') && panels.includes('recent-imagery-panel'), + ); }); test('spoken aliases resolve to real layers, keep every legacy phrase and never collide', () => { const aliases = voiceLayerAliasMap(); for (const id of aliases.values()) assert.ok(manifestIds.includes(id), id); const legacy = { - planes: 'flights', aircraft: 'flights', 'military flights': 'military', quakes: 'earthquakes', - 'space mission': 'rocket-launches', 'space missions': 'rocket-launches', missions: 'rocket-launches', - 'street traffic': 'traffic', cameras: 'cctv', 'internet radio': 'radio', 'radio stations': 'radio', - bikes: 'bikeshare', ais: 'ais-live-vessels', ships: 'ais-live-vessels', vessels: 'ais-live-vessels', - 'live vessels': 'ais-live-vessels', datacenters: 'local-datacenters', 'data centers': 'local-datacenters', - 'data centres': 'local-datacenters', dams: 'local-dams', 'submarine cables': 'telegeography-submarine-cables', - cables: 'telegeography-submarine-cables', telegeography: 'telegeography-submarine-cables', - firms: 'local-firms', fires: 'local-firms', 'active fires': 'local-firms', alpr: 'alpr-cameras', - 'alpr cameras': 'alpr-cameras', 'flock cameras': 'alpr-cameras', 'license plate readers': 'alpr-cameras', - 'license plate cameras': 'alpr-cameras', 'plate readers': 'alpr-cameras', 'local-adsb': 'local-adsb', - 'local adsb': 'local-adsb', 'local ads-b': 'local-adsb', 'my receiver': 'local-adsb', - 'my antenna': 'local-adsb', 'my sdr': 'local-adsb', + planes: 'flights', + aircraft: 'flights', + 'military flights': 'military', + quakes: 'earthquakes', + 'space mission': 'rocket-launches', + 'space missions': 'rocket-launches', + missions: 'rocket-launches', + 'street traffic': 'traffic', + cameras: 'cctv', + 'internet radio': 'radio', + 'radio stations': 'radio', + bikes: 'bikeshare', + ais: 'ais-live-vessels', + ships: 'ais-live-vessels', + vessels: 'ais-live-vessels', + 'live vessels': 'ais-live-vessels', + datacenters: 'local-datacenters', + 'data centers': 'local-datacenters', + 'data centres': 'local-datacenters', + dams: 'local-dams', + 'submarine cables': 'telegeography-submarine-cables', + cables: 'telegeography-submarine-cables', + telegeography: 'telegeography-submarine-cables', + firms: 'local-firms', + fires: 'local-firms', + 'active fires': 'local-firms', + alpr: 'alpr-cameras', + 'alpr cameras': 'alpr-cameras', + 'flock cameras': 'alpr-cameras', + 'license plate readers': 'alpr-cameras', + 'license plate cameras': 'alpr-cameras', + 'plate readers': 'alpr-cameras', + 'local-adsb': 'local-adsb', + 'local adsb': 'local-adsb', + 'local ads-b': 'local-adsb', + 'my receiver': 'local-adsb', + 'my antenna': 'local-adsb', + 'my sdr': 'local-adsb', }; - for (const [phrase, id] of Object.entries(legacy)) assert.equal(aliases.get(phrase), id, phrase); + for (const [phrase, id] of Object.entries(legacy)) + assert.equal(aliases.get(phrase), id, phrase); const added = { - buses: 'transit', trains: 'transit', hurricanes: 'weather-cyclones', cyclones: 'weather-cyclones', - 'weather radar': 'weather-radar', lightning: 'weather-lightning', 'military bases': 'military-installations', - 'recent imagery': 'recent-imagery', 'bike share': 'bikeshare', 'rocket launches': 'rocket-launches', - 'street level': 'street-level', mapillary: 'street-level', - wind: 'wind', directions: 'directions', 'fire perimeters': 'fire-perimeters', - 'wildfire perimeters': 'fire-perimeters', 'burn areas': 'fire-perimeters', + buses: 'transit', + trains: 'transit', + hurricanes: 'weather-cyclones', + cyclones: 'weather-cyclones', + 'weather radar': 'weather-radar', + lightning: 'weather-lightning', + 'military bases': 'military-installations', + 'recent imagery': 'recent-imagery', + 'bike share': 'bikeshare', + 'rocket launches': 'rocket-launches', + 'street level': 'street-level', + mapillary: 'street-level', + wind: 'wind', + directions: 'directions', + 'fire perimeters': 'fire-perimeters', + 'wildfire perimeters': 'fire-perimeters', + 'burn areas': 'fire-perimeters', + 'drone restrictions': 'drone-airspace', + 'drone airspace': 'drone-airspace', + 'no-fly zones': 'drone-airspace', + drones: 'drone-airspace', }; - for (const [phrase, id] of Object.entries(added)) assert.equal(aliases.get(phrase), id, phrase); + for (const [phrase, id] of Object.entries(added)) + assert.equal(aliases.get(phrase), id, phrase); const claimed = new Map(); for (const entry of VOICE_LAYER_MANIFEST) for (const alias of entry.aliases) { - assert.ok(!claimed.has(alias), `"${alias}" claimed by ${claimed.get(alias)} and ${entry.id}`); + assert.ok( + !claimed.has(alias), + `"${alias}" claimed by ${claimed.get(alias)} and ${entry.id}`, + ); claimed.set(alias, entry.id); } }); From a275c2c63047964de9b2ba5241d7e6cb450edb40 Mon Sep 17 00:00:00 2001 From: Matt Van Horn Date: Mon, 12 Oct 2026 19:39:15 +0000 Subject: [PATCH 02/83] test: pin voice schema and README layer count for drone airspace --- src/tooling/transitQa.test.mjs | 2 +- src/voice/actionSchemas.test.mjs | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/tooling/transitQa.test.mjs b/src/tooling/transitQa.test.mjs index 2ec277748..4e79bffbf 100644 --- a/src/tooling/transitQa.test.mjs +++ b/src/tooling/transitQa.test.mjs @@ -675,7 +675,7 @@ test('public transit documentation uses courtesy attribution and the approved gr ); assert.ok( readme.includes( - 'Twenty layers and map sources. **Seventeen have a keyless path.**', + 'Twenty-one layers and map sources. **Eighteen have a keyless path.**', ), ); assert.ok( diff --git a/src/voice/actionSchemas.test.mjs b/src/voice/actionSchemas.test.mjs index d3b8e8a90..a62c292f7 100644 --- a/src/voice/actionSchemas.test.mjs +++ b/src/voice/actionSchemas.test.mjs @@ -37,7 +37,8 @@ test('the complete Realtime tool payload pins the manifest-generated layer relea // omit top-level anyOf (OpenAI rejects it). Contacts list routing preserves // requested radius and center authority in descriptions only. // Street Level adds its toggle enum values and generated alias hint. - '9eb113454c5af5e32e27ffcdc7919d0e4600ba1d204f8f175e071152be41ad0f', + // Drone Restrictions adds toggle/query enum values and spoken aliases. + '3ceef7975c1437eacb2c602bf9728829b299532b7dd95534cf7d9eb47115fa54', ); }); From 052435473e878deb59b91b4bb0a009474f6d6feb Mon Sep 17 00:00:00 2001 From: Matt Van Horn Date: Wed, 14 Oct 2026 01:51:25 +0000 Subject: [PATCH 03/83] feat(cctv): add Houston TranStar camera pack Register the keyless TranStar snapshot catalog as a live CCTV source, synthesize frame URLs from validated numeric camera ids, and keep the nearest cameras to Houston's metro cores under the existing catalog ceiling. --- .env.example | 3 + CHANGELOG.md | 11 + DATA_SOURCES.md | 2 + README.md | 2 +- docs/CURRENT-STATE.md | 19 +- scripts/format-scope.json | 1 + server/providers/cctv/catalog.js | 6 + server/providers/cctv/constants.js | 31 +++ server/providers/cctv/normalize.js | 12 + server/providers/cctv/sources.js | 143 +++++++++++ src/data/cctvHouston.test.mjs | 379 +++++++++++++++++++++++++++++ src/data/dataCredits.js | 6 + 12 files changed, 613 insertions(+), 2 deletions(-) create mode 100644 src/data/cctvHouston.test.mjs diff --git a/.env.example b/.env.example index a2f8afa24..265377bf7 100644 --- a/.env.example +++ b/.env.example @@ -237,6 +237,9 @@ AISSTREAM_API_KEY= # CCTV_TORONTO_LIST_URL= # override the Toronto RESCU camera list URL # CCTV_LITHUANIA_ENABLED= # off by default; set 1 to load Via Lietuva / Eismoinfo road cameras (photo licence unconfirmed; see DATA_SOURCES.md) # CCTV_LITHUANIA_MAX_SOURCES=400 # cap Lithuanian cameras; no API key required +# CCTV_HOUSTON_ENABLED=1 # set 0 to disable the Houston TranStar cameras +# CCTV_HOUSTON_MAX_SOURCES=250 # cap cameras loaded from the Houston TranStar catalog (nearest metro cores first) +# CCTV_HOUSTON_CAMERAS_URL= # override the Houston TranStar catalog URL # CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs # CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry # CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured diff --git a/CHANGELOG.md b/CHANGELOG.md index 98d32365f..b4de32107 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,16 @@ # Changelog +- Add Houston traffic cameras from Houston TranStar as a CCTV pack. The camera + list is one keyless JSON request to + `traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js`; frames are + stills synthesized from each numeric camera id on `www.houstontranstar.org`, + so a catalog field cannot steer the proxy off-host. Cameras without a + snapshot id, and cameras outside Greater Houston, are dropped. By default + the 250 cameras nearest downtown, Uptown, the Medical Center, IAH and + Galveston load. `CCTV_HOUSTON_ENABLED=0` turns the pack off and + `CCTV_HOUSTON_MAX_SOURCES` changes the cap. The catalog-wide ceiling is + unchanged; extra cameras share it round-robin with the other packs. + - The tracked-aircraft descriptor now publishes `verticalRateMps`, on both the civil and military layers, so a consumer reading `getTrackedInfo()` no longer has to reach past it into the raw poll record for climb/descent. Reported, diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 0e04fb39b..913cad6d1 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -65,6 +65,7 @@ How to read this: | **Statens vegvesen** (catalog `ogckart-sn1.atlas.vegvesen.no`, frames `kamera.atlas.vegvesen.no`, HLS `kamera.vegvesen.no`) | CCTV camera catalog + frames + live HLS video, Norway road cameras | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) distributed by Statens vegvesen" | | **NYC DOT Traffic Management Center** (`webcams.nyctmc.org`) | CCTV camera catalog + frames, New York City five boroughs | NYC DOT requires developers to sign a [data-sharing agreement](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) for the camera feed. **Off by default**; enable only under your own agreement | "NYC DOT Traffic Management Center" + NYC DOT's required application disclaimer (in the Data attribution popover) | | **City of Toronto Open Data** (`opendata.toronto.ca`) | CCTV camera catalog + frames, Toronto city streets | [Open Government Licence – Toronto](https://www.toronto.ca/city-government/data-research-maps/open-data/open-data-licence/) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – Toronto" | +| **Houston TranStar** (`traffic.houstontranstar.org`, frames `www.houstontranstar.org`) | CCTV camera catalog + frames, Greater Houston | Public Houston TranStar traffic camera stills | "Houston TranStar" (courtesy) | | **GBFS (Lyft / BCycle / Social Bicycles / PBSC)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` | | **Bike Share Toronto** (`toronto.publicbikesystem.net`) | Bikeshare station availability, Toronto | Operator GBFS feed. No licence is published: the City's open-data package declares none and the feed has no `license_url`. Attribution is given to the operator. | "Bike Share Toronto (Toronto Parking Authority)" | | **Bike Itaú / Tembici** (`*.publicbikesystem.net`, seven Brazilian cities, Santiago and Bogotá) | Bikeshare station availability: São Paulo, Rio de Janeiro, Salvador, Recife, Porto Alegre, Brasília, Curitiba, Santiago de Chile and Bogotá | Operator GBFS feeds. No licence is published: the feeds' `system_information.json` carries no `license_url`, operator or attribution fields. Attribution is given to the operator. | "Bike Itaú (Tembici)"; "Tembici Bogotá" | @@ -166,6 +167,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei - **Statens vegvesen road cameras.** Keyless. The camera list is one GeoJSON request to the OGC API Features view of the official DATEX 3.1 CCTV site table (`ogckart-sn1.atlas.vegvesen.no/ogc/features/v1/collections/datex_3_1:CctvSimple/items`), so no DATEX account is needed; frames are stills on `kamera.atlas.vegvesen.no/api/images/`, and only that exact path per camera is registered. Cameras the feed reports as faulty are dropped. About 135 working cameras also publish live HLS (`kamera.vegvesen.no/public//manifest.m3u8`, MPEG-TS); those play as live video through the bounded in-memory HLS relay, with the still as the snapshot fallback, and only that exact manifest path per camera is registered. `CCTV_VEGVESEN_VIDEO=0` keeps every camera on stills. `orientationDescription` names the place a camera faces, not a bearing, so it goes into the label and headings use the id-hash fallback at low confidence. By default every live camera loads (~850); `CCTV_VEGVESEN_MAX_SOURCES` (8–1000) lowers the cap, keeping the cameras nearest Oslo, Bergen, Stavanger, Trondheim, Kristiansand, Bodø and Tromsø and `CCTV_VEGVESEN_ENABLED=0` turns the pack off. Frames are fetched at request time and never stored or redistributed. The NLOD attribution is registered in the Data attribution popover. - **NYC DOT Traffic Management Center cameras.** Off by default. NYC DOT's [developer data feeds page](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) asks developers who want the camera feed to contact `tmcdot@dot.nyc.gov` and sign a data-sharing agreement. This project does not hold one that covers downstream installations, so the pack loads only when an operator who holds their own agreement sets `CCTV_NYC_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to NYC DOT. The endpoint needs no key. The camera list is the public `webcams.nyctmc.org/api/cameras` catalog (~980 cameras across the five boroughs); frames are stills at `webcams.nyctmc.org/api/cameras//image`, synthesized from a validated UUID rather than taken from the payload, and only that host is registered. Cameras the feed reports as offline are dropped. The catalog publishes no camera facing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. Once enabled, the 250 cameras nearest the borough cores load; `CCTV_NYC_MAX_SOURCES` (8–400) changes the cap. Frames are fetched at request time and never stored or redistributed. NYC DOT asks every application that uses its data to carry its disclaimer; it is registered verbatim in the Data attribution popover. - **City of Toronto RESCU cameras.** Keyless. The camera list is the City's public `tmcearthcameras.json` on `opendata.toronto.ca` (dataset "Traffic Cameras", Transportation Services), served as JSONP and unwrapped with an anchored pattern. This is the municipal network the Ontario 511 pack does not carry: downtown arterials plus the Gardiner, DVP, Allen and Lake Shore. The list publishes no frame URL, so frame URLs are BUILT from the documented `CameraImages/loc####.jpg` convention and a strictly-digits camera number, pinning the frame proxy to the portal origin by construction; the list fetch refuses redirects and bounds the body it buffers. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `D1`-`D4` fields are the directions of the *comparison* images (static reference stills captured in 2016 under a different path), not a bearing for the live pan-tilt-zoom frame, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. A camera that is temporarily out of service still answers 200 with a byte-identical "Image Unavailable" placeholder rather than an error (about 6% of the network at any moment), so such a camera reads as healthy; the frame it shows is the agency's, not a synthesized one. The licence requires the attribution statement above. Disable with `CCTV_TORONTO_ENABLED=0`. +- **Houston TranStar cameras.** Keyless. The camera list is the public `traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js` catalog (~1,000 cameras across Greater Houston); frames are stills at `www.houstontranstar.org/snapshots/cctv/.jpg`, synthesized from a validated numeric id rather than taken from the payload, and only that host is registered. Cameras whose catalog id is `0` (a different, non-snapshot class) and cameras outside Greater Houston are dropped. Dedicated `dir` values (North/South/East/West) become high-confidence headings; `None` uses the shared id-hash fallback at low confidence. By default the 250 cameras nearest downtown, Uptown, the Medical Center, IAH and Galveston load; `CCTV_HOUSTON_MAX_SOURCES` (8–400) changes the cap and `CCTV_HOUSTON_ENABLED=0` turns the pack off. Frames are fetched at request time and never stored or redistributed. - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals. - **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All sixteen registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. The TTC feed is BusTime, the surface system: it carries buses and streetcars and no subway at all. About four in ten of its vehicles report no route at all (unassigned or deadheading), and those are shown with the feed default rather than a route-established mode. It sends no ETag or Last-Modified, so its conditional requests fall through to the ordinary snapshot cache. Three Golden Horseshoe neighbours run alongside it — MiWay, Hamilton HSR and Durham Region Transit — each bus-only and each polled only near its own region. Durham Region Transit publishes no bearing on any vehicle, so those sprites take their heading from observed movement like any other feed that omits it. Guelph, Kingston and Barrie extend the same pattern eastward and north. Hamilton, Guelph, Burlington and YRT publish route ids that are not the number on the vehicle, so those entries carry a route-name table taken from each operator’s own published GTFS; any id missing from a table renders unchanged rather than blank. A label is never invented: YRT runs a route `9101` that appears nowhere in its published routes, so it stays as the feed sends it. Burlington’s is the one that cannot be shortcut: `351` is route 1 while `3510` is route 10, so the table is a real join rather than a trimmed prefix. GO Transit / Metrolinx is deliberately absent: its REALTIME feed requires an API key issued by manual review, which the keyless admission rule excludes (its static schedule download is keyless, but carries no vehicle positions). Barrie and YRT/Viva each invite acceptance of their licence through a form, and each licence also states that using the data is itself acceptance; this deployment relies on that clause, which is recorded in both entries. Grand River Transit in Waterloo is absent for a different reason: it answers, but its feed header was months stale and it publishes no per-vehicle timestamps, so it would render vehicles that look live and are not. Ottawa’s OC Transpo falls to the same keyless admission rule as GO Transit — its realtime feed requires an API key. London Transit and Thunder Bay Transit publish vehicle positions over plain HTTP with no HTTPS endpoint available. Every other Ontario system — Brampton, Oakville, Niagara, Milton, Windsor, Greater Sudbury, UP Express and a dozen smaller operators — publishes static GTFS schedules only, with no realtime vehicle positions to register. Between the nine included and these exclusions, every Ontario realtime vehicle-position feed catalogued in the Mobility Database is accounted for. - **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply. diff --git a/README.md b/README.md index c49c53ece..867f14824 100644 --- a/README.md +++ b/README.md @@ -377,7 +377,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi | 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 | | ◍ **Air quality** | Live AQHI on the published 1–10+ health-risk scale, with a Weather-panel card showing the band legend and the highest readings in view. **Canada only** — ECCC is the first registered provider | ECCC MSC GeoMet | 🟢 | | 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds | -| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement) and opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | +| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit) · Houston (TranStar), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement) and opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | | 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 | | 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer and **FOLLOW**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key | | 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 | diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 8cb7b6d17..99cfc3234 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1072,6 +1072,23 @@ By default the 250 cameras nearest downtown Vancouver and Victoria load. The Open Government Licence – British Columbia attribution is registered in the Data attribution popover. +## Houston TranStar CCTV source pack + +The CCTV catalog adds Greater Houston traffic cameras from Houston TranStar +alongside the other live packs. The server reads the keyless camera list at +`https://traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js` and +keeps cameras with a numeric snapshot id inside the metro bounding box. Frame +URLs are built from each camera id under +`https://www.houstontranstar.org/snapshots/cctv/.jpg`; they are never taken +from the payload. Dedicated `dir` values become high-confidence headings; +`None` uses the shared id-hash fallback at low confidence. + +By default the 250 cameras nearest downtown, Uptown, the Medical Center, IAH +and Galveston load. `CCTV_HOUSTON_MAX_SOURCES` sets the pack cap (8–400) and +`CCTV_HOUSTON_ENABLED=0` turns the pack off. The catalog-wide ceiling stays at +4,000 and is shared round-robin, so adding this pack does not silently drop +another region. + ## Location control ownership City/POI rows, search/reset bindings, location readouts and the orbit indicator @@ -3366,7 +3383,7 @@ its criteria cannot be silently ignored. | Satellites | CelesTrak | `src/data/satellites.js` | `/api/celestrak` | 120s | | Space Missions (30d) | Launch Library 2 + CelesTrak | `src/data/rocketLaunches.js` | `/api/launches` + `/api/celestrak/active` | 5 min | | Traffic | Selectable TomTom / OpenStreetMap / Hybrid roads; optional TomTom flow (BYOK) | `src/data/traffic.js` | browser-direct OpenFreeMap tiles; `/api/tomtom` for flow | viewport-driven; capped tile cache | -| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) | +| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Houston TranStar + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) | | Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh | | Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) | | Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto, Bike Itaú (Brazil) and Bicing (Barcelona) + Urban Sharing, Entur, JCDecaux NO + nextbike (AMBici)) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 8bcc4c04d..84002f2cf 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -552,6 +552,7 @@ "src/data/cctvCataloniaSource.test.mjs", "src/data/cctvVegvesen.test.mjs", "src/data/cctvNyc.test.mjs", + "src/data/cctvHouston.test.mjs", "src/data/cctvNswSource.test.mjs", "src/data/cctvTxdotSource.test.mjs", "src/layers/cctv/source.test.mjs", diff --git a/server/providers/cctv/catalog.js b/server/providers/cctv/catalog.js index 75e58d2a6..0ed194660 100644 --- a/server/providers/cctv/catalog.js +++ b/server/providers/cctv/catalog.js @@ -25,6 +25,7 @@ import { loadVegvesenSourcesFromOpenData, loadNycSourcesFromOpenData, loadTorontoSourcesFromOpenData, + loadHoustonSourcesFromOpenData, } from './sources.js'; /** Env kill switch: unset or anything but "0" means enabled. */ @@ -144,6 +145,11 @@ const LIVE_PACKS = [ enabled: () => envOptIn('CCTV_LITHUANIA_ENABLED'), load: loadLithuaniaSources, }, + { + name: 'houston', + enabled: () => envEnabled('CCTV_HOUSTON_ENABLED'), + load: loadHoustonSourcesFromOpenData, + }, ]; /** * Load CCTV sources from a local JSON file (CCTV_SOURCES_FILE env or default). diff --git a/server/providers/cctv/constants.js b/server/providers/cctv/constants.js index 20b963770..43d804963 100644 --- a/server/providers/cctv/constants.js +++ b/server/providers/cctv/constants.js @@ -347,6 +347,37 @@ export const NYC_ANCHORS = [ { lat: 40.5795, lon: -74.1502 }, // Staten Island ]; +/** Houston TranStar cameras: one keyless JSON list for the metro (~1,000 + * cameras with a numeric id). Frames are stills synthesized from that id on + * the city's snapshot host — never taken from the payload. The catalog is + * served as `application/javascript`; asking for `application/json` gets HTTP + * 406, so the loader accepts any media type. */ +export const HOUSTON_CAMERAS_URL = + 'https://traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js'; +/** The only origin Houston camera frames may come from. The catalog's `url` + * is a `showWin(...)` viewer helper and is ignored; the registered path is + * always this origin plus `/snapshots/cctv/.jpg`. */ +export const HOUSTON_IMAGE_ORIGIN = 'https://www.houstontranstar.org/'; +export const HOUSTON_IMAGE_URL = (id) => + `${HOUSTON_IMAGE_ORIGIN}snapshots/cctv/${id}.jpg`; +/** Numeric TranStar snapshot ids: 3–5 digits, no leading zero. Id `0` is a + * different (non-snapshot) camera class in the same list. */ +export const HOUSTON_CAMERA_ID_RE = /^[1-9]\d{2,4}$/; +export const DEFAULT_HOUSTON_MAX_SOURCES = 250; +export const HOUSTON_MAX_CATALOG_BYTES = 4 * 1024 * 1024; +/** Ground-elevation prior in metres. The TranStar payload carries none, and + * Houston sits a few metres above sea level. */ +export const HOUSTON_GROUND_ELEVATION_M = 15; +/** Downtown Houston (Main @ Texas): the primary prioritization anchor. */ +export const HOUSTON_DOWNTOWN = { lat: 29.7604, lon: -95.3698 }; +export const HOUSTON_ANCHORS = [ + { lat: 29.7604, lon: -95.3698 }, // Downtown + { lat: 29.739, lon: -95.461 }, // Uptown / Galleria + { lat: 29.71, lon: -95.398 }, // Texas Medical Center + { lat: 29.9902, lon: -95.3368 }, // IAH + { lat: 29.3013, lon: -94.7977 }, // Galveston +]; + /** DelDOT CCTV: one keyless statewide JSON catalog; live video via RTMP-over-HTTP (rtmpt:80). */ export const DELDOT_CCTV_URL = 'https://tmc.deldot.gov/json/videocamera.json'; export const DEFAULT_DELDOT_MAX_SOURCES = 300; diff --git a/server/providers/cctv/normalize.js b/server/providers/cctv/normalize.js index 2d4e16e11..1b0e79a0c 100644 --- a/server/providers/cctv/normalize.js +++ b/server/providers/cctv/normalize.js @@ -388,6 +388,18 @@ export function isLikelyNycCoordinate(lat, lon) { ); } +/** Greater Houston (the Woodlands to Galveston, Katy to Baytown). Farther + * TranStar partners (Dallas, Columbus) are out of this city pack. */ +export function isLikelyHoustonCoordinate(lat, lon) { + return ( + isPlausibleLatLon(lat, lon) && + lat >= 29.2 && + lat <= 30.35 && + lon >= -96.05 && + lon <= -94.7 + ); +} + /** Mainland Norway, with slack for the coast and the Finnmark border. */ export function isLikelyNorwayCoordinate(lat, lon) { return ( diff --git a/server/providers/cctv/sources.js b/server/providers/cctv/sources.js index c3d11e752..895fe4c4d 100644 --- a/server/providers/cctv/sources.js +++ b/server/providers/cctv/sources.js @@ -96,6 +96,13 @@ import { NYC_MAX_CATALOG_BYTES, NYC_GROUND_ELEVATION_M, NYC_ANCHORS, + HOUSTON_CAMERAS_URL, + HOUSTON_IMAGE_URL, + HOUSTON_CAMERA_ID_RE, + DEFAULT_HOUSTON_MAX_SOURCES, + HOUSTON_MAX_CATALOG_BYTES, + HOUSTON_GROUND_ELEVATION_M, + HOUSTON_ANCHORS, } from './constants.js'; import { toFiniteNumber, @@ -116,6 +123,7 @@ import { isLikelyCataloniaCoordinate, decodeNumericEntities, isLikelyNycCoordinate, + isLikelyHoustonCoordinate, isLikelyNorwayCoordinate, isLikelyTorontoCoordinate, cameraDisplayCode, @@ -2784,3 +2792,138 @@ export async function loadTorontoSourcesFromOpenData() { return []; } } + +/** + * Label for one Houston TranStar camera: the interchange an operator + * recognises ("10 EAST @ SAN JACINTO"). + * + * @param {object} record - Raw TranStar camera row. + * @param {string} cameraId - Derived stable id. + * @returns {string} + */ +export function houstonCameraName(record, cameraId) { + const name = String(record?.location ?? '').trim(); + if (name) return name; + const short = String(cameraId || '').replace(/^houston-/, ''); + return short ? `Houston Camera ${short}` : 'Houston Camera'; +} + +/** + * One Houston TranStar camera row -> one catalog source, or null. + * + * Frame URLs are synthesized from a validated numeric id on the pinned + * snapshot origin. The payload's `url` is a `showWin(...)` viewer helper + * and is ignored so a catalog edit cannot steer the frame proxy off-host. + * Dedicated `dir` values (North/South/East/West) become high-confidence + * headings; `None`/blank fall back to the shared id-hash heading. + * + * @param {object} record - Raw TranStar camera row. + * @returns {?object} + */ +export function houstonCameraToSource(record) { + if (!record || typeof record !== 'object') return null; + + const lat = toFiniteNumber(record.lat); + const lon = toFiniteNumber(record.lng); + if (!isLikelyHoustonCoordinate(lat, lon)) return null; + + const rawId = String(record.id ?? '').trim(); + if (!HOUSTON_CAMERA_ID_RE.test(rawId)) return null; + const cameraId = `houston-${rawId}`; + const imageUrl = HOUSTON_IMAGE_URL(rawId); + const name = houstonCameraName(record, cameraId); + const extractedHeading = directionToHeading(record.dir, true); + const hasHeading = Number.isFinite(extractedHeading); + + return { + id: cameraId, + name, + city: 'Houston', + cityId: 'houston', + provider: 'Houston TranStar', + lat, + lon, + headingDeg: hasHeading ? extractedHeading : fallbackHeadingFromId(cameraId), + headingConfidence: hasHeading ? 'high' : 'low', + pitchDeg: hasHeading ? -24 : -18, + fovDeg: hasHeading ? 56 : 44, + rangeM: hasHeading ? 210 : 145, + mountHeightM: hasHeading ? 10 : 8, + groundElevationM: HOUSTON_GROUND_ELEVATION_M, + feedType: 'image', + url: imageUrl, + snapshotUrl: imageUrl, + sourceKind: 'houston-transtar', + license: 'Public Houston TranStar traffic camera', + code: cameraDisplayCode(name.toUpperCase()), + }; +} + +/** + * Fetch Houston TranStar cameras from the keyless `cctvSnapshots_json.js` + * catalog. Frames are stills on `www.houstontranstar.org`, registered from + * a validated numeric id rather than from the payload URL. The catalog host + * answers `Accept: application/json` with HTTP 406, so this request accepts + * any media type. + * + * @returns {Promise>} Normalized camera source objects. + */ +export async function loadHoustonSourcesFromOpenData() { + try { + const endpoint = + process.env.CCTV_HOUSTON_CAMERAS_URL || HOUSTON_CAMERAS_URL; + const resp = await fetch(endpoint, { + headers: { Accept: '*/*' }, + redirect: 'manual', + signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS), + }); + const discard = async () => { + try { + await resp.body?.cancel(); + } catch { + /* no-op */ + } + return []; + }; + if (resp.status >= 300 && resp.status < 400) { + console.warn( + '[CCTV] Houston catalog redirected; redirects are not followed', + ); + return discard(); + } + if (!resp.ok) { + console.warn('[CCTV] Houston camera download failed:', resp.status); + return discard(); + } + const payload = await readResponseJsonCapped( + resp, + HOUSTON_MAX_CATALOG_BYTES, + ); + const rows = Array.isArray(payload?.cameras) ? payload.cameras : []; + const cameras = []; + const seen = new Set(); + for (const record of rows) { + const camera = houstonCameraToSource(record); + if (!camera || seen.has(camera.id)) continue; + seen.add(camera.id); + cameras.push(camera); + } + const maxRaw = Number( + process.env.CCTV_HOUSTON_MAX_SOURCES || DEFAULT_HOUSTON_MAX_SOURCES, + ); + const maxCount = Number.isFinite(maxRaw) + ? Math.max(8, Math.min(400, Math.floor(maxRaw))) + : DEFAULT_HOUSTON_MAX_SOURCES; + const prioritized = prioritizeSources(cameras, maxCount, HOUSTON_ANCHORS); + console.log( + `[CCTV] Loaded Houston camera sources: ${cameras.length} (using nearest ${prioritized.length})`, + ); + return prioritized; + } catch (error) { + console.warn( + '[CCTV] Houston camera download error:', + error?.message || error, + ); + return []; + } +} diff --git a/src/data/cctvHouston.test.mjs b/src/data/cctvHouston.test.mjs new file mode 100644 index 000000000..97de95a84 --- /dev/null +++ b/src/data/cctvHouston.test.mjs @@ -0,0 +1,379 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; + +import { + houstonCameraName, + houstonCameraToSource, + loadHoustonSourcesFromOpenData, +} from '../../server/providers/cctv/sources.js'; +import { + DEFAULT_CCTV_MAX_SOURCES, + DEFAULT_HOUSTON_MAX_SOURCES, + HOUSTON_CAMERAS_URL, + HOUSTON_GROUND_ELEVATION_M, + HOUSTON_IMAGE_ORIGIN, + HOUSTON_IMAGE_URL, + HOUSTON_MAX_CATALOG_BYTES, +} from '../../server/providers/cctv/constants.js'; +import { + CAMERA_CODE_MAX_CHARS, + isLikelyHoustonCoordinate, +} from '../../server/providers/cctv/normalize.js'; +import { createCctvCatalog } from '../../server/providers/cctv/catalog.js'; +import { directionToHeading } from './directionText.js'; + +/** + * A response whose body is a live stream, plus a flag that flips when the + * stream is cancelled. A rejection path that returns without cancelling holds + * the transport open, so the flag is what the refusal tests actually assert. + */ +const streamingResponse = (init = {}) => { + const state = { cancelled: false }; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{')); + }, + cancel() { + state.cancelled = true; + }, + }); + return { response: new Response(body, init), state }; +}; + +const DOWNTOWN_ID = '1002'; +const GALLERIA_ID = '2001'; + +/** One TranStar camera row, shaped like the live `cctvSnapshots_json.js` payload. */ +const row = (overrides = {}) => ({ + location: '10 EAST @ SAN JACINTO', + id: DOWNTOWN_ID, + lat: '29.76824', + lng: '-95.355711', + dir: 'South', + fc: '6', + fp: '300', + url: "showWin('/layers/gc.aspx?cam=1002&loc=IH-10_East_at_SAN_JACINTO')", + ...overrides, +}); + +test('a Houston row maps to a source on the pinned HTTPS frame host', () => { + const source = houstonCameraToSource(row()); + assert.equal(source.id, `houston-${DOWNTOWN_ID}`); + assert.equal(source.name, '10 EAST @ SAN JACINTO'); + assert.equal(source.city, 'Houston'); + assert.equal(source.cityId, 'houston'); + assert.equal(source.provider, 'Houston TranStar'); + assert.equal(source.lat, 29.76824); + assert.equal(source.lon, -95.355711); + assert.equal(source.feedType, 'image'); + assert.equal(source.sourceKind, 'houston-transtar'); + assert.equal(source.license, 'Public Houston TranStar traffic camera'); + assert.equal(source.groundElevationM, HOUSTON_GROUND_ELEVATION_M); + assert.equal(source.url, HOUSTON_IMAGE_URL(DOWNTOWN_ID)); + assert.equal(source.url, source.snapshotUrl); + assert.ok(source.url.startsWith(HOUSTON_IMAGE_ORIGIN)); + assert.equal(source.headingConfidence, 'high'); + assert.equal(source.headingDeg, 180); +}); + +test('frame URLs are synthesized from the numeric id; the payload URL is ignored', () => { + const source = houstonCameraToSource( + row({ url: 'https://evil.example/cameras/x/image' }), + ); + assert.equal(source.url, HOUSTON_IMAGE_URL(DOWNTOWN_ID)); + assert.equal(source.snapshotUrl, source.url); +}); + +test('zero ids, off-city geometry and unusable ids are dropped', () => { + assert.equal(houstonCameraToSource(row({ id: '0' })), null); + assert.equal(houstonCameraToSource(row({ id: '00' })), null); + assert.equal(houstonCameraToSource(row({ id: '../etc/passwd' })), null); + assert.equal(houstonCameraToSource(row({ id: '12' })), null); + assert.equal(houstonCameraToSource(row({ id: '1002/../x' })), null); + // Dallas: a plausible lat/lon, but not a Houston camera. + assert.equal( + houstonCameraToSource(row({ lat: '32.7767', lng: '-96.797' })), + null, + ); + assert.equal(houstonCameraToSource(row({ lat: '0', lng: '0' })), null); + assert.equal(houstonCameraToSource(null), null); + // 3-digit and 5-digit snapshot ids stay. + assert.equal(houstonCameraToSource(row({ id: '102' })).id, 'houston-102'); + assert.equal(houstonCameraToSource(row({ id: '13711' })).id, 'houston-13711'); +}); + +test('bare cardinal dirs are high-confidence; None uses the id-hash fallback', () => { + assert.equal(directionToHeading('South', true), 180); + assert.equal(directionToHeading('None', true), NaN); + + const south = houstonCameraToSource(row({ dir: 'South' })); + assert.equal(south.headingConfidence, 'high'); + assert.equal(south.headingDeg, 180); + + const none = houstonCameraToSource(row({ dir: 'None' })); + assert.equal(none.headingConfidence, 'low'); + assert.ok(Number.isFinite(none.headingDeg)); + assert.equal( + none.headingDeg, + houstonCameraToSource(row({ dir: '' })).headingDeg, + ); +}); + +test('a nameless row still gets a label', () => { + assert.equal( + houstonCameraName({ location: ' 45 GULF @ WEST DALLAS ' }, 'houston-102'), + '45 GULF @ WEST DALLAS', + ); + assert.equal(houstonCameraName({}, 'houston-102'), 'Houston Camera 102'); + assert.equal(houstonCameraName({}, ''), 'Houston Camera'); +}); + +test('the unselected label is the interchange, trimmed to the code width', () => { + assert.equal(houstonCameraToSource(row()).code, '10 EAST @ SAN JACINTO'); + const long = houstonCameraToSource( + row({ + location: + 'IH-10 Katy Managed Lanes Eastbound at a very long interchange name that must truncate', + }), + ); + assert.equal(long.code.length, CAMERA_CODE_MAX_CHARS); + assert.ok(long.code.endsWith('…')); +}); + +test('the loader reads the keyless catalog and collapses duplicate ids', async (t) => { + t.mock.method(console, 'log', () => {}); + const requested = []; + t.mock.method(globalThis, 'fetch', async (url, init) => { + requested.push([String(url), init.headers.Accept, init.redirect]); + return Response.json({ + cameras: [ + row(), + row({ location: '10 EAST @ SAN JACINTO (duplicate)' }), + row({ + id: GALLERIA_ID, + location: '610 WEST LOOP @ SAN FELIPE', + lat: '29.749', + lng: '-95.461', + dir: 'North', + }), + row({ url: 'https://evil.example/x.jpg', id: '0' }), + ], + }); + }); + const cameras = await loadHoustonSourcesFromOpenData(); + assert.deepEqual(requested, [[HOUSTON_CAMERAS_URL, '*/*', 'manual']]); + assert.deepEqual( + cameras.map((camera) => camera.id), + [`houston-${GALLERIA_ID}`, `houston-${DOWNTOWN_ID}`], + 'nearest-to-anchor first: Uptown sits closer to its pin than downtown does', + ); + assert.equal(cameras[1].name, '10 EAST @ SAN JACINTO'); +}); + +test('an upstream failure yields an empty pack and releases the response', async (t) => { + t.mock.method(console, 'warn', () => {}); + const failed = streamingResponse({ status: 503 }); + t.mock.method(globalThis, 'fetch', async () => failed.response); + assert.deepEqual(await loadHoustonSourcesFromOpenData(), []); + assert.equal(failed.state.cancelled, true, 'the failed body is cancelled'); + + t.mock.restoreAll(); + t.mock.method(console, 'warn', () => {}); + t.mock.method(globalThis, 'fetch', async () => { + throw new Error('network down'); + }); + assert.deepEqual(await loadHoustonSourcesFromOpenData(), []); +}); + +test('the catalog fetch refuses redirects and oversized bodies', async (t) => { + t.mock.method(console, 'warn', () => {}); + const seen = []; + const redirected = streamingResponse({ + status: 302, + headers: { location: 'https://evil.example/cameras.js' }, + }); + t.mock.method(globalThis, 'fetch', async (url, init) => { + seen.push([String(url), init.redirect]); + return redirected.response; + }); + assert.deepEqual(await loadHoustonSourcesFromOpenData(), []); + assert.deepEqual(seen, [[HOUSTON_CAMERAS_URL, 'manual']]); + assert.equal( + redirected.state.cancelled, + true, + 'the redirect body is cancelled', + ); + + t.mock.restoreAll(); + t.mock.method(console, 'warn', () => {}); + t.mock.method(globalThis, 'fetch', async () => { + const body = JSON.stringify({ cameras: [row()] }); + return new Response(body, { + headers: { + 'Content-Type': 'application/javascript', + 'content-length': String(HOUSTON_MAX_CATALOG_BYTES + 1), + }, + }); + }); + assert.deepEqual(await loadHoustonSourcesFromOpenData(), []); + + t.mock.restoreAll(); + t.mock.method(console, 'warn', () => {}); + t.mock.method(globalThis, 'fetch', async () => { + const oversized = 'x'.repeat(HOUSTON_MAX_CATALOG_BYTES + 1024); + return new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(oversized)); + controller.close(); + }, + }), + { headers: { 'Content-Type': 'application/javascript' } }, + ); + }); + assert.deepEqual(await loadHoustonSourcesFromOpenData(), []); +}); + +test('a reduced pack cap keeps cameras nearest the metro anchors', async (t) => { + t.mock.method(console, 'log', () => {}); + const saved = process.env.CCTV_HOUSTON_MAX_SOURCES; + t.after(() => { + if (saved === undefined) delete process.env.CCTV_HOUSTON_MAX_SOURCES; + else process.env.CCTV_HOUSTON_MAX_SOURCES = saved; + }); + process.env.CCTV_HOUSTON_MAX_SOURCES = '8'; + const rows = [ + ...[29.32, 29.322, 29.324, 29.326, 29.328, 29.33].map((lat, i) => + row({ + id: String(3000 + i), + lat: String(lat), + lng: '-94.80', + location: `Galveston ${i}`, + dir: 'East', + }), + ), + ...[29.76, 29.759, 29.758, 29.757, 29.756, 29.755].map((lat, i) => + row({ + id: String(4000 + i), + lat: String(lat), + lng: '-95.3698', + location: `Downtown ${i}`, + dir: 'North', + }), + ), + ]; + t.mock.method(globalThis, 'fetch', async () => + Response.json({ cameras: rows }), + ); + const cameras = await loadHoustonSourcesFromOpenData(); + assert.equal(cameras.length, 8); + assert.deepEqual( + cameras.map((camera) => camera.name.slice(0, 8)), + [ + 'Downtown', + 'Downtown', + 'Downtown', + 'Downtown', + 'Downtown', + 'Downtown', + 'Galvesto', + 'Galvesto', + ], + 'downtown fills first; the Galveston anchor still keeps a share', + ); +}); + +/** + * Serve the Houston catalog to the Houston endpoint and an empty payload to + * every other pack, so one catalog refresh exercises the registration without + * reaching the network. + */ +const runCatalogWithMockedUpstreams = async (t) => { + const requested = []; + t.mock.method(console, 'log', () => {}); + t.mock.method(console, 'warn', () => {}); + t.mock.method(globalThis, 'fetch', async (url) => { + const href = String(url); + requested.push(href); + if (href === HOUSTON_CAMERAS_URL) { + return Response.json({ + cameras: [ + row(), + row({ + id: GALLERIA_ID, + location: '610 WEST LOOP @ SAN FELIPE', + lat: '29.749', + lng: '-95.461', + dir: 'West', + }), + ], + }); + } + return Response.json([]); + }); + const sources = await createCctvCatalog({ sourceRoot: '/nonexistent' })(); + return { requested, sources }; +}; + +test('the Houston lane is wired into the catalog and its loader runs', async (t) => { + const saved = { ...process.env }; + try { + delete process.env.CCTV_SOURCES_FILE; + delete process.env.CCTV_SOURCES_JSON; + delete process.env.CCTV_HOUSTON_ENABLED; + const { requested, sources } = await runCatalogWithMockedUpstreams(t); + assert.ok( + requested.includes(HOUSTON_CAMERAS_URL), + 'the catalog refresh invokes the Houston loader', + ); + assert.deepEqual( + sources.filter((s) => s.cityId === 'houston').map((s) => s.id), + [`houston-${GALLERIA_ID}`, `houston-${DOWNTOWN_ID}`], + 'Houston cameras reach the served catalog through the registered lane', + ); + } finally { + for (const key of Object.keys(process.env)) { + if (!(key in saved)) delete process.env[key]; + } + Object.assign(process.env, saved); + } +}); + +test('CCTV_HOUSTON_ENABLED=0 keeps the lane from being loaded at all', async (t) => { + const saved = { ...process.env }; + try { + delete process.env.CCTV_SOURCES_FILE; + delete process.env.CCTV_SOURCES_JSON; + process.env.CCTV_HOUSTON_ENABLED = '0'; + const { requested, sources } = await runCatalogWithMockedUpstreams(t); + assert.equal( + requested.includes(HOUSTON_CAMERAS_URL), + false, + 'the disabled lane never reaches its upstream', + ); + assert.deepEqual( + sources.filter((s) => s.cityId === 'houston'), + [], + ); + } finally { + for (const key of Object.keys(process.env)) { + if (!(key in saved)) delete process.env[key]; + } + Object.assign(process.env, saved); + } +}); + +test('the shipped catalog ceiling is not raised to make room for this pack', () => { + assert.equal(DEFAULT_CCTV_MAX_SOURCES, 4000); + assert.equal(DEFAULT_HOUSTON_MAX_SOURCES, 250); +}); + +test('isLikelyHoustonCoordinate spans the metro and rejects the rest', () => { + assert.equal(isLikelyHoustonCoordinate(29.7604, -95.3698), true); // Downtown + assert.equal(isLikelyHoustonCoordinate(29.3013, -94.7977), true); // Galveston + assert.equal(isLikelyHoustonCoordinate(29.9902, -95.3368), true); // IAH + assert.equal(isLikelyHoustonCoordinate(32.7767, -96.797), false); // Dallas + assert.equal(isLikelyHoustonCoordinate(30.2672, -97.7431), false); // Austin + assert.equal(isLikelyHoustonCoordinate(0, 0), false); + assert.equal(isLikelyHoustonCoordinate(NaN, -95.37), false); +}); diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index c13c7dd94..b74e8c061 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -300,6 +300,12 @@ export const DATA_CREDITS = [ 'Traffic cameras (Toronto): contains information licensed under the ' + 'Open Government Licence – Toronto', }, + { + key: 'houston-cctv', + html: + 'CCTV cameras & frames (Houston): ' + + 'Houston TranStar', + }, { key: 'gbfs', html: From 01a6734a5620a8ad4668f7a1ce0d65040f2f8bf7 Mon Sep 17 00:00:00 2001 From: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Date: Fri, 16 Oct 2026 18:22:41 -0700 Subject: [PATCH 04/83] fix(cctv): make the Houston TranStar pack opt-in TranStar's disclaimer prohibits copying, distribution and external framing of specific images without TxDOT's prior written permission, so the pack no longer loads by default. It now uses envOptIn and loads only when CCTV_HOUSTON_ENABLED=1, the same pattern as NYC DOT and Via Lietuva. - DATA_SOURCES.md: replace the licence column with the disclaimer's terms and a link, and document the opt-in - Ambient stills for Houston refresh every 3 minutes, matching TranStar's stated snapshot cadence - README, CHANGELOG, CURRENT-STATE and .env.example describe the pack as opt-in; the README camera count no longer includes Houston - Tests cover the default-off and =0 cases and the opted-in catalog path --- .env.example | 2 +- CHANGELOG.md | 11 --------- DATA_SOURCES.md | 4 +-- README.md | 2 +- changelog.d/1125-houston-cctv-pack.md | 13 ++++++++++ docs/CURRENT-STATE.md | 14 ++++++----- server/providers/cctv/catalog.js | 2 +- server/providers/cctv/sources.js | 3 ++- src/data/cctvHouston.test.mjs | 35 ++++++++++++++++----------- src/data/cctvLod.js | 2 ++ src/data/cctvLod.test.mjs | 1 + 11 files changed, 52 insertions(+), 37 deletions(-) create mode 100644 changelog.d/1125-houston-cctv-pack.md diff --git a/.env.example b/.env.example index 265377bf7..d176acf40 100644 --- a/.env.example +++ b/.env.example @@ -237,7 +237,7 @@ AISSTREAM_API_KEY= # CCTV_TORONTO_LIST_URL= # override the Toronto RESCU camera list URL # CCTV_LITHUANIA_ENABLED= # off by default; set 1 to load Via Lietuva / Eismoinfo road cameras (photo licence unconfirmed; see DATA_SOURCES.md) # CCTV_LITHUANIA_MAX_SOURCES=400 # cap Lithuanian cameras; no API key required -# CCTV_HOUSTON_ENABLED=1 # set 0 to disable the Houston TranStar cameras +# CCTV_HOUSTON_ENABLED= # set to 1 to opt in; off by default (TranStar terms) # CCTV_HOUSTON_MAX_SOURCES=250 # cap cameras loaded from the Houston TranStar catalog (nearest metro cores first) # CCTV_HOUSTON_CAMERAS_URL= # override the Houston TranStar catalog URL # CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs diff --git a/CHANGELOG.md b/CHANGELOG.md index b4de32107..98d32365f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,16 +1,5 @@ # Changelog -- Add Houston traffic cameras from Houston TranStar as a CCTV pack. The camera - list is one keyless JSON request to - `traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js`; frames are - stills synthesized from each numeric camera id on `www.houstontranstar.org`, - so a catalog field cannot steer the proxy off-host. Cameras without a - snapshot id, and cameras outside Greater Houston, are dropped. By default - the 250 cameras nearest downtown, Uptown, the Medical Center, IAH and - Galveston load. `CCTV_HOUSTON_ENABLED=0` turns the pack off and - `CCTV_HOUSTON_MAX_SOURCES` changes the cap. The catalog-wide ceiling is - unchanged; extra cameras share it round-robin with the other packs. - - The tracked-aircraft descriptor now publishes `verticalRateMps`, on both the civil and military layers, so a consumer reading `getTrackedInfo()` no longer has to reach past it into the raw poll record for climb/descent. Reported, diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 913cad6d1..92f623deb 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -65,7 +65,7 @@ How to read this: | **Statens vegvesen** (catalog `ogckart-sn1.atlas.vegvesen.no`, frames `kamera.atlas.vegvesen.no`, HLS `kamera.vegvesen.no`) | CCTV camera catalog + frames + live HLS video, Norway road cameras | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) distributed by Statens vegvesen" | | **NYC DOT Traffic Management Center** (`webcams.nyctmc.org`) | CCTV camera catalog + frames, New York City five boroughs | NYC DOT requires developers to sign a [data-sharing agreement](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) for the camera feed. **Off by default**; enable only under your own agreement | "NYC DOT Traffic Management Center" + NYC DOT's required application disclaimer (in the Data attribution popover) | | **City of Toronto Open Data** (`opendata.toronto.ca`) | CCTV camera catalog + frames, Toronto city streets | [Open Government Licence – Toronto](https://www.toronto.ca/city-government/data-research-maps/open-data/open-data-licence/) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – Toronto" | -| **Houston TranStar** (`traffic.houstontranstar.org`, frames `www.houstontranstar.org`) | CCTV camera catalog + frames, Greater Houston | Public Houston TranStar traffic camera stills | "Houston TranStar" (courtesy) | +| **Houston TranStar** (`traffic.houstontranstar.org`, frames `www.houstontranstar.org`) | CCTV camera catalog + frames, Greater Houston | No licence. The [TranStar disclaimer](https://traffic.houstontranstar.org/disclaimer.aspx) (Content Usage) prohibits copying or distribution, and external framing or linking to specific images, without TxDOT's prior written permission. Off unless `CCTV_HOUSTON_ENABLED=1` | "Houston TranStar" (courtesy) | | **GBFS (Lyft / BCycle / Social Bicycles / PBSC)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` | | **Bike Share Toronto** (`toronto.publicbikesystem.net`) | Bikeshare station availability, Toronto | Operator GBFS feed. No licence is published: the City's open-data package declares none and the feed has no `license_url`. Attribution is given to the operator. | "Bike Share Toronto (Toronto Parking Authority)" | | **Bike Itaú / Tembici** (`*.publicbikesystem.net`, seven Brazilian cities, Santiago and Bogotá) | Bikeshare station availability: São Paulo, Rio de Janeiro, Salvador, Recife, Porto Alegre, Brasília, Curitiba, Santiago de Chile and Bogotá | Operator GBFS feeds. No licence is published: the feeds' `system_information.json` carries no `license_url`, operator or attribution fields. Attribution is given to the operator. | "Bike Itaú (Tembici)"; "Tembici Bogotá" | @@ -167,7 +167,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei - **Statens vegvesen road cameras.** Keyless. The camera list is one GeoJSON request to the OGC API Features view of the official DATEX 3.1 CCTV site table (`ogckart-sn1.atlas.vegvesen.no/ogc/features/v1/collections/datex_3_1:CctvSimple/items`), so no DATEX account is needed; frames are stills on `kamera.atlas.vegvesen.no/api/images/`, and only that exact path per camera is registered. Cameras the feed reports as faulty are dropped. About 135 working cameras also publish live HLS (`kamera.vegvesen.no/public//manifest.m3u8`, MPEG-TS); those play as live video through the bounded in-memory HLS relay, with the still as the snapshot fallback, and only that exact manifest path per camera is registered. `CCTV_VEGVESEN_VIDEO=0` keeps every camera on stills. `orientationDescription` names the place a camera faces, not a bearing, so it goes into the label and headings use the id-hash fallback at low confidence. By default every live camera loads (~850); `CCTV_VEGVESEN_MAX_SOURCES` (8–1000) lowers the cap, keeping the cameras nearest Oslo, Bergen, Stavanger, Trondheim, Kristiansand, Bodø and Tromsø and `CCTV_VEGVESEN_ENABLED=0` turns the pack off. Frames are fetched at request time and never stored or redistributed. The NLOD attribution is registered in the Data attribution popover. - **NYC DOT Traffic Management Center cameras.** Off by default. NYC DOT's [developer data feeds page](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) asks developers who want the camera feed to contact `tmcdot@dot.nyc.gov` and sign a data-sharing agreement. This project does not hold one that covers downstream installations, so the pack loads only when an operator who holds their own agreement sets `CCTV_NYC_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to NYC DOT. The endpoint needs no key. The camera list is the public `webcams.nyctmc.org/api/cameras` catalog (~980 cameras across the five boroughs); frames are stills at `webcams.nyctmc.org/api/cameras//image`, synthesized from a validated UUID rather than taken from the payload, and only that host is registered. Cameras the feed reports as offline are dropped. The catalog publishes no camera facing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. Once enabled, the 250 cameras nearest the borough cores load; `CCTV_NYC_MAX_SOURCES` (8–400) changes the cap. Frames are fetched at request time and never stored or redistributed. NYC DOT asks every application that uses its data to carry its disclaimer; it is registered verbatim in the Data attribution popover. - **City of Toronto RESCU cameras.** Keyless. The camera list is the City's public `tmcearthcameras.json` on `opendata.toronto.ca` (dataset "Traffic Cameras", Transportation Services), served as JSONP and unwrapped with an anchored pattern. This is the municipal network the Ontario 511 pack does not carry: downtown arterials plus the Gardiner, DVP, Allen and Lake Shore. The list publishes no frame URL, so frame URLs are BUILT from the documented `CameraImages/loc####.jpg` convention and a strictly-digits camera number, pinning the frame proxy to the portal origin by construction; the list fetch refuses redirects and bounds the body it buffers. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `D1`-`D4` fields are the directions of the *comparison* images (static reference stills captured in 2016 under a different path), not a bearing for the live pan-tilt-zoom frame, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. A camera that is temporarily out of service still answers 200 with a byte-identical "Image Unavailable" placeholder rather than an error (about 6% of the network at any moment), so such a camera reads as healthy; the frame it shows is the agency's, not a synthesized one. The licence requires the attribution statement above. Disable with `CCTV_TORONTO_ENABLED=0`. -- **Houston TranStar cameras.** Keyless. The camera list is the public `traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js` catalog (~1,000 cameras across Greater Houston); frames are stills at `www.houstontranstar.org/snapshots/cctv/.jpg`, synthesized from a validated numeric id rather than taken from the payload, and only that host is registered. Cameras whose catalog id is `0` (a different, non-snapshot class) and cameras outside Greater Houston are dropped. Dedicated `dir` values (North/South/East/West) become high-confidence headings; `None` uses the shared id-hash fallback at low confidence. By default the 250 cameras nearest downtown, Uptown, the Medical Center, IAH and Galveston load; `CCTV_HOUSTON_MAX_SOURCES` (8–400) changes the cap and `CCTV_HOUSTON_ENABLED=0` turns the pack off. Frames are fetched at request time and never stored or redistributed. +- **Houston TranStar cameras.** Keyless. The camera list is the public `traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js` catalog (~1,000 cameras across Greater Houston); frames are stills at `www.houstontranstar.org/snapshots/cctv/.jpg`, synthesized from a validated numeric id rather than taken from the payload, and only that host is registered. Cameras whose catalog id is `0` (a different, non-snapshot class) and cameras outside Greater Houston are dropped. Dedicated `dir` values (North/South/East/West) become high-confidence headings; `None` uses the shared id-hash fallback at low confidence. TranStar's disclaimer is not a licence to ship the stills on by default (its FAQ grants permission-free use only for the RSS feeds), so the pack stays off unless `CCTV_HOUSTON_ENABLED=1`. Opting in loads the 250 cameras nearest downtown, Uptown, the Medical Center, IAH and Galveston; `CCTV_HOUSTON_MAX_SOURCES` (8–400) changes the cap. Snapshots update about every 3 minutes, which is also the ambient still cadence. Frames are fetched at request time and never stored or redistributed. - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals. - **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All sixteen registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. The TTC feed is BusTime, the surface system: it carries buses and streetcars and no subway at all. About four in ten of its vehicles report no route at all (unassigned or deadheading), and those are shown with the feed default rather than a route-established mode. It sends no ETag or Last-Modified, so its conditional requests fall through to the ordinary snapshot cache. Three Golden Horseshoe neighbours run alongside it — MiWay, Hamilton HSR and Durham Region Transit — each bus-only and each polled only near its own region. Durham Region Transit publishes no bearing on any vehicle, so those sprites take their heading from observed movement like any other feed that omits it. Guelph, Kingston and Barrie extend the same pattern eastward and north. Hamilton, Guelph, Burlington and YRT publish route ids that are not the number on the vehicle, so those entries carry a route-name table taken from each operator’s own published GTFS; any id missing from a table renders unchanged rather than blank. A label is never invented: YRT runs a route `9101` that appears nowhere in its published routes, so it stays as the feed sends it. Burlington’s is the one that cannot be shortcut: `351` is route 1 while `3510` is route 10, so the table is a real join rather than a trimmed prefix. GO Transit / Metrolinx is deliberately absent: its REALTIME feed requires an API key issued by manual review, which the keyless admission rule excludes (its static schedule download is keyless, but carries no vehicle positions). Barrie and YRT/Viva each invite acceptance of their licence through a form, and each licence also states that using the data is itself acceptance; this deployment relies on that clause, which is recorded in both entries. Grand River Transit in Waterloo is absent for a different reason: it answers, but its feed header was months stale and it publishes no per-vehicle timestamps, so it would render vehicles that look live and are not. Ottawa’s OC Transpo falls to the same keyless admission rule as GO Transit — its realtime feed requires an API key. London Transit and Thunder Bay Transit publish vehicle positions over plain HTTP with no HTTPS endpoint available. Every other Ontario system — Brampton, Oakville, Niagara, Milton, Windsor, Greater Sudbury, UP Express and a dozen smaller operators — publishes static GTFS schedules only, with no realtime vehicle positions to register. Between the nine included and these exclusions, every Ontario realtime vehicle-position feed catalogued in the Mobility Database is accounted for. - **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply. diff --git a/README.md b/README.md index 867f14824..6c2baec79 100644 --- a/README.md +++ b/README.md @@ -377,7 +377,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi | 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 | | ◍ **Air quality** | Live AQHI on the published 1–10+ health-risk scale, with a Weather-panel card showing the band legend and the highest readings in view. **Canada only** — ECCC is the first registered provider | ECCC MSC GeoMet | 🟢 | | 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds | -| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit) · Houston (TranStar), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement) and opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | +| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement), opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed) and opt-in Houston (TranStar, off unless CCTV_HOUSTON_ENABLED=1). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | | 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 | | 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer and **FOLLOW**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key | | 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 | diff --git a/changelog.d/1125-houston-cctv-pack.md b/changelog.d/1125-houston-cctv-pack.md new file mode 100644 index 000000000..dc573993f --- /dev/null +++ b/changelog.d/1125-houston-cctv-pack.md @@ -0,0 +1,13 @@ +- Add Houston traffic cameras from Houston TranStar as a CCTV pack. The camera + list is one keyless JSON request to + `traffic.houstontranstar.org/data/layers/cctvSnapshots_json.js`; frames are + stills synthesized from each numeric camera id on `www.houstontranstar.org`, + so a catalog field cannot steer the proxy off-host. Cameras without a + snapshot id, and cameras outside Greater Houston, are dropped. TranStar's + disclaimer prohibits redistribution and external framing without TxDOT's + written permission, so the pack is off unless `CCTV_HOUSTON_ENABLED=1`. + Opting in loads the 250 cameras nearest downtown, Uptown, the Medical + Center, IAH and Galveston; `CCTV_HOUSTON_MAX_SOURCES` changes that cap. + Snapshots update about every 3 minutes. The catalog-wide ceiling is + unchanged; extra cameras share it round-robin with the other packs + (mvanhorn, #1125). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 99cfc3234..0c8fe6261 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1083,11 +1083,13 @@ URLs are built from each camera id under from the payload. Dedicated `dir` values become high-confidence headings; `None` uses the shared id-hash fallback at low confidence. -By default the 250 cameras nearest downtown, Uptown, the Medical Center, IAH -and Galveston load. `CCTV_HOUSTON_MAX_SOURCES` sets the pack cap (8–400) and -`CCTV_HOUSTON_ENABLED=0` turns the pack off. The catalog-wide ceiling stays at -4,000 and is shared round-robin, so adding this pack does not silently drop -another region. +TranStar's disclaimer prohibits redistribution and external framing of specific +images without TxDOT's written permission, so the pack is off unless +`CCTV_HOUSTON_ENABLED=1`. Opting in loads the 250 cameras nearest downtown, +Uptown, the Medical Center, IAH and Galveston. `CCTV_HOUSTON_MAX_SOURCES` sets +the pack cap (8–400). Snapshots update about every 3 minutes. The catalog-wide +ceiling stays at 4,000 and is shared round-robin, so adding this pack does not +silently drop another region. ## Location control ownership @@ -3383,7 +3385,7 @@ its criteria cannot be silently ignored. | Satellites | CelesTrak | `src/data/satellites.js` | `/api/celestrak` | 120s | | Space Missions (30d) | Launch Library 2 + CelesTrak | `src/data/rocketLaunches.js` | `/api/launches` + `/api/celestrak/active` | 5 min | | Traffic | Selectable TomTom / OpenStreetMap / Hybrid roads; optional TomTom flow (BYOK) | `src/data/traffic.js` | browser-direct OpenFreeMap tiles; `/api/tomtom` for flow | viewport-driven; capped tile cache | -| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Houston TranStar + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) | +| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Houston TranStar (opt-in) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) | | Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh | | Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) | | Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto, Bike Itaú (Brazil) and Bicing (Barcelona) + Urban Sharing, Entur, JCDecaux NO + nextbike (AMBici)) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | diff --git a/server/providers/cctv/catalog.js b/server/providers/cctv/catalog.js index 0ed194660..2268b45ee 100644 --- a/server/providers/cctv/catalog.js +++ b/server/providers/cctv/catalog.js @@ -147,7 +147,7 @@ const LIVE_PACKS = [ }, { name: 'houston', - enabled: () => envEnabled('CCTV_HOUSTON_ENABLED'), + enabled: () => envOptIn('CCTV_HOUSTON_ENABLED'), load: loadHoustonSourcesFromOpenData, }, ]; diff --git a/server/providers/cctv/sources.js b/server/providers/cctv/sources.js index 895fe4c4d..f51d6bf35 100644 --- a/server/providers/cctv/sources.js +++ b/server/providers/cctv/sources.js @@ -2854,7 +2854,8 @@ export function houstonCameraToSource(record) { url: imageUrl, snapshotUrl: imageUrl, sourceKind: 'houston-transtar', - license: 'Public Houston TranStar traffic camera', + license: + 'TranStar disclaimer — redistribution and external framing need TxDOT written permission', code: cameraDisplayCode(name.toUpperCase()), }; } diff --git a/src/data/cctvHouston.test.mjs b/src/data/cctvHouston.test.mjs index 97de95a84..54aace016 100644 --- a/src/data/cctvHouston.test.mjs +++ b/src/data/cctvHouston.test.mjs @@ -67,7 +67,10 @@ test('a Houston row maps to a source on the pinned HTTPS frame host', () => { assert.equal(source.lon, -95.355711); assert.equal(source.feedType, 'image'); assert.equal(source.sourceKind, 'houston-transtar'); - assert.equal(source.license, 'Public Houston TranStar traffic camera'); + assert.equal( + source.license, + 'TranStar disclaimer — redistribution and external framing need TxDOT written permission', + ); assert.equal(source.groundElevationM, HOUSTON_GROUND_ELEVATION_M); assert.equal(source.url, HOUSTON_IMAGE_URL(DOWNTOWN_ID)); assert.equal(source.url, source.snapshotUrl); @@ -290,6 +293,7 @@ test('a reduced pack cap keeps cameras nearest the metro anchors', async (t) => */ const runCatalogWithMockedUpstreams = async (t) => { const requested = []; + t.mock.restoreAll(); t.mock.method(console, 'log', () => {}); t.mock.method(console, 'warn', () => {}); t.mock.method(globalThis, 'fetch', async (url) => { @@ -320,7 +324,7 @@ test('the Houston lane is wired into the catalog and its loader runs', async (t) try { delete process.env.CCTV_SOURCES_FILE; delete process.env.CCTV_SOURCES_JSON; - delete process.env.CCTV_HOUSTON_ENABLED; + process.env.CCTV_HOUSTON_ENABLED = '1'; const { requested, sources } = await runCatalogWithMockedUpstreams(t); assert.ok( requested.includes(HOUSTON_CAMERAS_URL), @@ -339,22 +343,25 @@ test('the Houston lane is wired into the catalog and its loader runs', async (t) } }); -test('CCTV_HOUSTON_ENABLED=0 keeps the lane from being loaded at all', async (t) => { +test('Houston stays unloaded unless CCTV_HOUSTON_ENABLED=1', async (t) => { const saved = { ...process.env }; try { delete process.env.CCTV_SOURCES_FILE; delete process.env.CCTV_SOURCES_JSON; - process.env.CCTV_HOUSTON_ENABLED = '0'; - const { requested, sources } = await runCatalogWithMockedUpstreams(t); - assert.equal( - requested.includes(HOUSTON_CAMERAS_URL), - false, - 'the disabled lane never reaches its upstream', - ); - assert.deepEqual( - sources.filter((s) => s.cityId === 'houston'), - [], - ); + for (const value of [undefined, '0']) { + if (value === undefined) delete process.env.CCTV_HOUSTON_ENABLED; + else process.env.CCTV_HOUSTON_ENABLED = value; + const { requested, sources } = await runCatalogWithMockedUpstreams(t); + assert.equal( + requested.includes(HOUSTON_CAMERAS_URL), + false, + 'the pack reaches TranStar only after an explicit opt-in', + ); + assert.deepEqual( + sources.filter((s) => s.cityId === 'houston'), + [], + ); + } } finally { for (const key of Object.keys(process.env)) { if (!(key in saved)) delete process.env[key]; diff --git a/src/data/cctvLod.js b/src/data/cctvLod.js index 44b2e8ba3..d91e3b4b3 100644 --- a/src/data/cctvLod.js +++ b/src/data/cctvLod.js @@ -34,6 +34,8 @@ const PROVIDER_STATIC_REFRESH_MS = Object.freeze({ 'ontario 511': 3 * 60 * 1000, // TxDOT publishes roughly once a minute; 3 min matches the other highway packs. txdot: 3 * 60 * 1000, + // TranStar says snapshots update about every 3 minutes. + 'houston transtar': 3 * 60 * 1000, // Digitraffic publishes a new weathercam frame on each station's // collectionInterval, 600 s for every station sampled; polling faster only // re-fetches the same JPEG. diff --git a/src/data/cctvLod.test.mjs b/src/data/cctvLod.test.mjs index 4f5982496..55821ae43 100644 --- a/src/data/cctvLod.test.mjs +++ b/src/data/cctvLod.test.mjs @@ -639,6 +639,7 @@ test('staticFrameRefreshMs follows known pack cadences and bounds explicit value assert.equal(staticFrameRefreshMs({ provider: 'Caltrans' }), 180_000); assert.equal(staticFrameRefreshMs({ provider: 'Ontario 511' }), 180_000); assert.equal(staticFrameRefreshMs({ provider: 'TxDOT' }), 180_000); + assert.equal(staticFrameRefreshMs({ provider: 'Houston TranStar' }), 180_000); // Digitraffic publishes one weathercam frame per 600 s collection interval. assert.equal(staticFrameRefreshMs({ provider: 'Fintraffic' }), 600_000); assert.equal( From 25ec9e10a35ea3b850d078b9d3de36735078899d Mon Sep 17 00:00:00 2001 From: bassem chagra Date: Thu, 22 Oct 2026 16:53:24 +0200 Subject: [PATCH 05/83] feat(cctv): vendor the Bosch MIC PTZ datasheet records the Amsterdam register names Adds MIC-7522-Z30B, MIC-7502-Z30B and MIC-7130-PB4 (MIC IP starlight 7000 HD) from the CCTV Camera Database v2.29.0 model-enrichment extract. Colour siblings (B/G/W) share one datasheet, so they and the register's x wildcard (MIC-7522-Z30x) are aliases of one record. The MIC datasheets state FOV without an axis, so these records add specs but no horizontal FOV range. Refreshes the dataset size in the module header and DATA_SOURCES.md. --- DATA_SOURCES.md | 7 +-- src/layers/cctv/modelSpecs.js | 67 ++++++++++++++++++++++++++--- src/layers/cctv/modelSpecs.test.mjs | 28 ++++++++++++ 3 files changed, 94 insertions(+), 8 deletions(-) diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 6d2ee498b..a9d676c85 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -208,7 +208,7 @@ The [Bhote Koshi event pack](public/events/bhote-koshi-2026/README.md), under `p | **Yale Bright Star Catalogue** (9,096 stars, sky background) | `sky/` | **Public domain** | ✅ (no restrictions) | "Yale Bright Star Catalogue (Hoffleit & Warren, via CDS)" (courtesy credit) | | **ESO Milky Way panorama** (eso0932a, blurred cube map, sky background) | `sky/` | **CC BY 4.0** | ✅ (attribution; changes stated) | "ESO/S. Brunier" | | **CCTV ground heights** (3,445 cameras) | `cctv_ground_heights/` | Precomputed camera placement heights, aligned to work with Google Photorealistic 3D Tiles (folder README) | — | — | -| **CCTV Camera Database** model-enrichment extract v2.26.0 (17 camera-model records) | `src/layers/cctv/modelSpecs.js` | **CC0-1.0** (public domain dedication) | ✅ (no restrictions) | "Specs: cctv-database.com" (courtesy — not legally required) | +| **CCTV Camera Database** model-enrichment extract v2.26.0 + v2.29.0 (20 camera-model records) | `src/layers/cctv/modelSpecs.js` | **CC0-1.0** (public domain dedication) | ✅ (no restrictions) | "Specs: cctv-database.com" (courtesy — not legally required) | ### ⚠️ TeleGeography is bundled but NonCommercial @@ -236,11 +236,12 @@ fixtures retain their provenance in `src/data/fixtures/README.md`. ### CCTV Camera Database (`src/layers/cctv/modelSpecs.js`) -The CCTV layer's hardware-model specs are a small verbatim extract (v2.26.0) of the +The CCTV layer's hardware-model specs are a small verbatim extract (v2.26.0, plus the Bosch MIC +records from v2.29.0) of the [CCTV Camera Database](https://github.com/ch-bas/cctv-camera-database), an open dataset of camera models transcribed from manufacturer datasheets, dedicated to the public domain under **CC0-1.0**. Only the records for hardware that the integrated public catalogs name (King -County, Sarasota County and Sioux Falls) are bundled, compiled into the module. Nothing is +County, Sarasota County, Sioux Falls and the City of Amsterdam register) are bundled, compiled into the module. Nothing is fetched at runtime. The records describe camera products (optics, resolution, night vision) only, never endpoints, stream paths or access data. CC0 requires no attribution; the CCTV HUD credits it anyway as `SPECS: CCTV-DATABASE.COM` beside any spec line it renders. diff --git a/src/layers/cctv/modelSpecs.js b/src/layers/cctv/modelSpecs.js index 2d93166a7..89fcf58d2 100644 --- a/src/layers/cctv/modelSpecs.js +++ b/src/layers/cctv/modelSpecs.js @@ -14,18 +14,25 @@ * * Spec records are vendored verbatim from the CCTV Camera Database * (https://github.com/ch-bas/cctv-camera-database, CC0-1.0) — an open dataset - * of 28,000+ camera models transcribed from manufacturer datasheets (84% with - * FOV, as of v2.26.0). This module vendors the records that the currently integrated public + * of 29,000+ camera models transcribed from manufacturer datasheets (76% with + * FOV, as of v2.29.0). This module vendors the records that the currently integrated public * catalogs publish hardware for — King County WA (Cohu, Bosch AUTODOME, Axis * Q6135-LE), plus the Bosch AUTODOME and Axis PTZ/multisensor models that - * appear in Sarasota County FL and Sioux Falls SD open data; packs that - * publish other hardware models can grow it, or vendor the full extract. + * appear in Sarasota County FL and Sioux Falls SD open data, and the Bosch MIC + * PTZs in the City of Amsterdam camera register; packs that publish other + * hardware models can grow it, or vendor the full extract. * * Scope guardrail: records describe camera PRODUCTS (optics, resolution, * night vision) — never endpoints, stream paths, or access/config data. */ -/** Verbatim records from the dataset's model-enrichment extract (v2.26.0). */ +/** + * Verbatim records from the dataset's model-enrichment extract (v2.26.0; the + * Bosch MIC records from v2.29.0). Bosch MIC order codes end in a colour + * letter (B black, G grey, W white) on one shared datasheet; those siblings, + * and the `x` wildcard registries write for the colour letter, are listed as + * aliases of the one vendored record. + */ const RECORDS = [ { id: 'axis-q6135-le', @@ -264,6 +271,56 @@ const RECORDS = [ autotracking: true, environment: ['outdoor'], }, + { + id: 'bosch-mic-7130-pb4', + brand: 'Bosch', + model: 'MIC-7130-PB4', + type: 'ptz', + url: 'https://www.cctv-database.com/camera/bosch-mic-7130-pb4/', + aliases: [ + 'MIC IP starlight 7000 HD', + 'MIC-7130-PG4', + 'MIC-7130-PW4', + 'MIC-7130-Px4', + ], + fov_deg: '2.1-59', + resolution: { mp: 1, w: 1280, h: 720, label: '720p' }, + ptz: true, + autotracking: true, + varifocal: true, + environment: ['outdoor'], + }, + { + id: 'bosch-mic-7502-z30b', + brand: 'Bosch', + model: 'MIC-7502-Z30B', + type: 'ptz', + url: 'https://www.cctv-database.com/camera/bosch-mic-7502-z30b/', + aliases: ['MIC-7502-Z30G', 'MIC-7502-Z30W', 'MIC-7502-Z30x'], + fov_deg: '2.3-63.7', + resolution: { mp: 2.13, w: 1920, h: 1080, label: '1080p' }, + ptz: true, + autotracking: true, + varifocal: true, + night_vision: { type: 'color' }, + environment: ['outdoor'], + }, + { + id: 'bosch-mic-7522-z30b', + brand: 'Bosch', + model: 'MIC-7522-Z30B', + type: 'ptz', + url: 'https://www.cctv-database.com/camera/bosch-mic-7522-z30b/', + aliases: ['MIC-7522-Z30G', 'MIC-7522-Z30W', 'MIC-7522-Z30x'], + fov_deg: '2.1-58.3', + // i18n-ignore: datasheet label + resolution: { mp: 2, w: 1920, h: 1080, label: '1080p HD' }, + ptz: true, + varifocal: true, + night_vision: { type: 'color' }, + environment: ['outdoor'], + ndaa_compliant: true, + }, { id: 'cohu-3920-idome', brand: 'Cohu', diff --git a/src/layers/cctv/modelSpecs.test.mjs b/src/layers/cctv/modelSpecs.test.mjs index 9fbdb00dd..852c6ad64 100644 --- a/src/layers/cctv/modelSpecs.test.mjs +++ b/src/layers/cctv/modelSpecs.test.mjs @@ -245,6 +245,34 @@ test('King County snapshot model strings still resolve', () => { ); }); +test('Amsterdam camera register model strings resolve', () => { + // The register writes the Bosch MIC colour letter as `x`. + assert.equal( + lookupModelSpec('Bosch MIC-7522-Z30x')?.id, + 'bosch-mic-7522-z30b', + ); + assert.equal( + lookupModelSpec('Bosch MIC-7502-Z30x')?.id, + 'bosch-mic-7502-z30b', + ); + assert.equal(lookupModelSpec('Bosch MIC-7130-Px4')?.id, 'bosch-mic-7130-pb4'); + assert.equal(lookupModelSpec('MIC-7522-Z30W')?.id, 'bosch-mic-7522-z30b'); + // Its ANPR units are not in the table: unknown, never a guess. + assert.equal(lookupModelSpec('CSG-8800'), null); + // The MIC and AUTODOME starlight 7000 HD (code-less names, so the brand + // is needed) stay apart. + assert.equal( + lookupModelSpec('MIC IP starlight 7000 HD', 'Bosch')?.id, + 'bosch-mic-7130-pb4', + ); + assert.equal( + lookupModelSpec('AUTODOME IP starlight 7000 HD', 'Bosch')?.id, + 'bosch-autodome-ip-starlight-7000-hd', + ); + // The MIC datasheets state FOV without an axis, so no horizontal range. + assert.equal(horizontalFovRangeDeg(lookupModelSpec('MIC-7522-Z30B')), null); +}); + test('catalog never overwrites an explicit FOV or a curated pose', () => { const [explicit, curated, cohu, pelco] = browserCatalog().buildCatalogFromSources([ From 6a7597d344960f7495fb3bdee2b34eed0a917c17 Mon Sep 17 00:00:00 2001 From: bassem chagra Date: Thu, 22 Oct 2026 17:01:50 +0200 Subject: [PATCH 06/83] docs(changelog): fragment for #1319 --- changelog.d/1319-cctv-bosch-mic-specs.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 changelog.d/1319-cctv-bosch-mic-specs.md diff --git a/changelog.d/1319-cctv-bosch-mic-specs.md b/changelog.d/1319-cctv-bosch-mic-specs.md new file mode 100644 index 000000000..bf1bd0b7f --- /dev/null +++ b/changelog.d/1319-cctv-bosch-mic-specs.md @@ -0,0 +1,3 @@ +- Vendor the Bosch MIC PTZ datasheet records that the City of Amsterdam camera + register names (MIC-7522, MIC-7502, MIC-7130), with the register's `x` colour + wildcard as an alias, from the CCTV Camera Database v2.29.0 (ch-bas, #1319). From dec6cb937ec290d514f2c44d8cdb87a1b4f7fba7 Mon Sep 17 00:00:00 2001 From: Said Turksever Date: Thu, 22 Oct 2026 23:27:00 +0100 Subject: [PATCH 07/83] test(cctv): verify Bosch MIC aliases and preserve estimated optics --- DATA_SOURCES.md | 10 ++++--- docs/CURRENT-STATE.md | 4 +++ src/layers/cctv/modelSpecs.js | 20 +++++++------- src/layers/cctv/modelSpecs.test.mjs | 41 +++++++++++++++++++++++++++++ 4 files changed, 62 insertions(+), 13 deletions(-) diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index a9d676c85..1602cfa49 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -236,13 +236,15 @@ fixtures retain their provenance in `src/data/fixtures/README.md`. ### CCTV Camera Database (`src/layers/cctv/modelSpecs.js`) -The CCTV layer's hardware-model specs are a small verbatim extract (v2.26.0, plus the Bosch MIC +The CCTV layer's hardware-model specs are a small extract (v2.26.0, plus the Bosch MIC records from v2.29.0) of the [CCTV Camera Database](https://github.com/ch-bas/cctv-camera-database), an open dataset of camera models transcribed from manufacturer datasheets, dedicated to the public domain under -**CC0-1.0**. Only the records for hardware that the integrated public catalogs name (King -County, Sarasota County, Sioux Falls and the City of Amsterdam register) are bundled, compiled into the module. Nothing is -fetched at runtime. The records describe camera products (optics, resolution, night vision) +**CC0-1.0**. Only the records for hardware named by the integrated public catalogs (King +County, Sarasota County and Sioux Falls), plus the City of Amsterdam register proposed +in [#251](https://github.com/bilawalsidhu/gods-eye-view/pull/251), are bundled, compiled into the module. Nothing is +fetched at runtime. The MIC specification fields are unchanged; product titles are shortened +to model codes and colour aliases are added locally. The records describe camera products (optics, resolution, night vision) only, never endpoints, stream paths or access data. CC0 requires no attribution; the CCTV HUD credits it anyway as `SPECS: CCTV-DATABASE.COM` beside any spec line it renders. diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index a9db91490..13174d759 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -4585,6 +4585,10 @@ silently demoting every later lookup for the session. multi-sensor datasheets publish a capability range, not the lens's current zoom, so those cameras keep their estimate and show `SPEC FOV min–max° (CAPABILITY)` beside it. Public camera state carries `fovSource`, `specFovMinDeg`, `specFovMaxDeg` and `spec` (#452). + The table also includes Bosch MIC-7130, MIC-7502 and MIC-7522 records for the Amsterdam + register proposed in #251, including its `x` colour aliases (#1319). Their axis-less + datasheet FOV does not supply a horizontal capability range or replace the estimated + FOV. Adding these records does not enable the proposed source pack. - **FIRMS**: no ground clamping (zero 3D-tiles height sampling), ≤18 screen-decluttered ambient labels, click-to-inspect detail card, 2.5k/3k sprite budgets viewport-clipped by FRP. - **CCTV v2 foundation:** a pitched frustum wireframe (4 corner rays + far-cap rectangle) with a monitor plane at the frustum's diff --git a/src/layers/cctv/modelSpecs.js b/src/layers/cctv/modelSpecs.js index 89fcf58d2..ba6702584 100644 --- a/src/layers/cctv/modelSpecs.js +++ b/src/layers/cctv/modelSpecs.js @@ -12,24 +12,26 @@ * Cameras without an identified model are untouched — unknown never implies * anything. * - * Spec records are vendored verbatim from the CCTV Camera Database + * Specification fields are vendored from the CCTV Camera Database * (https://github.com/ch-bas/cctv-camera-database, CC0-1.0) — an open dataset * of 29,000+ camera models transcribed from manufacturer datasheets (76% with - * FOV, as of v2.29.0). This module vendors the records that the currently integrated public + * FOV, as of v2.29.0). This module vendors records that the integrated public * catalogs publish hardware for — King County WA (Cohu, Bosch AUTODOME, Axis * Q6135-LE), plus the Bosch AUTODOME and Axis PTZ/multisensor models that - * appear in Sarasota County FL and Sioux Falls SD open data, and the Bosch MIC - * PTZs in the City of Amsterdam camera register; packs that publish other - * hardware models can grow it, or vendor the full extract. + * appear in Sarasota County FL and Sioux Falls SD open data, plus the Bosch MIC + * PTZs for the City of Amsterdam camera register proposed in #251; packs that + * publish other hardware models can grow it, or vendor the full extract. * * Scope guardrail: records describe camera PRODUCTS (optics, resolution, * night vision) — never endpoints, stream paths, or access/config data. */ /** - * Verbatim records from the dataset's model-enrichment extract (v2.26.0; the - * Bosch MIC records from v2.29.0). Bosch MIC order codes end in a colour - * letter (B black, G grey, W white) on one shared datasheet; those siblings, + * Records from the dataset's model-enrichment extract (v2.26.0; the Bosch MIC + * records from v2.29.0). MIC specification fields are unchanged; product + * titles are shortened to model codes and colour aliases are added locally. + * Bosch MIC order codes end in a colour letter (B black, G grey, W white) + * on one shared datasheet; those siblings, * and the `x` wildcard registries write for the colour letter, are listed as * aliases of the one vendored record. */ @@ -488,7 +490,7 @@ const NAME_INDEX = RECORDS.flatMap((record) => { * guesses). * @param {string} model - Hardware model as named by a source pack or seed. * @param {string} [manufacturer] - The source's manufacturer, when published. - * @returns {Object|null} The verbatim spec record, or null when unknown. + * @returns {Object|null} The matched spec record, or null when unknown. */ export function lookupModelSpec(model, manufacturer) { const query = words(model).map(stripVariantSuffix); diff --git a/src/layers/cctv/modelSpecs.test.mjs b/src/layers/cctv/modelSpecs.test.mjs index 852c6ad64..d76f37c24 100644 --- a/src/layers/cctv/modelSpecs.test.mjs +++ b/src/layers/cctv/modelSpecs.test.mjs @@ -273,6 +273,47 @@ test('Amsterdam camera register model strings resolve', () => { assert.equal(horizontalFovRangeDeg(lookupModelSpec('MIC-7522-Z30B')), null); }); +test('Bosch MIC colour aliases resolve without accepting unknown variants', () => { + for (const [pattern, id] of [ + ['MIC-7130-P?4', 'bosch-mic-7130-pb4'], + ['MIC-7502-Z30?', 'bosch-mic-7502-z30b'], + ['MIC-7522-Z30?', 'bosch-mic-7522-z30b'], + ]) { + for (const colour of ['B', 'G', 'W', 'x']) { + const model = pattern.replace('?', colour); + assert.equal(lookupModelSpec(model)?.id, id, model); + assert.equal(lookupModelSpec(`bosch ${model.toLowerCase()}`)?.id, id); + assert.equal(lookupModelSpec(model, 'Bosch')?.id, id); + assert.equal(lookupModelSpec(model, 'Axis'), null); + assert.equal(lookupModelSpec(`${model}-UNKNOWN`), null); + } + assert.equal(lookupModelSpec(pattern.replace('?', 'R')), null); + } +}); + +test('Bosch MIC catalog enrichment preserves estimated and explicit optics', () => { + for (const model of [ + 'Bosch MIC-7130-Px4', + 'Bosch MIC-7502-Z30x', + 'Bosch MIC-7522-Z30x', + ]) { + for (const pose of [{}, { fovDeg: 44 }, { poseSource: 'curated' }]) { + const [camera] = browserCatalog().buildCatalogFromSources([ + { id: 'amsterdam-mic', lat: 52.37, lon: 4.9, model, ...pose }, + ]); + assert.ok(camera.spec, model); + assert.equal(camera.fovDeg, pose.fovDeg ?? 74, model); + assert.equal(camera.fovSource, undefined); + assert.equal(camera.specFovMinDeg, undefined); + assert.equal(camera.specFovMaxDeg, undefined); + assert.equal(horizontalFovRangeDeg(camera.spec), null); + assert.equal(fixedHorizontalFovDeg(camera.spec), null); + assert.equal(specFovCapabilityToken(camera), null); + assert.match(specSummary(camera.spec), /^Bosch MIC-.* · PTZ$/); + } + } +}); + test('catalog never overwrites an explicit FOV or a curated pose', () => { const [explicit, curated, cohu, pelco] = browserCatalog().buildCatalogFromSources([ From 38d29f2beb22b5200b794751b32f1f21a0409e25 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Tue, 27 Oct 2026 13:09:32 +0400 Subject: [PATCH 08/83] feat(fires): FRP intensity filter, confidence switch and six-stop color ramp (HAL-676) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Color every FIRMS detection by fire radiative power on the Portolan cold → hot ramp (blue · teal · lime · amber · orange · red at 0 · 5 · 20 · 50 · 100 · 300 MW), replacing the three warm stops that made a 2 MW and a 40 MW fire look alike. Heat cells, glow sprites and card accents all read the same ramp (src/data/firmsIntensity.js). Add a minimum-FRP log slider whose track is painted with the ramp (so it doubles as the legend), an All / Nominal+ / High confidence switch and an "N of M shown" counter to the Active Fires row. The filter runs once in ingestion.applyFilter(), so markers, cards, heat cells, the context store, the top-fires query and the analyst records all see one set. Default is 0 MW / All: many of the strongest fires are rated low. Persist both options in the layer state (token w: minFrp `f`, confidence `c`), including the short share codec, and give the agent's get_active_fires tool matching min_frp_mw / confidence arguments. Layer panel gains generic `slider` and `segmented` row descriptors (src/ui/rowFilterControls.js). New strings in es / ru / zh-CN. Co-Authored-By: Claude Fable 5.1 --- scripts/format-scope.json | 6 +- scripts/package-boundaries.json | 17 +++ src/data/firmsCards.test.mjs | 22 ++- src/data/firmsIntensity.js | 209 +++++++++++++++++++++++++++++ src/data/firmsIntensity.test.mjs | 207 ++++++++++++++++++++++++++++ src/data/firmsLabels.js | 23 ++-- src/data/firmsLabels.test.mjs | 15 ++- src/data/layerState.js | 21 ++- src/data/layerState.test.mjs | 83 +++++++++++- src/i18n/locales/es.json | 14 +- src/i18n/locales/ru.json | 14 +- src/i18n/locales/zh-CN.json | 14 +- src/i18n/same-as-english.json | 5 +- src/layers/firms/ingestion.js | 32 ++++- src/layers/firms/lifecycle.js | 4 + src/layers/firms/model.js | 57 +++++--- src/layers/firms/policy.js | 19 ++- src/layers/firms/queries.js | 142 +++++++++++++++++++- src/layers/firms/rendering.js | 5 +- src/layers/firms/state.js | 28 ++++ src/share/viewCodec.js | 1 + src/tools/queries/hazards.js | 64 ++++++++- src/tools/queries/hazards.test.mjs | 36 +++++ src/ui/layerPanel.js | 54 +++++++- src/ui/rowFilterControls.js | 169 +++++++++++++++++++++++ src/ui/rowFilterControls.test.mjs | 207 ++++++++++++++++++++++++++++ src/ui/styles/radio.css | 102 ++++++++++++++ 27 files changed, 1500 insertions(+), 70 deletions(-) create mode 100644 src/data/firmsIntensity.js create mode 100644 src/data/firmsIntensity.test.mjs create mode 100644 src/ui/rowFilterControls.js create mode 100644 src/ui/rowFilterControls.test.mjs diff --git a/scripts/format-scope.json b/scripts/format-scope.json index ee23b8b6a..c7ae4e3cf 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -1117,5 +1117,9 @@ "src/testSupport/shareViewFixtures.mjs", "src/ui/shareLinks.test.mjs", "src/ui/shareButton.test.mjs", - "src/share/shareLinkPath.test.mjs" + "src/share/shareLinkPath.test.mjs", + "src/data/firmsIntensity.js", + "src/data/firmsIntensity.test.mjs", + "src/ui/rowFilterControls.js", + "src/ui/rowFilterControls.test.mjs" ] diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index b42d90fdc..ca2ecb542 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -495,6 +495,8 @@ "src/ui/railCards.js", "src/ui/railCardBlocks.js", "src/ui/slider.js", + "src/ui/rowFilterControls.js", + "src/ui/segmentedControl.js", "src/ui/chipGroup.js", "src/ui/rowList.js", "src/ui/railTimeline.js", @@ -670,6 +672,7 @@ "src/data/detectionPolicy.js", "src/data/installationFeedback.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateCoordinator.js", "src/data/layerStateTokenReservations.json", @@ -734,6 +737,7 @@ "src/ui/effects.js", "src/ui/frameRateMonitor.js", "src/ui/layerPanel.js", + "src/ui/rowFilterControls.js", "src/ui/weatherPanel.js", "src/ui/railCards.js", "src/ui/railCardBlocks.js", @@ -954,6 +958,7 @@ "src/data/contactMatch.js", "src/data/iconOrientation.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateTokenReservations.json", "src/data/modelVisualAnchor.js", @@ -1007,6 +1012,7 @@ "src/data/contactMatch.js", "src/data/iconOrientation.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateTokenReservations.json", "src/data/modelVisualAnchor.js", @@ -1052,6 +1058,7 @@ "src/data/contactMatch.js", "src/data/iconOrientation.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateCoordinator.js", "src/data/layerStateTokenReservations.json", @@ -1136,6 +1143,7 @@ "exports": ["./layers/firms"], "modules": [ "src/data/firmsAdapt.js", + "src/data/firmsIntensity.js", "src/data/firmsLabels.js", "src/data/iconOrientation.js", "src/layers/firms/anchors.js", @@ -1525,6 +1533,7 @@ "modules": [ "src/data/iconOrientation.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateTokenReservations.json", "src/data/radioCountry.js", @@ -1848,6 +1857,7 @@ "src/data/fireAnchors.js", "src/data/firmsAdapt.js", "src/data/firmsHeatmap.js", + "src/data/firmsIntensity.js", "src/data/firmsLabels.js", "src/data/flights.js", "src/data/flowMatch.js", @@ -2259,6 +2269,7 @@ "src/ui/effects.js", "src/ui/frameRateMonitor.js", "src/ui/layerPanel.js", + "src/ui/rowFilterControls.js", "src/ui/weatherPanel.js", "src/ui/railCards.js", "src/ui/railCardBlocks.js", @@ -2687,6 +2698,7 @@ "src/data/detectionDraw.js", "src/data/detectionPolicy.js", "src/data/firmsAdapt.js", + "src/data/firmsIntensity.js", "src/data/firmsLabels.js", "src/data/flowMatch.js", "src/data/focusDeemphasis.js", @@ -3283,6 +3295,7 @@ "src/i18n/marker.js", "src/data/installationFeedback.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateCoordinator.js", "src/data/layerStateTokenReservations.json", @@ -3899,6 +3912,7 @@ "src/styles/surveillance.js", "src/styles/thermal.js", "src/view/index.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateTokenReservations.json", "src/data/idSwitches.js", @@ -3975,6 +3989,8 @@ "src/tools/queries/brief.js", "src/tools/queries/maritime.js", "src/tools/queries/share.js", + "src/data/firmsIntensity.js", + "src/data/firmsAdapt.js", "src/data/idSwitches.js", "src/data/vesselTypes.js", "src/data/layerState.js", @@ -4107,6 +4123,7 @@ "modules": [ "src/view/index.js", "src/data/idSwitches.js", + "src/data/firmsIntensity.js", "src/data/layerState.js", "src/data/layerStateTokenReservations.json" ], diff --git a/src/data/firmsCards.test.mjs b/src/data/firmsCards.test.mjs index a0ee83c01..f82b8da80 100644 --- a/src/data/firmsCards.test.mjs +++ b/src/data/firmsCards.test.mjs @@ -68,11 +68,27 @@ test('buildFireCard: power that rounds to 10 MW drops the decimal', () => { assert.equal(selected.title, 'FIRE · 10 MW'); }); -test('buildFireCard: SNPP satellite code renders as SNPP, weak fire is not red', () => { +test('buildFireCard: SNPP satellite code renders as SNPP, weak fire is blue', () => { const card = buildFireCard({ fire: fire({ satellite: 'N', frp: 0.8, confidence: 0.3 }), position: {} }, NOW); assert.match(card.details[0], /SNPP$/); assert.equal(card.title, '▲ 0.8 MW'); - assert.notEqual(card.accent, accentForSeverity('red')); + assert.equal(card.accent, accentForSeverity('blue')); +}); + +test('buildFireCard: accent steps through the FRP ramp at 5 · 20 · 50 · 100 · 300 MW', () => { + const accentAt = (frp) => + buildFireCard({ fire: fire({ frp }), position: {} }, NOW).accent; + assert.equal(accentAt(4.9), accentForSeverity('blue')); + assert.equal(accentAt(5), accentForSeverity('teal')); + assert.equal(accentAt(20), accentForSeverity('lime')); + assert.equal(accentAt(50), accentForSeverity('amber')); + assert.equal(accentAt(100), accentForSeverity('orange')); + assert.equal(accentAt(300), accentForSeverity('red')); + // Confidence never moves the color: a low-confidence 864 MW fire stays red. + assert.equal( + buildFireCard({ fire: fire({ frp: 864, confidence: 0.3 }), position: {} }, NOW).accent, + accentForSeverity('red'), + ); }); test('buildSelectedFireCard: full detail card with coords, selected flag, no fade', () => { @@ -125,7 +141,7 @@ test('buildCellCard: singular noun and missing-age omission', () => { const card = buildCellCard(candidate, NOW); assert.equal(card.title, '1 FIRE'); assert.equal(card.details[0], 'max 9.9 MW'); - assert.equal(card.accent, accentForSeverity('yellow'), 'missing accent defaults to yellow'); + assert.equal(card.accent, accentForSeverity('blue'), 'missing accent defaults to the coldest stop'); }); test('FIRMS host policy keeps ambient cards bounded and selected cards protected', () => { diff --git a/src/data/firmsIntensity.js b/src/data/firmsIntensity.js new file mode 100644 index 000000000..417566346 --- /dev/null +++ b/src/data/firmsIntensity.js @@ -0,0 +1,209 @@ +/** + * @module firmsIntensity + * @description Fire intensity policy shared by the Active Fires layer and the + * agent fire tool: the six-stop cold → hot FRP color ramp, the log-scale + * slider that doubles as its legend, and the one filter every consumer + * (markers, cards, heat cells, the top-fires query, `get_active_fires`) runs + * before anything is rendered or selected. Pure — no Cesium, no DOM. + * + * Hue order follows the Portolan FIRMS explorer (`FRP_RAMP`, stepped, not + * interpolated). The thresholds are GEV's own, set on a real week's + * percentiles so every color appears: Portolan's 0/10/50/200/500/1000 MW + * leaves one red fire a week (HAL-675). + */ + +/** + * Stepped FRP ramp, coldest first. A detection takes the LAST stop whose + * `minFrp` it reaches. Names key the glow-sprite cache and the card accents. + */ +export const FRP_RAMP = Object.freeze([ + Object.freeze({ name: 'blue', minFrp: 0, hex: '#4a5bd4' }), + Object.freeze({ name: 'teal', minFrp: 5, hex: '#39a0a8' }), + Object.freeze({ name: 'lime', minFrp: 20, hex: '#c9cf4a' }), + Object.freeze({ name: 'amber', minFrp: 50, hex: '#f2b134' }), + Object.freeze({ name: 'orange', minFrp: 100, hex: '#e8722c' }), + Object.freeze({ name: 'red', minFrp: 300, hex: '#d1382a' }), +]); + +/** Slider ticks: the ramp thresholds, so slider, legend and markers agree. */ +export const FRP_SLIDER_TICKS = Object.freeze( + FRP_RAMP.map((stop) => stop.minFrp), +); + +/** Minimum-FRP values the share codec accepts, in whole megawatts. */ +export const MIN_FRP_MW_MIN = 0; +export const MIN_FRP_MW_MAX = 10_000; + +/** Confidence switch positions; VIIRS reports only low / nominal / high. */ +export const FIRMS_CONFIDENCE_MODES = Object.freeze(['all', 'nominal', 'high']); + +/** + * Normalized-confidence floor for a switch position. The thresholds match + * `confidenceBucket` in the layer model (0.3 / 0.6 / 0.9 after firmsAdapt). + */ +const CONFIDENCE_FLOOR = Object.freeze({ all: 0, nominal: 0.45, high: 0.75 }); + +export const DEFAULT_FIRE_FILTER = Object.freeze({ + minFrp: 0, + confidence: 'all', +}); + +/** + * The ramp stop for a fire radiative power. + * @param {number} frp Megawatts; non-finite or negative reads as 0. + * @returns {{name: string, minFrp: number, hex: string}} + */ +export function frpRampStop(frp) { + const value = Number.isFinite(frp) ? Math.max(0, frp) : 0; + let stop = FRP_RAMP[0]; + for (const candidate of FRP_RAMP) { + if (value >= candidate.minFrp) stop = candidate; + else break; + } + return stop; +} + +/** `#rrggbb` → `[r, g, b]` in 0..255. */ +export function hexToRgb(hex) { + const text = String(hex).replace('#', ''); + return [0, 2, 4].map((offset) => + Number.parseInt(text.slice(offset, offset + 2), 16), + ); +} + +/** + * Log-scale slider axis: position 0 means "no minimum" (0 MW), and the rest + * of the track runs 1 MW → SLIDER_MAX_MW logarithmically. The axis floor is 1 + * MW because the median detection is ~6 MW and a sub-megawatt floor hides + * nothing worth a tick. + */ +export const SLIDER_MAX_MW = 1000; +const LOG_MAX = Math.log10(SLIDER_MAX_MW); + +/** + * Track position in 0..1 for a minimum FRP. + * @param {number} minFrp Megawatts. + * @returns {number} + */ +export function sliderPositionForFrp(minFrp) { + if (!Number.isFinite(minFrp) || minFrp < 1) return 0; + return Math.min(1, Math.log10(minFrp) / LOG_MAX); +} + +/** + * Minimum FRP for a track position, rounded to whole megawatts so a slider + * value and its share-link form are the same number. + * @param {number} position 0..1. + * @returns {number} + */ +export function frpForSliderPosition(position) { + const p = Number.isFinite(position) ? Math.max(0, Math.min(1, position)) : 0; + if (p <= 0) return 0; + const frp = Math.pow(10, p * LOG_MAX); + return Math.max(1, Math.round(frp)); +} + +/** + * CSS `linear-gradient` painting the stepped ramp along the log axis, so the + * slider track IS the legend: the color under the thumb is the color of the + * weakest fire still shown. + * @returns {string} + */ +export function rampTrackGradient() { + const parts = []; + FRP_RAMP.forEach((stop, index) => { + const start = sliderPositionForFrp(stop.minFrp) * 100; + const next = FRP_RAMP[index + 1]; + const end = next ? sliderPositionForFrp(next.minFrp) * 100 : 100; + parts.push(`${stop.hex} ${start.toFixed(2)}% ${end.toFixed(2)}%`); + }); + return `linear-gradient(90deg, ${parts.join(', ')})`; +} + +/** + * Minimum FRP → whole megawatts inside the codec bounds, or null. + * @param {*} value + * @returns {?number} + */ +export function normalizeMinFrp(value) { + const number = + typeof value === 'number' + ? value + : typeof value === 'string' && value.trim() !== '' + ? Number(value) + : NaN; + if (!Number.isFinite(number)) return null; + const rounded = Math.round(number); + return rounded >= MIN_FRP_MW_MIN && rounded <= MIN_FRP_MW_MAX + ? rounded + : null; +} + +/** + * Confidence mode → one of FIRMS_CONFIDENCE_MODES, or null. + * @param {*} value + * @returns {?string} + */ +export function normalizeConfidenceMode(value) { + const text = String(value ?? '') + .trim() + .toLowerCase(); + return FIRMS_CONFIDENCE_MODES.includes(text) ? text : null; +} + +/** + * Sanitize a partial filter against the current one; unknown or invalid + * fields keep their current value. + * @param {Object} [params] + * @param {{minFrp: number, confidence: string}} [current] + * @returns {{minFrp: number, confidence: string}} + */ +export function normalizeFireFilter( + params = {}, + current = DEFAULT_FIRE_FILTER, +) { + const minFrp = normalizeMinFrp(params?.minFrp); + const confidence = normalizeConfidenceMode(params?.confidence); + return { + minFrp: minFrp === null ? current.minFrp : minFrp, + confidence: confidence === null ? current.confidence : confidence, + }; +} + +/** + * Whether one fire passes the filter. `confidence` is the normalized 0..1 + * value firmsAdapt produces; `frp` is megawatts. + * @param {{frp: number, confidence: number}} fire + * @param {{minFrp: number, confidence: string}} filter + * @returns {boolean} + */ +export function firePassesFilter(fire, filter = DEFAULT_FIRE_FILTER) { + const frp = Number.isFinite(fire?.frp) ? fire.frp : 0; + if (frp < (filter.minFrp || 0)) return false; + const floor = CONFIDENCE_FLOOR[filter.confidence] ?? 0; + if (floor <= 0) return true; + const confidence = Number.isFinite(fire?.confidence) ? fire.confidence : 0; + return confidence >= floor; +} + +/** + * The fires that pass the filter, in input order. Returns the SAME array when + * the filter is the default, so callers can tell "nothing hidden" cheaply. + * @template T + * @param {Array} fires Internal records (normalized confidence). + * @param {{minFrp: number, confidence: string}} filter + * @returns {Array} + */ +export function filterFires(fires, filter = DEFAULT_FIRE_FILTER) { + if (!Array.isArray(fires)) return []; + if (isDefaultFireFilter(filter)) return fires; + return fires.filter((fire) => firePassesFilter(fire, filter)); +} + +/** True when the filter hides nothing. */ +export function isDefaultFireFilter(filter) { + return ( + !(filter?.minFrp > 0) && + (filter?.confidence === 'all' || filter?.confidence == null) + ); +} diff --git a/src/data/firmsIntensity.test.mjs b/src/data/firmsIntensity.test.mjs new file mode 100644 index 000000000..558068e67 --- /dev/null +++ b/src/data/firmsIntensity.test.mjs @@ -0,0 +1,207 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + DEFAULT_FIRE_FILTER, + FIRMS_CONFIDENCE_MODES, + FRP_RAMP, + FRP_SLIDER_TICKS, + filterFires, + firePassesFilter, + frpForSliderPosition, + frpRampStop, + hexToRgb, + isDefaultFireFilter, + normalizeConfidenceMode, + normalizeFireFilter, + normalizeMinFrp, + rampTrackGradient, + sliderPositionForFrp, +} from './firmsIntensity.js'; + +test('the ramp has six cold → hot stops at 0 · 5 · 20 · 50 · 100 · 300 MW in Portolan hue order', () => { + assert.deepEqual( + FRP_RAMP.map((stop) => [stop.name, stop.minFrp, stop.hex]), + [ + ['blue', 0, '#4a5bd4'], + ['teal', 5, '#39a0a8'], + ['lime', 20, '#c9cf4a'], + ['amber', 50, '#f2b134'], + ['orange', 100, '#e8722c'], + ['red', 300, '#d1382a'], + ], + ); + assert.deepEqual([...FRP_SLIDER_TICKS], [0, 5, 20, 50, 100, 300]); + assert.ok(Object.isFrozen(FRP_RAMP)); +}); + +test('a detection takes the last stop it reaches; garbage reads as 0 MW', () => { + assert.equal(frpRampStop(0).name, 'blue'); + assert.equal(frpRampStop(4.99).name, 'blue'); + assert.equal(frpRampStop(5).name, 'teal'); + assert.equal(frpRampStop(19.9).name, 'teal'); + assert.equal(frpRampStop(20).name, 'lime'); + assert.equal(frpRampStop(50).name, 'amber'); + assert.equal(frpRampStop(100).name, 'orange'); + assert.equal(frpRampStop(299.9).name, 'orange'); + assert.equal(frpRampStop(300).name, 'red'); + assert.equal(frpRampStop(1246).name, 'red'); + assert.equal(frpRampStop(NaN).name, 'blue'); + assert.equal(frpRampStop(-3).name, 'blue'); + assert.equal(frpRampStop(undefined).name, 'blue'); +}); + +test('hex stops convert to the "r, g, b" accents the cards paint', () => { + assert.deepEqual(hexToRgb('#4a5bd4'), [74, 91, 212]); + assert.deepEqual(hexToRgb('d1382a'), [209, 56, 42]); +}); + +test('the log slider pins 0 MW at the left and runs 1 → 1000 MW; values round-trip', () => { + assert.equal(sliderPositionForFrp(0), 0); + assert.equal(sliderPositionForFrp(0.5), 0); + assert.equal(sliderPositionForFrp(1), 0); + assert.ok(Math.abs(sliderPositionForFrp(10) - 1 / 3) < 1e-9); + assert.ok(Math.abs(sliderPositionForFrp(100) - 2 / 3) < 1e-9); + assert.equal(sliderPositionForFrp(1000), 1); + assert.equal(sliderPositionForFrp(5000), 1, 'clamped past the axis end'); + assert.equal(frpForSliderPosition(0), 0); + assert.equal(frpForSliderPosition(1), 1000); + assert.equal(frpForSliderPosition(-1), 0); + assert.equal(frpForSliderPosition(2), 1000); + for (const frp of FRP_SLIDER_TICKS) { + assert.equal( + frpForSliderPosition(sliderPositionForFrp(frp)), + frp, + `tick ${frp} MW round-trips through the axis`, + ); + } + // Ticks spread along the track instead of bunching at one end. + const positions = FRP_SLIDER_TICKS.map(sliderPositionForFrp); + for (let i = 1; i < positions.length; i += 1) { + assert.ok(positions[i] - positions[i - 1] >= 0.09, `tick gap ${i}`); + } +}); + +test('the track gradient is the stepped ramp on the log axis, so it doubles as the legend', () => { + const gradient = rampTrackGradient(); + assert.match(gradient, /^linear-gradient\(90deg, /); + assert.match(gradient, /#4a5bd4 0\.00% 23\.3\d%/); + assert.match(gradient, /#39a0a8 23\.3\d% 43\.3\d%/); + assert.match(gradient, /#d1382a 82\.5\d% 100\.00%\)$/); + assert.equal(gradient.match(/#/g).length, FRP_RAMP.length); +}); + +test('minimum FRP normalizes to whole megawatts inside the codec bounds', () => { + assert.equal(normalizeMinFrp(0), 0); + assert.equal(normalizeMinFrp(19.6), 20); + assert.equal(normalizeMinFrp('100'), 100); + assert.equal(normalizeMinFrp(10000), 10000); + assert.equal(normalizeMinFrp(10001), null); + assert.equal(normalizeMinFrp(-1), null); + assert.equal(normalizeMinFrp('abc'), null); + assert.equal(normalizeMinFrp(''), null); + assert.equal(normalizeMinFrp(null), null); + assert.equal(normalizeMinFrp(NaN), null); +}); + +test('confidence modes are exactly all / nominal / high', () => { + assert.deepEqual([...FIRMS_CONFIDENCE_MODES], ['all', 'nominal', 'high']); + assert.equal(normalizeConfidenceMode(' High '), 'high'); + assert.equal(normalizeConfidenceMode('nominal'), 'nominal'); + assert.equal(normalizeConfidenceMode('low'), null); + assert.equal(normalizeConfidenceMode(0.9), null); + assert.equal(normalizeConfidenceMode(undefined), null); +}); + +test('a partial filter patches the current one and invalid fields keep their value', () => { + assert.deepEqual(normalizeFireFilter(), DEFAULT_FIRE_FILTER); + assert.deepEqual(normalizeFireFilter({ minFrp: 20 }), { + minFrp: 20, + confidence: 'all', + }); + assert.deepEqual( + normalizeFireFilter( + { confidence: 'high' }, + { minFrp: 50, confidence: 'all' }, + ), + { minFrp: 50, confidence: 'high' }, + ); + assert.deepEqual( + normalizeFireFilter( + { minFrp: 'nope', confidence: 'medium' }, + { minFrp: 50, confidence: 'nominal' }, + ), + { minFrp: 50, confidence: 'nominal' }, + ); + assert.equal(isDefaultFireFilter(DEFAULT_FIRE_FILTER), true); + assert.equal(isDefaultFireFilter({ minFrp: 0, confidence: 'all' }), true); + assert.equal(isDefaultFireFilter({ minFrp: 1, confidence: 'all' }), false); + assert.equal(isDefaultFireFilter({ minFrp: 0, confidence: 'high' }), false); +}); + +test('the filter keeps a fire at or above the floor and at or above the confidence mode', () => { + const low = { frp: 864, confidence: 0.3 }; + const nominal = { frp: 20, confidence: 0.6 }; + const high = { frp: 2, confidence: 0.9 }; + assert.equal(firePassesFilter(low), true, 'default shows everything'); + assert.equal( + firePassesFilter(nominal, { minFrp: 20, confidence: 'all' }), + true, + ); + assert.equal( + firePassesFilter(nominal, { minFrp: 21, confidence: 'all' }), + false, + ); + assert.equal( + firePassesFilter(low, { minFrp: 0, confidence: 'nominal' }), + false, + ); + assert.equal( + firePassesFilter(nominal, { minFrp: 0, confidence: 'nominal' }), + true, + ); + assert.equal( + firePassesFilter(high, { minFrp: 0, confidence: 'nominal' }), + true, + ); + assert.equal( + firePassesFilter(nominal, { minFrp: 0, confidence: 'high' }), + false, + ); + assert.equal(firePassesFilter(high, { minFrp: 0, confidence: 'high' }), true); + assert.equal( + firePassesFilter(high, { minFrp: 5, confidence: 'high' }), + false, + ); + // Missing numbers read as 0, never as a pass-through. + assert.equal(firePassesFilter({}, { minFrp: 1, confidence: 'all' }), false); + assert.equal( + firePassesFilter({ frp: 10 }, { minFrp: 0, confidence: 'nominal' }), + false, + ); +}); + +test('filterFires returns the same array for the default filter and preserves order otherwise', () => { + const fires = [ + { frp: 864, confidence: 0.3 }, + { frp: 20, confidence: 0.6 }, + { frp: 2, confidence: 0.9 }, + { frp: 120, confidence: 0.9 }, + ]; + assert.equal(filterFires(fires), fires); + assert.equal(filterFires(fires, { minFrp: 0, confidence: 'all' }), fires); + assert.deepEqual( + filterFires(fires, { minFrp: 20, confidence: 'all' }).map((f) => f.frp), + [864, 20, 120], + ); + assert.deepEqual( + filterFires(fires, { minFrp: 0, confidence: 'high' }).map((f) => f.frp), + [2, 120], + ); + assert.deepEqual( + filterFires(fires, { minFrp: 100, confidence: 'nominal' }).map( + (f) => f.frp, + ), + [120], + ); + assert.deepEqual(filterFires(null, { minFrp: 1, confidence: 'all' }), []); +}); diff --git a/src/data/firmsLabels.js b/src/data/firmsLabels.js index b9e9d1765..3af6a15a5 100644 --- a/src/data/firmsLabels.js +++ b/src/data/firmsLabels.js @@ -4,6 +4,8 @@ * dedicated card canvas moved into the shared world-overlay host. */ +import { FRP_RAMP, hexToRgb } from './firmsIntensity.js'; + /** Shipped ambient FIRMS card ceiling. Selected fires bypass this cohort. */ export const FIRMS_AMBIENT_COHORT_LIMIT = 18; export const FIRMS_OVERLAY_SOURCE_ID = 'firms'; @@ -23,20 +25,23 @@ export function fireDetectionKey(fire) { return `firms:${lat}:${lon}:${acq}:${source}`; } -/** Severity accent palette — matches the FIRMS glow-sprite color stops. */ -const ACCENT_RGB = Object.freeze({ - red: '224, 82, 82', - orange: '240, 178, 62', - yellow: '244, 227, 108', -}); +/** + * Severity accent palette — the FIRMS glow-sprite color stops (the six-stop + * FRP ramp in firmsIntensity.js) as "r, g, b" strings for card accents. + */ +const ACCENT_RGB = Object.freeze( + Object.fromEntries( + FRP_RAMP.map((stop) => [stop.name, hexToRgb(stop.hex).join(', ')]), + ), +); /** - * Severity stop name → "r, g, b" accent string (defaults to yellow). - * @param {string} stopName - 'red' | 'orange' | 'yellow'. + * Severity stop name → "r, g, b" accent string (defaults to the coldest stop). + * @param {string} stopName - A FRP_RAMP stop name ('blue' … 'red'). * @returns {string} */ export function accentForSeverity(stopName) { - return ACCENT_RGB[stopName] || ACCENT_RGB.yellow; + return ACCENT_RGB[stopName] || ACCENT_RGB[FRP_RAMP[0].name]; } /** diff --git a/src/data/firmsLabels.test.mjs b/src/data/firmsLabels.test.mjs index 8573d7330..62234bb2d 100644 --- a/src/data/firmsLabels.test.mjs +++ b/src/data/firmsLabels.test.mjs @@ -9,11 +9,16 @@ import { formatFrpMw, } from './firmsLabels.js'; -test('FIRMS formatting helpers retain the shipped severity palette', () => { - assert.equal(accentForSeverity('red'), '224, 82, 82'); - assert.equal(accentForSeverity('orange'), '240, 178, 62'); - assert.equal(accentForSeverity('yellow'), '244, 227, 108'); - assert.equal(accentForSeverity('chartreuse'), accentForSeverity('yellow')); +test('FIRMS card accents are the six FRP ramp stops, coldest as the fallback', () => { + // Portolan hue order; blue #4a5bd4 … red #d1382a (HAL-675). + assert.equal(accentForSeverity('blue'), '74, 91, 212'); + assert.equal(accentForSeverity('teal'), '57, 160, 168'); + assert.equal(accentForSeverity('lime'), '201, 207, 74'); + assert.equal(accentForSeverity('amber'), '242, 177, 52'); + assert.equal(accentForSeverity('orange'), '232, 114, 44'); + assert.equal(accentForSeverity('red'), '209, 56, 42'); + assert.equal(accentForSeverity('yellow'), accentForSeverity('blue')); + assert.equal(accentForSeverity('chartreuse'), accentForSeverity('blue')); }); test('FIRMS power that rounds to 10 MW drops the decimal', () => { diff --git a/src/data/layerState.js b/src/data/layerState.js index 089c8c08b..c68a67e67 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -298,6 +298,20 @@ function satelliteFilterOption() { } const OPTION_GROUPS = Object.freeze({ + // Active Fires intensity filter (HAL-676). `minFrp` is whole megawatts on + // the codec side; the panel's log slider rounds to it. Both default to + // "show everything": many of the strongest fires are rated low confidence. + // The bounds and modes mirror MIN_FRP_MW_MAX / FIRMS_CONFIDENCE_MODES in + // firmsIntensity.js (pinned by layerState.test); spelled out here because + // this file runs in isolation in the token-ledger tests. + 'local-firms': Object.freeze([ + boundedIntegerOption('minFrp', 'f', 0, { min: 0, max: 10000 }), + enumOption('confidence', 'c', 'all', ['all', 'nominal', 'high'], { + all: 'a', + nominal: 'n', + high: 'h', + }), + ]), traffic: Object.freeze([ enumOption('roadMode', 'r', null, ['tomtom', 'osm', 'hybrid'], { tomtom: 't', @@ -621,7 +635,12 @@ export const LAYER_STATE_REGISTRY = Object.freeze([ token: 'd', disposition: 'enabled-only', }), - Object.freeze({ id: 'local-firms', token: 'w', disposition: 'enabled-only' }), + Object.freeze({ + id: 'local-firms', + token: 'w', + disposition: 'enabled+options', + optionOwner: 'local-firms', + }), Object.freeze({ id: 'military', token: 'm', diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index d05fb9a95..d343fe20f 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -29,6 +29,7 @@ import radioLayer from './radio.js'; import { stampInitialShareGesture } from '../navigationPolicy.js'; import { MAX_SINCE_DAYS } from '../layers/streetLevel/policy.js'; import { SINCE_STOPS } from '../ui/streetLevelPresentation.js'; +import { FIRMS_CONFIDENCE_MODES, MIN_FRP_MW_MAX } from './firmsIntensity.js'; function deferred() { let resolve; @@ -2984,9 +2985,89 @@ test('Clear All cancels a saved restore still waiting to start', async () => { gate.resolve(); const results = await restore; for (const id of ['earthquakes', 'flights']) { - assert.equal(results.find((result) => result.layerId === id).succeeded, false); + assert.equal( + results.find((result) => result.layerId === id).succeeded, + false, + ); } assert.deepEqual([...manager.getEnabledLayerIds()], []); assert.equal(enabledLayerParam(sink), ''); coordinator.destroy(); }); + +/** Active Fires options decoded from a v2 link with these `lo` assignments. */ +function firmsOptions(lo) { + const params = new URLSearchParams([ + ['v', '2'], + ['l', 'w'], + ]); + if (lo) params.set('lo', lo); + return decodeLayerStateParams(params).options['local-firms']; +} + +test('Active Fires: the intensity filter rides share token w and round-trips', () => { + const state = createDefaultLayerState(); + assert.deepEqual(state.options['local-firms'], { + minFrp: 0, + confidence: 'all', + }); + state.enabledLayerIds = ['local-firms']; + state.options['local-firms'] = { minFrp: 20, confidence: 'high' }; + const encoded = encode(state); + assert.equal(new URLSearchParams(encoded).get('l'), 'w'); + // Other owners write their own explicit defaults into `lo` too (flights 3D, + // wind overlay); the fire assignments ride beside them. + const assignments = new URLSearchParams(encoded).get('lo').split('_'); + assert.ok(assignments.includes('w.f.20'), assignments.join(' ')); + assert.ok(assignments.includes('w.c.h'), assignments.join(' ')); + const decoded = decodeLayerStateParams(new URLSearchParams(encoded)); + assert.deepEqual(decoded.options['local-firms'], { + minFrp: 20, + confidence: 'high', + }); + assert.deepEqual( + parseStoredLayerState(serializeStoredLayerState(state)).options[ + 'local-firms' + ], + state.options['local-firms'], + ); + // Defaults are omitted from the link: a plain `l=w` is "show everything". + const plain = createDefaultLayerState(); + plain.enabledLayerIds = ['local-firms']; + assert.equal( + new URLSearchParams(encode(plain)) + .get('lo') + .split('_') + .filter((assignment) => assignment.startsWith('w.')).length, + 0, + ); + assert.deepEqual(firmsOptions(''), { minFrp: 0, confidence: 'all' }); +}); + +test('Active Fires: the codec accepts whole megawatts in bounds and the three confidence codes', () => { + // The codec spells its bounds out (it runs in isolation in the ledger + // tests); they must stay the layer's. + assert.equal(MIN_FRP_MW_MAX, 10000); + assert.deepEqual([...FIRMS_CONFIDENCE_MODES], ['all', 'nominal', 'high']); + assert.equal(firmsOptions('w.f.300').minFrp, 300); + assert.equal(firmsOptions('w.f.10000').minFrp, 10000); + assert.equal(firmsOptions('w.f.10001').minFrp, 0, 'out of bounds → default'); + assert.equal(firmsOptions('w.f.-5').minFrp, 0); + assert.equal( + firmsOptions('w.f.2.5').minFrp, + 0, + 'a third segment is rejected', + ); + assert.equal(firmsOptions('w.c.a').confidence, 'all'); + assert.equal(firmsOptions('w.c.n').confidence, 'nominal'); + assert.equal(firmsOptions('w.c.h').confidence, 'high'); + assert.equal( + firmsOptions('w.c.x').confidence, + 'all', + 'unknown code → default', + ); + assert.deepEqual(firmsOptions('w.c.n_w.f.5'), { + minFrp: 5, + confidence: 'nominal', + }); +}); diff --git a/src/i18n/locales/es.json b/src/i18n/locales/es.json index 0285917b0..ffee7fb87 100644 --- a/src/i18n/locales/es.json +++ b/src/i18n/locales/es.json @@ -3001,5 +3001,17 @@ "Your choice applies now, but could not be saved.": "Tu elección se aplica ahora, pero no se pudo guardar.", "Voice replies appear here.": "Las respuestas de voz aparecen aquí.", "Preview is off.": "La vista previa está desactivada.", - "Automatic stop is unavailable for this voice connection.": "La parada automática no está disponible para esta conexión de voz." + "Automatic stop is unavailable for this voice connection.": "La parada automática no está disponible para esta conexión de voz.", + "fire confidence filter|All": "Todas", + "fire confidence filter|Nominal+": "Nominal+", + "fire confidence filter|High": "Alta", + "Every detection — many of the strongest fires are rated low confidence": "Todas las detecciones — muchos de los incendios más intensos tienen confianza baja", + "Nominal and high confidence detections": "Detecciones de confianza nominal y alta", + "High confidence detections only": "Solo detecciones de confianza alta", + "Min FRP": "FRP mín.", + "Minimum fire radiative power, megawatts": "Potencia radiativa mínima del fuego, en megavatios", + "{value} MW": "{value} MW", + "fire intensity filter|All": "Todo", + "Detection confidence": "Confianza de la detección", + "{shown} of {total} shown": "{shown} de {total} mostrados" } diff --git a/src/i18n/locales/ru.json b/src/i18n/locales/ru.json index b1c1be229..1cd88757d 100644 --- a/src/i18n/locales/ru.json +++ b/src/i18n/locales/ru.json @@ -3070,5 +3070,17 @@ "Your choice applies now, but could not be saved.": "Ваш выбор применён, но сохранить его не удалось.", "Voice replies appear here.": "Здесь появляются голосовые ответы.", "Preview is off.": "Предпросмотр отключён.", - "Automatic stop is unavailable for this voice connection.": "Автоматическая остановка недоступна для этого голосового подключения." + "Automatic stop is unavailable for this voice connection.": "Автоматическая остановка недоступна для этого голосового подключения.", + "fire confidence filter|All": "Все", + "fire confidence filter|Nominal+": "Средняя+", + "fire confidence filter|High": "Высокая", + "Every detection — many of the strongest fires are rated low confidence": "Все детекции — у многих сильнейших пожаров достоверность низкая", + "Nominal and high confidence detections": "Детекции средней и высокой достоверности", + "High confidence detections only": "Только детекции высокой достоверности", + "Min FRP": "Мин. FRP", + "Minimum fire radiative power, megawatts": "Минимальная мощность излучения пожара, мегаватты", + "{value} MW": "{value} МВт", + "fire intensity filter|All": "Все", + "Detection confidence": "Достоверность детекции", + "{shown} of {total} shown": "показано {shown} из {total}" } diff --git a/src/i18n/locales/zh-CN.json b/src/i18n/locales/zh-CN.json index 9bf76700c..3044c45f8 100644 --- a/src/i18n/locales/zh-CN.json +++ b/src/i18n/locales/zh-CN.json @@ -2863,5 +2863,17 @@ "{count} placemarks": { "other": "{count} 个地标" }, - "S2 Underground publishes these KMZ products to GitHub. New uploads are detected within seconds and replace the previous edition automatically. Click a marker for its report.": "S2 Underground 在 GitHub 发布这些 KMZ 产品。新上传的内容会在数秒内被检测到，并自动替换旧版本。点击标记查看报告。" + "S2 Underground publishes these KMZ products to GitHub. New uploads are detected within seconds and replace the previous edition automatically. Click a marker for its report.": "S2 Underground 在 GitHub 发布这些 KMZ 产品。新上传的内容会在数秒内被检测到，并自动替换旧版本。点击标记查看报告。", + "fire confidence filter|All": "全部", + "fire confidence filter|Nominal+": "标称+", + "fire confidence filter|High": "高", + "Every detection — many of the strongest fires are rated low confidence": "所有检测——许多最强的火点置信度为低", + "Nominal and high confidence detections": "标称和高置信度的检测", + "High confidence detections only": "仅高置信度的检测", + "Min FRP": "最小 FRP", + "Minimum fire radiative power, megawatts": "最小火辐射功率（兆瓦）", + "{value} MW": "{value} MW", + "fire intensity filter|All": "全部", + "Detection confidence": "检测置信度", + "{shown} of {total} shown": "显示 {shown} / {total}" } diff --git a/src/i18n/same-as-english.json b/src/i18n/same-as-english.json index 05f5d3dee..5f1b14f8a 100644 --- a/src/i18n/same-as-english.json +++ b/src/i18n/same-as-english.json @@ -98,6 +98,7 @@ "WX", "audio {state}", "dB", + "fire confidence filter|Nominal+", "fire confidence|nominal", "ft", "hPa", @@ -120,6 +121,7 @@ "{value} KM", "{value} KM/H", "{value} KM/S", + "{value} MW", "ΔE", "ΔN", "▲ {frp} MW" @@ -168,9 +170,9 @@ "MM", "NVG", "REC", - "SRC {value}", "S2 GhostMaps", "S2 · {name}", + "SRC {value}", "Sentinel-2", "Techno", "Trance", @@ -195,6 +197,7 @@ "{type} SPD: {speed} HDG: {heading}", "{value} KM", "{value} KM/S", + "{value} MW", "ΔE", "ΔN", "▲ {frp} MW" diff --git a/src/layers/firms/ingestion.js b/src/layers/firms/ingestion.js index 494ab5f1e..215c35da3 100644 --- a/src/layers/firms/ingestion.js +++ b/src/layers/firms/ingestion.js @@ -1,5 +1,6 @@ import { adaptFirmsRecords } from '../../data/firmsAdapt.js'; import { fireDetectionKey } from '../../data/firmsLabels.js'; +import { filterFires } from '../../data/firmsIntensity.js'; import { msg } from '../../i18n/index.js'; export function createIngestion({ @@ -40,12 +41,13 @@ export function createIngestion({ layerState._stale = Boolean(payload?.stale); const previousSelection = layerState._selectedFire; layerState._selectedFire = null; - layerState._fires = adaptFirmsRecords(payload?.fires); - layerState._cellCacheByGrid.clear(); // aggregation is per-dataset — new fires, new cells - layerState._firesByFrp = [...layerState._fires].sort( + layerState._allFires = adaptFirmsRecords(payload?.fires); + // Sorted once per payload; applyFilter only filters, so a slider drag + // over 200k detections never re-sorts. + layerState._allFiresByFrp = [...layerState._allFires].sort( (a, b) => b.frp - a.frp, ); - layerState._count = layerState._fires.length; + applyFilter(); // Data age, not response age: a stale proxy payload truthfully reads old. layerState._lastUpdate = Number.isFinite(payload?.fetchedAt) ? payload.fetchedAt @@ -65,6 +67,7 @@ export function createIngestion({ } components.rendering.renderCurrentLod(true); if (reselected) components.selection.selectFire(reselected, false); + layerState._rowControlsListener?.(); } catch (error) { if ( request.signal.aborted || @@ -83,6 +86,25 @@ export function createIngestion({ } } + /** + * Run the intensity filter over the loaded payload. This is the ONE place + * the shown set is derived: `_fires` / `_firesByFrp` feed markers, cards, + * heat cells, the context store, the top-fires query and the analyst + * records, so none of them can disagree about which fires exist. Cells are + * aggregated per shown set, so the cache goes with it. + */ + + function applyFilter() { + layerState._fires = filterFires(layerState._allFires, layerState._filter); + layerState._cellCacheByGrid.clear(); + layerState._firesByFrp = filterFires( + layerState._allFiresByFrp, + layerState._filter, + ); + layerState._count = layerState._fires.length; + layerState._totalCount = layerState._allFires.length; + } + /** * Find the record in the freshly-loaded set matching a previous selection. * Identity is the detection itself (lat/lon/acquisition time) — indices @@ -109,5 +131,5 @@ export function createIngestion({ }, }; - return { loadHeatmap, findMatchingFire, methods }; + return { loadHeatmap, applyFilter, findMatchingFire, methods }; } diff --git a/src/layers/firms/lifecycle.js b/src/layers/firms/lifecycle.js index 7e0b42a00..1138504dd 100644 --- a/src/layers/firms/lifecycle.js +++ b/src/layers/firms/lifecycle.js @@ -91,10 +91,14 @@ export function createLifecycle({ components.rendering.removeDetectionCollections(viewer); components.selection.clearContextRegistrations(); clearSelectedEntityContextForLayer(id); + layerState._allFires = []; + layerState._allFiresByFrp = []; layerState._fires = []; layerState._firesByFrp = []; layerState._cellCacheByGrid.clear(); layerState._count = 0; + layerState._totalCount = 0; + layerState._rowControlsListener = null; layerState._cellCount = 0; layerState._lastUpdate = null; layerState._keyRequired = false; diff --git a/src/layers/firms/model.js b/src/layers/firms/model.js index 14b875bf8..a90066cc6 100644 --- a/src/layers/firms/model.js +++ b/src/layers/firms/model.js @@ -6,6 +6,7 @@ import { accentForSeverity, formatFrpMw, } from '../../data/firmsLabels.js'; +import { FRP_RAMP, frpRampStop } from '../../data/firmsIntensity.js'; import { VIEW_PADDING, DETECTION_COLOR_STOPS, @@ -119,27 +120,42 @@ export function createModel({ ); } - function heatColor(value, alpha) { - if (value > 0.72) return Cesium.Color.RED.withAlpha(alpha); - if (value > 0.42) return Cesium.Color.ORANGE.withAlpha(alpha); - return Cesium.Color.YELLOW.withAlpha(alpha); + /** + * The ramp stop for a fire radiative power (stepped, coldest first). + * Intensity alone drives the color: confidence feeds the card text and the + * switch, brightness is unreliable (saturates at 367 K) — see HAL-675. + * @param {number} frp - Megawatts. + * @returns {{name: string, minFrp: number, color: Cesium.Color}} + */ + + function frpColorStop(frp) { + const name = frpRampStop(frp).name; + return ( + DETECTION_COLOR_STOPS.find((stop) => stop.name === name) || + DETECTION_COLOR_STOPS[0] + ); } /** - * Pick a sprite color stop from FRP + confidence, reusing the same - * yellow → orange → red thresholds as the aggregated heat cells. + * Fill color for an aggregated heat cell: the ramp color of its strongest + * detection, so cells, markers and the slider legend read on one scale. + * @param {number} maxFrp - Strongest FRP in the cell, megawatts. + * @param {number} alpha - Fill alpha (heat-normalized by the caller). + * @returns {Cesium.Color} + */ + + function heatColor(maxFrp, alpha) { + return frpColorStop(maxFrp).color.withAlpha(alpha); + } + + /** + * Pick a sprite color stop for one detection from its FRP on the shared ramp. * @param {Object} fire - Detection record. * @returns {{name: string, color: Cesium.Color}} */ function detectionColorStop(fire) { - const heat = Math.min( - 1, - Math.sqrt(Math.max(0, fire.frp) / 150) * 0.85 + fire.confidence * 0.15, - ); - if (heat > 0.72) return DETECTION_COLOR_STOPS[0]; - if (heat > 0.42) return DETECTION_COLOR_STOPS[1]; - return DETECTION_COLOR_STOPS[2]; + return frpColorStop(fire?.frp); } /** FRP → core marker pixel size, clamped to 8..28px. */ @@ -418,7 +434,7 @@ export function createModel({ position: candidate.position, cullPosition: candidate.cullPosition || candidate.position, gapPx: 10, - accent: candidate.accent || accentForSeverity('yellow'), + accent: candidate.accent || accentForSeverity(FRP_RAMP[0].name), title: `${cell.count} ${noun}`, details: [parts.join(' · ')], selected: false, @@ -460,16 +476,14 @@ export function createModel({ } /** - * Heat-normalized cell score → severity accent, mirroring heatColor's - * red/orange/yellow thresholds. - * @param {number} normalized - 0..1 normalized heat score. + * Cell card accent: the ramp stop of the cell's strongest detection, + * mirroring heatColor. + * @param {number} maxFrp - Strongest FRP in the cell, megawatts. * @returns {string} "r, g, b" accent string. */ - function cellAccent(normalized) { - if (normalized > 0.72) return accentForSeverity('red'); - if (normalized > 0.42) return accentForSeverity('orange'); - return accentForSeverity('yellow'); + function cellAccent(maxFrp) { + return accentForSeverity(frpColorStop(maxFrp).name); } /** Coordinate line like "30.512°N 75.831°E". */ @@ -576,6 +590,7 @@ export function createModel({ cellIntersectsBounds, heatScore, heatColor, + frpColorStop, detectionColorStop, frpPixelSize, sizeBucket, diff --git a/src/layers/firms/policy.js b/src/layers/firms/policy.js index 673aab871..18ea88238 100644 --- a/src/layers/firms/policy.js +++ b/src/layers/firms/policy.js @@ -1,4 +1,5 @@ import { FIRMS_AMBIENT_COHORT_LIMIT } from '../../data/firmsLabels.js'; +import { FRP_RAMP } from '../../data/firmsIntensity.js'; import * as Cesium from 'cesium'; /** Client poll interval; the proxy's 30 min TTL is what guards upstream quota. */ @@ -88,10 +89,16 @@ export const CULL_LIFT_THRESHOLD_M = 10; export const CULL_LIFT_M = 12; -/** Color stops shared by cell heat fills and detection glow sprites. */ +/** + * Color stops shared by cell heat fills and detection glow sprites: the + * six-stop cold → hot FRP ramp (firmsIntensity.js), coldest first, as Cesium + * colors. Stop names key the glow-sprite cache and the card accents. + */ -export const DETECTION_COLOR_STOPS = [ - { name: 'red', color: Cesium.Color.RED }, - { name: 'orange', color: Cesium.Color.ORANGE }, - { name: 'yellow', color: Cesium.Color.YELLOW }, -]; +export const DETECTION_COLOR_STOPS = FRP_RAMP.map((stop) => + Object.freeze({ + name: stop.name, + minFrp: stop.minFrp, + color: Cesium.Color.fromCssColorString(stop.hex), + }), +); diff --git a/src/layers/firms/queries.js b/src/layers/firms/queries.js index 5f663cc97..220012133 100644 --- a/src/layers/firms/queries.js +++ b/src/layers/firms/queries.js @@ -1,7 +1,19 @@ import { fireDetectionKey } from '../../data/firmsLabels.js'; -import { t } from '../../i18n/index.js'; +import { + FIRMS_CONFIDENCE_MODES, + FRP_SLIDER_TICKS, + firePassesFilter, + frpForSliderPosition, + normalizeFireFilter, + rampTrackGradient, + sliderPositionForFrp, +} from '../../data/firmsIntensity.js'; +import { formatNumber, t } from '../../i18n/index.js'; import { REFRESH_INTERVAL_MS } from './policy.js'; +/** Slider resolution: whole-percent positions along the log axis. */ +const SLIDER_STEPS = 1000; + export function createQueries({ layerState, services, @@ -11,6 +23,25 @@ export function createQueries({ }) { const { id, name, icon, source } = config; + /** Whole-megawatt label for a slider value or tick ("0", "20", "1 000"). */ + function frpLabel(frp) { + return formatNumber(frp, { maximumFractionDigits: 0 }); + } + + const confidenceLabels = () => ({ + all: t('All', { context: 'fire confidence filter' }), + nominal: t('Nominal+', { context: 'fire confidence filter' }), + high: t('High', { context: 'fire confidence filter' }), + }); + + const confidenceTitles = () => ({ + all: t( + 'Every detection — many of the strongest fires are rated low confidence', + ), + nominal: t('Nominal and high confidence detections'), + high: t('High confidence detections only'), + }); + const methods = { /** Return the selected detection as a plain record for application consumers. */ getSelectedInfo() { @@ -51,6 +82,109 @@ export function createQueries({ // not this interval — is what protects the upstream FIRMS quota). updateInterval: REFRESH_INTERVAL_MS, + /** + * The intensity filter as layer options (share token `w`, owner + * `local-firms` in layerState.js). + * @returns {{minFrp: number, confidence: string}} + */ + getParams() { + return { ...layerState._filter }; + }, + + /** + * Apply a (partial) intensity filter. Invalid fields keep their current + * value. The filter runs in one place (ingestion.applyFilter) and the LOD + * rebuild re-derives markers, cells, cards and context registrations from + * the shown set; a selected fire that no longer passes is deselected. + * @param {{minFrp?: number, confidence?: string}} params + * @returns {boolean} True — the request is always accepted. + */ + setParams(params = {}) { + const next = normalizeFireFilter(params, layerState._filter); + if ( + next.minFrp === layerState._filter.minFrp && + next.confidence === layerState._filter.confidence + ) + return true; + layerState._filter = next; + if (!layerState._allFires.length) return true; + components.ingestion.applyFilter(); + if ( + layerState._selectedFire && + !firePassesFilter(layerState._selectedFire, next) + ) { + components.selection.clearFireSelection(); + } + if (layerState._enabled) components.rendering.renderCurrentLod(true); + layerState._rowControlsListener?.(); + return true; + }, + + /** + * Row controls under the Active Fires row: the log-scale minimum-FRP + * slider whose track is painted with the intensity ramp (so it doubles as + * the legend), the All / Nominal+ / High confidence switch, and the + * "N of M shown" counter. + */ + getRowControls() { + const { minFrp, confidence } = layerState._filter; + const labels = confidenceLabels(); + const titles = confidenceTitles(); + const shown = layerState._count; + const total = layerState._totalCount; + return { + slider: { + id: 'min-frp', + label: t('Min FRP'), + ariaLabel: t('Minimum fire radiative power, megawatts'), + valueText: + minFrp > 0 + ? t('{value} MW', { value: frpLabel(minFrp) }) + : t('All', { context: 'fire intensity filter' }), + min: 0, + max: SLIDER_STEPS, + step: 1, + value: Math.round(sliderPositionForFrp(minFrp) * SLIDER_STEPS), + track: rampTrackGradient(), + ticks: FRP_SLIDER_TICKS.map((frp) => ({ + at: sliderPositionForFrp(frp), + label: frpLabel(frp), + })), + params: (value) => ({ + minFrp: frpForSliderPosition(Number(value) / SLIDER_STEPS), + }), + }, + segmented: { + id: 'confidence', + ariaLabel: t('Detection confidence'), + options: FIRMS_CONFIDENCE_MODES.map((mode) => ({ + id: `confidence-${mode}`, + label: labels[mode], + title: titles[mode], + active: mode === confidence, + params: { confidence: mode }, + })), + }, + info: total + ? t('{shown} of {total} shown', { + shown: formatNumber(shown), + total: formatNumber(total), + }) + : '', + }; + }, + + /** + * Install the panel's "row controls changed" callback: a load or a filter + * change moves the counter, and nothing else repaints the row before the + * next scheduled refresh. + * @param {(() => void)|null} listener + */ + setRowControlsListener(listener) { + layerState._rowControlsListener = + typeof listener === 'function' ? listener : null; + }, + /** * Layer stats for the data panel. Degraded feed states surface through * `error` (established qa-failstate pattern: a dead feed must never look @@ -84,7 +218,9 @@ export function createQueries({ }); } return { + // Shown (filtered) detections; `total` is the loaded payload. count: layerState._count, + total: layerState._totalCount, cells: layerState._cellCount, lastUpdate: layerState._lastUpdate, loading: layerState._loading, @@ -176,8 +312,12 @@ export function createQueries({ _bindInteractionForTest(viewer, fires = []) { layerState._viewer = viewer; layerState._enabled = true; + layerState._allFires = fires; + layerState._allFiresByFrp = [...fires]; layerState._fires = fires; layerState._firesByFrp = [...fires]; + layerState._count = fires.length; + layerState._totalCount = fires.length; layerState._pickIndexById.clear(); layerState._labelLodDistance = 1e7; layerState._labelCandidates = fires.map((fire) => { diff --git a/src/layers/firms/rendering.js b/src/layers/firms/rendering.js index 8ae445f91..757fd4cf7 100644 --- a/src/layers/firms/rendering.js +++ b/src/layers/firms/rendering.js @@ -144,7 +144,7 @@ export function createRendering({ const score = components.model.heatScore(cell); const normalized = Math.min(1, Math.sqrt(score / maxScore)); const alpha = 0.16 + normalized * 0.5; - const color = components.model.heatColor(normalized, alpha); + const color = components.model.heatColor(cell.maxFrp, alpha); const centerLon = cell.lonCell + lod.gridDegrees / 2; const centerLat = cell.latCell + lod.gridDegrees / 2; const position = Cesium.Cartesian3.fromDegrees(centerLon, centerLat, 0); @@ -170,13 +170,12 @@ export function createRendering({ }); // Cells arrive heat-sorted, so candidate order doubles as label priority. - // Accent is resolved here because `normalized` only exists in this walk. layerState._labelCandidates.push({ position, cullPosition: components.model.cellCullPosition(centerLon, centerLat), fire: null, cell, - accent: components.model.cellAccent(normalized), + accent: components.model.cellAccent(cell.maxFrp), }); } diff --git a/src/layers/firms/state.js b/src/layers/firms/state.js index 48ae11db2..d84ee2026 100644 --- a/src/layers/firms/state.js +++ b/src/layers/firms/state.js @@ -1,4 +1,5 @@ import * as Cesium from 'cesium'; +import { DEFAULT_FIRE_FILTER } from '../../data/firmsIntensity.js'; export function createFirmsState({ services, config }) { const layerState = {}; @@ -39,12 +40,39 @@ export function createFirmsState({ services, config }) { layerState._error = null; + /** Every adapted detection in the current payload, before the filter. */ + + layerState._allFires = []; + + /** `_allFires` sorted by FRP descending, once per payload. */ + + layerState._allFiresByFrp = []; + + /** + * The detections that pass the intensity filter — the ONLY set rendering, + * selection, cells, the top-fires query and the analyst records ever walk. + */ + layerState._fires = []; layerState._firesByFrp = []; + /** Shown (filtered) detection count. */ + layerState._count = 0; + /** Loaded detection count before the filter ("N of M shown"). */ + + layerState._totalCount = 0; + + /** Minimum FRP (MW) and confidence mode; see firmsIntensity.js. */ + + layerState._filter = { ...DEFAULT_FIRE_FILTER }; + + /** Panel callback: repaint the row controls (counter) after a load. */ + + layerState._rowControlsListener = null; + layerState._cellCount = 0; layerState._lastUpdate = null; diff --git a/src/share/viewCodec.js b/src/share/viewCodec.js index 84ba8fbc1..18f64e0aa 100644 --- a/src/share/viewCodec.js +++ b/src/share/viewCodec.js @@ -255,6 +255,7 @@ export const SHARE_VIEW_FLAGS = freezeAll([ * written explicitly. */ export const SHARE_VIEW_LAYER_OPTION_ABSENT = Object.freeze({ + 'local-firms': Object.freeze({ minFrp: 0, confidence: 'all' }), traffic: Object.freeze({ roadMode: null }), 'weather-lightning': Object.freeze({ opacity: 'strong' }), 'weather-radar': Object.freeze({ opacity: 'strong' }), diff --git a/src/tools/queries/hazards.js b/src/tools/queries/hazards.js index 2b11c639c..e671db426 100644 --- a/src/tools/queries/hazards.js +++ b/src/tools/queries/hazards.js @@ -4,6 +4,14 @@ import { suggestView } from '../views.js'; import { defineTool, ToolError } from '../catalog.js'; import { AREA_SCHEMA, areaContains, resolveArea } from '../area.js'; import { LIMIT_SCHEMA, capRows, countNoun, isoTime } from '../results.js'; +import { + FIRMS_CONFIDENCE_MODES, + MIN_FRP_MW_MAX, + MIN_FRP_MW_MIN, + firePassesFilter, + normalizeFireFilter, +} from '../../data/firmsIntensity.js'; +import { normalizeConfidence } from '../../data/firmsAdapt.js'; function acquiredMsUtc(acqDate, acqTime) { if (typeof acqDate !== 'string' || !/^\d{4}-\d{2}-\d{2}$/.test(acqDate)) @@ -87,10 +95,30 @@ export const getActiveFires = defineTool({ title: 'Active fires', description: 'Satellite fire detections from NASA FIRMS in the last 24 hours within an ' + - 'area, most intense (fire radiative power) first.', + 'area, most intense (fire radiative power) first. Optionally limited to a ' + + 'minimum fire radiative power in megawatts and a confidence level — the ' + + "same filter as the Active Fires layer's intensity controls.", inputSchema: { type: 'object', - properties: { area: AREA_SCHEMA, limit: LIMIT_SCHEMA }, + properties: { + area: AREA_SCHEMA, + min_frp_mw: { + type: 'number', + minimum: MIN_FRP_MW_MIN, + maximum: MIN_FRP_MW_MAX, + description: + 'Hide detections weaker than this fire radiative power (MW). ' + + 'A typical detection is ~6 MW; 20 MW keeps the top tenth, 100 MW the top percent.', + }, + confidence: { + type: 'string', + enum: [...FIRMS_CONFIDENCE_MODES], + description: + '"all" (default), "nominal" (nominal and high), or "high". Many of the ' + + 'strongest fires are rated low confidence, so prefer "all" unless asked.', + }, + limit: LIMIT_SCHEMA, + }, required: ['area'], additionalProperties: false, }, @@ -103,9 +131,31 @@ export const getActiveFires = defineTool({ 'unavailable', 'Fire detections need a NASA FIRMS key configured for this server', ); + const filter = normalizeFireFilter({ + minFrp: args.min_frp_mw, + confidence: args.confidence, + }); const fires = snapshot.fires - .filter((fire) => areaContains(area, fire)) + .filter( + (fire) => + areaContains(area, fire) && + firePassesFilter( + { + frp: Number(fire.frp), + confidence: normalizeConfidence(fire.confidence), + }, + filter, + ), + ) .sort((a, b) => (b.frp ?? 0) - (a.frp ?? 0)); + const filterNotes = [ + ...(filter.minFrp > 0 ? [`≥ ${filter.minFrp} MW`] : []), + ...(filter.confidence !== 'all' + ? [ + `${filter.confidence === 'high' ? 'high' : 'nominal or high'} confidence`, + ] + : []), + ]; const result = capRows(fires, args.limit); // The proxy serves its last good snapshot when a refresh fails, and marks // satellites whose download failed. @@ -119,10 +169,16 @@ export const getActiveFires = defineTool({ ]; return { summary: - `${countNoun(fires.length, 'fire detection')} in ${area.label} in the last 24 hours` + + `${countNoun(fires.length, 'fire detection')}` + + (filterNotes.length ? ` (${filterNotes.join(', ')})` : '') + + ` in ${area.label} in the last 24 hours` + (notes.length ? ` (${notes.join('; ')}).` : '.'), data: { view: suggestView(services, { area, layers: ['local-firms'] }), + filter: { + min_frp_mw: filter.minFrp, + confidence: filter.confidence, + }, stale, missing_sources: missing, fetched_at: isoTime(snapshot.fetchedAt), diff --git a/src/tools/queries/hazards.test.mjs b/src/tools/queries/hazards.test.mjs index fa17158dd..c1d3eb43a 100644 --- a/src/tools/queries/hazards.test.mjs +++ b/src/tools/queries/hazards.test.mjs @@ -44,3 +44,39 @@ test('fire acquisition timestamps preserve valid leap days and UTC times', async assert.equal(result.data.rows[0].acquired, expected); } }); + +const WEEK = [ + { lat: 24.1, lon: 121.1, frp: 864, confidence: 'l', acqDate: '2026-10-01', acqTime: '0100' }, + { lat: 24.2, lon: 121.2, frp: 120, confidence: 'h', acqDate: '2026-10-01', acqTime: '0100' }, + { lat: 24.3, lon: 121.3, frp: 20, confidence: 'n', acqDate: '2026-10-01', acqTime: '0100' }, + { lat: 24.4, lon: 121.4, frp: 2, confidence: 'h', acqDate: '2026-10-01', acqTime: '0100' }, + { lat: 30, lon: 100, frp: 500, confidence: 'h', acqDate: '2026-10-01', acqTime: '0100' }, +]; +const TAIWAN = { area: { bbox: [120, 23, 122, 25] } }; + +test('get_active_fires shows every detection by default and reports the filter it used', async () => { + const result = await catalogWith(WEEK).call('get_active_fires', TAIWAN); + assert.deepEqual(result.data.rows.map((row) => row.frp_mw), [864, 120, 20, 2]); + assert.deepEqual(result.data.filter, { min_frp_mw: 0, confidence: 'all' }); + assert.match(result.summary, /^4 fire detections in /); +}); + +test('get_active_fires applies the same minimum-FRP and confidence filter as the layer', async () => { + const catalog = catalogWith(WEEK); + const strong = await catalog.call('get_active_fires', { ...TAIWAN, min_frp_mw: 20 }); + assert.deepEqual(strong.data.rows.map((row) => row.frp_mw), [864, 120, 20]); + assert.match(strong.summary, /^3 fire detections \(≥ 20 MW\) in /); + + const nominal = await catalog.call('get_active_fires', { ...TAIWAN, confidence: 'nominal' }); + assert.deepEqual(nominal.data.rows.map((row) => row.frp_mw), [120, 20, 2], 'the 864 MW low-confidence fire drops'); + assert.match(nominal.summary, /\(nominal or high confidence\)/); + + const both = await catalog.call('get_active_fires', { + ...TAIWAN, + min_frp_mw: 19.6, + confidence: 'high', + }); + assert.deepEqual(both.data.rows.map((row) => row.frp_mw), [120]); + assert.deepEqual(both.data.filter, { min_frp_mw: 20, confidence: 'high' }); + assert.match(both.summary, /\(≥ 20 MW, high confidence\)/); +}); diff --git a/src/ui/layerPanel.js b/src/ui/layerPanel.js index 18b71dabf..dc6566848 100644 --- a/src/ui/layerPanel.js +++ b/src/ui/layerPanel.js @@ -1,4 +1,5 @@ import { syncChipGroup } from './chipGroup.js'; +import { syncSegmentedGroup, syncSliderBlock } from './rowFilterControls.js'; import { syncRowList } from './rowList.js'; import { layerFeedState } from '../data/feedState.js'; export { layerFeedState } from '../data/feedState.js'; @@ -388,18 +389,42 @@ export class LayerPanel { const controls = document.createElement('div'); controls.className = 'data-toggle-controls'; this._bind(controls, 'click', (event) => { - const button = event.target?.closest?.('.data-toggle-chip'); + const button = + event.target?.closest?.('.data-toggle-chip') || + event.target?.closest?.('.data-toggle-segment'); if (!button || button.disabled) return; // Re-read the live descriptor rather than trusting the rendered // chip, so a stale row can never apply an inverted toggle. - const chip = this._rowControlsFor(layer.id)?.chips?.find( - (entry) => entry.id === button.dataset.chipId, - ); + const live = this._rowControlsFor(layer.id); + const chip = + live?.chips?.find( + (entry) => entry.id === button.dataset.chipId, + ) || + live?.segmented?.options?.find( + (entry) => entry.id === button.dataset.chipId, + ); if (!chip || chip.disabled || !this.isEnabled(layer.id)) return; if (typeof chip.onClick === 'function') chip.onClick(); else if (chip.params) this.setLayerParams(layer.id, chip.params, { origin: 'user' }); }); + // A row slider (e.g. the Active Fires minimum-FRP filter) maps its + // position to params through the live descriptor on every tick. + this._bind(controls, 'input', (event) => { + const input = event.target; + if (!input?.matches?.('.data-toggle-slider-input')) return; + const slider = this._rowControlsFor(layer.id)?.slider; + if ( + !slider || + slider.id !== input.dataset.sliderId || + typeof slider.params !== 'function' || + !this.isEnabled(layer.id) + ) + return; + this.setLayerParams(layer.id, slider.params(input.value), { + origin: 'user', + }); + }); row.appendChild(controls); // An ordered list below the chips, for a layer whose row carries a // sequence (turn-by-turn directions). Its own delegated listener, its @@ -521,13 +546,30 @@ export class LayerPanel { const chips = controls?.chips || []; const legend = controls?.legend || []; const infoText = controls?.info; + const slider = controls?.slider || null; + const segmented = controls?.segmented || null; this._syncRowList(listContainer, controls?.list || null); - container.hidden = chips.length === 0 && legend.length === 0 && !infoText; + container.hidden = + chips.length === 0 && + legend.length === 0 && + !infoText && + !slider && + !segmented; const info = container._rowControlsInfo || null; const firstLegend = container.querySelector('.data-toggle-legend-item'); - syncChipGroup(container, chips, { before: firstLegend || info }); + // Order: chips, switch, slider, legend, info. + syncSegmentedGroup(container, segmented, { before: firstLegend || info }); + const switchNode = container.querySelector('.data-toggle-segmented'); + syncSliderBlock(container, slider, { + before: switchNode?.nextSibling || firstLegend || info, + }); + const firstControl = + switchNode || container.querySelector('.data-toggle-slider'); + syncChipGroup(container, chips, { + before: firstControl || firstLegend || info, + }); const legendSignature = JSON.stringify( legend.map(({ label, color, count, blurb }) => [ diff --git a/src/ui/rowFilterControls.js b/src/ui/rowFilterControls.js new file mode 100644 index 000000000..b0299d24b --- /dev/null +++ b/src/ui/rowFilterControls.js @@ -0,0 +1,169 @@ +import { createSlider } from './slider.js'; +import { syncSegmentedControl } from './segmentedControl.js'; + +const set = (node, key, value) => { + if (node[key] !== value) node[key] = value; +}; +const attribute = (node, key, value) => { + if (node.getAttribute(key) !== value) node.setAttribute(key, value); +}; +// Custom properties need setProperty; test fixtures carry a plain style bag. +const styleVar = (node, name, value) => { + if (typeof node.style?.setProperty === 'function') { + if (value) node.style.setProperty(name, value); + else node.style.removeProperty(name); + } else if (node.style) node.style[name] = value || ''; +}; + +/** + * Reconcile a layer row's quantitative slider block in place (keyed by the + * descriptor id), so a refresh triggered by the slider's own input never + * replaces the node under the pointer. The caller owns dispatch: it listens + * for `input` on the row container and reads `.data-toggle-slider-input`. + * + * Descriptor: `{ id, label, ariaLabel, valueText, min, max, step, value, + * track, ticks: [{ at, label }] }` — `track` is a CSS background painting the + * runnable track (a color ramp doubles as the legend), `at` is 0..1. + * @param {HTMLElement} container The row's `.data-toggle-controls` node. + * @param {?object} slider Descriptor, or null to remove the block. + * @param {{before?: ?Node}} [options] Insertion anchor for a new block. + * @returns {?HTMLElement} The block, when rendered. + */ +export function syncSliderBlock(container, slider, { before = null } = {}) { + if (!container) return null; + const document = container.ownerDocument ?? globalThis.document; + const existing = [...container.children].find( + (node) => node.dataset?.sliderId, + ); + if (!slider) { + existing?.remove(); + return null; + } + let block = existing; + if (!block || block.dataset.sliderId !== slider.id) { + existing?.remove(); + block = document.createElement('div'); + block.className = 'data-toggle-slider'; + block.dataset.sliderId = slider.id; + const head = document.createElement('div'); + head.className = 'data-toggle-slider-head'; + const label = document.createElement('span'); + label.className = 'data-toggle-slider-label'; + const value = document.createElement('span'); + value.className = 'data-toggle-slider-value gev-slider-value'; + head.append(label, value); + const input = createSlider({ + documentRef: document, + className: 'data-toggle-slider-input', + min: slider.min, + max: slider.max, + step: slider.step, + value: slider.value, + }); + input.dataset.sliderId = slider.id; + const ticks = document.createElement('div'); + ticks.className = 'data-toggle-slider-ticks'; + block.append(head, input, ticks); + block._parts = { label, value, input, ticks }; + container.insertBefore(block, before); + } + const { label, value, input, ticks } = block._parts; + set(label, 'textContent', slider.label ?? ''); + set(value, 'textContent', slider.valueText ?? ''); + set(input, 'min', String(slider.min)); + set(input, 'max', String(slider.max)); + set(input, 'step', String(slider.step ?? 1)); + // Never move the thumb under an active drag: the layer's rounded value can + // sit a hair off the pointer, and a refresh runs on every input tick. + if (document.activeElement !== input) + set(input, 'value', String(slider.value)); + if (slider.ariaLabel) attribute(input, 'aria-label', slider.ariaLabel); + if (slider.valueText) attribute(input, 'aria-valuetext', slider.valueText); + const track = slider.track || ''; + if ((block.dataset.track || '') !== track) { + styleVar(input, '--slider-track', track); + block.dataset.track = track; + } + const signature = JSON.stringify(slider.ticks || []); + if (block.dataset.ticks !== signature) { + for (const node of [...ticks.children]) node.remove(); + for (const tick of slider.ticks || []) { + const node = document.createElement('span'); + node.className = 'data-toggle-slider-tick'; + styleVar(node, '--at', `${(tick.at * 100).toFixed(2)}%`); + node.textContent = tick.label; + ticks.appendChild(node); + } + block.dataset.ticks = signature; + } + return block; +} + +/** + * Reconcile a one-of-many switch (`.gev-segmented` radiogroup) in place. + * Buttons carry `data-chip-id` like chips so the row's delegated click handler + * can resolve them; the caller matches `.data-toggle-segment`. + * + * Descriptor: `{ id, ariaLabel, options: [{ id, label, title, active, + * disabled, params }] }`. + * @param {HTMLElement} container The row's `.data-toggle-controls` node. + * @param {?object} segmented Descriptor, or null to remove the group. + * @param {{before?: ?Node}} [options] Insertion anchor for a new group. + * @returns {?HTMLElement} The group, when rendered. + */ +export function syncSegmentedGroup( + container, + segmented, + { before = null } = {}, +) { + if (!container) return null; + const document = container.ownerDocument ?? globalThis.document; + const existing = [...container.children].find( + (node) => node.dataset?.segmentedId, + ); + if (!segmented) { + existing?.remove(); + return null; + } + let group = existing; + if (!group || group.dataset.segmentedId !== segmented.id) { + existing?.remove(); + group = document.createElement('div'); + group.className = 'gev-segmented data-toggle-segmented'; + group.dataset.segmentedId = segmented.id; + group.setAttribute('role', 'radiogroup'); + container.insertBefore(group, before); + } + if (segmented.ariaLabel) attribute(group, 'aria-label', segmented.ariaLabel); + const stale = new Map(); + for (const node of [...group.children]) + if (node.dataset?.chipId) stale.set(node.dataset.chipId, node); + const ordered = []; + for (const option of segmented.options || []) { + let button = stale.get(option.id); + stale.delete(option.id); + if (!button) { + button = document.createElement('button'); + button.type = 'button'; + button.className = 'gev-segment data-toggle-segment'; + button.dataset.chipId = option.id; + button.setAttribute('role', 'radio'); + } + set(button, 'textContent', option.label); + set(button, 'title', option.title || ''); + set(button, 'disabled', Boolean(option.disabled)); + ordered.push(button); + } + for (const node of stale.values()) node.remove(); + ordered.forEach((node, index) => { + if (group.children[index] !== node) + group.insertBefore(node, group.children[index] || null); + }); + const active = (segmented.options || []).find((option) => option.active); + syncSegmentedControl(ordered, active?.id, { + valueOf: (button) => button.dataset.chipId, + activeClass: 'active', + rovingFocus: true, + }); + return group; +} diff --git a/src/ui/rowFilterControls.test.mjs b/src/ui/rowFilterControls.test.mjs new file mode 100644 index 000000000..9cc8fb1d4 --- /dev/null +++ b/src/ui/rowFilterControls.test.mjs @@ -0,0 +1,207 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { syncSegmentedGroup, syncSliderBlock } from './rowFilterControls.js'; +import { railFixture } from './railTestFixture.mjs'; + +const sliderDescriptor = (overrides = {}) => ({ + id: 'min-frp', + label: 'MIN FRP', + ariaLabel: 'Minimum fire radiative power', + valueText: 'All', + min: 0, + max: 1000, + step: 1, + value: 0, + track: 'linear-gradient(90deg, #4a5bd4 0% 23%, #d1382a 82% 100%)', + ticks: [ + { at: 0, label: '0' }, + { at: 0.233, label: '5' }, + { at: 0.826, label: '300' }, + ], + params: (value) => ({ minFrp: Number(value) }), + ...overrides, +}); + +const segmentedDescriptor = (active = 'all') => ({ + id: 'confidence', + ariaLabel: 'Detection confidence', + options: ['all', 'nominal', 'high'].map((mode) => ({ + id: `confidence-${mode}`, + label: mode.toUpperCase(), + title: `${mode} title`, + active: mode === active, + params: { confidence: mode }, + })), +}); + +test('the slider block renders label, value, painted track and ticks, and is reconciled in place', () => { + const f = railFixture(); + const container = f.document.createElement('div'); + const block = syncSliderBlock(container, sliderDescriptor()); + assert.equal(block.className, 'data-toggle-slider'); + assert.equal(block.dataset.sliderId, 'min-frp'); + const input = block.querySelector('.data-toggle-slider-input'); + assert.equal(input.type, 'range'); + assert.equal(input.max, '1000'); + assert.equal(input.value, '0'); + assert.equal( + input.getAttribute('aria-label'), + 'Minimum fire radiative power', + ); + assert.equal(input.getAttribute('aria-valuetext'), 'All'); + assert.match(input.style['--slider-track'], /^linear-gradient/); + assert.equal( + block.querySelector('.data-toggle-slider-label').textContent, + 'MIN FRP', + ); + assert.equal( + block.querySelector('.data-toggle-slider-value').textContent, + 'All', + ); + const ticks = block.querySelector('.data-toggle-slider-ticks'); + assert.deepEqual( + ticks.children.map((node) => [node.textContent, node.style['--at']]), + [ + ['0', '0.00%'], + ['5', '23.30%'], + ['300', '82.60%'], + ], + ); + + const again = syncSliderBlock( + container, + sliderDescriptor({ value: 434, valueText: '20 MW' }), + ); + assert.equal(again, block, 'same node — a refresh never drops the thumb'); + assert.equal(input.value, '434'); + assert.equal( + block.querySelector('.data-toggle-slider-value').textContent, + '20 MW', + ); + assert.equal(container.children.length, 1); + + // A thumb under the pointer is never moved by a refresh. + input.focus(); + syncSliderBlock(container, sliderDescriptor({ value: 0 })); + assert.equal(input.value, '434'); + + syncSliderBlock(container, null); + assert.equal(container.children.length, 0); +}); + +test('the segmented group is a radiogroup whose active option reads checked, reconciled by id', () => { + const f = railFixture(); + const container = f.document.createElement('div'); + const group = syncSegmentedGroup(container, segmentedDescriptor('all')); + assert.equal(group.getAttribute('role'), 'radiogroup'); + assert.equal(group.getAttribute('aria-label'), 'Detection confidence'); + assert.ok(group.className.split(' ').includes('gev-segmented')); + const buttons = group.children; + assert.deepEqual( + buttons.map((b) => [ + b.dataset.chipId, + b.textContent, + b.getAttribute('aria-checked'), + b.getAttribute('tabindex'), + ]), + [ + ['confidence-all', 'ALL', 'true', '0'], + ['confidence-nominal', 'NOMINAL', 'false', '-1'], + ['confidence-high', 'HIGH', 'false', '-1'], + ], + ); + assert.equal(buttons[0].getAttribute('role'), 'radio'); + assert.equal(buttons[1].title, 'nominal title'); + + syncSegmentedGroup(container, segmentedDescriptor('high')); + assert.equal(group.children[0], buttons[0], 'buttons are reused'); + assert.equal(buttons[0].getAttribute('aria-checked'), 'false'); + assert.equal(buttons[2].getAttribute('aria-checked'), 'true'); + assert.ok(buttons[2].classList.contains('active')); + assert.equal(container.children.length, 1); + + syncSegmentedGroup(container, null); + assert.equal(container.children.length, 0); +}); + +test('a layer row dispatches slider input and switch clicks as user params', async () => { + const { LayerPanel } = await import('./layerPanel.js'); + const f = railFixture(); + const previousDocument = globalThis.document; + globalThis.document = f.document; + const calls = []; + let controls = { + slider: sliderDescriptor(), + segmented: segmentedDescriptor('all'), + info: '12 of 40 shown', + }; + const panel = new LayerPanel({ + getLayers: () => [ + { + id: 'local-firms', + name: 'Active Fires', + icon: '▲', + source: 'NASA FIRMS', + enabled: true, + showInTogglePanel: true, + stats: {}, + }, + ], + isEnabled: () => true, + setEnabled: () => {}, + setLayerParams: (...args) => calls.push(args), + hasRowControls: () => true, + subscribeRowControls: () => {}, + getRowControls: () => controls, + }); + try { + panel.mount(f.container); + const row = f.find((n) => n.dataset.layerId === 'local-firms'); + const container = row.querySelector('.data-toggle-controls'); + assert.equal(container.hidden, false); + assert.equal( + row.querySelector('.data-toggle-controls-info').textContent, + '12 of 40 shown', + ); + // Order: switch, slider, info. + assert.deepEqual( + container.children.map((node) => node.className.split(' ')[0]), + ['gev-segmented', 'data-toggle-slider', 'data-toggle-controls-info'], + ); + + const input = row.querySelector('.data-toggle-slider-input'); + input.value = '434'; + const inputEvent = new Event('input'); + Object.defineProperty(inputEvent, 'target', { value: input }); + container.dispatchEvent(inputEvent); + assert.deepEqual(calls.at(-1), [ + 'local-firms', + { minFrp: 434 }, + { origin: 'user' }, + ]); + + const high = row + .querySelector('.data-toggle-segmented') + .children.find((b) => b.dataset.chipId === 'confidence-high'); + high.click(); + assert.deepEqual(calls.at(-1), [ + 'local-firms', + { confidence: 'high' }, + { origin: 'user' }, + ]); + + // A stale descriptor never applies: the live one decides. + controls = { ...controls, segmented: segmentedDescriptor('high') }; + // The fixture DOM has no attribute selectors, so sync the row directly. + panel._syncRowControls(container, { id: 'local-firms', enabled: true }); + assert.equal(high.getAttribute('aria-checked'), 'true'); + assert.equal( + row.querySelector('.data-toggle-slider-input'), + input, + 'the slider survives a refresh', + ); + } finally { + panel.destroy(); + globalThis.document = previousDocument; + } +}); diff --git a/src/ui/styles/radio.css b/src/ui/styles/radio.css index 483321992..5047ae6f4 100644 --- a/src/ui/styles/radio.css +++ b/src/ui/styles/radio.css @@ -1035,6 +1035,108 @@ html:not([lang='en']) .data-row-list-text { padding-top: 3px; overflow-wrap: anywhere; } + +/* Optional per-layer one-of-many switch (Active Fires confidence). The shared + .gev-segmented surface, sized to the chip row. */ +.data-toggle-controls .data-toggle-segmented { + flex-basis: 100%; + max-width: 220px; +} + +.data-toggle-controls .data-toggle-segment { + min-height: 20px; + padding: 3px 6px; + font-size: 8px; + letter-spacing: 0.8px; +} + +/* Optional per-layer quantitative slider (Active Fires minimum FRP). The + track is painted by the layer (`--slider-track`, a stepped color ramp), so + the slider doubles as the marker legend; ticks sit under their thresholds. */ +.data-toggle-slider { + flex-basis: 100%; + display: grid; + gap: 2px; + padding-top: 2px; +} + +.data-toggle-slider-head { + display: flex; + justify-content: space-between; + align-items: baseline; + gap: 8px; + font-family: var(--font-mono); + font-size: 8px; + letter-spacing: 1.2px; + color: var(--text-dim); + text-transform: uppercase; +} + +.data-toggle-slider-value { + min-width: 0; + font-size: 9px; + letter-spacing: 0.5px; + text-transform: none; +} + +.data-toggle-slider-input { + height: 14px; +} + +.data-toggle-slider-input::-webkit-slider-runnable-track { + height: 5px; + border-radius: 3px; + background: var(--slider-track, rgba(255, 255, 255, 0.08)); + box-shadow: inset 0 0 0 1px rgba(0, 0, 0, 0.35); +} + +.data-toggle-slider-input::-moz-range-track, +.data-toggle-slider-input::-moz-range-progress { + height: 5px; + border-radius: 3px; + background: var(--slider-track, rgba(255, 255, 255, 0.08)); +} + +.data-toggle-slider-input::-webkit-slider-thumb { + width: 11px; + height: 11px; + margin-top: -3px; + background: var(--text-primary); + box-shadow: + 0 0 0 1px rgba(0, 0, 0, 0.6), + 0 0 6px rgba(255, 255, 255, 0.35); +} + +.data-toggle-slider-input::-moz-range-thumb { + width: 11px; + height: 11px; + background: var(--text-primary); + box-shadow: + 0 0 0 1px rgba(0, 0, 0, 0.6), + 0 0 6px rgba(255, 255, 255, 0.35); +} + +.data-toggle-slider-ticks { + position: relative; + height: 10px; + margin: 0 5px; + font-family: var(--font-mono); + font-size: 7px; + letter-spacing: 0.4px; + color: var(--text-dim); +} + +.data-toggle-slider-tick { + position: absolute; + top: 0; + left: var(--at); + transform: translateX(-50%); + white-space: nowrap; +} + +.data-toggle-slider-tick:first-child { + transform: none; +} /* ── Local RTL-SDR card (FM + Local ADS-B receiver) ── */ .sdr-radio-card { display: grid; From 22a87500c6b3296da1003a4101ca0ed80ba1f992 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Tue, 27 Oct 2026 18:30:58 +0400 Subject: [PATCH 09/83] docs(fires): current-state note and changelog fragment for the intensity filter (#1426) Describe the six-stop FRP ramp, the slider / confidence switch / counter, the single filter point, the share-token options and the tool arguments in docs/CURRENT-STATE.md, and add changelog.d/1426-fire-intensity-filter.md as CONTRIBUTING asks for runtime changes. Co-Authored-By: Claude Fable 5.1 --- changelog.d/1426-fire-intensity-filter.md | 8 ++++++++ docs/CURRENT-STATE.md | 24 +++++++++++++++++++++++ 2 files changed, 32 insertions(+) create mode 100644 changelog.d/1426-fire-intensity-filter.md diff --git a/changelog.d/1426-fire-intensity-filter.md b/changelog.d/1426-fire-intensity-filter.md new file mode 100644 index 000000000..fad9051e9 --- /dev/null +++ b/changelog.d/1426-fire-intensity-filter.md @@ -0,0 +1,8 @@ +- Color every Active Fires detection by fire radiative power on a six-stop + cold → hot ramp (blue · teal · lime · amber · orange · red at 0 · 5 · 20 · + 50 · 100 · 300 MW), and add a log-scale minimum-FRP slider whose track is the + ramp, an All / Nominal+ / High confidence switch and an "N of M shown" + counter to the layer row. The filter applies once before rendering, so + markers, cards, heat cells, context and the top-fires query agree; share + links keep it (`w` token: `f`, `c`), and `get_active_fires` takes matching + `min_frp_mw` and `confidence` arguments. Default 0 MW / All (gekh, #1426). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 0cb46940c..61699bda9 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2054,6 +2054,30 @@ Generated style sliders use the same name as their visible parameter label. The first-run suppression checkbox keeps its native wrapping label, so its accessible name remains "Don't show this again". Control behavior is unchanged. +## Active Fires intensity filter and color ramp + +Every FIRMS detection is colored by fire radiative power on a six-stop cold → +hot ramp: blue from 0 MW, teal from 5, lime from 20, amber from 50, orange from +100 and red from 300 MW (`FRP_RAMP` in `src/data/firmsIntensity.js`). Heat +cells, glow sprites and card accents read the same ramp; confidence and +brightness do not affect the color. + +The Active Fires row carries a minimum-FRP slider on a log scale whose track is +painted with the ramp and ticked at the same six values, an All / Nominal+ / +High confidence switch, and an "N of M shown" counter. The default is 0 MW and +All, because many of the strongest detections are rated low confidence. The +filter runs once in `ingestion.applyFilter()` before rendering, so markers, +cards, heat cells, the shared context store, the top-fires query and the +analyst records all see one filtered set; a selected fire that no longer passes +is deselected. The layer state keeps both options under the `w` (`local-firms`) +share token as `f` (whole megawatts, 0–10,000) and `c` (`a`, `n`, `h`), +including the short share codec; a link without them restores the defaults. + +The `get_active_fires` tool accepts its own `min_frp_mw` and `confidence` +(`all`, `nominal`, `high`) arguments, independent of the panel's setting, and +names the applied criteria in its notes. Stale-snapshot and missing-source +reporting are unchanged. + ## FIRMS source status The FIRMS CSV parser rejects impossible acquisition calendar dates before the proxy filters records to the trailing 24 hours, while retaining valid UTC acquisition times. From 1dad6b658a77b80465204cd10c5247cb4c2c084c Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Tue, 27 Oct 2026 18:50:03 +0400 Subject: [PATCH 10/83] feat(fires): caption the confidence switch and name the slider "Fire intensity (FRP)" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The switch had no visible word saying what ALL / NOMINAL+ / HIGH refer to, and "Min FRP" assumed the reader knows the acronym. The segmented row descriptor gains an optional visible `label` (a caption above the group); the fires row uses "Confidence" and "Fire intensity (FRP)", and the slider value reads "≥ N MW" so the threshold is still clear. Locales updated. Co-Authored-By: Claude Fable 5.1 --- docs/CURRENT-STATE.md | 7 ++++--- src/i18n/locales/es.json | 5 +++-- src/i18n/locales/ru.json | 5 +++-- src/i18n/locales/zh-CN.json | 5 +++-- src/i18n/same-as-english.json | 4 ++-- src/layers/firms/queries.js | 5 +++-- src/ui/rowFilterControls.js | 24 ++++++++++++++++++++++-- src/ui/rowFilterControls.test.mjs | 23 ++++++++++++++++++++--- src/ui/styles/radio.css | 10 ++++++++++ 9 files changed, 70 insertions(+), 18 deletions(-) diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 61699bda9..2132a910f 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2062,9 +2062,10 @@ hot ramp: blue from 0 MW, teal from 5, lime from 20, amber from 50, orange from cells, glow sprites and card accents read the same ramp; confidence and brightness do not affect the color. -The Active Fires row carries a minimum-FRP slider on a log scale whose track is -painted with the ramp and ticked at the same six values, an All / Nominal+ / -High confidence switch, and an "N of M shown" counter. The default is 0 MW and +The Active Fires row carries a "Fire intensity (FRP)" slider on a log scale +whose track is painted with the ramp and ticked at the same six values (its +value reads "All" or "≥ N MW"), a "Confidence" switch with All / Nominal+ / +High, and an "N of M shown" counter. The default is 0 MW and All, because many of the strongest detections are rated low confidence. The filter runs once in `ingestion.applyFilter()` before rendering, so markers, cards, heat cells, the shared context store, the top-fires query and the diff --git a/src/i18n/locales/es.json b/src/i18n/locales/es.json index ffee7fb87..babea8bfb 100644 --- a/src/i18n/locales/es.json +++ b/src/i18n/locales/es.json @@ -3008,9 +3008,10 @@ "Every detection — many of the strongest fires are rated low confidence": "Todas las detecciones — muchos de los incendios más intensos tienen confianza baja", "Nominal and high confidence detections": "Detecciones de confianza nominal y alta", "High confidence detections only": "Solo detecciones de confianza alta", - "Min FRP": "FRP mín.", + "Fire intensity (FRP)": "Intensidad (FRP)", + "≥ {value} MW": "≥ {value} MW", + "Confidence": "Confianza", "Minimum fire radiative power, megawatts": "Potencia radiativa mínima del fuego, en megavatios", - "{value} MW": "{value} MW", "fire intensity filter|All": "Todo", "Detection confidence": "Confianza de la detección", "{shown} of {total} shown": "{shown} de {total} mostrados" diff --git a/src/i18n/locales/ru.json b/src/i18n/locales/ru.json index 1cd88757d..6289ad38c 100644 --- a/src/i18n/locales/ru.json +++ b/src/i18n/locales/ru.json @@ -3077,9 +3077,10 @@ "Every detection — many of the strongest fires are rated low confidence": "Все детекции — у многих сильнейших пожаров достоверность низкая", "Nominal and high confidence detections": "Детекции средней и высокой достоверности", "High confidence detections only": "Только детекции высокой достоверности", - "Min FRP": "Мин. FRP", + "Fire intensity (FRP)": "Интенсивность (FRP)", + "≥ {value} MW": "≥ {value} МВт", + "Confidence": "Достоверность", "Minimum fire radiative power, megawatts": "Минимальная мощность излучения пожара, мегаватты", - "{value} MW": "{value} МВт", "fire intensity filter|All": "Все", "Detection confidence": "Достоверность детекции", "{shown} of {total} shown": "показано {shown} из {total}" diff --git a/src/i18n/locales/zh-CN.json b/src/i18n/locales/zh-CN.json index 3044c45f8..110f7a77c 100644 --- a/src/i18n/locales/zh-CN.json +++ b/src/i18n/locales/zh-CN.json @@ -2870,9 +2870,10 @@ "Every detection — many of the strongest fires are rated low confidence": "所有检测——许多最强的火点置信度为低", "Nominal and high confidence detections": "标称和高置信度的检测", "High confidence detections only": "仅高置信度的检测", - "Min FRP": "最小 FRP", + "Fire intensity (FRP)": "火势强度 (FRP)", + "≥ {value} MW": "≥ {value} MW", + "Confidence": "置信度", "Minimum fire radiative power, megawatts": "最小火辐射功率（兆瓦）", - "{value} MW": "{value} MW", "fire intensity filter|All": "全部", "Detection confidence": "检测置信度", "{shown} of {total} shown": "显示 {shown} / {total}" diff --git a/src/i18n/same-as-english.json b/src/i18n/same-as-english.json index 5f1b14f8a..5d0313965 100644 --- a/src/i18n/same-as-english.json +++ b/src/i18n/same-as-english.json @@ -121,7 +121,7 @@ "{value} KM", "{value} KM/H", "{value} KM/S", - "{value} MW", + "≥ {value} MW", "ΔE", "ΔN", "▲ {frp} MW" @@ -197,7 +197,7 @@ "{type} SPD: {speed} HDG: {heading}", "{value} KM", "{value} KM/S", - "{value} MW", + "≥ {value} MW", "ΔE", "ΔN", "▲ {frp} MW" diff --git a/src/layers/firms/queries.js b/src/layers/firms/queries.js index 220012133..44ce8573d 100644 --- a/src/layers/firms/queries.js +++ b/src/layers/firms/queries.js @@ -135,11 +135,11 @@ export function createQueries({ return { slider: { id: 'min-frp', - label: t('Min FRP'), + label: t('Fire intensity (FRP)'), ariaLabel: t('Minimum fire radiative power, megawatts'), valueText: minFrp > 0 - ? t('{value} MW', { value: frpLabel(minFrp) }) + ? t('≥ {value} MW', { value: frpLabel(minFrp) }) : t('All', { context: 'fire intensity filter' }), min: 0, max: SLIDER_STEPS, @@ -156,6 +156,7 @@ export function createQueries({ }, segmented: { id: 'confidence', + label: t('Confidence'), ariaLabel: t('Detection confidence'), options: FIRMS_CONFIDENCE_MODES.map((mode) => ({ id: `confidence-${mode}`, diff --git a/src/ui/rowFilterControls.js b/src/ui/rowFilterControls.js index b0299d24b..87a3e0bc6 100644 --- a/src/ui/rowFilterControls.js +++ b/src/ui/rowFilterControls.js @@ -104,8 +104,9 @@ export function syncSliderBlock(container, slider, { before = null } = {}) { * Buttons carry `data-chip-id` like chips so the row's delegated click handler * can resolve them; the caller matches `.data-toggle-segment`. * - * Descriptor: `{ id, ariaLabel, options: [{ id, label, title, active, - * disabled, params }] }`. + * Descriptor: `{ id, label, ariaLabel, options: [{ id, label, title, active, + * disabled, params }] }` — `label` is an optional visible caption above the + * buttons (the group itself is a flex item, so the caption is a sibling). * @param {HTMLElement} container The row's `.data-toggle-controls` node. * @param {?object} segmented Descriptor, or null to remove the group. * @param {{before?: ?Node}} [options] Insertion anchor for a new group. @@ -121,19 +122,38 @@ export function syncSegmentedGroup( const existing = [...container.children].find( (node) => node.dataset?.segmentedId, ); + const existingCaption = [...container.children].find( + (node) => node.dataset?.segmentedCaptionFor, + ); if (!segmented) { existing?.remove(); + existingCaption?.remove(); return null; } let group = existing; if (!group || group.dataset.segmentedId !== segmented.id) { existing?.remove(); + existingCaption?.remove(); group = document.createElement('div'); group.className = 'gev-segmented data-toggle-segmented'; group.dataset.segmentedId = segmented.id; group.setAttribute('role', 'radiogroup'); container.insertBefore(group, before); } + let caption = [...container.children].find( + (node) => node.dataset?.segmentedCaptionFor === segmented.id, + ); + if (segmented.label) { + if (!caption) { + caption = document.createElement('span'); + caption.className = 'data-toggle-segmented-label'; + caption.dataset.segmentedCaptionFor = segmented.id; + container.insertBefore(caption, group); + } + set(caption, 'textContent', segmented.label); + } else { + caption?.remove(); + } if (segmented.ariaLabel) attribute(group, 'aria-label', segmented.ariaLabel); const stale = new Map(); for (const node of [...group.children]) diff --git a/src/ui/rowFilterControls.test.mjs b/src/ui/rowFilterControls.test.mjs index 9cc8fb1d4..ac5d2c1d0 100644 --- a/src/ui/rowFilterControls.test.mjs +++ b/src/ui/rowFilterControls.test.mjs @@ -24,6 +24,7 @@ const sliderDescriptor = (overrides = {}) => ({ const segmentedDescriptor = (active = 'all') => ({ id: 'confidence', + label: 'Confidence', ariaLabel: 'Detection confidence', options: ['all', 'nominal', 'high'].map((mode) => ({ id: `confidence-${mode}`, @@ -96,6 +97,9 @@ test('the segmented group is a radiogroup whose active option reads checked, rec assert.equal(group.getAttribute('role'), 'radiogroup'); assert.equal(group.getAttribute('aria-label'), 'Detection confidence'); assert.ok(group.className.split(' ').includes('gev-segmented')); + const caption = container.children[container.children.indexOf(group) - 1]; + assert.equal(caption.className, 'data-toggle-segmented-label'); + assert.equal(caption.textContent, 'Confidence'); const buttons = group.children; assert.deepEqual( buttons.map((b) => [ @@ -118,10 +122,18 @@ test('the segmented group is a radiogroup whose active option reads checked, rec assert.equal(buttons[0].getAttribute('aria-checked'), 'false'); assert.equal(buttons[2].getAttribute('aria-checked'), 'true'); assert.ok(buttons[2].classList.contains('active')); + assert.equal(container.children.length, 2, 'caption and group'); + + syncSegmentedGroup(container, { ...segmentedDescriptor('high'), label: '' }); + assert.equal( + container.querySelector('.data-toggle-segmented-label'), + null, + 'an empty label removes the caption', + ); assert.equal(container.children.length, 1); syncSegmentedGroup(container, null); - assert.equal(container.children.length, 0); + assert.equal(container.children.length, 0, 'null removes group and caption'); }); test('a layer row dispatches slider input and switch clicks as user params', async () => { @@ -163,10 +175,15 @@ test('a layer row dispatches slider input and switch clicks as user params', asy row.querySelector('.data-toggle-controls-info').textContent, '12 of 40 shown', ); - // Order: switch, slider, info. + // Order: switch caption, switch, slider, info. assert.deepEqual( container.children.map((node) => node.className.split(' ')[0]), - ['gev-segmented', 'data-toggle-slider', 'data-toggle-controls-info'], + [ + 'data-toggle-segmented-label', + 'gev-segmented', + 'data-toggle-slider', + 'data-toggle-controls-info', + ], ); const input = row.querySelector('.data-toggle-slider-input'); diff --git a/src/ui/styles/radio.css b/src/ui/styles/radio.css index 5047ae6f4..e005c5288 100644 --- a/src/ui/styles/radio.css +++ b/src/ui/styles/radio.css @@ -1043,6 +1043,16 @@ html:not([lang='en']) .data-row-list-text { max-width: 220px; } +.data-toggle-segmented-label { + flex-basis: 100%; + padding-top: 2px; + font-family: var(--font-mono); + font-size: 8px; + letter-spacing: 1.2px; + color: var(--text-dim); + text-transform: uppercase; +} + .data-toggle-controls .data-toggle-segment { min-height: 20px; padding: 3px 6px; From 887b261195fe4a425e9cd8dfdd5027172b7c5342 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Tue, 27 Oct 2026 19:19:40 +0400 Subject: [PATCH 11/83] fix(fires): paint aggregated heat cells warm by density, not by strongest FRP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Coloring each far-zoom cell with the ramp stop of its strongest detection turned a burning region into a blue / teal / lime mosaic: neighbouring cells flipped hue on one outlier and the cold stops did not read as fire. Cells say where it burns, so they go back to the heat-normalized score (density, night share, max FRP) on the warm end of the ramp — lime, amber, red — with the cell card accent matching. Single detections keep the full six-stop FRP ramp. Co-Authored-By: Claude Fable 5.1 --- changelog.d/1426-fire-intensity-filter.md | 3 +- docs/CURRENT-STATE.md | 9 ++++-- src/layers/firms/model.js | 37 +++++++++++++++++------ src/layers/firms/rendering.js | 5 +-- 4 files changed, 39 insertions(+), 15 deletions(-) diff --git a/changelog.d/1426-fire-intensity-filter.md b/changelog.d/1426-fire-intensity-filter.md index fad9051e9..09e3c3a7d 100644 --- a/changelog.d/1426-fire-intensity-filter.md +++ b/changelog.d/1426-fire-intensity-filter.md @@ -2,7 +2,8 @@ cold → hot ramp (blue · teal · lime · amber · orange · red at 0 · 5 · 20 · 50 · 100 · 300 MW), and add a log-scale minimum-FRP slider whose track is the ramp, an All / Nominal+ / High confidence switch and an "N of M shown" - counter to the layer row. The filter applies once before rendering, so + counter to the layer row. Far-zoom heat cells keep a warm density fill + (lime · amber · red). The filter applies once before rendering, so markers, cards, heat cells, context and the top-fires query agree; share links keep it (`w` token: `f`, `c`), and `get_active_fires` takes matching `min_frp_mw` and `confidence` arguments. Default 0 MW / All (gekh, #1426). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 2132a910f..7e29d8675 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2058,9 +2058,12 @@ accessible name remains "Don't show this again". Control behavior is unchanged. Every FIRMS detection is colored by fire radiative power on a six-stop cold → hot ramp: blue from 0 MW, teal from 5, lime from 20, amber from 50, orange from -100 and red from 300 MW (`FRP_RAMP` in `src/data/firmsIntensity.js`). Heat -cells, glow sprites and card accents read the same ramp; confidence and -brightness do not affect the color. +100 and red from 300 MW (`FRP_RAMP` in `src/data/firmsIntensity.js`). Glow +sprites and fire-card accents read that ramp; confidence and brightness do +not affect the color. Aggregated heat cells (far zoom) instead take the warm +end of the ramp — lime, amber or red — from their heat-normalized score +(density, night share and strongest FRP), so a region reads as one field +rather than a per-cell mosaic; the cell card's accent matches its fill. The Active Fires row carries a "Fire intensity (FRP)" slider on a log scale whose track is painted with the ramp and ticked at the same six values (its diff --git a/src/layers/firms/model.js b/src/layers/firms/model.js index a90066cc6..83f573c98 100644 --- a/src/layers/firms/model.js +++ b/src/layers/firms/model.js @@ -137,15 +137,33 @@ export function createModel({ } /** - * Fill color for an aggregated heat cell: the ramp color of its strongest - * detection, so cells, markers and the slider legend read on one scale. - * @param {number} maxFrp - Strongest FRP in the cell, megawatts. + * Severity stop for an aggregated heat cell from its heat-normalized score + * (density, night share and strongest FRP together). Cells say *where* it + * burns, so they stay on the warm end of the ramp — lime, amber, red — and + * read as one field instead of a per-cell mosaic; the cold stops belong to + * single detections, which say *how hard* each one burns. + * @param {number} normalized - 0..1 heat score (sqrt-normalized to the view). + * @returns {{name: string, minFrp: number, color: Cesium.Color}} + */ + + function cellColorStop(normalized) { + const name = + normalized > 0.72 ? 'red' : normalized > 0.42 ? 'amber' : 'lime'; + return ( + DETECTION_COLOR_STOPS.find((stop) => stop.name === name) || + DETECTION_COLOR_STOPS[0] + ); + } + + /** + * Fill color for an aggregated heat cell. + * @param {number} normalized - 0..1 heat score (see {@link cellColorStop}). * @param {number} alpha - Fill alpha (heat-normalized by the caller). * @returns {Cesium.Color} */ - function heatColor(maxFrp, alpha) { - return frpColorStop(maxFrp).color.withAlpha(alpha); + function heatColor(normalized, alpha) { + return cellColorStop(normalized).color.withAlpha(alpha); } /** @@ -476,14 +494,14 @@ export function createModel({ } /** - * Cell card accent: the ramp stop of the cell's strongest detection, + * Cell card accent: the warm severity stop of the cell's heat score, * mirroring heatColor. - * @param {number} maxFrp - Strongest FRP in the cell, megawatts. + * @param {number} normalized - 0..1 heat score. * @returns {string} "r, g, b" accent string. */ - function cellAccent(maxFrp) { - return accentForSeverity(frpColorStop(maxFrp).name); + function cellAccent(normalized) { + return accentForSeverity(cellColorStop(normalized).name); } /** Coordinate line like "30.512°N 75.831°E". */ @@ -590,6 +608,7 @@ export function createModel({ cellIntersectsBounds, heatScore, heatColor, + cellColorStop, frpColorStop, detectionColorStop, frpPixelSize, diff --git a/src/layers/firms/rendering.js b/src/layers/firms/rendering.js index 757fd4cf7..8ae445f91 100644 --- a/src/layers/firms/rendering.js +++ b/src/layers/firms/rendering.js @@ -144,7 +144,7 @@ export function createRendering({ const score = components.model.heatScore(cell); const normalized = Math.min(1, Math.sqrt(score / maxScore)); const alpha = 0.16 + normalized * 0.5; - const color = components.model.heatColor(cell.maxFrp, alpha); + const color = components.model.heatColor(normalized, alpha); const centerLon = cell.lonCell + lod.gridDegrees / 2; const centerLat = cell.latCell + lod.gridDegrees / 2; const position = Cesium.Cartesian3.fromDegrees(centerLon, centerLat, 0); @@ -170,12 +170,13 @@ export function createRendering({ }); // Cells arrive heat-sorted, so candidate order doubles as label priority. + // Accent is resolved here because `normalized` only exists in this walk. layerState._labelCandidates.push({ position, cullPosition: components.model.cellCullPosition(centerLon, centerLat), fire: null, cell, - accent: components.model.cellAccent(cell.maxFrp), + accent: components.model.cellAccent(normalized), }); } From 6d6ad20ad3d171ade71911b9542d94d72855194d Mon Sep 17 00:00:00 2001 From: quanticsoul4772 <71096528+quanticsoul4772@users.noreply.github.com> Date: Thu, 29 Oct 2026 00:41:30 -0700 Subject: [PATCH 12/83] fix(infrastructure): ground-sample only stem bases the camera sees scene.sampleHeight reads only surface rendered for the current view. For a data centre or dam base outside the frustum or over the horizon it read through the Earth (about -6,344 km over Ashburn on Esri), failed the 9,000 m check and retried every 2 s indefinitely, one offscreen render per record. Over Ashburn that held rest frames at 58-589 ms. Unseen bases are now skipped without being stamped, so each is sampled on the first walk that sees it, and they no longer arm the retry render. Co-Authored-By: Claude Opus 5.5 --- docs/CURRENT-STATE.md | 4 ++- src/data/localGeojson.test.mjs | 53 +++++++++++++++++++++++++++++++++- src/data/localGeojsonCore.js | 44 +++++++++++++++++++++++++--- 3 files changed, 95 insertions(+), 6 deletions(-) diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 09ae87f1a..e5202e55c 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1959,7 +1959,9 @@ during continuous motion. Disable/re-enable resets selection cleanly. Close-up stem sizing uses camera-to-feature distance without a 5 km minimum. Unchanged overlay entries do not request another frame; moved positions, changed membership and re-enabling still publish. Ground sampling waits for -visible globe tiles to settle and uses valid loaded terrain as a floor while +visible globe tiles to settle, samples only markers whose base is in the +camera's view and above the horizon (an unseen base reads through the Earth +and would retry every 2 s), and uses valid loaded terrain as a floor while retaining rooftop and below-sea-level elevations. A hidden globe does not constrain photoreal geometry, and retry work remains bounded. Already sampled nearby markers follow higher settled terrain on existing bounded walks, without diff --git a/src/data/localGeojson.test.mjs b/src/data/localGeojson.test.mjs index 13cde65a5..89de41529 100644 --- a/src/data/localGeojson.test.mjs +++ b/src/data/localGeojson.test.mjs @@ -90,6 +90,8 @@ async function createRealLocalLayerHarness({ globalThis.window = { dispatchEvent() {} }; let sampleHeightImpl = sampleHeight; const sampleCalls = { count: 0 }; + // The camera frustum, reduced to the one answer ground sampling asks of it. + let baseVisibility = Cesium.Intersect.INSIDE; const overlayHost = { setVisible: (...args) => hostCalls.push(['visible', ...args]), setEntries: (...args) => hostCalls.push(['entries', ...args]), @@ -107,7 +109,14 @@ async function createRealLocalLayerHarness({ }, camera: { positionWC: Cesium.Cartesian3.fromDegrees(-97.695, 30.205, 100_000), - frustum: { fov: Math.PI / 3 }, + directionWC: new Cesium.Cartesian3(), + upWC: new Cesium.Cartesian3(), + frustum: { + fov: Math.PI / 3, + computeCullingVolume: () => ({ + computeVisibility: () => baseVisibility, + }), + }, moveEnd, flyTo() {}, }, @@ -151,6 +160,10 @@ async function createRealLocalLayerHarness({ sampleCalls, /** Swap scene.sampleHeight mid-test (e.g. tiles finally arrive). */ setSampleHeight(next) { sampleHeightImpl = next; }, + /** Put the feature's base inside (true) or outside the camera frustum. */ + setBaseInView(inView) { + baseVisibility = inView ? Cesium.Intersect.INSIDE : Cesium.Intersect.OUTSIDE; + }, cleanup() { if (originalWindow === undefined) delete globalThis.window; else globalThis.window = originalWindow; @@ -736,6 +749,44 @@ test('ground sampling waits for visible globe tiles before caching a height', as assert.ok(Math.abs(baseHeightM(env) - 117) < 0.01); }); +test('an unseen base is not sampled and arms no retry until it is in view', async (t) => { + // Out of view the scene has no surface under the base, so the depth sample + // would pass through the Earth (about -6,344 km) and fail on every retry. + const env = await createRealLocalLayerHarness({ + sampleHeightSupported: true, sampleHeight: () => 117, + }); + _resetRenderGovernorForTest(); + const governorScene = { renders: 0, requestRender() { this.renders += 1; } }; + installRenderGovernor({ scene: governorScene }); + governorScene.renders = 0; // discard the governor's own install settling frame + t.mock.timers.enable({ apis: ['setTimeout'] }); + const clock = installFakeClock(t); + t.after(() => { + env.layer.destroy(env.viewer); + env.cleanup(); + _resetRenderGovernorForTest(); + }); + env.viewer.scene.globe = { show: true, tilesLoaded: true, getHeight: () => 117 }; + setCameraAltitude(env, 20_000); + env.setBaseInView(false); + runArmedRetryChain(env, t, clock, 5); + assert.equal(env.sampleCalls.count, 0, 'an off-screen base must not be sampled'); + assert.equal(governorScene.renders, 0, 'an off-screen base must not keep a parked camera rendering'); + assert.ok(Math.abs(baseHeightM(env)) < 0.01); + + // In the frustum but over the horizon: 50 m up, about 67 km away. + env.setBaseInView(true); + env.viewer.camera.positionWC = Cesium.Cartesian3.fromDegrees(-97.0, 30.205, 50); + runArmedRetryChain(env, t, clock, 5); + assert.equal(env.sampleCalls.count, 0, 'a base over the horizon must not be sampled'); + assert.equal(governorScene.renders, 0); + + setCameraAltitude(env, 20_000); + env.preRender.raise(); + assert.equal(env.sampleCalls.count, 1, 'the base is sampled on the first walk that sees it'); + assert.ok(Math.abs(baseHeightM(env) - 117) < 0.01); +}); + test('settled terrain refinement lifts an already sampled stem without another GPU sample', async (t) => { const env = await createRealLocalLayerHarness({ sampleHeightSupported: true, sampleHeight: () => 117, diff --git a/src/data/localGeojsonCore.js b/src/data/localGeojsonCore.js index 1d04fa46c..9d97df61c 100644 --- a/src/data/localGeojsonCore.js +++ b/src/data/localGeojsonCore.js @@ -1000,9 +1000,9 @@ export function createLocalGeoJsonLayer( // governor the retry's preRender never arrives on a parked camera // and the stem stays at ellipsoid height (buried/floating) until // the user happens to move. Schedule the frame the retry needs. - // Gated on a sampleable scene and in-range records only, so a far - // camera (or a keyless scene) stays fully idle; the distance is - // only computed for still-unsampled stems. + // Gated on a sampleable scene and in-range, in-view records only, + // so a far camera (or a keyless scene) stays fully idle; the + // distance is only computed for still-unsampled stems. if ( canSampleGround && !record.groundSampled && @@ -1010,7 +1010,8 @@ export function createLocalGeoJsonLayer( Cesium.Cartesian3.distance( viewer.camera.positionWC, record.base, - ) < GROUND_SAMPLE_MAX_DISTANCE_M + ) < GROUND_SAMPLE_MAX_DISTANCE_M && + groundSampleInView(viewer, record) ) { groundRetryPending = true; } @@ -1130,9 +1131,44 @@ function refreshLocalTerrainFloor(viewer, record) { return true; } +const groundSampleSphere = new Cesium.BoundingSphere( + new Cesium.Cartesian3(), + 1, +); + +const groundSampleOccluder = new Cesium.EllipsoidalOccluder( + Cesium.Ellipsoid.WGS84, +); + +/** + * Whether the camera sees the record's base: inside the view frustum and + * above the horizon. scene.sampleHeight reads only surface rendered for the + * current view, so a sample of an unseen base passes through the Earth (about + * -6,344 km near Ashburn on Esri) and fails on every 2 s retry, each one a + * full offscreen render: 110 unseen data centres there held the page near 2 s + * a frame indefinitely. An unseen base keeps its stem at the unsampled height + * and is sampled on the first walk that sees it. + */ +function groundSampleInView(viewer, record) { + const camera = viewer.camera; + const culling = camera.frustum.computeCullingVolume( + camera.positionWC, + camera.directionWC, + camera.upWC, + ); + Cesium.Cartesian3.clone(record.base, groundSampleSphere.center); + if ( + culling.computeVisibility(groundSampleSphere) === Cesium.Intersect.OUTSIDE + ) + return false; + groundSampleOccluder.cameraPosition = camera.positionWC; + return groundSampleOccluder.isPointVisible(record.base); +} + function sampleLocalGroundHeight(viewer, record, now) { if (record.groundSampled || !viewer.scene.sampleHeightSupported) return false; if (now - record.lastGroundSampleMs < GROUND_SAMPLE_RETRY_MS) return false; + if (!groundSampleInView(viewer, record)) return false; record.lastGroundSampleMs = now; const globe = viewer.scene.globe; if (globe?.show && globe.tilesLoaded === false) return false; From 189b30b5459a1a476fed3833325d2f460ce07862 Mon Sep 17 00:00:00 2001 From: quanticsoul4772 <71096528+quanticsoul4772@users.noreply.github.com> Date: Thu, 29 Oct 2026 00:43:40 -0700 Subject: [PATCH 13/83] docs(changelog): add the infrastructure ground-sampling entry Co-Authored-By: Claude Opus 5.5 --- changelog.d/1464-infrastructure-ground-sample-in-view.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 changelog.d/1464-infrastructure-ground-sample-in-view.md diff --git a/changelog.d/1464-infrastructure-ground-sample-in-view.md b/changelog.d/1464-infrastructure-ground-sample-in-view.md new file mode 100644 index 000000000..8ab5d1d88 --- /dev/null +++ b/changelog.d/1464-infrastructure-ground-sample-in-view.md @@ -0,0 +1,4 @@ +- Ground-sample Data Centers and Dams markers only when the camera sees their + base. An off-screen base read through the Earth and retried every 2 s, which + held the page near 2 s a frame over Ashburn, Virginia on the Esri map + (quanticsoul4772, #1464). From ecc0ced47c4f765627910bf089e303251cd5e9b4 Mon Sep 17 00:00:00 2001 From: quanticsoul4772 <71096528+quanticsoul4772@users.noreply.github.com> Date: Thu, 29 Oct 2026 01:25:57 -0700 Subject: [PATCH 14/83] fix(infrastructure): test the ground, not the ellipsoid, for in-view sampling An unsampled base sits at ellipsoid height 0. Over high terrain seen at an angle that point projects below the screen while the ground above it is in view, so the gate skipped ground the camera could see and never sampled it. The gate now tests loaded globe terrain when there is some, and otherwise the whole span of heights a sample can accept (0 to 9,000 m). Co-Authored-By: Claude Opus 5.5 --- ...64-infrastructure-ground-sample-in-view.md | 6 +- docs/CURRENT-STATE.md | 8 ++- src/data/localGeojson.test.mjs | 58 +++++++++++++++++++ src/data/localGeojsonCore.js | 49 +++++++++++++--- 4 files changed, 106 insertions(+), 15 deletions(-) diff --git a/changelog.d/1464-infrastructure-ground-sample-in-view.md b/changelog.d/1464-infrastructure-ground-sample-in-view.md index 8ab5d1d88..b563c86b4 100644 --- a/changelog.d/1464-infrastructure-ground-sample-in-view.md +++ b/changelog.d/1464-infrastructure-ground-sample-in-view.md @@ -1,4 +1,4 @@ -- Ground-sample Data Centers and Dams markers only when the camera sees their - base. An off-screen base read through the Earth and retried every 2 s, which - held the page near 2 s a frame over Ashburn, Virginia on the Esri map +- Ground-sample Data Centers and Dams markers only when the camera sees the + ground under them. Unseen ground read through the Earth and retried every + 2 s; over Ashburn, Virginia on the Esri map, slow frames reached 2 s (quanticsoul4772, #1464). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index e5202e55c..065d67112 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1959,9 +1959,11 @@ during continuous motion. Disable/re-enable resets selection cleanly. Close-up stem sizing uses camera-to-feature distance without a 5 km minimum. Unchanged overlay entries do not request another frame; moved positions, changed membership and re-enabling still publish. Ground sampling waits for -visible globe tiles to settle, samples only markers whose base is in the -camera's view and above the horizon (an unseen base reads through the Earth -and would retry every 2 s), and uses valid loaded terrain as a floor while +visible globe tiles to settle, samples only markers whose ground is in the +camera's view and above the horizon (unseen ground reads through the Earth +and would retry every 2 s; until it is seen, a stem keeps its unsampled +height, which can show for a stem whose base is just off screen), and uses +valid loaded terrain as a floor while retaining rooftop and below-sea-level elevations. A hidden globe does not constrain photoreal geometry, and retry work remains bounded. Already sampled nearby markers follow higher settled terrain on existing bounded walks, without diff --git a/src/data/localGeojson.test.mjs b/src/data/localGeojson.test.mjs index 89de41529..1c47cb8c1 100644 --- a/src/data/localGeojson.test.mjs +++ b/src/data/localGeojson.test.mjs @@ -787,6 +787,64 @@ test('an unseen base is not sampled and arms no retry until it is in view', asyn assert.ok(Math.abs(baseHeightM(env) - 117) < 0.01); }); +for (const [label, globe] of [ + ['loaded terrain', { show: true, tilesLoaded: true, getHeight: () => 2_500 }], + ['a hidden globe', { show: false, tilesLoaded: true, getHeight: () => undefined }], +]) { + test(`high ground in view is sampled with ${label} although its unsampled base projects off screen`, async (t) => { + // A base on 2,500 m terrain, seen from 3 km away at a shallow angle: its + // ellipsoid-height point is far below the bottom of the screen. + const env = await createRealLocalLayerHarness({ + sampleHeightSupported: true, sampleHeight: () => 2_500, + }); + const clock = installFakeClock(t); + t.after(() => { env.layer.destroy(env.viewer); env.cleanup(); }); + env.viewer.scene.globe = globe; + const entity = env.dataSources[0].entities.values[0]; + const base = Cesium.Cartographic.fromCartesian(entity.__localBaseCartesian); + const ground = Cesium.Cartesian3.fromRadians(base.longitude, base.latitude, 2_500); + const position = Cesium.Cartesian3.fromRadians( + base.longitude, + base.latitude - 3_000 / 6_371_000, + 2_500 + 1_000, + ); + const direction = Cesium.Cartesian3.normalize( + Cesium.Cartesian3.subtract(ground, position, new Cesium.Cartesian3()), + new Cesium.Cartesian3(), + ); + const right = Cesium.Cartesian3.normalize( + Cesium.Cartesian3.cross( + direction, + Cesium.Ellipsoid.WGS84.geodeticSurfaceNormal(position), + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ); + const up = Cesium.Cartesian3.cross(right, direction, new Cesium.Cartesian3()); + const frustum = new Cesium.PerspectiveFrustum({ + fov: Math.PI / 3, + aspectRatio: 800 / 600, + near: 1, + far: 1e8, + }); + Object.assign(env.viewer.camera, { + positionWC: position, + directionWC: direction, + upWC: up, + frustum, + }); + const offScreen = frustum + .computeCullingVolume(position, direction, up) + .computeVisibility(new Cesium.BoundingSphere(entity.__localBaseCartesian, 1)); + assert.equal(offScreen, Cesium.Intersect.OUTSIDE, 'precondition: the ellipsoid-height base is off screen'); + + clock.advance(2_100); + env.preRender.raise(); + assert.equal(env.sampleCalls.count, 1, 'the ground in view is sampled'); + assert.ok(Math.abs(baseHeightM(env) - 2_500) < 0.01); + }); +} + test('settled terrain refinement lifts an already sampled stem without another GPU sample', async (t) => { const env = await createRealLocalLayerHarness({ sampleHeightSupported: true, sampleHeight: () => 117, diff --git a/src/data/localGeojsonCore.js b/src/data/localGeojsonCore.js index 9d97df61c..ec8fc824e 100644 --- a/src/data/localGeojsonCore.js +++ b/src/data/localGeojsonCore.js @@ -1140,29 +1140,60 @@ const groundSampleOccluder = new Cesium.EllipsoidalOccluder( Cesium.Ellipsoid.WGS84, ); +const groundSampleCarto = new Cesium.Cartographic(); +const groundSampleTop = new Cesium.Cartesian3(); + /** - * Whether the camera sees the record's base: inside the view frustum and - * above the horizon. scene.sampleHeight reads only surface rendered for the - * current view, so a sample of an unseen base passes through the Earth (about - * -6,344 km near Ashburn on Esri) and fails on every 2 s retry, each one a - * full offscreen render: 110 unseen data centres there held the page near 2 s - * a frame indefinitely. An unseen base keeps its stem at the unsampled height - * and is sampled on the first walk that sees it. + * Whether the camera can see the ground under the record: inside the view + * frustum and above the horizon. scene.sampleHeight reads only surface + * rendered for the current view, so a sample of unseen ground passes through + * the Earth (about -6,344 km near Ashburn on Esri) and fails on every 2 s + * retry, each one a full offscreen render; 110 unseen data centres there held + * rest frames at 58-589 ms. Unsampled bases sit on the ellipsoid, so the test + * uses loaded globe terrain when there is some, and otherwise every height a + * sample can accept (0 to GROUND_SAMPLE_MAX_ABS_HEIGHT_M): high ground in + * view is never mistaken for an unseen base. Unseen ground is sampled on the + * first walk that sees it. */ function groundSampleInView(viewer, record) { const camera = viewer.camera; + const globe = viewer.scene.globe; + const terrain = globe?.show ? globe.getHeight?.(record.carto) : undefined; + const known = + Number.isFinite(terrain) && + Math.abs(terrain) <= GROUND_SAMPLE_MAX_ABS_HEIGHT_M; + const span = GROUND_SAMPLE_MAX_ABS_HEIGHT_M / 2; + groundSampleCarto.longitude = record.carto.longitude; + groundSampleCarto.latitude = record.carto.latitude; + groundSampleCarto.height = known ? terrain : span; + Cesium.Cartographic.toCartesian( + groundSampleCarto, + Cesium.Ellipsoid.WGS84, + groundSampleSphere.center, + ); + groundSampleSphere.radius = known ? 1 : span; const culling = camera.frustum.computeCullingVolume( camera.positionWC, camera.directionWC, camera.upWC, ); - Cesium.Cartesian3.clone(record.base, groundSampleSphere.center); if ( culling.computeVisibility(groundSampleSphere) === Cesium.Intersect.OUTSIDE ) return false; + // The highest ground the sample could find is the last to drop below the + // horizon. + if (!known) groundSampleCarto.height = GROUND_SAMPLE_MAX_ABS_HEIGHT_M; groundSampleOccluder.cameraPosition = camera.positionWC; - return groundSampleOccluder.isPointVisible(record.base); + return groundSampleOccluder.isPointVisible( + known + ? groundSampleSphere.center + : Cesium.Cartographic.toCartesian( + groundSampleCarto, + Cesium.Ellipsoid.WGS84, + groundSampleTop, + ), + ); } function sampleLocalGroundHeight(viewer, record, now) { From f392eecbd969a0f8941b6ee2748bc58362a51865 Mon Sep 17 00:00:00 2001 From: quanticsoul4772 <71096528+quanticsoul4772@users.noreply.github.com> Date: Thu, 29 Oct 2026 01:47:05 -0700 Subject: [PATCH 15/83] fix(infrastructure): test ground below 0 m and unknown terrain correctly The ellipsoidal horizon test hid ground below the ellipsoid (low-geoid coasts, the Dead Sea) from a camera near it, so it is now tested at 0 m. With the globe hidden the ground is tested as the vertical from 0 to 9,000 m against the frustum planes, in place of a 4.5 km sphere that let in much unseen ground. Co-Authored-By: Claude Opus 5.5 --- docs/CURRENT-STATE.md | 11 +-- src/data/localGeojson.test.mjs | 128 +++++++++++++++++++++------------ src/data/localGeojsonCore.js | 77 +++++++++++--------- 3 files changed, 130 insertions(+), 86 deletions(-) diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 065d67112..4815e0fd1 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1959,11 +1959,12 @@ during continuous motion. Disable/re-enable resets selection cleanly. Close-up stem sizing uses camera-to-feature distance without a 5 km minimum. Unchanged overlay entries do not request another frame; moved positions, changed membership and re-enabling still publish. Ground sampling waits for -visible globe tiles to settle, samples only markers whose ground is in the -camera's view and above the horizon (unseen ground reads through the Earth -and would retry every 2 s; until it is seen, a stem keeps its unsampled -height, which can show for a stem whose base is just off screen), and uses -valid loaded terrain as a floor while +visible globe tiles to settle, and skips markers whose ground the camera +cannot see (unseen ground reads through the Earth and would retry every 2 s). +Ground is tested at loaded globe terrain, or across 0-9,000 m when the globe +is hidden, so photoreal views skip only ground well out of view. Until it is +seen, a stem keeps its unsampled height, which can show for a stem whose base +is just off screen. Sampling uses valid loaded terrain as a floor while retaining rooftop and below-sea-level elevations. A hidden globe does not constrain photoreal geometry, and retry work remains bounded. Already sampled nearby markers follow higher settled terrain on existing bounded walks, without diff --git a/src/data/localGeojson.test.mjs b/src/data/localGeojson.test.mjs index 1c47cb8c1..3e104592f 100644 --- a/src/data/localGeojson.test.mjs +++ b/src/data/localGeojson.test.mjs @@ -113,8 +113,13 @@ async function createRealLocalLayerHarness({ upWC: new Cesium.Cartesian3(), frustum: { fov: Math.PI / 3, + // No planes: everything is in view. One plane no point satisfies: + // nothing is. computeCullingVolume: () => ({ - computeVisibility: () => baseVisibility, + planes: + baseVisibility === Cesium.Intersect.OUTSIDE + ? [new Cesium.Cartesian4(0, 0, 0, -1)] + : [], }), }, moveEnd, @@ -787,64 +792,95 @@ test('an unseen base is not sampled and arms no retry until it is in view', asyn assert.ok(Math.abs(baseHeightM(env) - 117) < 0.01); }); -for (const [label, globe] of [ - ['loaded terrain', { show: true, tilesLoaded: true, getHeight: () => 2_500 }], - ['a hidden globe', { show: false, tilesLoaded: true, getHeight: () => undefined }], +/** + * Give the harness a real Cesium camera `southM` metres south of the base at + * `cameraM` metres above the ellipsoid, looking at the base's ground at + * `groundM` (or directly away from it). + */ +function aimRealCamera(env, { groundM, cameraM, southM, away = false }) { + const entity = env.dataSources[0].entities.values[0]; + const base = Cesium.Cartographic.fromCartesian(entity.__localBaseCartesian); + const ground = Cesium.Cartesian3.fromRadians(base.longitude, base.latitude, groundM); + const position = Cesium.Cartesian3.fromRadians( + base.longitude, + base.latitude - southM / 6_371_000, + cameraM, + ); + const direction = Cesium.Cartesian3.normalize( + away + ? Cesium.Cartesian3.subtract(position, ground, new Cesium.Cartesian3()) + : Cesium.Cartesian3.subtract(ground, position, new Cesium.Cartesian3()), + new Cesium.Cartesian3(), + ); + const right = Cesium.Cartesian3.normalize( + Cesium.Cartesian3.cross( + direction, + Cesium.Ellipsoid.WGS84.geodeticSurfaceNormal(position), + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ); + const up = Cesium.Cartesian3.cross(right, direction, new Cesium.Cartesian3()); + const frustum = new Cesium.PerspectiveFrustum({ + fov: Math.PI / 3, + aspectRatio: 800 / 600, + near: 1, + far: 1e8, + }); + Object.assign(env.viewer.camera, { + positionWC: position, + directionWC: direction, + upWC: up, + frustum, + }); + return frustum + .computeCullingVolume(position, direction, up) + .computeVisibility(new Cesium.BoundingSphere(entity.__localBaseCartesian, 1)); +} + +const loadedTerrain = (height) => ({ show: true, tilesLoaded: true, getHeight: () => height }); +const hiddenGlobe = { show: false, tilesLoaded: true, getHeight: () => undefined }; + +for (const { label, globe, groundM, cameraM, southM, baseOffScreen } of [ + // 2,500 m terrain from 3 km away at a shallow angle: the ellipsoid-height + // base is far below the bottom of the screen. + { label: 'high ground with loaded terrain', globe: loadedTerrain(2_500), groundM: 2_500, cameraM: 3_500, southM: 3_000, baseOffScreen: true }, + { label: 'high ground under a hidden globe', globe: hiddenGlobe, groundM: 2_500, cameraM: 3_500, southM: 3_000, baseOffScreen: true }, + // A low-geoid coast (about -63 m near Mumbai) from a camera 50 m above the + // ellipsoid: the ellipsoidal horizon would hide the ground itself. + { label: 'ground below the ellipsoid', globe: loadedTerrain(-63), groundM: -63, cameraM: 50, southM: 2_000, baseOffScreen: false }, ]) { - test(`high ground in view is sampled with ${label} although its unsampled base projects off screen`, async (t) => { - // A base on 2,500 m terrain, seen from 3 km away at a shallow angle: its - // ellipsoid-height point is far below the bottom of the screen. + test(`${label} in view is sampled`, async (t) => { const env = await createRealLocalLayerHarness({ - sampleHeightSupported: true, sampleHeight: () => 2_500, + sampleHeightSupported: true, sampleHeight: () => groundM, }); const clock = installFakeClock(t); t.after(() => { env.layer.destroy(env.viewer); env.cleanup(); }); env.viewer.scene.globe = globe; - const entity = env.dataSources[0].entities.values[0]; - const base = Cesium.Cartographic.fromCartesian(entity.__localBaseCartesian); - const ground = Cesium.Cartesian3.fromRadians(base.longitude, base.latitude, 2_500); - const position = Cesium.Cartesian3.fromRadians( - base.longitude, - base.latitude - 3_000 / 6_371_000, - 2_500 + 1_000, - ); - const direction = Cesium.Cartesian3.normalize( - Cesium.Cartesian3.subtract(ground, position, new Cesium.Cartesian3()), - new Cesium.Cartesian3(), - ); - const right = Cesium.Cartesian3.normalize( - Cesium.Cartesian3.cross( - direction, - Cesium.Ellipsoid.WGS84.geodeticSurfaceNormal(position), - new Cesium.Cartesian3(), - ), - new Cesium.Cartesian3(), - ); - const up = Cesium.Cartesian3.cross(right, direction, new Cesium.Cartesian3()); - const frustum = new Cesium.PerspectiveFrustum({ - fov: Math.PI / 3, - aspectRatio: 800 / 600, - near: 1, - far: 1e8, - }); - Object.assign(env.viewer.camera, { - positionWC: position, - directionWC: direction, - upWC: up, - frustum, - }); - const offScreen = frustum - .computeCullingVolume(position, direction, up) - .computeVisibility(new Cesium.BoundingSphere(entity.__localBaseCartesian, 1)); - assert.equal(offScreen, Cesium.Intersect.OUTSIDE, 'precondition: the ellipsoid-height base is off screen'); + const baseVisibility = aimRealCamera(env, { groundM, cameraM, southM }); + if (baseOffScreen) + assert.equal(baseVisibility, Cesium.Intersect.OUTSIDE, 'precondition: the ellipsoid-height base is off screen'); clock.advance(2_100); env.preRender.raise(); assert.equal(env.sampleCalls.count, 1, 'the ground in view is sampled'); - assert.ok(Math.abs(baseHeightM(env) - 2_500) < 0.01); + assert.ok(Math.abs(baseHeightM(env) - groundM) < 0.01); }); } +test('under a hidden globe, ground behind the camera is not sampled', async (t) => { + const env = await createRealLocalLayerHarness({ + sampleHeightSupported: true, sampleHeight: () => 2_500, + }); + const clock = installFakeClock(t); + t.after(() => { env.layer.destroy(env.viewer); env.cleanup(); }); + env.viewer.scene.globe = hiddenGlobe; + aimRealCamera(env, { groundM: 2_500, cameraM: 3_500, southM: 3_000, away: true }); + clock.advance(2_100); + env.preRender.raise(); + assert.equal(env.sampleCalls.count, 0); +}); + test('settled terrain refinement lifts an already sampled stem without another GPU sample', async (t) => { const env = await createRealLocalLayerHarness({ sampleHeightSupported: true, sampleHeight: () => 117, diff --git a/src/data/localGeojsonCore.js b/src/data/localGeojsonCore.js index ec8fc824e..596b78ed9 100644 --- a/src/data/localGeojsonCore.js +++ b/src/data/localGeojsonCore.js @@ -1131,17 +1131,25 @@ function refreshLocalTerrainFloor(viewer, record) { return true; } -const groundSampleSphere = new Cesium.BoundingSphere( - new Cesium.Cartesian3(), - 1, -); - const groundSampleOccluder = new Cesium.EllipsoidalOccluder( Cesium.Ellipsoid.WGS84, ); const groundSampleCarto = new Cesium.Cartographic(); -const groundSampleTop = new Cesium.Cartesian3(); +const groundSampleLow = new Cesium.Cartesian3(); +const groundSampleHigh = new Cesium.Cartesian3(); + +/** The record's vertical at `height` metres above the ellipsoid. */ +function groundSamplePoint(record, height, result) { + groundSampleCarto.longitude = record.carto.longitude; + groundSampleCarto.latitude = record.carto.latitude; + groundSampleCarto.height = height; + return Cesium.Cartographic.toCartesian( + groundSampleCarto, + Cesium.Ellipsoid.WGS84, + result, + ); +} /** * Whether the camera can see the ground under the record: inside the view @@ -1150,10 +1158,10 @@ const groundSampleTop = new Cesium.Cartesian3(); * the Earth (about -6,344 km near Ashburn on Esri) and fails on every 2 s * retry, each one a full offscreen render; 110 unseen data centres there held * rest frames at 58-589 ms. Unsampled bases sit on the ellipsoid, so the test - * uses loaded globe terrain when there is some, and otherwise every height a - * sample can accept (0 to GROUND_SAMPLE_MAX_ABS_HEIGHT_M): high ground in - * view is never mistaken for an unseen base. Unseen ground is sampled on the - * first walk that sees it. + * uses loaded globe terrain when there is some. Otherwise (a hidden globe) it + * tests the vertical from 0 to GROUND_SAMPLE_MAX_ABS_HEIGHT_M, every height a + * sample can accept, so high ground in view is never mistaken for an unseen + * base. Unseen ground is sampled on the first walk that sees it. */ function groundSampleInView(viewer, record) { const camera = viewer.camera; @@ -1162,37 +1170,36 @@ function groundSampleInView(viewer, record) { const known = Number.isFinite(terrain) && Math.abs(terrain) <= GROUND_SAMPLE_MAX_ABS_HEIGHT_M; - const span = GROUND_SAMPLE_MAX_ABS_HEIGHT_M / 2; - groundSampleCarto.longitude = record.carto.longitude; - groundSampleCarto.latitude = record.carto.latitude; - groundSampleCarto.height = known ? terrain : span; - Cesium.Cartographic.toCartesian( - groundSampleCarto, - Cesium.Ellipsoid.WGS84, - groundSampleSphere.center, - ); - groundSampleSphere.radius = known ? 1 : span; - const culling = camera.frustum.computeCullingVolume( + const low = groundSamplePoint(record, known ? terrain : 0, groundSampleLow); + const high = known + ? low + : groundSamplePoint( + record, + GROUND_SAMPLE_MAX_ABS_HEIGHT_M, + groundSampleHigh, + ); + // The vertical is unseen only when both ends are outside the same plane. + const { planes } = camera.frustum.computeCullingVolume( camera.positionWC, camera.directionWC, camera.upWC, ); - if ( - culling.computeVisibility(groundSampleSphere) === Cesium.Intersect.OUTSIDE - ) - return false; - // The highest ground the sample could find is the last to drop below the - // horizon. - if (!known) groundSampleCarto.height = GROUND_SAMPLE_MAX_ABS_HEIGHT_M; + for (const plane of planes) { + if ( + Cesium.Cartesian3.dot(plane, low) + plane.w < 0 && + Cesium.Cartesian3.dot(plane, high) + plane.w < 0 + ) + return false; + } + // The ellipsoidal occluder hides every point below the ellipsoid from a + // camera near it, so ground below 0 m (low-geoid coasts, the Dead Sea) is + // tested at 0 m; unknown ground at the top of the vertical. + const horizonHeight = known + ? Math.max(terrain, 0) + : GROUND_SAMPLE_MAX_ABS_HEIGHT_M; groundSampleOccluder.cameraPosition = camera.positionWC; return groundSampleOccluder.isPointVisible( - known - ? groundSampleSphere.center - : Cesium.Cartographic.toCartesian( - groundSampleCarto, - Cesium.Ellipsoid.WGS84, - groundSampleTop, - ), + groundSamplePoint(record, horizonHeight, groundSampleHigh), ); } From b778a43f1d7538a10d29f89b0611c29e3dabfe6e Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Thu, 29 Oct 2026 17:05:14 +0400 Subject: [PATCH 16/83] feat(fires): Snap Map-style heat raster and its globe imagery drape Detections are binned on a 2048x1024 equirectangular grid weighted 1 + log10(1 + FRP), Gaussian-blurred (longitude wraps), normalized on a log scale against the 99.5th percentile and mapped blue -> red with an alpha ramp that is fully transparent below the floor. heatImagery drapes the bytes as one full-globe SingleTileImageryProvider on the active imagery host. Co-Authored-By: Claude Opus 5.5 --- src/data/firmsHeatRaster.js | 176 ++++++++++++++++++++++++++ src/data/firmsHeatRaster.test.mjs | 73 +++++++++++ src/layers/firms/heatImagery.js | 71 +++++++++++ src/layers/firms/heatImagery.test.mjs | 87 +++++++++++++ 4 files changed, 407 insertions(+) create mode 100644 src/data/firmsHeatRaster.js create mode 100644 src/data/firmsHeatRaster.test.mjs create mode 100644 src/layers/firms/heatImagery.js create mode 100644 src/layers/firms/heatImagery.test.mjs diff --git a/src/data/firmsHeatRaster.js b/src/data/firmsHeatRaster.js new file mode 100644 index 000000000..52a020671 --- /dev/null +++ b/src/data/firmsHeatRaster.js @@ -0,0 +1,176 @@ +/** + * Snap Map–style heat raster for the Active Fires far-zoom bands: detections + * are binned on an equirectangular grid (row 0 = north, column 0 = 180°W), + * blurred, normalized on a log scale and mapped through HEAT_PALETTE. Pure: + * no Cesium, no DOM, so the whole pipeline is unit-tested in node. + */ + +/** Below this normalized heat nothing is painted: empty land stays empty. */ +export const HEAT_FLOOR = 0.03; + +export const HEAT_PALETTE = Object.freeze([ + { at: 0.0, rgb: [43, 108, 255], alpha: 0 }, + { at: 0.08, rgb: [43, 108, 255], alpha: 0.35 }, + { at: 0.25, rgb: [25, 195, 255], alpha: 0.55 }, + { at: 0.45, rgb: [47, 224, 122], alpha: 0.65 }, + { at: 0.65, rgb: [245, 230, 66], alpha: 0.75 }, + { at: 0.82, rgb: [255, 138, 31], alpha: 0.82 }, + { at: 1.0, rgb: [255, 42, 26], alpha: 0.88 }, +]); + +export function heatColorAt(t) { + if (!(t > HEAT_FLOOR)) return [0, 0, 0, 0]; + const v = Math.min(1, t); + let i = 1; + while (i < HEAT_PALETTE.length - 1 && HEAT_PALETTE[i].at < v) i += 1; + const a = HEAT_PALETTE[i - 1]; + const b = HEAT_PALETTE[i]; + const k = (v - a.at) / (b.at - a.at || 1); + const mix = (p, q) => Math.round(p + (q - p) * k); + return [ + mix(a.rgb[0], b.rgb[0]), + mix(a.rgb[1], b.rgb[1]), + mix(a.rgb[2], b.rgb[2]), + Math.round((a.alpha + (b.alpha - a.alpha) * k) * 255), + ]; +} + +const defaultWeight = (fire) => + 1 + Math.log10(1 + Math.max(0, Number(fire.frp) || 0)); + +export function binFires(fires, { width, height, weight = defaultWeight }) { + const grid = new Float32Array(width * height); + for (const fire of fires) { + if (!Number.isFinite(fire.lat) || !Number.isFinite(fire.lon)) continue; + const x = Math.min( + width - 1, + Math.max(0, Math.floor(((fire.lon + 180) / 360) * width)), + ); + const y = Math.min( + height - 1, + Math.max(0, Math.floor(((90 - fire.lat) / 180) * height)), + ); + grid[y * width + x] += weight(fire); + } + return grid; +} + +function kernel(sigma) { + const radius = Math.max(1, Math.ceil(sigma * 3)); + const k = new Float32Array(radius * 2 + 1); + let sum = 0; + for (let i = -radius; i <= radius; i++) { + k[i + radius] = Math.exp(-(i * i) / (2 * sigma * sigma)); + sum += k[i + radius]; + } + for (let i = 0; i < k.length; i++) k[i] /= sum; + return { k, radius }; +} + +export function blurGaussian(grid, width, height, sigma) { + const { k, radius } = kernel(sigma); + const tmp = new Float32Array(grid.length); + const out = new Float32Array(grid.length); + for (let y = 0; y < height; y++) { + const row = y * width; + for (let x = 0; x < width; x++) { + const v = grid[row + x]; + if (v === 0) continue; + for (let i = -radius; i <= radius; i++) { + const xx = (((x + i) % width) + width) % width; // longitude wraps + tmp[row + xx] += v * k[i + radius]; + } + } + } + for (let y = 0; y < height; y++) { + for (let x = 0; x < width; x++) { + const v = tmp[y * width + x]; + if (v === 0) continue; + for (let i = -radius; i <= radius; i++) { + const yy = Math.min(height - 1, Math.max(0, y + i)); // poles clamp + out[yy * width + x] += v * k[i + radius]; + } + } + } + return out; +} + +/** Bins of the log-density histogram used to find the reference quantile. */ +const QUANTILE_BINS = 4096; + +/** + * The p-quantile of the non-zero values, from a histogram of log1p(value): + * one linear pass instead of sorting ~350k burning pixels (a sort cost + * 200 ms+ and dominated the build). Bin width on the log axis is + * max/4096, far finer than the 256-step palette it feeds. + */ +function percentileOfNonZero(values, p) { + let count = 0; + let max = 0; + for (let i = 0; i < values.length; i++) { + const v = values[i]; + if (v > 0) { + count += 1; + if (v > max) max = v; + } + } + if (!count) return 0; + const logMax = Math.log1p(max); + const bins = new Uint32Array(QUANTILE_BINS); + const toBin = (QUANTILE_BINS - 1) / logMax; + for (let i = 0; i < values.length; i++) { + const v = values[i]; + if (v > 0) bins[Math.floor(Math.log1p(v) * toBin)] += 1; + } + const target = Math.floor(p * (count - 1)); + let seen = 0; + for (let b = 0; b < QUANTILE_BINS; b++) { + seen += bins[b]; + if (seen > target) return Math.expm1((b + 1) / toBin); + } + return max; +} + +/** HEAT_PALETTE sampled at 256 steps, so the per-pixel walk is one lookup. */ +const LUT_STEPS = 256; +let paletteLut = null; +function heatLut() { + if (!paletteLut) { + paletteLut = new Uint8ClampedArray(LUT_STEPS * 4); + for (let i = 0; i < LUT_STEPS; i++) { + paletteLut.set(heatColorAt(i / (LUT_STEPS - 1)), i * 4); + } + } + return paletteLut; +} + +export function buildHeatRaster( + fires, + { width, height, sigmaPx, percentile = 0.995 }, +) { + const density = blurGaussian( + binFires(fires, { width, height }), + width, + height, + sigmaPx, + ); + const reference = percentileOfNonZero(density, percentile); + const rgba = new Uint8ClampedArray(width * height * 4); + if (reference > 0) { + const lut = heatLut(); + const scale = (LUT_STEPS - 1) / Math.log1p(reference); + for (let i = 0; i < density.length; i++) { + const d = density[i]; + if (d <= 0) continue; + const step = Math.min(LUT_STEPS - 1, Math.round(Math.log1p(d) * scale)); + const l = step * 4; + if (lut[l + 3] === 0) continue; + const o = i * 4; + rgba[o] = lut[l]; + rgba[o + 1] = lut[l + 1]; + rgba[o + 2] = lut[l + 2]; + rgba[o + 3] = lut[l + 3]; + } + } + return { width, height, rgba, reference }; +} diff --git a/src/data/firmsHeatRaster.test.mjs b/src/data/firmsHeatRaster.test.mjs new file mode 100644 index 000000000..704c83427 --- /dev/null +++ b/src/data/firmsHeatRaster.test.mjs @@ -0,0 +1,73 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + HEAT_PALETTE, + heatColorAt, + binFires, + blurGaussian, + buildHeatRaster, +} from './firmsHeatRaster.js'; + +const fire = (lat, lon, frp = 10) => ({ lat, lon, frp }); + +test('palette runs cold to hot and starts fully transparent', () => { + assert.equal(HEAT_PALETTE[0].at, 0); + assert.equal(HEAT_PALETTE.at(-1).at, 1); + assert.deepEqual(heatColorAt(0), [0, 0, 0, 0]); + assert.equal(heatColorAt(0.01)[3], 0, 'below the floor nothing is painted'); + const low = heatColorAt(0.15); + const high = heatColorAt(1); + assert.ok(low[2] > low[0], 'low heat is blue'); + assert.ok(high[0] > high[2], 'the core is red'); + assert.ok(high[3] > low[3], 'hotter is more opaque'); +}); + +test('binFires drops each detection into its equirectangular pixel', () => { + const grid = binFires([fire(89.9, -179.9, 0), fire(-89.9, 179.9, 0)], { + width: 4, + height: 2, + }); + assert.equal(grid[0], 1, 'north-west corner'); + assert.equal(grid[7], 1, 'south-east corner'); + assert.equal( + grid.reduce((a, b) => a + b, 0), + 2, + ); +}); + +test('binFires weights strong fires slightly more', () => { + const grid = binFires([fire(0.1, 0.1, 999)], { width: 4, height: 2 }); + assert.ok(grid[2] > 3 && grid[2] < 5, String(grid[2])); +}); + +test('blurGaussian keeps mass and wraps across the antimeridian', () => { + const w = 16; + const h = 8; + const grid = new Float32Array(w * h); + grid[3 * w + 0] = 1; + const out = blurGaussian(grid, w, h, 1.5); + const sum = out.reduce((a, b) => a + b, 0); + assert.ok(Math.abs(sum - 1) < 1e-3, `mass ${sum}`); + assert.ok(out[3 * w + (w - 1)] > 0, 'spills to the east edge'); +}); + +test('buildHeatRaster is transparent where there are no fires', () => { + const r = buildHeatRaster([fire(40, -120), fire(40.1, -120.1)], { + width: 360, + height: 180, + sigmaPx: 2, + }); + assert.equal(r.rgba.length, 360 * 180 * 4); + const alphaAt = (lat, lon) => { + const x = Math.floor(((lon + 180) / 360) * 360); + const y = Math.floor(((90 - lat) / 180) * 180); + return r.rgba[(y * 360 + x) * 4 + 3]; + }; + assert.ok(alphaAt(40, -120) > 0, 'blob over the fires'); + assert.equal(alphaAt(-30, 60), 0, 'empty far away'); +}); + +test('buildHeatRaster with no fires is fully transparent', () => { + const r = buildHeatRaster([], { width: 8, height: 4, sigmaPx: 1 }); + assert.ok(r.rgba.every((v) => v === 0)); +}); diff --git a/src/layers/firms/heatImagery.js b/src/layers/firms/heatImagery.js new file mode 100644 index 000000000..f2f99fbe6 --- /dev/null +++ b/src/layers/firms/heatImagery.js @@ -0,0 +1,71 @@ +/** + * Drapes the Active Fires heat raster over the globe as one full-extent + * imagery tile on the active imagery host (globe or 3D-tiles), like the wind + * layer's scalar field. Owns the canvas and the imagery layer; nothing else + * in the fires layer touches imagery. + */ + +const defaultCanvas = (width, height) => { + const canvas = document.createElement('canvas'); + canvas.width = width; + canvas.height = height; + return canvas; +}; + +export function createHeatImagery({ + cesium, + getHost, + createCanvas = defaultCanvas, +}) { + let raster = null; + let layer = null; + let collection = null; + let show = true; + + function detach() { + if (layer && collection) collection.remove(layer, true); + layer = null; + collection = null; + } + + function attach() { + detach(); + if (!raster) return; + const host = getHost?.() ?? { collection: null, kind: 'none' }; + if (!host.collection) return; + const canvas = createCanvas(raster.width, raster.height); + const ctx = canvas.getContext('2d'); + const image = ctx.createImageData(raster.width, raster.height); + image.data.set(raster.rgba); + ctx.putImageData(image, 0, 0); + const provider = new cesium.SingleTileImageryProvider({ + url: canvas.toDataURL('image/png'), + tileWidth: raster.width, + tileHeight: raster.height, + rectangle: cesium.Rectangle.MAX_VALUE, + }); + collection = host.collection; + layer = collection.addImageryProvider(provider); + layer.show = show; + collection.raiseToTop?.(layer); + } + + return { + setRaster(next) { + raster = next; + attach(); + }, + setShow(value) { + show = Boolean(value); + if (layer) layer.show = show; + }, + rehome() { + const host = getHost?.(); + if ((host?.collection ?? null) !== collection) attach(); + }, + destroy() { + detach(); + raster = null; + }, + }; +} diff --git a/src/layers/firms/heatImagery.test.mjs b/src/layers/firms/heatImagery.test.mjs new file mode 100644 index 000000000..0fca7b97f --- /dev/null +++ b/src/layers/firms/heatImagery.test.mjs @@ -0,0 +1,87 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createHeatImagery } from './heatImagery.js'; + +function harness() { + const layers = []; + const removed = []; + const collection = { + addImageryProvider(provider) { + const layer = { provider, show: true, alpha: 1 }; + layers.push(layer); + return layer; + }, + remove(layer, destroy) { + layers.splice(layers.indexOf(layer), 1); + removed.push({ layer, destroy }); + }, + raiseToTop() {}, + }; + let host = { collection, kind: 'globe' }; + const cesium = { + Rectangle: { MAX_VALUE: { max: true } }, + SingleTileImageryProvider: class { + constructor(options) { + this.options = options; + } + }, + }; + const createCanvas = (w, h) => ({ + width: w, + height: h, + getContext: () => ({ + createImageData: (cw, ch) => ({ + data: new Uint8ClampedArray(cw * ch * 4), + }), + putImageData() {}, + }), + toDataURL: () => 'data:image/png;base64,AAAA', + }); + const imagery = createHeatImagery({ + cesium, + getHost: () => host, + createCanvas, + }); + return { imagery, layers, removed, collection, setHost: (h) => (host = h) }; +} + +const raster = { width: 4, height: 2, rgba: new Uint8ClampedArray(32) }; + +test('setRaster drapes one full-globe tile and replaces it on the next build', () => { + const h = harness(); + h.imagery.setRaster(raster); + assert.equal(h.layers.length, 1); + assert.deepEqual(h.layers[0].provider.options.rectangle, { max: true }); + assert.equal(h.layers[0].provider.options.tileWidth, 4); + h.imagery.setRaster(raster); + assert.equal(h.layers.length, 1, 'never two heat layers'); + assert.equal(h.removed[0].destroy, true); +}); + +test('setShow toggles without rebuilding', () => { + const h = harness(); + h.imagery.setRaster(raster); + const layer = h.layers[0]; + h.imagery.setShow(false); + assert.equal(layer.show, false); + h.imagery.setShow(true); + assert.equal(h.layers[0], layer); + assert.equal(layer.show, true); +}); + +test('no imagery host keeps nothing draped; rehome brings it back', () => { + const h = harness(); + h.setHost({ collection: null, kind: 'none' }); + h.imagery.setRaster(raster); + assert.equal(h.layers.length, 0); + h.setHost({ collection: h.collection, kind: 'globe' }); + h.imagery.rehome(); + assert.equal(h.layers.length, 1); +}); + +test('destroy removes the layer', () => { + const h = harness(); + h.imagery.setRaster(raster); + h.imagery.destroy(); + assert.equal(h.layers.length, 0); +}); From de87aaf86acfbf3fd358833c6478e93d9718d3f8 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Thu, 29 Oct 2026 17:11:32 +0400 Subject: [PATCH 17/83] feat(fires): far zoom shows a Snap Map-style heatmap instead of heat squares The global and regional LOD bands drape the heat raster on the active imagery host and draw no rectangle entities; the 2/1 degree aggregation now only feeds the "N FIRES" cards, whose accent is the strongest fire's FRP color. Markers below ~750 km hide the heatmap, as do disable and destroy. A stale raster is rebuilt immediately on first paint and with a 150 ms trailing debounce afterwards, so a slider drag does not re-encode on every step. The shell hands the layer its imagery host, so the heatmap follows globe and 3D-tiles map stacks. Co-Authored-By: Claude Opus 5.5 --- src/data/firmsInteraction.test.mjs | 75 ++++++++++++++++++ src/layers/firms/heatImagery.js | 4 + src/layers/firms/ingestion.js | 7 +- src/layers/firms/lifecycle.js | 24 ++++++ src/layers/firms/model.js | 48 ++---------- src/layers/firms/policy.js | 32 ++++++-- src/layers/firms/queries.js | 15 +++- src/layers/firms/rendering.js | 119 +++++++++++++++++++---------- src/layers/firms/state.js | 19 +++++ src/ui/layerBindings.js | 6 ++ 10 files changed, 257 insertions(+), 92 deletions(-) diff --git a/src/data/firmsInteraction.test.mjs b/src/data/firmsInteraction.test.mjs index d735df3f8..39f0382c8 100644 --- a/src/data/firmsInteraction.test.mjs +++ b/src/data/firmsInteraction.test.mjs @@ -392,3 +392,78 @@ test('disable cancels a pending source and a late response cannot repopulate the assert.equal(h.layer.getStats().loading, false); } finally { h.layer.destroy(h.viewer); h.cleanup(); } }); + +test('far zoom drapes one heatmap and draws no rectangles; markers and disable hide it', () => { + // A canvas stub for the heat drape and the detection glow sprites. + const priorDocument = globalThis.document; + const restoreDocument = () => { + if (priorDocument === undefined) delete globalThis.document; + else globalThis.document = priorDocument; + }; + globalThis.document = { + createElement: () => ({ + getContext: () => ({ + createImageData: (w, hh) => ({ + data: new Uint8ClampedArray(w * hh * 4), + }), + putImageData() {}, + createRadialGradient: () => ({ addColorStop() {} }), + fillRect() {}, + }), + toDataURL: () => 'data:image/png;base64,AAAA', + }), + }; + const h = harness({ + fires: [makeFire(), makeFire({ index: 8, lat: 30.6, lon: -98.3, frp: 12 })], + withDataSource: true, + cameraPosition: Cesium.Cartesian3.fromDegrees(-98, 30, 12_000_000), + }); + const added = []; + const removed = []; + h.viewer.scene.primitives = { + add() {}, + remove() {}, + raiseToTop() {}, + contains: () => true, + }; + h.viewer.scene.frameState = { mode: Cesium.SceneMode.SCENE3D }; + try { + const collection = { + addImageryProvider: (provider) => { + const layer = { provider, show: true }; + added.push(layer); + return layer; + }, + remove: (layer, destroy) => removed.push({ layer, destroy }), + raiseToTop() {}, + }; + h.layer.attachShellServices({ + imageryHost: () => ({ kind: 'globe', collection }), + }); + h.layer._renderLodForTest('global'); + assert.equal(added.length, 1, 'one heat imagery layer'); + assert.equal(added[0].show, true); + assert.equal(h.layer._entityCountForTest(), 0, 'no rectangle entities'); + assert.ok(h.published.length > 0, 'the far band still paints cards'); + + h.layer._renderLodForTest('regional'); + assert.equal(added.length, 1, 'switching far bands reuses the raster'); + + h.layer._renderLodForTest('close'); + assert.equal(added[0].show, false, 'the markers band hides the heatmap'); + + h.layer._renderLodForTest('global'); + assert.equal(added[0].show, true); + h.layer.disable(); + assert.equal(added[0].show, false, 'disable hides the heatmap'); + + h.layer.destroy(h.viewer); + assert.deepEqual( + removed.map((entry) => [entry.layer, entry.destroy]), + [[added[0], true]], + ); + } finally { + h.cleanup(); + restoreDocument(); + } +}); diff --git a/src/layers/firms/heatImagery.js b/src/layers/firms/heatImagery.js index f2f99fbe6..31fbf1f69 100644 --- a/src/layers/firms/heatImagery.js +++ b/src/layers/firms/heatImagery.js @@ -55,6 +55,10 @@ export function createHeatImagery({ raster = next; attach(); }, + /** True once a raster was handed over (painted or waiting for a host). */ + hasRaster() { + return raster !== null; + }, setShow(value) { show = Boolean(value); if (layer) layer.show = show; diff --git a/src/layers/firms/ingestion.js b/src/layers/firms/ingestion.js index 215c35da3..cb9dbd72f 100644 --- a/src/layers/firms/ingestion.js +++ b/src/layers/firms/ingestion.js @@ -89,14 +89,15 @@ export function createIngestion({ /** * Run the intensity filter over the loaded payload. This is the ONE place * the shown set is derived: `_fires` / `_firesByFrp` feed markers, cards, - * heat cells, the context store, the top-fires query and the analyst - * records, so none of them can disagree about which fires exist. Cells are - * aggregated per shown set, so the cache goes with it. + * the heatmap, the context store, the top-fires query and the analyst + * records, so none of them can disagree about which fires exist. Cells and + * the heat raster are built per shown set, so their caches go with it. */ function applyFilter() { layerState._fires = filterFires(layerState._allFires, layerState._filter); layerState._cellCacheByGrid.clear(); + layerState._heatRaster = null; layerState._firesByFrp = filterFires( layerState._allFiresByFrp, layerState._filter, diff --git a/src/layers/firms/lifecycle.js b/src/layers/firms/lifecycle.js index 1138504dd..72434ad46 100644 --- a/src/layers/firms/lifecycle.js +++ b/src/layers/firms/lifecycle.js @@ -1,5 +1,6 @@ import * as Cesium from 'cesium'; import { FIRMS_OVERLAY_SOURCE_ID } from '../../data/firmsLabels.js'; +import { LOD_LEVELS } from './policy.js'; export function createLifecycle({ layerState, @@ -14,6 +15,20 @@ export function createLifecycle({ const { id, overlayHost } = config; const methods = { + /** + * Shell services: `imageryHost()` resolves where draped imagery goes on + * the active map stack (globe or 3D tiles, src/maps/imageryHost.js). The + * heatmap moves with it; without a host it drapes on viewer.imageryLayers. + * @param {{imageryHost?: () => {collection: ?Object, kind: string}}|null} services + */ + attachShellServices(services) { + layerState._imageryHost = + typeof services?.imageryHost === 'function' + ? services.imageryHost + : null; + layerState._heatImagery?.rehome(); + }, + init(viewer) { if (layerState._destroyed) return; layerState._viewer = viewer; @@ -40,6 +55,8 @@ export function createLifecycle({ layerState._billboards.show = true; } overlayHost.setVisible(FIRMS_OVERLAY_SOURCE_ID, true); + if (LOD_LEVELS[layerState._currentLodIndex]?.mode === 'heat') + layerState._heatImagery?.setShow(true); components.viewport.installLodWatcher(); components.viewport.installMoveEndWatcher(); components.selection.installClickHandler(); @@ -64,6 +81,7 @@ export function createLifecycle({ components.selection.clearFireSelection(); if (layerState._dataSource) layerState._dataSource.show = false; if (layerState._billboards) layerState._billboards.show = false; + layerState._heatImagery?.setShow(false); overlayHost.clearSource(FIRMS_OVERLAY_SOURCE_ID); overlayHost.setVisible(FIRMS_OVERLAY_SOURCE_ID, false); clearSelectedEntityContextForLayer(id); @@ -96,6 +114,12 @@ export function createLifecycle({ layerState._fires = []; layerState._firesByFrp = []; layerState._cellCacheByGrid.clear(); + clearTimeout(layerState._heatRebuildTimer); + layerState._heatRebuildTimer = null; + layerState._heatImagery?.destroy(); + layerState._heatImagery = null; + layerState._heatRaster = null; + layerState._imageryHost = null; layerState._count = 0; layerState._totalCount = 0; layerState._rowControlsListener = null; diff --git a/src/layers/firms/model.js b/src/layers/firms/model.js index 83f573c98..c3d3be71a 100644 --- a/src/layers/firms/model.js +++ b/src/layers/firms/model.js @@ -136,36 +136,6 @@ export function createModel({ ); } - /** - * Severity stop for an aggregated heat cell from its heat-normalized score - * (density, night share and strongest FRP together). Cells say *where* it - * burns, so they stay on the warm end of the ramp — lime, amber, red — and - * read as one field instead of a per-cell mosaic; the cold stops belong to - * single detections, which say *how hard* each one burns. - * @param {number} normalized - 0..1 heat score (sqrt-normalized to the view). - * @returns {{name: string, minFrp: number, color: Cesium.Color}} - */ - - function cellColorStop(normalized) { - const name = - normalized > 0.72 ? 'red' : normalized > 0.42 ? 'amber' : 'lime'; - return ( - DETECTION_COLOR_STOPS.find((stop) => stop.name === name) || - DETECTION_COLOR_STOPS[0] - ); - } - - /** - * Fill color for an aggregated heat cell. - * @param {number} normalized - 0..1 heat score (see {@link cellColorStop}). - * @param {number} alpha - Fill alpha (heat-normalized by the caller). - * @returns {Cesium.Color} - */ - - function heatColor(normalized, alpha) { - return cellColorStop(normalized).color.withAlpha(alpha); - } - /** * Pick a sprite color stop for one detection from its FRP on the shared ramp. * @param {Object} fire - Detection record. @@ -432,8 +402,8 @@ export function createModel({ /** * Card model for an aggregated heat cell, e.g. title "14 FIRES", detail - * "max 210 MW · new 3h". Accent comes from the candidate (heat-normalized - * score is only known at renderCells time). Exported for unit tests. + * "max 210 MW · new 3h". Accent comes from the candidate (renderHeat sets + * it with {@link cellCardAccent}). Exported for unit tests. * @param {{cell: Object, position: Cesium.Cartesian3, accent: string}} candidate * @param {number} nowMs - Current epoch milliseconds. * @returns {Object} firmsLabels entry. @@ -494,14 +464,14 @@ export function createModel({ } /** - * Cell card accent: the warm severity stop of the cell's heat score, - * mirroring heatColor. - * @param {number} normalized - 0..1 heat score. + * Cell card accent: the cell's strongest fire on the FRP ramp, the same + * color its markers take once the camera is close enough to show them. + * @param {{maxFrp?: number}} cell - Aggregated cell. * @returns {string} "r, g, b" accent string. */ - function cellAccent(normalized) { - return accentForSeverity(cellColorStop(normalized).name); + function cellCardAccent(cell) { + return accentForSeverity(frpColorStop(cell?.maxFrp ?? 0).name); } /** Coordinate line like "30.512°N 75.831°E". */ @@ -607,8 +577,6 @@ export function createModel({ boundsContainPoint, cellIntersectsBounds, heatScore, - heatColor, - cellColorStop, frpColorStop, detectionColorStop, frpPixelSize, @@ -623,7 +591,7 @@ export function createModel({ buildFireCard, buildCellCard, applyFirmsOverlayPolicy, - cellAccent, + cellCardAccent, formatLatLon, formatFrp, formatAge, diff --git a/src/layers/firms/policy.js b/src/layers/firms/policy.js index 18ea88238..eb235bc81 100644 --- a/src/layers/firms/policy.js +++ b/src/layers/firms/policy.js @@ -7,9 +7,10 @@ import * as Cesium from 'cesium'; export const REFRESH_INTERVAL_MS = 600_000; /** - * LOD bands keyed by camera height. `cells` bands render aggregated - * ground-clamped heat rectangles; `detections` bands (< ~750km plus the - * local band) render individual fire detections as glow-sprite billboards. + * LOD bands keyed by camera height. `heat` bands (above ~3,000 km) drape a + * Snap Map–style heatmap over the globe (firmsHeatRaster.js); their + * `gridDegrees` aggregation only feeds the "N FIRES" cards. `detections` + * bands render individual fire detections as glow-sprite billboards. * Labels in every band go through the shared screen-space greedy declutter * (hard cap MAX_AMBIENT_LABELS) — there are no per-band label knobs. */ @@ -18,7 +19,7 @@ export const LOD_LEVELS = [ { id: 'global', minHeight: 9000000, - mode: 'cells', + mode: 'heat', gridDegrees: 2.0, maxCells: 1800, labelDistance: 12000000, @@ -26,7 +27,7 @@ export const LOD_LEVELS = [ { id: 'regional', minHeight: 3000000, - mode: 'cells', + mode: 'heat', gridDegrees: 1.0, maxCells: 3600, labelDistance: 8500000, @@ -47,6 +48,25 @@ export const LOD_LEVELS = [ }, ]; +/** + * Global heat raster for the `heat` bands: ≈0.18° per pixel, Gaussian σ in + * pixels. Built once per data refresh or filter change, never per camera move. + */ + +export const HEAT_RASTER = Object.freeze({ + width: 2048, + height: 1024, + sigmaPx: 3, +}); + +/** + * Trailing delay before a stale heatmap is rebuilt while one is already on + * screen: a slider drag fires many filter changes, and a build plus PNG + * encode costs a few hundred milliseconds. The first paint is immediate. + */ + +export const HEAT_REBUILD_MS = 150; + export const LOD_CHECK_MS = 650; /** +/-10% hysteresis on LOD band edges so slow zooms don't thrash rebuilds. */ @@ -90,7 +110,7 @@ export const CULL_LIFT_THRESHOLD_M = 10; export const CULL_LIFT_M = 12; /** - * Color stops shared by cell heat fills and detection glow sprites: the + * Color stops for detection glow sprites and card accents: the * six-stop cold → hot FRP ramp (firmsIntensity.js), coldest first, as Cesium * colors. Stop names key the glow-sprite cache and the card accents. */ diff --git a/src/layers/firms/queries.js b/src/layers/firms/queries.js index 44ce8573d..797734597 100644 --- a/src/layers/firms/queries.js +++ b/src/layers/firms/queries.js @@ -9,7 +9,7 @@ import { sliderPositionForFrp, } from '../../data/firmsIntensity.js'; import { formatNumber, t } from '../../i18n/index.js'; -import { REFRESH_INTERVAL_MS } from './policy.js'; +import { LOD_LEVELS, REFRESH_INTERVAL_MS } from './policy.js'; /** Slider resolution: whole-percent positions along the log axis. */ const SLIDER_STEPS = 1000; @@ -309,6 +309,19 @@ export function createQueries({ return result; }, + /** Test seam: render one named LOD band against a near-global view. */ + _renderLodForTest(lodId) { + const lod = LOD_LEVELS.find((level) => level.id === lodId); + if (lod.mode === 'detections') + components.rendering.renderDetections(lod, null); + else components.rendering.renderHeat(lod, null); + }, + + /** Test seam: entities drawn by the layer's data source. */ + _entityCountForTest() { + return layerState._dataSource?.entities?.values?.length ?? 0; + }, + /** Test seam that binds the production click path and indexes. */ _bindInteractionForTest(viewer, fires = []) { layerState._viewer = viewer; diff --git a/src/layers/firms/rendering.js b/src/layers/firms/rendering.js index 8ae445f91..84a6eed36 100644 --- a/src/layers/firms/rendering.js +++ b/src/layers/firms/rendering.js @@ -1,7 +1,14 @@ import * as Cesium from 'cesium'; import { FIRMS_OVERLAY_SOURCE_ID } from '../../data/firmsLabels.js'; import { horizonOccluder } from '../../data/iconOrientation.js'; -import { LOD_LEVELS, CONTEXT_TOP_N } from './policy.js'; +import { buildHeatRaster } from '../../data/firmsHeatRaster.js'; +import { + LOD_LEVELS, + CONTEXT_TOP_N, + HEAT_RASTER, + HEAT_REBUILD_MS, +} from './policy.js'; +import { createHeatImagery } from './heatImagery.js'; export function createRendering({ layerState, @@ -29,8 +36,11 @@ export function createRendering({ !layerState._dataSource || !layerState._fires.length || !layerState._viewer - ) + ) { + // A filter that leaves nothing shown must not leave the old heat behind. + if (!layerState._fires.length) layerState._heatImagery?.setShow(false); return false; + } const lodIndex = components.viewport.selectLodIndex( components.viewport.cameraHeight(), ); @@ -54,7 +64,7 @@ export function createRendering({ if (lod.mode === 'detections') { renderDetections(lod, bounds); } else { - renderCells(aggregateFires(lod, bounds), lod, bounds); + renderHeat(lod, bounds); } return true; } @@ -118,65 +128,89 @@ export function createRendering({ return clipped; } + /** The heat imagery adapter, created on first use. */ + + function heatImagery() { + if (!layerState._heatImagery) { + layerState._heatImagery = createHeatImagery({ + cesium: Cesium, + getHost: () => + layerState._imageryHost?.() ?? { + collection: layerState._viewer?.imageryLayers ?? null, + kind: 'globe', + }, + }); + } + return layerState._heatImagery; + } + + /** Build the heat raster from the shown set and drape it. */ + + function rebuildHeatRaster() { + clearTimeout(layerState._heatRebuildTimer); + layerState._heatRebuildTimer = null; + if (layerState._destroyed) return; + layerState._heatRaster = buildHeatRaster(layerState._fires, HEAT_RASTER); + heatImagery().setRaster(layerState._heatRaster); + } + /** - * Render aggregated heat cells (global/regional bands) as ground-clamped - * rectangles. Cell labels do NOT live on the entities anymore — they go - * through the same unclamped, screen-space-decluttered label pipeline as - * detections (see {@link rebuildAmbientLabels}). - * @param {Array} cells - Aggregated cells, heat-sorted descending. + * Make sure the draped heatmap matches the shown set. The first paint is + * immediate; later rebuilds (filter drags, refreshes) trail by + * HEAT_REBUILD_MS and keep the previous heatmap on screen meanwhile. + */ + + function ensureHeatRaster() { + if (layerState._heatRaster) return; + if (!heatImagery().hasRaster()) { + rebuildHeatRaster(); + return; + } + clearTimeout(layerState._heatRebuildTimer); + layerState._heatRebuildTimer = setTimeout( + rebuildHeatRaster, + HEAT_REBUILD_MS, + ); + } + + /** + * Far bands (global/regional): a Snap Map–style heatmap draped over the + * globe, transparent where nothing burns, plus the decluttered "N FIRES" + * cards from the coarse aggregation. No entities and no sprites are drawn; + * cards go through the same screen-space label pipeline as detections + * (see {@link rebuildAmbientLabels}). * @param {Object} lod - Active LOD descriptor. * @param {?Object} bounds - Padded view bounds in degrees, or null. */ - function renderCells(cells, lod, bounds) { + function renderHeat(lod, bounds) { ensureDetectionCollections(); layerState._dataSource.entities.removeAll(); if (layerState._billboards) layerState._billboards.removeAll(); layerState._pickIndexById.clear(); layerState._cullPositions.length = 0; - layerState._cellCount = cells.length; - const maxScore = Math.max(1, ...cells.map(components.model.heatScore)); + const imagery = heatImagery(); + // The map stack may have changed since the last far render. + imagery.rehome(); + ensureHeatRaster(); + imagery.setShow(layerState._enabled); + + const cells = aggregateFires(lod, bounds); + layerState._cellCount = cells.length; layerState._labelCandidates = []; layerState._labelLodDistance = lod.labelDistance; + // Cells arrive heat-sorted, so candidate order doubles as label priority. for (const cell of cells) { - const score = components.model.heatScore(cell); - const normalized = Math.min(1, Math.sqrt(score / maxScore)); - const alpha = 0.16 + normalized * 0.5; - const color = components.model.heatColor(normalized, alpha); const centerLon = cell.lonCell + lod.gridDegrees / 2; const centerLat = cell.latCell + lod.gridDegrees / 2; - const position = Cesium.Cartesian3.fromDegrees(centerLon, centerLat, 0); - - layerState._dataSource.entities.add({ - rectangle: { - coordinates: Cesium.Rectangle.fromDegrees( - cell.lonCell, - cell.latCell, - cell.lonCell + lod.gridDegrees, - cell.latCell + lod.gridDegrees, - ), - material: new Cesium.ColorMaterialProperty(color), - heightReference: Cesium.HeightReference.CLAMP_TO_GROUND, - }, - position, - properties: { - count: cell.count, - intensity: score, - maxFrp: cell.maxFrp, - night: cell.night, - }, - }); - - // Cells arrive heat-sorted, so candidate order doubles as label priority. - // Accent is resolved here because `normalized` only exists in this walk. layerState._labelCandidates.push({ - position, + position: Cesium.Cartesian3.fromDegrees(centerLon, centerLat, 0), cullPosition: components.model.cellCullPosition(centerLon, centerLat), fire: null, cell, - accent: components.model.cellAccent(normalized), + accent: components.model.cellCardAccent(cell), }); } @@ -202,6 +236,7 @@ export function createRendering({ */ function renderDetections(lod, bounds) { + layerState._heatImagery?.setShow(false); ensureDetectionCollections(); layerState._dataSource.entities.removeAll(); if (!layerState._billboards) return; @@ -345,7 +380,7 @@ export function createRendering({ return { renderCurrentLod, aggregateFires, - renderCells, + renderHeat, renderDetections, ensureDetectionCollections, removeDetectionCollections, diff --git a/src/layers/firms/state.js b/src/layers/firms/state.js index d84ee2026..9a0b031b2 100644 --- a/src/layers/firms/state.js +++ b/src/layers/firms/state.js @@ -131,6 +131,25 @@ export function createFirmsState({ services, config }) { layerState._cellCacheByGrid = new Map(); + /** + * Heat raster for the far bands (firmsHeatRaster.js); null = stale, rebuilt + * on the next far render. Cleared with the cell cache on every new shown set. + */ + + layerState._heatRaster = null; + + /** Pending debounced heat rebuild (see HEAT_REBUILD_MS). */ + + layerState._heatRebuildTimer = null; + + /** Imagery adapter (heatImagery.js), created on the first far render. */ + + layerState._heatImagery = null; + + /** Shell-provided imagery host resolver; falls back to viewer.imageryLayers. */ + + layerState._imageryHost = null; + /** Camera idle snapshot so the throttled LOD check is ~free when parked. */ layerState._camSnapValid = false; diff --git a/src/ui/layerBindings.js b/src/ui/layerBindings.js index fb8f29119..60128f8e0 100644 --- a/src/ui/layerBindings.js +++ b/src/ui/layerBindings.js @@ -141,6 +141,12 @@ export class LayerBindings { }); this._weatherShellModules.push(layer); } + // Active Fires drapes its far-zoom heatmap on the same imagery host. + const fires = this._dataManager?.layers?.get('local-firms')?.module; + if (typeof fires?.attachShellServices === 'function') { + fires.attachShellServices({ imageryHost: this.services.imageryHost }); + this._weatherShellModules.push(fires); + } } _persistAwarenessSelection(event, cleared = false) { From 52e428955f4c56249bec1573a542169026561ed2 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Thu, 29 Oct 2026 17:17:06 +0400 Subject: [PATCH 18/83] docs(fires): describe the far-zoom heatmap Co-Authored-By: Claude Opus 5.5 --- changelog.d/1426-fire-intensity-filter.md | 6 +++--- docs/CURRENT-STATE.md | 19 ++++++++++++++----- 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/changelog.d/1426-fire-intensity-filter.md b/changelog.d/1426-fire-intensity-filter.md index 09e3c3a7d..188f964ad 100644 --- a/changelog.d/1426-fire-intensity-filter.md +++ b/changelog.d/1426-fire-intensity-filter.md @@ -2,8 +2,8 @@ cold → hot ramp (blue · teal · lime · amber · orange · red at 0 · 5 · 20 · 50 · 100 · 300 MW), and add a log-scale minimum-FRP slider whose track is the ramp, an All / Nominal+ / High confidence switch and an "N of M shown" - counter to the layer row. Far-zoom heat cells keep a warm density fill - (lime · amber · red). The filter applies once before rendering, so - markers, cards, heat cells, context and the top-fires query agree; share + counter to the layer row. Far zoom shows a Snap Map–style heatmap instead + of squares, empty where nothing burns. The filter applies once before + rendering, so markers, cards, the heatmap, context and the top-fires query agree; share links keep it (`w` token: `f`, `c`), and `get_active_fires` takes matching `min_frp_mw` and `confidence` arguments. Default 0 MW / All (gekh, #1426). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 7e29d8675..336b32132 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2060,10 +2060,19 @@ Every FIRMS detection is colored by fire radiative power on a six-stop cold → hot ramp: blue from 0 MW, teal from 5, lime from 20, amber from 50, orange from 100 and red from 300 MW (`FRP_RAMP` in `src/data/firmsIntensity.js`). Glow sprites and fire-card accents read that ramp; confidence and brightness do -not affect the color. Aggregated heat cells (far zoom) instead take the warm -end of the ramp — lime, amber or red — from their heat-normalized score -(density, night share and strongest FRP), so a region reads as one field -rather than a per-cell mosaic; the cell card's accent matches its fill. +not affect the color. + +At far zoom (camera above ~3,000 km) the layer drapes a Snap Map–style +heatmap instead of individual markers: the shown detections are binned on a +global 2048 × 1024 grid, each weighted `1 + log10(1 + FRP)`, Gaussian-blurred, +normalized on a log scale against the 99.5th percentile and mapped from blue +through cyan, green, yellow and orange to red. Nothing is painted where +nothing burns (`src/data/firmsHeatRaster.js`). It is one full-globe imagery +tile on the active imagery host (`src/layers/firms/heatImagery.js`), rebuilt +on each data refresh or filter change (the first paint at once, later ones +150 ms after the last change) and hidden once markers take over below +~750 km. "N FIRES" cards from a 2° / 1° aggregation still label the hottest +areas; their accent is the strongest fire's FRP color. The Active Fires row carries a "Fire intensity (FRP)" slider on a log scale whose track is painted with the ramp and ticked at the same six values (its @@ -2071,7 +2080,7 @@ value reads "All" or "≥ N MW"), a "Confidence" switch with All / Nominal+ / High, and an "N of M shown" counter. The default is 0 MW and All, because many of the strongest detections are rated low confidence. The filter runs once in `ingestion.applyFilter()` before rendering, so markers, -cards, heat cells, the shared context store, the top-fires query and the +cards, the heatmap, the shared context store, the top-fires query and the analyst records all see one filtered set; a selected fire that no longer passes is deselected. The layer state keeps both options under the `w` (`local-firms`) share token as `f` (whole megawatts, 0–10,000) and `c` (`a`, `n`, `h`), From 2c422d2f3820707cf86de8c0f0618d57aade8a84 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Thu, 29 Oct 2026 17:26:23 +0400 Subject: [PATCH 19/83] build(boundaries): register the fires heat raster and imagery modules Co-Authored-By: Claude Opus 5.5 --- scripts/package-boundaries.json | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index 17772b58e..894e70a68 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -1174,6 +1174,7 @@ "modules": [ "src/data/firmsAdapt.js", "src/data/firmsIntensity.js", + "src/data/firmsHeatRaster.js", "src/data/firmsLabels.js", "src/data/iconOrientation.js", "src/layers/firms/anchors.js", @@ -1185,6 +1186,7 @@ "src/layers/firms/policy.js", "src/layers/firms/queries.js", "src/layers/firms/rendering.js", + "src/layers/firms/heatImagery.js", "src/layers/firms/selection.js", "src/layers/firms/source.js", "src/layers/firms/state.js", @@ -1916,6 +1918,7 @@ "src/data/firmsAdapt.js", "src/data/firmsHeatmap.js", "src/data/firmsIntensity.js", + "src/data/firmsHeatRaster.js", "src/data/firmsLabels.js", "src/data/flights.js", "src/data/flowMatch.js", @@ -2076,6 +2079,7 @@ "src/layers/firms/policy.js", "src/layers/firms/queries.js", "src/layers/firms/rendering.js", + "src/layers/firms/heatImagery.js", "src/layers/firms/selection.js", "src/layers/firms/source.js", "src/layers/firms/state.js", @@ -2760,6 +2764,7 @@ "src/data/detectionPolicy.js", "src/data/firmsAdapt.js", "src/data/firmsIntensity.js", + "src/data/firmsHeatRaster.js", "src/data/firmsLabels.js", "src/data/flowMatch.js", "src/data/focusDeemphasis.js", @@ -2890,6 +2895,7 @@ "src/layers/firms/policy.js", "src/layers/firms/queries.js", "src/layers/firms/rendering.js", + "src/layers/firms/heatImagery.js", "src/layers/firms/selection.js", "src/layers/firms/source.js", "src/layers/firms/state.js", From d32a4f62d33ef6dfe78d8e2b6f9373625c14e8e0 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Thu, 29 Oct 2026 17:36:09 +0400 Subject: [PATCH 20/83] i18n(zh-Hant): fire filter strings follow zh-CN wording (s2tw) Co-Authored-By: Claude Opus 5.5 --- src/i18n/locales/zh-Hant.json | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/i18n/locales/zh-Hant.json b/src/i18n/locales/zh-Hant.json index c71924233..9dadb3a50 100644 --- a/src/i18n/locales/zh-Hant.json +++ b/src/i18n/locales/zh-Hant.json @@ -2868,16 +2868,16 @@ "routing busy — too many routes at once": "路線服務繁忙：同時請求的路線過多", "route proxy error": "路線代理出錯", "fire confidence filter|All": "全部", - "fire confidence filter|Nominal+": "中+", + "fire confidence filter|Nominal+": "標稱+", "fire confidence filter|High": "高", - "Every detection — many of the strongest fires are rated low confidence": "所有探測——許多最強的火點可信度為低", - "Nominal and high confidence detections": "中、高可信度的探測", - "High confidence detections only": "僅高可信度的探測", + "Every detection — many of the strongest fires are rated low confidence": "所有檢測——許多最強的火點置信度為低", + "Nominal and high confidence detections": "標稱和高置信度的檢測", + "High confidence detections only": "僅高置信度的檢測", "Fire intensity (FRP)": "火勢強度 (FRP)", "≥ {value} MW": "≥ {value} MW", - "Confidence": "可信度", - "Minimum fire radiative power, megawatts": "最小火輻射功率（百萬瓦）", + "Confidence": "置信度", + "Minimum fire radiative power, megawatts": "最小火輻射功率（兆瓦）", "fire intensity filter|All": "全部", - "Detection confidence": "探測可信度", + "Detection confidence": "檢測置信度", "{shown} of {total} shown": "顯示 {shown} / {total}" } From bef47ee7c7cca963c3eeb145da5aad0bb08263bf Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Thu, 29 Oct 2026 18:25:33 +0400 Subject: [PATCH 21/83] fix(fires): weight the far-zoom heatmap by fire radiative power Heat was a detection count, so raising the minimum-FRP slider removed most detections and the hot spots faded to blue even where strong fires still burned. Each detection now adds its FRP in megawatts (0.5 MW floor), so weak fires carry little heat and filtering them out keeps strong-fire areas hot. On the live payload, hot spots that stay yellow or hotter at >=100 MW rise from 7% to 17% and their median heat from 0.10 to 0.38. --- docs/CURRENT-STATE.md | 4 +++- src/data/firmsHeatRaster.js | 11 ++++++++++- src/data/firmsHeatRaster.test.mjs | 33 +++++++++++++++++++++++++++---- 3 files changed, 42 insertions(+), 6 deletions(-) diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index b34636004..1bd9c82cd 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2096,7 +2096,9 @@ not affect the color. At far zoom (camera above ~3,000 km) the layer drapes a Snap Map–style heatmap instead of individual markers: the shown detections are binned on a -global 2048 × 1024 grid, each weighted `1 + log10(1 + FRP)`, Gaussian-blurred, +global 2048 × 1024 grid, each weighted by its FRP in megawatts (so heat is +radiated power, and raising the FRP slider does not dim areas where strong fires +burn), Gaussian-blurred, normalized on a log scale against the 99.5th percentile and mapped from blue through cyan, green, yellow and orange to red. Nothing is painted where nothing burns (`src/data/firmsHeatRaster.js`). It is one full-globe imagery diff --git a/src/data/firmsHeatRaster.js b/src/data/firmsHeatRaster.js index 52a020671..516c7676f 100644 --- a/src/data/firmsHeatRaster.js +++ b/src/data/firmsHeatRaster.js @@ -35,8 +35,17 @@ export function heatColorAt(t) { ]; } +/** Weight of a detection with no or zero FRP, in megawatts. */ +export const HEAT_MIN_WEIGHT_MW = 0.5; + +/** + * Heat is radiated power, not a head count: each detection adds its FRP in + * megawatts. Weak fires carry little power, so raising the minimum-FRP slider + * removes most detections without dimming the areas where strong fires burn + * (a count-based heat collapsed there and the hot spots faded). + */ const defaultWeight = (fire) => - 1 + Math.log10(1 + Math.max(0, Number(fire.frp) || 0)); + Math.max(HEAT_MIN_WEIGHT_MW, Number(fire.frp) || 0); export function binFires(fires, { width, height, weight = defaultWeight }) { const grid = new Float32Array(width * height); diff --git a/src/data/firmsHeatRaster.test.mjs b/src/data/firmsHeatRaster.test.mjs index 704c83427..2b51785e6 100644 --- a/src/data/firmsHeatRaster.test.mjs +++ b/src/data/firmsHeatRaster.test.mjs @@ -1,6 +1,7 @@ import test from 'node:test'; import assert from 'node:assert/strict'; import { + HEAT_MIN_WEIGHT_MW, HEAT_PALETTE, heatColorAt, binFires, @@ -23,7 +24,7 @@ test('palette runs cold to hot and starts fully transparent', () => { }); test('binFires drops each detection into its equirectangular pixel', () => { - const grid = binFires([fire(89.9, -179.9, 0), fire(-89.9, 179.9, 0)], { + const grid = binFires([fire(89.9, -179.9, 1), fire(-89.9, 179.9, 1)], { width: 4, height: 2, }); @@ -35,9 +36,33 @@ test('binFires drops each detection into its equirectangular pixel', () => { ); }); -test('binFires weights strong fires slightly more', () => { - const grid = binFires([fire(0.1, 0.1, 999)], { width: 4, height: 2 }); - assert.ok(grid[2] > 3 && grid[2] < 5, String(grid[2])); +test('binFires weights each detection by its radiative power', () => { + const grid = binFires( + [fire(0.1, 0.1, 250), fire(0.2, 0.2, 0), fire(0.3, 0.3, Number.NaN)], + { width: 4, height: 2 }, + ); + assert.equal(grid[2], 250 + 2 * HEAT_MIN_WEIGHT_MW); +}); + +test('filtering out weak fires keeps a strong fire hot', () => { + const strong = fire(0.1, 0.1, 300); + const weak = Array.from({ length: 40 }, () => fire(0.1, 0.1, 2)); + const reference = Array.from({ length: 10 }, () => fire(20.1, 20.1, 100)); + const options = { width: 360, height: 180, sigmaPx: 2 }; + const pixel = ((90 - 0.1) | 0) * 360 + ((0.1 + 180) | 0); + const red = (r) => r.rgba[pixel * 4]; + const alpha = (r) => r.rgba[pixel * 4 + 3]; + const unfiltered = buildHeatRaster([strong, ...weak, ...reference], options); + const filtered = buildHeatRaster([strong, ...reference], options); + assert.ok(alpha(filtered) > 0); + assert.ok( + alpha(filtered) >= alpha(unfiltered) * 0.9, + `alpha ${alpha(unfiltered)} -> ${alpha(filtered)}`, + ); + assert.ok( + red(filtered) >= red(unfiltered) * 0.9, + `red ${red(unfiltered)} -> ${red(filtered)}`, + ); }); test('blurGaussian keeps mass and wraps across the antimeridian', () => { From a4c7b20ca825a731531b61fc2935f1a47621248a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Itallo=20Brand=C3=A3o?= <45429643+IMBrandao@users.noreply.github.com> Date: Thu, 29 Oct 2026 16:11:10 -0300 Subject: [PATCH 22/83] feat(transit): add Rio de Janeiro buses and BRT (SMTR GTFS-RT, CC BY 4.0) Register Rio's two keyless GTFS-Realtime VehiclePositions feeds from the city transport secretariat (SMTR): municipal buses (SPPO) and the Mobi-Rio BRT, the Transit layer's first Latin American region. Both feeds publish Position.speed in km/h instead of m/s. A new optional registry field, speedUnit: 'kmh', makes buildTransitSnapshot convert it once, so every speedMps consumer stays correct and other feeds are untouched. Document the source in DATA_SOURCES.md and update the region lists in README.md and docs/CURRENT-STATE.md. Refs #1480 Co-Authored-By: Claude Opus 5.5 --- DATA_SOURCES.md | 3 +- README.md | 2 +- docs/CURRENT-STATE.md | 8 ++++-- src/data/transitFeeds.js | 50 ++++++++++++++++++++++++++++++++++ src/data/transitFeeds.test.mjs | 43 +++++++++++++++++++++++++++++ src/data/transitProxy.js | 21 +++++++++++++- src/data/transitProxy.test.mjs | 26 ++++++++++++++++++ 7 files changed, 147 insertions(+), 6 deletions(-) diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index e31d0d3f0..2b989c509 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -95,6 +95,7 @@ How to read this: | **Barrie Transit** (myridebarrie.ca) | Transit layer live vehicles, Barrie ON | Barrie Transit realtime data, published by the City of Barrie under a licence accepted by use of the data. The licence is revocable and requires no attribution. | Courtesy credit to the City of Barrie (the licence requires none, and its trademark clause is broad) | | **YRT/Viva** (rtu.york.ca) | Transit layer live vehicles, York Region ON | York Region Transit realtime data, first-party York Region infrastructure under the YRT Open Data Licence v2.0, which permits commercial use and makes attribution optional. Viva route ids are named by colour from York Region’s published GTFS. | "Contains public transit Information made available under YRT’s Open Data Licence" — the licence’s own wording, given though not required | | **Burlington Transit** (opendata.burlington.ca) | Transit layer live vehicles, Burlington ON | Burlington Transit realtime data under the City of Burlington Open Data Terms of Use, which permit any lawful purpose. The terms are revocable and must travel with the data, so the credit links them. Route ids are internal keys and are joined to the City’s published GTFS. | Credit to the City of Burlington plus a link to its Open Data Terms of Use, which the licence requires | +| **SMTR Rio de Janeiro** (dados.mobilidade.rio) | Transit layer live vehicles, Rio de Janeiro: municipal buses (SPPO) and the Mobi-Rio BRT, as two feeds | CC BY 4.0, stated on both data.rio catalog pages. Keyless GTFS-Realtime published by the Secretaria Municipal de Transportes, labelled beta, with a gateway limit of 5 requests a second and 60 a minute. Speeds arrive in km/h rather than m/s and are converted by the proxy. Route ids are internal keys and are shown raw. | Attribution to SMTR / Prefeitura da Cidade do Rio de Janeiro under CC BY 4.0 | | **Radio Browser** | Geolocated internet-radio station directory and station-level tags | Public-domain directory data under PDDL 1.0; individual broadcaster stream terms apply | "Radio Browser" plus a link to the selected broadcaster | | **Re:Earth Terrain** (Mapterhorn) | Terrain (keyless globe stacks — OSM etc. — + `/api/terrain/heights` ellipsoidal-height lookups) | Terrain mesh: CC BY 4.0; geoid: EGM2008 (NGA, public domain) | "Terrain (keyless globe stacks): Re:Earth Terrain / Mapterhorn (CC BY 4.0) / EGM2008 (NGA)" | | **OSRM on the FOSSGIS routing servers** (`routing.openstreetmap.de`) | Street-following routes for the Directions layer and voice route annotations, via `/api/route` | [FOSSGIS routing usage policy](https://routing.openstreetmap.de/about.html): "Display the required attribution and display a link to 'fix the map'", "Use a valid user agent and, if applicable, a correct referrer", "One request per second max", "No scraping, no heavy usage". The full policy is the German [FOSSGIS Nutzungsbedingungen](https://www.fossgis.de/arbeitsgruppen/osm-server/nutzungsbedingungen/); FOSSGIS also states that the server may be embedded in your own pages but "eine gewerbliche Nutzung ist nur mit Einschränkungen erlaubt" (commercial use only with restrictions). Route data derives from OpenStreetMap (ODbL 1.0) | "Routing: OSRM on the FOSSGIS servers" + a "fix the map" link — shown in the Data attribution popover | @@ -177,7 +178,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei - **NYC DOT Traffic Management Center cameras.** Off by default. NYC DOT's [developer data feeds page](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) asks developers who want the camera feed to contact `tmcdot@dot.nyc.gov` and sign a data-sharing agreement. This project does not hold one that covers downstream installations, so the pack loads only when an operator who holds their own agreement sets `CCTV_NYC_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to NYC DOT. The endpoint needs no key. The camera list is the public `webcams.nyctmc.org/api/cameras` catalog (~980 cameras across the five boroughs); frames are stills at `webcams.nyctmc.org/api/cameras//image`, synthesized from a validated UUID rather than taken from the payload, and only that host is registered. Cameras the feed reports as offline are dropped. The catalog publishes no camera facing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. Once enabled, the 250 cameras nearest the borough cores load; `CCTV_NYC_MAX_SOURCES` (8–400) changes the cap. Frames are fetched at request time and never stored or redistributed. NYC DOT asks every application that uses its data to carry its disclaimer; it is registered verbatim in the Data attribution popover. - **City of Toronto RESCU cameras.** Keyless. The camera list is the City's public `tmcearthcameras.json` on `opendata.toronto.ca` (dataset "Traffic Cameras", Transportation Services), served as JSONP and unwrapped with an anchored pattern. This is the municipal network the Ontario 511 pack does not carry: downtown arterials plus the Gardiner, DVP, Allen and Lake Shore. The list publishes no frame URL, so frame URLs are BUILT from the documented `CameraImages/loc####.jpg` convention and a strictly-digits camera number, pinning the frame proxy to the portal origin by construction; the list fetch refuses redirects and bounds the body it buffers. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `D1`-`D4` fields are the directions of the *comparison* images (static reference stills captured in 2016 under a different path), not a bearing for the live pan-tilt-zoom frame, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. A camera that is temporarily out of service still answers 200 with a byte-identical "Image Unavailable" placeholder rather than an error (about 6% of the network at any moment), so such a camera reads as healthy; the frame it shows is the agency's, not a synthesized one. The licence requires the attribution statement above. Disable with `CCTV_TORONTO_ENABLED=0`. - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals. -- **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All sixteen registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. The TTC feed is BusTime, the surface system: it carries buses and streetcars and no subway at all. About four in ten of its vehicles report no route at all (unassigned or deadheading), and those are shown with the feed default rather than a route-established mode. It sends no ETag or Last-Modified, so its conditional requests fall through to the ordinary snapshot cache. Three Golden Horseshoe neighbours run alongside it — MiWay, Hamilton HSR and Durham Region Transit — each bus-only and each polled only near its own region. Durham Region Transit publishes no bearing on any vehicle, so those sprites take their heading from observed movement like any other feed that omits it. Guelph, Kingston and Barrie extend the same pattern eastward and north. Hamilton, Guelph, Burlington and YRT publish route ids that are not the number on the vehicle, so those entries carry a route-name table taken from each operator’s own published GTFS; any id missing from a table renders unchanged rather than blank. A label is never invented: YRT runs a route `9101` that appears nowhere in its published routes, so it stays as the feed sends it. Burlington’s is the one that cannot be shortcut: `351` is route 1 while `3510` is route 10, so the table is a real join rather than a trimmed prefix. GO Transit / Metrolinx is deliberately absent: its REALTIME feed requires an API key issued by manual review, which the keyless admission rule excludes (its static schedule download is keyless, but carries no vehicle positions). Barrie and YRT/Viva each invite acceptance of their licence through a form, and each licence also states that using the data is itself acceptance; this deployment relies on that clause, which is recorded in both entries. Grand River Transit in Waterloo is absent for a different reason: it answers, but its feed header was months stale and it publishes no per-vehicle timestamps, so it would render vehicles that look live and are not. Ottawa’s OC Transpo falls to the same keyless admission rule as GO Transit — its realtime feed requires an API key. London Transit and Thunder Bay Transit publish vehicle positions over plain HTTP with no HTTPS endpoint available. Every other Ontario system — Brampton, Oakville, Niagara, Milton, Windsor, Greater Sudbury, UP Express and a dozen smaller operators — publishes static GTFS schedules only, with no realtime vehicle positions to register. Between the nine included and these exclusions, every Ontario realtime vehicle-position feed catalogued in the Mobility Database is accounted for. +- **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All eighteen registered feeds, covering seventeen regions, are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. The TTC feed is BusTime, the surface system: it carries buses and streetcars and no subway at all. About four in ten of its vehicles report no route at all (unassigned or deadheading), and those are shown with the feed default rather than a route-established mode. It sends no ETag or Last-Modified, so its conditional requests fall through to the ordinary snapshot cache. Three Golden Horseshoe neighbours run alongside it — MiWay, Hamilton HSR and Durham Region Transit — each bus-only and each polled only near its own region. Durham Region Transit publishes no bearing on any vehicle, so those sprites take their heading from observed movement like any other feed that omits it. Guelph, Kingston and Barrie extend the same pattern eastward and north. Hamilton, Guelph, Burlington and YRT publish route ids that are not the number on the vehicle, so those entries carry a route-name table taken from each operator’s own published GTFS; any id missing from a table renders unchanged rather than blank. A label is never invented: YRT runs a route `9101` that appears nowhere in its published routes, so it stays as the feed sends it. Burlington’s is the one that cannot be shortcut: `351` is route 1 while `3510` is route 10, so the table is a real join rather than a trimmed prefix. GO Transit / Metrolinx is deliberately absent: its REALTIME feed requires an API key issued by manual review, which the keyless admission rule excludes (its static schedule download is keyless, but carries no vehicle positions). Barrie and YRT/Viva each invite acceptance of their licence through a form, and each licence also states that using the data is itself acceptance; this deployment relies on that clause, which is recorded in both entries. Grand River Transit in Waterloo is absent for a different reason: it answers, but its feed header was months stale and it publishes no per-vehicle timestamps, so it would render vehicles that look live and are not. Ottawa’s OC Transpo falls to the same keyless admission rule as GO Transit — its realtime feed requires an API key. London Transit and Thunder Bay Transit publish vehicle positions over plain HTTP with no HTTPS endpoint available. Every other Ontario system — Brampton, Oakville, Niagara, Milton, Windsor, Greater Sudbury, UP Express and a dozen smaller operators — publishes static GTFS schedules only, with no realtime vehicle positions to register. Between the nine included and these exclusions, every Ontario realtime vehicle-position feed catalogued in the Mobility Database is accounted for. - **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and non-array or empty successful payloads are rejected atomically. Individual structurally malformed rows are skipped rather than failing the whole query; the response carries their bounded `rejectedStationCount` (at most 10,000), which the client adds to its own rejected rows and reports as degraded coverage without discarding the accepted generation. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply. - **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 6,000 — sized so a 31-day month stays inside TomTom's 200K monthly free allowance; still a configurable application safety ceiling, not a guarantee; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app). - **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line. diff --git a/README.md b/README.md index 745bec3d8..965c0e283 100644 --- a/README.md +++ b/README.md @@ -403,7 +403,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi | 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 | | 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer with full-screen **EXPAND**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key | | 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 | -| 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland, and nine in Ontario: Toronto (surface vehicles; the subway publishes no positions), Mississauga, Hamilton, Durham Region, York Region, Guelph, Burlington, Barrie and Kingston | Operator GTFS-Realtime feeds | 🟢 | +| 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland, Rio de Janeiro (buses and BRT), and nine in Ontario: Toronto (surface vehicles; the subway publishes no positions), Mississauga, Hamilton, Durham Region, York Region, Guelph, Burlington, Barrie and Kingston | Operator GTFS-Realtime feeds | 🟢 | | 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 | | 🧭 **Directions** | Click A and B on the globe for a street-following drive, walk or cycle route draped on the terrain with turn-by-turn steps — then FLY the camera along it. No key, no geocoder, no mic | OSRM on FOSSGIS servers (OpenStreetMap) | 🟢 | | 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 | diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 966bb3878..2adac270e 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -4295,11 +4295,13 @@ Transit is off by default, and share links use token `j`. The movement panel ord places Transit directly after Street Traffic and before Bike Share. Cameras and utilities retain their own groups. The layer polls registered feeds near the camera every 15 seconds, below a 3,000 km altitude -gate. Sixteen keyless regions are available: Boston, Austin, Minneapolis–St Paul, -Helsinki, the Netherlands, Norway, South East Queensland, and nine in +gate. Seventeen keyless regions are available: Boston, Austin, Minneapolis–St Paul, +Helsinki, the Netherlands, Norway, South East Queensland, Rio de Janeiro +(municipal buses and the Mobi-Rio BRT, two feeds), and nine in Ontario — Toronto (TTC surface vehicles; the subway publishes no positions), MiWay, Hamilton HSR, Durham Region, YRT/Viva, Guelph, -Burlington, Barrie and Kingston. Mode silhouettes +Burlington, Barrie and Kingston. Rio's feeds publish speed in km/h, which +the proxy converts to m/s before any card reads it. Mode silhouettes identify buses, trams, subways, trains, ferries and unclassified vehicles. Retained transit history uses the same outlier quarantine as live observations and diff --git a/src/data/transitFeeds.js b/src/data/transitFeeds.js index b8476a6ee..e871a2e89 100644 --- a/src/data/transitFeeds.js +++ b/src/data/transitFeeds.js @@ -385,7 +385,12 @@ function burlingtonRouteLabel(routeId) { * url: string, headers?: Record, * license: string, licenseUrl: string, attribution: string, * defaultMode: string, routeMode?: (routeId: string|null) => string, + * speedUnit?: 'kmh', * }>>} + * + * `speedUnit: 'kmh'` marks a feed that publishes `Position.speed` in km/h + * instead of the m/s the spec requires; the proxy converts it before anything + * downstream reads `speedMps`. */ export const TRANSIT_FEED_REGISTRY = Object.freeze([ Object.freeze({ @@ -729,6 +734,51 @@ export const TRANSIT_FEED_REGISTRY = Object.freeze([ defaultMode: 'bus', routeLabel: burlingtonRouteLabel, }), + Object.freeze({ + id: 'smtr-rio-onibus', + name: 'Rio buses (SPPO)', + operator: 'Secretaria Municipal de Transportes (SMTR), Rio de Janeiro', + region: 'Rio de Janeiro, RJ', + // Every vehicle in a 2026-10-29 snapshot, from Santa Cruz in the west to + // the Zona Sul, sat within 31 km of this point. + center: Object.freeze({ lat: -22.92, lon: -43.42 }), + loadRadiusKm: 45, + url: 'https://dados.mobilidade.rio/gtfs/realtime/onibus/vehicle-positions', + license: 'CC BY 4.0', + licenseUrl: + 'https://www.data.rio/documents/997bc16cc4344a5cb0e48ab06cbdfd1f', + attribution: 'SMTR / Prefeitura da Cidade do Rio de Janeiro (CC BY 4.0)', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'This work is licensed under a Creative Commons Attribution 4.0 International License', + note: 'The data.rio catalog page labels the feed [beta] and states a gateway limit of 5 requests a second and 60 a minute behind a 10-second cache, which the 15-second proxy TTL stays inside. It is the named replacement for the JSON GPS endpoints SMTR deprecated on 2026-10-07 and says it will switch off by 2026-11-30. SMTR relays the concessionaires’ ITS data and repairs trips with its own matching; the page places accuracy with the suppliers. Speeds arrive in km/h, not m/s (a 2026-10-29 snapshot peaked at 89), hence `speedUnit`. Route ids are internal keys: `O0313AAA0A` carried service 313 in the same day’s JSON feed. They are shown raw until a published table is joined.', + }), + defaultMode: 'bus', + speedUnit: 'kmh', + }), + Object.freeze({ + id: 'smtr-rio-brt', + name: 'Mobi-Rio BRT', + operator: + 'Secretaria Municipal de Transportes (SMTR), Rio de Janeiro, for the Mobi-Rio BRT fleet', + region: 'Rio de Janeiro, RJ', + center: Object.freeze({ lat: -22.92, lon: -43.42 }), + loadRadiusKm: 45, + url: 'https://dados.mobilidade.rio/gtfs/realtime/brt/vehicle-positions', + license: 'CC BY 4.0', + licenseUrl: + 'https://www.data.rio/documents/8bb196cc02174d25b77437bb2f6443d4', + attribution: 'SMTR / Prefeitura da Cidade do Rio de Janeiro (CC BY 4.0)', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'This work is licensed under a Creative Commons Attribution 4.0 International License', + note: 'Same catalog terms, [beta] label and gateway limits as the bus feed. The fleet is operated by Mobi-Rio and tracked by SONDA, whose data SMTR relays. Speeds are km/h here too (peak 62 in the same snapshot). Route ids are opaque (`20000461130`) and are shown raw.', + }), + defaultMode: 'bus', + speedUnit: 'kmh', + }), ]); /** diff --git a/src/data/transitFeeds.test.mjs b/src/data/transitFeeds.test.mjs index fd990af45..51903b7db 100644 --- a/src/data/transitFeeds.test.mjs +++ b/src/data/transitFeeds.test.mjs @@ -482,3 +482,46 @@ test('Burlington is live and carries its terms link, which its licence requires' assert.match(bur.licenseUrl, /Open%20Data%20Terms%20of%20Use\.pdf$/); assert.match(bur.url, /^https:\/\//, 'port 80 has no listener at all'); }); + +test('Rio buses and BRT cover the city from Centro to Santa Cruz, and not São Paulo', () => { + const ids = (lat, lon) => + transitFeedsInRange(lat, lon).map((feed) => feed.id); + for (const [place, lat, lon] of [ + ['Centro', -22.9068, -43.1729], + ['Barra da Tijuca', -23.0004, -43.3659], + ['Santa Cruz', -22.9194, -43.6847], + ]) { + assert.ok( + ids(lat, lon).includes('smtr-rio-onibus'), + `buses reach ${place}`, + ); + assert.ok(ids(lat, lon).includes('smtr-rio-brt'), `BRT reaches ${place}`); + } + assert.deepEqual( + ids(-23.5505, -46.6333).filter((id) => id.startsWith('smtr-rio')), + [], + 'São Paulo, 360 km away, never polls Rio', + ); +}); + +test('Rio feeds are CC BY 4.0, live, and declare the km/h speeds they publish', () => { + for (const id of ['smtr-rio-onibus', 'smtr-rio-brt']) { + const feed = getTransitFeed(id); + assert.equal(feed.defaultEnabled, true, `${id} is live`); + assert.equal(feed.license, 'CC BY 4.0'); + assert.match(feed.licenseUrl, /^https:\/\/www\.data\.rio\/documents\//); + assert.match(feed.attribution, /Prefeitura da Cidade do Rio de Janeiro/); + assert.equal(feed.speedUnit, 'kmh', `${id} publishes km/h`); + assert.equal(transitModeFor(feed, null), 'bus'); + } + const sources = readFileSync( + new URL('../../DATA_SOURCES.md', import.meta.url), + 'utf8', + ); + assert.ok( + sources + .split('\n') + .some((line) => line.startsWith('| **SMTR Rio de Janeiro')), + 'Rio has a DATA_SOURCES row', + ); +}); diff --git a/src/data/transitProxy.js b/src/data/transitProxy.js index 458240e9d..2923c9c32 100644 --- a/src/data/transitProxy.js +++ b/src/data/transitProxy.js @@ -224,6 +224,25 @@ export function repairVehicleTimestamps(vehicles, headerTimestamp, fetchedAtS) { }); } +/** + * Bring a feed's speeds to the m/s the spec requires. A feed registered with + * `speedUnit: 'kmh'` publishes km/h in `Position.speed`, and read as m/s a + * 44 km/h bus would be reported at 158 km/h. Converting here, once, keeps every + * consumer of `speedMps` honest without each one knowing about the feed. + * + * @param {object} feed Registry entry. + * @param {object[]} vehicles Normalized vehicle records. + * @returns {object[]} + */ +export function normalizeFeedSpeeds(feed, vehicles) { + if (feed?.speedUnit !== 'kmh') return vehicles; + return vehicles.map((vehicle) => + Number.isFinite(vehicle.speedMps) + ? { ...vehicle, speedMps: vehicle.speedMps / 3.6 } + : vehicle, + ); +} + /** * Decode upstream bytes into the JSON snapshot the browser consumes. * @@ -257,7 +276,7 @@ export function buildTransitSnapshot(feed, bytes, now = Date.now()) { truncated: decoded.truncated, count: decoded.vehicles.length, vehicles: repairVehicleTimestamps( - decoded.vehicles, + normalizeFeedSpeeds(feed, decoded.vehicles), decoded.timestamp, Math.floor(now / 1000), ), diff --git a/src/data/transitProxy.test.mjs b/src/data/transitProxy.test.mjs index dea10ab25..445e658f2 100644 --- a/src/data/transitProxy.test.mjs +++ b/src/data/transitProxy.test.mjs @@ -137,6 +137,7 @@ function encodeFeed({ const position = new PbfWriter(); position.writeFloatField(1, entity.lat); position.writeFloatField(2, entity.lon); + if (entity.speed != null) position.writeFloatField(5, entity.speed); vehicle.writeBytesField(2, position.finish()); if (entity.timestamp) vehicle.writeVarintField(5, entity.timestamp); const descriptor = new PbfWriter(); @@ -284,6 +285,31 @@ test('the snapshot builder fills timestamps from the header it decoded', () => { assert.equal(snapshot.feedTimestamp, 1_700_000_000); }); +test('a feed that publishes km/h has its speeds converted to m/s, and only that feed', () => { + // Rio's SMTR feeds put km/h in Position.speed; read as m/s, a 36 km/h bus + // would be reported at 130 km/h. + const bytes = encodeFeed({ + entities: [ + { id: 'moving', lat: -22.9, lon: -43.2, speed: 36 }, + { id: 'still', lat: -22.9, lon: -43.2, speed: 0 }, + { id: 'silent', lat: -22.9, lon: -43.2 }, + ], + }); + const rio = buildTransitSnapshot(getTransitFeed('smtr-rio-onibus'), bytes, 1); + const speeds = Object.fromEntries( + rio.vehicles.map((vehicle) => [vehicle.id, vehicle.speedMps]), + ); + assert.ok(Math.abs(speeds.moving - 10) < 1e-6, '36 km/h is 10 m/s'); + assert.equal(speeds.still, 0); + assert.equal(speeds.silent, null, 'an absent speed stays absent'); + const mbta = buildTransitSnapshot(getTransitFeed('mbta'), bytes, 1); + assert.equal( + mbta.vehicles.find((vehicle) => vehicle.id === 'moving').speedMps, + 36, + 'a spec-conforming feed is left alone', + ); +}); + test('conditional-request validators and gzip ride along with feed identification', () => { const entur = getTransitFeed('entur-norway'); const plain = transitUpstreamHeaders(entur); From 53d30c9e80d765b14eb2a2c35a957168cad2bf4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Itallo=20Brand=C3=A3o?= <45429643+IMBrandao@users.noreply.github.com> Date: Thu, 29 Oct 2026 16:30:47 -0300 Subject: [PATCH 23/83] docs(changelog): add fragment for #1481 Co-Authored-By: Claude Opus 5.5 --- changelog.d/1481-rio-transit.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 changelog.d/1481-rio-transit.md diff --git a/changelog.d/1481-rio-transit.md b/changelog.d/1481-rio-transit.md new file mode 100644 index 000000000..62b4ede04 --- /dev/null +++ b/changelog.d/1481-rio-transit.md @@ -0,0 +1,4 @@ +- Add Rio de Janeiro to the Transit layer: municipal buses and the Mobi-Rio BRT + from SMTR's keyless GTFS-Realtime feeds (CC BY 4.0), the layer's first Latin + American region. A feed that publishes speed in km/h declares + `speedUnit: 'kmh'` and the proxy converts it to m/s (IMBrandao, #1481). From 096383b43266fb6b13d3cc471be4b80f74f7876a Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Fri, 30 Oct 2026 21:00:13 +0400 Subject: [PATCH 24/83] Translate the fire filter strings into de, fr and ja --- src/i18n/locales/de.json | 15 ++++++++++++++- src/i18n/locales/fr.json | 15 ++++++++++++++- src/i18n/locales/ja.json | 15 ++++++++++++++- src/i18n/same-as-english.json | 3 +++ 4 files changed, 45 insertions(+), 3 deletions(-) diff --git a/src/i18n/locales/de.json b/src/i18n/locales/de.json index 8e383aded..f18b5ca67 100644 --- a/src/i18n/locales/de.json +++ b/src/i18n/locales/de.json @@ -2926,5 +2926,18 @@ "Malformed cyclone snapshot": "Fehlerhafter Wirbelsturm-Snapshot", "STALE · Roads: {source} · Flow: cached TomTom tiles": "VERALTET · Straßen: {source} · Verkehrsfluss: TomTom-Kacheln aus dem Cache", "routing busy — too many routes at once": "Routing-Dienst ausgelastet – zu viele Routen gleichzeitig", - "route proxy error": "Fehler des Routen-Proxys" + "route proxy error": "Fehler des Routen-Proxys", + "fire confidence filter|All": "Alle", + "fire confidence filter|Nominal+": "Mittel+", + "fire confidence filter|High": "Hoch", + "Every detection — many of the strongest fires are rated low confidence": "Alle Erkennungen — viele der stärksten Brände haben eine niedrige Konfidenz", + "Nominal and high confidence detections": "Erkennungen mit mittlerer und hoher Konfidenz", + "High confidence detections only": "Nur Erkennungen mit hoher Konfidenz", + "Fire intensity (FRP)": "Intensität (FRP)", + "≥ {value} MW": "≥ {value} MW", + "Confidence": "Konfidenz", + "Minimum fire radiative power, megawatts": "Minimale Strahlungsleistung des Brandes in Megawatt", + "fire intensity filter|All": "Alle", + "Detection confidence": "Konfidenz der Erkennung", + "{shown} of {total} shown": "{shown} von {total} angezeigt" } diff --git a/src/i18n/locales/fr.json b/src/i18n/locales/fr.json index 931e7c3ff..59641d143 100644 --- a/src/i18n/locales/fr.json +++ b/src/i18n/locales/fr.json @@ -2996,5 +2996,18 @@ "ESC to dismiss": "ESC pour fermer", "Tip: the GEV MIC button in the dock lets you talk to the map.": "Astuce : le bouton GEV MIC du dock permet de parler à la carte.", "Malformed cyclone snapshot": "Instantané cyclonique mal formé", - "Cyclone request timed out": "Délai dépassé pour la requête cyclonique" + "Cyclone request timed out": "Délai dépassé pour la requête cyclonique", + "fire confidence filter|All": "Toutes", + "fire confidence filter|Nominal+": "Nominale+", + "fire confidence filter|High": "Élevée", + "Every detection — many of the strongest fires are rated low confidence": "Toutes les détections — beaucoup des feux les plus intenses ont une confiance faible", + "Nominal and high confidence detections": "Détections de confiance nominale et élevée", + "High confidence detections only": "Détections de confiance élevée uniquement", + "Fire intensity (FRP)": "Intensité (FRP)", + "≥ {value} MW": "≥ {value} MW", + "Confidence": "Confiance", + "Minimum fire radiative power, megawatts": "Puissance radiative minimale du feu, en mégawatts", + "fire intensity filter|All": "Toutes", + "Detection confidence": "Confiance de la détection", + "{shown} of {total} shown": "{shown} sur {total} affichées" } diff --git a/src/i18n/locales/ja.json b/src/i18n/locales/ja.json index 5a48dc886..85757c1dd 100644 --- a/src/i18n/locales/ja.json +++ b/src/i18n/locales/ja.json @@ -2856,5 +2856,18 @@ "routing busy — too many routes at once": "ルート検索サービスが混雑しています。同時に要求されたルートが多すぎます", "route proxy error": "ルート検索プロキシでエラーが発生しました", "Malformed cyclone snapshot": "サイクロンのスナップショットの形式が不正です", - "Cyclone request timed out": "サイクロンのリクエストがタイムアウトしました" + "Cyclone request timed out": "サイクロンのリクエストがタイムアウトしました", + "fire confidence filter|All": "すべて", + "fire confidence filter|Nominal+": "中以上", + "fire confidence filter|High": "高", + "Every detection — many of the strongest fires are rated low confidence": "すべての検知 — 最も強い火災の多くは信頼度が低と判定されています", + "Nominal and high confidence detections": "信頼度が中・高の検知", + "High confidence detections only": "信頼度が高の検知のみ", + "Fire intensity (FRP)": "強度（FRP）", + "≥ {value} MW": "≥ {value} MW", + "Confidence": "信頼度", + "Minimum fire radiative power, megawatts": "火災放射強度の最小値（メガワット）", + "fire intensity filter|All": "すべて", + "Detection confidence": "検知の信頼度", + "{shown} of {total} shown": "{total}件中{shown}件を表示" } diff --git a/src/i18n/same-as-english.json b/src/i18n/same-as-english.json index dd0df9990..390b56e58 100644 --- a/src/i18n/same-as-english.json +++ b/src/i18n/same-as-english.json @@ -163,6 +163,7 @@ "{value} KM/S", "ΔN", "Ⅱ PAUSE", + "≥ {value} MW", "▲ {frp} MW" ], "es": [ @@ -469,6 +470,7 @@ "ΔE", "ΔN", "Ⅱ PAUSE", + "≥ {value} MW", "▲ {frp} MW" ], "it": [ @@ -688,6 +690,7 @@ "{type} SPD: {speed} HDG: {heading}", "ΔE", "ΔN", + "≥ {value} MW", "▲ {frp} MW" ], "pt": [ From 18d55f58b9a756cb758996eb2593e6b2967a956d Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Mon, 2 Nov 2026 13:48:28 +0400 Subject: [PATCH 25/83] Color fires from ember to white-hot instead of blue to red About 90% of detections are under 20 MW, so the old blue and teal stops painted most of the map in cold colors that do not read as fire. The ramp now runs ember, red, flame, orange, gold and white-hot at the same thresholds, each stop lighter than the one below, so the brightest marks are the strongest fires. The heatmap uses the same colors. --- changelog.d/1426-fire-intensity-filter.md | 12 ++--- docs/CURRENT-STATE.md | 17 ++++--- src/data/firmsCards.test.mjs | 26 +++++----- src/data/firmsHeatRaster.js | 18 ++++--- src/data/firmsHeatRaster.test.mjs | 10 ++-- src/data/firmsIntensity.js | 27 +++++----- src/data/firmsIntensity.test.mjs | 62 ++++++++++++++--------- src/data/firmsLabels.js | 4 +- src/data/firmsLabels.test.mjs | 20 ++++---- src/layers/firms/model.js | 2 +- src/layers/firms/policy.js | 2 +- src/ui/layerPanel/layerRows.test.mjs | 2 +- 12 files changed, 115 insertions(+), 87 deletions(-) diff --git a/changelog.d/1426-fire-intensity-filter.md b/changelog.d/1426-fire-intensity-filter.md index 188f964ad..c1d06643e 100644 --- a/changelog.d/1426-fire-intensity-filter.md +++ b/changelog.d/1426-fire-intensity-filter.md @@ -1,9 +1,9 @@ - Color every Active Fires detection by fire radiative power on a six-stop - cold → hot ramp (blue · teal · lime · amber · orange · red at 0 · 5 · 20 · - 50 · 100 · 300 MW), and add a log-scale minimum-FRP slider whose track is the - ramp, an All / Nominal+ / High confidence switch and an "N of M shown" - counter to the layer row. Far zoom shows a Snap Map–style heatmap instead - of squares, empty where nothing burns. The filter applies once before - rendering, so markers, cards, the heatmap, context and the top-fires query agree; share + ember → white-hot ramp (ember · red · flame · orange · gold · white-hot at + 0 · 5 · 20 · 50 · 100 · 300 MW, brighter as power rises), and add a + log-scale minimum-FRP slider whose track is the ramp, an All / Nominal+ / + High confidence switch and an "N of M shown" counter to the layer row. Far + zoom shows a Snap Map–style heatmap in the same colors instead of squares, + empty where nothing burns. The filter applies once before rendering, so markers, cards, the heatmap, context and the top-fires query agree; share links keep it (`w` token: `f`, `c`), and `get_active_fires` takes matching `min_frp_mw` and `confidence` arguments. Default 0 MW / All (gekh, #1426). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index a7d941001..5644960bc 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2175,19 +2175,22 @@ accessible name remains "Don't show this again". Control behavior is unchanged. ## Active Fires intensity filter and color ramp -Every FIRMS detection is colored by fire radiative power on a six-stop cold → -hot ramp: blue from 0 MW, teal from 5, lime from 20, amber from 50, orange from -100 and red from 300 MW (`FRP_RAMP` in `src/data/firmsIntensity.js`). Glow -sprites and fire-card accents read that ramp; confidence and brightness do -not affect the color. +Every FIRMS detection is colored by fire radiative power on a six-stop ember → +white-hot ramp: dark ember from 0 MW, red from 5, flame from 20, orange from +50, gold from 100 and pale white-hot from 300 MW (`FRP_RAMP` in +`src/data/firmsIntensity.js`). Each stop is lighter than the one below and +none is a cold hue, so the brightest marks are the strongest fires; ~90% of +detections are under 20 MW and stay dim red. Glow sprites and fire-card +accents read that ramp; confidence and brightness do not affect the color. At far zoom (camera above ~3,000 km) the layer drapes a Snap Map–style heatmap instead of individual markers: the shown detections are binned on a global 2048 × 1024 grid, each weighted by its FRP in megawatts (so heat is radiated power, and raising the FRP slider does not dim areas where strong fires burn), Gaussian-blurred, -normalized on a log scale against the 99.5th percentile and mapped from blue -through cyan, green, yellow and orange to red. Nothing is painted where +normalized on a log scale against the 99.5th percentile and mapped through the +same colors, from a faint ember edge through red, orange and gold to a pale +white-hot core. Nothing is painted where nothing burns (`src/data/firmsHeatRaster.js`). It is one full-globe imagery tile on the active imagery host (`src/layers/firms/heatImagery.js`), rebuilt on each data refresh or filter change (the first paint at once, later ones diff --git a/src/data/firmsCards.test.mjs b/src/data/firmsCards.test.mjs index d0ca7bd70..9d6157324 100644 --- a/src/data/firmsCards.test.mjs +++ b/src/data/firmsCards.test.mjs @@ -45,7 +45,7 @@ test('buildFireCard: title carries FRP, detail carries conf/age/satellite', () = assert.equal(card.details.length, 1); assert.equal(card.details[0], 'high · 2h · N20'); assert.equal(card.selected, false); - assert.equal(card.accent, accentForSeverity('red'), 'FRP 1520 is red-hot'); + assert.equal(card.accent, accentForSeverity('white-hot'), 'FRP 1520 is white-hot'); assert.deepEqual(card.position, { x: 1, y: 2, z: 3 }, 'uses the candidate position untouched'); }); @@ -69,26 +69,26 @@ test('buildFireCard: power that rounds to 10 MW drops the decimal', () => { assert.equal(selected.title, 'FIRE · 10 MW'); }); -test('buildFireCard: SNPP satellite code renders as SNPP, weak fire is blue', () => { +test('buildFireCard: SNPP satellite code renders as SNPP, weak fire is an ember', () => { const card = buildFireCard({ fire: fire({ satellite: 'N', frp: 0.8, confidence: 0.3 }), position: {} }, NOW); assert.match(card.details[0], /SNPP$/); assert.equal(card.title, '▲ 0.8 MW'); - assert.equal(card.accent, accentForSeverity('blue')); + assert.equal(card.accent, accentForSeverity('ember')); }); test('buildFireCard: accent steps through the FRP ramp at 5 · 20 · 50 · 100 · 300 MW', () => { const accentAt = (frp) => buildFireCard({ fire: fire({ frp }), position: {} }, NOW).accent; - assert.equal(accentAt(4.9), accentForSeverity('blue')); - assert.equal(accentAt(5), accentForSeverity('teal')); - assert.equal(accentAt(20), accentForSeverity('lime')); - assert.equal(accentAt(50), accentForSeverity('amber')); - assert.equal(accentAt(100), accentForSeverity('orange')); - assert.equal(accentAt(300), accentForSeverity('red')); + assert.equal(accentAt(4.9), accentForSeverity('ember')); + assert.equal(accentAt(5), accentForSeverity('red')); + assert.equal(accentAt(20), accentForSeverity('flame')); + assert.equal(accentAt(50), accentForSeverity('orange')); + assert.equal(accentAt(100), accentForSeverity('gold')); + assert.equal(accentAt(300), accentForSeverity('white-hot')); // Confidence never moves the color: a low-confidence 864 MW fire stays red. assert.equal( buildFireCard({ fire: fire({ frp: 864, confidence: 0.3 }), position: {} }, NOW).accent, - accentForSeverity('red'), + accentForSeverity('white-hot'), ); }); @@ -109,12 +109,12 @@ test('buildCellCard: plural noun, max FRP and newest age, accent passthrough', ( const candidate = { cell: { count: 14, maxFrp: 210.4, newestAcqMs: NOW - 3 * H, night: 2 }, position: { x: 0, y: 0, z: 0 }, - accent: accentForSeverity('orange'), + accent: accentForSeverity('gold'), }; const card = buildCellCard(candidate, NOW); assert.equal(card.title, '14 FIRES'); assert.equal(card.details[0], 'max 210 MW · new 3h'); - assert.equal(card.accent, accentForSeverity('orange')); + assert.equal(card.accent, accentForSeverity('gold')); }); test('buildCellCard: the count picks the plural form of the language', () => { @@ -166,7 +166,7 @@ test('buildCellCard: singular noun and missing-age omission', () => { const card = buildCellCard(candidate, NOW); assert.equal(card.title, '1 FIRE'); assert.equal(card.details[0], 'max 9.9 MW'); - assert.equal(card.accent, accentForSeverity('blue'), 'missing accent defaults to the coldest stop'); + assert.equal(card.accent, accentForSeverity('ember'), 'missing accent defaults to the weakest stop'); }); test('FIRMS host policy keeps ambient cards bounded and selected cards protected', () => { diff --git a/src/data/firmsHeatRaster.js b/src/data/firmsHeatRaster.js index 516c7676f..985699d7e 100644 --- a/src/data/firmsHeatRaster.js +++ b/src/data/firmsHeatRaster.js @@ -8,14 +8,18 @@ /** Below this normalized heat nothing is painted: empty land stays empty. */ export const HEAT_FLOOR = 0.03; +/** + * The FRP marker ramp as a continuous glow: faint ember at the edges, a + * pale white-hot core where the most power burns. + */ export const HEAT_PALETTE = Object.freeze([ - { at: 0.0, rgb: [43, 108, 255], alpha: 0 }, - { at: 0.08, rgb: [43, 108, 255], alpha: 0.35 }, - { at: 0.25, rgb: [25, 195, 255], alpha: 0.55 }, - { at: 0.45, rgb: [47, 224, 122], alpha: 0.65 }, - { at: 0.65, rgb: [245, 230, 66], alpha: 0.75 }, - { at: 0.82, rgb: [255, 138, 31], alpha: 0.82 }, - { at: 1.0, rgb: [255, 42, 26], alpha: 0.88 }, + { at: 0.0, rgb: [158, 42, 27], alpha: 0 }, + { at: 0.08, rgb: [158, 42, 27], alpha: 0.35 }, + { at: 0.25, rgb: [210, 64, 31], alpha: 0.55 }, + { at: 0.45, rgb: [240, 106, 40], alpha: 0.65 }, + { at: 0.65, rgb: [251, 154, 46], alpha: 0.75 }, + { at: 0.82, rgb: [253, 200, 74], alpha: 0.82 }, + { at: 1.0, rgb: [255, 243, 176], alpha: 0.9 }, ]); export function heatColorAt(t) { diff --git a/src/data/firmsHeatRaster.test.mjs b/src/data/firmsHeatRaster.test.mjs index 2b51785e6..a5cf7eccf 100644 --- a/src/data/firmsHeatRaster.test.mjs +++ b/src/data/firmsHeatRaster.test.mjs @@ -11,16 +11,20 @@ import { const fire = (lat, lon, frp = 10) => ({ lat, lon, frp }); -test('palette runs cold to hot and starts fully transparent', () => { +test('palette runs ember to white-hot and starts fully transparent', () => { assert.equal(HEAT_PALETTE[0].at, 0); assert.equal(HEAT_PALETTE.at(-1).at, 1); assert.deepEqual(heatColorAt(0), [0, 0, 0, 0]); assert.equal(heatColorAt(0.01)[3], 0, 'below the floor nothing is painted'); const low = heatColorAt(0.15); const high = heatColorAt(1); - assert.ok(low[2] > low[0], 'low heat is blue'); - assert.ok(high[0] > high[2], 'the core is red'); + assert.ok(low[0] > low[1] && low[1] > low[2], 'low heat is an ember red'); + assert.ok(high[1] > low[1] && high[2] > low[2], 'the core is paler'); assert.ok(high[3] > low[3], 'hotter is more opaque'); + for (let i = 1; i <= 100; i++) { + const [r, , b] = heatColorAt(i / 100); + assert.ok(r >= b, `no cold hue at ${i}%`); + } }); test('binFires drops each detection into its equirectangular pixel', () => { diff --git a/src/data/firmsIntensity.js b/src/data/firmsIntensity.js index 417566346..0b14dd3bd 100644 --- a/src/data/firmsIntensity.js +++ b/src/data/firmsIntensity.js @@ -1,28 +1,31 @@ /** * @module firmsIntensity * @description Fire intensity policy shared by the Active Fires layer and the - * agent fire tool: the six-stop cold → hot FRP color ramp, the log-scale + * agent fire tool: the six-stop ember → white-hot FRP color ramp, the log-scale * slider that doubles as its legend, and the one filter every consumer * (markers, cards, heat cells, the top-fires query, `get_active_fires`) runs * before anything is rendered or selected. Pure — no Cesium, no DOM. * - * Hue order follows the Portolan FIRMS explorer (`FRP_RAMP`, stepped, not - * interpolated). The thresholds are GEV's own, set on a real week's - * percentiles so every color appears: Portolan's 0/10/50/200/500/1000 MW - * leaves one red fire a week (HAL-675). + * The ramp (`FRP_RAMP`, stepped, not interpolated) runs like a fire cooling + * down: a dim ember for weak fires through red, orange and gold to a pale + * white-hot core, lightness rising with power, so the brightest marks are the + * strongest fires. No cold hues: ~90% of detections are under 20 MW and a + * blue start painted most of the map blue. The thresholds are GEV's own, set + * on a real week's percentiles so every color appears: Portolan's + * 0/10/50/200/500/1000 MW leaves one hottest fire a week (HAL-675). */ /** - * Stepped FRP ramp, coldest first. A detection takes the LAST stop whose + * Stepped FRP ramp, weakest first. A detection takes the LAST stop whose * `minFrp` it reaches. Names key the glow-sprite cache and the card accents. */ export const FRP_RAMP = Object.freeze([ - Object.freeze({ name: 'blue', minFrp: 0, hex: '#4a5bd4' }), - Object.freeze({ name: 'teal', minFrp: 5, hex: '#39a0a8' }), - Object.freeze({ name: 'lime', minFrp: 20, hex: '#c9cf4a' }), - Object.freeze({ name: 'amber', minFrp: 50, hex: '#f2b134' }), - Object.freeze({ name: 'orange', minFrp: 100, hex: '#e8722c' }), - Object.freeze({ name: 'red', minFrp: 300, hex: '#d1382a' }), + Object.freeze({ name: 'ember', minFrp: 0, hex: '#9e2a1b' }), + Object.freeze({ name: 'red', minFrp: 5, hex: '#d2401f' }), + Object.freeze({ name: 'flame', minFrp: 20, hex: '#f06a28' }), + Object.freeze({ name: 'orange', minFrp: 50, hex: '#fb9a2e' }), + Object.freeze({ name: 'gold', minFrp: 100, hex: '#fdc84a' }), + Object.freeze({ name: 'white-hot', minFrp: 300, hex: '#fff3b0' }), ]); /** Slider ticks: the ramp thresholds, so slider, legend and markers agree. */ diff --git a/src/data/firmsIntensity.test.mjs b/src/data/firmsIntensity.test.mjs index 558068e67..c1d22948b 100644 --- a/src/data/firmsIntensity.test.mjs +++ b/src/data/firmsIntensity.test.mjs @@ -18,40 +18,54 @@ import { sliderPositionForFrp, } from './firmsIntensity.js'; -test('the ramp has six cold → hot stops at 0 · 5 · 20 · 50 · 100 · 300 MW in Portolan hue order', () => { +test('the ramp has six ember → white-hot stops at 0 · 5 · 20 · 50 · 100 · 300 MW', () => { assert.deepEqual( FRP_RAMP.map((stop) => [stop.name, stop.minFrp, stop.hex]), [ - ['blue', 0, '#4a5bd4'], - ['teal', 5, '#39a0a8'], - ['lime', 20, '#c9cf4a'], - ['amber', 50, '#f2b134'], - ['orange', 100, '#e8722c'], - ['red', 300, '#d1382a'], + ['ember', 0, '#9e2a1b'], + ['red', 5, '#d2401f'], + ['flame', 20, '#f06a28'], + ['orange', 50, '#fb9a2e'], + ['gold', 100, '#fdc84a'], + ['white-hot', 300, '#fff3b0'], ], ); assert.deepEqual([...FRP_SLIDER_TICKS], [0, 5, 20, 50, 100, 300]); assert.ok(Object.isFrozen(FRP_RAMP)); }); +test('every stop is warm and each is lighter than the one below', () => { + const luma = ([r, g, b]) => 0.2126 * r + 0.7152 * g + 0.0722 * b; + let previous = -1; + for (const stop of FRP_RAMP) { + const rgb = hexToRgb(stop.hex); + assert.ok(rgb[0] >= rgb[2], `${stop.name} has no cold hue`); + assert.ok( + luma(rgb) > previous, + `${stop.name} is lighter than the stop below`, + ); + previous = luma(rgb); + } +}); + test('a detection takes the last stop it reaches; garbage reads as 0 MW', () => { - assert.equal(frpRampStop(0).name, 'blue'); - assert.equal(frpRampStop(4.99).name, 'blue'); - assert.equal(frpRampStop(5).name, 'teal'); - assert.equal(frpRampStop(19.9).name, 'teal'); - assert.equal(frpRampStop(20).name, 'lime'); - assert.equal(frpRampStop(50).name, 'amber'); - assert.equal(frpRampStop(100).name, 'orange'); - assert.equal(frpRampStop(299.9).name, 'orange'); - assert.equal(frpRampStop(300).name, 'red'); - assert.equal(frpRampStop(1246).name, 'red'); - assert.equal(frpRampStop(NaN).name, 'blue'); - assert.equal(frpRampStop(-3).name, 'blue'); - assert.equal(frpRampStop(undefined).name, 'blue'); + assert.equal(frpRampStop(0).name, 'ember'); + assert.equal(frpRampStop(4.99).name, 'ember'); + assert.equal(frpRampStop(5).name, 'red'); + assert.equal(frpRampStop(19.9).name, 'red'); + assert.equal(frpRampStop(20).name, 'flame'); + assert.equal(frpRampStop(50).name, 'orange'); + assert.equal(frpRampStop(100).name, 'gold'); + assert.equal(frpRampStop(299.9).name, 'gold'); + assert.equal(frpRampStop(300).name, 'white-hot'); + assert.equal(frpRampStop(1246).name, 'white-hot'); + assert.equal(frpRampStop(NaN).name, 'ember'); + assert.equal(frpRampStop(-3).name, 'ember'); + assert.equal(frpRampStop(undefined).name, 'ember'); }); test('hex stops convert to the "r, g, b" accents the cards paint', () => { - assert.deepEqual(hexToRgb('#4a5bd4'), [74, 91, 212]); + assert.deepEqual(hexToRgb('#9e2a1b'), [158, 42, 27]); assert.deepEqual(hexToRgb('d1382a'), [209, 56, 42]); }); @@ -84,9 +98,9 @@ test('the log slider pins 0 MW at the left and runs 1 → 1000 MW; values round- test('the track gradient is the stepped ramp on the log axis, so it doubles as the legend', () => { const gradient = rampTrackGradient(); assert.match(gradient, /^linear-gradient\(90deg, /); - assert.match(gradient, /#4a5bd4 0\.00% 23\.3\d%/); - assert.match(gradient, /#39a0a8 23\.3\d% 43\.3\d%/); - assert.match(gradient, /#d1382a 82\.5\d% 100\.00%\)$/); + assert.match(gradient, /#9e2a1b 0\.00% 23\.3\d%/); + assert.match(gradient, /#d2401f 23\.3\d% 43\.3\d%/); + assert.match(gradient, /#fff3b0 82\.5\d% 100\.00%\)$/); assert.equal(gradient.match(/#/g).length, FRP_RAMP.length); }); diff --git a/src/data/firmsLabels.js b/src/data/firmsLabels.js index 3af6a15a5..bf97982c2 100644 --- a/src/data/firmsLabels.js +++ b/src/data/firmsLabels.js @@ -36,8 +36,8 @@ const ACCENT_RGB = Object.freeze( ); /** - * Severity stop name → "r, g, b" accent string (defaults to the coldest stop). - * @param {string} stopName - A FRP_RAMP stop name ('blue' … 'red'). + * Severity stop name → "r, g, b" accent string (defaults to the weakest stop). + * @param {string} stopName - A FRP_RAMP stop name ('ember' … 'white-hot'). * @returns {string} */ export function accentForSeverity(stopName) { diff --git a/src/data/firmsLabels.test.mjs b/src/data/firmsLabels.test.mjs index 62234bb2d..a141d0d17 100644 --- a/src/data/firmsLabels.test.mjs +++ b/src/data/firmsLabels.test.mjs @@ -9,16 +9,16 @@ import { formatFrpMw, } from './firmsLabels.js'; -test('FIRMS card accents are the six FRP ramp stops, coldest as the fallback', () => { - // Portolan hue order; blue #4a5bd4 … red #d1382a (HAL-675). - assert.equal(accentForSeverity('blue'), '74, 91, 212'); - assert.equal(accentForSeverity('teal'), '57, 160, 168'); - assert.equal(accentForSeverity('lime'), '201, 207, 74'); - assert.equal(accentForSeverity('amber'), '242, 177, 52'); - assert.equal(accentForSeverity('orange'), '232, 114, 44'); - assert.equal(accentForSeverity('red'), '209, 56, 42'); - assert.equal(accentForSeverity('yellow'), accentForSeverity('blue')); - assert.equal(accentForSeverity('chartreuse'), accentForSeverity('blue')); +test('FIRMS card accents are the six FRP ramp stops, weakest as the fallback', () => { + // Ember #9e2a1b … white-hot #fff3b0, lightness rising with power. + assert.equal(accentForSeverity('ember'), '158, 42, 27'); + assert.equal(accentForSeverity('red'), '210, 64, 31'); + assert.equal(accentForSeverity('flame'), '240, 106, 40'); + assert.equal(accentForSeverity('orange'), '251, 154, 46'); + assert.equal(accentForSeverity('gold'), '253, 200, 74'); + assert.equal(accentForSeverity('white-hot'), '255, 243, 176'); + assert.equal(accentForSeverity('blue'), accentForSeverity('ember')); + assert.equal(accentForSeverity('chartreuse'), accentForSeverity('ember')); }); test('FIRMS power that rounds to 10 MW drops the decimal', () => { diff --git a/src/layers/firms/model.js b/src/layers/firms/model.js index 330d237e1..aefb94b21 100644 --- a/src/layers/firms/model.js +++ b/src/layers/firms/model.js @@ -121,7 +121,7 @@ export function createModel({ } /** - * The ramp stop for a fire radiative power (stepped, coldest first). + * The ramp stop for a fire radiative power (stepped, weakest first). * Intensity alone drives the color: confidence feeds the card text and the * switch, brightness is unreliable (saturates at 367 K) — see HAL-675. * @param {number} frp - Megawatts. diff --git a/src/layers/firms/policy.js b/src/layers/firms/policy.js index eb235bc81..faa55bbc3 100644 --- a/src/layers/firms/policy.js +++ b/src/layers/firms/policy.js @@ -111,7 +111,7 @@ export const CULL_LIFT_M = 12; /** * Color stops for detection glow sprites and card accents: the - * six-stop cold → hot FRP ramp (firmsIntensity.js), coldest first, as Cesium + * six-stop ember → white-hot FRP ramp (firmsIntensity.js), weakest first, as Cesium * colors. Stop names key the glow-sprite cache and the card accents. */ diff --git a/src/ui/layerPanel/layerRows.test.mjs b/src/ui/layerPanel/layerRows.test.mjs index 5e1d543d0..966c41cb6 100644 --- a/src/ui/layerPanel/layerRows.test.mjs +++ b/src/ui/layerPanel/layerRows.test.mjs @@ -75,7 +75,7 @@ const fireRow = (value, active = 'all') => ({ max: 1000, step: 1, value, - track: 'linear-gradient(90deg, #4a5bd4, #d1382a)', + track: 'linear-gradient(90deg, #9e2a1b, #fff3b0)', ticks: [ { at: 0, label: '0' }, { at: 0.5, label: '30' }, From 314292a006e6fda186f55c8434c9fdf97ff2cb54 Mon Sep 17 00:00:00 2001 From: German Khokhlov Date: Mon, 2 Nov 2026 14:10:57 +0400 Subject: [PATCH 26/83] Brighten the heatmap edge so weak fire regions stay visible The heat scale is set by the hottest regions, so most of the world sits at the low end. The ember edge was a dark red at 35% opacity and nearly vanished into forest and savanna imagery: over the Americas the spots were there but hard to see. The edge is now a bright red starting at 58% opacity, which reads at least as well as the old blue did. --- docs/CURRENT-STATE.md | 5 +++-- src/data/firmsHeatRaster.js | 21 ++++++++++++--------- src/data/firmsHeatRaster.test.mjs | 3 ++- 3 files changed, 17 insertions(+), 12 deletions(-) diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 5644960bc..dabc074ae 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2189,8 +2189,9 @@ global 2048 × 1024 grid, each weighted by its FRP in megawatts (so heat is radiated power, and raising the FRP slider does not dim areas where strong fires burn), Gaussian-blurred, normalized on a log scale against the 99.5th percentile and mapped through the -same colors, from a faint ember edge through red, orange and gold to a pale -white-hot core. Nothing is painted where +same hues, from a bright red edge through orange and gold to a pale +white-hot core. The edge starts more than half opaque, so regions far below +the hottest ones still read over forest and savanna imagery. Nothing is painted where nothing burns (`src/data/firmsHeatRaster.js`). It is one full-globe imagery tile on the active imagery host (`src/layers/firms/heatImagery.js`), rebuilt on each data refresh or filter change (the first paint at once, later ones diff --git a/src/data/firmsHeatRaster.js b/src/data/firmsHeatRaster.js index 985699d7e..f6aadff3b 100644 --- a/src/data/firmsHeatRaster.js +++ b/src/data/firmsHeatRaster.js @@ -9,17 +9,20 @@ export const HEAT_FLOOR = 0.03; /** - * The FRP marker ramp as a continuous glow: faint ember at the edges, a - * pale white-hot core where the most power burns. + * The FRP marker ramp as a continuous glow, from a red edge through orange + * and gold to a pale white-hot core. The edge is a bright red, not the + * markers' dark ember, and starts more than half opaque: most of the world + * sits low on a scale set by the hottest regions, and a dark translucent red + * vanishes into forest and savanna imagery. */ export const HEAT_PALETTE = Object.freeze([ - { at: 0.0, rgb: [158, 42, 27], alpha: 0 }, - { at: 0.08, rgb: [158, 42, 27], alpha: 0.35 }, - { at: 0.25, rgb: [210, 64, 31], alpha: 0.55 }, - { at: 0.45, rgb: [240, 106, 40], alpha: 0.65 }, - { at: 0.65, rgb: [251, 154, 46], alpha: 0.75 }, - { at: 0.82, rgb: [253, 200, 74], alpha: 0.82 }, - { at: 1.0, rgb: [255, 243, 176], alpha: 0.9 }, + { at: 0.0, rgb: [232, 58, 30], alpha: 0 }, + { at: 0.08, rgb: [232, 58, 30], alpha: 0.58 }, + { at: 0.25, rgb: [242, 86, 34], alpha: 0.7 }, + { at: 0.45, rgb: [248, 122, 40], alpha: 0.76 }, + { at: 0.65, rgb: [252, 166, 52], alpha: 0.82 }, + { at: 0.82, rgb: [254, 208, 92], alpha: 0.87 }, + { at: 1.0, rgb: [255, 245, 190], alpha: 0.92 }, ]); export function heatColorAt(t) { diff --git a/src/data/firmsHeatRaster.test.mjs b/src/data/firmsHeatRaster.test.mjs index a5cf7eccf..25c8b61fb 100644 --- a/src/data/firmsHeatRaster.test.mjs +++ b/src/data/firmsHeatRaster.test.mjs @@ -18,7 +18,8 @@ test('palette runs ember to white-hot and starts fully transparent', () => { assert.equal(heatColorAt(0.01)[3], 0, 'below the floor nothing is painted'); const low = heatColorAt(0.15); const high = heatColorAt(1); - assert.ok(low[0] > low[1] && low[1] > low[2], 'low heat is an ember red'); + assert.ok(low[0] > low[1] && low[1] > low[2], 'low heat is red'); + assert.ok(low[3] >= 0.5 * 255, 'low heat stays visible on any imagery'); assert.ok(high[1] > low[1] && high[2] > low[2], 'the core is paler'); assert.ok(high[3] > low[3], 'hotter is more opaque'); for (let i = 1; i <= 100; i++) { From af461febae51c176770163c4ec104f8f981287e8 Mon Sep 17 00:00:00 2001 From: manjunath22466 <301209272+manjunath22466@users.noreply.github.com> Date: Mon, 2 Nov 2026 18:39:44 +0530 Subject: [PATCH 27/83] feat(voice): add typed AI chat beside voice Chat sits next to the mic in the command dock. Replies are silent and never request the microphone; the first message connects with the selected provider and runs the same map actions as voice through each provider's text API (OpenAI OPENAI_CHAT_MODEL, Gemini GEMINI_CHAT_MODEL via generateContent, which never starts Live). Only one mode runs at a time. - Chat panel rendered by the voice control presenter: conversation, recent and quick-command suggestions, Up/Down history, attachments (map-layer references or small UTF-8 text files, sent only with the message) - Chat and voice panels sit above the dock or dock into the right rail's column; minimized bars sit centred above the dock, and height animates on swaps - A centred panel steps down to its bar for the Location and Visual Presets trays without moving focus, rests above an open tray, and docks right while expanded over one; it returns to the centre once the trays close - Minimized voice previews the last sentence from each side; the empty Voice Preview suggests things to say - Settings opened from chat keep chat's frame and show the shared AI provider and quick commands; voice Settings show Back to voice - Closing chat ends a typed session; the voice panel's close turns voice off - The dock widens its voice control for the chat button and restyles its wings, which show icons only at 900px and below; assistant panels share the left panels' glass, and Cyber keeps its own surfaces - Server text-chat handlers with body, time and per-IP limits (GEV_RATELIMIT_*_CHAT_PER_MIN), documented in SECURITY.md; tests and all eleven locales Co-authored-by: Darren Dignam <22195071+darrendignam@users.noreply.github.com> --- .env.example | 11 + README.md | 2 + SECURITY.md | 3 +- docs/CURRENT-STATE.md | 30 + scripts/format-scope.json | 20 +- scripts/package-boundaries.json | 35 + server/providers/common/textChat.js | 136 ++ server/providers/common/textInstructions.js | 31 + server/providers/common/textTools.js | 54 + server/providers/gemini.js | 5 + server/providers/gemini/chat.js | 172 ++ server/providers/openai.js | 5 + server/providers/openai/chat.js | 139 ++ server/providers/openai/instructions.js | 12 +- src/geminiChatServer.test.mjs | 162 ++ src/geminiComposition.test.mjs | 3 +- src/geminiServer.test.mjs | 5 +- src/i18n/locales/ar.json | 64 +- src/i18n/locales/de.json | 64 +- src/i18n/locales/es.json | 64 +- src/i18n/locales/fr.json | 64 +- src/i18n/locales/it.json | 64 +- src/i18n/locales/ja.json | 64 +- src/i18n/locales/pt.json | 64 +- src/i18n/locales/ru.json | 64 +- src/i18n/locales/uk.json | 64 +- src/i18n/locales/zh-CN.json | 64 +- src/i18n/locales/zh-Hant.json | 64 +- src/i18n/same-as-english.json | 10 + src/textChatServer.test.mjs | 276 ++++ src/ui/stores/voice.js | 22 +- src/ui/styles/ai-chat.css | 1656 +++++++++++++++++++ src/ui/styles/command-dock-sliding.css | 3 +- src/ui/views/panels/ChatPanel.vue | 92 ++ src/ui/views/panels/PanelLayoutButtons.vue | 18 + src/ui/views/panels/VoiceControl.vue | 45 +- src/voice/chatAttachments.js | 137 ++ src/voice/chatControl.js | 387 +++++ src/voice/chatPreferences.js | 113 ++ src/voice/control.js | 81 +- src/voice/geminiSession.js | 32 +- src/voice/geminiTextChat.js | 5 + src/voice/geminiTextChat.test.mjs | 492 ++++++ src/voice/panel/host.test.mjs | 356 +++- src/voice/panel/manualPanel.js | 27 +- src/voice/panel/manualPanel.test.mjs | 9 +- src/voice/panelLayout.js | 384 +++++ src/voice/providerCommands.js | 8 +- src/voice/realtimeInput.js | 7 +- src/voice/realtimeSession.js | 41 +- src/voice/session.test.mjs | 45 + src/voice/sessionCommands.js | 91 +- src/voice/textChat.js | 383 +++++ src/voice/textProtocol.js | 153 ++ src/voice/voiceMiniPreview.js | 68 + src/voice/voiceMiniPreview.test.mjs | 92 ++ style.css | 1 + 57 files changed, 6500 insertions(+), 63 deletions(-) create mode 100644 server/providers/common/textChat.js create mode 100644 server/providers/common/textInstructions.js create mode 100644 server/providers/common/textTools.js create mode 100644 server/providers/gemini/chat.js create mode 100644 server/providers/openai/chat.js create mode 100644 src/geminiChatServer.test.mjs create mode 100644 src/textChatServer.test.mjs create mode 100644 src/ui/styles/ai-chat.css create mode 100644 src/ui/views/panels/ChatPanel.vue create mode 100644 src/ui/views/panels/PanelLayoutButtons.vue create mode 100644 src/voice/chatAttachments.js create mode 100644 src/voice/chatControl.js create mode 100644 src/voice/chatPreferences.js create mode 100644 src/voice/geminiTextChat.js create mode 100644 src/voice/geminiTextChat.test.mjs create mode 100644 src/voice/panelLayout.js create mode 100644 src/voice/textChat.js create mode 100644 src/voice/textProtocol.js create mode 100644 src/voice/voiceMiniPreview.js create mode 100644 src/voice/voiceMiniPreview.test.mjs diff --git a/.env.example b/.env.example index c9700c5be..08b0df8f2 100644 --- a/.env.example +++ b/.env.example @@ -61,6 +61,12 @@ CESIUM_ION_TOKEN= # ChatGPT/Codex OAuth session; the browser still receives only a short-lived # Realtime client secret. Do not prefix with VITE_. OPENAI_API_KEY= +# Typed chat uses Chat Completions, independently of the voice model. It always +# bills OPENAI_API_KEY; GEV_PREFER_CODEX_OAUTH below applies to voice only. +OPENAI_CHAT_MODEL=gpt-4.1-mini +# Typed chat requests (/api/openai/chat) per IP per minute, separate from the +# voice throttle; 0 disables it. Not a billing cap. +# GEV_RATELIMIT_OPENAI_CHAT_PER_MIN=60 # the default # Optional: forbid API-key voice sessions. Select ChatGPT OAuth in Provider # Settings; missing or expired Codex sign-in then fails closed. # GEV_PREFER_CODEX_OAUTH=true @@ -269,3 +275,8 @@ OVERPASS_UPSTREAMS= # frame-ancestors sources such as "https://example.com", or * for any page. # Unset, no page may frame it. Other pages of the app can never be framed. # GEV_EMBED_FRAME_ANCESTORS= + +# Gemini typed chat uses generateContent; voice continues to use Gemini Live. +GEMINI_CHAT_MODEL=gemini-3.5-flash-lite +# Typed chat requests (/api/gemini/chat) per IP per minute; 0 disables it. +# GEV_RATELIMIT_GEMINI_CHAT_PER_MIN=60 # the default diff --git a/README.md b/README.md index 3a4c5fc43..d308e9b15 100644 --- a/README.md +++ b/README.md @@ -327,6 +327,8 @@ The standalone app offers **OpenAI** and **Gemini** in the GEV MIC provider choo Changing provider ends the current conversation. If voice is on, the new provider starts with the same open-mic or push-to-talk mode; a held Space key is not carried over. Gemini uses the same map actions and captions, but has no OpenAI cost meter or MINI tier. When Gemini reports that its session has expired, start voice again for a new conversation. The provider chooser does not change the OpenAI-backed AI HUD summary. +Next to the mic, **CHAT** opens a typed chat with silent replies. It uses the selected provider and the same map actions, and never asks for the microphone. OpenAI chat needs an API key (`OPENAI_CHAT_MODEL`); Gemini chat uses a separate text model (`GEMINI_CHAT_MODEL`). Chat and voice panels sit above the dock or dock to the right. + > Voice can use an **OpenAI API key** or, on the same machine, an opt-in signed-in **ChatGPT/Codex OAuth** session. The API-key path remains the default. Selecting ChatGPT OAuth in Provider Settings reuses an existing local sign-in or starts the Codex browser sign-in flow and selects OAuth when it completes. The OpenAI key still drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move. The OAuth option is experimental. OpenAI documents ChatGPT plan usage for the [Responses API](https://developers.openai.com/siwc/token-sharing-open-source/models-and-inference); a supported third-party Realtime contract for reusing Codex tokens has not been confirmed. A successful token mint does not establish billing or support terms. diff --git a/SECURITY.md b/SECURITY.md index 4423eead3..b32c5b60f 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -85,6 +85,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur Allowing `maps.googleapis.com` in `script-src` also trusts its JSONP endpoints, which can bypass the script allowlist if an HTML-injection bug exists; this is an accepted limitation of loading the Maps JavaScript API, alongside the existing `'unsafe-eval'` allowance. - **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `