diff --git a/CHANGELOG.md b/CHANGELOG.md index aaa32009a..53a71d87e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,36 @@ # Changelog +- Stop one retired bikeshare feed from reporting the whole layer as failed. + City feeds are fetched independently, but failure was recorded in a single + flag that only a successful *activation* cleared — and an already-loaded city + never activates again. A dead feed re-entering camera range therefore latched + LOAD FAILED permanently over a map still drawing every other city's stations. + Failure is now tracked per city: a city that fails beside others still + drawing marks the layer DEGRADED, polls that all fail over previously drawn + points read STALE without advancing freshness, and LOAD FAILED is reported + only when nothing is left to draw. A city that recovers clears its own + failure, a failure is dropped once its city leaves range or the camera rises + above the activation altitude, and late results from superseded requests are + ignored. + + Repointed the three feeds whose operators had moved: Reddy Bikeshare (Buffalo) + to Social Bicycles, Tugo (Tucson) and Trinity Metro (Fort Worth) to PBSC. + Dropped CoGo (Columbus) and Broward B-cycle (Fort Lauderdale), whose systems + have wound down — CoGo's feed reports no stations and last moved a bike in + March 2025. The three repointed feeds were verified live. + +- Add Bike Share Toronto (1,069 stations) to the bikeshare registry. The system + publishes the same PBSC GBFS v2 feeds as the Austin, Honolulu and Chattanooga + entries, so it needs no new plumbing; a registry test now holds every city's + feed URLs against the proxy host and path allowlists, so a city the proxy + would refuse fails the suite instead of rendering an empty layer. + +- GBFS v3 station names use English translations when available, then the first + usable translation or short name. Empty or malformed names no longer appear as + `[object Object]` or interrupt station loading; legacy string names remain + supported in the globe and bike-share tools. Thanks to + [Kristóf Kékesi](https://github.com/KristofKekesi) (#61). + - Overpass and military-installation disk caches no longer serve answers from a previously configured `OVERPASS_UPSTREAMS` instance after it is repointed. @@ -160,6 +191,30 @@ ## [Unreleased] +- Add a keyless Air quality layer to the Weather group, with Environment and + Climate Change Canada's AQHI as the first registered provider. Readings carry + `{ provider, scale, value, band }` so another network can be added without + inventing a shared numeric meaning across indices that do not share one; the + row and readout both state that coverage is Canada only. The WEATHER card + shows the band legend and the highest readings currently in view. AQHI is + reported as an integer from 1 with an open-ended "10+", so readings round to + that published form rather than the decimal precision the API returns. + Readings older than six hours are dropped, and the layer reports `stale` + through `getStats()` (also when every observation was dropped) so feed + provenance cannot describe an aged feed as live. Voice can switch it on and + query readings by value, band, station and observation time. + +- Bound six upstream body reads (FIRMS CSV, CelesTrak TLE groups, adsb.lol + `/v2/mil`, TomTom flow tiles, OpenSky `states/all` and adsbdb lookups) with + streaming size caps that cancel the body at the cap or the route's deadline; + the adsb.lol military poll gains a deadline of its own. An oversized OpenSky + snapshot is not retried, and an OpenSky 429 starts its cooldown before the + body is read. + +- Keep later satellites when a TLE catalog entry is missing an orbital line. + The parser skips the incomplete set and resumes at the next complete named + set, instead of losing the rest of the group (#906). + - Satellite clouds keep Clouds only / Full as quick endpoints and add a continuous cloud-filter slider: 0% is Full, 100% is the existing brightness key, and intermediate values blend the keyed alpha. The slider commits on release to diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 993271ef5..eeb5645fa 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -35,6 +35,7 @@ How to read this: | **The Space Devs — Launch Library 2 v2.3** | Recent launch, payload, stage, and recovery metadata for Space Missions (30d) | [The Space Devs terms of use](https://github.com/TheSpaceDevs/Tutorials/blob/main/faqs/faq_TSD.md#terms-of-use): data may be used and shared in any form; avoid forwarding it without added value; attribution is encouraged (not mandatory). [Official API limits](https://ll.thespacedevs.com/docs/): 15 unauthenticated calls/hour; optional token | "Launch Library 2 — The Space Devs" (courtesy attribution) | | **Esri World Imagery** (ArcGIS Online tile service) | The keyless satellite basemap — the default landing when no Google/ion credential is configured, and the "Esri Satellite" map stack | [Esri Master Agreement](https://www.esri.com/en-us/legal/terms/full-master-agreement): the public World Imagery service is usable in public-facing apps with attribution; no key is required for this classic endpoint, but Esri governs and can change access — an app at scale should review current ArcGIS Location Platform terms | "Powered by Esri — Source: Esri, Maxar, Earthstar Geographics, and the GIS User Community" (provider carries the service's own credit line) | | **USGS** | Earthquakes | U.S. public domain | "Data courtesy of the U.S. Geological Survey" | +| **Environment and Climate Change Canada** (MSC GeoMet OGC API) | Air quality (AQHI) — Canadian station observations | [ECCC Data Servers End-use Licence](https://eccc-msc.github.io/open-data/licence/readme_en/) (open, attribution required, no endorsement implied) | "Contains information licensed under the Environment and Climate Change Canada Data Servers End-use Licence" | | **NASA GIBS** (WMTS tiles + Worldview Snapshots) and **NASA CMR** (granule catalog) | Recent Imagery layer: HLS S30/L30 (Sentinel-2, Landsat 8/9) and VIIRS true-colour tiles draped on the map, per-day thumbnails and PNG exports; the HLS granule catalog (dates, cloud, footprints) for a selected box | U.S. public domain (NASA data); keyless, browser-direct (CORS `*`); no bulk use — one 30-day catalog query per box, tiles bounded to the box, six tile requests in flight | "We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS)." — [gibs.earthdata.nasa.gov](https://gibs.earthdata.nasa.gov), [HLS](https://lpdaac.usgs.gov/products/hlss30v002/) | | **NIFC WFIGS** (Wildland Fire Interagency Geospatial Services, ArcGIS feature service) | Fire Perimeters layer: current interagency wildfire incident perimeters, containment, and incident facts | U.S. public domain (interagency wildland-fire data published through the [NIFC Open Data portal](https://data-nifc.opendata.arcgis.com/)); fetched keyless through `/api/fire-perimeters` (5-minute server cache) with `exceededTransferLimit` paging | "Wildfire perimeters: National Interagency Fire Center (WFIGS)" | | **InciWeb** (inciweb.wildfire.gov) | Per-incident "InciWeb ↗" card links to official incident pages (narratives, evacuation/closure notices) | U.S. government public incident information. Links are matched best-effort by incident name and state, with incident page origin and update times checked on selection. Same-origin `/api/fire-perimeters/inciweb` routes cache the catalog for 1 hour and publication metadata for 30 minutes; linked pages remain InciWeb content | "Incident information: InciWeb (inciweb.wildfire.gov)" | @@ -60,7 +61,8 @@ How to read this: | **Open Calgary** (`data.calgary.ca`) | CCTV camera catalog + frames, Calgary | [Open Government Licence – City of Calgary](https://data.calgary.ca/stories/s/Open-Calgary-Terms-of-Use/u45n-7awa) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – City of Calgary" | | **Statens vegvesen** (catalog `ogckart-sn1.atlas.vegvesen.no`, frames `kamera.atlas.vegvesen.no`, HLS `kamera.vegvesen.no`) | CCTV camera catalog + frames + live HLS video, Norway road cameras | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) distributed by Statens vegvesen" | | **NYC DOT Traffic Management Center** (`webcams.nyctmc.org`) | CCTV camera catalog + frames, New York City five boroughs | NYC DOT requires developers to sign a [data-sharing agreement](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) for the camera feed. **Off by default**; enable only under your own agreement | "NYC DOT Traffic Management Center" + NYC DOT's required application disclaimer (in the Data attribution popover) | -| **GBFS (Lyft / BCycle)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` | +| **GBFS (Lyft / BCycle / Social Bicycles / PBSC)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` | +| **Bike Share Toronto** (`toronto.publicbikesystem.net`) | Bikeshare station availability, Toronto | Operator GBFS feed. No licence is published: the City's open-data package declares none and the feed has no `license_url`. Attribution is given to the operator. | "Bike Share Toronto (Toronto Parking Authority)" | | **Oslo / Bergen / Trondheim Bysykkel** (Urban Sharing, `gbfs.urbansharing.com`) | Bikeshare availability, Oslo, Bergen and Trondheim city bikes. Keyless GBFS; the proxy sends the `Client-Identifier` header the operators ask for | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED ([Oslo](https://oslobysykkel.no/en/open-data/realtime), [Bergen](https://bergenbysykkel.no/en/apne-data/sanntid), [Trondheim](https://trondheimbysykkel.no/en/open-data/realtime)) | "Contains data under the Norwegian licence for Open Government data (NLOD) made available by Oslo Bysykkel, Bergen Bysykkel and Trondheim Bysykkel" | | **Kolumbus / Farte Bysykkel** (Entur Mobility, `api.entur.io/mobility/v2/gbfs`) | Bikeshare availability, Kolumbus Bysykkel (Stavanger/Nord-Jæren) and Farte Bysykkel (Skien/Porsgrunn). Keyless GBFS; the proxy sends the `ET-Client-Name` header Entur requires and is limited to Entur's GBFS paths | [NLOD](https://data.norge.no/nlod/en/2.0) per the [Entur Mobility API docs](https://developer.entur.org/pages-mobility-docs-mobility-v2) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) made available by Entur (Kolumbus Bysykkel, Farte Bysykkel)" | | **JCDecaux Cyclocity** (`api.cyclocity.fr`) | Bikeshare availability, Bysykkel Lillestrøm. Keyless GBFS; the proxy is limited to JCDecaux contract paths | [Licence Ouverte / Etalab](https://developer.jcdecaux.com/files/Open-Licence-fr.pdf) (the feed's `license_url`) — attribution REQUIRED | "Bysykkel Lillestrøm: JCDecaux, Licence Ouverte" | @@ -81,6 +83,14 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei ### Notes on the live sources +- **Air quality (ECCC AQHI).** Keyless. Read from the MSC GeoMet OGC API (`api.weather.gc.ca`): the `aqhi-stations` catalog plus `aqhi-observations-realtime`. **Coverage is Canada only** — ECCC is the single registered provider, and the layer row and readout card both say so. Roughly 134 stations across ECCC's five administrative zones (Atlantic, Quebec, Ontario, Prairies & North, Pacific & Yukon). AQHI publishes hourly, so readings poll every 20 minutes and the station catalog, which changes a few times a year, is refetched only every 12 hours. Observations are requested with `latest=true`; without it the collection returns the full history, roughly 2,000 rows nationally. Readings older than six hours are dropped rather than displayed as current. + + Readings are stored as `{ provider, scale, value, band }` rather than a bare number, because air-quality indices are not interchangeable: AQHI runs 1-10+ on health risk, the US AQI runs 0-500 on concentration breakpoints, and Europe's EAQI is a five-step category. Naming the index a value belongs to is what lets another network be added later without inventing a shared numeric meaning that does not exist. + + AQHI is reported by ECCC as an integer from 1 to 10 with an open-ended "10+" above that. The API returns the underlying decimal (1.08, 2.32); the layer rounds to the published form rather than displaying a precision the index does not claim. + + **Quebec is thinly represented**, and that is upstream rather than a filter here: the province runs its own Info-Smog program, so only a handful of Quebec stations appear in ECCC's national feed. The reporting station count also varies hour to hour as stations report or fall silent. + - **Google Maps Platform.** You supply your own API key and are bound by [Google's ToS](https://cloud.google.com/maps-platform/terms). Google Maps Content (tiles, geocodes, places) **may not be cached, stored, rehosted, or committed** — this app only ever uses it live, which is the compliant pattern. The "Google" attribution is displayed on the globe and must stay visible. Restrict your key (see [SECURITY.md](SECURITY.md)). - **OpenSky Network.** Its license is **non-commercial**, and operational use of the REST API in a live product can require a prior written agreement with OpenSky — even for non-profit/government use. If you deploy this commercially, contact OpenSky for your own terms. The flights layer is a toggle and runs anonymously by default. - **adsb.lol flight fallback.** When OpenSky is unavailable and no last-good OpenSky response exists, the server requests a cached, capped 250 nm adsb.lol point snapshot around the current camera subpoint. This is regional observed context, not worldwide completeness; provenance is exposed in the Flights stats/context row. Military ICAOs remain reconciled through the existing dedicated military registry rather than duplicated. diff --git a/README.md b/README.md index 554f9b57f..68da35027 100644 --- a/README.md +++ b/README.md @@ -309,6 +309,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi | 🚢 **Live Vessels** | Thousands of ships worldwide | AISStream | 🟡 | | 🛰️ **Satellites** | 838-object catalog, color-coded by class with a live legend — the **DENSE** chip drops in the whole Starlink shell | CelesTrak | 🟢 | | 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 | +| ◍ **Air quality** | Live AQHI on the published 1–10+ health-risk scale, with a Weather-panel card showing the band legend and the highest readings in view. **Canada only** — ECCC is the first registered provider | ECCC MSC GeoMet | 🟢 | | 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds | | 📹 **CCTV Mesh** | ~3,900 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | | 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 | diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 47ced056b..83606412a 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1192,6 +1192,28 @@ Sharing asks of clients of the Oslo, Bergen and Trondheim Bysykkel feeds their GBFS tree: `api.entur.io` to `/mobility/v2/gbfs/` and `api.cyclocity.fr` (Bysykkel Lillestrøm) to `/contracts/`. +## Upstream body bounds + +FIRMS CSV, CelesTrak TLE groups, adsb.lol military responses, TomTom flow +tiles, OpenSky states/all and adsbdb lookups now read their bodies through the +shared streaming readers in `src/sources/httpBody.js`, so a declared or observed body +over the cap is refused with `RESPONSE_TOO_LARGE` and cancelled rather than +buffered. TomTom flow tiles use `readResponseBytesCapped` alongside +GTFS-Realtime. Each read takes the route's abort signal, so a body still +streaming when the deadline fires is cancelled. Caps are bounds, not budgets, sized well +above measured payloads: FIRMS 64 MiB per satellite source, CelesTrak 16 MiB +per group, OpenSky `states/all` 8 MiB, adsb.lol `/v2/mil` 8 MiB, TomTom tiles +8 MiB and adsbdb lookups 1 MiB. Where a route already had a deadline (FIRMS +60 s, CelesTrak 20 s, TomTom 15 s, adsbdb 8 s, and each 10 s OpenSky +`states/all` attempt) it already bounded the body read, because aborting a +fetch signal also aborts its body; the capped reader takes the same signal. +The adsb.lol military poll, which had no deadline, now waits at most 12 +seconds for headers and body together. An oversized OpenSky snapshot is not +retried, and an OpenSky 429 starts its cooldown from the response headers and +cancels the body unread. A rejected or timed-out body is an upstream miss and +follows each route's existing error and cached-data paths. FIRMS map-key +status, the OpenSky token exchange and CCTV media keep their current handling. + ## Remaining local service modules Overpass query validation, geometry simplification, disk caching and upstream @@ -3200,13 +3222,14 @@ its criteria cannot be silently ignored. | Live AIS Vessels 🚢 | AISStream websocket | `src/data/aisLiveVessels.js` | `/api/vessels` | 60s (+800ms visibility pass) | | Mapped Installations ⌖ | OpenFreeMap military areas; optional configured Overpass names and Google Places search | `src/data/militaryInstallations.js` | OpenFreeMap tiles, `/api/military-installations`, `/api/google/text-search` | viewport-driven; transient errors back off, missing capability switches to tiles | | Earthquakes | USGS | `src/data/earthquakes.js` | — | 60s | +| Air quality | ECCC MSC GeoMet OGC API (AQHI, Canada only) | `src/layers/airQuality/` | — | 20 min readings, 12 h station catalog | | Satellites | CelesTrak | `src/data/satellites.js` | `/api/celestrak` | 120s | | Space Missions (30d) | Launch Library 2 + CelesTrak | `src/data/rocketLaunches.js` | `/api/launches` + `/api/celestrak/active` | 5 min | | Traffic | Selectable TomTom / OpenStreetMap / Hybrid roads; optional TomTom flow (BYOK) | `src/data/traffic.js` | browser-direct OpenFreeMap tiles; `/api/tomtom` for flow | viewport-driven; capped tile cache | | CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + NYC DOT TMC (opt-in) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) | | Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh | | Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) | -| Bikeshare 🚲 | GBFS (Lyft + BCycle + Urban Sharing, Entur, JCDecaux NO) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | +| Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto + Urban Sharing, Entur, JCDecaux NO) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | | Directions 🧭 | OSRM on FOSSGIS servers (OpenStreetMap) | `src/data/directions.js` | `/api/route` (`steps=1`) | on placement / mode change | | Datacenters ▣ | OSM extract (bundled) | `src/data/localLayers.js` | — | static | | Dams ▰ | OpenInfraMap/OSM extract (bundled) | `src/data/localLayers.js` | — | static | diff --git a/package.json b/package.json index 1c2e5896d..9d8761215 100644 --- a/package.json +++ b/package.json @@ -178,6 +178,7 @@ "./layers/directions": "./src/layers/directions/index.js", "./layers/directions/source": "./src/layers/directions/source.js", "./layers/cctv": "./src/layers/cctv/index.js", + "./layers/air-quality": "./src/layers/airQuality/index.js", "./layers/alpr": "./src/layers/alpr/index.js", "./layers/radio": "./src/layers/radio/index.js", "./layers/submarine-cables": "./src/layers/submarineCables/index.js", @@ -238,6 +239,7 @@ "./layers/alpr/source": "./src/layers/alpr/source.js", "./layers/bikeshare/source": "./src/layers/bikeshare/source.js", "./layers/cctv/source": "./src/layers/cctv/source.js", + "./layers/air-quality/source": "./src/layers/airQuality/source.js", "./layers/earthquakes/source": "./src/layers/earthquakes/source.js", "./layers/perimeters/source": "./src/layers/perimeters/source.js", "./layers/gnss/source": "./src/layers/gnss/source.js", diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 6b67601a5..1f3d5e507 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -366,6 +366,12 @@ "src/layers/vessels/tracking.js", "src/layers/vessels/ownership.test.mjs", "src/data/earthquakes.js", + "src/app/layers/airQuality.js", + "src/layers/airQuality/index.js", + "src/layers/airQuality/model.js", + "src/layers/airQuality/model.test.mjs", + "src/layers/airQuality/ownership.test.mjs", + "src/layers/airQuality/source.js", "src/layers/earthquakes/index.js", "src/layers/earthquakes/model.js", "src/layers/earthquakes/source.js", @@ -734,6 +740,7 @@ "src/search/nominatim.js", "src/search/nominatim.test.mjs", "src/sources/httpBody.js", + "src/sources/httpBody.test.mjs", "src/sources/overpass.js", "src/sources/nominatim.js", "server/providers/transit.js", diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index 8c8dd13ba..dbca4b3b9 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -123,7 +123,12 @@ "traffic-provider": { "runtime": "node", "exports": ["./server/providers/traffic"], - "modules": ["server/providers/traffic.js", "src/data/tomtomTiles.js"], + "modules": [ + "server/providers/traffic.js", + "server/providers/common/http.js", + "src/sources/httpBody.js", + "src/data/tomtomTiles.js" + ], "external": [] }, "traffic-source": { @@ -139,7 +144,12 @@ "firms-provider": { "runtime": "node", "exports": ["./server/providers/firms"], - "modules": ["server/providers/firms.js", "src/data/firmsCsv.js"], + "modules": [ + "server/providers/firms.js", + "server/providers/common/http.js", + "src/sources/httpBody.js", + "src/data/firmsCsv.js" + ], "external": [] }, "gbfs-provider": { @@ -2419,7 +2429,11 @@ "src/layers/installations/classLabel.js", "src/data/cameraFraming.js", "src/data/geoImportPins.js", - "src/data/geoImportImages.js" + "src/data/geoImportImages.js", + "src/app/layers/airQuality.js", + "src/layers/airQuality/index.js", + "src/layers/airQuality/model.js", + "src/layers/airQuality/source.js" ], "external": [ "hls.js", @@ -2978,7 +2992,11 @@ "src/ui/unitLabels.js", "src/layers/recentImagery/display.js", "src/layers/installations/classLabel.js", - "src/data/cameraFraming.js" + "src/data/cameraFraming.js", + "src/app/layers/airQuality.js", + "src/layers/airQuality/index.js", + "src/layers/airQuality/model.js", + "src/layers/airQuality/source.js" ], "external": [ "hls.js", @@ -3390,7 +3408,9 @@ "src/layers/perimeters/source.js", "src/layers/perimeters/records.js", "src/layers/submarineCables/bundledSource.js", - "src/layers/gnss/source.js" + "src/layers/gnss/source.js", + "src/layers/airQuality/model.js", + "src/layers/airQuality/source.js" ], "external": [] }, @@ -3790,5 +3810,24 @@ "src/data/layerStateTokenReservations.json" ], "external": [] + }, + "air-quality": { + "exports": ["./layers/air-quality"], + "modules": [ + "src/layers/airQuality/index.js", + "src/layers/airQuality/model.js", + "src/layers/airQuality/source.js", + "src/sources/httpBody.js" + ], + "external": ["cesium"] + }, + "air-quality-source": { + "exports": ["./layers/air-quality/source"], + "modules": [ + "src/layers/airQuality/model.js", + "src/layers/airQuality/source.js", + "src/sources/httpBody.js" + ], + "external": [] } } diff --git a/scripts/qa-air-quality.mjs b/scripts/qa-air-quality.mjs new file mode 100644 index 000000000..68e6bd505 --- /dev/null +++ b/scripts/qa-air-quality.mjs @@ -0,0 +1,171 @@ +/** + * qa-air-quality.mjs — headless proof for the Air quality layer. + * + * Checks what unit tests cannot: that the live ECCC feed answers, that readings + * reach the app in the provider-shaped form other networks will share, and that + * the Weather readout descriptor carries a legend and an in-view list. + * + * AQHI is reported by ECCC as an integer from 1 with "10+" above ten. The API + * returns the underlying decimal, so a reading arriving as 1.08 — or a station + * with no reading surfacing as a confident 1 — is a real defect this catches. + * + * Run: node scripts/qa-air-quality.mjs --url http://localhost:4173 + */ + +import puppeteer from 'puppeteer'; + +const argv = process.argv.slice(2); +const getOpt = (name, dflt) => { + const i = argv.indexOf(name); + return i >= 0 && argv[i + 1] ? argv[i + 1] : dflt; +}; +const APP_URL = getOpt('--url', 'http://localhost:4173'); + +const results = []; +function record(name, ok, detail) { + results.push({ name, ok, detail }); + const tag = ok ? '\x1b[32mPASS\x1b[0m' : '\x1b[31mFAIL\x1b[0m'; + console.log(` [${tag}] ${name}${detail ? ` — ${detail}` : ''}`); +} +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +async function main() { + let exitCode = 0; + const consoleErrors = []; + const browser = await puppeteer.launch({ + headless: 'new', + args: [ + '--use-gl=swiftshader', + '--enable-unsafe-swiftshader', + '--no-sandbox', + '--disable-dev-shm-usage', + ], + defaultViewport: { width: 1400, height: 900 }, + }); + + try { + const page = await browser.newPage(); + page.on('pageerror', (e) => consoleErrors.push(String(e).slice(0, 160))); + page.on('console', (m) => { + if (m.type() === 'error') consoleErrors.push(m.text().slice(0, 160)); + }); + + await page.goto(APP_URL, { waitUntil: 'domcontentloaded', timeout: 90000 }); + await page.waitForFunction( + () => window.__godsEyeView?.viewer && window.__godsEyeView?.dataManager, + { timeout: 90000 }, + ); + await sleep(1500); + + const stats = await page.evaluate(async () => { + const dm = window.__godsEyeView.dataManager; + await dm.setEnabled('air-quality', true); + const mod = dm.layers.get('air-quality')?.module; + if (!mod) return { missing: true }; + let s = null; + for (let i = 0; i < 45; i++) { + await new Promise((r) => setTimeout(r, 1000)); + s = mod.getStats(); + if (s.count > 0 || s.error) break; + } + return s; + }); + + const loaded = !stats?.missing && !stats?.error && stats?.count > 0; + record('stations report a current index', loaded, + `count=${stats?.count} error=${stats?.error}`); + if (!loaded) exitCode = 1; + + const records = await page.evaluate( + () => + window.__godsEyeView.dataManager + .layers.get('air-quality') + ?.module.getAnalystRecords(500) || [], + ); + + // The provider-neutral contract other networks will share. + const shaped = records.every( + (r) => r.provider && r.scale && Number.isFinite(r.value) && r.band, + ); + record('every reading carries provider, scale, value and band', shaped, + records.length + ? `sample={provider:${records[0].provider}, scale:${records[0].scale}, value:${records[0].value}, band:${records[0].band}}` + : 'no records'); + if (!shaped) exitCode = 1; + + const published = records.every((r) => Number.isInteger(r.value) && r.value >= 1); + record('every AQHI value is in the published integer form, floored at 1', published, + `values=${[...new Set(records.map((r) => r.value))].sort((a, b) => a - b).join(',')}`); + if (!published) exitCode = 1; + + const zones = [...new Set(records.map((r) => r.zone).filter(Boolean))].sort(); + record('coverage spans multiple ECCC administrative zones', zones.length >= 3, + `zones=${zones.join(',') || 'none'} stations=${records.length}`); + if (zones.length < 3) exitCode = 1; + + // The card's whole point is "highest readings in view", so put the camera + // over Canada before asserting on it — the default view is over Austin, + // where an empty in-view list is correct and proves nothing. + await page.evaluate(() => { + const gev = window.__godsEyeView; + const ellipsoid = gev.viewer.scene.globe.ellipsoid; + const d2r = Math.PI / 180; + try { + gev.viewer.camera.cancelFlight(); + } catch { + /* no flight in progress */ + } + gev.viewer.camera.setView({ + destination: ellipsoid.cartographicToCartesian({ + longitude: -100 * d2r, + latitude: 55 * d2r, + height: 4_000_000, + }), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + gev.viewer.scene.requestRender?.(); + }); + await sleep(2500); + + const controls = await page.evaluate( + () => + window.__godsEyeView.dataManager + .layers.get('air-quality') + ?.module.getRowControls?.() || null, + ); + const readoutOk = + controls?.readout === true && + Array.isArray(controls?.legend?.colors) && + controls.legend.colors.length === controls.legend.labels.length && + Array.isArray(controls?.list?.items) && + controls.list.items.length > 0 && + controls.list.items.every((item) => item.id && item.label && item.value) && + /Canada only/.test(controls?.summary?.coverage || ''); + record('the Weather readout lists the worst readings in view, with a legend', readoutOk, + controls + ? `bands=${controls.legend?.labels?.join('/')} inView=${controls.list?.items?.length} coverage="${controls.summary?.coverage}"` + : 'no descriptor'); + if (!readoutOk) exitCode = 1; + + record('no console errors', consoleErrors.length === 0, + consoleErrors.length ? consoleErrors.slice(0, 2).join(' | ') : 'clean'); + if (consoleErrors.length) exitCode = 1; + } catch (e) { + console.error('\x1b[31mHarness error:\x1b[0m', e); + exitCode = 3; + } finally { + await browser.close(); + } + + const pass = results.filter((r) => r.ok === true).length; + const fail = results.filter((r) => r.ok === false).length; + console.log('\n' + '─'.repeat(60)); + console.log(` RESULT: ${pass} passed, ${fail} failed`); + console.log('─'.repeat(60) + '\n'); + process.exit(exitCode || (fail > 0 ? 1 : 0)); +} + +main().catch((e) => { + console.error(e); + process.exit(3); +}); diff --git a/server/providers/aircraft/adsb-lol.js b/server/providers/aircraft/adsb-lol.js index f67afed1d..8fc40170d 100644 --- a/server/providers/aircraft/adsb-lol.js +++ b/server/providers/aircraft/adsb-lol.js @@ -1,3 +1,5 @@ +import { readResponseTextCapped } from '../common/http.js'; + /** * Vite plugin: adsb.lol military aircraft proxy with 12 s response cache. * @@ -25,6 +27,10 @@ export function adsbLolProxy() { let _cooldownStatus = 0; /** Response cache TTL (ms). */ const CACHE_MS = 12000; + /** Upstream deadline (ms); the fetch previously had none and could hang a poll. */ + const UPSTREAM_TIMEOUT_MS = 12000; + /** /v2/mil measures ~67 KB; a relayed error page is bounded by the same cap. */ + const ADSBLOL_MIL_MAX_RESPONSE_BYTES = 8 * 1024 * 1024; /** Cooldown after a 429 when upstream sends no usable Retry-After (ms). */ const RATE_LIMIT_COOLDOWN_MS = 30000; /** Cooldown after a 5xx (ms). */ @@ -98,11 +104,17 @@ export function adsbLolProxy() { ); return; } + const signal = AbortSignal.timeout(UPSTREAM_TIMEOUT_MS); const upstream = await fetch('https://api.adsb.lol/v2/mil', { headers: { 'User-Agent': 'gods-eye-view-adsblol-proxy/1.0' }, + signal, }); if (upstream.ok) { - const body = await upstream.text(); + const body = await readResponseTextCapped( + upstream, + ADSBLOL_MIL_MAX_RESPONSE_BYTES, + signal, + ); _cache = body; _cacheAt = Date.now(); _cooldownUntil = 0; @@ -129,7 +141,11 @@ export function adsbLolProxy() { return; } } - const body = await upstream.text(); + const body = await readResponseTextCapped( + upstream, + ADSBLOL_MIL_MAX_RESPONSE_BYTES, + signal, + ); serve( res, upstream.status, diff --git a/server/providers/aircraft/enrichment.js b/server/providers/aircraft/enrichment.js index 400050151..9b7c1b1bb 100644 --- a/server/providers/aircraft/enrichment.js +++ b/server/providers/aircraft/enrichment.js @@ -1,5 +1,9 @@ import path from 'node:path'; import { promises as fsp } from 'node:fs'; +import { readResponseJsonCapped } from '../common/http.js'; + +/** Route and aircraft lookups are a few hundred bytes; 1 MiB is a hard bound. */ +const ADSBDB_MAX_RESPONSE_BYTES = 1024 * 1024; /** * adsbdb.com enrichment proxy: callsign → route (airline + origin/destination * airports) and hex → aircraft type/registration. Free community API — cached @@ -80,12 +84,16 @@ export function adsbdbProxy() { kind === 'route' ? `https://api.adsbdb.com/v0/callsign/${encodeURIComponent(key)}` : `https://api.adsbdb.com/v0/aircraft/${encodeURIComponent(key)}`; - const res = await fetch(url, { signal: AbortSignal.timeout(8000) }); + const signal = AbortSignal.timeout(8000); + const res = await fetch(url, { signal }); if (res.ok) { + const payload = await readResponseJsonCapped( + res, + ADSBDB_MAX_RESPONSE_BYTES, + signal, + ); const data = - kind === 'route' - ? parseRoute(await res.json()) - : parseAircraft(await res.json()); + kind === 'route' ? parseRoute(payload) : parseAircraft(payload); store[key] = { at: Date.now(), data }; // data may be null — negative cache dirty = true; return data; diff --git a/server/providers/aircraft/opensky.js b/server/providers/aircraft/opensky.js index 187049aea..752b3cf64 100644 --- a/server/providers/aircraft/opensky.js +++ b/server/providers/aircraft/opensky.js @@ -2,6 +2,7 @@ import { normalizeAdsbLolPointResponse } from '../../../src/data/adsbLolFallback import { coalesceProxyRequest, readResponseJsonCapped, + readResponseTextCapped, } from '../common/http.js'; import { requiredFiniteQueryNumber } from '../common/query.js'; // --------------------------------------------------------------------------- @@ -29,11 +30,34 @@ const OPENSKY_CACHE_MS = 9000; const OPENSKY_ATTEMPT_TIMEOUT_MS = 10_000; const OPENSKY_STATES_URL = 'https://opensky-network.org/api/states/all?extended=1'; +/** states/all?extended=1 measures ~0.8 MB anonymously; bounded, not budgeted. */ +const OPENSKY_STATES_MAX_RESPONSE_BYTES = 8 * 1024 * 1024; + +/** + * Credit-governor cooldown for a 429: OpenSky's retry-after bounded to + * 30 s … 30 min, or 2 min when the header is absent. + * @param {Response} upstream Rate-limited response. + * @returns {number} Cooldown (ms). + */ +function openSkyRateLimitCooldownMs(upstream) { + // headers.get() gives null when absent, and Number(null) is 0, not NaN. + const header = upstream.headers.get('x-rate-limit-retry-after-seconds'); + const retryAfterSec = + header == null || header.trim() === '' ? NaN : Number(header); + return Math.min( + Math.max( + Number.isFinite(retryAfterSec) ? retryAfterSec * 1000 : 120_000, + 30_000, + ), + 30 * 60_000, + ); +} /** * Fetch and read the global snapshot, giving each attempt * OPENSKY_ATTEMPT_TIMEOUT_MS and retrying once when an attempt times out or - * the connection fails. A second failure throws, which the proxy answers + * the connection fails. An oversized body is not retried: it would fail again + * and spend more credits. A second failure throws, which the proxy answers * from its stale cache or the regional fallback. * @param {Record} headers Request headers. * @returns {Promise<{upstream: Response, body: string}>} @@ -41,13 +65,27 @@ const OPENSKY_STATES_URL = async function fetchOpenSkyStates(headers) { for (let attempt = 1; ; attempt++) { try { - const upstream = await fetch(OPENSKY_STATES_URL, { - headers, - signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS), - }); - return { upstream, body: await upstream.text() }; + const signal = AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS); + const upstream = await fetch(OPENSKY_STATES_URL, { headers, signal }); + if (upstream.status === 429) { + // Start the cooldown from the headers and drop the rejected body + // unread, so a stalled or oversized 429 body cannot skip it. + _openskyCooldownUntil = + Date.now() + openSkyRateLimitCooldownMs(upstream); + void upstream.body?.cancel().catch(() => {}); + return { + upstream, + body: JSON.stringify({ error: 'OpenSky rate limited.' }), + }; + } + const body = await readResponseTextCapped( + upstream, + OPENSKY_STATES_MAX_RESPONSE_BYTES, + signal, + ); + return { upstream, body }; } catch (error) { - if (attempt >= 2) throw error; + if (attempt >= 2 || error?.code === 'RESPONSE_TOO_LARGE') throw error; console.warn( '[OpenSky] global snapshot attempt failed, retrying:', error?.message || error, @@ -544,19 +582,9 @@ export function openSkyProxy() { } if (upstream.status === 429) { reason = 'rate_limited'; - // Credit governor: honor OpenSky's retry-after (bounded 30 s … 30 min; - // 2 min when the header is absent) — no upstream attempts until then. - const retryAfterSec = Number( - upstream.headers.get('x-rate-limit-retry-after-seconds'), - ); - const cooldownMs = Math.min( - Math.max( - Number.isFinite(retryAfterSec) ? retryAfterSec * 1000 : 120_000, - 30_000, - ), - 30 * 60_000, - ); - _openskyCooldownUntil = now + cooldownMs; + // Credit governor: fetchOpenSkyStates already started the cooldown + // from OpenSky's retry-after — no upstream attempts until then. + const cooldownMs = openSkyRateLimitCooldownMs(upstream); // Serve the last-good body instead of the 429 when we have one — // the layer keeps rendering (STALE-cued) instead of dying. if (_openskyCacheBody && _openskyCacheStatus === 200) { diff --git a/server/providers/firms.js b/server/providers/firms.js index 636968348..c73ca10ce 100644 --- a/server/providers/firms.js +++ b/server/providers/firms.js @@ -2,6 +2,7 @@ import path from 'node:path'; import { promises as fsp } from 'node:fs'; import { filterTrailing24h, parseFirmsCsv } from '../../src/data/firmsCsv.js'; +import { readResponseTextCapped } from './common/http.js'; /** * NASA FIRMS live active-fire proxy with a memory + disk cache. @@ -35,6 +36,12 @@ export function firmsProxy() { 'VIIRS_SNPP_NRT', 'MODIS_NRT', ]; + /** + * Per-source body cap. A VIIRS world/2 pull is ~130k rows x ~100 B, about + * 13 MB per satellite in heavy fire season (MODIS runs smaller), so this is a + * bound, not a budget. + */ + const FIRMS_MAX_RESPONSE_BYTES = 64 * 1024 * 1024; const CACHE_DIR = path.join(process.cwd(), '.gev-cache'); const CACHE_PATH = path.join(CACHE_DIR, 'firms.json'); @@ -83,9 +90,12 @@ export function firmsProxy() { */ async function fetchSource(key, source) { const url = `https://firms.modaps.eosdis.nasa.gov/api/area/csv/${encodeURIComponent(key)}/${source}/world/2`; - const res = await fetch(url, { signal: AbortSignal.timeout(60_000) }); + const signal = AbortSignal.timeout(60_000); + const res = await fetch(url, { signal }); if (!res.ok) throw new Error(`HTTP ${res.status}`); - const records = parseFirmsCsv(await res.text()); + const records = parseFirmsCsv( + await readResponseTextCapped(res, FIRMS_MAX_RESPONSE_BYTES, signal), + ); if (records === null) throw new Error('non-CSV upstream response'); return records; } diff --git a/server/providers/space/celestrak.js b/server/providers/space/celestrak.js index a76cc590d..8baf59ba7 100644 --- a/server/providers/space/celestrak.js +++ b/server/providers/space/celestrak.js @@ -1,6 +1,7 @@ import path from 'node:path'; import { promises as fsp } from 'node:fs'; import { celestrakTleUrl } from '../../../src/data/spaceProviderRequests.js'; +import { readResponseTextCapped } from '../common/http.js'; /** * Vite plugin: CelesTrak TLE proxy. @@ -48,10 +49,14 @@ export function celestrakProxy() { } } + /** The largest group (`active`) measures ~2.7 MB; this bounds a runaway body. */ + const CELESTRAK_MAX_RESPONSE_BYTES = 16 * 1024 * 1024; + async function fetchUpstream(group) { const url = celestrakTleUrl(group); + const signal = AbortSignal.timeout(20000); const res = await fetch(url.toString(), { - signal: AbortSignal.timeout(20000), + signal, // CelesTrak 403s bulk groups (e.g. `active`) unless the request carries a // descriptive User-Agent with a contact point. headers: { @@ -60,7 +65,11 @@ export function celestrakProxy() { }, }); if (!res.ok) throw new Error(`HTTP ${res.status}`); - const body = await res.text(); + const body = await readResponseTextCapped( + res, + CELESTRAK_MAX_RESPONSE_BYTES, + signal, + ); // An upstream error page parses to zero TLEs — treat as failure, keep cache. if (!/^1 /m.test(body)) throw new Error('no TLE lines in response'); return { at: Date.now(), body }; diff --git a/server/providers/traffic.js b/server/providers/traffic.js index 2a3c2e60a..d53db6530 100644 --- a/server/providers/traffic.js +++ b/server/providers/traffic.js @@ -1,6 +1,7 @@ import path from 'node:path'; import { promises as fsp } from 'node:fs'; +import { readResponseBytesCapped } from './common/http.js'; import { isValidTileCoord as isValidTomTomTile, utcDayKey as tomtomUtcDayKey, @@ -48,6 +49,8 @@ export function tomtomProxy() { const DEFAULT_DAILY_BUDGET = 6000; const MEM_MAX_ENTRIES = 256; const UPSTREAM_TIMEOUT_MS = 15000; + /** Flow tiles are well under 1 MB; this bounds a runaway body, not a budget. */ + const TOMTOM_TILE_MAX_RESPONSE_BYTES = 8 * 1024 * 1024; /** @type {Map} tile key `z/x/y` -> cached tile (kept past TTL for serve-stale). */ const mem = new Map(); @@ -143,11 +146,15 @@ export function tomtomProxy() { 'https://api.tomtom.com/traffic/map/4/tile/flow/relative/' + `${z}/${x}/${y}.pbf?key=${encodeURIComponent(process.env.TOMTOM_API_KEY)}`; recordUpstreamFetch(); // attempts count — upstream bills the request either way - const res = await fetch(url, { - signal: AbortSignal.timeout(UPSTREAM_TIMEOUT_MS), - }); + const signal = AbortSignal.timeout(UPSTREAM_TIMEOUT_MS); + const res = await fetch(url, { signal }); if (!res.ok) throw new Error(`HTTP ${res.status}`); - const buf = Buffer.from(await res.arrayBuffer()); + const bytes = await readResponseBytesCapped( + res, + TOMTOM_TILE_MAX_RESPONSE_BYTES, + signal, + ); + const buf = Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength); if (buf.length === 0) throw new Error('empty tile body'); return buf; } diff --git a/src/app/constructCatalog.js b/src/app/constructCatalog.js index 184b51f17..832c174bc 100644 --- a/src/app/constructCatalog.js +++ b/src/app/constructCatalog.js @@ -24,6 +24,7 @@ import { createApplicationLocalAdsb } from './layers/localAdsb.js'; import { createApplicationAwareness } from './layers/militaryAwareness.js'; import { createApplicationFirms } from './layers/firms.js'; import { createApplicationEarthquakes } from './layers/earthquakes.js'; +import { createApplicationAirQuality } from './layers/airQuality.js'; import { createApplicationFirePerimeters } from './layers/perimeters.js'; import { createApplicationGnssIntegrity } from './layers/gnss.js'; import { createApplicationCables } from './layers/submarineCables.js'; @@ -118,6 +119,7 @@ export function createApplicationCatalog({ ...(resolveAsset ? { resolveAsset } : {}), }), createApplicationEarthquakes({ source: sources.earthquakes }), + createApplicationAirQuality({ source: sources['air-quality'] }), createApplicationFirePerimeters({ source: sources['fire-perimeters'], }), diff --git a/src/app/constructCatalog.test.mjs b/src/app/constructCatalog.test.mjs index f8ebb1e73..9e1059c33 100644 --- a/src/app/constructCatalog.test.mjs +++ b/src/app/constructCatalog.test.mjs @@ -40,7 +40,7 @@ test('catalogs construct distinct layers and classification from their supplied signal: b.signal, surface: fixtureSurface(b.signal), }); - assert.equal(first.layers.length, 31); + assert.equal(first.layers.length, 32); assert.ok(first.get('local-adsb'), 'Local ADS-B is registered'); assert.deepEqual( first.metadata.find(({ id }) => id === 'local-adsb'), @@ -154,7 +154,7 @@ test('Mapillary can be omitted or replaced without changing catalog membership', surface: fixtureSurface(lifetime.signal), }; const catalog = createApplicationCatalog(options); - assert.equal(catalog.layers.length, 31); + assert.equal(catalog.layers.length, 32); assert.deepEqual(catalog.get('street-level').providerIds, []); assert.equal( catalogControlServices(catalog).streetLevelLayer, diff --git a/src/app/layers/airQuality.js b/src/app/layers/airQuality.js new file mode 100644 index 000000000..1f25c06d9 --- /dev/null +++ b/src/app/layers/airQuality.js @@ -0,0 +1,6 @@ +import { createAirQualityLayer } from '../../layers/airQuality/index.js'; +import { overlayHost } from './overlayHost.js'; +/** Wire air-quality readings to the application overlay host. */ +export function createApplicationAirQuality(options) { + return createAirQualityLayer({ overlayHost, ...options }); +} diff --git a/src/app/sourceComposition.js b/src/app/sourceComposition.js index 0b516f5b4..0fc619142 100644 --- a/src/app/sourceComposition.js +++ b/src/app/sourceComposition.js @@ -46,6 +46,7 @@ export const CATALOG_SOURCE_CONTRACTS = Object.freeze({ cyclones: contract(['weather-cyclones'], ['getSnapshot']), earthquakes: contract(['earthquakes'], ['getSnapshot']), 'fire-perimeters': contract(['fire-perimeters'], ['getSnapshot']), + 'air-quality': contract(['air-quality'], ['getSnapshot']), 'gnss-integrity': contract(['gnss-integrity'], ['getSnapshot']), cables: contract(['telegeography-submarine-cables'], ['fetch']), transit: contract(['transit'], ['requestSnapshot', 'getHistory']), diff --git a/src/app/sourceComposition.test.mjs b/src/app/sourceComposition.test.mjs index cf8646e25..422308f1e 100644 --- a/src/app/sourceComposition.test.mjs +++ b/src/app/sourceComposition.test.mjs @@ -37,7 +37,7 @@ test('every acquisition source can be omitted without losing catalog membership' test('all acquisition sources may be absent, and enable fails before initializing or fetching', async (t) => { const actual = catalog({}, t); - assert.equal(actual.layers.length, 31); + assert.equal(actual.layers.length, 32); const manager = new LayerLifecycle( {}, { getSourceAvailability: actual.getSourceAvailability }, diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index 576f5b07d..f95070d04 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -169,6 +169,13 @@ export const DATA_CREDITS = [ }, OSM_CREDIT, OPENMAPTILES_CREDIT, + { + key: 'eccc-aqhi', + html: + 'Air quality (AQHI): Environment and Climate Change Canada — MSC GeoMet ' + + '(eccc-msc.github.io/open-data) ' + + '· ECCC Data Servers End-use Licence', + }, { key: 'overture-military-names', html: 'Military area names: Overture Maps Foundation (ODbL)', @@ -263,7 +270,9 @@ export const DATA_CREDITS = [ }, { key: 'gbfs', - html: 'Bikeshare availability: GBFS operator feeds (e.g. Austin BCycle)', + html: + 'Bikeshare availability: GBFS operator feeds ' + + '(e.g. Austin BCycle; Bike Share Toronto, operated by the Toronto Parking Authority)', }, { key: 'urbansharing-gbfs', diff --git a/src/data/gbfsSource.js b/src/data/gbfsSource.js index 8d4980948..57228554e 100644 --- a/src/data/gbfsSource.js +++ b/src/data/gbfsSource.js @@ -4,7 +4,7 @@ const GBFS_ALLOWED_HOSTS = new Set([ 'gbfs.bluebikes.com', 'gbfs.bcycle.com', 'gbfs.biketownpdx.com', - 'gbfs.cogobikeshare.com', + 'reddybikeshare.socialbicycles.com', 'austin.publicbikesystem.net', 'hon.publicbikesystem.net', 'chat.publicbikesystem.net', diff --git a/src/data/layerState.js b/src/data/layerState.js index 9428296aa..a07fac214 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -585,6 +585,11 @@ export const LAYER_STATE_TOKEN_RESERVATIONS = * owns stable URL ordering. */ export const LAYER_STATE_REGISTRY = Object.freeze([ + Object.freeze({ + id: 'air-quality', + token: '4', + disposition: 'enabled-only', + }), Object.freeze({ id: 'ais-live-vessels', token: 'a', diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index 631e5dfec..77b8f902f 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -202,8 +202,8 @@ function encode(state) { test('production registry is exact, canonical, and rejects incomplete contracts', async () => { assert.equal(validateLayerStateRegistry(), true); - assert.equal(REGISTERED_LAYER_IDS.length, 30); - assert.equal(new Set(REGISTERED_LAYER_IDS).size, 30); + assert.equal(REGISTERED_LAYER_IDS.length, 31); + assert.equal(new Set(REGISTERED_LAYER_IDS).size, 31); assert.ok(REGISTERED_LAYER_IDS.includes('transit')); assert.deepEqual(REGISTERED_LAYER_IDS, [...REGISTERED_LAYER_IDS].sort()); assert.deepEqual(LEGACY_LAYER_STATE_TOKENS, { @@ -242,19 +242,19 @@ test('production registry is exact, canonical, and rejects incomplete contracts' for (const [id, token] of Object.entries(LEGACY_LAYER_STATE_TOKENS)) { assert.equal(LAYER_STATE_TOKEN_RESERVATIONS[id], token); } - assert.equal(nextLayerStateToken(), '4'); + assert.equal(nextLayerStateToken(), '5'); assert.equal( nextLayerStateToken({ ...LAYER_STATE_TOKEN_RESERVATIONS, - alpha: '4', - bravo: '5', + alpha: '5', + bravo: '6', }), - '6', + '7', ); const digitsExhausted = { ...LAYER_STATE_TOKEN_RESERVATIONS, ...Object.fromEntries( - [...'456789'].map((digit) => [`prior-${digit}`, digit]), + [...'56789'].map((digit) => [`prior-${digit}`, digit]), ), }; assert.equal(nextLayerStateToken(digitsExhausted), '00'); @@ -307,8 +307,8 @@ test('production registry is exact, canonical, and rejects incomplete contracts' assert.equal( validateLayerStateAllocations(LAYER_STATE_TOKEN_RESERVATIONS, { ...LAYER_STATE_TOKEN_RESERVATIONS, - future: '4', - next: '5', + future: '5', + next: '6', }), true, ); @@ -318,7 +318,7 @@ test('production registry is exact, canonical, and rejects incomplete contracts' ...LAYER_STATE_TOKEN_RESERVATIONS, future: '00', }), - /next free token 4/, + /next free token 5/, ); const beforeLastDigit = { ...digitsExhausted }; delete beforeLastDigit['prior-9']; @@ -341,10 +341,10 @@ test('production registry is exact, canonical, and rejects incomplete contracts' assert.throws( () => validateLayerStateAllocations( - { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '4' }, - { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '4', competing: '4' }, + { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '5' }, + { ...LAYER_STATE_TOKEN_RESERVATIONS, merged: '5', competing: '5' }, ), - /next free token 5/, + /next free token 6/, ); assert.throws( () => validateLayerStateAllocations({ future: '00' }, { future: '01' }), diff --git a/src/data/layerStateTokenLedger.test.mjs b/src/data/layerStateTokenLedger.test.mjs index a38a43d51..7bcfd96cb 100644 --- a/src/data/layerStateTokenLedger.test.mjs +++ b/src/data/layerStateTokenLedger.test.mjs @@ -128,17 +128,23 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed { ...parseLayerStateTokenReservations(reservationRows) }, { ...LAYER_STATE_TOKEN_RESERVATIONS }, ); - assert.equal(Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length, 30); + assert.equal(Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length, 31); assert.deepEqual( { ...LAYER_STATE_TOKEN_RESERVATIONS }, { ...LEGACY_LAYER_STATE_TOKENS, 'street-level': '0', 'gnss-integrity': '3', + 'air-quality': '4', }, ); assert.throws( - () => parseLayerStateTokenReservations(reservationRows.slice(1)), + () => + parseLayerStateTokenReservations( + // Drop a legacy row by id; the first row sorts as air-quality, which + // is not a legacy reservation. + reservationRows.filter(([id]) => id !== 'ais-live-vessels'), + ), /Missing or changed legacy/, ); assert.throws( @@ -155,9 +161,9 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed /Duplicate layer-state token reservation/, ); assert.equal( - parseLayerStateTokenReservations([...reservationRows, ['future', '4']]) + parseLayerStateTokenReservations([...reservationRows, ['future', '5']]) .future, - '4', + '5', ); assert.throws( () => @@ -175,14 +181,14 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed test('published retired digits and competing PRs advance the merge-time allocation', () => { withPublishedBase( - { rows: [...reservationRows, ['retired-layer', '4']] }, + { rows: [...reservationRows, ['retired-layer', '5']] }, (cwd) => { const published = readPublishedLayerStateReservations('HEAD', cwd); - assert.equal(nextLayerStateToken(published), '5'); + assert.equal(nextLayerStateToken(published), '6'); assert.equal( validateLayerStateAllocations(published, { ...published, - newcomer: '5', + newcomer: '6', }), true, ); @@ -190,15 +196,15 @@ test('published retired digits and competing PRs advance the merge-time allocati () => validateLayerStateAllocations(published, { ...published, - competing: '4', + competing: '5', }), - /next free token 5/, + /next free token 6/, ); assert.throws( () => validateLayerStateAllocations(published, { ...LAYER_STATE_TOKEN_RESERVATIONS, - newcomer: '4', + newcomer: '5', }), /changed or removed: retired-layer/, ); @@ -206,8 +212,8 @@ test('published retired digits and competing PRs advance the merge-time allocati ); }); -test('PR B manually replaces provisional 4 with 5 after PR A publishes 4', () => { - withPublishedBase({ rows: [...reservationRows, ['pr-a', '4']] }, (cwd) => { +test('PR B manually replaces provisional 5 with 6 after PR A publishes 5', () => { + withPublishedBase({ rows: [...reservationRows, ['pr-a', '5']] }, (cwd) => { mkdirSync(path.join(cwd, 'scripts')); const fixtureChecker = path.join( cwd, @@ -228,8 +234,8 @@ test('PR B manually replaces provisional 4 with 5 after PR A publishes 4', () => writeCodecFixture( cwd, - [...reservationRows, ['pr-b', '4']], - [{ id: 'pr-b', token: '4' }], + [...reservationRows, ['pr-b', '5']], + [{ id: 'pr-b', token: '5' }], ); const beforeRebase = runCheck(); assert.equal(beforeRebase.status, 1); @@ -237,10 +243,10 @@ test('PR B manually replaces provisional 4 with 5 after PR A publishes 4', () => writeCodecFixture( cwd, - [...reservationRows, ['pr-a', '4'], ['pr-b', '4']], + [...reservationRows, ['pr-a', '5'], ['pr-b', '5']], [ - { id: 'pr-a', token: '4' }, - { id: 'pr-b', token: '4' }, + { id: 'pr-a', token: '5' }, + { id: 'pr-b', token: '5' }, ], ); const staleRebase = runCheck(); @@ -248,16 +254,16 @@ test('PR B manually replaces provisional 4 with 5 after PR A publishes 4', () => assert.match(staleRebase.stderr, /Duplicate layer-state token reservation/); const published = readPublishedLayerStateReservations('HEAD', cwd); - assert.equal(nextLayerStateToken(published), '5'); - const validBaselineRows = [...reservationRows, ['pr-a', '4']]; - writeCodecFixture(cwd, validBaselineRows, [{ id: 'pr-a', token: '4' }]); + assert.equal(nextLayerStateToken(published), '6'); + const validBaselineRows = [...reservationRows, ['pr-a', '5']]; + writeCodecFixture(cwd, validBaselineRows, [{ id: 'pr-a', token: '5' }]); const helper = spawnSync( process.execPath, [realpathSync(fixtureNext), 'pr-b'], { cwd, encoding: 'utf8' }, ); assert.equal(helper.status, 0, helper.stderr); - assert.match(helper.stdout, /pr-b: 5/); + assert.match(helper.stdout, /pr-b: 6/); assert.deepEqual( JSON.parse(readFileSync(path.join(cwd, LEDGER_PATH), 'utf8')), validBaselineRows, @@ -265,20 +271,20 @@ test('PR B manually replaces provisional 4 with 5 after PR A publishes 4', () => ); writeCodecFixture( cwd, - [...reservationRows, ['pr-a', '4'], ['pr-b', '5']], + [...reservationRows, ['pr-a', '5'], ['pr-b', '6']], [ - { id: 'pr-a', token: '4' }, - { id: 'pr-b', token: '5' }, + { id: 'pr-a', token: '5' }, + { id: 'pr-b', token: '6' }, ], ); const corrected = runCheck(); assert.equal(corrected.status, 0, corrected.stderr); - assert.match(corrected.stdout, /31 published, 1 new/); + assert.match(corrected.stdout, /32 published, 1 new/); }); }); test('allocation batches cross the last digit and base-36 pair boundaries in order', () => { - const digitRows = [...'456789'].map((digit) => [`retired-${digit}`, digit]); + const digitRows = [...'56789'].map((digit) => [`retired-${digit}`, digit]); const allDigits = parseLayerStateTokenReservations([ ...reservationRows, ...digitRows, @@ -330,7 +336,7 @@ test('allocation batches cross the last digit and base-36 pair boundaries in ord test('an isolated valid two-character fixture round-trips an l field beyond the old 64-character cap', async () => { const cwd = mkdtempSync(path.join(tmpdir(), 'gev-layer-token-width-')); try { - const priorDigits = [...'456789'].map((digit) => [ + const priorDigits = [...'56789'].map((digit) => [ `qa-prior-digit-${digit}`, digit, ]); @@ -367,7 +373,7 @@ test('an isolated valid two-character fixture round-trips an l field beyond the const params = new URLSearchParams([['v', '2']]); codec.encodeLayerStateParams(params, state); assert.ok(params.get('l').length > 64, 'fixture must cross the old cap'); - assert.equal(params.get('l').length, 71); + assert.equal(params.get('l').length, 73); const restored = codec.decodeLayerStateParams(params); assert.deepEqual(restored?.enabledLayerIds, expectedLayerIds); assert.deepEqual(restored?.options, state.options); @@ -385,7 +391,7 @@ test('checker reads a complete future base ledger and rejects retired-token reus }, (cwd) => { const published = readPublishedLayerStateReservations('HEAD', cwd); - assert.equal(Object.keys(published).length, 31); + assert.equal(Object.keys(published).length, 32); assert.equal(published['retired-layer'], '00'); assert.throws( () => diff --git a/src/data/layerStateTokenReservations.json b/src/data/layerStateTokenReservations.json index 34ba725b5..dd8dcd7c4 100644 --- a/src/data/layerStateTokenReservations.json +++ b/src/data/layerStateTokenReservations.json @@ -1,4 +1,5 @@ [ + ["air-quality", "4"], ["ais-live-vessels", "a"], ["alpr-cameras", "p"], ["bhote-koshi-2026", "h"], diff --git a/src/firstRunExperience.test.mjs b/src/firstRunExperience.test.mjs index 2615f9654..749123acd 100644 --- a/src/firstRunExperience.test.mjs +++ b/src/firstRunExperience.test.mjs @@ -994,10 +994,11 @@ test('the voice TOOL SCHEMA matches the pinned contract — the mission mapping // schema, which may not carry top-level anyOf), and Contacts requested-radius // list descriptions; the missions still ride existing tools. // Street Level and GNSS extend the manifest; broadcast caveats describe tracked and analyst aircraft. - assert.equal(block.length, 30511, 'serialized tool schema length drifted'); + // Air quality (ECCC AQHI) extends the manifest. + assert.equal(block.length, 30655, 'serialized tool schema length drifted'); assert.equal( crypto.createHash('sha256').update(block).digest('hex'), - '62d5df9cedf413403e4fd0a7cbe94bb2cd13736c3d8fedc84bcccd63ad253eb6', + '3b1d5e77c10a92f6543a4f9d1ff1870101a0306b65e5b8a6113111c81eda371e', 'the first-run missions must ride EXISTING tools: no schema edit, no cache bust', ); const instructions = fs.readFileSync( diff --git a/src/i18n/layerStatusText.test.mjs b/src/i18n/layerStatusText.test.mjs index 8b1d37f39..cb10d6f1e 100644 --- a/src/i18n/layerStatusText.test.mjs +++ b/src/i18n/layerStatusText.test.mjs @@ -10,8 +10,7 @@ import { setLocale, translateKnown } from './index.js'; const SITES = [ ['../layers/streetLevel/index.js', 'loading coverage...'], ['../layers/firms/ingestion.js', 'live feed unavailable'], - ['../layers/bikeshare/viewport.js', 'GBFS fetch error'], - ['../layers/bikeshare/ingestion.js', 'GBFS status update failed'], + ['../layers/bikeshare/controls.js', 'GBFS fetch error'], [ '../layers/streetLevel/viewerHost.js', 'Open the Street Level panel to view imagery', diff --git a/src/i18n/locales/es.json b/src/i18n/locales/es.json index a06512624..ee33dd3d6 100644 --- a/src/i18n/locales/es.json +++ b/src/i18n/locales/es.json @@ -2214,7 +2214,6 @@ "loading coverage...": "cargando cobertura...", "live feed unavailable": "fuente en vivo no disponible", "GBFS fetch error": "error al obtener datos GBFS", - "GBFS status update failed": "falló la actualización del estado GBFS", "Open the Street Level panel to view imagery": "Abre el panel \"A pie de calle\" para ver las imágenes", "Image could not be opened": "No se pudo abrir la imagen", "Mapillary rejected MAPILLARY_CLIENT_TOKEN — replace it in Provider Settings": "Mapillary rechazó MAPILLARY_CLIENT_TOKEN — sustitúyelo en Ajustes de proveedores", @@ -2783,5 +2782,16 @@ "Show the complete infrared image; 0% filter strength": "Muestra la imagen infrarroja completa; intensidad del filtro 0%", "FILTER": "FILTRO", "Cloud filter strength": "Intensidad del filtro de nubes", - "Panoramax answered HTTP {status}": "Panoramax respondió HTTP {status}" + "Panoramax answered HTTP {status}": "Panoramax respondió HTTP {status}", + "Highest air-quality readings in view": "Lecturas de calidad del aire más altas a la vista", + "Low": "Baja", + "Moderate": "Moderada", + "High": "Alta", + "Very High": "Muy alta", + "Air quality": "Calidad del aire", + "{n} sequences in view · click a line for its nearest photo": { + "one": "{n} secuencia a la vista · haz clic en una línea para su foto más cercana", + "many": "{n} secuencias a la vista · haz clic en una línea para su foto más cercana", + "other": "{n} secuencias a la vista · haz clic en una línea para su foto más cercana" + } } diff --git a/src/i18n/locales/ru.json b/src/i18n/locales/ru.json index 3e8e1f152..3c242f6bb 100644 --- a/src/i18n/locales/ru.json +++ b/src/i18n/locales/ru.json @@ -2250,7 +2250,6 @@ "loading coverage...": "загрузка покрытия...", "live feed unavailable": "поток в реальном времени недоступен", "GBFS fetch error": "ошибка загрузки GBFS", - "GBFS status update failed": "не удалось обновить статус GBFS", "Open the Street Level panel to view imagery": "Откройте панель «Уличные снимки», чтобы их посмотреть", "Image could not be opened": "Не удалось открыть снимок", "Mapillary rejected MAPILLARY_CLIENT_TOKEN — replace it in Provider Settings": "Mapillary отклонил MAPILLARY_CLIENT_TOKEN — замените его в «Настройках провайдеров»", @@ -2840,5 +2839,17 @@ "Show the complete infrared image; 0% filter strength": "Показать полное ИК-изображение; сила фильтра 0%", "FILTER": "ФИЛЬТР", "Cloud filter strength": "Сила фильтра облаков", - "Panoramax answered HTTP {status}": "Panoramax ответил: HTTP {status}" + "Panoramax answered HTTP {status}": "Panoramax ответил: HTTP {status}", + "Highest air-quality readings in view": "Наивысшие показатели качества воздуха в поле зрения", + "Low": "Низкий", + "Moderate": "Умеренный", + "High": "Высокий", + "Very High": "Очень высокий", + "Air quality": "Качество воздуха", + "{n} sequences in view · click a line for its nearest photo": { + "one": "{n} съёмка в кадре · щёлкните линию, чтобы открыть ближайшую фотографию", + "few": "{n} съёмки в кадре · щёлкните линию, чтобы открыть ближайшую фотографию", + "many": "{n} съёмок в кадре · щёлкните линию, чтобы открыть ближайшую фотографию", + "other": "{n} съёмки в кадре · щёлкните линию, чтобы открыть ближайшую фотографию" + } } diff --git a/src/layers/airQuality/index.js b/src/layers/airQuality/index.js new file mode 100644 index 000000000..9b7cd3876 --- /dev/null +++ b/src/layers/airQuality/index.js @@ -0,0 +1,287 @@ +import * as Cesium from 'cesium'; +import { t } from '../../i18n/index.js'; +import { + AIR_QUALITY_BANDS, + AIR_QUALITY_OVERLAY_SOURCE_ID, + AIR_QUALITY_OVERLAY_COHORT_LIMIT, + AIR_QUALITY_OVERLAY_COLLISION_CAPACITY, + AIR_QUALITY_PROVIDERS, + createAirQualityOverlayEntry, + mapAnalystRecord, + newestObservation, + readingsAreStale, + readingsInView, + selectAirQualityOverlayCohort, +} from './model.js'; +export * from './model.js'; +export { createEcccAirQualitySource } from './source.js'; + +/** Station markers are a point reading, not an extent. */ +const STATION_MARKER_RADIUS_M = 6_000; +/** Readings listed in the readout card. */ +const READOUT_LIMIT = 5; + +/** Own one air-quality display, its refresh lifecycle, and its readout card. */ +export function createAirQualityLayer({ source, overlayHost } = {}) { + if (typeof source?.getSnapshot !== 'function') + throw new TypeError('Air quality requires a snapshot source'); + if (!overlayHost) throw new TypeError('Air quality requires an overlay host'); + let _viewer = null; + let _request = null; + let _dataSource = null; + let _readings = []; + let _count = 0; + let _lastUpdate = null; + let _lastError = null; + let _enabled = false; + let rowControlsListener = null; + + const notify = () => rowControlsListener?.(); + + /** The camera's lon/lat rectangle in degrees, or null when unresolvable. */ + const viewRectangle = () => { + const rectangle = _viewer?.camera?.computeViewRectangle?.(); + if (!rectangle) return null; + const toDeg = Cesium.Math.toDegrees; + return { + west: toDeg(rectangle.west), + south: toDeg(rectangle.south), + east: toDeg(rectangle.east), + north: toDeg(rectangle.north), + }; + }; + + const layer = { + id: 'air-quality', + name: 'Air quality', + icon: '◍', + source: 'ECCC MSC GeoMet', + updateInterval: 1_200_000, + + init(viewer) { + if (_viewer) throw new Error('Air quality layer is already initialized'); + _viewer = viewer; + _dataSource = new Cesium.CustomDataSource('air-quality'); + _dataSource.show = false; + viewer.dataSources.add(_dataSource); + _readings = []; + _count = 0; + _lastUpdate = null; + _lastError = null; + _enabled = false; + overlayHost.setVisible(AIR_QUALITY_OVERLAY_SOURCE_ID, false); + }, + + enable() { + _enabled = true; + if (_dataSource) _dataSource.show = true; + overlayHost.setVisible(AIR_QUALITY_OVERLAY_SOURCE_ID, true); + notify(); + }, + + disable() { + // Abort in flight: a response arriving after disable must not redraw. + _request?.abort(); + _request = null; + _enabled = false; + if (_dataSource) _dataSource.show = false; + overlayHost.clearSource(AIR_QUALITY_OVERLAY_SOURCE_ID); + overlayHost.setVisible(AIR_QUALITY_OVERLAY_SOURCE_ID, false); + notify(); + }, + + async update() { + if (!_enabled || !_dataSource) return false; + _request?.abort(); + const request = new AbortController(); + _request = request; + try { + const readings = await source.getSnapshot({ signal: request.signal }); + // A superseded or aborted request must not touch the scene, and neither + // must one that lands after the layer was switched off. + if (request.signal.aborted || _request !== request || !_enabled) + return false; + _readings = readings; + this._render(); + _count = _readings.length; + _lastUpdate = Date.now(); + _lastError = null; + notify(); + return true; + } catch (error) { + if (request.signal.aborted || _request !== request) return false; + _lastError = error?.message || 'Air quality request failed'; + console.warn('[Data:AirQuality] Refresh failed:', _lastError); + notify(); + return false; + } finally { + if (_request === request) _request = null; + } + }, + + /** Rebuild entities and overlay labels from the current readings. */ + _render() { + if (!_dataSource) return; + _dataSource.entities.removeAll(); + const overlayEntries = []; + for (const reading of _readings) { + const position = Cesium.Cartesian3.fromDegrees( + reading.lon, + reading.lat, + ); + const color = Cesium.Color.fromCssColorString(reading.color); + _dataSource.entities.add( + new Cesium.Entity({ + id: `air-quality:${reading.id}`, + position, + ellipse: { + // Static axes, as in the earthquake layer: a per-frame axis + // re-tessellates the clamped ground primitive every frame. + semiMajorAxis: STATION_MARKER_RADIUS_M, + semiMinorAxis: STATION_MARKER_RADIUS_M, + material: new Cesium.ColorMaterialProperty(color.withAlpha(0.35)), + outline: true, + outlineColor: color.withAlpha(0.9), + outlineWidth: 2, + heightReference: Cesium.HeightReference.CLAMP_TO_GROUND, + }, + properties: { + provider: reading.provider, + scale: reading.scale, + value: reading.value, + band: reading.band, + name: reading.name, + zone: reading.zone, + risk: reading.risk, + observedMs: reading.observedMs, + note: reading.note, + lat: reading.lat, + lon: reading.lon, + }, + }), + ); + overlayEntries.push( + createAirQualityOverlayEntry({ + id: reading.id, + position, + title: `${reading.name} ${reading.label}`, + accent: reading.color, + value: reading.value, + }), + ); + } + if (_enabled) { + overlayHost.setEntries( + AIR_QUALITY_OVERLAY_SOURCE_ID, + selectAirQualityOverlayCohort(overlayEntries), + { + cohortLimit: AIR_QUALITY_OVERLAY_COHORT_LIMIT, + collisionCapacity: AIR_QUALITY_OVERLAY_COLLISION_CAPACITY, + moving: false, + }, + ); + } + }, + + destroy(viewer = _viewer) { + _request?.abort(); + _request = null; + _enabled = false; + overlayHost.clearSource(AIR_QUALITY_OVERLAY_SOURCE_ID); + overlayHost.setVisible(AIR_QUALITY_OVERLAY_SOURCE_ID, false); + if (_dataSource) { + viewer?.dataSources?.remove(_dataSource, true); + _dataSource = null; + } + _viewer = null; + _readings = []; + _count = 0; + _lastUpdate = null; + _lastError = null; + rowControlsListener = null; + }, + + /** + * Readout descriptor for the Weather panel: the band legend, and the worst + * readings currently in view. + * + * `readout: true` renders the toggle and source line only; the card owns + * the rest. Coverage is stated plainly because this layer is not global. + */ + getRowControls() { + const provider = AIR_QUALITY_PROVIDERS.eccc; + const inView = readingsInView(_readings, viewRectangle(), READOUT_LIMIT); + const newest = _readings.reduce( + (max, reading) => Math.max(max, reading.observedMs || 0), + 0, + ); + return { + readout: true, + summary: { + label: provider.label, + coverage: provider.coverage, + validTime: newest ? new Date(newest).toISOString() : undefined, + detail: `${provider.network} · ${provider.scale}`, + status: _lastError + ? _lastError + : _count === 0 + ? _lastUpdate === null + ? 'No readings available' + : 'No current readings' + : `${_count} stations reporting`, + units: provider.scale, + }, + legend: AIR_QUALITY_BANDS.map((band) => ({ + color: band.color, + label: band.label, + })), + legendCategorical: true, + list: { + ariaLabel: t('Highest air-quality readings in view'), + items: inView.map((reading, index) => ({ + id: reading.id, + ordinal: index + 1, + lead: `${reading.scale} ${reading.label}`, + text: `${reading.name} · ${reading.risk}`, + accent: reading.color, + })), + }, + }; + }, + + setRowControlsListener(listener) { + rowControlsListener = typeof listener === 'function' ? listener : null; + }, + + /** + * Snapshot readings as JSON-safe records for the analyst query engine. + * @param {number} [maxCount=500] + * @returns {Array} + */ + getAnalystRecords(maxCount = 500) { + if (!_enabled || !_readings.length) return []; + const limit = Number.isFinite(maxCount) + ? Math.max(1, Math.floor(maxCount)) + : 500; + return _readings + .slice(0, limit) + .map((reading, index) => mapAnalystRecord(reading, index)); + }, + + getStats() { + // `lastUpdate` is when a response was received; `observedAt` is when the + // network actually measured. Freshness is judged on the latter, so a feed + // that keeps answering with hours-old observations reports STALE rather + // than being narrated as live by the HUD and voice provenance envelope. + return { + count: _count, + lastUpdate: _lastUpdate, + error: _lastError, + observedAt: newestObservation(_readings), + // Only a completed refresh can be stale; before one there is no feed. + stale: _lastUpdate !== null && readingsAreStale(_readings), + }; + }, + }; + return layer; +} diff --git a/src/layers/airQuality/model.js b/src/layers/airQuality/model.js new file mode 100644 index 000000000..4854a3761 --- /dev/null +++ b/src/layers/airQuality/model.js @@ -0,0 +1,378 @@ +/** + * @module layers/airQuality/model + * + * Air-quality readings, normalized across networks. Pure — no Cesium, no + * network. + * + * Air-quality indices are not interchangeable numbers. Canada's AQHI runs 1 to + * 10+ on health risk; the US AQI runs 0 to 500 on concentration breakpoints; + * Europe's EAQI is a five-step category. A reading is therefore carried as + * `{ provider, scale, value, band }` — who measured it, which index it is, the + * value on that index's own terms, and the band it falls in — so a second + * network can be added without retrofitting a shared numeric meaning that does + * not exist. + * + * Only ECCC's AQHI is implemented here; `AIR_QUALITY_PROVIDERS` is the seam a + * later provider registers against. + */ +import { msg } from '../../i18n/index.js'; + +/** Band identifiers shared across providers, in ascending severity. */ +export const AIR_QUALITY_BANDS = Object.freeze([ + Object.freeze({ id: 'low', label: msg('Low'), color: '#4cc9f0' }), + Object.freeze({ id: 'moderate', label: msg('Moderate'), color: '#ffd43b' }), + Object.freeze({ id: 'high', label: msg('High'), color: '#ff6b35' }), + Object.freeze({ id: 'very-high', label: msg('Very High'), color: '#c1121f' }), +]); + +const BAND_BY_ID = new Map(AIR_QUALITY_BANDS.map((band) => [band.id, band])); + +/** Colour used when a station reports no usable reading. */ +export const AIR_QUALITY_UNKNOWN_COLOR = '#6b7785'; + +export const AIR_QUALITY_OVERLAY_SOURCE_ID = 'air-quality'; +export const AIR_QUALITY_OVERLAY_COHORT_LIMIT = 48; +export const AIR_QUALITY_OVERLAY_COLLISION_CAPACITY = 32; + +/** + * A reading this old is stale enough that displaying it would misinform. + * AQHI publishes hourly, so six hours is five missed publications. + */ +export const OBSERVATION_MAX_AGE_MS = 6 * 60 * 60 * 1000; + +/** + * How old the newest observation may be before the layer reports itself stale. + * + * This is deliberately separate from the age at which a reading is dropped. + * `lastUpdate` records when we last *received* a response, which stays current + * even when the network publishes nothing new; freshness has to be judged on + * the source's own observation time instead. AQHI publishes hourly, so a newest + * reading beyond two hours means at least one publication was missed and the + * feed should not be described as live. + */ +export const STALE_AFTER_MS = 2 * 60 * 60 * 1000; + +/** + * The newest observation time across a set of readings, or null when empty. + * @param {Array} readings + * @returns {number|null} Epoch ms. + */ +export function newestObservation(readings) { + if (!Array.isArray(readings) || readings.length === 0) return null; + let newest = null; + for (const reading of readings) { + const at = Number(reading?.observedMs); + if (Number.isFinite(at) && (newest === null || at > newest)) newest = at; + } + return newest; +} + +/** + * Is a set of readings stale, judged on source observation time? An empty + * set is stale: a refresh that kept no current observation has nothing live. + * @param {Array} readings + * @param {number} [now=Date.now()] + * @returns {boolean} + */ +export function readingsAreStale(readings, now = Date.now()) { + const newest = newestObservation(readings); + // No current observation at all — every row was dropped as too old or + // unusable — is the stalest a feed can be, not a nominal one. + if (newest === null) return true; + return now - newest > STALE_AFTER_MS; +} + +/** + * Registered provider descriptors. `coverage` is stated plainly because the + * layer is not global and the row must say so. + */ +export const AIR_QUALITY_PROVIDERS = Object.freeze({ + eccc: Object.freeze({ + id: 'eccc', + scale: 'AQHI', + label: msg('Air quality'), + network: 'Environment and Climate Change Canada', + coverage: 'Canada only · ECCC AQHI network', + /** ECCC's published band thresholds, evaluated in order. */ + bands: Object.freeze([ + Object.freeze({ id: 'low', max: 3 }), + Object.freeze({ id: 'moderate', max: 6 }), + Object.freeze({ id: 'high', max: 10 }), + Object.freeze({ id: 'very-high', max: Infinity }), + ]), + }), +}); + +/** Band descriptor for an id, or null. */ +export function airQualityBandById(bandId) { + return BAND_BY_ID.get(String(bandId ?? '')) || null; +} + +/** Accent colour for a band id, falling back to neutral grey. */ +export function airQualityColor(bandId) { + return airQualityBandById(bandId)?.color || AIR_QUALITY_UNKNOWN_COLOR; +} + +/** + * The published AQHI value for a raw ECCC reading: an integer from 1 up. + * + * ECCC reports whole numbers with a floor of 1 — there is no "AQHI 0" and no + * "AQHI 1.08". A raw value below 1 is a legitimate low reading, not an error, + * and is published as 1. + * + * @param {number|string|null|undefined} raw - Raw `aqhi` value from the API. + * @returns {number|null} Published integer AQHI, or null when unusable. + */ +export function aqhiDisplayValue(raw) { + // Guard the empty forms BEFORE coercing: Number(null) and Number('') are both + // 0, which would otherwise floor to a confident "AQHI 1" for a station that + // reported nothing at all. + if (raw === null || raw === undefined || raw === '') return null; + if (typeof raw !== 'number' && typeof raw !== 'string') return null; + if (typeof raw === 'string' && raw.trim() === '') return null; + const value = Number(raw); + if (!Number.isFinite(value) || value < 0) return null; + return Math.max(1, Math.round(value)); +} + +/** + * The band id for a value on a provider's scale. + * @param {string} providerId + * @param {number|null} value - Already in the provider's published form. + * @returns {string|null} + */ +export function bandForValue(providerId, value) { + const provider = AIR_QUALITY_PROVIDERS[String(providerId ?? '')]; + if (!provider || !Number.isFinite(value)) return null; + return provider.bands.find((band) => value <= band.max)?.id ?? null; +} + +/** + * Display text for a value on a provider's scale. + * AQHI collapses everything above ten into the open-ended "10+". + * @param {string} providerId + * @param {number|null} value + * @returns {string} + */ +export function valueLabel(providerId, value) { + if (!Number.isFinite(value)) return '--'; + if (providerId === 'eccc') return value > 10 ? '10+' : String(value); + return String(value); +} + +/** Plain-language risk phrase for a band id. */ +export function bandRiskText(bandId) { + const band = airQualityBandById(bandId); + return band ? `${band.label} health risk` : null; +} + +/** + * Validate a station FeatureCollection into an id-keyed catalog. + * + * Atomic: a structurally broken response returns null so the caller keeps its + * last good catalog. A station missing coordinates is skipped, since one + * unusable station is not evidence the catalog is broken. + * + * @param {object} geojson - GeoJSON FeatureCollection from `aqhi-stations`. + * @returns {Map|null} Stations by location_id, or null. + */ +export function normalizeAqhiStations(geojson) { + if (!Array.isArray(geojson?.features)) return null; + const stations = new Map(); + for (const feature of geojson.features) { + const properties = feature?.properties; + if ( + !properties || + typeof properties !== 'object' || + Array.isArray(properties) + ) + return null; + const coordinates = feature?.geometry?.coordinates; + if (!Array.isArray(coordinates) || coordinates.length < 2) continue; + const [lon, lat] = coordinates.map(Number); + if (!Number.isFinite(lon) || !Number.isFinite(lat)) continue; + if (Math.abs(lat) > 90 || Math.abs(lon) > 180) return null; + const id = String(properties.location_id ?? feature.id ?? '').trim(); + if (!id) continue; + stations.set(id, { + id, + name: String(properties.location_name_en || id).trim(), + // ECCC's administrative zone: atl, que, ont, pnr, pyr. + zone: String(properties['eccc_administrative-zone'] || '').trim(), + lat, + lon, + }); + } + return stations; +} + +/** + * Join latest ECCC observations onto known stations into provider-shaped rows. + * + * The catalog is the only thing that supplies a reading its coordinates, so an + * observation without a matching station cannot be placed and is dropped. + * Readings older than OBSERVATION_MAX_AGE_MS are dropped too — a six-hour-old + * air-quality number presented as current is worse than no number. + * + * @param {object} geojson - FeatureCollection from `aqhi-observations-realtime`. + * @param {Map} stations - Known stations by location_id. + * @param {number} [now=Date.now()] - Clock seam for tests. + * @returns {Array|null} Normalized readings, or null when malformed. + */ +export function normalizeAqhiObservations(geojson, stations, now = Date.now()) { + if (!Array.isArray(geojson?.features)) return null; + if (!(stations instanceof Map)) return null; + const provider = AIR_QUALITY_PROVIDERS.eccc; + const newest = new Map(); + for (const feature of geojson.features) { + const properties = feature?.properties; + if ( + !properties || + typeof properties !== 'object' || + Array.isArray(properties) + ) + return null; + const stationId = String(properties.location_id ?? '').trim(); + const station = stations.get(stationId); + if (!station) continue; + + const value = aqhiDisplayValue(properties.aqhi); + if (value === null) continue; + const observedMs = Date.parse(properties.observation_datetime); + if (!Number.isFinite(observedMs)) continue; + if (now - observedMs > OBSERVATION_MAX_AGE_MS) continue; + + const previous = newest.get(stationId); + if (previous && previous.observedMs >= observedMs) continue; + const band = bandForValue(provider.id, value); + newest.set(stationId, { + id: `${provider.id}:${stationId}`, + provider: provider.id, + scale: provider.scale, + value, + band, + stationId, + name: station.name, + zone: station.zone, + lat: station.lat, + lon: station.lon, + observedMs, + color: airQualityColor(band), + label: valueLabel(provider.id, value), + risk: bandRiskText(band), + note: String(properties.special_notes_en || '').trim() || null, + }); + } + return [...newest.values()].sort( + (a, b) => b.value - a.value || a.stationId.localeCompare(b.stationId), + ); +} + +/** + * Build the source-owned presentation for one station label. + * @param {object} input + * @returns {object} + */ +export function createAirQualityOverlayEntry({ + id, + position, + title, + accent, + value, +}) { + return { + id: String(id), + position, + variant: 'label', + title, + accent, + // Worse air wins the label budget — the reading people need to see first. + priority: Math.round(Number(value) || 0) * 1000, + collisionGroup: 'ambient-label', + paintLane: 'ambient-label', + interactive: false, + edgeFade: 'keyhole', + horizonCull: true, + terrainOcclusion: false, + gapPx: 14, + verticalOnly: true, + placement: 'above', + }; +} + +/** Keep the worst readings, with stable identity as the tie-break. */ +export function selectAirQualityOverlayCohort( + entries, + limit = AIR_QUALITY_OVERLAY_COHORT_LIMIT, +) { + const cap = Math.max( + 0, + Math.min(AIR_QUALITY_OVERLAY_COHORT_LIMIT, Math.floor(Number(limit) || 0)), + ); + if (!Array.isArray(entries) || cap === 0) return []; + return entries + .slice() + .sort( + (a, b) => + b.priority - a.priority || String(a.id).localeCompare(String(b.id)), + ) + .slice(0, cap); +} + +/** + * The worst readings inside a lon/lat rectangle, for the readout card. + * + * A null rectangle means the camera could not resolve one (a fully oblique or + * space-level view); everything is then in scope rather than nothing, so the + * card degrades to the worst readings overall instead of going blank. + * + * @param {Array} readings + * @param {?{west:number, south:number, east:number, north:number}} rect - Degrees. + * @param {number} [limit=5] + * @returns {Array} + */ +export function readingsInView(readings, rect, limit = 5) { + if (!Array.isArray(readings)) return []; + const cap = Math.max(0, Math.floor(Number(limit) || 0)); + if (cap === 0) return []; + const inside = !rect + ? readings + : readings.filter((reading) => { + if (reading.lat < rect.south || reading.lat > rect.north) return false; + // A rectangle crossing the antimeridian has west > east. + return rect.west <= rect.east + ? reading.lon >= rect.west && reading.lon <= rect.east + : reading.lon >= rect.west || reading.lon <= rect.east; + }); + return inside + .slice() + .sort((a, b) => b.value - a.value || a.stationId.localeCompare(b.stationId)) + .slice(0, cap); +} + +/** + * Map one reading to a JSON-safe analyst record. Pure — no Cesium types. + * @param {object|null|undefined} raw + * @param {number} [index=0] + * @returns {object} + */ +export function mapAnalystRecord(raw, index = 0) { + const num = (v) => (Number.isFinite(v) ? v : null); + const text = (v) => { + const t = String(v ?? '').trim(); + return t || null; + }; + return { + id: text(raw?.id) || `AIR-${String(index).padStart(4, '0')}`, + provider: text(raw?.provider), + scale: text(raw?.scale), + value: num(raw?.value), + band: text(raw?.band), + name: text(raw?.name), + zone: text(raw?.zone), + risk: text(raw?.risk), + lat: num(raw?.lat), + lon: num(raw?.lon), + observedMs: num(raw?.observedMs), + }; +} diff --git a/src/layers/airQuality/model.test.mjs b/src/layers/airQuality/model.test.mjs new file mode 100644 index 000000000..b4f1a643e --- /dev/null +++ b/src/layers/airQuality/model.test.mjs @@ -0,0 +1,269 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + AIR_QUALITY_BANDS, + AIR_QUALITY_PROVIDERS, + AIR_QUALITY_UNKNOWN_COLOR, + OBSERVATION_MAX_AGE_MS, + STALE_AFTER_MS, + airQualityColor, + aqhiDisplayValue, + bandForValue, + bandRiskText, + mapAnalystRecord, + normalizeAqhiObservations, + normalizeAqhiStations, + readingsAreStale, + readingsInView, + selectAirQualityOverlayCohort, + valueLabel, +} from './model.js'; + +const NOW = Date.parse('2026-09-24T18:30:00Z'); +const RECENT = '2026-09-24T18:00:00Z'; +const collection = (features) => ({ type: 'FeatureCollection', features }); +const station = (id, name, zone, lon, lat) => ({ + type: 'Feature', + geometry: { type: 'Point', coordinates: [lon, lat] }, + properties: { + location_id: id, + location_name_en: name, + 'eccc_administrative-zone': zone, + }, +}); +const observation = (id, aqhi, when = RECENT) => ({ + type: 'Feature', + properties: { location_id: id, aqhi, observation_datetime: when }, +}); + +const STATIONS = collection([ + station('IAKID', 'Calgary', 'pnr', -114.0575, 51.0458), + station('OAAAA', 'Toronto', 'ont', -79.3832, 43.6532), + station('AAAAA', 'Halifax', 'atl', -63.5752, 44.6488), +]); + +// ── provider-shaped records ──────────────────────────────────────────────── + +test('readings carry the provider-neutral {provider, scale, value, band} shape', () => { + // Indices are not interchangeable numbers: AQHI is 1-10+ on health risk, + // US AQI is 0-500 on concentration. The shape names which index a value is on + // so a second network can be added without inventing a shared meaning. + const stations = normalizeAqhiStations(STATIONS); + const [row] = normalizeAqhiObservations( + collection([observation('IAKID', 2.32)]), + stations, + NOW, + ); + assert.equal(row.provider, 'eccc'); + assert.equal(row.scale, 'AQHI'); + assert.equal(row.value, 2); + assert.equal(row.band, 'low'); +}); + +test('bands map from a value on a named provider scale', () => { + assert.equal(bandForValue('eccc', 1), 'low'); + assert.equal(bandForValue('eccc', 3), 'low'); + assert.equal(bandForValue('eccc', 4), 'moderate'); + assert.equal(bandForValue('eccc', 7), 'high'); + assert.equal(bandForValue('eccc', 11), 'very-high'); + // An unregistered provider cannot be banded rather than being guessed at. + assert.equal(bandForValue('unknown-network', 5), null); + assert.equal(bandForValue('eccc', null), null); +}); + +test('the provider states its coverage plainly', () => { + assert.match(AIR_QUALITY_PROVIDERS.eccc.coverage, /Canada only/); + assert.equal(AIR_QUALITY_PROVIDERS.eccc.scale, 'AQHI'); +}); + +test("values above ten collapse to the scale's open-ended label", () => { + assert.equal(valueLabel('eccc', 10), '10'); + assert.equal(valueLabel('eccc', 11), '10+'); + assert.equal(valueLabel('eccc', null), '--'); +}); + +test('every band has a distinct colour, and an unknown band is neutral', () => { + const colors = AIR_QUALITY_BANDS.map((b) => b.color); + assert.equal(new Set(colors).size, colors.length); + assert.equal(airQualityColor('nonsense'), AIR_QUALITY_UNKNOWN_COLOR); + assert.equal(bandRiskText('high'), 'High health risk'); + assert.equal(bandRiskText(null), null); +}); + +// ── published form ───────────────────────────────────────────────────────── + +test('AQHI rounds to the published integer form and floors at 1', () => { + assert.equal(aqhiDisplayValue(1.08), 1); + assert.equal(aqhiDisplayValue(3.5), 4); + assert.equal(aqhiDisplayValue(0), 1); + assert.equal(aqhiDisplayValue('3'), 3); +}); + +test('an absent reading never becomes a confident AQHI 1', () => { + // Number(null) and Number('') are both 0, which would floor to 1. + for (const empty of [null, undefined, '', ' ', NaN, true, {}, [], -1]) { + assert.equal(aqhiDisplayValue(empty), null, `${JSON.stringify(empty)}`); + } +}); + +// ── snapshot validation ──────────────────────────────────────────────────── + +test('stations normalize into an id-keyed catalog carrying their zone', () => { + const stations = normalizeAqhiStations(STATIONS); + assert.equal(stations.size, 3); + assert.equal(stations.get('OAAAA').zone, 'ont'); +}); + +test('malformed payloads are rejected atomically', () => { + assert.equal(normalizeAqhiStations(null), null); + assert.equal(normalizeAqhiStations(collection([{ properties: null }])), null); + const stations = normalizeAqhiStations(STATIONS); + assert.equal(normalizeAqhiObservations(null, stations, NOW), null); + assert.equal( + normalizeAqhiObservations(collection([]), 'not a map', NOW), + null, + ); +}); + +test('an observation with no matching station is dropped', () => { + const stations = normalizeAqhiStations(STATIONS); + const rows = normalizeAqhiObservations( + collection([observation('IAKID', 2), observation('ZZZZZ', 9)]), + stations, + NOW, + ); + assert.equal(rows.length, 1); +}); + +test('a stale reading is dropped rather than shown as current', () => { + const stations = normalizeAqhiStations(STATIONS); + const old = new Date(NOW - OBSERVATION_MAX_AGE_MS - 60_000).toISOString(); + const rows = normalizeAqhiObservations( + collection([observation('IAKID', 3, old), observation('OAAAA', 4)]), + stations, + NOW, + ); + assert.deepEqual( + rows.map((r) => r.stationId), + ['OAAAA'], + ); +}); + +test('a snapshot whose every observation was too old is stale, not nominal', () => { + const stations = normalizeAqhiStations(STATIONS); + const old = new Date(NOW - OBSERVATION_MAX_AGE_MS - 60_000).toISOString(); + const rows = normalizeAqhiObservations( + collection([observation('IAKID', 3, old), observation('OAAAA', 4, old)]), + stations, + NOW, + ); + assert.deepEqual(rows, []); + assert.equal(readingsAreStale(rows, NOW), true); + assert.equal(readingsAreStale([{ observedMs: NOW - 60_000 }], NOW), false); + assert.equal( + readingsAreStale([{ observedMs: NOW - STALE_AFTER_MS - 1 }], NOW), + true, + ); +}); + +test('the newest reading per station wins and worst air sorts first', () => { + const stations = normalizeAqhiStations(STATIONS); + const rows = normalizeAqhiObservations( + collection([ + observation('IAKID', 3, '2026-09-24T16:00:00Z'), + observation('IAKID', 7, '2026-09-24T18:00:00Z'), + observation('OAAAA', 5), + ]), + stations, + NOW, + ); + assert.deepEqual( + rows.map((r) => r.value), + [7, 5], + ); +}); + +// ── readout selection ────────────────────────────────────────────────────── + +const reading = (id, value, lat, lon) => ({ + id, + stationId: id, + value, + lat, + lon, + name: id, +}); + +test('the readout lists the worst readings inside the view rectangle', () => { + const rows = [ + reading('west', 9, 51.0, -114.0), + reading('east', 4, 43.6, -79.3), + reading('mild', 2, 51.1, -114.1), + ]; + const inView = readingsInView( + rows, + { west: -115, south: 50, east: -113, north: 52 }, + 5, + ); + assert.deepEqual( + inView.map((r) => r.id), + ['west', 'mild'], + ); +}); + +test('an unresolvable view rectangle degrades to the worst overall, not to blank', () => { + // A space-level or fully oblique camera cannot produce a rectangle; the card + // should still say something useful. + const rows = [reading('a', 3, 0, 0), reading('b', 8, 10, 10)]; + assert.deepEqual( + readingsInView(rows, null, 5).map((r) => r.id), + ['b', 'a'], + ); + assert.deepEqual(readingsInView(rows, null, 0), []); + assert.deepEqual(readingsInView(null, null, 5), []); +}); + +test('a view rectangle crossing the antimeridian still selects correctly', () => { + // west > east is how a dateline-crossing rectangle is expressed. + const rows = [reading('near', 5, 0, 179), reading('far', 9, 0, 100)]; + const inView = readingsInView( + rows, + { west: 170, south: -10, east: -170, north: 10 }, + 5, + ); + assert.deepEqual( + inView.map((r) => r.id), + ['near'], + ); +}); + +test('the overlay cohort is capped and deterministic', () => { + const entries = Array.from({ length: 120 }, (_, i) => ({ + id: `s${i}`, + priority: i % 11, + })); + const cohort = selectAirQualityOverlayCohort(entries); + assert.ok(cohort.length <= 48); + assert.deepEqual( + cohort.map((e) => e.id), + selectAirQualityOverlayCohort(entries).map((e) => e.id), + ); +}); + +test('analyst records are JSON-safe and name the provider and scale', () => { + const stations = normalizeAqhiStations(STATIONS); + const [row] = normalizeAqhiObservations( + collection([observation('IAKID', 2)]), + stations, + NOW, + ); + const record = mapAnalystRecord(row, 0); + assert.deepEqual(JSON.parse(JSON.stringify(record)), record); + assert.equal(record.provider, 'eccc'); + assert.equal(record.scale, 'AQHI'); + const empty = mapAnalystRecord(undefined, 7); + assert.equal(empty.id, 'AIR-0007'); + for (const [key, value] of Object.entries(empty)) { + assert.notEqual(value, undefined, `${key} must not be undefined`); + } +}); diff --git a/src/layers/airQuality/ownership.test.mjs b/src/layers/airQuality/ownership.test.mjs new file mode 100644 index 000000000..78ab8a0bb --- /dev/null +++ b/src/layers/airQuality/ownership.test.mjs @@ -0,0 +1,275 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createAirQualityLayer } from './index.js'; +import { layerFeedState } from '../../data/feedState.js'; +import { STALE_AFTER_MS } from './model.js'; + +function harness(source) { + const sources = []; + const events = []; + const viewer = { + dataSources: { + add(value) { + sources.push(value); + }, + remove(value) { + sources.splice(sources.indexOf(value), 1); + }, + }, + }; + const layer = createAirQualityLayer({ + source, + overlayHost: { + setEntries(...args) { + events.push(args); + }, + setVisible() {}, + clearSource() {}, + }, + }); + layer.init(viewer); + layer.enable(viewer); + return { layer, viewer, sources, events }; +} + +const reading = (id, value, overrides = {}) => ({ + id: `eccc:${id}`, + provider: 'eccc', + scale: 'AQHI', + value, + band: 'low', + stationId: id, + name: id, + zone: 'ont', + lat: 43.65, + lon: -79.38, + observedMs: 1_700_000_000_000, + color: '#4cc9f0', + label: String(value), + risk: 'Low health risk', + note: null, + ...overrides, +}); + +const staticSource = (rows) => ({ getSnapshot: async () => rows }); + +test('a layer requires both a snapshot source and an overlay host', () => { + assert.throws(() => createAirQualityLayer(), /snapshot source/); + assert.throws( + () => createAirQualityLayer({ source: staticSource([]) }), + /overlay host/, + ); +}); + +test('initialization is single-use and owns exactly one data source', () => { + const { layer, viewer, sources } = harness(staticSource([])); + assert.equal(sources.length, 1); + assert.throws(() => layer.init(viewer), /already initialized/); + layer.destroy(viewer); + assert.equal(sources.length, 0); +}); + +test('a refresh renders readings and publishes an overlay cohort', async () => { + const { layer, events } = harness( + staticSource([reading('A', 2), reading('B', 7)]), + ); + assert.equal(await layer.update(), true); + assert.equal(layer.getStats().count, 2); + assert.equal(layer.getStats().error, null); + assert.equal(events.at(-1)[0], 'air-quality'); +}); + +// ── the abort + enabled guard ────────────────────────────────────────────── + +test('a response landing after disable cannot redraw the scene', async () => { + // The failure this prevents: disable() clears the overlay, then an in-flight + // response resolves and repopulates it behind the user's back. + let release; + const gate = new Promise((resolve) => { + release = resolve; + }); + const { layer, events } = harness({ + getSnapshot: async ({ signal } = {}) => { + await gate; + signal?.throwIfAborted(); + return [reading('A', 5)]; + }, + }); + const pending = layer.update(); + layer.disable(); + const publishedBefore = events.length; + release(); + assert.equal(await pending, false, 'a late response must not be applied'); + assert.equal( + events.length, + publishedBefore, + 'no overlay write after disable', + ); + assert.equal(layer.getStats().count, 0); +}); + +test('disable aborts the request in flight', async () => { + let seenSignal = null; + const { layer } = harness({ + getSnapshot: async ({ signal } = {}) => { + seenSignal = signal; + await new Promise((resolve) => setTimeout(resolve, 40)); + signal?.throwIfAborted(); + return []; + }, + }); + const pending = layer.update(); + layer.disable(); + await pending; + assert.ok(seenSignal, 'a signal was passed through to the source'); + assert.equal(seenSignal.aborted, true); +}); + +test('a superseded refresh does not clobber the newer one', async () => { + let call = 0; + const { layer } = harness({ + getSnapshot: async ({ signal } = {}) => { + call += 1; + const mine = call; + await new Promise((resolve) => setTimeout(resolve, mine === 1 ? 40 : 1)); + signal?.throwIfAborted(); + return [reading(`call-${mine}`, mine)]; + }, + }); + const first = layer.update(); + const second = layer.update(); + const [firstResult, secondResult] = await Promise.all([first, second]); + assert.equal(firstResult, false, 'the superseded call reports no update'); + assert.equal(secondResult, true); + assert.equal(layer.getAnalystRecords()[0].name, 'call-2'); +}); + +test('update is a no-op while the layer is disabled', async () => { + const { layer } = harness(staticSource([reading('A', 2)])); + layer.disable(); + assert.equal(await layer.update(), false); + assert.equal(layer.getStats().count, 0); +}); + +// ── readout descriptor ───────────────────────────────────────────────────── + +test('the readout descriptor carries the band legend and coverage', async () => { + const { layer } = harness(staticSource([reading('A', 2), reading('B', 9)])); + await layer.update(); + const controls = layer.getRowControls(); + assert.equal(controls.readout, true); + assert.match(controls.summary.coverage, /Canada only/); + // The Weather panel's legend shape: one {color, label} per band. + assert.equal(controls.legendCategorical, true); + assert.deepEqual( + controls.legend.map((entry) => entry.label), + ['Low', 'Moderate', 'High', 'Very High'], + ); + assert.ok( + controls.legend.every((entry) => /^#[0-9a-f]{6}$/i.test(entry.color)), + ); + // Worst first, and every item keyed. + assert.deepEqual( + controls.list.items.map((i) => i.text), + ['B · Low health risk', 'A · Low health risk'], + ); + assert.ok(controls.list.items.every((i) => i.id)); + assert.match(controls.summary.status, /2 stations reporting/); +}); + +test('the readout reports an error rather than a silent empty card', async () => { + const { layer } = harness({ + getSnapshot: async () => { + throw new Error('ECCC HTTP 503'); + }, + }); + await layer.update(); + assert.match(layer.getRowControls().summary.status, /503/); +}); + +test('row-control listeners are notified on refresh and teardown-safe', async () => { + const { layer, viewer } = harness(staticSource([reading('A', 2)])); + let calls = 0; + layer.setRowControlsListener(() => { + calls += 1; + }); + await layer.update(); + assert.ok(calls >= 1, 'a refresh notifies the card'); + layer.destroy(viewer); + const after = calls; + layer.setRowControlsListener(() => { + calls += 1; + }); + assert.equal(calls, after); +}); + +// ── feed-state honesty ───────────────────────────────────────────────────── + +test('freshness is judged on observation time, not on when we last fetched', async () => { + // lastUpdate stays current every time the network answers, even when it + // answers with nothing new. Judging freshness on it would let the HUD and + // voice describe hours-old air quality as live. + const aged = Date.now() - STALE_AFTER_MS - 60_000; + const { layer } = harness( + staticSource([reading('A', 2, { observedMs: aged })]), + ); + await layer.update(); + const stats = layer.getStats(); + assert.ok( + stats.lastUpdate >= Date.now() - 5_000, + 'we did just receive a response', + ); + assert.equal(stats.observedAt, aged, 'but the observation itself is old'); + assert.equal(stats.stale, true); + assert.equal( + layerFeedState(stats), + 'stale', + 'provenance must not read this as nominal', + ); +}); + +test('a current reading reports nominal', async () => { + const { layer } = harness( + staticSource([reading('A', 2, { observedMs: Date.now() - 10 * 60_000 })]), + ); + await layer.update(); + const stats = layer.getStats(); + assert.equal(stats.stale, false); + assert.equal(layerFeedState(stats), 'nominal'); +}); + +test('a refresh that kept no current observation reports stale, not nominal', async () => { + // Every observation older than OBSERVATION_MAX_AGE_MS is dropped, so a feed + // that only has hours-old rows yields an empty snapshot. That is the + // stalest a feed can be; it must not read as a healthy empty layer. + const { layer } = harness(staticSource([])); + assert.equal(layer.getStats().stale, false, 'no refresh yet, no verdict'); + await layer.update(); + const stats = layer.getStats(); + assert.equal(stats.count, 0); + assert.equal(stats.observedAt, null); + assert.equal(stats.stale, true); + assert.equal(layerFeedState(stats), 'stale'); + assert.equal(layer.getRowControls().summary.status, 'No current readings'); +}); + +test('a failed refresh over good readings reports degraded, not nominal', async () => { + let fail = false; + const { layer } = harness({ + getSnapshot: async () => { + if (fail) throw new Error('ECCC HTTP 503'); + return [reading('A', 2, { observedMs: Date.now() - 10 * 60_000 })]; + }, + }); + await layer.update(); + assert.equal(layerFeedState(layer.getStats()), 'nominal'); + fail = true; + await layer.update(); + const stats = layer.getStats(); + assert.ok(stats.count > 0, 'the previous readings are still displayed'); + assert.equal( + layerFeedState(stats), + 'degraded', + 'showing old data after a failure is degraded', + ); +}); diff --git a/src/layers/airQuality/source.js b/src/layers/airQuality/source.js new file mode 100644 index 000000000..8fb4808ec --- /dev/null +++ b/src/layers/airQuality/source.js @@ -0,0 +1,104 @@ +import { readResponseJsonCapped } from '../../sources/httpBody.js'; +import { normalizeAqhiStations, normalizeAqhiObservations } from './model.js'; + +const COLLECTIONS_ROOT = 'https://api.weather.gc.ca/collections'; +const STATIONS_URL = `${COLLECTIONS_ROOT}/aqhi-stations/items`; +const OBSERVATIONS_URL = `${COLLECTIONS_ROOT}/aqhi-observations-realtime/items`; + +/** Comfortably above Canada's ~134 stations without inviting a huge page. */ +const STATION_LIMIT = 500; +const OBSERVATION_LIMIT = 500; +/** Station geometry is near-static — a few changes a year. Refetch twice a day. */ +const STATION_TTL_MS = 12 * 60 * 60 * 1000; +/** A full 500-row GeoJSON page is a few hundred KB; anything past this is not a page. */ +const MAX_RESPONSE_BYTES = 4 * 1024 * 1024; + +/** + * Request and validate a complete ECCC AQHI snapshot before it can replace + * displayed readings. The first registered air-quality provider. + * + * Two collections are involved. `aqhi-stations` is the catalog — it supplies + * every reading its coordinates and is cached for twelve hours because it + * barely changes. `aqhi-observations-realtime` is the hourly reading feed, + * requested with `latest=true` so the response is current values rather than + * the full observation history, which is roughly 2,000 rows nationally. + * + * The station catalog is only replaced when a refresh actually returns + * stations, so a transient empty response cannot blank the layer. + * + * @param {object} [options] + * @param {Function} [options.fetchImpl] - Injected fetch, for tests. + * @param {() => number} [options.now] - Clock seam, for tests. + * @param {number} [options.timeoutMs=15000] - Deadline for each request. + * @returns {{getSnapshot: Function}} + */ +export function createEcccAirQualitySource({ + fetchImpl = (...args) => globalThis.fetch(...args), + now = () => Date.now(), + timeoutMs = 15_000, +} = {}) { + /** @type {Map} */ + let stations = new Map(); + let stationsFetchedAt = 0; + + /** One bounded GeoJSON page: deadline, caller abort, byte cap, row cap. */ + const getJson = async (url, limit, signal) => { + const controller = new AbortController(); + const abort = () => controller.abort(signal.reason); + signal?.addEventListener('abort', abort, { once: true }); + const timer = setTimeout( + () => controller.abort(new Error('ECCC request timed out')), + timeoutMs, + ); + try { + signal?.throwIfAborted(); + const response = await fetchImpl(url, { + headers: { Accept: 'application/json' }, + signal: controller.signal, + }); + if (!response.ok) throw new Error(`ECCC HTTP ${response.status}`); + const payload = await readResponseJsonCapped( + response, + MAX_RESPONSE_BYTES, + controller.signal, + ); + // The server's `limit` is a request, not a guarantee. + if (Array.isArray(payload?.features) && payload.features.length > limit) + return { ...payload, features: payload.features.slice(0, limit) }; + return payload; + } finally { + clearTimeout(timer); + signal?.removeEventListener('abort', abort); + } + }; + + return { + async getSnapshot({ signal } = {}) { + const clock = now(); + if (stations.size === 0 || clock - stationsFetchedAt > STATION_TTL_MS) { + const payload = await getJson( + `${STATIONS_URL}?f=json&limit=${STATION_LIMIT}`, + STATION_LIMIT, + signal, + ); + const next = normalizeAqhiStations(payload); + if (!next) throw new Error('Malformed AQHI station response'); + // An empty refresh must not wipe a good catalog. + if (next.size > 0) { + stations = next; + stationsFetchedAt = clock; + } + } + if (stations.size === 0) throw new Error('No AQHI stations available'); + + const payload = await getJson( + `${OBSERVATIONS_URL}?f=json&latest=true&limit=${OBSERVATION_LIMIT}`, + OBSERVATION_LIMIT, + signal, + ); + const readings = normalizeAqhiObservations(payload, stations, clock); + if (!readings) throw new Error('Malformed AQHI observation response'); + return readings; + }, + }; +} diff --git a/src/layers/airQuality/source.test.mjs b/src/layers/airQuality/source.test.mjs new file mode 100644 index 000000000..b9ffe34a6 --- /dev/null +++ b/src/layers/airQuality/source.test.mjs @@ -0,0 +1,197 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createEcccAirQualitySource } from './source.js'; + +const NOW = Date.parse('2026-09-24T18:30:00Z'); +const HOUR = 60 * 60 * 1000; +const station = (id) => ({ + type: 'Feature', + geometry: { type: 'Point', coordinates: [-79.38, 43.65] }, + properties: { location_id: id, location_name_en: id }, +}); +const observation = (id, at = NOW, aqhi = 3) => ({ + type: 'Feature', + properties: { + location_id: id, + aqhi, + observation_datetime: new Date(at - 10 * 60_000).toISOString(), + }, +}); +const json = (body, init) => + new Response(JSON.stringify(body), { + headers: { 'content-type': 'application/json' }, + ...init, + }); + +/** A fetch that serves the station catalog and observations from mutable state. */ +function feed() { + const state = { + clock: NOW, + stations: [station('A')], + stationIds: null, + urls: [], + }; + state.source = createEcccAirQualitySource({ + now: () => state.clock, + fetchImpl: async (url) => { + state.urls.push(url); + return json({ + type: 'FeatureCollection', + // Every known station reports ten minutes before the current clock. + features: url.includes('aqhi-stations') + ? state.stations + : (state.stationIds || ['A']).map((id) => + observation(id, state.clock), + ), + }); + }, + }); + state.stationRequests = () => + state.urls.filter((url) => url.includes('aqhi-stations')).length; + return state; +} + +test('the station catalog is cached for twelve hours, then refetched', async () => { + const state = feed(); + await state.source.getSnapshot(); + state.clock = NOW + 12 * HOUR; + await state.source.getSnapshot(); + assert.equal(state.stationRequests(), 1, 'within the TTL the cache serves'); + state.clock = NOW + 12 * HOUR + 1; + await state.source.getSnapshot(); + assert.equal( + state.stationRequests(), + 2, + 'past the TTL the catalog refreshes', + ); +}); + +test('an empty catalog refresh keeps the previous catalog', async () => { + const state = feed(); + await state.source.getSnapshot(); + state.stations = []; + state.clock = NOW + 13 * HOUR; + const readings = await state.source.getSnapshot(); + assert.deepEqual( + readings.map((reading) => reading.stationId), + ['A'], + ); + // The empty refresh did not count as fresh, so the next call retries it. + await state.source.getSnapshot(); + assert.equal(state.stationRequests(), 3); +}); + +test('an empty first catalog is an error, not an empty layer', async () => { + const state = feed(); + state.stations = []; + await assert.rejects(state.source.getSnapshot(), /No AQHI stations/); +}); + +test('requests ask for JSON, the latest observations only, and a bounded page', async () => { + const state = feed(); + await state.source.getSnapshot(); + const [stations, observations] = state.urls.map((url) => new URL(url)); + assert.equal(stations.pathname, '/collections/aqhi-stations/items'); + assert.equal(stations.searchParams.get('f'), 'json'); + assert.equal(stations.searchParams.get('limit'), '500'); + assert.equal( + observations.pathname, + '/collections/aqhi-observations-realtime/items', + ); + // Without latest=true the collection returns the full history (~2,000 rows). + assert.equal(observations.searchParams.get('latest'), 'true'); + assert.equal(observations.searchParams.get('limit'), '500'); +}); + +test('rows past the requested limit are ignored', async () => { + const state = feed(); + state.stations = Array.from({ length: 600 }, (_, index) => + station(`S${String(index).padStart(3, '0')}`), + ); + state.stationIds = state.stations.map((row) => row.properties.location_id); + const readings = await state.source.getSnapshot(); + assert.equal(readings.length, 500); +}); + +test('a malformed catalog and HTTP errors reject', async () => { + await assert.rejects( + createEcccAirQualitySource({ + fetchImpl: async () => json({ features: 'no' }), + }).getSnapshot(), + /Malformed AQHI station/, + ); + await assert.rejects( + createEcccAirQualitySource({ + fetchImpl: async () => json({}, { status: 503 }), + }).getSnapshot(), + /ECCC HTTP 503/, + ); +}); + +test('an oversized response is refused before it is parsed', async () => { + const big = 'x'.repeat(4 * 1024 * 1024 + 1); + await assert.rejects( + createEcccAirQualitySource({ + fetchImpl: async () => + new Response(new Blob([big]).stream(), { + headers: { 'content-type': 'application/json' }, + }), + }).getSnapshot(), + (error) => error.code === 'RESPONSE_TOO_LARGE', + ); + await assert.rejects( + createEcccAirQualitySource({ + fetchImpl: async () => + json({}, { headers: { 'content-length': String(5 * 1024 * 1024) } }), + }).getSnapshot(), + (error) => error.code === 'RESPONSE_TOO_LARGE', + ); +}); + +test('a caller abort cancels the request and detaches its listener', async () => { + const caller = new AbortController(); + let requestSignal = null; + const source = createEcccAirQualitySource({ + fetchImpl: (_url, { signal }) => { + requestSignal = signal; + return new Promise((_resolve, reject) => + signal.addEventListener('abort', () => reject(signal.reason), { + once: true, + }), + ); + }, + }); + const pending = source.getSnapshot({ signal: caller.signal }); + caller.abort(new Error('layer disabled')); + await assert.rejects(pending, /layer disabled/); + assert.equal(requestSignal.aborted, true); + const already = new AbortController(); + already.abort(new Error('before start')); + let calls = 0; + await assert.rejects( + createEcccAirQualitySource({ + fetchImpl: async () => { + calls += 1; + return json({ features: [] }); + }, + }).getSnapshot({ signal: already.signal }), + /before start/, + ); + assert.equal(calls, 0); +}); + +test('a request that outlives its deadline is aborted', async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + const source = createEcccAirQualitySource({ + timeoutMs: 1_000, + fetchImpl: (_url, { signal }) => + new Promise((_resolve, reject) => + signal.addEventListener('abort', () => reject(signal.reason), { + once: true, + }), + ), + }); + const pending = source.getSnapshot(); + t.mock.timers.tick(1_000); + await assert.rejects(pending, /ECCC request timed out/); +}); diff --git a/src/layers/bikeshare/controls.js b/src/layers/bikeshare/controls.js index 4d9552b2d..5f73a2dc9 100644 --- a/src/layers/bikeshare/controls.js +++ b/src/layers/bikeshare/controls.js @@ -1,4 +1,4 @@ -import { t } from '../../i18n/index.js'; +import { msg, t } from '../../i18n/index.js'; import { STATUS_POLL_MS } from './policy.js'; export function createControls({ state: layerState, services, parts, source }) { @@ -57,7 +57,7 @@ export function createControls({ state: layerState, services, parts, source }) { /** * Return current layer statistics for the UI status display. - * @returns {{ count: number, lastUpdate: number|null, loading: boolean, loadingLabel?: string, error?: string }} + * @returns {{ count: number, lastUpdate: number|null, loading: boolean, loadingLabel?: string, error?: string, degraded?: boolean, stale?: boolean }} */ getStats() { const stats = { @@ -73,7 +73,21 @@ export function createControls({ state: layerState, services, parts, source }) { }) : t('scanning nearby systems...'); } - if (layerState._error) stats.error = layerState._error; + // A failed city is only the layer's failure when it left nothing on the + // map. While other cities still render, the partial failure reads + // DEGRADED; when every in-range city failed and only previously drawn + // points remain, they read STALE. + if (layerState._failedCityIds.size > 0) { + if (layerState._stationRenderMap.size === 0) { + stats.error = msg('GBFS fetch error'); + } else { + stats.degraded = true; + const allFailed = Array.from(layerState._activeCityIds).every( + (cityId) => layerState._failedCityIds.has(cityId), + ); + if (allFailed) stats.stale = true; + } + } return stats; }, }; diff --git a/src/layers/bikeshare/failureReporting.test.mjs b/src/layers/bikeshare/failureReporting.test.mjs new file mode 100644 index 000000000..89a87bdad --- /dev/null +++ b/src/layers/bikeshare/failureReporting.test.mjs @@ -0,0 +1,376 @@ +// BIKESHARE FAILURE REPORTING — one dead city is not a dead layer. +// +// City feeds are fetched independently, but the layer reported failure through +// a single flag. A city whose operator had retired its feed set that flag on +// every proximity check, and the only code that cleared it ran on a successful +// *activation* — which an already-loaded city never repeats. The result was a +// permanent LOAD FAILED over a map drawing a thousand live stations. +// +// These cases pin the rule that replaced it: the layer reports failure only +// when it has nothing to show, and a city that comes back clears its own +// failure rather than waiting for a neighbour to succeed. +// +// Run with: npm test +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { createState } from './state.js'; +import { createControls } from './controls.js'; + +function makeLayer() { + const services = { + overlays: { + setOverlayEntries() {}, + setOverlaySourceVisible() {}, + clearOverlaySource() {}, + }, + }; + const state = createState({ services }); + const parts = { queries: { collectDetectableStations: () => [] } }; + const { methods } = createControls({ state, services, parts, source: {} }); + return { state, methods }; +} + +/** Stand in for a rendered station without reaching into the renderer. */ +function renderStations(state, count) { + for (let i = 0; i < count; i += 1) state._stationRenderMap.set(`s${i}`, {}); + state._count = state._stationRenderMap.size; +} + +test('a failed city does not fail a layer that is drawing stations', () => { + const { state, methods } = makeLayer(); + renderStations(state, 1069); + state._failedCityIds.add('buffalo-reddy'); + + const stats = methods.getStats(); + assert.equal(stats.count, 1069); + assert.equal( + stats.error, + undefined, + 'a layer drawing 1069 stations reported LOAD FAILED', + ); +}); + +test('the layer reports failure when a failed city left it with nothing', () => { + const { state, methods } = makeLayer(); + state._failedCityIds.add('buffalo-reddy'); + + const stats = methods.getStats(); + assert.equal(stats.count, 0); + assert.equal(stats.error, 'GBFS fetch error'); +}); + +test('a layer with nothing to draw and nothing failed is not an error', () => { + const { state, methods } = makeLayer(); + + // Out of range of every city is empty, not broken. + assert.equal(methods.getStats().error, undefined); +}); + +test('a recovered city clears its own failure', () => { + const { state, methods } = makeLayer(); + state._failedCityIds.add('buffalo-reddy'); + assert.equal(methods.getStats().error, 'GBFS fetch error'); + + state._failedCityIds.delete('buffalo-reddy'); + assert.equal( + methods.getStats().error, + undefined, + 'the failure outlived the city that caused it', + ); +}); + +// The cases above pin how getStats() reads failure state. The cases below +// drive the real proximity check, status poll and lifecycle against a fake +// GBFS source, so the failure state they read is the state the layer builds. + +import * as Cesium from 'cesium'; +import { CITY_BY_ID } from './registry.js'; +import { createIngestion } from './ingestion.js'; +import { createModel } from './model.js'; +import { createViewport } from './viewport.js'; +import { createRendering } from './rendering.js'; +import { createLifecycle } from './lifecycle.js'; + +const TORONTO = 'toronto-bike-share'; +const BUFFALO = 'buffalo-reddy'; +// Within range of both Toronto and Buffalo. +const BETWEEN = { lat: 43.25, lon: -79.1 }; +// Within range of Toronto only. +const TORONTO_ONLY = { lat: 43.7, lon: -79.6 }; +// Within range of Buffalo only. +const BUFFALO_ONLY = { lat: 42.5, lon: -78.0 }; +const LOW_M = 10_000; + +// disable() detaches a keydown listener; node has no document. +globalThis.document ??= { + addEventListener() {}, + removeEventListener() {}, +}; + +/** A GBFS source whose per-city answers each test can change. */ +function makeSource() { + const cityOf = (url) => { + for (const [id, city] of CITY_BY_ID) { + if (url === city.stationInformationUrl) return { id, kind: 'info' }; + if (url === city.stationStatusUrl) return { id, kind: 'status' }; + } + throw new Error(`unexpected GBFS url ${url}`); + }; + const source = { + dead: new Set(), + deadStatus: new Set(), + /** Optional per-call override: (cityId, kind) => Promise | undefined. */ + hook: null, + async getStations(url, { signal } = {}) { + const { id, kind } = cityOf(url); + const hooked = source.hook?.(id, kind); + if (hooked) return hooked; + await new Promise((resolve) => setTimeout(resolve, 1)); + signal?.throwIfAborted(); + if (source.dead.has(id)) throw new Error('GBFS HTTP 404'); + if (kind === 'status' && source.deadStatus.has(id)) + throw new Error('GBFS HTTP 503'); + const city = CITY_BY_ID.get(id); + if (kind === 'info') { + return { + data: { + stations: [ + { + station_id: `${id}-1`, + name: `${id} dock`, + lat: city.centerLat, + lon: city.centerLon, + capacity: 10, + }, + ], + }, + }; + } + return { + data: { + stations: [ + { + station_id: `${id}-1`, + num_bikes_available: 3, + num_docks_available: 7, + is_installed: true, + is_renting: true, + is_returning: true, + }, + ], + }, + }; + }, + }; + return source; +} + +/** Assemble the layer from its real parts around a camera the test moves. */ +function makeLiveLayer() { + const source = makeSource(); + const services = { + render: { governorRequestRender() {} }, + overlays: { + setOverlayEntries() {}, + setOverlaySourceVisible() {}, + clearOverlaySource() {}, + }, + sprites: { registerSpriteCollection() {}, restoreSpriteOrder() {} }, + picking: { registerPickOwner() {}, unregisterPickOwner() {} }, + }; + const state = createState({ services }); + const parts = {}; + const context = { state, services, parts, source }; + parts.ingestion = createIngestion(context); + parts.model = createModel(context); + parts.viewport = createViewport(context); + parts.rendering = createRendering(context); + // Selection owns a DOM click handler; nothing here selects a station. + parts.selection = { + _installClickHandler() {}, + _clearSelection() {}, + _onKeyDown() {}, + }; + parts.queries = { collectDetectableStations: () => [] }; + parts.lifecycle = createLifecycle(context); + const { methods: controls } = createControls(context); + + const camera = { ...BETWEEN, height: LOW_M }; + const viewer = { + camera: { + get positionCartographic() { + return Cesium.Cartographic.fromDegrees( + camera.lon, + camera.lat, + camera.height, + ); + }, + computeViewRectangle: () => undefined, + changed: { addEventListener() {}, removeEventListener() {} }, + }, + scene: { globe: {}, primitives: { add() {}, remove() {} } }, + }; + parts.lifecycle.methods.init(viewer); + + return { + state, + source, + stats: () => controls.getStats(), + moveTo(place, height = LOW_M) { + Object.assign(camera, place, { height }); + return parts.viewport.runProximityCheck(); + }, + async enable() { + parts.lifecycle.methods.enable(viewer); + await settle(); + }, + disable: () => parts.lifecycle.methods.disable(viewer), + poll: () => parts.ingestion.methods.update(), + }; +} + +/** Let fire-and-forget activation started by enable() finish. */ +async function settle() { + for (let i = 0; i < 5; i += 1) + await new Promise((resolve) => setTimeout(resolve, 2)); +} + +function deferred() { + let resolve, reject; + const promise = new Promise((res, rej) => { + resolve = res; + reject = rej; + }); + return { promise, resolve, reject }; +} + +test('a city that fails beside one that renders reads DEGRADED, not healthy', async () => { + const layer = makeLiveLayer(); + layer.source.dead.add(BUFFALO); + await layer.enable(); + + const stats = layer.stats(); + assert.equal(stats.count, 1, 'Toronto should still draw'); + assert.equal(stats.error, undefined); + assert.equal(stats.degraded, true, 'the Buffalo failure was hidden'); + assert.equal(stats.stale, undefined); +}); + +test('a failed city that comes back clears its own failure', async () => { + const layer = makeLiveLayer(); + layer.source.dead.add(BUFFALO); + await layer.enable(); + assert.equal(layer.stats().degraded, true); + + layer.source.dead.clear(); + await layer.moveTo(BETWEEN); + const stats = layer.stats(); + assert.equal(stats.count, 2); + assert.equal(stats.degraded, undefined); + assert.equal(stats.error, undefined); +}); + +test('a status poll that recovers clears the failure the last poll set', async () => { + const layer = makeLiveLayer(); + await layer.moveTo(TORONTO_ONLY); + await layer.enable(); + layer.source.deadStatus.add(TORONTO); + await layer.poll(); + assert.equal(layer.stats().degraded, true); + + layer.source.deadStatus.clear(); + await layer.poll(); + assert.equal(layer.stats().degraded, undefined); + assert.equal(layer.stats().stale, undefined); +}); + +test('a failure does not outlive its city leaving range', async () => { + const layer = makeLiveLayer(); + layer.source.dead.add(BUFFALO); + await layer.enable(); + await layer.moveTo(BUFFALO_ONLY); + assert.equal(layer.stats().error, 'GBFS fetch error'); + + // Low over the open Atlantic: nothing in range is empty, not broken. + await layer.moveTo({ lat: 30, lon: -40 }); + const stats = layer.stats(); + assert.equal(stats.count, 0); + assert.equal(stats.error, undefined, 'LOAD FAILED over empty ocean'); + assert.equal(stats.degraded, undefined); +}); + +test('zooming out above the altitude gate drops a city failure', async () => { + const layer = makeLiveLayer(); + layer.source.dead.add(BUFFALO); + await layer.enable(); + await layer.moveTo(BUFFALO_ONLY); + assert.equal(layer.stats().error, 'GBFS fetch error'); + + await layer.moveTo(BUFFALO_ONLY, 200_000); + const stats = layer.stats(); + assert.equal(stats.count, 0); + assert.equal(stats.error, undefined, 'LOAD FAILED above the gate'); +}); + +test('a late rejection from before a disable/re-enable is ignored', async () => { + const layer = makeLiveLayer(); + await layer.moveTo(BUFFALO_ONLY); + const first = deferred(); + // The first info request hangs and ignores its abort signal. + layer.source.hook = (id, kind) => { + if (id !== BUFFALO || kind !== 'info') return undefined; + layer.source.hook = null; + return first.promise; + }; + await layer.enable(); + layer.disable(); + await layer.enable(); + assert.equal(layer.stats().count, 1, 're-enable should draw Buffalo'); + + first.reject(new Error('GBFS HTTP 500')); + await settle(); + const stats = layer.stats(); + assert.equal(stats.count, 1, 'a stale rejection tore down live points'); + assert.equal(stats.error, undefined); + assert.equal(stats.degraded, undefined); +}); + +test('a late status poll rejection from before a disable/re-enable is ignored', async () => { + const layer = makeLiveLayer(); + await layer.moveTo(TORONTO_ONLY); + await layer.enable(); + const first = deferred(); + layer.source.hook = (id, kind) => { + if (id !== TORONTO || kind !== 'status') return undefined; + layer.source.hook = null; + return first.promise; + }; + const poll = layer.poll(); + layer.disable(); + await layer.enable(); + + first.reject(new Error('GBFS HTTP 500')); + await poll; + const stats = layer.stats(); + assert.equal(stats.count, 1); + assert.equal(stats.degraded, undefined); + assert.equal(stats.error, undefined); +}); + +test('every status poll failing over cached points reads STALE, not fresh', async () => { + const layer = makeLiveLayer(); + await layer.moveTo(TORONTO_ONLY); + await layer.enable(); + const loadedAt = layer.stats().lastUpdate; + assert.ok(loadedAt, 'activation should record freshness'); + + layer.source.deadStatus.add(TORONTO); + await new Promise((resolve) => setTimeout(resolve, 5)); + await layer.poll(); + const stats = layer.stats(); + assert.equal(stats.count, 1, 'the cached points stay drawn'); + assert.equal(stats.error, undefined); + assert.equal(stats.degraded, true); + assert.equal(stats.stale, true); + assert.equal(stats.lastUpdate, loadedAt, 'a failed poll advanced freshness'); +}); diff --git a/src/layers/bikeshare/ingestion.js b/src/layers/bikeshare/ingestion.js index 64a4a3b96..f2127e53e 100644 --- a/src/layers/bikeshare/ingestion.js +++ b/src/layers/bikeshare/ingestion.js @@ -1,4 +1,3 @@ -import { msg } from '../../i18n/index.js'; import { CITY_BY_ID } from './registry.js'; export function createIngestion({ @@ -172,31 +171,34 @@ export function createIngestion({ const generation = layerState._proximityGeneration; const cityIds = Array.from(layerState._activeCityIds); + const isCurrent = (cityId) => + layerState._enabled && + layerState._activeCityIds.has(cityId) && + generation === layerState._proximityGeneration; + let refreshed = false; await Promise.all( cityIds.map(async (cityId) => { try { const statusMap = await loadCityStationStatus(cityId, generation); - if ( - !layerState._enabled || - !layerState._activeCityIds.has(cityId) || - generation !== layerState._proximityGeneration - ) - return; + if (!isCurrent(cityId)) return; parts.rendering.applyStatusToPoints(cityId, statusMap); + layerState._failedCityIds.delete(cityId); + refreshed = true; } catch (error) { - if (error?.name === 'AbortError') return; + if (error?.name === 'AbortError' || !isCurrent(cityId)) return; console.warn( `[Data:Bikeshare] ${cityId} status update error:`, error, ); - // English data (stats reach the voice tools); the panel translates it. - layerState._error = msg('GBFS status update failed'); + layerState._failedCityIds.add(cityId); } }), ); layerState._count = layerState._stationRenderMap.size; - if (layerState._count > 0) layerState._lastUpdate = Date.now(); + // Freshness advances only on a poll that actually refreshed a city. + if (refreshed && layerState._count > 0) + layerState._lastUpdate = Date.now(); }, }; diff --git a/src/layers/bikeshare/lifecycle.js b/src/layers/bikeshare/lifecycle.js index 17af85f11..3e46b5972 100644 --- a/src/layers/bikeshare/lifecycle.js +++ b/src/layers/bikeshare/lifecycle.js @@ -51,7 +51,7 @@ export function createLifecycle({ layerState._lastUpdate = null; layerState._loading = false; layerState._loadingOps = 0; - layerState._error = null; + layerState._failedCityIds.clear(); layerState._limitWarned = false; layerState._overlayHost.setVisible( @@ -75,7 +75,7 @@ export function createLifecycle({ */ enable(viewer) { layerState._enabled = true; - layerState._error = null; + layerState._failedCityIds.clear(); layerState._pointCollection.show = true; layerState._overlayHost.setVisible( BIKESHARE_SELECTED_OVERLAY_SOURCE_ID, diff --git a/src/layers/bikeshare/registry.js b/src/layers/bikeshare/registry.js index 5592078bc..78ebce9ce 100644 --- a/src/layers/bikeshare/registry.js +++ b/src/layers/bikeshare/registry.js @@ -234,18 +234,6 @@ const RAW_GBFS_CITY_REGISTRY = [ 'https://hon.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', provider: 'Biki', }, - { - id: 'columbus-cogo', - city: 'Columbus, OH', - centerLat: 39.9612, - centerLon: -82.9988, - loadRadiusKm: 90, - stationInformationUrl: - 'https://gbfs.cogobikeshare.com/gbfs/2.3/en/station_information.json', - stationStatusUrl: - 'https://gbfs.cogobikeshare.com/gbfs/2.3/en/station_status.json', - provider: 'CoGo', - }, { id: 'chattanooga-bikechatt', city: 'Chattanooga, TN', @@ -258,6 +246,18 @@ const RAW_GBFS_CITY_REGISTRY = [ 'https://chat.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', provider: 'Bike Chattanooga', }, + { + id: 'toronto-bike-share', + city: 'Toronto, ON', + centerLat: 43.6532, + centerLon: -79.3832, + loadRadiusKm: 90, + stationInformationUrl: + 'https://toronto.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://toronto.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', + provider: 'Bike Share Toronto', + }, bcycleEntry({ id: 'boulder-bcycle', city: 'Boulder, CO', @@ -325,14 +325,6 @@ const RAW_GBFS_CITY_REGISTRY = [ systemId: 'bcycle_pacersbikeshare', provider: 'Pacers Bikeshare', }), - bcycleEntry({ - id: 'fort-lauderdale-broward', - city: 'Fort Lauderdale, FL', - centerLat: 26.1224, - centerLon: -80.1373, - systemId: 'bcycle_broward', - provider: 'Broward B-cycle', - }), bcycleEntry({ id: 'memphis-bcycle', city: 'Memphis, TN', @@ -347,22 +339,32 @@ const RAW_GBFS_CITY_REGISTRY = [ centerLon: -93.625, systemId: 'bcycle_desmoines', }), - bcycleEntry({ + { + // Tugo moved off BCycle to PBSC; the bcycle_tugo feed is gone (404). id: 'tucson-tugo', city: 'Tucson, AZ', centerLat: 32.2226, centerLon: -110.9747, - systemId: 'bcycle_tugo', + loadRadiusKm: 100, + stationInformationUrl: + 'https://tucson.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://tucson.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', provider: 'Tugo', - }), - bcycleEntry({ + }, + { + // Trinity Metro moved off BCycle to PBSC; the bcycle_fortworth feed is gone. id: 'fort-worth-trinity', city: 'Fort Worth, TX', centerLat: 32.7555, centerLon: -97.3308, - systemId: 'bcycle_fortworth', + loadRadiusKm: 100, + stationInformationUrl: + 'https://fortworth.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://fortworth.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', provider: 'Trinity Metro', - }), + }, bcycleEntry({ id: 'omaha-heartland', city: 'Omaha, NE', @@ -386,14 +388,20 @@ const RAW_GBFS_CITY_REGISTRY = [ centerLon: -82.394, systemId: 'bcycle_greenville', }), - bcycleEntry({ + { + // Reddy left BCycle for Social Bicycles; the bcycle_reddy feed is gone + // (404). This is the endpoint the MobilityData GBFS registry lists. id: 'buffalo-reddy', city: 'Buffalo, NY', centerLat: 42.8864, centerLon: -78.8784, - systemId: 'bcycle_reddy', + loadRadiusKm: 100, + stationInformationUrl: + 'https://reddybikeshare.socialbicycles.com/opendata/station_information.json', + stationStatusUrl: + 'https://reddybikeshare.socialbicycles.com/opendata/station_status.json', provider: 'Reddy Bikeshare', - }), + }, bcycleEntry({ id: 'las-vegas-rtc-bike-share', city: 'Las Vegas, NV', diff --git a/src/layers/bikeshare/registry.test.mjs b/src/layers/bikeshare/registry.test.mjs new file mode 100644 index 000000000..74a103079 --- /dev/null +++ b/src/layers/bikeshare/registry.test.mjs @@ -0,0 +1,58 @@ +// BIKESHARE REGISTRY — a city entry is only useful if the proxy will serve it. +// +// A registry entry and the GBFS proxy allowlist are edited in different files, +// so a city can look correctly registered while every one of its requests is +// rejected at the proxy. These cases hold the two together: each entry is +// checked against the same host and path predicates the proxy enforces, so a +// new city that the proxy would refuse fails here rather than silently +// rendering an empty layer. +// +// Run with: npm test +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { GBFS_CITY_REGISTRY, CITY_BY_ID } from './registry.js'; +import { isAllowedGbfsHost, isAllowedGbfsPath } from '../../data/gbfsSource.js'; + +test('every registered city is reachable through the GBFS proxy allowlist', () => { + for (const city of GBFS_CITY_REGISTRY) { + for (const raw of [city.stationInformationUrl, city.stationStatusUrl]) { + const url = new URL(raw); + assert.equal( + isAllowedGbfsHost(url.hostname), + true, + `${city.id}: host ${url.hostname} is not allowlisted`, + ); + assert.equal( + isAllowedGbfsPath(url.pathname, url.hostname), + true, + `${city.id}: path ${url.pathname} is not allowlisted`, + ); + } + } +}); + +test('Bike Share Toronto is registered against the PBSC v2 feeds', () => { + const toronto = CITY_BY_ID.get('toronto-bike-share'); + assert.ok(toronto, 'toronto-bike-share is missing from the registry'); + assert.equal(toronto.city, 'Toronto, ON'); + assert.equal(toronto.provider, 'Bike Share Toronto'); + assert.equal( + toronto.stationInformationUrl, + 'https://toronto.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + ); + assert.equal( + toronto.stationStatusUrl, + 'https://toronto.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', + ); +}); + +test('the Toronto center sits inside the system service area', () => { + const toronto = CITY_BY_ID.get('toronto-bike-share'); + // Bounding box of the 1,069 live stations, sampled from the published + // station_information feed: 43.588..43.813 N, -79.603..-79.123 W. + assert.ok(toronto.centerLat > 43.588 && toronto.centerLat < 43.813); + assert.ok(toronto.centerLon > -79.603 && toronto.centerLon < -79.123); + // The system spans roughly 37 km east-west, so the load radius has to clear + // half that from the center or downtown pans lose the outer stations. + assert.ok(toronto.loadRadiusKm >= 25); +}); diff --git a/src/layers/bikeshare/state.js b/src/layers/bikeshare/state.js index 381384147..3617914bf 100644 --- a/src/layers/bikeshare/state.js +++ b/src/layers/bikeshare/state.js @@ -99,9 +99,17 @@ export function createState({ services }) { layerState._loadingOps = 0; - /** Most recent error message string, or null. */ + /** + * @type {Set} City ids whose most recent fetch failed. + * + * Per city rather than one layer-wide flag: feeds are fetched independently, + * so a retired operator must not mark a layer that other cities are still + * populating as failed. Entries are dropped when their city leaves range. + * The layer reports failure only when these cost it everything it had to + * draw, and degraded otherwise — see `getStats()` in controls.js. + */ - layerState._error = null; + layerState._failedCityIds = new Set(); /** Whether the MAX_TOTAL_POINTS cap warning has already been logged. */ diff --git a/src/layers/bikeshare/viewport.js b/src/layers/bikeshare/viewport.js index 92c7803b6..25a4df7d0 100644 --- a/src/layers/bikeshare/viewport.js +++ b/src/layers/bikeshare/viewport.js @@ -1,5 +1,4 @@ import * as Cesium from 'cesium'; -import { msg } from '../../i18n/index.js'; import { GBFS_CITY_REGISTRY } from './registry.js'; import { ACTIVATION_EXIT_ALTITUDE_M, @@ -123,6 +122,9 @@ export function createViewport({ state: layerState, services, parts, source }) { const cityIds = Array.from(layerState._activeCityIds); for (const cityId of cityIds) deactivateCity(cityId); layerState._activeCityIds.clear(); + // A failure belongs to a city in range; none are once everything is off. + layerState._failedCityIds.clear(); + layerState._count = 0; } /** @@ -169,12 +171,19 @@ export function createViewport({ state: layerState, services, parts, source }) { return; parts.rendering.applyStatusToPoints(cityId, statusMap); layerState._lastUpdate = Date.now(); - layerState._error = null; + layerState._failedCityIds.delete(cityId); } catch (error) { if (error?.name === 'AbortError') return; + // A rejection from superseded work must not mark (or tear down) a city + // that a newer proximity check or re-enable now owns. + if ( + !layerState._enabled || + !layerState._activeCityIds.has(cityId) || + generation !== layerState._proximityGeneration + ) + return; console.warn(`[Data:Bikeshare] ${cityId} activate error:`, error); - // English data (stats reach the voice tools); the panel translates it. - layerState._error = msg('GBFS fetch error'); + layerState._failedCityIds.add(cityId); deactivateCity(cityId); layerState._activeCityIds.delete(cityId); } finally { @@ -210,6 +219,10 @@ export function createViewport({ state: layerState, services, parts, source }) { deactivateCity(cityId); } } + // A failure never outlives its city's range. + for (const cityId of layerState._failedCityIds) { + if (!nextActive.has(cityId)) layerState._failedCityIds.delete(cityId); + } layerState._activeCityIds = nextActive; if (layerState._activeCityIds.size === 0) { diff --git a/src/layers/cyclones/index.js b/src/layers/cyclones/index.js index 67d1288fc..bcd66923a 100644 --- a/src/layers/cyclones/index.js +++ b/src/layers/cyclones/index.js @@ -437,6 +437,7 @@ export function createCyclonesLayer({ })), }, chips: [], + legendCategorical: true, legend: storm ? [ { diff --git a/src/layers/transit/lifecycle.test.mjs b/src/layers/transit/lifecycle.test.mjs index 795bbc654..8a2d39edf 100644 --- a/src/layers/transit/lifecycle.test.mjs +++ b/src/layers/transit/lifecycle.test.mjs @@ -3586,6 +3586,7 @@ test('actual Traffic, Bikeshare and Transit lifecycles retain sensitivity in eve _pointCollection: {}, _overlayHost: { setVisible: noop }, _cityRuntime: new Map(), + _failedCityIds: new Set(), }, services: { sprites: { restoreSpriteOrder: noop }, diff --git a/src/overlays/worldOverlayAllocation.test.mjs b/src/overlays/worldOverlayAllocation.test.mjs index bbaa4f168..e29785d48 100644 --- a/src/overlays/worldOverlayAllocation.test.mjs +++ b/src/overlays/worldOverlayAllocation.test.mjs @@ -11,6 +11,7 @@ import { EARTHQUAKE_OVERLAY_COHORT_LIMIT } from '../data/earthquakes.js'; import { ROCKET_MISSION_AMBIENT_OVERLAY_COHORT_LIMIT } from '../data/rocketLaunches.js'; import { RADIO_OVERLAY_COHORT_LIMIT } from '../data/radio.js'; import { CABLE_REFERENCE_LABEL_WINNER_CAP } from '../data/telegeographySubmarineCables.js'; +import { AIR_QUALITY_OVERLAY_COHORT_LIMIT } from '../layers/airQuality/model.js'; import { isCalibratedAllocationRuntime } from '../../scripts/run-unit-tests.mjs'; /** @@ -34,6 +35,7 @@ import { isCalibratedAllocationRuntime } from '../../scripts/run-unit-tests.mjs' * ---------------------+---------+------------+---------+----------------+-------------+------------- * generic below cap | 60 | 60 | 60 | 3182 | 53.0 | 4,100 * generic above cap | 250 | 250 | 96 | 10022 | 40.1 | 13,000 + * air quality | 48 | 48 | 32 | 3900 | 81.3 | 5,200 * local infrastructure | 320 | 320 | 192 | 39478 | 123.4 | 49,000 * Phase 3 + FIRMS | 338 | 338 | 210 | 41220 | 122.0 | 53,600 * Phase 3 + vessels | 451 | 451 | 311 | 67252 | 149.1 | 87,500 @@ -259,6 +261,23 @@ const WORKLOADS = [ maxBytesPerFrame: 19_000, saturated: true, }, + { + // The air-quality station cohort on its own. Measured standalone rather + // than stacked on the Phase-5 host (the submarine-cable precedent): these + // stations are sparse and, with the single registered provider, + // continental, so what matters is its own cohort cost. + // + // Re-measured against the current worker: median 3,900 B/frame, max 4,114, + // and 85.7 B/candidate at the ceiling — inside the shared 154 B/candidate + // bound, so no per-source exception is taken. The 5,200 frame budget + // carries 26.4% headroom over the measured maximum. + name: 'with the air-quality station cohort live', + profile: 'air-quality', + entries: AIR_QUALITY_OVERLAY_COHORT_LIMIT, + candidates: AIR_QUALITY_OVERLAY_COHORT_LIMIT, + maxBytesPerFrame: 5_200, + saturated: true, + }, { name: 'with the Dense detection lane active over 5,000 observations', profile: 'phase6-detection', diff --git a/src/overlays/worldOverlayAllocation.worker.mjs b/src/overlays/worldOverlayAllocation.worker.mjs index 49573cf1f..c9237eb95 100644 --- a/src/overlays/worldOverlayAllocation.worker.mjs +++ b/src/overlays/worldOverlayAllocation.worker.mjs @@ -26,6 +26,12 @@ import { TRACKED_OVERLAY_SOURCE_OPTIONS, } from '../data/trackedReadout.js'; import { CCTV_AMBIENT_CARD_MAX } from '../data/cctvLod.js'; +import { + AIR_QUALITY_OVERLAY_COHORT_LIMIT, + AIR_QUALITY_OVERLAY_COLLISION_CAPACITY, + AIR_QUALITY_OVERLAY_SOURCE_ID, + createAirQualityOverlayEntry, +} from '../layers/airQuality/model.js'; import { CCTV_OVERLAY_SOURCE_ID, createCctvThumbnailOverlayEntry, @@ -1110,6 +1116,69 @@ function buildSubmarineCablesWorkload(count) { ); } +/** + * The air-quality station cohort at full size. + * + * Measured standalone (the submarine-cable precedent) rather than stacked on + * the Phase-5 host: air-quality stations are sparse and, with the single + * currently-registered provider, continental — so the question a budget has to + * answer is what its own cohort costs, not what it adds to a worldwide worst + * case it never shares a viewport with. The entry shape is the same static + * `ambient-label` the earthquake layer publishes, so the per-candidate ceiling + * is the meaningful gate. + */ +function buildAirQualityWorkload(count) { + if (count !== AIR_QUALITY_OVERLAY_COHORT_LIMIT) { + throw new Error( + `air-quality requires ${AIR_QUALITY_OVERLAY_COHORT_LIMIT} entries`, + ); + } + const workload = { + entries: [], + positions: [], + drifts: [], + registrations: [], + }; + const columns = Math.max(1, Math.ceil(Math.sqrt((count * 16) / 9))); + const rows = Math.max(1, Math.ceil(count / columns)); + const spanX = columns > 1 ? 1.68 / (columns - 1) : 0; + const spanY = rows > 1 ? 1.56 / (rows - 1) : 0; + const stations = []; + for (let index = 0; index < count; index++) { + const column = index % columns; + const row = Math.floor(index / columns); + const baseX = -0.84 + column * spanX; + const baseY = -0.78 + row * spanY; + const position = new Cesium.Cartesian3(baseX, baseY, 0); + workload.positions.push(position); + workload.drifts.push({ + baseX, + baseY, + phase: index * 0.27, + rate: 0.32 + (index % 6) * 0.04, + }); + const entry = createAirQualityOverlayEntry({ + id: `air-${index}`, + position, + // i18n-ignore: synthetic allocation-benchmark fixture, not shown as UI + title: `Station ${index} ${1 + (index % 10)}`, + accent: '#4cc9f0', + value: 1 + (index % 10), + }); + entry.horizonCull = false; + stations.push(entry); + } + workload.registrations.push({ + sourceId: AIR_QUALITY_OVERLAY_SOURCE_ID, + entries: stations, + options: { + cohortLimit: AIR_QUALITY_OVERLAY_COHORT_LIMIT, + collisionCapacity: AIR_QUALITY_OVERLAY_COLLISION_CAPACITY, + }, + }); + return workload; +} + function buildPhase6DetectionWorkload(count) { const random = makeRandom(0xd37ec710); const positions = []; @@ -1215,9 +1284,13 @@ function main() { ? buildPhase5RocketMissionWorkload(ENTRY_COUNT) : PROFILE === 'all-live-radio' ? buildAllLiveRadioWorkload(ENTRY_COUNT) - : PROFILE === 'submarine-cables' - ? buildSubmarineCablesWorkload(ENTRY_COUNT) - : buildWorkload(ENTRY_COUNT); + : PROFILE === 'air-quality' + ? buildAirQualityWorkload(ENTRY_COUNT) + : PROFILE === 'submarine-cables' + ? buildSubmarineCablesWorkload( + ENTRY_COUNT, + ) + : buildWorkload(ENTRY_COUNT); const { entries, positions, drifts } = workload; const solveIntervalMs = Number(process.env.GEV_ALLOC_SOLVE_MS) || 125; const detectionActive = !!workload.detectionLayer; diff --git a/src/sources/httpBody.js b/src/sources/httpBody.js index 031e7ba51..9c8a28f04 100644 --- a/src/sources/httpBody.js +++ b/src/sources/httpBody.js @@ -55,7 +55,9 @@ export async function readResponseJsonCapped(response, maxBytes, signal) { /** * Read a fetch() Response body as bytes with the same hard cap as - * readResponseTextCapped — for protobuf upstreams (GTFS-Realtime). + * readResponseTextCapped — for protobuf and tile upstreams (GTFS-Realtime, + * TomTom flow). Same early Content-Length rejection, running cap while + * streaming and cancellation on abort, so callers can treat the two alike. * Throws { code:'RESPONSE_TOO_LARGE' }. */ export async function readResponseBytesCapped(response, maxBytes, signal) { diff --git a/src/sources/httpBody.test.mjs b/src/sources/httpBody.test.mjs new file mode 100644 index 000000000..aa8e77d1a --- /dev/null +++ b/src/sources/httpBody.test.mjs @@ -0,0 +1,133 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { readResponseBytesCapped } from './httpBody.js'; + +/** + * A Response that streams `chunk` forever until the reader cancels it. The + * zero high-water mark keeps the stream from pulling eagerly at construction, + * so `pulls()` counts only what a reader actually asked for. + */ +function endlessResponse(chunk, onCancel, headers = {}) { + let pulls = 0; + const stream = new ReadableStream( + { + pull(controller) { + pulls += 1; + controller.enqueue(chunk); + }, + cancel: onCancel, + }, + { highWaterMark: 0 }, + ); + return { response: new Response(stream, { headers }), pulls: () => pulls }; +} + +function chunkedResponse(chunks, headers = {}) { + const stream = new ReadableStream( + { + start(controller) { + for (const chunk of chunks) controller.enqueue(chunk); + controller.close(); + }, + }, + { highWaterMark: 0 }, + ); + return new Response(stream, { headers }); +} + +const bytes = (n, fill) => new Uint8Array(n).fill(fill); + +test('returns the exact bytes of a body that fits, untouched by text decoding', async () => { + // 0xFF and 0xFE are invalid UTF-8 lead bytes; a text path would mangle them. + const payload = Uint8Array.from([0xff, 0xfe, 0x00, 0x7f, 0x80, 0x01]); + const buf = await readResponseBytesCapped(chunkedResponse([payload]), 1024); + assert.ok(buf instanceof Uint8Array); + assert.deepEqual([...buf], [...payload]); +}); + +test('joins chunks in order and reports the total length', async () => { + const buf = await readResponseBytesCapped( + chunkedResponse([bytes(3, 1), bytes(2, 2), bytes(1, 3)]), + 1024, + ); + assert.deepEqual([...buf], [1, 1, 1, 2, 2, 3]); +}); + +test('refuses on a declared Content-Length without pulling the body', async () => { + let cancelled = false; + const { response, pulls } = endlessResponse( + bytes(512, 0), + () => { + cancelled = true; + }, + { 'content-length': '4096' }, + ); + await assert.rejects(readResponseBytesCapped(response, 1024), { + code: 'RESPONSE_TOO_LARGE', + }); + assert.equal(pulls(), 0); + assert.ok(cancelled); +}); + +test('refuses a chunked body the moment it passes the cap and stops pulling', async () => { + let cancelled = false; + const { response, pulls } = endlessResponse(bytes(512, 0), () => { + cancelled = true; + }); + await assert.rejects(readResponseBytesCapped(response, 1024), { + code: 'RESPONSE_TOO_LARGE', + }); + // Three 512-byte chunks is the first total over 1024; an uncapped read of + // this never-ending stream would not terminate at all. + assert.ok(pulls() <= 3, `pulled ${pulls()} chunks`); + assert.ok(cancelled); +}); + +test('accepts a body exactly at the cap', async () => { + const buf = await readResponseBytesCapped( + chunkedResponse([bytes(512, 7), bytes(512, 7)]), + 1024, + ); + assert.equal(buf.byteLength, 1024); +}); + +test('cancels a body still streaming when the signal aborts, and never returns a partial one', async () => { + const controller = new AbortController(); + let cancelled = false; + let pulls = 0; + const stream = new ReadableStream( + { + pull(ctrl) { + pulls += 1; + ctrl.enqueue(bytes(64, 0)); + if (pulls === 2) controller.abort(); + }, + cancel() { + cancelled = true; + }, + }, + { highWaterMark: 0 }, + ); + await assert.rejects( + readResponseBytesCapped(new Response(stream), 1 << 20, controller.signal), + ); + assert.ok(cancelled); + assert.ok(pulls <= 3, `pulled ${pulls} chunks after abort`); +}); + +test('refuses immediately when the signal is already aborted', async () => { + const controller = new AbortController(); + controller.abort(); + const { response, pulls } = endlessResponse(bytes(8, 0), () => {}); + await assert.rejects( + readResponseBytesCapped(response, 1024, controller.signal), + ); + assert.equal(pulls(), 0); +}); + +test('falls back to arrayBuffer() for a body with no stream reader', async () => { + const response = new Response(bytes(16, 9)); + const buf = await readResponseBytesCapped(response, 1024); + assert.equal(buf.byteLength, 16); + assert.equal(buf[0], 9); +}); diff --git a/src/sources/reference.js b/src/sources/reference.js index 8631d3ad2..35c0087b7 100644 --- a/src/sources/reference.js +++ b/src/sources/reference.js @@ -1,4 +1,5 @@ import { createUsgsEarthquakeSource } from '../layers/earthquakes/source.js'; +import { createEcccAirQualitySource } from '../layers/airQuality/source.js'; import { createWfigsPerimeterSource } from '../layers/perimeters/source.js'; import { createBundledCableSource } from '../layers/submarineCables/bundledSource.js'; import { createAdsbGnssSource } from '../layers/gnss/source.js'; @@ -7,6 +8,7 @@ import { createAdsbGnssSource } from '../layers/gnss/source.js'; export function createReferenceSources() { return { earthquakes: createUsgsEarthquakeSource(), + 'air-quality': createEcccAirQualitySource(), 'fire-perimeters': createWfigsPerimeterSource(), cables: createBundledCableSource(), 'gnss-integrity': createAdsbGnssSource(), diff --git a/src/sources/reference.test.mjs b/src/sources/reference.test.mjs index 9f9cc9d92..d4f56033e 100644 --- a/src/sources/reference.test.mjs +++ b/src/sources/reference.test.mjs @@ -14,15 +14,18 @@ test('reference factories retain compatibility without starting acquisition or s const second = createReferenceSources(); assert.deepEqual(Object.keys(first), [ 'earthquakes', + 'air-quality', 'fire-perimeters', 'cables', 'gnss-integrity', ]); assert.notEqual(first.earthquakes, second.earthquakes); + assert.notEqual(first['air-quality'], second['air-quality']); assert.notEqual(first['fire-perimeters'], second['fire-perimeters']); assert.notEqual(first.cables, second.cables); assert.notEqual(first['gnss-integrity'], second['gnss-integrity']); assert.equal(typeof first.earthquakes.getSnapshot, 'function'); + assert.equal(typeof first['air-quality'].getSnapshot, 'function'); assert.equal(typeof first['fire-perimeters'].getSnapshot, 'function'); assert.equal(typeof first.cables.fetch, 'function'); assert.equal(typeof first['gnss-integrity'].getSnapshot, 'function'); diff --git a/src/tooling/environmentProviders.test.mjs b/src/tooling/environmentProviders.test.mjs index f8f02595c..63795c16d 100644 --- a/src/tooling/environmentProviders.test.mjs +++ b/src/tooling/environmentProviders.test.mjs @@ -53,6 +53,29 @@ function isolate(t, env = {}) { } const json = (res) => JSON.parse(res.body); +/** A response that streams 1 MiB chunks of `text` well past every cap, then ends. */ +function oversizedResponse(text) { + const state = { pulled: 0, cancelled: false }; + const chunk = new TextEncoder().encode( + text.repeat(Math.ceil((1 << 20) / text.length)), + ); + const response = new Response( + new ReadableStream( + { + pull(controller) { + if (++state.pulled > 40) controller.close(); + else controller.enqueue(chunk); + }, + cancel() { + state.cancelled = true; + }, + }, + { highWaterMark: 0 }, + ), + ); + return { response, state }; +} + test('standalone composition mounts every extracted provider exactly once without acquisition', (t) => { t.mock.method(globalThis, 'fetch', () => { throw Error('construction must not fetch'); @@ -168,6 +191,29 @@ test('traffic middleware preserves keyless mode, caching, stale budget fallback assert.equal(calls, 2); }); +test('traffic middleware answers 502 to an oversized tile without caching it', async (t) => { + isolate(t, { TOMTOM_API_KEY: 'fixture-key', TOMTOM_DAILY_TILE_BUDGET: '' }); + const bodies = []; + t.mock.method(globalThis, 'fetch', async () => { + const oversized = oversizedResponse('\x01'); + bodies.push(oversized.state); + return oversized.response; + }); + const request = install(tomtomProxy()); + const first = await request('/flow/8/1/1.pbf'); + assert.equal(first.status, 502); + assert.equal(bodies[0].cancelled, true); + assert.ok(bodies[0].pulled <= 9, 'reading stops at the 8 MiB cap'); + assert.ok( + !fsp.writeFile.mock.calls.some(({ arguments: [file] }) => + String(file).endsWith('.pbf'), + ), + 'no tile reached the disk cache', + ); + assert.equal((await request('/flow/8/1/1.pbf')).status, 502); + assert.equal(bodies.length, 2, 'nothing was cached in memory'); +}); + test('FIRMS retains a large successful source during partial failure and filters stale data at serve time', async (t) => { isolate(t, { FIRMS_MAP_KEY: '' }); let now = Date.UTC(2026, 8, 12, 12); diff --git a/src/tooling/liveProviders.test.mjs b/src/tooling/liveProviders.test.mjs index d7b740668..a60a37cea 100644 --- a/src/tooling/liveProviders.test.mjs +++ b/src/tooling/liveProviders.test.mjs @@ -37,6 +37,30 @@ function install(plugin, preview = false) { }; } +/** A response that streams 1 MiB chunks of `text` well past every cap, then ends. */ +function oversizedResponse(text, init = {}) { + const state = { pulled: 0, cancelled: false }; + const chunk = new TextEncoder().encode( + text.repeat(Math.ceil((1 << 20) / text.length)), + ); + const response = new Response( + new ReadableStream( + { + pull(controller) { + if (++state.pulled > 40) controller.close(); + else controller.enqueue(chunk); + }, + cancel() { + state.cancelled = true; + }, + }, + { highWaterMark: 0 }, + ), + init, + ); + return { response, state }; +} + function environment(t, values) { for (const [key, value] of Object.entries(values)) { const original = process.env[key]; @@ -193,6 +217,130 @@ test('OpenSky global snapshot attempts are time-limited, retried once, then fall assert.equal(stalled.response.headers['x-flight-source'], 'adsb.lol'); }); +const openSkyAnonEnv = { + OPENSKY_CLIENT_ID: undefined, + OPENSKY_CLIENT_SECRET: undefined, + OPENSKY_AUTH_MODE: 'anon', + OPENSKY_USERNAME: undefined, + OPENSKY_PASSWORD: undefined, +}; +const regionalFallback = () => + Response.json({ + now: Date.now() / 1000, + ac: [{ hex: 'abc123', lat: 30, lon: -97, alt_baro: 10000 }], + }); + +test('OpenSky 429 starts the cooldown from headers and cancels the body unread', async (t) => { + environment(t, openSkyAnonEnv); + t.mock.method(console, 'warn', () => {}); + t.mock.method(console, 'error', () => {}); + let states = 0; + let limited; + t.mock.method(globalThis, 'fetch', async (url) => { + if (url.includes('/states/')) { + states++; + limited = oversizedResponse('x', { + status: 429, + headers: { 'x-rate-limit-retry-after-seconds': '600' }, + }); + return limited.response; + } + if (url.includes('/lat/')) return regionalFallback(); + throw Error(`Unexpected URL: ${url}`); + }); + const fresh = await import( + `../../server/providers/aircraft/opensky.js?ratelimit=${Math.random()}` + ); + const request = install(fresh.openSkyProxy()); + const first = await request('/api/flights', '?lat=30&lon=-97'); + assert.equal(first.statusCode, 200); + assert.equal(first.headers['x-flight-source'], 'adsb.lol'); + assert.equal(states, 1, 'a rate-limited snapshot is not retried'); + assert.equal(limited.state.cancelled, true); + assert.ok(limited.state.pulled <= 1, 'the 429 body is not read'); + const cooling = await request('/api/flights', '?lat=30&lon=-97'); + assert.equal(cooling.statusCode, 200); + assert.equal( + cooling.headers['x-opensky-auth-reason'], + 'opensky_cooldown_regional_fallback', + ); + assert.equal(states, 1, 'the cooldown holds off the next upstream attempt'); +}); + +test('OpenSky 429 without retry-after cools down for the 2 minute default', async (t) => { + environment(t, openSkyAnonEnv); + t.mock.method(console, 'warn', () => {}); + t.mock.method(console, 'error', () => {}); + const start = Date.now(); + let now = start; + t.mock.method(Date, 'now', () => now); + let states = 0; + t.mock.method(globalThis, 'fetch', async (url) => { + if (url.includes('/states/')) { + states++; + return new Response('{}', { status: 429 }); + } + if (url.includes('/lat/')) return regionalFallback(); + throw Error(`Unexpected URL: ${url}`); + }); + const fresh = await import( + `../../server/providers/aircraft/opensky.js?ratelimit-default=${Math.random()}` + ); + const request = install(fresh.openSkyProxy()); + await request('/api/flights', '?lat=30&lon=-97'); + assert.equal(states, 1); + // Past the 30 s floor but inside the 2 minute default. + now = start + 90_000; + await request('/api/flights', '?lat=30&lon=-97'); + assert.equal(states, 1, 'a missing header is not read as a zero wait'); +}); + +test('OpenSky does not retry a snapshot body over the cap', async (t) => { + environment(t, openSkyAnonEnv); + t.mock.method(console, 'warn', () => {}); + t.mock.method(console, 'error', () => {}); + const bodies = []; + t.mock.method(globalThis, 'fetch', async (url) => { + if (url.includes('/states/')) { + const oversized = oversizedResponse(' '); + bodies.push(oversized.state); + return oversized.response; + } + if (url.includes('/lat/')) return regionalFallback(); + throw Error(`Unexpected URL: ${url}`); + }); + const fresh = await import( + `../../server/providers/aircraft/opensky.js?oversize=${Math.random()}` + ); + const response = await install(fresh.openSkyProxy())( + '/api/flights', + '?lat=30&lon=-97', + ); + assert.equal(response.statusCode, 200); + assert.equal(response.headers['x-flight-source'], 'adsb.lol'); + assert.equal(bodies.length, 1, 'an oversized body is not fetched twice'); + assert.equal(bodies[0].cancelled, true); + assert.ok(bodies[0].pulled <= 9, 'reading stops at the 8 MiB cap'); +}); + +test('military aircraft route answers 502 to an oversized body without caching it', async (t) => { + t.mock.method(console, 'error', () => {}); + const bodies = []; + t.mock.method(globalThis, 'fetch', async () => { + const oversized = oversizedResponse('{"ac":[]}'); + bodies.push(oversized.state); + return oversized.response; + }); + const request = install(providers.adsbLolProxy()); + const first = await request('/api/military'); + assert.equal(first.statusCode, 502); + assert.equal(bodies[0].cancelled, true); + assert.ok(bodies[0].pulled <= 9, 'reading stops at the 8 MiB cap'); + const again = await request('/api/military'); + assert.equal(again.statusCode, 502, 'nothing was cached'); + assert.equal(bodies.length, 2); +}); + test('military aircraft route preserves fresh cache and stale response after upstream failure', async (t) => { let now = Date.now(); let calls = 0; diff --git a/src/tooling/spaceProviders.test.mjs b/src/tooling/spaceProviders.test.mjs index 6ccd8d57c..443ca5b96 100644 --- a/src/tooling/spaceProviders.test.mjs +++ b/src/tooling/spaceProviders.test.mjs @@ -47,6 +47,29 @@ function isolateDisk(t) { t.mock.method(fsp, 'writeFile', async () => {}); } +/** A response that streams 1 MiB chunks of `text` well past every cap, then ends. */ +function oversizedResponse(text) { + const state = { pulled: 0, cancelled: false }; + const chunk = new TextEncoder().encode( + text.repeat(Math.ceil((1 << 20) / text.length)), + ); + const response = new Response( + new ReadableStream( + { + pull(controller) { + if (++state.pulled > 40) controller.close(); + else controller.enqueue(chunk); + }, + cancel() { + state.cancelled = true; + }, + }, + { highWaterMark: 0 }, + ), + ); + return { response, state }; +} + test('portable requests keep fixed origins, encode group data and preserve the 30-day UTC window', () => { const tle = celestrakTleUrl('stations&FORMAT=json'); assert.equal(tle.origin, 'https://celestrak.org'); @@ -120,6 +143,27 @@ test('exported CelesTrak plugin coalesces refreshes, retains stale TLEs, and rea assert.equal(disk.body, tle); }); +test('CelesTrak answers 502 to an oversized TLE body without caching it', async (t) => { + isolateDisk(t); + t.mock.method(console, 'warn', () => {}); + const bodies = []; + t.mock.method(globalThis, 'fetch', async () => { + const oversized = oversizedResponse( + 'ISS\n1 25544U fixture\n2 25544 fixture\n', + ); + bodies.push(oversized.state); + return oversized.response; + }); + const request = install(celestrakProxy()); + const first = await request('/api/celestrak', '/stations'); + assert.equal(first.status, 502); + assert.equal(bodies[0].cancelled, true); + assert.ok(bodies[0].pulled <= 17, 'reading stops at the 16 MiB cap'); + assert.equal(fsp.writeFile.mock.callCount(), 0); + assert.equal((await request('/api/celestrak', '/stations')).status, 502); + assert.equal(bodies.length, 2, 'nothing was cached in memory'); +}); + for (const preview of [false, true]) test(`exported launch plugin preserves optional server auth and cache in ${preview ? 'preview' : 'development'}`, async (t) => { isolateDisk(t); diff --git a/src/ui/layerPanel.js b/src/ui/layerPanel.js index 44cfd1bd1..5f1cff5fe 100644 --- a/src/ui/layerPanel.js +++ b/src/ui/layerPanel.js @@ -90,6 +90,7 @@ const PANEL_GROUPS = [ { label: msg('Weather'), ids: [ + 'air-quality', 'wind', 'weather-radar', 'weather-satellite', @@ -569,6 +570,7 @@ export class LayerPanel { 'weather-satellite', 'weather-lightning', 'weather-cyclones', + 'air-quality', ].includes(layer.id), ) .map((layer) => ({ diff --git a/src/ui/layerPanel.test.mjs b/src/ui/layerPanel.test.mjs index 5e7305f80..b4b771e1a 100644 --- a/src/ui/layerPanel.test.mjs +++ b/src/ui/layerPanel.test.mjs @@ -329,3 +329,31 @@ test('the unavailable chip may abbreviate, but its accessible name keeps the ful assert.equal(button.dataset.feedState, 'unavailable'); assert.match(attrs.get('aria-label'), /^Active Fires: UNAVAILABLE\./); }); + +test('the Weather readout receives enabled air-quality alongside weather layers', async () => { + const { LayerPanel } = await import('./layerPanel.js'); + const layers = ['wind', 'air-quality', 'other'].map((id) => ({ + id, + icon: '*', + enabled: true, + })); + layers.push({ id: 'weather-radar', icon: '*', enabled: false }); + const panel = new LayerPanel({ + getLayers: () => layers, + getRowControls: (id) => ({ readout: true, summary: { label: id } }), + }); + let entries = null; + panel._weatherPanel = { + update: (next) => { + entries = next; + }, + }; + panel._refreshWeatherPanel(); + assert.deepEqual( + entries.map(({ id, summary }) => [id, summary.label]), + [ + ['wind', 'wind'], + ['air-quality', 'air-quality'], + ], + ); +}); diff --git a/src/ui/streetLevelPresentation.js b/src/ui/streetLevelPresentation.js index 579452336..0f468edd6 100644 --- a/src/ui/streetLevelPresentation.js +++ b/src/ui/streetLevelPresentation.js @@ -257,29 +257,45 @@ function presentMeta(state) { ); const hint = translateHint(state.coverage.hint); if (state.coverage.count > 0) { - // A provider whose lines are not sequences carries its wording as data - // (coverageLabel); its count line needs its own tn pair so every language - // gets real plural forms. New wording: add a pair here and to the catalogs. - const count = state.coverage.count; - const countLine = - state.coverage.label?.noun === 'streets' - ? tn( - count, - '{n} street in view · click a line for its nearest photo', - '{n} streets in view · click a line for its nearest photo', - { n: formatNumber(count) }, - ) - : tn( - count, - '{n} sequence in view · click a line for its photos', - '{n} sequences in view · click a line for its photos', - { n: formatNumber(count) }, - ); + const countLine = coverageCountLine(state.coverage); return [countLine, hint].filter(Boolean).join(' · '); } return hint; } +/** + * The coverage count line. Providers carry their wording as data + * (coverageLabel noun/action); each combination needs its own literal tn + * pair so every language gets real plural forms — the noun cannot inflect + * through a placeholder in Russian. New wording: add a pair here and to the + * catalogs (es.json/ru.json). + * @param {{count: number, label?: {noun?: string, action?: string}|null}} coverage + */ +function coverageCountLine(coverage) { + const count = coverage.count; + const values = { n: formatNumber(count) }; + if (coverage.label?.action === 'click a line for its nearest photo') + return coverage.label.noun === 'streets' + ? tn( + count, + '{n} street in view · click a line for its nearest photo', + '{n} streets in view · click a line for its nearest photo', + values, + ) + : tn( + count, + '{n} sequence in view · click a line for its nearest photo', + '{n} sequences in view · click a line for its nearest photo', + values, + ); + return tn( + count, + '{n} sequence in view · click a line for its photos', + '{n} sequences in view · click a line for its photos', + values, + ); +} + /** @param {{now?: number}} [options] Clock for the SINCE readout. */ export function presentStreetLevelPanel(state, { now = Date.now() } = {}) { const enabled = state.enabled === true; diff --git a/src/ui/weatherPanel.js b/src/ui/weatherPanel.js index f6a1b6cef..24ff83fd7 100644 --- a/src/ui/weatherPanel.js +++ b/src/ui/weatherPanel.js @@ -11,8 +11,9 @@ const ORDER = [ 'weather-radar', 'weather-satellite', 'weather-lightning', + 'air-quality', ]; -const OBSERVED = new Set(ORDER.slice(2)); +const OBSERVED = new Set(ORDER.slice(2, 5)); const utc = (time) => Number.isFinite(Date.parse(time)) ? `${formatUtcDayMonth(time)} ${utcClock(time)}` @@ -166,7 +167,10 @@ export function createWeatherPanel({ ? t('LATEST · newest per product') : historyTime(state.target), }); - const models = active.map(({ id, icon, summary, legend = [], list }) => { + const models = active.map((entry) => { + const { id, icon, summary, legend = [], list } = entry; + // Discrete classes (storm geometry, AQHI bands) list swatches, not a ramp. + const categorical = entry.legendCategorical === true; let detail = summary.detail; const product = state.products.find((item) => item.id === id); if (OBSERVED.has(id)) { @@ -207,17 +211,15 @@ export function createWeatherPanel({ id: 'legend', type: 'legend', legend: { - categorical: id === 'weather-cyclones', + categorical, colors: legend.map(({ color }) => color), labels: legend.map(({ label }) => label), - units: id === 'weather-cyclones' ? '' : summary.units, + units: categorical ? '' : summary.units, zeroIndex: legend.findIndex(({ label }) => label === '0'), }, }; const blocks = [ - ...(id === 'weather-cyclones' && list?.items?.length - ? [{ id: 'storms', type: 'list', list }] - : []), + ...(list?.items?.length ? [{ id: 'list', type: 'list', list }] : []), { id: 'details', type: 'lines', lines }, ...(legend.length ? [legendBlock] : []), ...(summary.settings?.length diff --git a/src/ui/weatherPanel.test.mjs b/src/ui/weatherPanel.test.mjs index 79f6802b5..868f7d9ba 100644 --- a/src/ui/weatherPanel.test.mjs +++ b/src/ui/weatherPanel.test.mjs @@ -3,6 +3,7 @@ import assert from 'node:assert/strict'; import { setLocale } from '../i18n/index.js'; import { createWeatherPanel } from './weatherPanel.js'; import { railFixture } from './railTestFixture.mjs'; +import { createAirQualityLayer } from '../layers/airQuality/index.js'; const ticks = ['2026-09-22T01:00:00Z', '2026-09-22T01:05:00Z']; const wind = { id: 'wind', @@ -167,6 +168,7 @@ test('cyclone readout retains advisory, position, intensity, geometry, coverage, }, ], }, + legendCategorical: true, legend: [ { label: 'Advisory center / forecast track', color: '#7fe6ed' }, { label: 'Center-track uncertainty cone', color: '#7fe6ed44' }, @@ -411,7 +413,7 @@ test('cyclones lead, and observed history owns one bordered group with only acti assert.deepEqual(opened(f), ['weather-cyclones']); assert.equal( card(f, cyclone.id).children[1].children[0].dataset.blockId, - 'storms', + 'list', ); view.update([wind, satellite]); assert.equal(group.children[1].textContent, 'Satellite clouds'); @@ -686,3 +688,67 @@ test('history time keeps the hour whatever the length of the localized date (K-6 } } }); + +test('air-quality readout renders its band legend, Canada coverage and worst-first list', async () => { + const observedMs = Date.now() - 10 * 60_000; + const reading = (stationId, value, band, name) => ({ + id: `eccc:${stationId}`, + provider: 'eccc', + scale: 'AQHI', + value, + band, + stationId, + name, + zone: 'on', + lat: 43.65, + lon: -79.38, + observedMs, + color: '#4cc9f0', + label: String(value), + risk: `${band} risk`, + note: null, + }); + const layer = createAirQualityLayer({ + source: { + getSnapshot: async () => [ + reading('B', 8, 'high', 'Hamilton'), + reading('A', 2, 'low', 'Toronto'), + ], + }, + overlayHost: { setEntries() {}, setVisible() {}, clearSource() {} }, + }); + layer.init({ dataSources: { add() {}, remove() {} } }); + layer.enable(); + await layer.update(); + const f = fixture(); + const view = createWeatherPanel(f); + view.update([{ id: layer.id, icon: layer.icon, ...layer.getRowControls() }]); + const aq = card(f, 'air-quality'); + assert.ok(aq, 'the air-quality card is shown'); + assert.equal(aq.dataset.open, 'true'); + assert.match( + f.find((n) => n.classList?.contains?.('rail-card-badge'), aq).textContent, + /Canada only/, + ); + assert.match(line(f, 'air-quality', 'status').textContent, /2 stations/); + const entries = f.find((n) => n.className === 'rail-card-legend-entries', aq); + assert.equal(entries.hidden, false); + assert.deepEqual( + entries.children.map((entry) => entry.children[1].textContent), + ['Low', 'Moderate', 'High', 'Very High'], + ); + assert.equal( + f.find((n) => n.className === 'rail-card-ramp', aq).hidden, + true, + ); + const list = f.find((n) => n.dataset.blockId === 'list', aq); + assert.equal(list.hidden, false); + assert.deepEqual( + list.children.map((item) => + item.children[0].children.map((part) => part.textContent).join(' '), + ), + ['AQHI 8 Hamilton · high risk', 'AQHI 2 Toronto · low risk'], + ); + view.destroy(); + layer.destroy(); +}); diff --git a/src/voice/actionSchemas.test.mjs b/src/voice/actionSchemas.test.mjs index c57b4a1ab..550d725e4 100644 --- a/src/voice/actionSchemas.test.mjs +++ b/src/voice/actionSchemas.test.mjs @@ -37,7 +37,8 @@ test('the complete Realtime tool payload pins the manifest-generated layer relea // omit top-level anyOf (OpenAI rejects it). Contacts list routing preserves // requested radius and center authority in descriptions only. // Street Level and GNSS extend the manifest; tracked and analyst aircraft describe broadcasts. - '9398815a573ab29ecc3ce527f6e1a2ad883804e7696e0444aebb11f973700d4b', + // Air quality (ECCC AQHI) extends the manifest. + '775ba323384b4c1b62fb8415bac52aecc3fc668e20bedcfa57e426dcd1da3969', ); }); diff --git a/src/voice/layerManifest.js b/src/voice/layerManifest.js index 286b8a5d8..eb0f09909 100644 --- a/src/voice/layerManifest.js +++ b/src/voice/layerManifest.js @@ -250,6 +250,24 @@ const layers = [ context: false, noQuery: 'map overlay; no countable records', }, + { + id: 'air-quality', + aliases: ['air quality', 'aqhi', 'smog', 'air pollution'], + context: false, + query: { + fields: { + value: ['n', 'AQHI'], + band: ['t'], + scale: ['t'], + provider: ['t'], + name: ['t'], + zone: ['t'], + risk: ['t'], + observedMs: ['ms'], + }, + timeField: 'observedMs', + }, + }, { id: 'traffic', aliases: ['street traffic', 'road traffic', 'congestion'],