diff --git a/CHANGELOG.md b/CHANGELOG.md index b0aa586a3..aaa32009a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,8 @@ # Changelog +- Overpass and military-installation disk caches no longer serve answers from a + previously configured `OVERPASS_UPSTREAMS` instance after it is repointed. + - A Radio Browser mirror answering 204, 205 or 304, or a status outside 200–599, no longer crashes the local dev/preview server. Null-body statuses get an empty body; anything else that cannot become a response fails that @@ -58,6 +61,10 @@ - Regional articles retain unknown publication times when compact source dates do not identify a real calendar instant. +- Add Detroit (MoGo), Pittsburgh (POGOH), Aspen (WE-cycle) and Kona & Hilo + (HIBike) to the GBFS bikeshare registry, the layer's first stations in + Michigan. + - Treat explicitly current, complete empty vessel coverage as a successful update, clearing obsolete contacts while retaining stale or incomplete snapshots. @@ -153,6 +160,36 @@ ## [Unreleased] +- Satellite clouds keep Clouds only / Full as quick endpoints and add a continuous + cloud-filter slider: 0% is Full, 100% is the existing brightness key, and + intermediate values blend the keyed alpha. The slider commits on release to + avoid restaging imagery for every drag event. Filter percentages round-trip in + share links; legacy mode codes retain their old endpoint meanings. Background + panel refreshes preserve the pending drag value until release. Contributed by + [NotProniss](https://github.com/NotProniss) (#1084). + +- Street Level: Panoramax is the fourth provider, keyless. Light its chip + (orange; off by default, one provider at a time): from orbit dots show + where the federated commons has pictures, and below about 60 km its + sequences draw as lines from `api.panoramax.xyz` vector tiles. A click on a + line opens that sequence's picture nearest the click without moving the + camera, flat or 360°, with cones along the sequence around it and ‹ › / + ← → following the picture's own links. The 360° and since filters apply, + in the lines and in the lookup. The caption names each picture's author + and licence (CC BY-SA 4.0, Licence Ouverte 2.0…) and links to it in the + Panoramax viewer. HD originals load once, only when the viewer is wider + than the 2,048 px version (a panorama: expanded), and every download stops + when the photo changes; an image whose header declares more than 64 MP is + refused before it decodes. Share links carry it as + `0.m.0_0.r.google-0*kartaview-0*panoramax-1` (`0.m.0_0.r.panoramax-1` works + too). KartaView's photo viewer + is now shared (`photoViewer.js`, `panorama.js`); in it, for KartaView too, + Esc clears the selected drive before it collapses the panel. Lines are + simplified and drawn within a point budget (a tile's unsimplified lines + over Paris hung the tab), decoded in slices so the globe keeps moving, + and kept; the cones' tiles have their own connection, and the open + picture's tile draws its cones before its neighbours arrive. + - Flights and Military Flights highlight emergency transponder broadcasts in fleet billboards, 3D models and Cockpit contacts. Colors update when a status starts or clears. Emergency squawks outrank ADS-B priority statuses; GEV @@ -235,6 +272,17 @@ panel, resize it from any edge or corner, and double-click the header to dock it again. The position persists across reloads. +- Add Norway's city bikes to the Bike Share layer: Oslo, Bergen and + Trondheim Bysykkel (Urban Sharing), Kolumbus Bysykkel in Stavanger and + Farte Bysykkel in Skien/Porsgrunn (Entur Mobility), and Bysykkel + Lillestrøm (JCDecaux), about 760 stations in all. Every feed is keyless + GBFS. `gbfs.urbansharing.com`, `api.entur.io` and `api.cyclocity.fr` join + the proxy allowlist; the last two are kept to their GBFS paths. The proxy + now sends `Client-Identifier: gods-eye-view` on every request and + `ET-Client-Name: gods-eye-view-bikeshare` to Entur, as the operators ask, + and accepts Entur's extensionless feed paths. Attributed under NLOD and + Licence Ouverte. `get_bike_share` covers the six systems too. + - Add Norway's road cameras from Statens vegvesen as a CCTV pack. The camera list is one keyless GeoJSON request to the agency's OGC API view of its DATEX 3.1 CCTV table, so no DATEX account is needed. Every working camera diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index d0897a04f..993271ef5 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -23,6 +23,7 @@ How to read this: | **Mapillary** (vector tiles, Graph API, MapillaryJS) | Street Level: coverage lines, image cones and the embedded viewer. Coverage tiles cached for 24 h | Imagery [CC BY-SA 4.0](https://www.mapillary.com/terms); API under the Mapillary terms with a client token | "© Mapillary contributors, CC BY-SA 4.0", shown on the globe while the provider is on | | **Google Street View** (Maps JavaScript API) | Street Level, on the Google 3D map only (Google's terms): Google's panorama viewer and nearest-panorama lookups; panorama dates and addresses kept for the session only | Google Maps Platform ToS (your own key + billing; every panorama load is billed) | "Street View imagery © Google" on the globe while the provider is on, and Google's logo and terms in the viewer | | **KartaView** (`api.kartaview.org` coverage tiles and photo API, `cdn.kartaview.org` image proxy, `storage*.openstreetcam.org` photos) | Street Level: coverage lines (z13–14 JSON tiles; with the imagery filter on, KartaView's filtered PNG tiles as a mask), nearest-photo lookups, the photo's sequence and the photo itself, fetched by the browser. Keyless; nothing cached beyond the session | Imagery [CC BY-SA 4.0](https://kartaview.org/terms) (KartaView terms §4); the public API needs no token | "© KartaView contributors, CC BY-SA 4.0", shown on the globe while the provider is on | +| **Panoramax** (`api.panoramax.xyz` meta-catalogue: vector tiles and STAC search; pictures from the federated instance holding each one, e.g. `panoramax.openstreetmap.fr`, `panoramax.ign.fr`) | Street Level: coverage lines and density points, nearest-picture lookups, a sequence's pictures as cones, and the picture itself, fetched by the browser. Keyless; tiles cached by the browser for an hour (the API's own header) | Per picture: mostly [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/), IGN's [Licence Ouverte 2.0](https://www.etalab.gouv.fr/licence-ouverte-open-licence/); the API needs no token | "© Panoramax contributors, licence per picture" on the globe while the provider is on; the caption names each picture's author and licence | | **OpenStreetMap** | Map and place data (roads, military areas and names, ALPR, place search, outlines); via vector tiles, hourly ALPR extract, bundled data, Nominatim and optional self-configured Overpass | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | © OpenStreetMap contributors | | **Google Map Tiles API** (Photorealistic 3D Tiles) + Places/Geocoding | The 3D globe, voice scene context, and on-demand nearby installation search | Google Maps Platform ToS (proprietary, your own key + billing) | "Google" / "Google Maps" logo — **shown in-app**, required | | **OpenSky Network** | Primary worldwide live-flight snapshot | Non-commercial research/education license | Schäfer et al., _"Bringing Up OpenSky"_, IPSN 2014 + opensky-network.org | @@ -60,6 +61,9 @@ How to read this: | **Statens vegvesen** (catalog `ogckart-sn1.atlas.vegvesen.no`, frames `kamera.atlas.vegvesen.no`, HLS `kamera.vegvesen.no`) | CCTV camera catalog + frames + live HLS video, Norway road cameras | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) distributed by Statens vegvesen" | | **NYC DOT Traffic Management Center** (`webcams.nyctmc.org`) | CCTV camera catalog + frames, New York City five boroughs | NYC DOT requires developers to sign a [data-sharing agreement](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) for the camera feed. **Off by default**; enable only under your own agreement | "NYC DOT Traffic Management Center" + NYC DOT's required application disclaimer (in the Data attribution popover) | | **GBFS (Lyft / BCycle)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` | +| **Oslo / Bergen / Trondheim Bysykkel** (Urban Sharing, `gbfs.urbansharing.com`) | Bikeshare availability, Oslo, Bergen and Trondheim city bikes. Keyless GBFS; the proxy sends the `Client-Identifier` header the operators ask for | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED ([Oslo](https://oslobysykkel.no/en/open-data/realtime), [Bergen](https://bergenbysykkel.no/en/apne-data/sanntid), [Trondheim](https://trondheimbysykkel.no/en/open-data/realtime)) | "Contains data under the Norwegian licence for Open Government data (NLOD) made available by Oslo Bysykkel, Bergen Bysykkel and Trondheim Bysykkel" | +| **Kolumbus / Farte Bysykkel** (Entur Mobility, `api.entur.io/mobility/v2/gbfs`) | Bikeshare availability, Kolumbus Bysykkel (Stavanger/Nord-Jæren) and Farte Bysykkel (Skien/Porsgrunn). Keyless GBFS; the proxy sends the `ET-Client-Name` header Entur requires and is limited to Entur's GBFS paths | [NLOD](https://data.norge.no/nlod/en/2.0) per the [Entur Mobility API docs](https://developer.entur.org/pages-mobility-docs-mobility-v2) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) made available by Entur (Kolumbus Bysykkel, Farte Bysykkel)" | +| **JCDecaux Cyclocity** (`api.cyclocity.fr`) | Bikeshare availability, Bysykkel Lillestrøm. Keyless GBFS; the proxy is limited to JCDecaux contract paths | [Licence Ouverte / Etalab](https://developer.jcdecaux.com/files/Open-Licence-fr.pdf) (the feed's `license_url`) — attribution REQUIRED | "Bysykkel Lillestrøm: JCDecaux, Licence Ouverte" | | **MBTA / MassDOT** (cdn.mbta.com) | Transit layer live vehicles and up to 15 minutes of recently observed positions, Greater Boston | MBTA / MassDOT open realtime vehicle data, used for the live view and up to fifteen minutes of recently observed positions; published for developer use and not against their use. | Attribution to MBTA / MassDOT (courtesy text) | | **CapMetro** (data.texas.gov) | Transit layer live vehicles, Austin TX | CapMetro open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to CapMetro (courtesy text) | | **Metro Transit** (Metropolitan Council) | Transit layer live vehicles, Minneapolis–St Paul | Metro Transit / Metropolitan Council open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to Metro Transit / Metropolitan Council (courtesy text) | diff --git a/README.md b/README.md index 95c8a24e0..554f9b57f 100644 --- a/README.md +++ b/README.md @@ -312,7 +312,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi | 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds | | 📹 **CCTV Mesh** | ~3,900 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | | 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 | -| 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer and **FOLLOW**. Mapillary, Google Street View and KartaView; Panoramax planned | Mapillary · Google · KartaView | 🟢 KartaView · 🟡 Mapillary token · 🔴 Google Maps key | +| 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer and **FOLLOW**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key | | 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 | | 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland | Operator GTFS-Realtime feeds | 🟢 | | 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 | @@ -350,13 +350,13 @@ _The Space Missions layer replaying a Falcon 9 ascent — labeled `RECONSTRUCTED ## 📷 Street Level -> Street-level photos from any provider in one panel. This build ships **Mapillary** (a free client token), **Google Street View** (your Google Maps key) and **KartaView** (no key). +> Street-level photos from any provider in one panel. This build ships **Mapillary** (a free client token), **Google Street View** (your Google Maps key), **KartaView** and **Panoramax** (no key). -**Setup.** At [mapillary.com/dashboard/developers](https://www.mapillary.com/dashboard/developers), register an application with **Read** access and copy its **Client Token** (`MLY|…`). Paste it into **POWER UP → Mapillary**, or set `MAPILLARY_CLIENT_TOKEN` in `.env` and restart. Without it the panel reads **KEY REQUIRED**. **Google Street View** uses `GOOGLE_MAPS_API_KEY` with the **Maps JavaScript API** enabled for it in Google Cloud; its chip starts off, since every panorama is billed to the key. **KartaView** needs nothing: its public API is keyless. +**Setup.** At [mapillary.com/dashboard/developers](https://www.mapillary.com/dashboard/developers), register an application with **Read** access and copy its **Client Token** (`MLY|…`). Paste it into **POWER UP → Mapillary**, or set `MAPILLARY_CLIENT_TOKEN` in `.env` and restart. Without it the panel reads **KEY REQUIRED**. **Google Street View** uses `GOOGLE_MAPS_API_KEY` with the **Maps JavaScript API** enabled for it in Google Cloud; its chip starts off, since every panorama is billed to the key. **KartaView** and **Panoramax** need nothing: their public APIs are keyless. -**Use.** Turn on **Street Level** under DATA LAYERS → Cameras, or with the pill in the STREET LEVEL panel header. Each provider has a chip in its own colour (Mapillary green), and one is lit at a time: lighting one switches the others off. Mapillary, which is free, is lit by default. The **IMAGERY** (ALL / 360° / FLAT) and **SINCE** filters apply to Mapillary and KartaView; Street View panoramas are all 360°, so with Street View lit the filters and legend are hidden. Click a coverage line for its image cones and a cone to open the photo. Light **KARTAVIEW** (purple) and zoom in below about 6 km: its lines are the streets KartaView has photos of, and a click on one opens the photo nearest the click, flat or 360° (drag to look around), with cones along its drive; **‹ ›** or ← → step through the drive (Shift+← →, ↑ ↓ and + − look around a 360° photo; pinch zooms on a phone). Photos KartaView has archived are skipped, and where all are archived the panel says so. For KartaView, **SINCE** goes by upload date, as KartaView's own map does (it can filter by nothing else), and the panel notes it under the slider; KartaView filters its lines on its own server, and a filtered area KartaView has not drawn before can take several seconds, and the unfiltered lines show meanwhile. On the **Google 3D** map (Google's terms keep Street View off other maps, where it does not appear), light **GOOGLE STREET VIEW** and click a street from below 3 km, or press **OPEN STREET VIEW** for the view centre, to open Google's panorama there; step along its arrows. Above the photo: **EXPAND**, **FIT / FILL** (Mapillary, and flat KartaView photos), and **FOLLOW** (the globe camera follows the photo's view; Google 3D map only). The caption links to the image on the provider's site, and the globe credits each provider that is on ("© Mapillary contributors, CC BY-SA 4.0", "Street View imagery © Google", "© KartaView contributors, CC BY-SA 4.0"). KartaView's own photo pages fail for signed-out visitors, so its link opens the full-size photo. +**Use.** Turn on **Street Level** under DATA LAYERS → Cameras, or with the pill in the STREET LEVEL panel header. Each provider has a chip in its own colour (Mapillary green), and one is lit at a time: lighting one switches the others off. Mapillary, which is free, is lit by default. The **IMAGERY** (ALL / 360° / FLAT) and **SINCE** filters apply to Mapillary, KartaView and Panoramax; Street View panoramas are all 360°, so with Street View lit the filters and legend are hidden. Click a coverage line for its image cones and a cone to open the photo. Light **KARTAVIEW** (purple) and zoom in below about 6 km: its lines are the streets KartaView has photos of, and a click on one opens the photo nearest the click, flat or 360° (drag to look around), with cones along its drive; **‹ ›** or ← → step through the drive (Shift+← →, ↑ ↓ and + − look around a 360° photo; pinch zooms on a phone). Photos KartaView has archived are skipped, and where all are archived the panel says so. For KartaView, **SINCE** goes by upload date, as KartaView's own map does (it can filter by nothing else), and the panel notes it under the slider; KartaView filters its lines on its own server, and a filtered area KartaView has not drawn before can take several seconds, and the unfiltered lines show meanwhile. Light **PANORAMAX** (orange) for the open, federated commons (strongest in France and Belgium): from orbit its dots show where it has pictures, and below about 60 km its lines are sequences. A click on a line opens that sequence's picture nearest the click without moving the camera, flat or 360°, with cones along the sequence around it; **‹ ›** or ← → step through it, and both filters apply. Its caption names the picture's authors (or the instance holding it) and licence (mostly CC BY-SA 4.0; pictures on the IGN instance are Licence Ouverte 2.0). On the **Google 3D** map (Google's terms keep Street View off other maps, where it does not appear), light **GOOGLE STREET VIEW** and click a street from below 3 km, or press **OPEN STREET VIEW** for the view centre, to open Google's panorama there; step along its arrows. Above the photo: **EXPAND**, **FIT / FILL** (Mapillary, and flat KartaView and Panoramax photos), and **FOLLOW** (the globe camera follows the photo's view; Google 3D map only). The caption links to the image on the provider's site, and the globe credits each provider that is on ("© Mapillary contributors, CC BY-SA 4.0", "Street View imagery © Google", "© KartaView contributors, CC BY-SA 4.0", "© Panoramax contributors, licence per picture"). KartaView's own photo pages fail for signed-out visitors, so its link opens the full-size photo. -Drag the panel header to float the panel, and resize it from any edge. Double-click the header or collapse it to put it back in the rail. Share links carry the provider switches and filters (`0.m.0` Mapillary off, `0.m.0_0.r.google-1` Street View on, `0.m.0_0.r.google-0*kartaview-1` KartaView on, as the app writes it (`0.m.0_0.r.kartaview-1` works too; a link that leaves out `m` keeps Mapillary, the default, lit), `0.p.p` panoramas only, `0.s.365` the last year). To add a provider, implement the contract in `src/layers/streetLevel/registry.js` and register it in `src/app/layers/streetLevel.js`. +Drag the panel header to float the panel, and resize it from any edge. Double-click the header or collapse it to put it back in the rail. Share links carry the provider switches and filters (`0.m.0` Mapillary off, `0.m.0_0.r.google-1` Street View on, `0.m.0_0.r.google-0*kartaview-1*panoramax-0` KartaView on and `0.m.0_0.r.google-0*kartaview-0*panoramax-1` Panoramax on, as the app writes them (`0.m.0_0.r.kartaview-1` and `0.m.0_0.r.panoramax-1` work too; a link that leaves out `m` keeps Mapillary, the default, lit), `0.p.p` panoramas only, `0.s.365` the last year). To add a provider, implement the contract in `src/layers/streetLevel/registry.js` and register it in `src/app/layers/streetLevel.js`. ## 🎖️ Field Missions diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 5df22e452..3eb78c290 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -256,8 +256,15 @@ Clouds only applies a soft brightness ramp to decoded pixels once, using Cesium's sRGB-to-linear conversion (`channel ** 2.2`) and smoothstep from 0.40 to 0.70. The old 0.55 threshold is the ramp midpoint. RGB and source alpha are preserved in Full image mode; both modes use the chosen layer opacity. This -is a display filter, not a cloud mask. Satellite share links retain the display -mode; observation history remains transient and links open latest. +is a display filter, not a cloud mask. The cloud-filter slider is a continuous +0–100% strength: 0% is Full, 100% is the existing Clouds only key, and +intermediate values blend their alpha. The value commits on slider release to +avoid restaging for every pointer event. Status refreshes retain the uncommitted +drag preview; changed model values or disabled controls supersede it. +Satellite share links carry the filter +strength; legacy `f`/`a` values keep their Clouds only / Full meanings. Soft and +Vivid remain layer-opacity presets (40%/70%); history remains transient and links +open latest. Global infrared fetches one capped 4 MiB, 2048×1024 mosaic per frame and decodes and processes it before staging. Globe hosts crop that canvas into 256 px tiles to maximum level 3 on a geographic 2×1 root grid bounded to the manifest extent, @@ -302,7 +309,8 @@ release commits immediately. Latest selects the newest frame per product, and history labels the actual displayed frame as synced or nearest. Missing-frame messages use each product's eligibility gap (30 minutes, or three hours for the global mosaic). Wind shows forecast valid and issue times and does not follow -history. Satellite clouds uses Clouds only / Full image configuration labels. +history. Satellite clouds uses Clouds only / Full controls, a continuous cloud +filter slider and Soft/Vivid opacity presets in its right-rail settings. Left weather rows retain only the toggle and one status line with a source tooltip. Configuration and the cyclone Storms list live in the cards, as do the @@ -1174,6 +1182,15 @@ The station reader addresses the proxy as `/api/gbfs/`; the feed address is read from the path, and a query-string form is refused with a 400. GBFS refuses upstream redirects and enforces its 5 MiB response cap while streaming. The 12-second deadline includes reading the body, and rejected or stalled downloads are cancelled. Development and preview use the same handler. +Every upstream request carries `Client-Identifier: gods-eye-view`, which Urban +Sharing asks of clients of the Oslo, Bergen and Trondheim Bysykkel feeds +(`gbfs.urbansharing.com`); other operators ignore it. Requests to Entur +(`api.entur.io`, Kolumbus and Farte Bysykkel) also carry +`ET-Client-Name: gods-eye-view-bikeshare`. Entur serves its feeds without a +`.json` suffix, so the endpoint check accepts `station_information` and +`station_status` with or without it. Hosts that front more than GBFS are kept to +their GBFS tree: `api.entur.io` to `/mobility/v2/gbfs/` and `api.cyclocity.fr` +(Bysykkel Lillestrøm) to `/contracts/`. ## Remaining local service modules @@ -3189,7 +3206,7 @@ its criteria cannot be silently ignored. | CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + NYC DOT TMC (opt-in) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) | | Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh | | Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) | -| Bikeshare 🚲 | GBFS (Lyft + BCycle) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | +| Bikeshare 🚲 | GBFS (Lyft + BCycle + Urban Sharing, Entur, JCDecaux NO) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | | Directions 🧭 | OSRM on FOSSGIS servers (OpenStreetMap) | `src/data/directions.js` | `/api/route` (`steps=1`) | on placement / mode change | | Datacenters ▣ | OSM extract (bundled) | `src/data/localLayers.js` | — | static | | Dams ▰ | OpenInfraMap/OSM extract (bundled) | `src/data/localLayers.js` | — | static | @@ -3238,7 +3255,7 @@ provider draws in one colour (`PROVIDER_COLORS` in `src/layers/streetLevel/policy.js`); 360° cones are rings and the selected sequence is GEV cyan. Share options: `m` (Mapillary), `r` (the other providers' switches by id, e.g. `google-1` for Street View), `p` (`a`/`p`/`f` panoramas), `s` (since, days). Mapillary, Google -Street View and KartaView are registered. Without +Street View, KartaView and Panoramax are registered. Without `MAPILLARY_CLIENT_TOKEN` the panel reads KEY REQUIRED and the controls are disabled. With it, coverage draws as z0–5 overview points from orbit and z11–14 sequence lines below 60 km; the proxy strips the unused `image` layer from z14 @@ -3434,14 +3451,88 @@ photo that was opening is drawn on the new canvas instead. FIT / FILL hide for panoramas (`supportsRenderModes`). The caption has no place name (KartaView's is where the drive began) and links to the full-size photo through the image proxy, since kartaview.org's photo -pages fail for signed-out visitors. +pages fail for signed-out visitors. The viewer itself is `photoViewer.js`, +shared with Panoramax: each provider hands it a `PhotoSources` table (first +and sharper flat and panorama URLs, and the caption fields). + +Panoramax (`providers/panoramax/`) is keyless and browser-side: the +meta-catalogue `api.panoramax.xyz` (CORS open) federates every instance, and +each picture is served by the instance holding it (OpenStreetMap France, +IGN, MapComplete…; IGN's redirect to its OVH object storage). Its chip +(orange, `#ff6f00`, Panoramax's own) starts off. Coverage comes from its +vector tiles (`/api/map/{z}/{x}/{y}.mvt`, an hour's HTTP cache): sequence +lines from z14 below 1.8 km above ground, z13 to 5 km, z12 to 12 km, z11 +to 30 km and z10 to 60 km, as Mapillary's (a dense z10 tile over Paris is +~2.1 MB and 8,600 sequences, 750,000 points). `decode.js` simplifies each +line (Douglas–Peucker to the segment, 4 units of a 4,096 tile: about a +pixel where it is drawn, 6–10% of a GPS track's points), and the tiles in +view draw their longest lines within 90,000 points (`maxPoints`, +`keepOrder`), shared out once loading settles so a sparse tile keeps all +it has and a dense one (a city centre) takes the rest; the selected +sequence is always drawn. Every point becomes ~8 ground-line vertices: +unsimplified, the lines hung the tab. Decoded tiles are kept while the +provider is on (16, ten minutes, `tileCache.js`, shared with KartaView), +and above +60 km the `grid` layer's 0.1° +density cells, binned to at most 96×96 points a tile (a z0 tile holds +~70,000 cells) and not counted as sequences: the panel says to zoom in. +A tile decodes in slices of about 8 ms, the page getting a turn between +them (a 2 MB z10 tile is ~200 ms of work). A busy API (429/503) holds the +tiles back for its Retry-After, which the browser cannot read here (the API +exposes no headers), so the wait doubles from 5 s to 5 minutes until it +answers; an answer may take as long as it keeps arriving (silent for 15 s +is given up), so a slow link still gets its megabyte tiles. Both +filters apply: lines carry their day and camera type, and a cell stays while +it holds pictures of the kind asked for (cells are undated). A click on a +line asks the core (`actions.openNearest`) for that sequence's picture +nearest the click, within the camera height / 100 (50–400 m: from high up +a click lands well off a line), without framing, then any sequence's within +50 m. Each lookup is two searches side by side, ranked by distance in the +browser: `place_position` (nearest first, but even at +`place_fov_tolerance=180` it leaves out pictures looking away from the +point) and a bounding box (those too, roughly nearest first, so 20 are +asked for: ~60 kB, against ~600 kB for 100); the filters go to both (`filter=field_of_view=360`, `datetime=/..`). A line is dated +by its sequence's first day, so a sequence still growing can hide under +SINCE; a line with no camera type or day stays drawn. Lines keep the +decoded longest-first order under the per-tile cap (`keepOrder`). Once the picture is on screen +its sequence is highlighted and drawn as cones from the z15 tiles around it +(their `pictures` layer, ~110 bytes a picture, against ~7.5 kB for a STAC +item), plus the neighbours within 150 m of an edge, on their own connection +(coverage tiles take three, searches two); each tile's list for the sequence +is kept. The open picture's own tile draws at once and the neighbours join +as they arrive; one that fails is not counted, so a picture opening in it +re-centres the cones there. The cones re-centre when a picture opens outside the tiles +they were drawn from, judged from the moment their load starts, and centre +only on a picture actually shown (never on a lookup that came back after +the user moved on); a lookup nobody waits for any more is stopped. ‹ › follow the picture's own `prev`/`next` +links, so stepping costs one small search per picture. The viewer shows the +2,048 px SD derivative (equirectangular for a 360° picture, decoded at +2,048 × 1,024) and swaps in the HD original only when the viewer is wider +than SD (flat) or, for a panorama, expanded; once, shrunk while it decodes. +When SD fails a flat picture falls back to the thumbnail, and to the +original only in the expanded viewer. Every download of a photo, first or +sharper, stops when another opens or the viewer closes. Originals wider than +8,192 px are never fetched; a file's header (JPEG, PNG, WebP, GIF) is read +before it decodes, and one declaring more than 64 MP or 16,384 px a side +is refused undecoded, as is an `` that decodes larger and any file +over 48 MB. The caption names every +producer (an uploading app and its photographer; never an account named by +an email address), or else the instance holding the picture ("via +panoramax.ign.fr"), and the picture's licence (CC BY-SA 4.0, Licence +Ouverte 2.0…; StreetPose `license`), and links to the picture in the +Panoramax viewer. Esc anywhere in the docked viewer clears the selected +sequence before the panel can take it (KartaView too). After a GPU reset +the viewer draws the same picture again itself (`onReopen`), leaving the +core's lookups alone. The MCP globe panel allows the meta-catalogue and the +three biggest instances; a picture on another instance shows only in the +app. Street Level has two browser gates. `npm run qa:street-level -- --url ` runs against real Mapillary and needs `MAPILLARY_CLIENT_TOKEN`. `npm run qa:street-level:fixtures -- --url ` answers every Mapillary request from fixtures (`providers/mapillary/coverageFixture.mjs`, -`scripts/fixtures/street-level/`), so it needs no network and covers the whole -photo flow. CI runs the fixture gate and `qa:panel-resize` +`scripts/fixtures/street-level/`), and KartaView's and Panoramax's too, so it +needs no network and covers the whole photo flow. CI runs the fixture gate and `qa:panel-resize` against a production build served by `vite preview` with a dummy token. Directions is a keyless front end to the routing the voice agent already @@ -4166,6 +4257,10 @@ for, resolved lazily after env loading. It replaces the empty default list; there is no built-in fallback. Local/private instances are allowed. The stable `gods-eye-view/…` User-Agent is unchanged. URLs never appear in response headers or upstream errors, including old cached endpoint metadata. +Disk cache entries for both routes carry a SHA-256 tag of the configured +instance set (order does not matter) and are served only while it matches, so +repointing `OVERPASS_UPSTREAMS` refetches instead of serving another instance's +answers; untagged entries stop serving once an instance is configured. The shared transport guards `/api/overpass` and `/api/military-installations`. With no configured upstream, each route serves existing cache data (including diff --git a/scripts/fixtures/street-level/panoramax.mjs b/scripts/fixtures/street-level/panoramax.mjs new file mode 100644 index 000000000..acca30e0f --- /dev/null +++ b/scripts/fixtures/street-level/panoramax.mjs @@ -0,0 +1,283 @@ +import { tileBounds } from '../../../src/layers/streetLevel/tileMath.js'; +import { encodePanoramaxTile } from '../../../src/testSupport/panoramaxFixtures.mjs'; +import { metresApart, photoJpeg } from './mapillaryGraph.mjs'; + +/** + * Panoramax's meta-catalogue and one instance's pictures answered from + * fixtures, so the gate's Panoramax steps need no network. Two sequences + * drive east along the same Paris street (PNX_LINE), their pictures + * interleaved, so a click on one line must open that line's picture, not + * the other's that is as near. Every seventh picture of the first is 360°. + * The meta-catalogue's pictures live on panoramax.openstreetmap.fr; the + * first sequence is CC BY-SA 4.0 by "Fixture Mapper", the second Licence + * Ouverte by an account named by an email address (never shown). + */ + +export const PNX_SEQUENCES = Object.freeze([ + 'a1a1a1a1-0000-4000-8000-000000000001', + 'b2b2b2b2-0000-4000-8000-000000000002', +]); +export const PNX_LINE = Object.freeze({ + lat: 48.8874, + west: 2.3445, + east: 2.3545, +}); +export const PNX_PARK = Object.freeze({ + lon: 2.3495, + lat: 48.8874, + height: 700, + pitch: -1.4, +}); +export const PNX_CREATOR = 'Fixture Mapper'; +const PNX_SPACING_M = 10; +const METRES_PER_DEG_LON = 111_320 * Math.cos((PNX_LINE.lat * Math.PI) / 180); +const API = 'https://api.panoramax.xyz/api'; +const HOST = 'https://panoramax.openstreetmap.fr'; + +const CORS = Object.freeze({ 'Access-Control-Allow-Origin': '*' }); + +/** Calls answered since the last reset, by kind. */ +let asked = { search: 0, hd: 0 }; + +export function resetPanoramaxFixture() { + asked = { search: 0, hd: 0 }; +} + +/** How many HD originals were downloaded since the last reset. */ +export function panoramaxHdDownloads() { + return asked.hd; +} + +/** + * Both sequences' pictures, west to east. The second's sit 5 m along from + * the first's, on the same street. + */ +export function panoramaxPictures() { + const step = PNX_SPACING_M / METRES_PER_DEG_LON; + const count = Math.floor((PNX_LINE.east - PNX_LINE.west) / step); + return PNX_SEQUENCES.flatMap((sequenceId, s) => + Array.from({ length: count }, (_, i) => ({ + id: `${sequenceId.slice(0, 8)}-${String(i).padStart(4, '0')}`, + sequenceId, + index: i, + count, + lon: Number((PNX_LINE.west + (i + s / 2) * step).toFixed(7)), + lat: PNX_LINE.lat, + isPano: s === 0 && i % 7 === 3, + datetime: + s === 0 ? '2025-06-01T10:00:00+00:00' : '2023-03-01T09:00:00+00:00', + })), + ); +} + +function item(picture, pictures) { + const sibling = (delta) => + pictures.find( + (p) => + p.sequenceId === picture.sequenceId && + p.index === picture.index + delta, + ); + const link = (rel, other) => + other + ? [ + { + rel, + href: `${API}/collections/${other.sequenceId}/items/${other.id}`, + }, + ] + : []; + const first = picture.sequenceId === PNX_SEQUENCES[0]; + const files = `${HOST}/derivates/${picture.id}`; + return { + type: 'Feature', + id: picture.id, + collection: picture.sequenceId, + geometry: { type: 'Point', coordinates: [picture.lon, picture.lat] }, + providers: [ + { + name: first ? PNX_CREATOR : 'someone@fixture.example.fr', + roles: ['producer'], + }, + ], + properties: { + datetime: picture.datetime, + license: first ? 'CC-BY-SA-4.0' : 'etalab-2.0', + 'view:azimuth': 90, + 'pers:interior_orientation': picture.isPano + ? { field_of_view: 360, sensor_array_dimensions: [5760, 2880] } + : { field_of_view: 70, sensor_array_dimensions: [4032, 3024] }, + }, + assets: { + hd: { href: `${files}/hd.jpg` }, + sd: { href: `${files}/sd.jpg` }, + thumb: { href: `${files}/thumb.jpg` }, + }, + links: [ + ...link('prev', sibling(-1)), + ...link('next', sibling(1)), + { rel: 'via', href: HOST }, + ], + }; +} + +/** The tile `z/x/y` in the meta-catalogue's shape. */ +export function panoramaxTile(z, x, y) { + const tile = { x, y, z }; + const { west, east, south, north } = tileBounds(x, y, z); + if (PNX_LINE.lat < south || PNX_LINE.lat > north) return new Uint8Array(0); + const from = Math.max(PNX_LINE.west, west); + const to = Math.min(PNX_LINE.east, east); + if (from >= to) return new Uint8Array(0); + if (z <= 5) + return encodePanoramaxTile(tile, { + grid: [{ lon: (from + to) / 2, lat: PNX_LINE.lat, pano: 30, flat: 170 }], + }); + if (z < 7) return new Uint8Array(0); + const sequences = PNX_SEQUENCES.map((id, s) => ({ + id, + type: s === 0 ? 'equirectangular' : 'flat', + date: s === 0 ? '2025-06-01' : '2023-03-01', + parts: [ + [ + [from, PNX_LINE.lat], + [to, PNX_LINE.lat], + ], + ], + })); + if (z < 15) return encodePanoramaxTile(tile, { sequences }); + const pictures = panoramaxPictures() + .filter((p) => p.lon >= west && p.lon < east) + .map((p) => ({ + id: p.id, + lon: p.lon, + lat: p.lat, + sequenceId: p.sequenceId, + heading: 90, + ts: p.datetime.replace('T', ' ').replace('+00:00', '+00'), + type: p.isPano ? 'equirectangular' : 'flat', + })); + return encodePanoramaxTile(tile, { sequences, pictures }); +} + +/** A call named by host, path shape and parameter names. */ +export function describePanoramaxCall(method, url) { + const keys = [...url.searchParams.keys()].sort(); + const path = url.pathname + .replace(/\/map\/\d+\/\d+\/\d+\.mvt$/, '/map/{z}/{x}/{y}.mvt') + .replace( + /^\/derivates\/[\w-]+\/(sd|hd|thumb)\.jpg$/, + '/derivates/{id}/$1.jpg', + ); + return `${method} ${url.hostname}${path}${keys.length ? `?${keys.join('&')}` : ''}`; +} + +const json = (body) => ({ + status: 200, + contentType: 'application/geo+json', + headers: CORS, + body: JSON.stringify(body), + known: true, +}); + +/** + * Answer one Panoramax request; `known` is false for a call no fixture + * covers, so the gate fails on a new one instead of missing it. + */ +export function answerPanoramaxRequest({ method, url }) { + const address = new URL(url); + if (method === 'OPTIONS') + return { status: 204, headers: CORS, body: '', known: true }; + const pictures = panoramaxPictures(); + if (address.hostname === new URL(HOST).hostname) { + const file = address.pathname.match( + /^\/derivates\/([\w-]+)\/(sd|hd|thumb)\.jpg$/, + ); + if (!file || !pictures.some((p) => p.id === file[1])) + return { status: 404, headers: CORS, body: '', known: false }; + if (file[2] === 'hd') asked.hd++; + return { + status: 200, + contentType: 'image/jpeg', + headers: CORS, + body: photoJpeg(), + known: true, + }; + } + if (address.hostname !== 'api.panoramax.xyz') + return { status: 404, headers: CORS, body: '', known: false }; + const tile = address.pathname.match(/^\/api\/map\/(\d+)\/(\d+)\/(\d+)\.mvt$/); + if (tile) { + const bytes = panoramaxTile( + Number(tile[1]), + Number(tile[2]), + Number(tile[3]), + ); + return bytes.length + ? { + status: 200, + contentType: 'application/vnd.mapbox-vector-tile', + headers: CORS, + body: Buffer.from(bytes), + known: true, + } + : { status: 204, headers: CORS, body: '', known: true }; + } + if (address.pathname !== '/api/search') + return { status: 404, headers: CORS, body: '', known: false }; + asked.search++; + const q = address.searchParams; + if (q.has('ids')) { + const wanted = q.get('ids').split(','); + return json({ + type: 'FeatureCollection', + features: pictures + .filter((p) => wanted.includes(p.id)) + .map((p) => item(p, pictures)), + }); + } + const filter = q.get('filter') || ''; + const since = q.get('datetime')?.split('/')[0]; + const kept = pictures + .filter( + (p) => !q.get('collections') || p.sequenceId === q.get('collections'), + ) + .filter((p) => + filter === 'field_of_view=360' + ? p.isPano + : filter === 'field_of_view<360' + ? !p.isPano + : true, + ) + .filter((p) => !since || Date.parse(p.datetime) >= Date.parse(since)); + // A box: every picture in it, nearest its centre first, as the live API + // returns them (roughly: the browser ranks them again). + if (q.has('bbox')) { + const [west, south, east, north] = q.get('bbox').split(',').map(Number); + const centre = { lon: (west + east) / 2, lat: (south + north) / 2 }; + return json({ + type: 'FeatureCollection', + features: kept + .filter( + (p) => + p.lon >= west && p.lon <= east && p.lat >= south && p.lat <= north, + ) + .map((p) => ({ p, metres: metresApart(centre, p) })) + .sort((a, b) => a.metres - b.metres || a.p.id.localeCompare(b.p.id)) + .slice(0, Number(q.get('limit')) || 10) + .map(({ p }) => item(p, pictures)), + }); + } + const [lon, lat] = (q.get('place_position') || '').split(',').map(Number); + const radius = Number((q.get('place_distance') || '0-50').split('-')[1]); + // As the API does, even at the widest tolerance: a place behind a + // picture (every one looks east) is not in it. + const facing = (p) => lon >= p.lon - 1e-7; + const near = kept + .filter(facing) + .map((p) => ({ p, metres: metresApart({ lon, lat }, p) })) + .filter(({ metres }) => metres <= radius) + .sort((a, b) => a.metres - b.metres) + .slice(0, Number(q.get('limit')) || 10) + .map(({ p }) => item(p, pictures)); + return json({ type: 'FeatureCollection', features: near }); +} diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 34e12d6c2..6b67601a5 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -932,15 +932,33 @@ "src/layers/streetLevel/providers/kartaview/source.js", "src/layers/streetLevel/providers/kartaview/decode.js", "src/layers/streetLevel/providers/kartaview/viewer.js", - "src/layers/streetLevel/providers/kartaview/panorama.js", + "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", "src/layers/streetLevel/sequenceCones.js", "src/layers/streetLevel/providers/kartaview/policy.test.mjs", "src/layers/streetLevel/providers/kartaview/decode.test.mjs", "src/layers/streetLevel/providers/kartaview/source.test.mjs", - "src/layers/streetLevel/providers/kartaview/panorama.test.mjs", + "src/layers/streetLevel/panorama.test.mjs", "src/layers/streetLevel/providers/kartaview/viewer.test.mjs", "src/layers/streetLevel/providers/kartaview/provider.test.mjs", + "src/layers/streetLevel/providers/panoramax/index.js", + "src/layers/streetLevel/providers/panoramax/policy.js", + "src/layers/streetLevel/providers/panoramax/source.js", + "src/layers/streetLevel/providers/panoramax/decode.js", + "src/layers/streetLevel/providers/panoramax/viewer.js", + "src/layers/streetLevel/photoViewer.js", + "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", + "src/layers/streetLevel/tileCache.js", + "src/layers/streetLevel/providers/panoramax/policy.test.mjs", + "src/layers/streetLevel/providers/panoramax/decode.test.mjs", + "src/layers/streetLevel/providers/panoramax/source.test.mjs", + "src/layers/streetLevel/providers/panoramax/viewer.test.mjs", + "src/layers/streetLevel/providers/panoramax/provider.test.mjs", + "src/testSupport/panoramaxFixtures.mjs", + "src/layers/streetLevel/photoViewer.test.mjs", + "src/layers/streetLevel/limiter.test.mjs", + "src/layers/streetLevel/imageSize.test.mjs", "src/layers/streetLevel/lineCoverage.test.mjs", "src/testSupport/kartaViewFakes.mjs", "src/testSupport/googleStreetViewFakes.mjs", @@ -957,6 +975,7 @@ "src/qaChrome.test.mjs", "scripts/fixtures/street-level/mapillaryGraph.mjs", "scripts/fixtures/street-level/kartaView.mjs", + "scripts/fixtures/street-level/panoramax.mjs", "src/layers/streetLevel/filter.test.mjs", "src/layers/streetLevel/registry.test.mjs", "src/layers/streetLevel/params.test.mjs", diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index a02bcd40c..08760e422 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -1357,7 +1357,16 @@ "src/layers/streetLevel/providers/kartaview/source.js", "src/layers/streetLevel/providers/kartaview/decode.js", "src/layers/streetLevel/providers/kartaview/viewer.js", - "src/layers/streetLevel/providers/kartaview/panorama.js", + "src/layers/streetLevel/providers/panoramax/index.js", + "src/layers/streetLevel/providers/panoramax/policy.js", + "src/layers/streetLevel/providers/panoramax/source.js", + "src/layers/streetLevel/providers/panoramax/decode.js", + "src/layers/streetLevel/providers/panoramax/viewer.js", + "src/layers/streetLevel/photoViewer.js", + "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", + "src/layers/streetLevel/tileCache.js", + "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", "src/layers/streetLevel/sequenceCones.js", "src/layers/streetLevel/providers/mapillary/source.js", @@ -2365,7 +2374,16 @@ "src/layers/streetLevel/providers/kartaview/source.js", "src/layers/streetLevel/providers/kartaview/decode.js", "src/layers/streetLevel/providers/kartaview/viewer.js", - "src/layers/streetLevel/providers/kartaview/panorama.js", + "src/layers/streetLevel/providers/panoramax/index.js", + "src/layers/streetLevel/providers/panoramax/policy.js", + "src/layers/streetLevel/providers/panoramax/source.js", + "src/layers/streetLevel/providers/panoramax/decode.js", + "src/layers/streetLevel/providers/panoramax/viewer.js", + "src/layers/streetLevel/photoViewer.js", + "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", + "src/layers/streetLevel/tileCache.js", + "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", "src/layers/streetLevel/sequenceCones.js", "src/layers/streetLevel/providers/mapillary/source.js", @@ -2922,7 +2940,16 @@ "src/layers/streetLevel/providers/kartaview/source.js", "src/layers/streetLevel/providers/kartaview/decode.js", "src/layers/streetLevel/providers/kartaview/viewer.js", - "src/layers/streetLevel/providers/kartaview/panorama.js", + "src/layers/streetLevel/providers/panoramax/index.js", + "src/layers/streetLevel/providers/panoramax/policy.js", + "src/layers/streetLevel/providers/panoramax/source.js", + "src/layers/streetLevel/providers/panoramax/decode.js", + "src/layers/streetLevel/providers/panoramax/viewer.js", + "src/layers/streetLevel/photoViewer.js", + "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", + "src/layers/streetLevel/tileCache.js", + "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", "src/layers/streetLevel/sequenceCones.js", "src/layers/streetLevel/providers/mapillary/source.js", diff --git a/scripts/qa-street-level.mjs b/scripts/qa-street-level.mjs index c47a3f9f8..6f87762cf 100644 --- a/scripts/qa-street-level.mjs +++ b/scripts/qa-street-level.mjs @@ -3,8 +3,9 @@ * Browser QA for the Street Level layer against a running server. Run with * `npm run qa:street-level -- --url http://localhost:4173`. * - * `--fixtures` (the CI mode) answers every *.mapillary.com and KartaView - * request from fixtures, so nothing reaches Mapillary or KartaView. The server still needs a token + * `--fixtures` (the CI mode) answers every *.mapillary.com, KartaView and + * Panoramax request from fixtures, so nothing reaches those providers. The + * server still needs a token * because the bundle carries it; CI uses a dummy one. Fixture runs also stage * a rejected key and a keyless second server. * @@ -46,6 +47,17 @@ import { kartaViewPhotos, resetKartaViewFixture, } from './fixtures/street-level/kartaView.mjs'; +import { + answerPanoramaxRequest, + describePanoramaxCall, + PNX_CREATOR, + PNX_LINE, + PNX_PARK, + PNX_SEQUENCES, + panoramaxHdDownloads, + panoramaxPictures, + resetPanoramaxFixture, +} from './fixtures/street-level/panoramax.mjs'; const PANEL_ID = 'street-level-panel'; @@ -62,6 +74,7 @@ export const EXPECTED_PROVIDERS = Object.freeze([ 'mapillary', 'google', 'kartaview', + 'panoramax', ]); /** @@ -245,6 +258,8 @@ export async function serveFixtures(page, fixture) { { urlPattern: '*://api.kartaview.org/*' }, { urlPattern: '*://cdn.kartaview.org/*' }, { urlPattern: '*://*.openstreetcam.org/*' }, + { urlPattern: '*://api.panoramax.xyz/*' }, + { urlPattern: '*://panoramax.openstreetmap.fr/*' }, ], }); const fulfil = (requestId, { status, headers = {}, contentType, body }) => @@ -280,6 +295,14 @@ export async function serveFixtures(page, fixture) { answer = answerKartaViewRequest({ method, url: request.url }); fixture.calls.add(call); if (!answer.known) fixture.unknown.push(call); + } else if ( + address.hostname === 'api.panoramax.xyz' || + address.hostname === 'panoramax.openstreetmap.fr' + ) { + const call = describePanoramaxCall(method, address); + answer = answerPanoramaxRequest({ method, url: request.url }); + fixture.calls.add(call); + if (!answer.known) fixture.unknown.push(call); } else if ( address.pathname === '/api/mapillary/status' && fixture.configured !== undefined @@ -1054,7 +1077,7 @@ async function main() { assert.equal(street.renderModes, true, 'FIT / FILL for a flat photo'); assert.ok( await settle(page, () => { - const img = document.querySelector('.sl-kv-photo'); + const img = document.querySelector('.sl-photo-flat'); return img && !img.hidden && img.naturalWidth > 0; }), 'the photo is on screen', @@ -1098,7 +1121,7 @@ async function main() { ); } const before = street.imageId; - await clickPanelControl(page, '.sl-kv-step-next'); + await clickPanelControl(page, '.sl-photo-step-next'); assert.ok( await uiUntil( (u, was) => @@ -1150,10 +1173,10 @@ async function main() { assert.equal(u.street.error, null); assert.equal(u.street.renderModes, false, 'no frame to fit'); assert.equal( - await page.$eval('.sl-kv-pano', (canvas) => canvas.hidden), + await page.$eval('.sl-photo-pano', (canvas) => canvas.hidden), false, ); - const box = await page.$eval('.sl-kv-pano', (canvas) => { + const box = await page.$eval('.sl-photo-pano', (canvas) => { const r = canvas.getBoundingClientRect(); return { x: r.x + r.width / 2, y: r.y + r.height / 2 }; }); @@ -1170,7 +1193,7 @@ async function main() { // A GPU reset: the panorama comes back on a new canvas, and drags. assert.ok( await page.evaluate(() => { - const canvas = document.querySelector('.sl-kv-pano'); + const canvas = document.querySelector('.sl-photo-pano'); window.__kvLostCanvas = canvas; const lose = canvas .getContext('webgl') @@ -1184,7 +1207,7 @@ async function main() { await settle( page, () => { - const canvas = document.querySelector('.sl-kv-pano'); + const canvas = document.querySelector('.sl-photo-pano'); return ( canvas && canvas !== window.__kvLostCanvas && @@ -1231,6 +1254,349 @@ async function main() { await setStack(original); } + /** + * Panoramax: keyless orange coverage with both filters, a line click + * opening that sequence's picture nearest the click (author, licence, + * cones, ›), a 360° picture whose original is fetched once on EXPAND. + * Live runs park over Paris, where Panoramax has pictures. Runs on a + * draped map; leaves Mapillary lit. + */ + async function panoramaxSteps() { + resetPanoramaxFixture(); + const ui = () => + page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('street-level') + .module.getUIState(), + ); + const uiUntil = (test, arg = null, options) => + settle( + page, + `(${test})(window.__godsEyeView.dataManager.layers.get('street-level').module.getUIState(), ${JSON.stringify(arg)})`, + undefined, + options, + ); + const lit = (id) => + uiUntil( + (u, want) => + u.providers.find((p) => p.id === want)?.on === true && + u.providers.filter((p) => p.on).length === 1, + id, + ); + const module = (call, ...args) => + page.evaluate( + (name, list) => + window.__godsEyeView.dataManager.layers + .get('street-level') + .module[name](...list), + call, + args, + ); + const stack = () => + page.evaluate(() => + window.__godsEyeView.mapStackController.getActiveId(), + ); + const setStack = async (id) => { + await page.evaluate( + (stackId) => + window.__godsEyeView.mapStackController.setStack(stackId), + id, + ); + await settle( + page, + (stackId) => + window.__godsEyeView.mapStackController.getActiveId() === stackId, + id, + ); + }; + const camera = () => + page.evaluate(() => { + const c = window.__godsEyeView.viewer.camera.positionCartographic; + return { + lon: (c.longitude * 180) / Math.PI, + lat: (c.latitude * 180) / Math.PI, + height: c.height, + }; + }); + const pnxChip = '#sl-provider-chips [data-chip-id="panoramax"]'; + const original = await stack(); + await module('closeViewer'); + await setStack('esri-imagery'); + for (let attempt = 0; attempt < 4; attempt++) { + await page.evaluate((at) => { + const v = window.__godsEyeView.viewer; + v.camera.cancelFlight?.(); + const C = v.camera.positionCartographic.constructor; + v.camera.setView({ + destination: v.scene.globe.ellipsoid.cartographicToCartesian( + C.fromDegrees(at.lon, at.lat, at.height), + ), + orientation: { heading: 0, pitch: at.pitch, roll: 0 }, + }); + }, PNX_PARK); + await sleep(1000); + } + try { + await step( + 'Panoramax: its chip draws keyless orange coverage, keeps both filters, credits Panoramax and lights in the share link', + async () => { + await clickPanelControl(page, pnxChip); + assert.ok(await lit('panoramax'), 'Panoramax alone is lit'); + await waitForCoverage(page); + const u = await ui(); + assert.equal(u.keyRequired, false); + assert.deepEqual(u.filterControls, { pano: true, since: true }); + assert.equal( + await page.$eval(pnxChip, (chip) => + chip.style.getPropertyValue('--chip-color'), + ), + '#ff6f00', + ); + assert.match( + await page.$eval('#sl-coverage-meta', (node) => node.textContent), + /sequences in view · click a line for its nearest photo/, + ); + await page.waitForFunction( + () => + document.body.innerHTML.includes('Panoramax contributors'), + { timeout: 15_000 }, + ); + assert.ok( + await settle(page, () => + (new URLSearchParams(location.hash.slice(1)).get('lo') || '') + .split('_') + .some( + (entry) => + entry.startsWith('0.') && entry.includes('panoramax-1'), + ), + ), + `the share link lights Panoramax: ${await page.evaluate(() => location.hash)}`, + ); + }, + ); + await step( + 'Panoramax: a click on a line opens that sequence’s picture nearest the click, camera still, with author and licence, cones and ›', + async ({ skip }) => { + // Points on the street: the fixture's, or live pictures there. + let points = null; + if (fixtures) + points = Array.from({ length: 9 }, (_, i) => [ + PNX_LINE.west + + ((i + 2) / 13) * (PNX_LINE.east - PNX_LINE.west), + PNX_LINE.lat, + ]); + else { + const near = await fetch( + `https://api.panoramax.xyz/api/search?place_position=${PNX_PARK.lon},${PNX_PARK.lat}&place_distance=0-150&place_fov_tolerance=180&limit=12`, + ) + .then((r) => r.json()) + .catch(() => null); + points = near?.features?.map((f) => f.geometry.coordinates); + if (!points?.length) + return skip('no Panoramax picture near the park'); + } + const handle = await page + .waitForFunction( + findPickPoint, + { timeout: 60_000, polling: 500 }, + { + prefix: 'pnx:seq:', + id: null, + points, + billboardIds: null, + }, + ) + .catch(() => null); + if (!handle && !fixtures) return skip('no Panoramax line in view'); + assert.ok(handle, 'a Panoramax line is pickable'); + let at = await handle.jsonValue(); + if (fixtures) { + // Click the top line where the other sequence's pictures are: + // the nearest picture is then the other's, and only the line + // click's own-sequence lookup opens the clicked line's. + const others = panoramaxPictures() + .filter((p) => p.sequenceId !== at.id) + .slice(40, 100) + .map((p) => [p.lon, p.lat]); + const aimed = await page + .waitForFunction( + findPickPoint, + { timeout: 60_000, polling: 500 }, + { + prefix: 'pnx:seq:', + id: at.id, + points: others, + billboardIds: null, + }, + ) + .catch(() => null); + assert.ok( + aimed, + 'the line is pickable over the other’s pictures', + ); + at = await aimed.jsonValue(); + } + const parked = await camera(); + await page.mouse.click(at.x, at.y); + await page.waitForFunction( + () => { + const s = window.__godsEyeView.dataManager.layers + .get('street-level') + .module.getUIState().street; + return (s.imageId && !s.loading) || s.error; + }, + { timeout: 60_000 }, + ); + const street = (await ui()).street; + assert.equal(street.error, null, `viewer error: ${street.error}`); + assert.equal(street.providerId, 'panoramax'); + assert.equal( + street.sequenceId, + at.id, + 'the picture is from the line clicked', + ); + assert.ok(street.license, 'the caption names the licence'); + assert.match( + street.externalUrl, + /^https:\/\/api\.panoramax\.xyz\/#pic=/, + ); + await sleep(1500); + const now = await camera(); + assert.ok( + Math.abs(now.lon - parked.lon) < 1e-6 && + Math.abs(now.lat - parked.lat) < 1e-6 && + Math.abs(now.height - parked.height) < 0.5, + `the camera did not move: ${JSON.stringify({ parked, now })}`, + ); + if (fixtures) { + const caption = await page.$eval( + '#sl-image-by', + (node) => node.textContent, + ); + if (street.sequenceId === PNX_SEQUENCES[0]) + assert.equal(caption, `Image by ${PNX_CREATOR} · CC BY-SA 4.0`); + else + assert.equal( + caption, + 'via panoramax.openstreetmap.fr · Licence Ouverte 2.0', + 'an account named by an email address is not shown; the instance is', + ); + const pictures = panoramaxPictures().filter( + (p) => p.sequenceId === street.sequenceId, + ); + assert.ok( + await uiUntil( + (u, want) => + u.sequence.selectedId === want && + !u.sequence.loading && + u.sequence.images > 0, + street.sequenceId, + { timeout: 30_000 }, + ), + `cones along the sequence: ${JSON.stringify((await ui()).sequence)}`, + ); + assert.ok((await ui()).sequence.images <= pictures.length); + } + const before = street.imageId; + await clickPanelControl(page, '.sl-photo-step-next'); + assert.ok( + await uiUntil( + (u, was) => + u.street.imageId && + u.street.imageId !== was && + !u.street.loading && + !u.street.error, + before, + { timeout: 30_000 }, + ), + '› opened the next picture', + ); + const after = (await ui()).street; + assert.equal(after.sequenceId, street.sequenceId); + if (fixtures) { + const all = panoramaxPictures(); + const from = all.find((p) => p.id === before); + assert.equal( + after.imageId, + all.find( + (p) => + p.sequenceId === from.sequenceId && + p.index === from.index + 1, + ).id, + '› followed the picture’s next link', + ); + } + }, + ); + if (fixtures) + await step( + 'Panoramax: a 360° picture opens on the panorama without FIT / FILL, drags, and EXPAND fetches its original once', + async () => { + const pano = panoramaxPictures().find((p) => p.isPano); + await module('openImage', 'panoramax', pano.id); + assert.ok( + await uiUntil( + (u, want) => + u.street.imageId === want && + !u.street.loading && + u.street.isPano, + pano.id, + { timeout: 60_000 }, + ), + `the panorama opened: ${JSON.stringify((await ui()).street)}`, + ); + const u = await ui(); + assert.equal(u.street.renderModes, false, 'no frame to fit'); + const box = await page.$eval('.sl-photo-pano', (canvas) => { + const r = canvas.getBoundingClientRect(); + return { x: r.x + r.width / 2, y: r.y + r.height / 2 }; + }); + const bearing = u.street.bearing; + await page.mouse.move(box.x, box.y); + await page.mouse.down(); + for (let i = 1; i <= 8; i++) + await page.mouse.move(box.x - i * 15, box.y); + await page.mouse.up(); + assert.ok( + await uiUntil((v, was) => v.street.bearing !== was, bearing), + 'dragging the panorama turned its bearing', + ); + assert.equal(panoramaxHdDownloads(), 0, 'docked: SD only'); + await clickPanelControl(page, '#sl-viewer-expand'); + assert.ok( + await settle(page, () => + Boolean(document.querySelector('.sl-viewer-wrap-expanded')), + ), + 'the viewer expanded', + ); + const deadline = Date.now() + 30_000; + while (panoramaxHdDownloads() === 0 && Date.now() < deadline) + await sleep(250); + await sleep(2000); + assert.equal( + panoramaxHdDownloads(), + 1, + 'the original, once, for the expanded viewer', + ); + await page.keyboard.press('Escape'); + }, + ); + else + await skipStep( + 'fixtures only', + 'Panoramax: a 360° picture opens on the panorama without FIT / FILL, drags, and EXPAND fetches its original once', + ); + } finally { + await module('closeViewer'); + await clickPanelControl( + page, + '#sl-provider-chips [data-chip-id="mapillary"]', + ); + assert.ok(await lit('mapillary'), 'Mapillary is lit again'); + await setStack(original); + } + } + async function keyedSteps() { // A late startup flight only shows itself by moving the camera, so park // and check it stays put rather than reading a state. @@ -2549,6 +2915,7 @@ async function main() { }, ); await kartaViewSteps(); + await panoramaxSteps(); await step( 'a reload with the layer on keeps a collapsed panel collapsed, and the share link carries its ui token', async () => { @@ -2632,7 +2999,7 @@ async function main() { ); if (fixtures) await step( - 'every Mapillary and KartaView request was answered from a fixture', + 'every Mapillary, KartaView and Panoramax request was answered from a fixture', () => { console.log( ` (answered: ${[...fixture.calls].sort().join(' | ')})`, diff --git a/server/providers/gbfs.js b/server/providers/gbfs.js index 9741bb4e6..990642c8b 100644 --- a/server/providers/gbfs.js +++ b/server/providers/gbfs.js @@ -3,6 +3,7 @@ import { isAllowedGbfsHost, isAllowedGbfsPath, gbfsCacheControl, + gbfsUpstreamHeaders, } from '../../src/data/gbfsSource.js'; // --------------------------------------------------------------------------- @@ -11,6 +12,9 @@ import { /** Upstream fetch timeout for GBFS requests (ms). */ const GBFS_PROXY_TIMEOUT_MS = 12000; +/** Value of the Client-Identifier header sent with every upstream request. */ +export const GBFS_CLIENT_IDENTIFIER = 'gods-eye-view'; + export const GBFS_MAX_BODY_BYTES = 5 * 1024 * 1024; // 5 MB function gbfsRedirectHost(location, requestUrl) { @@ -60,6 +64,10 @@ export async function fetchGbfsUpstream( headers: { Accept: 'application/json', 'User-Agent': 'gods-eye-view-gbfs-proxy/1.0', + // Urban Sharing (the Norwegian city-bike feeds) asks every client to + // name itself in this header; other operators ignore it. + 'Client-Identifier': GBFS_CLIENT_IDENTIFIER, + ...gbfsUpstreamHeaders(new URL(url).hostname), }, redirect: 'manual', signal: controller.signal, @@ -181,7 +189,7 @@ export function gbfsProxy() { return; } - if (!isAllowedGbfsPath(upstreamUrl.pathname)) { + if (!isAllowedGbfsPath(upstreamUrl.pathname, upstreamUrl.hostname)) { res.writeHead(400, { 'Content-Type': 'application/json', 'Cache-Control': 'no-store', diff --git a/server/providers/military-installations.js b/server/providers/military-installations.js index 08ad87d9f..7d252b8ed 100644 --- a/server/providers/military-installations.js +++ b/server/providers/military-installations.js @@ -9,7 +9,10 @@ import { fetchOverpassPayload, overpassNotConfigured, } from './overpass/transport.js'; -import { resolveOverpassUpstreams } from './overpass/constants.js'; +import { + resolveOverpassUpstreams, + resolveOverpassUpstreamTag, +} from './overpass/constants.js'; import { _militaryInstallationCache, trimMilitaryInstallationCache, @@ -74,7 +77,13 @@ function militaryInstallationsProxy() { retrievedAt: new Date().toISOString(), status: 'ready', }; - const entry = { payload, cachedAt: Date.now() }; + // Tagged with the configured upstream set so the disk layer does not serve + // this answer back after the operator repoints (see overpass/cache.js). + const entry = { + payload, + cachedAt: Date.now(), + upstreams: resolveOverpassUpstreamTag(), + }; _militaryInstallationCache.set(key, entry); trimMilitaryInstallationCache(); writeMilitaryInstallationDisk(key, entry); diff --git a/server/providers/military-installations/cache.js b/server/providers/military-installations/cache.js index c77627e29..4d77981e6 100644 --- a/server/providers/military-installations/cache.js +++ b/server/providers/military-installations/cache.js @@ -5,6 +5,7 @@ import { MILITARY_INSTALLATION_DISK_DIR, MILITARY_INSTALLATION_MAX_CACHE, } from './constants.js'; +import { cachedByConfiguredUpstream } from '../overpass/constants.js'; import path from 'node:path'; import { createHash } from 'node:crypto'; import { promises as fsp } from 'node:fs'; @@ -114,6 +115,9 @@ async function readMilitaryInstallationDisk( const entry = JSON.parse( await fsp.readFile(militaryInstallationDiskPath(cacheKey, dir), 'utf8'), ); + // Same reason as the Overpass disk cache: the key is the bbox alone, so an + // entry written against a different OVERPASS_UPSTREAMS is otherwise a hit. + if (!cachedByConfiguredUpstream(entry)) return null; if (!militaryInstallationDiskFresh(entry, maxAgeMs)) return null; return migrateMilitaryInstallationEntry(entry); } catch { diff --git a/server/providers/overpass/cache.js b/server/providers/overpass/cache.js index b4e53ab21..442f599a6 100644 --- a/server/providers/overpass/cache.js +++ b/server/providers/overpass/cache.js @@ -5,6 +5,7 @@ import { OVERPASS_DISK_DIR, OVERPASS_CACHE_MS, OVERPASS_CACHE_MAX_ENTRIES, + cachedByConfiguredUpstream, } from './constants.js'; import path from 'node:path'; import { createHash } from 'node:crypto'; @@ -47,6 +48,11 @@ async function readOverpassDisk(cacheKey, maxAgeMs) { // Older versions persisted 4xx refusals with normal data TTLs. Ignore // them on both fresh and stale reads so an upgrade can recover immediately. if (!overpassPayloadIsData(payload)) return null; + // The key is the query alone, so a file written by a different upstream — + // including the public instances that were the default before #742 — is a + // hit for the same query. Refuse it: the operator's configured instance + // must be what answers, on the stale path too. + if (!cachedByConfiguredUpstream(payload)) return null; if (Date.now() - payload.cachedAt > maxAgeMs) return null; return payload; } catch { diff --git a/server/providers/overpass/constants.js b/server/providers/overpass/constants.js index 40f244d11..6cfd7257a 100644 --- a/server/providers/overpass/constants.js +++ b/server/providers/overpass/constants.js @@ -1,4 +1,5 @@ import path from 'node:path'; +import { createHash } from 'node:crypto'; // --------------------------------------------------------------------------- // Overpass API proxy constants and cache state @@ -27,16 +28,68 @@ function parseOverpassUpstreams(raw) { return endpoints.slice(0, 8); } -let upstreamMemo = { raw: null, endpoints: [] }; +let upstreamMemo = { raw: null, endpoints: [], tag: 'none' }; + +/** + * Short, stable identifier for the configured upstream set. Cached responses + * carry it so an answer produced by a different instance is not served after + * the operator repoints `OVERPASS_UPSTREAMS` — the disk cache outlives the + * configuration change by 7 days, a month for boundary queries, and forever on + * the serve-stale path. It is a hash, never the URLs, and it is only ever + * compared on read: it is not sent to clients and does not appear in errors. + */ +function upstreamTag(endpoints) { + if (!endpoints.length) return 'none'; + // Sorted, so reordering the same instances for failover keeps the cache. + return createHash('sha256') + .update([...endpoints].sort().join('\n')) + .digest('hex'); +} /** Resolve after environment loading. Public Overpass instances are not used by default. */ function resolveOverpassUpstreams() { - const raw = process.env.OVERPASS_UPSTREAMS || ''; - if (upstreamMemo.raw !== raw) - upstreamMemo = { raw, endpoints: parseOverpassUpstreams(raw) }; + readUpstreamMemo(); return [...upstreamMemo.endpoints]; } +/** The configured-upstream tag stored alongside, and checked against, cache entries. */ +function resolveOverpassUpstreamTag() { + return readUpstreamMemo().tag; +} + +let reportedForeignCache = false; + +/** + * Was this cached entry produced by the upstream set configured now? + * + * With nothing configured there is no instance to contradict, and serving + * last-good data at any age is the point of the cache (the proxy's + * not-configured path depends on it), so any entry is accepted. Once an + * operator does configure an instance, only its own answers are served: an + * entry with no tag, or one from a different instance, is refused rather than + * matched on the query alone. Shared by both caches fed by these upstreams. + */ +function cachedByConfiguredUpstream(entry) { + const tag = resolveOverpassUpstreamTag(); + if (tag === 'none' || entry?.upstreams === tag) return true; + if (!reportedForeignCache) { + reportedForeignCache = true; + console.warn( + '[Overpass] ignoring cached responses produced by a different OVERPASS_UPSTREAMS; they will be refetched from the configured instance.', + ); + } + return false; +} + +function readUpstreamMemo() { + const raw = process.env.OVERPASS_UPSTREAMS || ''; + if (upstreamMemo.raw !== raw) { + const endpoints = parseOverpassUpstreams(raw); + upstreamMemo = { raw, endpoints, tag: upstreamTag(endpoints) }; + } + return upstreamMemo; +} + /** * TTL for FRESH cached Overpass responses (ms). Road geometry is static for * months — the original 45 s TTL forced a public-mirror round-trip on nearly @@ -153,6 +206,8 @@ export { OVERPASS_MAX_RESPONSE_BYTES, parseOverpassUpstreams, resolveOverpassUpstreams, + resolveOverpassUpstreamTag, + cachedByConfiguredUpstream, OVERPASS_USER_AGENT, OVERPASS_TIMEOUT_MS, }; diff --git a/server/providers/overpass/transport.js b/server/providers/overpass/transport.js index 86b9eae15..38c31bb08 100644 --- a/server/providers/overpass/transport.js +++ b/server/providers/overpass/transport.js @@ -1,6 +1,7 @@ import { OVERPASS_MAX_RESPONSE_BYTES, resolveOverpassUpstreams, + resolveOverpassUpstreamTag, OVERPASS_USER_AGENT, OVERPASS_TIMEOUT_MS, } from './constants.js'; @@ -187,6 +188,9 @@ async function fetchOverpassPayload( contentType: 'application/json', // Never retain a secret-bearing endpoint in cache or response metadata. endpoint: 'configured', + // Which configured instance produced this, as a hash rather than a URL, + // so a cached answer is not reused after the operator repoints. + upstreams: resolveOverpassUpstreamTag(), rateLimited: false, }; } catch { diff --git a/src/app/layers/streetLevel.js b/src/app/layers/streetLevel.js index f176d97de..b441d12d7 100644 --- a/src/app/layers/streetLevel.js +++ b/src/app/layers/streetLevel.js @@ -2,6 +2,7 @@ import { createStreetLevelLayer } from '../../layers/streetLevel/index.js'; import { createMapillaryProvider } from '../../layers/streetLevel/providers/mapillary/index.js'; import { createGoogleProvider } from '../../layers/streetLevel/providers/google/index.js'; import { createKartaViewProvider } from '../../layers/streetLevel/providers/kartaview/index.js'; +import { createPanoramaxProvider } from '../../layers/streetLevel/providers/panoramax/index.js'; import * as sprites from '../../data/spriteOrder.js'; import * as picking from '../../data/pickRegistry.js'; import * as input from '../../data/inputOwnership.js'; @@ -12,8 +13,8 @@ import * as render from '../../renderGovernor.js'; const googleApiKey = () => globalThis.window?.__GOOGLE_MAPS_API_KEY__ || null; /** - * Mapillary, Google Street View and KartaView, once the Mapillary source is - * supplied; else none. + * Mapillary, Google Street View, KartaView and Panoramax, once the Mapillary + * source is supplied; else none. */ function defaultProviders(sources) { if (sources.mapillary == null) return []; @@ -21,6 +22,7 @@ function defaultProviders(sources) { createMapillaryProvider({ source: sources.mapillary }), createGoogleProvider({ getApiKey: googleApiKey }), createKartaViewProvider(), + createPanoramaxProvider(), ]; } diff --git a/src/app/layers/streetLevel.test.mjs b/src/app/layers/streetLevel.test.mjs index c2c3ebb16..e82e762e5 100644 --- a/src/app/layers/streetLevel.test.mjs +++ b/src/app/layers/streetLevel.test.mjs @@ -28,7 +28,7 @@ test('explicit providers replace the default source and an empty list removes it ); }); -test('the defaults are Mapillary (given its source), Street View and KartaView, which start off and travel as provider switches', async () => { +test('the defaults are Mapillary (given its source), Street View, KartaView and Panoramax, which start off and travel as provider switches', async () => { const { createDefaultLayerState } = await import('../../data/layerState.js'); const mapillary = { getStatus() {}, @@ -37,7 +37,12 @@ test('the defaults are Mapillary (given its source), Street View and KartaView, nearestImages() {}, }; const layer = createApplicationStreetLevel({ sources: { mapillary } }); - assert.deepEqual(layer.providerIds, ['mapillary', 'google', 'kartaview']); + assert.deepEqual(layer.providerIds, [ + 'mapillary', + 'google', + 'kartaview', + 'panoramax', + ]); // A fresh page and a restored default agree: Street View is billed, so off. layer.setParams(createDefaultLayerState().options['street-level']); assert.equal(layer.getParams().google, false); @@ -45,12 +50,25 @@ test('the defaults are Mapillary (given its source), Street View and KartaView, // A link that lit Street View carries it in the shared switch field. layer.setParams({ providerSwitches: 'google-1' }); assert.equal(layer.getParams().google, true); - assert.equal(layer.getParams().providerSwitches, 'google-1*kartaview-0'); + assert.equal( + layer.getParams().providerSwitches, + 'google-1*kartaview-0*panoramax-0', + ); // KartaView is keyless and free, but one source shows at a time. layer.setParams({ providerSwitches: 'kartaview-1' }); assert.equal(layer.getParams().kartaview, true); assert.equal(layer.getParams().google, false); assert.equal(layer.getParams().mapillary, false); + // As is Panoramax: lighting it puts KartaView out. + layer.setParams({ providerSwitches: 'panoramax-1' }); + assert.equal(layer.getParams().panoramax, true); + assert.equal(layer.getParams().kartaview, false); + assert.equal(layer.getParams().mapillary, false); + // The form the app writes (the README and CHANGELOG quote it). + assert.equal( + layer.getParams().providerSwitches, + 'google-0*kartaview-0*panoramax-1', + ); // Without the application's Street Level source there are none. assert.deepEqual(createApplicationStreetLevel({}).providerIds, []); }); diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index 9f0b8b2d1..576f5b07d 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -265,6 +265,26 @@ export const DATA_CREDITS = [ key: 'gbfs', html: 'Bikeshare availability: GBFS operator feeds (e.g. Austin BCycle)', }, + { + key: 'urbansharing-gbfs', + html: + 'City bikes (Norway): contains data under the ' + + 'Norwegian licence for Open Government data (NLOD) ' + + 'made available by Oslo Bysykkel, Bergen Bysykkel and Trondheim Bysykkel', + }, + { + key: 'entur-mobility-gbfs', + html: + 'City bikes (Stavanger, Skien/Porsgrunn): contains data under the ' + + 'Norwegian licence for Open Government data (NLOD) ' + + 'made available by Entur (Kolumbus Bysykkel, Farte Bysykkel)', + }, + { + key: 'jcdecaux-gbfs', + html: + 'City bikes (Lillestrøm): Bysykkel Lillestrøm, JCDecaux — ' + + 'Licence Ouverte', + }, { key: 'osrm-routing', // The service asks for its attribution to carry a "fix the map" link, so diff --git a/src/data/gbfsProxy.test.mjs b/src/data/gbfsProxy.test.mjs index 191cb57ba..c95256137 100644 --- a/src/data/gbfsProxy.test.mjs +++ b/src/data/gbfsProxy.test.mjs @@ -1,6 +1,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; import { + GBFS_CLIENT_IDENTIFIER, GBFS_MAX_BODY_BYTES, fetchGbfsUpstream, } from '../../server/providers/gbfs.js'; @@ -56,6 +57,41 @@ test('GBFS upstream fetch passes a direct 200 through unchanged', async () => { }); }); +test('GBFS upstream fetch names the app in the Client-Identifier header', async () => { + let observedHeaders = null; + await fetchGbfsUpstream( + 'https://gbfs.urbansharing.com/oslobysykkel.no/station_status.json', + { + fetchImpl: async (_url, options) => { + observedHeaders = options.headers; + return new Response(STATION_BODY, { status: 200 }); + }, + }, + ); + assert.equal(GBFS_CLIENT_IDENTIFIER, 'gods-eye-view'); + assert.equal(observedHeaders['Client-Identifier'], GBFS_CLIENT_IDENTIFIER); +}); + +test('GBFS upstream fetch adds ET-Client-Name for Entur and only for Entur', async () => { + const headersFor = async (url) => { + let observed = null; + await fetchGbfsUpstream(url, { + fetchImpl: async (_url, options) => { + observed = options.headers; + return new Response(STATION_BODY, { status: 200 }); + }, + }); + return observed; + }; + const entur = await headersFor( + 'https://api.entur.io/mobility/v2/gbfs/v2/kolumbusbysykkel/station_status', + ); + assert.equal(entur['ET-Client-Name'], 'gods-eye-view-bikeshare'); + assert.equal(entur['Client-Identifier'], GBFS_CLIENT_IDENTIFIER); + const lyft = await headersFor(STATION_URL); + assert.equal(lyft['ET-Client-Name'], undefined); +}); + test('GBFS upstream fetch forwards a non-redirect error status for the middleware to relay', async () => { const result = await fetchGbfsUpstream(STATION_URL, { fetchImpl: async () => new Response(null, { status: 404 }), diff --git a/src/data/gbfsSource.js b/src/data/gbfsSource.js index a15a806d9..8d4980948 100644 --- a/src/data/gbfsSource.js +++ b/src/data/gbfsSource.js @@ -8,6 +8,31 @@ const GBFS_ALLOWED_HOSTS = new Set([ 'austin.publicbikesystem.net', 'hon.publicbikesystem.net', 'chat.publicbikesystem.net', + 'gbfs.urbansharing.com', + 'api.entur.io', + 'api.cyclocity.fr', +]); + +/** + * Hosts that serve more than GBFS keep the proxy to their GBFS tree: Entur's + * api.entur.io fronts every Entur API, and api.cyclocity.fr every JCDecaux + * contract endpoint. + */ +const GBFS_HOST_PATH_PREFIXES = new Map([ + ['api.entur.io', '/mobility/v2/gbfs/'], + ['api.cyclocity.fr', '/contracts/'], +]); + +/** + * Identifying headers an operator asks every GBFS client to send. Entur rate + * limits or blocks consumers that do not send ET-Client-Name, in the form + * "company-application". + */ +const GBFS_HOST_HEADERS = new Map([ + [ + 'api.entur.io', + Object.freeze({ 'ET-Client-Name': 'gods-eye-view-bikeshare' }), + ], ]); /** @@ -28,13 +53,40 @@ export function isAllowedGbfsHost(hostname) { } /** - * Only allow station_information.json and station_status.json endpoints. + * Only allow station_information and station_status endpoints. The `.json` + * suffix is optional because Entur serves its feeds without one. When a + * hostname is given and that host serves more than GBFS, the path must also + * sit under the host's GBFS prefix. * * @param {string} pathname + * @param {string} [hostname] * @returns {boolean} */ -export function isAllowedGbfsPath(pathname) { - return /\/station_(information|status)\.json$/i.test(String(pathname || '')); +export function isAllowedGbfsPath(pathname, hostname = '') { + const path = String(pathname || ''); + if (!/\/station_(information|status)(\.json)?$/i.test(path)) return false; + const prefix = GBFS_HOST_PATH_PREFIXES.get( + String(hostname || '') + .trim() + .toLowerCase(), + ); + return !prefix || path.startsWith(prefix); +} + +/** + * Extra request headers an upstream GBFS host asks clients to send. + * + * @param {string} hostname + * @returns {Record} + */ +export function gbfsUpstreamHeaders(hostname) { + return ( + GBFS_HOST_HEADERS.get( + String(hostname || '') + .trim() + .toLowerCase(), + ) || {} + ); } /** @@ -47,7 +99,7 @@ export function isAllowedGbfsPath(pathname) { * @returns {string} Cache-Control header value. */ export function gbfsCacheControl(pathname) { - if (/\/station_information\.json$/i.test(String(pathname || ''))) { + if (/\/station_information(\.json)?$/i.test(String(pathname || ''))) { return 'public, max-age=300'; } return 'no-store'; diff --git a/src/data/installationProxy.test.mjs b/src/data/installationProxy.test.mjs index 08b14de99..308ca9dba 100644 --- a/src/data/installationProxy.test.mjs +++ b/src/data/installationProxy.test.mjs @@ -26,6 +26,7 @@ import { validMilitaryInstallationBox, writeMilitaryInstallationDisk, } from '../../vite.config.js'; +import { resolveOverpassUpstreamTag } from '../../server/providers/overpass/constants.js'; const DAY_MS = 86_400_000; test('installation failure reasons disclose no raw upstream error and do not guess overload', () => { @@ -274,3 +275,30 @@ test('an unwritable cache directory degrades to a miss instead of throwing', asy await fsp.rm(dir, { recursive: true, force: true }); } }); + +test('an installation entry is not served after OVERPASS_UPSTREAMS is repointed', async (t) => { + const prior = process.env.OVERPASS_UPSTREAMS; + t.after(() => { + if (prior === undefined) delete process.env.OVERPASS_UPSTREAMS; + else process.env.OVERPASS_UPSTREAMS = prior; + }); + const dir = tempCacheDir(); + const key = '30.250,-97.750,30.300,-97.700'; + try { + process.env.OVERPASS_UPSTREAMS = 'https://a.example/api/interpreter,https://b.example/api/interpreter'; + const tag = resolveOverpassUpstreamTag(); + await writeMilitaryInstallationDisk(key, { ...entry(Date.now()), upstreams: tag }, dir); + assert.ok(await readMilitaryInstallationDisk(key, TTL_MS, dir), 'its own entry is a hit'); + process.env.OVERPASS_UPSTREAMS = 'https://b.example/api/interpreter,https://a.example/api/interpreter'; + assert.equal(resolveOverpassUpstreamTag(), tag, 'reordering the same instances keeps the tag'); + process.env.OVERPASS_UPSTREAMS = 'https://c.example/api/interpreter'; + assert.equal(await readMilitaryInstallationDisk(key, TTL_MS, dir), null, 'fresh read refused'); + assert.equal(await readMilitaryInstallationDisk(key, Infinity, dir), null, 'stale read refused'); + await writeMilitaryInstallationDisk('legacy', entry(Date.now()), dir); + assert.equal(await readMilitaryInstallationDisk('legacy', Infinity, dir), null, 'untagged entry refused'); + delete process.env.OVERPASS_UPSTREAMS; + assert.ok(await readMilitaryInstallationDisk(key, Infinity, dir), 'nothing configured accepts any entry'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/src/data/layerState.js b/src/data/layerState.js index 57fecef8a..9428296aa 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -299,6 +299,24 @@ function imageryPinOption(key, token) { }); } +function satelliteFilterOption() { + const percentage = boundedIntegerOption('infrared', 'i', 100, { + min: 0, + max: 100, + }); + return Object.freeze({ + ...percentage, + normalize: (value) => + value === 'filtered' + ? 100 + : value === 'full' + ? 0 + : percentage.normalize(value), + decode: (value) => + value === 'f' ? 100 : value === 'a' ? 0 : percentage.decode(value), + }); +} + const OPTION_GROUPS = Object.freeze({ traffic: Object.freeze([ enumOption('roadMode', 'r', null, ['tomtom', 'osm', 'hybrid'], { @@ -320,10 +338,7 @@ const OPTION_GROUPS = Object.freeze({ }), ]), 'weather-satellite': Object.freeze([ - enumOption('infrared', 'i', 'filtered', ['filtered', 'full'], { - filtered: 'f', - full: 'a', - }), + satelliteFilterOption(), enumOption('opacity', 'o', 'strong', ['light', 'strong'], { light: 'l', strong: 's', diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index 30f251b7b..631e5dfec 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -2422,23 +2422,74 @@ test('wind appearance shares round trip while old links retain weather defaults' assert.deepEqual(invalid.options.wind, defaults); }); -test('observed weather round trips product and opacity without persisting historical playback', () => { +test('observed weather round trips product and presets without persisting historical playback', () => { const state = normalizeLayerState({ enabledLayerIds: ['weather-radar', 'weather-satellite'], options: { 'weather-radar': { opacity: 'light', play: true }, - 'weather-satellite': { product: 'clouds', opacity: 'light', step: -1 }, + 'weather-satellite': { + product: 'clouds', + opacity: 'light', + infrared: 'filtered', + step: -1, + }, }, }); const params = new URLSearchParams(encode(state)); const decoded = decodeLayerStateParams(params); assert.deepEqual(decoded, state); assert.equal(state.options['weather-satellite'].product, 'clouds'); + assert.equal(state.options['weather-satellite'].opacity, 'light'); + assert.equal(state.options['weather-satellite'].infrared, 100); assert.equal(Object.hasOwn(state.options['weather-radar'], 'play'), false); }); -test('satellite infrared display mode round trips and invalid or absent values use filtered', () => { - for (const infrared of ['full', 'filtered', undefined, 'invalid']) { +test('satellite filter strength percentages round-trip and legacy mode codes remain readable', () => { + const state = normalizeLayerState({ + enabledLayerIds: ['weather-satellite'], + options: { 'weather-satellite': { infrared: 53, opacity: 'light' } }, + }); + const params = encodeLayerStateParams( + new URLSearchParams([['v', '2']]), + state, + ); + assert.deepEqual(decodeLayerStateParams(params), state); + assert.equal(state.options['weather-satellite'].infrared, 53); + assert.equal(state.options['weather-satellite'].opacity, 'light'); + for (const [legacy, expected] of [ + ['f', 100], + ['a', 0], + ]) { + const decoded = decodeLayerStateParams( + new URLSearchParams(`v=2&l=o&lo=o.i.${legacy}`), + ); + assert.equal(decoded.options['weather-satellite'].infrared, expected); + } + for (const [legacy, expected] of [ + ['filtered', 100], + ['full', 0], + ]) { + const normalized = normalizeLayerState({ + enabledLayerIds: ['weather-satellite'], + options: { 'weather-satellite': { infrared: legacy } }, + }); + assert.equal(normalized.options['weather-satellite'].infrared, expected); + } + const invalid = normalizeLayerState({ + enabledLayerIds: ['weather-satellite'], + options: { 'weather-satellite': { infrared: 101 } }, + }); + assert.equal(invalid.options['weather-satellite'].infrared, 100); +}); + +test('satellite cloud filter round trips and invalid or absent values default to Clouds only', () => { + for (const [infrared, expected] of [ + ['full', 0], + ['filtered', 100], + [53, 53], + [undefined, 100], + ['invalid', 100], + ]) { const state = normalizeLayerState({ enabledLayerIds: ['weather-satellite'], options: { @@ -2454,10 +2505,7 @@ test('satellite infrared display mode round trips and invalid or absent values u decodeLayerStateParams(new URLSearchParams(encode(state))), state, ); - assert.equal( - state.options['weather-satellite'].infrared, - infrared === 'full' ? 'full' : 'filtered', - ); + assert.equal(state.options['weather-satellite'].infrared, expected); assert.equal( Object.hasOwn(state.options['weather-satellite'], 'step'), false, diff --git a/src/i18n/locales/es.json b/src/i18n/locales/es.json index 5fa201f0b..d9d0315e2 100644 --- a/src/i18n/locales/es.json +++ b/src/i18n/locales/es.json @@ -949,8 +949,6 @@ "many": "{n} calles a la vista · haz clic en una línea para su foto más cercana", "other": "{n} calles a la vista · haz clic en una línea para su foto más cercana" }, - "KartaView photo. ← and → step along the sequence. On a 360° photo, drag or use Shift+← →, ↑ ↓ to look around, and + − to zoom": "Foto de KartaView. ← y → avanzan por la secuencia. En una foto de 360°, arrastra o usa Mayús+← →, ↑ ↓ para mirar alrededor, y + − para hacer zoom", - "KartaView street-level photo": "Foto a pie de calle de KartaView", "Camera follows view: move the globe camera wherever the street-level view looks": "La cámara sigue la vista: mueve la cámara del globo hacia donde mira la vista a pie de calle", "Camera follow needs the Google 3D map: choose Google 3D under MAP SOURCE": "Seguir con la cámara requiere el mapa Google 3D: elige Google 3D en FUENTE DEL MAPA", "Switch a provider on to draw its coverage.": "Activa un proveedor para dibujar su cobertura.", @@ -1293,9 +1291,7 @@ "Hourly global mosaic; usually 2–3 hours delayed": "Mosaico global horario; normalmente con 2–3 horas de retraso", "GOES regional clouds; approximately 5-minute updates": "Nubes regionales de GOES; actualizaciones cada 5 minutos aproximadamente", "Clouds only": "Solo nubes", - "Dim everything but the bright, cold cloud tops; a brightness filter, not a cloud mask": "Atenúa todo excepto las cimas de nubes brillantes y frías; un filtro de brillo, no una máscara de nubes", "Full": "Completa", - "The complete infrared image at the chosen opacity": "La imagen infrarroja completa con la opacidad elegida", "Soft": "Suave", "Vivid": "Intensa", "Image opacity; does not alter the observed values": "Opacidad de la imagen; no altera los valores observados", @@ -2761,5 +2757,27 @@ "Analyst query": "Consulta analítica", "Next ISS pass": "Próximo paso de la EEI", "Next satellite pass": "Próximo paso del satélite", - "Action": "Acción" + "Action": "Acción", + "The photo could not be loaded": "No se pudo cargar la foto", + "{name} photo. ← and → step along the sequence. On a 360° photo, drag or use Shift+← →, ↑ ↓ to look around, and + − to zoom": "Foto de {name}. ← y → avanzan por la secuencia. En una foto de 360°, arrastra o usa Mayús+← →, ↑ ↓ para mirar alrededor, y + − para hacer zoom", + "{name} street-level photo": "Foto a pie de calle de {name}", + "Previous photo in this sequence (←)": "Foto anterior de la secuencia (←)", + "Next photo in this sequence (→)": "Foto siguiente de la secuencia (→)", + "The Panoramax photo could not be loaded": "No se pudo cargar la foto de Panoramax", + "Panoramax is busy — coverage resumes on its own": "Panoramax está saturado — la cobertura se reanuda sola", + "no Panoramax photos in view": "no hay fotos de Panoramax a la vista", + "no Panoramax photos here match the filters": "aquí no hay fotos de Panoramax que cumplan los filtros", + "Panoramax sent a map tile that could not be read": "Panoramax envió una tesela del mapa que no se pudo leer", + "This Panoramax photo is no longer available": "Esta foto de Panoramax ya no está disponible", + "zoom in to see Panoramax coverage": "acércate para ver la cobertura de Panoramax", + "Panoramax did not answer in time — try again": "Panoramax no respondió a tiempo — inténtalo de nuevo", + "Panoramax could not be reached — check the connection": "No se pudo conectar con Panoramax — comprueba la conexión", + "Panoramax sent too much data": "Panoramax envió demasiados datos", + "Panoramax is busy": "Panoramax está saturado", + "Panoramax sent an unreadable answer": "Panoramax envió una respuesta que no se pudo leer", + "Apply the full cloud brightness key; 100% filter strength": "Aplica el filtro de brillo de nubes completo; intensidad del filtro 100%", + "Show the complete infrared image; 0% filter strength": "Muestra la imagen infrarroja completa; intensidad del filtro 0%", + "FILTER": "FILTRO", + "Cloud filter strength": "Intensidad del filtro de nubes", + "Panoramax answered HTTP {status}": "Panoramax respondió HTTP {status}" } diff --git a/src/i18n/locales/ru.json b/src/i18n/locales/ru.json index 0b3e4186b..ab61f56cc 100644 --- a/src/i18n/locales/ru.json +++ b/src/i18n/locales/ru.json @@ -959,8 +959,6 @@ "many": "{n} улиц в кадре · щёлкните линию, чтобы открыть ближайшую фотографию", "other": "{n} улицы в кадре · щёлкните линию, чтобы открыть ближайшую фотографию" }, - "KartaView photo. ← and → step along the sequence. On a 360° photo, drag or use Shift+← →, ↑ ↓ to look around, and + − to zoom": "Фотография KartaView. ← и → листают съёмку. На 360°-фотографии перетаскивайте или используйте Shift+← →, ↑ ↓ для обзора, + − для масштаба", - "KartaView street-level photo": "Уличная фотография KartaView", "Camera follows view: move the globe camera wherever the street-level view looks": "Камера следует за видом: камера глобуса поворачивается туда, куда смотрит уличный снимок", "Camera follow needs the Google 3D map: choose Google 3D under MAP SOURCE": "Слежению камеры нужна карта Google 3D: выберите Google 3D в разделе «ИСТОЧНИК КАРТЫ»", "Switch a provider on to draw its coverage.": "Включите источник, чтобы показать его покрытие.", @@ -1310,9 +1308,7 @@ "Hourly global mosaic; usually 2–3 hours delayed": "Ежечасная глобальная мозаика; обычно с задержкой 2–3 ч", "GOES regional clouds; approximately 5-minute updates": "Региональная облачность GOES; обновление примерно каждые 5 мин", "Clouds only": "Только облака", - "Dim everything but the bright, cold cloud tops; a brightness filter, not a cloud mask": "Затемнить всё, кроме ярких холодных вершин облаков; фильтр яркости, а не маска облачности", "Full": "Полный", - "The complete infrared image at the chosen opacity": "Полное ИК-изображение с выбранной непрозрачностью", "Soft": "Мягко", "Vivid": "Ярко", "Image opacity; does not alter the observed values": "Непрозрачность изображения; не меняет наблюдаемые значения", @@ -2818,5 +2814,27 @@ "Analyst query": "Аналитический запрос", "Next ISS pass": "Следующий пролёт МКС", "Next satellite pass": "Следующий пролёт спутника", - "Action": "Действие" + "Action": "Действие", + "The photo could not be loaded": "Не удалось загрузить фотографию", + "{name} photo. ← and → step along the sequence. On a 360° photo, drag or use Shift+← →, ↑ ↓ to look around, and + − to zoom": "Фотография {name}. ← и → листают съёмку. На 360°-фотографии перетаскивайте или используйте Shift+← →, ↑ ↓ для обзора, + − для масштаба", + "{name} street-level photo": "Уличная фотография {name}", + "Previous photo in this sequence (←)": "Предыдущая фотография съёмки (←)", + "Next photo in this sequence (→)": "Следующая фотография съёмки (→)", + "The Panoramax photo could not be loaded": "Не удалось загрузить фотографию Panoramax", + "Panoramax is busy — coverage resumes on its own": "Panoramax перегружен — покрытие загрузится само", + "no Panoramax photos in view": "фотографий Panoramax в зоне видимости нет", + "no Panoramax photos here match the filters": "здесь нет фотографий Panoramax, подходящих под фильтры", + "Panoramax sent a map tile that could not be read": "Panoramax прислал тайл карты, который не удалось прочитать", + "This Panoramax photo is no longer available": "Эта фотография Panoramax больше недоступна", + "zoom in to see Panoramax coverage": "приблизьтесь, чтобы увидеть покрытие Panoramax", + "Panoramax did not answer in time — try again": "Panoramax не ответил вовремя — попробуйте снова", + "Panoramax could not be reached — check the connection": "Не удалось связаться с Panoramax — проверьте подключение", + "Panoramax sent too much data": "Panoramax прислал слишком много данных", + "Panoramax is busy": "Panoramax перегружен", + "Panoramax sent an unreadable answer": "Panoramax прислал ответ, который не удалось прочитать", + "Apply the full cloud brightness key; 100% filter strength": "Применить полный фильтр яркости облаков; сила фильтра 100%", + "Show the complete infrared image; 0% filter strength": "Показать полное ИК-изображение; сила фильтра 0%", + "FILTER": "ФИЛЬТР", + "Cloud filter strength": "Сила фильтра облаков", + "Panoramax answered HTTP {status}": "Panoramax ответил: HTTP {status}" } diff --git a/src/layers/bikeshare/proxyRoute.test.mjs b/src/layers/bikeshare/proxyRoute.test.mjs index 88b98f31b..9bb04ced0 100644 --- a/src/layers/bikeshare/proxyRoute.test.mjs +++ b/src/layers/bikeshare/proxyRoute.test.mjs @@ -17,6 +17,7 @@ import { createModel } from './model.js'; import { CITY_BY_ID } from './registry.js'; const AUSTIN = CITY_BY_ID.get('austin-capmetro'); +const OSLO = CITY_BY_ID.get('oslo-bysykkel'); const STATION_INFORMATION = { data: { @@ -204,6 +205,101 @@ test('the proxy relays its cache policy and upstream host through the mounted ro ); }); +test('the Norwegian city-bike systems are served through the mounted route', async () => { + for (const [id, systemId] of [ + ['oslo-bysykkel', 'oslobysykkel.no'], + ['bergen-bysykkel', 'bergenbysykkel.no'], + ['trondheim-bysykkel', 'trondheimbysykkel.no'], + ]) { + const system = CITY_BY_ID.get(id); + assert.ok(system, `${id} is registered`); + assert.deepEqual(system.hosts, ['gbfs.urbansharing.com']); + assert.equal( + system.stationStatusUrl, + `https://gbfs.urbansharing.com/${systemId}/station_status.json`, + ); + } + + await withUpstream( + { + [OSLO.stationInformationUrl]: STATION_INFORMATION, + [OSLO.stationStatusUrl]: STATION_STATUS, + }, + async (requested) => { + const source = createBikeshareSource({ fetchImpl: mountGbfsProxy() }); + const info = await source.getStations(OSLO.stationInformationUrl); + const status = await source.getStations(OSLO.stationStatusUrl); + assert.deepEqual(requested, [ + OSLO.stationInformationUrl, + OSLO.stationStatusUrl, + ]); + assert.equal(info.data.stations.length, 1); + assert.equal(status.data.stations.length, 1); + + const response = await mountGbfsProxy()( + '/api/gbfs/' + encodeURIComponent(OSLO.stationStatusUrl), + ); + assert.equal(response.status, 200); + assert.equal( + response.headers.get('x-gbfs-upstream'), + 'gbfs.urbansharing.com', + ); + }, + ); +}); + +test('every Norwegian city-bike system is registered and reachable through the mounted route', async () => { + const expected = { + 'stavanger-kolumbus': + 'https://api.entur.io/mobility/v2/gbfs/v2/kolumbusbysykkel/station_status', + 'skien-farte': + 'https://api.entur.io/mobility/v2/gbfs/v2/fartebysykkel/station_status', + 'lillestrom-bysykkel': + 'https://api.cyclocity.fr/contracts/lillestrom/gbfs/station_status.json', + }; + const upstream = {}; + for (const [id, statusUrl] of Object.entries(expected)) { + const system = CITY_BY_ID.get(id); + assert.ok(system, `${id} is registered`); + assert.equal(system.stationStatusUrl, statusUrl); + upstream[system.stationInformationUrl] = STATION_INFORMATION; + upstream[system.stationStatusUrl] = STATION_STATUS; + } + + await withUpstream(upstream, async (requested) => { + const source = createBikeshareSource({ fetchImpl: mountGbfsProxy() }); + for (const id of Object.keys(expected)) { + const system = CITY_BY_ID.get(id); + const info = await source.getStations(system.stationInformationUrl); + const status = await source.getStations(system.stationStatusUrl); + assert.equal(info.data.stations.length, 1, `${id} information`); + assert.equal(status.data.stations.length, 1, `${id} status`); + } + assert.equal(requested.length, 6); + + // Entur's extensionless station_information keeps its 5-minute cache. + const kolumbus = CITY_BY_ID.get('stavanger-kolumbus'); + const response = await mountGbfsProxy()( + '/api/gbfs/' + encodeURIComponent(kolumbus.stationInformationUrl), + ); + assert.equal(response.headers.get('cache-control'), 'public, max-age=300'); + }); +}); + +test('the mounted route keeps multi-API hosts to their GBFS tree', async () => { + await withUpstream({}, async (requested) => { + const mounted = mountGbfsProxy(); + for (const target of [ + 'https://api.entur.io/journey-planner/v3/station_status', + 'https://api.cyclocity.fr/other/station_status.json', + ]) { + const response = await mounted('/api/gbfs/' + encodeURIComponent(target)); + assert.equal(response.status, 400, target); + } + assert.deepEqual(requested, []); + }); +}); + test('the query-string shape the client used to send is refused by the mounted route', async () => { await withUpstream( { [AUSTIN.stationStatusUrl]: STATION_STATUS }, diff --git a/src/layers/bikeshare/registry.js b/src/layers/bikeshare/registry.js index ff752f664..5592078bc 100644 --- a/src/layers/bikeshare/registry.js +++ b/src/layers/bikeshare/registry.js @@ -23,6 +23,47 @@ function bcycleEntry({ ...buildBcycleUrls(systemId), }; } +function urbanSharingEntry({ + id, + city, + centerLat, + centerLon, + systemId, + provider, + loadRadiusKm = 100, +}) { + return { + id, + city, + centerLat, + centerLon, + loadRadiusKm, + provider, + stationInformationUrl: `https://gbfs.urbansharing.com/${systemId}/station_information.json`, + stationStatusUrl: `https://gbfs.urbansharing.com/${systemId}/station_status.json`, + }; +} +function enturEntry({ + id, + city, + centerLat, + centerLon, + systemId, + provider, + loadRadiusKm = 100, +}) { + const base = `https://api.entur.io/mobility/v2/gbfs/v2/${systemId}`; + return { + id, + city, + centerLat, + centerLon, + loadRadiusKm, + provider, + stationInformationUrl: `${base}/station_information`, + stationStatusUrl: `${base}/station_status`, + }; +} const RAW_GBFS_CITY_REGISTRY = [ { id: 'nyc-citibike', @@ -48,6 +89,56 @@ const RAW_GBFS_CITY_REGISTRY = [ 'https://gbfs.lyft.com/gbfs/2.3/chi/en/station_status.json', provider: 'Divvy', }, + { + id: 'detroit-mogo', + city: 'Detroit, MI', + centerLat: 42.3314, + centerLon: -83.0458, + loadRadiusKm: 90, + stationInformationUrl: + 'https://detroit.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://detroit.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', + provider: 'MoGo', + }, + { + id: 'pittsburgh-pogoh', + city: 'Pittsburgh, PA', + centerLat: 40.4406, + centerLon: -79.9959, + loadRadiusKm: 90, + stationInformationUrl: + 'https://pittsburgh.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://pittsburgh.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', + provider: 'POGOH', + }, + { + id: 'aspen-we-cycle', + city: 'Aspen, CO', + centerLat: 39.1911, + centerLon: -106.8175, + loadRadiusKm: 90, + stationInformationUrl: + 'https://aspen.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://aspen.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', + provider: 'WE-cycle', + }, + // HIBike serves both Kailua-Kona and Hilo, about 95 km apart; the centre sits + // between them so a camera over either town loads the whole system. + { + id: 'hawaii-island-hibike', + city: 'Kona & Hilo, HI', + centerLat: 19.68, + centerLon: -155.54, + loadRadiusKm: 90, + stationInformationUrl: + 'https://kona.publicbikesystem.net/customer/gbfs/v2/en/station_information.json', + stationStatusUrl: + 'https://kona.publicbikesystem.net/customer/gbfs/v2/en/station_status.json', + provider: 'HIBike', + }, { id: 'dc-capital-bikeshare', city: 'Washington, DC', @@ -318,6 +409,59 @@ const RAW_GBFS_CITY_REGISTRY = [ centerLon: -119.6982, systemId: 'bcycle_santabarbara', }), + urbanSharingEntry({ + id: 'oslo-bysykkel', + city: 'Oslo, Norway', + centerLat: 59.9139, + centerLon: 10.7522, + systemId: 'oslobysykkel.no', + provider: 'Oslo Bysykkel', + }), + urbanSharingEntry({ + id: 'bergen-bysykkel', + city: 'Bergen, Norway', + centerLat: 60.3913, + centerLon: 5.3221, + systemId: 'bergenbysykkel.no', + provider: 'Bergen Bysykkel', + }), + urbanSharingEntry({ + id: 'trondheim-bysykkel', + city: 'Trondheim, Norway', + centerLat: 63.4305, + centerLon: 10.3951, + systemId: 'trondheimbysykkel.no', + provider: 'Trondheim Bysykkel', + }), + enturEntry({ + id: 'stavanger-kolumbus', + city: 'Stavanger, Norway', + centerLat: 58.9, + centerLon: 5.72, + systemId: 'kolumbusbysykkel', + provider: 'Kolumbus Bysykkel', + }), + enturEntry({ + id: 'skien-farte', + city: 'Skien/Porsgrunn, Norway', + centerLat: 59.16, + centerLon: 9.64, + systemId: 'fartebysykkel', + provider: 'Farte Bysykkel', + loadRadiusKm: 40, + }), + { + id: 'lillestrom-bysykkel', + city: 'Lillestrøm, Norway', + centerLat: 59.9556, + centerLon: 11.0492, + loadRadiusKm: 40, + stationInformationUrl: + 'https://api.cyclocity.fr/contracts/lillestrom/gbfs/station_information.json', + stationStatusUrl: + 'https://api.cyclocity.fr/contracts/lillestrom/gbfs/station_status.json', + provider: 'Bysykkel Lillestrøm', + }, ]; function normalizeRegistryEntry(entry) { const id = String(entry?.id || '') @@ -346,11 +490,13 @@ function normalizeRegistryEntry(entry) { ) { throw new Error(`GBFS entry "${id}" must use https URLs`); } - // Validate that URLs end with expected GBFS endpoint filenames - if (!/\/station_information\.json$/i.test(stationInformationUrl.pathname)) { + // Validate that URLs end with expected GBFS endpoint names (Entur omits .json) + if ( + !/\/station_information(\.json)?$/i.test(stationInformationUrl.pathname) + ) { throw new Error(`GBFS entry "${id}" station_information URL is invalid`); } - if (!/\/station_status\.json$/i.test(stationStatusUrl.pathname)) { + if (!/\/station_status(\.json)?$/i.test(stationStatusUrl.pathname)) { throw new Error(`GBFS entry "${id}" station_status URL is invalid`); } diff --git a/src/layers/bikeshare/usPbscCities.test.mjs b/src/layers/bikeshare/usPbscCities.test.mjs new file mode 100644 index 000000000..aba2d20ec --- /dev/null +++ b/src/layers/bikeshare/usPbscCities.test.mjs @@ -0,0 +1,77 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { CITY_BY_ID } from './registry.js'; +import { createModel } from './model.js'; +import { createViewport } from './viewport.js'; +import { isAllowedGbfsHost, isAllowedGbfsPath } from '../../data/gbfsSource.js'; + +// Each city's PBSC host, and the stations farthest from its registered centre +// in the live feed (checked 2026-10-13). HIBike's centre sits between its two +// towns, so it lists the outermost station in each. +const CITIES = { + 'detroit-mogo': { + host: 'detroit.publicbikesystem.net', + provider: 'MoGo', + stations: [{ name: 'Robina Ave & W Twelve Mile Rd', lat: 42.50288, lon: -83.18921 }], + }, + 'pittsburgh-pogoh': { + host: 'pittsburgh.publicbikesystem.net', + provider: 'POGOH', + stations: [{ name: 'Wilkinsburg Park & Ride', lat: 40.446951, lon: -79.888593 }], + }, + 'aspen-we-cycle': { + host: 'aspen.publicbikesystem.net', + provider: 'WE-cycle', + stations: [{ name: 'Cowen Dr | Hwy 133', lat: 39.413115, lon: -107.220634 }], + }, + 'hawaii-island-hibike': { + host: 'kona.publicbikesystem.net', + provider: 'HIBike', + stations: [ + { name: 'Kona Community Center', lat: 19.642932, lon: -156.004962 }, + { name: 'Kealoha Beach Park, Hilo', lat: 19.733317, lon: -155.029202 }, + ], + }, +}; + +// The production range check, with no viewer or fetches behind it. +const parts = { model: createModel({ state: {}, services: {}, parts: {}, source: null }) }; +const { computeInRangeCities } = createViewport({ + state: {}, + services: { render: { governorRequestRender() {} } }, + parts, + source: null, +}); + +test('the added US PBSC systems are registered and readable through the proxy', () => { + for (const [id, expected] of Object.entries(CITIES)) { + const city = CITY_BY_ID.get(id); + assert.ok(city, `${id} is registered`); + assert.equal(city.provider, expected.provider); + assert.deepEqual(city.hosts, [expected.host]); + for (const feed of [city.stationInformationUrl, city.stationStatusUrl]) { + const url = new URL(feed); + assert.ok(isAllowedGbfsHost(url.hostname), `${id} host ${url.hostname}`); + assert.ok(isAllowedGbfsPath(url.pathname), `${id} path ${url.pathname}`); + } + } +}); + +test('a camera over each outermost station activates its system', () => { + for (const [id, expected] of Object.entries(CITIES)) { + for (const station of expected.stations) { + assert.ok( + computeInRangeCities({ lat: station.lat, lon: station.lon }).has(id), + `${station.name} activates ${id}`, + ); + } + } +}); + +test('HIBike stays inactive away from Hawaii Island', () => { + // Honolulu is about 270 km from the HIBike centre. + assert.equal( + computeInRangeCities({ lat: 21.3069, lon: -157.8583 }).has('hawaii-island-hibike'), + false, + ); +}); diff --git a/src/layers/streetLevel/imageSize.js b/src/layers/streetLevel/imageSize.js new file mode 100644 index 000000000..dc1131495 --- /dev/null +++ b/src/layers/streetLevel/imageSize.js @@ -0,0 +1,64 @@ +/** + * An image's pixel size read from its header, before anything decodes it: + * a picture host can serve a small file that declares enormous dimensions, + * and a decode of it alone can take the tab (or the GPU process) down. + * JPEG (SOFn), PNG (IHDR), WebP (VP8, VP8L, VP8X) and GIF; null for any + * other format, or a header this does not reach. + * @param {Uint8Array} bytes The start of the file (a JPEG's EXIF can push its SOF past 64 kB). + * @returns {{width: number, height: number}|null} + */ +export function imageSize(bytes) { + if (!bytes || bytes.length < 12) return null; + const b = bytes; + const be16 = (i) => (b[i] << 8) | b[i + 1]; + const le16 = (i) => b[i] | (b[i + 1] << 8); + const le24 = (i) => b[i] | (b[i + 1] << 8) | (b[i + 2] << 16); + const ascii = (i, text) => + [...text].every((char, k) => b[i + k] === char.charCodeAt(0)); + // JPEG: walk the markers to the first start-of-frame. + if (b[0] === 0xff && b[1] === 0xd8) { + let i = 2; + while (i + 9 < b.length) { + if (b[i] !== 0xff) return null; + const marker = b[i + 1]; + // Fill bytes, and markers with no length. + if (marker === 0xff) { + i++; + continue; + } + if (marker === 0x01 || (marker >= 0xd0 && marker <= 0xd9)) { + i += 2; + continue; + } + const isFrame = + marker >= 0xc0 && + marker <= 0xcf && + marker !== 0xc4 && + marker !== 0xc8 && + marker !== 0xcc; + if (isFrame) return { width: be16(i + 7), height: be16(i + 5) }; + i += 2 + be16(i + 2); + } + return null; + } + // PNG: IHDR is the first chunk. + if (b[0] === 0x89 && ascii(1, 'PNG') && ascii(12, 'IHDR')) { + const u32 = (i) => + ((b[i] << 24) | (b[i + 1] << 16) | (b[i + 2] << 8) | b[i + 3]) >>> 0; + return { width: u32(16), height: u32(20) }; + } + if (ascii(0, 'RIFF') && ascii(8, 'WEBP') && b.length >= 30) { + if (ascii(12, 'VP8 ')) + return { width: le16(26) & 0x3fff, height: le16(28) & 0x3fff }; + if (ascii(12, 'VP8L')) + return { + width: 1 + (((b[22] & 0x3f) << 8) | b[21]), + height: + 1 + (((b[24] & 0x0f) << 10) | (b[23] << 2) | ((b[22] & 0xc0) >> 6)), + }; + if (ascii(12, 'VP8X')) return { width: 1 + le24(24), height: 1 + le24(27) }; + return null; + } + if (ascii(0, 'GIF8')) return { width: le16(6), height: le16(8) }; + return null; +} diff --git a/src/layers/streetLevel/imageSize.test.mjs b/src/layers/streetLevel/imageSize.test.mjs new file mode 100644 index 000000000..e83978596 --- /dev/null +++ b/src/layers/streetLevel/imageSize.test.mjs @@ -0,0 +1,82 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { imageSize } from './imageSize.js'; + +const bytes = (...parts) => + Uint8Array.from( + parts.flat().map((v) => (typeof v === 'string' ? v.charCodeAt(0) : v)), + ); +const ascii = (text) => [...text]; + +test('a JPEG’s size is read from its first frame, past an EXIF segment', () => { + const exif = [0xff, 0xe1, 0x00, 0x10, ...new Array(14).fill(0)]; + const sof = [0xff, 0xc0, 0x00, 0x11, 0x08, 0x0b, 0xc0, 0x0f, 0xc0]; + const jpeg = bytes([0xff, 0xd8], exif, sof, new Array(20).fill(0)); + assert.deepEqual(imageSize(jpeg), { width: 0x0fc0, height: 0x0bc0 }); + // Progressive (SOF2) too; a DHT (C4) is not a frame. + const dht = [0xff, 0xc4, 0x00, 0x04, 0, 0]; + const sof2 = [0xff, 0xc2, 0x00, 0x11, 0x08, 0x00, 0x10, 0x00, 0x20]; + assert.deepEqual( + imageSize(bytes([0xff, 0xd8], dht, sof2, new Array(10).fill(0))), + { + width: 32, + height: 16, + }, + ); +}); + +test('PNG, WebP (lossy, lossless, extended) and GIF sizes', () => { + const png = bytes( + [0x89], + ascii('PNG'), + [0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13], + ascii('IHDR'), + [0, 0, 0x10, 0, 0, 0, 0x08, 0], + ); + assert.deepEqual(imageSize(png), { width: 4096, height: 2048 }); + const riff = (chunk, rest) => + bytes(ascii('RIFF'), [0, 0, 0, 0], ascii('WEBP'), ascii(chunk), rest); + const vp8 = riff('VP8 ', [...new Array(10).fill(0), 0x00, 0x08, 0x00, 0x04]); + assert.deepEqual(imageSize(vp8), { width: 2048, height: 1024 }); + // VP8L: 14-bit width-1 and height-1 packed after the 0x2f signature. + const w = 99; + const h = 49; + const packed = w | (h << 14); + const vp8l = riff('VP8L', [ + 0, + 0, + 0, + 0, + 0x2f, + packed & 0xff, + (packed >> 8) & 0xff, + (packed >> 16) & 0xff, + (packed >> 24) & 0xff, + 0, + 0, + 0, + 0, + 0, + 0, + ]); + assert.deepEqual(imageSize(vp8l), { width: 100, height: 50 }); + const vp8x = riff( + 'VP8X', + [0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0x0f, 0x00, 0xff, 0x07, 0x00], + ); + assert.deepEqual(imageSize(vp8x), { width: 4096, height: 2048 }); + const gif = bytes(ascii('GIF89a'), [0x40, 0x01, 0xf0, 0x00], [0, 0]); + assert.deepEqual(imageSize(gif), { width: 320, height: 240 }); +}); + +test('anything else, or a header cut short, has no size', () => { + assert.equal(imageSize(null), null); + assert.equal(imageSize(new Uint8Array(4)), null); + assert.equal(imageSize(bytes(ascii(''))), null); + assert.equal( + imageSize( + bytes([0xff, 0xd8, 0xff, 0xe1, 0xff, 0xff], new Array(8).fill(0)), + ), + null, + ); +}); diff --git a/src/layers/streetLevel/limiter.js b/src/layers/streetLevel/limiter.js new file mode 100644 index 000000000..c986bf356 --- /dev/null +++ b/src/layers/streetLevel/limiter.js @@ -0,0 +1,49 @@ +/** + * A gate letting at most `limit` tasks run at once, first in first out, or + * newest first with `lifo` (the last request is the one the user waits on). + */ +export function createLimiter(limit, { lifo = false } = {}) { + let running = 0; + const queue = []; + function next() { + while (running < limit && queue.length) { + const { task, resolve, reject, signal } = lifo + ? queue.pop() + : queue.shift(); + if (signal?.aborted) { + reject(signal.reason); + continue; + } + running++; + Promise.resolve() + .then(task) + .then(resolve, reject) + .finally(() => { + running--; + next(); + }); + } + } + return { + run(task, { signal } = {}) { + return new Promise((resolve, reject) => { + const item = { task, resolve, reject, signal }; + queue.push(item); + signal?.addEventListener( + 'abort', + () => { + const at = queue.indexOf(item); + if (at === -1) return; + queue.splice(at, 1); + reject(signal.reason); + }, + { once: true }, + ); + next(); + }); + }, + get pending() { + return running + queue.length; + }, + }; +} diff --git a/src/layers/streetLevel/limiter.test.mjs b/src/layers/streetLevel/limiter.test.mjs new file mode 100644 index 000000000..1137fe2f6 --- /dev/null +++ b/src/layers/streetLevel/limiter.test.mjs @@ -0,0 +1,84 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createLimiter } from './limiter.js'; + +const tick = () => new Promise((resolve) => setTimeout(resolve, 0)); + +/** A task that runs until `finish()`. */ +function held(log, name) { + let finish; + const done = new Promise((resolve) => (finish = resolve)); + return { + task: () => { + log.push(name); + return done.then(() => name); + }, + finish: () => finish(), + }; +} + +test('at most `limit` tasks run; queued ones start first in, first out', async () => { + const log = []; + const gate = createLimiter(2); + const tasks = ['a', 'b', 'c', 'd'].map((name) => held(log, name)); + const results = tasks.map(({ task }) => gate.run(task)); + await tick(); + assert.deepEqual(log, ['a', 'b']); + assert.equal(gate.pending, 4); + tasks[0].finish(); + await tick(); + assert.deepEqual(log, ['a', 'b', 'c']); + for (const { finish } of tasks) finish(); + assert.deepEqual(await Promise.all(results), ['a', 'b', 'c', 'd']); + assert.equal(gate.pending, 0); +}); + +test('lifo starts the newest queued task next', async () => { + const log = []; + const gate = createLimiter(1, { lifo: true }); + const tasks = ['a', 'b', 'c'].map((name) => held(log, name)); + const results = tasks.map(({ task }) => gate.run(task)); + await tick(); + tasks[0].finish(); + await tick(); + assert.deepEqual(log, ['a', 'c']); + tasks[2].finish(); + tasks[1].finish(); + await Promise.all(results); +}); + +test('a queued task whose signal aborts never runs, leaves the queue and rejects with the reason', async () => { + const log = []; + const gate = createLimiter(1); + const first = held(log, 'a'); + const running = gate.run(first.task); + const controller = new AbortController(); + const queued = gate.run(held(log, 'b').task, { signal: controller.signal }); + await tick(); + assert.equal(gate.pending, 2); + controller.abort(new Error('overtaken')); + await assert.rejects(queued, /overtaken/); + assert.equal(gate.pending, 1, 'out of the queue'); + first.finish(); + await running; + await tick(); + assert.deepEqual(log, ['a'], 'b never ran'); +}); + +test('a task already aborted when its turn comes is skipped', async () => { + const log = []; + const gate = createLimiter(1); + const first = held(log, 'a'); + const running = gate.run(first.task); + const signal = { + aborted: false, + reason: new Error('gone'), + addEventListener() {}, + }; + const queued = gate.run(held(log, 'b').task, { signal }); + signal.aborted = true; + first.finish(); + await running; + await assert.rejects(queued, /gone/); + assert.deepEqual(log, ['a']); +}); diff --git a/src/layers/streetLevel/lineCoverage.js b/src/layers/streetLevel/lineCoverage.js index 48d783a54..a52853f5b 100644 --- a/src/layers/streetLevel/lineCoverage.js +++ b/src/layers/streetLevel/lineCoverage.js @@ -13,6 +13,8 @@ import { msg } from '../../i18n/index.js'; import { MESH_SAMPLE_RADIUS_M } from './meshSampler.js'; import { tileBounds, tilesForBbox } from './tileMath.js'; +/** A tile's shared point cap must move by this much to redraw it. */ +const SHARE_HYSTERESIS = 0.1; /** Draped lines stay at most this long while a tile's cast lines build. */ export const SWAP_MAX_WAIT_MS = 4000; /** Tiles touched by new mesh samples are redrawn at most this often. */ @@ -73,6 +75,8 @@ export function meshBoxInRange(box, centre) { * @property {number} viewRangeMinM Street-zoom coverage reaches at least this far… * @property {number} viewRangePerHeight …or camera height times this. * @property {number} viewNearM Ground always covered around a tilted camera. + * @property {boolean} [keepOrder] Cap a tile's lines in the order decoded (default: newest first). + * @property {number} [maxPoints] Line points drawn at once, over every tile (default: no limit). */ /** @@ -95,6 +99,7 @@ export function meshBoxInRange(box, centre) { * @param {LineCoverageTuning} options.tuning * @param {{keyRejected?: string, rateLimited: string, tooHigh?: string}} options.messages * @param {(sequence: object, filter: object) => boolean} [options.passes] Whether the imagery filter keeps a line or point. + * @param {(point: object) => number} [options.overviewPointSize] An overview point's size in pixels (default `tuning.overviewPointPx`). */ export function createLineCoverage({ state, @@ -106,9 +111,13 @@ export function createLineCoverage({ tuning, messages, passes = passesImageryFilter, + overviewPointSize = null, }) { const { render } = state.services; const perTileSequenceCap = Math.floor(tuning.maxSequences / tuning.maxTiles); + const perTilePointCap = tuning.maxPoints + ? Math.floor(tuning.maxPoints / tuning.maxTiles) + : Infinity; /** Colour for a line: the provider's, GEV cyan while selected. */ function sequenceColor({ selected = false } = {}) { @@ -144,18 +153,114 @@ export function createLineCoverage({ /** * A tile's sequences, filtered then capped: capping first could leave a - * dense tile of newer flat captures with no 360° lines at all. + * dense tile of newer flat captures with no 360° lines at all. `measure` + * only records what the lines need, for a draw that follows. */ - function drawnSequences(entry) { + function drawnSequences(entry, { measure = false } = {}) { const drawn = []; const current = filter(); + const selectedId = state.sequence.selectedId ?? null; + const cap = entry.pointCap ?? perTilePointCap; + let points = 0; + let demand = 0; for (const sequence of entry.sequenceList) { - if (drawn.length >= perTileSequenceCap) break; - if (passes(sequence, current)) drawn.push(sequence); + if (!passes(sequence, current)) continue; + const size = pointsOf(sequence); + demand += size; + // The selected sequence is always drawn: its highlight is the answer + // to a click, whatever the caps leave out. It is outside the cap, so + // it is not counted against it. + if (sequence.id === selectedId) { + drawn.push(sequence); + continue; + } + if (drawn.length >= perTileSequenceCap) continue; + // A line past the point budget is skipped; a shorter one may still fit. + if (points + size > cap) continue; + points += size; + drawn.push(sequence); + } + // What every passing line would take: the budget is shared by it. + entry.pointDemand = demand; + if (!measure) { + entry.pointsDrawn = points; + // The cap this draw used: a share is judged against it, not against + // later shares that redrew nothing (small steps add up). + entry.drawnCap = cap; } return drawn; } + /** + * Share the point budget across the tiles in view (water-filling): a + * sparse tile keeps what it needs and the rest goes to the dense ones, + * so a city centre is not drawn thinner than its suburbs. Sets each + * tile's cap; returns the tiles whose cap changes what they draw. + */ + function shareCaps() { + const changed = []; + if (!tuning.maxPoints) return changed; + const entries = [...state.coverage.tiles.values()].filter( + (entry) => entry.kind === 'sequence', + ); + const byDemand = [...entries].sort( + (a, b) => (a.pointDemand || 0) - (b.pointDemand || 0), + ); + let left = tuning.maxPoints; + byDemand.forEach((entry, i) => { + const share = left / (byDemand.length - i); + const cap = Math.floor(Math.min(entry.pointDemand || 0, share)); + left -= cap; + const before = entry.drawnCap ?? perTilePointCap; + entry.pointCap = cap; + // Only a change that alters what is drawn, by enough to be worth a + // rebuild: more room for a tile that had lines left out, or well + // under what it draws now (a small overshoot stays until the next + // natural redraw). + const grows = + cap > before * (1 + SHARE_HYSTERESIS) && + entry.pointsDrawn < entry.pointDemand; + const shrinks = cap < entry.pointsDrawn * (1 - SHARE_HYSTERESIS); + if (grows || shrinks) changed.push(entry); + }); + return changed; + } + + /** + * Once loading settles: redraw the tiles the shared budget changes, + * keeping their old lines until the new ones are ready (no blink). + */ + function shareBudget() { + const changed = shareCaps(); + for (const entry of changed) redrawEntry(entry, { swap: true }); + if (changed.length) { + requestRender(); + notify(); + } + } + + /** + * Draw one tile again from its decoded lines (filter, caps or selection + * changed). With `swap`, its old lines stay until the new ones are ready. + */ + function redrawEntry(entry, { swap = false } = {}) { + cancelCast(entry); + entry.castRequested = false; + entry.meshParts = null; + if (swap && entry.kind === 'sequence' && entry.primitives?.length) { + redrawTile(entry); + return; + } + detachPrimitive(entry); + attachPrimitive(entry); + } + + function pointsOf(sequence) { + let count = 0; + for (const part of sequence.parts) count += part.length; + return count; + } + /** * Batched primitives for a tile's sequence parts: cast polylines where * terrain heights are cached, draped ground lines otherwise. Each batch @@ -268,7 +373,22 @@ export function createLineCoverage({ /** Rebuild a tile's lines; the old ones stay up until the new are ready. */ function redrawTile(entry) { - const previous = entry.primitives; + let previous = entry.primitives; + if (entry.swap) { + // A swap still building: what is on screen is its old lines and + // whichever new ones are ready. Those stay; the rest never showed. + const shown = [...entry.swap.old]; + const building = []; + for (const record of previous) + (record.primitive.ready && !record.primitive.isDestroyed?.() + ? shown + : building + ).push(record); + entry.swap.stop(); + entry.swap = null; + removePrimitives(building); + previous = shown; + } entry.primitives = []; attachPrimitive(entry); removeWhenReady(entry, previous); @@ -427,7 +547,7 @@ export function createLineCoverage({ collection.add({ position: Cesium.Cartesian3.fromDegrees(point.lon, point.lat), color: tint, - pixelSize: tuning.overviewPointPx, + pixelSize: overviewPointSize?.(point) ?? tuning.overviewPointPx, // Google 3D terrain and clouds must not hide the near side's dots; // the horizon cull hides the far side's, which this lets through. disableDepthTestDistance: Number.POSITIVE_INFINITY, @@ -560,10 +680,13 @@ export function createLineCoverage({ north: bounds.north + pad, }; if (kind === 'sequence') { - // Newest first; the per-tile cap applies to what passes the filter. - entry.sequenceList = decoded.sequences.sort( - (a, b) => (b.capturedAt || 0) - (a.capturedAt || 0), - ); + // Newest first, unless the provider ordered them itself; the + // per-tile cap applies to what passes the filter. + entry.sequenceList = tuning.keepOrder + ? decoded.sequences + : decoded.sequences.sort( + (a, b) => (b.capturedAt || 0) - (a.capturedAt || 0), + ); entry.sequences = new Map(); entry.total = decoded.sequences.length; } else { @@ -595,7 +718,10 @@ export function createLineCoverage({ // must not drop a newer request's entry (which is what counts as loading). if (state.coverage.pending.get(key) === controller) { state.coverage.pending.delete(key); - if (!state.coverage.pending.size) purgeStale(); + if (!state.coverage.pending.size) { + purgeStale(); + shareBudget(); + } } notify(); } @@ -735,7 +861,10 @@ export function createLineCoverage({ state.coverage.pending.delete(key); } for (const tile of tiles) loadTile(tile, kind); - if (!state.coverage.pending.size) purgeStale(); + if (!state.coverage.pending.size) { + purgeStale(); + shareBudget(); + } // The camera moved: cells that were out of the sampler's range may not be, // and lines left draped by a failed, partial or dropped cast get another try. if (terrainMode()) @@ -788,13 +917,12 @@ export function createLineCoverage({ /** Rebuild every loaded tile from its decoded cache (after a filter change). */ function rebuild() { purgeStale(); - for (const entry of state.coverage.tiles.values()) { - cancelCast(entry); - entry.castRequested = false; - entry.meshParts = null; - detachPrimitive(entry); - attachPrimitive(entry); - } + // What each tile's lines need under the new filter, then the shares, + // then one redraw each. + for (const entry of state.coverage.tiles.values()) + if (entry.kind === 'sequence') drawnSequences(entry, { measure: true }); + shareCaps(); + for (const entry of state.coverage.tiles.values()) redrawEntry(entry); requestRender(); notify(); } @@ -848,6 +976,17 @@ export function createLineCoverage({ /** Recolour one sequence, every part of it, in place (selection highlight). */ function recolorSequence(id, selected) { + // Selected but left out by a cap: draw it (it is always admitted). + if (selected && (tuning.maxPoints || 0) > 0) + for (const entry of state.coverage.tiles.values()) + if ( + entry.kind === 'sequence' && + !entry.sequences.has(id) && + entry.sequenceList.some( + (sequence) => sequence.id === id && passes(sequence, filter()), + ) + ) + redrawEntry(entry, { swap: true }); for (const entry of drawnEntries()) { const sequence = entry.sequences.get(id); if (!sequence) continue; diff --git a/src/layers/streetLevel/lineCoverage.test.mjs b/src/layers/streetLevel/lineCoverage.test.mjs index 88308673c..7fdc19773 100644 --- a/src/layers/streetLevel/lineCoverage.test.mjs +++ b/src/layers/streetLevel/lineCoverage.test.mjs @@ -25,7 +25,15 @@ const TUNING = Object.freeze({ */ function setup( t, - { zoomFor, messages = { rateLimited: 'busy' }, passes, fail } = {}, + { + zoomFor, + messages = { rateLimited: 'busy' }, + passes, + fail, + decoded = null, + tuning = TUNING, + overviewPointSize = null, + } = {}, ) { const centre = tileCentre(-121.4944, 38.5816); const viewer = coverageViewer( @@ -68,6 +76,7 @@ function setup( fetchTile: async (tile) => { asked.push(tile); if (fail) throw fail; + if (decoded) return decoded(centre, tile); return { sequences: [ { @@ -94,9 +103,10 @@ function setup( colors: { coverage: '#635bff', selected: '#00e5ff' }, pickPrefix: 'test:line:', renderKey: 'test-coverage', - tuning: TUNING, + tuning, messages, ...(passes ? { passes } : {}), + ...(overviewPointSize ? { overviewPointSize } : {}), }); t.after(() => { coverage.clear(); @@ -140,3 +150,383 @@ test('a refused key without its own message keeps the error it came with', async assert.ok(await until(() => state.keyRejected === true)); assert.equal(state.coverage.lastError, 'the source refused the key'); }); + +/** Three lines in one tile: an old long one, then two newer short ones. */ +function threeLines(centre) { + const line = (id, capturedAt, span) => ({ + id, + parts: [ + [ + [centre.lon - span, centre.lat], + [centre.lon + span, centre.lat], + ], + ], + capturedAt, + isPano: false, + }); + return { + sequences: [ + line('old-long', 1, 0.002), + line('new-short', 2, 0.0002), + line('newer-short', 3, 0.0001), + ], + overview: [], + }; +} + +test('the per-tile cap keeps the newest lines, or the provider’s own order with keepOrder', async (t) => { + const capped = { ...TUNING, maxTiles: 1, maxSequences: 2 }; + const drawnIds = (state) => + [...state.coverage.tiles.values()].flatMap((tile) => [ + ...tile.sequences.keys(), + ]); + const newest = setup(t, { decoded: threeLines, tuning: capped }); + newest.coverage.refresh(); + assert.ok(await until(() => newest.state.coverage.tiles.size === 1)); + assert.deepEqual(drawnIds(newest.state).sort(), ['new-short', 'newer-short']); + const kept = setup(t, { + decoded: threeLines, + tuning: { ...capped, keepOrder: true }, + }); + kept.coverage.refresh(); + assert.ok(await until(() => kept.state.coverage.tiles.size === 1)); + assert.deepEqual(drawnIds(kept.state), ['old-long', 'new-short']); +}); + +test('overview points take the provider’s size, each its own', async (t) => { + const { state, coverage, viewer } = setup(t, { + zoomFor: () => ({ zoom: 3, kind: 'overview' }), + tuning: { ...TUNING, overviewMaxTiles: 4 }, + decoded: (centre) => ({ + sequences: [], + overview: [ + { id: 'a', lon: centre.lon, lat: centre.lat, weight: 1 }, + { id: 'b', lon: centre.lon + 0.01, lat: centre.lat, weight: 50_000 }, + ], + }), + overviewPointSize: (point) => (point.weight > 1 ? 7 : 3), + }); + coverage.refresh(); + assert.ok(await until(() => state.coverage.tiles.size > 0)); + const sizes = []; + for (const primitive of viewer.scene.primitives.items) + for (let i = 0; i < (primitive.length ?? 0); i++) + sizes.push(primitive.get(i).pixelSize); + assert.deepEqual(sizes.sort(), [3, 7]); +}); + +test('a point budget skips lines past it, and a shorter one later may still fit', async (t) => { + const { state, coverage } = setup(t, { + tuning: { + ...TUNING, + maxTiles: 1, + maxSequences: 10, + maxPoints: 5, + keepOrder: true, + }, + decoded: (centre) => { + const line = (id, n) => ({ + id, + parts: [ + Array.from({ length: n }, (_, i) => [ + centre.lon - 0.001 + i * 0.0001, + centre.lat, + ]), + ], + capturedAt: 0, + isPano: false, + }); + return { + sequences: [line('three', 3), line('four', 4), line('two', 2)], + overview: [], + }; + }, + }); + coverage.refresh(); + assert.ok(await until(() => state.coverage.tiles.size === 1)); + const [tile] = state.coverage.tiles.values(); + assert.deepEqual([...tile.sequences.keys()], ['three', 'two']); +}); + +/** A line of `n` points, its id unique to the tile. */ +function pointsLine(id, n, tile, centre) { + return { + id, + parts: [ + Array.from({ length: n }, (_, i) => [ + centre.lon - 0.001 + i * 0.00001, + centre.lat, + ]), + ], + capturedAt: 0, + isPano: false, + }; +} + +test('the point budget is shared: a sparse tile keeps its few, a dense one gets the rest', async (t) => { + const tuning = { + ...TUNING, + maxTiles: 9, + maxSequences: 900, + maxPoints: 90, + keepOrder: true, + }; + const { state, coverage } = setup(t, { + // Street zoom far enough in that several tiles are in view. + zoomFor: () => ({ zoom: 18, kind: 'sequence' }), + tuning, + decoded: (centre, tile) => { + const dense = tile.x % 2 === 0 && tile.y % 2 === 0; + const lines = dense + ? Array.from({ length: 30 }, (_, i) => + pointsLine(`d${tile.x}/${tile.y}/${i}`, 5, tile, centre), + ) + : [pointsLine(`s${tile.x}/${tile.y}`, 2, tile, centre)]; + return { sequences: lines, overview: [] }; + }, + }); + coverage.refresh(); + assert.ok( + await until( + () => state.coverage.tiles.size > 1 && !state.coverage.pending.size, + ), + ); + const entries = [...state.coverage.tiles.values()]; + const drawn = (entry) => entry.pointsDrawn; + const total = entries.reduce((sum, entry) => sum + drawn(entry), 0); + assert.ok(total <= tuning.maxPoints, `within the budget: ${total}`); + const sparse = entries.filter((entry) => entry.pointDemand === 2); + const dense = entries.filter((entry) => entry.pointDemand === 150); + assert.ok(sparse.length && dense.length, 'both kinds in view'); + for (const entry of sparse) + assert.equal(drawn(entry), 2, 'a sparse tile keeps all'); + const fair = Math.floor(tuning.maxPoints / tuning.maxTiles); + const left = tuning.maxPoints - sparse.length * 2; + for (const entry of dense) + assert.ok( + drawn(entry) > fair || dense.length * fair >= left, + `a dense tile gets more than the even share: ${drawn(entry)}`, + ); + assert.ok( + total > tuning.maxPoints - 5 * dense.length, + 'little budget unused', + ); +}); + +test('a selected sequence the budget left out is drawn, highlighted', async (t) => { + const { state, coverage } = setup(t, { + tuning: { + ...TUNING, + maxTiles: 1, + maxSequences: 10, + maxPoints: 5, + keepOrder: true, + }, + decoded: (centre, tile) => ({ + sequences: [ + pointsLine('long', 5, tile, centre), + pointsLine('short', 3, tile, centre), + ], + overview: [], + }), + }); + coverage.refresh(); + assert.ok(await until(() => state.coverage.tiles.size === 1)); + const [tile] = state.coverage.tiles.values(); + assert.deepEqual([...tile.sequences.keys()], ['long']); + state.sequence.selectedId = 'short'; + coverage.recolorSequence('short', true); + assert.deepEqual([...tile.sequences.keys()].sort(), ['long', 'short']); +}); + +/** Count the line primitives a coverage adds to the scene. */ +function countAdds(viewer) { + const counts = { added: 0 }; + for (const collection of [ + viewer.scene.groundPrimitives, + viewer.scene.primitives, + ]) { + const add = collection.add.bind(collection); + collection.add = (primitive) => { + counts.added++; + return add(primitive); + }; + } + return counts; +} + +test('a tile redrawn for a new share keeps its old lines until the new ones are ready', async (t) => { + const tuning = { + ...TUNING, + maxTiles: 9, + maxSequences: 900, + maxPoints: 90, + keepOrder: true, + }; + const { state, coverage, viewer } = setup(t, { + zoomFor: () => ({ zoom: 18, kind: 'sequence' }), + tuning, + decoded: (centre, tile) => { + const dense = tile.x % 2 === 0 && tile.y % 2 === 0; + return { + sequences: dense + ? Array.from({ length: 30 }, (_, i) => + pointsLine(`d${tile.x}/${tile.y}/${i}`, 5, tile, centre), + ) + : [pointsLine(`s${tile.x}/${tile.y}`, 2, tile, centre)], + overview: [], + }; + }, + }); + coverage.refresh(); + assert.ok( + await until( + () => state.coverage.tiles.size > 1 && !state.coverage.pending.size, + ), + ); + const redrawn = [...state.coverage.tiles.values()].filter( + (entry) => entry.swap, + ); + assert.ok(redrawn.length > 0, 'a dense tile took a bigger share'); + for (const entry of redrawn) + for (const { primitive } of entry.swap.old) + assert.ok( + viewer.scene.groundPrimitives.items.has(primitive) || + viewer.scene.primitives.items.has(primitive), + 'the old lines are still on the globe while the new ones build', + ); +}); + +test('a second redraw while a swap still builds keeps the lines on screen', async (t) => { + const { state, coverage, viewer } = setup(t, { + zoomFor: () => ({ zoom: 18, kind: 'sequence' }), + tuning: { + ...TUNING, + maxTiles: 9, + maxSequences: 900, + maxPoints: 90, + keepOrder: true, + }, + decoded: (centre, tile) => { + const dense = tile.x % 2 === 0 && tile.y % 2 === 0; + return { + sequences: dense + ? Array.from({ length: 30 }, (_, i) => + pointsLine(`d${tile.x}/${tile.y}/${i}`, 5, tile, centre), + ) + : [pointsLine(`s${tile.x}/${tile.y}`, 2, tile, centre)], + overview: [], + }; + }, + }); + coverage.refresh(); + assert.ok( + await until( + () => state.coverage.tiles.size > 1 && !state.coverage.pending.size, + ), + ); + // A tile mid-swap (its new lines never become ready here) with a line + // the cap left out: selecting that line redraws the tile again. + const entry = [...state.coverage.tiles.values()].find( + (tile) => + tile.swap && + tile.sequenceList.some((sequence) => !tile.sequences.has(sequence.id)), + ); + assert.ok(entry, 'a swapping tile with a line left out'); + const onScreen = [...entry.swap.old]; + const building = entry.primitives.map(({ primitive }) => primitive); + const left = entry.sequenceList.find( + (sequence) => !entry.sequences.has(sequence.id), + ); + state.sequence.selectedId = left.id; + coverage.recolorSequence(left.id, true); + assert.ok(entry.sequences.has(left.id), 'the selected line is drawn'); + const onGlobe = (primitive) => + viewer.scene.groundPrimitives.items.has(primitive) || + viewer.scene.primitives.items.has(primitive); + for (const { primitive } of onScreen) + assert.ok( + onGlobe(primitive), + 'the lines on screen stay until the new ones are ready', + ); + for (const primitive of building) + assert.ok(!onGlobe(primitive), 'the first swap’s unready lines are gone'); + assert.ok(entry.swap, 'still swapping'); +}); + +test('a selected line outside the cap does not make its tile redraw on every settle, nor does one the filter hides', async (t) => { + let pano = 'all'; + const { state, coverage, viewer } = setup(t, { + tuning: { + ...TUNING, + maxTiles: 1, + maxSequences: 10, + maxPoints: 5, + keepOrder: true, + }, + decoded: (centre, tile) => ({ + sequences: [ + pointsLine('long', 5, tile, centre), + { ...pointsLine('short', 3, tile, centre), isPano: true }, + ], + overview: [], + }), + }); + state.context.getFilter = () => ({ pano, sinceMs: null }); + coverage.refresh(); + assert.ok(await until(() => state.coverage.tiles.size === 1)); + state.sequence.selectedId = 'short'; + coverage.recolorSequence('short', true); + const counts = countAdds(viewer); + for (let i = 0; i < 3; i++) { + coverage.refresh(); + await new Promise((resolve) => setTimeout(resolve, 20)); + } + assert.equal(counts.added, 0, 'settles redraw nothing'); + // A selection the filter hides is not drawn, so it is not redrawn for. + pano = 'flat'; + coverage.rebuild(); + const after = countAdds(viewer); + for (let i = 0; i < 3; i++) coverage.recolorSequence('short', true); + assert.equal(after.added, 0); +}); + +test('small share increases add up: a retained tile redraws once its share has grown past the threshold since its last draw', async (t) => { + const tuning = { + ...TUNING, + maxTiles: 1, + maxSequences: 900, + maxPoints: 100, + keepOrder: true, + }; + const { state, coverage } = setup(t, { + tuning, + decoded: (centre, tile) => ({ + sequences: Array.from({ length: 30 }, (_, i) => + pointsLine(`l${i}`, 4, tile, centre), + ), + overview: [], + }), + }); + coverage.refresh(); + assert.ok( + await until( + () => state.coverage.tiles.size === 1 && !state.coverage.pending.size, + ), + ); + const [tile] = state.coverage.tiles.values(); + assert.equal(tile.pointsDrawn, 100); + const settle = async () => { + coverage.refresh(); + await until(() => !state.coverage.pending.size); + await new Promise((resolve) => setTimeout(resolve, 10)); + }; + // +8%: under the threshold, nothing is redrawn. + tuning.maxPoints = 108; + await settle(); + assert.equal(tile.pointsDrawn, 100); + // +8% again: +16% since the tile was last drawn, so it is redrawn. + tuning.maxPoints = 116; + await settle(); + assert.equal(tile.pointsDrawn, 116, 'the grown share is drawn'); +}); diff --git a/src/layers/streetLevel/providers/kartaview/panorama.js b/src/layers/streetLevel/panorama.js similarity index 98% rename from src/layers/streetLevel/providers/kartaview/panorama.js rename to src/layers/streetLevel/panorama.js index 50f348a9b..72f5718a0 100644 --- a/src/layers/streetLevel/providers/kartaview/panorama.js +++ b/src/layers/streetLevel/panorama.js @@ -188,13 +188,15 @@ export function createPanoramaRenderer(canvas) { gl.UNSIGNED_BYTE, image, ); + // Read before the close: a closed bitmap measures 0×0. + const { width, height } = image; // The pixels are on the GPU now; a bitmap's copy can go. image.close?.(); // Linear only: a mipmapped lookup draws a seam where longitude wraps. gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR); gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR); const repeat = - isPowerOfTwo(image.width) && isPowerOfTwo(image.height) + isPowerOfTwo(width) && isPowerOfTwo(height) ? gl.REPEAT : gl.CLAMP_TO_EDGE; gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, repeat); diff --git a/src/layers/streetLevel/providers/kartaview/panorama.test.mjs b/src/layers/streetLevel/panorama.test.mjs similarity index 86% rename from src/layers/streetLevel/providers/kartaview/panorama.test.mjs rename to src/layers/streetLevel/panorama.test.mjs index 5bb6380d1..6a0c7c8b8 100644 --- a/src/layers/streetLevel/providers/kartaview/panorama.test.mjs +++ b/src/layers/streetLevel/panorama.test.mjs @@ -167,3 +167,33 @@ test('an image uploads as RGBA and its bitmap is released; closing hands the tex ); assert.equal(renderer.hasImage(), false); }); + +test('a power-of-two bitmap wraps in longitude even though it is closed once uploaded', (t) => { + const { gl, renderer } = stubPanorama(t); + const wraps = []; + gl.texParameteri = (target, name, value) => wraps.push([name, value]); + gl.TEXTURE_WRAP_S = 'WRAP_S'; + gl.REPEAT = 'REPEAT'; + gl.CLAMP_TO_EDGE = 'CLAMP'; + const bitmap = { + width: 2048, + height: 1024, + // A closed ImageBitmap measures 0×0. + close() { + this.width = 0; + this.height = 0; + }, + }; + renderer.setImage(bitmap); + assert.deepEqual( + wraps.find(([name]) => name === 'WRAP_S'), + ['WRAP_S', 'REPEAT'], + ); + wraps.length = 0; + renderer.setImage({ width: 2000, height: 1000 }); + assert.deepEqual( + wraps.find(([name]) => name === 'WRAP_S'), + ['WRAP_S', 'CLAMP'], + 'other sizes clamp (WebGL 1 repeats powers of two only)', + ); +}); diff --git a/src/layers/streetLevel/photoViewer.js b/src/layers/streetLevel/photoViewer.js new file mode 100644 index 000000000..705fa8524 --- /dev/null +++ b/src/layers/streetLevel/photoViewer.js @@ -0,0 +1,926 @@ +import { + createPanoramaRenderer, + PANO_FOV_DEFAULT_DEG, + wrapDegrees, +} from './panorama.js'; +import { imageSize } from './imageSize.js'; +import { msg, t } from '../../i18n/index.js'; + +/** A photo download (tens of kilobytes to a few megabytes). */ +export const IMAGE_TIMEOUT_MS = 15_000; + +/** A picture file or decoded side past these is refused (a photo is a few MB). */ +export const MAX_IMAGE_BYTES = 48 * 1024 * 1024; +export const MAX_IMAGE_SIDE = 16_384; +/** 64 MP: 256 MB once decoded to RGBA. A 16,384 px square is four times that. */ +export const MAX_IMAGE_PIXELS = 8_192 * 8_192; +/** Where in a file to look for its dimensions (a JPEG's EXIF comes first). */ +const HEADER_BYTES = 256 * 1024; + +/** Whether a picture this size is past what the viewer decodes. */ +export function tooManyPixels(width, height) { + return ( + width > MAX_IMAGE_SIDE || + height > MAX_IMAGE_SIDE || + width * height > MAX_IMAGE_PIXELS + ); +} + +/** Panorama texture widths: enough for a docked viewer, and for an expanded one. */ +export const PANO_MID_SIZE = Object.freeze([2048, 1024]); +export const PANO_FULL_SIZE = Object.freeze([4096, 2048]); + +/** + * Load an image within the timeout. A CORS-clean one (for WebGL) is decoded + * off the main thread into an ImageBitmap where the browser can: uploading + * a decoded bitmap costs a fraction of uploading an (90 ms → 15 ms for + * 4096×2048 on a phone-class CPU). `resize` shrinks a bitmap while decoding + * it (a panorama larger than any texture the viewer uploads). + * @param {string} url + * @param {{crossOrigin?: boolean, timeoutMs?: number, resize?: [number, number]|null, failure?: string}} [options] + */ +export function loadImage( + url, + { + crossOrigin = true, + timeoutMs = IMAGE_TIMEOUT_MS, + resize = null, + failure = msg('The photo could not be loaded'), + signal = null, + } = {}, +) { + if ( + crossOrigin && + url && + typeof globalThis.createImageBitmap === 'function' && + typeof globalThis.fetch === 'function' + ) + return loadBitmap(url, timeoutMs, resize, failure, signal); + return new Promise((resolve, reject) => { + if (!url) return reject(new Error(failure)); + if (signal?.aborted) return reject(new Error(failure)); + const image = new Image(); + if (crossOrigin) image.crossOrigin = 'anonymous'; + image.decoding = 'async'; + // Stop the download: another photo opened, or the time ran out. + const stop = () => { + clearTimeout(timer); + signal?.removeEventListener?.('abort', stop); + image.onload = image.onerror = null; + image.src = ''; + reject(new Error(failure)); + }; + const timer = setTimeout(stop, timeoutMs); + signal?.addEventListener?.('abort', stop, { once: true }); + const settle = () => { + clearTimeout(timer); + signal?.removeEventListener?.('abort', stop); + }; + image.onload = () => { + settle(); + // An reads no header first: refuse what it decoded too large. + if (tooManyPixels(image.naturalWidth || 0, image.naturalHeight || 0)) { + image.src = ''; + reject(oversized(failure)); + return; + } + resolve(image); + }; + image.onerror = () => { + settle(); + reject(new Error(failure)); + }; + image.src = url; + }); +} + +async function loadBitmap(url, timeoutMs, resize, failure, signal) { + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + // The caller's abort (the photo changed) stops a multi-megabyte download. + const onAbort = () => controller.abort(); + if (signal?.aborted) controller.abort(); + signal?.addEventListener?.('abort', onAbort, { once: true }); + try { + const response = await fetch(url, { + mode: 'cors', + signal: controller.signal, + }); + if (!response.ok) throw new Error(failure); + // Picture hosts are not ours: refuse a file or an image past any real + // photo's size before it is decoded. + const length = Number(response.headers?.get?.('content-length')); + if (length > MAX_IMAGE_BYTES) throw oversized(failure); + const blob = await boundedBlob(response, failure); + // Its declared size, before anything decodes it. + const size = imageSize( + new Uint8Array(await blob.slice(0, HEADER_BYTES).arrayBuffer()), + ); + if (size && tooManyPixels(size.width, size.height)) + throw oversized(failure); + const bitmap = await (resize + ? createImageBitmap(blob, { + resizeWidth: resize[0], + resizeHeight: resize[1], + resizeQuality: 'high', + }) + : createImageBitmap(blob)); + // (A resized decode reports its new size: this guards unresized ones.) + if (tooManyPixels(bitmap.width, bitmap.height)) { + bitmap.close?.(); + throw oversized(failure); + } + return bitmap; + } catch (error) { + // A size refusal stays one: the viewer must not fetch it another way. + if (error?.tooLarge) throw error; + throw new Error(failure); + } finally { + clearTimeout(timer); + signal?.removeEventListener?.('abort', onAbort); + } +} + +/** + * A picture refused for its size (`tooLarge`): the viewer will not load + * that file another way, an having no size limit at all. + */ +export function oversized(failure) { + return Object.assign(new Error(failure), { tooLarge: true }); +} + +/** + * The body as a Blob, read with a running count so a host that sends no + * length (or a false one) cannot fill memory before the cap is checked. + */ +async function boundedBlob(response, failure) { + const reader = response.body?.getReader?.(); + if (!reader) { + const blob = await response.blob(); + if (blob.size > MAX_IMAGE_BYTES) throw oversized(failure); + return blob; + } + const chunks = []; + let total = 0; + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + total += value.byteLength; + if (total > MAX_IMAGE_BYTES) { + reader.cancel().catch(() => {}); + throw oversized(failure); + } + chunks.push(value); + } + return new Blob(chunks, { + type: response.headers?.get?.('content-type') || 'image/jpeg', + }); +} + +/** The panorama size a canvas this wide can show sharply at this field of view. */ +export function panoSizeFor(canvasPixels, fovDeg) { + const needed = (Math.max(1, canvasPixels) * 360) / Math.max(1, fovDeg); + return needed > PANO_MID_SIZE[0] ? PANO_FULL_SIZE : PANO_MID_SIZE; +} + +/** + * @typedef {object} PhotoSources Where a provider's photo comes from. + * @property {(photo: object) => boolean} hasPano A 360° photo with an equirectangular image. + * @property {(photo: object) => {url: string, width: number, resize?: [number, number]|null}|null} panoFirst + * The panorama shown first, its texture width (0 when it is only a preview), and a size to shrink it to. + * @property {(photo: object, size: [number, number], view: {expanded: boolean}) => {url: string, resize?: [number, number]|null, timeoutMs?: number}|null} panoSharp + * A sharper panorama for a canvas that needs `size`, or null; `expanded` while the viewer is. + * @property {(photo: object, view: {expanded: boolean}) => Array} flatFirst Flat versions to try in turn (`expanded` while the viewer is). + * @property {(photo: object, pixels: number) => string|null} flatSharp + * A sharper flat version for a picture this many device pixels wide, or null. + * @property {(photo: object) => {title?: string|null, creator?: string|null, license?: string|null, externalUrl: string, capturedAtPrecision?: 'day'|'month'}} describe + * @property {(photo: object) => number|null} [bearingOf] Where the camera looked (default `photo.heading`). + */ + +/** + * Viewer adapter for providers whose photos are plain image files: a flat + * photo in an , a 360° photo on a WebGL panorama (drag to look, wheel + * or pinch to zoom), and ‹ › to step along the sequence (also ← →). Poses + * follow the view direction. + * + * Photos are normalized by the provider: `{id, sequenceId, lon, lat, + * heading, isPano, capturedAt}` plus whatever its `sources` read. + * + * @param {object} options + * @param {string} options.providerId + * @param {string} options.name Provider name for labels and messages, e.g. 'KartaView'. + * @param {PhotoSources} options.sources + * @param {(photoId: string) => Promise} options.getPhoto A normalized photo. + * @param {(photo: object, delta: -1|1) => Promise} options.step The photo `delta` along the sequence, or null at its end. + * @param {(imageId: string) => void} options.onStep Open the photo ‹ › chose (through the core, so loading and errors show). + * @param {(photo: object) => void} [options.onShown] A photo is on screen. + * @param {(photo: object) => void} [options.onShowing] The photo this open shows, just before its pose. + * @param {(imageId: string) => void} [options.onReopen] Draw the same photo again after a GPU reset (default: `onStep`). + * @param {() => boolean} [options.onEscape] Esc in the docked viewer: true when it was used (e.g. a selection cleared). + * @param {object} [options.render] Render governor. + * @param {(url: string, options?: object) => Promise} [options.load] + * @param {(canvas: HTMLCanvasElement) => object} [options.createRenderer] + * @returns {import('./registry.js').ViewerAdapter} + */ +export function createPhotoViewer({ + providerId, + name, + sources, + getPhoto, + step, + onStep, + onShown = () => {}, + onShowing = () => {}, + onReopen = onStep, + onEscape = () => false, + render, + load = loadImage, + createRenderer = createPanoramaRenderer, +}) { + const LOAD_FAILED = `The ${name} photo could not be loaded`; + const listeners = new Set(); + let root = null; + let picture = null; + let canvas = null; + let prevButton = null; + let nextButton = null; + let renderer = null; + /** WebGL failed once: 360° photos show their front view instead. */ + let rendererFailed = false; + let photo = null; + /** What shows: 'flat', 'pano' or null. */ + let showing = null; + let openSeq = 0; + let renderMode = 'letterbox'; + let drag = null; + let poseFrame = 0; + let removeListeners = []; + /** The document the viewer was built in (new canvases come from it). */ + let documentOf = null; + /** A context loss's look direction, for the reopen of that photo. */ + let restoreView = null; + /** The panel's viewer box, where Esc is listened for (null: the root). */ + let escapeTarget = null; + /** Width of the panorama texture on screen (0: the preview). */ + let panoWidth = 0; + /** The sharper panorama on its way, for which open (resizes come in bursts). */ + let panoComing = null; + /** A resize came while it loaded: look again once it lands. */ + let lookAgain = false; + /** The sharper flat version on screen, and the one on its way. */ + let flatSharp = { shown: null, loading: null }; + /** The open in progress: another open, a close or an unmount stops its downloads. */ + let openAbort = null; + /** The photo an open is fetching: ‹ › step on from it, not the old one. */ + let openingId = null; + /** ‹ › chose this photo: keep the panorama's look direction for it. */ + let keepViewFor = null; + /** ‹ › presses run one after another, each from where the last one went. */ + let steps = Promise.resolve(); + + /** Files refused for their size: never loaded again, by any path. */ + const refused = new Set(); + async function loadAs(url, options) { + if (refused.has(url)) throw oversized(LOAD_FAILED); + try { + return await load(url, { failure: LOAD_FAILED, ...options }); + } catch (error) { + if (error?.tooLarge) refused.add(url); + throw error; + } + } + + function requestRender() { + render?.governorRequestRender?.(`${providerId}-viewer`); + } + + function emit() { + if (!photo || !showing) return; + // A lost GPU context, still being recovered: the open that redraws the + // panorama poses once it is on screen. + if (showing === 'pano' && !renderer) return; + const view = showing === 'pano' ? renderer.getView() : null; + const heading = sources.bearingOf + ? sources.bearingOf(photo) + : photo.heading; + const described = sources.describe(photo); + const pose = { + providerId, + imageId: photo.id, + position: { lon: photo.lon, lat: photo.lat }, + bearing: + heading === null || heading === undefined + ? null + : view + ? wrapDegrees(heading + view.yaw) + : heading, + tilt: view ? view.pitch : 0, + altitude: null, + isPano: showing === 'pano', + capturedAt: photo.capturedAt, + capturedAtPrecision: described.capturedAtPrecision || 'day', + title: described.title ?? null, + creator: described.creator ?? null, + license: described.license ?? null, + sequenceId: photo.sequenceId, + externalUrl: described.externalUrl, + }; + for (const listener of [...listeners]) { + try { + listener(pose); + } catch { + /* listener errors are the core's to log */ + } + } + requestRender(); + } + + /** Re-emit on the next frame: a drag moves the view many times a frame. */ + function emitSoon() { + if (poseFrame) return; + poseFrame = requestAnimationFrame(() => { + poseFrame = 0; + emit(); + }); + } + + function syncButtons() { + const enabled = Boolean(photo?.sequenceId) && showing !== null; + if (prevButton) prevButton.disabled = !enabled; + if (nextButton) nextButton.disabled = !enabled; + if (enabled) checkEnds(photo); + } + + /** Disable ‹ or › at once where the sequence ends (it also warms the neighbours). */ + function checkEnds(target) { + for (const [delta, button] of [ + [-1, prevButton], + [1, nextButton], + ]) + step(target, delta).then( + (id) => { + if (photo === target && button && !id) disable(button); + }, + () => {}, + ); + } + + /** Disable a step button, keeping keyboard focus in the viewer. */ + function disable(button) { + if (button.ownerDocument?.activeElement === button) root?.focus?.(); + button.disabled = true; + } + + function show(kind) { + showing = kind; + if (picture) picture.hidden = kind !== 'flat'; + if (canvas) canvas.hidden = kind !== 'pano'; + if (root) root.dataset.kind = kind || ''; + syncButtons(); + } + + function applyRenderMode() { + if (picture) + picture.style.objectFit = renderMode === 'fill' ? 'cover' : 'contain'; + } + + function ensureRenderer() { + if (renderer || rendererFailed || !canvas) return renderer; + try { + renderer = createRenderer(canvas); + } catch { + rendererFailed = true; + renderer = null; + } + return renderer; + } + + /** + * A lost WebGL context (GPU reset, memory pressure) takes the panorama + * with it: drop the renderer, and show the photo again on a new one. + */ + function onContextLost(event) { + event.preventDefault?.(); + const lost = renderer; + renderer = null; + // The new renderer starts facing the photo's heading: keep the user's look. + if (showing === 'pano' && photo && lost) + restoreView = { id: photo.id, view: lost.getView() }; + try { + lost?.destroy(); + } catch { + /* already gone */ + } + // A lost canvas hands back its lost context: draw on a new element. + const fresh = makeCanvas(documentOf); + fresh.hidden = canvas.hidden; + canvas.replaceWith?.(fresh); + canvas = fresh; + // Show the photo again through the core, unless a newer one is opening. + if (showing === 'pano' && photo && openingId === null) { + const again = photo.id; + show(null); + openingId = again; + onReopen(again); + } + } + + /** The panorama canvas, wired for drag, pinch, wheel and context loss. */ + function makeCanvas(doc) { + const element = doc.createElement('canvas'); + element.className = 'sl-photo-pano'; + element.hidden = true; + listen(element, 'pointerdown', onPointerDown); + listen(element, 'pointermove', onPointerMove); + listen(element, 'pointerup', onPointerUp); + listen(element, 'pointercancel', onPointerUp); + listen(element, 'wheel', onWheel, { passive: false }); + listen(element, 'webglcontextlost', onContextLost); + return element; + } + + /** + * Step along the sequence from the photo showing, or from the one on its + * way after an earlier press, and open it through the core. + */ + function stepBy(delta) { + steps = steps.then(async () => { + const fromId = openingId ?? photo?.id; + if (!fromId || !root) return; + let from; + let nextId = null; + try { + from = await getPhoto(fromId); + if (from.sequenceId) nextId = await step(from, delta); + } catch { + // Not the end of the sequence, just no answer: the press is lost. + return; + } + // Closed, or another photo was opened meanwhile. + if ((openingId ?? photo?.id) !== fromId) return; + if (!nextId) { + const button = delta < 0 ? prevButton : nextButton; + if (button && photo?.id === fromId) disable(button); + return; + } + openingId = nextId; + keepViewFor = nextId; + onStep(nextId); + }); + } + + /** Pointers on the panorama: one drags the view, two pinch the zoom. */ + const pointers = new Map(); + let pinch = null; + + function pinchSpan() { + const [a, b] = [...pointers.values()]; + return Math.max(1, Math.hypot(a.x - b.x, a.y - b.y)); + } + + function onPointerDown(event) { + if (showing !== 'pano' || !renderer) return; + if (event.pointerType === 'mouse' && event.button !== 0) return; + pointers.set(event.pointerId, { x: event.clientX, y: event.clientY }); + canvas.setPointerCapture?.(event.pointerId); + if (pointers.size === 2) { + drag = null; + pinch = { span: pinchSpan(), fov: renderer.getView().fov }; + return; + } + drag = { + id: event.pointerId, + x: event.clientX, + y: event.clientY, + view: renderer.getView(), + }; + canvas.classList.add('sl-photo-dragging'); + } + + function onPointerMove(event) { + if (pointers.has(event.pointerId)) + pointers.set(event.pointerId, { x: event.clientX, y: event.clientY }); + if (pinch && pointers.size === 2 && renderer) { + renderer.setView({ fov: pinch.fov * (pinch.span / pinchSpan()) }); + return; + } + if (!drag || event.pointerId !== drag.id || !renderer) return; + const { fov } = drag.view; + const width = Math.max(1, canvas.clientWidth); + const height = Math.max(1, canvas.clientHeight); + const vertical = fov; + const horizontal = fov * (width / height); + renderer.setView({ + yaw: drag.view.yaw - ((event.clientX - drag.x) / width) * horizontal, + pitch: drag.view.pitch + ((event.clientY - drag.y) / height) * vertical, + }); + emitSoon(); + } + + function onPointerUp(event) { + pointers.delete(event.pointerId); + if (pointers.size < 2) pinch = null; + if (!drag || event.pointerId !== drag.id) return; + drag = null; + canvas.classList.remove('sl-photo-dragging'); + } + + function onWheel(event) { + if (showing !== 'pano' || !renderer) return; + event.preventDefault(); + const { fov } = renderer.getView(); + renderer.setView({ fov: fov * Math.exp(event.deltaY * 0.001) }); + } + + /** + * Docked, Esc anywhere in the viewer's box (the photo, its toolbar, its + * caption) is the provider's before the panel's, which would collapse; + * expanded, the dialog's own Esc shrinks it. + */ + function onEscapeKey(event) { + if ( + event.key !== 'Escape' || + event.shiftKey || + event.defaultPrevented || + root?.closest?.('.sl-viewer-wrap-expanded') || + !onEscape() + ) + return; + event.preventDefault(); + event.stopPropagation(); + } + + function onKeyDown(event) { + if (!escapeTarget) onEscapeKey(event); + if ( + event.defaultPrevented || + event.altKey || + event.ctrlKey || + event.metaKey + ) + return; + const pano = showing === 'pano' && renderer; + const turn = { ArrowLeft: -1, ArrowRight: 1 }[event.key]; + // Shift+← → look around a panorama; ← → step along the sequence. + if (turn && event.shiftKey && pano) { + event.preventDefault(); + const { yaw, fov } = renderer.getView(); + renderer.setView({ yaw: yaw + (turn * fov) / 4 }); + emitSoon(); + } else if (turn) { + event.preventDefault(); + stepBy(turn); + } else if ((event.key === 'ArrowUp' || event.key === 'ArrowDown') && pano) { + event.preventDefault(); + const { pitch, fov } = renderer.getView(); + renderer.setView({ + pitch: pitch + ((event.key === 'ArrowUp' ? 1 : -1) * fov) / 6, + }); + emitSoon(); + } else if (['+', '=', '-', '_'].includes(event.key) && pano) { + event.preventDefault(); + const { fov } = renderer.getView(); + renderer.setView({ + fov: fov * (event.key === '+' || event.key === '=' ? 0.85 : 1 / 0.85), + }); + } + } + + function listen(target, type, handler, options) { + target.addEventListener(type, handler, options); + removeListeners.push(() => + target.removeEventListener(type, handler, options), + ); + } + + function build(host) { + const doc = host.ownerDocument; + documentOf = doc; + root = doc.createElement('div'); + root.className = 'sl-photo-viewer'; + root.dataset.provider = providerId; + root.tabIndex = 0; + root.setAttribute('role', 'group'); + root.setAttribute( + 'aria-label', + t( + '{name} photo. ← and → step along the sequence. On a 360° photo, drag or use Shift+← →, ↑ ↓ to look around, and + − to zoom', + { name }, + ), + ); + picture = doc.createElement('img'); + picture.className = 'sl-photo-flat'; + picture.alt = t('{name} street-level photo', { name }); + picture.draggable = false; + picture.hidden = true; + canvas = makeCanvas(doc); + const nav = (label, glyph, delta) => { + const button = doc.createElement('button'); + button.type = 'button'; + button.className = `sl-photo-step sl-photo-step-${delta < 0 ? 'prev' : 'next'}`; + button.textContent = glyph; + button.title = label; + button.setAttribute('aria-label', label); + button.disabled = true; + listen(button, 'click', (event) => { + event.stopPropagation(); + stepBy(delta); + }); + return button; + }; + prevButton = nav(t('Previous photo in this sequence (←)'), '‹', -1); + nextButton = nav(t('Next photo in this sequence (→)'), '›', 1); + root.append(picture, canvas, prevButton, nextButton); + host.append(root); + listen(root, 'keydown', onKeyDown); + // The viewer's box in the panel; without one, the root's own handler + // takes Esc (one listener per element and type). + escapeTarget = host.closest?.('.sl-viewer-wrap') ?? null; + if (escapeTarget) listen(escapeTarget, 'keydown', onEscapeKey); + applyRenderMode(); + } + + function teardown() { + pointers.clear(); + pinch = null; + for (const remove of removeListeners) remove(); + removeListeners = []; + if (poseFrame) cancelAnimationFrame(poseFrame); + poseFrame = 0; + drag = null; + renderer?.destroy(); + renderer = null; + root?.remove(); + root = picture = canvas = prevButton = nextButton = null; + showing = null; + photo = null; + } + + /** Device pixels across the flat photo's frame (its longer side). */ + function viewerPixels() { + const ratio = globalThis.devicePixelRatio || 1; + const element = showing === 'pano' ? canvas : picture; + const side = Math.max( + element?.clientWidth || 0, + element?.clientHeight || 0, + ); + return (side || root?.clientWidth || 0) * ratio; + } + + /** A sharper version once the first is on screen, if nothing replaced it. */ + async function upgrade(target, seq, kind) { + // A 360° photo shown flat (no WebGL) keeps its front view. + if (kind === 'flat' && target.isPano) return; + // Another photo is opening: the one still shown is not worth more bytes. + if (openingId !== null && openingId !== target.id) return; + const signal = openAbort?.signal; + if (kind === 'pano') { + const size = panoSizeNow(); + if (size[0] <= panoWidth) return; + // Resizes come in bursts while the panel animates: one load at a time + // for this photo, and one more look once it lands. + if (panoComing?.seq === seq) { + lookAgain = true; + return; + } + const sharp = sources.panoSharp(target, size, { + expanded: isExpanded(), + }); + if (!sharp?.url) return; + const coming = { seq, width: size[0], controller: new AbortController() }; + panoComing = coming; + lookAgain = false; + try { + const image = await loadAs(sharp.url, { + crossOrigin: true, + signal: coming.controller.signal, + resize: sharp.resize ?? null, + ...(sharp.timeoutMs ? { timeoutMs: sharp.timeoutMs } : {}), + }); + if (seq !== openSeq || photo !== target || showing !== 'pano') { + image.close?.(); + return; + } + if (!renderer || size[0] <= panoWidth) { + image.close?.(); + return; + } + renderer.setImage(image); + panoWidth = size[0]; + requestRender(); + } catch { + /* the first version stays */ + } finally { + if (panoComing === coming) { + panoComing = null; + if (lookAgain && seq === openSeq && photo === target) { + lookAgain = false; + upgrade(target, seq, 'pano'); + } + } + } + return; + } + const url = sources.flatSharp(target, viewerPixels()); + if (!url || url === flatSharp.shown || url === flatSharp.loading) return; + const sharp = flatSharp; + sharp.loading = url; + try { + const image = await loadAs(url, { crossOrigin: false, signal }); + if (seq !== openSeq || photo !== target || showing !== 'flat') return; + picture.src = image.src; + sharp.shown = url; + requestRender(); + } catch { + /* the first version stays; a later resize may try again */ + } finally { + // Discarded or failed: nothing is on its way, so it may be asked again. + if (sharp.loading === url) sharp.loading = null; + } + } + + /** The viewer is the expanded dialog, not docked in the panel. */ + function isExpanded() { + return Boolean(root?.closest?.('.sl-viewer-wrap-expanded')); + } + + /** Another photo, or none: a sharper panorama still on its way is not needed. */ + function dropSharpPanorama() { + panoComing?.controller?.abort(); + panoComing = null; + lookAgain = false; + } + + /** The sharp panorama this viewer needs now: 2,048 docked, 4,096 expanded. */ + function panoSizeNow() { + const ratio = Math.min(globalThis.devicePixelRatio || 1, 2); + const width = (canvas?.clientWidth || 0) * ratio; + return panoSizeFor(width, renderer?.getView().fov ?? PANO_FOV_DEFAULT_DEG); + } + + async function showPano(target, seq, keepView, signal) { + // If the panorama fails, the caller shows the front view rather than + // waiting on it twice. + const first = sources.panoFirst(target); + if (!first?.url) throw new Error(LOAD_FAILED); + const image = await loadAs(first.url, { + crossOrigin: true, + signal, + // Never decoded at more than the texture it becomes. + resize: first.resize ?? PANO_MID_SIZE, + }); + if (seq !== openSeq) { + image.close?.(); + return false; + } + // The context may have been lost while this loaded: build on the new + // canvas, or let the caller show the front view. + if (!ensureRenderer()) { + image.close?.(); + throw new Error(LOAD_FAILED); + } + const restore = restoreView?.id === target.id ? restoreView.view : null; + restoreView = null; + if (restore) renderer.setView(restore); + else if (!keepView || showing !== 'pano') + renderer.setView({ yaw: 0, pitch: 0, fov: PANO_FOV_DEFAULT_DEG }); + photo = target; + // Shown first: a hidden canvas measures 0 and would be sized 1×1. + show('pano'); + renderer.setImage(image); + panoWidth = first.width || 0; + // A sharper version of the photo before was dropped by `open`; one of + // this photo, asked by a resize meanwhile, is still worth having. + return true; + } + + async function showFlat(target, seq, signal) { + const urls = sources + .flatFirst(target, { expanded: isExpanded() }) + .filter(Boolean); + let image; + let failure = null; + for (const url of urls) { + try { + image = await loadAs(url, { crossOrigin: false, signal }); + break; + } catch (error) { + failure = error; + if (seq !== openSeq) return false; + } + } + if (!image) throw failure || new Error(LOAD_FAILED); + if (seq !== openSeq) return false; + picture.src = image.src; + flatSharp = { shown: null, loading: null }; + photo = target; + show('flat'); + return true; + } + + async function openPhoto(id, seq, keepView, signal) { + const target = await getPhoto(id, { signal }); + if (seq !== openSeq) return; + let shown = false; + if (sources.hasPano(target) && ensureRenderer()) { + try { + shown = await showPano(target, seq, keepView, signal); + } catch { + // The panorama failed: its front view is better than nothing. + if (seq !== openSeq) return; + shown = await showFlat(target, seq, signal); + } + } else shown = await showFlat(target, seq, signal); + if (!shown) return; + onShowing(target); + emit(); + onShown(target); + upgrade(target, seq, showing); + } + + const adapter = { + async mount(host) { + if (!host) throw new Error('Street Level viewer has no host element'); + if (root && root.parentNode === host) return; + teardown(); + rendererFailed = false; + build(host); + }, + + /** Resolves once the photo is on screen and its pose was emitted. */ + async open(imageId) { + if (!root) throw new Error(`${name} viewer is not mounted`); + const id = String(imageId); + const seq = ++openSeq; + dropSharpPanorama(); + // The previous open's downloads stop: only this photo is wanted now. + openAbort?.abort(); + const opening = (openAbort = new AbortController()); + const keepView = keepViewFor === id; + keepViewFor = null; + openingId = id; + try { + await openPhoto(id, seq, keepView, opening.signal); + } catch (error) { + // A later open, or a close, stopped this one: that is not a failure. + if (seq !== openSeq) return; + throw error; + } finally { + if (openingId === id && seq === openSeq) openingId = null; + } + }, + + close() { + openSeq++; + dropSharpPanorama(); + openAbort?.abort(); + openAbort = null; + openingId = null; + keepViewFor = null; + photo = null; + flatSharp = { shown: null, loading: null }; + show(null); + if (picture) picture.removeAttribute('src'); + renderer?.clear?.(); + }, + + unmount() { + openSeq++; + dropSharpPanorama(); + openAbort?.abort(); + openAbort = null; + refused.clear(); + openingId = null; + keepViewFor = null; + flatSharp = { shown: null, loading: null }; + teardown(); + }, + + resize() { + renderer?.resize(); + // Expanded: a docked viewer's panorama or flat version is now too soft. + if (showing === 'pano' && photo && panoSizeNow()[0] > panoWidth) + upgrade(photo, openSeq, 'pano'); + else if (showing === 'flat' && photo) upgrade(photo, openSeq, 'flat'); + }, + + /** FIT / FILL frame a flat photo; a panorama has no frame to fit. */ + supportsRenderModes: () => showing !== 'pano', + + setRenderMode(mode) { + renderMode = mode === 'fill' ? 'fill' : 'letterbox'; + applyRenderMode(); + }, + + /** The photo's direction became known (its drive loaded): pose again. */ + refreshPose() { + emit(); + }, + + onPose(listener) { + listeners.add(listener); + return () => listeners.delete(listener); + }, + }; + return adapter; +} diff --git a/src/layers/streetLevel/photoViewer.test.mjs b/src/layers/streetLevel/photoViewer.test.mjs new file mode 100644 index 000000000..09df91c2e --- /dev/null +++ b/src/layers/streetLevel/photoViewer.test.mjs @@ -0,0 +1,538 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createPhotoViewer } from './photoViewer.js'; +import { fakeHost, until } from '../../testSupport/kartaViewFakes.mjs'; + +const PHOTOS = { + a: { id: 'a', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: true }, + b: { id: 'b', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: true }, + f: { id: 'f', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: false }, + g: { id: 'g', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: false }, +}; + +/** Sources whose panoramas and sharper flat versions are named after the photo. */ +const SOURCES = { + hasPano: (photo) => photo.isPano, + panoFirst: (photo) => ({ url: `${photo.id}-first`, width: 2048 }), + panoSharp: (photo) => ({ url: `${photo.id}-sharp` }), + flatFirst: (photo) => [`${photo.id}-first`], + flatSharp: (photo, pixels) => (pixels > 1000 ? `${photo.id}-sharp` : null), + describe: () => ({ externalUrl: 'https://example.org' }), +}; + +/** A viewer whose loads wait until `release(url)` (or fail with `fail(url)`). */ +function setup() { + const pending = new Map(); + const loads = []; + const images = []; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: (url) => + new Promise((resolve, reject) => { + loads.push(url); + pending.set(url, { resolve, reject }); + }), + createRenderer: () => ({ + setImage: (image) => images.push(image.src), + setView() {}, + getView: () => ({ yaw: 0, pitch: 0, fov: 80 }), + resize() {}, + clear() {}, + destroy() {}, + }), + }); + const release = (url) => { + const closed = { value: false }; + pending.get(url).resolve({ src: url, close: () => (closed.value = true) }); + pending.delete(url); + return closed; + }; + const fail = (url) => { + pending.get(url).reject(new Error('gone')); + pending.delete(url); + }; + return { viewer, loads, images, release, fail, host: fakeHost() }; +} + +const tick = () => new Promise((resolve) => setTimeout(resolve, 0)); +const parts = (host) => { + const [picture, canvas] = host.children[0].children; + return { picture, canvas }; +}; + +test('a panorama that arrives after another photo opened is released, not shown', async () => { + const env = setup(); + await env.viewer.mount(env.host); + const first = env.viewer.open('a'); + await tick(); + const second = env.viewer.open('b'); + await tick(); + const stale = env.release('a-first'); + await first; + assert.equal(stale.value, true, 'its bitmap is closed'); + env.release('b-first'); + await second; + assert.deepEqual(env.images, ['b-first']); +}); + +test('one sharp panorama download per open, even across a quick switch and a burst of resizes', async (t) => { + const env = setup(); + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + await env.viewer.mount(env.host); + const { canvas } = parts(env.host); + canvas.clientWidth = 1800; + const opening = env.viewer.open('a'); + await tick(); + env.release('a-first'); + await opening; + await tick(); + assert.ok(env.loads.includes('a-sharp'), 'expanded: the sharp one is asked'); + // Photo b opens while a's sharp version is still on its way. + const next = env.viewer.open('b'); + await tick(); + env.release('b-first'); + await next; + await tick(); + assert.ok(env.loads.includes('b-sharp')); + // a's download finishes late; resizes must not ask for b's again. + env.release('a-sharp'); + await tick(); + env.viewer.resize(); + env.viewer.resize(); + await tick(); + assert.equal(env.loads.filter((url) => url === 'b-sharp').length, 1); + env.release('b-sharp'); + await tick(); + assert.deepEqual(env.images, ['a-first', 'b-first', 'b-sharp']); +}); + +test('a sharper flat version that failed is tried again on the next resize', async (t) => { + const env = setup(); + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + await env.viewer.mount(env.host); + const { picture } = parts(env.host); + picture.clientWidth = 1200; + const opening = env.viewer.open('f'); + await tick(); + env.release('f-first'); + await opening; + await tick(); + env.fail('f-sharp'); + await tick(); + assert.equal(picture.src, 'f-first'); + env.viewer.resize(); + await tick(); + assert.equal(env.loads.filter((url) => url === 'f-sharp').length, 2); + env.release('f-sharp'); + assert.ok(await until(() => picture.src === 'f-sharp')); +}); + +test('Esc: the docked viewer offers it to the provider first; expanded, it is the dialog’s', async () => { + let asked = 0; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + onEscape: () => ++asked > 0, + load: async (url) => ({ src: url }), + }); + const host = fakeHost(); + await viewer.mount(host); + const root = host.children[0]; + const press = () => { + const event = { key: 'Escape', prevented: false, stopped: false }; + event.preventDefault = () => (event.prevented = true); + event.stopPropagation = () => (event.stopped = true); + root.fire('keydown', event); + return event; + }; + const docked = press(); + assert.equal(asked, 1); + assert.equal(docked.prevented && docked.stopped, true); + root.closest = (selector) => + selector.includes('sl-viewer-wrap-expanded') ? {} : null; + const expanded = press(); + assert.equal(asked, 1, 'not asked while expanded'); + assert.equal(expanded.prevented, false); +}); + +test('a sharp panorama still downloading when another photo opens is aborted', async (t) => { + const signals = []; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: (url, options) => + url.endsWith('-sharp') + ? new Promise((_, reject) => { + signals.push(options.signal); + options.signal?.addEventListener('abort', () => + reject(new Error('aborted')), + ); + }) + : Promise.resolve({ src: url }), + createRenderer: () => ({ + setImage() {}, + setView() {}, + getView: () => ({ yaw: 0, pitch: 0, fov: 80 }), + resize() {}, + clear() {}, + destroy() {}, + }), + }); + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + const host = fakeHost(); + await viewer.mount(host); + parts(host).canvas.clientWidth = 1800; + await viewer.open('a'); + await tick(); + assert.equal(signals.length, 1); + assert.equal(signals[0].aborted, false); + await viewer.open('f'); + assert.equal(signals[0].aborted, true, 'not needed any more'); +}); + +test('a lost WebGL context while the first panorama loads releases its bitmap', async () => { + let closed = false; + let built = 0; + let land = null; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: (url) => + new Promise((resolve) => { + land = () => resolve({ src: url, close: () => (closed = true) }); + }), + // The first renderer is lost mid-load; a new one cannot be built. + createRenderer: () => { + built++; + if (built > 1) throw new Error('context lost'); + return { + setImage() {}, + setView() {}, + getView: () => ({ yaw: 0, pitch: 0, fov: 80 }), + resize() {}, + clear() {}, + destroy() {}, + }; + }, + }); + const host = fakeHost(); + await viewer.mount(host); + const opening = viewer.open('a'); + assert.ok(await until(() => land !== null), 'the panorama is loading'); + parts(host).canvas.fire('webglcontextlost', {}); + const first = land; + first(); + // The panorama failed: the front view is loaded flat instead. + assert.ok(await until(() => land !== first), 'the flat fallback loads'); + land(); + await opening; + assert.equal(built, 2, 'a new renderer was tried'); + assert.equal(closed, true, 'the bitmap no renderer took is closed'); +}); + +test('docked, Esc anywhere in the viewer’s box (toolbar, caption) is offered to the provider', async () => { + let asked = 0; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + onEscape: () => ++asked > 0, + load: async (url) => ({ src: url }), + }); + const host = fakeHost(); + const wrap = fakeHost(); + host.closest = (selector) => (selector === '.sl-viewer-wrap' ? wrap : null); + await viewer.mount(host); + const flags = { stopped: false, prevented: false }; + // Focus back on EXPAND after a shrink: the key reaches the wrap, not the photo. + wrap.fire('keydown', { + key: 'Escape', + stopPropagation: () => (flags.stopped = true), + preventDefault: () => (flags.prevented = true), + }); + assert.equal(asked, 1); + assert.deepEqual(flags, { stopped: true, prevented: true }); + viewer.unmount(); + wrap.fire('keydown', { key: 'Escape' }); + assert.equal(asked, 1, 'unmounted: the listener is gone'); +}); + +test('a picture file or bitmap past any real photo’s size is refused', async (t) => { + const { loadImage, MAX_IMAGE_BYTES, MAX_IMAGE_SIDE } = + await import('./photoViewer.js'); + const saved = { + fetch: globalThis.fetch, + createImageBitmap: globalThis.createImageBitmap, + }; + t.after(() => Object.assign(globalThis, saved)); + let closed = 0; + const bitmap = (width, height) => ({ width, height, close: () => closed++ }); + globalThis.createImageBitmap = async (blob) => blob.bitmap; + const respond = ({ length, size, width = 100, height = 50 }) => + (globalThis.fetch = async () => ({ + ok: true, + headers: new Map([['content-length', String(length ?? '')]]), + blob: async () => ({ + size: size ?? 10, + bitmap: bitmap(width, height), + slice: () => ({ arrayBuffer: async () => new ArrayBuffer(0) }), + }), + })); + const refusedForSize = (error) => { + assert.match(error.message, /could not be loaded/); + assert.equal(error.tooLarge, true, 'marked as a size refusal'); + return true; + }; + respond({ length: MAX_IMAGE_BYTES + 1 }); + await assert.rejects(loadImage('https://x/a.jpg'), refusedForSize); + respond({ size: MAX_IMAGE_BYTES + 1 }); + await assert.rejects(loadImage('https://x/a.jpg'), refusedForSize); + respond({ width: MAX_IMAGE_SIDE + 1 }); + await assert.rejects(loadImage('https://x/a.jpg'), refusedForSize); + assert.equal(closed, 1, 'the oversize bitmap is released'); + // No length given, a body that keeps coming: cut off while it streams. + let pulled = 0; + globalThis.fetch = async () => ({ + ok: true, + headers: new Map(), + body: new ReadableStream({ + pull(controller) { + pulled++; + controller.enqueue(new Uint8Array(8 * 1024 * 1024)); + }, + }), + }); + await assert.rejects(loadImage('https://x/a.jpg'), refusedForSize); + assert.ok(pulled * 8 * 1024 * 1024 <= MAX_IMAGE_BYTES + 16 * 1024 * 1024); + // Other failures are not size refusals. + globalThis.fetch = async () => ({ ok: false, headers: new Map() }); + await assert.rejects( + loadImage('https://x/a.jpg'), + (error) => !error.tooLarge, + ); + respond({}); + assert.equal((await loadImage('https://x/a.jpg')).width, 100); +}); + +/** A viewer whose loads are held, recording each load's signal by url. */ +function heldViewer() { + const pending = new Map(); + const signals = new Map(); + const loads = []; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: (url, options = {}) => + new Promise((resolve, reject) => { + loads.push(url); + signals.set(url, options.signal); + options.signal?.addEventListener('abort', () => + reject(new Error('aborted')), + ); + pending.set(url, { resolve, reject }); + }), + }); + const release = (url) => { + pending.get(url).resolve({ src: url }); + pending.delete(url); + }; + return { viewer, pending, signals, loads, release, host: fakeHost() }; +} + +test('a new open, a close or an unmount stops the downloads of the open before it', async () => { + const env = heldViewer(); + await env.viewer.mount(env.host); + const first = env.viewer.open('f'); + assert.ok(await until(() => env.signals.has('f-first'))); + const second = env.viewer.open('g'); + assert.equal(env.signals.get('f-first').aborted, true, 'f is not wanted now'); + await first.catch(() => {}); + assert.ok(await until(() => env.signals.has('g-first'))); + env.viewer.close(); + assert.equal(env.signals.get('g-first').aborted, true, 'closed'); + await second.catch(() => {}); + const third = env.viewer.open('f'); + assert.ok(await until(() => env.pending.has('f-first'))); + env.viewer.unmount(); + assert.equal(env.signals.get('f-first').aborted, true, 'unmounted'); + await third.catch(() => {}); +}); + +test('a sharper flat version on its way stops when another photo opens, and is asked again later', async (t) => { + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + const env = heldViewer(); + await env.viewer.mount(env.host); + parts(env.host).picture.clientWidth = 1200; + const opening = env.viewer.open('f'); + assert.ok(await until(() => env.pending.has('f-first'))); + env.release('f-first'); + await opening; + assert.ok( + await until(() => env.pending.has('f-sharp')), + 'the 1,200 px viewer asks for more', + ); + // Resizes while it loads (the panel animating) do not ask again. + env.viewer.resize(); + env.viewer.resize(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(env.loads.filter((url) => url === 'f-sharp').length, 1); + // g opens (and fails): f's sharp download stops, f stays on screen. + const failing = env.viewer.open('g'); + assert.equal(env.signals.get('f-sharp').aborted, true); + assert.ok(await until(() => env.pending.has('g-first'))); + env.pending.get('g-first').reject(new Error('gone')); + env.pending.delete('g-first'); + await failing.catch(() => {}); + // The discarded response does not block f's sharp version for good. + env.signals.delete('f-sharp'); + env.viewer.resize(); + assert.ok(await until(() => env.signals.has('f-sharp')), 'asked again'); +}); + +test('a resize while another photo opens does not upgrade the photo still shown', async (t) => { + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + const env = heldViewer(); + await env.viewer.mount(env.host); + const opening = env.viewer.open('f'); + assert.ok(await until(() => env.pending.has('f-first'))); + env.release('f-first'); + await opening; + // g is opening; EXPAND resizes the viewer meanwhile. + const next = env.viewer.open('g'); + assert.ok(await until(() => env.pending.has('g-first'))); + parts(env.host).picture.clientWidth = 1200; + env.viewer.resize(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(env.pending.has('f-sharp'), false, 'f is on its way out'); + env.release('g-first'); + await next; + assert.ok(await until(() => env.pending.has('g-sharp')), 'g gets it'); +}); + +test('a panorama is always decoded at a bounded size', async () => { + const env = heldViewer(); + const options = []; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: async (url, opts) => { + options.push(opts); + return { src: url }; + }, + createRenderer: () => ({ + setImage() {}, + setView() {}, + getView: () => ({ yaw: 0, pitch: 0, fov: 80 }), + resize() {}, + clear() {}, + destroy() {}, + }), + }); + await viewer.mount(env.host); + await viewer.open('a'); + // SOURCES' panoFirst names no size: the viewer still bounds it. + assert.deepEqual(options[0].resize, [2048, 1024]); +}); + +test('a header that declares a huge image is refused before anything decodes it', async (t) => { + const { loadImage } = await import('./photoViewer.js'); + const saved = { + fetch: globalThis.fetch, + createImageBitmap: globalThis.createImageBitmap, + }; + t.after(() => Object.assign(globalThis, saved)); + let decoded = 0; + globalThis.createImageBitmap = async () => { + decoded++; + return { width: 10, height: 10 }; + }; + // A 1 kB PNG that says it is 16,384 × 16,384. + const png = new Uint8Array(1024); + png.set([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13]); + png.set( + [...'IHDR'].map((c) => c.charCodeAt(0)), + 12, + ); + png.set([0, 0, 0x40, 0, 0, 0, 0x40, 0], 16); + globalThis.fetch = async () => new Response(png); + await assert.rejects( + loadImage('https://x/bomb.png'), + (error) => error.tooLarge === true, + ); + assert.equal(decoded, 0, 'never decoded'); +}); + +test('an that turns out too large is refused and let go', async (t) => { + const { loadImage, MAX_IMAGE_SIDE } = await import('./photoViewer.js'); + const saved = globalThis.Image; + t.after(() => (globalThis.Image = saved)); + let size = [MAX_IMAGE_SIDE + 1, 100]; + const images = []; + globalThis.Image = class { + constructor() { + images.push(this); + } + set src(value) { + this._src = value; + if (value) + queueMicrotask(() => { + [this.naturalWidth, this.naturalHeight] = size; + this.onload?.(); + }); + } + get src() { + return this._src; + } + }; + await assert.rejects( + loadImage('https://x/a.jpg', { crossOrigin: false }), + (error) => error.tooLarge === true, + ); + assert.equal(images.at(-1).src, '', 'its pixels let go'); + size = [4000, 3000]; + const ok = await loadImage('https://x/b.jpg', { crossOrigin: false }); + assert.equal(ok.naturalWidth, 4000); + // An aborted load stops and rejects. + size = [10, 10]; + const controller = new AbortController(); + const aborted = loadImage('https://x/c.jpg', { + crossOrigin: false, + signal: controller.signal, + }); + controller.abort(); + await assert.rejects(aborted); + assert.equal(images.at(-1).src, '', 'its download stops'); +}); diff --git a/src/layers/streetLevel/policy.js b/src/layers/streetLevel/policy.js index 58a962e4c..a5e95d7d8 100644 --- a/src/layers/streetLevel/policy.js +++ b/src/layers/streetLevel/policy.js @@ -13,6 +13,7 @@ export const PROVIDER_COLORS = Object.freeze({ mapillary: '#05cb63', google: '#4285f4', kartaview: '#635bff', + panoramax: '#ff6f00', }); /** Colours every provider shares: the selection highlight and the marker. */ diff --git a/src/layers/streetLevel/providers/kartaview/index.js b/src/layers/streetLevel/providers/kartaview/index.js index 4f339c421..1cf2513f3 100644 --- a/src/layers/streetLevel/providers/kartaview/index.js +++ b/src/layers/streetLevel/providers/kartaview/index.js @@ -1,4 +1,7 @@ import { createLineCoverage } from '../../lineCoverage.js'; +import { createLru, createTileCache } from '../../tileCache.js'; + +export { createTileCache } from '../../tileCache.js'; import { createSequenceCones, thinImages } from '../../sequenceCones.js'; import { metresBetween } from '../../view.js'; import { @@ -42,9 +45,6 @@ import { const PHOTO_CACHE_SIZE = 4000; /** Sequence pages kept in memory. */ const PAGE_CACHE_SIZE = 12; -/** Decoded coverage tiles kept, and for how long (masks too). */ -const TILE_CACHE_SIZE = 24; -const TILE_CACHE_TTL_MS = 10 * 60_000; /** z14 near the street, z13 a little higher, nothing above. */ export function kartaViewZoomFor(heightM) { @@ -55,54 +55,6 @@ export function kartaViewZoomFor(heightM) { }; } -/** Least-recently-used map: reading an entry renews it. */ -function createLru(limit) { - const map = new Map(); - return { - get(key) { - if (!map.has(key)) return undefined; - const value = map.get(key); - map.delete(key); - map.set(key, value); - return value; - }, - set(key, value) { - map.delete(key); - map.set(key, value); - while (map.size > limit) map.delete(map.keys().next().value); - }, - delete: (key) => map.delete(key), - clear: () => map.clear(), - }; -} - -/** - * Loaded coverage tiles by `z/x/y`, at most `size`, each for `ttlMs`. KartaView - * sends tiles uncompressed and uncacheable (hundreds of kB, seconds each), so - * a pan back or a zoom across z13/z14 would fetch them all again. - */ -export function createTileCache({ - load, - size = TILE_CACHE_SIZE, - ttlMs = TILE_CACHE_TTL_MS, - now = Date.now, - keyOf = (tile) => `${tile.z}/${tile.x}/${tile.y}`, -}) { - const kept = createLru(size); - return { - async get(tile, options) { - const key = keyOf(tile); - const entry = kept.get(key); - if (entry && now() - entry.at < ttlMs) return entry.tile; - const loaded = await load(tile, options); - kept.set(key, { at: now(), tile: loaded }); - return loaded; - }, - /** Drop every tile (the provider was switched off). */ - clear: () => kept.clear(), - }; -} - /** The 1-based page of a sequence that holds `index`, if no photo is missing. */ function pageOf(index) { return Math.floor(Math.max(0, index) / SEQUENCE_PAGE_SIZE) + 1; @@ -497,6 +449,13 @@ export function createKartaViewProvider({ msg('The KartaView photo could not be loaded'), ), ), + // Esc in the viewer clears the selected drive, as the panel says. + onEscape: () => { + if (!state.sequence.selectedId) return false; + cones.clearSelection(); + drawnPage = null; + return true; + }, render: context.services?.render, ...viewerOptions, }); diff --git a/src/layers/streetLevel/providers/kartaview/policy.js b/src/layers/streetLevel/providers/kartaview/policy.js index a0946a731..137b9cc78 100644 --- a/src/layers/streetLevel/providers/kartaview/policy.js +++ b/src/layers/streetLevel/providers/kartaview/policy.js @@ -103,9 +103,6 @@ export const SEQUENCE_PAGE_EDGE = 12; /** Equirectangular panorama sizes: a quick preview, then the sharp one. */ export const PANO_PREVIEW_SIZE = Object.freeze([1024, 512]); -/** Enough for a docked viewer; an expanded one gets the full size. */ -export const PANO_MID_SIZE = Object.freeze([2048, 1024]); -export const PANO_FULL_SIZE = Object.freeze([4096, 2048]); /** A flat photo's first version, then its sharp one. */ export const FLAT_VIEW_SIZE = Object.freeze([1280, 1280]); export const FLAT_FULL_SIZE = Object.freeze([2560, 2560]); diff --git a/src/layers/streetLevel/providers/kartaview/provider.test.mjs b/src/layers/streetLevel/providers/kartaview/provider.test.mjs index 706988904..a043063fa 100644 --- a/src/layers/streetLevel/providers/kartaview/provider.test.mjs +++ b/src/layers/streetLevel/providers/kartaview/provider.test.mjs @@ -896,6 +896,27 @@ test('a photo no page lists steps to the photos either side of its index', async assert.equal(env.poses.at(-1), 's-4'); }); +test('Esc in the docked viewer clears the selected drive and is consumed; with none, it goes on to the panel', async (t) => { + const env = await mounted({ sequences: { s: [0, 1, 2, 3] } }); + await withCones(t, env); + await env.instance.viewer.open('s-1'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => env.instance.sequenceStats().images === 4)); + const root = env.next.parentNode; + const press = () => { + const flags = { stopped: false, prevented: false }; + root.fire('keydown', { + key: 'Escape', + stopPropagation: () => (flags.stopped = true), + preventDefault: () => (flags.prevented = true), + }); + return flags; + }; + assert.deepEqual(press(), { stopped: true, prevented: true }); + assert.equal(env.instance.sequenceStats().selectedId, null, 'drive cleared'); + assert.deepEqual(press(), { stopped: false, prevented: false }); +}); + test('the panel counts the drive’s photos, not the thinned page of them the cones show', async (t) => { const env = await mounted({ sequences: { s: Array.from({ length: 40 }, (_, i) => i) }, diff --git a/src/layers/streetLevel/providers/kartaview/source.js b/src/layers/streetLevel/providers/kartaview/source.js index dada2ba65..89ed9409e 100644 --- a/src/layers/streetLevel/providers/kartaview/source.js +++ b/src/layers/streetLevel/providers/kartaview/source.js @@ -22,6 +22,9 @@ import { photoFilterParams, } from './policy.js'; import { msg } from '../../../../i18n/index.js'; +import { createLimiter } from '../../limiter.js'; + +export { createLimiter } from '../../limiter.js'; /** * An HTTP or API failure; `retryAfterSec` is set while KartaView rate-limits, @@ -72,56 +75,6 @@ function retryAfterSeconds(response) { return Number.isFinite(value) && value >= 0 ? value : null; } -/** - * A gate letting at most `limit` tasks run at once, first in first out, or - * newest first with `lifo` (the last request is the one the user waits on). - */ -export function createLimiter(limit, { lifo = false } = {}) { - let running = 0; - const queue = []; - function next() { - while (running < limit && queue.length) { - const { task, resolve, reject, signal } = lifo - ? queue.pop() - : queue.shift(); - if (signal?.aborted) { - reject(signal.reason); - continue; - } - running++; - Promise.resolve() - .then(task) - .then(resolve, reject) - .finally(() => { - running--; - next(); - }); - } - } - return { - run(task, { signal } = {}) { - return new Promise((resolve, reject) => { - const item = { task, resolve, reject, signal }; - queue.push(item); - signal?.addEventListener( - 'abort', - () => { - const at = queue.indexOf(item); - if (at === -1) return; - queue.splice(at, 1); - reject(signal.reason); - }, - { once: true }, - ); - next(); - }); - }, - get pending() { - return running + queue.length; - }, - }; -} - /** * KartaView's public API, browser-side and keyless (it allows any origin). * It speaks HTTP/1.1, so the browser opens six connections to it: tiles take diff --git a/src/layers/streetLevel/providers/kartaview/viewer.js b/src/layers/streetLevel/providers/kartaview/viewer.js index 92be74757..e9c5c0c0d 100644 --- a/src/layers/streetLevel/providers/kartaview/viewer.js +++ b/src/layers/streetLevel/providers/kartaview/viewer.js @@ -1,702 +1,93 @@ import { - createPanoramaRenderer, - PANO_FOV_DEFAULT_DEG, - wrapDegrees, -} from './panorama.js'; -import { msg, t } from '../../../../i18n/index.js'; + createPhotoViewer, + loadImage as loadAnyImage, +} from '../../photoViewer.js'; import { FLAT_FULL_SIZE, FLAT_VIEW_SIZE, IMAGE_TIMEOUT_MS, + KARTAVIEW_NAME, KARTAVIEW_PROVIDER_ID, - PANO_FULL_SIZE, - PANO_MID_SIZE, PANO_PREVIEW_SIZE, cameraBearing, kartaViewPhotoUrl, resizedImageUrl, } from './policy.js'; +import { msg } from '../../../../i18n/index.js'; + +export { panoSizeFor } from '../../photoViewer.js'; + /** English data: the Street Level panel translates it (translateKnown). */ const LOAD_FAILED = msg('The KartaView photo could not be loaded'); -/** - * Load an image within the timeout. A CORS-clean one (for WebGL) is decoded - * off the main thread into an ImageBitmap where the browser can: uploading - * a decoded bitmap costs a fraction of uploading an (90 ms → 15 ms for - * 4096×2048 on a phone-class CPU). - */ -export function loadImage( - url, - { crossOrigin = true, timeoutMs = IMAGE_TIMEOUT_MS } = {}, -) { - if ( - crossOrigin && - url && - typeof globalThis.createImageBitmap === 'function' && - typeof globalThis.fetch === 'function' - ) - return loadBitmap(url, timeoutMs); - return new Promise((resolve, reject) => { - if (!url) return reject(new Error(LOAD_FAILED)); - const image = new Image(); - if (crossOrigin) image.crossOrigin = 'anonymous'; - image.decoding = 'async'; - const timer = setTimeout(() => { - image.src = ''; - reject(new Error(LOAD_FAILED)); - }, timeoutMs); - image.onload = () => { - clearTimeout(timer); - resolve(image); - }; - image.onerror = () => { - clearTimeout(timer); - reject(new Error(LOAD_FAILED)); - }; - image.src = url; +/** Load a KartaView photo (see photoViewer.js `loadImage`). */ +export function loadImage(url, options = {}) { + return loadAnyImage(url, { + timeoutMs: IMAGE_TIMEOUT_MS, + failure: LOAD_FAILED, + ...options, }); } -async function loadBitmap(url, timeoutMs) { - const controller = new AbortController(); - const timer = setTimeout(() => controller.abort(), timeoutMs); - try { - const response = await fetch(url, { - mode: 'cors', - signal: controller.signal, - }); - if (!response.ok) throw new Error(LOAD_FAILED); - return await createImageBitmap(await response.blob()); - } catch { - throw new Error(LOAD_FAILED); - } finally { - clearTimeout(timer); - } -} - -/** Free a decoded bitmap nobody will draw (an has nothing to free). */ -function release(image) { - image?.close?.(); -} - -/** The panorama size a canvas this wide can show sharply at this field of view. */ -export function panoSizeFor(canvasPixels, fovDeg) { - const needed = (Math.max(1, canvasPixels) * 360) / Math.max(1, fovDeg); - return needed > PANO_MID_SIZE[0] ? PANO_FULL_SIZE : PANO_MID_SIZE; -} +/** + * Where KartaView photos come from. Only KartaView's image proxy can shrink + * a 13,000 px panorama, so panoramas and flat upgrades go through it. + * @type {import('../../photoViewer.js').PhotoSources} + */ +const KARTAVIEW_SOURCES = Object.freeze({ + hasPano: (photo) => Boolean(photo.isPano && photo.urls.pano), + panoFirst: (photo) => ({ + url: resizedImageUrl(photo.urls.pano, PANO_PREVIEW_SIZE), + width: 0, + resize: PANO_PREVIEW_SIZE, + }), + panoSharp: (photo, size) => ({ + url: resizedImageUrl(photo.urls.pano, size), + }), + // The proxy's 1,280 px version is a few hundred kB; KartaView's own + // "large thumbnail" can be 1.7 MB. Either serves when the other fails. + // A 360° photo's front view is only on storage. + flatFirst: (photo) => [ + photo.isPano ? null : resizedImageUrl(photo.urls.full, FLAT_VIEW_SIZE), + photo.urls.view, + ], + // A docked viewer shows the 1,280 px version in full: the 2,560 px one + // (~0.6 MB a photo) waits until the viewer is bigger than that. + flatSharp: (photo, pixels) => + pixels > FLAT_VIEW_SIZE[0] + ? resizedImageUrl(photo.urls.full, FLAT_FULL_SIZE) + : null, + // The camera's direction, which the drive's track may refine. + bearingOf: (photo) => cameraBearing(photo), + describe: (photo) => ({ + // KartaView's only place name is the sequence's, where the drive began. + title: null, + creator: null, + externalUrl: kartaViewPhotoUrl(photo), + }), +}); /** * Viewer adapter for KartaView photos: a flat photo in an , a 360° - * photo on a WebGL panorama (drag to look, wheel to zoom), and ‹ › to step - * along the sequence (also ← →). Poses follow the view direction. + * photo on a WebGL panorama, and ‹ › along the sequence (photoViewer.js). * * @param {object} options * @param {(photoId: string) => Promise} options.getPhoto A normalized photo (policy.js normalizePhoto). - * @param {(photo: object, delta: -1|1) => Promise} options.step The photo `delta` along the sequence, or null at its end. - * @param {(imageId: string) => void} options.onStep Open the photo ‹ › chose (through the core, so loading and errors show). - * @param {(photo: object) => void} [options.onShown] A photo is on screen. - * @param {(imageId: string) => void} [options.onReopen] Draw the same photo again after a GPU reset (default: `onStep`). - * @param {object} [options.render] Render governor. + * @param {(photo: object, delta: -1|1) => Promise} options.step + * @param {(imageId: string) => void} options.onStep + * @param {(photo: object) => void} [options.onShown] + * @param {object} [options.render] * @param {(url: string, options?: object) => Promise} [options.load] * @param {(canvas: HTMLCanvasElement) => object} [options.createRenderer] * @returns {import('../../registry.js').ViewerAdapter} */ -export function createKartaViewViewer({ - getPhoto, - step, - onStep, - onShown = () => {}, - onReopen = onStep, - render, - load = loadImage, - createRenderer = createPanoramaRenderer, -}) { - const listeners = new Set(); - let root = null; - let picture = null; - let canvas = null; - let prevButton = null; - let nextButton = null; - let renderer = null; - /** WebGL failed once: 360° photos show their front view instead. */ - let rendererFailed = false; - let photo = null; - /** What shows: 'flat', 'pano' or null. */ - let showing = null; - let openSeq = 0; - let renderMode = 'letterbox'; - let drag = null; - let poseFrame = 0; - let removeListeners = []; - /** The document the viewer was built in (new canvases come from it). */ - let documentOf = null; - /** A context loss's look direction, for the reopen of that photo. */ - let restoreView = null; - /** Width of the panorama texture on screen (0: the preview). */ - let panoWidth = 0; - /** The flat photo on screen is the 2,560 px version. */ - let flatSharp = false; - /** The sharper version loading ({seq, kind}): a resize starts no second one. */ - let upgrading = null; - /** A resize came while it loaded: look again once it lands. */ - let upgradeAgain = false; - /** The photo an open is fetching: ‹ › step on from it, not the old one. */ - let openingId = null; - /** ‹ › chose this photo: keep the panorama's look direction for it. */ - let keepViewFor = null; - /** ‹ › presses run one after another, each from where the last one went. */ - let steps = Promise.resolve(); - - function requestRender() { - render?.governorRequestRender?.('kartaview-viewer'); - } - - function emit() { - if (!photo || !showing) return; - // A lost GPU context, still being recovered: the open that redraws the - // panorama poses once it is on screen. - if (showing === 'pano' && !renderer) return; - const view = showing === 'pano' ? renderer.getView() : null; - const heading = cameraBearing(photo); - const pose = { - providerId: KARTAVIEW_PROVIDER_ID, - imageId: photo.id, - position: { lon: photo.lon, lat: photo.lat }, - bearing: - heading === null - ? null - : view - ? wrapDegrees(heading + view.yaw) - : heading, - tilt: view ? view.pitch : 0, - altitude: null, - isPano: showing === 'pano', - capturedAt: photo.capturedAt, - capturedAtPrecision: 'day', - // KartaView's only place name is the sequence's, where the drive began. - title: null, - creator: null, - sequenceId: photo.sequenceId, - externalUrl: kartaViewPhotoUrl(photo), - }; - for (const listener of [...listeners]) { - try { - listener(pose); - } catch { - /* listener errors are the core's to log */ - } - } - requestRender(); - } - - /** Re-emit on the next frame: a drag moves the view many times a frame. */ - function emitSoon() { - if (poseFrame) return; - poseFrame = requestAnimationFrame(() => { - poseFrame = 0; - emit(); - }); - } - - function syncButtons() { - const enabled = Boolean(photo?.sequenceId) && showing !== null; - if (prevButton) prevButton.disabled = !enabled; - if (nextButton) nextButton.disabled = !enabled; - if (enabled) checkEnds(photo); - } - - /** Disable ‹ or › at once where the sequence ends (it also warms the neighbours). */ - function checkEnds(target) { - for (const [delta, button] of [ - [-1, prevButton], - [1, nextButton], - ]) - step(target, delta).then( - (id) => { - if (photo === target && button && !id) disable(button); - }, - () => {}, - ); - } - - /** Disable a step button, keeping keyboard focus in the viewer. */ - function disable(button) { - if (button.ownerDocument?.activeElement === button) root?.focus?.(); - button.disabled = true; - } - - function show(kind) { - showing = kind; - if (picture) picture.hidden = kind !== 'flat'; - if (canvas) canvas.hidden = kind !== 'pano'; - if (root) root.dataset.kind = kind || ''; - syncButtons(); - } - - function applyRenderMode() { - if (picture) - picture.style.objectFit = renderMode === 'fill' ? 'cover' : 'contain'; - } - - function ensureRenderer() { - if (renderer || rendererFailed || !canvas) return renderer; - try { - renderer = createRenderer(canvas); - } catch { - rendererFailed = true; - renderer = null; - } - return renderer; - } - - /** - * A lost WebGL context (GPU reset, memory pressure) takes the panorama - * with it: drop the renderer, and show the photo again on a new one. - */ - function onContextLost(event) { - event.preventDefault?.(); - const lost = renderer; - renderer = null; - // The new renderer starts facing the photo's heading: keep the user's look. - if (showing === 'pano' && photo && lost) - restoreView = { id: photo.id, view: lost.getView() }; - try { - lost?.destroy(); - } catch { - /* already gone */ - } - // A lost canvas hands back its lost context: draw on a new element. - const fresh = makeCanvas(documentOf); - fresh.hidden = canvas.hidden; - canvas.replaceWith?.(fresh); - canvas = fresh; - // Show the photo again through the core, unless a newer one is opening. - if (showing === 'pano' && photo && openingId === null) { - const again = photo.id; - show(null); - openingId = again; - onReopen(again); - } - } - - /** The panorama canvas, wired for drag, pinch, wheel and context loss. */ - function makeCanvas(doc) { - const element = doc.createElement('canvas'); - element.className = 'sl-kv-pano'; - element.hidden = true; - listen(element, 'pointerdown', onPointerDown); - listen(element, 'pointermove', onPointerMove); - listen(element, 'pointerup', onPointerUp); - listen(element, 'pointercancel', onPointerUp); - listen(element, 'wheel', onWheel, { passive: false }); - listen(element, 'webglcontextlost', onContextLost); - return element; - } - - /** - * Step along the sequence from the photo showing, or from the one on its - * way after an earlier press, and open it through the core. - */ - function stepBy(delta) { - steps = steps.then(async () => { - const fromId = openingId ?? photo?.id; - if (!fromId || !root) return; - let from; - let nextId = null; - try { - from = await getPhoto(fromId); - if (from.sequenceId) nextId = await step(from, delta); - } catch { - // Not the end of the sequence, just no answer: the press is lost. - return; - } - // Closed, or another photo was opened meanwhile. - if ((openingId ?? photo?.id) !== fromId) return; - if (!nextId) { - const button = delta < 0 ? prevButton : nextButton; - if (button && photo?.id === fromId) disable(button); - return; - } - openingId = nextId; - keepViewFor = nextId; - onStep(nextId); - }); - } - - /** Pointers on the panorama: one drags the view, two pinch the zoom. */ - const pointers = new Map(); - let pinch = null; - - function pinchSpan() { - const [a, b] = [...pointers.values()]; - return Math.max(1, Math.hypot(a.x - b.x, a.y - b.y)); - } - - function onPointerDown(event) { - if (showing !== 'pano' || !renderer) return; - if (event.pointerType === 'mouse' && event.button !== 0) return; - pointers.set(event.pointerId, { x: event.clientX, y: event.clientY }); - canvas.setPointerCapture?.(event.pointerId); - if (pointers.size === 2) { - drag = null; - pinch = { span: pinchSpan(), fov: renderer.getView().fov }; - return; - } - drag = { - id: event.pointerId, - x: event.clientX, - y: event.clientY, - view: renderer.getView(), - }; - canvas.classList.add('sl-kv-dragging'); - } - - function onPointerMove(event) { - if (pointers.has(event.pointerId)) - pointers.set(event.pointerId, { x: event.clientX, y: event.clientY }); - if (pinch && pointers.size === 2 && renderer) { - renderer.setView({ fov: pinch.fov * (pinch.span / pinchSpan()) }); - return; - } - if (!drag || event.pointerId !== drag.id || !renderer) return; - const { fov } = drag.view; - const width = Math.max(1, canvas.clientWidth); - const height = Math.max(1, canvas.clientHeight); - const vertical = fov; - const horizontal = fov * (width / height); - renderer.setView({ - yaw: drag.view.yaw - ((event.clientX - drag.x) / width) * horizontal, - pitch: drag.view.pitch + ((event.clientY - drag.y) / height) * vertical, - }); - emitSoon(); - } - - function onPointerUp(event) { - pointers.delete(event.pointerId); - if (pointers.size < 2) pinch = null; - if (!drag || event.pointerId !== drag.id) return; - drag = null; - canvas.classList.remove('sl-kv-dragging'); - } - - function onWheel(event) { - if (showing !== 'pano' || !renderer) return; - event.preventDefault(); - const { fov } = renderer.getView(); - renderer.setView({ fov: fov * Math.exp(event.deltaY * 0.001) }); - } - - function onKeyDown(event) { - if ( - event.defaultPrevented || - event.altKey || - event.ctrlKey || - event.metaKey - ) - return; - const pano = showing === 'pano' && renderer; - const turn = { ArrowLeft: -1, ArrowRight: 1 }[event.key]; - // Shift+← → look around a panorama; ← → step along the sequence. - if (turn && event.shiftKey && pano) { - event.preventDefault(); - const { yaw, fov } = renderer.getView(); - renderer.setView({ yaw: yaw + (turn * fov) / 4 }); - emitSoon(); - } else if (turn) { - event.preventDefault(); - stepBy(turn); - } else if ((event.key === 'ArrowUp' || event.key === 'ArrowDown') && pano) { - event.preventDefault(); - const { pitch, fov } = renderer.getView(); - renderer.setView({ - pitch: pitch + ((event.key === 'ArrowUp' ? 1 : -1) * fov) / 6, - }); - emitSoon(); - } else if (['+', '=', '-', '_'].includes(event.key) && pano) { - event.preventDefault(); - const { fov } = renderer.getView(); - renderer.setView({ - fov: fov * (event.key === '+' || event.key === '=' ? 0.85 : 1 / 0.85), - }); - } - } - - function listen(target, type, handler, options) { - target.addEventListener(type, handler, options); - removeListeners.push(() => - target.removeEventListener(type, handler, options), - ); - } - - function build(host) { - const doc = host.ownerDocument; - documentOf = doc; - root = doc.createElement('div'); - root.className = 'sl-kv-viewer'; - root.tabIndex = 0; - root.setAttribute('role', 'group'); - root.setAttribute( - 'aria-label', - t( - 'KartaView photo. ← and → step along the sequence. On a 360° photo, drag or use Shift+← →, ↑ ↓ to look around, and + − to zoom', - ), - ); - picture = doc.createElement('img'); - picture.className = 'sl-kv-photo'; - picture.alt = t('KartaView street-level photo'); - picture.draggable = false; - picture.hidden = true; - canvas = makeCanvas(doc); - const nav = (label, glyph, delta) => { - const button = doc.createElement('button'); - button.type = 'button'; - button.className = `sl-kv-step sl-kv-step-${delta < 0 ? 'prev' : 'next'}`; - button.textContent = glyph; - button.title = label; - button.setAttribute('aria-label', label); - button.disabled = true; - listen(button, 'click', (event) => { - event.stopPropagation(); - stepBy(delta); - }); - return button; - }; - prevButton = nav('Previous photo in this sequence (←)', '‹', -1); - nextButton = nav('Next photo in this sequence (→)', '›', 1); - root.append(picture, canvas, prevButton, nextButton); - host.append(root); - listen(root, 'keydown', onKeyDown); - applyRenderMode(); - } - - function teardown() { - pointers.clear(); - pinch = null; - for (const remove of removeListeners) remove(); - removeListeners = []; - if (poseFrame) cancelAnimationFrame(poseFrame); - poseFrame = 0; - drag = null; - renderer?.destroy(); - renderer = null; - root?.remove(); - root = picture = canvas = prevButton = nextButton = null; - showing = null; - photo = null; - } - - /** A sharper version once the first is on screen, if nothing replaced it. */ - async function upgrade(target, seq, kind) { - // A 360° photo shown flat (no WebGL) keeps its front view. - if (kind === 'flat' && target.isPano) return; - const size = kind === 'pano' ? panoSizeNow() : FLAT_FULL_SIZE; - if (kind === 'pano' && size[0] <= panoWidth) return; - // A docked viewer shows the 1,280 px version in full: the 2,560 px one - // (~0.6 MB a photo) waits until the viewer is bigger than that. - if (kind === 'flat' && (flatSharp || flatPixels() <= FLAT_VIEW_SIZE[0])) - return; - const url = - kind === 'pano' - ? resizedImageUrl(target.urls.pano, size) - : resizedImageUrl(target.urls.full, size); - if (!url) return; - // Resizes come in bursts while the panel animates: one load at a time - // for this photo, and one more look once it lands. - if (upgrading?.seq === seq && upgrading.kind === kind) { - upgradeAgain = true; - return; - } - const pending = (upgrading = { seq, kind }); - upgradeAgain = false; - let image = null; - try { - image = await load(url, { crossOrigin: kind === 'pano' }); - if (seq !== openSeq || photo !== target || showing !== kind) return; - if (kind === 'pano') { - if (!renderer || size[0] <= panoWidth) return; - renderer.setImage(image); - image = null; - panoWidth = size[0]; - } else { - picture.src = image.src; - image = null; - flatSharp = true; - } - requestRender(); - } catch { - /* the first version stays */ - } finally { - release(image); - if (upgrading === pending) { - upgrading = null; - if (upgradeAgain && seq === openSeq && photo === target) { - upgradeAgain = false; - upgrade(target, seq, showing); - } - } - } - } - - /** Device pixels across the flat photo's frame. */ - function flatPixels() { - const ratio = globalThis.devicePixelRatio || 1; - return ( - Math.max(picture?.clientWidth || 0, picture?.clientHeight || 0) * ratio - ); - } - - /** The sharp panorama this viewer needs now: 2,048 docked, 4,096 expanded. */ - function panoSizeNow() { - const ratio = Math.min(globalThis.devicePixelRatio || 1, 2); - const width = (canvas?.clientWidth || 0) * ratio; - return panoSizeFor(width, renderer?.getView().fov ?? PANO_FOV_DEFAULT_DEG); - } - - async function showPano(target, seq, keepView) { - // Only the proxy can shrink a 13,000 px panorama: if it fails, the - // caller shows the front view rather than waiting on it twice. - const image = await load( - resizedImageUrl(target.urls.pano, PANO_PREVIEW_SIZE), - { crossOrigin: true }, - ); - if (seq !== openSeq) { - release(image); - return false; - } - // The context may have been lost while this loaded: build on the new - // canvas, or let the caller show the front view. - if (!ensureRenderer()) { - release(image); - throw new Error(LOAD_FAILED); - } - const restore = restoreView?.id === target.id ? restoreView.view : null; - restoreView = null; - if (restore) renderer.setView(restore); - else if (!keepView || showing !== 'pano') - renderer.setView({ yaw: 0, pitch: 0, fov: PANO_FOV_DEFAULT_DEG }); - photo = target; - // Shown first: a hidden canvas measures 0 and would be sized 1×1. - show('pano'); - renderer.setImage(image); - panoWidth = 0; - return true; - } - - async function showFlat(target, seq) { - // The proxy's 1,280 px version is a few hundred kB; KartaView's own - // "large thumbnail" can be 1.7 MB. Either serves when the other fails. - // A 360° photo's front view is only on storage. - const proxied = target.isPano - ? null - : resizedImageUrl(target.urls.full, FLAT_VIEW_SIZE); - const sources = [proxied, target.urls.view].filter(Boolean); - let image; - let failure = null; - for (const url of sources) { - try { - image = await load(url, { crossOrigin: false }); - break; - } catch (error) { - failure = error; - if (seq !== openSeq) return false; - } - } - if (!image) throw failure || new Error(LOAD_FAILED); - if (seq !== openSeq) return false; - picture.src = image.src; - flatSharp = false; - photo = target; - show('flat'); - return true; - } - - async function openPhoto(id, seq, keepView) { - const target = await getPhoto(id); - if (seq !== openSeq) return; - let shown = false; - if (target.isPano && target.urls.pano && ensureRenderer()) { - try { - shown = await showPano(target, seq, keepView); - } catch { - // The panorama failed: its front view is better than nothing. - if (seq !== openSeq) return; - shown = await showFlat(target, seq); - } - } else shown = await showFlat(target, seq); - if (!shown) return; - emit(); - onShown(target); - upgrade(target, seq, showing); - } - - const adapter = { - async mount(host) { - if (!host) throw new Error('Street Level viewer has no host element'); - if (root && root.parentNode === host) return; - teardown(); - rendererFailed = false; - build(host); - }, - - /** Resolves once the photo is on screen and its pose was emitted. */ - async open(imageId) { - if (!root) throw new Error('KartaView viewer is not mounted'); - const id = String(imageId); - const seq = ++openSeq; - const keepView = keepViewFor === id; - keepViewFor = null; - openingId = id; - try { - await openPhoto(id, seq, keepView); - } finally { - if (openingId === id && seq === openSeq) openingId = null; - } - }, - - close() { - openSeq++; - openingId = null; - keepViewFor = null; - photo = null; - show(null); - if (picture) picture.removeAttribute('src'); - renderer?.clear?.(); - }, - - unmount() { - openSeq++; - openingId = null; - keepViewFor = null; - teardown(); - }, - - resize() { - renderer?.resize(); - // Expanded: a docked viewer's first version is now too soft. - if (showing === 'pano' && photo && panoSizeNow()[0] > panoWidth) - upgrade(photo, openSeq, 'pano'); - if (showing === 'flat' && photo && !flatSharp) - upgrade(photo, openSeq, 'flat'); - }, - - /** FIT / FILL frame a flat photo; a panorama has no frame to fit. */ - supportsRenderModes: () => showing !== 'pano', - - setRenderMode(mode) { - renderMode = mode === 'fill' ? 'fill' : 'letterbox'; - applyRenderMode(); - }, - - /** The photo's direction became known (its drive loaded): pose again. */ - refreshPose() { - emit(); - }, - - onPose(listener) { - listeners.add(listener); - return () => listeners.delete(listener); - }, - }; - return adapter; +export function createKartaViewViewer({ load = loadImage, ...options }) { + return createPhotoViewer({ + providerId: KARTAVIEW_PROVIDER_ID, + name: KARTAVIEW_NAME, + sources: KARTAVIEW_SOURCES, + load, + ...options, + }); } diff --git a/src/layers/streetLevel/providers/kartaview/viewer.test.mjs b/src/layers/streetLevel/providers/kartaview/viewer.test.mjs index 1391b727c..9ca5174c1 100644 --- a/src/layers/streetLevel/providers/kartaview/viewer.test.mjs +++ b/src/layers/streetLevel/providers/kartaview/viewer.test.mjs @@ -883,6 +883,11 @@ test('the panorama is as sharp as the viewer needs: 2,048 docked, 4,096 once exp }); test('a CORS image is decoded off the main thread into an ImageBitmap where the browser can', async (t) => { + // A blob whose header names no size (the viewer then reads none). + const fakeBlob = { + size: 4, + slice: () => ({ arrayBuffer: async () => new ArrayBuffer(0) }), + }; const saved = { fetch: globalThis.fetch, bitmap: globalThis.createImageBitmap, @@ -896,7 +901,7 @@ test('a CORS image is decoded off the main thread into an ImageBitmap where the fetched.push({ url, mode: init.mode }); return url.includes('gone') ? { ok: false, status: 409 } - : { ok: true, blob: async () => 'blob' }; + : { ok: true, blob: async () => fakeBlob }; }; globalThis.createImageBitmap = async (blob) => ({ blob, @@ -906,7 +911,7 @@ test('a CORS image is decoded off the main thread into an ImageBitmap where the const bitmap = await loadImage('https://cdn.kartaview.org/a', { crossOrigin: true, }); - assert.deepEqual(bitmap, { blob: 'blob', width: 4096, height: 2048 }); + assert.deepEqual(bitmap, { blob: fakeBlob, width: 4096, height: 2048 }); assert.deepEqual(fetched, [ { url: 'https://cdn.kartaview.org/a', mode: 'cors' }, ]); diff --git a/src/layers/streetLevel/providers/mapillary/coverageFixture.mjs b/src/layers/streetLevel/providers/mapillary/coverageFixture.mjs index 8fb94821b..9f9f0fa2f 100644 --- a/src/layers/streetLevel/providers/mapillary/coverageFixture.mjs +++ b/src/layers/streetLevel/providers/mapillary/coverageFixture.mjs @@ -1,6 +1,6 @@ import { PbfWriter } from 'pbf'; -const zigzag = (value) => (value << 1) ^ (value >> 31); +export const zigzag = (value) => (value << 1) ^ (value >> 31); /** Tile-local [x, y] (0..extent) of a lon/lat inside tile (x, y, z). */ export function lonLatToTileLocal([lon, lat], { x, y, z }, extent = 4096) { @@ -13,7 +13,7 @@ export function lonLatToTileLocal([lon, lat], { x, y, z }, extent = 4096) { } /** One MVT layer writer with deduplicated keys and appended values. */ -function layerWriter(name, extent) { +export function layerWriter(name, extent) { const keys = new Map(); const values = []; const features = []; @@ -62,7 +62,7 @@ function layerWriter(name, extent) { } /** MVT MoveTo/LineTo commands; the cursor carries over between parts. */ -function lineGeometry(parts) { +export function lineGeometry(parts) { const out = []; let cx = 0; let cy = 0; diff --git a/src/layers/streetLevel/providers/panoramax/decode.js b/src/layers/streetLevel/providers/panoramax/decode.js new file mode 100644 index 000000000..b8cbfeacc --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/decode.js @@ -0,0 +1,291 @@ +import { PbfReader } from 'pbf'; +import { VectorTile } from '@mapbox/vector-tile'; +import { tileLocalToLonLat } from '../../tileMath.js'; +import { + DECODE_SLICE_MS, + OVERVIEW_BINS_PER_AXIS, + SIMPLIFY_TOLERANCE_UNITS, + panoramaxTimeMs, +} from './policy.js'; + +/** + * Douglas–Peucker in tile units: the points of `line` ({x, y}) that keep + * it within `tolerance` of the original (measured to each segment, so + * dead ends and U-turns stay), first and last always kept. + * Iterative, so a long track cannot overflow the stack. + */ +export function simplifyLine(line, tolerance) { + if (line.length < 3 || !(tolerance > 0)) return line; + const keep = new Uint8Array(line.length); + keep[0] = 1; + keep[line.length - 1] = 1; + const spans = [[0, line.length - 1]]; + while (spans.length) { + const [from, to] = spans.pop(); + const a = line[from]; + const b = line[to]; + const dx = b.x - a.x; + const dy = b.y - a.y; + const span2 = dx * dx + dy * dy; + let far = -1; + let farthest = tolerance; + for (let i = from + 1; i < to; i++) { + const p = line[i]; + // Distance to the segment, not the infinite line: a dead end or a + // U-turn (a point beyond the chord's ends) is kept. + const t = span2 + ? Math.max( + 0, + Math.min(1, ((p.x - a.x) * dx + (p.y - a.y) * dy) / span2), + ) + : 0; + const d = Math.hypot(p.x - (a.x + t * dx), p.y - (a.y + t * dy)); + if (d > farthest) { + farthest = d; + far = i; + } + } + if (far === -1) continue; + keep[far] = 1; + spans.push([from, far], [far, to]); + } + return line.filter((_, i) => keep[i]); +} + +/** Epoch ms of a sequence's day ('2025-08-11'), or 0 when undated. */ +function dayMs(value) { + return panoramaxTimeMs(typeof value === 'string' ? value : '') ?? 0; +} + +/** + * Decode a Panoramax coverage tile (`/api/map/{z}/{x}/{y}.mvt`). + * + * - `sequences` (z7–15): one line per sequence, its parts kept apart so no + * segment is drawn across a gap, with its day and camera type; longest + * first, so a cap on lines drawn drops the shortest. + * - `grid` (z0–6): picture-density cells, as overview points binned to at + * most OVERVIEW_BINS_PER_AXIS² a tile (a z0 tile holds ~70,000 cells). A + * cell has 360° and flat counts but no date. + * - `pictures` (z15) is left for `decodePictures`. + * + * In one go; `decodeCoverageTileSliced` gives the page a turn between + * slices. + * @param {Uint8Array} bytes + * @param {{x: number, y: number, z: number}} address + * @returns {{sequences: Array<{id: string, parts: Array>, capturedAt: number, isPano: boolean}>, overview: Array<{id: string, lon: number, lat: number, capturedAt: null, isPano: boolean, hasPano: boolean, hasFlat: boolean, weight: number}>}} + */ +export function decodeCoverageTile(bytes, address) { + const steps = coverageSteps(bytes, address); + let step = steps.next(); + while (!step.done) step = steps.next(); + return step.value; +} + +/** Give the page a turn: input and rendering run before the next slice. */ +function nextTask() { + if (typeof globalThis.scheduler?.yield === 'function') + return globalThis.scheduler.yield(); + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +/** + * `decodeCoverageTile` in slices of about `sliceMs`, the page getting a + * turn between them: a 2 MB z10 tile over Paris is ~200 ms of decoding on + * a fast laptop, more on a phone, and would freeze the globe while the + * camera approaches. Stops at the next slice once `signal` aborts. + * @param {Uint8Array} bytes + * @param {{x: number, y: number, z: number}} address + * @param {{signal?: AbortSignal, sliceMs?: number, pause?: () => Promise, now?: () => number}} [options] + */ +export async function decodeCoverageTileSliced( + bytes, + address, + { + signal, + sliceMs = DECODE_SLICE_MS, + pause = nextTask, + now = () => performance.now(), + } = {}, +) { + const steps = coverageSteps(bytes, address); + let started = now(); + let step = steps.next(); + while (!step.done) { + if (now() - started >= sliceMs) { + await pause(); + signal?.throwIfAborted(); + started = now(); + } + step = steps.next(); + } + return step.value; +} + +/** The decode, a step a feature (see `decodeCoverageTile`). */ +function* coverageSteps(bytes, address) { + const result = { sequences: [], overview: [] }; + if (!bytes || !bytes.length) return result; + const tile = new VectorTile(new PbfReader(bytes)); + const { x, y, z } = address; + + const sequences = tile.layers.sequences; + if (sequences) { + const measured = []; + for (let i = 0; i < sequences.length; i++) { + const feature = sequences.feature(i); + const props = feature.properties || {}; + const parts = []; + let length = 0; + const tolerance = (SIMPLIFY_TOLERANCE_UNITS * sequences.extent) / 4096; + for (const line of feature.loadGeometry()) { + if (line.length < 2) continue; + for (let k = 1; k < line.length; k++) + length += Math.hypot( + line[k].x - line[k - 1].x, + line[k].y - line[k - 1].y, + ); + // Simplified before anything else: GPS tracks carry a point every + // few metres, and each becomes ~8 ground-line vertices on the GPU. + parts.push( + simplifyLine(line, tolerance).map((point) => + tileLocalToLonLat(point.x, point.y, sequences.extent, x, y, z), + ), + ); + } + if (!parts.length || props.id == null) continue; + const sequence = { + id: String(props.id), + parts, + capturedAt: dayMs(props.date), + // A sequence with no camera type is neither (the filters keep it). + isPano: + props.type === 'equirectangular' + ? true + : props.type === 'flat' + ? false + : null, + }; + measured.push({ sequence, length }); + yield; + } + // A turn before the sort: over a city it is thousands of lines. + yield; + measured.sort((a, b) => b.length - a.length); + result.sequences = measured.map(({ sequence }) => sequence); + } + + const grid = tile.layers.grid; + if (grid) { + const bins = new Map(); + const size = grid.extent / OVERVIEW_BINS_PER_AXIS; + for (let i = 0; i < grid.length; i++) { + const feature = grid.feature(i); + const props = feature.properties || {}; + const point = feature.loadGeometry()?.[0]?.[0]; + if (i % 64 === 63) yield; + if (!point) continue; + const pano = Number(props.nb_360_pictures) || 0; + const flat = Number(props.nb_flat_pictures) || 0; + const weight = Number(props.nb_pictures) || pano + flat; + if (weight <= 0) continue; + const key = `${Math.floor(point.x / size)}:${Math.floor(point.y / size)}`; + const bin = bins.get(key); + if (!bin) { + bins.set(key, { point, weight, pano, flat, top: weight }); + continue; + } + // The bin sits on its busiest cell and counts them all. + bin.weight += weight; + bin.pano += pano; + bin.flat += flat; + if (weight > bin.top) { + bin.top = weight; + bin.point = point; + } + } + for (const [key, bin] of bins) { + const [lon, lat] = tileLocalToLonLat( + bin.point.x, + bin.point.y, + grid.extent, + x, + y, + z, + ); + result.overview.push({ + id: `${z}/${x}/${y}/${key}`, + lon, + lat, + capturedAt: null, + isPano: bin.pano > bin.flat, + hasPano: bin.pano > 0, + hasFlat: bin.flat > 0, + weight: bin.weight, + }); + } + } + return result; +} + +/** A picture's sequence ids: the tile stores them as a JSON array in a string. */ +function sequenceIdsOf(props) { + const raw = props.sequences; + if (typeof raw === 'string' && raw.startsWith('[')) { + try { + const ids = JSON.parse(raw); + if (Array.isArray(ids)) return ids.map(String); + } catch { + /* fall through to the first sequence */ + } + } + return props.first_sequence != null ? [String(props.first_sequence)] : []; +} + +/** + * The pictures of a z15 tile that belong to `sequenceId`, as cones draw + * them (unsorted). + * @param {Uint8Array} bytes + * @param {{x: number, y: number, z: number}} address + * @param {string} sequenceId + * @returns {Array<{id: string, lon: number, lat: number, compassAngle: number, capturedAt: number, isPano: boolean}>} + */ +export function decodePictures(bytes, address, sequenceId) { + if (!bytes || !bytes.length) return []; + const tile = new VectorTile(new PbfReader(bytes)); + const layer = tile.layers.pictures; + if (!layer) return []; + const { x, y, z } = address; + const wanted = String(sequenceId); + const out = []; + for (let i = 0; i < layer.length; i++) { + const feature = layer.feature(i); + const props = feature.properties || {}; + // A cheap test first: most pictures in a tile are other sequences'. + if ( + typeof props.sequences === 'string' && + !props.sequences.includes(wanted) + ) + continue; + if (!sequenceIdsOf(props).includes(wanted) || props.id == null) continue; + const point = feature.loadGeometry()?.[0]?.[0]; + if (!point) continue; + const [lon, lat] = tileLocalToLonLat( + point.x, + point.y, + layer.extent, + x, + y, + z, + ); + const heading = Number(props.heading); + out.push({ + id: String(props.id), + lon, + lat, + compassAngle: Number.isFinite(heading) ? heading : 0, + capturedAt: panoramaxTimeMs(String(props.ts ?? '')) ?? 0, + isPano: props.type === 'equirectangular', + }); + } + return out; +} diff --git a/src/layers/streetLevel/providers/panoramax/decode.test.mjs b/src/layers/streetLevel/providers/panoramax/decode.test.mjs new file mode 100644 index 000000000..7caa25a63 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/decode.test.mjs @@ -0,0 +1,317 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + decodeCoverageTile, + decodeCoverageTileSliced, + decodePictures, + simplifyLine, +} from './decode.js'; +import { OVERVIEW_BINS_PER_AXIS } from './policy.js'; +import { encodePanoramaxTile } from '../../../../testSupport/panoramaxFixtures.mjs'; +import { tileCentre } from '../../../../testSupport/streetLevelFakes.mjs'; + +const near = (a, b, eps = 1e-4) => Math.abs(a - b) < eps; + +test('sequence lines keep their parts apart, with their day and camera type', () => { + const { tile, lon, lat } = tileCentre(2.35, 48.85); + const bytes = encodePanoramaxTile(tile, { + sequences: [ + { + id: 'pano-seq', + type: 'equirectangular', + date: '2024-03-05', + parts: [ + [ + [lon - 0.001, lat], + [lon, lat], + ], + [ + [lon + 0.0005, lat], + [lon + 0.001, lat], + ], + ], + }, + { + id: 'flat-seq', + parts: [ + [ + [lon, lat - 0.001], + [lon, lat], + ], + ], + }, + ], + }); + const { sequences, overview } = decodeCoverageTile(bytes, tile); + assert.equal(overview.length, 0); + assert.equal(sequences.length, 2); + const pano = sequences.find((s) => s.id === 'pano-seq'); + const flat = sequences.find((s) => s.id === 'flat-seq'); + assert.equal(pano.isPano, true); + assert.equal(pano.capturedAt, Date.UTC(2024, 2, 5)); + assert.equal(pano.parts.length, 2, 'the gap is not bridged'); + assert.ok(near(pano.parts[1][1][0], lon + 0.001)); + assert.equal(flat.isPano, false); + const untyped = decodeCoverageTile( + encodePanoramaxTile(tile, { + sequences: [ + { + id: 'u', + type: '', + parts: [ + [ + [lon, lat], + [lon + 0.001, lat], + ], + ], + }, + ], + }), + tile, + ).sequences[0]; + assert.equal(untyped.isPano, null, 'no camera type: neither'); +}); + +test('density cells bin to a bounded number of points that keep the counts and kinds', () => { + const tile = { x: 0, y: 0, z: 0 }; + // Two cells in one bin (3.75° wide at z0) far enough apart to tell + // which it sits on, one far away, one empty. + const grid = [ + { lon: 0.3, lat: 48.85, pano: 0, flat: 5 }, + { lon: 3.4, lat: 48.85, pano: 30, flat: 0 }, + { lon: -73.9, lat: 40.7, pano: 1, flat: 0 }, + { lon: 10, lat: 10, pano: 0, flat: 0 }, + ]; + const { overview } = decodeCoverageTile( + encodePanoramaxTile(tile, { grid }), + tile, + ); + assert.equal(overview.length, 2, 'Paris cells share a bin; empty cells go'); + const paris = overview.find((p) => p.lon > 0); + assert.equal(paris.weight, 35); + assert.equal(paris.hasPano, true); + assert.equal(paris.hasFlat, true); + assert.equal(paris.isPano, true, 'mostly 360°'); + assert.equal(paris.capturedAt, null, 'cells are undated'); + assert.ok( + near(paris.lon, 3.4, 0.2), + `sits on the busiest cell: ${paris.lon}`, + ); + const nyc = overview.find((p) => p.lon < 0); + assert.deepEqual([nyc.hasPano, nyc.hasFlat, nyc.weight], [true, false, 1]); +}); + +test('lines come longest first, so a cap on lines drawn drops the shortest', () => { + const { tile, lon, lat } = tileCentre(2.35, 48.85); + const span = (id, metres) => ({ + id, + parts: [ + [ + [lon, lat], + [lon + metres / 73_000, lat], + ], + ], + }); + const { sequences } = decodeCoverageTile( + encodePanoramaxTile(tile, { + sequences: [span('short', 20), span('long', 900), span('mid', 300)], + }), + tile, + ); + assert.deepEqual( + sequences.map((s) => s.id), + ['long', 'mid', 'short'], + ); +}); + +test('a dense grid tile never draws more than the bins allow', () => { + const tile = { x: 0, y: 0, z: 0 }; + const grid = []; + for (let i = 0; i < 300; i++) + for (let j = 0; j < 60; j++) + grid.push({ lon: -170 + i * 1.1, lat: -60 + j * 2 }); + const { overview } = decodeCoverageTile( + encodePanoramaxTile(tile, { grid }), + tile, + ); + assert.ok(overview.length <= OVERVIEW_BINS_PER_AXIS ** 2); + assert.ok(overview.length > 1000); + assert.equal( + overview.reduce((sum, p) => sum + p.weight, 0), + grid.length, + 'every picture is still counted', + ); +}); + +/** A tile with many lines and a dense grid, for the sliced decode. */ +function busyTile() { + const { tile, lon, lat } = tileCentre(2.35, 48.85); + const sequences = Array.from({ length: 200 }, (_, i) => ({ + id: `s${i}`, + parts: [ + [ + [lon, lat + i / 1e6], + [lon + (i + 1) / 73_000, lat + i / 1e6], + ], + ], + })); + const grid = Array.from({ length: 500 }, (_, i) => ({ + lon: lon + (i % 25) / 1e4, + lat: lat + Math.floor(i / 25) / 1e4, + })); + return { tile, bytes: encodePanoramaxTile(tile, { sequences, grid }) }; +} + +test('a sliced decode gives the page turns and decodes the same as one in one go', async () => { + const { tile, bytes } = busyTile(); + let clock = 0; + let pauses = 0; + const sliced = await decodeCoverageTileSliced(bytes, tile, { + sliceMs: 8, + // Each step "takes" a millisecond: a turn every eight features. + now: () => clock++, + pause: async () => { + pauses++; + }, + }); + assert.ok(pauses > 20, `${pauses} turns`); + assert.deepEqual(sliced, decodeCoverageTile(bytes, tile)); +}); + +test('a sliced decode stops at the next turn once aborted', async () => { + const { tile, bytes } = busyTile(); + const controller = new AbortController(); + let clock = 0; + let pauses = 0; + await assert.rejects( + decodeCoverageTileSliced(bytes, tile, { + signal: controller.signal, + now: () => (clock += 10), + pause: async () => { + if (++pauses === 2) controller.abort(); + }, + }), + { name: 'AbortError' }, + ); + assert.equal(pauses, 2, 'no work after the abort'); +}); + +test('a fast sliced decode takes no turns', async () => { + const { tile, bytes } = busyTile(); + let pauses = 0; + await decodeCoverageTileSliced(bytes, tile, { + now: () => 0, + pause: async () => { + pauses++; + }, + }); + assert.equal(pauses, 0); +}); + +test('an empty tile decodes to nothing', () => { + for (const bytes of [new Uint8Array(0), null, undefined]) + assert.deepEqual(decodeCoverageTile(bytes, { x: 0, y: 0, z: 0 }), { + sequences: [], + overview: [], + }); + assert.deepEqual( + decodePictures(new Uint8Array(0), { x: 0, y: 0, z: 15 }, 's'), + [], + ); +}); + +test('a z15 tile’s pictures of one sequence become cones; other sequences’ are skipped', () => { + const { tile, lon, lat } = tileCentre(2.35, 48.85, 15); + const bytes = encodePanoramaxTile(tile, { + pictures: [ + { + id: 'a', + lon, + lat, + sequenceId: 's1', + heading: 45, + type: 'equirectangular', + ts: '2023-12-12 07:56:37+00', + }, + { id: 'b', lon: lon + 0.0001, lat, sequenceId: 's2' }, + { id: 'c', lon: lon + 0.0002, lat, sequenceId: 's1' }, + ], + }); + const images = decodePictures(bytes, tile, 's1'); + assert.deepEqual( + images.map((i) => i.id), + ['a', 'c'], + ); + assert.equal(images[0].compassAngle, 45); + assert.equal(images[0].isPano, true); + assert.equal(images[0].capturedAt, Date.UTC(2023, 11, 12, 7, 56, 37)); + assert.ok(near(images[1].lon, lon + 0.0002)); + assert.deepEqual(decodePictures(bytes, tile, 'nope'), []); + // A tile without pictures (below z15) has none to give. + const lines = encodePanoramaxTile(tile, { + sequences: [ + { + id: 's1', + parts: [ + [ + [lon, lat], + [lon + 0.001, lat], + ], + ], + }, + ], + }); + assert.deepEqual(decodePictures(lines, tile, 's1'), []); +}); + +test('lines are simplified within a few tile units, ends and corners kept', () => { + const zigzag = Array.from({ length: 1000 }, (_, i) => ({ + x: i, + y: i % 2 ? 1 : 0, + })); + assert.deepEqual( + simplifyLine(zigzag, 4), + [zigzag[0], zigzag.at(-1)], + 'a 1-unit wobble goes', + ); + const corner = [ + { x: 0, y: 0 }, + { x: 50, y: 1 }, + { x: 100, y: 0 }, + { x: 100, y: 100 }, + ]; + assert.deepEqual(simplifyLine(corner, 4), [corner[0], corner[2], corner[3]]); + assert.equal(simplifyLine(corner.slice(0, 2), 4).length, 2); + // A loop back to its start (no chord) keeps its far point. + const loop = [ + { x: 0, y: 0 }, + { x: 40, y: 0 }, + { x: 0, y: 0 }, + ]; + assert.equal(simplifyLine(loop, 4).length, 3); + assert.equal(simplifyLine(zigzag, 0).length, 1000, 'no tolerance, no change'); + // A dead end: out to 100 and back to 50 keeps its far end. + const deadEnd = [ + { x: 0, y: 0 }, + { x: 100, y: 0 }, + { x: 50, y: 0 }, + ]; + assert.deepEqual(simplifyLine(deadEnd, 4), deadEnd); +}); + +test('a decoded GPS track keeps a fraction of its points', () => { + const { tile, lon, lat } = tileCentre(2.35, 48.85); + const track = Array.from({ length: 500 }, (_, i) => [ + lon - 0.004 + i * 0.000016, + lat + Math.sin(i / 40) * 0.0004, + ]); + const [sequence] = decodeCoverageTile( + encodePanoramaxTile(tile, { sequences: [{ id: 'gps', parts: [track] }] }), + tile, + ).sequences; + assert.ok( + sequence.parts[0].length < 100, + `${sequence.parts[0].length} points`, + ); + assert.ok(sequence.parts[0].length > 5); +}); diff --git a/src/layers/streetLevel/providers/panoramax/index.js b/src/layers/streetLevel/providers/panoramax/index.js new file mode 100644 index 000000000..d3a3c7922 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/index.js @@ -0,0 +1,644 @@ +import { createLineCoverage, sequenceIdFromPick } from '../../lineCoverage.js'; +import { createSequenceCones, thinImages } from '../../sequenceCones.js'; +import { passesImageryFilter } from '../../filter.js'; +import { + lonToTileX, + latToTileY, + overviewZoomForHeight, + tileBounds, +} from '../../tileMath.js'; +import { metresBetween } from '../../view.js'; +import { createLru, createTileCache } from '../../tileCache.js'; +import { decodeCoverageTileSliced, decodePictures } from './decode.js'; +import { createPanoramaxSource } from './source.js'; +import { createPanoramaxViewer } from './viewer.js'; +import { msg } from '../../../../i18n/index.js'; +import { + COLORS, + COVERAGE_LINES_BELOW_M, + COVERAGE_TUNING, + COVERAGE_Z11_BELOW_M, + COVERAGE_Z12_BELOW_M, + COVERAGE_Z13_BELOW_M, + COVERAGE_Z14_BELOW_M, + GONE_MESSAGE, + LINE_CLICK_HINT, + LINE_CLICK_RADIUS_MAX_M, + LINE_CLICK_RADIUS_PER_HEIGHT, + NEAREST_LIMIT, + NEAREST_RADIUS_M, + FILTERED_HINT, + NO_COVERAGE_HINT, + OVERVIEW_HINT, + PANORAMAX_CREDIT_HTML, + PANORAMAX_LABEL, + PANORAMAX_NAME, + PANORAMAX_PROVIDER_ID, + PICK_PREFIX, + PICTURE_TILE_EDGE_M, + PICTURE_TILE_ZOOM, + RATE_LIMITED_MESSAGE, + UNREADABLE_TILE_MESSAGE, + normalizeItem, +} from './policy.js'; + +export { PANORAMAX_PROVIDER_ID } from './policy.js'; + +/** + * A tile's bytes decoded by `decode`; a body that is not a vector tile + * (an HTML error page, a cut-off answer) reads as such, not as a parser's + * "Unimplemented type: 4". + */ +async function readable(decode) { + try { + return await decode(); + } catch (error) { + if (error?.name === 'AbortError' || error?.status !== undefined) + throw error; + throw new Error(UNREADABLE_TILE_MESSAGE); + } +} + +/** + * A density dot's size: busier places draw bigger (2.5 px for one picture, + * 8 px from a few hundred thousand), so a city does not read like farmland. + */ +export function overviewPointSize(point) { + const weight = Math.max(1, Number(point?.weight) || 1); + return Math.min(8, 2.5 + Math.log10(weight)); +} + +/** Pictures kept in memory by id (lookups and ‹ › fill it). */ +const PHOTO_CACHE_SIZE = 2000; +/** Sequences whose cone centre is remembered. */ +const ANCHOR_CACHE_SIZE = 50; +/** Decoded coverage tiles kept (a simplified dense z10 tile is ~4 MB). */ +const COVERAGE_TILE_CACHE_SIZE = 16; +/** One sequence's pictures in a z15 tile, kept for re-centring. */ +const PICTURE_LIST_CACHE_SIZE = 32; + +/** + * Sequence lines at z10–14 below 60 km; above, the density grid at z5 and + * coarser. + */ +export function panoramaxZoomFor(heightM) { + if (!Number.isFinite(heightM)) return null; + if (heightM <= COVERAGE_LINES_BELOW_M) { + const zoom = + heightM <= COVERAGE_Z14_BELOW_M + ? 14 + : heightM <= COVERAGE_Z13_BELOW_M + ? 13 + : heightM <= COVERAGE_Z12_BELOW_M + ? 12 + : heightM <= COVERAGE_Z11_BELOW_M + ? 11 + : 10; + return { zoom, kind: 'sequence' }; + } + return { + zoom: Math.min(5, overviewZoomForHeight(heightM) ?? 5), + kind: 'overview', + }; +} + +/** + * The imagery filter for lines and grid cells. A cell (undated, with 360° + * and flat counts) stays while it holds pictures of the kind asked for. A + * line is dated by its sequence's first day, so a sequence still growing + * may hide under "since" while its newer pictures can be opened; a line + * with no camera type or no day is kept (a click finds out). + */ +export function passesCoverage(record, filter) { + if (!filter) return true; + if ('hasPano' in record) { + if (filter.pano === 'pano') return record.hasPano; + if (filter.pano === 'flat') return record.hasFlat; + return true; + } + return passesImageryFilter(record, { + pano: typeof record.isPano === 'boolean' ? filter.pano : 'all', + sinceMs: record.capturedAt ? filter.sinceMs : null, + }); +} + +/** The z15 tile holding a point. */ +function pictureTileOf({ lon, lat }) { + const z = PICTURE_TILE_ZOOM; + return { x: lonToTileX(lon, z), y: latToTileY(lat, z), z }; +} + +/** + * The z15 tiles whose pictures surround a point: its own first, and the + * neighbours it sits within PICTURE_TILE_EDGE_M of (at most nine). + */ +export function pictureTilesAround(point) { + const home = pictureTileOf(point); + const box = tileBounds(home.x, home.y, home.z); + const near = (lon, lat) => + metresBetween(point, { lon, lat }) <= PICTURE_TILE_EDGE_M; + // Near both edges at once where tiles are narrow (high latitudes). + const dxs = [0]; + if (near(box.west, point.lat)) dxs.push(-1); + if (near(box.east, point.lat)) dxs.push(1); + const dys = [0]; + if (near(point.lon, box.north)) dys.push(-1); + if (near(point.lon, box.south)) dys.push(1); + const max = 2 ** home.z; + const tiles = []; + for (const dy of dys) + for (const dx of dxs) { + const y = home.y + dy; + if (y < 0 || y >= max) continue; + tiles.push({ x: (home.x + dx + max) % max, y, z: home.z }); + } + return tiles; +} + +const tileKey = ({ x, y, z }) => `${z}/${x}/${y}`; + +/** + * Panoramax Street Level provider: the federated, keyless, open + * street-level commons. Coverage from its vector tiles (density points + * from orbit, sequence lines near the ground); a click on a line opens that + * sequence's picture nearest the click, with cones along the sequence + * around it and ‹ › to step through it. Flat pictures show as photos, 360° + * ones on the shared WebGL panorama. + * @param {{source?: ReturnType, viewerOptions?: object}} [options] + * @returns {import('../../registry.js').StreetLevelProvider} + */ +export function createPanoramaxProvider({ + source = createPanoramaxSource(), + viewerOptions = {}, +} = {}) { + return Object.freeze({ + id: PANORAMAX_PROVIDER_ID, + name: PANORAMAX_NAME, + label: PANORAMAX_LABEL, + requiresKeyId: null, + pickPrefix: PICK_PREFIX.root, + colors: COLORS, + credit: Object.freeze({ html: PANORAMAX_CREDIT_HTML }), + defaultOn: false, + coverageLabel: Object.freeze({ + noun: 'sequences', + action: LINE_CLICK_HINT, + }), + + create(context) { + const state = { + context, + services: context.services, + viewer: null, + keyRequired: false, + keyRejected: false, + statusKnown: true, + coverage: { + zoom: null, + tiles: new Map(), + stale: new Map(), + staleTimer: null, + pending: new Map(), + lastError: null, + holdUntil: 0, + holdTimer: null, + debounceTimer: null, + removeCameraListener: null, + terrainReady: null, + hint: '', + kind: null, + }, + sequence: { + selectedId: null, + images: [], + cache: new Map(), + collection: null, + loading: false, + abort: null, + }, + }; + const photos = createLru(PHOTO_CACHE_SIZE); + /** Where each sequence's cones centre: the picture last opened in it, or a click. */ + const anchors = createLru(ANCHOR_CACHE_SIZE); + /** The z15 tiles the cones were drawn from. */ + let drawn = null; + /** A line click: its sequence's pictures win the next nearest lookup there. */ + let preferred = null; + + function remember(item) { + const photo = normalizeItem(item); + if (photo) photos.set(photo.id, photo); + return photo; + } + + /** + * Lookups in flight, shared: ‹ › and the viewer ask for the same + * picture. A lookup stops once every caller waiting on it has given up + * (its search would hold one of the two API connections for nothing). + */ + const inflight = new Map(); + function getPhoto(photoId, { signal } = {}) { + const id = String(photoId); + const cached = photos.get(id); + if (cached) return Promise.resolve(cached); + if (signal?.aborted) return Promise.reject(signal.reason); + let lookup = inflight.get(id); + if (!lookup) { + const controller = new AbortController(); + lookup = { controller, waiting: 0 }; + lookup.promise = source + .getItem(id, { signal: controller.signal }) + .catch((error) => { + // Ids are UUIDs: the API answers 400 to anything else. + if (error?.status === 400) return null; + throw error; + }) + .then((item) => { + const photo = item && remember(item); + if (!photo) throw new Error(GONE_MESSAGE); + return photo; + }) + .finally(() => { + if (inflight.get(id) === lookup) inflight.delete(id); + }); + lookup.promise.catch(() => {}); + inflight.set(id, lookup); + } + const shared = lookup; + shared.waiting++; + if (!signal) shared.waiting = Infinity; + return new Promise((resolve, reject) => { + const leave = () => { + reject(signal.reason); + shared.waiting--; + // A caller in the same tick (the same photo opened again) may + // still join: stop only if nobody has. + queueMicrotask(() => { + if (shared.waiting > 0) return; + shared.controller.abort(signal.reason); + if (inflight.get(id) === shared) inflight.delete(id); + }); + }; + signal?.addEventListener('abort', leave, { once: true }); + shared.promise.then( + (photo) => { + signal?.removeEventListener('abort', leave); + resolve(photo); + }, + (error) => { + signal?.removeEventListener('abort', leave); + reject(error); + }, + ); + }); + } + + /** + * How far from a line click its sequence's pictures are looked for: + * from high up the click may land tens of metres off the line. + */ + function lineClickRadius() { + const height = state.viewer?.camera?.positionCartographic?.height; + if (!Number.isFinite(height)) return NEAREST_RADIUS_M; + return Math.min( + LINE_CLICK_RADIUS_MAX_M, + Math.max(NEAREST_RADIUS_M, height * LINE_CLICK_RADIUS_PER_HEIGHT), + ); + } + + /** The picture `delta` along its sequence, from its own links (null at an end). */ + async function step(photo, delta) { + if (!photo?.sequenceId) return null; + return (delta < 0 ? photo.prevId : photo.nextId) || null; + } + + // Decoded tiles, kept a while: panning back, or across a zoom band and + // back, neither fetches nor decodes them again. + const coverageTiles = createTileCache({ + size: COVERAGE_TILE_CACHE_SIZE, + load: (tile, { signal }) => + readable(async () => + decodeCoverageTileSliced( + await source.getTile(tile, { signal }), + tile, + { signal }, + ), + ), + }); + /** A z15 tile's pictures of one sequence, by `z/x/y:sequence`. */ + const pictureLists = createLru(PICTURE_LIST_CACHE_SIZE); + /** + * Only finished lists are kept: a load in flight belongs to its own + * selection, whose abort (a re-centre) must not empty the next one's. + */ + async function picturesIn(tile, sequenceId, signal) { + const key = `${tileKey(tile)}:${sequenceId}`; + const kept = pictureLists.get(key); + if (kept) return kept; + const list = await readable(async () => + decodePictures( + await source.getTile(tile, { signal, lane: 'pictures' }), + tile, + sequenceId, + ), + ); + pictureLists.set(key, list); + return list; + } + + const coverage = createLineCoverage({ + state, + fetchTile: (tile, options) => coverageTiles.get(tile, options), + zoomFor: panoramaxZoomFor, + colors: COLORS, + pickPrefix: PICK_PREFIX.sequence, + renderKey: 'panoramax-coverage', + tuning: COVERAGE_TUNING, + messages: { rateLimited: RATE_LIMITED_MESSAGE }, + passes: passesCoverage, + overviewPointSize, + }); + + const cones = createSequenceCones({ + state, + async loadImages(sequenceId, { signal }) { + const anchor = anchors.get(sequenceId); + if (!anchor) return []; + const tiles = pictureTilesAround(anchor); + // Known from the start: a picture opening while these load is + // judged against them (re-centred if outside, left alone inside). + const covering = { sequenceId, tiles: new Set(tiles.map(tileKey)) }; + drawn = covering; + const conesOf = (lists) => + thinImages( + [ + ...new Map( + lists.flat().map((image) => [image.id, image]), + ).values(), + ].sort((a, b) => a.capturedAt - b.capturedAt), + ); + const keyOf = (tile) => `${tileKey(tile)}:${sequenceId}`; + // The home tile is needed now (it holds the open picture). + const home = await picturesIn(tiles[0], sequenceId, signal); + signal?.throwIfAborted(); + // Neighbours already read draw with it; the rest (each up to + // 1.4 MB in central Paris) join the cones as they arrive. + const neighbours = tiles.slice(1); + const kept = neighbours + .map((tile) => pictureLists.get(keyOf(tile))) + .filter(Boolean); + const missing = neighbours.filter( + (tile) => !pictureLists.get(keyOf(tile)), + ); + if (missing.length) + Promise.allSettled( + missing.map((tile) => picturesIn(tile, sequenceId, signal)), + ).then((lists) => { + if (signal?.aborted || drawn !== covering) return; + dropFailed(covering, missing, lists); + // Only a tile that arrived now redraws (no loop on failures). + if (lists.some((list) => list.status === 'fulfilled')) + cones.select(sequenceId, { force: true }); + }); + return conesOf([home, ...kept]); + }, + colors: COLORS, + pickPrefix: PICK_PREFIX.image, + spriteId: 'street-level:panoramax-cones', + renderKey: 'panoramax-sequence', + onHighlight: (sequenceId, selected) => + coverage.recolorSequence(sequenceId, selected), + }); + + /** + * A tile that failed is not covered: a picture opening in it later + * re-centres the cones there (and asks for the tile again). + */ + function dropFailed(covering, tiles, lists) { + lists.forEach((list, i) => { + if (list.status === 'rejected') + covering.tiles.delete(tileKey(tiles[i])); + }); + } + + /** + * The cones show the z15 tiles around one picture: once a picture of + * the selected sequence opens outside them (‹ ›, a cone), re-centre. + */ + function recentreCones(photo) { + if (!photo?.sequenceId) return; + if (state.sequence.selectedId !== photo.sequenceId) return; + if (drawn?.sequenceId !== photo.sequenceId) return; + if (drawn.tiles.has(tileKey(pictureTileOf(photo)))) return; + cones.select(photo.sequenceId, { force: true }); + } + + const viewer = createPanoramaxViewer({ + getPhoto, + // The cones centre on the picture actually shown, never on a + // lookup that came back after the user moved on. + onShowing: (photo) => { + if (photo.sequenceId) anchors.set(photo.sequenceId, photo); + }, + // After a GPU reset the same picture is drawn again by the viewer + // itself: through the core it would cancel a lookup the user + // started. A failure still shows. + onReopen: (photoId) => + viewer + .open(photoId) + .catch((error) => + context.actions.reportError( + error?.message || + msg('The Panoramax photo could not be loaded'), + ), + ), + step, + onStep: (photoId) => + context.actions.openImage(photoId, { frame: false }), + onShown: recentreCones, + // Esc in the viewer clears the selected sequence, as the panel says. + onEscape: () => { + if (!state.sequence.selectedId) return false; + cones.clearSelection(); + drawn = null; + return true; + }, + render: context.services?.render, + ...viewerOptions, + }); + + /** Nearest of `items` within `radius` metres of a point, or null. */ + function nearestOf(items, point, radius = NEAREST_RADIUS_M) { + let best = null; + for (const item of items) { + const photo = remember(item); + if (!photo) continue; + const metres = metresBetween(point, photo); + if (metres > radius) continue; + if (!best || metres < best.metres) best = { photo, metres }; + } + return best?.photo ?? null; + } + + return { + async status() { + return { configured: true }; + }, + + init(cesiumViewer) { + state.viewer = cesiumViewer; + cones.ensureCollections(cesiumViewer); + cones.setVisible(false); + }, + + activate(cesiumViewer) { + state.viewer = cesiumViewer; + cones.setVisible(true); + coverage.attach(cesiumViewer); + }, + + deactivate() { + coverage.detach(); + coverage.clear(); + coverage.resetErrors(); + cones.clearSelection(); + cones.setVisible(false); + drawn = null; + preferred = null; + // Off: decoded tiles and lists go (several MB each over a city). + coverageTiles.clear(); + pictureLists.clear(); + }, + + destroy(cesiumViewer) { + coverage.detach(); + coverage.clear(); + coverage.resetErrors(); + cones.destroy(cesiumViewer); + }, + + refreshCoverage: () => coverage.refresh(), + + /** The filter changed: redraw. Each draw reads it from the context. */ + setFilter() { + coverage.rebuild(); + cones.rerender(); + }, + + setSurface() { + coverage.setSurface(); + cones.rerender(); + }, + + coverageStats() { + const { kind } = state.coverage; + const lines = kind === 'sequence'; + const shown = coverage.sequenceCount(); + // Density dots are not sequences: they are not counted as such. + const count = lines ? shown : 0; + const loading = state.coverage.pending.size > 0; + const settled = !loading && !state.coverage.lastError; + let fallback = ''; + // Tiles that held something the filter hid all of. + const filteredOut = [...state.coverage.tiles.values()].some( + (tile) => tile.total > 0, + ); + if (kind === 'overview') + fallback = + shown > 0 || !settled + ? OVERVIEW_HINT + : filteredOut + ? FILTERED_HINT + : NO_COVERAGE_HINT; + else if (lines && settled && count === 0) + fallback = filteredOut ? FILTERED_HINT : NO_COVERAGE_HINT; + return { + count, + zoom: state.coverage.zoom, + kind: state.coverage.kind, + loading, + hint: state.coverage.hint || fallback, + error: state.coverage.lastError, + keyRequired: false, + keyRejected: false, + }; + }, + + /** + * A line opens its sequence's picture nearest the click (the cones + * follow once it is on screen); a cone opens its picture. + */ + handlePick(pickId, { groundPoint } = {}) { + if (pickId.startsWith(PICK_PREFIX.image)) { + context.actions.openImage(pickId.slice(PICK_PREFIX.image.length)); + return true; + } + const sequenceId = sequenceIdFromPick(pickId, PICK_PREFIX.sequence); + if (!sequenceId) return false; + const point = groundPoint?.(); + // No ground under the click (the sky beyond a line): nothing to open. + if (!point) return true; + anchors.set(sequenceId, { ...point, sequenceId }); + preferred = { + sequenceId, + lon: point.lon, + lat: point.lat, + radius: lineClickRadius(), + }; + context.actions.openNearest(point); + return true; + }, + + selectSequence: (sequenceId) => + cones.select(sequenceId, { force: true }), + clearSequence: () => { + cones.clearSelection(); + drawn = null; + }, + sequenceStats: () => ({ + selectedId: state.sequence.selectedId, + images: state.sequence.images.length, + loading: state.sequence.loading, + }), + + /** + * The nearest picture within the search radius that passes the + * imagery filter, or null. After a click on a line, that line's + * sequence is asked first. + */ + async nearestImage({ lat, lon }, { signal } = {}) { + const point = { lat, lon }; + const filter = context.getFilter(); + // Only the lookup the click itself asked for (the same point). + const wanted = + preferred && metresBetween(preferred, point) < 1 ? preferred : null; + preferred = null; + const ask = (collections, radius = NEAREST_RADIUS_M) => + source.nearestItems( + { + lat, + lon, + radius, + limit: NEAREST_LIMIT, + collections, + pano: filter.pano, + sinceMs: filter.sinceMs, + }, + { signal }, + ); + if (wanted) { + const hit = nearestOf( + await ask(wanted.sequenceId, wanted.radius), + point, + wanted.radius, + ); + if (hit) return hit.id; + } + return nearestOf(await ask(null), point)?.id ?? null; + }, + + viewer, + }; + }, + }); +} diff --git a/src/layers/streetLevel/providers/panoramax/policy.js b/src/layers/streetLevel/providers/panoramax/policy.js new file mode 100644 index 000000000..55036cb32 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/policy.js @@ -0,0 +1,322 @@ +import { COLORS as SHARED_COLORS, PROVIDER_COLORS } from '../../policy.js'; +import { msg } from '../../../../i18n/index.js'; + +export { NEAREST_RADIUS_M } from '../../policy.js'; + +/** Identity and tuning for the Panoramax street-level provider (keyless). */ +export const PANORAMAX_PROVIDER_ID = 'panoramax'; +export const PANORAMAX_NAME = 'Panoramax'; +export const PANORAMAX_LABEL = 'PANORAMAX'; + +/** + * The Panoramax meta-catalogue: one keyless, CORS-enabled STAC API over + * every federated instance (OpenStreetMap France, IGN, MapComplete…). Its + * pictures are served by the instance that holds them. + */ +export const PANORAMAX_API = 'https://api.panoramax.xyz/api'; +/** The meta-catalogue's web viewer, where the caption link goes. */ +export const PANORAMAX_VIEWER = 'https://api.panoramax.xyz/'; + +/** Stable id prefixes for picked primitives; every one starts with `pnx:`. */ +export const PICK_PREFIX = Object.freeze({ + root: 'pnx:', + sequence: 'pnx:seq:', + image: 'pnx:img:', +}); + +export const COLORS = Object.freeze({ + coverage: PROVIDER_COLORS.panoramax, + selected: SHARED_COLORS.selected, +}); + +/** + * Sequence lines come from z7–15 tiles; z10–14 are drawn below 60 km, as + * Mapillary's are (a dense tile over Paris: z10 ~2.1 MB and 8,600 + * sequences, z11 ~1.4 MB, z12 ~800 kB). Higher up, the z0–5 `grid` layer's + * density cells (0.1°, ~10 km) draw as points: lower down they would be a + * handful of dots over a whole city, which reads as no coverage. + */ +export const COVERAGE_Z14_BELOW_M = 1_800; +export const COVERAGE_Z13_BELOW_M = 5_000; +export const COVERAGE_Z12_BELOW_M = 12_000; +export const COVERAGE_Z11_BELOW_M = 30_000; +export const COVERAGE_LINES_BELOW_M = 60_000; +export const COVERAGE_TUNING = Object.freeze({ + maxTiles: 9, + overviewMaxTiles: 6, + // Each tile draws at most 1,600 lines, its longest first (as decoded, + // `keepOrder`): a dense z11 + // tile over Paris holds 3,800, and a lower cap left the city centre + // looking emptier than its suburbs. (KartaView draws as many.) + maxSequences: 14_400, + keepOrder: true, + // Points drawn at most, over every tile (each becomes ~8 ground-line + // vertices, ~1.4 kB of geometry): the simplified Paris z10 tile alone + // holds 57,000, unsimplified 750,000, which hung the tab. + maxPoints: 90_000, + lineWidthPx: 2.5, + overviewPointPx: 4, + moveDebounceMs: 320, + viewRangeMinM: 2_500, + viewRangePerHeight: 10, + viewNearM: 1_000, +}); +/** + * Lines are simplified (Douglas–Peucker) to this many tile units of a + * 4,096 extent: about a pixel where each zoom is drawn (1.6 m at z14, + * within GPS noise), and 6–10% of the points GPS tracks carry. + */ +export const SIMPLIFY_TOLERANCE_UNITS = 4; + +/** + * Coverage tiles decode in slices of this many ms, the page getting a turn + * between them (decode.js `decodeCoverageTileSliced`): under a frame. + */ +export const DECODE_SLICE_MS = 8; + +/** + * Grid cells per overview tile, at most: a z0 tile holds ~70,000 cells + * (1.9 MB), drawn as this many binned points. + */ +export const OVERVIEW_BINS_PER_AXIS = 96; + +export const RATE_LIMITED_MESSAGE = msg( + 'Panoramax is busy — coverage resumes on its own', +); +export const NO_COVERAGE_HINT = msg('no Panoramax photos in view'); +/** Lines in view, all hidden by the imagery filter. */ +export const FILTERED_HINT = msg('no Panoramax photos here match the filters'); +export const UNREADABLE_TILE_MESSAGE = msg( + 'Panoramax sent a map tile that could not be read', +); +export const GONE_MESSAGE = msg('This Panoramax photo is no longer available'); +/** From high up the dots are where pictures are, not sequences to click. */ +export const OVERVIEW_HINT = msg('zoom in to see Panoramax coverage'); + +/** Cones come from z15 tiles' `pictures` layer (~110 bytes a picture). */ +export const PICTURE_TILE_ZOOM = 15; +/** Within this of a z15 tile's edge, the neighbouring tile's pictures join. */ +export const PICTURE_TILE_EDGE_M = 150; + +export const TILE_TIMEOUT_MS = 20_000; +/** + * Once an answer has started, it may take as long as it keeps coming: a + * 2 MB tile on a slow link outlasts any whole-request timeout, so only a + * stream silent this long is given up. + */ +export const STREAM_IDLE_TIMEOUT_MS = 15_000; +/** Busy (429/503) without a Retry-After the browser can read: back off from this. */ +export const BUSY_BACKOFF_BASE_SEC = 5; +export const API_TIMEOUT_MS = 12_000; +/** Real tiles are under 6 MB (z6 over France); refuse anything absurd. */ +export const MAX_RESPONSE_BYTES = 16 * 1024 * 1024; +/** + * The meta-catalogue speaks HTTP/1.1 only, so the browser opens six + * connections to it: coverage tiles take three, searches two and the + * cones' tiles one, so a click's lookup never queues behind megabyte tiles + * (it timed out doing so). + */ +export const API_CONCURRENCY = 2; +export const TILE_CONCURRENCY = 3; +/** The selected sequence's z15 tiles (cones) take the sixth. */ +export const PICTURE_TILE_CONCURRENCY = 1; + +/** Pictures asked for around a point; the API returns them nearest first. */ +export const NEAREST_LIMIT = 10; +/** + * The widest the API allows: the point may be beside a picture, but a + * picture looking away from it is still left out (the bounding-box search + * beside it has those). + */ +export const NEAREST_FOV_TOLERANCE_DEG = 180; +/** + * Pictures in the bounding box around a click, at most. The API returns + * them roughly nearest first (the first ten of 100 are the first ten of + * 10), and the browser ranks them exactly: 20 is ~60 kB, 100 ~600 kB. + */ +export const NEAREST_BOX_LIMIT = 20; +/** + * A line clicked from high up may be tens of metres off the click: its + * sequence is searched within the camera height / 100 (50–400 m). + */ +export const LINE_CLICK_RADIUS_PER_HEIGHT = 0.01; +export const LINE_CLICK_RADIUS_MAX_M = 400; + +/** A full-resolution panorama is only fetched up to this width (memory). */ +export const PANO_HD_MAX_WIDTH = 8_192; +/** An HD panorama can be several megabytes. */ +export const HD_TIMEOUT_MS = 30_000; +/** Panoramax's SD derivative is 2,048 px wide, its thumbnail 500 px. */ +export const SD_WIDTH = 2_048; + +/** On-globe credit: licences are per picture, named in the caption. */ +export const PANORAMAX_CREDIT_HTML = + 'Street Level: imagery © Panoramax contributors, licence per picture'; + +export const LINE_CLICK_HINT = 'click a line for its nearest photo'; + +/** Licence ids Panoramax uses, as the caption names them. */ +const LICENSE_LABELS = Object.freeze({ + 'CC-BY-SA-4.0': 'CC BY-SA 4.0', + 'CC-BY-4.0': 'CC BY 4.0', + 'CC0-1.0': 'CC0', + 'etalab-2.0': 'Licence Ouverte 2.0', +}); + +/** The caption's licence name for a STAC licence id, or null. */ +export function licenseLabel(license) { + if (typeof license !== 'string' || !license.trim()) return null; + const id = license.trim(); + if (id === 'proprietary' || id === 'other') return null; + return LICENSE_LABELS[id] || id.slice(0, 40); +} + +/** Deep link to a picture in the Panoramax web viewer. */ +export function panoramaxPictureUrl(pictureId) { + const id = String(pictureId ?? '').trim(); + if (!id) return PANORAMAX_VIEWER; + return `${PANORAMAX_VIEWER}#${new URLSearchParams({ pic: id, focus: 'pic' })}`; +} + +/** Epoch ms for a capture time ('2023-12-12T07:54:47+00:00', '2023-12-12 07:56:37+00'), or null. */ +export function panoramaxTimeMs(value) { + if (typeof value !== 'string' || !value) return null; + // Tile timestamps are Postgres text: a space and an hour-only offset. + const iso = /\d[T ]\d/.test(value) + ? value + .replace(' ', 'T') + .replace(/([+-]\d{2})$/, '$1:00') + .replace(/\.(\d{3})\d+/, '.$1') + : value; + const ms = Date.parse(iso); + return Number.isFinite(ms) && ms > Date.UTC(1990, 0, 1) ? ms : null; +} + +/** An https address, or null (asset links come from federated instances). */ +function httpsUrl(value) { + try { + const url = new URL(String(value)); + return url.protocol === 'https:' ? url.href : null; + } catch { + return null; + } +} + +/** Last path segment of an item's `rel` link: the linked picture's id. */ +function linkedId(item, rel) { + const href = item?.links?.find?.((link) => link?.rel === rel)?.href; + if (typeof href !== 'string') return null; + const id = href.split(/[?#]/)[0].split('/').filter(Boolean).pop(); + if (!id) return null; + try { + return decodeURIComponent(id); + } catch { + // A malformed escape is no link, not a failed lookup. + return null; + } +} + +/** + * An email address anywhere in a name, at any of its '@'s. Linear: names + * come from any instance, and each run between '@'s is scanned once. + */ +function looksLikeEmail(name) { + return /[^\s@]@[^\s@]+\.[^\s@]/.test(name); +} + +/** + * Who took the picture. Producers are account names, and some accounts + * are named by an email address: those are not repeated in the caption. + */ +function creatorOf(item) { + const props = item.properties || {}; + const names = [ + props['geovisio:producer'], + ...(Array.isArray(item.providers) + ? item.providers + .filter((p) => p?.roles?.includes?.('producer')) + .map((p) => p.name) + : []), + ]; + // Every producer is credited (an uploading app and its photographer), + // each name once, none that is an email address. + const credited = []; + const seen = new Set(); + for (const value of names) { + if (typeof value !== 'string') continue; + const name = value.trim().slice(0, 80); + if (!name || looksLikeEmail(value)) continue; + const key = name.toLowerCase(); + if (seen.has(key)) continue; + seen.add(key); + credited.push(name); + } + return credited.length ? credited.join(', ').slice(0, 120) : null; +} + +/** The federated instance holding a picture (its `via` link's host), or null. */ +function instanceOf(item) { + const href = item?.links?.find?.((link) => link?.rel === 'via')?.href; + try { + const url = new URL(String(href)); + return url.protocol === 'https:' ? url.hostname.slice(0, 80) : null; + } catch { + return null; + } +} + +/** A 360° picture: a full-sphere field of view or an equirectangular projection. */ +export function isEquirectangular(item) { + const props = item?.properties || {}; + const orientation = props['pers:interior_orientation']; + if (Number(orientation?.field_of_view) >= 360) return true; + return props['pers:projection'] === 'equirectangular'; +} + +/** + * A STAC item (from search or a collection) in the shape the provider and + * viewer use, or null without an id or a position. + * @returns {{id: string, sequenceId: string|null, lon: number, lat: number, heading: number|null, isPano: boolean, capturedAt: number|null, creator: string|null, license: string|null, width: number|null, prevId: string|null, nextId: string|null, urls: {thumb: string|null, sd: string|null, hd: string|null}}|null} + */ +export function normalizeItem(item) { + if (!item || item.id == null) return null; + // Only numbers: Number(null) and Number('') would put a picture at 0,0. + const [lon, lat] = Array.isArray(item.geometry?.coordinates) + ? item.geometry.coordinates.map((value) => + typeof value === 'number' ? value : NaN, + ) + : [NaN, NaN]; + if (!Number.isFinite(lat) || !Number.isFinite(lon)) return null; + if (Math.abs(lat) > 90 || Math.abs(lon) > 180) return null; + const props = item.properties || {}; + const assets = item.assets || {}; + const azimuth = + props['view:azimuth'] === '' || props['view:azimuth'] == null + ? NaN + : Number(props['view:azimuth']); + const width = Number( + props['pers:interior_orientation']?.sensor_array_dimensions?.[0], + ); + return { + id: String(item.id), + sequenceId: item.collection != null ? String(item.collection) : null, + lon, + lat, + heading: Number.isFinite(azimuth) ? ((azimuth % 360) + 360) % 360 : null, + isPano: isEquirectangular(item), + capturedAt: panoramaxTimeMs(props.datetime), + creator: creatorOf(item), + // Licences ask for the source to be named: where no author can be + // named, the caption names the instance holding the picture. + via: instanceOf(item), + license: licenseLabel(props.license), + width: Number.isFinite(width) && width > 0 ? width : null, + prevId: linkedId(item, 'prev'), + nextId: linkedId(item, 'next'), + urls: { + thumb: httpsUrl(assets.thumb?.href), + sd: httpsUrl(assets.sd?.href), + hd: httpsUrl(assets.hd?.href), + }, + }; +} diff --git a/src/layers/streetLevel/providers/panoramax/policy.test.mjs b/src/layers/streetLevel/providers/panoramax/policy.test.mjs new file mode 100644 index 000000000..1bb1ea578 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/policy.test.mjs @@ -0,0 +1,281 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + COLORS, + PICK_PREFIX, + isEquirectangular, + licenseLabel, + normalizeItem, + panoramaxPictureUrl, + panoramaxTimeMs, +} from './policy.js'; +import { PROVIDER_COLORS } from '../../policy.js'; +import { stacItem } from '../../../../testSupport/panoramaxFixtures.mjs'; + +test('Panoramax draws in its own orange, and every pick id starts pnx:', () => { + assert.equal(COLORS.coverage, '#ff6f00'); + assert.equal(PROVIDER_COLORS.panoramax, '#ff6f00'); + for (const prefix of Object.values(PICK_PREFIX)) + assert.ok(prefix.startsWith('pnx:')); + // One colour per source. + assert.equal( + new Set(Object.values(PROVIDER_COLORS)).size, + Object.keys(PROVIDER_COLORS).length, + ); +}); + +test('capture times parse in the API’s ISO form, the tiles’ Postgres form, and as bare days', () => { + assert.equal( + panoramaxTimeMs('2023-12-12T07:54:47+00:00'), + Date.UTC(2023, 11, 12, 7, 54, 47), + ); + assert.equal( + panoramaxTimeMs('2023-12-12 07:56:37+00'), + Date.UTC(2023, 11, 12, 7, 56, 37), + ); + assert.equal( + panoramaxTimeMs('2025-08-13 07:45:51.98+00'), + Date.UTC(2025, 7, 13, 7, 45, 51, 980), + ); + // Microseconds are cut to what Date.parse reads. + assert.equal( + panoramaxTimeMs('2025-08-03T14:38:51.838000+00:00'), + Date.UTC(2025, 7, 3, 14, 38, 51, 838), + ); + // A sequence's day: its trailing "-11" is the day, not a UTC offset. + assert.equal(panoramaxTimeMs('2025-08-11'), Date.UTC(2025, 7, 11)); + assert.equal( + panoramaxTimeMs('2025-08-13 09:00:00+02'), + Date.UTC(2025, 7, 13, 7), + ); + for (const bad of ['', 'junk', '1970-01-01', null, 42]) + assert.equal(panoramaxTimeMs(bad), null, String(bad)); +}); + +test('a STAC item becomes a picture: position, heading, 360°, licence, links and assets', () => { + const photo = normalizeItem( + stacItem({ + id: 'a', + lon: 2.35, + lat: 48.85, + sequenceId: 's', + heading: 370, + pano: true, + license: 'etalab-2.0', + prevId: 'p', + nextId: 'n', + }), + ); + assert.deepEqual( + { + id: photo.id, + sequenceId: photo.sequenceId, + lon: photo.lon, + lat: photo.lat, + heading: photo.heading, + isPano: photo.isPano, + license: photo.license, + width: photo.width, + prevId: photo.prevId, + nextId: photo.nextId, + creator: photo.creator, + }, + { + id: 'a', + sequenceId: 's', + lon: 2.35, + lat: 48.85, + heading: 10, + isPano: true, + license: 'Licence Ouverte 2.0', + width: 5760, + prevId: 'p', + nextId: 'n', + creator: 'tester', + }, + ); + assert.equal( + photo.urls.sd, + 'https://panoramax.example.org/pictures/a/sd.jpg', + ); + assert.equal(photo.capturedAt, Date.UTC(2025, 5, 1, 10)); +}); + +test('a picture without links, heading or a usable position', () => { + const bare = normalizeItem({ + id: 7, + geometry: { coordinates: [1, 2] }, + properties: {}, + }); + assert.equal(bare.id, '7'); + assert.equal(bare.heading, null); + assert.equal(bare.prevId, null); + assert.equal(bare.nextId, null); + assert.equal(bare.sequenceId, null); + assert.equal(bare.isPano, false); + assert.equal(bare.capturedAt, null); + assert.deepEqual(bare.urls, { thumb: null, sd: null, hd: null }); + for (const item of [ + null, + { geometry: { coordinates: [1, 2] } }, + { id: 'x', geometry: { coordinates: ['a', 2] } }, + { id: 'x', geometry: { coordinates: [181, 2] } }, + { id: 'x', geometry: { coordinates: [1, 91] } }, + { id: 'x' }, + ]) + assert.equal(normalizeItem(item), null, JSON.stringify(item)); +}); + +test('asset links that are not https are dropped', () => { + const item = stacItem({ id: 'a', lon: 1, lat: 2 }); + item.assets.sd.href = 'http://insecure.example/sd.jpg'; + item.assets.hd.href = 'javascript:alert(1)'; + const photo = normalizeItem(item); + assert.equal(photo.urls.sd, null); + assert.equal(photo.urls.hd, null); + assert.ok(photo.urls.thumb.startsWith('https://')); +}); + +test('an account named by an email address is not shown as the creator', () => { + const named = (producer) => + normalizeItem(stacItem({ id: 'a', lon: 1, lat: 2, producer })).creator; + assert.equal(named('someone@city.example.fr'), null); + assert.equal(named('Robot8A'), 'Robot8A'); + assert.equal(named(' padded '), 'padded'); + assert.equal(named('x'.repeat(200)).length, 80); + // The producer role among the providers serves when the property is missing. + const item = stacItem({ id: 'a', lon: 1, lat: 2 }); + delete item.properties['geovisio:producer']; + item.providers = [ + { name: 'host', roles: ['host'] }, + { name: 'mapper', roles: ['producer'] }, + ]; + assert.equal(normalizeItem(item).creator, 'mapper'); +}); + +test('360°: a full field of view, or the equirectangular projection', () => { + assert.equal( + isEquirectangular({ + properties: { 'pers:interior_orientation': { field_of_view: 360 } }, + }), + true, + ); + assert.equal( + isEquirectangular({ properties: { 'pers:projection': 'equirectangular' } }), + true, + ); + assert.equal( + isEquirectangular({ + properties: { 'pers:interior_orientation': { field_of_view: 92 } }, + }), + false, + ); + assert.equal(isEquirectangular(null), false); +}); + +test('licences read as the caption names them', () => { + assert.equal(licenseLabel('CC-BY-SA-4.0'), 'CC BY-SA 4.0'); + assert.equal(licenseLabel('etalab-2.0'), 'Licence Ouverte 2.0'); + assert.equal(licenseLabel('CC0-1.0'), 'CC0'); + assert.equal(licenseLabel('ODbL-1.0'), 'ODbL-1.0'); + assert.equal(licenseLabel('proprietary'), null); + assert.equal(licenseLabel(''), null); + assert.equal(licenseLabel(undefined), null); + assert.equal(licenseLabel('x'.repeat(100)).length, 40); +}); + +test('the caption link opens the picture in the Panoramax viewer', () => { + assert.equal( + panoramaxPictureUrl('d5467bbb-4357'), + 'https://api.panoramax.xyz/#pic=d5467bbb-4357&focus=pic', + ); + assert.equal(panoramaxPictureUrl(''), 'https://api.panoramax.xyz/'); + assert.equal(panoramaxPictureUrl(null), 'https://api.panoramax.xyz/'); + assert.ok(panoramaxPictureUrl('a&b=c').includes('pic=a%26b%3Dc')); +}); + +test('hostile item data: a malformed link escape, null coordinates, a huge producer name', () => { + const item = stacItem({ id: 'a', lon: 1, lat: 2, nextId: 'n' }); + item.links.push({ + rel: 'prev', + href: 'https://api.panoramax.xyz/api/collections/s/items/%E0%A4%A', + }); + const photo = normalizeItem(item); + assert.equal(photo.prevId, null, 'no link, no throw'); + assert.equal(photo.nextId, 'n'); + for (const coordinates of [[null, null], ['', ''], ['2.35', '48.85'], [2.35]]) + assert.equal( + normalizeItem({ id: 'x', geometry: { coordinates } }), + null, + JSON.stringify(coordinates), + ); + // A million characters without spaces is checked in linear time. + const started = Date.now(); + const huge = normalizeItem( + stacItem({ id: 'a', lon: 1, lat: 2, producer: 'a'.repeat(1_000_000) }), + ); + assert.ok(Date.now() - started < 500, `${Date.now() - started} ms`); + assert.equal(huge.creator.length, 80); + const named = (producer) => + normalizeItem(stacItem({ id: 'a', lon: 1, lat: 2, producer })).creator; + assert.equal(named(`${'x'.repeat(200_000)}@example.org`), null); + assert.equal( + named('Carto@Cité'), + 'Carto@Cité', + 'an @ without a domain is a name', + ); + // The address need not follow the first '@'. + assert.equal(named('@team someone@example.org'), null); + assert.equal(named('Carto@Cité (mapper@example.org)'), null); + assert.equal(named(`${'@'.repeat(100_000)}x`), `${'@'.repeat(80)}`); +}); + +test('thousands of producers are credited in linear time', () => { + const item = stacItem({ id: 'a', lon: 1, lat: 2 }); + item.providers = Array.from({ length: 50_000 }, (_, i) => ({ + name: `producer-${i}`, + roles: ['producer'], + })); + const started = performance.now(); + const { creator } = normalizeItem(item); + assert.ok(performance.now() - started < 1000); + assert.ok(creator.startsWith('tester, producer-0, producer-1')); + assert.ok(creator.length <= 120); +}); + +test('the instance holding a picture is named (its via host), never as its author', () => { + const item = stacItem({ + id: 'a', + lon: 1, + lat: 2, + producer: 'someone@ign.example.fr', + }); + item.links.push({ rel: 'via', href: 'https://panoramax.ign.fr' }); + assert.equal(normalizeItem(item).creator, null); + assert.equal(normalizeItem(item).via, 'panoramax.ign.fr'); + item.links.at(-1).href = 'http://insecure.example'; + assert.equal(normalizeItem(item).via, null); + item.links.at(-1).href = 'not a url'; + assert.equal(normalizeItem(item).via, null); +}); + +test('every producer is credited once: the uploading app and its photographer', () => { + const item = stacItem({ id: 'a', lon: 1, lat: 2, producer: 'mapcomplete' }); + item.providers = [ + { name: 'mapcomplete', roles: ['producer'] }, + { name: 'pyrog', roles: ['producer'] }, + { name: 'MapComplete', roles: ['producer'] }, + { name: 'someone@example.org', roles: ['producer'] }, + { name: 'host', roles: ['host'] }, + ]; + assert.equal(normalizeItem(item).creator, 'mapcomplete, pyrog'); + const hidden = stacItem({ id: 'b', lon: 1, lat: 2, producer: 'a@b.org' }); + hidden.providers = [{ name: 'c@d.org', roles: ['producer'] }]; + assert.equal(normalizeItem(hidden).creator, null); + const many = stacItem({ id: 'c', lon: 1, lat: 2 }); + many.providers = Array.from({ length: 50 }, (_, i) => ({ + name: `producer-${i}`, + roles: ['producer'], + })); + assert.ok(normalizeItem(many).creator.length <= 120); +}); diff --git a/src/layers/streetLevel/providers/panoramax/provider.test.mjs b/src/layers/streetLevel/providers/panoramax/provider.test.mjs new file mode 100644 index 000000000..4bee380ae --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/provider.test.mjs @@ -0,0 +1,1246 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + createPanoramaxProvider, + overviewPointSize, + panoramaxZoomFor, + passesCoverage, + pictureTilesAround, +} from './index.js'; +import { + FILTERED_HINT, + NO_COVERAGE_HINT, + OVERVIEW_HINT, + PICK_PREFIX, +} from './policy.js'; +import { validateProviders } from '../../registry.js'; +import { tileBounds } from '../../tileMath.js'; +import { fakeHost, until } from '../../../../testSupport/kartaViewFakes.mjs'; +import { + coverageViewer, + tileCentre, +} from '../../../../testSupport/streetLevelFakes.mjs'; +import { + encodePanoramaxTile, + stacItem, +} from '../../../../testSupport/panoramaxFixtures.mjs'; + +const ORIGIN = { lon: 2.3495, lat: 48.8874 }; +/** A point `metres` east of ORIGIN. */ +const east = (metres) => ({ + lon: ORIGIN.lon + metres / 73_300, + lat: ORIGIN.lat, +}); + +/** + * A sequence `seq` of `count` pictures 5 m apart heading east from ORIGIN + * (`offset` m in), each linked to its neighbours. + */ +function sequence(seq, count, { offset = 0, pano = false } = {}) { + return Array.from({ length: count }, (_, i) => { + const at = east(offset + i * 5); + return stacItem({ + id: `${seq}-${i}`, + sequenceId: seq, + ...at, + pano, + prevId: i ? `${seq}-${i - 1}` : null, + nextId: i < count - 1 ? `${seq}-${i + 1}` : null, + }); + }); +} + +/** A source over `items`, answering nearest lookups by distance, recording calls. */ +function fakeSource({ items = [], tiles = () => new Uint8Array(0) } = {}) { + const calls = { nearest: [], items: [], tiles: [] }; + const metres = (a, b) => + Math.hypot((a.lon - b.lon) * 73_300, (a.lat - b.lat) * 111_200); + return { + calls, + getTile: async (tile) => { + calls.tiles.push(`${tile.z}/${tile.x}/${tile.y}`); + return tiles(tile); + }, + getItem: async (id) => { + calls.items.push(id); + return items.find((item) => item.id === id) ?? null; + }, + nearestItems: async (query) => { + calls.nearest.push(query); + const at = { lon: query.lon, lat: query.lat }; + return items + .filter( + (item) => !query.collections || item.collection === query.collections, + ) + .map((item) => ({ + item, + d: metres(at, { + lon: item.geometry.coordinates[0], + lat: item.geometry.coordinates[1], + }), + })) + .filter(({ d }) => d <= query.radius) + .sort((a, b) => a.d - b.d) + .slice(0, query.limit) + .map(({ item }) => item); + }, + }; +} + +function fakeContext({ filter = { pano: 'all', sinceMs: null } } = {}) { + const actions = { opened: [], nearest: [], errors: [], instance: null }; + return { + actions, + context: { + services: {}, + getFilter: () => filter, + isActive: () => true, + groundCaster: null, + meshSampler: null, + getSurface: () => 'draped', + notify() {}, + actions: { + openImage: async (id, options) => { + actions.opened.push({ id, options }); + // As the core does, errors included (a test may not mount a viewer). + await actions.instance?.viewer.open(id).catch((error) => { + actions.errors.push(error.message); + }); + return true; + }, + openNearest: (point) => actions.nearest.push(point), + reportError: (message) => actions.errors.push(message), + }, + }, + }; +} + +function setup(sourceOptions, contextOptions) { + const source = fakeSource(sourceOptions); + const loads = []; + const definition = createPanoramaxProvider({ + source, + viewerOptions: { + load: async (url) => { + loads.push(url); + return { src: url }; + }, + createRenderer: () => { + throw new Error('no WebGL in Node'); + }, + }, + }); + const { context, actions } = fakeContext(contextOptions); + const instance = definition.create(context); + actions.instance = instance; + return { definition, instance, source, actions, loads }; +} + +test('Panoramax registers keyless and off by default, with both filters and its own wording', () => { + const { definition } = setup(); + assert.equal(definition.id, 'panoramax'); + assert.equal(definition.label, 'PANORAMAX'); + assert.equal(definition.requiresKeyId, null); + assert.equal(definition.defaultOn, false); + assert.equal(definition.filters, undefined, 'pano and since both apply'); + assert.notEqual(definition.highlightsSequence, false, 'lines are sequences'); + assert.match(definition.credit.html, /Panoramax<\/a> contributors/); + assert.equal( + definition.coverageLabel.action, + 'click a line for its nearest photo', + ); + assert.doesNotThrow(() => validateProviders([definition])); +}); + +test('lines at z10–14 below 60 km, as Mapillary’s; density points from z5 down to z0 above', () => { + assert.deepEqual(panoramaxZoomFor(500), { zoom: 14, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(1_800), { zoom: 14, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(4_000), { zoom: 13, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(12_000), { zoom: 12, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(25_000), { zoom: 11, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(30_000), { zoom: 11, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(30_001), { zoom: 10, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(60_000), { zoom: 10, kind: 'sequence' }); + assert.deepEqual(panoramaxZoomFor(60_001), { zoom: 5, kind: 'overview' }); + assert.deepEqual(panoramaxZoomFor(100_000), { zoom: 5, kind: 'overview' }); + assert.deepEqual(panoramaxZoomFor(2_000_000), { zoom: 3, kind: 'overview' }); + assert.deepEqual(panoramaxZoomFor(20_000_000), { zoom: 0, kind: 'overview' }); + assert.equal(panoramaxZoomFor(NaN), null); +}); + +test('a line with no camera type or no day passes the filters: a click finds out', () => { + const filter = { pano: 'pano', sinceMs: Date.UTC(2024, 0, 1) }; + assert.equal( + passesCoverage({ isPano: null, capturedAt: Date.UTC(2025, 0, 1) }, filter), + true, + ); + assert.equal(passesCoverage({ isPano: true, capturedAt: 0 }, filter), true); + assert.equal(passesCoverage({ isPano: false, capturedAt: 0 }, filter), false); + assert.equal( + passesCoverage({ isPano: null, capturedAt: Date.UTC(2020, 0, 1) }, filter), + false, + ); +}); + +test('the imagery filter: lines by type and day, density cells by what they hold', () => { + const old = Date.UTC(2020, 0, 1); + const since = { pano: 'all', sinceMs: Date.UTC(2024, 0, 1) }; + assert.equal( + passesCoverage({ isPano: false, capturedAt: old }, since), + false, + ); + assert.equal( + passesCoverage( + { isPano: true, capturedAt: Date.UTC(2025, 0, 1) }, + { + pano: 'pano', + sinceMs: null, + }, + ), + true, + ); + const cell = { hasPano: false, hasFlat: true, capturedAt: null }; + assert.equal(passesCoverage(cell, { pano: 'pano', sinceMs: null }), false); + assert.equal(passesCoverage(cell, { pano: 'flat', sinceMs: null }), true); + assert.equal(passesCoverage(cell, since), true, 'cells are undated: kept'); + assert.equal(passesCoverage(cell, null), true); +}); + +test('cones read the z15 tile around a picture, and its neighbours near an edge', () => { + const home = tileCentre(ORIGIN.lon, ORIGIN.lat, 15); + assert.equal(pictureTilesAround(home).length, 1, 'mid-tile: one tile'); + const box = tileBounds(home.tile.x, home.tile.y, 15); + const corner = { lon: box.west + 0.0002, lat: box.north - 0.0002 }; + const tiles = pictureTilesAround(corner).map((t) => `${t.x}/${t.y}`); + const { x, y } = home.tile; + assert.deepEqual(tiles, [ + `${x}/${y}`, + `${x - 1}/${y}`, + `${x}/${y - 1}`, + `${x - 1}/${y - 1}`, + ]); + // Across the antimeridian the column wraps. + const dateLine = pictureTilesAround({ lon: -179.9999, lat: 0.5 }); + assert.equal(dateLine[1].x, 2 ** 15 - 1); +}); + +test('the nearest picture within 50 m that passes the filter; filters go to the API', async () => { + const items = [ + ...sequence('far', 1, { offset: 80 }), + ...sequence('a', 3, { offset: 10 }), + ]; + const { instance, source } = setup( + { items }, + { filter: { pano: 'pano', sinceMs: 123 } }, + ); + const id = await instance.nearestImage(east(0)); + assert.equal(id, 'a-0'); + assert.equal(source.calls.nearest.length, 1); + assert.equal(source.calls.nearest[0].pano, 'pano'); + assert.equal(source.calls.nearest[0].sinceMs, 123); + assert.equal(source.calls.nearest[0].collections, null); + assert.equal(await instance.nearestImage(east(300)), null, 'none near'); +}); + +test('a click on a line opens that sequence’s picture nearest the click, not another’s', async () => { + const items = [ + ...sequence('other', 2, { offset: 0 }), + ...sequence('clicked', 4, { offset: 8 }), + ]; + const { instance, actions, source } = setup({ items }); + assert.equal( + instance.handlePick(`${PICK_PREFIX.sequence}clicked~1`, { + groundPoint: () => east(1), + }), + true, + ); + assert.deepEqual(actions.nearest, [east(1)]); + // The core asks the provider for the nearest picture to that point. + assert.equal(await instance.nearestImage(east(1)), 'clicked-0'); + assert.equal(source.calls.nearest[0].collections, 'clicked'); + // Only once: the next lookup (OPEN, another click) is any sequence's. + assert.equal(await instance.nearestImage(east(1)), 'other-0'); +}); + +test('a line click whose sequence has nothing near falls back to any picture there', async () => { + const items = [ + ...sequence('near', 1), + ...sequence('clicked', 1, { offset: 400 }), + ]; + const { instance, source } = setup({ items }); + instance.handlePick(`${PICK_PREFIX.sequence}clicked`, { + groundPoint: () => east(0), + }); + assert.equal(await instance.nearestImage(east(0)), 'near-0'); + assert.deepEqual( + source.calls.nearest.map((q) => q.collections), + ['clicked', null], + ); +}); + +test('a cone opens its picture; foreign ids are not picked', () => { + const { instance, actions } = setup(); + assert.equal(instance.handlePick(`${PICK_PREFIX.image}abc`), true); + assert.equal(actions.opened[0].id, 'abc'); + assert.equal(instance.handlePick('kv:line:1'), false); + assert.equal(instance.handlePick('pnx:other'), false); +}); + +/** A provider and its viewer mounted, opening pictures through the fake core. */ +async function mounted(sourceOptions, contextOptions) { + const env = setup(sourceOptions, contextOptions); + const poses = []; + env.instance.viewer.onPose((pose) => poses.push(pose.imageId)); + const host = fakeHost(); + await env.instance.viewer.mount(host); + const [, , prev, next] = host.children[0].children; + return { ...env, poses, prev, next }; +} + +test('› and ‹ step along the picture’s links without moving the camera, asking only for the next picture', async () => { + const items = sequence('s', 3); + const env = await mounted({ items }); + await env.instance.viewer.open('s-0'); + assert.ok(await until(() => env.prev.disabled), 'the first picture: no ‹'); + env.next.fire('click'); + assert.ok(await until(() => env.poses.at(-1) === 's-1')); + assert.deepEqual(env.actions.opened.at(-1), { + id: 's-1', + options: { frame: false }, + }); + env.next.fire('click'); + assert.ok(await until(() => env.poses.at(-1) === 's-2')); + assert.ok(await until(() => env.next.disabled), 'the last picture: no ›'); + assert.deepEqual(env.source.calls.items, ['s-0', 's-1', 's-2']); + env.prev.fire('click'); + assert.ok(await until(() => env.poses.at(-1) === 's-1')); + assert.equal(env.source.calls.items.length, 3, 's-1 came from the cache'); +}); + +test('one lookup for a picture asked twice at once', async () => { + const items = sequence('s', 1); + const { instance, source } = setup({ items }); + let release; + const gate = new Promise((resolve) => (release = resolve)); + const getItem = source.getItem; + source.getItem = async (id) => { + await gate; + return getItem(id); + }; + const host = fakeHost(); + await instance.viewer.mount(host); + const first = instance.viewer.open('s-0'); + const second = instance.viewer.open('s-0'); + release(); + await Promise.all([first, second]); + assert.deepEqual(source.calls.items, ['s-0']); +}); + +test('a picture Panoramax does not know, or an id it refuses, fails the open with a reason', async () => { + const env = await mounted({ items: [] }); + await assert.rejects( + env.instance.viewer.open('ghost'), + /no longer available/, + ); + env.source.getItem = async () => { + throw Object.assign(new Error('Panoramax answered HTTP 400'), { + status: 400, + }); + }; + await assert.rejects( + env.instance.viewer.open('not-a-uuid'), + /no longer available/, + ); + env.source.getItem = async () => { + throw Object.assign(new Error('Panoramax is busy'), { status: 503 }); + }; + await assert.rejects(env.instance.viewer.open('busy'), /busy/); +}); + +/** Cones on a stand-in Cesium viewer; `count()` reads how many are drawn. */ +async function withCones(t, env) { + const Cesium = await import('cesium'); + // The cone glyph draws on a canvas: any drawing call is fine. + const drawing = new Proxy({}, { get: () => () => {}, set: () => true }); + const saved = globalThis.document; + globalThis.document = { + createElement: () => ({ getContext: () => drawing }), + }; + t.after(() => (globalThis.document = saved)); + let collection = null; + env.instance.init({ + scene: { + frameState: { mode: Cesium.SceneMode.SCENE3D }, + updateHeight: () => () => {}, + getHeight: () => undefined, + primitives: { add: (c) => (collection = c), remove() {} }, + preRender: { addEventListener: () => () => {} }, + }, + camera: { positionWC: new Cesium.Cartesian3(0, 0, 1e8) }, + }); + return { + count: () => collection?.length ?? 0, + ids: () => + Array.from({ length: collection?.length ?? 0 }, (_, i) => + String(collection.get(i).id).replace(PICK_PREFIX.image, ''), + ), + }; +} + +/** z15 tiles holding every item of `items` as a picture. */ +const pictureTiles = (items) => (tile) => { + if (tile.z !== 15) return new Uint8Array(0); + const box = tileBounds(tile.x, tile.y, tile.z); + const inside = items.filter( + ({ + geometry: { + coordinates: [lon, lat], + }, + }) => + lon >= box.west && lon < box.east && lat <= box.north && lat > box.south, + ); + return encodePanoramaxTile(tile, { + pictures: inside.map((item) => ({ + id: item.id, + lon: item.geometry.coordinates[0], + lat: item.geometry.coordinates[1], + sequenceId: item.collection, + ts: item.properties.datetime.replace('T', ' ').replace('+00:00', '+00'), + })), + }); +}; + +test('once a picture is on screen its sequence’s cones load from the z15 tile around it', async (t) => { + const items = [...sequence('s', 6), ...sequence('x', 3, { offset: 2 })]; + const env = await mounted({ items, tiles: pictureTiles(items) }); + const cones = await withCones(t, env); + await env.instance.viewer.open('s-2'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + assert.equal(env.instance.sequenceStats().selectedId, 's'); + assert.equal(env.instance.sequenceStats().images, 6, 'only sequence s'); + assert.equal(cones.count(), 6); + assert.ok(env.source.calls.tiles.every((key) => key.startsWith('15/'))); +}); + +test('a click on a line with no ground under it changes nothing: the open sequence keeps its cones', async (t) => { + const items = [...sequence('x', 6), ...sequence('y', 3, { offset: 3 })]; + const env = await mounted({ items, tiles: pictureTiles(items) }); + const cones = await withCones(t, env); + await env.instance.viewer.open('x-1'); + await env.instance.selectSequence('x'); + assert.ok(await until(() => cones.count() === 6)); + const asked = env.source.calls.tiles.length; + assert.equal( + env.instance.handlePick(`${PICK_PREFIX.sequence}y`, { + groundPoint: () => null, + }), + true, + 'the click is the provider’s, even with nothing to open', + ); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(env.instance.sequenceStats().selectedId, 'x'); + assert.equal(cones.count(), 6); + assert.equal(env.source.calls.tiles.length, asked); + assert.deepEqual(env.actions.nearest, []); +}); + +test('a sequence selected with nothing to centre on draws no cones, and a picture opening in it then does not throw', async (t) => { + const items = sequence('s', 3); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + await env.instance.selectSequence('s'); + assert.equal(env.instance.sequenceStats().images, 0); + assert.deepEqual(env.source.calls.tiles, []); + // Its first picture opens: the cones load around it. + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => env.instance.sequenceStats().images === 3)); +}); + +test('a line clicked from high up looks for its sequence further off; the radius grows with height, up to 400 m', async () => { + const items = [ + ...sequence('near', 1), + ...sequence('line', 1, { offset: 220 }), + ]; + const { instance, source } = setup({ items }); + const height = (h) => + instance.init({ + camera: { positionCartographic: { height: h } }, + scene: { + primitives: { add() {}, remove() {} }, + preRender: { addEventListener: () => () => {} }, + }, + }); + const click = () => + instance.handlePick(`${PICK_PREFIX.sequence}line`, { + groundPoint: () => east(0), + }); + height(25_000); + click(); + assert.equal( + await instance.nearestImage(east(0)), + 'line-0', + '220 m off, from 25 km', + ); + assert.equal(source.calls.nearest.at(-1).radius, 250); + height(1_000); + click(); + assert.equal( + await instance.nearestImage(east(0)), + 'near-0', + 'from 1 km only 50 m: any sequence’s nearer picture', + ); + height(900_000); + click(); + await instance.nearestImage(east(0)); + assert.equal( + source.calls.nearest.filter((q) => q.collections === 'line').at(-1).radius, + 400, + 'capped', + ); +}); + +test('a lookup at another point does not take a line click’s sequence', async () => { + const items = [ + ...sequence('other', 1), + ...sequence('clicked', 1, { offset: 8 }), + ]; + const { instance, source } = setup({ items }); + instance.handlePick(`${PICK_PREFIX.sequence}clicked`, { + groundPoint: () => east(1), + }); + // OPEN at the view centre, a few metres away, overtakes the click. + assert.equal(await instance.nearestImage(east(3)), 'other-0'); + assert.deepEqual( + source.calls.nearest.map((q) => q.collections), + [null], + ); +}); + +test('the cones come in capture order, and a broken neighbour tile only thins them', async (t) => { + const items = sequence('s', 5).map((item, i) => ({ + ...item, + properties: { + ...item.properties, + // Captured in reverse of their position along the street. + datetime: `2025-06-01T10:00:0${9 - i}+00:00`, + }, + })); + const tiles = pictureTiles(items); + const env = await mounted({ items, tiles }); + const cones = await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => cones.count() === 5)); + assert.deepEqual(cones.ids(), ['s-4', 's-3', 's-2', 's-1', 's-0']); + // A neighbour answers with garbage: the home tile's cones still draw. +}); + +test('a broken neighbour z15 tile only thins the cones; an unreadable home tile reads as such', async (t) => { + // At a tile corner: the home tile and three neighbours are asked. + const home = tileCentre(ORIGIN.lon, ORIGIN.lat, 15); + const box = tileBounds(home.tile.x, home.tile.y, 15); + const at = { lon: box.west + 0.0003, lat: box.north - 0.0002 }; + const items = [0, 1, 2].map((i) => + stacItem({ + id: `c-${i}`, + sequenceId: 'c', + lon: at.lon + i * 0.00005, + lat: at.lat, + }), + ); + const tiles = pictureTiles(items); + const broken = new TextEncoder().encode('oops'); + const homeKey = `15/${home.tile.x}/${home.tile.y}`; + const env = await mounted({ + items, + tiles: (tile) => + `${tile.z}/${tile.x}/${tile.y}` === homeKey ? tiles(tile) : broken, + }); + const cones = await withCones(t, env); + await env.instance.viewer.open('c-0'); + await env.instance.selectSequence('c'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + assert.equal(cones.count(), 3, 'the home tile’s cones draw'); + assert.ok(env.source.calls.tiles.length > 1, 'neighbours were asked'); + // Another sequence whose home tile is unreadable too. + const env2 = await mounted({ items, tiles: () => broken }); + await withCones(t, env2); + await env2.instance.viewer.open('c-0'); + await env2.instance.selectSequence('c'); + assert.ok(await until(() => env2.actions.errors.some(Boolean))); + assert.match(env2.actions.errors.filter(Boolean).at(-1), /could not be read/); +}); + +test('cones re-centred into tiles already read do not read them again', async (t) => { + const items = sequence('long', 400); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + await env.instance.viewer.open('long-0'); + await env.instance.selectSequence('long'); + assert.ok(await until(() => env.instance.sequenceStats().images > 0)); + await env.instance.viewer.open('long-399'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + const asked = env.source.calls.tiles.length; + // Back to the start: its tiles are kept. + await env.instance.viewer.open('long-0'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + await new Promise((resolve) => setTimeout(resolve, 10)); + assert.equal(env.source.calls.tiles.length, asked); + assert.ok(env.instance.sequenceStats().images > 0); +}); + +test('› within the tiles the cones came from reloads nothing', async (t) => { + const items = sequence('s', 4); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => env.instance.sequenceStats().images === 4)); + const asked = env.source.calls.tiles.length; + env.next.fire('click'); + assert.ok(await until(() => env.poses.at(-1) === 's-1')); + await new Promise((resolve) => setTimeout(resolve, 10)); + assert.equal(env.source.calls.tiles.length, asked); +}); + +test('the cones re-centre when › walks into another z15 tile', async (t) => { + // 400 pictures 5 m apart span 2 km: more than two z15 tiles (~800 m). + const items = sequence('long', 400); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + await env.instance.viewer.open('long-0'); + await env.instance.selectSequence('long'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + const firstTiles = new Set(env.source.calls.tiles); + const first = env.instance.sequenceStats().images; + assert.ok(first > 0 && first < 400, `part of the sequence: ${first}`); + // Jump to the far end: the cones follow. + await env.instance.viewer.open('long-399'); + assert.ok( + await until(() => + env.source.calls.tiles.some((key) => !firstTiles.has(key)), + ), + 'a new tile was read', + ); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + assert.ok(env.instance.sequenceStats().images > 0); +}); + +/** The provider drawing coverage on a stand-in viewer above one z14 tile. */ +function coverageSetup( + t, + { sequences = [], grid = [], filter, height = 900 } = {}, +) { + const centre = tileCentre(ORIGIN.lon, ORIGIN.lat); + let hold = null; + const env = setup( + { + tiles: (tile) => { + const bytes = encodePanoramaxTile(tile, { sequences, grid }); + return hold ? hold.then(() => bytes) : bytes; + }, + }, + { filter }, + ); + const viewer = coverageViewer({ ...centre, height }, t); + env.instance.init(viewer); + t.after(() => env.instance.destroy(viewer)); + return { + ...env, + viewer, + centre, + holdTiles() { + let release; + hold = new Promise((resolve) => (release = resolve)); + return () => { + hold = null; + release(); + }; + }, + }; +} + +const line = (id, options = {}) => ({ + id, + parts: [ + [ + [ORIGIN.lon - 0.001, ORIGIN.lat], + [ORIGIN.lon + 0.001, ORIGIN.lat], + ], + ], + ...options, +}); + +test('sequence lines draw near the ground, and the imagery filter thins them', async (t) => { + const env = coverageSetup(t, { + sequences: [ + line('new-pano', { type: 'equirectangular', date: '2025-06-01' }), + line('old-flat', { date: '2019-06-01' }), + ], + }); + env.instance.activate(env.viewer); + assert.ok(await until(() => env.instance.coverageStats().count === 2)); + assert.equal(env.instance.coverageStats().kind, 'sequence'); + assert.equal(env.instance.coverageStats().keyRequired, false); + env.instance.deactivate(); + + const filtered = coverageSetup(t, { + filter: { pano: 'pano', sinceMs: null }, + sequences: [ + line('new-pano', { type: 'equirectangular' }), + line('old-flat'), + ], + }); + filtered.instance.activate(filtered.viewer); + assert.ok(await until(() => filtered.instance.coverageStats().count === 1)); +}); + +test('coverage hints: nothing while loading, “no pictures” once an empty view has loaded', async (t) => { + const env = coverageSetup(t); + const release = env.holdTiles(); + env.instance.activate(env.viewer); + assert.ok(await until(() => env.instance.coverageStats().loading)); + assert.equal(env.instance.coverageStats().hint, ''); + release(); + assert.ok(await until(() => !env.instance.coverageStats().loading)); + assert.equal(env.instance.coverageStats().hint, NO_COVERAGE_HINT); +}); + +test('from high up the density grid draws points, not counted as sequences, and the panel says to zoom in', async (t) => { + const env = coverageSetup(t, { + height: 200_000, + grid: [{ lon: ORIGIN.lon, lat: ORIGIN.lat, pano: 2, flat: 3 }], + }); + env.instance.activate(env.viewer); + assert.ok( + await until(() => env.instance.coverageStats().kind === 'overview'), + ); + assert.ok(await until(() => !env.instance.coverageStats().loading)); + assert.equal(env.instance.coverageStats().zoom, 5); + assert.equal(env.instance.coverageStats().count, 0, 'dots are not sequences'); + assert.equal(env.instance.coverageStats().hint, OVERVIEW_HINT); +}); + +test('switching off clears the cones and forgets a pending line click', async (t) => { + const items = sequence('s', 3); + const env = await mounted({ items, tiles: pictureTiles(items) }); + const cones = await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => cones.count() === 3), 'cones to clear'); + env.instance.handlePick(`${PICK_PREFIX.sequence}s`, { + groundPoint: () => east(0), + }); + env.instance.deactivate(); + assert.equal(env.instance.sequenceStats().selectedId, null); + assert.equal(cones.count(), 0, 'the cones are gone'); + // The click's preference is gone: a later lookup asks any sequence. + await env.instance.nearestImage(east(0)); + assert.equal(env.source.calls.nearest.at(-1).collections, null); +}); + +test('lines in view that the filter hides all of say so, not that there is nothing here', async (t) => { + const env = coverageSetup(t, { + filter: { pano: 'pano', sinceMs: null }, + sequences: [line('flat-only')], + }); + env.instance.activate(env.viewer); + assert.ok( + await until( + () => + !env.instance.coverageStats().loading && + env.instance.coverageStats().zoom !== null, + ), + ); + assert.ok( + await until(() => env.instance.coverageStats().hint === FILTERED_HINT), + ); + assert.equal(env.instance.coverageStats().count, 0); +}); + +test('from high up over empty sea the panel says there is nothing, not to zoom in', async (t) => { + const env = coverageSetup(t, { height: 200_000, grid: [] }); + env.instance.activate(env.viewer); + assert.ok( + await until(() => env.instance.coverageStats().kind === 'overview'), + ); + assert.ok( + await until(() => env.instance.coverageStats().hint === NO_COVERAGE_HINT), + ); +}); + +test('density dots grow with the pictures they stand for', () => { + assert.equal(overviewPointSize({ weight: 1 }), 2.5); + assert.equal(overviewPointSize({ weight: 100 }), 4.5); + assert.equal(overviewPointSize({ weight: 1e9 }), 8); + assert.equal(overviewPointSize({}), 2.5); + assert.ok( + overviewPointSize({ weight: 50_000 }) > + overviewPointSize({ weight: 50 }) + 2, + ); +}); + +test('Esc in the docked viewer clears the selected sequence before the panel can collapse', async (t) => { + const items = sequence('s', 3); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => env.instance.sequenceStats().images === 3)); + const root = env.next.parentNode; + const flags = { stopped: false, prevented: false }; + root.fire('keydown', { + key: 'Escape', + stopPropagation: () => (flags.stopped = true), + preventDefault: () => (flags.prevented = true), + }); + assert.equal(env.instance.sequenceStats().selectedId, null); + assert.deepEqual( + flags, + { stopped: true, prevented: true }, + 'consumed: the panel never sees it', + ); + // Nothing selected: Esc goes on to the panel. + const again = { key: 'Escape', defaultPrevented: false }; + const result = { prevented: false }; + root.fire('keydown', { + ...again, + preventDefault() { + result.prevented = true; + }, + }); + assert.equal(result.prevented, false); +}); + +test('cones near the east and south edges read those neighbours; near both edges of a narrow tile, both sides', () => { + const home = tileCentre(ORIGIN.lon, ORIGIN.lat, 15); + const box = tileBounds(home.tile.x, home.tile.y, 15); + const { x, y } = home.tile; + const keys = (point) => pictureTilesAround(point).map((t) => `${t.x}/${t.y}`); + assert.deepEqual(keys({ lon: box.east - 0.0002, lat: box.south + 0.0002 }), [ + `${x}/${y}`, + `${x + 1}/${y}`, + `${x}/${y + 1}`, + `${x + 1}/${y + 1}`, + ]); + // Svalbard: a z15 tile is ~250 m wide, so its middle is near both sides. + const north = tileCentre(15.6, 78.22, 15); + const around = keys(north); + for (const dx of [-1, 1]) + assert.ok( + around.includes(`${north.tile.x + dx}/${north.tile.y}`), + `${dx} in ${around}`, + ); + assert.equal(around[0], `${north.tile.x}/${north.tile.y}`, 'its own first'); +}); + +test('a search hit outside the radius (a bounding box’s corner) is not the nearest', async () => { + const corner = stacItem({ id: 'corner', sequenceId: 'c', ...east(65) }); + const { instance, source } = setup({ items: [corner] }); + source.nearestItems = async () => [corner]; + assert.equal(await instance.nearestImage(east(0)), null); +}); + +test('a line clicked from low down still searches its own sequence within 50 m', async () => { + const items = [ + ...sequence('other', 1, { offset: 2 }), + ...sequence('clicked', 1, { offset: 30 }), + ]; + const { instance, source } = setup({ items }); + instance.init({ + camera: { positionCartographic: { height: 300 } }, + scene: { + primitives: { add() {}, remove() {} }, + preRender: { addEventListener: () => () => {} }, + }, + }); + instance.handlePick(`${PICK_PREFIX.sequence}clicked`, { + groundPoint: () => east(0), + }); + assert.equal(await instance.nearestImage(east(0)), 'clicked-0'); + assert.equal(source.calls.nearest[0].radius, 50); +}); + +test('cones re-centre on a picture that opens while they are still loading', async (t) => { + const items = sequence('long', 400); + const tiles = pictureTiles(items); + let release = null; + const env = await mounted({ items, tiles }); + await withCones(t, env); + await env.instance.viewer.open('long-0'); + env.source.getTile = async (tile) => { + env.source.calls.tiles.push(`${tile.z}/${tile.x}/${tile.y}`); + if (release === null) await new Promise((go) => (release = go)); + return tiles(tile); + }; + const first = env.instance.selectSequence('long'); + assert.ok(await until(() => release !== null), 'the first load is held'); + // The far end opens while the cones around long-0 still load. + await env.instance.viewer.open('long-399'); + release(); + await first; + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + const [lon, lat] = items[399].geometry.coordinates; + const [far] = pictureTilesAround({ lon, lat }); + assert.ok( + env.source.calls.tiles.includes(`15/${far.x}/${far.y}`), + `the far picture's tile was read: ${env.source.calls.tiles}`, + ); +}); + +test('a filter that hides every density dot says so', async (t) => { + const env = coverageSetup(t, { + height: 200_000, + filter: { pano: 'pano', sinceMs: null }, + grid: [{ lon: ORIGIN.lon, lat: ORIGIN.lat, pano: 0, flat: 5 }], + }); + env.instance.activate(env.viewer); + assert.ok( + await until(() => env.instance.coverageStats().hint === FILTERED_HINT), + ); +}); + +test('dots are drawn at their own sizes, and the busiest is the biggest', async (t) => { + const env = coverageSetup(t, { + height: 200_000, + grid: [ + { lon: ORIGIN.lon, lat: ORIGIN.lat, pano: 0, flat: 1 }, + { lon: ORIGIN.lon + 3, lat: ORIGIN.lat, pano: 30_000, flat: 20_000 }, + ], + }); + env.instance.activate(env.viewer); + const sizes = () => { + const out = []; + for (const primitive of env.viewer.scene.primitives.items) + for (let i = 0; i < (primitive.length ?? 0); i++) + if (primitive.get(i).pixelSize) out.push(primitive.get(i).pixelSize); + return out.sort((a, b) => a - b); + }; + assert.ok(await until(() => sizes().length === 2)); + assert.deepEqual(sizes(), [ + overviewPointSize({ weight: 1 }), + overviewPointSize({ weight: 50_000 }), + ]); +}); + +test('lines keep the decoded (longest-first) order under a budget of points', async () => { + const { COVERAGE_TUNING } = await import('./policy.js'); + assert.equal(COVERAGE_TUNING.keepOrder, true); + assert.ok(COVERAGE_TUNING.maxPoints > 0); +}); + +test('a coverage tile decoded a moment ago is not fetched again; switched off, the tiles are let go', async (t) => { + const env = coverageSetup(t, { sequences: [line('a')] }); + const asked = []; + const getTile = env.source.getTile; + env.source.getTile = (tile, options) => { + asked.push(`${tile.z}/${tile.x}/${tile.y}`); + return getTile(tile, options); + }; + env.instance.activate(env.viewer); + assert.ok(await until(() => env.instance.coverageStats().count === 1)); + const z14 = asked.length; + // Up a zoom band and back down: the z14 tile comes from memory. + env.viewer.view.height = 3_000; + env.instance.refreshCoverage(); + assert.ok(await until(() => env.instance.coverageStats().zoom === 13)); + assert.ok(await until(() => !env.instance.coverageStats().loading)); + const withZ13 = asked.length; + env.viewer.view.height = 900; + env.instance.refreshCoverage(); + assert.ok(await until(() => env.instance.coverageStats().zoom === 14)); + assert.ok(await until(() => env.instance.coverageStats().count === 1)); + assert.equal(asked.length, withZ13, 'nothing fetched again'); + assert.ok(z14 > 0); + // Off and on: the memory went back, so the tile is fetched again. + env.instance.deactivate(); + env.instance.activate(env.viewer); + assert.ok(await until(() => env.instance.coverageStats().count === 1)); + assert.ok(asked.length > withZ13); +}); + +test('a re-centre while the cones still load does not inherit the aborted load’s tiles', async (t) => { + const items = sequence('long', 400); + const tiles = pictureTiles(items); + const env = await mounted({ items, tiles }); + const cones = await withCones(t, env); + let hold = null; + env.source.getTile = async (tile, { signal } = {}) => { + env.source.calls.tiles.push(`${tile.z}/${tile.x}/${tile.y}`); + if (hold) await hold.promise; + signal?.throwIfAborted(); + return tiles(tile); + }; + await env.instance.viewer.open('long-0'); + const tileOf = (item) => { + const [lon, lat] = item.geometry.coordinates; + return pictureTilesAround({ lon, lat })[0]; + }; + const first = new Set( + pictureTilesAround({ + lon: items[0].geometry.coordinates[0], + lat: items[0].geometry.coordinates[1], + }).map((tile) => `${tile.x}/${tile.y}`), + ); + const out = items.findIndex((item) => { + const tile = tileOf(item); + return !first.has(`${tile.x}/${tile.y}`); + }); + hold = {}; + hold.promise = new Promise((resolve) => (hold.release = resolve)); + const loading = env.instance.selectSequence('long'); + await new Promise((resolve) => setTimeout(resolve, 5)); + // The far picture opens while the first load is held: the cones re-centre. + await env.instance.viewer.open(`long-${out}`); + await new Promise((resolve) => setTimeout(resolve, 5)); + const release = hold.release; + hold = null; + release(); + await loading; + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + // Every picture in the tiles the second load covers has its cone. + const [lon, lat] = items[out].geometry.coordinates; + const second = new Set( + pictureTilesAround({ lon, lat }).map((tile) => `${tile.x}/${tile.y}`), + ); + const covered = items.filter((item) => { + const tile = tileOf(item); + return second.has(`${tile.x}/${tile.y}`); + }); + assert.ok(covered.length > 0); + // The neighbours join as they arrive: wait for them, then every picture + // in the re-centred tiles has its cone (the shared ones too). + const coneCount = () => { + const ids = new Set(cones.ids()); + return covered.filter((item) => ids.has(item.id)).length; + }; + assert.ok( + await until(() => coneCount() === covered.length), + `${coneCount()} of ${covered.length}`, + ); +}); + +test('after a GPU reset the viewer draws the same picture itself, leaving the core’s lookups alone; a failure is reported', async () => { + const items = sequence('p', 2, { pano: true }); + let failLoads = false; + const view = { yaw: 0, pitch: 0, fov: 90 }; + const definition = createPanoramaxProvider({ + source: fakeSource({ items }), + viewerOptions: { + load: async (url) => { + if (failLoads) + throw new Error('The Panoramax photo could not be loaded'); + return { src: url, width: 2048, height: 1024 }; + }, + createRenderer: () => ({ + setImage() {}, + setView: (next) => Object.assign(view, next), + getView: () => ({ ...view }), + resize() {}, + clear() {}, + destroy() {}, + }), + }, + }); + const { context, actions } = fakeContext(); + const instance = definition.create(context); + actions.instance = instance; + const host = fakeHost(); + await instance.viewer.mount(host); + const poses = []; + instance.viewer.onPose((pose) => poses.push(pose.imageId)); + await instance.viewer.open('p-0'); + const canvas = () => host.children[0].children[1]; + const kind = () => host.children[0].dataset.kind; + assert.equal(kind(), 'pano'); + canvas().fire('webglcontextlost'); + assert.ok(await until(() => kind() === 'pano'), 'drawn again'); + assert.deepEqual(poses, ['p-0', 'p-0']); + assert.deepEqual(actions.opened, [], 'not through the core'); + failLoads = true; + canvas().fire('webglcontextlost'); + assert.ok(await until(() => actions.errors.some(Boolean))); + assert.match(actions.errors.filter(Boolean)[0], /could not be loaded/); +}); + +/** A source whose lookup of `heldId` waits for `release()` and ignores aborts. */ +function holdLookup(source, heldId) { + const held = { release: null, signals: [] }; + const getItem = source.getItem; + source.getItem = async (id, options = {}) => { + held.signals.push({ id, signal: options.signal }); + if (id === heldId) await new Promise((go) => (held.release = go)); + return getItem(id); + }; + return held; +} + +test('a lookup that comes back after the user moved on does not move the cones', async (t) => { + const items = sequence('long', 400); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + const held = holdLookup(env.source, 'long-0'); + const late = env.instance.viewer.open('long-0'); + assert.ok(await until(() => held.release !== null)); + await env.instance.viewer.open('long-399'); + await late; + // The first lookup answers anyway (a source that ignores the abort). + held.release(); + await new Promise((resolve) => setTimeout(resolve, 0)); + await env.instance.selectSequence('long'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + const [lon, lat] = items[399].geometry.coordinates; + const [far] = pictureTilesAround({ lon, lat }); + const [lon0, lat0] = items[0].geometry.coordinates; + const [near] = pictureTilesAround({ lon: lon0, lat: lat0 }); + assert.equal(env.source.calls.tiles[0], `15/${far.x}/${far.y}`); + assert.ok( + !env.source.calls.tiles.includes(`15/${near.x}/${near.y}`), + 'nothing around the picture the user left', + ); +}); + +test('a lookup nobody waits for any more is stopped; one still wanted is not', async () => { + const items = sequence('s', 3); + const env = await mounted({ items }); + const held = holdLookup(env.source, 's-0'); + const first = env.instance.viewer.open('s-0'); + assert.ok(await until(() => held.release !== null)); + const [{ signal }] = held.signals; + assert.equal(signal.aborted, false); + await env.instance.viewer.open('s-1'); + await first; + await Promise.resolve(); + assert.equal(signal.aborted, true, 'the user moved on: its search stops'); + // The same picture asked again at once shares the lookup, which goes on. + held.release = null; + held.signals.length = 0; + const again = env.instance.viewer.open('s-2'); + const twice = env.instance.viewer.open('s-2'); + await Promise.all([again, twice]); + assert.equal(held.signals.length, 1); + assert.equal(held.signals[0].signal.aborted, false); + assert.equal(env.poses.at(-1), 's-2'); +}); + +/** + * Two pictures of sequence `e` either side of a z15 tile's west edge, + * half-way down it: the cones around the first read its tile and the + * west neighbour. + */ +function acrossWestEdge() { + const home = tileCentre(ORIGIN.lon, ORIGIN.lat, 15); + const box = tileBounds(home.tile.x, home.tile.y, 15); + const lat = (box.north + box.south) / 2; + const items = [ + stacItem({ + id: 'e-0', + sequenceId: 'e', + lon: box.west + 0.00005, + lat, + nextId: 'e-1', + }), + stacItem({ + id: 'e-1', + sequenceId: 'e', + lon: box.west - 0.00005, + lat, + prevId: 'e-0', + }), + ]; + const west = `15/${home.tile.x - 1}/${home.tile.y}`; + return { items, west }; +} + +test( + 'the open picture’s own tile draws its cones at once; a neighbour’s join as it arrives', + { timeout: 10_000 }, + async (t) => { + const { items, west } = acrossWestEdge(); + const tiles = pictureTiles(items); + let release = null; + const env = await mounted({ + items, + tiles: async (tile) => { + if (`${tile.z}/${tile.x}/${tile.y}` === west) + await new Promise((go) => (release = go)); + return tiles(tile); + }, + }); + const cones = await withCones(t, env); + await env.instance.viewer.open('e-0'); + const [lon, lat] = items[0].geometry.coordinates; + assert.equal( + pictureTilesAround({ lon, lat }).length, + 2, + 'the west neighbour is read too', + ); + // Not awaited: a load that waited for the neighbour would hang here. + env.instance.selectSequence('e'); + assert.ok(await until(() => release !== null), 'the neighbour is held'); + assert.ok( + await until(() => cones.count() === 1), + 'the home tile’s cone shows', + ); + assert.deepEqual(cones.ids(), ['e-0']); + release(); + assert.ok(await until(() => cones.count() === 2), 'the neighbour’s joins'); + }, +); + +test('a neighbour tile that failed is not counted as shown: › into it re-centres the cones there', async (t) => { + const { items, west } = acrossWestEdge(); + const tiles = pictureTiles(items); + let failing = true; + const env = await mounted({ + items, + tiles: (tile) => { + if (failing && `${tile.z}/${tile.x}/${tile.y}` === west) + throw Object.assign(new Error('Panoramax is busy'), { status: 503 }); + return tiles(tile); + }, + }); + const cones = await withCones(t, env); + await env.instance.viewer.open('e-0'); + await env.instance.selectSequence('e'); + assert.ok(await until(() => cones.count() === 1)); + // Let the neighbour's failure land. + assert.ok(await until(() => env.source.calls.tiles.includes(west))); + await new Promise((resolve) => setTimeout(resolve, 10)); + failing = false; + const asked = env.source.calls.tiles.filter((key) => key === west).length; + env.next.fire('click'); + assert.ok(await until(() => env.poses.at(-1) === 'e-1')); + assert.ok( + await until( + () => env.source.calls.tiles.filter((key) => key === west).length > asked, + ), + 'the failed tile is asked again', + ); + assert.ok(await until(() => cones.count() === 2), 'e-1 has its cone'); +}); + +test('a busy or timed-out tile keeps its own message; only a body that is not a tile reads as unreadable', async (t) => { + const items = sequence('s', 3); + const failures = [ + Object.assign(new Error('Panoramax is busy'), { status: 429 }), + Object.assign(new Error('Panoramax did not answer in time — try again'), { + status: 0, + }), + ]; + for (const failure of failures) { + const env = await mounted({ + items, + tiles: () => { + throw failure; + }, + }); + await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => env.actions.errors.some(Boolean))); + assert.equal(env.actions.errors.filter(Boolean).at(-1), failure.message); + } +}); diff --git a/src/layers/streetLevel/providers/panoramax/source.js b/src/layers/streetLevel/providers/panoramax/source.js new file mode 100644 index 000000000..bfdd99057 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/source.js @@ -0,0 +1,306 @@ +import { createLimiter } from '../../limiter.js'; +import { msg } from '../../../../i18n/index.js'; +import { + API_CONCURRENCY, + API_TIMEOUT_MS, + BUSY_BACKOFF_BASE_SEC, + MAX_RESPONSE_BYTES, + NEAREST_BOX_LIMIT, + NEAREST_FOV_TOLERANCE_DEG, + NEAREST_LIMIT, + NEAREST_RADIUS_M, + PANORAMAX_API, + PICTURE_TILE_CONCURRENCY, + STREAM_IDLE_TIMEOUT_MS, + TILE_CONCURRENCY, + TILE_TIMEOUT_MS, +} from './policy.js'; + +/** Longest wait a Retry-After (or the backoff) asks for. */ +const RETRY_AFTER_MAX_SEC = 300; + +/** An HTTP or API failure; `retryAfterSec` is set while Panoramax is busy. */ +export class PanoramaxSourceError extends Error { + constructor(message, { status = 0, retryAfterSec = null } = {}) { + super(message); + this.name = 'PanoramaxSourceError'; + this.status = status; + this.retryAfterSec = retryAfterSec; + } +} + +const NO_ANSWER = msg('Panoramax did not answer in time — try again'); +const UNREACHABLE = msg( + 'Panoramax could not be reached — check the connection', +); + +/** + * Seconds from a Retry-After header (delay seconds or an HTTP date), or + * null. The meta-catalogue exposes no headers to the browser, so in the + * app this is usually null and the caller backs off on its own. + */ +export function retryAfterSeconds(response, now = Date.now()) { + const header = response.headers?.get?.('retry-after'); + if (header == null || header === '') return null; + let value = Number(header); + if (!Number.isFinite(value)) { + const at = Date.parse(header); + if (!Number.isFinite(at)) return null; + value = Math.ceil((at - now) / 1000); + } + if (value < 0) return null; + // A maintenance page's hour must not freeze coverage for an hour. + return Math.min(RETRY_AFTER_MAX_SEC, Math.max(1, value)); +} + +/** + * The body as bytes, refusing more than MAX_RESPONSE_BYTES whatever the + * headers say. `onChunk` hears each piece arrive (the idle timeout). + */ +async function readBounded(response, onChunk = () => {}) { + const tooMuch = () => + new PanoramaxSourceError(msg('Panoramax sent too much data'), { + status: response.status, + }); + const length = Number(response.headers?.get?.('content-length')); + if (length > MAX_RESPONSE_BYTES) throw tooMuch(); + const reader = response.body?.getReader?.(); + if (!reader) { + const bytes = new Uint8Array(await response.arrayBuffer()); + if (bytes.byteLength > MAX_RESPONSE_BYTES) throw tooMuch(); + return bytes; + } + const chunks = []; + let total = 0; + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + onChunk(); + total += value.byteLength; + if (total > MAX_RESPONSE_BYTES) { + reader.cancel().catch(() => {}); + throw tooMuch(); + } + chunks.push(value); + } + const body = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + body.set(chunk, offset); + offset += chunk.byteLength; + } + return body; +} + +/** + * Every network path the Panoramax provider uses, browser-side and keyless: + * the meta-catalogue allows any origin. Vector tiles for coverage and + * cones, STAC search for single pictures and nearest ones. + * @param {{fetchImpl?: typeof fetch, api?: string}} [options] + */ +export function createPanoramaxSource({ + fetchImpl = (...args) => globalThis.fetch(...args), + api = PANORAMAX_API, +} = {}) { + const root = api.replace(/\/+$/, ''); + const apiLimiter = createLimiter(API_CONCURRENCY, { lifo: true }); + const tileLimiter = createLimiter(TILE_CONCURRENCY); + // The cones' z15 tiles have their own connection: a selection never + // waits behind coverage tiles. + const pictureLimiter = createLimiter(PICTURE_TILE_CONCURRENCY, { + lifo: true, + }); + + /** Busy answers in a row with no readable Retry-After (the backoff grows). */ + let busyStreak = 0; + + /** + * GET: its headers within `timeoutMs`, then a body that may take as long + * as it keeps arriving (silent for STREAM_IDLE_TIMEOUT_MS is given up). + * The caller's abort wins over both. + */ + async function get(url, { signal, timeoutMs, empty = [] }) { + const controller = new AbortController(); + const onAbort = () => controller.abort(signal.reason); + if (signal?.aborted) throw signal.reason; + signal?.addEventListener('abort', onAbort, { once: true }); + const giveUp = () => controller.abort(new PanoramaxSourceError(NO_ANSWER)); + let timer = setTimeout(giveUp, timeoutMs); + try { + const response = await fetchImpl(url, { signal: controller.signal }); + if (empty.includes(response.status)) { + busyStreak = 0; + return null; + } + if (response.status === 429 || response.status === 503) { + const told = retryAfterSeconds(response); + // Not told how long: back off, longer each time (5, 10, 20… 300 s). + const wait = + told ?? + Math.min( + RETRY_AFTER_MAX_SEC, + BUSY_BACKOFF_BASE_SEC * 2 ** Math.min(busyStreak, 8), + ); + busyStreak++; + throw new PanoramaxSourceError(msg('Panoramax is busy'), { + status: response.status, + retryAfterSec: wait, + }); + } + if (!response.ok) + throw new PanoramaxSourceError( + `Panoramax answered HTTP ${response.status}`, + { status: response.status }, + ); + busyStreak = 0; + // The answer has begun: from here only silence is timed. + clearTimeout(timer); + timer = setTimeout(giveUp, STREAM_IDLE_TIMEOUT_MS); + return await readBounded(response, () => { + clearTimeout(timer); + timer = setTimeout(giveUp, STREAM_IDLE_TIMEOUT_MS); + }); + } catch (error) { + if (signal?.aborted) throw signal.reason; + if (controller.signal.aborted) throw controller.signal.reason; + if (error instanceof PanoramaxSourceError) throw error; + // Offline, or a CORS or TLS failure: not a timeout, not worth a retry. + throw new PanoramaxSourceError(UNREACHABLE); + } finally { + clearTimeout(timer); + signal?.removeEventListener('abort', onAbort); + } + } + + /** + * A STAC search, through the API gate (newest request first). One that + * timed out is asked once more: a search is quick once the connection + * is free. + */ + async function search(params, { signal } = {}) { + const url = `${root}/search?${new URLSearchParams(params)}`; + const once = () => + apiLimiter.run(() => get(url, { signal, timeoutMs: API_TIMEOUT_MS }), { + signal, + }); + let bytes; + try { + bytes = await once(); + } catch (error) { + if (signal?.aborted || error?.message !== NO_ANSWER) throw error; + bytes = await once(); + } + let payload; + try { + payload = JSON.parse(new TextDecoder().decode(bytes)); + } catch { + throw new PanoramaxSourceError( + msg('Panoramax sent an unreadable answer'), + ); + } + return Array.isArray(payload?.features) ? payload.features : []; + } + + return { + api: root, + + /** Raw vector-tile bytes; an empty array where there is no coverage. */ + async getTile({ x, y, z }, { signal, lane = 'coverage' } = {}) { + const url = `${root}/map/${z}/${x}/${y}.mvt`; + const gate = lane === 'pictures' ? pictureLimiter : tileLimiter; + const bytes = await gate.run( + () => + get(url, { signal, timeoutMs: TILE_TIMEOUT_MS, empty: [204, 404] }), + { signal }, + ); + return bytes ?? new Uint8Array(0); + }, + + /** One picture (a STAC item) by id, or null. */ + async getItem(id, { signal } = {}) { + const [item] = await search({ ids: String(id), limit: '1' }, { signal }); + return item && String(item.id) === String(id) ? item : null; + }, + + /** + * Pictures within `radius` metres of a point, unranked: the caller ranks + * them by distance. Two searches side by side, since neither finds them + * all: `place_position` (nearest first) leaves out pictures looking + * away from the point, whatever the tolerance; a bounding box has them, + * roughly nearest first (not exactly: the caller ranks the union). + * `collections` keeps one sequence's, `pano` and `sinceMs` filter as + * the imagery filter does. + */ + async nearestItems( + { + lat, + lon, + radius = NEAREST_RADIUS_M, + limit = NEAREST_LIMIT, + collections = null, + pano = 'all', + sinceMs = null, + }, + { signal } = {}, + ) { + const metres = Math.min(1000, Math.max(1, Math.round(radius))); + const shared = {}; + if (collections) shared.collections = String(collections); + if (pano === 'pano') shared.filter = 'field_of_view=360'; + else if (pano === 'flat') shared.filter = 'field_of_view<360'; + if (Number.isFinite(sinceMs) && sinceMs > 0) + shared.datetime = `${new Date(sinceMs).toISOString()}/..`; + const dLat = metres / 111_320; + const dLon = + metres / (111_320 * Math.max(0.01, Math.cos((lat * Math.PI) / 180))); + // Clamped to the map: the API finds nothing in a box past ±180°. + const box = [ + Math.max(-180, lon - dLon), + Math.max(-90, lat - dLat), + Math.min(180, lon + dLon), + Math.min(90, lat + dLat), + ].map((v) => v.toFixed(7)); + // One search failing (busy, no answer) leaves the other's pictures; + // only both failing fails the lookup. The caller's abort stops both. + const both = new AbortController(); + const onAbort = () => both.abort(signal.reason); + if (signal?.aborted) throw signal.reason; + signal?.addEventListener('abort', onAbort, { once: true }); + const run = (params) => + search(params, { signal: both.signal }).catch((error) => { + if (!both.signal.aborted && !signal?.aborted) failures.push(error); + throw error; + }); + const failures = []; + let settled; + try { + settled = await Promise.allSettled([ + run({ + place_position: `${Number(lon).toFixed(7)},${Number(lat).toFixed(7)}`, + place_distance: `0-${metres}`, + place_fov_tolerance: String(NEAREST_FOV_TOLERANCE_DEG), + limit: String(Math.min(100, Math.max(1, limit))), + ...shared, + }), + run({ + bbox: box.join(','), + limit: String(NEAREST_BOX_LIMIT), + ...shared, + }), + ]); + } finally { + signal?.removeEventListener('abort', onAbort); + } + if (signal?.aborted) throw signal.reason; + const found = settled + .filter((result) => result.status === 'fulfilled') + .map((result) => result.value); + if (!found.length) throw failures[0] ?? settled[0].reason; + const byId = new Map(); + for (const item of found.flat()) + if (item?.id != null && !byId.has(String(item.id))) + byId.set(String(item.id), item); + return [...byId.values()]; + }, + }; +} diff --git a/src/layers/streetLevel/providers/panoramax/source.test.mjs b/src/layers/streetLevel/providers/panoramax/source.test.mjs new file mode 100644 index 000000000..db2b1160d --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/source.test.mjs @@ -0,0 +1,513 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + PanoramaxSourceError, + createPanoramaxSource, + retryAfterSeconds, +} from './source.js'; +import { + MAX_RESPONSE_BYTES, + NEAREST_BOX_LIMIT, + STREAM_IDLE_TIMEOUT_MS, + TILE_TIMEOUT_MS, +} from './policy.js'; + +/** A fetch answering from `respond(url)`, recording every URL asked. */ +function fakeFetch(respond) { + const urls = []; + const fetchImpl = async (url, { signal } = {}) => { + urls.push(url); + const answer = await respond(url, signal); + if (answer instanceof Response) return answer; + const { status = 200, body = '', headers = {} } = answer; + return new Response(status === 204 ? null : body, { status, headers }); + }; + return { urls, fetchImpl }; +} + +const json = (value) => ({ body: JSON.stringify(value) }); + +test('a tile is the raw vector-tile bytes; 204 and 404 mean no coverage', async () => { + const { urls, fetchImpl } = fakeFetch((url) => + url.includes('/14/') + ? { body: new Uint8Array([1, 2, 3]) } + : { status: 204 }, + ); + const source = createPanoramaxSource({ fetchImpl }); + assert.deepEqual( + [...(await source.getTile({ x: 8299, y: 5634, z: 14 }))], + [1, 2, 3], + ); + assert.equal(urls[0], 'https://api.panoramax.xyz/api/map/14/8299/5634.mvt'); + assert.equal((await source.getTile({ x: 1, y: 1, z: 13 })).length, 0); + const missing = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ status: 404 })).fetchImpl, + }); + assert.equal((await missing.getTile({ x: 1, y: 1, z: 13 })).length, 0); +}); + +test('a busy API says so and when to come back; other failures name the status', async () => { + const busy = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ + status: 429, + headers: { 'retry-after': '12' }, + })).fetchImpl, + }); + await assert.rejects(busy.getTile({ x: 0, y: 0, z: 0 }), (error) => { + assert.ok(error instanceof PanoramaxSourceError); + assert.equal(error.status, 429); + assert.equal(error.retryAfterSec, 12); + return true; + }); + const down = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ status: 503 })).fetchImpl, + }); + await assert.rejects(down.getTile({ x: 0, y: 0, z: 0 }), (error) => { + assert.equal(error.retryAfterSec, 5, 'a default wait without the header'); + return true; + }); + const broken = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ status: 500 })).fetchImpl, + }); + await assert.rejects(broken.getItem('x'), /HTTP 500/); +}); + +test('an oversized answer is refused while it streams', async () => { + let pulled = 0; + const huge = new ReadableStream({ + pull(controller) { + pulled++; + controller.enqueue(new Uint8Array(4 * 1024 * 1024)); + }, + }); + const source = createPanoramaxSource({ + fetchImpl: async () => new Response(huge, { status: 200 }), + }); + await assert.rejects(source.getTile({ x: 0, y: 0, z: 0 }), /too much data/); + assert.ok(pulled * 4 * 1024 * 1024 <= MAX_RESPONSE_BYTES + 8 * 1024 * 1024); + // A declared length over the cap is refused before reading. + const declared = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ + body: 'x', + headers: { 'content-length': String(MAX_RESPONSE_BYTES + 1) }, + })).fetchImpl, + }); + await assert.rejects(declared.getItem('a'), /too much data/); +}); + +test('a picture by id is a STAC search; another id in the answer is not taken for it', async () => { + const { urls, fetchImpl } = fakeFetch((url) => + url.includes('ids=good') + ? json({ features: [{ id: 'good' }] }) + : json({ features: [{ id: 'other' }] }), + ); + const source = createPanoramaxSource({ fetchImpl }); + assert.deepEqual(await source.getItem('good'), { id: 'good' }); + assert.equal(await source.getItem('bad'), null); + const asked = new URL(urls[0]); + assert.equal(asked.pathname, '/api/search'); + assert.equal(asked.searchParams.get('ids'), 'good'); + assert.equal(asked.searchParams.get('limit'), '1'); +}); + +test('nearest pictures: a place search and a box search side by side, both filtered as the imagery filter asks', async () => { + const { urls, fetchImpl } = fakeFetch(() => json({ features: [] })); + const source = createPanoramaxSource({ fetchImpl }); + await source.nearestItems({ lat: 48.85, lon: 2.35 }); + await source.nearestItems({ + lat: 48.85, + lon: 2.35, + collections: 'seq', + pano: 'pano', + sinceMs: Date.UTC(2025, 0, 1), + radius: 5000, + limit: 500, + }); + await source.nearestItems({ lat: 48.85, lon: 2.35, pano: 'flat' }); + const asked = urls.map((u) => new URL(u).searchParams); + assert.equal(asked.length, 6, 'two searches a lookup'); + const place = asked.filter((q) => q.has('place_position')); + const box = asked.filter((q) => q.has('bbox')); + assert.equal(place.length, 3); + assert.equal(box.length, 3); + const [plain, filtered, flat] = place; + assert.equal(plain.get('place_position'), '2.3500000,48.8500000'); + assert.equal(plain.get('place_distance'), '0-50'); + assert.equal(plain.get('place_fov_tolerance'), '180'); + assert.equal(plain.get('collections'), null); + assert.equal(plain.get('filter'), null); + assert.equal(plain.get('datetime'), null); + assert.equal(filtered.get('collections'), 'seq'); + assert.equal(filtered.get('filter'), 'field_of_view=360'); + assert.equal(filtered.get('datetime'), '2025-01-01T00:00:00.000Z/..'); + assert.equal(filtered.get('place_distance'), '0-1000', 'capped'); + assert.equal(filtered.get('limit'), '100', 'capped'); + assert.equal(flat.get('filter'), 'field_of_view<360'); + // The box spans the radius each way, and carries the same filters. + const [w, s, e, n] = box[0].get('bbox').split(',').map(Number); + assert.ok(Math.abs((n - s) * 111_320 - 100) < 1, `${n - s}°`); + assert.ok( + Math.abs((e - w) * 111_320 * Math.cos((48.85 * Math.PI) / 180) - 100) < 1, + ); + assert.equal(box[1].get('collections'), 'seq'); + assert.equal(box[1].get('filter'), 'field_of_view=360'); + assert.equal(box[1].get('datetime'), '2025-01-01T00:00:00.000Z/..'); + assert.equal(box[2].get('filter'), 'field_of_view<360'); +}); + +test('a picture both searches find is listed once', async () => { + const source = createPanoramaxSource({ + fetchImpl: fakeFetch((url) => + url.includes('bbox') + ? json({ features: [{ id: 'b' }, { id: 'a' }, { id: null }] }) + : json({ features: [{ id: 'a' }] }), + ).fetchImpl, + }); + const items = await source.nearestItems({ lat: 1, lon: 2 }); + assert.deepEqual( + items.map((i) => i.id), + ['a', 'b'], + ); +}); + +test('the newest search goes first when the API gate is full', async () => { + const waiting = []; + const order = []; + const source = createPanoramaxSource({ + fetchImpl: async (url) => { + order.push(new URL(url).searchParams.get('ids')); + await new Promise((resolve) => waiting.push(resolve)); + return new Response(JSON.stringify({ features: [] })); + }, + }); + const all = ['1', '2', '3', '4', '5', '6'].map((id) => source.getItem(id)); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(order, ['1', '2'], 'two at once'); + waiting.shift()(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(order.at(-1), '6', 'the latest waiting one next'); + let done = false; + Promise.allSettled(all).then(() => (done = true)); + while (!done) { + waiting.splice(0).forEach((go) => go()); + await new Promise((resolve) => setTimeout(resolve, 0)); + } +}); + +test('an unreadable search answer, or one with no features, is handled', async () => { + const garbled = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ body: '' })).fetchImpl, + }); + await assert.rejects(garbled.getItem('a'), /unreadable/); + const empty = createPanoramaxSource({ + fetchImpl: fakeFetch(() => json({ type: 'FeatureCollection' })).fetchImpl, + }); + assert.deepEqual(await empty.nearestItems({ lat: 0, lon: 0 }), []); +}); + +test('the caller’s abort wins; a silent API times out with a readable message', async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + const hang = (url, signal) => + new Promise((_, reject) => + signal.addEventListener('abort', () => reject(signal.reason)), + ); + const source = createPanoramaxSource({ + fetchImpl: fakeFetch(hang).fetchImpl, + }); + const controller = new AbortController(); + const asked = source.getItem('a', { signal: controller.signal }); + await Promise.resolve(); + controller.abort(new Error('overtaken')); + await assert.rejects(asked, /overtaken/); + const slow = source.getTile({ x: 0, y: 0, z: 0 }); + // Let the request start before the clock runs out. + for (let i = 0; i < 5; i++) await Promise.resolve(); + t.mock.timers.tick(60_000); + await assert.rejects(slow, /did not answer in time/); +}); + +test('a network failure reads as unreachable, not as a stack trace', async () => { + const source = createPanoramaxSource({ + fetchImpl: async () => { + throw new TypeError('Failed to fetch'); + }, + }); + await assert.rejects(source.getTile({ x: 0, y: 0, z: 0 }), (error) => { + assert.ok(error instanceof PanoramaxSourceError); + assert.match(error.message, /could not be reached/); + return true; + }); +}); + +test('a search that timed out is asked once more; a second timeout is reported', async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + let calls = 0; + const hang = (signal) => + new Promise((_, reject) => + signal.addEventListener('abort', () => reject(signal.reason)), + ); + const source = createPanoramaxSource({ + fetchImpl: async (url, { signal }) => { + calls++; + if (calls === 1) return hang(signal); + return new Response(JSON.stringify({ features: [{ id: 'a' }] })); + }, + }); + const asked = source.getItem('a'); + for (let i = 0; i < 5; i++) await Promise.resolve(); + t.mock.timers.tick(60_000); + assert.deepEqual(await asked, { id: 'a' }); + assert.equal(calls, 2); + const stuck = createPanoramaxSource({ + fetchImpl: async (url, { signal }) => hang(signal), + }); + const never = stuck.getItem('b'); + for (let round = 0; round < 2; round++) { + for (let i = 0; i < 10; i++) await Promise.resolve(); + t.mock.timers.tick(60_000); + } + await assert.rejects(never, /did not answer in time/); +}); + +test('the HTTP/1.1 API never gets more than five requests at once: three tiles, two searches', async () => { + const { API_CONCURRENCY, TILE_CONCURRENCY } = await import('./policy.js'); + assert.ok(API_CONCURRENCY + TILE_CONCURRENCY <= 5); + let open = 0; + let most = 0; + const waiting = []; + const source = createPanoramaxSource({ + fetchImpl: async (url) => { + open++; + most = Math.max(most, open); + await new Promise((resolve) => waiting.push(resolve)); + open--; + return url.includes('/map/') + ? new Response(null, { status: 204 }) + : new Response(JSON.stringify({ features: [] })); + }, + }); + const all = [ + ...Array.from({ length: 8 }, (_, i) => + source.getTile({ x: i, y: 0, z: 14 }), + ), + ...Array.from({ length: 4 }, (_, i) => source.getItem(String(i))), + ]; + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(most, 5); + let done = false; + Promise.allSettled(all).then(() => (done = true)); + while (!done) { + waiting.splice(0).forEach((go) => go()); + await new Promise((resolve) => setTimeout(resolve, 0)); + } + assert.equal(most, 5, 'never more'); +}); + +test('one of the two nearest searches failing leaves the other’s pictures; both failing fails the lookup', async () => { + const half = createPanoramaxSource({ + fetchImpl: fakeFetch((url) => + url.includes('bbox') + ? { status: 503 } + : json({ features: [{ id: 'facing' }] }), + ).fetchImpl, + }); + assert.deepEqual( + (await half.nearestItems({ lat: 1, lon: 2 })).map((i) => i.id), + ['facing'], + ); + const none = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ status: 503 })).fetchImpl, + }); + await assert.rejects(none.nearestItems({ lat: 1, lon: 2 }), /busy/); +}); + +test('only a timeout is asked again: a busy or failing API is asked once, an unreachable one says so', async () => { + for (const status of [429, 500, 503]) { + const { urls, fetchImpl } = fakeFetch(() => ({ status })); + const source = createPanoramaxSource({ fetchImpl }); + await assert.rejects(source.getItem('a')); + assert.equal(urls.length, 1, `HTTP ${status}: one fetch`); + } + let calls = 0; + const offline = createPanoramaxSource({ + fetchImpl: async () => { + calls++; + throw new TypeError('Failed to fetch'); + }, + }); + await assert.rejects(offline.getItem('a'), /could not be reached/); + assert.equal(calls, 1, 'not retried'); +}); + +test('a Retry-After is honoured up to five minutes', async () => { + const wait = async (header) => { + const source = createPanoramaxSource({ + fetchImpl: fakeFetch(() => ({ + status: 503, + headers: { 'retry-after': header }, + })).fetchImpl, + }); + return source + .getTile({ x: 0, y: 0, z: 0 }) + .catch((error) => error.retryAfterSec); + }; + assert.equal(await wait('3600'), 300); + assert.equal(await wait('99999999999'), 300); + assert.equal(await wait('0'), 1); + assert.equal(await wait('12'), 12); +}); + +test('a Retry-After may be an HTTP date', () => { + const now = Date.parse('2026-10-15T12:00:00Z'); + const response = (value) => + new Response(null, { status: 429, headers: { 'retry-after': value } }); + assert.equal( + retryAfterSeconds(response('Thu, 15 Oct 2026 12:01:00 GMT'), now), + 60, + ); + assert.equal( + retryAfterSeconds(response('Thu, 15 Oct 2026 14:00:00 GMT'), now), + 300, + ); + assert.equal( + retryAfterSeconds(response('Thu, 15 Oct 2026 11:00:00 GMT'), now), + null, + 'a date already past', + ); + assert.equal(retryAfterSeconds(response('soon'), now), null); + assert.equal( + retryAfterSeconds(new Response(null, { status: 429 }), now), + null, + ); +}); + +test('busy without a readable Retry-After (the browser hides it): the wait doubles, and resets once answered', async () => { + let busy = true; + const source = createPanoramaxSource({ + fetchImpl: fakeFetch(() => + busy ? { status: 429 } : { body: new Uint8Array([1]) }, + ).fetchImpl, + }); + const wait = () => + source.getTile({ x: 0, y: 0, z: 0 }).then( + () => 'ok', + (error) => error.retryAfterSec, + ); + const waits = []; + for (let i = 0; i < 9; i++) waits.push(await wait()); + assert.deepEqual(waits, [5, 10, 20, 40, 80, 160, 300, 300, 300]); + busy = false; + assert.equal(await wait(), 'ok'); + busy = true; + assert.equal(await wait(), 5, 'back to the first wait'); +}); + +test( + 'a body that keeps arriving is not cut off by the request timeout; one that stalls is', + { timeout: 10_000 }, + async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + const settle = async () => { + for (let i = 0; i < 20; i++) await Promise.resolve(); + }; + // Headers at once, then a chunk every 10 s for twice the tile timeout. + const chunks = Math.ceil((2 * TILE_TIMEOUT_MS) / 10_000); + let push; + const slowBody = (signal) => + new Response( + new ReadableStream({ + start(controller) { + push = (last) => { + controller.enqueue(new Uint8Array([7])); + if (last) controller.close(); + }; + signal.addEventListener('abort', () => + controller.error(signal.reason), + ); + }, + }), + { status: 200 }, + ); + const source = createPanoramaxSource({ + fetchImpl: async (url, { signal }) => slowBody(signal), + }); + const slow = source.getTile({ x: 0, y: 0, z: 0 }); + await settle(); + for (let i = 1; i <= chunks; i++) { + t.mock.timers.tick(10_000); + push(i === chunks); + await settle(); + } + assert.equal((await slow).length, chunks, 'every chunk read'); + const stalled = source.getTile({ x: 0, y: 0, z: 0 }); + await settle(); + push(false); + await settle(); + t.mock.timers.tick(STREAM_IDLE_TIMEOUT_MS); + await assert.rejects(stalled, /did not answer in time/); + }, +); + +test('the box search asks for a few pictures: the API returns them nearest first', async () => { + const { urls, fetchImpl } = fakeFetch(() => json({ features: [] })); + const source = createPanoramaxSource({ fetchImpl }); + await source.nearestItems({ lon: 2.35, lat: 48.85 }); + const box = urls + .map((url) => new URL(url)) + .find((url) => url.searchParams.has('bbox')); + assert.ok(box, 'a box search'); + assert.equal(box.searchParams.get('limit'), String(NEAREST_BOX_LIMIT)); + assert.ok(NEAREST_BOX_LIMIT <= 20); +}); + +test('the cones’ tiles have their own connection: never behind coverage tiles, the newest first', async () => { + const { PICTURE_TILE_CONCURRENCY, TILE_CONCURRENCY } = + await import('./policy.js'); + assert.equal(PICTURE_TILE_CONCURRENCY, 1); + const held = new Map(); + const asked = []; + const source = createPanoramaxSource({ + fetchImpl: (url) => { + asked.push(url.replace(/^.*\/map\//, '')); + return new Promise((resolve) => + held.set(url.replace(/^.*\/map\//, ''), () => + resolve(new Response(null, { status: 204 })), + ), + ); + }, + }); + const settle = () => new Promise((resolve) => setTimeout(resolve, 0)); + // Coverage fills its lane and queues. + const coverage = Array.from({ length: TILE_CONCURRENCY + 2 }, (_, i) => + source.getTile({ x: i, y: 0, z: 10 }), + ); + await settle(); + assert.equal(asked.length, TILE_CONCURRENCY); + // A cone tile goes at once; two more wait, the newer first. + const pictures = [1, 2, 3].map((x) => + source.getTile({ x, y: 9, z: 15 }, { lane: 'pictures' }), + ); + await settle(); + assert.deepEqual(asked.slice(TILE_CONCURRENCY), ['15/1/9.mvt']); + held.get('15/1/9.mvt')(); + await settle(); + assert.deepEqual(asked.slice(TILE_CONCURRENCY), ['15/1/9.mvt', '15/3/9.mvt']); + for (let round = 0; round < 6; round++) { + for (const go of held.values()) go(); + await settle(); + } + await Promise.all([...coverage, ...pictures]); +}); + +test('a lookup near the antimeridian keeps its box on the map', async () => { + const { urls, fetchImpl } = fakeFetch(() => json({ features: [] })); + const source = createPanoramaxSource({ fetchImpl }); + await source.nearestItems({ lat: -16, lon: 179.9999 }); + const box = urls + .map((u) => new URL(u).searchParams.get('bbox')) + .find(Boolean) + .split(',') + .map(Number); + assert.equal(box[2], 180); + assert.ok(box[0] < 180); +}); diff --git a/src/layers/streetLevel/providers/panoramax/viewer.js b/src/layers/streetLevel/providers/panoramax/viewer.js new file mode 100644 index 000000000..902e95156 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/viewer.js @@ -0,0 +1,97 @@ +import { + PANO_FULL_SIZE, + createPhotoViewer, + loadImage, +} from '../../photoViewer.js'; +import { + HD_TIMEOUT_MS, + PANORAMAX_NAME, + PANORAMAX_PROVIDER_ID, + PANO_HD_MAX_WIDTH, + SD_WIDTH, + panoramaxPictureUrl, +} from './policy.js'; + +/** An HD original small enough to decode (its width is known and under the cap). */ +function hdDecodable(photo) { + return Boolean( + photo.urls.hd && photo.width && photo.width <= PANO_HD_MAX_WIDTH, + ); +} + +/** + * Where Panoramax pictures come from: the instance holding each one serves + * a 2,048 px SD derivative (equirectangular for a 360° picture) and the HD + * original. SD shows first; HD only once the viewer is wide enough to use + * it. A 360° original is shrunk to the texture size while it decodes. + * @type {import('../../photoViewer.js').PhotoSources} + */ +export const PANORAMAX_SOURCES = Object.freeze({ + hasPano: (photo) => + Boolean(photo.isPano && (photo.urls.sd || hdDecodable(photo))), + // Without an SD derivative the original goes straight to the texture size. + panoFirst: (photo) => { + // Decoded at SD's own size, whatever the host actually serves. + if (photo.urls.sd) + return { + url: photo.urls.sd, + width: SD_WIDTH, + resize: [SD_WIDTH, SD_WIDTH / 2], + }; + if (!hdDecodable(photo)) return null; + const width = Math.min(photo.width, PANO_FULL_SIZE[0]); + return { url: photo.urls.hd, width, resize: [width, width / 2] }; + }, + // The original is several megabytes with no server to shrink it: only + // for the expanded viewer (a docked one on a dense screen asks too). + panoSharp: (photo, size, { expanded = false } = {}) => { + const width = photo.width || 0; + if (!expanded || !photo.urls.hd || !photo.urls.sd) return null; + // An original no wider than the SD one adds nothing; a huge one would + // take hundreds of megabytes to decode. + if (width <= SD_WIDTH || width > PANO_HD_MAX_WIDTH) return null; + const target = Math.min(size[0], width); + return { + url: photo.urls.hd, + resize: [target, target / 2], + timeoutMs: HD_TIMEOUT_MS, + }; + }, + // A 360° picture shown flat (no WebGL) is its whole equirectangular image. + // When SD fails the thumbnail stands in; the original (megabytes) only + // for the expanded viewer, and only one of a size it may decode. + flatFirst: (photo, { expanded = false } = {}) => [ + photo.urls.sd, + expanded && hdDecodable(photo) ? photo.urls.hd : null, + photo.urls.thumb, + ], + // The original once the viewer is wider than SD, if its width is known + // and decodable (a 200 MP phone photo is 16,000 px across). + flatSharp: (photo, pixels) => + photo.urls.sd && hdDecodable(photo) && pixels > SD_WIDTH + ? photo.urls.hd + : null, + describe: (photo) => ({ + // Where no author can be named, the instance holding the photo is. + title: !photo.creator && photo.via ? `via ${photo.via}` : null, + creator: photo.creator, + license: photo.license, + externalUrl: panoramaxPictureUrl(photo.id), + }), +}); + +/** + * Viewer adapter for Panoramax pictures: flat in an , 360° on the + * shared WebGL panorama, ‹ › along the sequence (photoViewer.js). + * @param {object} options See photoViewer.js `createPhotoViewer`. + * @returns {import('../../registry.js').ViewerAdapter} + */ +export function createPanoramaxViewer({ load = loadImage, ...options }) { + return createPhotoViewer({ + providerId: PANORAMAX_PROVIDER_ID, + name: PANORAMAX_NAME, + sources: PANORAMAX_SOURCES, + load, + ...options, + }); +} diff --git a/src/layers/streetLevel/providers/panoramax/viewer.test.mjs b/src/layers/streetLevel/providers/panoramax/viewer.test.mjs new file mode 100644 index 000000000..3f8ae4789 --- /dev/null +++ b/src/layers/streetLevel/providers/panoramax/viewer.test.mjs @@ -0,0 +1,341 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { PANORAMAX_SOURCES, createPanoramaxViewer } from './viewer.js'; +import { normalizeItem } from './policy.js'; +import { fakeHost, until } from '../../../../testSupport/kartaViewFakes.mjs'; +import { stacItem } from '../../../../testSupport/panoramaxFixtures.mjs'; + +const photo = (options) => + normalizeItem(stacItem({ lon: 2.35, lat: 48.85, ...options })); + +const PHOTOS = { + flat: photo({ id: 'flat', prevId: 'p0', nextId: 'flat2', heading: 30 }), + flat2: photo({ id: 'flat2', prevId: 'flat', heading: 40 }), + pano: photo({ id: 'pano', pano: true, heading: 100, width: 5760 }), + bigPano: photo({ id: 'bigPano', pano: true, width: 11_000 }), + smallPano: photo({ id: 'smallPano', pano: true, width: 2048 }), + email: photo({ id: 'email', producer: 'someone@city.example.fr' }), +}; + +function fakeRenderer() { + const view = { yaw: 0, pitch: 0, fov: 80 }; + const images = []; + return { + images, + setImage: (image) => images.push(image), + setView(next) { + Object.assign(view, next); + }, + getView: () => ({ ...view }), + resize() {}, + clear() {}, + destroy() {}, + }; +} + +function setup() { + const loads = []; + const renderers = []; + const stepped = []; + const viewer = createPanoramaxViewer({ + getPhoto: async (id) => { + if (!PHOTOS[id]) throw new Error('no such picture'); + return PHOTOS[id]; + }, + step: async (from, delta) => (delta < 0 ? from.prevId : from.nextId), + onStep: (id) => { + stepped.push(id); + viewer.open(id).catch(() => {}); + }, + load: async (url, options) => { + loads.push({ url, ...options }); + return { src: url, width: 1, height: 1 }; + }, + createRenderer: () => { + const renderer = fakeRenderer(); + renderers.push(renderer); + return renderer; + }, + }); + const poses = []; + viewer.onPose((pose) => poses.push(pose)); + return { viewer, poses, loads, renderers, stepped, host: fakeHost() }; +} + +const parts = (host) => { + const root = host.children[0]; + const [picture, canvas, prev, next] = root.children; + return { root, picture, canvas, prev, next }; +}; + +test('a flat picture shows its SD version with author, licence and a Panoramax link', async () => { + const { viewer, poses, loads, host } = setup(); + await viewer.mount(host); + const { root, picture, canvas } = parts(host); + assert.equal(root.dataset.provider, 'panoramax', 'its colour in the CSS'); + assert.match(root.attributes['aria-label'], /^Panoramax photo/); + await viewer.open('flat'); + assert.equal(picture.hidden, false); + assert.equal(canvas.hidden, true); + assert.equal( + picture.src, + 'https://panoramax.example.org/pictures/flat/sd.jpg', + ); + assert.deepEqual( + loads.map((l) => l.url), + [PHOTOS.flat.urls.sd], + ); + const pose = poses.at(-1); + assert.equal(pose.providerId, 'panoramax'); + assert.equal(pose.bearing, 30); + assert.equal(pose.isPano, false); + assert.equal(pose.creator, 'tester'); + assert.equal(pose.license, 'CC BY-SA 4.0'); + assert.equal(pose.sequenceId, 'seq-1'); + assert.equal( + pose.externalUrl, + 'https://api.panoramax.xyz/#pic=flat&focus=pic', + ); + assert.equal(viewer.supportsRenderModes(), true); +}); + +test('no creator in the caption for an account named by an email address', async () => { + const { viewer, poses, host } = setup(); + await viewer.mount(host); + await viewer.open('email'); + assert.equal(poses.at(-1).creator, null); +}); + +test('HD replaces SD only once the viewer is wide enough to use it', async (t) => { + const { viewer, loads, host } = setup(); + await viewer.mount(host); + const { picture, root } = parts(host); + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + picture.clientWidth = 400; + await viewer.open('flat'); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(loads.length, 1, 'docked: SD is sharp enough'); + // EXPAND on a wide screen. + picture.clientWidth = root.clientWidth = 2600; + viewer.resize(); + assert.ok(await until(() => picture.src === PHOTOS.flat.urls.hd)); + viewer.resize(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal( + loads.filter((l) => l.url === PHOTOS.flat.urls.hd).length, + 1, + 'asked once', + ); +}); + +test('a 360° picture shows its SD equirectangular image, then HD shrunk to 4,096 when expanded', async (t) => { + const { viewer, poses, loads, renderers, host } = setup(); + await viewer.mount(host); + const { canvas, picture } = parts(host); + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + canvas.clientWidth = 400; + await viewer.open('pano'); + assert.equal(canvas.hidden, false); + assert.equal(picture.hidden, true); + assert.equal(viewer.supportsRenderModes(), false, 'no frame to fit'); + const { signal, ...first } = loads[0]; + assert.deepEqual(first, { + url: PHOTOS.pano.urls.sd, + crossOrigin: true, + resize: [2048, 1024], + failure: 'The Panoramax photo could not be loaded', + }); + assert.equal(signal.aborted, false, 'the open can stop it'); + assert.equal(poses.at(-1).isPano, true); + assert.equal(poses.at(-1).bearing, 100); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(loads.length, 1, 'docked: the 2,048 px SD is enough'); + // Docked on a dense screen the canvas wants more, but the original is + // megabytes: not until the viewer is expanded. + canvas.clientWidth = 1800; + viewer.resize(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(loads.length, 1, 'docked: still SD only'); + const root = canvas.parentNode; + root.closest = (selector) => + selector.includes('sl-viewer-wrap-expanded') ? {} : null; + // EXPAND resizes more than once while the panel settles. + viewer.resize(); + viewer.resize(); + viewer.resize(); + assert.ok(await until(() => renderers[0].images.length === 2)); + assert.equal( + loads.filter((l) => l.url === PHOTOS.pano.urls.hd).length, + 1, + 'the original is fetched once', + ); + const hd = loads.at(-1); + assert.equal(hd.url, PHOTOS.pano.urls.hd); + assert.deepEqual(hd.resize, [4096, 2048]); + assert.equal(hd.timeoutMs, 30_000); + assert.ok(hd.signal, 'abortable'); +}); + +test('HD panoramas too big to decode, or no bigger than SD, are never fetched; nor while docked', () => { + const size = [4096, 2048]; + const expanded = { expanded: true }; + assert.equal( + PANORAMAX_SOURCES.panoSharp(PHOTOS.pano, size, { expanded: false }), + null, + ); + assert.equal(PANORAMAX_SOURCES.panoSharp(PHOTOS.pano, size), null); + assert.equal( + PANORAMAX_SOURCES.panoSharp(PHOTOS.bigPano, size, expanded), + null, + ); + assert.equal( + PANORAMAX_SOURCES.panoSharp(PHOTOS.smallPano, size, expanded), + null, + ); + assert.deepEqual( + PANORAMAX_SOURCES.panoSharp({ ...PHOTOS.pano, width: 3000 }, size, expanded) + .resize, + [3000, 1500], + 'never enlarged', + ); + // Without an SD derivative the HD one is the first and only. + const hdOnly = { ...PHOTOS.pano, urls: { ...PHOTOS.pano.urls, sd: null } }; + assert.deepEqual(PANORAMAX_SOURCES.panoFirst(hdOnly), { + url: hdOnly.urls.hd, + width: 4096, + resize: [4096, 2048], + }); + // …unless it is too big to decode: then no panorama, and the thumbnail. + const hugeOnly = { ...hdOnly, width: 11_904 }; + assert.equal(PANORAMAX_SOURCES.panoFirst(hugeOnly), null); + assert.equal(PANORAMAX_SOURCES.hasPano(hugeOnly), false); + assert.deepEqual(PANORAMAX_SOURCES.flatFirst(hugeOnly).filter(Boolean), [ + hugeOnly.urls.thumb, + ]); + assert.equal(PANORAMAX_SOURCES.panoSharp(hdOnly, size, expanded), null); + assert.equal(PANORAMAX_SOURCES.flatSharp(PHOTOS.flat, 2048), null); + assert.equal( + PANORAMAX_SOURCES.flatSharp(PHOTOS.flat, 2049), + PHOTOS.flat.urls.hd, + ); + assert.equal( + PANORAMAX_SOURCES.hasPano({ isPano: true, urls: { sd: null, hd: null } }), + false, + ); +}); + +test('› and ‹ follow the picture’s own links; the ends disable the buttons', async () => { + const { viewer, poses, stepped, host } = setup(); + await viewer.mount(host); + const { next, prev } = parts(host); + await viewer.open('flat'); + next.fire('click'); + assert.ok(await until(() => poses.at(-1)?.imageId === 'flat2')); + assert.deepEqual(stepped, ['flat2']); + assert.ok(await until(() => next.disabled), 'flat2 is the last'); + assert.equal(prev.disabled, false); +}); + +test('without WebGL a 360° picture shows its whole equirectangular image flat', async () => { + const loads = []; + const viewer = createPanoramaxViewer({ + getPhoto: async () => PHOTOS.pano, + step: async () => null, + onStep() {}, + load: async (url) => { + loads.push(url); + return { src: url }; + }, + createRenderer: () => { + throw new Error('no WebGL'); + }, + }); + const host = fakeHost(); + await viewer.mount(host); + await viewer.open('pano'); + const { picture } = parts(host); + assert.equal(picture.hidden, false); + assert.equal(picture.src, PHOTOS.pano.urls.sd); +}); + +test('a picture whose SD fails falls back to the thumbnail; the original only for the expanded viewer', async () => { + const tried = []; + const viewer = createPanoramaxViewer({ + getPhoto: async () => PHOTOS.flat, + step: async () => null, + onStep() {}, + load: async (url) => { + tried.push(url); + if (!url.endsWith('thumb.jpg')) throw new Error('gone'); + return { src: url }; + }, + }); + const host = fakeHost(); + await viewer.mount(host); + await viewer.open('flat'); + assert.deepEqual(tried, [PHOTOS.flat.urls.sd, PHOTOS.flat.urls.thumb]); + assert.equal(parts(host).picture.src, PHOTOS.flat.urls.thumb); + // Expanded, the original is worth its megabytes. + tried.length = 0; + parts(host).root.closest = (selector) => + selector.includes('sl-viewer-wrap-expanded') ? {} : null; + await viewer.open('flat'); + assert.deepEqual(tried, [ + PHOTOS.flat.urls.sd, + PHOTOS.flat.urls.hd, + PHOTOS.flat.urls.thumb, + ]); +}); + +test('a photo with no author to name is captioned with the instance holding it', () => { + const photo = normalizeItem( + stacItem({ id: 'x', lon: 1, lat: 2, producer: 'someone@city.example.fr' }), + ); + photo.via = 'panoramax.ign.fr'; + assert.deepEqual(PANORAMAX_SOURCES.describe(photo), { + title: 'via panoramax.ign.fr', + creator: null, + license: 'CC BY-SA 4.0', + externalUrl: 'https://api.panoramax.xyz/#pic=x&focus=pic', + }); + const named = { ...photo, creator: 'Robot8A' }; + assert.equal(PANORAMAX_SOURCES.describe(named).title, null); +}); + +test('an SD panorama refused for its size is not fetched again as a flat photo', async () => { + const loads = []; + const viewer = createPanoramaxViewer({ + getPhoto: async () => PHOTOS.pano, + step: async () => null, + onStep() {}, + load: async (url, options = {}) => { + loads.push({ url, crossOrigin: options.crossOrigin }); + // The picture host's SD file is past the viewer's size limits. + if (url === PHOTOS.pano.urls.sd) + throw Object.assign(new Error('too big'), { tooLarge: true }); + return { src: url }; + }, + createRenderer: () => ({ + setImage() {}, + setView() {}, + getView: () => ({ yaw: 0, pitch: 0, fov: 80 }), + resize() {}, + clear() {}, + destroy() {}, + }), + }); + const host = fakeHost(); + await viewer.mount(host); + await viewer.open('pano'); + const sdLoads = loads.filter((l) => l.url === PHOTOS.pano.urls.sd); + assert.equal(sdLoads.length, 1, 'tried once, as a bitmap, and refused'); + assert.equal(sdLoads[0].crossOrigin, true); + const { picture } = parts(host); + assert.equal(picture.hidden, false, 'the flat fallback shows'); + assert.notEqual(picture.src, PHOTOS.pano.urls.sd); + // Opened again later: the refused file is still not asked for. + await viewer.open('pano'); + assert.equal(loads.filter((l) => l.url === PHOTOS.pano.urls.sd).length, 1); +}); diff --git a/src/layers/streetLevel/registry.js b/src/layers/streetLevel/registry.js index 3451447e4..ce7640137 100644 --- a/src/layers/streetLevel/registry.js +++ b/src/layers/streetLevel/registry.js @@ -83,6 +83,7 @@ export function filtersOf(provider) { * @property {'day'|'month'} [capturedAtPrecision] How exactly the provider dates images (default 'day'). * @property {string|null} [title] A place name for the caption, e.g. the address. * @property {string|null} creator + * @property {string|null} [license] The image's licence when it is per image, e.g. 'CC BY-SA 4.0'. * @property {string|null} sequenceId * @property {string} externalUrl Deep link to the image on the provider's site. */ diff --git a/src/layers/streetLevel/state.js b/src/layers/streetLevel/state.js index fd92e05ec..a7bf85e37 100644 --- a/src/layers/streetLevel/state.js +++ b/src/layers/streetLevel/state.js @@ -26,6 +26,7 @@ export function freshStreet() { renderModes: true, sequenceId: null, creator: null, + license: null, externalUrl: null, loading: false, error: null, diff --git a/src/layers/streetLevel/tileCache.js b/src/layers/streetLevel/tileCache.js new file mode 100644 index 000000000..d6c0a91e5 --- /dev/null +++ b/src/layers/streetLevel/tileCache.js @@ -0,0 +1,51 @@ +/** Decoded coverage tiles kept, and for how long. */ +export const TILE_CACHE_SIZE = 24; +export const TILE_CACHE_TTL_MS = 10 * 60_000; + +/** Least-recently-used map: reading an entry renews it. */ +export function createLru(limit) { + const map = new Map(); + return { + get(key) { + if (!map.has(key)) return undefined; + const value = map.get(key); + map.delete(key); + map.set(key, value); + return value; + }, + set(key, value) { + map.delete(key); + map.set(key, value); + while (map.size > limit) map.delete(map.keys().next().value); + }, + delete: (key) => map.delete(key), + clear: () => map.clear(), + }; +} + +/** + * Loaded (decoded) coverage tiles by `keyOf` (default `z/x/y`), at most `size`, each for + * `ttlMs`: a pan back, or a zoom across bands and back, neither fetches nor + * decodes them again. A load that fails is not kept. + */ +export function createTileCache({ + load, + size = TILE_CACHE_SIZE, + ttlMs = TILE_CACHE_TTL_MS, + now = Date.now, + keyOf = (tile) => `${tile.z}/${tile.x}/${tile.y}`, +}) { + const kept = createLru(size); + return { + async get(tile, options) { + const key = keyOf(tile); + const entry = kept.get(key); + if (entry && now() - entry.at < ttlMs) return entry.tile; + const loaded = await load(tile, options); + kept.set(key, { at: now(), tile: loaded }); + return loaded; + }, + /** Forget every tile (the layer is off: its memory goes back). */ + clear: () => kept.clear(), + }; +} diff --git a/src/layers/streetLevel/viewerHost.js b/src/layers/streetLevel/viewerHost.js index 89e9ad843..ce7363a13 100644 --- a/src/layers/streetLevel/viewerHost.js +++ b/src/layers/streetLevel/viewerHost.js @@ -39,6 +39,7 @@ export function createViewerHost({ state, parts }) { title: pose.title || null, sequenceId: pose.sequenceId ?? null, creator: pose.creator || null, + license: pose.license || null, externalUrl: pose.externalUrl || null, // FIT / FILL, unless this image has no frame to fit (a panorama). renderModes: diff --git a/src/layers/streetLevel/viewerHost.test.mjs b/src/layers/streetLevel/viewerHost.test.mjs index 070fe3d91..259934b73 100644 --- a/src/layers/streetLevel/viewerHost.test.mjs +++ b/src/layers/streetLevel/viewerHost.test.mjs @@ -673,3 +673,15 @@ test('an adapter that can frame but says this image has no frame offers no FIT / await host.open('mapillary', 'flat-1'); assert.equal(state.street.renderModes, true, 'a flat image can be framed'); }); + +test('a pose’s licence reaches the caption state, and a closed viewer forgets it', async () => { + const { state, host, google } = twoProviders(); + await host.open('google', 'g-1'); + google.emitPose('g-1', { providerId: 'google', license: 'CC BY-SA 4.0' }); + assert.equal(state.street.license, 'CC BY-SA 4.0'); + google.emitPose('g-1', { providerId: 'google' }); + assert.equal(state.street.license, null, 'none given: none shown'); + google.emitPose('g-1', { providerId: 'google', license: 'CC0' }); + host.close(); + assert.equal(state.street.license, null); +}); diff --git a/src/layers/weather/index.js b/src/layers/weather/index.js index 16f876cb0..db62ea98a 100644 --- a/src/layers/weather/index.js +++ b/src/layers/weather/index.js @@ -69,7 +69,7 @@ export function createWeatherLayer({ const satellite = !radar && !lightning; let product = radar ? 'radar' : lightning ? 'lightning' : 'clouds-regional'; let opacity = 'strong'; - let infrared = 'filtered'; + let infrared = 100; let viewer = null, rendering = null, manifest = null, @@ -405,11 +405,20 @@ export function createWeatherLayer({ } }, setParams(params = {}) { + const nextInfrared = satellite + ? params.infrared === 'full' + ? 0 + : params.infrared === 'filtered' + ? 100 + : params.infrared + : null; const infraredChanged = satellite && - ['filtered', 'full'].includes(params.infrared) && - params.infrared !== infrared; - if (infraredChanged) infrared = params.infrared; + Number.isInteger(nextInfrared) && + nextInfrared >= 0 && + nextInfrared <= 100 && + nextInfrared !== infrared; + if (infraredChanged) infrared = nextInfrared; if (['light', 'strong'].includes(params.opacity)) { opacity = params.opacity; rendering?.setAlpha(opacity === 'light' ? 0.4 : satellite ? 0.7 : 0.8); @@ -677,18 +686,20 @@ export function createWeatherLayer({ { id: 'filtered', label: t('Clouds only'), - active: infrared === 'filtered', - params: { infrared: 'filtered' }, + active: infrared === 100, + params: { infrared: 100 }, title: t( - 'Dim everything but the bright, cold cloud tops; a brightness filter, not a cloud mask', + 'Apply the full cloud brightness key; 100% filter strength', ), }, { id: 'full', label: t('Full'), - active: infrared === 'full', - params: { infrared: 'full' }, - title: t('The complete infrared image at the chosen opacity'), + active: infrared === 0, + params: { infrared: 0 }, + title: t( + 'Show the complete infrared image; 0% filter strength', + ), }, ] : []), @@ -750,7 +761,21 @@ export function createWeatherLayer({ { id: 'image', label: t('IMAGE'), - chips: controls.chips.filter(({ params }) => params.infrared), + chips: controls.chips.filter( + ({ params }) => params.infrared !== undefined, + ), + range: { + id: 'cloud-filter', + param: 'infrared', + min: 0, + max: 100, + step: 1, + value: infrared, + label: t('FILTER'), + unit: '%', + ariaLabel: t('Cloud filter strength'), + commitOnChange: true, + }, }, ] : []), diff --git a/src/layers/weather/infraredAlpha.js b/src/layers/weather/infraredAlpha.js index 007c7486f..f42bda603 100644 --- a/src/layers/weather/infraredAlpha.js +++ b/src/layers/weather/infraredAlpha.js @@ -1,10 +1,18 @@ export const INFRARED_ALPHA_T0 = 0.4; export const INFRARED_ALPHA_T1 = 0.7; -/** Return display pixels without changing RGB or the source RGBA bytes. */ -export function infraredAlpha(rgba, mode = 'filtered') { +/** Return display pixels without changing RGB or the source RGBA bytes. + * `strength` is 0–100: zero is the full image and 100 is the cloud key. + */ +export function infraredAlpha(rgba, strength = 'filtered') { const pixels = new Uint8ClampedArray(rgba); - if (mode === 'full') return pixels; + const amount = + strength === 'full' + ? 0 + : strength === 'filtered' || !Number.isFinite(strength) + ? 1 + : Math.max(0, Math.min(100, strength)) / 100; + if (amount === 0) return pixels; for (let i = 0; i < pixels.length; i += 4) { // Match Cesium's czm_srgbToLinear conversion in draped imagery. const linearMax = @@ -17,7 +25,8 @@ export function infraredAlpha(rgba, mode = 'filtered') { (INFRARED_ALPHA_T1 - INFRARED_ALPHA_T0), ), ); - pixels[i + 3] = rgba[i + 3] * t * t * (3 - 2 * t); + const filtered = t * t * (3 - 2 * t); + pixels[i + 3] = rgba[i + 3] * (1 - amount + filtered * amount); } return pixels; } diff --git a/src/layers/weather/infraredAlpha.test.mjs b/src/layers/weather/infraredAlpha.test.mjs index f0ee95516..950271a9d 100644 --- a/src/layers/weather/infraredAlpha.test.mjs +++ b/src/layers/weather/infraredAlpha.test.mjs @@ -40,6 +40,21 @@ test('filtered infrared preserves RGB and input bytes, scales existing alpha, an ); }); +test('filter strength interpolates between the full infrared image and the cloud key', () => { + const pixels = new Uint8ClampedArray([ + 0, 0, 0, 255, 194, 0, 0, 200, 12, 50, 217, 128, + ]); + assert.deepEqual([...infraredAlpha(pixels, 0)], [...pixels]); + assert.deepEqual( + infraredAlpha(pixels, 100), + infraredAlpha(pixels, 'filtered'), + ); + assert.deepEqual( + [...infraredAlpha(pixels, 50)], + [0, 0, 0, 128, 194, 0, 0, 149, 12, 50, 217, 128], + ); +}); + test('full infrared preserves every channel including partial and zero alpha', () => { const pixels = new Uint8ClampedArray([ 12, 50, 194, 80, 0, 0, 0, 255, 255, 200, 0, 0, diff --git a/src/layers/weather/infraredImage.test.mjs b/src/layers/weather/infraredImage.test.mjs index f3937abc3..61e532564 100644 --- a/src/layers/weather/infraredImage.test.mjs +++ b/src/layers/weather/infraredImage.test.mjs @@ -32,6 +32,15 @@ test('decoded pixels are processed once on a same-sized canvas', () => { assert.equal(output.height, 1); assert.equal(output.image, image); assert.deepEqual([...output.pixels], [194, 0, 0, 98]); + assert.deepEqual( + [...processInfraredImage(image, 50, canvas).pixels], + [194, 0, 0, 149], + 'half-strength filter retains a midpoint haze', + ); + assert.deepEqual( + [...processInfraredImage(image, 0, canvas).pixels], + [194, 0, 0, 200], + ); assert.deepEqual( [...processInfraredImage(image, 'full', canvas).pixels], [194, 0, 0, 200], diff --git a/src/layers/weather/weather.test.mjs b/src/layers/weather/weather.test.mjs index a80ad6560..1a4ba5177 100644 --- a/src/layers/weather/weather.test.mjs +++ b/src/layers/weather/weather.test.mjs @@ -444,11 +444,14 @@ function layerHarness({ let active = null; let clearCount = 0; let hidden = false; + let alpha = null; const rendering = { rehome() { active?.finish(false); }, - setAlpha() {}, + setAlpha(value) { + alpha = value; + }, setHidden(value) { hidden = value; if (value) active?.finish(false); @@ -514,6 +517,7 @@ function layerHarness({ documentRef, moveEnd, clears: () => clearCount, + getAlpha: () => alpha, }; } @@ -1591,8 +1595,8 @@ test('satellite mode chips restage history without changing time or latest-follo await flush(); h.layer.setParams({ infrared: 'full' }); assert.equal(h.stages[2].time, times[1]); - assert.equal(h.stages[2].infrared, 'full'); - assert.equal(h.layer.getParams().infrared, 'full'); + assert.equal(h.stages[2].infrared, 0); + assert.equal(h.layer.getParams().infrared, 0); assert.deepEqual( h.layer .getRowControls() @@ -1605,11 +1609,59 @@ test('satellite mode chips restage history without changing time or latest-follo assert.equal(h.layer.getDiagnostics().followLatest, false); assert.equal(h.layer.getDiagnostics().time, times[1]); h.layer.setParams({ infrared: 'invalid' }); - assert.equal(h.layer.getParams().infrared, 'full'); + assert.equal(h.layer.getParams().infrared, 0); assert.equal(h.stages.length, 3); h.layer.destroy(); }); +test('satellite cloud filter spans Full to Clouds only without changing layer opacity', () => { + const h = layerHarness({ id: 'weather-satellite' }); + assert.equal(h.layer.getParams().infrared, 100); + assert.equal(h.layer.getParams().opacity, 'strong'); + assert.equal(h.getAlpha(), 0.7); + const setting = h.layer + .getRowControls() + .summary.settings.find(({ id }) => id === 'image'); + assert.equal(setting.range.param, 'infrared'); + assert.equal(setting.range.value, 100); + assert.equal(setting.range.min, 0); + assert.equal(setting.range.max, 100); + assert.equal(setting.range.step, 1); + assert.equal(setting.range.commitOnChange, true); + + const full = h.layer.getRowControls().chips.find(({ id }) => id === 'full'); + h.layer.setParams(full.params); + assert.equal(h.layer.getParams().infrared, 0); + assert.equal(h.getAlpha(), 0.7); + assert.equal( + h.layer.getRowControls().chips.find(({ id }) => id === 'full').active, + true, + ); + + h.layer.setParams({ infrared: 53 }); + assert.equal(h.layer.getParams().infrared, 53); + assert.equal(h.getAlpha(), 0.7, 'filter strength is not overall opacity'); + assert.equal( + h.layer.getRowControls().summary.settings.find(({ id }) => id === 'image') + .range.value, + 53, + ); + h.layer.setParams({ opacity: 'light' }); + assert.equal(h.layer.getParams().opacity, 'light'); + assert.equal( + h.getAlpha(), + 0.4, + 'Soft remains the independent opacity preset', + ); + h.layer.setParams({ infrared: 101 }); + assert.equal( + h.layer.getParams().infrared, + 53, + 'invalid filter values are ignored', + ); + h.layer.destroy(); +}); + test('a rapid third selection waits for retirement before installing, and clear cancels that wait', async () => { for (const clear of [false, true]) { const h = renderingHarness(); @@ -1914,8 +1966,14 @@ test('observed descriptors keep configuration only and label satellite clouds by ); assert.equal( controls.chips.find(({ id }) => id === 'filtered').title, - 'Dim everything but the bright, cold cloud tops; a brightness filter, not a cloud mask', + 'Apply the full cloud brightness key; 100% filter strength', ); + const imageSetting = controls.summary.settings.find( + ({ id }) => id === 'image', + ); + assert.equal(imageSetting.range.param, 'infrared'); + assert.equal(imageSetting.range.value, 100); + assert.equal(imageSetting.range.commitOnChange, true); h.layer.setParams({ product: 'clouds' }); assert.equal( h.layer.getRowControls().summary.coverage, @@ -2196,7 +2254,7 @@ test('layer warms the next advertised observation only after a successful playin warmed.map(({ time }) => time), [times[0]], ); - assert.deepEqual(warmed[0].options, { infrared: 'filtered' }); + assert.deepEqual(warmed[0].options, { infrared: 100 }); const next = clock.setTarget(times[0]); await flush(); h.stages.at(-1).finish(); diff --git a/src/overpassProxy.test.mjs b/src/overpassProxy.test.mjs index 71b62aa2d..ed4539fda 100644 --- a/src/overpassProxy.test.mjs +++ b/src/overpassProxy.test.mjs @@ -12,6 +12,7 @@ import { createHash, randomUUID } from 'node:crypto'; import path from 'node:path'; import { Readable } from 'node:stream'; import createViteConfig, { fetchOverpassPayload, overpassPayloadIsData, readOverpassDisk } from '../vite.config.js'; +import { resolveOverpassUpstreamTag } from '../server/providers/overpass/constants.js'; const DATA = { status: 200, body: '{"elements":[]}' }; @@ -155,7 +156,13 @@ test('coalesced outage callers both receive last-good data, never a cached refus const directory = path.join(process.cwd(), '.gev-cache', 'overpass'); const file = path.join(directory, `${createHash('sha1').update(body).digest('hex')}.json`); await mkdir(directory, { recursive: true }); - const stale = { ...DATA, cachedAt: Date.now() - 40 * 86400000 }; + // Last-good data was fetched from the instance configured above, so it + // carries that instance's tag the way the transport now writes it. + const stale = { + ...DATA, + cachedAt: Date.now() - 40 * 86400000, + upstreams: resolveOverpassUpstreamTag(), + }; await writeFile(file, JSON.stringify(stale)); const entered = Promise.withResolvers(); const release = Promise.withResolvers(); @@ -188,3 +195,46 @@ test('coalesced outage callers both receive last-good data, never a cached refus } } }); + +test('a disk entry is not served after OVERPASS_UPSTREAMS is repointed', async (t) => { + const prior = process.env.OVERPASS_UPSTREAMS; + t.after(() => { + if (prior === undefined) delete process.env.OVERPASS_UPSTREAMS; + else process.env.OVERPASS_UPSTREAMS = prior; + }); + const key = `overpass-upstream-switch-${randomUUID()}`; + const directory = path.join(process.cwd(), '.gev-cache', 'overpass'); + const file = path.join( + directory, + `${createHash('sha1').update(key).digest('hex')}.json`, + ); + await mkdir(directory, { recursive: true }); + try { + // Written while one instance was configured... + process.env.OVERPASS_UPSTREAMS = 'https://first.example/api/interpreter'; + const first = resolveOverpassUpstreamTag(); + await writeFile( + file, + JSON.stringify({ ...DATA, cachedAt: Date.now(), upstreams: first }), + ); + assert.ok(await readOverpassDisk(key, 60000), 'its own entry is a hit'); + + // ...is not an answer once the operator points at a different one, on the + // stale path either: the key is the query alone, so it would otherwise hit. + process.env.OVERPASS_UPSTREAMS = 'https://second.example/api/interpreter'; + assert.notEqual(resolveOverpassUpstreamTag(), first, 'tag follows the env'); + assert.equal(await readOverpassDisk(key, 60000), null, 'fresh read refused'); + assert.equal( + await readOverpassDisk(key, Infinity), + null, + 'stale read refused', + ); + + // With nothing configured there is no instance to contradict, and serving + // last-good data is what the not-configured path exists for. + delete process.env.OVERPASS_UPSTREAMS; + assert.ok(await readOverpassDisk(key, Infinity), 'unconfigured still serves'); + } finally { + await unlink(file); + } +}); diff --git a/src/testSupport/panoramaxFixtures.mjs b/src/testSupport/panoramaxFixtures.mjs new file mode 100644 index 000000000..47e807604 --- /dev/null +++ b/src/testSupport/panoramaxFixtures.mjs @@ -0,0 +1,129 @@ +// Panoramax stand-ins for tests and the hermetic QA gate: vector tiles in +// the meta-catalogue's shape, and STAC items. +import { PbfWriter } from 'pbf'; +import { + layerWriter, + lineGeometry, + lonLatToTileLocal, + zigzag, +} from '../layers/streetLevel/providers/mapillary/coverageFixture.mjs'; + +/** + * Encode a Panoramax `/api/map/{z}/{x}/{y}.mvt` tile from lon/lat features. + * @param {{x: number, y: number, z: number}} tile + * @param {{sequences?: Array<{id: string, date?: string, type?: string, parts: Array>}>, grid?: Array<{lon: number, lat: number, pano?: number, flat?: number}>, pictures?: Array<{id: string, lon: number, lat: number, sequenceId: string, heading?: number, ts?: string, type?: string}>, extent?: number}} [content] + */ +export function encodePanoramaxTile( + tile, + { sequences = [], grid = [], pictures = [], extent = 4096 } = {}, +) { + const pbf = new PbfWriter(); + if (sequences.length) { + const layer = layerWriter('sequences', extent); + for (const sequence of sequences) + layer.add( + 2, + lineGeometry( + sequence.parts.map((part) => + part.map((point) => lonLatToTileLocal(point, tile, extent)), + ), + ), + { + id: sequence.id, + type: sequence.type ?? 'flat', + date: sequence.date ?? '2025-06-01', + model: 'Test camera', + }, + ); + pbf.writeMessage(3, (_, out) => layer.write(out), null); + } + const point = (lon, lat) => { + const [px, py] = lonLatToTileLocal([lon, lat], tile, extent); + return [9, zigzag(px), zigzag(py)]; + }; + if (grid.length) { + const layer = layerWriter('grid', extent); + for (const cell of grid) { + const pano = cell.pano ?? 0; + const flat = cell.flat ?? 1; + layer.add(1, point(cell.lon, cell.lat), { + nb_pictures: pano + flat, + nb_360_pictures: pano, + nb_flat_pictures: flat, + }); + } + pbf.writeMessage(3, (_, out) => layer.write(out), null); + } + if (pictures.length) { + const layer = layerWriter('pictures', extent); + for (const picture of pictures) + layer.add(1, point(picture.lon, picture.lat), { + id: picture.id, + ts: picture.ts ?? '2025-06-01 10:00:00+00', + heading: picture.heading ?? 90, + sequences: JSON.stringify([picture.sequenceId]), + first_sequence: picture.sequenceId, + type: picture.type ?? 'flat', + }); + pbf.writeMessage(3, (_, out) => layer.write(out), null); + } + return new Uint8Array(pbf.finish()); +} + +/** + * A STAC item as the meta-catalogue returns it. + * @param {{id: string, lon: number, lat: number, sequenceId?: string, heading?: number, pano?: boolean, datetime?: string, producer?: string, license?: string, width?: number, prevId?: string|null, nextId?: string|null, host?: string}} options + */ +export function stacItem({ + id, + lon, + lat, + sequenceId = 'seq-1', + heading = 90, + pano = false, + datetime = '2025-06-01T10:00:00+00:00', + producer = 'tester', + license = 'CC-BY-SA-4.0', + width = pano ? 5760 : 4032, + prevId = null, + nextId = null, + host = 'https://panoramax.example.org', +}) { + const api = 'https://api.panoramax.xyz/api'; + const links = [ + { rel: 'self', href: `${api}/collections/${sequenceId}/items/${id}` }, + ]; + if (prevId) + links.push({ + rel: 'prev', + href: `${api}/collections/${sequenceId}/items/${prevId}`, + }); + if (nextId) + links.push({ + rel: 'next', + href: `${api}/collections/${sequenceId}/items/${nextId}`, + }); + return { + type: 'Feature', + id, + collection: sequenceId, + geometry: { type: 'Point', coordinates: [lon, lat] }, + providers: [{ name: producer, roles: ['producer'] }], + properties: { + datetime, + license, + 'view:azimuth': heading, + 'geovisio:producer': producer, + 'pers:interior_orientation': { + ...(pano ? { field_of_view: 360 } : {}), + sensor_array_dimensions: [width, pano ? width / 2 : 3024], + }, + }, + assets: { + hd: { href: `${host}/pictures/${id}/hd.jpg` }, + sd: { href: `${host}/pictures/${id}/sd.jpg` }, + thumb: { href: `${host}/pictures/${id}/thumb.jpg` }, + }, + links, + }; +} diff --git a/src/tooling/streetLevelQa.test.mjs b/src/tooling/streetLevelQa.test.mjs index 6a29b4f69..cd5ff501c 100644 --- a/src/tooling/streetLevelQa.test.mjs +++ b/src/tooling/streetLevelQa.test.mjs @@ -750,6 +750,115 @@ test('KartaView fixtures draw the drive, answer nearest first, and refuse the st ); }); +test('Panoramax fixtures: two sequences on one street, searches honour sequence and filters, unknown calls are flagged', async () => { + const { + answerPanoramaxRequest, + describePanoramaxCall, + PNX_LINE, + PNX_SEQUENCES, + panoramaxHdDownloads, + panoramaxPictures, + panoramaxTile, + resetPanoramaxFixture, + } = await import('../../scripts/fixtures/street-level/panoramax.mjs'); + const { decodeCoverageTile, decodePictures } = + await import('../layers/streetLevel/providers/panoramax/decode.js'); + const { normalizeItem } = + await import('../layers/streetLevel/providers/panoramax/policy.js'); + resetPanoramaxFixture(); + const pictures = panoramaxPictures(); + assert.deepEqual( + [...new Set(pictures.map((p) => p.sequenceId))], + [...PNX_SEQUENCES], + ); + assert.ok(pictures.some((p) => p.isPano)); + // Coverage: both lines in a z14 tile on the street; cones in z15. + const z = 14; + const x = lonToTileX((PNX_LINE.west + PNX_LINE.east) / 2, z); + const y = latToTileY(PNX_LINE.lat, z); + const lines = decodeCoverageTile(panoramaxTile(z, x, y), { x, y, z }); + assert.deepEqual( + lines.sequences.map((s) => s.id).sort(), + [...PNX_SEQUENCES].sort(), + ); + const t15 = { + x: lonToTileX(pictures[20].lon, 15), + y: latToTileY(PNX_LINE.lat, 15), + z: 15, + }; + assert.ok( + decodePictures(panoramaxTile(15, t15.x, t15.y), t15, PNX_SEQUENCES[0]) + .length > 0, + ); + // A search for one sequence near the street gives only its pictures, nearest first. + const at = pictures[30]; + const ask = (query) => + JSON.parse( + answerPanoramaxRequest({ + method: 'GET', + url: `https://api.panoramax.xyz/api/search?${query}`, + }).body, + ).features.map(normalizeItem); + const own = ask( + `place_position=${at.lon},${at.lat}&place_distance=0-50&limit=5&collections=${PNX_SEQUENCES[1]}`, + ); + assert.ok(own.length > 0); + assert.ok(own.every((p) => p.sequenceId === PNX_SEQUENCES[1])); + assert.equal(own[0].nextId && typeof own[0].nextId, 'string'); + const panos = ask( + `place_position=${at.lon},${at.lat}&place_distance=0-200&limit=50&filter=field_of_view%3D360`, + ); + assert.ok(panos.length > 0 && panos.every((p) => p.isPano)); + assert.deepEqual( + ask( + `place_position=${at.lon},${at.lat}&place_distance=0-200&limit=50&datetime=2024-01-01T00:00:00.000Z/..`, + ).every((p) => p.sequenceId === PNX_SEQUENCES[0]), + true, + 'the second sequence is older', + ); + // A box has every picture in it, those looking away included; a place + // search leaves out pictures the point is behind (all look east). + const d = 0.0002; + const boxed = ask( + `bbox=${at.lon - d},${at.lat - d},${at.lon + d},${at.lat + d}&limit=100`, + ); + assert.ok( + boxed.some((p) => p.lon < at.lon) && boxed.some((p) => p.lon > at.lon), + ); + const placed = ask( + `place_position=${at.lon},${at.lat}&place_distance=0-30&limit=50`, + ); + assert.ok(placed.length > 0); + assert.ok( + placed.every((p) => p.lon <= at.lon + 1e-7), + 'none it is behind', + ); + const one = ask(`ids=${at.id}&limit=1`); + assert.equal(one[0].id, at.id); + // Its pictures, and only those, are served; HD downloads are counted. + const file = (name) => + answerPanoramaxRequest({ + method: 'GET', + url: `https://panoramax.openstreetmap.fr/derivates/${name}`, + }); + assert.equal(file(`${at.id}/sd.jpg`).status, 200); + assert.equal(file(`${at.id}/hd.jpg`).status, 200); + assert.equal(panoramaxHdDownloads(), 1); + assert.equal(file('nope/sd.jpg').known, false); + const unknown = answerPanoramaxRequest({ + method: 'GET', + url: 'https://api.panoramax.xyz/api/collections/x/items', + }); + assert.equal(unknown.known, false); + assert.equal( + describePanoramaxCall( + 'GET', + new URL('https://api.panoramax.xyz/api/map/14/1/2.mvt'), + ), + 'GET api.panoramax.xyz/api/map/{z}/{x}/{y}.mvt', + ); +}); + /** The alpha of a fixture PNG (8-bit RGBA, unfiltered rows, as it writes them). */ function fixturePngAlpha(bytes) { const width = bytes.readUInt32BE(16); diff --git a/src/tools/globePanel.js b/src/tools/globePanel.js index 16e32eec6..0bfa5c2df 100644 --- a/src/tools/globePanel.js +++ b/src/tools/globePanel.js @@ -15,9 +15,9 @@ export const PANEL_WORKER_PRELUDE_PATH = 'cesium/worker-prelude.js'; /** * Sites the app's browser code loads from directly: map imagery, 3D tiles, * terrain, fonts, and Street Level's Mapillary Graph API and image CDN (its - * coverage tiles come through the app's server), Google Street View and - * KartaView. Everything from the app's - * own server comes through the MCP server. + * coverage tiles come through the app's server), Google Street View, + * KartaView and Panoramax. Everything from the app's own server comes + * through the MCP server. */ const PROVIDER_ORIGINS = Object.freeze([ 'https://tile.googleapis.com', @@ -49,6 +49,14 @@ const PROVIDER_ORIGINS = Object.freeze([ 'https://api.kartaview.org', 'https://cdn.kartaview.org', 'https://*.openstreetcam.org', + // Panoramax's meta-catalogue, and the federated instances holding most of + // its pictures (a picture on another instance shows only in the app). + 'https://api.panoramax.xyz', + 'https://panoramax.openstreetmap.fr', + 'https://panoramax.ign.fr', + // IGN's pictures redirect to its OVH object storage. + 'https://panoramax-storage-public-fast.s3.gra.perf.cloud.ovh.net', + 'https://panoramax.mapcomplete.org', ]); export const MCP_APP_MIME_TYPE = 'text/html;profile=mcp-app'; diff --git a/src/tools/queries/mobility.test.mjs b/src/tools/queries/mobility.test.mjs index 7c7050e88..011e431fd 100644 --- a/src/tools/queries/mobility.test.mjs +++ b/src/tools/queries/mobility.test.mjs @@ -457,6 +457,51 @@ test('bike share names the systems it did not search', async () => { ); }); +test('bike share over Oslo searches only the nearby Norwegian systems', async () => { + const requested = []; + const result = await composeCatalog({ + tools: coreTools, + services: { + bikeshare: { + systems: GBFS_CITY_REGISTRY, + getStations: async (url) => { + requested.push(url); + return { data: { stations: [] } }; + }, + }, + }, + }).call('get_bike_share', { + area: { lat: 59.9139, lon: 10.7522, radius_km: 5 }, + }); + assert.deepEqual(result.data.systems.sort(), [ + 'Lillestrøm, Norway', + 'Oslo, Norway', + ]); + assert.deepEqual(result.data.systems_not_searched, []); + assert.ok( + requested.every( + (url) => + url.startsWith('https://gbfs.urbansharing.com/oslobysykkel.no/') || + url.startsWith('https://api.cyclocity.fr/contracts/lillestrom/'), + ), + ); +}); + +test('bike share over Stavanger searches Kolumbus Bysykkel', async () => { + const result = await composeCatalog({ + tools: coreTools, + services: { + bikeshare: { + systems: GBFS_CITY_REGISTRY, + getStations: async () => ({ data: { stations: [] } }), + }, + }, + }).call('get_bike_share', { + area: { lat: 58.97, lon: 5.7331, radius_km: 5 }, + }); + assert.deepEqual(result.data.systems, ['Stavanger, Norway']); +}); + test('traffic counts only roads inside a radius area and reports missing tiles', async () => { const flow = (segments, partial = false) => ({ getStatus: async () => ({ hasKey: true }), diff --git a/src/ui/layerPanel.js b/src/ui/layerPanel.js index ed76c8c44..44cfd1bd1 100644 --- a/src/ui/layerPanel.js +++ b/src/ui/layerPanel.js @@ -164,6 +164,10 @@ const FEED_ERROR_PATTERNS = [ /^KartaView answered HTTP (\d+)$/, ([, status]) => t('KartaView answered HTTP {status}', { status }), ], + [ + /^Panoramax answered HTTP (\d+)$/, + ([, status]) => t('Panoramax answered HTTP {status}', { status }), + ], [ /^KartaView refused the request \(code (-?\d+)\)$/, ([, code]) => t('KartaView refused the request (code {code})', { code }), diff --git a/src/ui/railCardBlocks.js b/src/ui/railCardBlocks.js index f9f979a97..e1d37356c 100644 --- a/src/ui/railCardBlocks.js +++ b/src/ui/railCardBlocks.js @@ -39,6 +39,7 @@ export function createRailCardBlocks({ container, cardId, onParams }) { }; const dispose = (block) => { block.removeClick?.(); + block.removeInput?.(); block.children?.destroy(); block.root.remove(); }; @@ -83,6 +84,66 @@ export function createRailCardBlocks({ container, cardId, onParams }) { ); block.chips = make('div', 'rail-card-chips', root); block.chips.setAttribute('role', 'group'); + block.rangeRow = make('div', 'rail-card-setting-range-row', root); + block.rangeLabel = make( + 'span', + 'rail-card-setting-range-label', + block.rangeRow, + ); + block.rangeInput = make( + 'input', + 'rail-card-setting-range', + block.rangeRow, + ); + block.rangeInput.type = 'range'; + block.rangeValue = make( + 'span', + 'rail-card-setting-range-value', + block.rangeRow, + ); + const rangeParams = () => { + const range = block.props.range; + const value = Number(block.rangeInput.value); + if ( + !range?.param || + range.disabled || + !Number.isFinite(value) || + value < Number(range.min) || + value > Number(range.max) + ) + return null; + return { params: { [range.param]: value } }; + }; + const onInput = () => { + const range = block.props.range; + const value = Number(block.rangeInput.value); + const params = rangeParams(); + if (!params) return; + if (range.commitOnChange) block.rangeDraft = { range, value }; + set( + block.rangeValue, + 'textContent', + `${value}${range.unit || ''}`, + ); + attribute( + block.rangeInput, + 'aria-valuetext', + `${value}${range.unit || ''}`, + ); + if (!range.commitOnChange) dispatch(params); + }; + const onChange = () => { + if (!block.props.range?.commitOnChange) return; + const params = rangeParams(); + block.rangeDraft = null; + if (params) dispatch(params); + }; + block.rangeInput.addEventListener('input', onInput); + block.rangeInput.addEventListener('change', onChange); + block.removeInput = () => { + block.rangeInput.removeEventListener('input', onInput); + block.rangeInput.removeEventListener('change', onChange); + }; bind((event) => { const button = event.target?.closest?.('[data-chip-id]'); dispatch( @@ -133,17 +194,59 @@ export function createRailCardBlocks({ container, cardId, onParams }) { blocks.set(props.id, block); } block.props = props; - if (props.type === 'settings') + if (props.type === 'settings') { block.children.update( (props.settings || []).map((setting) => ({ ...setting, type: 'setting', })), ); - else if (props.type === 'setting') { + } else if (props.type === 'setting') { set(block.label, 'textContent', props.label); attribute(block.chips, 'aria-label', props.label); syncChipGroup(block.chips, props.chips); + const range = props.range; + set(block.rangeRow, 'hidden', !range); + const draft = block.rangeDraft; + if ( + draft && + (!range || + range.disabled || + !range.commitOnChange || + ['id', 'param', 'min', 'max', 'step', 'value'].some( + (key) => range[key] !== draft.range[key], + )) + ) + block.rangeDraft = null; + if (range) { + const min = Number(range.min); + const max = Number(range.max); + const step = Number(range.step); + const candidate = Number(block.rangeDraft?.value ?? range.value); + const value = Math.min( + max, + Math.max(min, Number.isFinite(candidate) ? candidate : min), + ); + set(block.rangeLabel, 'textContent', range.label || ''); + set(block.rangeLabel, 'hidden', !range.label); + set(block.rangeInput.dataset, 'rangeId', range.id || props.id); + set(block.rangeInput, 'min', String(min)); + set(block.rangeInput, 'max', String(max)); + set(block.rangeInput, 'step', String(step)); + set(block.rangeInput, 'value', String(value)); + set(block.rangeInput, 'disabled', Boolean(range.disabled)); + attribute( + block.rangeInput, + 'aria-label', + range.ariaLabel || props.label, + ); + attribute( + block.rangeInput, + 'aria-valuetext', + `${value}${range.unit || ''}`, + ); + set(block.rangeValue, 'textContent', `${value}${range.unit || ''}`); + } } else if (props.type === 'list') syncRowList(block.root, props.list); else if (props.type === 'result') { set(block.label, 'textContent', props.label); diff --git a/src/ui/railCards.test.mjs b/src/ui/railCards.test.mjs index e465e3cc7..74629bebb 100644 --- a/src/ui/railCards.test.mjs +++ b/src/ui/railCards.test.mjs @@ -119,6 +119,96 @@ test('footer buttons retain focus and current callbacks; links use safe new tabs button.click(); assert.deepEqual(calls, [2]); }); +test('setting ranges stay below presets and commit expensive filters on release', () => { + const f = railFixture(); + const calls = []; + const view = createRailCards({ + ...f, + onParams: (...args) => calls.push(args), + }); + const model = (value) => ({ + ...card, + blocks: [ + { + id: 'settings', + type: 'settings', + settings: [ + { + id: 'image', + label: 'IMAGE', + chips: [ + { + id: 'clouds-only', + label: 'Clouds only', + params: { infrared: 100 }, + }, + { id: 'full', label: 'Full', params: { infrared: 0 } }, + ], + range: { + id: 'cloud-filter', + param: 'infrared', + min: 0, + max: 100, + step: 1, + value, + label: 'FILTER', + unit: '%', + ariaLabel: 'Cloud filter strength', + commitOnChange: true, + }, + }, + ], + }, + ], + }); + view.update([model(100)]); + const setting = f.find((node) => node.dataset.blockId === 'image'); + assert.equal(setting.children[1].className, 'rail-card-chips'); + assert.equal(setting.children[2].className, 'rail-card-setting-range-row'); + assert.equal(setting.children[2].children[0].textContent, 'FILTER'); + const slider = f.find((node) => node.dataset.rangeId === 'cloud-filter'); + assert.equal(slider.tagName, 'INPUT'); + assert.equal(slider.type, 'range'); + assert.equal(slider.min, '0'); + assert.equal(slider.max, '100'); + assert.equal(slider.step, '1'); + assert.equal(slider.value, '100'); + assert.equal(slider.getAttribute('aria-label'), 'Cloud filter strength'); + slider.focus(); + slider.value = '53'; + slider.dispatchEvent(new Event('input')); + assert.equal(calls.length, 0, 'expensive image work waits until release'); + assert.equal( + f.find((node) => node.className === 'rail-card-setting-range-value') + .textContent, + '53%', + ); + view.update([model(100)]); + assert.equal(slider.value, '53', 'a status refresh retains the pending drag'); + assert.equal( + slider.getAttribute('aria-valuetext'), + '53%', + 'the accessible preview also survives a refresh', + ); + slider.dispatchEvent(new Event('change')); + assert.deepEqual(calls.at(-1), ['a', { infrared: 53 }]); + view.update([model(53)]); + assert.equal(slider.value, '53'); + assert.equal(f.document.activeElement, slider); + slider.value = '41'; + slider.dispatchEvent(new Event('input')); + view.update([model(70)]); + assert.equal( + slider.value, + '70', + 'an authoritative value supersedes the draft', + ); + view.destroy(); + slider.value = '41'; + slider.dispatchEvent(new Event('change')); + assert.equal(calls.length, 1, 'destroy removes the range listener'); +}); + test('legend validates colors, preserves the freezing anchor and ignores CSSOM normalization', () => { let writes = 0; const f = railFixture(() => writes++); diff --git a/src/ui/streetLevelPresentation.js b/src/ui/streetLevelPresentation.js index 0ddced9ac..579452336 100644 --- a/src/ui/streetLevelPresentation.js +++ b/src/ui/streetLevelPresentation.js @@ -104,6 +104,10 @@ const ERROR_PATTERNS = [ /^KartaView answered HTTP (\d+)$/, ([, status]) => t('KartaView answered HTTP {status}', { status }), ], + [ + /^Panoramax answered HTTP (\d+)$/, + ([, status]) => t('Panoramax answered HTTP {status}', { status }), + ], [ /^KartaView refused the request \(code (-?\d+)\)$/, ([, code]) => t('KartaView refused the request (code {code})', { code }), @@ -172,6 +176,7 @@ function presentViewer(state) { if (street.title) left.push(street.title); if (street.creator) left.push(t('Image by {creator}', { creator: street.creator })); + if (street.license) left.push(street.license); return { open: street.open === true, loading: street.loading === true && !street.imageId, diff --git a/src/ui/streetLevelPresentation.test.mjs b/src/ui/streetLevelPresentation.test.mjs index ccbfaa687..40cc4d14a 100644 --- a/src/ui/streetLevelPresentation.test.mjs +++ b/src/ui/streetLevelPresentation.test.mjs @@ -541,3 +541,25 @@ test('KEY REQUIRED points at Street View when it would draw with the Google key' assert.equal(plain.status.text, 'KEY REQUIRED'); assert.notEqual(plain.meta, guidance); }); + +test('the caption names an image’s own licence after its author, and leaves it out when there is none', () => { + const caption = (street) => + presentStreetLevelPanel({ + enabled: true, + providers: [], + coverage: {}, + legend: [], + filter: {}, + sequence: {}, + street: { open: true, imageId: '1', ...street }, + }).viewer.captionLeft; + assert.equal( + caption({ creator: 'Fixture Mapper', license: 'CC BY-SA 4.0' }), + 'Image by Fixture Mapper · CC BY-SA 4.0', + ); + assert.equal( + caption({ license: 'Licence Ouverte 2.0' }), + 'Licence Ouverte 2.0', + ); + assert.equal(caption({ creator: 'Someone' }), 'Image by Someone'); +}); diff --git a/src/ui/styles/street-level.css b/src/ui/styles/street-level.css index 1914c8fba..db3db5d2e 100644 --- a/src/ui/styles/street-level.css +++ b/src/ui/styles/street-level.css @@ -7,6 +7,9 @@ /* KartaView's purple for lines; a lighter one where it is text (WCAG AA). */ --sl-kartaview: #635bff; --sl-kartaview-text: #8f8aff; + /* Panoramax's orange; the darker one sits behind white text. */ + --sl-panoramax: #ff6f00; + --sl-panoramax-fill: #b84e00; --sl-text: 0.62rem; --sl-text-small: 0.6rem; } @@ -646,8 +649,11 @@ html:not([lang='en']) .sl-meta-row .sl-nearest-btn { font-size: 1.3rem; } -/* ── KartaView photo viewer ──────────────────────────────────────────────── */ -.sl-kv-viewer { +/* ── Photo viewer (KartaView, Panoramax) ─────────────────────────────────── */ +.sl-photo-viewer { + /* The lit source's colour: a focus ring, and behind the white ‹ › glyphs. */ + --sl-photo-ring: var(--sl-kartaview-text); + --sl-photo-fill: color-mix(in srgb, var(--sl-kartaview) 85%, transparent); position: absolute; inset: 0; background: #000; @@ -655,23 +661,29 @@ html:not([lang='en']) .sl-meta-row .sl-nearest-btn { overflow: hidden; } -.sl-kv-viewer:focus-visible { +.sl-photo-viewer:focus-visible { outline: none; } /* The photo and canvas fill the viewer and would cover a ring drawn on it: draw the ring on a layer above them (clicks pass through). */ -.sl-kv-viewer:focus-visible::after { +.sl-photo-viewer:focus-visible::after { content: ''; position: absolute; inset: 0; z-index: 3; - border: 2px solid var(--sl-kartaview-text); + border: 2px solid var(--sl-photo-ring); pointer-events: none; } -.sl-kv-photo, -.sl-kv-pano { +/* White on #ff6f00 is 2.8:1; on the darker orange behind ‹ › it is 5.1:1. */ +.sl-photo-viewer[data-provider='panoramax'] { + --sl-photo-ring: var(--sl-panoramax); + --sl-photo-fill: var(--sl-panoramax-fill); +} + +.sl-photo-flat, +.sl-photo-pano { position: absolute; inset: 0; width: 100%; @@ -679,25 +691,25 @@ html:not([lang='en']) .sl-meta-row .sl-nearest-btn { user-select: none; } -.sl-kv-photo { +.sl-photo-flat { object-fit: contain; } -.sl-kv-pano { +.sl-photo-pano { cursor: grab; touch-action: none; } -.sl-kv-pano.sl-kv-dragging { +.sl-photo-pano.sl-photo-dragging { cursor: grabbing; } -.sl-kv-photo[hidden], -.sl-kv-pano[hidden] { +.sl-photo-flat[hidden], +.sl-photo-pano[hidden] { display: none; } -.sl-kv-step { +.sl-photo-step { position: absolute; top: 50%; z-index: 2; @@ -713,27 +725,28 @@ html:not([lang='en']) .sl-meta-row .sl-nearest-btn { cursor: pointer; } -.sl-kv-step-prev { +.sl-photo-step-prev { left: 6px; } -.sl-kv-step-next { +.sl-photo-step-next { right: 6px; } -.sl-kv-step:hover:not(:disabled), -.sl-kv-step:focus-visible { - background: color-mix(in srgb, var(--sl-kartaview) 85%, transparent); +.sl-photo-step:hover:not(:disabled), +.sl-photo-step:focus-visible { + background: var(--sl-photo-fill); outline: none; } -.sl-kv-step:disabled { +.sl-photo-step:disabled { opacity: 0.3; cursor: default; } /* Nothing to step through until a photo shows. */ -.sl-kv-viewer:not([data-kind='flat']):not([data-kind='pano']) .sl-kv-step { +.sl-photo-viewer:not([data-kind='flat']):not([data-kind='pano']) + .sl-photo-step { display: none; } diff --git a/src/ui/styles/weather.css b/src/ui/styles/weather.css index d632bb1bf..c8b2d544c 100644 --- a/src/ui/styles/weather.css +++ b/src/ui/styles/weather.css @@ -349,6 +349,36 @@ html:not([lang='en']) .rail-card-setting { .rail-card-setting .rail-card-chips { margin-top: 0; } +.rail-card-setting-range-row { + display: flex; + grid-column: 2; + align-items: center; + gap: 7px; + min-width: 0; +} +.rail-card-setting-range-label { + flex: 0 0 6ch; + color: var(--text-dim); + font: 8px var(--font-mono); + white-space: nowrap; +} +.rail-card-setting-range { + width: 100%; + min-width: 0; + accent-color: var(--accent); + cursor: pointer; +} +.rail-card-setting-range:focus-visible { + outline: 2px solid var(--text-primary); + outline-offset: 3px; +} +.rail-card-setting-range-value { + flex: 0 0 4ch; + white-space: nowrap; + color: var(--text-primary); + font: 9px var(--font-mono); + text-align: right; +} .rail-card-result { display: grid; gap: 4px;