diff --git a/CHANGELOG.md b/CHANGELOG.md index df6497222..a7db1c6a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -153,13 +153,18 @@ ← → following the picture's own links. The 360° and since filters apply, in the lines and in the lookup. The caption names each picture's author and licence (CC BY-SA 4.0, Licence Ouverte 2.0…) and links to it in the - Panoramax viewer; HD originals load only for an expanded viewer, once. - Share links carry it as `0.m.0_0.r.panoramax-1`. KartaView's photo viewer + Panoramax viewer. HD originals load once, only when the viewer is wider + than the 2,048 px version (a panorama: expanded), and every download stops + when the photo changes; an image whose header declares more than 64 MP is + refused before it decodes. Share links carry it as + `0.m.0_0.r.google-0*kartaview-0*panoramax-1` (`0.m.0_0.r.panoramax-1` works + too). KartaView's photo viewer is now shared (`photoViewer.js`, `panorama.js`); in it, for KartaView too, Esc clears the selected drive before it collapses the panel. Lines are simplified and drawn within a point budget (a tile's unsimplified lines - over Paris hung the tab), decoded tiles are kept, and the cones' tiles - have their own connection. + over Paris hung the tab), decoded in slices so the globe keeps moving, + and kept; the cones' tiles have their own connection, and the open + picture's tile draws its cones before its neighbours arrive. - Flights and Military Flights highlight emergency transponder broadcasts in fleet billboards, 3D models and Cockpit contacts. Colors update when a status diff --git a/README.md b/README.md index ee19953ab..554f9b57f 100644 --- a/README.md +++ b/README.md @@ -354,9 +354,9 @@ _The Space Missions layer replaying a Falcon 9 ascent — labeled `RECONSTRUCTED **Setup.** At [mapillary.com/dashboard/developers](https://www.mapillary.com/dashboard/developers), register an application with **Read** access and copy its **Client Token** (`MLY|…`). Paste it into **POWER UP → Mapillary**, or set `MAPILLARY_CLIENT_TOKEN` in `.env` and restart. Without it the panel reads **KEY REQUIRED**. **Google Street View** uses `GOOGLE_MAPS_API_KEY` with the **Maps JavaScript API** enabled for it in Google Cloud; its chip starts off, since every panorama is billed to the key. **KartaView** and **Panoramax** need nothing: their public APIs are keyless. -**Use.** Turn on **Street Level** under DATA LAYERS → Cameras, or with the pill in the STREET LEVEL panel header. Each provider has a chip in its own colour (Mapillary green), and one is lit at a time: lighting one switches the others off. Mapillary, which is free, is lit by default. The **IMAGERY** (ALL / 360° / FLAT) and **SINCE** filters apply to Mapillary and KartaView; Street View panoramas are all 360°, so with Street View lit the filters and legend are hidden. Click a coverage line for its image cones and a cone to open the photo. Light **KARTAVIEW** (purple) and zoom in below about 6 km: its lines are the streets KartaView has photos of, and a click on one opens the photo nearest the click, flat or 360° (drag to look around), with cones along its drive; **‹ ›** or ← → step through the drive (Shift+← →, ↑ ↓ and + − look around a 360° photo; pinch zooms on a phone). Photos KartaView has archived are skipped, and where all are archived the panel says so. For KartaView, **SINCE** goes by upload date, as KartaView's own map does (it can filter by nothing else), and the panel notes it under the slider; KartaView filters its lines on its own server, and a filtered area KartaView has not drawn before can take several seconds, and the unfiltered lines show meanwhile. Light **PANORAMAX** (orange) for the open, federated commons (strongest in France and Belgium): from orbit its dots show where it has pictures, and below about 60 km its lines are sequences. A click on a line opens that sequence's picture nearest the click without moving the camera, flat or 360°, with cones along the sequence around it; **‹ ›** or ← → step through it, and both filters apply. Its caption names the picture's authors (or the instance holding it) and licence (mostly CC BY-SA 4.0; pictures on the IGN instance are Licence Ouverte 2.0). On the **Google 3D** map (Google's terms keep Street View off other maps, where it does not appear), light **GOOGLE STREET VIEW** and click a street from below 3 km, or press **OPEN STREET VIEW** for the view centre, to open Google's panorama there; step along its arrows. Above the photo: **EXPAND**, **FIT / FILL** (Mapillary, and flat KartaView and Panoramax photos), and **FOLLOW** (the globe camera follows the photo's view; Google 3D map only). The caption links to the image on the provider's site, and the globe credits each provider that is on ("© Mapillary contributors, CC BY-SA 4.0", "Street View imagery © Google", "© KartaView contributors, CC BY-SA 4.0", "© Panoramax contributors, licence per picture"). KartaView's own photo pages fail for signed-out visitors, so its link opens the full-size photo. +**Use.** Turn on **Street Level** under DATA LAYERS → Cameras, or with the pill in the STREET LEVEL panel header. Each provider has a chip in its own colour (Mapillary green), and one is lit at a time: lighting one switches the others off. Mapillary, which is free, is lit by default. The **IMAGERY** (ALL / 360° / FLAT) and **SINCE** filters apply to Mapillary, KartaView and Panoramax; Street View panoramas are all 360°, so with Street View lit the filters and legend are hidden. Click a coverage line for its image cones and a cone to open the photo. Light **KARTAVIEW** (purple) and zoom in below about 6 km: its lines are the streets KartaView has photos of, and a click on one opens the photo nearest the click, flat or 360° (drag to look around), with cones along its drive; **‹ ›** or ← → step through the drive (Shift+← →, ↑ ↓ and + − look around a 360° photo; pinch zooms on a phone). Photos KartaView has archived are skipped, and where all are archived the panel says so. For KartaView, **SINCE** goes by upload date, as KartaView's own map does (it can filter by nothing else), and the panel notes it under the slider; KartaView filters its lines on its own server, and a filtered area KartaView has not drawn before can take several seconds, and the unfiltered lines show meanwhile. Light **PANORAMAX** (orange) for the open, federated commons (strongest in France and Belgium): from orbit its dots show where it has pictures, and below about 60 km its lines are sequences. A click on a line opens that sequence's picture nearest the click without moving the camera, flat or 360°, with cones along the sequence around it; **‹ ›** or ← → step through it, and both filters apply. Its caption names the picture's authors (or the instance holding it) and licence (mostly CC BY-SA 4.0; pictures on the IGN instance are Licence Ouverte 2.0). On the **Google 3D** map (Google's terms keep Street View off other maps, where it does not appear), light **GOOGLE STREET VIEW** and click a street from below 3 km, or press **OPEN STREET VIEW** for the view centre, to open Google's panorama there; step along its arrows. Above the photo: **EXPAND**, **FIT / FILL** (Mapillary, and flat KartaView and Panoramax photos), and **FOLLOW** (the globe camera follows the photo's view; Google 3D map only). The caption links to the image on the provider's site, and the globe credits each provider that is on ("© Mapillary contributors, CC BY-SA 4.0", "Street View imagery © Google", "© KartaView contributors, CC BY-SA 4.0", "© Panoramax contributors, licence per picture"). KartaView's own photo pages fail for signed-out visitors, so its link opens the full-size photo. -Drag the panel header to float the panel, and resize it from any edge. Double-click the header or collapse it to put it back in the rail. Share links carry the provider switches and filters (`0.m.0` Mapillary off, `0.m.0_0.r.google-1` Street View on, `0.m.0_0.r.google-0*kartaview-1` KartaView on, as the app writes it (`0.m.0_0.r.kartaview-1` works too, and `0.m.0_0.r.panoramax-1` lights Panoramax; a link that leaves out `m` keeps Mapillary, the default, lit), `0.p.p` panoramas only, `0.s.365` the last year). To add a provider, implement the contract in `src/layers/streetLevel/registry.js` and register it in `src/app/layers/streetLevel.js`. +Drag the panel header to float the panel, and resize it from any edge. Double-click the header or collapse it to put it back in the rail. Share links carry the provider switches and filters (`0.m.0` Mapillary off, `0.m.0_0.r.google-1` Street View on, `0.m.0_0.r.google-0*kartaview-1*panoramax-0` KartaView on and `0.m.0_0.r.google-0*kartaview-0*panoramax-1` Panoramax on, as the app writes them (`0.m.0_0.r.kartaview-1` and `0.m.0_0.r.panoramax-1` work too; a link that leaves out `m` keeps Mapillary, the default, lit), `0.p.p` panoramas only, `0.s.365` the last year). To add a provider, implement the contract in `src/layers/streetLevel/registry.js` and register it in `src/app/layers/streetLevel.js`. ## 🎖️ Field Missions diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 8e5bff676..28c59191a 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -3403,7 +3403,13 @@ provider is on (16, ten minutes, `tileCache.js`, shared with KartaView), and above 60 km the `grid` layer's 0.1° density cells, binned to at most 96×96 points a tile (a z0 tile holds -~70,000 cells) and not counted as sequences: the panel says to zoom in. Both +~70,000 cells) and not counted as sequences: the panel says to zoom in. +A tile decodes in slices of about 8 ms, the page getting a turn between +them (a 2 MB z10 tile is ~200 ms of work). A busy API (429/503) holds the +tiles back for its Retry-After, which the browser cannot read here (the API +exposes no headers), so the wait doubles from 5 s to 5 minutes until it +answers; an answer may take as long as it keeps arriving (silent for 15 s +is given up), so a slow link still gets its megabyte tiles. Both filters apply: lines carry their day and camera type, and a cell stays while it holds pictures of the kind asked for (cells are undated). A click on a line asks the core (`actions.openNearest`) for that sequence's picture @@ -3412,8 +3418,8 @@ a click lands well off a line), without framing, then any sequence's within 50 m. Each lookup is two searches side by side, ranked by distance in the browser: `place_position` (nearest first, but even at `place_fov_tolerance=180` it leaves out pictures looking away from the -point) and a bounding box (those too, but in no order); the filters go to -both (`filter=field_of_view=360`, `datetime=/..`). A line is dated +point) and a bounding box (those too, roughly nearest first, so 20 are +asked for: ~60 kB, against ~600 kB for 100); the filters go to both (`filter=field_of_view=360`, `datetime=/..`). A line is dated by its sequence's first day, so a sequence still growing can hide under SINCE; a line with no camera type or day stays drawn. Lines keep the decoded longest-first order under the per-tile cap (`keepOrder`). Once the picture is on screen @@ -3421,20 +3427,31 @@ its sequence is highlighted and drawn as cones from the z15 tiles around it (their `pictures` layer, ~110 bytes a picture, against ~7.5 kB for a STAC item), plus the neighbours within 150 m of an edge, on their own connection (coverage tiles take three, searches two); each tile's list for the sequence -is kept, and the cones re-centre when a picture opens outside the tiles -they were drawn from, judged from the moment their load starts. ‹ › follow the picture's own `prev`/`next` +is kept. The open picture's own tile draws at once and the neighbours join +as they arrive; one that fails is not counted, so a picture opening in it +re-centres the cones there. The cones re-centre when a picture opens outside the tiles +they were drawn from, judged from the moment their load starts, and centre +only on a picture actually shown (never on a lookup that came back after +the user moved on); a lookup nobody waits for any more is stopped. ‹ › follow the picture's own `prev`/`next` links, so stepping costs one small search per picture. The viewer shows the -2,048 px SD derivative (equirectangular for a 360° picture) and swaps in the -HD original only when the viewer is wider than SD (flat) or, for a -panorama, expanded; once, shrunk while it decodes, and aborted if another -photo opens; originals wider than 8,192 px are never fetched, and any -picture file over 48 MB or 16,384 px is refused. The caption names every +2,048 px SD derivative (equirectangular for a 360° picture, decoded at +2,048 × 1,024) and swaps in the HD original only when the viewer is wider +than SD (flat) or, for a panorama, expanded; once, shrunk while it decodes. +When SD fails a flat picture falls back to the thumbnail, and to the +original only in the expanded viewer. Every download of a photo, first or +sharper, stops when another opens or the viewer closes. Originals wider than +8,192 px are never fetched; a file's header (JPEG, PNG, WebP, GIF) is read +before it decodes, and one declaring more than 64 MP or 16,384 px a side +is refused undecoded, as is an `` that decodes larger and any file +over 48 MB. The caption names every producer (an uploading app and its photographer; never an account named by an email address), or else the instance holding the picture ("via panoramax.ign.fr"), and the picture's licence (CC BY-SA 4.0, Licence Ouverte 2.0…; StreetPose `license`), and links to the picture in the Panoramax viewer. Esc anywhere in the docked viewer clears the selected -sequence before the panel can take it (KartaView too). The MCP globe panel allows the meta-catalogue and the +sequence before the panel can take it (KartaView too). After a GPU reset +the viewer draws the same picture again itself (`onReopen`), leaving the +core's lookups alone. The MCP globe panel allows the meta-catalogue and the three biggest instances; a picture on another instance shows only in the app. diff --git a/scripts/fixtures/street-level/panoramax.mjs b/scripts/fixtures/street-level/panoramax.mjs index 7506eb806..acca30e0f 100644 --- a/scripts/fixtures/street-level/panoramax.mjs +++ b/scripts/fixtures/street-level/panoramax.mjs @@ -249,9 +249,11 @@ export function answerPanoramaxRequest({ method, url }) { : true, ) .filter((p) => !since || Date.parse(p.datetime) >= Date.parse(since)); - // A box: every picture in it, in no particular order (id order here). + // A box: every picture in it, nearest its centre first, as the live API + // returns them (roughly: the browser ranks them again). if (q.has('bbox')) { const [west, south, east, north] = q.get('bbox').split(',').map(Number); + const centre = { lon: (west + east) / 2, lat: (south + north) / 2 }; return json({ type: 'FeatureCollection', features: kept @@ -259,9 +261,10 @@ export function answerPanoramaxRequest({ method, url }) { (p) => p.lon >= west && p.lon <= east && p.lat >= south && p.lat <= north, ) - .sort((a, b) => a.id.localeCompare(b.id)) + .map((p) => ({ p, metres: metresApart(centre, p) })) + .sort((a, b) => a.metres - b.metres || a.p.id.localeCompare(b.p.id)) .slice(0, Number(q.get('limit')) || 10) - .map((p) => item(p, pictures)), + .map(({ p }) => item(p, pictures)), }); } const [lon, lat] = (q.get('place_position') || '').split(',').map(Number); diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 1eb4f7014..f8c76fea4 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -943,6 +943,7 @@ "src/layers/streetLevel/providers/panoramax/viewer.js", "src/layers/streetLevel/photoViewer.js", "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", "src/layers/streetLevel/tileCache.js", "src/layers/streetLevel/providers/panoramax/policy.test.mjs", "src/layers/streetLevel/providers/panoramax/decode.test.mjs", @@ -952,6 +953,7 @@ "src/testSupport/panoramaxFixtures.mjs", "src/layers/streetLevel/photoViewer.test.mjs", "src/layers/streetLevel/limiter.test.mjs", + "src/layers/streetLevel/imageSize.test.mjs", "src/layers/streetLevel/lineCoverage.test.mjs", "src/testSupport/kartaViewFakes.mjs", "src/testSupport/googleStreetViewFakes.mjs", diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index 98e8d17ec..e7b494fba 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -1244,6 +1244,7 @@ "src/layers/streetLevel/providers/panoramax/viewer.js", "src/layers/streetLevel/photoViewer.js", "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", "src/layers/streetLevel/tileCache.js", "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", @@ -2225,6 +2226,7 @@ "src/layers/streetLevel/providers/panoramax/viewer.js", "src/layers/streetLevel/photoViewer.js", "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", "src/layers/streetLevel/tileCache.js", "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", @@ -2777,6 +2779,7 @@ "src/layers/streetLevel/providers/panoramax/viewer.js", "src/layers/streetLevel/photoViewer.js", "src/layers/streetLevel/limiter.js", + "src/layers/streetLevel/imageSize.js", "src/layers/streetLevel/tileCache.js", "src/layers/streetLevel/panorama.js", "src/layers/streetLevel/lineCoverage.js", diff --git a/src/app/layers/streetLevel.test.mjs b/src/app/layers/streetLevel.test.mjs index 16e2cc4b5..e82e762e5 100644 --- a/src/app/layers/streetLevel.test.mjs +++ b/src/app/layers/streetLevel.test.mjs @@ -64,6 +64,11 @@ test('the defaults are Mapillary (given its source), Street View, KartaView and assert.equal(layer.getParams().panoramax, true); assert.equal(layer.getParams().kartaview, false); assert.equal(layer.getParams().mapillary, false); + // The form the app writes (the README and CHANGELOG quote it). + assert.equal( + layer.getParams().providerSwitches, + 'google-0*kartaview-0*panoramax-1', + ); // Without the application's Street Level source there are none. assert.deepEqual(createApplicationStreetLevel({}).providerIds, []); }); diff --git a/src/layers/streetLevel/imageSize.js b/src/layers/streetLevel/imageSize.js new file mode 100644 index 000000000..dc1131495 --- /dev/null +++ b/src/layers/streetLevel/imageSize.js @@ -0,0 +1,64 @@ +/** + * An image's pixel size read from its header, before anything decodes it: + * a picture host can serve a small file that declares enormous dimensions, + * and a decode of it alone can take the tab (or the GPU process) down. + * JPEG (SOFn), PNG (IHDR), WebP (VP8, VP8L, VP8X) and GIF; null for any + * other format, or a header this does not reach. + * @param {Uint8Array} bytes The start of the file (a JPEG's EXIF can push its SOF past 64 kB). + * @returns {{width: number, height: number}|null} + */ +export function imageSize(bytes) { + if (!bytes || bytes.length < 12) return null; + const b = bytes; + const be16 = (i) => (b[i] << 8) | b[i + 1]; + const le16 = (i) => b[i] | (b[i + 1] << 8); + const le24 = (i) => b[i] | (b[i + 1] << 8) | (b[i + 2] << 16); + const ascii = (i, text) => + [...text].every((char, k) => b[i + k] === char.charCodeAt(0)); + // JPEG: walk the markers to the first start-of-frame. + if (b[0] === 0xff && b[1] === 0xd8) { + let i = 2; + while (i + 9 < b.length) { + if (b[i] !== 0xff) return null; + const marker = b[i + 1]; + // Fill bytes, and markers with no length. + if (marker === 0xff) { + i++; + continue; + } + if (marker === 0x01 || (marker >= 0xd0 && marker <= 0xd9)) { + i += 2; + continue; + } + const isFrame = + marker >= 0xc0 && + marker <= 0xcf && + marker !== 0xc4 && + marker !== 0xc8 && + marker !== 0xcc; + if (isFrame) return { width: be16(i + 7), height: be16(i + 5) }; + i += 2 + be16(i + 2); + } + return null; + } + // PNG: IHDR is the first chunk. + if (b[0] === 0x89 && ascii(1, 'PNG') && ascii(12, 'IHDR')) { + const u32 = (i) => + ((b[i] << 24) | (b[i + 1] << 16) | (b[i + 2] << 8) | b[i + 3]) >>> 0; + return { width: u32(16), height: u32(20) }; + } + if (ascii(0, 'RIFF') && ascii(8, 'WEBP') && b.length >= 30) { + if (ascii(12, 'VP8 ')) + return { width: le16(26) & 0x3fff, height: le16(28) & 0x3fff }; + if (ascii(12, 'VP8L')) + return { + width: 1 + (((b[22] & 0x3f) << 8) | b[21]), + height: + 1 + (((b[24] & 0x0f) << 10) | (b[23] << 2) | ((b[22] & 0xc0) >> 6)), + }; + if (ascii(12, 'VP8X')) return { width: 1 + le24(24), height: 1 + le24(27) }; + return null; + } + if (ascii(0, 'GIF8')) return { width: le16(6), height: le16(8) }; + return null; +} diff --git a/src/layers/streetLevel/imageSize.test.mjs b/src/layers/streetLevel/imageSize.test.mjs new file mode 100644 index 000000000..e83978596 --- /dev/null +++ b/src/layers/streetLevel/imageSize.test.mjs @@ -0,0 +1,82 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { imageSize } from './imageSize.js'; + +const bytes = (...parts) => + Uint8Array.from( + parts.flat().map((v) => (typeof v === 'string' ? v.charCodeAt(0) : v)), + ); +const ascii = (text) => [...text]; + +test('a JPEG’s size is read from its first frame, past an EXIF segment', () => { + const exif = [0xff, 0xe1, 0x00, 0x10, ...new Array(14).fill(0)]; + const sof = [0xff, 0xc0, 0x00, 0x11, 0x08, 0x0b, 0xc0, 0x0f, 0xc0]; + const jpeg = bytes([0xff, 0xd8], exif, sof, new Array(20).fill(0)); + assert.deepEqual(imageSize(jpeg), { width: 0x0fc0, height: 0x0bc0 }); + // Progressive (SOF2) too; a DHT (C4) is not a frame. + const dht = [0xff, 0xc4, 0x00, 0x04, 0, 0]; + const sof2 = [0xff, 0xc2, 0x00, 0x11, 0x08, 0x00, 0x10, 0x00, 0x20]; + assert.deepEqual( + imageSize(bytes([0xff, 0xd8], dht, sof2, new Array(10).fill(0))), + { + width: 32, + height: 16, + }, + ); +}); + +test('PNG, WebP (lossy, lossless, extended) and GIF sizes', () => { + const png = bytes( + [0x89], + ascii('PNG'), + [0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13], + ascii('IHDR'), + [0, 0, 0x10, 0, 0, 0, 0x08, 0], + ); + assert.deepEqual(imageSize(png), { width: 4096, height: 2048 }); + const riff = (chunk, rest) => + bytes(ascii('RIFF'), [0, 0, 0, 0], ascii('WEBP'), ascii(chunk), rest); + const vp8 = riff('VP8 ', [...new Array(10).fill(0), 0x00, 0x08, 0x00, 0x04]); + assert.deepEqual(imageSize(vp8), { width: 2048, height: 1024 }); + // VP8L: 14-bit width-1 and height-1 packed after the 0x2f signature. + const w = 99; + const h = 49; + const packed = w | (h << 14); + const vp8l = riff('VP8L', [ + 0, + 0, + 0, + 0, + 0x2f, + packed & 0xff, + (packed >> 8) & 0xff, + (packed >> 16) & 0xff, + (packed >> 24) & 0xff, + 0, + 0, + 0, + 0, + 0, + 0, + ]); + assert.deepEqual(imageSize(vp8l), { width: 100, height: 50 }); + const vp8x = riff( + 'VP8X', + [0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0x0f, 0x00, 0xff, 0x07, 0x00], + ); + assert.deepEqual(imageSize(vp8x), { width: 4096, height: 2048 }); + const gif = bytes(ascii('GIF89a'), [0x40, 0x01, 0xf0, 0x00], [0, 0]); + assert.deepEqual(imageSize(gif), { width: 320, height: 240 }); +}); + +test('anything else, or a header cut short, has no size', () => { + assert.equal(imageSize(null), null); + assert.equal(imageSize(new Uint8Array(4)), null); + assert.equal(imageSize(bytes(ascii(''))), null); + assert.equal( + imageSize( + bytes([0xff, 0xd8, 0xff, 0xe1, 0xff, 0xff], new Array(8).fill(0)), + ), + null, + ); +}); diff --git a/src/layers/streetLevel/lineCoverage.js b/src/layers/streetLevel/lineCoverage.js index ab9b996db..d57a98bd0 100644 --- a/src/layers/streetLevel/lineCoverage.js +++ b/src/layers/streetLevel/lineCoverage.js @@ -247,7 +247,6 @@ export function createLineCoverage({ entry.castRequested = false; entry.meshParts = null; if (swap && entry.kind === 'sequence' && entry.primitives?.length) { - finishSwap(entry); redrawTile(entry); return; } @@ -373,7 +372,22 @@ export function createLineCoverage({ /** Rebuild a tile's lines; the old ones stay up until the new are ready. */ function redrawTile(entry) { - const previous = entry.primitives; + let previous = entry.primitives; + if (entry.swap) { + // A swap still building: what is on screen is its old lines and + // whichever new ones are ready. Those stay; the rest never showed. + const shown = [...entry.swap.old]; + const building = []; + for (const record of previous) + (record.primitive.ready && !record.primitive.isDestroyed?.() + ? shown + : building + ).push(record); + entry.swap.stop(); + entry.swap = null; + removePrimitives(building); + previous = shown; + } entry.primitives = []; attachPrimitive(entry); removeWhenReady(entry, previous); diff --git a/src/layers/streetLevel/lineCoverage.test.mjs b/src/layers/streetLevel/lineCoverage.test.mjs index b0b5f7e94..7fdc19773 100644 --- a/src/layers/streetLevel/lineCoverage.test.mjs +++ b/src/layers/streetLevel/lineCoverage.test.mjs @@ -397,6 +397,63 @@ test('a tile redrawn for a new share keeps its old lines until the new ones are ); }); +test('a second redraw while a swap still builds keeps the lines on screen', async (t) => { + const { state, coverage, viewer } = setup(t, { + zoomFor: () => ({ zoom: 18, kind: 'sequence' }), + tuning: { + ...TUNING, + maxTiles: 9, + maxSequences: 900, + maxPoints: 90, + keepOrder: true, + }, + decoded: (centre, tile) => { + const dense = tile.x % 2 === 0 && tile.y % 2 === 0; + return { + sequences: dense + ? Array.from({ length: 30 }, (_, i) => + pointsLine(`d${tile.x}/${tile.y}/${i}`, 5, tile, centre), + ) + : [pointsLine(`s${tile.x}/${tile.y}`, 2, tile, centre)], + overview: [], + }; + }, + }); + coverage.refresh(); + assert.ok( + await until( + () => state.coverage.tiles.size > 1 && !state.coverage.pending.size, + ), + ); + // A tile mid-swap (its new lines never become ready here) with a line + // the cap left out: selecting that line redraws the tile again. + const entry = [...state.coverage.tiles.values()].find( + (tile) => + tile.swap && + tile.sequenceList.some((sequence) => !tile.sequences.has(sequence.id)), + ); + assert.ok(entry, 'a swapping tile with a line left out'); + const onScreen = [...entry.swap.old]; + const building = entry.primitives.map(({ primitive }) => primitive); + const left = entry.sequenceList.find( + (sequence) => !entry.sequences.has(sequence.id), + ); + state.sequence.selectedId = left.id; + coverage.recolorSequence(left.id, true); + assert.ok(entry.sequences.has(left.id), 'the selected line is drawn'); + const onGlobe = (primitive) => + viewer.scene.groundPrimitives.items.has(primitive) || + viewer.scene.primitives.items.has(primitive); + for (const { primitive } of onScreen) + assert.ok( + onGlobe(primitive), + 'the lines on screen stay until the new ones are ready', + ); + for (const primitive of building) + assert.ok(!onGlobe(primitive), 'the first swap’s unready lines are gone'); + assert.ok(entry.swap, 'still swapping'); +}); + test('a selected line outside the cap does not make its tile redraw on every settle, nor does one the filter hides', async (t) => { let pano = 'all'; const { state, coverage, viewer } = setup(t, { diff --git a/src/layers/streetLevel/photoViewer.js b/src/layers/streetLevel/photoViewer.js index 3de2c1af7..079cf82c8 100644 --- a/src/layers/streetLevel/photoViewer.js +++ b/src/layers/streetLevel/photoViewer.js @@ -3,6 +3,7 @@ import { PANO_FOV_DEFAULT_DEG, wrapDegrees, } from './panorama.js'; +import { imageSize } from './imageSize.js'; /** A photo download (tens of kilobytes to a few megabytes). */ export const IMAGE_TIMEOUT_MS = 15_000; @@ -10,6 +11,19 @@ export const IMAGE_TIMEOUT_MS = 15_000; /** A picture file or decoded side past these is refused (a photo is a few MB). */ export const MAX_IMAGE_BYTES = 48 * 1024 * 1024; export const MAX_IMAGE_SIDE = 16_384; +/** 64 MP: 256 MB once decoded to RGBA. A 16,384 px square is four times that. */ +export const MAX_IMAGE_PIXELS = 8_192 * 8_192; +/** Where in a file to look for its dimensions (a JPEG's EXIF comes first). */ +const HEADER_BYTES = 256 * 1024; + +/** Whether a picture this size is past what the viewer decodes. */ +export function tooManyPixels(width, height) { + return ( + width > MAX_IMAGE_SIDE || + height > MAX_IMAGE_SIDE || + width * height > MAX_IMAGE_PIXELS + ); +} /** Panorama texture widths: enough for a docked viewer, and for an expanded one. */ export const PANO_MID_SIZE = Object.freeze([2048, 1024]); @@ -43,19 +57,36 @@ export function loadImage( return loadBitmap(url, timeoutMs, resize, failure, signal); return new Promise((resolve, reject) => { if (!url) return reject(new Error(failure)); + if (signal?.aborted) return reject(new Error(failure)); const image = new Image(); if (crossOrigin) image.crossOrigin = 'anonymous'; image.decoding = 'async'; - const timer = setTimeout(() => { + // Stop the download: another photo opened, or the time ran out. + const stop = () => { + clearTimeout(timer); + signal?.removeEventListener?.('abort', stop); + image.onload = image.onerror = null; image.src = ''; reject(new Error(failure)); - }, timeoutMs); - image.onload = () => { + }; + const timer = setTimeout(stop, timeoutMs); + signal?.addEventListener?.('abort', stop, { once: true }); + const settle = () => { clearTimeout(timer); + signal?.removeEventListener?.('abort', stop); + }; + image.onload = () => { + settle(); + // An reads no header first: refuse what it decoded too large. + if (tooManyPixels(image.naturalWidth || 0, image.naturalHeight || 0)) { + image.src = ''; + reject(oversized(failure)); + return; + } resolve(image); }; image.onerror = () => { - clearTimeout(timer); + settle(); reject(new Error(failure)); }; image.src = url; @@ -80,6 +111,12 @@ async function loadBitmap(url, timeoutMs, resize, failure, signal) { const length = Number(response.headers?.get?.('content-length')); if (length > MAX_IMAGE_BYTES) throw oversized(failure); const blob = await boundedBlob(response, failure); + // Its declared size, before anything decodes it. + const size = imageSize( + new Uint8Array(await blob.slice(0, HEADER_BYTES).arrayBuffer()), + ); + if (size && tooManyPixels(size.width, size.height)) + throw oversized(failure); const bitmap = await (resize ? createImageBitmap(blob, { resizeWidth: resize[0], @@ -88,7 +125,7 @@ async function loadBitmap(url, timeoutMs, resize, failure, signal) { }) : createImageBitmap(blob)); // (A resized decode reports its new size: this guards unresized ones.) - if (bitmap.width > MAX_IMAGE_SIDE || bitmap.height > MAX_IMAGE_SIDE) { + if (tooManyPixels(bitmap.width, bitmap.height)) { bitmap.close?.(); throw oversized(failure); } @@ -152,7 +189,7 @@ export function panoSizeFor(canvasPixels, fovDeg) { * The panorama shown first, its texture width (0 when it is only a preview), and a size to shrink it to. * @property {(photo: object, size: [number, number], view: {expanded: boolean}) => {url: string, resize?: [number, number]|null, timeoutMs?: number}|null} panoSharp * A sharper panorama for a canvas that needs `size`, or null; `expanded` while the viewer is. - * @property {(photo: object) => Array} flatFirst Flat versions to try in turn. + * @property {(photo: object, view: {expanded: boolean}) => Array} flatFirst Flat versions to try in turn (`expanded` while the viewer is). * @property {(photo: object, pixels: number) => string|null} flatSharp * A sharper flat version for a picture this many device pixels wide, or null. * @property {(photo: object) => {title?: string|null, creator?: string|null, license?: string|null, externalUrl: string, capturedAtPrecision?: 'day'|'month'}} describe @@ -176,6 +213,7 @@ export function panoSizeFor(canvasPixels, fovDeg) { * @param {(photo: object, delta: -1|1) => Promise} options.step The photo `delta` along the sequence, or null at its end. * @param {(imageId: string) => void} options.onStep Open the photo ‹ › chose (through the core, so loading and errors show). * @param {(photo: object) => void} [options.onShown] A photo is on screen. + * @param {(photo: object) => void} [options.onShowing] The photo this open shows, just before its pose. * @param {(imageId: string) => void} [options.onReopen] Draw the same photo again after a GPU reset (default: `onStep`). * @param {() => boolean} [options.onEscape] Esc in the docked viewer: true when it was used (e.g. a selection cleared). * @param {object} [options.render] Render governor. @@ -191,6 +229,7 @@ export function createPhotoViewer({ step, onStep, onShown = () => {}, + onShowing = () => {}, onReopen = onStep, onEscape = () => false, render, @@ -227,8 +266,10 @@ export function createPhotoViewer({ let panoComing = null; /** A resize came while it loaded: look again once it lands. */ let lookAgain = false; - /** The sharper flat version on screen or on its way (one download each). */ - let flatSharpUrl = null; + /** The sharper flat version on screen, and the one on its way. */ + let flatSharp = { shown: null, loading: null }; + /** The open in progress: another open, a close or an unmount stops its downloads. */ + let openAbort = null; /** The photo an open is fetching: ‹ › step on from it, not the old one. */ let openingId = null; /** ‹ › chose this photo: keep the panorama's look direction for it. */ @@ -626,6 +667,9 @@ export function createPhotoViewer({ async function upgrade(target, seq, kind) { // A 360° photo shown flat (no WebGL) keeps its front view. if (kind === 'flat' && target.isPano) return; + // Another photo is opening: the one still shown is not worth more bytes. + if (openingId !== null && openingId !== target.id) return; + const signal = openAbort?.signal; if (kind === 'pano') { const size = panoSizeNow(); if (size[0] <= panoWidth) return; @@ -636,7 +680,7 @@ export function createPhotoViewer({ return; } const sharp = sources.panoSharp(target, size, { - expanded: Boolean(root?.closest?.('.sl-viewer-wrap-expanded')), + expanded: isExpanded(), }); if (!sharp?.url) return; const coming = { seq, width: size[0], controller: new AbortController() }; @@ -674,19 +718,28 @@ export function createPhotoViewer({ return; } const url = sources.flatSharp(target, viewerPixels()); - if (!url || url === flatSharpUrl) return; - flatSharpUrl = url; + if (!url || url === flatSharp.shown || url === flatSharp.loading) return; + const sharp = flatSharp; + sharp.loading = url; try { - const image = await loadAs(url, { crossOrigin: false }); + const image = await loadAs(url, { crossOrigin: false, signal }); if (seq !== openSeq || photo !== target || showing !== 'flat') return; picture.src = image.src; + sharp.shown = url; requestRender(); } catch { - // The first version stays; a later resize may try again. - if (flatSharpUrl === url && photo === target) flatSharpUrl = null; + /* the first version stays; a later resize may try again */ + } finally { + // Discarded or failed: nothing is on its way, so it may be asked again. + if (sharp.loading === url) sharp.loading = null; } } + /** The viewer is the expanded dialog, not docked in the panel. */ + function isExpanded() { + return Boolean(root?.closest?.('.sl-viewer-wrap-expanded')); + } + /** Another photo, or none: a sharper panorama still on its way is not needed. */ function dropSharpPanorama() { panoComing?.controller?.abort(); @@ -701,14 +754,16 @@ export function createPhotoViewer({ return panoSizeFor(width, renderer?.getView().fov ?? PANO_FOV_DEFAULT_DEG); } - async function showPano(target, seq, keepView) { + async function showPano(target, seq, keepView, signal) { // If the panorama fails, the caller shows the front view rather than // waiting on it twice. const first = sources.panoFirst(target); if (!first?.url) throw new Error(LOAD_FAILED); const image = await loadAs(first.url, { crossOrigin: true, - resize: first.resize ?? null, + signal, + // Never decoded at more than the texture it becomes. + resize: first.resize ?? PANO_MID_SIZE, }); if (seq !== openSeq) { image.close?.(); @@ -730,17 +785,20 @@ export function createPhotoViewer({ show('pano'); renderer.setImage(image); panoWidth = first.width || 0; - panoComing = null; + // A sharper version of the photo before was dropped by `open`; one of + // this photo, asked by a resize meanwhile, is still worth having. return true; } - async function showFlat(target, seq) { - const urls = sources.flatFirst(target).filter(Boolean); + async function showFlat(target, seq, signal) { + const urls = sources + .flatFirst(target, { expanded: isExpanded() }) + .filter(Boolean); let image; let failure = null; for (const url of urls) { try { - image = await loadAs(url, { crossOrigin: false }); + image = await loadAs(url, { crossOrigin: false, signal }); break; } catch (error) { failure = error; @@ -750,26 +808,27 @@ export function createPhotoViewer({ if (!image) throw failure || new Error(LOAD_FAILED); if (seq !== openSeq) return false; picture.src = image.src; - flatSharpUrl = null; + flatSharp = { shown: null, loading: null }; photo = target; show('flat'); return true; } - async function openPhoto(id, seq, keepView) { - const target = await getPhoto(id); + async function openPhoto(id, seq, keepView, signal) { + const target = await getPhoto(id, { signal }); if (seq !== openSeq) return; let shown = false; if (sources.hasPano(target) && ensureRenderer()) { try { - shown = await showPano(target, seq, keepView); + shown = await showPano(target, seq, keepView, signal); } catch { // The panorama failed: its front view is better than nothing. if (seq !== openSeq) return; - shown = await showFlat(target, seq); + shown = await showFlat(target, seq, signal); } - } else shown = await showFlat(target, seq); + } else shown = await showFlat(target, seq, signal); if (!shown) return; + onShowing(target); emit(); onShown(target); upgrade(target, seq, showing); @@ -790,11 +849,18 @@ export function createPhotoViewer({ const id = String(imageId); const seq = ++openSeq; dropSharpPanorama(); + // The previous open's downloads stop: only this photo is wanted now. + openAbort?.abort(); + const opening = (openAbort = new AbortController()); const keepView = keepViewFor === id; keepViewFor = null; openingId = id; try { - await openPhoto(id, seq, keepView); + await openPhoto(id, seq, keepView, opening.signal); + } catch (error) { + // A later open, or a close, stopped this one: that is not a failure. + if (seq !== openSeq) return; + throw error; } finally { if (openingId === id && seq === openSeq) openingId = null; } @@ -803,10 +869,12 @@ export function createPhotoViewer({ close() { openSeq++; dropSharpPanorama(); + openAbort?.abort(); + openAbort = null; openingId = null; keepViewFor = null; photo = null; - flatSharpUrl = null; + flatSharp = { shown: null, loading: null }; show(null); if (picture) picture.removeAttribute('src'); renderer?.clear?.(); @@ -815,10 +883,12 @@ export function createPhotoViewer({ unmount() { openSeq++; dropSharpPanorama(); + openAbort?.abort(); + openAbort = null; refused.clear(); openingId = null; keepViewFor = null; - flatSharpUrl = null; + flatSharp = { shown: null, loading: null }; teardown(); }, diff --git a/src/layers/streetLevel/photoViewer.test.mjs b/src/layers/streetLevel/photoViewer.test.mjs index 758a1e25b..09df91c2e 100644 --- a/src/layers/streetLevel/photoViewer.test.mjs +++ b/src/layers/streetLevel/photoViewer.test.mjs @@ -7,6 +7,7 @@ const PHOTOS = { a: { id: 'a', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: true }, b: { id: 'b', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: true }, f: { id: 'f', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: false }, + g: { id: 'g', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: false }, }; /** Sources whose panoramas and sharper flat versions are named after the photo. */ @@ -294,7 +295,11 @@ test('a picture file or bitmap past any real photo’s size is refused', async ( (globalThis.fetch = async () => ({ ok: true, headers: new Map([['content-length', String(length ?? '')]]), - blob: async () => ({ size: size ?? 10, bitmap: bitmap(width, height) }), + blob: async () => ({ + size: size ?? 10, + bitmap: bitmap(width, height), + slice: () => ({ arrayBuffer: async () => new ArrayBuffer(0) }), + }), })); const refusedForSize = (error) => { assert.match(error.message, /could not be loaded/); @@ -331,3 +336,203 @@ test('a picture file or bitmap past any real photo’s size is refused', async ( respond({}); assert.equal((await loadImage('https://x/a.jpg')).width, 100); }); + +/** A viewer whose loads are held, recording each load's signal by url. */ +function heldViewer() { + const pending = new Map(); + const signals = new Map(); + const loads = []; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: (url, options = {}) => + new Promise((resolve, reject) => { + loads.push(url); + signals.set(url, options.signal); + options.signal?.addEventListener('abort', () => + reject(new Error('aborted')), + ); + pending.set(url, { resolve, reject }); + }), + }); + const release = (url) => { + pending.get(url).resolve({ src: url }); + pending.delete(url); + }; + return { viewer, pending, signals, loads, release, host: fakeHost() }; +} + +test('a new open, a close or an unmount stops the downloads of the open before it', async () => { + const env = heldViewer(); + await env.viewer.mount(env.host); + const first = env.viewer.open('f'); + assert.ok(await until(() => env.signals.has('f-first'))); + const second = env.viewer.open('g'); + assert.equal(env.signals.get('f-first').aborted, true, 'f is not wanted now'); + await first.catch(() => {}); + assert.ok(await until(() => env.signals.has('g-first'))); + env.viewer.close(); + assert.equal(env.signals.get('g-first').aborted, true, 'closed'); + await second.catch(() => {}); + const third = env.viewer.open('f'); + assert.ok(await until(() => env.pending.has('f-first'))); + env.viewer.unmount(); + assert.equal(env.signals.get('f-first').aborted, true, 'unmounted'); + await third.catch(() => {}); +}); + +test('a sharper flat version on its way stops when another photo opens, and is asked again later', async (t) => { + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + const env = heldViewer(); + await env.viewer.mount(env.host); + parts(env.host).picture.clientWidth = 1200; + const opening = env.viewer.open('f'); + assert.ok(await until(() => env.pending.has('f-first'))); + env.release('f-first'); + await opening; + assert.ok( + await until(() => env.pending.has('f-sharp')), + 'the 1,200 px viewer asks for more', + ); + // Resizes while it loads (the panel animating) do not ask again. + env.viewer.resize(); + env.viewer.resize(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(env.loads.filter((url) => url === 'f-sharp').length, 1); + // g opens (and fails): f's sharp download stops, f stays on screen. + const failing = env.viewer.open('g'); + assert.equal(env.signals.get('f-sharp').aborted, true); + assert.ok(await until(() => env.pending.has('g-first'))); + env.pending.get('g-first').reject(new Error('gone')); + env.pending.delete('g-first'); + await failing.catch(() => {}); + // The discarded response does not block f's sharp version for good. + env.signals.delete('f-sharp'); + env.viewer.resize(); + assert.ok(await until(() => env.signals.has('f-sharp')), 'asked again'); +}); + +test('a resize while another photo opens does not upgrade the photo still shown', async (t) => { + t.after(() => delete globalThis.devicePixelRatio); + globalThis.devicePixelRatio = 1; + const env = heldViewer(); + await env.viewer.mount(env.host); + const opening = env.viewer.open('f'); + assert.ok(await until(() => env.pending.has('f-first'))); + env.release('f-first'); + await opening; + // g is opening; EXPAND resizes the viewer meanwhile. + const next = env.viewer.open('g'); + assert.ok(await until(() => env.pending.has('g-first'))); + parts(env.host).picture.clientWidth = 1200; + env.viewer.resize(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(env.pending.has('f-sharp'), false, 'f is on its way out'); + env.release('g-first'); + await next; + assert.ok(await until(() => env.pending.has('g-sharp')), 'g gets it'); +}); + +test('a panorama is always decoded at a bounded size', async () => { + const env = heldViewer(); + const options = []; + const viewer = createPhotoViewer({ + providerId: 'test', + name: 'Test', + sources: SOURCES, + getPhoto: async (id) => PHOTOS[id], + step: async () => null, + onStep() {}, + load: async (url, opts) => { + options.push(opts); + return { src: url }; + }, + createRenderer: () => ({ + setImage() {}, + setView() {}, + getView: () => ({ yaw: 0, pitch: 0, fov: 80 }), + resize() {}, + clear() {}, + destroy() {}, + }), + }); + await viewer.mount(env.host); + await viewer.open('a'); + // SOURCES' panoFirst names no size: the viewer still bounds it. + assert.deepEqual(options[0].resize, [2048, 1024]); +}); + +test('a header that declares a huge image is refused before anything decodes it', async (t) => { + const { loadImage } = await import('./photoViewer.js'); + const saved = { + fetch: globalThis.fetch, + createImageBitmap: globalThis.createImageBitmap, + }; + t.after(() => Object.assign(globalThis, saved)); + let decoded = 0; + globalThis.createImageBitmap = async () => { + decoded++; + return { width: 10, height: 10 }; + }; + // A 1 kB PNG that says it is 16,384 × 16,384. + const png = new Uint8Array(1024); + png.set([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13]); + png.set( + [...'IHDR'].map((c) => c.charCodeAt(0)), + 12, + ); + png.set([0, 0, 0x40, 0, 0, 0, 0x40, 0], 16); + globalThis.fetch = async () => new Response(png); + await assert.rejects( + loadImage('https://x/bomb.png'), + (error) => error.tooLarge === true, + ); + assert.equal(decoded, 0, 'never decoded'); +}); + +test('an that turns out too large is refused and let go', async (t) => { + const { loadImage, MAX_IMAGE_SIDE } = await import('./photoViewer.js'); + const saved = globalThis.Image; + t.after(() => (globalThis.Image = saved)); + let size = [MAX_IMAGE_SIDE + 1, 100]; + const images = []; + globalThis.Image = class { + constructor() { + images.push(this); + } + set src(value) { + this._src = value; + if (value) + queueMicrotask(() => { + [this.naturalWidth, this.naturalHeight] = size; + this.onload?.(); + }); + } + get src() { + return this._src; + } + }; + await assert.rejects( + loadImage('https://x/a.jpg', { crossOrigin: false }), + (error) => error.tooLarge === true, + ); + assert.equal(images.at(-1).src, '', 'its pixels let go'); + size = [4000, 3000]; + const ok = await loadImage('https://x/b.jpg', { crossOrigin: false }); + assert.equal(ok.naturalWidth, 4000); + // An aborted load stops and rejects. + size = [10, 10]; + const controller = new AbortController(); + const aborted = loadImage('https://x/c.jpg', { + crossOrigin: false, + signal: controller.signal, + }); + controller.abort(); + await assert.rejects(aborted); + assert.equal(images.at(-1).src, '', 'its download stops'); +}); diff --git a/src/layers/streetLevel/providers/kartaview/viewer.js b/src/layers/streetLevel/providers/kartaview/viewer.js index d33adef97..9b1d0884b 100644 --- a/src/layers/streetLevel/providers/kartaview/viewer.js +++ b/src/layers/streetLevel/providers/kartaview/viewer.js @@ -37,6 +37,7 @@ const KARTAVIEW_SOURCES = Object.freeze({ panoFirst: (photo) => ({ url: resizedImageUrl(photo.urls.pano, PANO_PREVIEW_SIZE), width: 0, + resize: PANO_PREVIEW_SIZE, }), panoSharp: (photo, size) => ({ url: resizedImageUrl(photo.urls.pano, size), diff --git a/src/layers/streetLevel/providers/kartaview/viewer.test.mjs b/src/layers/streetLevel/providers/kartaview/viewer.test.mjs index 1391b727c..9ca5174c1 100644 --- a/src/layers/streetLevel/providers/kartaview/viewer.test.mjs +++ b/src/layers/streetLevel/providers/kartaview/viewer.test.mjs @@ -883,6 +883,11 @@ test('the panorama is as sharp as the viewer needs: 2,048 docked, 4,096 once exp }); test('a CORS image is decoded off the main thread into an ImageBitmap where the browser can', async (t) => { + // A blob whose header names no size (the viewer then reads none). + const fakeBlob = { + size: 4, + slice: () => ({ arrayBuffer: async () => new ArrayBuffer(0) }), + }; const saved = { fetch: globalThis.fetch, bitmap: globalThis.createImageBitmap, @@ -896,7 +901,7 @@ test('a CORS image is decoded off the main thread into an ImageBitmap where the fetched.push({ url, mode: init.mode }); return url.includes('gone') ? { ok: false, status: 409 } - : { ok: true, blob: async () => 'blob' }; + : { ok: true, blob: async () => fakeBlob }; }; globalThis.createImageBitmap = async (blob) => ({ blob, @@ -906,7 +911,7 @@ test('a CORS image is decoded off the main thread into an ImageBitmap where the const bitmap = await loadImage('https://cdn.kartaview.org/a', { crossOrigin: true, }); - assert.deepEqual(bitmap, { blob: 'blob', width: 4096, height: 2048 }); + assert.deepEqual(bitmap, { blob: fakeBlob, width: 4096, height: 2048 }); assert.deepEqual(fetched, [ { url: 'https://cdn.kartaview.org/a', mode: 'cors' }, ]); diff --git a/src/layers/streetLevel/providers/panoramax/decode.js b/src/layers/streetLevel/providers/panoramax/decode.js index 0a354d06b..b8cbfeacc 100644 --- a/src/layers/streetLevel/providers/panoramax/decode.js +++ b/src/layers/streetLevel/providers/panoramax/decode.js @@ -2,6 +2,7 @@ import { PbfReader } from 'pbf'; import { VectorTile } from '@mapbox/vector-tile'; import { tileLocalToLonLat } from '../../tileMath.js'; import { + DECODE_SLICE_MS, OVERVIEW_BINS_PER_AXIS, SIMPLIFY_TOLERANCE_UNITS, panoramaxTimeMs, @@ -66,11 +67,62 @@ function dayMs(value) { * most OVERVIEW_BINS_PER_AXIS² a tile (a z0 tile holds ~70,000 cells). A * cell has 360° and flat counts but no date. * - `pictures` (z15) is left for `decodePictures`. + * + * In one go; `decodeCoverageTileSliced` gives the page a turn between + * slices. * @param {Uint8Array} bytes * @param {{x: number, y: number, z: number}} address * @returns {{sequences: Array<{id: string, parts: Array>, capturedAt: number, isPano: boolean}>, overview: Array<{id: string, lon: number, lat: number, capturedAt: null, isPano: boolean, hasPano: boolean, hasFlat: boolean, weight: number}>}} */ export function decodeCoverageTile(bytes, address) { + const steps = coverageSteps(bytes, address); + let step = steps.next(); + while (!step.done) step = steps.next(); + return step.value; +} + +/** Give the page a turn: input and rendering run before the next slice. */ +function nextTask() { + if (typeof globalThis.scheduler?.yield === 'function') + return globalThis.scheduler.yield(); + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +/** + * `decodeCoverageTile` in slices of about `sliceMs`, the page getting a + * turn between them: a 2 MB z10 tile over Paris is ~200 ms of decoding on + * a fast laptop, more on a phone, and would freeze the globe while the + * camera approaches. Stops at the next slice once `signal` aborts. + * @param {Uint8Array} bytes + * @param {{x: number, y: number, z: number}} address + * @param {{signal?: AbortSignal, sliceMs?: number, pause?: () => Promise, now?: () => number}} [options] + */ +export async function decodeCoverageTileSliced( + bytes, + address, + { + signal, + sliceMs = DECODE_SLICE_MS, + pause = nextTask, + now = () => performance.now(), + } = {}, +) { + const steps = coverageSteps(bytes, address); + let started = now(); + let step = steps.next(); + while (!step.done) { + if (now() - started >= sliceMs) { + await pause(); + signal?.throwIfAborted(); + started = now(); + } + step = steps.next(); + } + return step.value; +} + +/** The decode, a step a feature (see `decodeCoverageTile`). */ +function* coverageSteps(bytes, address) { const result = { sequences: [], overview: [] }; if (!bytes || !bytes.length) return result; const tile = new VectorTile(new PbfReader(bytes)); @@ -78,7 +130,7 @@ export function decodeCoverageTile(bytes, address) { const sequences = tile.layers.sequences; if (sequences) { - const lengths = new Map(); + const measured = []; for (let i = 0; i < sequences.length; i++) { const feature = sequences.feature(i); const props = feature.properties || {}; @@ -113,10 +165,13 @@ export function decodeCoverageTile(bytes, address) { ? false : null, }; - lengths.set(sequence, length); - result.sequences.push(sequence); + measured.push({ sequence, length }); + yield; } - result.sequences.sort((a, b) => lengths.get(b) - lengths.get(a)); + // A turn before the sort: over a city it is thousands of lines. + yield; + measured.sort((a, b) => b.length - a.length); + result.sequences = measured.map(({ sequence }) => sequence); } const grid = tile.layers.grid; @@ -127,6 +182,7 @@ export function decodeCoverageTile(bytes, address) { const feature = grid.feature(i); const props = feature.properties || {}; const point = feature.loadGeometry()?.[0]?.[0]; + if (i % 64 === 63) yield; if (!point) continue; const pano = Number(props.nb_360_pictures) || 0; const flat = Number(props.nb_flat_pictures) || 0; diff --git a/src/layers/streetLevel/providers/panoramax/decode.test.mjs b/src/layers/streetLevel/providers/panoramax/decode.test.mjs index c98ff3461..7caa25a63 100644 --- a/src/layers/streetLevel/providers/panoramax/decode.test.mjs +++ b/src/layers/streetLevel/providers/panoramax/decode.test.mjs @@ -1,6 +1,11 @@ import assert from 'node:assert/strict'; import test from 'node:test'; -import { decodeCoverageTile, decodePictures, simplifyLine } from './decode.js'; +import { + decodeCoverageTile, + decodeCoverageTileSliced, + decodePictures, + simplifyLine, +} from './decode.js'; import { OVERVIEW_BINS_PER_AXIS } from './policy.js'; import { encodePanoramaxTile } from '../../../../testSupport/panoramaxFixtures.mjs'; import { tileCentre } from '../../../../testSupport/streetLevelFakes.mjs'; @@ -69,10 +74,11 @@ test('sequence lines keep their parts apart, with their day and camera type', () test('density cells bin to a bounded number of points that keep the counts and kinds', () => { const tile = { x: 0, y: 0, z: 0 }; - // Two cells in one bin, one far away, one empty. + // Two cells in one bin (3.75° wide at z0) far enough apart to tell + // which it sits on, one far away, one empty. const grid = [ - { lon: 2.35, lat: 48.85, pano: 0, flat: 5 }, - { lon: 2.36, lat: 48.85, pano: 30, flat: 0 }, + { lon: 0.3, lat: 48.85, pano: 0, flat: 5 }, + { lon: 3.4, lat: 48.85, pano: 30, flat: 0 }, { lon: -73.9, lat: 40.7, pano: 1, flat: 0 }, { lon: 10, lat: 10, pano: 0, flat: 0 }, ]; @@ -87,7 +93,10 @@ test('density cells bin to a bounded number of points that keep the counts and k assert.equal(paris.hasFlat, true); assert.equal(paris.isPano, true, 'mostly 360°'); assert.equal(paris.capturedAt, null, 'cells are undated'); - assert.ok(near(paris.lon, 2.36, 0.2), 'sits on the busiest cell'); + assert.ok( + near(paris.lon, 3.4, 0.2), + `sits on the busiest cell: ${paris.lon}`, + ); const nyc = overview.find((p) => p.lon < 0); assert.deepEqual([nyc.hasPano, nyc.hasFlat, nyc.weight], [true, false, 1]); }); @@ -134,6 +143,71 @@ test('a dense grid tile never draws more than the bins allow', () => { ); }); +/** A tile with many lines and a dense grid, for the sliced decode. */ +function busyTile() { + const { tile, lon, lat } = tileCentre(2.35, 48.85); + const sequences = Array.from({ length: 200 }, (_, i) => ({ + id: `s${i}`, + parts: [ + [ + [lon, lat + i / 1e6], + [lon + (i + 1) / 73_000, lat + i / 1e6], + ], + ], + })); + const grid = Array.from({ length: 500 }, (_, i) => ({ + lon: lon + (i % 25) / 1e4, + lat: lat + Math.floor(i / 25) / 1e4, + })); + return { tile, bytes: encodePanoramaxTile(tile, { sequences, grid }) }; +} + +test('a sliced decode gives the page turns and decodes the same as one in one go', async () => { + const { tile, bytes } = busyTile(); + let clock = 0; + let pauses = 0; + const sliced = await decodeCoverageTileSliced(bytes, tile, { + sliceMs: 8, + // Each step "takes" a millisecond: a turn every eight features. + now: () => clock++, + pause: async () => { + pauses++; + }, + }); + assert.ok(pauses > 20, `${pauses} turns`); + assert.deepEqual(sliced, decodeCoverageTile(bytes, tile)); +}); + +test('a sliced decode stops at the next turn once aborted', async () => { + const { tile, bytes } = busyTile(); + const controller = new AbortController(); + let clock = 0; + let pauses = 0; + await assert.rejects( + decodeCoverageTileSliced(bytes, tile, { + signal: controller.signal, + now: () => (clock += 10), + pause: async () => { + if (++pauses === 2) controller.abort(); + }, + }), + { name: 'AbortError' }, + ); + assert.equal(pauses, 2, 'no work after the abort'); +}); + +test('a fast sliced decode takes no turns', async () => { + const { tile, bytes } = busyTile(); + let pauses = 0; + await decodeCoverageTileSliced(bytes, tile, { + now: () => 0, + pause: async () => { + pauses++; + }, + }); + assert.equal(pauses, 0); +}); + test('an empty tile decodes to nothing', () => { for (const bytes of [new Uint8Array(0), null, undefined]) assert.deepEqual(decodeCoverageTile(bytes, { x: 0, y: 0, z: 0 }), { diff --git a/src/layers/streetLevel/providers/panoramax/index.js b/src/layers/streetLevel/providers/panoramax/index.js index b6cfa93c8..3ce1f5d21 100644 --- a/src/layers/streetLevel/providers/panoramax/index.js +++ b/src/layers/streetLevel/providers/panoramax/index.js @@ -9,7 +9,7 @@ import { } from '../../tileMath.js'; import { metresBetween } from '../../view.js'; import { createLru, createTileCache } from '../../tileCache.js'; -import { decodeCoverageTile, decodePictures } from './decode.js'; +import { decodeCoverageTileSliced, decodePictures } from './decode.js'; import { createPanoramaxSource } from './source.js'; import { createPanoramaxViewer } from './viewer.js'; import { @@ -230,16 +230,23 @@ export function createPanoramaxProvider({ return photo; } - /** Lookups in flight, shared: ‹ › and the viewer ask for the same picture. */ + /** + * Lookups in flight, shared: ‹ › and the viewer ask for the same + * picture. A lookup stops once every caller waiting on it has given up + * (its search would hold one of the two API connections for nothing). + */ const inflight = new Map(); - function getPhoto(photoId) { + function getPhoto(photoId, { signal } = {}) { const id = String(photoId); const cached = photos.get(id); if (cached) return Promise.resolve(cached); + if (signal?.aborted) return Promise.reject(signal.reason); let lookup = inflight.get(id); if (!lookup) { - lookup = source - .getItem(id) + const controller = new AbortController(); + lookup = { controller, waiting: 0 }; + lookup.promise = source + .getItem(id, { signal: controller.signal }) .catch((error) => { // Ids are UUIDs: the API answers 400 to anything else. if (error?.status === 400) return null; @@ -250,10 +257,39 @@ export function createPanoramaxProvider({ if (!photo) throw new Error(GONE_MESSAGE); return photo; }) - .finally(() => inflight.delete(id)); + .finally(() => { + if (inflight.get(id) === lookup) inflight.delete(id); + }); + lookup.promise.catch(() => {}); inflight.set(id, lookup); } - return lookup; + const shared = lookup; + shared.waiting++; + if (!signal) shared.waiting = Infinity; + return new Promise((resolve, reject) => { + const leave = () => { + reject(signal.reason); + shared.waiting--; + // A caller in the same tick (the same photo opened again) may + // still join: stop only if nobody has. + queueMicrotask(() => { + if (shared.waiting > 0) return; + shared.controller.abort(signal.reason); + if (inflight.get(id) === shared) inflight.delete(id); + }); + }; + signal?.addEventListener('abort', leave, { once: true }); + shared.promise.then( + (photo) => { + signal?.removeEventListener('abort', leave); + resolve(photo); + }, + (error) => { + signal?.removeEventListener('abort', leave); + reject(error); + }, + ); + }); } /** @@ -281,7 +317,11 @@ export function createPanoramaxProvider({ size: COVERAGE_TILE_CACHE_SIZE, load: (tile, { signal }) => readable(async () => - decodeCoverageTile(await source.getTile(tile, { signal }), tile), + decodeCoverageTileSliced( + await source.getTile(tile, { signal }), + tile, + { signal }, + ), ), }); /** A z15 tile's pictures of one sequence, by `z/x/y:sequence`. */ @@ -326,21 +366,40 @@ export function createPanoramaxProvider({ const tiles = pictureTilesAround(anchor); // Known from the start: a picture opening while these load is // judged against them (re-centred if outside, left alone inside). - drawn = { sequenceId, tiles: new Set(tiles.map(tileKey)) }; - // The home tile must load; a neighbour that fails only thins the cones. - const lists = await Promise.allSettled( - tiles.map((tile) => picturesIn(tile, sequenceId, signal)), - ); + const covering = { sequenceId, tiles: new Set(tiles.map(tileKey)) }; + drawn = covering; + const conesOf = (lists) => + thinImages( + [ + ...new Map( + lists.flat().map((image) => [image.id, image]), + ).values(), + ].sort((a, b) => a.capturedAt - b.capturedAt), + ); + const keyOf = (tile) => `${tileKey(tile)}:${sequenceId}`; + // The home tile is needed now (it holds the open picture). + const home = await picturesIn(tiles[0], sequenceId, signal); signal?.throwIfAborted(); - if (lists[0].status === 'rejected') throw lists[0].reason; - const byId = new Map( - lists - .flatMap((list) => list.value ?? []) - .map((image) => [image.id, image]), - ); - return thinImages( - [...byId.values()].sort((a, b) => a.capturedAt - b.capturedAt), + // Neighbours already read draw with it; the rest (each up to + // 1.4 MB in central Paris) join the cones as they arrive. + const neighbours = tiles.slice(1); + const kept = neighbours + .map((tile) => pictureLists.get(keyOf(tile))) + .filter(Boolean); + const missing = neighbours.filter( + (tile) => !pictureLists.get(keyOf(tile)), ); + if (missing.length) + Promise.allSettled( + missing.map((tile) => picturesIn(tile, sequenceId, signal)), + ).then((lists) => { + if (signal?.aborted || drawn !== covering) return; + dropFailed(covering, missing, lists); + // Only a tile that arrived now redraws (no loop on failures). + if (lists.some((list) => list.status === 'fulfilled')) + cones.select(sequenceId, { force: true }); + }); + return conesOf([home, ...kept]); }, colors: COLORS, pickPrefix: PICK_PREFIX.image, @@ -350,6 +409,17 @@ export function createPanoramaxProvider({ coverage.recolorSequence(sequenceId, selected), }); + /** + * A tile that failed is not covered: a picture opening in it later + * re-centres the cones there (and asks for the tile again). + */ + function dropFailed(covering, tiles, lists) { + lists.forEach((list, i) => { + if (list.status === 'rejected') + covering.tiles.delete(tileKey(tiles[i])); + }); + } + /** * The cones show the z15 tiles around one picture: once a picture of * the selected sequence opens outside them (‹ ›, a cone), re-centre. @@ -363,11 +433,23 @@ export function createPanoramaxProvider({ } const viewer = createPanoramaxViewer({ - getPhoto: async (photoId) => { - const photo = await getPhoto(photoId); + getPhoto, + // The cones centre on the picture actually shown, never on a + // lookup that came back after the user moved on. + onShowing: (photo) => { if (photo.sequenceId) anchors.set(photo.sequenceId, photo); - return photo; }, + // After a GPU reset the same picture is drawn again by the viewer + // itself: through the core it would cancel a lookup the user + // started. A failure still shows. + onReopen: (photoId) => + viewer + .open(photoId) + .catch((error) => + context.actions.reportError( + error?.message || 'The Panoramax photo could not be loaded', + ), + ), step, onStep: (photoId) => context.actions.openImage(photoId, { frame: false }), @@ -449,9 +531,9 @@ export function createPanoramaxProvider({ coverageStats() { const { kind } = state.coverage; const lines = kind === 'sequence'; - const drawn = coverage.sequenceCount(); + const shown = coverage.sequenceCount(); // Density dots are not sequences: they are not counted as such. - const count = lines ? drawn : 0; + const count = lines ? shown : 0; const loading = state.coverage.pending.size > 0; const settled = !loading && !state.coverage.lastError; let fallback = ''; @@ -461,7 +543,7 @@ export function createPanoramaxProvider({ ); if (kind === 'overview') fallback = - drawn > 0 || !settled + shown > 0 || !settled ? OVERVIEW_HINT : filteredOut ? FILTERED_HINT diff --git a/src/layers/streetLevel/providers/panoramax/policy.js b/src/layers/streetLevel/providers/panoramax/policy.js index 7e8ccca0d..198b8bb8b 100644 --- a/src/layers/streetLevel/providers/panoramax/policy.js +++ b/src/layers/streetLevel/providers/panoramax/policy.js @@ -66,6 +66,13 @@ export const COVERAGE_TUNING = Object.freeze({ * within GPS noise), and 6–10% of the points GPS tracks carry. */ export const SIMPLIFY_TOLERANCE_UNITS = 4; + +/** + * Coverage tiles decode in slices of this many ms, the page getting a turn + * between them (decode.js `decodeCoverageTileSliced`): under a frame. + */ +export const DECODE_SLICE_MS = 8; + /** * Grid cells per overview tile, at most: a z0 tile holds ~70,000 cells * (1.9 MB), drawn as this many binned points. @@ -89,6 +96,14 @@ export const PICTURE_TILE_ZOOM = 15; export const PICTURE_TILE_EDGE_M = 150; export const TILE_TIMEOUT_MS = 20_000; +/** + * Once an answer has started, it may take as long as it keeps coming: a + * 2 MB tile on a slow link outlasts any whole-request timeout, so only a + * stream silent this long is given up. + */ +export const STREAM_IDLE_TIMEOUT_MS = 15_000; +/** Busy (429/503) without a Retry-After the browser can read: back off from this. */ +export const BUSY_BACKOFF_BASE_SEC = 5; export const API_TIMEOUT_MS = 12_000; /** Real tiles are under 6 MB (z6 over France); refuse anything absurd. */ export const MAX_RESPONSE_BYTES = 16 * 1024 * 1024; @@ -111,8 +126,12 @@ export const NEAREST_LIMIT = 10; * beside it has those). */ export const NEAREST_FOV_TOLERANCE_DEG = 180; -/** Pictures in the bounding box around a click, at most. */ -export const NEAREST_BOX_LIMIT = 100; +/** + * Pictures in the bounding box around a click, at most. The API returns + * them roughly nearest first (the first ten of 100 are the first ten of + * 10), and the browser ranks them exactly: 20 is ~60 kB, 100 ~600 kB. + */ +export const NEAREST_BOX_LIMIT = 20; /** * A line clicked from high up may be tens of metres off the click: its * sequence is searched within the camera height / 100 (50–400 m). @@ -194,10 +213,12 @@ function linkedId(item, rel) { } } -/** An email address anywhere in a name (linear: names come from any instance). */ +/** + * An email address anywhere in a name, at any of its '@'s. Linear: names + * come from any instance, and each run between '@'s is scanned once. + */ function looksLikeEmail(name) { - const at = name.indexOf('@'); - return at > 0 && name.indexOf('.', at + 2) !== -1; + return /[^\s@]@[^\s@]+\.[^\s@]/.test(name); } /** @@ -217,12 +238,14 @@ function creatorOf(item) { // Every producer is credited (an uploading app and its photographer), // each name once, none that is an email address. const credited = []; + const seen = new Set(); for (const value of names) { if (typeof value !== 'string') continue; const name = value.trim().slice(0, 80); if (!name || looksLikeEmail(value)) continue; - if (credited.some((other) => other.toLowerCase() === name.toLowerCase())) - continue; + const key = name.toLowerCase(); + if (seen.has(key)) continue; + seen.add(key); credited.push(name); } return credited.length ? credited.join(', ').slice(0, 120) : null; diff --git a/src/layers/streetLevel/providers/panoramax/policy.test.mjs b/src/layers/streetLevel/providers/panoramax/policy.test.mjs index 770f52a26..1bb1ea578 100644 --- a/src/layers/streetLevel/providers/panoramax/policy.test.mjs +++ b/src/layers/streetLevel/providers/panoramax/policy.test.mjs @@ -224,6 +224,23 @@ test('hostile item data: a malformed link escape, null coordinates, a huge produ 'Carto@Cité', 'an @ without a domain is a name', ); + // The address need not follow the first '@'. + assert.equal(named('@team someone@example.org'), null); + assert.equal(named('Carto@Cité (mapper@example.org)'), null); + assert.equal(named(`${'@'.repeat(100_000)}x`), `${'@'.repeat(80)}`); +}); + +test('thousands of producers are credited in linear time', () => { + const item = stacItem({ id: 'a', lon: 1, lat: 2 }); + item.providers = Array.from({ length: 50_000 }, (_, i) => ({ + name: `producer-${i}`, + roles: ['producer'], + })); + const started = performance.now(); + const { creator } = normalizeItem(item); + assert.ok(performance.now() - started < 1000); + assert.ok(creator.startsWith('tester, producer-0, producer-1')); + assert.ok(creator.length <= 120); }); test('the instance holding a picture is named (its via host), never as its author', () => { diff --git a/src/layers/streetLevel/providers/panoramax/provider.test.mjs b/src/layers/streetLevel/providers/panoramax/provider.test.mjs index 031f376d7..4bee380ae 100644 --- a/src/layers/streetLevel/providers/panoramax/provider.test.mjs +++ b/src/layers/streetLevel/providers/panoramax/provider.test.mjs @@ -726,12 +726,16 @@ test('from high up the density grid draws points, not counted as sequences, and test('switching off clears the cones and forgets a pending line click', async (t) => { const items = sequence('s', 3); const env = await mounted({ items, tiles: pictureTiles(items) }); - await withCones(t, env); + const cones = await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => cones.count() === 3), 'cones to clear'); env.instance.handlePick(`${PICK_PREFIX.sequence}s`, { groundPoint: () => east(0), }); env.instance.deactivate(); assert.equal(env.instance.sequenceStats().selectedId, null); + assert.equal(cones.count(), 0, 'the cones are gone'); // The click's preference is gone: a later lookup asks any sequence. await env.instance.nearestImage(east(0)); assert.equal(env.source.calls.nearest.at(-1).collections, null); @@ -1001,15 +1005,242 @@ test('a re-centre while the cones still load does not inherit the aborted load const second = new Set( pictureTilesAround({ lon, lat }).map((tile) => `${tile.x}/${tile.y}`), ); - const ids = new Set(cones.ids()); const covered = items.filter((item) => { const tile = tileOf(item); return second.has(`${tile.x}/${tile.y}`); }); assert.ok(covered.length > 0); - assert.equal( - covered.filter((item) => ids.has(item.id)).length, - covered.length, - 'every picture in the re-centred tiles has its cone (the shared ones too)', + // The neighbours join as they arrive: wait for them, then every picture + // in the re-centred tiles has its cone (the shared ones too). + const coneCount = () => { + const ids = new Set(cones.ids()); + return covered.filter((item) => ids.has(item.id)).length; + }; + assert.ok( + await until(() => coneCount() === covered.length), + `${coneCount()} of ${covered.length}`, ); }); + +test('after a GPU reset the viewer draws the same picture itself, leaving the core’s lookups alone; a failure is reported', async () => { + const items = sequence('p', 2, { pano: true }); + let failLoads = false; + const view = { yaw: 0, pitch: 0, fov: 90 }; + const definition = createPanoramaxProvider({ + source: fakeSource({ items }), + viewerOptions: { + load: async (url) => { + if (failLoads) + throw new Error('The Panoramax photo could not be loaded'); + return { src: url, width: 2048, height: 1024 }; + }, + createRenderer: () => ({ + setImage() {}, + setView: (next) => Object.assign(view, next), + getView: () => ({ ...view }), + resize() {}, + clear() {}, + destroy() {}, + }), + }, + }); + const { context, actions } = fakeContext(); + const instance = definition.create(context); + actions.instance = instance; + const host = fakeHost(); + await instance.viewer.mount(host); + const poses = []; + instance.viewer.onPose((pose) => poses.push(pose.imageId)); + await instance.viewer.open('p-0'); + const canvas = () => host.children[0].children[1]; + const kind = () => host.children[0].dataset.kind; + assert.equal(kind(), 'pano'); + canvas().fire('webglcontextlost'); + assert.ok(await until(() => kind() === 'pano'), 'drawn again'); + assert.deepEqual(poses, ['p-0', 'p-0']); + assert.deepEqual(actions.opened, [], 'not through the core'); + failLoads = true; + canvas().fire('webglcontextlost'); + assert.ok(await until(() => actions.errors.some(Boolean))); + assert.match(actions.errors.filter(Boolean)[0], /could not be loaded/); +}); + +/** A source whose lookup of `heldId` waits for `release()` and ignores aborts. */ +function holdLookup(source, heldId) { + const held = { release: null, signals: [] }; + const getItem = source.getItem; + source.getItem = async (id, options = {}) => { + held.signals.push({ id, signal: options.signal }); + if (id === heldId) await new Promise((go) => (held.release = go)); + return getItem(id); + }; + return held; +} + +test('a lookup that comes back after the user moved on does not move the cones', async (t) => { + const items = sequence('long', 400); + const env = await mounted({ items, tiles: pictureTiles(items) }); + await withCones(t, env); + const held = holdLookup(env.source, 'long-0'); + const late = env.instance.viewer.open('long-0'); + assert.ok(await until(() => held.release !== null)); + await env.instance.viewer.open('long-399'); + await late; + // The first lookup answers anyway (a source that ignores the abort). + held.release(); + await new Promise((resolve) => setTimeout(resolve, 0)); + await env.instance.selectSequence('long'); + assert.ok(await until(() => !env.instance.sequenceStats().loading)); + const [lon, lat] = items[399].geometry.coordinates; + const [far] = pictureTilesAround({ lon, lat }); + const [lon0, lat0] = items[0].geometry.coordinates; + const [near] = pictureTilesAround({ lon: lon0, lat: lat0 }); + assert.equal(env.source.calls.tiles[0], `15/${far.x}/${far.y}`); + assert.ok( + !env.source.calls.tiles.includes(`15/${near.x}/${near.y}`), + 'nothing around the picture the user left', + ); +}); + +test('a lookup nobody waits for any more is stopped; one still wanted is not', async () => { + const items = sequence('s', 3); + const env = await mounted({ items }); + const held = holdLookup(env.source, 's-0'); + const first = env.instance.viewer.open('s-0'); + assert.ok(await until(() => held.release !== null)); + const [{ signal }] = held.signals; + assert.equal(signal.aborted, false); + await env.instance.viewer.open('s-1'); + await first; + await Promise.resolve(); + assert.equal(signal.aborted, true, 'the user moved on: its search stops'); + // The same picture asked again at once shares the lookup, which goes on. + held.release = null; + held.signals.length = 0; + const again = env.instance.viewer.open('s-2'); + const twice = env.instance.viewer.open('s-2'); + await Promise.all([again, twice]); + assert.equal(held.signals.length, 1); + assert.equal(held.signals[0].signal.aborted, false); + assert.equal(env.poses.at(-1), 's-2'); +}); + +/** + * Two pictures of sequence `e` either side of a z15 tile's west edge, + * half-way down it: the cones around the first read its tile and the + * west neighbour. + */ +function acrossWestEdge() { + const home = tileCentre(ORIGIN.lon, ORIGIN.lat, 15); + const box = tileBounds(home.tile.x, home.tile.y, 15); + const lat = (box.north + box.south) / 2; + const items = [ + stacItem({ + id: 'e-0', + sequenceId: 'e', + lon: box.west + 0.00005, + lat, + nextId: 'e-1', + }), + stacItem({ + id: 'e-1', + sequenceId: 'e', + lon: box.west - 0.00005, + lat, + prevId: 'e-0', + }), + ]; + const west = `15/${home.tile.x - 1}/${home.tile.y}`; + return { items, west }; +} + +test( + 'the open picture’s own tile draws its cones at once; a neighbour’s join as it arrives', + { timeout: 10_000 }, + async (t) => { + const { items, west } = acrossWestEdge(); + const tiles = pictureTiles(items); + let release = null; + const env = await mounted({ + items, + tiles: async (tile) => { + if (`${tile.z}/${tile.x}/${tile.y}` === west) + await new Promise((go) => (release = go)); + return tiles(tile); + }, + }); + const cones = await withCones(t, env); + await env.instance.viewer.open('e-0'); + const [lon, lat] = items[0].geometry.coordinates; + assert.equal( + pictureTilesAround({ lon, lat }).length, + 2, + 'the west neighbour is read too', + ); + // Not awaited: a load that waited for the neighbour would hang here. + env.instance.selectSequence('e'); + assert.ok(await until(() => release !== null), 'the neighbour is held'); + assert.ok( + await until(() => cones.count() === 1), + 'the home tile’s cone shows', + ); + assert.deepEqual(cones.ids(), ['e-0']); + release(); + assert.ok(await until(() => cones.count() === 2), 'the neighbour’s joins'); + }, +); + +test('a neighbour tile that failed is not counted as shown: › into it re-centres the cones there', async (t) => { + const { items, west } = acrossWestEdge(); + const tiles = pictureTiles(items); + let failing = true; + const env = await mounted({ + items, + tiles: (tile) => { + if (failing && `${tile.z}/${tile.x}/${tile.y}` === west) + throw Object.assign(new Error('Panoramax is busy'), { status: 503 }); + return tiles(tile); + }, + }); + const cones = await withCones(t, env); + await env.instance.viewer.open('e-0'); + await env.instance.selectSequence('e'); + assert.ok(await until(() => cones.count() === 1)); + // Let the neighbour's failure land. + assert.ok(await until(() => env.source.calls.tiles.includes(west))); + await new Promise((resolve) => setTimeout(resolve, 10)); + failing = false; + const asked = env.source.calls.tiles.filter((key) => key === west).length; + env.next.fire('click'); + assert.ok(await until(() => env.poses.at(-1) === 'e-1')); + assert.ok( + await until( + () => env.source.calls.tiles.filter((key) => key === west).length > asked, + ), + 'the failed tile is asked again', + ); + assert.ok(await until(() => cones.count() === 2), 'e-1 has its cone'); +}); + +test('a busy or timed-out tile keeps its own message; only a body that is not a tile reads as unreadable', async (t) => { + const items = sequence('s', 3); + const failures = [ + Object.assign(new Error('Panoramax is busy'), { status: 429 }), + Object.assign(new Error('Panoramax did not answer in time — try again'), { + status: 0, + }), + ]; + for (const failure of failures) { + const env = await mounted({ + items, + tiles: () => { + throw failure; + }, + }); + await withCones(t, env); + await env.instance.viewer.open('s-0'); + await env.instance.selectSequence('s'); + assert.ok(await until(() => env.actions.errors.some(Boolean))); + assert.equal(env.actions.errors.filter(Boolean).at(-1), failure.message); + } +}); diff --git a/src/layers/streetLevel/providers/panoramax/source.js b/src/layers/streetLevel/providers/panoramax/source.js index 1217b19bf..9678233ad 100644 --- a/src/layers/streetLevel/providers/panoramax/source.js +++ b/src/layers/streetLevel/providers/panoramax/source.js @@ -1,10 +1,8 @@ import { createLimiter } from '../../limiter.js'; - -/** Longest wait a Retry-After is honoured for. */ -const RETRY_AFTER_MAX_SEC = 300; import { API_CONCURRENCY, API_TIMEOUT_MS, + BUSY_BACKOFF_BASE_SEC, MAX_RESPONSE_BYTES, NEAREST_BOX_LIMIT, NEAREST_FOV_TOLERANCE_DEG, @@ -12,10 +10,14 @@ import { NEAREST_RADIUS_M, PANORAMAX_API, PICTURE_TILE_CONCURRENCY, + STREAM_IDLE_TIMEOUT_MS, TILE_CONCURRENCY, TILE_TIMEOUT_MS, } from './policy.js'; +/** Longest wait a Retry-After (or the backoff) asks for. */ +const RETRY_AFTER_MAX_SEC = 300; + /** An HTTP or API failure; `retryAfterSec` is set while Panoramax is busy. */ export class PanoramaxSourceError extends Error { constructor(message, { status = 0, retryAfterSec = null } = {}) { @@ -29,17 +31,30 @@ export class PanoramaxSourceError extends Error { const NO_ANSWER = 'Panoramax did not answer in time — try again'; const UNREACHABLE = 'Panoramax could not be reached — check the connection'; -/** Seconds from a Retry-After header, or null. */ -function retryAfterSeconds(response) { +/** + * Seconds from a Retry-After header (delay seconds or an HTTP date), or + * null. The meta-catalogue exposes no headers to the browser, so in the + * app this is usually null and the caller backs off on its own. + */ +export function retryAfterSeconds(response, now = Date.now()) { const header = response.headers?.get?.('retry-after'); - const value = header == null || header === '' ? NaN : Number(header); - if (!Number.isFinite(value) || value < 0) return null; + if (header == null || header === '') return null; + let value = Number(header); + if (!Number.isFinite(value)) { + const at = Date.parse(header); + if (!Number.isFinite(at)) return null; + value = Math.ceil((at - now) / 1000); + } + if (value < 0) return null; // A maintenance page's hour must not freeze coverage for an hour. return Math.min(RETRY_AFTER_MAX_SEC, Math.max(1, value)); } -/** The body as bytes, refusing more than MAX_RESPONSE_BYTES whatever the headers say. */ -async function readBounded(response) { +/** + * The body as bytes, refusing more than MAX_RESPONSE_BYTES whatever the + * headers say. `onChunk` hears each piece arrive (the idle timeout). + */ +async function readBounded(response, onChunk = () => {}) { const tooMuch = () => new PanoramaxSourceError('Panoramax sent too much data', { status: response.status, @@ -57,6 +72,7 @@ async function readBounded(response) { for (;;) { const { done, value } = await reader.read(); if (done) break; + onChunk(); total += value.byteLength; if (total > MAX_RESPONSE_BYTES) { reader.cancel().catch(() => {}); @@ -92,30 +108,55 @@ export function createPanoramaxSource({ lifo: true, }); - /** GET within `timeoutMs`; the caller's abort wins over the timeout. */ + /** Busy answers in a row with no readable Retry-After (the backoff grows). */ + let busyStreak = 0; + + /** + * GET: its headers within `timeoutMs`, then a body that may take as long + * as it keeps arriving (silent for STREAM_IDLE_TIMEOUT_MS is given up). + * The caller's abort wins over both. + */ async function get(url, { signal, timeoutMs, empty = [] }) { const controller = new AbortController(); const onAbort = () => controller.abort(signal.reason); if (signal?.aborted) throw signal.reason; signal?.addEventListener('abort', onAbort, { once: true }); - const timer = setTimeout( - () => controller.abort(new PanoramaxSourceError(NO_ANSWER)), - timeoutMs, - ); + const giveUp = () => controller.abort(new PanoramaxSourceError(NO_ANSWER)); + let timer = setTimeout(giveUp, timeoutMs); try { const response = await fetchImpl(url, { signal: controller.signal }); - if (empty.includes(response.status)) return null; - if (response.status === 429 || response.status === 503) + if (empty.includes(response.status)) { + busyStreak = 0; + return null; + } + if (response.status === 429 || response.status === 503) { + const told = retryAfterSeconds(response); + // Not told how long: back off, longer each time (5, 10, 20… 300 s). + const wait = + told ?? + Math.min( + RETRY_AFTER_MAX_SEC, + BUSY_BACKOFF_BASE_SEC * 2 ** Math.min(busyStreak, 8), + ); + busyStreak++; throw new PanoramaxSourceError('Panoramax is busy', { status: response.status, - retryAfterSec: retryAfterSeconds(response) ?? 5, + retryAfterSec: wait, }); + } if (!response.ok) throw new PanoramaxSourceError( `Panoramax answered HTTP ${response.status}`, { status: response.status }, ); - return await readBounded(response); + busyStreak = 0; + // The answer has begun: from here only silence is timed. + clearTimeout(timer); + timer = setTimeout(giveUp, STREAM_IDLE_TIMEOUT_MS); + return await readBounded(response, () => { + clearTimeout(timer); + timer = setTimeout(giveUp, STREAM_IDLE_TIMEOUT_MS); + }); } catch (error) { if (signal?.aborted) throw signal.reason; if (controller.signal.aborted) throw controller.signal.reason; @@ -180,8 +221,8 @@ export function createPanoramaxSource({ * Pictures within `radius` metres of a point, unranked: the caller ranks * them by distance. Two searches side by side, since neither finds them * all: `place_position` (nearest first) leaves out pictures looking - * away from the point, whatever the tolerance; a bounding box has them - * but in no order, so in a dense spot its page may miss the nearest. + * away from the point, whatever the tolerance; a bounding box has them, + * roughly nearest first (not exactly: the caller ranks the union). * `collections` keeps one sequence's, `pano` and `sinceMs` filter as * the imagery filter does. */ diff --git a/src/layers/streetLevel/providers/panoramax/source.test.mjs b/src/layers/streetLevel/providers/panoramax/source.test.mjs index 63eb6ff9e..db2b1160d 100644 --- a/src/layers/streetLevel/providers/panoramax/source.test.mjs +++ b/src/layers/streetLevel/providers/panoramax/source.test.mjs @@ -1,7 +1,16 @@ import assert from 'node:assert/strict'; import test from 'node:test'; -import { PanoramaxSourceError, createPanoramaxSource } from './source.js'; -import { MAX_RESPONSE_BYTES } from './policy.js'; +import { + PanoramaxSourceError, + createPanoramaxSource, + retryAfterSeconds, +} from './source.js'; +import { + MAX_RESPONSE_BYTES, + NEAREST_BOX_LIMIT, + STREAM_IDLE_TIMEOUT_MS, + TILE_TIMEOUT_MS, +} from './policy.js'; /** A fetch answering from `respond(url)`, recording every URL asked. */ function fakeFetch(respond) { @@ -348,6 +357,148 @@ test('a Retry-After is honoured up to five minutes', async () => { assert.equal(await wait('12'), 12); }); +test('a Retry-After may be an HTTP date', () => { + const now = Date.parse('2026-10-15T12:00:00Z'); + const response = (value) => + new Response(null, { status: 429, headers: { 'retry-after': value } }); + assert.equal( + retryAfterSeconds(response('Thu, 15 Oct 2026 12:01:00 GMT'), now), + 60, + ); + assert.equal( + retryAfterSeconds(response('Thu, 15 Oct 2026 14:00:00 GMT'), now), + 300, + ); + assert.equal( + retryAfterSeconds(response('Thu, 15 Oct 2026 11:00:00 GMT'), now), + null, + 'a date already past', + ); + assert.equal(retryAfterSeconds(response('soon'), now), null); + assert.equal( + retryAfterSeconds(new Response(null, { status: 429 }), now), + null, + ); +}); + +test('busy without a readable Retry-After (the browser hides it): the wait doubles, and resets once answered', async () => { + let busy = true; + const source = createPanoramaxSource({ + fetchImpl: fakeFetch(() => + busy ? { status: 429 } : { body: new Uint8Array([1]) }, + ).fetchImpl, + }); + const wait = () => + source.getTile({ x: 0, y: 0, z: 0 }).then( + () => 'ok', + (error) => error.retryAfterSec, + ); + const waits = []; + for (let i = 0; i < 9; i++) waits.push(await wait()); + assert.deepEqual(waits, [5, 10, 20, 40, 80, 160, 300, 300, 300]); + busy = false; + assert.equal(await wait(), 'ok'); + busy = true; + assert.equal(await wait(), 5, 'back to the first wait'); +}); + +test( + 'a body that keeps arriving is not cut off by the request timeout; one that stalls is', + { timeout: 10_000 }, + async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + const settle = async () => { + for (let i = 0; i < 20; i++) await Promise.resolve(); + }; + // Headers at once, then a chunk every 10 s for twice the tile timeout. + const chunks = Math.ceil((2 * TILE_TIMEOUT_MS) / 10_000); + let push; + const slowBody = (signal) => + new Response( + new ReadableStream({ + start(controller) { + push = (last) => { + controller.enqueue(new Uint8Array([7])); + if (last) controller.close(); + }; + signal.addEventListener('abort', () => + controller.error(signal.reason), + ); + }, + }), + { status: 200 }, + ); + const source = createPanoramaxSource({ + fetchImpl: async (url, { signal }) => slowBody(signal), + }); + const slow = source.getTile({ x: 0, y: 0, z: 0 }); + await settle(); + for (let i = 1; i <= chunks; i++) { + t.mock.timers.tick(10_000); + push(i === chunks); + await settle(); + } + assert.equal((await slow).length, chunks, 'every chunk read'); + const stalled = source.getTile({ x: 0, y: 0, z: 0 }); + await settle(); + push(false); + await settle(); + t.mock.timers.tick(STREAM_IDLE_TIMEOUT_MS); + await assert.rejects(stalled, /did not answer in time/); + }, +); + +test('the box search asks for a few pictures: the API returns them nearest first', async () => { + const { urls, fetchImpl } = fakeFetch(() => json({ features: [] })); + const source = createPanoramaxSource({ fetchImpl }); + await source.nearestItems({ lon: 2.35, lat: 48.85 }); + const box = urls + .map((url) => new URL(url)) + .find((url) => url.searchParams.has('bbox')); + assert.ok(box, 'a box search'); + assert.equal(box.searchParams.get('limit'), String(NEAREST_BOX_LIMIT)); + assert.ok(NEAREST_BOX_LIMIT <= 20); +}); + +test('the cones’ tiles have their own connection: never behind coverage tiles, the newest first', async () => { + const { PICTURE_TILE_CONCURRENCY, TILE_CONCURRENCY } = + await import('./policy.js'); + assert.equal(PICTURE_TILE_CONCURRENCY, 1); + const held = new Map(); + const asked = []; + const source = createPanoramaxSource({ + fetchImpl: (url) => { + asked.push(url.replace(/^.*\/map\//, '')); + return new Promise((resolve) => + held.set(url.replace(/^.*\/map\//, ''), () => + resolve(new Response(null, { status: 204 })), + ), + ); + }, + }); + const settle = () => new Promise((resolve) => setTimeout(resolve, 0)); + // Coverage fills its lane and queues. + const coverage = Array.from({ length: TILE_CONCURRENCY + 2 }, (_, i) => + source.getTile({ x: i, y: 0, z: 10 }), + ); + await settle(); + assert.equal(asked.length, TILE_CONCURRENCY); + // A cone tile goes at once; two more wait, the newer first. + const pictures = [1, 2, 3].map((x) => + source.getTile({ x, y: 9, z: 15 }, { lane: 'pictures' }), + ); + await settle(); + assert.deepEqual(asked.slice(TILE_CONCURRENCY), ['15/1/9.mvt']); + held.get('15/1/9.mvt')(); + await settle(); + assert.deepEqual(asked.slice(TILE_CONCURRENCY), ['15/1/9.mvt', '15/3/9.mvt']); + for (let round = 0; round < 6; round++) { + for (const go of held.values()) go(); + await settle(); + } + await Promise.all([...coverage, ...pictures]); +}); + test('a lookup near the antimeridian keeps its box on the map', async () => { const { urls, fetchImpl } = fakeFetch(() => json({ features: [] })); const source = createPanoramaxSource({ fetchImpl }); diff --git a/src/layers/streetLevel/providers/panoramax/viewer.js b/src/layers/streetLevel/providers/panoramax/viewer.js index 3e26e4d56..902e95156 100644 --- a/src/layers/streetLevel/providers/panoramax/viewer.js +++ b/src/layers/streetLevel/providers/panoramax/viewer.js @@ -31,7 +31,13 @@ export const PANORAMAX_SOURCES = Object.freeze({ Boolean(photo.isPano && (photo.urls.sd || hdDecodable(photo))), // Without an SD derivative the original goes straight to the texture size. panoFirst: (photo) => { - if (photo.urls.sd) return { url: photo.urls.sd, width: SD_WIDTH }; + // Decoded at SD's own size, whatever the host actually serves. + if (photo.urls.sd) + return { + url: photo.urls.sd, + width: SD_WIDTH, + resize: [SD_WIDTH, SD_WIDTH / 2], + }; if (!hdDecodable(photo)) return null; const width = Math.min(photo.width, PANO_FULL_SIZE[0]); return { url: photo.urls.hd, width, resize: [width, width / 2] }; @@ -51,15 +57,20 @@ export const PANORAMAX_SOURCES = Object.freeze({ timeoutMs: HD_TIMEOUT_MS, }; }, - // A 360° picture shown flat (no WebGL) is its whole equirectangular image; - // an original too big to decode is skipped for the thumbnail. - flatFirst: (photo) => [ + // A 360° picture shown flat (no WebGL) is its whole equirectangular image. + // When SD fails the thumbnail stands in; the original (megabytes) only + // for the expanded viewer, and only one of a size it may decode. + flatFirst: (photo, { expanded = false } = {}) => [ photo.urls.sd, - !photo.isPano || hdDecodable(photo) ? photo.urls.hd : null, + expanded && hdDecodable(photo) ? photo.urls.hd : null, photo.urls.thumb, ], + // The original once the viewer is wider than SD, if its width is known + // and decodable (a 200 MP phone photo is 16,000 px across). flatSharp: (photo, pixels) => - photo.urls.sd && photo.urls.hd && pixels > SD_WIDTH ? photo.urls.hd : null, + photo.urls.sd && hdDecodable(photo) && pixels > SD_WIDTH + ? photo.urls.hd + : null, describe: (photo) => ({ // Where no author can be named, the instance holding the photo is. title: !photo.creator && photo.via ? `via ${photo.via}` : null, diff --git a/src/layers/streetLevel/providers/panoramax/viewer.test.mjs b/src/layers/streetLevel/providers/panoramax/viewer.test.mjs index 614a50e13..3f8ae4789 100644 --- a/src/layers/streetLevel/providers/panoramax/viewer.test.mjs +++ b/src/layers/streetLevel/providers/panoramax/viewer.test.mjs @@ -140,12 +140,14 @@ test('a 360° picture shows its SD equirectangular image, then HD shrunk to 4,09 assert.equal(canvas.hidden, false); assert.equal(picture.hidden, true); assert.equal(viewer.supportsRenderModes(), false, 'no frame to fit'); - assert.deepEqual(loads[0], { + const { signal, ...first } = loads[0]; + assert.deepEqual(first, { url: PHOTOS.pano.urls.sd, crossOrigin: true, - resize: null, + resize: [2048, 1024], failure: 'The Panoramax photo could not be loaded', }); + assert.equal(signal.aborted, false, 'the open can stop it'); assert.equal(poses.at(-1).isPano, true); assert.equal(poses.at(-1).bearing, 100); await new Promise((resolve) => setTimeout(resolve, 0)); @@ -258,7 +260,7 @@ test('without WebGL a 360° picture shows its whole equirectangular image flat', assert.equal(picture.src, PHOTOS.pano.urls.sd); }); -test('a picture whose SD fails falls back to HD, then the thumbnail', async () => { +test('a picture whose SD fails falls back to the thumbnail; the original only for the expanded viewer', async () => { const tried = []; const viewer = createPanoramaxViewer({ getPhoto: async () => PHOTOS.flat, @@ -273,12 +275,18 @@ test('a picture whose SD fails falls back to HD, then the thumbnail', async () = const host = fakeHost(); await viewer.mount(host); await viewer.open('flat'); + assert.deepEqual(tried, [PHOTOS.flat.urls.sd, PHOTOS.flat.urls.thumb]); + assert.equal(parts(host).picture.src, PHOTOS.flat.urls.thumb); + // Expanded, the original is worth its megabytes. + tried.length = 0; + parts(host).root.closest = (selector) => + selector.includes('sl-viewer-wrap-expanded') ? {} : null; + await viewer.open('flat'); assert.deepEqual(tried, [ PHOTOS.flat.urls.sd, PHOTOS.flat.urls.hd, PHOTOS.flat.urls.thumb, ]); - assert.equal(parts(host).picture.src, PHOTOS.flat.urls.thumb); }); test('a photo with no author to name is captioned with the instance holding it', () => {