mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 10:24:02 +00:00
Street Level: Panoramax review follow-ups (gekh, #1134)
Image loading (photoViewer.js) - One AbortController per open: a new open, close or unmount stops the previous photo's first and sharper downloads (an <img> by src = ''). A collapsed viewer never falls back to the original; Panoramax's flat fallback is the thumbnail. A superseded open resolves quietly. - Flat HD tracked as "loading" and "shown" apart: a discarded response no longer blocks later upgrades, and resizes while it loads ask once. - No upgrade of the photo still shown while another opens. - Size limits before decoding: imageSize.js reads JPEG/PNG/WebP/GIF headers; more than 64 MP or 16,384 px a side is refused undecoded, an <img> that decodes larger is refused and let go, flat HD needs a known width of at most 8,192 px, and panoramas always decode with a resize. Panoramax provider - Retry-After parsed as seconds or an HTTP date; the browser cannot read it here, so busy answers back off 5 s doubling to 300 s, reset once answered. - Headers timed, then the body by an idle timeout (15 s of silence), so a slow link still gets its megabyte tiles. - Cones centre only on the picture shown (onShowing), never on a late lookup; lookups nobody waits for are aborted (shared, refcounted). - onReopen: a GPU reset redraws in the viewer, not through the core. - Cones draw the home z15 tile at once and add neighbours as they arrive; a neighbour that failed is not counted as drawn. - Coverage tiles decode in ~8 ms slices (a 2 MB z10 tile over Paris was one ~85-200 ms task); sorted by measured length without a Map. - Box search asks for 20, not 100 (the API returns them nearest first). - looksLikeEmail checks every '@'; creatorOf de-duplicates with a Set. lineCoverage.js - A redraw while a swap still builds keeps what is on screen (the swap's old lines and its ready new ones) and drops only the unready ones. Tests for each (mutation-checked), two vacuous tests made real, the cones' pictures lane and readable()'s busy/timeout paths covered; the fixture's box search is nearest first. Docs: README filters and share links, CHANGELOG HD wording, CURRENT-STATE. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Said TurkseverandClaude Opus 5.5 committed
1 parent
e3bc431f9a
commit
b645a767c8
25 files changed
+1362
-135
No files matched your search
+9
-4
@@ -153,13 +153,18 @@
|
||||
← → following the picture's own links. The 360° and since filters apply,
|
||||
in the lines and in the lookup. The caption names each picture's author
|
||||
and licence (CC BY-SA 4.0, Licence Ouverte 2.0…) and links to it in the
|
||||
Panoramax viewer; HD originals load only for an expanded viewer, once.
|
||||
Share links carry it as `0.m.0_0.r.panoramax-1`. KartaView's photo viewer
|
||||
Panoramax viewer. HD originals load once, only when the viewer is wider
|
||||
than the 2,048 px version (a panorama: expanded), and every download stops
|
||||
when the photo changes; an image whose header declares more than 64 MP is
|
||||
refused before it decodes. Share links carry it as
|
||||
`0.m.0_0.r.google-0*kartaview-0*panoramax-1` (`0.m.0_0.r.panoramax-1` works
|
||||
too). KartaView's photo viewer
|
||||
is now shared (`photoViewer.js`, `panorama.js`); in it, for KartaView too,
|
||||
Esc clears the selected drive before it collapses the panel. Lines are
|
||||
simplified and drawn within a point budget (a tile's unsimplified lines
|
||||
over Paris hung the tab), decoded tiles are kept, and the cones' tiles
|
||||
have their own connection.
|
||||
over Paris hung the tab), decoded in slices so the globe keeps moving,
|
||||
and kept; the cones' tiles have their own connection, and the open
|
||||
picture's tile draws its cones before its neighbours arrive.
|
||||
|
||||
- Flights and Military Flights highlight emergency transponder broadcasts in
|
||||
fleet billboards, 3D models and Cockpit contacts. Colors update when a status
|
||||
|
||||
@@ -354,9 +354,9 @@ _The Space Missions layer replaying a Falcon 9 ascent — labeled `RECONSTRUCTED
|
||||
|
||||
**Setup.** At [mapillary.com/dashboard/developers](https://www.mapillary.com/dashboard/developers), register an application with **Read** access and copy its **Client Token** (`MLY|…`). Paste it into **POWER UP → Mapillary**, or set `MAPILLARY_CLIENT_TOKEN` in `.env` and restart. Without it the panel reads **KEY REQUIRED**. **Google Street View** uses `GOOGLE_MAPS_API_KEY` with the **Maps JavaScript API** enabled for it in Google Cloud; its chip starts off, since every panorama is billed to the key. **KartaView** and **Panoramax** need nothing: their public APIs are keyless.
|
||||
|
||||
**Use.** Turn on **Street Level** under DATA LAYERS → Cameras, or with the pill in the STREET LEVEL panel header. Each provider has a chip in its own colour (Mapillary green), and one is lit at a time: lighting one switches the others off. Mapillary, which is free, is lit by default. The **IMAGERY** (ALL / 360° / FLAT) and **SINCE** filters apply to Mapillary and KartaView; Street View panoramas are all 360°, so with Street View lit the filters and legend are hidden. Click a coverage line for its image cones and a cone to open the photo. Light **KARTAVIEW** (purple) and zoom in below about 6 km: its lines are the streets KartaView has photos of, and a click on one opens the photo nearest the click, flat or 360° (drag to look around), with cones along its drive; **‹ ›** or ← → step through the drive (Shift+← →, ↑ ↓ and + − look around a 360° photo; pinch zooms on a phone). Photos KartaView has archived are skipped, and where all are archived the panel says so. For KartaView, **SINCE** goes by upload date, as KartaView's own map does (it can filter by nothing else), and the panel notes it under the slider; KartaView filters its lines on its own server, and a filtered area KartaView has not drawn before can take several seconds, and the unfiltered lines show meanwhile. Light **PANORAMAX** (orange) for the open, federated commons (strongest in France and Belgium): from orbit its dots show where it has pictures, and below about 60 km its lines are sequences. A click on a line opens that sequence's picture nearest the click without moving the camera, flat or 360°, with cones along the sequence around it; **‹ ›** or ← → step through it, and both filters apply. Its caption names the picture's authors (or the instance holding it) and licence (mostly CC BY-SA 4.0; pictures on the IGN instance are Licence Ouverte 2.0). On the **Google 3D** map (Google's terms keep Street View off other maps, where it does not appear), light **GOOGLE STREET VIEW** and click a street from below 3 km, or press **OPEN STREET VIEW** for the view centre, to open Google's panorama there; step along its arrows. Above the photo: **EXPAND**, **FIT / FILL** (Mapillary, and flat KartaView and Panoramax photos), and **FOLLOW** (the globe camera follows the photo's view; Google 3D map only). The caption links to the image on the provider's site, and the globe credits each provider that is on ("© Mapillary contributors, CC BY-SA 4.0", "Street View imagery © Google", "© KartaView contributors, CC BY-SA 4.0", "© Panoramax contributors, licence per picture"). KartaView's own photo pages fail for signed-out visitors, so its link opens the full-size photo.
|
||||
**Use.** Turn on **Street Level** under DATA LAYERS → Cameras, or with the pill in the STREET LEVEL panel header. Each provider has a chip in its own colour (Mapillary green), and one is lit at a time: lighting one switches the others off. Mapillary, which is free, is lit by default. The **IMAGERY** (ALL / 360° / FLAT) and **SINCE** filters apply to Mapillary, KartaView and Panoramax; Street View panoramas are all 360°, so with Street View lit the filters and legend are hidden. Click a coverage line for its image cones and a cone to open the photo. Light **KARTAVIEW** (purple) and zoom in below about 6 km: its lines are the streets KartaView has photos of, and a click on one opens the photo nearest the click, flat or 360° (drag to look around), with cones along its drive; **‹ ›** or ← → step through the drive (Shift+← →, ↑ ↓ and + − look around a 360° photo; pinch zooms on a phone). Photos KartaView has archived are skipped, and where all are archived the panel says so. For KartaView, **SINCE** goes by upload date, as KartaView's own map does (it can filter by nothing else), and the panel notes it under the slider; KartaView filters its lines on its own server, and a filtered area KartaView has not drawn before can take several seconds, and the unfiltered lines show meanwhile. Light **PANORAMAX** (orange) for the open, federated commons (strongest in France and Belgium): from orbit its dots show where it has pictures, and below about 60 km its lines are sequences. A click on a line opens that sequence's picture nearest the click without moving the camera, flat or 360°, with cones along the sequence around it; **‹ ›** or ← → step through it, and both filters apply. Its caption names the picture's authors (or the instance holding it) and licence (mostly CC BY-SA 4.0; pictures on the IGN instance are Licence Ouverte 2.0). On the **Google 3D** map (Google's terms keep Street View off other maps, where it does not appear), light **GOOGLE STREET VIEW** and click a street from below 3 km, or press **OPEN STREET VIEW** for the view centre, to open Google's panorama there; step along its arrows. Above the photo: **EXPAND**, **FIT / FILL** (Mapillary, and flat KartaView and Panoramax photos), and **FOLLOW** (the globe camera follows the photo's view; Google 3D map only). The caption links to the image on the provider's site, and the globe credits each provider that is on ("© Mapillary contributors, CC BY-SA 4.0", "Street View imagery © Google", "© KartaView contributors, CC BY-SA 4.0", "© Panoramax contributors, licence per picture"). KartaView's own photo pages fail for signed-out visitors, so its link opens the full-size photo.
|
||||
|
||||
Drag the panel header to float the panel, and resize it from any edge. Double-click the header or collapse it to put it back in the rail. Share links carry the provider switches and filters (`0.m.0` Mapillary off, `0.m.0_0.r.google-1` Street View on, `0.m.0_0.r.google-0*kartaview-1` KartaView on, as the app writes it (`0.m.0_0.r.kartaview-1` works too, and `0.m.0_0.r.panoramax-1` lights Panoramax; a link that leaves out `m` keeps Mapillary, the default, lit), `0.p.p` panoramas only, `0.s.365` the last year). To add a provider, implement the contract in `src/layers/streetLevel/registry.js` and register it in `src/app/layers/streetLevel.js`.
|
||||
Drag the panel header to float the panel, and resize it from any edge. Double-click the header or collapse it to put it back in the rail. Share links carry the provider switches and filters (`0.m.0` Mapillary off, `0.m.0_0.r.google-1` Street View on, `0.m.0_0.r.google-0*kartaview-1*panoramax-0` KartaView on and `0.m.0_0.r.google-0*kartaview-0*panoramax-1` Panoramax on, as the app writes them (`0.m.0_0.r.kartaview-1` and `0.m.0_0.r.panoramax-1` work too; a link that leaves out `m` keeps Mapillary, the default, lit), `0.p.p` panoramas only, `0.s.365` the last year). To add a provider, implement the contract in `src/layers/streetLevel/registry.js` and register it in `src/app/layers/streetLevel.js`.
|
||||
|
||||
## 🎖️ Field Missions
|
||||
|
||||
|
||||
+28
-11
@@ -3403,7 +3403,13 @@ provider is on (16, ten minutes, `tileCache.js`, shared with KartaView),
|
||||
and above
|
||||
60 km the `grid` layer's 0.1°
|
||||
density cells, binned to at most 96×96 points a tile (a z0 tile holds
|
||||
~70,000 cells) and not counted as sequences: the panel says to zoom in. Both
|
||||
~70,000 cells) and not counted as sequences: the panel says to zoom in.
|
||||
A tile decodes in slices of about 8 ms, the page getting a turn between
|
||||
them (a 2 MB z10 tile is ~200 ms of work). A busy API (429/503) holds the
|
||||
tiles back for its Retry-After, which the browser cannot read here (the API
|
||||
exposes no headers), so the wait doubles from 5 s to 5 minutes until it
|
||||
answers; an answer may take as long as it keeps arriving (silent for 15 s
|
||||
is given up), so a slow link still gets its megabyte tiles. Both
|
||||
filters apply: lines carry their day and camera type, and a cell stays while
|
||||
it holds pictures of the kind asked for (cells are undated). A click on a
|
||||
line asks the core (`actions.openNearest`) for that sequence's picture
|
||||
@@ -3412,8 +3418,8 @@ a click lands well off a line), without framing, then any sequence's within
|
||||
50 m. Each lookup is two searches side by side, ranked by distance in the
|
||||
browser: `place_position` (nearest first, but even at
|
||||
`place_fov_tolerance=180` it leaves out pictures looking away from the
|
||||
point) and a bounding box (those too, but in no order); the filters go to
|
||||
both (`filter=field_of_view=360`, `datetime=<since>/..`). A line is dated
|
||||
point) and a bounding box (those too, roughly nearest first, so 20 are
|
||||
asked for: ~60 kB, against ~600 kB for 100); the filters go to both (`filter=field_of_view=360`, `datetime=<since>/..`). A line is dated
|
||||
by its sequence's first day, so a sequence still growing can hide under
|
||||
SINCE; a line with no camera type or day stays drawn. Lines keep the
|
||||
decoded longest-first order under the per-tile cap (`keepOrder`). Once the picture is on screen
|
||||
@@ -3421,20 +3427,31 @@ its sequence is highlighted and drawn as cones from the z15 tiles around it
|
||||
(their `pictures` layer, ~110 bytes a picture, against ~7.5 kB for a STAC
|
||||
item), plus the neighbours within 150 m of an edge, on their own connection
|
||||
(coverage tiles take three, searches two); each tile's list for the sequence
|
||||
is kept, and the cones re-centre when a picture opens outside the tiles
|
||||
they were drawn from, judged from the moment their load starts. ‹ › follow the picture's own `prev`/`next`
|
||||
is kept. The open picture's own tile draws at once and the neighbours join
|
||||
as they arrive; one that fails is not counted, so a picture opening in it
|
||||
re-centres the cones there. The cones re-centre when a picture opens outside the tiles
|
||||
they were drawn from, judged from the moment their load starts, and centre
|
||||
only on a picture actually shown (never on a lookup that came back after
|
||||
the user moved on); a lookup nobody waits for any more is stopped. ‹ › follow the picture's own `prev`/`next`
|
||||
links, so stepping costs one small search per picture. The viewer shows the
|
||||
2,048 px SD derivative (equirectangular for a 360° picture) and swaps in the
|
||||
HD original only when the viewer is wider than SD (flat) or, for a
|
||||
panorama, expanded; once, shrunk while it decodes, and aborted if another
|
||||
photo opens; originals wider than 8,192 px are never fetched, and any
|
||||
picture file over 48 MB or 16,384 px is refused. The caption names every
|
||||
2,048 px SD derivative (equirectangular for a 360° picture, decoded at
|
||||
2,048 × 1,024) and swaps in the HD original only when the viewer is wider
|
||||
than SD (flat) or, for a panorama, expanded; once, shrunk while it decodes.
|
||||
When SD fails a flat picture falls back to the thumbnail, and to the
|
||||
original only in the expanded viewer. Every download of a photo, first or
|
||||
sharper, stops when another opens or the viewer closes. Originals wider than
|
||||
8,192 px are never fetched; a file's header (JPEG, PNG, WebP, GIF) is read
|
||||
before it decodes, and one declaring more than 64 MP or 16,384 px a side
|
||||
is refused undecoded, as is an `<img>` that decodes larger and any file
|
||||
over 48 MB. The caption names every
|
||||
producer (an uploading app and its photographer; never an account named by
|
||||
an email address), or else the instance holding the picture ("via
|
||||
panoramax.ign.fr"), and the picture's licence (CC BY-SA 4.0, Licence
|
||||
Ouverte 2.0…; StreetPose `license`), and links to the picture in the
|
||||
Panoramax viewer. Esc anywhere in the docked viewer clears the selected
|
||||
sequence before the panel can take it (KartaView too). The MCP globe panel allows the meta-catalogue and the
|
||||
sequence before the panel can take it (KartaView too). After a GPU reset
|
||||
the viewer draws the same picture again itself (`onReopen`), leaving the
|
||||
core's lookups alone. The MCP globe panel allows the meta-catalogue and the
|
||||
three biggest instances; a picture on another instance shows only in the
|
||||
app.
|
||||
|
||||
|
||||
@@ -249,9 +249,11 @@ export function answerPanoramaxRequest({ method, url }) {
|
||||
: true,
|
||||
)
|
||||
.filter((p) => !since || Date.parse(p.datetime) >= Date.parse(since));
|
||||
// A box: every picture in it, in no particular order (id order here).
|
||||
// A box: every picture in it, nearest its centre first, as the live API
|
||||
// returns them (roughly: the browser ranks them again).
|
||||
if (q.has('bbox')) {
|
||||
const [west, south, east, north] = q.get('bbox').split(',').map(Number);
|
||||
const centre = { lon: (west + east) / 2, lat: (south + north) / 2 };
|
||||
return json({
|
||||
type: 'FeatureCollection',
|
||||
features: kept
|
||||
@@ -259,9 +261,10 @@ export function answerPanoramaxRequest({ method, url }) {
|
||||
(p) =>
|
||||
p.lon >= west && p.lon <= east && p.lat >= south && p.lat <= north,
|
||||
)
|
||||
.sort((a, b) => a.id.localeCompare(b.id))
|
||||
.map((p) => ({ p, metres: metresApart(centre, p) }))
|
||||
.sort((a, b) => a.metres - b.metres || a.p.id.localeCompare(b.p.id))
|
||||
.slice(0, Number(q.get('limit')) || 10)
|
||||
.map((p) => item(p, pictures)),
|
||||
.map(({ p }) => item(p, pictures)),
|
||||
});
|
||||
}
|
||||
const [lon, lat] = (q.get('place_position') || '').split(',').map(Number);
|
||||
|
||||
@@ -943,6 +943,7 @@
|
||||
"src/layers/streetLevel/providers/panoramax/viewer.js",
|
||||
"src/layers/streetLevel/photoViewer.js",
|
||||
"src/layers/streetLevel/limiter.js",
|
||||
"src/layers/streetLevel/imageSize.js",
|
||||
"src/layers/streetLevel/tileCache.js",
|
||||
"src/layers/streetLevel/providers/panoramax/policy.test.mjs",
|
||||
"src/layers/streetLevel/providers/panoramax/decode.test.mjs",
|
||||
@@ -952,6 +953,7 @@
|
||||
"src/testSupport/panoramaxFixtures.mjs",
|
||||
"src/layers/streetLevel/photoViewer.test.mjs",
|
||||
"src/layers/streetLevel/limiter.test.mjs",
|
||||
"src/layers/streetLevel/imageSize.test.mjs",
|
||||
"src/layers/streetLevel/lineCoverage.test.mjs",
|
||||
"src/testSupport/kartaViewFakes.mjs",
|
||||
"src/testSupport/googleStreetViewFakes.mjs",
|
||||
|
||||
@@ -1244,6 +1244,7 @@
|
||||
"src/layers/streetLevel/providers/panoramax/viewer.js",
|
||||
"src/layers/streetLevel/photoViewer.js",
|
||||
"src/layers/streetLevel/limiter.js",
|
||||
"src/layers/streetLevel/imageSize.js",
|
||||
"src/layers/streetLevel/tileCache.js",
|
||||
"src/layers/streetLevel/panorama.js",
|
||||
"src/layers/streetLevel/lineCoverage.js",
|
||||
@@ -2225,6 +2226,7 @@
|
||||
"src/layers/streetLevel/providers/panoramax/viewer.js",
|
||||
"src/layers/streetLevel/photoViewer.js",
|
||||
"src/layers/streetLevel/limiter.js",
|
||||
"src/layers/streetLevel/imageSize.js",
|
||||
"src/layers/streetLevel/tileCache.js",
|
||||
"src/layers/streetLevel/panorama.js",
|
||||
"src/layers/streetLevel/lineCoverage.js",
|
||||
@@ -2777,6 +2779,7 @@
|
||||
"src/layers/streetLevel/providers/panoramax/viewer.js",
|
||||
"src/layers/streetLevel/photoViewer.js",
|
||||
"src/layers/streetLevel/limiter.js",
|
||||
"src/layers/streetLevel/imageSize.js",
|
||||
"src/layers/streetLevel/tileCache.js",
|
||||
"src/layers/streetLevel/panorama.js",
|
||||
"src/layers/streetLevel/lineCoverage.js",
|
||||
|
||||
@@ -64,6 +64,11 @@ test('the defaults are Mapillary (given its source), Street View, KartaView and
|
||||
assert.equal(layer.getParams().panoramax, true);
|
||||
assert.equal(layer.getParams().kartaview, false);
|
||||
assert.equal(layer.getParams().mapillary, false);
|
||||
// The form the app writes (the README and CHANGELOG quote it).
|
||||
assert.equal(
|
||||
layer.getParams().providerSwitches,
|
||||
'google-0*kartaview-0*panoramax-1',
|
||||
);
|
||||
// Without the application's Street Level source there are none.
|
||||
assert.deepEqual(createApplicationStreetLevel({}).providerIds, []);
|
||||
});
|
||||
@@ -0,0 +1,64 @@
|
||||
/**
|
||||
* An image's pixel size read from its header, before anything decodes it:
|
||||
* a picture host can serve a small file that declares enormous dimensions,
|
||||
* and a decode of it alone can take the tab (or the GPU process) down.
|
||||
* JPEG (SOFn), PNG (IHDR), WebP (VP8, VP8L, VP8X) and GIF; null for any
|
||||
* other format, or a header this does not reach.
|
||||
* @param {Uint8Array} bytes The start of the file (a JPEG's EXIF can push its SOF past 64 kB).
|
||||
* @returns {{width: number, height: number}|null}
|
||||
*/
|
||||
export function imageSize(bytes) {
|
||||
if (!bytes || bytes.length < 12) return null;
|
||||
const b = bytes;
|
||||
const be16 = (i) => (b[i] << 8) | b[i + 1];
|
||||
const le16 = (i) => b[i] | (b[i + 1] << 8);
|
||||
const le24 = (i) => b[i] | (b[i + 1] << 8) | (b[i + 2] << 16);
|
||||
const ascii = (i, text) =>
|
||||
[...text].every((char, k) => b[i + k] === char.charCodeAt(0));
|
||||
// JPEG: walk the markers to the first start-of-frame.
|
||||
if (b[0] === 0xff && b[1] === 0xd8) {
|
||||
let i = 2;
|
||||
while (i + 9 < b.length) {
|
||||
if (b[i] !== 0xff) return null;
|
||||
const marker = b[i + 1];
|
||||
// Fill bytes, and markers with no length.
|
||||
if (marker === 0xff) {
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (marker === 0x01 || (marker >= 0xd0 && marker <= 0xd9)) {
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
const isFrame =
|
||||
marker >= 0xc0 &&
|
||||
marker <= 0xcf &&
|
||||
marker !== 0xc4 &&
|
||||
marker !== 0xc8 &&
|
||||
marker !== 0xcc;
|
||||
if (isFrame) return { width: be16(i + 7), height: be16(i + 5) };
|
||||
i += 2 + be16(i + 2);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
// PNG: IHDR is the first chunk.
|
||||
if (b[0] === 0x89 && ascii(1, 'PNG') && ascii(12, 'IHDR')) {
|
||||
const u32 = (i) =>
|
||||
((b[i] << 24) | (b[i + 1] << 16) | (b[i + 2] << 8) | b[i + 3]) >>> 0;
|
||||
return { width: u32(16), height: u32(20) };
|
||||
}
|
||||
if (ascii(0, 'RIFF') && ascii(8, 'WEBP') && b.length >= 30) {
|
||||
if (ascii(12, 'VP8 '))
|
||||
return { width: le16(26) & 0x3fff, height: le16(28) & 0x3fff };
|
||||
if (ascii(12, 'VP8L'))
|
||||
return {
|
||||
width: 1 + (((b[22] & 0x3f) << 8) | b[21]),
|
||||
height:
|
||||
1 + (((b[24] & 0x0f) << 10) | (b[23] << 2) | ((b[22] & 0xc0) >> 6)),
|
||||
};
|
||||
if (ascii(12, 'VP8X')) return { width: 1 + le24(24), height: 1 + le24(27) };
|
||||
return null;
|
||||
}
|
||||
if (ascii(0, 'GIF8')) return { width: le16(6), height: le16(8) };
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { imageSize } from './imageSize.js';
|
||||
|
||||
const bytes = (...parts) =>
|
||||
Uint8Array.from(
|
||||
parts.flat().map((v) => (typeof v === 'string' ? v.charCodeAt(0) : v)),
|
||||
);
|
||||
const ascii = (text) => [...text];
|
||||
|
||||
test('a JPEG’s size is read from its first frame, past an EXIF segment', () => {
|
||||
const exif = [0xff, 0xe1, 0x00, 0x10, ...new Array(14).fill(0)];
|
||||
const sof = [0xff, 0xc0, 0x00, 0x11, 0x08, 0x0b, 0xc0, 0x0f, 0xc0];
|
||||
const jpeg = bytes([0xff, 0xd8], exif, sof, new Array(20).fill(0));
|
||||
assert.deepEqual(imageSize(jpeg), { width: 0x0fc0, height: 0x0bc0 });
|
||||
// Progressive (SOF2) too; a DHT (C4) is not a frame.
|
||||
const dht = [0xff, 0xc4, 0x00, 0x04, 0, 0];
|
||||
const sof2 = [0xff, 0xc2, 0x00, 0x11, 0x08, 0x00, 0x10, 0x00, 0x20];
|
||||
assert.deepEqual(
|
||||
imageSize(bytes([0xff, 0xd8], dht, sof2, new Array(10).fill(0))),
|
||||
{
|
||||
width: 32,
|
||||
height: 16,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('PNG, WebP (lossy, lossless, extended) and GIF sizes', () => {
|
||||
const png = bytes(
|
||||
[0x89],
|
||||
ascii('PNG'),
|
||||
[0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13],
|
||||
ascii('IHDR'),
|
||||
[0, 0, 0x10, 0, 0, 0, 0x08, 0],
|
||||
);
|
||||
assert.deepEqual(imageSize(png), { width: 4096, height: 2048 });
|
||||
const riff = (chunk, rest) =>
|
||||
bytes(ascii('RIFF'), [0, 0, 0, 0], ascii('WEBP'), ascii(chunk), rest);
|
||||
const vp8 = riff('VP8 ', [...new Array(10).fill(0), 0x00, 0x08, 0x00, 0x04]);
|
||||
assert.deepEqual(imageSize(vp8), { width: 2048, height: 1024 });
|
||||
// VP8L: 14-bit width-1 and height-1 packed after the 0x2f signature.
|
||||
const w = 99;
|
||||
const h = 49;
|
||||
const packed = w | (h << 14);
|
||||
const vp8l = riff('VP8L', [
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0x2f,
|
||||
packed & 0xff,
|
||||
(packed >> 8) & 0xff,
|
||||
(packed >> 16) & 0xff,
|
||||
(packed >> 24) & 0xff,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
]);
|
||||
assert.deepEqual(imageSize(vp8l), { width: 100, height: 50 });
|
||||
const vp8x = riff(
|
||||
'VP8X',
|
||||
[0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0x0f, 0x00, 0xff, 0x07, 0x00],
|
||||
);
|
||||
assert.deepEqual(imageSize(vp8x), { width: 4096, height: 2048 });
|
||||
const gif = bytes(ascii('GIF89a'), [0x40, 0x01, 0xf0, 0x00], [0, 0]);
|
||||
assert.deepEqual(imageSize(gif), { width: 320, height: 240 });
|
||||
});
|
||||
|
||||
test('anything else, or a header cut short, has no size', () => {
|
||||
assert.equal(imageSize(null), null);
|
||||
assert.equal(imageSize(new Uint8Array(4)), null);
|
||||
assert.equal(imageSize(bytes(ascii('<html><body>'))), null);
|
||||
assert.equal(
|
||||
imageSize(
|
||||
bytes([0xff, 0xd8, 0xff, 0xe1, 0xff, 0xff], new Array(8).fill(0)),
|
||||
),
|
||||
null,
|
||||
);
|
||||
});
|
||||
@@ -247,7 +247,6 @@ export function createLineCoverage({
|
||||
entry.castRequested = false;
|
||||
entry.meshParts = null;
|
||||
if (swap && entry.kind === 'sequence' && entry.primitives?.length) {
|
||||
finishSwap(entry);
|
||||
redrawTile(entry);
|
||||
return;
|
||||
}
|
||||
@@ -373,7 +372,22 @@ export function createLineCoverage({
|
||||
|
||||
/** Rebuild a tile's lines; the old ones stay up until the new are ready. */
|
||||
function redrawTile(entry) {
|
||||
const previous = entry.primitives;
|
||||
let previous = entry.primitives;
|
||||
if (entry.swap) {
|
||||
// A swap still building: what is on screen is its old lines and
|
||||
// whichever new ones are ready. Those stay; the rest never showed.
|
||||
const shown = [...entry.swap.old];
|
||||
const building = [];
|
||||
for (const record of previous)
|
||||
(record.primitive.ready && !record.primitive.isDestroyed?.()
|
||||
? shown
|
||||
: building
|
||||
).push(record);
|
||||
entry.swap.stop();
|
||||
entry.swap = null;
|
||||
removePrimitives(building);
|
||||
previous = shown;
|
||||
}
|
||||
entry.primitives = [];
|
||||
attachPrimitive(entry);
|
||||
removeWhenReady(entry, previous);
|
||||
|
||||
@@ -397,6 +397,63 @@ test('a tile redrawn for a new share keeps its old lines until the new ones are
|
||||
);
|
||||
});
|
||||
|
||||
test('a second redraw while a swap still builds keeps the lines on screen', async (t) => {
|
||||
const { state, coverage, viewer } = setup(t, {
|
||||
zoomFor: () => ({ zoom: 18, kind: 'sequence' }),
|
||||
tuning: {
|
||||
...TUNING,
|
||||
maxTiles: 9,
|
||||
maxSequences: 900,
|
||||
maxPoints: 90,
|
||||
keepOrder: true,
|
||||
},
|
||||
decoded: (centre, tile) => {
|
||||
const dense = tile.x % 2 === 0 && tile.y % 2 === 0;
|
||||
return {
|
||||
sequences: dense
|
||||
? Array.from({ length: 30 }, (_, i) =>
|
||||
pointsLine(`d${tile.x}/${tile.y}/${i}`, 5, tile, centre),
|
||||
)
|
||||
: [pointsLine(`s${tile.x}/${tile.y}`, 2, tile, centre)],
|
||||
overview: [],
|
||||
};
|
||||
},
|
||||
});
|
||||
coverage.refresh();
|
||||
assert.ok(
|
||||
await until(
|
||||
() => state.coverage.tiles.size > 1 && !state.coverage.pending.size,
|
||||
),
|
||||
);
|
||||
// A tile mid-swap (its new lines never become ready here) with a line
|
||||
// the cap left out: selecting that line redraws the tile again.
|
||||
const entry = [...state.coverage.tiles.values()].find(
|
||||
(tile) =>
|
||||
tile.swap &&
|
||||
tile.sequenceList.some((sequence) => !tile.sequences.has(sequence.id)),
|
||||
);
|
||||
assert.ok(entry, 'a swapping tile with a line left out');
|
||||
const onScreen = [...entry.swap.old];
|
||||
const building = entry.primitives.map(({ primitive }) => primitive);
|
||||
const left = entry.sequenceList.find(
|
||||
(sequence) => !entry.sequences.has(sequence.id),
|
||||
);
|
||||
state.sequence.selectedId = left.id;
|
||||
coverage.recolorSequence(left.id, true);
|
||||
assert.ok(entry.sequences.has(left.id), 'the selected line is drawn');
|
||||
const onGlobe = (primitive) =>
|
||||
viewer.scene.groundPrimitives.items.has(primitive) ||
|
||||
viewer.scene.primitives.items.has(primitive);
|
||||
for (const { primitive } of onScreen)
|
||||
assert.ok(
|
||||
onGlobe(primitive),
|
||||
'the lines on screen stay until the new ones are ready',
|
||||
);
|
||||
for (const primitive of building)
|
||||
assert.ok(!onGlobe(primitive), 'the first swap’s unready lines are gone');
|
||||
assert.ok(entry.swap, 'still swapping');
|
||||
});
|
||||
|
||||
test('a selected line outside the cap does not make its tile redraw on every settle, nor does one the filter hides', async (t) => {
|
||||
let pano = 'all';
|
||||
const { state, coverage, viewer } = setup(t, {
|
||||
|
||||
@@ -3,6 +3,7 @@ import {
|
||||
PANO_FOV_DEFAULT_DEG,
|
||||
wrapDegrees,
|
||||
} from './panorama.js';
|
||||
import { imageSize } from './imageSize.js';
|
||||
|
||||
/** A photo download (tens of kilobytes to a few megabytes). */
|
||||
export const IMAGE_TIMEOUT_MS = 15_000;
|
||||
@@ -10,6 +11,19 @@ export const IMAGE_TIMEOUT_MS = 15_000;
|
||||
/** A picture file or decoded side past these is refused (a photo is a few MB). */
|
||||
export const MAX_IMAGE_BYTES = 48 * 1024 * 1024;
|
||||
export const MAX_IMAGE_SIDE = 16_384;
|
||||
/** 64 MP: 256 MB once decoded to RGBA. A 16,384 px square is four times that. */
|
||||
export const MAX_IMAGE_PIXELS = 8_192 * 8_192;
|
||||
/** Where in a file to look for its dimensions (a JPEG's EXIF comes first). */
|
||||
const HEADER_BYTES = 256 * 1024;
|
||||
|
||||
/** Whether a picture this size is past what the viewer decodes. */
|
||||
export function tooManyPixels(width, height) {
|
||||
return (
|
||||
width > MAX_IMAGE_SIDE ||
|
||||
height > MAX_IMAGE_SIDE ||
|
||||
width * height > MAX_IMAGE_PIXELS
|
||||
);
|
||||
}
|
||||
|
||||
/** Panorama texture widths: enough for a docked viewer, and for an expanded one. */
|
||||
export const PANO_MID_SIZE = Object.freeze([2048, 1024]);
|
||||
@@ -43,19 +57,36 @@ export function loadImage(
|
||||
return loadBitmap(url, timeoutMs, resize, failure, signal);
|
||||
return new Promise((resolve, reject) => {
|
||||
if (!url) return reject(new Error(failure));
|
||||
if (signal?.aborted) return reject(new Error(failure));
|
||||
const image = new Image();
|
||||
if (crossOrigin) image.crossOrigin = 'anonymous';
|
||||
image.decoding = 'async';
|
||||
const timer = setTimeout(() => {
|
||||
// Stop the download: another photo opened, or the time ran out.
|
||||
const stop = () => {
|
||||
clearTimeout(timer);
|
||||
signal?.removeEventListener?.('abort', stop);
|
||||
image.onload = image.onerror = null;
|
||||
image.src = '';
|
||||
reject(new Error(failure));
|
||||
}, timeoutMs);
|
||||
image.onload = () => {
|
||||
};
|
||||
const timer = setTimeout(stop, timeoutMs);
|
||||
signal?.addEventListener?.('abort', stop, { once: true });
|
||||
const settle = () => {
|
||||
clearTimeout(timer);
|
||||
signal?.removeEventListener?.('abort', stop);
|
||||
};
|
||||
image.onload = () => {
|
||||
settle();
|
||||
// An <img> reads no header first: refuse what it decoded too large.
|
||||
if (tooManyPixels(image.naturalWidth || 0, image.naturalHeight || 0)) {
|
||||
image.src = '';
|
||||
reject(oversized(failure));
|
||||
return;
|
||||
}
|
||||
resolve(image);
|
||||
};
|
||||
image.onerror = () => {
|
||||
clearTimeout(timer);
|
||||
settle();
|
||||
reject(new Error(failure));
|
||||
};
|
||||
image.src = url;
|
||||
@@ -80,6 +111,12 @@ async function loadBitmap(url, timeoutMs, resize, failure, signal) {
|
||||
const length = Number(response.headers?.get?.('content-length'));
|
||||
if (length > MAX_IMAGE_BYTES) throw oversized(failure);
|
||||
const blob = await boundedBlob(response, failure);
|
||||
// Its declared size, before anything decodes it.
|
||||
const size = imageSize(
|
||||
new Uint8Array(await blob.slice(0, HEADER_BYTES).arrayBuffer()),
|
||||
);
|
||||
if (size && tooManyPixels(size.width, size.height))
|
||||
throw oversized(failure);
|
||||
const bitmap = await (resize
|
||||
? createImageBitmap(blob, {
|
||||
resizeWidth: resize[0],
|
||||
@@ -88,7 +125,7 @@ async function loadBitmap(url, timeoutMs, resize, failure, signal) {
|
||||
})
|
||||
: createImageBitmap(blob));
|
||||
// (A resized decode reports its new size: this guards unresized ones.)
|
||||
if (bitmap.width > MAX_IMAGE_SIDE || bitmap.height > MAX_IMAGE_SIDE) {
|
||||
if (tooManyPixels(bitmap.width, bitmap.height)) {
|
||||
bitmap.close?.();
|
||||
throw oversized(failure);
|
||||
}
|
||||
@@ -152,7 +189,7 @@ export function panoSizeFor(canvasPixels, fovDeg) {
|
||||
* The panorama shown first, its texture width (0 when it is only a preview), and a size to shrink it to.
|
||||
* @property {(photo: object, size: [number, number], view: {expanded: boolean}) => {url: string, resize?: [number, number]|null, timeoutMs?: number}|null} panoSharp
|
||||
* A sharper panorama for a canvas that needs `size`, or null; `expanded` while the viewer is.
|
||||
* @property {(photo: object) => Array<string>} flatFirst Flat versions to try in turn.
|
||||
* @property {(photo: object, view: {expanded: boolean}) => Array<string>} flatFirst Flat versions to try in turn (`expanded` while the viewer is).
|
||||
* @property {(photo: object, pixels: number) => string|null} flatSharp
|
||||
* A sharper flat version for a picture this many device pixels wide, or null.
|
||||
* @property {(photo: object) => {title?: string|null, creator?: string|null, license?: string|null, externalUrl: string, capturedAtPrecision?: 'day'|'month'}} describe
|
||||
@@ -176,6 +213,7 @@ export function panoSizeFor(canvasPixels, fovDeg) {
|
||||
* @param {(photo: object, delta: -1|1) => Promise<string|null>} options.step The photo `delta` along the sequence, or null at its end.
|
||||
* @param {(imageId: string) => void} options.onStep Open the photo ‹ › chose (through the core, so loading and errors show).
|
||||
* @param {(photo: object) => void} [options.onShown] A photo is on screen.
|
||||
* @param {(photo: object) => void} [options.onShowing] The photo this open shows, just before its pose.
|
||||
* @param {(imageId: string) => void} [options.onReopen] Draw the same photo again after a GPU reset (default: `onStep`).
|
||||
* @param {() => boolean} [options.onEscape] Esc in the docked viewer: true when it was used (e.g. a selection cleared).
|
||||
* @param {object} [options.render] Render governor.
|
||||
@@ -191,6 +229,7 @@ export function createPhotoViewer({
|
||||
step,
|
||||
onStep,
|
||||
onShown = () => {},
|
||||
onShowing = () => {},
|
||||
onReopen = onStep,
|
||||
onEscape = () => false,
|
||||
render,
|
||||
@@ -227,8 +266,10 @@ export function createPhotoViewer({
|
||||
let panoComing = null;
|
||||
/** A resize came while it loaded: look again once it lands. */
|
||||
let lookAgain = false;
|
||||
/** The sharper flat version on screen or on its way (one download each). */
|
||||
let flatSharpUrl = null;
|
||||
/** The sharper flat version on screen, and the one on its way. */
|
||||
let flatSharp = { shown: null, loading: null };
|
||||
/** The open in progress: another open, a close or an unmount stops its downloads. */
|
||||
let openAbort = null;
|
||||
/** The photo an open is fetching: ‹ › step on from it, not the old one. */
|
||||
let openingId = null;
|
||||
/** ‹ › chose this photo: keep the panorama's look direction for it. */
|
||||
@@ -626,6 +667,9 @@ export function createPhotoViewer({
|
||||
async function upgrade(target, seq, kind) {
|
||||
// A 360° photo shown flat (no WebGL) keeps its front view.
|
||||
if (kind === 'flat' && target.isPano) return;
|
||||
// Another photo is opening: the one still shown is not worth more bytes.
|
||||
if (openingId !== null && openingId !== target.id) return;
|
||||
const signal = openAbort?.signal;
|
||||
if (kind === 'pano') {
|
||||
const size = panoSizeNow();
|
||||
if (size[0] <= panoWidth) return;
|
||||
@@ -636,7 +680,7 @@ export function createPhotoViewer({
|
||||
return;
|
||||
}
|
||||
const sharp = sources.panoSharp(target, size, {
|
||||
expanded: Boolean(root?.closest?.('.sl-viewer-wrap-expanded')),
|
||||
expanded: isExpanded(),
|
||||
});
|
||||
if (!sharp?.url) return;
|
||||
const coming = { seq, width: size[0], controller: new AbortController() };
|
||||
@@ -674,19 +718,28 @@ export function createPhotoViewer({
|
||||
return;
|
||||
}
|
||||
const url = sources.flatSharp(target, viewerPixels());
|
||||
if (!url || url === flatSharpUrl) return;
|
||||
flatSharpUrl = url;
|
||||
if (!url || url === flatSharp.shown || url === flatSharp.loading) return;
|
||||
const sharp = flatSharp;
|
||||
sharp.loading = url;
|
||||
try {
|
||||
const image = await loadAs(url, { crossOrigin: false });
|
||||
const image = await loadAs(url, { crossOrigin: false, signal });
|
||||
if (seq !== openSeq || photo !== target || showing !== 'flat') return;
|
||||
picture.src = image.src;
|
||||
sharp.shown = url;
|
||||
requestRender();
|
||||
} catch {
|
||||
// The first version stays; a later resize may try again.
|
||||
if (flatSharpUrl === url && photo === target) flatSharpUrl = null;
|
||||
/* the first version stays; a later resize may try again */
|
||||
} finally {
|
||||
// Discarded or failed: nothing is on its way, so it may be asked again.
|
||||
if (sharp.loading === url) sharp.loading = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** The viewer is the expanded dialog, not docked in the panel. */
|
||||
function isExpanded() {
|
||||
return Boolean(root?.closest?.('.sl-viewer-wrap-expanded'));
|
||||
}
|
||||
|
||||
/** Another photo, or none: a sharper panorama still on its way is not needed. */
|
||||
function dropSharpPanorama() {
|
||||
panoComing?.controller?.abort();
|
||||
@@ -701,14 +754,16 @@ export function createPhotoViewer({
|
||||
return panoSizeFor(width, renderer?.getView().fov ?? PANO_FOV_DEFAULT_DEG);
|
||||
}
|
||||
|
||||
async function showPano(target, seq, keepView) {
|
||||
async function showPano(target, seq, keepView, signal) {
|
||||
// If the panorama fails, the caller shows the front view rather than
|
||||
// waiting on it twice.
|
||||
const first = sources.panoFirst(target);
|
||||
if (!first?.url) throw new Error(LOAD_FAILED);
|
||||
const image = await loadAs(first.url, {
|
||||
crossOrigin: true,
|
||||
resize: first.resize ?? null,
|
||||
signal,
|
||||
// Never decoded at more than the texture it becomes.
|
||||
resize: first.resize ?? PANO_MID_SIZE,
|
||||
});
|
||||
if (seq !== openSeq) {
|
||||
image.close?.();
|
||||
@@ -730,17 +785,20 @@ export function createPhotoViewer({
|
||||
show('pano');
|
||||
renderer.setImage(image);
|
||||
panoWidth = first.width || 0;
|
||||
panoComing = null;
|
||||
// A sharper version of the photo before was dropped by `open`; one of
|
||||
// this photo, asked by a resize meanwhile, is still worth having.
|
||||
return true;
|
||||
}
|
||||
|
||||
async function showFlat(target, seq) {
|
||||
const urls = sources.flatFirst(target).filter(Boolean);
|
||||
async function showFlat(target, seq, signal) {
|
||||
const urls = sources
|
||||
.flatFirst(target, { expanded: isExpanded() })
|
||||
.filter(Boolean);
|
||||
let image;
|
||||
let failure = null;
|
||||
for (const url of urls) {
|
||||
try {
|
||||
image = await loadAs(url, { crossOrigin: false });
|
||||
image = await loadAs(url, { crossOrigin: false, signal });
|
||||
break;
|
||||
} catch (error) {
|
||||
failure = error;
|
||||
@@ -750,26 +808,27 @@ export function createPhotoViewer({
|
||||
if (!image) throw failure || new Error(LOAD_FAILED);
|
||||
if (seq !== openSeq) return false;
|
||||
picture.src = image.src;
|
||||
flatSharpUrl = null;
|
||||
flatSharp = { shown: null, loading: null };
|
||||
photo = target;
|
||||
show('flat');
|
||||
return true;
|
||||
}
|
||||
|
||||
async function openPhoto(id, seq, keepView) {
|
||||
const target = await getPhoto(id);
|
||||
async function openPhoto(id, seq, keepView, signal) {
|
||||
const target = await getPhoto(id, { signal });
|
||||
if (seq !== openSeq) return;
|
||||
let shown = false;
|
||||
if (sources.hasPano(target) && ensureRenderer()) {
|
||||
try {
|
||||
shown = await showPano(target, seq, keepView);
|
||||
shown = await showPano(target, seq, keepView, signal);
|
||||
} catch {
|
||||
// The panorama failed: its front view is better than nothing.
|
||||
if (seq !== openSeq) return;
|
||||
shown = await showFlat(target, seq);
|
||||
shown = await showFlat(target, seq, signal);
|
||||
}
|
||||
} else shown = await showFlat(target, seq);
|
||||
} else shown = await showFlat(target, seq, signal);
|
||||
if (!shown) return;
|
||||
onShowing(target);
|
||||
emit();
|
||||
onShown(target);
|
||||
upgrade(target, seq, showing);
|
||||
@@ -790,11 +849,18 @@ export function createPhotoViewer({
|
||||
const id = String(imageId);
|
||||
const seq = ++openSeq;
|
||||
dropSharpPanorama();
|
||||
// The previous open's downloads stop: only this photo is wanted now.
|
||||
openAbort?.abort();
|
||||
const opening = (openAbort = new AbortController());
|
||||
const keepView = keepViewFor === id;
|
||||
keepViewFor = null;
|
||||
openingId = id;
|
||||
try {
|
||||
await openPhoto(id, seq, keepView);
|
||||
await openPhoto(id, seq, keepView, opening.signal);
|
||||
} catch (error) {
|
||||
// A later open, or a close, stopped this one: that is not a failure.
|
||||
if (seq !== openSeq) return;
|
||||
throw error;
|
||||
} finally {
|
||||
if (openingId === id && seq === openSeq) openingId = null;
|
||||
}
|
||||
@@ -803,10 +869,12 @@ export function createPhotoViewer({
|
||||
close() {
|
||||
openSeq++;
|
||||
dropSharpPanorama();
|
||||
openAbort?.abort();
|
||||
openAbort = null;
|
||||
openingId = null;
|
||||
keepViewFor = null;
|
||||
photo = null;
|
||||
flatSharpUrl = null;
|
||||
flatSharp = { shown: null, loading: null };
|
||||
show(null);
|
||||
if (picture) picture.removeAttribute('src');
|
||||
renderer?.clear?.();
|
||||
@@ -815,10 +883,12 @@ export function createPhotoViewer({
|
||||
unmount() {
|
||||
openSeq++;
|
||||
dropSharpPanorama();
|
||||
openAbort?.abort();
|
||||
openAbort = null;
|
||||
refused.clear();
|
||||
openingId = null;
|
||||
keepViewFor = null;
|
||||
flatSharpUrl = null;
|
||||
flatSharp = { shown: null, loading: null };
|
||||
teardown();
|
||||
},
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ const PHOTOS = {
|
||||
a: { id: 'a', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: true },
|
||||
b: { id: 'b', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: true },
|
||||
f: { id: 'f', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: false },
|
||||
g: { id: 'g', sequenceId: 's', lon: 1, lat: 2, heading: 0, isPano: false },
|
||||
};
|
||||
|
||||
/** Sources whose panoramas and sharper flat versions are named after the photo. */
|
||||
@@ -294,7 +295,11 @@ test('a picture file or bitmap past any real photo’s size is refused', async (
|
||||
(globalThis.fetch = async () => ({
|
||||
ok: true,
|
||||
headers: new Map([['content-length', String(length ?? '')]]),
|
||||
blob: async () => ({ size: size ?? 10, bitmap: bitmap(width, height) }),
|
||||
blob: async () => ({
|
||||
size: size ?? 10,
|
||||
bitmap: bitmap(width, height),
|
||||
slice: () => ({ arrayBuffer: async () => new ArrayBuffer(0) }),
|
||||
}),
|
||||
}));
|
||||
const refusedForSize = (error) => {
|
||||
assert.match(error.message, /could not be loaded/);
|
||||
@@ -331,3 +336,203 @@ test('a picture file or bitmap past any real photo’s size is refused', async (
|
||||
respond({});
|
||||
assert.equal((await loadImage('https://x/a.jpg')).width, 100);
|
||||
});
|
||||
|
||||
/** A viewer whose loads are held, recording each load's signal by url. */
|
||||
function heldViewer() {
|
||||
const pending = new Map();
|
||||
const signals = new Map();
|
||||
const loads = [];
|
||||
const viewer = createPhotoViewer({
|
||||
providerId: 'test',
|
||||
name: 'Test',
|
||||
sources: SOURCES,
|
||||
getPhoto: async (id) => PHOTOS[id],
|
||||
step: async () => null,
|
||||
onStep() {},
|
||||
load: (url, options = {}) =>
|
||||
new Promise((resolve, reject) => {
|
||||
loads.push(url);
|
||||
signals.set(url, options.signal);
|
||||
options.signal?.addEventListener('abort', () =>
|
||||
reject(new Error('aborted')),
|
||||
);
|
||||
pending.set(url, { resolve, reject });
|
||||
}),
|
||||
});
|
||||
const release = (url) => {
|
||||
pending.get(url).resolve({ src: url });
|
||||
pending.delete(url);
|
||||
};
|
||||
return { viewer, pending, signals, loads, release, host: fakeHost() };
|
||||
}
|
||||
|
||||
test('a new open, a close or an unmount stops the downloads of the open before it', async () => {
|
||||
const env = heldViewer();
|
||||
await env.viewer.mount(env.host);
|
||||
const first = env.viewer.open('f');
|
||||
assert.ok(await until(() => env.signals.has('f-first')));
|
||||
const second = env.viewer.open('g');
|
||||
assert.equal(env.signals.get('f-first').aborted, true, 'f is not wanted now');
|
||||
await first.catch(() => {});
|
||||
assert.ok(await until(() => env.signals.has('g-first')));
|
||||
env.viewer.close();
|
||||
assert.equal(env.signals.get('g-first').aborted, true, 'closed');
|
||||
await second.catch(() => {});
|
||||
const third = env.viewer.open('f');
|
||||
assert.ok(await until(() => env.pending.has('f-first')));
|
||||
env.viewer.unmount();
|
||||
assert.equal(env.signals.get('f-first').aborted, true, 'unmounted');
|
||||
await third.catch(() => {});
|
||||
});
|
||||
|
||||
test('a sharper flat version on its way stops when another photo opens, and is asked again later', async (t) => {
|
||||
t.after(() => delete globalThis.devicePixelRatio);
|
||||
globalThis.devicePixelRatio = 1;
|
||||
const env = heldViewer();
|
||||
await env.viewer.mount(env.host);
|
||||
parts(env.host).picture.clientWidth = 1200;
|
||||
const opening = env.viewer.open('f');
|
||||
assert.ok(await until(() => env.pending.has('f-first')));
|
||||
env.release('f-first');
|
||||
await opening;
|
||||
assert.ok(
|
||||
await until(() => env.pending.has('f-sharp')),
|
||||
'the 1,200 px viewer asks for more',
|
||||
);
|
||||
// Resizes while it loads (the panel animating) do not ask again.
|
||||
env.viewer.resize();
|
||||
env.viewer.resize();
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
assert.equal(env.loads.filter((url) => url === 'f-sharp').length, 1);
|
||||
// g opens (and fails): f's sharp download stops, f stays on screen.
|
||||
const failing = env.viewer.open('g');
|
||||
assert.equal(env.signals.get('f-sharp').aborted, true);
|
||||
assert.ok(await until(() => env.pending.has('g-first')));
|
||||
env.pending.get('g-first').reject(new Error('gone'));
|
||||
env.pending.delete('g-first');
|
||||
await failing.catch(() => {});
|
||||
// The discarded response does not block f's sharp version for good.
|
||||
env.signals.delete('f-sharp');
|
||||
env.viewer.resize();
|
||||
assert.ok(await until(() => env.signals.has('f-sharp')), 'asked again');
|
||||
});
|
||||
|
||||
test('a resize while another photo opens does not upgrade the photo still shown', async (t) => {
|
||||
t.after(() => delete globalThis.devicePixelRatio);
|
||||
globalThis.devicePixelRatio = 1;
|
||||
const env = heldViewer();
|
||||
await env.viewer.mount(env.host);
|
||||
const opening = env.viewer.open('f');
|
||||
assert.ok(await until(() => env.pending.has('f-first')));
|
||||
env.release('f-first');
|
||||
await opening;
|
||||
// g is opening; EXPAND resizes the viewer meanwhile.
|
||||
const next = env.viewer.open('g');
|
||||
assert.ok(await until(() => env.pending.has('g-first')));
|
||||
parts(env.host).picture.clientWidth = 1200;
|
||||
env.viewer.resize();
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
assert.equal(env.pending.has('f-sharp'), false, 'f is on its way out');
|
||||
env.release('g-first');
|
||||
await next;
|
||||
assert.ok(await until(() => env.pending.has('g-sharp')), 'g gets it');
|
||||
});
|
||||
|
||||
test('a panorama is always decoded at a bounded size', async () => {
|
||||
const env = heldViewer();
|
||||
const options = [];
|
||||
const viewer = createPhotoViewer({
|
||||
providerId: 'test',
|
||||
name: 'Test',
|
||||
sources: SOURCES,
|
||||
getPhoto: async (id) => PHOTOS[id],
|
||||
step: async () => null,
|
||||
onStep() {},
|
||||
load: async (url, opts) => {
|
||||
options.push(opts);
|
||||
return { src: url };
|
||||
},
|
||||
createRenderer: () => ({
|
||||
setImage() {},
|
||||
setView() {},
|
||||
getView: () => ({ yaw: 0, pitch: 0, fov: 80 }),
|
||||
resize() {},
|
||||
clear() {},
|
||||
destroy() {},
|
||||
}),
|
||||
});
|
||||
await viewer.mount(env.host);
|
||||
await viewer.open('a');
|
||||
// SOURCES' panoFirst names no size: the viewer still bounds it.
|
||||
assert.deepEqual(options[0].resize, [2048, 1024]);
|
||||
});
|
||||
|
||||
test('a header that declares a huge image is refused before anything decodes it', async (t) => {
|
||||
const { loadImage } = await import('./photoViewer.js');
|
||||
const saved = {
|
||||
fetch: globalThis.fetch,
|
||||
createImageBitmap: globalThis.createImageBitmap,
|
||||
};
|
||||
t.after(() => Object.assign(globalThis, saved));
|
||||
let decoded = 0;
|
||||
globalThis.createImageBitmap = async () => {
|
||||
decoded++;
|
||||
return { width: 10, height: 10 };
|
||||
};
|
||||
// A 1 kB PNG that says it is 16,384 × 16,384.
|
||||
const png = new Uint8Array(1024);
|
||||
png.set([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 13]);
|
||||
png.set(
|
||||
[...'IHDR'].map((c) => c.charCodeAt(0)),
|
||||
12,
|
||||
);
|
||||
png.set([0, 0, 0x40, 0, 0, 0, 0x40, 0], 16);
|
||||
globalThis.fetch = async () => new Response(png);
|
||||
await assert.rejects(
|
||||
loadImage('https://x/bomb.png'),
|
||||
(error) => error.tooLarge === true,
|
||||
);
|
||||
assert.equal(decoded, 0, 'never decoded');
|
||||
});
|
||||
|
||||
test('an <img> that turns out too large is refused and let go', async (t) => {
|
||||
const { loadImage, MAX_IMAGE_SIDE } = await import('./photoViewer.js');
|
||||
const saved = globalThis.Image;
|
||||
t.after(() => (globalThis.Image = saved));
|
||||
let size = [MAX_IMAGE_SIDE + 1, 100];
|
||||
const images = [];
|
||||
globalThis.Image = class {
|
||||
constructor() {
|
||||
images.push(this);
|
||||
}
|
||||
set src(value) {
|
||||
this._src = value;
|
||||
if (value)
|
||||
queueMicrotask(() => {
|
||||
[this.naturalWidth, this.naturalHeight] = size;
|
||||
this.onload?.();
|
||||
});
|
||||
}
|
||||
get src() {
|
||||
return this._src;
|
||||
}
|
||||
};
|
||||
await assert.rejects(
|
||||
loadImage('https://x/a.jpg', { crossOrigin: false }),
|
||||
(error) => error.tooLarge === true,
|
||||
);
|
||||
assert.equal(images.at(-1).src, '', 'its pixels let go');
|
||||
size = [4000, 3000];
|
||||
const ok = await loadImage('https://x/b.jpg', { crossOrigin: false });
|
||||
assert.equal(ok.naturalWidth, 4000);
|
||||
// An aborted <img> load stops and rejects.
|
||||
size = [10, 10];
|
||||
const controller = new AbortController();
|
||||
const aborted = loadImage('https://x/c.jpg', {
|
||||
crossOrigin: false,
|
||||
signal: controller.signal,
|
||||
});
|
||||
controller.abort();
|
||||
await assert.rejects(aborted);
|
||||
assert.equal(images.at(-1).src, '', 'its download stops');
|
||||
});
|
||||
@@ -37,6 +37,7 @@ const KARTAVIEW_SOURCES = Object.freeze({
|
||||
panoFirst: (photo) => ({
|
||||
url: resizedImageUrl(photo.urls.pano, PANO_PREVIEW_SIZE),
|
||||
width: 0,
|
||||
resize: PANO_PREVIEW_SIZE,
|
||||
}),
|
||||
panoSharp: (photo, size) => ({
|
||||
url: resizedImageUrl(photo.urls.pano, size),
|
||||
|
||||
@@ -883,6 +883,11 @@ test('the panorama is as sharp as the viewer needs: 2,048 docked, 4,096 once exp
|
||||
});
|
||||
|
||||
test('a CORS image is decoded off the main thread into an ImageBitmap where the browser can', async (t) => {
|
||||
// A blob whose header names no size (the viewer then reads none).
|
||||
const fakeBlob = {
|
||||
size: 4,
|
||||
slice: () => ({ arrayBuffer: async () => new ArrayBuffer(0) }),
|
||||
};
|
||||
const saved = {
|
||||
fetch: globalThis.fetch,
|
||||
bitmap: globalThis.createImageBitmap,
|
||||
@@ -896,7 +901,7 @@ test('a CORS image is decoded off the main thread into an ImageBitmap where the
|
||||
fetched.push({ url, mode: init.mode });
|
||||
return url.includes('gone')
|
||||
? { ok: false, status: 409 }
|
||||
: { ok: true, blob: async () => 'blob' };
|
||||
: { ok: true, blob: async () => fakeBlob };
|
||||
};
|
||||
globalThis.createImageBitmap = async (blob) => ({
|
||||
blob,
|
||||
@@ -906,7 +911,7 @@ test('a CORS image is decoded off the main thread into an ImageBitmap where the
|
||||
const bitmap = await loadImage('https://cdn.kartaview.org/a', {
|
||||
crossOrigin: true,
|
||||
});
|
||||
assert.deepEqual(bitmap, { blob: 'blob', width: 4096, height: 2048 });
|
||||
assert.deepEqual(bitmap, { blob: fakeBlob, width: 4096, height: 2048 });
|
||||
assert.deepEqual(fetched, [
|
||||
{ url: 'https://cdn.kartaview.org/a', mode: 'cors' },
|
||||
]);
|
||||
|
||||
@@ -2,6 +2,7 @@ import { PbfReader } from 'pbf';
|
||||
import { VectorTile } from '@mapbox/vector-tile';
|
||||
import { tileLocalToLonLat } from '../../tileMath.js';
|
||||
import {
|
||||
DECODE_SLICE_MS,
|
||||
OVERVIEW_BINS_PER_AXIS,
|
||||
SIMPLIFY_TOLERANCE_UNITS,
|
||||
panoramaxTimeMs,
|
||||
@@ -66,11 +67,62 @@ function dayMs(value) {
|
||||
* most OVERVIEW_BINS_PER_AXIS² a tile (a z0 tile holds ~70,000 cells). A
|
||||
* cell has 360° and flat counts but no date.
|
||||
* - `pictures` (z15) is left for `decodePictures`.
|
||||
*
|
||||
* In one go; `decodeCoverageTileSliced` gives the page a turn between
|
||||
* slices.
|
||||
* @param {Uint8Array} bytes
|
||||
* @param {{x: number, y: number, z: number}} address
|
||||
* @returns {{sequences: Array<{id: string, parts: Array<Array<[number, number]>>, capturedAt: number, isPano: boolean}>, overview: Array<{id: string, lon: number, lat: number, capturedAt: null, isPano: boolean, hasPano: boolean, hasFlat: boolean, weight: number}>}}
|
||||
*/
|
||||
export function decodeCoverageTile(bytes, address) {
|
||||
const steps = coverageSteps(bytes, address);
|
||||
let step = steps.next();
|
||||
while (!step.done) step = steps.next();
|
||||
return step.value;
|
||||
}
|
||||
|
||||
/** Give the page a turn: input and rendering run before the next slice. */
|
||||
function nextTask() {
|
||||
if (typeof globalThis.scheduler?.yield === 'function')
|
||||
return globalThis.scheduler.yield();
|
||||
return new Promise((resolve) => setTimeout(resolve, 0));
|
||||
}
|
||||
|
||||
/**
|
||||
* `decodeCoverageTile` in slices of about `sliceMs`, the page getting a
|
||||
* turn between them: a 2 MB z10 tile over Paris is ~200 ms of decoding on
|
||||
* a fast laptop, more on a phone, and would freeze the globe while the
|
||||
* camera approaches. Stops at the next slice once `signal` aborts.
|
||||
* @param {Uint8Array} bytes
|
||||
* @param {{x: number, y: number, z: number}} address
|
||||
* @param {{signal?: AbortSignal, sliceMs?: number, pause?: () => Promise<void>, now?: () => number}} [options]
|
||||
*/
|
||||
export async function decodeCoverageTileSliced(
|
||||
bytes,
|
||||
address,
|
||||
{
|
||||
signal,
|
||||
sliceMs = DECODE_SLICE_MS,
|
||||
pause = nextTask,
|
||||
now = () => performance.now(),
|
||||
} = {},
|
||||
) {
|
||||
const steps = coverageSteps(bytes, address);
|
||||
let started = now();
|
||||
let step = steps.next();
|
||||
while (!step.done) {
|
||||
if (now() - started >= sliceMs) {
|
||||
await pause();
|
||||
signal?.throwIfAborted();
|
||||
started = now();
|
||||
}
|
||||
step = steps.next();
|
||||
}
|
||||
return step.value;
|
||||
}
|
||||
|
||||
/** The decode, a step a feature (see `decodeCoverageTile`). */
|
||||
function* coverageSteps(bytes, address) {
|
||||
const result = { sequences: [], overview: [] };
|
||||
if (!bytes || !bytes.length) return result;
|
||||
const tile = new VectorTile(new PbfReader(bytes));
|
||||
@@ -78,7 +130,7 @@ export function decodeCoverageTile(bytes, address) {
|
||||
|
||||
const sequences = tile.layers.sequences;
|
||||
if (sequences) {
|
||||
const lengths = new Map();
|
||||
const measured = [];
|
||||
for (let i = 0; i < sequences.length; i++) {
|
||||
const feature = sequences.feature(i);
|
||||
const props = feature.properties || {};
|
||||
@@ -113,10 +165,13 @@ export function decodeCoverageTile(bytes, address) {
|
||||
? false
|
||||
: null,
|
||||
};
|
||||
lengths.set(sequence, length);
|
||||
result.sequences.push(sequence);
|
||||
measured.push({ sequence, length });
|
||||
yield;
|
||||
}
|
||||
result.sequences.sort((a, b) => lengths.get(b) - lengths.get(a));
|
||||
// A turn before the sort: over a city it is thousands of lines.
|
||||
yield;
|
||||
measured.sort((a, b) => b.length - a.length);
|
||||
result.sequences = measured.map(({ sequence }) => sequence);
|
||||
}
|
||||
|
||||
const grid = tile.layers.grid;
|
||||
@@ -127,6 +182,7 @@ export function decodeCoverageTile(bytes, address) {
|
||||
const feature = grid.feature(i);
|
||||
const props = feature.properties || {};
|
||||
const point = feature.loadGeometry()?.[0]?.[0];
|
||||
if (i % 64 === 63) yield;
|
||||
if (!point) continue;
|
||||
const pano = Number(props.nb_360_pictures) || 0;
|
||||
const flat = Number(props.nb_flat_pictures) || 0;
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { decodeCoverageTile, decodePictures, simplifyLine } from './decode.js';
|
||||
import {
|
||||
decodeCoverageTile,
|
||||
decodeCoverageTileSliced,
|
||||
decodePictures,
|
||||
simplifyLine,
|
||||
} from './decode.js';
|
||||
import { OVERVIEW_BINS_PER_AXIS } from './policy.js';
|
||||
import { encodePanoramaxTile } from '../../../../testSupport/panoramaxFixtures.mjs';
|
||||
import { tileCentre } from '../../../../testSupport/streetLevelFakes.mjs';
|
||||
@@ -69,10 +74,11 @@ test('sequence lines keep their parts apart, with their day and camera type', ()
|
||||
|
||||
test('density cells bin to a bounded number of points that keep the counts and kinds', () => {
|
||||
const tile = { x: 0, y: 0, z: 0 };
|
||||
// Two cells in one bin, one far away, one empty.
|
||||
// Two cells in one bin (3.75° wide at z0) far enough apart to tell
|
||||
// which it sits on, one far away, one empty.
|
||||
const grid = [
|
||||
{ lon: 2.35, lat: 48.85, pano: 0, flat: 5 },
|
||||
{ lon: 2.36, lat: 48.85, pano: 30, flat: 0 },
|
||||
{ lon: 0.3, lat: 48.85, pano: 0, flat: 5 },
|
||||
{ lon: 3.4, lat: 48.85, pano: 30, flat: 0 },
|
||||
{ lon: -73.9, lat: 40.7, pano: 1, flat: 0 },
|
||||
{ lon: 10, lat: 10, pano: 0, flat: 0 },
|
||||
];
|
||||
@@ -87,7 +93,10 @@ test('density cells bin to a bounded number of points that keep the counts and k
|
||||
assert.equal(paris.hasFlat, true);
|
||||
assert.equal(paris.isPano, true, 'mostly 360°');
|
||||
assert.equal(paris.capturedAt, null, 'cells are undated');
|
||||
assert.ok(near(paris.lon, 2.36, 0.2), 'sits on the busiest cell');
|
||||
assert.ok(
|
||||
near(paris.lon, 3.4, 0.2),
|
||||
`sits on the busiest cell: ${paris.lon}`,
|
||||
);
|
||||
const nyc = overview.find((p) => p.lon < 0);
|
||||
assert.deepEqual([nyc.hasPano, nyc.hasFlat, nyc.weight], [true, false, 1]);
|
||||
});
|
||||
@@ -134,6 +143,71 @@ test('a dense grid tile never draws more than the bins allow', () => {
|
||||
);
|
||||
});
|
||||
|
||||
/** A tile with many lines and a dense grid, for the sliced decode. */
|
||||
function busyTile() {
|
||||
const { tile, lon, lat } = tileCentre(2.35, 48.85);
|
||||
const sequences = Array.from({ length: 200 }, (_, i) => ({
|
||||
id: `s${i}`,
|
||||
parts: [
|
||||
[
|
||||
[lon, lat + i / 1e6],
|
||||
[lon + (i + 1) / 73_000, lat + i / 1e6],
|
||||
],
|
||||
],
|
||||
}));
|
||||
const grid = Array.from({ length: 500 }, (_, i) => ({
|
||||
lon: lon + (i % 25) / 1e4,
|
||||
lat: lat + Math.floor(i / 25) / 1e4,
|
||||
}));
|
||||
return { tile, bytes: encodePanoramaxTile(tile, { sequences, grid }) };
|
||||
}
|
||||
|
||||
test('a sliced decode gives the page turns and decodes the same as one in one go', async () => {
|
||||
const { tile, bytes } = busyTile();
|
||||
let clock = 0;
|
||||
let pauses = 0;
|
||||
const sliced = await decodeCoverageTileSliced(bytes, tile, {
|
||||
sliceMs: 8,
|
||||
// Each step "takes" a millisecond: a turn every eight features.
|
||||
now: () => clock++,
|
||||
pause: async () => {
|
||||
pauses++;
|
||||
},
|
||||
});
|
||||
assert.ok(pauses > 20, `${pauses} turns`);
|
||||
assert.deepEqual(sliced, decodeCoverageTile(bytes, tile));
|
||||
});
|
||||
|
||||
test('a sliced decode stops at the next turn once aborted', async () => {
|
||||
const { tile, bytes } = busyTile();
|
||||
const controller = new AbortController();
|
||||
let clock = 0;
|
||||
let pauses = 0;
|
||||
await assert.rejects(
|
||||
decodeCoverageTileSliced(bytes, tile, {
|
||||
signal: controller.signal,
|
||||
now: () => (clock += 10),
|
||||
pause: async () => {
|
||||
if (++pauses === 2) controller.abort();
|
||||
},
|
||||
}),
|
||||
{ name: 'AbortError' },
|
||||
);
|
||||
assert.equal(pauses, 2, 'no work after the abort');
|
||||
});
|
||||
|
||||
test('a fast sliced decode takes no turns', async () => {
|
||||
const { tile, bytes } = busyTile();
|
||||
let pauses = 0;
|
||||
await decodeCoverageTileSliced(bytes, tile, {
|
||||
now: () => 0,
|
||||
pause: async () => {
|
||||
pauses++;
|
||||
},
|
||||
});
|
||||
assert.equal(pauses, 0);
|
||||
});
|
||||
|
||||
test('an empty tile decodes to nothing', () => {
|
||||
for (const bytes of [new Uint8Array(0), null, undefined])
|
||||
assert.deepEqual(decodeCoverageTile(bytes, { x: 0, y: 0, z: 0 }), {
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
} from '../../tileMath.js';
|
||||
import { metresBetween } from '../../view.js';
|
||||
import { createLru, createTileCache } from '../../tileCache.js';
|
||||
import { decodeCoverageTile, decodePictures } from './decode.js';
|
||||
import { decodeCoverageTileSliced, decodePictures } from './decode.js';
|
||||
import { createPanoramaxSource } from './source.js';
|
||||
import { createPanoramaxViewer } from './viewer.js';
|
||||
import {
|
||||
@@ -230,16 +230,23 @@ export function createPanoramaxProvider({
|
||||
return photo;
|
||||
}
|
||||
|
||||
/** Lookups in flight, shared: ‹ › and the viewer ask for the same picture. */
|
||||
/**
|
||||
* Lookups in flight, shared: ‹ › and the viewer ask for the same
|
||||
* picture. A lookup stops once every caller waiting on it has given up
|
||||
* (its search would hold one of the two API connections for nothing).
|
||||
*/
|
||||
const inflight = new Map();
|
||||
function getPhoto(photoId) {
|
||||
function getPhoto(photoId, { signal } = {}) {
|
||||
const id = String(photoId);
|
||||
const cached = photos.get(id);
|
||||
if (cached) return Promise.resolve(cached);
|
||||
if (signal?.aborted) return Promise.reject(signal.reason);
|
||||
let lookup = inflight.get(id);
|
||||
if (!lookup) {
|
||||
lookup = source
|
||||
.getItem(id)
|
||||
const controller = new AbortController();
|
||||
lookup = { controller, waiting: 0 };
|
||||
lookup.promise = source
|
||||
.getItem(id, { signal: controller.signal })
|
||||
.catch((error) => {
|
||||
// Ids are UUIDs: the API answers 400 to anything else.
|
||||
if (error?.status === 400) return null;
|
||||
@@ -250,10 +257,39 @@ export function createPanoramaxProvider({
|
||||
if (!photo) throw new Error(GONE_MESSAGE);
|
||||
return photo;
|
||||
})
|
||||
.finally(() => inflight.delete(id));
|
||||
.finally(() => {
|
||||
if (inflight.get(id) === lookup) inflight.delete(id);
|
||||
});
|
||||
lookup.promise.catch(() => {});
|
||||
inflight.set(id, lookup);
|
||||
}
|
||||
return lookup;
|
||||
const shared = lookup;
|
||||
shared.waiting++;
|
||||
if (!signal) shared.waiting = Infinity;
|
||||
return new Promise((resolve, reject) => {
|
||||
const leave = () => {
|
||||
reject(signal.reason);
|
||||
shared.waiting--;
|
||||
// A caller in the same tick (the same photo opened again) may
|
||||
// still join: stop only if nobody has.
|
||||
queueMicrotask(() => {
|
||||
if (shared.waiting > 0) return;
|
||||
shared.controller.abort(signal.reason);
|
||||
if (inflight.get(id) === shared) inflight.delete(id);
|
||||
});
|
||||
};
|
||||
signal?.addEventListener('abort', leave, { once: true });
|
||||
shared.promise.then(
|
||||
(photo) => {
|
||||
signal?.removeEventListener('abort', leave);
|
||||
resolve(photo);
|
||||
},
|
||||
(error) => {
|
||||
signal?.removeEventListener('abort', leave);
|
||||
reject(error);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -281,7 +317,11 @@ export function createPanoramaxProvider({
|
||||
size: COVERAGE_TILE_CACHE_SIZE,
|
||||
load: (tile, { signal }) =>
|
||||
readable(async () =>
|
||||
decodeCoverageTile(await source.getTile(tile, { signal }), tile),
|
||||
decodeCoverageTileSliced(
|
||||
await source.getTile(tile, { signal }),
|
||||
tile,
|
||||
{ signal },
|
||||
),
|
||||
),
|
||||
});
|
||||
/** A z15 tile's pictures of one sequence, by `z/x/y:sequence`. */
|
||||
@@ -326,21 +366,40 @@ export function createPanoramaxProvider({
|
||||
const tiles = pictureTilesAround(anchor);
|
||||
// Known from the start: a picture opening while these load is
|
||||
// judged against them (re-centred if outside, left alone inside).
|
||||
drawn = { sequenceId, tiles: new Set(tiles.map(tileKey)) };
|
||||
// The home tile must load; a neighbour that fails only thins the cones.
|
||||
const lists = await Promise.allSettled(
|
||||
tiles.map((tile) => picturesIn(tile, sequenceId, signal)),
|
||||
);
|
||||
const covering = { sequenceId, tiles: new Set(tiles.map(tileKey)) };
|
||||
drawn = covering;
|
||||
const conesOf = (lists) =>
|
||||
thinImages(
|
||||
[
|
||||
...new Map(
|
||||
lists.flat().map((image) => [image.id, image]),
|
||||
).values(),
|
||||
].sort((a, b) => a.capturedAt - b.capturedAt),
|
||||
);
|
||||
const keyOf = (tile) => `${tileKey(tile)}:${sequenceId}`;
|
||||
// The home tile is needed now (it holds the open picture).
|
||||
const home = await picturesIn(tiles[0], sequenceId, signal);
|
||||
signal?.throwIfAborted();
|
||||
if (lists[0].status === 'rejected') throw lists[0].reason;
|
||||
const byId = new Map(
|
||||
lists
|
||||
.flatMap((list) => list.value ?? [])
|
||||
.map((image) => [image.id, image]),
|
||||
);
|
||||
return thinImages(
|
||||
[...byId.values()].sort((a, b) => a.capturedAt - b.capturedAt),
|
||||
// Neighbours already read draw with it; the rest (each up to
|
||||
// 1.4 MB in central Paris) join the cones as they arrive.
|
||||
const neighbours = tiles.slice(1);
|
||||
const kept = neighbours
|
||||
.map((tile) => pictureLists.get(keyOf(tile)))
|
||||
.filter(Boolean);
|
||||
const missing = neighbours.filter(
|
||||
(tile) => !pictureLists.get(keyOf(tile)),
|
||||
);
|
||||
if (missing.length)
|
||||
Promise.allSettled(
|
||||
missing.map((tile) => picturesIn(tile, sequenceId, signal)),
|
||||
).then((lists) => {
|
||||
if (signal?.aborted || drawn !== covering) return;
|
||||
dropFailed(covering, missing, lists);
|
||||
// Only a tile that arrived now redraws (no loop on failures).
|
||||
if (lists.some((list) => list.status === 'fulfilled'))
|
||||
cones.select(sequenceId, { force: true });
|
||||
});
|
||||
return conesOf([home, ...kept]);
|
||||
},
|
||||
colors: COLORS,
|
||||
pickPrefix: PICK_PREFIX.image,
|
||||
@@ -350,6 +409,17 @@ export function createPanoramaxProvider({
|
||||
coverage.recolorSequence(sequenceId, selected),
|
||||
});
|
||||
|
||||
/**
|
||||
* A tile that failed is not covered: a picture opening in it later
|
||||
* re-centres the cones there (and asks for the tile again).
|
||||
*/
|
||||
function dropFailed(covering, tiles, lists) {
|
||||
lists.forEach((list, i) => {
|
||||
if (list.status === 'rejected')
|
||||
covering.tiles.delete(tileKey(tiles[i]));
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The cones show the z15 tiles around one picture: once a picture of
|
||||
* the selected sequence opens outside them (‹ ›, a cone), re-centre.
|
||||
@@ -363,11 +433,23 @@ export function createPanoramaxProvider({
|
||||
}
|
||||
|
||||
const viewer = createPanoramaxViewer({
|
||||
getPhoto: async (photoId) => {
|
||||
const photo = await getPhoto(photoId);
|
||||
getPhoto,
|
||||
// The cones centre on the picture actually shown, never on a
|
||||
// lookup that came back after the user moved on.
|
||||
onShowing: (photo) => {
|
||||
if (photo.sequenceId) anchors.set(photo.sequenceId, photo);
|
||||
return photo;
|
||||
},
|
||||
// After a GPU reset the same picture is drawn again by the viewer
|
||||
// itself: through the core it would cancel a lookup the user
|
||||
// started. A failure still shows.
|
||||
onReopen: (photoId) =>
|
||||
viewer
|
||||
.open(photoId)
|
||||
.catch((error) =>
|
||||
context.actions.reportError(
|
||||
error?.message || 'The Panoramax photo could not be loaded',
|
||||
),
|
||||
),
|
||||
step,
|
||||
onStep: (photoId) =>
|
||||
context.actions.openImage(photoId, { frame: false }),
|
||||
@@ -449,9 +531,9 @@ export function createPanoramaxProvider({
|
||||
coverageStats() {
|
||||
const { kind } = state.coverage;
|
||||
const lines = kind === 'sequence';
|
||||
const drawn = coverage.sequenceCount();
|
||||
const shown = coverage.sequenceCount();
|
||||
// Density dots are not sequences: they are not counted as such.
|
||||
const count = lines ? drawn : 0;
|
||||
const count = lines ? shown : 0;
|
||||
const loading = state.coverage.pending.size > 0;
|
||||
const settled = !loading && !state.coverage.lastError;
|
||||
let fallback = '';
|
||||
@@ -461,7 +543,7 @@ export function createPanoramaxProvider({
|
||||
);
|
||||
if (kind === 'overview')
|
||||
fallback =
|
||||
drawn > 0 || !settled
|
||||
shown > 0 || !settled
|
||||
? OVERVIEW_HINT
|
||||
: filteredOut
|
||||
? FILTERED_HINT
|
||||
|
||||
@@ -66,6 +66,13 @@ export const COVERAGE_TUNING = Object.freeze({
|
||||
* within GPS noise), and 6–10% of the points GPS tracks carry.
|
||||
*/
|
||||
export const SIMPLIFY_TOLERANCE_UNITS = 4;
|
||||
|
||||
/**
|
||||
* Coverage tiles decode in slices of this many ms, the page getting a turn
|
||||
* between them (decode.js `decodeCoverageTileSliced`): under a frame.
|
||||
*/
|
||||
export const DECODE_SLICE_MS = 8;
|
||||
|
||||
/**
|
||||
* Grid cells per overview tile, at most: a z0 tile holds ~70,000 cells
|
||||
* (1.9 MB), drawn as this many binned points.
|
||||
@@ -89,6 +96,14 @@ export const PICTURE_TILE_ZOOM = 15;
|
||||
export const PICTURE_TILE_EDGE_M = 150;
|
||||
|
||||
export const TILE_TIMEOUT_MS = 20_000;
|
||||
/**
|
||||
* Once an answer has started, it may take as long as it keeps coming: a
|
||||
* 2 MB tile on a slow link outlasts any whole-request timeout, so only a
|
||||
* stream silent this long is given up.
|
||||
*/
|
||||
export const STREAM_IDLE_TIMEOUT_MS = 15_000;
|
||||
/** Busy (429/503) without a Retry-After the browser can read: back off from this. */
|
||||
export const BUSY_BACKOFF_BASE_SEC = 5;
|
||||
export const API_TIMEOUT_MS = 12_000;
|
||||
/** Real tiles are under 6 MB (z6 over France); refuse anything absurd. */
|
||||
export const MAX_RESPONSE_BYTES = 16 * 1024 * 1024;
|
||||
@@ -111,8 +126,12 @@ export const NEAREST_LIMIT = 10;
|
||||
* beside it has those).
|
||||
*/
|
||||
export const NEAREST_FOV_TOLERANCE_DEG = 180;
|
||||
/** Pictures in the bounding box around a click, at most. */
|
||||
export const NEAREST_BOX_LIMIT = 100;
|
||||
/**
|
||||
* Pictures in the bounding box around a click, at most. The API returns
|
||||
* them roughly nearest first (the first ten of 100 are the first ten of
|
||||
* 10), and the browser ranks them exactly: 20 is ~60 kB, 100 ~600 kB.
|
||||
*/
|
||||
export const NEAREST_BOX_LIMIT = 20;
|
||||
/**
|
||||
* A line clicked from high up may be tens of metres off the click: its
|
||||
* sequence is searched within the camera height / 100 (50–400 m).
|
||||
@@ -194,10 +213,12 @@ function linkedId(item, rel) {
|
||||
}
|
||||
}
|
||||
|
||||
/** An email address anywhere in a name (linear: names come from any instance). */
|
||||
/**
|
||||
* An email address anywhere in a name, at any of its '@'s. Linear: names
|
||||
* come from any instance, and each run between '@'s is scanned once.
|
||||
*/
|
||||
function looksLikeEmail(name) {
|
||||
const at = name.indexOf('@');
|
||||
return at > 0 && name.indexOf('.', at + 2) !== -1;
|
||||
return /[^\s@]@[^\s@]+\.[^\s@]/.test(name);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -217,12 +238,14 @@ function creatorOf(item) {
|
||||
// Every producer is credited (an uploading app and its photographer),
|
||||
// each name once, none that is an email address.
|
||||
const credited = [];
|
||||
const seen = new Set();
|
||||
for (const value of names) {
|
||||
if (typeof value !== 'string') continue;
|
||||
const name = value.trim().slice(0, 80);
|
||||
if (!name || looksLikeEmail(value)) continue;
|
||||
if (credited.some((other) => other.toLowerCase() === name.toLowerCase()))
|
||||
continue;
|
||||
const key = name.toLowerCase();
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
credited.push(name);
|
||||
}
|
||||
return credited.length ? credited.join(', ').slice(0, 120) : null;
|
||||
|
||||
@@ -224,6 +224,23 @@ test('hostile item data: a malformed link escape, null coordinates, a huge produ
|
||||
'Carto@Cité',
|
||||
'an @ without a domain is a name',
|
||||
);
|
||||
// The address need not follow the first '@'.
|
||||
assert.equal(named('@team [email protected]'), null);
|
||||
assert.equal(named('Carto@Cité ([email protected])'), null);
|
||||
assert.equal(named(`${'@'.repeat(100_000)}x`), `${'@'.repeat(80)}`);
|
||||
});
|
||||
|
||||
test('thousands of producers are credited in linear time', () => {
|
||||
const item = stacItem({ id: 'a', lon: 1, lat: 2 });
|
||||
item.providers = Array.from({ length: 50_000 }, (_, i) => ({
|
||||
name: `producer-${i}`,
|
||||
roles: ['producer'],
|
||||
}));
|
||||
const started = performance.now();
|
||||
const { creator } = normalizeItem(item);
|
||||
assert.ok(performance.now() - started < 1000);
|
||||
assert.ok(creator.startsWith('tester, producer-0, producer-1'));
|
||||
assert.ok(creator.length <= 120);
|
||||
});
|
||||
|
||||
test('the instance holding a picture is named (its via host), never as its author', () => {
|
||||
|
||||
@@ -726,12 +726,16 @@ test('from high up the density grid draws points, not counted as sequences, and
|
||||
test('switching off clears the cones and forgets a pending line click', async (t) => {
|
||||
const items = sequence('s', 3);
|
||||
const env = await mounted({ items, tiles: pictureTiles(items) });
|
||||
await withCones(t, env);
|
||||
const cones = await withCones(t, env);
|
||||
await env.instance.viewer.open('s-0');
|
||||
await env.instance.selectSequence('s');
|
||||
assert.ok(await until(() => cones.count() === 3), 'cones to clear');
|
||||
env.instance.handlePick(`${PICK_PREFIX.sequence}s`, {
|
||||
groundPoint: () => east(0),
|
||||
});
|
||||
env.instance.deactivate();
|
||||
assert.equal(env.instance.sequenceStats().selectedId, null);
|
||||
assert.equal(cones.count(), 0, 'the cones are gone');
|
||||
// The click's preference is gone: a later lookup asks any sequence.
|
||||
await env.instance.nearestImage(east(0));
|
||||
assert.equal(env.source.calls.nearest.at(-1).collections, null);
|
||||
@@ -1001,15 +1005,242 @@ test('a re-centre while the cones still load does not inherit the aborted load
|
||||
const second = new Set(
|
||||
pictureTilesAround({ lon, lat }).map((tile) => `${tile.x}/${tile.y}`),
|
||||
);
|
||||
const ids = new Set(cones.ids());
|
||||
const covered = items.filter((item) => {
|
||||
const tile = tileOf(item);
|
||||
return second.has(`${tile.x}/${tile.y}`);
|
||||
});
|
||||
assert.ok(covered.length > 0);
|
||||
assert.equal(
|
||||
covered.filter((item) => ids.has(item.id)).length,
|
||||
covered.length,
|
||||
'every picture in the re-centred tiles has its cone (the shared ones too)',
|
||||
// The neighbours join as they arrive: wait for them, then every picture
|
||||
// in the re-centred tiles has its cone (the shared ones too).
|
||||
const coneCount = () => {
|
||||
const ids = new Set(cones.ids());
|
||||
return covered.filter((item) => ids.has(item.id)).length;
|
||||
};
|
||||
assert.ok(
|
||||
await until(() => coneCount() === covered.length),
|
||||
`${coneCount()} of ${covered.length}`,
|
||||
);
|
||||
});
|
||||
|
||||
test('after a GPU reset the viewer draws the same picture itself, leaving the core’s lookups alone; a failure is reported', async () => {
|
||||
const items = sequence('p', 2, { pano: true });
|
||||
let failLoads = false;
|
||||
const view = { yaw: 0, pitch: 0, fov: 90 };
|
||||
const definition = createPanoramaxProvider({
|
||||
source: fakeSource({ items }),
|
||||
viewerOptions: {
|
||||
load: async (url) => {
|
||||
if (failLoads)
|
||||
throw new Error('The Panoramax photo could not be loaded');
|
||||
return { src: url, width: 2048, height: 1024 };
|
||||
},
|
||||
createRenderer: () => ({
|
||||
setImage() {},
|
||||
setView: (next) => Object.assign(view, next),
|
||||
getView: () => ({ ...view }),
|
||||
resize() {},
|
||||
clear() {},
|
||||
destroy() {},
|
||||
}),
|
||||
},
|
||||
});
|
||||
const { context, actions } = fakeContext();
|
||||
const instance = definition.create(context);
|
||||
actions.instance = instance;
|
||||
const host = fakeHost();
|
||||
await instance.viewer.mount(host);
|
||||
const poses = [];
|
||||
instance.viewer.onPose((pose) => poses.push(pose.imageId));
|
||||
await instance.viewer.open('p-0');
|
||||
const canvas = () => host.children[0].children[1];
|
||||
const kind = () => host.children[0].dataset.kind;
|
||||
assert.equal(kind(), 'pano');
|
||||
canvas().fire('webglcontextlost');
|
||||
assert.ok(await until(() => kind() === 'pano'), 'drawn again');
|
||||
assert.deepEqual(poses, ['p-0', 'p-0']);
|
||||
assert.deepEqual(actions.opened, [], 'not through the core');
|
||||
failLoads = true;
|
||||
canvas().fire('webglcontextlost');
|
||||
assert.ok(await until(() => actions.errors.some(Boolean)));
|
||||
assert.match(actions.errors.filter(Boolean)[0], /could not be loaded/);
|
||||
});
|
||||
|
||||
/** A source whose lookup of `heldId` waits for `release()` and ignores aborts. */
|
||||
function holdLookup(source, heldId) {
|
||||
const held = { release: null, signals: [] };
|
||||
const getItem = source.getItem;
|
||||
source.getItem = async (id, options = {}) => {
|
||||
held.signals.push({ id, signal: options.signal });
|
||||
if (id === heldId) await new Promise((go) => (held.release = go));
|
||||
return getItem(id);
|
||||
};
|
||||
return held;
|
||||
}
|
||||
|
||||
test('a lookup that comes back after the user moved on does not move the cones', async (t) => {
|
||||
const items = sequence('long', 400);
|
||||
const env = await mounted({ items, tiles: pictureTiles(items) });
|
||||
await withCones(t, env);
|
||||
const held = holdLookup(env.source, 'long-0');
|
||||
const late = env.instance.viewer.open('long-0');
|
||||
assert.ok(await until(() => held.release !== null));
|
||||
await env.instance.viewer.open('long-399');
|
||||
await late;
|
||||
// The first lookup answers anyway (a source that ignores the abort).
|
||||
held.release();
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
await env.instance.selectSequence('long');
|
||||
assert.ok(await until(() => !env.instance.sequenceStats().loading));
|
||||
const [lon, lat] = items[399].geometry.coordinates;
|
||||
const [far] = pictureTilesAround({ lon, lat });
|
||||
const [lon0, lat0] = items[0].geometry.coordinates;
|
||||
const [near] = pictureTilesAround({ lon: lon0, lat: lat0 });
|
||||
assert.equal(env.source.calls.tiles[0], `15/${far.x}/${far.y}`);
|
||||
assert.ok(
|
||||
!env.source.calls.tiles.includes(`15/${near.x}/${near.y}`),
|
||||
'nothing around the picture the user left',
|
||||
);
|
||||
});
|
||||
|
||||
test('a lookup nobody waits for any more is stopped; one still wanted is not', async () => {
|
||||
const items = sequence('s', 3);
|
||||
const env = await mounted({ items });
|
||||
const held = holdLookup(env.source, 's-0');
|
||||
const first = env.instance.viewer.open('s-0');
|
||||
assert.ok(await until(() => held.release !== null));
|
||||
const [{ signal }] = held.signals;
|
||||
assert.equal(signal.aborted, false);
|
||||
await env.instance.viewer.open('s-1');
|
||||
await first;
|
||||
await Promise.resolve();
|
||||
assert.equal(signal.aborted, true, 'the user moved on: its search stops');
|
||||
// The same picture asked again at once shares the lookup, which goes on.
|
||||
held.release = null;
|
||||
held.signals.length = 0;
|
||||
const again = env.instance.viewer.open('s-2');
|
||||
const twice = env.instance.viewer.open('s-2');
|
||||
await Promise.all([again, twice]);
|
||||
assert.equal(held.signals.length, 1);
|
||||
assert.equal(held.signals[0].signal.aborted, false);
|
||||
assert.equal(env.poses.at(-1), 's-2');
|
||||
});
|
||||
|
||||
/**
|
||||
* Two pictures of sequence `e` either side of a z15 tile's west edge,
|
||||
* half-way down it: the cones around the first read its tile and the
|
||||
* west neighbour.
|
||||
*/
|
||||
function acrossWestEdge() {
|
||||
const home = tileCentre(ORIGIN.lon, ORIGIN.lat, 15);
|
||||
const box = tileBounds(home.tile.x, home.tile.y, 15);
|
||||
const lat = (box.north + box.south) / 2;
|
||||
const items = [
|
||||
stacItem({
|
||||
id: 'e-0',
|
||||
sequenceId: 'e',
|
||||
lon: box.west + 0.00005,
|
||||
lat,
|
||||
nextId: 'e-1',
|
||||
}),
|
||||
stacItem({
|
||||
id: 'e-1',
|
||||
sequenceId: 'e',
|
||||
lon: box.west - 0.00005,
|
||||
lat,
|
||||
prevId: 'e-0',
|
||||
}),
|
||||
];
|
||||
const west = `15/${home.tile.x - 1}/${home.tile.y}`;
|
||||
return { items, west };
|
||||
}
|
||||
|
||||
test(
|
||||
'the open picture’s own tile draws its cones at once; a neighbour’s join as it arrives',
|
||||
{ timeout: 10_000 },
|
||||
async (t) => {
|
||||
const { items, west } = acrossWestEdge();
|
||||
const tiles = pictureTiles(items);
|
||||
let release = null;
|
||||
const env = await mounted({
|
||||
items,
|
||||
tiles: async (tile) => {
|
||||
if (`${tile.z}/${tile.x}/${tile.y}` === west)
|
||||
await new Promise((go) => (release = go));
|
||||
return tiles(tile);
|
||||
},
|
||||
});
|
||||
const cones = await withCones(t, env);
|
||||
await env.instance.viewer.open('e-0');
|
||||
const [lon, lat] = items[0].geometry.coordinates;
|
||||
assert.equal(
|
||||
pictureTilesAround({ lon, lat }).length,
|
||||
2,
|
||||
'the west neighbour is read too',
|
||||
);
|
||||
// Not awaited: a load that waited for the neighbour would hang here.
|
||||
env.instance.selectSequence('e');
|
||||
assert.ok(await until(() => release !== null), 'the neighbour is held');
|
||||
assert.ok(
|
||||
await until(() => cones.count() === 1),
|
||||
'the home tile’s cone shows',
|
||||
);
|
||||
assert.deepEqual(cones.ids(), ['e-0']);
|
||||
release();
|
||||
assert.ok(await until(() => cones.count() === 2), 'the neighbour’s joins');
|
||||
},
|
||||
);
|
||||
|
||||
test('a neighbour tile that failed is not counted as shown: › into it re-centres the cones there', async (t) => {
|
||||
const { items, west } = acrossWestEdge();
|
||||
const tiles = pictureTiles(items);
|
||||
let failing = true;
|
||||
const env = await mounted({
|
||||
items,
|
||||
tiles: (tile) => {
|
||||
if (failing && `${tile.z}/${tile.x}/${tile.y}` === west)
|
||||
throw Object.assign(new Error('Panoramax is busy'), { status: 503 });
|
||||
return tiles(tile);
|
||||
},
|
||||
});
|
||||
const cones = await withCones(t, env);
|
||||
await env.instance.viewer.open('e-0');
|
||||
await env.instance.selectSequence('e');
|
||||
assert.ok(await until(() => cones.count() === 1));
|
||||
// Let the neighbour's failure land.
|
||||
assert.ok(await until(() => env.source.calls.tiles.includes(west)));
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
failing = false;
|
||||
const asked = env.source.calls.tiles.filter((key) => key === west).length;
|
||||
env.next.fire('click');
|
||||
assert.ok(await until(() => env.poses.at(-1) === 'e-1'));
|
||||
assert.ok(
|
||||
await until(
|
||||
() => env.source.calls.tiles.filter((key) => key === west).length > asked,
|
||||
),
|
||||
'the failed tile is asked again',
|
||||
);
|
||||
assert.ok(await until(() => cones.count() === 2), 'e-1 has its cone');
|
||||
});
|
||||
|
||||
test('a busy or timed-out tile keeps its own message; only a body that is not a tile reads as unreadable', async (t) => {
|
||||
const items = sequence('s', 3);
|
||||
const failures = [
|
||||
Object.assign(new Error('Panoramax is busy'), { status: 429 }),
|
||||
Object.assign(new Error('Panoramax did not answer in time — try again'), {
|
||||
status: 0,
|
||||
}),
|
||||
];
|
||||
for (const failure of failures) {
|
||||
const env = await mounted({
|
||||
items,
|
||||
tiles: () => {
|
||||
throw failure;
|
||||
},
|
||||
});
|
||||
await withCones(t, env);
|
||||
await env.instance.viewer.open('s-0');
|
||||
await env.instance.selectSequence('s');
|
||||
assert.ok(await until(() => env.actions.errors.some(Boolean)));
|
||||
assert.equal(env.actions.errors.filter(Boolean).at(-1), failure.message);
|
||||
}
|
||||
});
|
||||
@@ -1,10 +1,8 @@
|
||||
import { createLimiter } from '../../limiter.js';
|
||||
|
||||
/** Longest wait a Retry-After is honoured for. */
|
||||
const RETRY_AFTER_MAX_SEC = 300;
|
||||
import {
|
||||
API_CONCURRENCY,
|
||||
API_TIMEOUT_MS,
|
||||
BUSY_BACKOFF_BASE_SEC,
|
||||
MAX_RESPONSE_BYTES,
|
||||
NEAREST_BOX_LIMIT,
|
||||
NEAREST_FOV_TOLERANCE_DEG,
|
||||
@@ -12,10 +10,14 @@ import {
|
||||
NEAREST_RADIUS_M,
|
||||
PANORAMAX_API,
|
||||
PICTURE_TILE_CONCURRENCY,
|
||||
STREAM_IDLE_TIMEOUT_MS,
|
||||
TILE_CONCURRENCY,
|
||||
TILE_TIMEOUT_MS,
|
||||
} from './policy.js';
|
||||
|
||||
/** Longest wait a Retry-After (or the backoff) asks for. */
|
||||
const RETRY_AFTER_MAX_SEC = 300;
|
||||
|
||||
/** An HTTP or API failure; `retryAfterSec` is set while Panoramax is busy. */
|
||||
export class PanoramaxSourceError extends Error {
|
||||
constructor(message, { status = 0, retryAfterSec = null } = {}) {
|
||||
@@ -29,17 +31,30 @@ export class PanoramaxSourceError extends Error {
|
||||
const NO_ANSWER = 'Panoramax did not answer in time — try again';
|
||||
const UNREACHABLE = 'Panoramax could not be reached — check the connection';
|
||||
|
||||
/** Seconds from a Retry-After header, or null. */
|
||||
function retryAfterSeconds(response) {
|
||||
/**
|
||||
* Seconds from a Retry-After header (delay seconds or an HTTP date), or
|
||||
* null. The meta-catalogue exposes no headers to the browser, so in the
|
||||
* app this is usually null and the caller backs off on its own.
|
||||
*/
|
||||
export function retryAfterSeconds(response, now = Date.now()) {
|
||||
const header = response.headers?.get?.('retry-after');
|
||||
const value = header == null || header === '' ? NaN : Number(header);
|
||||
if (!Number.isFinite(value) || value < 0) return null;
|
||||
if (header == null || header === '') return null;
|
||||
let value = Number(header);
|
||||
if (!Number.isFinite(value)) {
|
||||
const at = Date.parse(header);
|
||||
if (!Number.isFinite(at)) return null;
|
||||
value = Math.ceil((at - now) / 1000);
|
||||
}
|
||||
if (value < 0) return null;
|
||||
// A maintenance page's hour must not freeze coverage for an hour.
|
||||
return Math.min(RETRY_AFTER_MAX_SEC, Math.max(1, value));
|
||||
}
|
||||
|
||||
/** The body as bytes, refusing more than MAX_RESPONSE_BYTES whatever the headers say. */
|
||||
async function readBounded(response) {
|
||||
/**
|
||||
* The body as bytes, refusing more than MAX_RESPONSE_BYTES whatever the
|
||||
* headers say. `onChunk` hears each piece arrive (the idle timeout).
|
||||
*/
|
||||
async function readBounded(response, onChunk = () => {}) {
|
||||
const tooMuch = () =>
|
||||
new PanoramaxSourceError('Panoramax sent too much data', {
|
||||
status: response.status,
|
||||
@@ -57,6 +72,7 @@ async function readBounded(response) {
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
onChunk();
|
||||
total += value.byteLength;
|
||||
if (total > MAX_RESPONSE_BYTES) {
|
||||
reader.cancel().catch(() => {});
|
||||
@@ -92,30 +108,55 @@ export function createPanoramaxSource({
|
||||
lifo: true,
|
||||
});
|
||||
|
||||
/** GET within `timeoutMs`; the caller's abort wins over the timeout. */
|
||||
/** Busy answers in a row with no readable Retry-After (the backoff grows). */
|
||||
let busyStreak = 0;
|
||||
|
||||
/**
|
||||
* GET: its headers within `timeoutMs`, then a body that may take as long
|
||||
* as it keeps arriving (silent for STREAM_IDLE_TIMEOUT_MS is given up).
|
||||
* The caller's abort wins over both.
|
||||
*/
|
||||
async function get(url, { signal, timeoutMs, empty = [] }) {
|
||||
const controller = new AbortController();
|
||||
const onAbort = () => controller.abort(signal.reason);
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
signal?.addEventListener('abort', onAbort, { once: true });
|
||||
const timer = setTimeout(
|
||||
() => controller.abort(new PanoramaxSourceError(NO_ANSWER)),
|
||||
timeoutMs,
|
||||
);
|
||||
const giveUp = () => controller.abort(new PanoramaxSourceError(NO_ANSWER));
|
||||
let timer = setTimeout(giveUp, timeoutMs);
|
||||
try {
|
||||
const response = await fetchImpl(url, { signal: controller.signal });
|
||||
if (empty.includes(response.status)) return null;
|
||||
if (response.status === 429 || response.status === 503)
|
||||
if (empty.includes(response.status)) {
|
||||
busyStreak = 0;
|
||||
return null;
|
||||
}
|
||||
if (response.status === 429 || response.status === 503) {
|
||||
const told = retryAfterSeconds(response);
|
||||
// Not told how long: back off, longer each time (5, 10, 20… 300 s).
|
||||
const wait =
|
||||
told ??
|
||||
Math.min(
|
||||
RETRY_AFTER_MAX_SEC,
|
||||
BUSY_BACKOFF_BASE_SEC * 2 ** Math.min(busyStreak, 8),
|
||||
);
|
||||
busyStreak++;
|
||||
throw new PanoramaxSourceError('Panoramax is busy', {
|
||||
status: response.status,
|
||||
retryAfterSec: retryAfterSeconds(response) ?? 5,
|
||||
retryAfterSec: wait,
|
||||
});
|
||||
}
|
||||
if (!response.ok)
|
||||
throw new PanoramaxSourceError(
|
||||
`Panoramax answered HTTP ${response.status}`,
|
||||
{ status: response.status },
|
||||
);
|
||||
return await readBounded(response);
|
||||
busyStreak = 0;
|
||||
// The answer has begun: from here only silence is timed.
|
||||
clearTimeout(timer);
|
||||
timer = setTimeout(giveUp, STREAM_IDLE_TIMEOUT_MS);
|
||||
return await readBounded(response, () => {
|
||||
clearTimeout(timer);
|
||||
timer = setTimeout(giveUp, STREAM_IDLE_TIMEOUT_MS);
|
||||
});
|
||||
} catch (error) {
|
||||
if (signal?.aborted) throw signal.reason;
|
||||
if (controller.signal.aborted) throw controller.signal.reason;
|
||||
@@ -180,8 +221,8 @@ export function createPanoramaxSource({
|
||||
* Pictures within `radius` metres of a point, unranked: the caller ranks
|
||||
* them by distance. Two searches side by side, since neither finds them
|
||||
* all: `place_position` (nearest first) leaves out pictures looking
|
||||
* away from the point, whatever the tolerance; a bounding box has them
|
||||
* but in no order, so in a dense spot its page may miss the nearest.
|
||||
* away from the point, whatever the tolerance; a bounding box has them,
|
||||
* roughly nearest first (not exactly: the caller ranks the union).
|
||||
* `collections` keeps one sequence's, `pano` and `sinceMs` filter as
|
||||
* the imagery filter does.
|
||||
*/
|
||||
|
||||
@@ -1,7 +1,16 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { PanoramaxSourceError, createPanoramaxSource } from './source.js';
|
||||
import { MAX_RESPONSE_BYTES } from './policy.js';
|
||||
import {
|
||||
PanoramaxSourceError,
|
||||
createPanoramaxSource,
|
||||
retryAfterSeconds,
|
||||
} from './source.js';
|
||||
import {
|
||||
MAX_RESPONSE_BYTES,
|
||||
NEAREST_BOX_LIMIT,
|
||||
STREAM_IDLE_TIMEOUT_MS,
|
||||
TILE_TIMEOUT_MS,
|
||||
} from './policy.js';
|
||||
|
||||
/** A fetch answering from `respond(url)`, recording every URL asked. */
|
||||
function fakeFetch(respond) {
|
||||
@@ -348,6 +357,148 @@ test('a Retry-After is honoured up to five minutes', async () => {
|
||||
assert.equal(await wait('12'), 12);
|
||||
});
|
||||
|
||||
test('a Retry-After may be an HTTP date', () => {
|
||||
const now = Date.parse('2026-10-15T12:00:00Z');
|
||||
const response = (value) =>
|
||||
new Response(null, { status: 429, headers: { 'retry-after': value } });
|
||||
assert.equal(
|
||||
retryAfterSeconds(response('Thu, 15 Oct 2026 12:01:00 GMT'), now),
|
||||
60,
|
||||
);
|
||||
assert.equal(
|
||||
retryAfterSeconds(response('Thu, 15 Oct 2026 14:00:00 GMT'), now),
|
||||
300,
|
||||
);
|
||||
assert.equal(
|
||||
retryAfterSeconds(response('Thu, 15 Oct 2026 11:00:00 GMT'), now),
|
||||
null,
|
||||
'a date already past',
|
||||
);
|
||||
assert.equal(retryAfterSeconds(response('soon'), now), null);
|
||||
assert.equal(
|
||||
retryAfterSeconds(new Response(null, { status: 429 }), now),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
test('busy without a readable Retry-After (the browser hides it): the wait doubles, and resets once answered', async () => {
|
||||
let busy = true;
|
||||
const source = createPanoramaxSource({
|
||||
fetchImpl: fakeFetch(() =>
|
||||
busy ? { status: 429 } : { body: new Uint8Array([1]) },
|
||||
).fetchImpl,
|
||||
});
|
||||
const wait = () =>
|
||||
source.getTile({ x: 0, y: 0, z: 0 }).then(
|
||||
() => 'ok',
|
||||
(error) => error.retryAfterSec,
|
||||
);
|
||||
const waits = [];
|
||||
for (let i = 0; i < 9; i++) waits.push(await wait());
|
||||
assert.deepEqual(waits, [5, 10, 20, 40, 80, 160, 300, 300, 300]);
|
||||
busy = false;
|
||||
assert.equal(await wait(), 'ok');
|
||||
busy = true;
|
||||
assert.equal(await wait(), 5, 'back to the first wait');
|
||||
});
|
||||
|
||||
test(
|
||||
'a body that keeps arriving is not cut off by the request timeout; one that stalls is',
|
||||
{ timeout: 10_000 },
|
||||
async (t) => {
|
||||
t.mock.timers.enable({ apis: ['setTimeout'] });
|
||||
const settle = async () => {
|
||||
for (let i = 0; i < 20; i++) await Promise.resolve();
|
||||
};
|
||||
// Headers at once, then a chunk every 10 s for twice the tile timeout.
|
||||
const chunks = Math.ceil((2 * TILE_TIMEOUT_MS) / 10_000);
|
||||
let push;
|
||||
const slowBody = (signal) =>
|
||||
new Response(
|
||||
new ReadableStream({
|
||||
start(controller) {
|
||||
push = (last) => {
|
||||
controller.enqueue(new Uint8Array([7]));
|
||||
if (last) controller.close();
|
||||
};
|
||||
signal.addEventListener('abort', () =>
|
||||
controller.error(signal.reason),
|
||||
);
|
||||
},
|
||||
}),
|
||||
{ status: 200 },
|
||||
);
|
||||
const source = createPanoramaxSource({
|
||||
fetchImpl: async (url, { signal }) => slowBody(signal),
|
||||
});
|
||||
const slow = source.getTile({ x: 0, y: 0, z: 0 });
|
||||
await settle();
|
||||
for (let i = 1; i <= chunks; i++) {
|
||||
t.mock.timers.tick(10_000);
|
||||
push(i === chunks);
|
||||
await settle();
|
||||
}
|
||||
assert.equal((await slow).length, chunks, 'every chunk read');
|
||||
const stalled = source.getTile({ x: 0, y: 0, z: 0 });
|
||||
await settle();
|
||||
push(false);
|
||||
await settle();
|
||||
t.mock.timers.tick(STREAM_IDLE_TIMEOUT_MS);
|
||||
await assert.rejects(stalled, /did not answer in time/);
|
||||
},
|
||||
);
|
||||
|
||||
test('the box search asks for a few pictures: the API returns them nearest first', async () => {
|
||||
const { urls, fetchImpl } = fakeFetch(() => json({ features: [] }));
|
||||
const source = createPanoramaxSource({ fetchImpl });
|
||||
await source.nearestItems({ lon: 2.35, lat: 48.85 });
|
||||
const box = urls
|
||||
.map((url) => new URL(url))
|
||||
.find((url) => url.searchParams.has('bbox'));
|
||||
assert.ok(box, 'a box search');
|
||||
assert.equal(box.searchParams.get('limit'), String(NEAREST_BOX_LIMIT));
|
||||
assert.ok(NEAREST_BOX_LIMIT <= 20);
|
||||
});
|
||||
|
||||
test('the cones’ tiles have their own connection: never behind coverage tiles, the newest first', async () => {
|
||||
const { PICTURE_TILE_CONCURRENCY, TILE_CONCURRENCY } =
|
||||
await import('./policy.js');
|
||||
assert.equal(PICTURE_TILE_CONCURRENCY, 1);
|
||||
const held = new Map();
|
||||
const asked = [];
|
||||
const source = createPanoramaxSource({
|
||||
fetchImpl: (url) => {
|
||||
asked.push(url.replace(/^.*\/map\//, ''));
|
||||
return new Promise((resolve) =>
|
||||
held.set(url.replace(/^.*\/map\//, ''), () =>
|
||||
resolve(new Response(null, { status: 204 })),
|
||||
),
|
||||
);
|
||||
},
|
||||
});
|
||||
const settle = () => new Promise((resolve) => setTimeout(resolve, 0));
|
||||
// Coverage fills its lane and queues.
|
||||
const coverage = Array.from({ length: TILE_CONCURRENCY + 2 }, (_, i) =>
|
||||
source.getTile({ x: i, y: 0, z: 10 }),
|
||||
);
|
||||
await settle();
|
||||
assert.equal(asked.length, TILE_CONCURRENCY);
|
||||
// A cone tile goes at once; two more wait, the newer first.
|
||||
const pictures = [1, 2, 3].map((x) =>
|
||||
source.getTile({ x, y: 9, z: 15 }, { lane: 'pictures' }),
|
||||
);
|
||||
await settle();
|
||||
assert.deepEqual(asked.slice(TILE_CONCURRENCY), ['15/1/9.mvt']);
|
||||
held.get('15/1/9.mvt')();
|
||||
await settle();
|
||||
assert.deepEqual(asked.slice(TILE_CONCURRENCY), ['15/1/9.mvt', '15/3/9.mvt']);
|
||||
for (let round = 0; round < 6; round++) {
|
||||
for (const go of held.values()) go();
|
||||
await settle();
|
||||
}
|
||||
await Promise.all([...coverage, ...pictures]);
|
||||
});
|
||||
|
||||
test('a lookup near the antimeridian keeps its box on the map', async () => {
|
||||
const { urls, fetchImpl } = fakeFetch(() => json({ features: [] }));
|
||||
const source = createPanoramaxSource({ fetchImpl });
|
||||
|
||||
@@ -31,7 +31,13 @@ export const PANORAMAX_SOURCES = Object.freeze({
|
||||
Boolean(photo.isPano && (photo.urls.sd || hdDecodable(photo))),
|
||||
// Without an SD derivative the original goes straight to the texture size.
|
||||
panoFirst: (photo) => {
|
||||
if (photo.urls.sd) return { url: photo.urls.sd, width: SD_WIDTH };
|
||||
// Decoded at SD's own size, whatever the host actually serves.
|
||||
if (photo.urls.sd)
|
||||
return {
|
||||
url: photo.urls.sd,
|
||||
width: SD_WIDTH,
|
||||
resize: [SD_WIDTH, SD_WIDTH / 2],
|
||||
};
|
||||
if (!hdDecodable(photo)) return null;
|
||||
const width = Math.min(photo.width, PANO_FULL_SIZE[0]);
|
||||
return { url: photo.urls.hd, width, resize: [width, width / 2] };
|
||||
@@ -51,15 +57,20 @@ export const PANORAMAX_SOURCES = Object.freeze({
|
||||
timeoutMs: HD_TIMEOUT_MS,
|
||||
};
|
||||
},
|
||||
// A 360° picture shown flat (no WebGL) is its whole equirectangular image;
|
||||
// an original too big to decode is skipped for the thumbnail.
|
||||
flatFirst: (photo) => [
|
||||
// A 360° picture shown flat (no WebGL) is its whole equirectangular image.
|
||||
// When SD fails the thumbnail stands in; the original (megabytes) only
|
||||
// for the expanded viewer, and only one of a size it may decode.
|
||||
flatFirst: (photo, { expanded = false } = {}) => [
|
||||
photo.urls.sd,
|
||||
!photo.isPano || hdDecodable(photo) ? photo.urls.hd : null,
|
||||
expanded && hdDecodable(photo) ? photo.urls.hd : null,
|
||||
photo.urls.thumb,
|
||||
],
|
||||
// The original once the viewer is wider than SD, if its width is known
|
||||
// and decodable (a 200 MP phone photo is 16,000 px across).
|
||||
flatSharp: (photo, pixels) =>
|
||||
photo.urls.sd && photo.urls.hd && pixels > SD_WIDTH ? photo.urls.hd : null,
|
||||
photo.urls.sd && hdDecodable(photo) && pixels > SD_WIDTH
|
||||
? photo.urls.hd
|
||||
: null,
|
||||
describe: (photo) => ({
|
||||
// Where no author can be named, the instance holding the photo is.
|
||||
title: !photo.creator && photo.via ? `via ${photo.via}` : null,
|
||||
|
||||
@@ -140,12 +140,14 @@ test('a 360° picture shows its SD equirectangular image, then HD shrunk to 4,09
|
||||
assert.equal(canvas.hidden, false);
|
||||
assert.equal(picture.hidden, true);
|
||||
assert.equal(viewer.supportsRenderModes(), false, 'no frame to fit');
|
||||
assert.deepEqual(loads[0], {
|
||||
const { signal, ...first } = loads[0];
|
||||
assert.deepEqual(first, {
|
||||
url: PHOTOS.pano.urls.sd,
|
||||
crossOrigin: true,
|
||||
resize: null,
|
||||
resize: [2048, 1024],
|
||||
failure: 'The Panoramax photo could not be loaded',
|
||||
});
|
||||
assert.equal(signal.aborted, false, 'the open can stop it');
|
||||
assert.equal(poses.at(-1).isPano, true);
|
||||
assert.equal(poses.at(-1).bearing, 100);
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
@@ -258,7 +260,7 @@ test('without WebGL a 360° picture shows its whole equirectangular image flat',
|
||||
assert.equal(picture.src, PHOTOS.pano.urls.sd);
|
||||
});
|
||||
|
||||
test('a picture whose SD fails falls back to HD, then the thumbnail', async () => {
|
||||
test('a picture whose SD fails falls back to the thumbnail; the original only for the expanded viewer', async () => {
|
||||
const tried = [];
|
||||
const viewer = createPanoramaxViewer({
|
||||
getPhoto: async () => PHOTOS.flat,
|
||||
@@ -273,12 +275,18 @@ test('a picture whose SD fails falls back to HD, then the thumbnail', async () =
|
||||
const host = fakeHost();
|
||||
await viewer.mount(host);
|
||||
await viewer.open('flat');
|
||||
assert.deepEqual(tried, [PHOTOS.flat.urls.sd, PHOTOS.flat.urls.thumb]);
|
||||
assert.equal(parts(host).picture.src, PHOTOS.flat.urls.thumb);
|
||||
// Expanded, the original is worth its megabytes.
|
||||
tried.length = 0;
|
||||
parts(host).root.closest = (selector) =>
|
||||
selector.includes('sl-viewer-wrap-expanded') ? {} : null;
|
||||
await viewer.open('flat');
|
||||
assert.deepEqual(tried, [
|
||||
PHOTOS.flat.urls.sd,
|
||||
PHOTOS.flat.urls.hd,
|
||||
PHOTOS.flat.urls.thumb,
|
||||
]);
|
||||
assert.equal(parts(host).picture.src, PHOTOS.flat.urls.thumb);
|
||||
});
|
||||
|
||||
test('a photo with no author to name is captioned with the instance holding it', () => {
|
||||
|
||||
Reference in new issue
Block a user