diff --git a/SECURITY.md b/SECURITY.md index 6f3dc4ce6..21affbcab 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -132,7 +132,7 @@ God's Eye View's tools are also served to MCP clients: over stdio (`npm run mcp` - **What the panel's requests can reach.** In clients that display MCP Apps, the panel loads the app through the `panel_request` tool, which requests paths on the app's server. It is marked for the panel only and requires a key that each MCP server puts in its panel page, which keeps it from clients that list tools to the model without loading the panel. That key is not access control: any MCP client can read the panel page, key included, and then reach the app's files and data routes through `panel_request`, much as its other tools reach the data. Provider Settings, credential and model endpoints, `/mcp` and the development server's internal routes are refused to every caller. - **The panel carries the two browser keys.** The panel build (`npm run build:panel`) includes the same Google Maps key and Cesium ion token as the app, and they run inside the client's panel page, on the client's site. A key restricted by HTTP referrer must also allow that page, or the panel cannot load tiles. To keep the app's own keys narrower, build the panel with separately restricted keys in the environment of `npm run build:panel`. - **Embed mode is not framable by default.** `?embed=1` documents keep `X-Frame-Options: DENY` and `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names the pages that may frame them; a framing page can change what the app shows. Other documents can never be framed. -- **A plain link never loads a billed Street View panorama without a click.** A share link with `sl=google,,` flies there, switches to Google 3D and lights Street View, then stops: the panel is armed and the photo loads only when the visitor presses OPEN GOOGLE STREET VIEW (one lookup and one panorama load on the browser Maps key, no retries, no geocoding or server call). A view posted into the embedded app (the God's Eye View panel, or a page `GEV_EMBED_FRAME_ANCESTORS` allows), and the panel page's own opening link for a tool result issued in the last five minutes, are that client's own request and open the photo; a client that mounts the panel again later (reopening a conversation) hands it the old result, which only arms, and so does any later result on the same mount issued more than five minutes ago; adding `?embed=1` to a published link does not lift the gate (in a frame the link only arms; the panel that holds OPEN is hidden there, so a posted view is the way to open a photo). The OPEN press uses the link's point only while the ground under the screen centre is within 1 km of it; with the sky in the middle it opens nothing. The armed lookup dies when its source is switched off, when its map leaves, or when any other photo opens or closes. A link pasted into an open tab follows the same rule, and a link's source switch and layer are never saved as the recipient's own preferences. A link that follows an entity ignores `sl`. Mapillary, KartaView and Panoramax links open their photo at once; none of them bills per image. +- **A plain link never loads a billed Street View panorama without a click.** A share link with `sl=google,,` flies there, switches to Google 3D and lights Street View, then stops: the panel is armed and the photo loads only when the visitor presses OPEN GOOGLE STREET VIEW (one lookup and one panorama load on the browser Maps key, no retries, no geocoding or server call). A view posted into the embedded app (the God's Eye View panel, or a page `GEV_EMBED_FRAME_ANCESTORS` allows), and the panel page's own opening link for a tool result issued in the last five minutes, are that client's own request and open the photo; freshness is judged only by the result's `issued_at`, so a result older than five minutes only arms, whether the panel was mounted again (reopening a conversation) or got it later on the same mount, while one re-sent or remounted within five minutes of issue opens the photo again; adding `?embed=1` to a published link does not lift the gate (in a frame the link only arms; the panel that holds OPEN is hidden there, so a posted view is the way to open a photo). The OPEN press uses the link's point only while the ground under the screen centre is within 1 km of it; with the sky in the middle it opens nothing. The armed lookup dies when its source is switched off, when its map leaves, or when any other photo opens or closes. A link pasted into an open tab follows the same rule, and a link's source switch and layer are never saved as the recipient's own preferences. A link that follows an entity ignores `sl`. Mapillary, KartaView and Panoramax links open their photo at once; none of them bills per image. ## Scope & expectations diff --git a/changelog.d/1113-show-street-level.md b/changelog.d/1113-show-street-level.md index ca56834ec..89f47726e 100644 --- a/changelog.d/1113-show-street-level.md +++ b/changelog.d/1113-show-street-level.md @@ -1,40 +1,15 @@ - "Show me X on Street View" (or on Mapillary, KartaView or Panoramax): a `show_street_level` voice and typed action flies to the place, switches to Google 3D when the source needs it, lights that source in Street Level, - turns the layer on and opens the nearest photo, reporting which step failed. - Views and share links carry the same request (`street_level`, `sl=`), so - `show_in_gods_eye_view` and a link can open a panorama at a place. Street - View lookups from a geocoded place search up to 250 m; clicks keep 50 m. A - plain link with `sl=google` arms Street View and leaves the billed load to a - press of OPEN GOOGLE STREET VIEW, which aims at the view centre again once - the visitor has moved more than 1 km away; embedded views open it, and the - God's Eye View panel only for a result issued in the last five minutes. On - a phone, a photo that opens or a link that arms opens the Cameras sheet; - in cockpit, clean view, recording or scene playback the action says the - panel is hidden instead of loading a photo nobody sees. Embedded photos - remain available over the clean globe. Mapillary's - nearest-photo search asks for 100 photos, so the nearest and the filter's - pick come from the spot's photos rather than an arbitrary eight. The source the - action lights stays lit like a chip click. An unavailable map stack now - reports the controller's own reason instead of always asking for a Cesium - ion token. When nothing is found and a 360°, flat or SINCE filter the source - honours is on, the answer names the filter. Cancelled lookups clear their - loading state over an existing photo and release their caller abort listeners. - Embedded photos show loading errors inside the overlay, and the OPEN prompt - uses the recipient's selected language. A link's photos never save its - layers as the recipient's preferences, and a link pasted into an open tab - opens or arms them too. With no place named, the action looks under the - screen centre as OPEN does; an armed OPEN aimed at the sky opens nothing; a - failing source is reported as an error, not as missing imagery; and - Mapillary's nearest search gives up after 12 seconds. On a phone the panel - scrolls into view, and in embed mode EXPAND stays open while cockpit, - recording and scene playback hide the photo. The action checks again after - every wait: a navigation or a drag of the globe, or a mode that hides the - panel, stops it before a photo loads for a place the user left or in a - panel nobody sees, also between finding a photo and loading it. A request - that finds nothing closes the previous photo, and one that finds a photo - replaces it in place, so EXPAND stays. A link whose restore flight the - visitor interrupted, or a posted view the user overtook, leaves their - camera and photo alone, and a photo found but not loaded keeps its error. - Mapillary's timeout message is translated. The panel's later results open a billed photo only when they are - fresh too (Said Turksever, @asturksever, #1113). + turns the layer on and opens the nearest photo, saying which step failed. + With no place named it looks under the screen centre, as OPEN does. Views + and share links carry the same request (`street_level`, `sl=`). A plain + `sl=google` link never loads a billed panorama by itself: it arms Street + View and leaves the load to OPEN GOOGLE STREET VIEW; embedded views open it, + and so does the God's Eye View panel for a result issued in the last five + minutes. A request stops when the user navigates or hides the panel before + its photo loads; a new photo replaces the open one in place, so EXPAND + stays. When nothing is found and a 360°, flat or SINCE filter is on, the + answer names the filter. An unavailable map stack now reports the + controller's own reason instead of always asking for a Cesium ion token + (Said Turksever, @asturksever, #1113). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 6983e1a88..32494ab44 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1094,7 +1094,7 @@ Voice controls bind to a protocol-independent session factory. The default WebRT The standalone app offers OpenAI and Gemini Live through the **AI Provider** setting, shared by voice and typed chat. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter. -Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 34 tools using separate description-only metadata; a session is offered 33 of them unless `OVERPASS_UPSTREAMS` is set, which adds `osm_query` and its one prompt sentence (`server/providers/openai/tools.js`). Description customization cannot replace argument types, enum values or required fields. +Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 35 tools using separate description-only metadata; a session is offered 34 of them unless `OVERPASS_UPSTREAMS` is set, which adds `osm_query` and its one prompt sentence (`server/providers/openai/tools.js`). Description customization cannot replace argument types, enum values or required fields. Voice areas live in a store owned by the action runner for the page (`src/data/areaStore.js`); the model gets an `areaId`, a rounded bounding box and a summary, never geometry. `resolve_area` (`src/voice/areaActions.js`, `src/data/areaResolver.js`) checks the store, bundled Natural Earth regions, then bundled countries and states by name — a name several prominent units share returns up to four candidates unless the view is inside exactly one — and then the annotation outline ladder (`resolveAnnotationTarget`: bundled counties, Census places and neighborhoods, the guarded `/api/geocode/outline`, and Overpass only when configured). Administrative geometry is stored only when its source identity agrees with the requested unit, by name or by a form the bundled matcher reads as that unit at its level ("State of Texas", "Province of Bagmati", "USA", "UK"; not "Great Britain" or "Korea"); stale, mismatched, synthesized or otherwise unverified administrative boundaries are refused rather than relabelled. Clarification IDs use the bounded `ne::` grammar and are accepted only from the current resolver's candidate memory. A lookup past 8 s answers `AREA_TIMEOUT` and keeps filling the caches. `around=true` asks the same ladder for the grounds enclosing a landmark (Nominatim, then the OpenFreeMap area holding it) and otherwise stores a 250 m buffer marked approximate. `analyst_query` scopes add `area`, `drawn` (newest draw-tool outline) and `annotation` (newest or named outline); region scopes use the same resolver within 3 s. Drawn outlines are a simplified copy (≤1,200 vertices); counting reads the stored geometry (`src/data/areaGeometry.js`). `find_imagery` (`src/voice/imageryActions.js`) sets the Recent Imagery box from an area, the view or the turn's pointer snapshot and pins the best acquisition; any hand change to the panel or explicit layer OFF while it searches ends the request (`DISPLACED`) without pinning or success narration. `osm_query` asks `/api/osm/features` for one curated preset in a bounded box; the route answers `OVERPASS_NOT_CONFIGURED` without an operator Overpass. Results go to the session-only `osm-places` layer and become the analyst's last answer; a list capped at its limit stays incomplete in follow-ups, which keep its "listed the first N" caveat and report counts as floors. An explicit OSM Places OFF while the provider is pending aborts the work and prevents re-enable, result, credit and follow-up-memory commits. The voice answer deadline also aborts the client provider request, so the caller does not leave a timed-out request without a cancellation owner. The server's whole search (22 s across every configured endpoint, with Overpass told 20 s) ends before that 25 s deadline, because a browser that stops waiting cancels the upstream query and leaves nothing cached for a retry. @@ -5079,8 +5079,8 @@ source has its key (a keyless source explains itself and changes nothing, as a chip does), flies to the place at street framing and waits for arrival (a name, coordinates, a pointed spot, a numbered item, or nothing for the ground under the screen centre, as OPEN uses: `aim: 'screen-centre'` with -`armed: false`, so sky refuses and a link's armed point is neither used nor -dropped), switches to the map the source needs (Google Street View: Google 3D), +`armed: false`, so sky refuses and a link's armed point is not used and is +kept until a photo opens), switches to the map the source needs (Google Street View: Google 3D), lights that source through `setLayerParams` with origin `voice` (one source at a time, kept in the share link like a chip click; a link runs the action with origin `programmatic`, for the source switch and the layer, so it never writes @@ -5147,11 +5147,16 @@ checks again after every await (flight, map switch, layer on, lookup): a newer turn, a shell navigation stamp (`_navigationGeneration`, re-taken where its own flight lands) or a press or wheel on the globe stops the remaining stages (`superseded: true`), and cockpit, clean view, recording or scene -playback entered meanwhile ends it at the `view` stage; a photo that opened -after either is closed again. The hidden-panel check also runs before the -flight, so a recording's camera never moves for it. A photo already open -closes before the new request arms or looks up, so a place with no imagery -shows its own error, not the previous place's photo. A plain +playback entered meanwhile ends it at the `view` stage. During the lookup the +same checks run before a found photo loads (`proceed`), so nothing billed +loads for a request that was given up, and a lookup given up leaves the +panel as it was; a photo that opened after either is closed again. The +hidden-panel check also runs before the flight, so a recording's camera never +moves for it. A photo already open stays up while the new request looks (the +viewer is replaced, not closed, so EXPAND stays, also when the request +switches the source) and closes if the request opens nothing, so a place with +no imagery shows its own error, not the previous place's photo; a link that +only arms closes it at once. A plain link never loads a photo from a source that bills every load (`billedPerImage` on the provider; Google): the action runs with `autoOpen: false`, arms the lookup on the layer (`armNearest`: the link's diff --git a/src/app/embed.js b/src/app/embed.js index 1f18db5e0..d1ad4653c 100644 --- a/src/app/embed.js +++ b/src/app/embed.js @@ -341,10 +341,16 @@ export function installViews({ // A restore whose camera never landed (the recipient dragged, searched or // opened another place during its flight) leaves the camera with them. const cameraLanded = (outcome) => outcome?.share?.camera === 'applied'; + // The generation the restore's own flight owned: a navigation after it + // landed, while the layers still loaded, has moved on from it. + const restoreGeneration = (outcome) => + Number.isInteger(outcome?.navigationToken) + ? outcome.navigationToken + : shell._navigationGeneration; let generationAtRestore = null; let startSuperseded = false; let queue = ready.then((outcome) => { - generationAtRestore = shell._navigationGeneration; + generationAtRestore = restoreGeneration(outcome); startSuperseded = outcome?.status === 'superseded' || (Boolean(outcome?.share) && !cameraLanded(outcome)); @@ -414,7 +420,7 @@ export function installViews({ new URLSearchParams(String(location?.hash || '').replace(/^#/, '')), ); if (!streetLevel) return; - const generation = shell._navigationGeneration; + const generation = restoreGeneration(event.detail); queue = linkStep(() => openLinkedStreetLevel(streetLevel, { generation })); }; windowRef?.addEventListener?.( diff --git a/src/layers/streetLevel/index.js b/src/layers/streetLevel/index.js index 96992e347..5e13e647d 100644 --- a/src/layers/streetLevel/index.js +++ b/src/layers/streetLevel/index.js @@ -266,8 +266,11 @@ export function createStreetLevelLayer({ if (providerAvailable(entry)) parts.credits.show(state.viewer, entry.def); } + /** Requests replacing the open photo that switch its source keep it up. */ + let viewerHolds = 0; + function deactivate(entry) { - parts.viewerHost.unmount(entry.def.id); + if (!viewerHolds) parts.viewerHost.unmount(entry.def.id); entry.instance.deactivate(); parts.credits.hide(state.viewer, entry.def); } @@ -280,6 +283,13 @@ export function createStreetLevelLayer({ nearestLookup = null; } + /** Close the image and deselect its sequence on the map. */ + function closeViewer() { + abortNearest(); + parts.viewerHost.close(); + parts.clearSequences(); + } + /** Ground under the screen centre; null for sky, undefined when it cannot aim. */ function screenCentreGround() { const canvas = state.viewer?.scene?.canvas; @@ -692,15 +702,17 @@ export function createStreetLevelLayer({ * street); clicks keep the default. A caller's `signal` retires the * lookup: nothing opens after it fires. Providers without * `nearestImage` are skipped. `armed: false` (a request of its own, such - * as the assistant's) neither uses nor drops a link's armed point. Why + * as the assistant's) does not use a link's armed point, and keeps it + * while it looks; a photo it opens ends it, as any opened photo does. Why * nothing opened is left in `street.nearestOutcome`: 'empty' (no imagery * in reach), 'sky' (the screen centre is not ground), or null (an error). * `replace: true` (a request that takes over the panel) keeps an open * photo up while it looks, so the viewer and EXPAND stay, and closes it * when the lookup opens nothing: the panel shows this request's outcome. - * `proceed()` is asked once the lookup has found an image and before it - * loads; false retires the request (a photo that bills every load must - * not load for a request its caller has given up). + * `proceed()` is asked before a found image loads and before an empty + * or failed lookup reports; false retires the request and leaves the + * panel as it was (a photo that bills every load must not load for a + * request its caller has given up). */ async openNearest( point, @@ -722,9 +734,7 @@ export function createStreetLevelLayer({ const armed = useArmed ? state.street.armed : null; if (useArmed) state.street.armed = null; const closeReplaced = () => { - if (!replace || !state.street.open) return; - parts.viewerHost.close(); - parts.clearSequences(); + if (replace && state.street.open) closeViewer(); }; if (!point && aim === 'screen-centre' && armed) { // The button promises the centre of the view: the link's spot only @@ -812,6 +822,9 @@ export function createStreetLevelLayer({ if (proceed && proceed() !== true) return retired(); nearestLookup = null; callerSignal?.removeEventListener('abort', onCallerAbort); + // The photo asked for stays on screen: a sequence playing in the + // photo it replaces stops. + if (replace && state.street.open) parts.viewerHost.stopPlayback(); // The caller's signal still counts while the viewer mounts and loads. return openImage(entry.def.id, imageId, { frame, @@ -821,6 +834,9 @@ export function createStreetLevelLayer({ callerSignal?.removeEventListener('abort', onCallerAbort); // Replaced while it looked: the panel is the newer lookup's. if (nearestLookup !== lookup) return false; + // Given up by its caller while it looked: the open photo stays, and + // the panel reports nothing of this lookup. + if (proceed && proceed() !== true) return retired(); nearestLookup = null; closeReplaced(); // Switched off while it looked: nothing to open, nothing to report. @@ -848,11 +864,23 @@ export function createStreetLevelLayer({ notify(); return false; }, - /** Close the image and deselect its sequence on the map. */ - closeViewer() { - abortNearest(); - parts.viewerHost.close(); - parts.clearSequences(); + closeViewer, + /** + * Keep the open photo up while a request that replaces it switches the + * source (the viewer and EXPAND are replaced, not closed). The returned + * release closes it if its source is still off by then. + */ + holdViewer() { + viewerHolds += 1; + let held = true; + return () => { + if (!held) return; + held = false; + viewerHolds -= 1; + if (viewerHolds) return; + for (const entry of state.providers.values()) + if (!entry.on) parts.viewerHost.unmount(entry.def.id); + }; }, /** * Prepare a nearest lookup for the next OPEN press without running it: a diff --git a/src/layers/streetLevel/index.test.mjs b/src/layers/streetLevel/index.test.mjs index ce6a91abc..92cb2338e 100644 --- a/src/layers/streetLevel/index.test.mjs +++ b/src/layers/streetLevel/index.test.mjs @@ -366,7 +366,11 @@ function fakeViewer() { } /** A layer over `providers`, initialised and enabled on a stand-in viewer. */ -async function enabledLayer(t, providers = [fakeProvider()]) { +async function enabledLayer( + t, + providers = [fakeProvider()], + { services, setup } = {}, +) { const saved = globalThis.document; // A pose moves the position marker, whose glyph is drawn on a canvas. const drawing = new Proxy({}, { get: () => () => ({ addColorStop() {} }) }); @@ -374,7 +378,11 @@ async function enabledLayer(t, providers = [fakeProvider()]) { createElement: () => ({ getContext: () => drawing }), }); const viewer = fakeViewer(); - const layer = createStreetLevelLayer({ providers }); + setup?.(viewer); + const layer = createStreetLevelLayer({ + providers, + ...(services ? { services } : {}), + }); layer.init(viewer); layer.enable(viewer); t.after(() => { @@ -1944,7 +1952,6 @@ test('the panel says why a lit Street View draws nothing off Google 3D', async ( }); test('OPEN aimed at the sky after a link armed opens nothing: never the far link point', async (t) => { - const { createStreetLevelLayer: create } = await import('./index.js'); const scenePick = await import('../../data/scenePick.js'); const lookups = []; const provider = fakeProvider({ @@ -1953,26 +1960,18 @@ test('OPEN aimed at the sky after a link armed opens nothing: never the far link return null; }, }); - const saved = globalThis.document; - const drawing = new Proxy({}, { get: () => () => ({ addColorStop() {} }) }); - globalThis.document = Object.assign(new EventTarget(), { - createElement: () => ({ getContext: () => drawing }), - }); - const viewer = fakeViewer(); - // Every pick misses the ground: the screen centre is sky. - Object.assign(viewer.scene, { - pickPositionSupported: true, - pickPosition: () => undefined, - }); - viewer.globe = { pick: () => undefined }; - viewer.scene.canvas.clientWidth = 1000; - viewer.scene.canvas.clientHeight = 600; - const layer = create({ providers: [provider], services: { scenePick } }); - layer.init(viewer); - layer.enable(viewer); - t.after(() => { - layer.destroy(); - globalThis.document = saved; + const { layer } = await enabledLayer(t, [provider], { + services: { scenePick }, + setup: (viewer) => { + // Every pick misses the ground: the screen centre is sky. + Object.assign(viewer.scene, { + pickPositionSupported: true, + pickPosition: () => undefined, + }); + viewer.globe = { pick: () => undefined }; + viewer.scene.canvas.clientWidth = 1000; + viewer.scene.canvas.clientHeight = 600; + }, }); layer.armNearest({ lat: 48.8584, lon: 2.2945 }, { radiusM: 250 }); assert.equal( @@ -1986,7 +1985,6 @@ test('OPEN aimed at the sky after a link armed opens nothing: never the far link }); test('a request of its own (armed: false) neither uses nor drops the link’s armed point', async (t) => { - const { createStreetLevelLayer: create } = await import('./index.js'); const scenePick = await import('../../data/scenePick.js'); const lookups = []; const provider = fakeProvider({ @@ -1995,25 +1993,18 @@ test('a request of its own (armed: false) neither uses nor drops the link’s ar return null; }, }); - const saved = globalThis.document; - const drawing = new Proxy({}, { get: () => () => ({ addColorStop() {} }) }); - globalThis.document = Object.assign(new EventTarget(), { - createElement: () => ({ getContext: () => drawing }), - }); - const viewer = fakeViewer(); const centre = Cesium.Cartesian3.fromDegrees(-121.4937, 38.5799, 0); - Object.assign(viewer.scene, { - pickPositionSupported: true, - pickPosition: (at) => (at.x === 500 && at.y === 300 ? centre : undefined), - }); - viewer.scene.canvas.clientWidth = 1000; - viewer.scene.canvas.clientHeight = 600; - const layer = create({ providers: [provider], services: { scenePick } }); - layer.init(viewer); - layer.enable(viewer); - t.after(() => { - layer.destroy(); - globalThis.document = saved; + const { layer } = await enabledLayer(t, [provider], { + services: { scenePick }, + setup: (viewer) => { + Object.assign(viewer.scene, { + pickPositionSupported: true, + pickPosition: (at) => + at.x === 500 && at.y === 300 ? centre : undefined, + }); + viewer.scene.canvas.clientWidth = 1000; + viewer.scene.canvas.clientHeight = 600; + }, }); // About 120 m away: an OPEN press would take the link's spot. layer.armNearest({ lat: 38.5808, lon: -121.494 }, { radiusM: 250 }); @@ -2030,25 +2021,12 @@ test('a request of its own (armed: false) neither uses nor drops the link’s ar }); test('a failing source is not reported as missing imagery', async (t) => { - const { createStreetLevelLayer: create } = await import('./index.js'); const provider = fakeProvider({ nearestImage: async () => { throw new Error('Mapillary graph HTTP 500'); }, }); - const saved = globalThis.document; - const drawing = new Proxy({}, { get: () => () => ({ addColorStop() {} }) }); - globalThis.document = Object.assign(new EventTarget(), { - createElement: () => ({ getContext: () => drawing }), - }); - const viewer = fakeViewer(); - const layer = create({ providers: [provider] }); - layer.init(viewer); - layer.enable(viewer); - t.after(() => { - layer.destroy(); - globalThis.document = saved; - }); + const { layer } = await enabledLayer(t, [provider]); assert.equal(await layer.openNearest({ lat: 1, lon: 2 }), false); const street = layer.getUIState().street; assert.equal(street.nearestOutcome, null); diff --git a/src/layers/streetLevel/lineCoverage.js b/src/layers/streetLevel/lineCoverage.js index b71fe7cd7..80a86836c 100644 --- a/src/layers/streetLevel/lineCoverage.js +++ b/src/layers/streetLevel/lineCoverage.js @@ -532,9 +532,8 @@ export function createLineCoverage({ } /** - * The sequence lines drawn now, as `[lon, lat]` parts per sequence: for - * diagnostics and the browser gate, which aims its clicks at them instead - * of scanning pixels: what each tile drew (`attachPrimitive` keeps it in + * The sequence lines drawn now, as `[lon, lat]` parts per sequence, for + * diagnostics: what each tile drew (`attachPrimitive` keeps it in * `entry.sequences`). Read-only copies. */ function drawnLines() { diff --git a/src/layers/streetLevel/providers/mapillary/index.js b/src/layers/streetLevel/providers/mapillary/index.js index 97d3460b3..c40bed62c 100644 --- a/src/layers/streetLevel/providers/mapillary/index.js +++ b/src/layers/streetLevel/providers/mapillary/index.js @@ -6,7 +6,6 @@ import { import { createSequenceCones, thinImages } from '../../sequenceCones.js'; import { coverageZoomForHeight } from '../../tileMath.js'; import { passesImageryFilter } from '../../filter.js'; -import { NEAREST_RADIUS_M } from '../../policy.js'; import { metresBetween } from '../../view.js'; import { decodeCoverageTile } from './decode.js'; import { createMapillaryViewer } from './viewer.js'; @@ -20,7 +19,6 @@ import { MAPILLARY_NAME, MAPILLARY_PROVIDER_ID, MAPILLARY_SOURCE_METHODS, - NEAREST_LIMIT, PICK_PREFIX, RATE_LIMITED_MESSAGE, } from './policy.js'; @@ -206,10 +204,7 @@ export function createMapillaryProvider({ source, photoViewer = null }) { * cone). The core aborts `signal` once a newer request overtakes it. */ async nearestImage({ lat, lon }, { signal } = {}) { - const images = await source.nearestImages( - { lat, lon, radius: NEAREST_RADIUS_M, limit: NEAREST_LIMIT }, - { signal }, - ); + const images = await source.nearestImages({ lat, lon }, { signal }); // The API returns the images in the radius in no particular order. const metres = (record) => { const [lon2, lat2] = (record.computed_geometry || record.geometry) diff --git a/src/layers/streetLevel/providers/mapillary/provider.test.mjs b/src/layers/streetLevel/providers/mapillary/provider.test.mjs index 31ec665ad..aa1f2463f 100644 --- a/src/layers/streetLevel/providers/mapillary/provider.test.mjs +++ b/src/layers/streetLevel/providers/mapillary/provider.test.mjs @@ -200,11 +200,7 @@ test('nearestImage opens the closest image that passes the filter, not the first ); assert.equal(await flat.nearestImage({ lat: 1, lon: 2 }, { signal }), 'near'); assert.equal(requests[0].options.signal, signal, 'the core can abort it'); - assert.equal(requests[0].query.radius, 50); - // The API cuts at the limit before any order: a small one would sort and - // filter a handful of arbitrary photos (a 360° filter could find none). - assert.equal(requests[0].query.limit, 100, "the API's most"); - assert.deepEqual([requests[0].query.lat, requests[0].query.lon], [1, 2]); + assert.deepEqual(requests[0].query, { lat: 1, lon: 2 }); const any = createMapillaryProvider({ source }).create(fakeContext()); assert.equal(await any.nearestImage({ lat: 1, lon: 2 }), 'near-pano'); const tooOld = createMapillaryProvider({ source }).create( diff --git a/src/layers/streetLevel/providers/mapillary/replacePlay.test.mjs b/src/layers/streetLevel/providers/mapillary/replacePlay.test.mjs new file mode 100644 index 000000000..a8a2984c7 --- /dev/null +++ b/src/layers/streetLevel/providers/mapillary/replacePlay.test.mjs @@ -0,0 +1,56 @@ +import '../../../../testSupport/registerDom.mjs'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { registerHooks } from 'node:module'; +import { createStreetLevelLayer } from '../../index.js'; +import { fakeStreetLevelProvider, fakeCesiumViewer } from '../../../../testSupport/streetLevelFakes.mjs'; +import { createMapillaryViewer } from './viewer.js'; +import { showStreetLevel } from '../../../../voice/gevActions.js'; + +// Library boundary fixture: moveTo changes the image without stopping PLAY, +// matching installed MapillaryJS Viewer.moveTo / Navigator.moveTo$. +const fakeLibrary = `export class Viewer { + constructor() { globalThis.__replacePlayViewer = this; this.playing = false; this.id = null; this.stops = 0; } + on() {} remove() {} resize() {} setRenderMode() {} + getComponent() { return { play: () => {this.playing = true;}, stop: () => {this.playing = false; this.stops++;} }; } + async moveTo(id) { this.id = id; return { id, sequenceId: 'fixture-sequence', cameraType: 'perspective' }; } + async getPosition() { return { lng: 2, lat: 48 }; } + async getPointOfView() { return { bearing: 0 }; } +} +export class GraphDataProvider {} +export const RenderMode = { Fill: 1, Letterbox: 0 };`; +const hooks = registerHooks({ + resolve(specifier, context, next) { + if (specifier === 'mapillary-js') return { url: 'data:text/javascript,' + encodeURIComponent(fakeLibrary), shortCircuit: true }; + if (specifier.endsWith('mapillary.css')) return { url: 'data:text/javascript,export default {}', shortCircuit: true }; + return next(specifier, context); + }, +}); +globalThis.fetch = async () => { throw new Error('Unexpected network use in offline test'); }; +test('a photo the action asks for stops sequence PLAY', async t => { + const provider = fakeStreetLevelProvider({ nearestImage: async () => 'B' }); + const create = provider.create; + provider.create = context => ({ ...create(context), viewer: createMapillaryViewer({ source: { token: '' } }) }); + const viewer = fakeCesiumViewer(); viewer.canvas = viewer.scene.canvas; + const layer = createStreetLevelLayer({ providers: [provider] }); + layer.init(viewer); layer.enable(viewer); + layer.attachViewerHost(document.body.appendChild(document.createElement('div'))); + t.after(() => { layer.destroy(); hooks.deregister(); delete globalThis.__replacePlayViewer; document.body.replaceChildren(); }); + await Promise.resolve(); + await layer.openImage('mapillary', 'A', { frame: false }); + const libraryViewer = globalThis.__replacePlayViewer; + libraryViewer.getComponent('sequence').play(); + assert.equal(libraryViewer.playing, true); + const manager = { + layers: new Map([['street-level', { module: layer }]]), isEnabled: () => true, + getLayerParams: () => layer.getParams(), setLayerParams: (_id, params) => layer.setParams(params), + }; + const result = await showStreetLevel({ provider: 'mapillary', latitude: 48, longitude: 2 }, { + viewer, dataManager: manager, styleManager: { _navigationGeneration: 1 }, + runGevAction: async () => ({ ok: true, latitude: 48, longitude: 2, label: 'fixture destination' }), + }); + assert.equal(result.ok, true); + assert.equal(libraryViewer.id, 'B'); + assert.equal(layer.getUIState().street.open, true); + assert.equal(libraryViewer.playing, false, 'requested image must stay selected rather than continue sequence PLAY'); +}); diff --git a/src/layers/streetLevel/providers/mapillary/source.js b/src/layers/streetLevel/providers/mapillary/source.js index 156771de6..032240e7b 100644 --- a/src/layers/streetLevel/providers/mapillary/source.js +++ b/src/layers/streetLevel/providers/mapillary/source.js @@ -196,18 +196,15 @@ export function createMapillarySource({ return Array.isArray(payload?.data) ? payload.data : []; }, - /** Images within `radius` metres of a point, in no particular order. */ - async nearestImages( - { lat, lon, radius = NEAREST_RADIUS_M, limit = NEAREST_LIMIT }, - { signal } = {}, - ) { + /** Images within `NEAREST_RADIUS_M` of a point, in no particular order. */ + async nearestImages({ lat, lon }, { signal } = {}) { const payload = await graph( 'images', { lat: String(lat), lng: String(lon), - radius: String(Math.min(50, Math.max(1, radius))), - limit: String(Math.min(100, Math.max(1, limit))), + radius: String(NEAREST_RADIUS_M), + limit: String(NEAREST_LIMIT), fields: NEAREST_IMAGE_FIELDS, }, { signal, timeoutMs: NEAREST_TIMEOUT_MS }, diff --git a/src/layers/streetLevel/providers/mapillary/source.test.mjs b/src/layers/streetLevel/providers/mapillary/source.test.mjs index d7fb8b7ee..9afc41291 100644 --- a/src/layers/streetLevel/providers/mapillary/source.test.mjs +++ b/src/layers/streetLevel/providers/mapillary/source.test.mjs @@ -126,24 +126,21 @@ test('a rejected key and a rate limit are told apart from other tile errors', as ); }); -test('nearestImages asks the graph API around a point, its radius and limit clamped', async () => { +test('nearestImages asks the graph API around a point, within 50 m, for at most 100 images', async () => { const { calls, fetchImpl } = fakeFetch(() => jsonResponse({ data: [{ id: '1', geometry: { coordinates: [1, 2] } }] }), ); const source = createMapillarySource({ token: 'MLY|1|abc', fetchImpl }); - const images = await source.nearestImages({ - lat: 38.5, - lon: -121.5, - radius: 500, - limit: 500, - }); + const images = await source.nearestImages({ lat: 38.5, lon: -121.5 }); assert.equal(images.length, 1); const url = new URL(calls[0].url); assert.equal(url.pathname, '/images'); assert.equal(url.searchParams.get('lat'), '38.5'); assert.equal(url.searchParams.get('lng'), '-121.5'); - assert.equal(url.searchParams.get('radius'), '50', 'the API maximum'); - assert.equal(url.searchParams.get('limit'), '100'); + assert.equal(url.searchParams.get('radius'), '50'); + // The API cuts at the limit before any order: a small one would sort and + // filter a handful of arbitrary photos (a 360° filter could find none). + assert.equal(url.searchParams.get('limit'), '100', "the API's most"); assert.match(url.searchParams.get('fields'), /computed_geometry/); assert.match(url.searchParams.get('fields'), /is_pano/); }); diff --git a/src/layers/streetLevel/providers/mapillary/viewer.js b/src/layers/streetLevel/providers/mapillary/viewer.js index 4ec2654e4..c270ba796 100644 --- a/src/layers/streetLevel/providers/mapillary/viewer.js +++ b/src/layers/streetLevel/providers/mapillary/viewer.js @@ -174,6 +174,15 @@ export function createMapillaryViewer({ source, render } = {}) { current = null; } + /** A playing sequence keeps stepping through (and downloading) images. */ + function stopPlayback() { + try { + viewer?.getComponent('sequence')?.stop(); + } catch { + /* sequence component not available */ + } + } + return { async mount(host) { if (!host) throw new Error('Street Level viewer has no host element'); @@ -195,15 +204,12 @@ export function createMapillaryViewer({ source, render } = {}) { close() { pendingOpen = null; current = null; - // A playing sequence would keep stepping through (and downloading) - // images in the hidden viewer. - try { - viewer?.getComponent('sequence')?.stop(); - } catch { - /* sequence component not available */ - } + // Hidden, the viewer must not keep playing. + stopPlayback(); }, + stopPlayback, + unmount() { pendingOpen = null; // A build still waiting for the library gives up when it arrives. diff --git a/src/layers/streetLevel/registry.js b/src/layers/streetLevel/registry.js index 4abf39412..a79cbd675 100644 --- a/src/layers/streetLevel/registry.js +++ b/src/layers/streetLevel/registry.js @@ -51,7 +51,7 @@ export function filtersOf(provider) { * @property {(sequenceId: string) => Promise} [selectSequence] * @property {() => void} [clearSequence] * @property {() => {selectedId: string|null, images: number, loading: boolean, error?: string|null}} [sequenceStats] - * @property {() => Array<{id: string, parts: Array>}>} [coverageLines] The coverage lines drawn now, as [lon, lat] parts (diagnostics and the browser gate). + * @property {() => Array<{id: string, parts: Array>}>} [coverageLines] The coverage lines drawn now, as [lon, lat] parts (diagnostics). * @property {(point: {lat: number, lon: number}, options?: {signal?: AbortSignal, radiusM?: number}) => Promise} [nearestImage] For ground clicks and `openNearest`; an aborted lookup gives up (it may reject with an AbortError). `radiusM` asks for a wider search than NEAREST_RADIUS_M; a provider that cannot search wider ignores it. * @property {ViewerAdapter} viewer * diff --git a/src/layers/streetLevel/viewerHost.js b/src/layers/streetLevel/viewerHost.js index f56d2487f..acaba443e 100644 --- a/src/layers/streetLevel/viewerHost.js +++ b/src/layers/streetLevel/viewerHost.js @@ -175,6 +175,11 @@ export function createViewerHost({ state, parts }) { active?.adapter.resize(); } + /** Stop a sequence playing in the open photo (adapters that play). */ + function stopPlayback() { + active?.adapter.stopPlayback?.(); + } + function setRenderMode(mode) { state.street.renderMode = mode === 'fill' ? 'fill' : 'letterbox'; active?.adapter.setRenderMode?.(state.street.renderMode); @@ -197,5 +202,5 @@ export function createViewerHost({ state, parts }) { active = null; } - return { attach, open, close, resize, setRenderMode, unmount }; + return { attach, open, close, resize, setRenderMode, stopPlayback, unmount }; } diff --git a/src/streetLevelLifecycle.test.mjs b/src/streetLevelLifecycle.test.mjs new file mode 100644 index 000000000..3a08a44c3 --- /dev/null +++ b/src/streetLevelLifecycle.test.mjs @@ -0,0 +1,194 @@ +import './testSupport/registerDom.mjs'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createStreetLevelLayer } from './layers/streetLevel/index.js'; +import { fakeStreetLevelProvider, fakeCesiumViewer } from './testSupport/streetLevelFakes.mjs'; +import { showStreetLevel } from './voice/gevActions.js'; +import { installViews } from './app/embed.js'; +import { ShareRestoration } from './ui/shareRestoration.js'; +import { createView } from './view/index.js'; +import { StreetLevelControls } from './ui/streetLevelControls.js'; +import { createApp, nextTick } from 'vue'; +import { createPinia } from 'pinia'; +import StreetLevelPanel from './ui/views/panels/StreetLevelPanel.vue'; +import { useStreetLevelStore } from './ui/stores/streetLevel.js'; + +// Street-level request lifecycle across the action, the layer, links and the +// panel. Offline: providers and camera flights are fixtures. +globalThis.fetch = async () => { throw new Error('Unexpected network use in offline test'); }; +const settle = () => new Promise(resolve => setTimeout(resolve, 0)); +function deferred() { + let resolve; + const promise = new Promise(done => { resolve = done; }); + return { promise, resolve }; +} +async function harness(t, { nearestImage = async () => 'A', failImage = null, billed = false, extraProviders = [] } = {}) { + document.body.className = ''; + document.body.replaceChildren(); + const provider = fakeStreetLevelProvider({ nearestImage }); + provider.billedPerImage = billed; + const originalCreate = provider.create; + const opened = []; + provider.create = context => { + const instance = originalCreate(context); + let pose; + instance.viewer.onPose = listener => { pose = listener; return () => { pose = null; }; }; + instance.viewer.open = async id => { + opened.push(id); + if (id === failImage) throw new Error('Fixture image load failed'); + pose?.({ imageId: id }); + }; + return instance; + }; + const viewer = fakeCesiumViewer(); + viewer.canvas = viewer.scene.canvas; + viewer.canvas.setPointerCapture = () => {}; + const layer = createStreetLevelLayer({ providers: [provider, ...extraProviders] }); + layer.init(viewer); + layer.enable(viewer); + const host = document.body.appendChild(document.createElement('div')); + layer.attachViewerHost(host); + t.after(() => { layer.destroy(); document.body.className = ''; document.body.replaceChildren(); }); + await settle(); + assert.equal(layer.providerStatus('mapillary').billed, billed); + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + getAll: () => [{ id: 'street-level', enabled: true }], + isEnabled: () => true, + getLayerParams: () => ({ mapillary: true }), + setLayerParams: (_id, params) => layer.setParams(params), + }; + const styleManager = { _navigationGeneration: 1 }; + const flights = []; + const context = { + viewer, dataManager, styleManager, + runOptions: { signal: new AbortController().signal }, + runGevAction: async (name, args) => { + assert.equal(name, 'fly_to_location'); + flights.push(args); + return { ok: true, latitude: 48, longitude: 2, label: 'fixture point' }; + }, + }; + let result; + const run = async (name, args) => { + if (name !== 'show_street_level') return { ok: true }; + result = await showStreetLevel(args, context); + return result; + }; + const view = createView({ camera: { lat: 48, lon: 2 }, layers: ['street-level'], street_level: { provider: 'mapillary', lat: 48, lon: 2 } }); + return { layer, viewer, dataManager, context, run, view, opened, flights, result: () => result }; +} + +test('a landed link does not reclaim a camera the user moved while its layers load', async t => { + const h = await harness(t); + const location = { search: '', hash: '' }; + const shell = h.context.styleManager; + shell.initialRestorePromise = Promise.resolve(); + const remove = installViews({ ...h, shell, location, windowRef: window }); + t.after(remove); + await settle(); + location.hash = '#sl=mapillary,48,2'; + const layers = deferred(); + const landed = deferred(); + const restoration = Object.assign(Object.create(ShareRestoration.prototype), { + _shareRestoreGeneration: 1, _disposed: false, + shareLinkManager: { + applyState: async () => { landed.resolve(); return { camera: 'applied' }; }, + completeInitialRestore() {}, + }, + _sharedVessel: () => null, + }); + const restoring = restoration._runShareRestore({}, { + applyCamera: true, navigationToken: 1, layersPromise: layers.promise, + settle: detail => window.dispatchEvent(new CustomEvent('gev:share-navigation-restore-settled', { detail })), + }); + await landed.promise; + shell._navigationGeneration++; + layers.resolve([]); + await restoring; + await settle(); await settle(); + assert.equal(h.flights.length, 0, 'an old landed link must not fly after later navigation'); +}); + +test('a replacement from another source keeps EXPAND', async t => { + const h = await harness(t, { + extraProviders: [fakeStreetLevelProvider({ id: 'panoramax', pickPrefix: 'pm:', nearestImage: async () => 'B' })], + }); + const root = document.body.appendChild(document.createElement('div')); + root.innerHTML = '
'; + const wrap = root.querySelector('#sl-viewer-wrap'); + let fullscreen = null; + Object.defineProperty(document, 'fullscreenElement', { configurable: true, get: () => fullscreen }); + const oldExit = document.exitFullscreen; + // As a browser does: the change is announced after the switch. + const announce = () => setTimeout(() => document.dispatchEvent(new Event('fullscreenchange')), 0); + document.exitFullscreen = async () => { fullscreen = null; announce(); }; + wrap.requestFullscreen = async () => { fullscreen = wrap; announce(); }; + const store = { attach: () => 1, detach() {}, publish() {}, setExpandAvailable() {}, setExpanded() {}, setEmbedOverlay() {} }; + const controls = new StreetLevelControls({ root, layer: h.layer, actions: {}, store }); + const off = h.layer.subscribe(state => controls.render(state)); + t.after(() => { off(); controls.destroy(); delete document.fullscreenElement; document.exitFullscreen = oldExit; }); + assert.equal(await h.layer.openImage('mapillary', 'previous', { frame: false }), true); + await controls._toggleExpanded(); + assert.equal(controls.isViewerExpanded(), true); + const result = await showStreetLevel({ provider: 'panoramax', latitude: 48, longitude: 2 }, h.context); + assert.equal(result.ok, true); + assert.equal(h.layer.getUIState().street.providerId, 'panoramax'); + assert.equal(controls.isViewerExpanded(), true, 'replacement must preserve the expanded viewing surface'); +}); + +for (const outcome of ['empty', 'error', 'image']) { + test(`a replacement retired by a navigation keeps the open photo (${outcome})`, async t => { + const lookup = deferred(); + const entered = deferred(); + const h = await harness(t, { nearestImage: async () => { + entered.resolve(); await lookup.promise; + if (outcome === 'error') throw new Error('Fixture lookup failure'); + return outcome === 'image' ? 'B' : null; + } }); + await h.layer.openImage('mapillary', 'A', { frame: false }); + const action = showStreetLevel({ provider: 'mapillary', latitude: 48, longitude: 2 }, h.context); + await entered.promise; + h.context.styleManager._navigationGeneration++; + lookup.resolve(); + const result = await action; + assert.ok(result.cancelled || result.superseded); + assert.deepEqual(h.opened, ['A']); + assert.equal(h.layer.getUIState().street.open, true, 'retired lookup must preserve A'); + assert.equal(h.layer.getUIState().street.error, null, 'retired lookup must not publish an error'); + }); +} + +test('a failed replacement shows its error inside the expanded viewer', async t => { + const h = await harness(t, { nearestImage: async () => 'B', failImage: 'B' }); + const root = document.body.appendChild(document.createElement('div')); + const pinia = createPinia(); + const app = createApp(StreetLevelPanel); app.use(pinia); app.mount(root); + const store = useStreetLevelStore(pinia); + const wrap = root.querySelector('#sl-viewer-wrap'); + let fullscreen = null; + Object.defineProperty(document, 'fullscreenElement', { configurable: true, get: () => fullscreen }); + const oldExit = document.exitFullscreen; + // As a browser does: the change is announced after the switch. + const announce = () => setTimeout(() => document.dispatchEvent(new Event('fullscreenchange')), 0); + document.exitFullscreen = async () => { fullscreen = null; announce(); }; + wrap.requestFullscreen = async () => { fullscreen = wrap; announce(); }; + const controls = new StreetLevelControls({ root, layer: h.layer, actions: {}, store }); + const off = h.layer.subscribe(state => controls.render(state)); + t.after(() => { off(); controls.destroy(); app.unmount(); delete document.fullscreenElement; document.exitFullscreen = oldExit; }); + await h.layer.openImage('mapillary', 'A', { frame: false }); + await nextTick(); + await controls._toggleExpanded(); + await settle(); + const result = await showStreetLevel({ provider: 'mapillary', latitude: 48, longitude: 2 }, h.context); + await nextTick(); + assert.equal(result.ok, false); + assert.equal(controls.isViewerExpanded(), true); + assert.equal(document.fullscreenElement, wrap); + assert.equal(h.layer.getUIState().street.imageId, 'A'); + assert.match(h.layer.getUIState().street.error, /Fixture image load failed/); + assert.equal(wrap.contains(root.querySelector('#sl-error')), false); + const error = wrap.querySelector('#sl-viewer-error'); + assert.match(error.textContent, /Fixture image load failed/); + assert.equal(error.hidden, false, 'error must be visible inside fullscreen'); +}); diff --git a/src/ui/shareRestoration.js b/src/ui/shareRestoration.js index 9fc4c6f45..4921d991e 100644 --- a/src/ui/shareRestoration.js +++ b/src/ui/shareRestoration.js @@ -263,7 +263,9 @@ export class ShareRestoration { // the shared flight having landed. const vessel = this._sharedVessel(state); if (vessel) void this._restoreSharedVessel(vessel, current); - settle({ status: 'settled', share, layers, tracking }); + // The token tells listeners whether anyone navigated after the + // restore's flight landed (while the layers were still loading). + settle({ status: 'settled', share, layers, tracking, navigationToken }); } catch (error) { if (!current()) return; this.shareLinkManager.completeInitialRestore(state.restoreToken); diff --git a/src/ui/views/panels/StreetLevelPanel.vue b/src/ui/views/panels/StreetLevelPanel.vue index 1661574bb..fa3c593a5 100644 --- a/src/ui/views/panels/StreetLevelPanel.vue +++ b/src/ui/views/panels/StreetLevelPanel.vue @@ -142,7 +142,7 @@ const vThumb = { - +
@@ -159,7 +159,7 @@ const vThumb = { {{ t('PROVIDERS') }}
- +
diff --git a/src/view/index.js b/src/view/index.js index c22f2c3f6..11c3b8c7d 100644 --- a/src/view/index.js +++ b/src/view/index.js @@ -354,10 +354,11 @@ export function createView({ ); if (needs) map = needs; // The point defaults to where the camera looks: a view "of X" opens - // photos at X, also when the camera sits behind it. + // photos at X, also when the camera sits behind it. It is a pair: half + // of one is not mixed with the camera's other half. + const given = [latitude(streetLevel.lat), longitude(streetLevel.lon)]; const seen = lookAtPoint(camera); - const lat = latitude(streetLevel.lat) ?? seen.lat; - const lon = longitude(streetLevel.lon) ?? seen.lon; + const [lat, lon] = given.includes(undefined) ? [seen.lat, seen.lon] : given; street = Object.freeze({ ...(provider ? { provider } : {}), lat: round(clamp(lat, -90, 90), 5), diff --git a/src/view/index.test.mjs b/src/view/index.test.mjs index 1389b058b..9fda7d149 100644 --- a/src/view/index.test.mjs +++ b/src/view/index.test.mjs @@ -208,6 +208,16 @@ test('street-level photos travel in the link, light their source and bring their assert.equal(streetLevelFromParams(new URLSearchParams('sl=bing,1,2')), null); assert.equal(streetLevelFromParams(new URLSearchParams('lat=1')), null); assert.equal(createView({ camera: { lat: 1, lon: 2 } }).street_level, null); + // Half a point is not mixed with the camera's other half. + const half = createView({ + camera: { lat: 1, lon: 2 }, + street_level: { provider: 'mapillary', lat: 40 }, + }); + assert.deepEqual(half.street_level, { + provider: 'mapillary', + lat: 1, + lon: 2, + }); }); test('street-level photos: a tilted camera opens them where it looks, a link cannot pair them with the wrong map, and follow is refused', async () => { diff --git a/src/voice/actionSchemas.test.mjs b/src/voice/actionSchemas.test.mjs index c2324067c..c2f084d06 100644 --- a/src/voice/actionSchemas.test.mjs +++ b/src/voice/actionSchemas.test.mjs @@ -58,32 +58,11 @@ test('the complete Realtime tool payload pins the manifest-generated layer relea // control_cockpit states the own-layer entry rule (Contacts not required). // move_camera tilt takes an optional pitchDeg angle, described as // absolute with its -45 default, with steps sent as a direction. - // show_street_level (map, source, photo) is the only change on top of main. - SHOW_STREET_LEVEL_PAYLOAD_DIGEST, - ); - // Every shipped tool is byte-identical to the previous release: the new - // tool is the whole difference, so a later edit cannot hide in a re-pin. - const shipped = createHash('sha256') - .update( - JSON.stringify( - stable( - GEV_REALTIME_TOOLS.filter( - (tool) => - !['set_cyber_sonar', 'show_street_level'].includes(tool.name), - ), - ), - ), - ) - .digest('hex'); - assert.equal( - shipped, - '6a0d154e68beb787c4e20f12d29a5c86577dc343f3e71d8b5651998ea45d8c3c', + // show_street_level (map, source, photo) adds one tool. + '0c27f62062001e6a07241cd301c3d357f55e9b74f66cd2904d16590510337468', ); }); -const SHOW_STREET_LEVEL_PAYLOAD_DIGEST = - '0c27f62062001e6a07241cd301c3d357f55e9b74f66cd2904d16590510337468'; - test('descriptions customize wording without changing immutable shared arguments', () => { const descriptions = { fly_to_location: { diff --git a/src/voice/deixis.js b/src/voice/deixis.js index c78ccb54a..d224eb661 100644 --- a/src/voice/deixis.js +++ b/src/voice/deixis.js @@ -143,6 +143,8 @@ function resolveFly(args, { pointer, referents }) { entity: { layerId: target.layerId, id: target.id, + // The answer names it, not its id ("near cam-9"). + ...(target.label ? { label: target.label } : {}), ...(hasPosition(target) ? { lat: target.lat, lon: target.lon } : {}), }, }, diff --git a/src/voice/deixis.test.mjs b/src/voice/deixis.test.mjs index cd7f7374d..20ae0530f 100644 --- a/src/voice/deixis.test.mjs +++ b/src/voice/deixis.test.mjs @@ -104,7 +104,7 @@ test('referents: numbered items resolve for track, fly and context', () => { const last = resolveDeicticArgs('track_entity', { query: 'x', referent: -1 }, { referents }); assert.equal(last.args.query, 'ccc333', '-1 is the last one'); const fly = resolveDeicticArgs('fly_to_location', { referent: 1 }, { referents }); - assert.deepEqual(fly.args, { entity: { layerId: 'flights', id: 'aaa111' } }, 'moving contacts are located at run time'); + assert.deepEqual(fly.args, { entity: { layerId: 'flights', id: 'aaa111', label: 'SWA12' } }, 'moving contacts are located at run time'); const context = resolveDeicticArgs('get_entity_context', { referent: 3 }, { referents }); assert.equal(context.args.scope, 'target'); assert.equal(context.args.target.label, 'AAL9'); diff --git a/src/voice/gevActions.js b/src/voice/gevActions.js index f8a906dc2..a4cb18a16 100644 --- a/src/voice/gevActions.js +++ b/src/voice/gevActions.js @@ -3813,24 +3813,26 @@ function streetLevelPanelHidden() { } /** - * "Show me on Street View / on Mapillary": fly to the place, make sure - * the imagery source's map is up, light that source in Street Level, turn the - * layer on and open the nearest image. One source at a time, as the panel - * does; the choice persists like a chip click. The key is checked before - * anything changes, and each stage reports its own failure so speech can say - * which part did not happen. With no place at all, the view centre is used. - * `autoOpen: false` (links, not the model) stops short of loading a photo - * from a source that bills every load: the panel is armed and the OPEN - * button is one click away. + * "Show me on Street View / on Mapillary": fly there, light the source + * (and its map) and open the nearest image. Each stage reports its own + * failure, so speech can say which part did not happen. `autoOpen: false` + * (links) stops short of a billed load: the panel is armed for OPEN. */ export async function showStreetLevel(args, context) { // Every await can hand the request's moment to someone else: a newer turn, // a navigation or gesture that takes the camera, or a mode that hides the // panel. Each stage after the place is settled checks again. const owner = watchCameraOwner(context.viewer, context.styleManager); + // A photo held up while the request switches its source is let go however + // the request ends: replaced by then, or closed if its source stayed off. + let releaseViewer = null; + const holdViewer = (layerModule) => { + releaseViewer ??= layerModule.holdViewer?.() ?? null; + }; try { - return await requestStreetLevel(args, { ...context, owner }); + return await requestStreetLevel(args, { ...context, owner, holdViewer }); } finally { + releaseViewer?.(); owner.stop(); } } @@ -3840,9 +3842,9 @@ async function requestStreetLevel( { dataManager, styleManager, - placeSearch, runGevAction, owner, + holdViewer = () => {}, runOptions = {}, current = () => true, reportProgress = () => {}, @@ -3868,11 +3870,14 @@ async function requestStreetLevel( const query = String(args.query || '').trim(); const latitude = Number(args.latitude); const longitude = Number(args.longitude); + const coordinatesGiven = args.latitude != null && args.longitude != null; const hasCoordinates = - args.latitude != null && - args.longitude != null && - Number.isFinite(latitude) && - Number.isFinite(longitude); + coordinatesGiven && Math.abs(latitude) <= 90 && Math.abs(longitude) <= 180; + if (coordinatesGiven && !hasCoordinates) + return fail( + 'location', + 'Latitude must be within ±90° and longitude within ±180°', + ); // A referent that is a layer entity (deixis): located when the flight starts. const entity = args.entity && typeof args.entity === 'object' && args.entity.id @@ -3952,29 +3957,11 @@ async function requestStreetLevel( atViewCentre = !query && !hasCoordinates && !entity; let point = null; let location = null; - if (args.fly === false || atViewCentre) { + // No place at all: the layer looks at the ground under the screen centre, + // as the OPEN button does. `fly: false` (tests) keeps the camera for a + // point given by its coordinates. + if (atViewCentre || (args.fly === false && hasCoordinates)) { if (hasCoordinates) point = { lat: latitude, lon: longitude }; - else if (entity) { - // Where it is now, as the flying path does: the coordinates a list - // saved go stale as soon as the item moves or unloads. - const located = locateLayerEntity(dataManager, entity); - if (!located) return fail('location', `${label} is no longer loaded`); - point = { lat: located.lat, lon: located.lon }; - } else if (query) { - reportProgress('search', query); - const outcome = await placeSearch.geocode(query, { - signal: runOptions.signal, - }); - const place = outcome?.place; - if (!current()) - return fail('location', 'Street-level request was cancelled'); - if (!Number.isFinite(place?.lat) || !Number.isFinite(place?.lng)) - return fail('location', `Could not find ${query}`); - point = { lat: place.lat, lon: place.lng }; - label = place.label || query; - } - // No place at all: the layer looks at the ground under the screen - // centre, as the OPEN button does. } else { location = await runGevAction( 'fly_to_location', @@ -3995,8 +3982,10 @@ async function requestStreetLevel( location?.error || `Could not arrive at ${location?.label || label}`, { cancelled: !current() || Boolean(location?.cancelled), location }, ); - // An item keeps its name: the flight to it reports only coordinates. - label = entity ? label : location.label || label; + // An item, or a named place flown to by its coordinates, keeps its name: + // such a flight reports only coordinates. + label = + entity || (query && hasCoordinates) ? label : location.label || label; if ( !Number.isFinite(location.latitude) || !Number.isFinite(location.longitude) @@ -4037,7 +4026,10 @@ async function requestStreetLevel( if (stop) return stop; } - // Light the source the way a chip click does, so the choice is kept. + // Light the source the way a chip click does, so the choice is kept. An + // open photo of another source stays up until this request replaces it. + if (layerModule.getUIState?.()?.street?.open === true) + holdViewer(layerModule); const switches = Object.fromEntries( providerIds.map((id) => [id, id === provider]), ); diff --git a/src/voice/gevActions.test.mjs b/src/voice/gevActions.test.mjs index cf4cbff6f..7ccaabc8d 100644 --- a/src/voice/gevActions.test.mjs +++ b/src/voice/gevActions.test.mjs @@ -6660,11 +6660,11 @@ function createStreetLevelHarness({ return false; } } - // Like the layer: a replacing lookup that opens nothing closes the photo. + // Like the layer: a lookup its caller gave up leaves the panel alone, + // and a replacing lookup that opens nothing closes the photo. + if (options.proceed && options.proceed() !== true) return false; const closeReplaced = () => { - if (!options.replace || !street.open) return; - order.push('close'); - Object.assign(street, { open: false, imageId: null, error: null, nearestOutcome: null }); + if (options.replace && street.open) layer.closeViewer(); }; if (imageryError) { closeReplaced(); @@ -6678,8 +6678,6 @@ function createStreetLevelHarness({ street.error = `No street-level imagery within ${options.radiusM ?? 50} m of ${point ? 'that point' : 'the view centre'}`; return false; } - // Found: it loads only if the request still wants it. - if (options.proceed && options.proceed() !== true) return false; street.open = true; street.imageId = nearest; street.title = '10th St, Sacramento'; @@ -6762,6 +6760,14 @@ test('show_street_level flies first, then switches the map, lights the source, t assert.equal(result.imageId, 'pano-a'); assert.equal(result.title, '10th St, Sacramento'); assert.equal(result.flew, true); + // A name given with its coordinates is what the answer says. + const named = createStreetLevelHarness(); + const naming = named.runner('show_street_level', { + query: 'Capitol Mall', latitude: 38.5799, longitude: -121.4937, provider: 'google', + }); + await new Promise((resolve) => setImmediate(resolve)); + named.finishFlight(); + assert.equal((await naming).label, 'Capitol Mall'); }); test('show_street_level on Mapillary keeps the map, searches at the default radius and skips what is already on', async () => { @@ -6799,6 +6805,11 @@ test('show_street_level reports each stage it could not pass', async () => { assert.equal(none.label, 'the view centre'); assert.deepEqual(centre.order, ['params:google:voice', 'open:google:250:false:view-centre']); + const offGlobe = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); + const outOfRange = await offGlobe.runner('show_street_level', { latitude: 95, longitude: 2, provider: 'google' }); + assert.deepEqual([outOfRange.ok, outOfRange.stage], [false, 'location']); + assert.deepEqual(offGlobe.order, [], 'nothing flies, switches or looks up'); + const unknown = await createStreetLevelHarness().runner('show_street_level', { latitude: 1, longitude: 2, provider: 'openstreetcam', fly: false, }); @@ -6865,15 +6876,6 @@ test('show_street_level opens photos of a numbered item or a pointed spot throug assert.equal(result.ok, true, result.error); assert.equal(result.label, 'Camera 9'); assert.deepEqual(order, ['fly', 'params:google:voice', 'open:google:250:false']); - // Without a flight the entity's own position is used. - const still = withCamera(createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' })); - const kept = await still.runner('show_street_level', { - provider: 'google', - fly: false, - entity: { layerId: 'cctv', id: 'cam-9', lat: 38.58, lon: -121.5 }, - }); - assert.equal(kept.ok, true, kept.error); - assert.deepEqual(still.order, ['params:google:voice', 'open:google:250:false']); }); test('show_street_level hands the point from a named place, a preset and a curated POI to the lookup', async () => { @@ -7003,43 +7005,9 @@ test('show_street_level tells a failed enable, a delisted source, a key refused assert.deepEqual(unconfigured.order, [], 'nothing changed'); }); -test('show_street_level without a flight geocodes a name, locates an item, and reports what it cannot place', async () => { +test('show_street_level reports a flight that lands without a position', async () => { globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; const hit = createStreetLevelHarness({ enabled: true, activeStack: 'osm' }); - const points = []; - const open = hit.layer.openNearest; - hit.layer.openNearest = (point, options) => { - points.push(point); - return open(point, options); - }; - const runner = createGevActionRunner({ - viewer: createVoiceNavigationHarness().viewer, - styleManager: { mapStackController: { getActiveId: () => 'osm' }, setMapStack: async () => ({ ok: true }) }, - dataManager: hit.dataManager, - placeSearch: { - geocode: async (query) => - query === 'Capitol Mall' - ? { place: { lat: 38.5792, lng: -121.4958, label: 'Capitol Mall, Sacramento' } } - : { place: null, answered: true }, - }, - }); - const found = await runner('show_street_level', { query: 'Capitol Mall', provider: 'mapillary', fly: false }); - assert.equal(found.ok, true, found.error); - assert.equal(found.label, 'Capitol Mall, Sacramento'); - assert.deepEqual(points.at(-1), { lat: 38.5792, lon: -121.4958 }); - assert.equal(found.flew, false); - - const miss = await runner('show_street_level', { query: 'Atlantis Boulevard', provider: 'mapillary', fly: false }); - assert.deepEqual([miss.ok, miss.stage], [false, 'location']); - assert.equal(miss.say, "Couldn't find Atlantis Boulevard."); - assert.equal(points.length, 1, 'a miss asks for no imagery'); - - const gone = await runner('show_street_level', { - provider: 'mapillary', fly: false, entity: { layerId: 'cctv', id: 'cam-gone', label: 'Camera 7' }, - }); - assert.deepEqual([gone.ok, gone.stage], [false, 'location']); - assert.equal(gone.say, "Couldn't find Camera 7."); - // A flight that reports no coordinates cannot be followed by a lookup. const noPoint = createGevActionRunner({ viewer: createVoiceNavigationHarness().viewer, @@ -7105,20 +7073,6 @@ test('show_street_level says "refused its key" when Google already refused it, a assert.match(dark.error, /could not be switched on/); }); -test('show_street_level without a flight needs the item where it is now, not where a list saw it', async () => { - globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; - const harness = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); - // The referent carries the coordinates the list saved, but it is no longer loaded. - const stale = await harness.runner('show_street_level', { - provider: 'google', - fly: false, - entity: { layerId: 'cctv', id: 'cam-gone', label: 'Camera 7', lat: 38.58, lon: -121.5 }, - }); - assert.deepEqual([stale.ok, stale.stage], [false, 'location']); - assert.match(stale.error, /no longer loaded/); - assert.ok(!harness.order.some((entry) => entry.startsWith('open:')), 'nothing looked up at the stale point'); -}); - test('activeImageryFilter reports only the narrowing filters the lit source honours', async () => { const { activeImageryFilter } = await import('./gevActions.js'); const both = { pano: true, since: true }; @@ -7281,32 +7235,6 @@ test('a partial analyst card shows one partial chip', () => { assert.equal(new Set(labels).size, labels.length, JSON.stringify(labels)); }); -test('show_street_level superseded during its geocode changes nothing', async () => { - globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; - const h = createStreetLevelHarness({ enabled: true }); - let current = true; - const runner = createGevActionRunner({ - viewer: createVoiceNavigationHarness().viewer, - styleManager: { - mapStackController: { getActiveId: () => 'osm' }, - setMapStack: async (id) => { - h.order.push(`map:${id}`); - return { ok: true }; - }, - }, - dataManager: h.dataManager, - placeSearch: { - geocode: async () => { - current = false; - return { place: { lat: 1, lng: 2, label: 'X' } }; - }, - }, - }); - const result = await runner('show_street_level', { query: 'X', provider: 'google', fly: false }, { isCurrent: () => current }); - assert.deepEqual(h.order.filter((step) => /^(map|params|open|arm)/.test(step)), [], 'the stale turn switched nothing'); - assert.deepEqual([result.ok, result.stage, result.cancelled], [false, 'location', true]); -}); - test('show_street_level superseded during its flight switches nothing', async () => { globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; const h = createStreetLevelHarness(); diff --git a/src/voice/pointAndAsk.test.mjs b/src/voice/pointAndAsk.test.mjs index 290fc2537..b1942c85f 100644 --- a/src/voice/pointAndAsk.test.mjs +++ b/src/voice/pointAndAsk.test.mjs @@ -343,17 +343,9 @@ test('deixis prompt lint: one short rule, and every sentinel it names exists in tool('annotate_map').annotations.items.properties.target.description, /"pointer"/, ); - // Budget: the whole Realtime prompt (instructions + tools), measured in - // characters. It was 56,308 after the area/imagery/OSM merge; stating each - // policy once (WHERE precedence, AREAS routing, AFTER TOOLS speech) brought - // it to 51,646. Voice geometry (resolve_area, find_imagery, osm_query and - // the analyst area scopes) raised the ceiling to 55,500; tightening the - // AREAS rule and the area tool descriptions left it at 55,103 as measured - // below (every tool, osm_query included); main's Sea Marks and Ski Pistes - // layers bring it to 55,210, and the tilt pitchDeg argument (with its -45 - // default) and main's cockpit entry rule to 55,339. show_street_level (the - // tool and its routing rule, ~940) raises it to 56,500. A new tool or rule - // has to fit under this ceiling. + // Budget: the whole Realtime prompt (instructions + every tool, osm_query + // included), in characters. Every session pays for it in latency and cost, + // so a new tool or rule has to fit under this ceiling or earn a raise. const size = instructions.length + JSON.stringify(GEV_REALTIME_TOOLS).length; assert.ok(size <= 56_500, `prompt size ${size}`); });