diff --git a/README.md b/README.md index 47cdf9535..cbc136b83 100644 --- a/README.md +++ b/README.md @@ -382,7 +382,7 @@ Click **GEV MIC**, grant the microphone, and just talk. This is more than a voic - **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director. - **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token. If you choose **USE CHATGPT OAUTH** in Provider Settings, the local server uses that sign-in only to mint the same kind of short-lived Realtime credential, and refuses the OAuth path for non-loopback clients. -Thirty tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook: +Thirty-two tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook: **🎥 Direct it** — drone-operator camera verbs: diff --git a/SECURITY.md b/SECURITY.md index 4423eead3..8c43bcee5 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -129,6 +129,7 @@ God's Eye View's tools are also served to MCP clients: over stdio (`npm run mcp` - **What the panel's requests can reach.** In clients that display MCP Apps, the panel loads the app through the `panel_request` tool, which requests paths on the app's server. It is marked for the panel only and requires a key that each MCP server puts in its panel page, which keeps it from clients that list tools to the model without loading the panel. That key is not access control: any MCP client can read the panel page, key included, and then reach the app's files and data routes through `panel_request`, much as its other tools reach the data. Provider Settings, credential and model endpoints, `/mcp` and the development server's internal routes are refused to every caller. - **The panel carries the two browser keys.** The panel build (`npm run build:panel`) includes the same Google Maps key and Cesium ion token as the app, and they run inside the client's panel page, on the client's site. A key restricted by HTTP referrer must also allow that page, or the panel cannot load tiles. To keep the app's own keys narrower, build the panel with separately restricted keys in the environment of `npm run build:panel`. - **Embed mode is not framable by default.** `?embed=1` documents keep `X-Frame-Options: DENY` and `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names the pages that may frame them; a framing page can change what the app shows. Other documents can never be framed. +- **A plain link never loads a billed Street View panorama without a click.** A share link with `sl=google,,` flies there, switches to Google 3D and lights Street View, then stops: the panel is armed and the photo loads only when the visitor presses OPEN GOOGLE STREET VIEW (one lookup and one panorama load on the browser Maps key, no retries, no geocoding or server call). A view posted into the embedded app (the God's Eye View panel, or a page `GEV_EMBED_FRAME_ANCESTORS` allows), and the panel page's own opening link, are that client's own request and open the photo; adding `?embed=1` to a published link does not lift the gate (in a frame the link only arms; the panel that holds OPEN is hidden there, so a posted view is the way to open a photo). The armed lookup dies when its source is switched off, when its map leaves, or when any other photo opens or closes. A link that follows an entity ignores `sl`. Mapillary, KartaView and Panoramax links open their photo at once; none of them bills per image. ## Scope & expectations diff --git a/changelog.d/1113-show-street-level.md b/changelog.d/1113-show-street-level.md new file mode 100644 index 000000000..36cf415df --- /dev/null +++ b/changelog.d/1113-show-street-level.md @@ -0,0 +1,14 @@ +- "Show me X on Street View" (or on Mapillary, KartaView or Panoramax): a + `show_street_level` voice and typed action flies to the place, switches to + Google 3D when the source needs it, lights that source in Street Level, + turns the layer on and opens the nearest photo, reporting which step failed. + Views and share links carry the same request (`street_level`, `sl=`), so + `show_in_gods_eye_view` and a link can open a panorama at a place. Street + View lookups from a geocoded place search up to 250 m; clicks keep 50 m. A + plain link with `sl=google` arms Street View and leaves the billed load to a + press of OPEN GOOGLE STREET VIEW; embedded views open it. The source the + action lights stays lit like a chip click. An unavailable map stack now + reports the controller's own reason instead of always asking for a Cesium + ion token. When nothing is found and a 360°, flat or SINCE filter the source + honours is on, the answer names the filter (Said Turksever, @asturksever, + #1113). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 35d81dd05..fe1083dd3 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -828,7 +828,7 @@ Voice controls bind to a protocol-independent session factory. The default WebRT The standalone app offers OpenAI and Gemini Live through an explicit provider chooser. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter. -Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields. +Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 32 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields. `createGevActionRunner` routes `control_cctv` enable/disable through the shared layer visibility lifecycle. Results distinguish successful changes from @@ -4328,6 +4328,53 @@ core's lookups alone. The MCP globe panel allows the meta-catalogue and the three biggest instances; a picture on another instance shows only in the app. +**"Show me X on Street View."** The `show_street_level` assistant action +(voice and typed; `src/voice/gevActions.js`) is one call: it checks the +source has its key (a keyless source explains itself and changes nothing, as +a chip does), flies to the place at street framing and waits for arrival (a +name, coordinates, a pointed spot, a numbered item, or nothing for the view +centre), switches to the map the source needs (Google Street View: Google 3D), +lights that source through `setLayerParams` with origin `voice` (one source at +a time, kept in the share link like a chip click), turns the layer on and +calls `openNearest`. `provider` is `google` for "Street View", `mapillary`, +`kartaview` or `panoramax` by name, else the lit source. Stages fail separately (`location`, `map`, +`provider`, `layer`, `key`, `imagery`; an empty `imagery` result names a +narrowing 360°/flat or SINCE filter the source honours, as `filter`) and a +newer turn retires the lookup +(`openNearest`'s `signal` option), so nothing opens after the user moved on. +For a source without a coverage layer the lookup widens to 250 m +(`NEAREST_RADIUS_MAX_M`; Google honours it, Mapillary's API caps at 50 m); +clicks keep 50 m. The layer exposes `requiredMapStack(id)` and +`providerStatus(id)` for callers that must decide before the panel lists a +source, and `coverageLines(id)` (the lines a provider draws now, current and +stale tiles, as `[lon, lat]` parts) for diagnostics and the browser gate, +which aims its line clicks at that geometry, waits for the asynchronous +ground primitives to be ready, and clicks in the same JavaScript turn as the +pick so a rebuilt frame cannot come between them. Views carry the same request as `street_level` (`src/view/index.js`: +source and point, defaulting to where the camera looks; a `google` source +brings `map: photoreal` and refuses another map; not with `follow`), links as +`sl=,,`; `show_in_gods_eye_view` accepts it, the globe panel +applies it through the action, and a plain link runs the action once restored +(one shot: the rewritten hash drops `sl`). In embed mode the rail holding the +panel is hidden, so an open photo moves to an overlay on the globe +(`sl-viewer-wrap-embed`) and back when it closes; Esc closes it (in full +screen, Esc leaves full screen first). A caller's abort is +honoured through the viewer's mount and load, not just the lookup, and only +the lookup that still owns the panel may change its spinner or error. A +passive link's photos stand down if anyone navigated (a gesture, a search, +the assistant) while the link waited behind its own marks. A plain +link never loads a photo from a source that bills every load +(`billedPerImage` on the provider; Google): the action runs with +`autoOpen: false`, arms the lookup on the layer (`armNearest`: the link's +point and radius, consumed only by the next OPEN press (`aim: +'screen-centre'`); any other lookup, open, close, disable, its source +switched off or its map leaving drops it), opens the panel and a toast +points at OPEN GOOGLE STREET VIEW. Only the panel page's own link (inline +embed) and posted views open the photo; `?embed=1` on a published link only +arms, without the panel nudge or toast (the rail is hidden in a frame). A link that follows an entity ignores `sl`. +The look-at default applies below 20 km, at a pitch steeper than -5° and +within 10 km ahead; otherwise the camera's own point is used. + Street Level has two browser gates. `npm run qa:street-level -- --url ` runs against real Mapillary and needs `MAPILLARY_CLIENT_TOKEN`. `npm run qa:street-level:fixtures -- --url ` answers every Mapillary request @@ -5361,7 +5408,7 @@ and unreachable upstream (502/504) separately from road geometry. - **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware uses `OPENAI_API_KEY` by default, or the signed-in local ChatGPT/Codex OAuth access token when OAuth was explicitly selected in Provider Settings, and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. Both long-lived credentials stay server-side; OAuth minting is loopback-only. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token. - **Session defaults** (env-tunable): model `gpt-realtime-2` (or `gpt-realtime-2.1-mini` when the MINI tier is selected — see the model-tier entry below), voice `marin`, reasoning effort `low`, semantic VAD with low eagerness, no response interruption, context window truncated to ~3,000 post-instruction tokens with 0.5 retention ratio — the conversational window stays short because map state is fetched live per turn. -- **Thirty tools** (argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `next_iss_pass`, and `next_satellite_pass`. +- **Thirty-two tools** (argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `show_street_level`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_voice_inactivity_timeout`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `next_iss_pass`, and `next_satellite_pass`. > **Reading `npm test` totals:** the count depends on the Node major. The two > GC-bracketed allocation microbenchmarks (`src/data/focusAllocations.test.mjs` diff --git a/docs/TOOLS.md b/docs/TOOLS.md index 80fd80b13..2effba947 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -102,11 +102,26 @@ action draws (pins, highlights, areas, arrows, routes and labels at a place name or coordinates). Links carry them in the `an` parameter, and the app draws them once the link has been restored. +A view can open street-level photos too: `street_level` names a source +(`google` for Street View, `mapillary`, `kartaview`, `panoramax`, or none for +whichever the panel has lit) and a point, which defaults to the spot the camera looks at. Street View +shows only on the Google 3D map, so a `google` source sets `map` to +`photoreal` and any other map is refused; a view cannot both follow an entity +and open photos. The `street-level` layer is turned on. Links carry it as +`sl=,,`; once the link has restored, the app runs its +`show_street_level` action, which flies to the point at street framing and +opens the nearest photo. A plain link with a `google` source stops before the +billed load: the panel is armed and OPEN GOOGLE STREET VIEW is one click away. +A view posted into the embedded app, or the panel page's own opening link, +opens the photo, since it is the client's own request; `?embed=1` on a +published link does not. Mapillary, KartaView and Panoramax links open the +photo at once; none of them bills per image. A link that follows an entity ignores `sl`. + Tools take a view as `VIEW_ARGUMENTS`: an `area` to frame from above, or a -`camera`, plus `layers`, `style`, `map`, `follow` and `annotations`. A -tilted camera over an area looks at its center from behind; camera fields -given with an area override its framing. `resolveViewArguments` turns them -into a view. +`camera`, plus `layers`, `style`, `map`, `follow`, `annotations` and +`street_level`. A tilted camera over an area looks at its center from behind; +camera fields given with an area override its framing. `resolveViewArguments` +turns them into a view. ### The God's Eye View panel @@ -299,5 +314,5 @@ and the development server's internal routes. See SECURITY.md. | `situation_brief` | `weather` | Weather, earthquakes, fires, aircraft, ships and cyclones for an area, by section | | `military_awareness` | `military` | Military and other aircraft, ships and military installations within 250 km of a point, by section | | `get_hud_caption` | `weather`, `summary` | The app's heads-up display caption for an area | -| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks and something to follow | +| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks, something to follow, or street-level photos (Street View or Mapillary) at a place | | `panel_request` | `app` | Panel only: loads a path from the app's server for the God's Eye View panel | diff --git a/scripts/qa-street-level.mjs b/scripts/qa-street-level.mjs index 0dd841acf..12e350212 100644 --- a/scripts/qa-street-level.mjs +++ b/scripts/qa-street-level.mjs @@ -46,6 +46,15 @@ const VIEWPORT = { width: 1440, height: 900 }; /** Over downtown Sacramento, above the fixture photo line, looking down. */ const PARK = { lon: PHOTO_LINE.lon, lat: 38.5816, height: 900, pitch: -1.3 }; const DAY_MS = 86_400_000; +/** A Sacramento street with Street View, for the sl=google link step. */ +const STREET_VIEW_STREET = Object.freeze({ lon: -121.49372, lat: 38.57986 }); +/** + * What a Street View lookup and load cost: the Maps JS API's + * SingleImageSearch (the nearest lookup), GetMetadata and the panorama tiles. + * Script loads are not counted. + */ +const STREET_VIEW_REQUEST = + /MapsJsInternalService\/(SingleImageSearch|GetMetadata)|GeoPhotoService|streetviewpixels-pa\.googleapis\.com/; /** * Coverage for any street-zoom tile: a grid of 360°/flat, recent/old @@ -234,6 +243,28 @@ async function boot(page, url) { }); } +/** + * Open a share link as a fresh document. Changing only the hash of a live + * page and reloading races the app's share-link writer, which can put its own + * state back in the hash before the reload reads it. The page's fixtures (a + * CDP session on the page) carry over. + */ +async function bootLink(page, link) { + await page.goto('about:blank'); + await boot(page, link); +} + +/** The camera's position in degrees and metres. */ +const cameraAt = (page) => + page.evaluate(() => { + const c = window.__godsEyeView.viewer.camera.positionCartographic; + return { + lon: (c.longitude * 180) / Math.PI, + lat: (c.latitude * 180) / Math.PI, + height: c.height, + }; + }); + /** Run `test(uiState, arg)` in the page until it is true. */ const uiUntil = (page, test, arg = null, timeout = 30_000) => page.waitForFunction( @@ -750,6 +781,219 @@ async function main() { await uiUntil(page, litAlone, 'mapillary'); await park(page); + const base = url.split('#')[0]; + await fixtureStep( + 'a link to street-level photos (sl=) opens the photo at its point once the link restores', + async () => { + const photo = fixture.images[0]; + // The camera sits ~330 m north of the photo: the lookup must use + // the link's point, not the view centre. + const cameraLat = Number((photo.lat + 0.003).toFixed(6)); + await bootLink( + page, + `${base}#v=2&lat=${cameraLat}&lon=${photo.lon}&alt=600&heading=0&pitch=-60&roll=0&sl=mapillary,${photo.lat},${photo.lon}`, + ); + await uiUntil( + page, + (u) => (u.street.imageId && !u.street.loading) || u.street.error, + null, + 90_000, + ); + const { street } = await getUI(page); + assert.equal(street.error, null, `viewer error: ${street.error}`); + assert.equal(street.providerId, 'mapillary'); + assert.equal( + street.imageId, + photo.id, + `the photo at the link's point, not the one nearest the camera: ${street.imageId}`, + ); + assert.equal( + await page.evaluate(() => location.hash.includes('sl=')), + false, + 'the rewritten link no longer carries the one-shot request', + ); + }, + ); + + await fixtureStep( + 'the show_street_level action opens the nearest photo at a point without a pointer', + async () => { + await page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('street-level') + .module.closeViewer(), + ); + const photo = fixture.images[0]; + const result = await page.evaluate( + (at) => + window.__godsEyeView.voiceCommands.runner('show_street_level', { + latitude: at.lat, + longitude: at.lon, + provider: 'mapillary', + fly: false, + }), + { lat: photo.lat, lon: photo.lon }, + ); + assert.equal( + result.ok, + true, + `action failed: ${result.stage} ${result.error}`, + ); + assert.equal(result.provider, 'mapillary'); + assert.ok( + fixture.images.some((image) => image.id === result.imageId), + `a fixture photo opened: ${result.imageId}`, + ); + assert.match(result.say || '', /^Mapillary at /); + }, + ); + + await step( + 'an sl=google link never loads a photo by itself: keyless it stops at the key, keyed it arms and waits for OPEN', + async ({ skipped }) => { + const point = STREET_VIEW_STREET; + // ~330 m north of the street: a flight would move the camera, and a + // press that ignored the armed point would open something else. + const cameraLat = Number((point.lat + 0.003).toFixed(6)); + const googleRequests = []; + const onRequest = (request) => { + if (STREET_VIEW_REQUEST.test(request.url())) + googleRequests.push(request.url()); + }; + page.on('request', onRequest); + try { + await bootLink( + page, + `${base}#v=2&lat=${cameraLat}&lon=${point.lon}&alt=600&heading=0&pitch=-60&roll=0&map=osm&sl=google,${point.lat},${point.lon}`, + ); + await page.waitForFunction( + (want) => { + const c = + window.__godsEyeView.viewer.camera.positionCartographic; + return Math.abs((c.latitude * 180) / Math.PI - want) < 0.0005; + }, + { timeout: 60_000 }, + cameraLat, + ); + const googleKey = await page.evaluate(() => + Boolean(window.__GOOGLE_MAPS_API_KEY__), + ); + if (!googleKey) { + // Without a key the action stops at the key, before any flight, + // map switch or source change, and the direct call says why. + const direct = await page.evaluate( + (at) => + window.__godsEyeView.voiceCommands.runner( + 'show_street_level', + { + latitude: at.lat, + longitude: at.lon, + provider: 'google', + fly: false, + autoOpen: false, + }, + ), + point, + ); + assert.equal(direct.ok, false); + assert.equal( + direct.stage, + 'key', + `${direct.stage} ${direct.error}`, + ); + const after = await page.evaluate(() => { + const m = + window.__godsEyeView.dataManager.layers.get( + 'street-level', + ).module; + const u = m.getUIState(); + return { + stack: window.__godsEyeView.mapStackController.getActiveId(), + google: m.getParams().google === true, + armed: u.street.armed, + open: u.street.open, + }; + }); + assert.deepEqual( + after, + { stack: 'osm', google: false, armed: null, open: false }, + 'a keyless Street View link changes nothing', + ); + const still = await cameraAt(page); + assert.ok( + Math.abs(still.lat - cameraLat) < 0.0005, + `the camera did not fly: ${still.lat}`, + ); + assert.deepEqual(googleRequests, [], 'no Street View request'); + return; + } + const photoreal = await page.evaluate(() => + window.__godsEyeView.mapStackController.isStackAvailable( + 'photoreal', + ), + ); + if (!photoreal) return skipped('no Google 3D'); + await uiUntil( + page, + (u) => + u.street.armed !== null && + u.providers.some((p) => p.id === 'google' && p.on), + null, + 90_000, + ); + const beforePress = googleRequests.length; + const waiting = await getUI(page); + assert.equal( + waiting.street.open, + false, + 'nothing open without a click', + ); + assert.equal(waiting.street.imageId, null); + assert.deepEqual( + waiting.street.armed.point, + { lat: point.lat, lon: point.lon }, + 'armed at the link point', + ); + assert.equal( + beforePress, + 0, + 'no Street View request before the press', + ); + await expandPanel(page); + await press(page, '#sl-nearest-btn'); + await uiUntil( + page, + (u) => (u.street.imageId && !u.street.loading) || u.street.error, + null, + 90_000, + ); + const { street } = await getUI(page); + assert.equal(street.error, null, `viewer error: ${street.error}`); + assert.equal(street.providerId, 'google'); + assert.ok( + googleRequests.length >= 1, + 'the press made the Street View request', + ); + const metres = Math.hypot( + (street.position.lat - point.lat) * 111_320, + (street.position.lon - point.lon) * + 111_320 * + Math.cos((point.lat * Math.PI) / 180), + ); + assert.ok( + metres <= 250, + `within 250 m of the link point: ${Math.round(metres)} m`, + ); + // Back to Mapillary: the keyless page restores the saved layer + // and expects its KEY REQUIRED. + await press(page, '#sl-provider-chips [data-chip-id="mapillary"]'); + await uiUntil(page, litAlone, 'mapillary'); + } finally { + page.off('request', onRequest); + } + }, + ); + await fixtureStep( 'keyless server (second page): KEY REQUIRED gates the controls', async () => { diff --git a/scripts/qa-voice-routing.mjs b/scripts/qa-voice-routing.mjs index 0d89b525a..cb75b32ba 100644 --- a/scripts/qa-voice-routing.mjs +++ b/scripts/qa-voice-routing.mjs @@ -889,6 +889,7 @@ const SLOW_TOOLS = new Set([ 'annotate_map', 'fly_to_location', 'select_nearest_aircraft', + 'show_street_level', 'analyst_query', 'fly_route', 'frame_overhead', diff --git a/scripts/voice-bench/phrases.mjs b/scripts/voice-bench/phrases.mjs index 9d1bcc8ba..6fb6f5de8 100644 --- a/scripts/voice-bench/phrases.mjs +++ b/scripts/voice-bench/phrases.mjs @@ -35,6 +35,14 @@ export const CORE_PHRASES = Object.freeze([ { phrase: 'Show me the whole earth', expect: 'zoom_to_globe' }, { phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead', typedArgs: [where('frame_overhead', (a) => a.target === 'flights')] }, + // — street-level photos: one composite action, never fly + basemap — + { phrase: 'Show me the Eiffel Tower on Street View', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'google' && /eiffel/i.test(a.query || ''))] }, + { phrase: 'Street view of 10th Street in Sacramento', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'google')] }, + { phrase: 'Show me Times Square on Mapillary', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'mapillary')] }, + { phrase: 'Switch to Google Street View', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'google' && !a.query)] }, + { phrase: 'Show me the Eiffel Tower', expect: 'fly_to_location' }, + { phrase: 'Turn on street level', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'street-level' && a.enabled === true)] }, + // — the satellites trap: data layer, never basemap — { phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } }, { phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'satellites' && a.enabled === false)] }, diff --git a/server/providers/openai/instructions.js b/server/providers/openai/instructions.js index acf576d13..471ee21ad 100644 --- a/server/providers/openai/instructions.js +++ b/server/providers/openai/instructions.js @@ -28,7 +28,7 @@ function realtimeInstructions( 'READING THE VIEW: with no selected object or overlay entities, answer from get_entity_context\'s basemap context (tiles source, centre target, reverse-geocoded place, altitude, style, enabled layers); say there is nothing only when that is unavailable too. Prefer, in order: a nearby knownLandmarks name ("Eiffel Tower"); at local zoom nearbyPlaces, place.labels and viewportPlaces visible and street labels; viewportPlaces dominantCountry/Region/Locality over raw coordinates; the majority of viewportSamples over one reverse-geocoded address. Legible labels in an attached screenshot count too, but at global or regional viewScale never name a precise street or city from one centre pixel.', 'Do not mention disabled layers or stale selections.', 'POINTING: pointer_context says what the cursor is on (pointing:: resolved by the tools; target "none": not pointing). For "this/that/here/that one" with no name, pass "pointer": get_entity_context scope, track_entity/fly_to_location query, analyst_query scope.kind ("around here" is pointer, not view), annotate_map target. "The second/last one" is referent:n from the last numbered list. Without a pointer, ask or follow WHERE.', - 'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, a searched fly_to_location, select_nearest_aircraft, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.', + 'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, a searched fly_to_location, select_nearest_aircraft, show_street_level, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.', 'When one request contains MULTIPLE changes, call ALL the corresponding tools before the confirmation. Never confirm a partial subset; if one fails, say which.', 'AFTER TOOLS: speak the result\'s say line once, lightly rephrased at most, never repeating your preamble. For a requested analyst list or ranking, also name up to three returned items with the relevant returned values; keep their order. A count alone does not answer that request; if say is null, finish any remaining requested tools before confirming. Never add a count, list or action claim that the executed tool results do not support. Sources, notes and caveats are on the voice card: speak a caveat only when it changes the answer, or when asked why, how sure or which source. Lower bounds ("at least 500"), partial answers and stale, degraded or unavailable feed states in say are part of the answer — always preserve them. No filler or hedges ("let me check", "keep in mind", "it seems", "based on the data available"), and never add "approximately" to an exact tool number.', 'For "show/open/turn on" layer requests, enable the matching layer. For "hide/close/turn off", disable it.', @@ -55,6 +55,7 @@ function realtimeInstructions( '"Track/follow " (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.', '"Show me which planes are overhead"/"frame the ships"/"show me the satellites above" use frame_overhead with the matching target.', 'Confirmations echo the RESULTING state, never the request: "HUD minimal layout", "Density twenty-five percent", "Bing aerial imagery", "Tracking UAL428", "Framed fourteen aircraft". On ok=false, state the failure plainly: "Nothing matched UAL999", "No ships within 120 kilometers". Never claim an action without ok=true in the tool result.', + 'STREET LEVEL: "X on Street View/Mapillary/KartaView/Panoramax", "street view of X" → show_street_level once, not fly_to_location; no place = view centre. Street View is never a basemap.', 'For destination requests such as "take me to Italy", "go to NYC", or "show me the Eiffel Tower", call fly_to_location. Prefer known city IDs when available; otherwise pass the plain place query.', 'Navigation-only requests ("take me to X", "go to X", "fly to X") are NOT descriptions: call fly_to_location alone and do NOT also call annotate_map, unless the user explicitly asks to mark the place or you go on to explain specific places there. Never drop a point pin on a region-scale natural feature (a mountain range, desert, sea, or forest) — a single point in the middle of the Rockies is meaningless. If the user explicitly asks to mark such a region, prefer type=area.', 'For country and city destinations, omit rangeM so GEV frames the whole country or city in view. For landmarks and buildings, omit rangeM so GEV chooses a close landmark view.', diff --git a/server/providers/openai/toolDescriptions.js b/server/providers/openai/toolDescriptions.js index 23d5da9de..5225511be 100644 --- a/server/providers/openai/toolDescriptions.js +++ b/server/providers/openai/toolDescriptions.js @@ -65,6 +65,28 @@ export const ACTION_DESCRIPTIONS = { }, }, }, + show_street_level: { + description: + 'Fly to a place and open its nearest street-level photo; Street View switches to Google 3D (billed). No query: view centre.', + $position: 1, + parameters: { + properties: { + query: { + description: 'Place or address; "pointer" = under the cursor.', + $position: 1, + }, + provider: { + description: + 'google = Street View; others by name; omit: the lit source.', + $position: 1, + }, + referent: { + description: 'Numbered item n from the last result list.', + $position: 1, + }, + }, + }, + }, adjust_camera_zoom: { description: 'Move the current Cesium camera closer to or farther from what it is presently looking at. Use for relative zoom requests without changing location.', diff --git a/src/app/embed.js b/src/app/embed.js index 21e685a91..a3dec5b84 100644 --- a/src/app/embed.js +++ b/src/app/embed.js @@ -13,12 +13,18 @@ import * as Cesium from 'cesium'; import { announceNavigationAuthority } from '../navigationPolicy.js'; -import { annotationsFromParams, createView } from '../view/index.js'; +import { + annotationsFromParams, + createView, + followFromParams, + streetLevelFromParams, +} from '../view/index.js'; const EMBED_VIEW_MESSAGE = 'gev:view'; const EMBED_APPLIED_MESSAGE = 'gev:view-applied'; const EMBED_READY_MESSAGE = 'gev:ready'; +const STREET_LEVEL_LAYER = 'street-level'; const FOLLOW_LAYERS = { aircraft: 'flights', military_aircraft: 'military', @@ -165,6 +171,26 @@ function flyTo(viewer, camera) { }); } +/** + * The `show_street_level` arguments for a view's street-level photos. A plain + * link (`autoOpen: false`) never loads a photo from a source that bills every + * load: the app arms the panel and the OPEN button is one click away. A view + * posted into the embedded app is the client's own request and opens. + */ +function streetLevelArgs(streetLevel, { autoOpen = true } = {}) { + return { + latitude: streetLevel.lat, + longitude: streetLevel.lon, + ...(streetLevel.provider ? { provider: streetLevel.provider } : {}), + fly: true, + ...(autoOpen ? {} : { autoOpen: false }), + }; +} + +const STREET_LEVEL_PANEL_ID = 'street-level-panel'; +const ARMED_TOAST = + 'Street View is ready: press OPEN GOOGLE STREET VIEW to load it'; + /** * Make the app show a view: style and map, exactly the view's layers, * annotations, the camera, and the followed entity. Each step runs through the @@ -198,6 +224,8 @@ export async function applyView( await act('set_layer_visibility', { layerId: layer.id, enabled: false }); } for (const layerId of view.layers) { + // The street-level action lights its source first, then the layer. + if (view.street_level && layerId === STREET_LEVEL_LAYER) continue; if (!dataManager.isEnabled(layerId)) await act('set_layer_visibility', { layerId, enabled: true }); } @@ -207,6 +235,18 @@ export async function applyView( await act('clear_annotations', {}); if (view.annotations.length) await act('annotate_map', { annotations: [...view.annotations] }); + if (view.street_level) { + // The photos frame the camera: the action flies to the point at street + // level, lights the source and opens the nearest photo. When it fails + // before flying, the view's own camera still applies. + const result = await act( + 'show_street_level', + streetLevelArgs(view.street_level), + ); + if (result?.ok !== true && !result?.location) + steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) }); + return steps; + } if (!view.follow) { steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) }); return steps; @@ -254,12 +294,56 @@ export function installViews({ windowRef = globalThis.window, }) { const ready = Promise.resolve(shell.initialRestorePromise).catch(() => {}); - const linked = annotationsFromParams( - new URLSearchParams(String(location?.hash || '').replace(/^#/, '')), + const hashParams = new URLSearchParams( + String(location?.hash || '').replace(/^#/, ''), ); + // What the link itself asks for runs first, one thing at a time; posted + // views queue behind it. One failing step does not take the next with it. + // The camera's owner at restore: anyone who navigates while the link's + // steps wait (a gesture, a search, the assistant) takes it, and a passive + // link must not fly over them. + let generationAtRestore = null; + let queue = ready.then(() => { + generationAtRestore = shell._navigationGeneration; + }); + const navigatedSinceRestore = () => + Number.isFinite(generationAtRestore) && + shell._navigationGeneration !== generationAtRestore; + const linkStep = (work) => + queue.then(async () => { + if (signal?.aborted) return; + try { + await work(); + } catch { + /* reported by the action itself */ + } + }); + const linked = annotationsFromParams(hashParams); if (linked.length) - void ready.then(() => { - if (!signal?.aborted) void run('annotate_map', { annotations: linked }); + queue = linkStep(() => run('annotate_map', { annotations: linked })); + // A link to street-level photos opens them once the link has restored. A + // link anyone can publish stops short of a billed load and leaves it to a + // click; only the panel page's own link (inline embed) is the client's + // request. A link that follows an entity keeps the camera on it. + const streetLevel = followFromParams(hashParams) + ? null + : streetLevelFromParams(hashParams); + if (streetLevel) + queue = linkStep(async () => { + if (navigatedSinceRestore()) return; + const result = await run( + 'show_street_level', + streetLevelArgs(streetLevel, { autoOpen: isEmbeddedInline() }), + ); + // The restore window keeps the panel collapsed; OPEN must be in view. + // Embed mode hides the rail that holds it, so there the link only arms + // (a posted view is the way to open a photo in a frame). + if (result?.armed === true && !isEmbedded(location)) { + shell.setPanelCollapsed?.(STREET_LEVEL_PANEL_ID, false, { + explicit: true, + }); + shell._showToast?.(ARMED_TOAST); + } }); if (!isEmbedded(location)) return () => {}; @@ -278,7 +362,6 @@ export function installViews({ peer.postMessage(message, origin); }; // Views apply one at a time, in the order they arrive. - let queue = ready; const onMessage = (event) => { if (event.source !== peer || event.data?.type !== EMBED_VIEW_MESSAGE) return; diff --git a/src/app/embed.test.mjs b/src/app/embed.test.mjs index 5fa949342..f7aae019e 100644 --- a/src/app/embed.test.mjs +++ b/src/app/embed.test.mjs @@ -8,7 +8,7 @@ import { panelAwareVisibility, } from './embed.js'; import { LayerLifecycle } from '../data/lifecycle.js'; -import { createView } from '../view/index.js'; +import { createView, viewToParams } from '../view/index.js'; const fakeViewer = () => { const flights = []; @@ -393,3 +393,475 @@ test('a followed aircraft owns the camera, and cockpit view is entered after it' /no cockpit view/, ); }); + +test('a view with street-level photos lets the action fly and open them', async () => { + const viewer = fakeViewer(); + const { calls, run } = recorder(); + const view = createView({ + camera: { lat: 38.5799, lon: -121.4937, altitude_m: 500 }, + street_level: { provider: 'google' }, + }); + const steps = await applyView(view, { + viewer, + dataManager: fakeLayers(new Set()), + run, + }); + assert.deepEqual( + calls.map(([name]) => name), + [ + 'set_map_stack', + 'stop_tracking', + 'clear_annotations', + 'show_street_level', + ], + ); + assert.deepEqual(calls.at(-1)[1], { + latitude: 38.5799, + longitude: -121.4937, + provider: 'google', + fly: true, + }); + assert.equal(viewer.flights.length, 0, 'the action frames the camera'); + assert.ok(steps.every((step) => step.ok)); + assert.equal(steps.at(-1).step, 'show_street_level'); +}); + +test('a link to street-level photos opens them after it restores', async () => { + const { calls, run } = recorder(); + const toasts = []; + const panels = []; + installViews({ + shell: { + initialRestorePromise: Promise.resolve(), + _showToast: (text) => toasts.push(text), + setPanelCollapsed: (...args) => panels.push(args), + }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + location: { + search: '', + hash: '#v=2&lat=1&lon=2&sl=mapillary,38.58000,-121.49000', + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(calls, [ + [ + 'show_street_level', + { + latitude: 38.58, + longitude: -121.49, + provider: 'mapillary', + fly: true, + autoOpen: false, + }, + ], + ]); + assert.deepEqual( + [toasts, panels], + [[], []], + 'an opened photo needs no toast or panel nudge', + ); + // A failure (no key) is not an armed state either. + const failed = recorder({ + show_street_level: () => ({ + ok: false, + stage: 'key', + error: 'needs its key', + }), + }); + installViews({ + shell: { + initialRestorePromise: Promise.resolve(), + _showToast: (text) => toasts.push(text), + setPanelCollapsed: (...args) => panels.push(args), + }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run: failed.run, + location: { + search: '', + hash: '#v=2&lat=1&lon=2&sl=google,1.00000,2.00000', + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual([toasts, panels], [[], []]); +}); + +test('a link that follows an entity keeps the camera: its photos are not opened', async () => { + const { calls, run } = recorder(); + const params = viewToParams( + createView({ + camera: { lat: 1, lon: 2 }, + follow: { kind: 'satellite', id: '25544' }, + }), + ); + params.set('sl', 'mapillary,1.00000,2.00000'); + installViews({ + shell: { initialRestorePromise: Promise.resolve() }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + location: { search: '', hash: `#${params}` }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(calls, []); +}); + +test('a failing link step does not take the next one with it', async () => { + const calls = []; + const run = async (name, args) => { + calls.push(name); + if (name === 'annotate_map') throw new Error('marks failed'); + return { ok: true, armed: false, ...args }; + }; + const an = encodeURIComponent( + JSON.stringify([{ type: 'pin', target: 'Austin' }]), + ); + installViews({ + shell: { initialRestorePromise: Promise.resolve() }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + location: { + search: '', + hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`, + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(calls, ['annotate_map', 'show_street_level']); +}); + +test('a plain link never auto-loads a billed photo: it arms the panel and says so; an embedded view opens', async () => { + const toasts = []; + const panels = []; + const { calls, run } = recorder({ + show_street_level: () => ({ ok: true, armed: true }), + }); + installViews({ + shell: { + initialRestorePromise: Promise.resolve(), + _showToast: (text) => toasts.push(text), + setPanelCollapsed: (id, collapsed, options) => + panels.push([id, collapsed, options]), + }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + location: { + search: '', + hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372', + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(calls[0][1].autoOpen, false); + assert.deepEqual(toasts, [ + 'Street View is ready: press OPEN GOOGLE STREET VIEW to load it', + ]); + assert.deepEqual( + panels, + [['street-level-panel', false, { explicit: true }]], + 'the panel opens so OPEN is in view', + ); + // Adding ?embed=1 to a published link does not lift the gate: the page is + // still a plain top-level load. + const framedToasts = []; + const framedPanels = []; + const framed = recorder({ + show_street_level: () => ({ ok: true, armed: true }), + }); + const windowRef = new EventTarget(); + windowRef.document = { body: { classList: { add() {} } } }; + windowRef.parent = { postMessage() {} }; + installViews({ + shell: { + initialRestorePromise: Promise.resolve(), + setCleanView() {}, + _showToast: (text) => framedToasts.push(text), + setPanelCollapsed: (...args) => framedPanels.push(args), + }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run: framed.run, + location: { + search: '?embed=1', + hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372', + }, + windowRef, + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(framed.calls[0][1].autoOpen, false); + // In a frame the rail that holds OPEN is hidden: no nudge, no toast. + assert.deepEqual([framedToasts, framedPanels], [[], []]); + // Only the panel page's own link (inline embed) is the client's request. + globalThis.GEV_EMBED_INLINE = true; + try { + const inline = recorder({ show_street_level: () => ({ ok: true }) }); + const inlineWindow = new EventTarget(); + inlineWindow.document = { body: { classList: { add() {} } } }; + inlineWindow.postMessage = () => {}; + // The inline panel keeps rendering from a timer; give it inert clocks. + Object.assign(inlineWindow, { + requestAnimationFrame: () => 1, + cancelAnimationFrame() {}, + setTimeout: () => 1, + clearTimeout() {}, + setInterval: () => 1, + clearInterval() {}, + }); + const stop = installViews({ + shell: { initialRestorePromise: Promise.resolve(), setCleanView() {} }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run: inline.run, + location: { + search: '', + hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372', + }, + windowRef: inlineWindow, + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal('autoOpen' in inline.calls[0][1], false); + stop(); + } finally { + delete globalThis.GEV_EMBED_INLINE; + } + // applyView (a posted view) opens too. + const posted = recorder(); + await applyView( + createView({ + camera: { lat: 1, lon: 2 }, + street_level: { provider: 'google' }, + }), + { + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run: posted.run, + }, + ); + assert.equal('autoOpen' in posted.calls.at(-1)[1], false); +}); + +test('a view whose photos fail before any flight still gets its camera', async () => { + const viewer = fakeViewer(); + const { calls, run } = recorder({ + show_street_level: () => ({ + ok: false, + stage: 'key', + error: 'needs its key', + }), + }); + const view = createView({ + camera: { lat: 1, lon: 2, altitude_m: 500 }, + street_level: { provider: 'google' }, + }); + const steps = await applyView(view, { + viewer, + dataManager: fakeLayers(new Set()), + run, + }); + assert.equal(viewer.flights.length, 1, 'the view flies itself'); + assert.deepEqual( + steps.map((step) => [step.step, step.ok]), + [ + ['set_map_stack', true], + ['stop_tracking', true], + ['clear_annotations', true], + ['show_street_level', false], + ['camera', true], + ], + ); + assert.ok(calls.some(([name]) => name === 'show_street_level')); + // A flight that happened (the action reports where) is not repeated. + const flown = fakeViewer(); + const { run: flownRun } = recorder({ + show_street_level: () => ({ + ok: false, + stage: 'imagery', + location: 'Somewhere', + error: 'no imagery', + }), + }); + await applyView(view, { + viewer: flown, + dataManager: fakeLayers(new Set()), + run: flownRun, + }); + assert.equal(flown.flights.length, 0); +}); + +test('a link runs its marks and photos in order, and posted views wait behind them', async () => { + const log = []; + let releaseStreetLevel = null; + let releaseMarks = null; + const run = async (name, args) => { + log.push(`start:${name}`); + if (name === 'annotate_map') + await new Promise((resolve) => { + releaseMarks = resolve; + }); + if (name === 'show_street_level') + await new Promise((resolve) => { + releaseStreetLevel = resolve; + }); + log.push(`end:${name}`); + return { + ok: true, + ...(name === 'show_street_level' ? { location: 'x' } : {}), + }; + }; + const windowRef = new EventTarget(); + windowRef.document = { body: { classList: { add() {} } } }; + // A framing page: answers go to it, not to the app's own window. + windowRef.parent = { + postMessage: (message) => log.push(`post:${message.type}`), + }; + const an = encodeURIComponent( + JSON.stringify([{ type: 'pin', target: 'Austin' }]), + ); + installViews({ + shell: { initialRestorePromise: Promise.resolve(), setCleanView() {} }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + location: { + search: '?embed=1', + hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`, + }, + windowRef, + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + // A view arrives while the link's photos are still opening. + windowRef.dispatchEvent( + Object.assign(new Event('message'), { + source: windowRef.parent, + origin: 'null', + data: { + type: 'gev:view', + id: 'v1', + view: { camera: { lat: 3, lon: 4 } }, + }, + }), + ); + await new Promise((resolve) => setTimeout(resolve, 0)); + const actions = () => log.filter((entry) => !entry.startsWith('post:')); + assert.deepEqual( + actions(), + ['start:annotate_map'], + 'the photos wait for the marks', + ); + releaseMarks(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(actions().slice(0, 3), [ + 'start:annotate_map', + 'end:annotate_map', + 'start:show_street_level', + ]); + assert.ok( + !log.includes('start:stop_tracking'), + 'the posted view waits for the link', + ); + releaseStreetLevel(); + for ( + let tick = 0; + tick < 20 && !log.includes('post:gev:view-applied'); + tick++ + ) + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal( + log.indexOf('end:show_street_level') < log.indexOf('start:stop_tracking'), + true, + ); + assert.ok(log.includes('post:gev:view-applied'), log.join(' ')); +}); + +test('an aborted install never runs the link photos', async () => { + const { calls, run } = recorder(); + const controller = new AbortController(); + controller.abort(); + installViews({ + shell: { initialRestorePromise: Promise.resolve() }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + signal: controller.signal, + location: { + search: '', + hash: '#v=2&lat=1&lon=2&sl=google,1.00000,2.00000', + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(calls, []); +}); + +test('a passive link stands down when someone navigated while it waited behind its marks', async () => { + const calls = []; + let releaseMarks = null; + const shell = { + initialRestorePromise: Promise.resolve(), + _navigationGeneration: 7, + }; + const run = async (name) => { + calls.push(name); + if (name === 'annotate_map') + await new Promise((resolve) => { + releaseMarks = resolve; + }); + return { ok: true }; + }; + const an = encodeURIComponent( + JSON.stringify([{ type: 'pin', target: 'Austin' }]), + ); + installViews({ + shell, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run, + location: { + search: '', + hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`, + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(calls, ['annotate_map']); + shell._navigationGeneration = 8; // the recipient searched, or the assistant flew + releaseMarks(); + await new Promise((resolve) => setTimeout(resolve, 0)); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual( + calls, + ['annotate_map'], + 'the link did not fly over the newer navigation', + ); + + // Unchanged navigation: the photos open as before. + const quiet = []; + installViews({ + shell: { + initialRestorePromise: Promise.resolve(), + _navigationGeneration: 3, + }, + viewer: fakeViewer(), + dataManager: fakeLayers(new Set()), + run: async (name) => { + quiet.push(name); + return { ok: true }; + }, + location: { + search: '', + hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`, + }, + windowRef: new EventTarget(), + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(quiet, ['annotate_map', 'show_street_level']); +}); diff --git a/src/firstRunExperience.test.mjs b/src/firstRunExperience.test.mjs index 87229e5c5..6603382c4 100644 --- a/src/firstRunExperience.test.mjs +++ b/src/firstRunExperience.test.mjs @@ -982,8 +982,9 @@ test('the voice TOOL SCHEMA matches the pinned contract — the mission mapping // Canonical serialization pins every tool name, description, property and // ordering while allowing source formatting. Derived from the unchanged // release schema before formatting (the previous source-byte pin passed). + // show_street_level (street-level photos) is a later, separate tool. const legacyTools = structuredClone(GEV_REALTIME_TOOLS).filter( - (tool) => tool.name !== 'set_cyber_sonar', + (tool) => !['set_cyber_sonar', 'show_street_level'].includes(tool.name), ); const hudLayout = legacyTools.find((tool) => tool.name === 'set_hud') .parameters.properties.layout; diff --git a/src/i18n/locales/ar.json b/src/i18n/locales/ar.json index e2e05348d..faed66510 100644 --- a/src/i18n/locales/ar.json +++ b/src/i18n/locales/ar.json @@ -3222,5 +3222,13 @@ "Audio": "الصوت", "UTILITIES": "الأدوات", "Exit cockpit to fly to a drawing": "اخرج من القمرة للانتقال إلى رسم", - "Directions, drawing and your own files": "الاتجاهات والرسم وملفاتك الخاصة" + "Directions, drawing and your own files": "الاتجاهات والرسم وملفاتك الخاصة", + "Press OPEN to load the photo": "اضغط «فتح» لتحميل الصورة", + "Captured {month}": "التُقطت في {month}", + "this spot": "هذا الموضع", + "the view centre": "مركز العرض", + "{source} at {place}": "{source} في {place}", + "Street level at {place}": "مستوى الشارع في {place}", + "Switching map": "جارٍ تبديل الخريطة", + "Opening imagery": "جارٍ فتح الصور" } diff --git a/src/i18n/locales/de.json b/src/i18n/locales/de.json index 0abaa4ddf..d98a33930 100644 --- a/src/i18n/locales/de.json +++ b/src/i18n/locales/de.json @@ -2942,5 +2942,13 @@ "Audio": "Audio", "UTILITIES": "WERKZEUGE", "Exit cockpit to fly to a drawing": "Cockpit verlassen, um zu einer Zeichnung zu fliegen", - "Directions, drawing and your own files": "Route, Zeichnen und eigene Dateien" + "Directions, drawing and your own files": "Route, Zeichnen und eigene Dateien", + "Press OPEN to load the photo": "Auf ÖFFNEN klicken, um das Foto zu laden", + "Captured {month}": "Aufgenommen im {month}", + "this spot": "diese Stelle", + "the view centre": "die Bildmitte", + "{source} at {place}": "{source} bei {place}", + "Street level at {place}": "Straßenebene bei {place}", + "Switching map": "Karte wird gewechselt", + "Opening imagery": "Bilder werden geöffnet" } diff --git a/src/i18n/locales/es.json b/src/i18n/locales/es.json index 65997812d..8d530a7b4 100644 --- a/src/i18n/locales/es.json +++ b/src/i18n/locales/es.json @@ -906,6 +906,8 @@ "Bike Share": "Bicis compartidas", "Cameras": "Cámaras", "Street Level": "A pie de calle", + "Press OPEN to load the photo": "Pulse ABRIR para cargar la foto", + "Captured {month}": "Capturada en {month}", "Mapped ALPR Cameras": "Cámaras ALPR mapeadas", "Data Centers": "Centros de datos", "Active Fires": "Incendios activos", @@ -1745,6 +1747,10 @@ "Mark {targets}": "Marcar: {targets}", "Fly to {place}": "Volar a {place}", "Nearest aircraft to {place}": "Aeronave más cercana a {place}", + "{source} at {place}": "{source} en {place}", + "Street level at {place}": "A pie de calle en {place}", + "this spot": "este punto", + "the view centre": "el centro de la vista", "Read the view": "Consultar la vista", "Check view state": "Comprobar el estado de la vista", "Frame {target}": "Encuadrar {target}", @@ -1752,6 +1758,8 @@ "Tracing outline": "Trazando contorno", "Looking up place": "Buscando lugar", "Flying": "Volando", + "Switching map": "Cambiando de mapa", + "Opening imagery": "Abriendo imágenes", "Turning on layer": "Activando capa", "Loading aircraft": "Cargando aeronaves", "Picking nearest": "Eligiendo la más cercana", diff --git a/src/i18n/locales/fr.json b/src/i18n/locales/fr.json index 0d1460e1d..5e7457e76 100644 --- a/src/i18n/locales/fr.json +++ b/src/i18n/locales/fr.json @@ -3012,5 +3012,13 @@ "Audio": "Audio", "UTILITIES": "OUTILS", "Exit cockpit to fly to a drawing": "Quittez le cockpit pour rejoindre un dessin", - "Directions, drawing and your own files": "Itinéraire, dessin et vos propres fichiers" + "Directions, drawing and your own files": "Itinéraire, dessin et vos propres fichiers", + "Press OPEN to load the photo": "Appuyez sur OUVRIR pour charger la photo", + "Captured {month}": "Prise en {month}", + "this spot": "cet endroit", + "the view centre": "le centre de la vue", + "{source} at {place}": "{source} à {place}", + "Street level at {place}": "Niveau rue à {place}", + "Switching map": "Changement de carte", + "Opening imagery": "Ouverture des images" } diff --git a/src/i18n/locales/it.json b/src/i18n/locales/it.json index d460440e2..cf642ae4e 100644 --- a/src/i18n/locales/it.json +++ b/src/i18n/locales/it.json @@ -3012,5 +3012,13 @@ "Audio": "Audio", "UTILITIES": "STRUMENTI", "Exit cockpit to fly to a drawing": "Esci dal cockpit per volare a un disegno", - "Directions, drawing and your own files": "Indicazioni, disegno e i tuoi file" + "Directions, drawing and your own files": "Indicazioni, disegno e i tuoi file", + "Press OPEN to load the photo": "Premi APRI per caricare la foto", + "Captured {month}": "Scattata a {month}", + "this spot": "questo punto", + "the view centre": "il centro della vista", + "{source} at {place}": "{source} a {place}", + "Street level at {place}": "Vista stradale a {place}", + "Switching map": "Cambio mappa", + "Opening imagery": "Apertura delle immagini" } diff --git a/src/i18n/locales/ja.json b/src/i18n/locales/ja.json index 237c55f6c..e41b78411 100644 --- a/src/i18n/locales/ja.json +++ b/src/i18n/locales/ja.json @@ -2872,5 +2872,13 @@ "Audio": "オーディオ", "UTILITIES": "ユーティリティ", "Exit cockpit to fly to a drawing": "描画の位置へ移動するにはコックピットを終了してください", - "Directions, drawing and your own files": "ルート案内、描画、自分のファイル" + "Directions, drawing and your own files": "ルート案内、描画、自分のファイル", + "Press OPEN to load the photo": "「開く」を押して写真を読み込む", + "Captured {month}": "{month}撮影", + "this spot": "この地点", + "the view centre": "表示中心", + "{source} at {place}": "{place}の{source}", + "Street level at {place}": "{place}のストリートレベル", + "Switching map": "地図を切り替え中", + "Opening imagery": "画像を開いています" } diff --git a/src/i18n/locales/pt.json b/src/i18n/locales/pt.json index 25f53f162..e6ccfd9f8 100644 --- a/src/i18n/locales/pt.json +++ b/src/i18n/locales/pt.json @@ -3012,5 +3012,13 @@ "Audio": "Áudio", "UTILITIES": "UTILITÁRIOS", "Exit cockpit to fly to a drawing": "Saia da cabine para voar até um desenho", - "Directions, drawing and your own files": "Rotas, desenho e seus próprios arquivos" + "Directions, drawing and your own files": "Rotas, desenho e seus próprios arquivos", + "Press OPEN to load the photo": "Clique em ABRIR para carregar a foto", + "Captured {month}": "Capturada em {month}", + "this spot": "este ponto", + "the view centre": "o centro da vista", + "{source} at {place}": "{source} em {place}", + "Street level at {place}": "Nível da rua em {place}", + "Switching map": "Mudando de mapa", + "Opening imagery": "Abrindo imagens" } diff --git a/src/i18n/locales/ru.json b/src/i18n/locales/ru.json index f3217b079..aa5f47c26 100644 --- a/src/i18n/locales/ru.json +++ b/src/i18n/locales/ru.json @@ -915,6 +915,8 @@ "Bike Share": "Прокат велосипедов", "Cameras": "Камеры", "Street Level": "Уличные снимки", + "Press OPEN to load the photo": "Нажмите ОТКРЫТЬ, чтобы загрузить снимок", + "Captured {month}": "Снято: {month}", "Mapped ALPR Cameras": "Камеры ALPR на карте", "Data Centers": "Дата-центры", "Active Fires": "Активные пожары", @@ -1778,6 +1780,10 @@ "Mark {targets}": "Отметить {targets}", "Fly to {place}": "Перелёт: {place}", "Nearest aircraft to {place}": "Ближайший самолёт: {place}", + "{source} at {place}": "{source}: {place}", + "Street level at {place}": "Уличные снимки: {place}", + "this spot": "эта точка", + "the view centre": "центр вида", "Read the view": "Осмотреть видимую область", "Check view state": "Проверить состояние вида", "Frame {target}": "Показать в кадре: {target}", @@ -1785,6 +1791,8 @@ "Tracing outline": "Построение контура", "Looking up place": "Поиск места", "Flying": "Перелёт", + "Switching map": "Смена карты", + "Opening imagery": "Открытие снимков", "Turning on layer": "Включение слоя", "Loading aircraft": "Загрузка самолётов", "Picking nearest": "Выбор ближайших", diff --git a/src/i18n/locales/uk.json b/src/i18n/locales/uk.json index 720b0a7bd..2fbc3db48 100644 --- a/src/i18n/locales/uk.json +++ b/src/i18n/locales/uk.json @@ -3082,5 +3082,13 @@ "Audio": "Аудіо", "UTILITIES": "ІНСТРУМЕНТИ", "Exit cockpit to fly to a drawing": "Вийдіть із кабіни, щоб перелетіти до малюнка", - "Directions, drawing and your own files": "Маршрути, малювання та ваші файли" + "Directions, drawing and your own files": "Маршрути, малювання та ваші файли", + "Press OPEN to load the photo": "Натисніть «ВІДКРИТИ», щоб завантажити фото", + "Captured {month}": "Знято: {month}", + "this spot": "це місце", + "the view centre": "центр карти", + "{source} at {place}": "{source}: {place}", + "Street level at {place}": "Вуличні знімки: {place}", + "Switching map": "Перемикання карти", + "Opening imagery": "Відкриття знімків" } diff --git a/src/i18n/locales/zh-CN.json b/src/i18n/locales/zh-CN.json index b9d8799c4..7978bff4d 100644 --- a/src/i18n/locales/zh-CN.json +++ b/src/i18n/locales/zh-CN.json @@ -1737,6 +1737,8 @@ "Bike Share": "共享单车", "Cameras": "摄像头", "Street Level": "街景", + "Press OPEN to load the photo": "点击“打开”加载照片", + "Captured {month}": "拍摄于 {month}", "Mapped ALPR Cameras": "地图标注 ALPR 摄像头", "Data Centers": "数据中心", "Active Fires": "活跃火点", @@ -2416,6 +2418,10 @@ "Fly to {place}": "飞往 {place}", "Fly to Location": "飞往地点", "Nearest aircraft to {place}": "距离 {place} 最近的飞机", + "{source} at {place}": "{place} 的 {source}", + "Street level at {place}": "{place} 的街景", + "this spot": "此处", + "the view centre": "视图中心", "Nearest aircraft to Here": "距离此处最近的飞机", "Layer on": "图层已开启", "Layer off": "图层已关闭", @@ -2428,6 +2434,8 @@ "Tracing outline": "正在绘制轮廓", "Looking up place": "正在查询地点", "Flying": "飞行中", + "Switching map": "正在切换地图", + "Opening imagery": "正在打开影像", "Turning on layer": "正在开启图层", "Loading aircraft": "正在加载飞机", "Picking nearest": "正在选取最近目标", diff --git a/src/i18n/locales/zh-Hant.json b/src/i18n/locales/zh-Hant.json index 06ce0fcaa..a0760dc1b 100644 --- a/src/i18n/locales/zh-Hant.json +++ b/src/i18n/locales/zh-Hant.json @@ -2872,5 +2872,13 @@ "Right panels": "右側面板", "UTILITIES": "工具", "Exit cockpit to fly to a drawing": "請先退出座艙,再飛往繪製位置", - "Directions, drawing and your own files": "路線導航、繪製和你自己的文件" + "Directions, drawing and your own files": "路線導航、繪製和你自己的文件", + "Press OPEN to load the photo": "點擊“打開”載入照片", + "Captured {month}": "拍攝於 {month}", + "this spot": "此處", + "the view centre": "視圖中心", + "{source} at {place}": "{place} 的 {source}", + "Street level at {place}": "{place} 的街景", + "Switching map": "正在切換地圖", + "Opening imagery": "正在打開影像" } diff --git a/src/layers/streetLevel/index.js b/src/layers/streetLevel/index.js index 1f7b0f7f0..409fdf384 100644 --- a/src/layers/streetLevel/index.js +++ b/src/layers/streetLevel/index.js @@ -21,6 +21,7 @@ import { AIM_AT_GROUND_MESSAGE, keyStatusLabel, NEAREST_RADIUS_M, + NEAREST_RADIUS_MAX_M, POSITION_PICK_ID, STREET_LEVEL_LAYER_ID, } from './policy.js'; @@ -157,6 +158,12 @@ export function createStreetLevelLayer({ const owner = state.providers.get(state.street.providerId); if (state.street.open && owner && !providerAvailable(owner)) layer.closeViewer(); + // An armed lookup for a source whose map left dies too. + const armedFor = state.street.armed?.providerIds?.map((id) => + state.providers.get(id), + ); + if (armedFor?.some((entry) => entry && !providerAvailable(entry))) + state.street.armed = null; for (const entry of state.providers.values()) { if (!entry.def.requiresMapStack) continue; if (state.enabled && entry.on && providerAvailable(entry)) @@ -275,8 +282,14 @@ export function createStreetLevelLayer({ } /** `frame: false` keeps the camera where it is (the user picked the spot). */ - function openImage(providerId, imageId, { frame = true } = {}) { + function openImage( + providerId, + imageId, + { frame = true, signal = null } = {}, + ) { + if (signal?.aborted) return Promise.resolve(false); abortNearest(); + state.street.armed = null; const entry = state.providers.get(providerId); if (entry && !providerAvailable(entry)) { state.street.loading = false; @@ -284,7 +297,7 @@ export function createStreetLevelLayer({ notify(); return Promise.resolve(false); } - return parts.viewerHost.open(providerId, imageId, { frame }); + return parts.viewerHost.open(providerId, imageId, { frame, signal }); } function unavailableReason(entry) { @@ -330,6 +343,9 @@ export function createStreetLevelLayer({ function switchProvider(entry, on) { if (entry.on === on) return; entry.on = on; + // A lookup armed for this source dies with it. + if (!on && state.street.armed?.providerIds?.includes(entry.def.id)) + state.street.armed = null; if (!state.enabled) return; if (on) activate(entry); else deactivate(entry); @@ -466,6 +482,37 @@ export function createStreetLevelLayer({ }, /** Registered providers, in chip order. */ providerIds: defs.map((def) => def.id), + /** The map stack a provider needs (`requiresMapStack`), or null for any. */ + requiredMapStack(providerId) { + const def = defs.find((entry) => entry.id === providerId); + return def?.requiresMapStack || null; + }, + /** The coverage lines a provider draws now (see `coverageLines`). */ + coverageLines(providerId) { + return state.providers.get(providerId)?.instance.coverageLines?.() ?? []; + }, + /** + * A provider's name and key state on any map (the panel lists a provider + * only on the map it needs). `configured` is null until its status call + * answers; `keyRequired` covers a missing or refused key. + */ + providerStatus(providerId) { + const entry = state.providers.get(providerId); + if (!entry) return null; + const stats = entry.instance.coverageStats(); + return { + id: entry.def.id, + name: entry.def.name, + on: entry.on, + configured: entry.status ? entry.status.configured === true : null, + keyRequired: stats.keyRequired === true || stats.keyRejected === true, + keyRejected: stats.keyRejected === true, + groundClick: entry.def.groundClick === true, + billed: entry.def.billedPerImage === true, + available: providerAvailable(entry), + error: stats.error || null, + }; + }, getSourceAvailability() { const available = defs.length > 0; return { @@ -612,13 +659,34 @@ export function createStreetLevelLayer({ /** * Open the nearest image of the active providers (or `providerIds`) near * a point, else the view centre, or with `aim: 'screen-centre'` the ground - * under the screen centre (sky: nothing). Providers without + * under the screen centre (sky: nothing). `radiusM` widens the search for + * providers that can (a geocoded landmark is rarely within 50 m of a + * street); clicks keep the default. A caller's `signal` retires the + * lookup: nothing opens after it fires. Providers without * `nearestImage` are skipped. */ async openNearest( point, - { providerIds = null, frame = true, aim = 'view' } = {}, + { + providerIds = null, + frame = true, + aim = 'view', + radiusM = null, + signal: callerSignal = null, + } = {}, ) { + if (callerSignal?.aborted) return false; + // An OPEN press after a link armed a lookup opens what the link asked; + // any other lookup is a new request and drops the armed one. + const armed = state.street.armed; + state.street.armed = null; + if (!point && aim === 'screen-centre' && armed) { + point = armed.point; + radiusM ??= armed.radiusM; + } + const radius = Number.isFinite(radiusM) + ? Math.min(NEAREST_RADIUS_MAX_M, Math.max(1, radiusM)) + : NEAREST_RADIUS_M; let view = point; if (!view && aim === 'screen-centre') { view = screenCentreGround(); @@ -637,6 +705,21 @@ export function createStreetLevelLayer({ const lookup = new AbortController(); nearestLookup = lookup; const { signal } = lookup; + const onCallerAbort = () => lookup.abort(); + callerSignal?.addEventListener('abort', onCallerAbort, { once: true }); + const retired = () => { + callerSignal?.removeEventListener('abort', onCallerAbort); + // Only the lookup that still owns the panel may touch it: a newer one + // has its own spinner and error. + if (nearestLookup !== lookup) return false; + nearestLookup = null; + // The caller moved on: the panel shows nothing of this lookup. + if (state.street.loading && !state.street.open) { + state.street.loading = false; + notify(); + } + return false; + }; state.street.loading = true; state.street.error = null; notify(); @@ -651,23 +734,35 @@ export function createStreetLevelLayer({ try { imageId = await entry.instance.nearestImage( { lat: view.lat, lon: view.lon }, - { signal }, + radius === NEAREST_RADIUS_M + ? { signal } + : { signal, radiusM: radius }, ); } catch (error) { + if (callerSignal?.aborted) return retired(); // Overtaken: whatever replaced it owns the panel, errors included. if (signal.aborted) return false; // One provider failing (no key, offline) must not hide the others. lastError = error; continue; } + if (callerSignal?.aborted) return retired(); if (signal.aborted) return false; if (!state.enabled) break; // Switched off, or its map left, while it looked: try the next. if (!imageId || !entry.on || !providerAvailable(entry)) continue; nearestLookup = null; - return openImage(entry.def.id, imageId, { frame }); + callerSignal?.removeEventListener('abort', onCallerAbort); + // The caller's signal still counts while the viewer mounts and loads. + return openImage(entry.def.id, imageId, { + frame, + signal: callerSignal, + }); } - if (nearestLookup === lookup) nearestLookup = null; + callerSignal?.removeEventListener('abort', onCallerAbort); + // Replaced while it looked: the panel is the newer lookup's. + if (nearestLookup !== lookup) return false; + nearestLookup = null; // Switched off while it looked: nothing to open, nothing to report. const stillOn = entries.some( (entry) => entry.on && providerAvailable(entry), @@ -679,11 +774,13 @@ export function createStreetLevelLayer({ lastError?.message || (point ? t('No street-level imagery within {radius} m of that point', { - radius: NEAREST_RADIUS_M, + radius, }) : t( 'No street-level imagery within {radius} m of the view centre', - { radius: NEAREST_RADIUS_M }, + { + radius, + }, )); state.street.loading = false; notify(); @@ -695,6 +792,23 @@ export function createStreetLevelLayer({ parts.viewerHost.close(); parts.clearSequences(); }, + /** + * Prepare a nearest lookup for the next OPEN press without running it: a + * link to a source that bills every load stops here. The press opens at + * this point with this radius; any other lookup, open or close, the + * source switched off, or its map leaving clears it. + */ + armNearest(point, { providerIds = null, radiusM = null } = {}) { + state.street.armed = { + point: + Number.isFinite(point?.lat) && Number.isFinite(point?.lon) + ? { lat: point.lat, lon: point.lon } + : null, + providerIds: Array.isArray(providerIds) ? [...providerIds] : null, + radiusM: Number.isFinite(radiusM) ? radiusM : null, + }; + notify(); + }, setViewerRenderMode: (mode) => parts.viewerHost.setRenderMode(mode), resizeViewer: () => parts.viewerHost.resize(), }; diff --git a/src/layers/streetLevel/index.test.mjs b/src/layers/streetLevel/index.test.mjs index 6c79a52ca..a82dc53c5 100644 --- a/src/layers/streetLevel/index.test.mjs +++ b/src/layers/streetLevel/index.test.mjs @@ -12,6 +12,9 @@ import { fakeStreetLevelProvider as fakeProvider, } from '../../testSupport/streetLevelFakes.mjs'; import { setLocale } from '../../i18n/index.js'; +// Loaded up front: it brings Vue, which reads `document` once when it loads, +// and several tests here stand a bare fake document in for the page. +import { createGevActionRunner } from '../../voice/gevActions.js'; /** * A layer over `source` and `photoViewer`, initialised on a stand-in viewer @@ -1118,6 +1121,674 @@ test('switching source by params clears a failed lookup’s error; reapplying th assert.equal(second.getUIState().street.error, kept); }); +test('openNearest searches wider on request, never past the cap, and says how far it looked', async (t) => { + const { layer, fake } = await withStreetView(t, { nearest: () => null }); + layer.setProviderEnabled('google', true); + await settle(); + await layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false, radiusM: 250 }, + ); + assert.equal(fake.lookups.at(-1).radius, 250); + assert.equal( + layer.getUIState().street.error, + 'No street-level imagery within 250 m of that point', + ); + await layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false, radiusM: 5000 }, + ); + assert.equal(fake.lookups.at(-1).radius, 250, 'capped'); + await layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false }, + ); + assert.equal(fake.lookups.at(-1).radius, 50, 'a click keeps the default'); +}); + +test('a caller’s signal retires a nearest lookup: nothing opens after it fires', async (t) => { + let answer = null; + const { layer, fake } = await withStreetView(t, { + nearest: () => 'pano-a', + }); + const lib = fake.library; + const service = lib.StreetViewService.prototype.getPanorama; + lib.StreetViewService.prototype.getPanorama = function (request) { + return new Promise((resolve, reject) => { + answer = () => service.call(this, request).then(resolve, reject); + }); + }; + layer.setProviderEnabled('google', true); + await settle(); + const controller = new AbortController(); + const pending = layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false, signal: controller.signal }, + ); + await settle(); + controller.abort(); + answer?.(); + assert.equal(await pending, false); + const street = layer.getUIState().street; + assert.equal(street.open, false, 'the retired lookup opened nothing'); + assert.equal(street.loading, false, 'no stuck spinner'); + assert.equal(street.error, null, 'and reports no missing imagery'); + const refused = await layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false, signal: controller.signal }, + ); + assert.equal(refused, false, 'an already-aborted signal asks nothing'); +}); + +test('providerStatus reports name, key state and billing on any map', async (t) => { + const { layer } = await withStreetView(t, { mapStack: 'osm' }); + const google = layer.providerStatus('google'); + assert.equal(google.name, 'Google Street View'); + assert.equal(google.billed, true, 'every Street View load is billed'); + assert.equal(google.available, false, 'not on this map'); + assert.equal(google.keyRequired, false); + assert.equal(layer.providerStatus('mapillary').billed, false); + assert.equal(layer.providerStatus('nope'), null); + assert.equal(layer.requiredMapStack('google'), 'photoreal'); + assert.equal(layer.requiredMapStack('mapillary'), null); +}); + +test('an armed lookup asks nothing until the OPEN press, then opens at its point and radius once', async (t) => { + const { layer, fake, mapillaryLookups } = await withStreetView(t); + layer.setProviderEnabled('google', true); + await settle(); + const before = fake.lookups.length; + layer.armNearest( + { lat: 38.57986, lon: -121.49372 }, + { providerIds: ['google'], radiusM: 250 }, + ); + await settle(); + await new Promise((resolve) => setTimeout(resolve, 20)); + // The click gate: arming is free. No provider was asked, nothing opened. + assert.equal( + fake.lookups.length, + before, + 'no Google lookup before the press', + ); + assert.deepEqual(mapillaryLookups, [], 'no Mapillary lookup either'); + assert.equal(layer.getUIState().street.open, false); + assert.equal(layer.getUIState().street.loading, false); + assert.deepEqual(layer.getUIState().street.armed, { + point: { lat: 38.57986, lon: -121.49372 }, + providerIds: ['google'], + radiusM: 250, + }); + // The OPEN button: no point, aimed at the screen centre. + const opened = await layer.openNearest(undefined, { + providerIds: ['google'], + frame: false, + aim: 'screen-centre', + }); + assert.equal(opened, true); + assert.equal( + fake.lookups.length, + before + 1, + 'exactly one lookup for the press', + ); + const request = fake.lookups.at(-1); + assert.deepEqual( + request.location, + { lat: 38.57986, lng: -121.49372 }, + 'the armed point, not the screen centre', + ); + assert.equal(request.radius, 250); + assert.equal(layer.getUIState().street.armed, null, 'used once'); + layer.armNearest({ lat: 1, lon: 2 }, { providerIds: ['google'] }); + layer.closeViewer(); + assert.equal(layer.getUIState().street.armed, null, 'closing clears it'); + layer.armNearest({ lat: 1, lon: 2 }, { providerIds: ['google'] }); + await layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false }, + ); + assert.deepEqual( + fake.lookups.at(-1).location, + { lat: 38.58, lng: -121.49 }, + 'an explicit point wins', + ); + assert.equal(layer.getUIState().street.armed, null); + // Only the OPEN press consumes the armed point: "street view here" after a + // pan is a new request at the view centre, and it drops the armed one. + layer.closeViewer(); + layer.armNearest( + { lat: 10, lon: 20 }, + { providerIds: ['google'], radiusM: 250 }, + ); + await layer.openNearest(null, { providerIds: ['google'], frame: false }); + const centre = fake.lookups.at(-1); + assert.notDeepEqual( + centre.location, + { lat: 10, lng: 20 }, + 'the view centre, not the armed point', + ); + assert.equal(centre.radius, 50, 'and the default radius'); + assert.equal(layer.getUIState().street.armed, null, 'dropped'); + // Opening any image, or switching the layer off, clears it too. + layer.armNearest({ lat: 10, lon: 20 }, { providerIds: ['google'] }); + await layer.openImage('google', 'pano-a'); + assert.equal(layer.getUIState().street.armed, null, 'cleared by an open'); + layer.armNearest({ lat: 10, lon: 20 }, { providerIds: ['google'] }); + layer.disable(); + assert.equal(layer.getUIState().street.armed, null, 'cleared by disable'); +}); + +test('a caller’s signal: an aborted one asks nothing, and a provider that ignores the abort still opens nothing', async (t) => { + const { layer, fake } = await withStreetView(t); + layer.setProviderEnabled('google', true); + await settle(); + const before = fake.lookups.length; + const done = new AbortController(); + done.abort(); + assert.equal( + await layer.openNearest( + { lat: 38.58, lon: -121.49 }, + { providerIds: ['google'], frame: false, signal: done.signal }, + ), + false, + ); + assert.equal( + fake.lookups.length, + before, + 'an already-aborted signal asks nothing', + ); +}); + +test('a provider that ignores the abort and answers late still opens nothing', async (t) => { + let seen = null; + let release = null; + const slow = fakeProvider({ + nearestImage: async (_point, { signal } = {}) => { + seen = signal; + await new Promise((resolve) => { + release = resolve; + }); + return 'late-image'; + }, + }); + const { layer: ignoring } = await enabledLayer(t, [slow]); + const { fakeHost } = + await import('../../testSupport/googleStreetViewFakes.mjs'); + ignoring.attachViewerHost(fakeHost()); + const controller = new AbortController(); + const pending = ignoring.openNearest( + { lat: 1, lon: 2 }, + { frame: false, signal: controller.signal }, + ); + await settle(); + controller.abort(); + assert.equal(seen?.aborted, true, 'the provider sees the abort'); + release(); + assert.equal(await pending, false); + const street = ignoring.getUIState().street; + assert.equal(street.open, false, 'the late answer opened nothing'); + assert.equal(street.loading, false, 'no stuck spinner'); + assert.equal(street.error, null); +}); + +test('providerStatus reports a keyless source on any map', async (t) => { + const keyless = await withStreetView(t, { + mapStack: 'osm', + mapillaryKeyless: true, + }); + await settle(); + const mapillary = keyless.layer.providerStatus('mapillary'); + assert.equal(mapillary.configured, false); + assert.equal(mapillary.keyRequired, true); + assert.equal(mapillary.keyRejected, false); +}); + +test('providerStatus reports a refused key on any map', async (t) => { + const refused = await withStreetView(t, { mapStack: 'osm' }); + refused.loader.refuseKey(); + await settle(); + const google = refused.layer.providerStatus('google'); + assert.equal( + google.keyRequired, + true, + 'a refused key gates like a missing one', + ); + assert.equal(google.keyRejected, true); + assert.equal(google.available, false, 'still not on this map'); +}); + +test('an armed lookup dies with its source: switched off, or its map gone', async (t) => { + const { layer, stack } = await withStreetView(t); + layer.setProviderEnabled('google', true); + await settle(); + layer.armNearest( + { lat: 1, lon: 2 }, + { providerIds: ['google'], radiusM: 250 }, + ); + layer.setProviderEnabled('google', false); + assert.equal( + layer.getUIState().street.armed, + null, + 'switching the source off clears it', + ); + layer.setProviderEnabled('google', true); + layer.armNearest( + { lat: 1, lon: 2 }, + { providerIds: ['google'], radiusM: 250 }, + ); + // Lighting Mapillary instead (one source at a time) turns Google off too. + layer.setProviderEnabled('mapillary', true); + assert.equal( + layer.getUIState().street.armed, + null, + 'another source lit clears it', + ); + layer.setProviderEnabled('google', true); + layer.armNearest( + { lat: 1, lon: 2 }, + { providerIds: ['google'], radiusM: 250 }, + ); + // A map-stack notify that keeps Google 3D (a silent re-settle) keeps it. + stack.switchTo('photoreal'); + assert.notEqual( + layer.getUIState().street.armed, + null, + 'a notify on the same map keeps it', + ); + stack.switchTo('osm'); + assert.equal( + layer.getUIState().street.armed, + null, + 'leaving Google 3D clears it', + ); + // Arming notifies, so a panel can show the state. + let notified = 0; + const unsubscribe = layer.subscribe(() => notified++); + stack.switchTo('photoreal'); + await settle(); + const before = notified; + layer.armNearest({ lat: 1, lon: 2 }, { providerIds: ['google'] }); + await settle(); + assert.ok(notified > before, 'subscribers hear about the armed lookup'); + unsubscribe(); +}); + +test('a lookup armed with no point opens under the middle of the screen on the press', async (t) => { + const { createStreetLevelLayer: create } = await import('./index.js'); + const scenePick = await import('../../data/scenePick.js'); + const lookups = []; + const provider = fakeProvider({ + nearestImage: async (point, options) => { + lookups.push({ ...point, radiusM: options?.radiusM ?? null }); + return null; + }, + }); + const saved = globalThis.document; + const drawing = new Proxy({}, { get: () => () => ({ addColorStop() {} }) }); + globalThis.document = Object.assign(new EventTarget(), { + createElement: () => ({ getContext: () => drawing }), + }); + const viewer = fakeViewer(); + const centre = Cesium.Cartesian3.fromDegrees(-121.4937, 38.5799, 0); + Object.assign(viewer.scene, { + pickPositionSupported: true, + pickPosition: (at) => (at.x === 500 && at.y === 300 ? centre : undefined), + }); + viewer.scene.canvas.clientWidth = 1000; + viewer.scene.canvas.clientHeight = 600; + // Registered before the layer exists, so a throwing create() still + // restores the document; destroy() runs first, while the stub is in place. + let layer = null; + t.after(() => { + layer?.destroy(); + globalThis.document = saved; + }); + layer = create({ providers: [provider], services: { scenePick } }); + layer.init(viewer); + layer.enable(viewer); + layer.armNearest(null, { providerIds: [provider.id], radiusM: 250 }); + assert.equal( + layer.getUIState().street.armed.point, + null, + 'no point stored, not 0,0', + ); + await layer.openNearest(undefined, { aim: 'screen-centre' }); + assert.equal(lookups.length, 1); + assert.ok( + Math.abs(lookups[0].lat - 38.5799) < 1e-9, + 'the screen centre, not Null Island', + ); + assert.ok(Math.abs(lookups[0].lon - -121.4937) < 1e-9); + assert.equal(lookups[0].radiusM, 250, 'with the armed radius'); +}); + +test('the action over the real layer: a billed link arms without a lookup, and one press makes exactly one at the point', async (t) => { + const { layer, fake, viewer } = await withStreetView(t); + // The runner wires pointer tracking to the clock and camera. + viewer.clock ??= { onTick: { addEventListener: () => () => {} } }; + viewer.camera.moveEnd ??= { addEventListener() {} }; + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + isEnabled: () => true, + getLayerParams: () => layer.getParams(), + setLayerParams: (_id, params) => { + layer.setParams(params); + return true; + }, + getAll: () => [{ id: 'street-level', name: 'Street Level', enabled: true }], + }; + const styleManager = { + mapStackController: { getActiveId: () => 'photoreal' }, + setMapStack: async () => ({ ok: true, activeStack: 'photoreal' }), + }; + const runner = createGevActionRunner({ viewer, styleManager, dataManager }); + const before = fake.lookups.length; + const armed = await runner('show_street_level', { + latitude: 38.57986, + longitude: -121.49372, + provider: 'google', + fly: false, + autoOpen: false, + }); + assert.equal(armed.ok, true, armed.error); + assert.equal(armed.armed, true); + assert.equal(armed.providerName, 'Google Street View'); + await settle(); + assert.equal(fake.lookups.length, before, 'arming made no Google request'); + assert.deepEqual( + layer + .getUIState() + .providers.filter((p) => p.on) + .map((p) => p.id), + ['google'], + ); + assert.equal(layer.getUIState().street.open, false); + // The OPEN button. + assert.equal( + await layer.openNearest(undefined, { + providerIds: ['google'], + frame: false, + aim: 'screen-centre', + }), + true, + ); + assert.equal(fake.lookups.length, before + 1, 'one request for the press'); + assert.deepEqual(fake.lookups.at(-1).location, { + lat: 38.57986, + lng: -121.49372, + }); + assert.equal(fake.lookups.at(-1).radius, 250); + assert.equal(layer.getUIState().street.imageId, 'pano-a'); + // The same call without the gate opens at once, with one request. + layer.closeViewer(); + const opened = await runner('show_street_level', { + latitude: 38.57986, + longitude: -121.49372, + provider: 'google', + fly: false, + }); + assert.equal(opened.ok, true, opened.error); + assert.equal(opened.imageId, 'pano-a'); + assert.equal(fake.lookups.length, before + 2); +}); + +test('coverageLines hands back what a provider draws, and nothing for providers without lines', async (t) => { + const drawing = fakeProvider(); + const create = drawing.create; + drawing.create = (context) => ({ + ...create(context), + coverageLines: () => [ + { + id: 's1', + parts: [ + [ + [1, 2], + [1, 2.001], + ], + ], + }, + ], + }); + const { layer } = await enabledLayer(t, [drawing]); + assert.deepEqual(layer.coverageLines(drawing.id), [ + { + id: 's1', + parts: [ + [ + [1, 2], + [1, 2.001], + ], + ], + }, + ]); + assert.deepEqual(layer.coverageLines('nope'), []); +}); + +test('coverageLines is empty for a provider that draws no coverage', async (t) => { + const { layer } = await enabledLayer(t, [fakeProvider()]); + assert.deepEqual(layer.coverageLines(fakeProvider().id), []); +}); + +test('openNearest keeps honouring the caller after the lookup: an abort while the viewer loads opens nothing', async (t) => { + let releaseMount = null; + const provider = fakeProvider({ nearestImage: async () => 'img-1' }); + const create = provider.create; + provider.create = (context) => { + const instance = create(context); + const mount = instance.viewer.mount; + instance.viewer.mount = async (...args) => { + await new Promise((resolve) => { + releaseMount = resolve; + }); + return mount(...args); + }; + return instance; + }; + const { layer } = await enabledLayer(t, [provider]); + const { fakeHost } = + await import('../../testSupport/googleStreetViewFakes.mjs'); + layer.attachViewerHost(fakeHost()); + const controller = new AbortController(); + const pending = layer.openNearest( + { lat: 1, lon: 2 }, + { frame: false, signal: controller.signal }, + ); + for (let i = 0; i < 20 && !releaseMount; i++) await settle(); + assert.ok(releaseMount, 'the lookup answered and the viewer is mounting'); + controller.abort(); + releaseMount(); + assert.equal(await pending, false); + assert.deepEqual(provider.calls.open, [], 'the photo was never opened'); + assert.equal(layer.getUIState().street.open, false); + assert.equal(layer.getUIState().street.loading, false); +}); + +test('an aborted older lookup leaves its replacement’s spinner alone', async (t) => { + const answers = []; + const provider = fakeProvider({ + nearestImage: (_point, { signal } = {}) => + new Promise((resolve, reject) => { + answers.push(resolve); + signal?.addEventListener('abort', () => + reject(Object.assign(new Error('aborted'), { name: 'AbortError' })), + ); + }), + }); + const { layer } = await enabledLayer(t, [provider]); + const { fakeHost } = + await import('../../testSupport/googleStreetViewFakes.mjs'); + layer.attachViewerHost(fakeHost()); + const older = new AbortController(); + const first = layer.openNearest( + { lat: 1, lon: 2 }, + { frame: false, signal: older.signal }, + ); + await settle(); + // A newer request replaces it, and its caller withdraws the old one too. + const second = layer.openNearest({ lat: 3, lon: 4 }, { frame: false }); + older.abort(); + assert.equal(await first, false); + assert.equal( + layer.getUIState().street.loading, + true, + 'the newer lookup is still loading', + ); + answers.at(-1)(null); + assert.equal(await second, false); + assert.equal(layer.getUIState().street.loading, false); + assert.match( + layer.getUIState().street.error, + /No street-level imagery/, + 'the newer lookup reports its own result', + ); +}); + +test('show_street_level opens a KartaView photo, and a plain link opens it at once since it is not billed', async (t) => { + const mapillary = fakeProvider(); + let nearest = 'kv-1'; + const kartaview = fakeProvider({ + id: 'kartaview', + pickPrefix: 'kv:', + nearestImage: async () => nearest, + }); + kartaview.name = 'KartaView'; + kartaview.label = 'KARTAVIEW'; + kartaview.defaultOn = false; + // As the real provider: 360° / flat and SINCE (the setFilter log stays in + // the fake's closure). + kartaview.filters = Object.freeze(['pano', 'since']); + const { layer, viewer } = await enabledLayer(t, [mapillary, kartaview]); + const { fakeHost } = + await import('../../testSupport/googleStreetViewFakes.mjs'); + layer.attachViewerHost(fakeHost()); + viewer.clock ??= { onTick: { addEventListener: () => () => {} } }; + viewer.camera.moveEnd ??= { addEventListener() {} }; + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + isEnabled: () => true, + getLayerParams: () => layer.getParams(), + setLayerParams: (_id, params) => { + layer.setParams(params); + return true; + }, + getAll: () => [{ id: 'street-level', name: 'Street Level', enabled: true }], + }; + const styleManager = { + mapStackController: { getActiveId: () => 'osm' }, + setMapStack: async () => ({ ok: false, error: 'no map switch expected' }), + }; + const runner = createGevActionRunner({ viewer, styleManager, dataManager }); + // The way a plain sl=kartaview link calls it: autoOpen false. + const result = await runner('show_street_level', { + latitude: 52.52, + longitude: 13.405, + provider: 'kartaview', + fly: false, + autoOpen: false, + }); + assert.equal(result.ok, true, result.error); + assert.equal(result.armed, undefined, 'not armed: KartaView bills nothing'); + assert.equal(result.provider, 'kartaview'); + assert.equal(result.providerName, 'KartaView'); + assert.deepEqual( + layer + .getUIState() + .providers.filter((p) => p.on) + .map((p) => p.id), + ['kartaview'], + ); + assert.deepEqual(kartaview.calls.open, ['kv-1']); + assert.equal(layer.providerStatus('kartaview').billed, false); + assert.equal(layer.requiredMapStack('kartaview'), null); + + // With 360° only on, an empty search names the filter, not just "no imagery". + layer.closeViewer?.(); + layer.setParams({ pano: 'pano' }); + nearest = null; + const empty = await runner('show_street_level', { + latitude: 52.52, + longitude: 13.405, + provider: 'kartaview', + fly: false, + }); + assert.deepEqual([empty.ok, empty.stage], [false, 'imagery']); + assert.equal(empty.filter?.pano, 'pano'); + assert.match(empty.error, /with the 360° filter on$/); + layer.setParams({ pano: 'all' }); + const plain = await runner('show_street_level', { + latitude: 52.52, + longitude: 13.405, + provider: 'kartaview', + fly: false, + }); + assert.equal(plain.ok, false); + assert.equal(plain.filter, undefined, 'no filter, no hint'); + assert.doesNotMatch(plain.error, /filter/); +}); + +test('show_street_level switches from KartaView to Panoramax and opens its photo at once, unbilled', async (t) => { + const mapillary = fakeProvider(); + const kartaview = fakeProvider({ id: 'kartaview', pickPrefix: 'kv:' }); + kartaview.name = 'KartaView'; + kartaview.defaultOn = false; + const panoramax = fakeProvider({ + id: 'panoramax', + pickPrefix: 'px:', + nearestImage: async () => 'px-1', + }); + panoramax.name = 'Panoramax'; + panoramax.label = 'PANORAMAX'; + panoramax.defaultOn = false; + const { layer, viewer } = await enabledLayer(t, [ + mapillary, + kartaview, + panoramax, + ]); + const { fakeHost } = + await import('../../testSupport/googleStreetViewFakes.mjs'); + layer.attachViewerHost(fakeHost()); + viewer.clock ??= { onTick: { addEventListener: () => () => {} } }; + viewer.camera.moveEnd ??= { addEventListener() {} }; + layer.setParams({ mapillary: false, kartaview: true, panoramax: false }); + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + isEnabled: () => true, + getLayerParams: () => layer.getParams(), + setLayerParams: (_id, params) => { + layer.setParams(params); + return true; + }, + getAll: () => [{ id: 'street-level', name: 'Street Level', enabled: true }], + }; + const styleManager = { + mapStackController: { getActiveId: () => 'osm' }, + setMapStack: async () => ({ ok: false, error: 'no map switch expected' }), + }; + const runner = createGevActionRunner({ viewer, styleManager, dataManager }); + // The way a plain sl=panoramax link calls it: autoOpen false. + const result = await runner('show_street_level', { + latitude: 48.8874, + longitude: 2.3495, + provider: 'panoramax', + fly: false, + autoOpen: false, + }); + assert.equal(result.ok, true, result.error); + assert.equal(result.armed, undefined, 'not armed: Panoramax bills nothing'); + assert.equal(result.providerName, 'Panoramax'); + assert.deepEqual( + layer + .getUIState() + .providers.filter((p) => p.on) + .map((p) => p.id), + ['panoramax'], + 'KartaView switched off', + ); + assert.deepEqual(panoramax.calls.open, ['px-1']); + assert.deepEqual(kartaview.calls.open, [], 'nothing asked of KartaView'); + assert.equal(layer.providerStatus('panoramax').billed, false); + assert.equal(layer.requiredMapStack('panoramax'), null); +}); + test('the panel says why a lit Street View draws nothing off Google 3D', async (t) => { const { presentStreetLevelPanel } = await import('../../ui/streetLevelPresentation.js'); diff --git a/src/layers/streetLevel/lineCoverage.js b/src/layers/streetLevel/lineCoverage.js index bab1af46d..b71fe7cd7 100644 --- a/src/layers/streetLevel/lineCoverage.js +++ b/src/layers/streetLevel/lineCoverage.js @@ -531,6 +531,25 @@ export function createLineCoverage({ }; } + /** + * The sequence lines drawn now, as `[lon, lat]` parts per sequence: for + * diagnostics and the browser gate, which aims its clicks at them instead + * of scanning pixels: what each tile drew (`attachPrimitive` keeps it in + * `entry.sequences`). Read-only copies. + */ + function drawnLines() { + const lines = []; + for (const entry of state.coverage.tiles.values()) + for (const sequence of entry.sequences.values()) + lines.push({ + id: sequence.id, + parts: (sequence.parts || []).map((part) => + part.map(([lon, lat]) => [lon, lat]), + ), + }); + return lines; + } + return { attach, detach, @@ -543,5 +562,6 @@ export function createLineCoverage({ highlight, sequenceCount, stats, + drawnLines, }; } diff --git a/src/layers/streetLevel/lineCoverage.test.mjs b/src/layers/streetLevel/lineCoverage.test.mjs index 60bae7eb3..dd9282119 100644 --- a/src/layers/streetLevel/lineCoverage.test.mjs +++ b/src/layers/streetLevel/lineCoverage.test.mjs @@ -654,3 +654,67 @@ test('the selected sequence is drawn and highlighted even past the per-tile cap' assert.ok(state.coverage.highlight, 'the selected line is highlighted'); coverage.clear(); }); + +test('drawnLines reports the lines on the globe as [lon, lat] copies, filtered like the drawing', async () => { + const { source, state, coverage, centre } = setup(); + assert.deepEqual(coverage.drawnLines(), [], 'nothing before a tile'); + const sequences = [ + { + id: 'flat-1', + parts: [ + [ + [centre.lon - 0.001, centre.lat], + [centre.lon + 0.001, centre.lat], + ], + ], + capturedAt: 2_000_000_000_000, + }, + { + id: 'pano-1', + parts: [ + [ + [centre.lon, centre.lat - 0.001], + [centre.lon, centre.lat], + [centre.lon, centre.lat + 0.001], + ], + ], + capturedAt: 1_000_000_000_000, + isPano: true, + }, + ]; + state.filter = { pano: 'pano', sinceDays: 0 }; + coverage.refresh(); + source.calls[0].resolve(encodeCoverageTile(centre.tile, { sequences })); + await settle(); + const lines = coverage.drawnLines(); + assert.deepEqual( + lines.map((line) => line.id), + ['pano-1'], + 'only the drawn (filtered) lines', + ); + assert.equal(lines[0].parts.length, 1); + assert.equal(lines[0].parts[0].length, 3); + // Tile coordinates are quantised to the tile grid (a few metres at z14). + const [midLon, midLat] = lines[0].parts[0][1]; + assert.ok( + Math.abs(midLon - centre.lon) < 1e-4 && + Math.abs(midLat - centre.lat) < 1e-4, + `coordinates as [lon, lat]: ${midLon}, ${midLat} vs ${centre.lon}, ${centre.lat}`, + ); + assert.ok( + lines[0].parts[0][0][1] < lines[0].parts[0][2][1], + 'vertex order kept: south to north', + ); + // Copies: a caller cannot bend the drawn line. + lines[0].parts[0][1][0] = 0; + assert.notEqual(coverage.drawnLines()[0].parts[0][1][0], 0); + state.filter = { pano: 'all', sinceDays: 0 }; + coverage.rebuild(); + assert.deepEqual( + coverage + .drawnLines() + .map((line) => line.id) + .sort(), + ['flat-1', 'pano-1'], + ); +}); diff --git a/src/layers/streetLevel/policy.js b/src/layers/streetLevel/policy.js index 4bf7f953d..e135307e7 100644 --- a/src/layers/streetLevel/policy.js +++ b/src/layers/streetLevel/policy.js @@ -40,6 +40,14 @@ export const FILTER_DEFAULT = Object.freeze({ pano: 'all', sinceDays: 0 }); /** Nearest-image search radius in metres. */ export const NEAREST_RADIUS_M = 50; +/** + * Widest nearest-image radius a caller may ask for, in metres. A geocoded + * landmark sits at its centroid, often further than 50 m from the nearest + * street; providers that can search wider (Google) honour it, the rest keep + * their own cap. + */ +export const NEAREST_RADIUS_MAX_M = 250; + /** The Google 3D map stack: Street View may be shown only on it. */ export const GOOGLE_3D_MAP_STACK_ID = 'photoreal'; diff --git a/src/layers/streetLevel/providers/google/index.js b/src/layers/streetLevel/providers/google/index.js index d32ee95f6..8af24275c 100644 --- a/src/layers/streetLevel/providers/google/index.js +++ b/src/layers/streetLevel/providers/google/index.js @@ -12,6 +12,7 @@ import { GROUND_CLICK_HINT, KEY_REJECTED_MESSAGE, NEAREST_RADIUS_M, + NEAREST_RADIUS_MAX_M, NEAREST_TIMEOUT_MS, NO_ANSWER_MESSAGE, PICK_PREFIX, @@ -67,6 +68,7 @@ export function createGoogleProvider({ requiresMapStack: GOOGLE_3D_MAP_STACK_ID, // Every panorama is billed to the Maps key. defaultOn: false, + billedPerImage: true, // All 360°, and no coverage for a date filter to thin. filters: [], marker: Object.freeze({ kind: 'view-cone', color: COLORS.coverage }), @@ -136,8 +138,12 @@ export function createGoogleProvider({ * Nearest Google-collected outdoor panorama, or null. The imagery * filters do not apply (see `filters`). */ - async nearestImage({ lat, lon }, { signal } = {}) { + async nearestImage({ lat, lon }, { signal, radiusM } = {}) { if (loader.authFailed()) throw new Error(KEY_REJECTED_MESSAGE); + // Wider than the shared default only on request, never past the cap. + const radius = Number.isFinite(radiusM) + ? Math.min(NEAREST_RADIUS_MAX_M, Math.max(1, radiusM)) + : NEAREST_RADIUS_M; signal?.throwIfAborted(); // Set once the caller has its answer (aborted, timed out, refused): // a lookup still waiting on Google's script then asks nothing. @@ -150,7 +156,7 @@ export function createGoogleProvider({ try { const { data } = await service.getPanorama({ location: { lat, lng: lon }, - radius: NEAREST_RADIUS_M, + radius, // Both: Google's own imagery, outdoors (sources intersect). sources: [GOOGLE, OUTDOOR], preference: lib.StreetViewPreference.NEAREST, diff --git a/src/layers/streetLevel/providers/google/policy.js b/src/layers/streetLevel/providers/google/policy.js index e0fa40cfb..d8ec0bb69 100644 --- a/src/layers/streetLevel/providers/google/policy.js +++ b/src/layers/streetLevel/providers/google/policy.js @@ -1,7 +1,7 @@ import { COLORS as SHARED_COLORS, PROVIDER_COLORS } from '../../policy.js'; import { msg } from '../../../../i18n/index.js'; -export { NEAREST_RADIUS_M } from '../../policy.js'; +export { NEAREST_RADIUS_M, NEAREST_RADIUS_MAX_M } from '../../policy.js'; export const GOOGLE_PROVIDER_ID = 'google'; export const GOOGLE_NAME = msg('Google Street View'); diff --git a/src/layers/streetLevel/providers/google/provider.test.mjs b/src/layers/streetLevel/providers/google/provider.test.mjs index 3314f0048..a78d9201f 100644 --- a/src/layers/streetLevel/providers/google/provider.test.mjs +++ b/src/layers/streetLevel/providers/google/provider.test.mjs @@ -291,3 +291,13 @@ test('the panorama a lookup found is dated without a second lookup', async () => 'no getPanorama({pano}) for a panorama the lookup already described', ); }); + +test('a wider lookup radius is honoured up to the cap', async () => { + const { instance, fake } = setup(); + await instance.nearestImage({ lat: 38.58, lon: -121.49 }, { radiusM: 250 }); + assert.equal(fake.lookups.at(-1).radius, 250); + await instance.nearestImage({ lat: 38.58, lon: -121.49 }, { radiusM: 9999 }); + assert.equal(fake.lookups.at(-1).radius, 250); + await instance.nearestImage({ lat: 38.58, lon: -121.49 }, { radiusM: NaN }); + assert.equal(fake.lookups.at(-1).radius, 50); +}); diff --git a/src/layers/streetLevel/providers/kartaview/index.js b/src/layers/streetLevel/providers/kartaview/index.js index 6e75f967a..523f6920c 100644 --- a/src/layers/streetLevel/providers/kartaview/index.js +++ b/src/layers/streetLevel/providers/kartaview/index.js @@ -535,6 +535,7 @@ export function createKartaViewProvider({ loading: state.sequence.loading, error: state.sequence.error, }), + coverageLines: () => coverage.drawnLines(), /** * The nearest photo within the search radius that can be shown, or diff --git a/src/layers/streetLevel/providers/mapillary/index.js b/src/layers/streetLevel/providers/mapillary/index.js index 928922195..901df886c 100644 --- a/src/layers/streetLevel/providers/mapillary/index.js +++ b/src/layers/streetLevel/providers/mapillary/index.js @@ -191,6 +191,7 @@ export function createMapillaryProvider({ source, photoViewer = null }) { loading: state.sequence.loading, error: state.sequence.error, }), + coverageLines: () => parts.coverage.drawnLines(), viewer: photoViewer || diff --git a/src/layers/streetLevel/providers/panoramax/index.js b/src/layers/streetLevel/providers/panoramax/index.js index e855c9c72..805d304b9 100644 --- a/src/layers/streetLevel/providers/panoramax/index.js +++ b/src/layers/streetLevel/providers/panoramax/index.js @@ -526,6 +526,9 @@ export function createPanoramaxProvider({ cones.clearSelection(); drawn = null; }, + // The lines drawn now (current and stale tiles), as [lon, lat] + // parts, for diagnostics and the browser gate's line clicks. + coverageLines: () => coverage.drawnLines(), sequenceStats: () => ({ selectedId: state.sequence.selectedId, images: state.sequence.images.length, diff --git a/src/layers/streetLevel/providers/panoramax/provider.test.mjs b/src/layers/streetLevel/providers/panoramax/provider.test.mjs index 628805ce1..4ad4e9655 100644 --- a/src/layers/streetLevel/providers/panoramax/provider.test.mjs +++ b/src/layers/streetLevel/providers/panoramax/provider.test.mjs @@ -1182,3 +1182,9 @@ test('a busy or timed-out tile keeps its own message; only a body that is not a assert.equal(env.instance.sequenceStats().error, failure.message); } }); + +test('coverageLines exposes the drawn lines, none before any tile', () => { + const { instance } = setup(); + assert.equal(typeof instance.coverageLines, 'function'); + assert.deepEqual(instance.coverageLines(), []); +}); diff --git a/src/layers/streetLevel/registry.js b/src/layers/streetLevel/registry.js index 5e51929a6..2c3fe42ca 100644 --- a/src/layers/streetLevel/registry.js +++ b/src/layers/streetLevel/registry.js @@ -20,6 +20,7 @@ export function filtersOf(provider) { * @property {boolean} [groundClick] No coverage: a ground click at street zoom opens its nearest image. * @property {string} [requiresMapStack] Usable only on this map stack. * @property {boolean} [defaultOn] Default true; the first that defaults on is lit. + * @property {boolean} [billedPerImage] Every image load costs the key's owner: a link never opens one without a click. * @property {{kind: 'view-cone', color: string}} [marker] Default: GEV's pin. * @property {Array<'pano'|'since'>} [filters] Filters it honours (default both). * @property {{noun: string, action: string}} [coverageLabel] How the panel counts its lines and says what a click does (default: sequences, "click a line for its photos"). @@ -48,7 +49,8 @@ export function filtersOf(provider) { * @property {(sequenceId: string) => Promise} [selectSequence] * @property {() => void} [clearSequence] * @property {() => {selectedId: string|null, images: number, loading: boolean, error?: string|null}} [sequenceStats] - * @property {(point: {lat: number, lon: number}, options?: {signal?: AbortSignal}) => Promise} [nearestImage] For ground clicks and `openNearest`; an aborted lookup gives up (it may reject with an AbortError). + * @property {() => Array<{id: string, parts: Array>}>} [coverageLines] The coverage lines drawn now, as [lon, lat] parts (diagnostics and the browser gate). + * @property {(point: {lat: number, lon: number}, options?: {signal?: AbortSignal, radiusM?: number}) => Promise} [nearestImage] For ground clicks and `openNearest`; an aborted lookup gives up (it may reject with an AbortError). `radiusM` asks for a wider search than NEAREST_RADIUS_M; a provider that cannot search wider ignores it. * @property {ViewerAdapter} viewer * * @typedef {object} ViewerAdapter diff --git a/src/layers/streetLevel/state.js b/src/layers/streetLevel/state.js index f9e5e4c88..7408b7dee 100644 --- a/src/layers/streetLevel/state.js +++ b/src/layers/streetLevel/state.js @@ -29,6 +29,8 @@ export function freshStreet() { externalUrl: null, loading: false, error: null, + // A lookup a link prepared but left to a click: {point, providerIds, radiusM}. + armed: null, }; } diff --git a/src/layers/streetLevel/viewerHost.js b/src/layers/streetLevel/viewerHost.js index 16c992afd..f56d2487f 100644 --- a/src/layers/streetLevel/viewerHost.js +++ b/src/layers/streetLevel/viewerHost.js @@ -99,12 +99,17 @@ export function createViewerHost({ state, parts }) { } /** - * Open an image: true once its first pose is in, false if it failed or was - * overtaken. `frame: false` leaves the camera where it is. + * Open an image: true once its first pose is in, false if it failed, was + * overtaken, or `signal` fired first. An abort while the viewer mounts or + * loads closes the photo: nothing opens for a request that was withdrawn. */ - async function open(providerId, imageId, { frame = true } = {}) { + async function open( + providerId, + imageId, + { frame = true, signal = null } = {}, + ) { const entry = state.providers.get(providerId); - if (!entry || !imageId) return false; + if (!entry || !imageId || signal?.aborted) return false; if (!state.street.host) { // A nearest-image lookup may have set it; nothing is loading now. state.street.loading = false; @@ -115,6 +120,10 @@ export function createViewerHost({ state, parts }) { } const seq = ++openSeq; const current = () => seq === openSeq; + const onAbort = () => { + if (current()) close(); + }; + signal?.addEventListener('abort', onAbort, { once: true }); // Claimed now, honoured after loading only if nothing newer took the camera. const ticket = frame ? parts.framing.begin() : null; Object.assign(state.street, { @@ -138,6 +147,7 @@ export function createViewerHost({ state, parts }) { if (current()) state.street.error = error?.message || msg('Image could not be opened'); } finally { + signal?.removeEventListener('abort', onAbort); if (current()) state.street.loading = false; notify(); } diff --git a/src/layers/streetLevel/viewerHost.test.mjs b/src/layers/streetLevel/viewerHost.test.mjs index fdcb74e21..c7e7d0007 100644 --- a/src/layers/streetLevel/viewerHost.test.mjs +++ b/src/layers/streetLevel/viewerHost.test.mjs @@ -557,6 +557,51 @@ test('an adapter that can frame but says this image has no frame offers no FIT / assert.equal(state.street.renderModes, true, 'a flat image can be framed'); }); +test('a caller that withdraws while the viewer is still mounting gets no photo', async () => { + const gate = deferred(); + const adapter = fakeAdapter({ gate }); + const { state, host } = harness(adapter); + const controller = new AbortController(); + const pending = host.open('mapillary', 'a', { signal: controller.signal }); + assert.equal(state.street.open, true, 'opening'); + controller.abort(); // the library is still loading + gate.resolve(); + assert.equal(await pending, false); + assert.deepEqual(adapter.calls.open, [], 'the image was never opened'); + assert.equal(state.street.open, false, 'the photo is closed'); + assert.equal(state.street.loading, false, 'no stuck spinner'); + assert.equal(state.street.imageId, null); +}); + +test('a caller that withdraws while the photo is loading gets it closed', async () => { + const openGate = deferred(); + const adapter = fakeAdapter({ openGate }); + const { state, host } = harness(adapter); + const controller = new AbortController(); + const pending = host.open('mapillary', 'a', { signal: controller.signal }); + await settle(); + assert.deepEqual(adapter.calls.open, ['a'], 'loading'); + controller.abort(); + openGate.resolve(); + assert.equal(await pending, false); + assert.equal(state.street.open, false, 'the withdrawn photo is closed'); + assert.equal(state.street.imageId, null); + // An already-withdrawn request asks nothing at all. + const done = new AbortController(); + done.abort(); + assert.equal( + await host.open('mapillary', 'b', { signal: done.signal }), + false, + ); + assert.deepEqual(adapter.calls.open, ['a']); + // A signal that never fires changes nothing. + assert.equal( + await host.open('mapillary', 'c', { signal: new AbortController().signal }), + true, + ); + assert.equal(state.street.imageId, 'c'); +}); + test('a pose’s licence reaches the caption state, and a closed viewer forgets it', async () => { const { state, host, google } = twoProviders(); await host.open('google', 'g-1'); diff --git a/src/locations.js b/src/locations.js index d17186905..5da9432c4 100644 --- a/src/locations.js +++ b/src/locations.js @@ -380,6 +380,8 @@ export async function searchAndFlyTo(viewer, query, options = {}) { }); return { label, + latitude: lat, + longitude: lng, navigationMode: 'natural-region-swath', rangeM: swath.rangeM, }; @@ -415,6 +417,8 @@ export async function searchAndFlyTo(viewer, query, options = {}) { if (flight) { return { label, + latitude: lat, + longitude: lng, navigationMode, rangeM: null, }; @@ -468,6 +472,8 @@ export async function searchAndFlyTo(viewer, query, options = {}) { }); return { label, + latitude: lat, + longitude: lng, navigationMode: requestedRange ? 'explicit-range' : options.forceClose diff --git a/src/locations.test.mjs b/src/locations.test.mjs index 8d2a9503c..10e3b53fb 100644 --- a/src/locations.test.mjs +++ b/src/locations.test.mjs @@ -778,3 +778,35 @@ test('a precise search without an outline frames against the resolved ground, no const eye = target.height + flight.offset.range * Math.sin(-flight.offset.pitch); assert.ok(eye > 171 + 100, `eye ${eye} must clear the 171 m surface`); }); + +test('searchAndFlyTo reports the place it flew to, whichever framing it chose', async () => { + // A city frames its viewport. + const city = await runSearch(stubViewer(), {}); + assert.equal(city.latitude, 30.2672); + assert.equal(city.longitude, -97.7431); + // A precise place flies to a landmark. + const place = await runSearch(stubViewer(), {}, { + query: 'Texas State Capitol', + result: { + formatted_address: '1100 Congress Ave, Austin, TX', + types: ['establishment', 'point_of_interest'], + geometry: { location: { lat: 30.2747, lng: -97.7403 } }, + }, + }); + assert.equal(place.latitude, 30.2747); + assert.equal(place.longitude, -97.7403); + // A natural region frames a swath. + const region = await runSearch(stubViewer(), {}, { + query: 'Rocky Mountains', + result: { + formatted_address: 'Rocky Mountains', + types: ['natural_feature', 'establishment'], + geometry: { + location: { lat: 43.7, lng: -110.7 }, + viewport: { southwest: { lat: 35, lng: -118 }, northeast: { lat: 52, lng: -104 } }, + }, + }, + }); + assert.equal(region.latitude, 43.7); + assert.equal(region.longitude, -110.7); +}); diff --git a/src/radioMarkup.test.mjs b/src/radioMarkup.test.mjs index 74a44e77d..552d3b417 100644 --- a/src/radioMarkup.test.mjs +++ b/src/radioMarkup.test.mjs @@ -25,11 +25,12 @@ const css = readStylesheet(new URL('../style.css', import.meta.url)); function realtimeTools() { return GEV_REALTIME_TOOLS; } -test('Realtime schema exposes the authoritative 31-tool inventory', () => { +test('Realtime schema exposes the authoritative 32-tool inventory', () => { const tools = realtimeTools(); - assert.equal(tools.length, 31); + assert.equal(tools.length, 32); const names = tools.map((tool) => tool.name); - assert.equal(new Set(names).size, 31, 'tool names are unique'); + assert.equal(new Set(names).size, 32, 'tool names are unique'); + assert.ok(names.includes('show_street_level')); assert.ok(names.includes('set_context_mode')); assert.ok(names.includes('control_cockpit')); assert.ok(names.includes('set_voice_inactivity_timeout')); @@ -199,6 +200,8 @@ test('no unchanged Realtime tool definition drifts silently', () => { 'track_entity', // Point-and-ask: 'pointer' sentinels and referent arguments. 'annotate_map', + // Street-level photos: a new composite tool, not an edit to a shipped one. + 'show_street_level', ]); const unchanged = realtimeTools() .filter((tool) => !TOUCHED.has(tool.name)) diff --git a/src/streetLevelMarkup.test.mjs b/src/streetLevelMarkup.test.mjs index 7e2e32314..602a3b065 100644 --- a/src/streetLevelMarkup.test.mjs +++ b/src/streetLevelMarkup.test.mjs @@ -115,3 +115,17 @@ test('OPEN STREET VIEW keeps its width beside a long coverage line', () => { ); assert.match(panelCss, /\.sl-meta-row \.sl-meta \{[^}]*min-width: 0;/); }); + +test('embed mode shows an open photo over the globe, filling the overlay', () => { + const overlay = css.match( + /body\.ui-embed \.sl-viewer-wrap-embed \{([^}]*)\}/, + ); + assert.ok(overlay, 'embed styles the viewer it moves over the globe'); + for (const rule of ['position: fixed', 'inset: 12px', 'z-index: 1000']) + assert.ok(overlay[1].includes(rule), rule); + // The photo takes the overlay's height, as it does full screen. + assert.match( + css, + /body\.ui-embed \.sl-viewer-wrap-embed \.sl-viewer \{[^}]*flex: 1 1 auto/, + ); +}); diff --git a/src/tools/queries/share.js b/src/tools/queries/share.js index 5284e8a06..3de7902b0 100644 --- a/src/tools/queries/share.js +++ b/src/tools/queries/share.js @@ -47,10 +47,26 @@ export async function resolveViewArguments(args, { services, signal, tools }) { ? { ...args.camera } : { ...base.camera, ...args.camera }; let label = null; + // Street-level photos open at the area's centre unless a point is given; + // a new area moves a passed-through view's photos with it. + let streetLevel = + args.street_level && typeof args.street_level === 'object' + ? { ...args.street_level } + : base.street_level && typeof base.street_level === 'object' + ? args.area + ? { provider: base.street_level.provider } + : { ...base.street_level } + : null; if (args.area) { const area = await resolveArea(args.area, { services, signal }); label = area.label; const framed = cameraForArea(area); + if ( + streetLevel && + streetLevel.lat === undefined && + streetLevel.lon === undefined + ) + streetLevel = { ...streetLevel, lat: framed.lat, lon: framed.lon }; // A tilted view of an area looks at its center from behind, unless the // caller placed the camera itself. camera = @@ -84,6 +100,7 @@ export async function resolveViewArguments(args, { services, signal, tools }) { map: args.map ?? base.map ?? null, follow, annotations: args.annotations ?? base.annotations ?? [], + street_level: streetLevel, }); } catch (error) { // A view passed through can hold anything; report it as bad arguments. @@ -113,7 +130,9 @@ export const showInGodsEyeView = defineTool({ 'it. Pass the view another answer returned, or describe one: an area ' + 'framed from above or a camera position, with chosen data layers, visual ' + 'style and map, optionally following an aircraft (or riding in its ' + - 'cockpit view) or a satellite, and with marks drawn on the map.', + 'cockpit view) or a satellite, with marks drawn on the map, or opening ' + + 'street-level photos (Google Street View, Mapillary, KartaView or ' + + 'Panoramax) at a place.', inputSchema: { type: 'object', properties: VIEW_ARGUMENTS, diff --git a/src/tools/queries/share.test.mjs b/src/tools/queries/share.test.mjs index 7be532150..aac74a776 100644 --- a/src/tools/queries/share.test.mjs +++ b/src/tools/queries/share.test.mjs @@ -113,3 +113,104 @@ test('links carry a camera, style, map and an entity to follow', async () => { /Give a view, an area, or a camera with lat and lon/, ); }); + +test('links can open street-level photos at the area, on the map the source needs', async () => { + const { viewFromParams } = await import('../../view/index.js'); + const catalog = composeCatalog({ + tools: coreTools, + services: { app: { baseUrl: 'http://localhost:4173/' } }, + }); + const result = await catalog.call('show_in_gods_eye_view', { + area: { lat: 38.5799, lon: -121.4937, radius_km: 1 }, + street_level: { provider: 'google' }, + }); + const { view } = result.data; + assert.equal(view.map, 'photoreal'); + assert.deepEqual(view.layers, ['street-level']); + assert.deepEqual(view.street_level, { + provider: 'google', + lat: 38.5799, + lon: -121.4937, + }); + const params = new URLSearchParams(new URL(result.data.url).hash.slice(1)); + assert.equal(params.get('sl'), 'google,38.57990,-121.49370'); + assert.equal(params.get('map'), 'photoreal'); + assert.deepEqual(viewFromParams(params), view); + + const point = await catalog.call('show_in_gods_eye_view', { + view, + street_level: { provider: 'mapillary', lat: 38.58, lon: -121.49 }, + }); + assert.deepEqual(point.data.view.street_level, { + provider: 'mapillary', + lat: 38.58, + lon: -121.49, + }); + await assert.rejects( + catalog.call('show_in_gods_eye_view', { + area: { lat: 0, lon: 0, radius_km: 1 }, + map: 'osm', + street_level: { provider: 'google' }, + }), + /photoreal/, + ); +}); + +test('a new area moves a passed-through view’s photos, and a tilted area still opens them at its centre', async () => { + const catalog = composeCatalog({ + tools: coreTools, + services: { app: { baseUrl: 'http://localhost:4173/' } }, + }); + const first = await catalog.call('show_in_gods_eye_view', { + area: { lat: 38.5799, lon: -121.4937, radius_km: 1 }, + street_level: { provider: 'google' }, + }); + const second = await catalog.call('show_in_gods_eye_view', { + view: first.data.view, + area: { lat: 42.3314, lon: -83.0458, radius_km: 1 }, + }); + assert.deepEqual(second.data.view.street_level, { + provider: 'google', + lat: 42.3314, + lon: -83.0458, + }); + const tilted = await catalog.call('show_in_gods_eye_view', { + area: { lat: 48.8584, lon: 2.2945, radius_km: 1 }, + camera: { pitch_deg: -30 }, + street_level: { provider: 'mapillary' }, + }); + assert.notEqual( + tilted.data.view.camera.lat, + 48.8584, + 'the camera sits behind the target', + ); + assert.deepEqual(tilted.data.view.street_level, { + provider: 'mapillary', + lat: 48.8584, + lon: 2.2945, + }); + await assert.rejects( + catalog.call('show_in_gods_eye_view', { + area: { lat: 0, lon: 0, radius_km: 1 }, + follow: { kind: 'aircraft', id: 'AE1234' }, + street_level: {}, + }), + /cannot both follow/, + ); +}); + +test('a view passed through alone keeps its street-level photos', async () => { + const catalog = composeCatalog({ + tools: coreTools, + services: { app: { baseUrl: 'http://localhost:4173/' } }, + }); + const first = await catalog.call('show_in_gods_eye_view', { + area: { lat: 38.5799, lon: -121.4937, radius_km: 1 }, + street_level: { provider: 'mapillary' }, + }); + const again = await catalog.call('show_in_gods_eye_view', { + view: first.data.view, + }); + assert.deepEqual(again.data.view.street_level, first.data.view.street_level); + assert.deepEqual(again.data.view, first.data.view); +}); diff --git a/src/ui/applicationShell.js b/src/ui/applicationShell.js index c3ad27c14..2dd75cd3b 100644 --- a/src/ui/applicationShell.js +++ b/src/ui/applicationShell.js @@ -1318,9 +1318,12 @@ export class StyleManager extends ShellFacade { }; } if (!target.available) { + // The controller knows why (Google 3D: no Maps key; others: no ion token). return { ok: false, - error: `${target.label} requires a Cesium ion token`, + error: + target.unavailableReason || + `${target.label} requires a Cesium ion token`, activeStack: this.mapStackController.getActiveId(), }; } diff --git a/src/ui/setMapStackReason.test.mjs b/src/ui/setMapStackReason.test.mjs new file mode 100644 index 000000000..ca412116c --- /dev/null +++ b/src/ui/setMapStackReason.test.mjs @@ -0,0 +1,33 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { StyleManager } from './applicationShell.js'; + +test('an unavailable map stack reports the controller’s own reason, not always an ion token', async () => { + const controller = { + getStacks: () => [ + { + id: 'photoreal', + label: 'Google 3D', + available: false, + unavailableReason: 'Google 3D needs GOOGLE_MAPS_API_KEY', + }, + { id: 'bing-aerial', label: 'Bing aerial', available: false }, + ], + getActiveId: () => 'osm', + }; + const shell = { mapStackController: controller }; + const photoreal = await StyleManager.prototype.setMapStack.call( + shell, + 'photoreal', + ); + assert.deepEqual(photoreal, { + ok: false, + error: 'Google 3D needs GOOGLE_MAPS_API_KEY', + activeStack: 'osm', + }); + const bing = await StyleManager.prototype.setMapStack.call( + shell, + 'bing-aerial', + ); + assert.equal(bing.error, 'Bing aerial requires a Cesium ion token'); +}); diff --git a/src/ui/streetLevelControls.js b/src/ui/streetLevelControls.js index d680bc00f..001157f7c 100644 --- a/src/ui/streetLevelControls.js +++ b/src/ui/streetLevelControls.js @@ -11,6 +11,11 @@ const RENDER_MODE_KEY = 'gev:street-level:render-mode'; /** classes of the modes that hide every panel; an expanded photo leaves with them. */ const PANEL_HIDING_MODES = ['ui-clean-view', 'recording-mode', 'cockpit-mode']; +/** True on an embedded page (`?embed=1` or the inline panel). */ +function isEmbedPage() { + return globalThis.document?.body?.classList?.contains('ui-embed') === true; +} + /** * Drives the Street Level panel body: provider chips, filters and viewer. * The panel view renders what this presents (`gev-street-level` store); the @@ -137,6 +142,15 @@ export class StreetLevelControls { }); } this.store.setExpandAvailable(Boolean(this._expandMode)); + // Embed mode: Esc closes an open photo, since the rail it would go back to + // is hidden. Full screen keeps its own Esc (above), which leaves it first. + this.listen(document, 'keydown', (event) => { + if (event.key !== 'Escape' || !this._embedHome || this.isViewerExpanded()) + return; + event.preventDefault(); + event.stopPropagation(); + this.layer.closeViewer?.(); + }); try { const stored = localStorage.getItem(RENDER_MODE_KEY); if (stored === 'fill' || stored === 'letterbox') @@ -293,6 +307,32 @@ export class StreetLevelControls { this._requestResize(); } + /** + * Embed mode hides the right rail, panel and viewer with it: an open photo + * moves to as an overlay on the globe (`sl-viewer-wrap-embed`, see + * embed.css) and returns to its place in the panel when it closes. The + * panel view keeps patching the wrap where it is: its siblings are static, + * so Vue never inserts next to it. + */ + _syncEmbedOverlay(open) { + const wrap = this._elements.viewerWrap; + if (!wrap || !document.body) return; + const lift = open === true && isEmbedPage(); + if (lift && !this._embedHome) { + this._embedHome = { parent: wrap.parentNode, next: wrap.nextSibling }; + document.body.appendChild(wrap); + wrap.classList.add('sl-viewer-wrap-embed'); + this._requestResize(); + } else if (!lift && this._embedHome) { + wrap.classList.remove('sl-viewer-wrap-embed'); + const { parent, next } = this._embedHome; + this._embedHome = null; + if (parent?.isConnected) + parent.insertBefore(wrap, next?.parentNode === parent ? next : null); + this._requestResize(); + } + } + connect() { this._unsubscribe?.(); this._unsubscribe = null; @@ -350,6 +390,9 @@ export class StreetLevelControls { const open = view.viewer.open; if (open && !this._wasOpen) this.setCollapsed(false, { persist: false }); this._wasOpen = open; + // Embed mode hides the rail the panel sits in: an open photo is shown over + // the globe instead, and goes back into the panel when it closes. + this._syncEmbedOverlay(open); } destroy() { @@ -362,6 +405,7 @@ export class StreetLevelControls { if (this._expandMode === 'popover') this._elements.viewerWrap?.removeAttribute('popover'); this._watchPanelHidingModes(false); + this._syncEmbedOverlay(false); this.listeners.abort(); this._resizeObserver?.disconnect(); this._resizeObserver = null; diff --git a/src/ui/streetLevelControls.test.mjs b/src/ui/streetLevelControls.test.mjs index 89b3c0964..9a38c0b98 100644 --- a/src/ui/streetLevelControls.test.mjs +++ b/src/ui/streetLevelControls.test.mjs @@ -921,3 +921,103 @@ test('a window changed elsewhere under a focused SINCE thumb leaves its readout assert.equal(since.getAttribute('aria-valuetext'), label()); controls.destroy(); })); + +/** In embed mode, the viewer moved out of the hidden rail onto the globe. */ +const overGlobe = (dom) => + dom.wrap.parentNode === dom.document.body && + dom.wrap.classList.contains('sl-viewer-wrap-embed'); + +const escape = () => + document.dispatchEvent( + new KeyboardEvent('keydown', { key: 'Escape', bubbles: true }), + ); + +test('in embed mode an open photo leaves the hidden rail for an overlay on the globe, and goes back when it closes', () => + withDom(async (dom) => { + dom.document.body.classList.add('ui-embed'); + const home = dom.wrap.parentNode; + const after = dom.wrap.nextSibling; + const { layer, controls } = await stubPanel(dom, uiState({ open: false })); + assert.equal(overGlobe(dom), false, 'nothing to show yet'); + await layer.publish(uiState({ open: true })); + assert.equal(overGlobe(dom), true, 'shown over the globe'); + assert.equal(dom.wrap.hidden, false, 'the panel view still renders it'); + await layer.publish(uiState({ open: true, renderMode: 'fill' })); + assert.equal(overGlobe(dom), true, 'an update keeps it there'); + assert.equal( + dom.wrap.querySelector('[aria-checked="true"]')?.textContent.trim(), + 'FILL', + 'and the panel view keeps patching it there', + ); + await layer.publish(uiState({ open: false })); + assert.equal(overGlobe(dom), false); + assertSameNode(dom.wrap.parentNode, home, 'back in the panel'); + assertSameNode(dom.wrap.nextSibling, after, 'in its own place'); + assert.equal(dom.wrap.hidden, true); + controls.destroy(); + })); + +test('outside embed mode a photo stays in the panel', () => + withDom(async (dom) => { + const home = dom.wrap.parentNode; + const { layer, controls } = await stubPanel(dom, uiState({ open: false })); + await layer.publish(uiState({ open: true })); + assert.equal(overGlobe(dom), false); + assertSameNode(dom.wrap.parentNode, home); + controls.destroy(); + })); + +test('in embed mode Esc closes the photo, and the next photo is shown over the globe again', () => + withDom(async (dom) => { + dom.document.body.classList.add('ui-embed'); + const { layer, controls } = await stubPanel(dom, uiState({ open: false })); + let closed = 0; + layer.closeViewer = () => { + closed++; + layer.publish(uiState({ open: false })); + }; + await layer.publish(uiState({ open: true })); + escape(); + await nextTick(); + assert.equal(closed, 1, 'Esc closed the photo'); + assert.equal(overGlobe(dom), false); + await layer.publish(uiState({ open: true })); + assert.equal(overGlobe(dom), true, 'the next photo is shown'); + controls.destroy(); + })); + +test('in embed mode Esc in full screen leaves full screen and keeps the photo', () => + withDom(async (dom) => { + dom.document.body.classList.add('ui-embed'); + const { layer, controls } = await stubPanel(dom, uiState({ open: false })); + let closed = 0; + layer.closeViewer = () => closed++; + await layer.publish(uiState({ open: true })); + await dom.wrap.requestFullscreen(); + await settle(); + assert.equal(controls.isViewerExpanded(), true); + escape(); + assert.equal(closed, 0, 'the photo stays open'); + controls.destroy(); + })); + +test('outside embed mode Esc does not close a photo in the panel', () => + withDom(async (dom) => { + const { layer, controls } = await stubPanel(dom, uiState({ open: true })); + let closed = 0; + layer.closeViewer = () => closed++; + escape(); + assert.equal(closed, 0); + controls.destroy(); + })); + +test('destroying the panel in embed mode puts an open photo back in the panel', () => + withDom(async (dom) => { + dom.document.body.classList.add('ui-embed'); + const home = dom.wrap.parentNode; + const { layer, controls } = await stubPanel(dom, uiState({ open: false })); + await layer.publish(uiState({ open: true })); + controls.destroy(); + assert.equal(overGlobe(dom), false); + assertSameNode(dom.wrap.parentNode, home); + })); diff --git a/src/ui/styles/embed.css b/src/ui/styles/embed.css index 5528a76da..a376ac5da 100644 --- a/src/ui/styles/embed.css +++ b/src/ui/styles/embed.css @@ -13,3 +13,19 @@ body.ui-embed #world-overlay-actions, body.ui-embed #view-switcher { display: none !important; } + +/* Street-level photos: the rail that holds the panel is hidden here, so an + open photo shows over the globe (streetLevelControls moves the viewer). */ +body.ui-embed .sl-viewer-wrap-embed { + position: fixed; + inset: 12px; + z-index: 1000; +} + +body.ui-embed .sl-viewer-wrap-embed .sl-viewer-stage, +body.ui-embed .sl-viewer-wrap-embed .sl-viewer { + flex: 1 1 auto; + height: auto; + min-height: 0; + aspect-ratio: auto; +} diff --git a/src/view/index.js b/src/view/index.js index 022914277..5caecdd70 100644 --- a/src/view/index.js +++ b/src/view/index.js @@ -11,6 +11,7 @@ import { decodeLayerStateParams, encodeLayerStateParams, } from '../data/layerState.js'; +import { encodeIdSwitches } from '../data/idSwitches.js'; import { RETIRED_STYLES } from '../visualStylePolicy.js'; /** Visual styles by internal name, with the name share links use. */ @@ -65,6 +66,63 @@ const FOLLOWABLE = Object.freeze({ export const VIEW_FOLLOW_KINDS = Object.freeze(Object.keys(FOLLOWABLE)); +/** + * Street-level imagery sources a view can open, the layer that shows them and + * the map a source needs (Google's terms allow Street View beside Google maps + * only). Literal here: this module stays portable. + */ +const STREET_LEVEL_LAYER = 'street-level'; +export const VIEW_STREET_LEVEL_PROVIDERS = Object.freeze([ + 'google', + 'mapillary', + 'kartaview', + 'panoramax', +]); +const STREET_LEVEL_MAP = Object.freeze({ google: 'photoreal' }); +const STREET_LEVEL_PARAM = 'sl'; +const EARTH_RADIUS_M = 6_371_000; + +/** Above this a camera frames a region, not a street: photos open under it. */ +const LOOK_AT_MAX_ALTITUDE_M = 20_000; +/** Flatter than this the look-at point is at the horizon: use the camera's. */ +const LOOK_AT_MIN_PITCH_DEG = -5; +/** Further ahead than this the camera frames a region: use its own point. */ +const LOOK_AT_MAX_DISTANCE_M = 10_000; + +/** + * The ground point a street-scale camera looks at: straight below it when it + * looks down, else ahead along its heading by altitude / tan(pitch). A camera + * with no altitude, above street scale, aimed at the horizon, or looking + * further than 10 km ahead gives its own point. Longitude wraps at the + * antimeridian. + */ +function lookAtPoint(camera) { + const pitch = Number(camera.pitch_deg ?? -90); + const altitude = Number(camera.altitude_m); + if ( + !(altitude > 0) || + altitude > LOOK_AT_MAX_ALTITUDE_M || + !(pitch > -89) || + pitch > LOOK_AT_MIN_PITCH_DEG + ) + return { lat: camera.lat, lon: camera.lon }; + const ground = altitude / Math.tan((-pitch * Math.PI) / 180); + if (ground > LOOK_AT_MAX_DISTANCE_M) + return { lat: camera.lat, lon: camera.lon }; + const heading = ((Number(camera.heading_deg) || 0) * Math.PI) / 180; + const dLat = + ((ground * Math.cos(heading)) / EARTH_RADIUS_M) * (180 / Math.PI); + const cosLat = Math.cos((camera.lat * Math.PI) / 180) || 1e-6; + const dLon = + ((ground * Math.sin(heading)) / (EARTH_RADIUS_M * cosLat)) * + (180 / Math.PI); + const lon = camera.lon + dLon; + return { + lat: clamp(camera.lat + dLat, -90, 90), + lon: lon > 180 ? lon - 360 : lon < -180 ? lon + 360 : lon, + }; +} + /** Annotation kinds the app draws, as in its annotate_map action. */ export const VIEW_ANNOTATION_TYPES = Object.freeze([ 'pin', @@ -162,6 +220,21 @@ export const VIEW_PROPERTIES = Object.freeze({ additionalProperties: false, }, }), + street_level: Object.freeze({ + type: 'object', + description: + 'Open street-level photos at a point: Google Street View (google; the ' + + 'map becomes Google 3D), Mapillary, KartaView or Panoramax. lat/lon ' + + 'default to the spot the camera looks at. The app flies there and ' + + 'opens the nearest photo; a plain link with google only arms it, ' + + 'leaving the billed load to a click. Not with follow.', + properties: { + provider: { type: 'string', enum: [...VIEW_STREET_LEVEL_PROVIDERS] }, + lat: { type: 'number', minimum: -90, maximum: 90 }, + lon: { type: 'number', minimum: -180, maximum: 180 }, + }, + additionalProperties: false, + }), follow: Object.freeze({ type: 'object', description: @@ -251,6 +324,7 @@ export function createView({ map = null, follow = null, annotations = [], + street_level: streetLevel = null, } = {}) { if (!Number.isFinite(camera?.lat) || !Number.isFinite(camera?.lon)) throw new TypeError('A view needs a camera lat and lon'); @@ -264,10 +338,37 @@ export function createView({ throw new TypeError(`Cannot follow a ${follow.kind}`); if (follow?.cockpit === true && !followed.cockpit) throw new TypeError(`A ${follow.kind} has no cockpit view`); + let street = null; + if (streetLevel) { + if (follow) + throw new TypeError( + 'A view cannot both follow an entity and open street-level photos', + ); + const provider = streetLevel.provider ?? null; + if (provider !== null && !VIEW_STREET_LEVEL_PROVIDERS.includes(provider)) + throw new TypeError(`Unknown street-level source: ${provider}`); + const needs = provider ? STREET_LEVEL_MAP[provider] || null : null; + if (needs && map !== null && map !== needs) + throw new TypeError( + `${provider} street-level imagery shows only on the ${needs} map`, + ); + if (needs) map = needs; + // The point defaults to where the camera looks: a view "of X" opens + // photos at X, also when the camera sits behind it. + const seen = lookAtPoint(camera); + const lat = latitude(streetLevel.lat) ?? seen.lat; + const lon = longitude(streetLevel.lon) ?? seen.lon; + street = Object.freeze({ + ...(provider ? { provider } : {}), + lat: round(clamp(lat, -90, 90), 5), + lon: round(clamp(lon, -180, 180), 5), + }); + } const enabled = new Set( layers.filter((layer) => REGISTERED_LAYER_IDS.includes(layer)), ); if (followed) enabled.add(followed.layer); + if (street) enabled.add(STREET_LEVEL_LAYER); return Object.freeze({ camera: Object.freeze({ lat: round(clamp(camera.lat, -90, 90), 4), @@ -296,6 +397,7 @@ export function createView({ .map(annotationOf) .filter(Boolean), ), + street_level: street, }); } @@ -321,8 +423,29 @@ export function viewToParams(view) { ...state.options[followed.owner], [followed.key]: view.follow.id, }; + // The restored layer lights the source the photos come from. + const provider = view.street_level?.provider; + if (provider) + state.options[STREET_LEVEL_LAYER] = { + ...state.options[STREET_LEVEL_LAYER], + mapillary: provider === 'mapillary', + providerSwitches: encodeIdSwitches( + VIEW_STREET_LEVEL_PROVIDERS.filter((id) => id !== 'mapillary').map( + (id) => [id, id === provider], + ), + ), + }; encodeLayerStateParams(params, state); } + if (view.street_level) + params.set( + STREET_LEVEL_PARAM, + [ + view.street_level.provider ?? '', + view.street_level.lat.toFixed(5), + view.street_level.lon.toFixed(5), + ].join(','), + ); if (view.annotations.length) { const encoded = JSON.stringify(view.annotations); if (encoded.length <= MAX_ANNOTATION_PARAM_CHARS) @@ -345,6 +468,40 @@ export function annotationsFromParams(params) { } } +/** + * The street-level photos a share link opens (`sl=,,`, the + * source optional), or null. + */ +export function streetLevelFromParams(params) { + const raw = params.get(STREET_LEVEL_PARAM); + if (!raw) return null; + // Three fields, plain decimal degrees: that alone bounds the value. + const parts = raw.split(','); + if (parts.length !== 3) return null; + const [provider, lat, lon] = parts; + if (provider && !VIEW_STREET_LEVEL_PROVIDERS.includes(provider)) return null; + // Plain decimal degrees only: no exponents, hex or blanks. + const DECIMAL = /^-?\d{1,3}(\.\d{1,8})?$/; + if (!DECIMAL.test(lat) || !DECIMAL.test(lon)) return null; + const point = { lat: latitude(Number(lat)), lon: longitude(Number(lon)) }; + if (point.lat === undefined || point.lon === undefined) return null; + return { ...(provider ? { provider } : {}), ...point }; +} + +/** The entity a decoded layer state follows, or null. */ +function followFromState(state) { + for (const [kind, followed] of Object.entries(FOLLOWABLE)) { + const id = state?.options?.[followed.owner]?.[followed.key]; + if (id !== null && id !== undefined) return { kind, id: String(id) }; + } + return null; +} + +/** The entity a share link follows, or null. */ +export function followFromParams(params) { + return followFromState(decodeLayerStateParams(params)); +} + /** The view a set of share-link hash parameters restores, or null. */ export function viewFromParams(params) { const lat = Number.parseFloat(params.get('lat')); @@ -355,13 +512,12 @@ export function viewFromParams(params) { return Number.isFinite(value) ? value : undefined; }; const state = decodeLayerStateParams(params); - let follow = null; - for (const [kind, followed] of Object.entries(FOLLOWABLE)) { - const id = state?.options?.[followed.owner]?.[followed.key]; - if (id !== null && id !== undefined && !follow) - follow = { kind, id: String(id) }; - } + const follow = followFromState(state); const map = params.get('map'); + // A link is untrusted input: its photos win over a map their source cannot + // show on, and a followed entity keeps the camera. + const street = follow ? null : streetLevelFromParams(params); + const needs = street?.provider ? STREET_LEVEL_MAP[street.provider] : null; return createView({ camera: { lat, @@ -376,9 +532,10 @@ export function viewFromParams(params) { : Object.hasOwn(STYLES_BY_URL_NAME, params.get('style')) ? STYLES_BY_URL_NAME[params.get('style')] : null, - map: VIEW_MAPS.includes(map) ? map : null, + map: needs ? needs : VIEW_MAPS.includes(map) ? map : null, follow, annotations: annotationsFromParams(params), + street_level: street, }); } diff --git a/src/view/index.test.mjs b/src/view/index.test.mjs index 66630e6a0..1389b058b 100644 --- a/src/view/index.test.mjs +++ b/src/view/index.test.mjs @@ -127,6 +127,379 @@ test('annotations travel in the link, keeping only what the app draws', async () assert.equal(viewToParams(many).has('an'), false); }); +test('street-level photos travel in the link, light their source and bring their map', async () => { + const { streetLevelFromParams, VIEW_STREET_LEVEL_PROVIDERS } = + await import('./index.js'); + const { decodeLayerStateParams } = await import('../data/layerState.js'); + assert.deepEqual(VIEW_STREET_LEVEL_PROVIDERS, [ + 'google', + 'mapillary', + 'kartaview', + 'panoramax', + ]); + const view = createView({ + camera: { lat: 38.5799, lon: -121.4937, altitude_m: 400 }, + street_level: { provider: 'google' }, + }); + assert.equal(view.map, 'photoreal', 'Street View brings Google 3D'); + assert.deepEqual(view.layers, ['street-level']); + assert.deepEqual(view.street_level, { + provider: 'google', + lat: 38.5799, + lon: -121.4937, + }); + const params = viewToParams(view); + assert.equal(params.get('sl'), 'google,38.57990,-121.49370'); + assert.deepEqual(viewFromParams(params), view); + const state = decodeLayerStateParams(params); + assert.deepEqual(state.enabledLayerIds, ['street-level']); + assert.equal(state.options['street-level'].mapillary, false); + assert.equal( + state.options['street-level'].providerSwitches, + 'google-1*kartaview-0*panoramax-0', + 'Street View on, KartaView off', + ); + + const mapillary = createView({ + camera: { lat: 1, lon: 2 }, + map: 'osm', + street_level: { provider: 'mapillary', lat: 1.00001, lon: 2.00001 }, + }); + assert.equal(mapillary.map, 'osm', 'Mapillary shows on any map'); + assert.deepEqual(viewFromParams(viewToParams(mapillary)), mapillary); + const lit = decodeLayerStateParams(viewToParams(mapillary)).options[ + 'street-level' + ]; + assert.equal(lit.mapillary, true); + assert.equal( + lit.providerSwitches, + 'google-0*kartaview-0*panoramax-0', + 'a Mapillary link lights Mapillary only', + ); + const anySource = createView({ + camera: { lat: 1, lon: 2 }, + street_level: {}, + }); + assert.deepEqual(anySource.street_level, { lat: 1, lon: 2 }); + assert.equal(viewToParams(anySource).get('sl'), ',1.00000,2.00000'); + assert.deepEqual(viewFromParams(viewToParams(anySource)), anySource); + + assert.throws( + () => + createView({ + camera: { lat: 1, lon: 2 }, + map: 'osm', + street_level: { provider: 'google' }, + }), + /photoreal/, + ); + assert.throws( + () => + createView({ + camera: { lat: 1, lon: 2 }, + street_level: { provider: 'openstreetcam' }, + }), + /Unknown street-level source/, + ); + assert.equal( + streetLevelFromParams(new URLSearchParams('sl=google,x,y')), + null, + ); + assert.equal(streetLevelFromParams(new URLSearchParams('sl=bing,1,2')), null); + assert.equal(streetLevelFromParams(new URLSearchParams('lat=1')), null); + assert.equal(createView({ camera: { lat: 1, lon: 2 } }).street_level, null); +}); + +test('street-level photos: a tilted camera opens them where it looks, a link cannot pair them with the wrong map, and follow is refused', async () => { + const { streetLevelFromParams } = await import('./index.js'); + // Looking north from 1,000 m at 45°: the ground point is ~1 km north. + const tilted = createView({ + camera: { + lat: 48.8494, + lon: 2.2945, + altitude_m: 1000, + heading_deg: 0, + pitch_deg: -45, + }, + street_level: { provider: 'google' }, + }); + assert.ok( + tilted.street_level.lat > 48.858 && tilted.street_level.lat < 48.859, + String(tilted.street_level.lat), + ); + assert.equal(tilted.street_level.lon, 2.2945); + const down = createView({ + camera: { lat: 1, lon: 2, altitude_m: 1000 }, + street_level: {}, + }); + assert.deepEqual(down.street_level, { lat: 1, lon: 2 }); + // A hand-edited link: the photos win over the map. + const link = viewFromParams( + new URLSearchParams('v=2&lat=1&lon=2&map=osm&sl=google,1,2'), + ); + assert.equal(link.map, 'photoreal'); + assert.deepEqual(link.street_level, { provider: 'google', lat: 1, lon: 2 }); + assert.throws( + () => + createView({ + camera: { lat: 1, lon: 2 }, + follow: { kind: 'aircraft', id: 'a1' }, + street_level: {}, + }), + /cannot both follow/, + ); + // A followed link drops its photos rather than fight over the camera. + const followed = viewFromParams( + viewToParams( + createView({ + camera: { lat: 1, lon: 2 }, + follow: { kind: 'satellite', id: '25544' }, + }), + ), + ); + assert.equal(followed.street_level, null); + // Strict parsing: three fields, real numbers, in range, bounded length. + for (const bad of [ + 'google,1', + 'google,1,2,3', + 'google,1abc,2', + 'google,91,0', + 'google,,2', + 'google,1,2'.padEnd(65, '0'), + ]) + assert.equal( + streetLevelFromParams(new URLSearchParams({ sl: bad })), + null, + bad, + ); +}); + +test('the look-at default is street-scale only, follows the heading, and wraps at the antimeridian', async () => { + const { followFromParams, streetLevelFromParams } = + await import('./index.js'); + const at = (camera) => + createView({ camera, street_level: { provider: 'mapillary' } }) + .street_level; + // Looking east from 1,000 m at 45° in Paris: ~1 km east is ~0.0137° of longitude. + const east = at({ + lat: 48.85, + lon: 2.2945, + altitude_m: 1000, + heading_deg: 90, + pitch_deg: -45, + }); + assert.ok(Math.abs(east.lon - (2.2945 + 0.01365)) < 0.0005, String(east.lon)); + assert.ok(Math.abs(east.lat - 48.85) < 0.0002, String(east.lat)); + // Looking west from the same place lands west. + assert.ok( + at({ + lat: 48.85, + lon: 2.2945, + altitude_m: 1000, + heading_deg: 270, + pitch_deg: -45, + }).lon < 2.2945, + ); + // At the horizon, with no altitude, or above street scale: the camera's own point. + assert.deepEqual(at({ lat: 1, lon: 2, altitude_m: 1000, pitch_deg: 0 }), { + provider: 'mapillary', + lat: 1, + lon: 2, + }); + assert.deepEqual(at({ lat: 1, lon: 2, pitch_deg: -30 }), { + provider: 'mapillary', + lat: 1, + lon: 2, + }); + assert.deepEqual(at({ lat: 1, lon: 2, altitude_m: 50_000, pitch_deg: -30 }), { + provider: 'mapillary', + lat: 1, + lon: 2, + }); + // Across the antimeridian the longitude wraps instead of clamping. + const wrapped = at({ + lat: 0, + lon: 179.999, + altitude_m: 1000, + heading_deg: 90, + pitch_deg: -45, + }); + assert.ok(wrapped.lon < -179.9, String(wrapped.lon)); + + // A followed link keeps its camera: the photos are dropped even when both are in the hash. + const followed = viewToParams( + createView({ + camera: { lat: 1, lon: 2 }, + follow: { kind: 'satellite', id: '25544' }, + }), + ); + followed.set('sl', 'google,1.00000,2.00000'); + assert.deepEqual(followFromParams(followed), { + kind: 'satellite', + id: '25544', + }); + assert.equal(viewFromParams(followed).street_level, null); + assert.equal(followFromParams(new URLSearchParams('v=2&lat=1&lon=2')), null); + + // Strict decimal degrees: no exponents, hex, blanks or over-long values. + for (const bad of [ + 'google,1e1,2', + 'google,0x10,2', + 'google, 1,2', + 'google,1.,2', + `google,${'1.0000000'.padEnd(60, '0')},2`, + ]) + assert.equal( + streetLevelFromParams(new URLSearchParams({ sl: bad })), + null, + bad, + ); + assert.deepEqual( + streetLevelFromParams(new URLSearchParams({ sl: ',-1.5,2' })), + { lat: -1.5, lon: 2 }, + ); +}); + +test('the look-at thresholds are pinned: 20 km up, -5° of pitch, 10 km ahead', () => { + const at = (camera) => createView({ camera, street_level: {} }).street_level; + const own = { lat: 1, lon: 2 }; + const moved = (point) => point.lat !== own.lat || point.lon !== own.lon; + // Altitude ceiling, at a steep pitch that keeps the distance under 10 km. + assert.ok( + moved(at({ ...own, altitude_m: 19_999, pitch_deg: -70 })), + '19,999 m applies', + ); + assert.ok( + !moved(at({ ...own, altitude_m: 20_001, pitch_deg: -70 })), + '20,001 m does not', + ); + // Pitch cutoff, at 100 m so the distance cap cannot decide it (1.4 km ahead). + assert.ok( + moved(at({ ...own, altitude_m: 100, pitch_deg: -6 })), + '-6° applies', + ); + assert.ok( + !moved(at({ ...own, altitude_m: 100, pitch_deg: -4 })), + '-4° does not', + ); + // Distance cap, pinned either side of 10 km at 1,000 m: -5.8° looks 9.84 km + // ahead, -5.6° 10.2 km. + assert.ok( + moved(at({ ...own, altitude_m: 1000, pitch_deg: -5.8 })), + '9.84 km ahead applies', + ); + assert.ok( + !moved(at({ ...own, altitude_m: 1000, pitch_deg: -5.6 })), + '10.2 km ahead does not', + ); + assert.ok( + !moved(at({ ...own, altitude_m: 10_000, pitch_deg: -30 })), + '17 km ahead does not', + ); + assert.ok( + moved(at({ ...own, altitude_m: 5_000, pitch_deg: -30 })), + '8.7 km ahead applies', + ); + // Near-vertical stays put (no tan blow-up). + assert.ok(!moved(at({ ...own, altitude_m: 1000, pitch_deg: -89.5 }))); + // Western wrap and the pole clamp. + const west = at({ + lat: 0, + lon: -179.999, + altitude_m: 1000, + heading_deg: 270, + pitch_deg: -45, + }); + assert.ok(west.lon > 179.9, String(west.lon)); + const pole = at({ + lat: 89.9995, + lon: 0, + altitude_m: 1000, + heading_deg: 0, + pitch_deg: -45, + }); + assert.ok(pole.lat <= 90, String(pole.lat)); +}); + +test('sl rejects a bad longitude slot and over-precise values, and no length bound hides the regex', async () => { + const { streetLevelFromParams } = await import('./index.js'); + for (const bad of [ + 'google,1,1e1', + 'google,1,0x10', + 'google,1, 2', + 'google,1,2.', + 'google,1.123456789,2', + 'google,1,2.123456789', + ]) + assert.equal( + streetLevelFromParams(new URLSearchParams({ sl: bad })), + null, + bad, + ); + assert.deepEqual( + streetLevelFromParams( + new URLSearchParams({ sl: 'google,-89.12345678,179.12345678' }), + ), + { provider: 'google', lat: -89.12345678, lon: 179.12345678 }, + ); +}); + +test('KartaView photos travel in a link and light KartaView alone, on any map', async () => { + const { streetLevelFromParams } = await import('./index.js'); + const { decodeLayerStateParams } = await import('../data/layerState.js'); + const view = createView({ + camera: { lat: 52.52, lon: 13.405, altitude_m: 400 }, + map: 'osm', + street_level: { provider: 'kartaview' }, + }); + assert.equal(view.map, 'osm', 'KartaView needs no particular map'); + assert.deepEqual(view.layers, ['street-level']); + const params = viewToParams(view); + assert.equal(params.get('sl'), 'kartaview,52.52000,13.40500'); + assert.deepEqual(viewFromParams(params), view); + const lit = decodeLayerStateParams(params).options['street-level']; + assert.equal(lit.mapillary, false); + assert.equal( + lit.providerSwitches, + 'google-0*kartaview-1*panoramax-0', + 'KartaView on, the others off', + ); + assert.deepEqual( + streetLevelFromParams(new URLSearchParams({ sl: 'kartaview,1,2' })), + { + provider: 'kartaview', + lat: 1, + lon: 2, + }, + ); +}); + +test('Panoramax photos travel in a link and light Panoramax alone, on any map', async () => { + const { streetLevelFromParams } = await import('./index.js'); + const { decodeLayerStateParams } = await import('../data/layerState.js'); + const view = createView({ + camera: { lat: 48.8874, lon: 2.3495, altitude_m: 400 }, + map: 'osm', + street_level: { provider: 'panoramax' }, + }); + assert.equal(view.map, 'osm', 'Panoramax needs no particular map'); + assert.deepEqual(view.layers, ['street-level']); + const params = viewToParams(view); + assert.equal(params.get('sl'), 'panoramax,48.88740,2.34950'); + assert.deepEqual(viewFromParams(params), view); + const lit = decodeLayerStateParams(params).options['street-level']; + assert.equal(lit.mapillary, false); + assert.equal( + lit.providerSwitches, + 'google-0*kartaview-0*panoramax-1', + 'Panoramax on, the others off', + ); + assert.deepEqual( + streetLevelFromParams(new URLSearchParams({ sl: 'panoramax,1,2' })), + { provider: 'panoramax', lat: 1, lon: 2 }, + ); +}); + test('a view naming the retired Snow style opens on Normal', () => { const camera = { lat: 1, lon: 2 }; assert.equal(createView({ camera, style: 'snow' }).style, 'normal'); diff --git a/src/voice/actionSchemas.js b/src/voice/actionSchemas.js index 9ec47028c..3e19d25a0 100644 --- a/src/voice/actionSchemas.js +++ b/src/voice/actionSchemas.js @@ -103,6 +103,36 @@ const schemas = [ required: ['layerId'], }, }, + { + name: 'show_street_level', + parameters: { + type: 'object', + additionalProperties: false, + properties: { + query: { + type: 'string', + maxLength: 160, + }, + latitude: { + type: 'number', + minimum: -90, + maximum: 90, + }, + longitude: { + type: 'number', + minimum: -180, + maximum: 180, + }, + provider: { + type: 'string', + enum: ['google', 'mapillary', 'kartaview', 'panoramax'], + }, + referent: { + type: 'integer', + }, + }, + }, + }, { name: 'adjust_camera_zoom', parameters: { diff --git a/src/voice/actionSchemas.test.mjs b/src/voice/actionSchemas.test.mjs index d15a7c25e..9bf2caaaa 100644 --- a/src/voice/actionSchemas.test.mjs +++ b/src/voice/actionSchemas.test.mjs @@ -38,21 +38,38 @@ test('the complete Realtime tool payload pins the manifest-generated layer relea // consolidated tool wording (each policy stated once); model-facing tools // omit top-level anyOf (OpenAI rejects it). Contacts list routing preserves // requested radius and center authority in descriptions only. - // Street Level and GNSS extend the manifest; tracked and analyst aircraft describe broadcasts. - // Air quality (ECCC AQHI) extends the manifest. - // Globe rotation adds one motion; visual keyhole fit and explicit - // reset guidance/defaults update only move_camera and mode descriptions. - // S2 GhostMaps extends the generated layer enums. - // The Utilities panel (Directions, Draw, Import) adds one panel target. - // Panel adds named voice targets and timeout; shared Panel/Cockpit guidance - // is consolidated without changing either feature's arguments. - // Snow leaves the set_visual_style enum. - // The Operator HUD layout leaves set_hud's layout enum. - // The layer alias hint leaves out aliases the layer id already spells. + // Street Level, GNSS and air quality (ECCC AQHI) extend the manifest; + // tracked and analyst aircraft describe broadcasts; globe rotation adds + // one motion; S2 GhostMaps, the Utilities panel target and the voice panel + // (named targets, timeout) follow; Snow leaves the set_visual_style enum, the Operator HUD layout + // leaves set_hud's layout enum and the layer alias hint leaves out aliases + // the layer id already spells; show_street_level (map, source, photo) is + // the only change on top of main. + SHOW_STREET_LEVEL_PAYLOAD_DIGEST, + ); + // Every shipped tool is byte-identical to the previous release: the new + // tool is the whole difference, so a later edit cannot hide in a re-pin. + const shipped = createHash('sha256') + .update( + JSON.stringify( + stable( + GEV_REALTIME_TOOLS.filter( + (tool) => + !['set_cyber_sonar', 'show_street_level'].includes(tool.name), + ), + ), + ), + ) + .digest('hex'); + assert.equal( + shipped, 'dc2bac1be93a503dc88d637beea2f1bc457e1f9167422bd91548b44756a8aca5', ); }); +const SHOW_STREET_LEVEL_PAYLOAD_DIGEST = + '2198c1f95cfbc19e89ba8814ad0fd8303a74a7b254703f2e7a4893f5798d9aed'; + test('descriptions customize wording without changing immutable shared arguments', () => { const descriptions = { fly_to_location: { @@ -112,6 +129,7 @@ test('all legacy action arguments are byte-identical after removing the delibera 'next_satellite_pass', 'set_cyber_sonar', 'set_voice_inactivity_timeout', + 'show_street_level', ].includes(tool.name), ); // Layer enums are generated from the voice layer manifest and pinned by diff --git a/src/voice/deixis.js b/src/voice/deixis.js index 25d7939eb..0c21f297e 100644 --- a/src/voice/deixis.js +++ b/src/voice/deixis.js @@ -330,6 +330,7 @@ function resolveCropPixels(args, { imageFrame, cameraKey }) { const RESOLVERS = Object.freeze({ track_entity: resolveTrack, fly_to_location: resolveFly, + show_street_level: resolveFly, get_entity_context: resolveContext, analyst_query: resolveAnalyst, annotate_map: resolveAnnotate, diff --git a/src/voice/deixis.test.mjs b/src/voice/deixis.test.mjs index bd2e66f1b..cd7f7374d 100644 --- a/src/voice/deixis.test.mjs +++ b/src/voice/deixis.test.mjs @@ -180,3 +180,10 @@ test('numbered custom-labeled pins retain their resolved coordinates', () => { assert.deepEqual(resolveDeicticArgs('fly_to_location', { referent: 1 }, { referents }).args, { latitude: 30.27, longitude: -97.74 }); }); + +test('show_street_level: "street view of this" opens photos at the pointed ground point', () => { + const out = resolveDeicticArgs('show_street_level', { query: 'pointer', provider: 'google' }, { pointer: ground }); + assert.deepEqual(out.args, { provider: 'google', latitude: 30.27, longitude: -97.74 }); + assert.equal(out.used.source, 'pointer'); + assert.deepEqual(resolveDeicticArgs('show_street_level', { query: 'Eiffel Tower' }, { pointer: ground }).args, { query: 'Eiffel Tower' }); +}); diff --git a/src/voice/gevActions.js b/src/voice/gevActions.js index 70636dd4e..e2c4cf964 100644 --- a/src/voice/gevActions.js +++ b/src/voice/gevActions.js @@ -34,6 +34,10 @@ import { isContextRecordActive, } from '../data/contextStore.js'; import { CCTV_FOCUS_RESULT } from '../layers/cctv/index.js'; +import { + NEAREST_RADIUS_MAX_M as STREET_LEVEL_WIDE_RADIUS_M, + STREET_LEVEL_LAYER_ID, +} from '../layers/streetLevel/policy.js'; import { createAnalystEngine, normalizeAnalystSpec, @@ -731,6 +735,18 @@ export function createGevActionRunner({ }; } + if (name === 'show_street_level') { + return showStreetLevel(args, { + dataManager, + styleManager, + placeSearch, + runGevAction, + runOptions, + current, + reportProgress, + }); + } + if (name === 'set_visual_style') { const style = normalizeStyle(args.style); if (!style) @@ -2582,6 +2598,29 @@ function stopAllTracking(viewer, dataManager) { return { ok: true, action: 'stop_tracking', released }; } +/** + * The imagery filter the lit source honours when it is narrower than the + * default (360° or flat only, or SINCE), or null. + * @returns {{ pano?: string, sinceDays?: number, label: string } | null} + */ +export function activeImageryFilter(ui) { + const filter = ui?.filter || {}; + const controls = ui?.filterControls || {}; + const out = {}; + const names = []; + if (controls.pano && (filter.pano === 'pano' || filter.pano === 'flat')) { + out.pano = filter.pano; + names.push(filter.pano === 'pano' ? '360°' : 'flat-photo'); + } + if (controls.since && Number(filter.sinceDays) > 0) { + out.sinceDays = Number(filter.sinceDays); + names.push('SINCE'); + } + if (!names.length) return null; + out.label = `${names.join(' and ')} filter${names.length > 1 ? 's' : ''}`; + return out; +} + /** * Frames entities near the current view target with a cinematic pull-back: * oblique high pitch for aircraft/ships, shallow wide pitch for satellites. @@ -2589,6 +2628,7 @@ function stopAllTracking(viewer, dataManager) { * detection so the framed entities are labeled, and reports * detectionEnabled so the voice agent can mention labels are on. */ + async function frameOverhead(viewer, dataManager, styleManager, args = {}) { const targetRaw = String(args.target || 'flights').toLowerCase(); const layerId = @@ -3245,6 +3285,311 @@ function normalizeStyle(value) { return null; } +/** + * "Show me on Street View / on Mapillary": fly to the place, make sure + * the imagery source's map is up, light that source in Street Level, turn the + * layer on and open the nearest image. One source at a time, as the panel + * does; the choice persists like a chip click. The key is checked before + * anything changes, and each stage reports its own failure so speech can say + * which part did not happen. With no place at all, the view centre is used. + * `autoOpen: false` (links, not the model) stops short of loading a photo + * from a source that bills every load: the panel is armed and the OPEN + * button is one click away. + */ +export async function showStreetLevel( + args, + { + dataManager, + styleManager, + placeSearch, + runGevAction, + runOptions = {}, + current = () => true, + reportProgress = () => {}, + }, +) { + const action = 'show_street_level'; + let provider = null; + let providerName = null; + let label = null; + const fail = (stage, error, extra = {}) => ({ + ok: false, + action, + stage, + cancelled: !current(), + error, + ...(provider ? { provider } : {}), + ...(providerName ? { providerName } : {}), + ...(label ? { label } : {}), + ...extra, + }); + const query = String(args.query || '').trim(); + const latitude = Number(args.latitude); + const longitude = Number(args.longitude); + const hasCoordinates = + args.latitude != null && + args.longitude != null && + Number.isFinite(latitude) && + Number.isFinite(longitude); + // A referent that is a layer entity (deixis): located when the flight starts. + const entity = + args.entity && typeof args.entity === 'object' && args.entity.id + ? args.entity + : null; + const layerModule = + dataManager?.layers?.get?.(STREET_LEVEL_LAYER_ID)?.module || null; + if (!layerModule || typeof layerModule.openNearest !== 'function') + return fail('provider', 'Street Level is unavailable'); + const providerIds = Array.isArray(layerModule.providerIds) + ? layerModule.providerIds + : []; + + // The source: asked for, else the one lit now, else the first registered. + const requested = String(args.provider || '') + .trim() + .toLowerCase(); + if (requested) { + if (!providerIds.includes(requested)) + return fail( + 'provider', + `${requested} is not a street-level imagery source here`, + ); + provider = requested; + } else { + const params = dataManager.getLayerParams?.(STREET_LEVEL_LAYER_ID) || {}; + provider = + providerIds.find((id) => params[id] === true) || providerIds[0] || null; + } + if (!provider) + return fail('provider', 'No street-level imagery source is configured'); + const status = layerModule.providerStatus?.(provider) || null; + providerName = status?.name || provider; + + // The key first, on any map: a chip without a key explains it and switches + // nothing, and so does this. + if (status && (status.keyRequired === true || status.configured === false)) + return fail('key', status.error || `${providerName} needs its key`, { + keyRejected: status.keyRejected === true, + keyAlternatives: layerModule.getUIState?.()?.keyAlternatives || [], + }); + + // Then the place: a name that geocodes to nothing changes nothing else. + label = + query || + (entity ? String(entity.label || entity.id) : '') || + (hasCoordinates ? `${latitude.toFixed(4)}, ${longitude.toFixed(4)}` : '') || + 'the view centre'; + const atViewCentre = !query && !hasCoordinates && !entity; + let point = null; + let location = null; + if (args.fly === false || atViewCentre) { + if (hasCoordinates) point = { lat: latitude, lon: longitude }; + else if (entity) { + // Where it is now, as the flying path does: the coordinates a list + // saved go stale as soon as the item moves or unloads. + const located = locateLayerEntity(dataManager, entity); + if (!located) return fail('location', `${label} is no longer loaded`); + point = { lat: located.lat, lon: located.lon }; + } else if (query) { + reportProgress('search', query); + const outcome = await placeSearch.geocode(query, { + signal: runOptions.signal, + }); + const place = outcome?.place; + if (!current()) + return fail('location', 'Street-level request was cancelled'); + if (!Number.isFinite(place?.lat) || !Number.isFinite(place?.lng)) + return fail('location', `Could not find ${query}`); + point = { lat: place.lat, lon: place.lng }; + label = place.label || query; + } + // No place at all: the layer looks under the view centre. + } else { + location = await runGevAction( + 'fly_to_location', + { + ...(entity + ? { entity } + : hasCoordinates + ? { latitude, longitude } + : { query }), + viewMode: 'close', + waitForArrival: true, + }, + runOptions, + ); + if (location?.ok !== true || !current()) + return fail( + 'location', + location?.error || `Could not arrive at ${location?.label || label}`, + { cancelled: !current() || Boolean(location?.cancelled), location }, + ); + // An item keeps its name: the flight to it reports only coordinates. + label = entity ? label : location.label || label; + if ( + !Number.isFinite(location.latitude) || + !Number.isFinite(location.longitude) + ) + return fail( + 'location', + `${label} has no position to look for imagery at`, + { location }, + ); + point = { lat: location.latitude, lon: location.longitude }; + } + + // The source's map, when it needs one (Street View: Google 3D only). + const requiredStack = layerModule.requiredMapStack?.(provider) || null; + const activeStack = styleManager?.mapStackController?.getActiveId?.() ?? null; + if (requiredStack && activeStack !== requiredStack) { + reportProgress('map', requiredStack); + const map = await runGevAction( + 'set_map_stack', + { stack: requiredStack }, + runOptions, + ); + if (map?.ok !== true || !current()) + return fail( + 'map', + map?.error || + `${providerName} needs the ${requiredStack} map, which is not available`, + { map, location }, + ); + } + + // Light the source the way a chip click does, so the choice is kept. + const switches = Object.fromEntries( + providerIds.map((id) => [id, id === provider]), + ); + let accepted = true; + if (typeof dataManager.setLayerParams === 'function') + accepted = + dataManager.setLayerParams(STREET_LEVEL_LAYER_ID, switches, { + origin: 'voice', + }) !== false; + else layerModule.setProviderEnabled?.(provider, true); + if (!accepted) + return fail('provider', `${providerName} could not be switched on`, { + location, + }); + let layer = null; + if (!dataManager.isEnabled?.(STREET_LEVEL_LAYER_ID)) { + layer = await runGevAction( + 'set_layer_visibility', + { layerId: STREET_LEVEL_LAYER_ID, enabled: true }, + runOptions, + ); + if (layer?.ok !== true || !current()) + return fail( + 'layer', + layer?.error || 'Street Level could not be enabled', + { cancelled: !current() || Boolean(layer?.cancelled), layer, location }, + ); + } + + // Listed now that its map is up, and lit. + const ui = layerModule.getUIState?.() || {}; + const snapshot = + (ui.providers || []).find((entry) => entry.id === provider) || null; + if (!snapshot) + return fail( + 'provider', + ui.coverage?.hint || `${providerName} is not available on this map`, + { location }, + ); + if (snapshot.on !== true) + return fail('provider', `${providerName} could not be switched on`, { + location, + }); + + // A billed source never loads from a link without a click: everything is + // ready and OPEN is one press away at the view centre. + if (args.autoOpen === false && status?.billed === true) { + // The next OPEN press loads exactly what the link asked for. + layerModule.armNearest?.(point, { + providerIds: [provider], + ...(snapshot.groundClick === true + ? { radiusM: STREET_LEVEL_WIDE_RADIUS_M } + : {}), + }); + return { + ok: true, + action, + provider, + providerName, + label, + armed: true, + flew: location !== null, + ...(location ? { location: location.label } : {}), + }; + } + + // A source with no coverage to click (Street View) is searched wider: a + // geocoded landmark is rarely within 50 m of a street. A newer turn + // retires the lookup: nothing opens after the user moved on. + reportProgress('imagery', label); + const opened = await layerModule.openNearest(point, { + providerIds: [provider], + frame: false, + ...(runOptions.signal ? { signal: runOptions.signal } : {}), + ...(snapshot.groundClick === true + ? { radiusM: STREET_LEVEL_WIDE_RADIUS_M } + : {}), + }); + const street = layerModule.getUIState?.()?.street || {}; + // A cancelled request opens nothing (the lookup and the viewer both honour + // the turn's signal); a photo that did open is reported as open. + if (opened !== true && !current()) + return fail('imagery', 'Street-level request was cancelled', { + cancelled: true, + location, + }); + if (opened !== true) { + // Why nothing opened: the key, a change under the lookup, or no imagery. + const after = layerModule.providerStatus?.(provider) || status || {}; + if (after.keyRequired === true || after.keyRejected === true) + return fail( + 'key', + street.error || after.error || `${providerName} needs its key`, + { keyRejected: after.keyRejected === true, location }, + ); + if ( + !dataManager.isEnabled?.(STREET_LEVEL_LAYER_ID) || + after.on === false || + after.available === false + ) + return fail( + 'imagery', + street.error || 'Street Level changed while it looked for imagery', + { cancelled: true, location }, + ); + // An imagery filter the source honours can hide every photo here; say + // so, or the answer reads as "no coverage" when one switch would do. + const filter = activeImageryFilter(layerModule.getUIState?.()); + const reason = street.error || `No ${providerName} imagery near ${label}`; + return fail( + 'imagery', + filter ? `${reason} with the ${filter.label} on` : reason, + { location, ...(filter ? { filter } : {}) }, + ); + } + return { + ok: true, + action, + provider, + providerName, + label, + imageId: street.imageId ?? null, + title: street.title ?? null, + capturedAt: street.capturedAt ?? null, + capturedAtPrecision: street.capturedAtPrecision ?? null, + position: street.position ?? null, + externalUrl: street.externalUrl ?? null, + flew: location !== null, + ...(location ? { location: location.label } : {}), + }; +} + async function flyToRequestedLocation( viewer, args, @@ -3309,11 +3654,25 @@ async function flyToRequestedLocation( ); if (result === false) return cancelled(CITY_POIS[locationId]?.name || locationId); + // Where the camera went: the first POI for a close view, else the middle + // of the city's overview bounds. + const city = CITY_POIS[locationId]; + const overview = + !(rangeM || args.viewMode === 'close') && Boolean(city?.viewBounds); + const centre = overview + ? { + lat: + (city.viewBounds.southwest.lat + city.viewBounds.northeast.lat) / 2, + lon: + (city.viewBounds.southwest.lng + city.viewBounds.northeast.lng) / 2, + } + : city?.pois?.[0] || null; const response = { ok: Boolean(result), action: 'fly_to_location', locationId, label: CITY_POIS[locationId]?.name || locationId, + ...(centre ? { latitude: centre.lat, longitude: centre.lon } : {}), rangeM: result?.range ? Math.round(result.range) : rangeM || null, navigationMode: rangeM ? 'explicit-range' @@ -3374,6 +3733,7 @@ async function flyToRequestedLocation( action: 'fly_to_location', query, label: poi?.name || query, + ...(poi ? { latitude: poi.lat, longitude: poi.lon } : {}), navigationMode: rangeM ? 'preset-poi-range' : 'preset-poi', rangeM: result?.range ? Math.round(result.range) : rangeM || null, }; @@ -3404,6 +3764,10 @@ async function flyToRequestedLocation( action: 'fly_to_location', query, label: destination?.label || query, + ...(Number.isFinite(destination?.latitude) && + Number.isFinite(destination?.longitude) + ? { latitude: destination.latitude, longitude: destination.longitude } + : {}), navigationMode: destination?.navigationMode || null, rangeM: destination?.rangeM || rangeM || null, }; diff --git a/src/voice/gevActions.test.mjs b/src/voice/gevActions.test.mjs index f11abc5ae..ec0c76e5d 100644 --- a/src/voice/gevActions.test.mjs +++ b/src/voice/gevActions.test.mjs @@ -20,6 +20,7 @@ import { normalizeStackId, } from './gevActions.js'; import { MAP_STACKS } from '../mapStackController.js'; +import { KEY_REJECTED_MESSAGE } from '../layers/streetLevel/providers/google/policy.js'; import { GEV_REALTIME_TOOLS } from '../../server/providers/openai/tools.js'; import { presentResult } from './resultDisplay.js'; import { attachVoiceResult } from './speech.js'; @@ -3987,6 +3988,538 @@ test('analyst-only lookup distinguishes duplicate names and uses canonical aircr assert.equal(plane.latitude, 36, 'canonical ID resolves the current position, not old coordinates'); }); +/** A Street Level layer stub, a map stack and a data manager for show_street_level. */ +function createStreetLevelHarness({ + activeStack = 'osm', + lit = { mapillary: true, google: false }, + enabled = false, + keyRequired = false, + configured = !keyRequired, + nearest = 'pano-a', + paramsAccepted = true, + enableResult = true, + listGoogle = true, + duringLookup = null, + keyRejectedAtStart = false, + lightFails = false, +} = {}) { + const order = []; + const { viewer, styleManager } = createVoiceNavigationHarness(); + let completeFlight = null; + viewer.camera.lookAt = () => {}; + viewer.camera.flyToBoundingSphere = (_sphere, options) => { + order.push('fly'); + completeFlight = options.complete; + }; + styleManager.runImmediateLocationNavigation = (navigate) => + styleManager.runImmediateNavigation('location', navigate); + let stack = activeStack; + styleManager.mapStackController = { getActiveId: () => stack }; + styleManager.setMapStack = async (id) => { + order.push(`map:${id}`); + stack = id; + return { ok: true, activeStack: id, error: null }; + }; + let params = { ...lit }; + let keyRejected = keyRejectedAtStart; + const street = { imageId: null, error: null }; + const names = { google: 'Google Street View', mapillary: 'Mapillary' }; + const layer = { + providerIds: ['mapillary', 'google'], + requiredMapStack: (id) => (id === 'google' ? 'photoreal' : null), + providerStatus: (id) => + names[id] + ? { + id, + name: names[id], + on: params[id] === true, + configured, + keyRequired: keyRequired || keyRejected, + keyRejected, + error: keyRejected ? KEY_REJECTED_MESSAGE : keyRequired ? `${names[id]} needs its key` : null, + groundClick: id === 'google', + billed: id === 'google', + available: id !== 'google' || stack === 'photoreal', + } + : null, + getUIState: () => ({ + providers: ['mapillary', 'google'] + .filter((id) => id !== 'google' || (stack === 'photoreal' && listGoogle)) + .map((id) => ({ + id, + name: id === 'google' ? 'Google Street View' : 'Mapillary', + on: params[id] === true, + configured: !keyRequired, + keyRequired, + groundClick: id === 'google', + hint: keyRequired ? `${id} needs its key` : '', + error: null, + })), + keyAlternatives: [], + coverage: { hint: 'not on this map' }, + street: { ...street }, + }), + armNearest(point, options) { + order.push(`arm:${options.providerIds.join('+')}:${options.radiusM ?? 'default'}:${point ? `${point.lat},${point.lon}` : 'view-centre'}`); + }, + async openNearest(point, options) { + order.push(`open:${options.providerIds.join('+')}:${options.radiusM ?? 'default'}:${options.frame}${point ? '' : ':view-centre'}`); + if (options.signal?.aborted) return false; + if (duringLookup) { + const effect = duringLookup(); + if (effect === 'key-rejected') { + keyRejected = true; + street.error = KEY_REJECTED_MESSAGE; + return false; + } + if (effect === 'layer-off') { + enabled = false; + return false; + } + } + if (!nearest) { + street.error = `No street-level imagery within ${options.radiusM ?? 50} m of ${point ? 'that point' : 'the view centre'}`; + return false; + } + street.imageId = nearest; + street.title = '10th St, Sacramento'; + street.position = point; + return true; + }, + }; + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + isEnabled: () => enabled, + getLayerParams: () => ({ ...params }), + setLayerParams(id, next, { origin }) { + order.push(`params:${Object.entries(next).filter(([, on]) => on).map(([k]) => k).join('+')}:${origin}`); + if (paramsAccepted === false) return false; + if (!lightFails) params = { ...params, ...next }; + return true; + }, + async setEnabled(id) { + order.push(`enable:${id}`); + if (enableResult === false) return false; + enabled = true; + return true; + }, + getAll: () => [{ id: 'street-level', name: 'Street Level', enabled }], + }; + const runner = createGevActionRunner({ viewer, styleManager, dataManager }); + return { runner, order, finishFlight: () => completeFlight?.(), layer, dataManager }; +} + +/** A loaded CCTV camera the entity branch can locate. */ +function withCamera(harness) { + harness.dataManager.layers.set('cctv', { + module: { + findByQuery: (id) => (id === 'cam-9' ? { latitude: 38.58, longitude: -121.5 } : null), + }, + }); + const wasEnabled = harness.dataManager.isEnabled; + harness.dataManager.isEnabled = (id) => id === 'cctv' || wasEnabled(id); + return harness; +} + +test('show_street_level flies first, then switches the map, lights the source, turns the layer on and opens the nearest photo', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { runner, order, finishFlight } = createStreetLevelHarness(); + const pending = runner('show_street_level', { latitude: 38.5799, longitude: -121.4937, provider: 'google' }); + await new Promise((resolve) => setImmediate(resolve)); + assert.deepEqual(order, ['fly'], 'nothing changes until the place is reached'); + finishFlight(); + const result = await pending; + assert.equal(result.ok, true, result.error); + assert.deepEqual(order, [ + 'fly', + 'map:photoreal', + 'params:google:voice', + 'enable:street-level', + 'open:google:250:false', + ]); + assert.equal(result.provider, 'google'); + assert.equal(result.providerName, 'Google Street View'); + assert.equal(result.imageId, 'pano-a'); + assert.equal(result.title, '10th St, Sacramento'); + assert.equal(result.flew, true); +}); + +test('show_street_level on Mapillary keeps the map, searches at the default radius and skips what is already on', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { runner, order } = createStreetLevelHarness({ enabled: true, activeStack: 'osm' }); + const result = await runner('show_street_level', { + latitude: 38.58, + longitude: -121.49, + provider: 'mapillary', + fly: false, + }); + assert.equal(result.ok, true, result.error); + assert.deepEqual(order, ['params:mapillary:voice', 'open:mapillary:default:false']); + assert.equal(result.flew, false); +}); + +test('show_street_level without a source uses the one lit in the panel', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { runner, order } = createStreetLevelHarness({ + activeStack: 'photoreal', + lit: { mapillary: false, google: true }, + enabled: true, + }); + const result = await runner('show_street_level', { latitude: 1, longitude: 2, fly: false }); + assert.equal(result.ok, true, result.error); + assert.equal(result.provider, 'google'); + assert.deepEqual(order, ['params:google:voice', 'open:google:250:false']); +}); + +test('show_street_level reports each stage it could not pass', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const centre = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); + const none = await centre.runner('show_street_level', { provider: 'google' }); + assert.equal(none.ok, true, 'no place at all means the view centre'); + assert.equal(none.label, 'the view centre'); + assert.deepEqual(centre.order, ['params:google:voice', 'open:google:250:false:view-centre']); + + const unknown = await createStreetLevelHarness().runner('show_street_level', { + latitude: 1, longitude: 2, provider: 'openstreetcam', fly: false, + }); + assert.deepEqual([unknown.ok, unknown.stage], [false, 'provider']); + assert.match(unknown.error, /openstreetcam/); + + const keylessHarness = createStreetLevelHarness({ enabled: true, keyRequired: true }); + const keyless = await keylessHarness.runner('show_street_level', { + latitude: 1, longitude: 2, provider: 'google', fly: false, + }); + assert.deepEqual([keyless.ok, keyless.stage], [false, 'key']); + assert.match(keyless.error, /needs its key/); + assert.equal(keyless.providerName, 'Google Street View'); + assert.deepEqual(keylessHarness.order, [], 'a keyless source changes nothing: no flight, no map, no switch'); + + const refusedSwitch = await createStreetLevelHarness({ enabled: true, paramsAccepted: false }).runner( + 'show_street_level', + { latitude: 1, longitude: 2, provider: 'google', fly: false }, + ); + assert.deepEqual([refusedSwitch.ok, refusedSwitch.stage], [false, 'provider']); + assert.match(refusedSwitch.error, /could not be switched on/); + + const empty = await createStreetLevelHarness({ enabled: true, nearest: null }).runner( + 'show_street_level', + { latitude: 1, longitude: 2, provider: 'google', fly: false }, + ); + assert.deepEqual([empty.ok, empty.stage], [false, 'imagery']); + assert.match(empty.error, /within 250 m/); +}); + +test('show_street_level stops at the map when the source’s map is refused, before touching the panel', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { layer, order } = createStreetLevelHarness({ enabled: true }); + const reason = 'Google 3D needs GOOGLE_MAPS_API_KEY'; + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + isEnabled: () => true, + getLayerParams: () => ({ mapillary: true, google: false }), + setLayerParams: () => order.push('params'), + getAll: () => [], + }; + const { viewer, styleManager } = createVoiceNavigationHarness(); + styleManager.mapStackController = { getActiveId: () => 'osm' }; + styleManager.setMapStack = async () => ({ ok: false, activeStack: 'osm', error: reason }); + const runner = createGevActionRunner({ viewer, styleManager, dataManager }); + const result = await runner('show_street_level', { latitude: 1, longitude: 2, provider: 'google', fly: false }); + assert.deepEqual([result.ok, result.stage, result.error], [false, 'map', reason]); + assert.deepEqual(order, [], 'the source is not switched when its map is refused'); +}); + +test('show_street_level opens photos of a numbered item or a pointed spot through its entity', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { runner, order, finishFlight } = withCamera( + createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }), + ); + const pending = runner('show_street_level', { + provider: 'google', + entity: { layerId: 'cctv', id: 'cam-9', label: 'Camera 9', lat: 38.58, lon: -121.5 }, + }); + await new Promise((resolve) => setImmediate(resolve)); + assert.deepEqual(order, ['fly'], 'the entity position is flown to'); + finishFlight(); + const result = await pending; + assert.equal(result.ok, true, result.error); + assert.equal(result.label, 'Camera 9'); + assert.deepEqual(order, ['fly', 'params:google:voice', 'open:google:250:false']); + // Without a flight the entity's own position is used. + const still = withCamera(createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' })); + const kept = await still.runner('show_street_level', { + provider: 'google', + fly: false, + entity: { layerId: 'cctv', id: 'cam-9', lat: 38.58, lon: -121.5 }, + }); + assert.equal(kept.ok, true, kept.error); + assert.deepEqual(still.order, ['params:google:voice', 'open:google:250:false']); +}); + +test('show_street_level hands the point from a named place, a preset and a curated POI to the lookup', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const points = []; + const make = () => { + const harness = createStreetLevelHarness({ enabled: true, activeStack: 'osm' }); + const open = harness.layer.openNearest; + harness.layer.openNearest = async (point, options) => { + points.push(point); + return open(point, options); + }; + return harness; + }; + // A geocoded name: the flight reports where it landed. + const named = make(); + const viewerFor = () => ({ ...createVoiceNavigationHarness().viewer }); + const runnerWithSearch = createGevActionRunner({ + viewer: viewerFor(), + styleManager: { mapStackController: { getActiveId: () => 'osm' }, setMapStack: async () => ({ ok: true }) }, + dataManager: { + layers: new Map([['street-level', { module: named.layer }]]), + isEnabled: () => true, + getLayerParams: () => ({ mapillary: true, google: false }), + setLayerParams: () => true, + getAll: () => [], + }, + searchNavigation: async (_viewer, query, options) => { + options.onComplete?.(); + return { label: `${query}, CA`, latitude: 38.5799, longitude: -121.4937, navigationMode: 'precise-place', rangeM: 250 }; + }, + }); + const geocoded = await runnerWithSearch('show_street_level', { query: '10th Street, Sacramento', provider: 'mapillary' }); + assert.equal(geocoded.ok, true, geocoded.error); + assert.equal(geocoded.label, '10th Street, Sacramento, CA'); + assert.deepEqual(points.at(-1), { lat: 38.5799, lon: -121.4937 }); + // A curated POI and a city preset fly to their hand-tuned poses and report them. + for (const [args, expected] of [ + [{ query: 'Golden Gate Bridge', provider: 'mapillary' }, { lat: 37.8199, lon: -122.4783 }], + [{ query: 'austin', provider: 'mapillary' }, null], + ]) { + const harness = make(); + const pending = harness.runner('show_street_level', args); + await new Promise((resolve) => setImmediate(resolve)); + harness.finishFlight(); + const result = await pending; + assert.equal(result.ok, true, `${args.query}: ${result.error}`); + const point = points.at(-1); + if (expected) assert.deepEqual(point, expected); + else assert.ok(Number.isFinite(point.lat) && Number.isFinite(point.lon), 'a preset flight reports its point'); + } +}); + +test('show_street_level retires its lookup when the turn is superseded, and says so', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { runner, order, layer } = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); + const controller = new AbortController(); + let currentTurn = true; + layer.openNearest = async (_point, options) => { + order.push(`open:signal:${Boolean(options.signal)}`); + controller.abort(); + currentTurn = false; + return false; + }; + const result = await runner( + 'show_street_level', + { latitude: 1, longitude: 2, provider: 'google', fly: false }, + { signal: controller.signal, isCurrent: () => currentTurn }, + ); + assert.deepEqual([result.ok, result.cancelled, result.stage], [false, true, 'imagery']); + assert.doesNotMatch(result.error, /No .* imagery/); + assert.ok(order.includes('open:signal:true'), 'the lookup gets the turn signal'); +}); + +test('show_street_level with autoOpen:false arms a billed source and leaves the load to a click; a free source still opens', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const armedHarness = createStreetLevelHarness({ enabled: true, activeStack: 'osm' }); + const armed = await armedHarness.runner('show_street_level', { + latitude: 38.5799, longitude: -121.4937, provider: 'google', fly: false, autoOpen: false, + }); + assert.equal(armed.ok, true, armed.error); + assert.equal(armed.armed, true); + assert.equal(armed.imageId, undefined); + assert.deepEqual( + armedHarness.order, + ['map:photoreal', 'params:google:voice', 'arm:google:250:38.5799,-121.4937'], + 'map and source are ready, the lookup is armed at the point, nothing is loaded', + ); + assert.match(armed.say, /^Google Street View is ready at .*press OPEN/); + assert.deepEqual( + { ...armed, say: undefined, display: undefined }, + { ok: true, action: 'show_street_level', provider: 'google', providerName: 'Google Street View', label: '38.5799, -121.4937', armed: true, flew: false, say: undefined, display: undefined }, + ); + const free = createStreetLevelHarness({ enabled: true, activeStack: 'osm' }); + const opened = await free.runner('show_street_level', { + latitude: 38.5799, longitude: -121.4937, provider: 'mapillary', fly: false, autoOpen: false, + }); + assert.equal(opened.ok, true, opened.error); + assert.equal(opened.armed, undefined); + assert.equal(opened.imageId, 'pano-a'); +}); + +test('show_street_level tells a failed enable, a delisted source, a key refused mid-lookup and a layer switched off apart', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const at = { latitude: 1, longitude: 2, provider: 'google', fly: false }; + const notEnabled = await createStreetLevelHarness({ activeStack: 'photoreal', enableResult: false }).runner('show_street_level', at); + assert.deepEqual([notEnabled.ok, notEnabled.stage, notEnabled.cancelled], [false, 'layer', false]); + assert.match(notEnabled.say, /Couldn't turn on Street Level/); + + const delisted = await createStreetLevelHarness({ enabled: true, activeStack: 'photoreal', listGoogle: false }).runner('show_street_level', at); + assert.deepEqual([delisted.ok, delisted.stage], [false, 'provider']); + assert.equal(delisted.error, 'not on this map'); + + const refused = await createStreetLevelHarness({ enabled: true, activeStack: 'photoreal', duringLookup: () => 'key-rejected' }).runner('show_street_level', at); + assert.deepEqual([refused.ok, refused.stage, refused.keyRejected], [false, 'key', true]); + assert.equal(refused.say, 'Google Street View refused its key.'); + + const switchedOff = await createStreetLevelHarness({ enabled: true, activeStack: 'photoreal', duringLookup: () => 'layer-off' }).runner('show_street_level', at); + assert.deepEqual([switchedOff.ok, switchedOff.stage, switchedOff.cancelled], [false, 'imagery', true]); + assert.equal(switchedOff.say, undefined, 'a request the user withdrew is not spoken'); + + // A source whose status call says it is not configured is key-gated even + // before its coverage reports a missing key. + const unconfigured = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal', configured: false }); + const gated = await unconfigured.runner('show_street_level', at); + assert.deepEqual([gated.ok, gated.stage], [false, 'key']); + assert.deepEqual(unconfigured.order, [], 'nothing changed'); +}); + +test('show_street_level without a flight geocodes a name, locates an item, and reports what it cannot place', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const hit = createStreetLevelHarness({ enabled: true, activeStack: 'osm' }); + const points = []; + const open = hit.layer.openNearest; + hit.layer.openNearest = (point, options) => { + points.push(point); + return open(point, options); + }; + const runner = createGevActionRunner({ + viewer: createVoiceNavigationHarness().viewer, + styleManager: { mapStackController: { getActiveId: () => 'osm' }, setMapStack: async () => ({ ok: true }) }, + dataManager: hit.dataManager, + placeSearch: { + geocode: async (query) => + query === 'Capitol Mall' + ? { place: { lat: 38.5792, lng: -121.4958, label: 'Capitol Mall, Sacramento' } } + : { place: null, answered: true }, + }, + }); + const found = await runner('show_street_level', { query: 'Capitol Mall', provider: 'mapillary', fly: false }); + assert.equal(found.ok, true, found.error); + assert.equal(found.label, 'Capitol Mall, Sacramento'); + assert.deepEqual(points.at(-1), { lat: 38.5792, lon: -121.4958 }); + assert.equal(found.flew, false); + + const miss = await runner('show_street_level', { query: 'Atlantis Boulevard', provider: 'mapillary', fly: false }); + assert.deepEqual([miss.ok, miss.stage], [false, 'location']); + assert.equal(miss.say, "Couldn't find Atlantis Boulevard."); + assert.equal(points.length, 1, 'a miss asks for no imagery'); + + const gone = await runner('show_street_level', { + provider: 'mapillary', fly: false, entity: { layerId: 'cctv', id: 'cam-gone', label: 'Camera 7' }, + }); + assert.deepEqual([gone.ok, gone.stage], [false, 'location']); + assert.equal(gone.say, "Couldn't find Camera 7."); + + // A flight that reports no coordinates cannot be followed by a lookup. + const noPoint = createGevActionRunner({ + viewer: createVoiceNavigationHarness().viewer, + styleManager: { mapStackController: { getActiveId: () => 'osm' }, setMapStack: async () => ({ ok: true }) }, + dataManager: hit.dataManager, + searchNavigation: async (_viewer, query, options) => { + options.onComplete?.(); + return { label: query, navigationMode: 'city-overview', rangeM: null }; + }, + }); + const unplaced = await noPoint('show_street_level', { query: 'Somewhere', provider: 'mapillary' }); + assert.deepEqual([unplaced.ok, unplaced.stage], [false, 'location']); + assert.match(unplaced.error, /no position/); +}); + +test('fly_to_location reports the overview centre for a city framed as a whole, and the first POI for a close view', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const { viewer, styleManager } = createVoiceNavigationHarness(); + viewer.camera.lookAt = () => {}; + viewer.camera.flyTo = (options) => options.complete?.(); + viewer.camera.flyToBoundingSphere = (_s, options) => options.complete?.(); + styleManager.runImmediateLocationNavigation = (navigate) => styleManager.runImmediateNavigation('location', navigate); + const runner = createGevActionRunner({ viewer, styleManager, dataManager: { layers: new Map(), getAll: () => [] } }); + const overview = await runner('fly_to_location', { locationId: 'austin' }); + assert.equal(overview.ok, true); + assert.ok(Math.abs(overview.latitude - 30.31) < 0.001, String(overview.latitude)); + assert.ok(Math.abs(overview.longitude - -97.75) < 0.001, String(overview.longitude)); + const close = await runner('fly_to_location', { locationId: 'austin', viewMode: 'close' }); + assert.deepEqual([close.latitude, close.longitude], [30.2747, -97.7403]); +}); + +test('show_street_level arms the view centre when a billed link names no place, and keeps the flight label when it flew', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const centre = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); + const armed = await centre.runner('show_street_level', { provider: 'google', autoOpen: false }); + assert.equal(armed.ok, true, armed.error); + assert.equal(armed.armed, true); + assert.deepEqual(centre.order, ['params:google:voice', 'arm:google:250:view-centre'], 'armed at the view centre, nothing opened'); + const flown = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); + const pending = flown.runner('show_street_level', { latitude: 38.5799, longitude: -121.4937, provider: 'google', autoOpen: false }); + await new Promise((resolve) => setImmediate(resolve)); + flown.finishFlight(); + const result = await pending; + assert.equal(result.armed, true); + assert.equal(result.flew, true); + assert.equal(result.location, '38.5799, -121.4937'); + assert.ok(!flown.order.some((entry) => entry.startsWith('open:')), 'no load after the flight'); +}); + +test('show_street_level says "refused its key" when Google already refused it, and fails when the source will not light', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const refusedHarness = createStreetLevelHarness({ enabled: true, keyRejectedAtStart: true }); + const refused = await refusedHarness.runner('show_street_level', { latitude: 1, longitude: 2, provider: 'google', fly: false }); + assert.deepEqual([refused.ok, refused.stage, refused.keyRejected], [false, 'key', true]); + assert.equal(refused.error, KEY_REJECTED_MESSAGE); + assert.equal(refused.say, 'Google Street View refused its key.'); + assert.deepEqual(refusedHarness.order, [], 'nothing changed'); + const dark = await createStreetLevelHarness({ enabled: true, activeStack: 'photoreal', lightFails: true }).runner( + 'show_street_level', + { latitude: 1, longitude: 2, provider: 'google', fly: false }, + ); + assert.deepEqual([dark.ok, dark.stage], [false, 'provider']); + assert.match(dark.error, /could not be switched on/); +}); + +test('show_street_level without a flight needs the item where it is now, not where a list saw it', async () => { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const harness = createStreetLevelHarness({ enabled: true, activeStack: 'photoreal' }); + // The referent carries the coordinates the list saved, but it is no longer loaded. + const stale = await harness.runner('show_street_level', { + provider: 'google', + fly: false, + entity: { layerId: 'cctv', id: 'cam-gone', label: 'Camera 7', lat: 38.58, lon: -121.5 }, + }); + assert.deepEqual([stale.ok, stale.stage], [false, 'location']); + assert.match(stale.error, /no longer loaded/); + assert.ok(!harness.order.some((entry) => entry.startsWith('open:')), 'nothing looked up at the stale point'); +}); + +test('activeImageryFilter reports only the narrowing filters the lit source honours', async () => { + const { activeImageryFilter } = await import('./gevActions.js'); + const both = { pano: true, since: true }; + assert.equal(activeImageryFilter({ filter: { pano: 'all', sinceDays: 0 }, filterControls: both }), null); + assert.deepEqual(activeImageryFilter({ filter: { pano: 'pano', sinceDays: 0 }, filterControls: both }), { + pano: 'pano', + label: '360° filter', + }); + assert.deepEqual(activeImageryFilter({ filter: { pano: 'flat', sinceDays: 365 }, filterControls: both }), { + pano: 'flat', + sinceDays: 365, + label: 'flat-photo and SINCE filters', + }); + assert.equal( + activeImageryFilter({ filter: { pano: 'pano', sinceDays: 365 }, filterControls: { pano: false, since: false } }), + null, + 'Street View honours neither', + ); + assert.equal(activeImageryFilter(undefined), null); +}); + test('set_panel_open accepts the Authoring name for the scene panel', async () => { const calls = []; const runner = createGevActionRunner({ diff --git a/src/voice/layerManifest.js b/src/voice/layerManifest.js index 962bf6949..44801dc0a 100644 --- a/src/voice/layerManifest.js +++ b/src/voice/layerManifest.js @@ -352,7 +352,12 @@ const layers = [ }, { id: 'street-level', - aliases: ['street level', 'street-level imagery', 'mapillary'], + aliases: [ + 'street level', + 'street-level imagery', + 'mapillary', + 'street view', + ], context: false, noQuery: 'street-level imagery; no countable records', }, diff --git a/src/voice/layerManifest.test.mjs b/src/voice/layerManifest.test.mjs index ab382c460..b35d83cc5 100644 --- a/src/voice/layerManifest.test.mjs +++ b/src/voice/layerManifest.test.mjs @@ -119,6 +119,7 @@ test('spoken aliases resolve to real layers, keep every legacy phrase and never 'weather radar': 'weather-radar', lightning: 'weather-lightning', 'military bases': 'military-installations', 'recent imagery': 'recent-imagery', 'bike share': 'bikeshare', 'rocket launches': 'rocket-launches', 'street level': 'street-level', mapillary: 'street-level', + 'street view': 'street-level', wind: 'wind', directions: 'directions', 'fire perimeters': 'fire-perimeters', 'wildfire perimeters': 'fire-perimeters', 'burn areas': 'fire-perimeters', }; diff --git a/src/voice/pointerReticle.js b/src/voice/pointerReticle.js index 5a3a64fdc..2a5dae764 100644 --- a/src/voice/pointerReticle.js +++ b/src/voice/pointerReticle.js @@ -27,6 +27,7 @@ const CAMERA_TOOLS = new Set([ 'fly_route', 'frame_overhead', 'select_nearest_aircraft', + 'show_street_level', 'control_cockpit', ]); diff --git a/src/voice/speech.js b/src/voice/speech.js index a0f65b573..f481d1338 100644 --- a/src/voice/speech.js +++ b/src/voice/speech.js @@ -24,6 +24,7 @@ import { analystHeadline } from './resultDisplay.js'; import { + formatDate, formatNumber, getLocale, msg, @@ -564,6 +565,87 @@ function selectNearestAircraft(result) { }; } +/** Month and year an image was captured, or ''. */ +function capturedMonth(capturedAt) { + const ms = Number(capturedAt); + if (!Number.isFinite(ms) || ms <= 0) return ''; + try { + // Dates are UTC months (Google: the first of the month). + return formatDate(new Date(ms), { + month: 'short', + year: 'numeric', + timeZone: 'UTC', + }); + } catch { + return ''; + } +} + +function showStreetLevel(result) { + // `location` is the flight's own result on a failed flight; its label is + // the place, never the object. + const place = spokenLabel( + result?.label || + (typeof result?.location === 'string' + ? result.location + : result?.location?.label) || + 'there', + ); + const source = spokenLabel(result?.providerName || 'Street Level'); + // The card in the interface language; the spoken line stays English. + const shownSource = result?.providerName + ? spokenLabel(result.providerName) + : t('Street Level'); + const shown = spokenLabel(localizePlaceName(result?.label || '')) || place; + if (result?.cancelled) return null; + if (!result?.ok) { + let say; + if (result?.stage === 'location') say = `Couldn't find ${place}.`; + else if (result?.stage === 'map') + say = `${source} needs the Google 3D map, which isn't available.`; + else if (result?.stage === 'key') + say = result?.keyRejected + ? `${source} refused its key.` + : `${source} needs its key.`; + else if (result?.stage === 'layer') say = "Couldn't turn on Street Level."; + else if (result?.stage === 'imagery') + say = result?.filter?.label + ? `No ${source} imagery near ${place} with the ${result.filter.label} on.` + : `No ${source} imagery near ${place}.`; + else say = `Couldn't open ${source}.`; + return { + say, + display: { + title: shownSource, + lines: errorLines(result), + sources: [], + }, + }; + } + if (result.armed === true) + return { + say: sentence(`${source} is ready at ${place}: press OPEN to load it`), + display: { + title: shownSource, + lines: [shown, t('Press OPEN to load the photo')], + sources: [{ label: shownSource }], + }, + }; + const title = spokenLabel(result.title || '', 60); + const month = capturedMonth(result.capturedAt); + return { + say: sentence(`${source} at ${title || place}`), + display: { + title: shownSource, + lines: [ + title || shown, + month ? t('Captured {month}', { month }) : null, + ].filter(Boolean), + sources: [{ label: shownSource }], + }, + }; +} + const SPOKEN_ROUTE = Object.freeze({ between: (origin, destination) => `${origin} to ${destination}`, from: (origin) => `from ${origin}`, @@ -771,6 +853,7 @@ export const SPEECH_BUILDERS = Object.freeze({ fly_to_location: flyToLocation, frame_overhead: frameOverhead, select_nearest_aircraft: selectNearestAircraft, + show_street_level: showStreetLevel, get_entity_context: getEntityContext, get_current_view_state: getCurrentViewState, annotate_map: annotateMap, @@ -826,6 +909,14 @@ function annotationTargets(args) { .filter(Boolean); } +/** Street-level sources as a plan step names them. */ +const STREET_LEVEL_SOURCE_NAMES = Object.freeze({ + google: 'Street View', + mapillary: 'Mapillary', + kartaview: 'KartaView', + panoramax: 'Panoramax', +}); + // frame_overhead targets as the schema names them. function frameStepLabel(target) { switch (target) { @@ -897,6 +988,17 @@ export function planStepLabel(name, args = {}) { ? t('Nearest aircraft to {place}', { place: shownPlace(place, 32) }) : t('Nearest aircraft to Here'); } + case 'show_street_level': { + const source = STREET_LEVEL_SOURCE_NAMES[args.provider]; + const place = args.query + ? shownPlace(args.query, 32) + : args.latitude != null + ? t('this spot') + : t('the view centre'); + return source + ? t('{source} at {place}', { source, place }) + : t('Street level at {place}', { place }); + } case 'set_layer_visibility': { const on = args.enabled !== false; const label = @@ -940,6 +1042,8 @@ export function narrationLabel(name, args = {}) { presetPlaceName(args.locationQuery || args.locationId || ''), 32, ); + case 'show_street_level': + return shownPlace(args.query || '', 32); default: return ''; } @@ -954,6 +1058,8 @@ const STEP_LABELS = Object.freeze({ layer: msg('Turning on layer'), refresh: msg('Loading aircraft'), nearest: msg('Picking nearest'), + map: msg('Switching map'), + imagery: msg('Opening imagery'), }); /** On-screen label for a progress step. */ @@ -988,6 +1094,12 @@ export function progressLine(step, label) { return 'Loading aircraft there.'; case 'nearest': return 'Picking the nearest aircraft.'; + case 'map': + return place === 'photoreal' + ? 'Switching to Google 3D.' + : 'Switching the map.'; + case 'imagery': + return place ? `Looking for imagery at ${place}.` : null; case 'still': return place ? `Still working on ${place}.` : 'Still working on it.'; default: diff --git a/src/voice/speech.test.mjs b/src/voice/speech.test.mjs index 04b04d9b4..eaaaed4cb 100644 --- a/src/voice/speech.test.mjs +++ b/src/voice/speech.test.mjs @@ -489,6 +489,154 @@ test('labels are bounded and cleaned before they are spoken', () => { ); }); +test('street-level speech names the source and the place, and says which step failed', () => { + const ok = envelope('show_street_level', { + ok: true, + action: 'show_street_level', + provider: 'google', + providerName: 'Google Street View', + label: '10th Street, Sacramento', + title: '1000 10th St, Sacramento, CA', + capturedAt: Date.UTC(2024, 4, 1), // Google dates: the first of the month, UTC + }); + assert.equal(ok.say, 'Google Street View at 1000 10th St, Sacramento, CA.'); + assertSpeakable(ok.say); + assert.equal(ok.display.title, 'Google Street View'); + assert.equal(ok.display.lines[1], 'Captured May 2024'); + + const stages = { + location: "Couldn't find Nowhere.", + map: "Google Street View needs the Google 3D map, which isn't available.", + key: 'Google Street View needs its key.', + layer: "Couldn't turn on Street Level.", + imagery: 'No Google Street View imagery near Nowhere.', + }; + for (const [stage, say] of Object.entries(stages)) { + const out = envelope('show_street_level', { + ok: false, + stage, + label: 'Nowhere', + providerName: 'Google Street View', + error: 'detail', + }); + assert.equal(out.say, say, stage); + assertSpeakable(out.say); + assert.deepEqual(out.display.lines, ['detail']); + } + const armed = envelope('show_street_level', { + ok: true, + armed: true, + providerName: 'Google Street View', + label: '10th St, Sacramento', + }); + assert.equal(armed.say, 'Google Street View is ready at 10th St, Sacramento: press OPEN to load it.'); + assertSpeakable(armed.say); + assert.deepEqual(armed.display.lines, ['10th St, Sacramento', 'Press OPEN to load the photo']); + const unfound = envelope('show_street_level', { + ok: false, + stage: 'location', + location: { ok: false, action: 'fly_to_location', label: 'Nowhere Town' }, + }); + assert.equal(unfound.say, "Couldn't find Nowhere Town."); + assert.equal(unfound.display.title, 'Street Level'); + const refused = envelope('show_street_level', { + ok: false, + stage: 'key', + keyRejected: true, + providerName: 'Google Street View', + }); + assert.equal(refused.say, 'Google Street View refused its key.'); + assert.equal( + envelope('show_street_level', { ok: false, cancelled: true }).say, + undefined, + ); + assert.equal( + planStepLabel('show_street_level', { + query: 'Eiffel Tower', + provider: 'google', + }), + 'Street View at Eiffel Tower', + ); + assert.equal( + planStepLabel('show_street_level', { + query: 'Times Square', + provider: 'mapillary', + }), + 'Mapillary at Times Square', + ); + assert.equal( + progressStepLabel('imagery', 'Eiffel Tower'), + 'Opening imagery: Eiffel Tower', + ); + assert.equal(progressLine('map', 'photoreal'), 'Switching to Google 3D.'); + assert.equal( + progressLine('imagery', 'Eiffel Tower'), + 'Looking for imagery at Eiffel Tower.', + ); +}); + +test('the capture month is a UTC month whatever the timezone, and labels name the place', async () => { + const { execFileSync } = await import('node:child_process'); + const speech = new URL('./speech.js', import.meta.url).href; + // Google dates a panorama 2024-05 as the first of the month, UTC. West of + // Greenwich that instant is still April; the card must still say May. + const out = execFileSync( + process.execPath, + [ + '--input-type=module', + '-e', + `import { attachVoiceResult } from ${JSON.stringify(speech)}; + const r = attachVoiceResult('show_street_level', { ok: true, providerName: 'Google Street View', label: 'x', capturedAt: Date.UTC(2024, 4, 1) }); + process.stdout.write(r.display.lines[1]);`, + ], + { env: { ...process.env, TZ: 'America/Los_Angeles' }, encoding: 'utf8' }, + ); + assert.equal(out, 'Captured May 2024'); + const { narrationLabel } = await import('./speech.js'); + assert.equal(narrationLabel('show_street_level', { query: 'Eiffel Tower' }), 'Eiffel Tower'); + assert.equal(narrationLabel('show_street_level', {}), ''); + assert.equal( + planStepLabel('show_street_level', { latitude: 1, longitude: 2 }), + 'Street level at this spot', + ); + assert.equal(planStepLabel('show_street_level', {}), 'Street level at the view centre'); +}); + +test('the plan step names KartaView as a source', () => { + assert.equal( + planStepLabel('show_street_level', { + query: 'Alexanderplatz', + provider: 'kartaview', + }), + 'KartaView at Alexanderplatz', + ); +}); + +test('an empty street-level search names the imagery filter that hid the photos', () => { + const out = envelope('show_street_level', { + ok: false, + stage: 'imagery', + label: 'Alexanderplatz', + providerName: 'KartaView', + filter: { pano: 'pano', label: '360° filter' }, + }); + assert.equal( + out.say, + 'No KartaView imagery near Alexanderplatz with the 360° filter on.', + ); + assertSpeakable(out.say); +}); + +test('the plan step names Panoramax as a source', () => { + assert.equal( + planStepLabel('show_street_level', { + query: 'Montmartre', + provider: 'panoramax', + }), + 'Panoramax at Montmartre', + ); +}); + test('progress step labels join through the catalog colon', () => { setLocale('fr'); try {