diff --git a/README.md b/README.md index dc119497a..4d5998aca 100644 --- a/README.md +++ b/README.md @@ -384,7 +384,7 @@ Click **GEV MIC**, grant the microphone, and just talk. This is more than a voic - **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director. - **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token. If you choose **USE CHATGPT OAUTH** in Provider Settings, the local server uses that sign-in only to mint the same kind of short-lived Realtime credential, and refuses the OAuth path for non-loopback clients. -Thirty-two tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook: +Thirty-five tools (thirty-four unless an Overpass server is set), four jobs — the commands below come straight from the product's voice test suite and tool playbook: **🎥 Direct it** — drone-operator camera verbs: diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 0c3577644..e93e21d79 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -5282,14 +5282,19 @@ app. newer turn, a shell navigation stamp (`_navigationGeneration`, re-taken where its own flight lands) or a press or wheel on the globe stops it (`superseded: true`). The same checks run before a found photo loads - (`proceed`), so nothing billed loads for a request that was given up, and a - lookup given up leaves the panel as it was; a photo that opened after either - is closed again. + (`proceed`), so nothing billed loads for a request that was given up. A + lookup given up leaves the panel as it was when the request kept the photo's + source; one that had switched the source off closes that source's photo with + it. A photo that opened after either is closed again. - **Replacing a photo.** An open photo stays up while the new request looks (`openNearest` `replace: true`; the viewer is replaced, not closed, so EXPAND - stays, also across sources through `holdViewer`) and closes if nothing - opens, so a place with no imagery shows its own error; a link that only arms - closes it at once. A replacement stops a playing Mapillary sequence. + stays, also across sources through `holdViewer`, which holds only the action's + own switch-off of the photo's source: a source switched by hand meanwhile + still stops the load) and closes if nothing opens, so a place with no imagery + shows its own error; a link that only arms closes it at once. A replacement + from another source drops the old photo's details, and the request's error or + armed OPEN outlives the old viewer. A replacement stops a playing Mapillary + sequence. - **Outcome.** `noImagery` marks an empty result, `imageryError` a source or viewer failure and `aim: 'sky'` a sky centre (from `street.nearestOutcome`); an empty result names a narrowing 360°/flat or SINCE filter the source @@ -5312,8 +5317,8 @@ error. Views carry the request as `street_level` (`src/view/index.js`): a source and a point, which defaults to where the camera looks (below 20 km, at a pitch -steeper than -5° and within 10 km ahead; else the camera's own point, also -for half a point). A `google` source brings `map: photoreal` and refuses +steeper than -5° and within 10 km ahead; else the camera's own point); half a +point is not mixed with the camera's: both coordinates come from that default. A `google` source brings `map: photoreal` and refuses another map; `street_level` does not combine with `follow`. `show_in_gods_eye_view`, and the embedded app for a posted view with no point, measure the view's altitude from the terrain under the camera @@ -6411,7 +6416,7 @@ and unreachable upstream (502/504) separately from road geometry. - **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware uses `OPENAI_API_KEY` by default, or the signed-in local ChatGPT/Codex OAuth access token when OAuth was explicitly selected in Provider Settings, and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. Both long-lived credentials stay server-side; OAuth minting is loopback-only. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token. With no OpenAI key the token route answers `503` with code `API_KEY_MISSING`, and the voice control shows `Needs OPENAI_API_KEY — add it in Provider Settings` in the interface language, reworded on a language change, without the microphone-permission hint. - **Session defaults** (env-tunable): model `gpt-realtime-2` (or `gpt-realtime-2.1-mini` when the MINI tier is selected — see the model-tier entry below), voice `marin`, reasoning effort `low`, semantic VAD with low eagerness, no response interruption, context window truncated to ~3,000 post-instruction tokens with 0.5 retention ratio — the conversational window stays short because map state is fetched live per turn. -- **Thirty-two tools** (argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `show_street_level`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_voice_inactivity_timeout`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `next_iss_pass`, and `next_satellite_pass`. +- **Thirty-five tools** (thirty-four in a session without `OVERPASS_UPSTREAMS`, which adds `osm_query`; argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `show_street_level`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_voice_inactivity_timeout`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `resolve_area`, `find_imagery`, `osm_query`, `next_iss_pass`, and `next_satellite_pass`. > **Reading `npm test` totals:** the count depends on the Node major. The two > GC-bracketed allocation microbenchmarks (`src/data/focusAllocations.test.mjs` diff --git a/src/app/embed.js b/src/app/embed.js index f4954e720..13e928df1 100644 --- a/src/app/embed.js +++ b/src/app/embed.js @@ -12,7 +12,10 @@ */ import * as Cesium from 'cesium'; -import { announceNavigationAuthority } from '../navigationPolicy.js'; +import { + announceNavigationAuthority, + watchCameraOwner, +} from '../navigationPolicy.js'; import { t, translateKnown } from '../i18n/index.js'; import { GOOGLE_LABEL } from '../layers/streetLevel/providers/google/policy.js'; import { @@ -234,10 +237,10 @@ export async function applyView( }, ) { const steps = []; - const act = async (name, args) => { + const act = async (name, args, options) => { let result = null; try { - result = await run(name, args); + result = await run(name, args, options); } catch (error) { result = { ok: false, error: error?.message || String(error) }; } @@ -270,9 +273,12 @@ export async function applyView( if (view.street_level) { // The photos frame the camera; when the action fails before flying, the // view's own camera still applies. + // A posted view replays someone else's view, as a link does: the source + // it lights is not saved as the visitor's own choice. const result = await act( 'show_street_level', streetLevelArgs(view.street_level, { autoOpen }), + { origin: 'programmatic' }, ); if (result?.ok !== true) { // Overtaken (a click on the map, a newer request): hands off. @@ -397,8 +403,10 @@ export function installViews({ // OPEN must be in view, but embed mode hides the rail that holds it: in // a frame a posted view is the way to open a photo. if (result?.armed === true && !isEmbedded(location)) { + // Opened for this link only: the recipient's saved layout stays theirs. shell.setPanelCollapsed?.(STREET_LEVEL_PANEL_ID, false, { explicit: true, + persist: false, }); // On a phone the panel sits in a tab sheet: open it so OPEN shows. shell.revealPanel?.(STREET_LEVEL_PANEL_ID); @@ -490,10 +498,25 @@ export function installViews({ queue = queue.then(async () => { if (signal?.aborted) return; if (unplaced && view.street_level) { - const point = await groundedLookAt(view.camera, terrain, signal).catch( - () => null, - ); + // Whoever takes the camera while the terrain answers (a drag, a + // search, the assistant) keeps it: the view stands down. + const owner = watchCameraOwner(viewer, shell); + const point = await groundedLookAt(view.camera, terrain, signal) + .catch(() => null) + .finally(() => owner.stop()); if (signal?.aborted) return; + if (owner.taken()) { + post( + { + type: EMBED_APPLIED_MESSAGE, + id, + ok: false, + steps: [{ step: 'street_level', ok: false, superseded: true }], + }, + replyOrigin, + ); + return; + } if (point) view = createView({ ...event.data.view, diff --git a/src/app/embed.test.mjs b/src/app/embed.test.mjs index cb34287a5..bcc5ada9b 100644 --- a/src/app/embed.test.mjs +++ b/src/app/embed.test.mjs @@ -85,6 +85,19 @@ test('a view is applied through the app actions in order', async () => { assert.ok(steps.some((step) => step.step === 'camera')); }); +test("a view's street-level step replays the sender's view: nothing it switches is the visitor's choice", async () => { + const { options, run } = recorder(); + await applyView( + createView({ + camera: { lat: 48.86, lon: 2.29 }, + street_level: { provider: 'panoramax', lat: 48.8584, lon: 2.2945 }, + }), + { viewer: fakeViewer(), dataManager: fakeLayers(new Set()), run }, + ); + const [, runOptions] = options.find(([name]) => name === 'show_street_level'); + assert.equal(runOptions?.origin, 'programmatic'); +}); + test('a followed entity is retried until its layer has it', async () => { let tries = 0; const { run } = recorder({ @@ -739,8 +752,8 @@ test('a plain link never auto-loads a billed photo: it arms the panel and says s ]); assert.deepEqual( panels, - [['street-level-panel', false, { explicit: true }]], - 'the panel opens so OPEN is in view', + [['street-level-panel', false, { explicit: true, persist: false }]], + "the panel opens so OPEN is in view, without saving the layout as the recipient's", ); assert.deepEqual(revealed, ['street-level-panel'], 'its phone sheet too'); // Adding ?embed=1 to a published link does not lift the gate: the page is diff --git a/src/i18n/locales/zh-Hant.json b/src/i18n/locales/zh-Hant.json index e9c6c300d..b10c726b2 100644 --- a/src/i18n/locales/zh-Hant.json +++ b/src/i18n/locales/zh-Hant.json @@ -3330,7 +3330,7 @@ "Refresh map still": "重新整理地圖靜態畫面", "The map still uses a separate request and may show a different moment.": "地圖靜態畫面使用獨立請求，顯示的時間可能不同。", "Video plays from Québec 511, which sees your IP address.": "影片從 Québec 511 播放，Québec 511 會看到你的 IP 位址。", - "Press OPEN to load the photo": "點擊“打開”載入照片", + "Press OPEN to load the photo": "點擊“打開”加載照片", "Captured {month}": "拍攝於 {month}", "this spot": "此處", "the view centre": "視圖中心", @@ -3338,9 +3338,9 @@ "Street level at {place}": "{place} 的街景", "Switching map": "正在切換地圖", "Opening imagery": "正在打開影像", - "Street View is ready: press {open} to load it": "街景已就緒：按 {open} 載入", - "{source} cannot open this link's street-level photo without a working key": "沒有有效金鑰，{source} 無法開啟此連結的街景照片", - "Street-level photos are hidden in the current view. Leave it first.": "目前的檢視隱藏了街景照片。請先離開此檢視。", + "Street View is ready: press {open} to load it": "街景已就緒：按 {open} 加載", + "{source} cannot open this link's street-level photo without a working key": "沒有有效密鑰，{source} 無法打開此鏈接的街景照片", + "Street-level photos are hidden in the current view. Leave it first.": "當前視圖隱藏了街景照片。請先退出該視圖。", "STATIONS": "太空站", "Crewed stations and the craft that visit them": "載人太空站及來訪飛行器", "IMAGING": "成像", diff --git a/src/layers/streetLevel/index.js b/src/layers/streetLevel/index.js index 6b0aa2d27..bee453af8 100644 --- a/src/layers/streetLevel/index.js +++ b/src/layers/streetLevel/index.js @@ -266,11 +266,11 @@ export function createStreetLevelLayer({ if (providerAvailable(entry)) parts.credits.show(state.viewer, entry.def); } - /** Requests replacing the open photo that switch its source keep it up. */ - let viewerHolds = 0; + /** Sources whose open photo a replacing request keeps up while it switches them off. */ + const heldViewers = new Map(); function deactivate(entry) { - if (!viewerHolds) parts.viewerHost.unmount(entry.def.id); + if (!heldViewers.has(entry.def.id)) parts.viewerHost.unmount(entry.def.id); entry.instance.deactivate(); parts.credits.hide(state.viewer, entry.def); } @@ -854,19 +854,29 @@ export function createStreetLevelLayer({ }, closeViewer, /** - * Keep the open photo up while a request that replaces it switches the - * source, so EXPAND stays. Release closes it if its source is still off. + * Keep the open photo up while a request that replaces it switches its + * source off, so EXPAND stays; only that source's own switch-off is held, + * a source switched by hand meanwhile still stops a load. Release frees + * the old viewer. */ holdViewer() { - viewerHolds += 1; + const id = state.street.open ? state.street.providerId : null; + if (!id) return () => {}; + heldViewers.set(id, (heldViewers.get(id) || 0) + 1); let held = true; return () => { if (!held) return; held = false; - viewerHolds -= 1; - if (viewerHolds) return; - for (const entry of state.providers.values()) - if (!entry.on) parts.viewerHost.unmount(entry.def.id); + const left = heldViewers.get(id) - 1; + if (left > 0) return void heldViewers.set(id, left); + heldViewers.delete(id); + if (state.providers.get(id)?.on !== false) return; + // Its photo still up (the request gave up) closes with its source; + // otherwise the request replaced or closed it, and the panel keeps + // what that request left (an error, an armed OPEN). + if (state.street.open && state.street.providerId === id) + parts.viewerHost.unmount(id); + else parts.viewerHost.release(id); }; }, /** diff --git a/src/layers/streetLevel/providers/mapillary/policy.js b/src/layers/streetLevel/providers/mapillary/policy.js index 78ca14a49..e885d377c 100644 --- a/src/layers/streetLevel/providers/mapillary/policy.js +++ b/src/layers/streetLevel/providers/mapillary/policy.js @@ -76,8 +76,8 @@ export const SEQUENCE_IMAGE_FIELDS = /** * Images asked for in a nearest-image search: the API cuts at the limit - * before any order, so this is its most, and the distance sort and the - * imagery filter choose among all of the spot's photos. + * before any order, so this is its most; the distance sort and the imagery + * filter choose among the photos this one request returns. */ export const NEAREST_LIMIT = 100; /** diff --git a/src/layers/streetLevel/viewerHost.js b/src/layers/streetLevel/viewerHost.js index acaba443e..9e3b18901 100644 --- a/src/layers/streetLevel/viewerHost.js +++ b/src/layers/streetLevel/viewerHost.js @@ -73,6 +73,19 @@ export function createViewerHost({ state, parts }) { active.unsubscribe(); active.adapter.unmount(); active = null; + // The photo being replaced belonged to that source: none of its details + // (id, place, author, link) describe the one loading now. + const { open, loading, error, providerId, providerName, providerLabel } = + state.street; + Object.assign(state.street, freshStreet(), { + open, + loading, + error, + providerId, + providerName, + providerLabel, + }); + parts.marker.clear(); } const adapter = entry.instance.viewer; const promise = (async () => { @@ -202,5 +215,27 @@ export function createViewerHost({ state, parts }) { active = null; } - return { attach, open, close, resize, setRenderMode, stopPlayback, unmount }; + /** + * Destroy `providerId`'s viewer and leave the panel's state alone: what a + * request leaves behind once it replaced or closed that source's photo. + */ + function release(providerId) { + if (active?.id !== providerId && mounting?.id !== providerId) return; + const pending = mounting; + mounting = null; + active?.unsubscribe(); + (active?.adapter ?? pending?.adapter)?.unmount(); + active = null; + } + + return { + attach, + open, + close, + resize, + setRenderMode, + stopPlayback, + unmount, + release, + }; } diff --git a/src/navigationPolicy.js b/src/navigationPolicy.js index 2980b21e6..0813b773e 100644 --- a/src/navigationPolicy.js +++ b/src/navigationPolicy.js @@ -181,3 +181,37 @@ export function reassertNavigationHandoff({ release?.(); return true; } + +/** Gestures on the globe that take the camera from a pending request. */ +const CAMERA_GESTURES = ['pointerdown', 'wheel']; + +/** + * Whether the camera is still a request's: a navigation the shell stamps (a + * search, the assistant, a link) or a press or wheel on the globe hands it to + * someone else. `claim()` makes the camera as it is now the request's (after + * its own flight); `stop()` removes the gesture listeners. + */ +export function watchCameraOwner(viewer, navigation) { + let generation = navigation?._navigationGeneration; + const canvas = viewer?.canvas ?? viewer?.scene?.canvas ?? null; + let taken = false; + const onGesture = () => { + taken = true; + }; + for (const type of CAMERA_GESTURES) + canvas?.addEventListener?.(type, onGesture, { passive: true }); + return { + claim: () => { + generation = navigation?._navigationGeneration; + taken = false; + }, + taken: () => + taken || + (Number.isFinite(generation) && + navigation._navigationGeneration !== generation), + stop: () => { + for (const type of CAMERA_GESTURES) + canvas?.removeEventListener?.(type, onGesture); + }, + }; +} diff --git a/src/streetLevelLifecycle.test.mjs b/src/streetLevelLifecycle.test.mjs index 3a08a44c3..b64ef9ea0 100644 --- a/src/streetLevelLifecycle.test.mjs +++ b/src/streetLevelLifecycle.test.mjs @@ -6,7 +6,6 @@ import { fakeStreetLevelProvider, fakeCesiumViewer } from './testSupport/streetL import { showStreetLevel } from './voice/gevActions.js'; import { installViews } from './app/embed.js'; import { ShareRestoration } from './ui/shareRestoration.js'; -import { createView } from './view/index.js'; import { StreetLevelControls } from './ui/streetLevelControls.js'; import { createApp, nextTick } from 'vue'; import { createPinia } from 'pinia'; @@ -22,11 +21,10 @@ function deferred() { const promise = new Promise(done => { resolve = done; }); return { promise, resolve }; } -async function harness(t, { nearestImage = async () => 'A', failImage = null, billed = false, extraProviders = [] } = {}) { +async function harness(t, { nearestImage = async () => 'A', failImage = null, extraProviders = [] } = {}) { document.body.className = ''; document.body.replaceChildren(); const provider = fakeStreetLevelProvider({ nearestImage }); - provider.billedPerImage = billed; const originalCreate = provider.create; const opened = []; provider.create = context => { @@ -50,7 +48,6 @@ async function harness(t, { nearestImage = async () => 'A', failImage = null, bi layer.attachViewerHost(host); t.after(() => { layer.destroy(); document.body.className = ''; document.body.replaceChildren(); }); await settle(); - assert.equal(layer.providerStatus('mapillary').billed, billed); const dataManager = { layers: new Map([['street-level', { module: layer }]]), getAll: () => [{ id: 'street-level', enabled: true }], @@ -69,14 +66,9 @@ async function harness(t, { nearestImage = async () => 'A', failImage = null, bi return { ok: true, latitude: 48, longitude: 2, label: 'fixture point' }; }, }; - let result; - const run = async (name, args) => { - if (name !== 'show_street_level') return { ok: true }; - result = await showStreetLevel(args, context); - return result; - }; - const view = createView({ camera: { lat: 48, lon: 2 }, layers: ['street-level'], street_level: { provider: 'mapillary', lat: 48, lon: 2 } }); - return { layer, viewer, dataManager, context, run, view, opened, flights, result: () => result }; + const run = (name, args) => + name === 'show_street_level' ? showStreetLevel(args, context) : Promise.resolve({ ok: true }); + return { layer, viewer, dataManager, context, run, opened, flights }; } test('a landed link does not reclaim a camera the user moved while its layers load', async t => { diff --git a/src/streetLevelReplacement.test.mjs b/src/streetLevelReplacement.test.mjs new file mode 100644 index 000000000..2d931d707 --- /dev/null +++ b/src/streetLevelReplacement.test.mjs @@ -0,0 +1,201 @@ +import './testSupport/registerDom.mjs'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createStreetLevelLayer } from './layers/streetLevel/index.js'; +import { fakeStreetLevelProvider, fakeCesiumViewer } from './testSupport/streetLevelFakes.mjs'; +import { showStreetLevel } from './voice/gevActions.js'; +import { installViews } from './app/embed.js'; + +// Replacing a street-level photo, across sources and against the user's own +// moves; posted views waiting on terrain. Offline: providers, lookups, terrain +// and flights are fixtures. +globalThis.fetch = async () => { throw new Error('Unexpected network use in offline test'); }; +const settle = () => new Promise(resolve => setTimeout(resolve, 0)); +function deferred() { + let resolve; + const promise = new Promise(done => { resolve = done; }); + return { promise, resolve }; +} +function provider(id, { nearest = async () => 'B', load = async () => {}, billed = false } = {}) { + const def = fakeStreetLevelProvider({ id, pickPrefix: id + ':', nearestImage: nearest }); + def.name = id; + def.label = id.toUpperCase(); + def.billedPerImage = billed; + const create = def.create; + def.create = context => { + const instance = create(context); + let pose; + instance.viewer.onPose = listener => { pose = listener; return () => { pose = null; }; }; + instance.viewer.open = async imageId => { + def.calls.open.push(imageId); + await load(imageId); + pose?.({ imageId, position: { lat: 48, lon: 2 }, creator: 'fixture photographer', + externalUrl: 'https://www.mapillary.com/app/?pKey=' + imageId }); + }; + return instance; + }; + return def; +} +async function harness(t, defs) { + document.body.className = ''; + document.body.replaceChildren(); + const viewer = fakeCesiumViewer(); + viewer.canvas = viewer.scene.canvas; + viewer.canvas.setPointerCapture = () => {}; + const layer = createStreetLevelLayer({ providers: defs }); + layer.init(viewer); + layer.enable(viewer); + layer.attachViewerHost(document.body.appendChild(document.createElement('div'))); + t.after(() => { layer.destroy(); document.body.className = ''; document.body.replaceChildren(); }); + await settle(); + const dataManager = { + layers: new Map([['street-level', { module: layer }]]), + getAll: () => [{ id: 'street-level', enabled: true }], + isEnabled: () => true, + getLayerParams: () => ({ mapillary: true }), + setLayerParams: (_id, params) => layer.setParams(params), + }; + const shell = { _navigationGeneration: 1, initialRestorePromise: Promise.resolve(), setCleanView() {} }; + const flights = []; + const context = { viewer, dataManager, styleManager: shell, + runOptions: { signal: new AbortController().signal }, + runGevAction: async (name, args) => { + assert.equal(name, 'fly_to_location'); + flights.push(args); + return { ok: true, latitude: args.latitude, longitude: args.longitude, label: 'fixture point' }; + }, + }; + const run = (name, args) => name === 'show_street_level' ? showStreetLevel(args, context) : Promise.resolve({ ok: true }); + return { layer, viewer, dataManager, shell, context, run, flights }; +} +async function embed(t, h, terrain) { + const replies = []; + const peer = { postMessage: message => replies.push(message) }; + const windowRef = new EventTarget(); + windowRef.parent = peer; + windowRef.document = document; + const remove = installViews({ ...h, terrain, windowRef, location: { search: '?embed=1', hash: '' } }); + t.after(remove); + await settle(); + const post = (id, view) => { + const event = new Event('message'); + Object.assign(event, { source: peer, origin: 'https://host.example', data: { type: 'gev:view', id, view } }); + windowRef.dispatchEvent(event); + }; + return { post, replies }; +} +const args = provider => ({ provider, latitude: 48, longitude: 2, fly: false }); +const view = pitch => ({ camera: { lat: 48, lon: 2, altitude_m: 1000, pitch_deg: pitch }, street_level: { provider: 'mapillary' } }); + +for (const outcome of ['empty', 'error']) { + test('a replacement from another source that opens nothing keeps its explanation (' + outcome + ')', async t => { + const a = provider('mapillary'); + const b = provider('panoramax', { nearest: async () => { + if (outcome === 'error') throw new Error('Fixture lookup failure'); + return null; + } }); + const h = await harness(t, [a, b]); + assert.equal(await h.layer.openImage('mapillary', 'A', { frame: false }), true); + const result = await showStreetLevel(args('panoramax'), h.context); + const street = h.layer.getUIState().street; + t.diagnostic(JSON.stringify({ result, error: street.error, nearestOutcome: street.nearestOutcome })); + assert.equal(result.ok, false); + assert.equal(street.open, false); + assert.ok(street.error, 'replacement explanation must survive release'); + if (outcome === 'empty') assert.equal(street.nearestOutcome, 'empty'); + }); +} +test('an arming link from another source keeps its armed point', async t => { + const h = await harness(t, [provider('mapillary'), provider('google', { billed: true })]); + assert.equal(await h.layer.openImage('mapillary', 'A', { frame: false }), true); + const result = await showStreetLevel({ ...args('google'), autoOpen: false }, h.context); + const street = h.layer.getUIState().street; + t.diagnostic(JSON.stringify({ result, armed: street.armed })); + assert.equal(result.armed, true); + assert.deepEqual(street.armed?.point, { lat: 48, lon: 2 }); +}); +test('a same-source lookup that finds nothing keeps its explanation', async t => { + const h = await harness(t, [provider('mapillary', { nearest: async () => null })]); + await h.layer.openImage('mapillary', 'A', { frame: false }); + const result = await showStreetLevel(args('mapillary'), h.context); + assert.equal(result.noImagery, true); + assert.ok(h.layer.getUIState().street.error); + assert.equal(h.layer.getUIState().street.nearestOutcome, 'empty'); +}); + +for (const takeover of ['navigation', 'gesture', 'none']) { + test('a posted view waiting on terrain yields to the user (' + takeover + ')', async t => { + const a = provider('mapillary'); + const h = await harness(t, [a]); + const entered = deferred(); + const terrain = deferred(); + const e = await embed(t, h, { getHeights: async () => { entered.resolve(); return terrain.promise; } }); + e.post(1, view(-45)); + await entered.promise; + if (takeover === 'navigation') h.shell._navigationGeneration++; + if (takeover === 'gesture') h.viewer.canvas.dispatchEvent(new Event('pointerdown')); + terrain.resolve([{ ellipsoid: 200 }]); + for (let n = 0; n < 8 && !e.replies.some(r => r.id === 1); n++) await settle(); + const reply = e.replies.find(r => r.id === 1); + t.diagnostic(JSON.stringify({ takeover, flights: h.flights, opened: a.calls.open, reply })); + assert.ok(reply, 'host receives completion'); + assert.equal(h.flights.length, takeover === 'none' ? 1 : 0, 'superseded view must not reclaim the camera'); + assert.equal(a.calls.open.length, takeover === 'none' ? 1 : 0); + assert.equal(reply.ok, takeover === 'none'); + }); +} + +test('a posted view looking at the horizon does not wait on terrain or hold the queue', async t => { + const h = await harness(t, [provider('mapillary')]); + const terrain = deferred(); + let terrainCalls = 0; + const e = await embed(t, h, { getHeights: () => { terrainCalls++; return terrain.promise; } }); + e.post(1, view(0)); + e.post(2, { ...view(-90), street_level: { provider: 'mapillary', lat: 49, lon: 3 } }); + for (let n = 0; n < 8; n++) await settle(); + const before = e.replies.filter(r => r.id).map(r => r.id); + terrain.resolve([{ ellipsoid: 200 }]); + for (let n = 0; n < 8 && !e.replies.some(r => r.id === 2); n++) await settle(); + const after = e.replies.filter(r => r.id).map(r => r.id); + t.diagnostic(JSON.stringify({ terrainCalls, before, after, flights: h.flights })); + assert.deepEqual(after, [1, 2], 'both complete after irrelevant terrain is released'); + assert.deepEqual(before, [1, 2], 'both should complete without any terrain response'); + assert.equal(terrainCalls, 0); +}); + +for (const initial of [true, false]) { + test('a source switched by hand during the load stops the request (photo open before: ' + initial + ')', async t => { + const entered = deferred(); + const load = deferred(); + const a = provider('mapillary', { load: async id => { if (id === 'B') { entered.resolve(); await load.promise; } } }); + const h = await harness(t, [a, provider('panoramax')]); + if (initial) await h.layer.openImage('mapillary', 'A', { frame: false }); + const request = showStreetLevel(args('mapillary'), h.context); + await entered.promise; + h.layer.setProviderEnabled('panoramax', true); + load.resolve(); + const result = await request; + const street = h.layer.getUIState().street; + t.diagnostic(JSON.stringify({ initial, result, open: street.open, sourceOn: h.layer.providerStatus('mapillary').on })); + assert.equal(h.layer.providerStatus('mapillary').on, false); + assert.equal(street.open, false); + assert.equal(result.ok, false, 'request for the disabled source must not report success'); + assert.equal(result.cancelled, true); + }); +} + +test('a failed replacement from another source does not keep the old photo\'s details', async t => { + const h = await harness(t, [provider('mapillary'), provider('panoramax', { load: async () => { throw new Error('Fixture image load failure'); } })]); + await h.layer.openImage('mapillary', 'A', { frame: false }); + const before = h.layer.getUIState().street; + assert.equal(before.imageId, 'A'); + assert.ok(before.externalUrl); + const result = await showStreetLevel(args('panoramax'), h.context); + const street = h.layer.getUIState().street; + t.diagnostic(JSON.stringify({ result, street })); + assert.equal(result.ok, false); + assert.equal(street.providerId, 'panoramax'); + assert.match(street.error, /Fixture image load failure/); + assert.deepEqual({ imageId: street.imageId, position: street.position, creator: street.creator, externalUrl: street.externalUrl }, + { imageId: null, position: null, creator: null, externalUrl: null }, 'destroyed provider photo must not be attributed to replacement'); +}); diff --git a/src/view/index.js b/src/view/index.js index 7d70e1c8c..cd5f20f31 100644 --- a/src/view/index.js +++ b/src/view/index.js @@ -131,7 +131,13 @@ export function lookAtPoint(camera) { */ export async function groundedLookAt(camera, terrain, signal) { if (typeof terrain?.getHeights !== 'function') return null; - if (!Number.isFinite(camera?.altitude_m) || !(camera.pitch_deg > -89)) + // Looking down or at the horizon the point is the camera's own: the ground + // under it changes nothing, so it is not asked for. + if ( + !Number.isFinite(camera?.altitude_m) || + !(camera.pitch_deg > -89) || + camera.pitch_deg > LOOK_AT_MIN_PITCH_DEG + ) return null; let heights; try { @@ -381,6 +387,12 @@ export function createView({ // photos at X, also when the camera sits behind it. It is a pair: half // of one is not mixed with the camera's other half. const given = [latitude(streetLevel.lat), longitude(streetLevel.lon)]; + // A coordinate given off the globe is an error, not a reason to guess. + for (const [index, key] of ['lat', 'lon'].entries()) + if (streetLevel[key] != null && given[index] === undefined) + throw new TypeError( + `Street-level ${key} is not a valid coordinate: ${streetLevel[key]}`, + ); const seen = lookAtPoint(camera); const [lat, lon] = given.includes(undefined) ? [seen.lat, seen.lon] : given; street = Object.freeze({ diff --git a/src/view/index.test.mjs b/src/view/index.test.mjs index 9fda7d149..046d76b3c 100644 --- a/src/view/index.test.mjs +++ b/src/view/index.test.mjs @@ -208,6 +208,21 @@ test('street-level photos travel in the link, light their source and bring their assert.equal(streetLevelFromParams(new URLSearchParams('sl=bing,1,2')), null); assert.equal(streetLevelFromParams(new URLSearchParams('lat=1')), null); assert.equal(createView({ camera: { lat: 1, lon: 2 } }).street_level, null); + // A point off the globe is refused, like an unknown source. + for (const bad of [ + { lat: 95, lon: 2 }, + { lat: 48, lon: 200 }, + { lat: Number.NaN, lon: 2 }, + ]) + assert.throws( + () => + createView({ + camera: { lat: 1, lon: 2 }, + street_level: { provider: 'mapillary', ...bad }, + }), + TypeError, + JSON.stringify(bad), + ); // Half a point is not mixed with the camera's other half. const half = createView({ camera: { lat: 1, lon: 2 }, diff --git a/src/voice/gevActions.js b/src/voice/gevActions.js index 14956c6c9..14b8102c3 100644 --- a/src/voice/gevActions.js +++ b/src/voice/gevActions.js @@ -60,6 +60,7 @@ import { import * as defaultFloorServices from '../data/groundFloor.js'; import { isPickedWorldPosition } from '../data/scenePick.js'; import { unavailablePlaceSearch } from '../search/placeSearch.js'; +import { watchCameraOwner } from '../navigationPolicy.js'; import * as defaultAnnotationResolver from '../annotations/annotationResolver.js'; import { normalizeRadioCountryInput } from '../data/radioCountry.js'; import { TR3B_CLASS } from '../data/tr3bRegistry.js'; @@ -2984,7 +2985,6 @@ export function activeImageryFilter(ui) { * detection so the framed entities are labeled, and reports * detectionEnabled so the voice agent can mention labels are on. */ - async function frameOverhead(viewer, dataManager, styleManager, args = {}) { const targetRaw = String(args.target || 'flights').toLowerCase(); const layerId = @@ -3763,40 +3763,6 @@ function actionIntentOrigin(runOptions) { return runOptions?.origin === 'programmatic' ? 'programmatic' : 'voice'; } -/** Gestures on the globe that take the camera from a pending request. */ -const CAMERA_GESTURES = ['pointerdown', 'wheel']; - -/** - * Whether the camera is still a request's: a navigation the shell stamps (a - * search, the assistant, a link) or a press or wheel on the globe hands it to - * someone else. `claim()` makes the camera as it is now the request's (after - * its own flight); `stop()` removes the gesture listeners. - */ -function watchCameraOwner(viewer, navigation) { - let generation = navigation?._navigationGeneration; - const canvas = viewer?.canvas ?? viewer?.scene?.canvas ?? null; - let taken = false; - const onGesture = () => { - taken = true; - }; - for (const type of CAMERA_GESTURES) - canvas?.addEventListener?.(type, onGesture, { passive: true }); - return { - claim: () => { - generation = navigation?._navigationGeneration; - taken = false; - }, - taken: () => - taken || - (Number.isFinite(generation) && - navigation._navigationGeneration !== generation), - stop: () => { - for (const type of CAMERA_GESTURES) - canvas?.removeEventListener?.(type, onGesture); - }, - }; -} - /** * Whether cockpit, clean view, recording or scene playback hides the panel a * street-level photo opens in. Embed mode uses clean view for its globe but @@ -4034,7 +4000,8 @@ async function requestStreetLevel( // Light the source the way a chip click does, so the choice is kept. An // open photo of another source stays up until this request replaces it. - if (layerModule.getUIState?.()?.street?.open === true) + const openPhoto = layerModule.getUIState?.()?.street; + if (openPhoto?.open === true && openPhoto.providerId !== provider) holdViewer(layerModule); const switches = Object.fromEntries( providerIds.map((id) => [id, id === provider]),