diff --git a/changelog.d/1520-voice-camera-tilt.md b/changelog.d/1520-voice-camera-tilt.md new file mode 100644 index 000000000..f46ae3c51 --- /dev/null +++ b/changelog.d/1520-voice-camera-tilt.md @@ -0,0 +1,6 @@ +- Voice "tilt the camera" works from the top-down view a fly-to lands on: the + camera swings about the place under the crosshair, keeps its heading, and can + tilt to a spoken angle ("tilt 45 degrees") or a default oblique view. A tilt, + orbit, pan or rotate said with a fly-to ("fly to Paris and orbit it") now + waits for the flight to land, and is dropped if you move the map meanwhile + (Manjunath, @manjunath22466, #1520). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 5166e4a2a..c3e88d64f 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -5693,7 +5693,7 @@ and unreachable upstream (502/504) separately from road geometry. > branch: **2,281 on Node 25.6.1** (allocation suites skipped) = 2,255 on > `main` + 26 route pins; the same tree on Node 24 reports 2,295. -- **Camera verbs** (`src/cameraVerbs.js`) — one motion slot, driven per clock tick. `move_camera` orbits/pans/tilts/rotates; `fly_route` is a cinematic dolly along an existing route annotation. `rotate_globe` rotates the world-space camera rigidly around Earth's center using its screen-up axis, preserving geocentric range and framing. Left/right describe visible globe motion, while `rotate` turns the camera in place. Omitted globe direction/mode default to right/continuous; explicit values prevail. Other camera motions keep their once default. Admission requires 3D, no Cockpit/tracking owner and the whole globe inside the existing keyhole circle, using the shared keyhole/projection geometry and vertical camera FOV with a rotation-local bounding-sphere cone for off-axis containment. Canvas-edge clipping alone does not block a rigid rotation or its unchanged Globe View preset. There is no altitude cutoff. An already smaller, fully framed globe rotates directly; repeated nudges, direction and speed changes recheck current framing without demanding a reset merely because ellipsoid altitude varied. A close-up or overlap refuses without moving the camera. When the unchanged Globe View preset fits, voice says “Globe rotation isn’t available from this view. You can say ‘Reset to Globe View and rotate the globe.’” without asking a permission question or automatically resetting. Explicit Reset/Show Globe View commands remain direct; an explicit reset-and-rotate request waits for reset success before requesting rotation. If OpenAI dispatches both tools together, rotation waits for earlier same-response resets and is skipped if any fails or is cancelled. OpenAI rejects later tool calls in the same response after a globe refusal, including calls arriving while globe admission is pending, and preserves response-scoped `tool_choice: none` for the refusal follow-up; Gemini rejects further tool execution, including queued sibling calls, until fresh typed input or admitted speech. Model interruption, advisory captions and microphone toggles do not release that guard. There is no pending Yes/No offer. Outside the refused response, explicit-command interpretation and avoiding repeated resets remain model-guided; a distinct model-issued reset call is not an app-enforced proof of user intent. A failed standalone reset can still be retried by the model, and a reset result does not guarantee the camera has fully settled. Before suggesting a reset, a cloned camera checks the unchanged Globe View destination against the same keyhole/depth rules without moving the live camera. If that preset cannot fit, `globe-reset-unavailable` asks for further zoom and the refusal instructs further zoom instead of a reset; this also covers ultrawide views. Unsupported view geometry or non-WGS84 projection assumptions fail closed without the closer-zoom instruction. Supported camera inputs/navigation intents cancel rotation before their action. Finite radius greater than the ellipsoid's maximum radius proves whole-arc clearance at constant centre-distance; no low-altitude swept solver is needed. The unchanged Globe View reset preset remains 18,000,000 m. Terrain and buildings remain outside the guarantee. +- **Camera verbs** (`src/cameraVerbs.js`) — one motion slot, driven per clock tick. `move_camera` orbits/pans/tilts/rotates; `fly_route` is a cinematic dolly along an existing route annotation. `rotate_globe` rotates the world-space camera rigidly around Earth's center using its screen-up axis, preserving geocentric range and framing. Left/right describe visible globe motion, while `rotate` turns the camera in place. `tilt` swings the camera about the ground under the crosshair, so the place stays centred and the heading is kept. It tilts in place instead with no ground under the crosshair, from beyond 1,000 km of range, or while an entity is tracked (voice releases the follow camera before moving, so a voice tilt from a tracked view still pivots); pitch stays within -89° to -5°, clamped only in the direction of travel so a top-down (-90°) view can tilt up out of it. Angles and limits are the camera's own pitch; at regional range the pivot's pitch is up to about 8° shallower (the Earth curves away), so the swing solves for the pivot pitch that gives the camera the angle asked for. A pivot swing toward the horizon lowers the camera (height above the pivot is range × sin|pitch|), so the swing stops lowering it at 150 m above the pivot (not above the terrain behind the camera) and the rest of the tilt happens in place. Beyond about 17 km of range the swing never hands over to an in-place tilt (that would lift the crosshair onto the sky); it keeps the camera at least 0.5° above the place's own horizon, which from roughly 500 km stops it short of -5° with the place still centred: a tilt to an angle beyond that stops at the angle reached and says so (`pitchDeg` is the angle reached, with a `note`), a continuous tilt stops there, and a further tilt up is refused as at the limit. A zoom during a pivoting tilt scales the range it holds, as for orbit, but a zoom in stops at the range that keeps that 150 m clearance; a zoom that can move no further (that floor, the 80 m floor, or the cap: 1,000 km for a tilt, where pivoting stops, and 5,000 km for an orbit) is reported as not zoomed. A zoom during a tilt to an angle re-judges the place's horizon limit at the new range: when that moves where the tilt ends, the tilt eases from where it is to the new end (holding the camera's pitch through the range change; a camera the zoom leaves past the new range's limit steps down to it with the zoom, since past it the place would drop behind the Earth), and the zoom's result names that angle (`pitchDeg`) with a `note`. Once the tilt continues in place, a zoom moves the camera like any zoom. A tilt with `pitchDeg` (or with neither angle nor direction, which means -45°) eases to that angle; a positive angle means the same angle below the horizon. `pitchDeg` is absolute, not a step: sent with a `direction` it contradicts, the tilt is refused. An angle the camera already holds answers `unchanged` but still takes navigation, so it stops a running motion and releases the follow camera. A tilt left or right is refused. A voice tilt, orbit, pan or rotate sent while a voice `fly_to_location` is still looking up or flying ("fly to London and tilt the camera" or "fly to Paris and orbit it" arrives as two sibling calls) waits for that flight to land, up to 15 s (moving to a later fly-to in the same reply as soon as it begins, whatever the earlier one reports), so it neither cancels the flight nor gets replaced by it; if the flight is cancelled, finds nothing or times out, the user grabs, touches or scrolls the map during it, or another navigation (a location-panel search, share link or file import, which leave the flight flying) takes the camera during it, the motion is not run and says so. A motion that runs after a fly-to of the same reply has already found nothing (Gemini runs one turn's calls one by one, and passes its turn as the reply) is refused the same way rather than moving the old view; a motion in a later reply runs. `stop` and `rotate_globe` do not wait, and a motion sent before its fly-to is not reordered: it runs at once, and the fly-to's result names the motion it stopped (`stoppedMotion`). Orbit still arms (`armed: 'waiting-for-arrival'`) when it starts during any other camera flight, and captures its target once the tween queue has been quiet for 0.8 s. Omitted globe direction/mode default to right/continuous; explicit values prevail. Other camera motions keep their once default. Admission requires 3D, no Cockpit/tracking owner and the whole globe inside the existing keyhole circle, using the shared keyhole/projection geometry and vertical camera FOV with a rotation-local bounding-sphere cone for off-axis containment. Canvas-edge clipping alone does not block a rigid rotation or its unchanged Globe View preset. There is no altitude cutoff. An already smaller, fully framed globe rotates directly; repeated nudges, direction and speed changes recheck current framing without demanding a reset merely because ellipsoid altitude varied. A close-up or overlap refuses without moving the camera. When the unchanged Globe View preset fits, voice says “Globe rotation isn’t available from this view. You can say ‘Reset to Globe View and rotate the globe.’” without asking a permission question or automatically resetting. Explicit Reset/Show Globe View commands remain direct; an explicit reset-and-rotate request waits for reset success before requesting rotation. If OpenAI dispatches both tools together, rotation waits for earlier same-response resets and is skipped if any fails or is cancelled. OpenAI rejects later tool calls in the same response after a globe refusal, including calls arriving while globe admission is pending, and preserves response-scoped `tool_choice: none` for the refusal follow-up; Gemini rejects further tool execution, including queued sibling calls, until fresh typed input or admitted speech. Model interruption, advisory captions and microphone toggles do not release that guard. There is no pending Yes/No offer. Outside the refused response, explicit-command interpretation and avoiding repeated resets remain model-guided; a distinct model-issued reset call is not an app-enforced proof of user intent. A failed standalone reset can still be retried by the model, and a reset result does not guarantee the camera has fully settled. Before suggesting a reset, a cloned camera checks the unchanged Globe View destination against the same keyhole/depth rules without moving the live camera. If that preset cannot fit, `globe-reset-unavailable` asks for further zoom and the refusal instructs further zoom instead of a reset; this also covers ultrawide views. Unsupported view geometry or non-WGS84 projection assumptions fail closed without the closer-zoom instruction. Supported camera inputs/navigation intents cancel rotation before their action. Finite radius greater than the ellipsoid's maximum radius proves whole-arc clearance at constant centre-distance; no low-altitude swept solver is needed. The unchanged Globe View reset preset remains 18,000,000 m. Terrain and buildings remain outside the guarantee. - **The route dolly is shaped, not linear** (2026-08-20). A trapezoid speed profile (smoothstep up, cruise, smoothstep down, distance taken as the closed-form integral) eases both ends without changing the pace — duration is still `totalM / ROUTE_M_S[speed]`, with one exception: a 0.5 s minimum keeps a degenerate route from being an instant teleport, so routes under 10 m (slow) / 20 m (normal) / 45 m (fast) fly SLOWER than the speed word, never faster. Turns bank up to **10°** (a 90° street corner settles near 7.5°), measured as a triangular pulse over a 4 s window centred on the camera so the roll leads in and unwinds after. Altitude breathes ±20 m around the 260 m mean and lifts up to 26 m into turns. Pitch is LOCKED at −32°; heading comes from a gaze that leads the path by 6.5 s of travel. `prefers-reduced-motion` zeroes the roll and the altitude shaping and keeps the easing. - **Invariants — do not "fix" these.** (1) **Every** release levels the roll: completion unwinds it through the ease-out envelope, and `interruptCameraMotion` zeroes it synchronously (heading/pitch/position preserved). Cesium keeps the last up vector it was handed, so skipping this leaves the user holding a tilted horizon. (2) Heading is interpolated as an ANGLE about the local up, never as a Cartesian lerp — a lerp cannot cross an antipodal pair, so an out-and-back route looked backwards for the entire return leg; `signedTurnRad` branches the exact-180° case deterministically because the cross product's sign there is a floating-point coin toss. (3) A COLD floor cell is missing data, not flat ground: route vertices carry height 0, so trusting them flew a mountain corridor at 260 m above the ELLIPSOID. The corridor warm is fire-and-forget, so the dolly ARMS — camera untouched, no teleport onto the route — for up to 1.2 s waiting for real floor data, falls back to a rendered-mesh probe (`scene.sampleHeight`) when the DEM stays cold — latched to ONE firing per flight, for exactly the cells the cache could not answer, so a route costs at most 8 `sampleHeight` calls however long it arms — and otherwise holds the launch altitude for the whole route. A corridor is only RESOLVED when every cell is accounted for: one warm cell says nothing about the ground under the other seven, and treating it as an answer let the dolly descend to 460 m over a 1,600 m rendered surface. The probe reads the rendered surface at the current LOD (rooftops and primitives included) — a better estimate than nothing, not a guaranteed upper bound. Never descend blind. The floor is SMOOTHED in both directions for the ride while the hard clearance clamp reads the RAW sample, so a cell boundary cannot pop the eye but a cliff is still cleared on the frame it is seen. A pre-departure floor is adopted whole; one arriving mid-flight is eased onto, and the eye's descent rate is capped. - Pins: `src/routeCinematics.test.mjs` (26 tests), with `scripts/qa-flyroute-mutations.mjs` reverting each fix individually to prove they are load-bearing (19 named defects); rendered proof `scripts/qa-flyroute-cinema.mjs` (drives the real voice runner, measures the real Cesium camera every frame, writes a labelled contact sheet). diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 1b0d1454c..e5b46f7b3 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -1127,6 +1127,8 @@ "scripts/qa-status-badges.mjs", "src/dockerTrustedPeers.test.mjs", "src/cameraGlobeRotation.test.mjs", + "src/cameraTilt.test.mjs", + "src/ui/shellFacade.test.mjs", "src/voice/globeResetConfirmation.test.mjs", "src/hudDms.test.mjs", "src/hudSummaryGate.test.mjs", diff --git a/server/providers/openai/toolDescriptions.js b/server/providers/openai/toolDescriptions.js index 891145021..c7e78c7d6 100644 --- a/server/providers/openai/toolDescriptions.js +++ b/server/providers/openai/toolDescriptions.js @@ -523,7 +523,7 @@ export const ACTION_DESCRIPTIONS = { properties: { direction: { description: - 'Required for pan/tilt/rotate; orbit/rotate_globe default right.', + 'Pan/rotate/tilt steps; orbit/rotate_globe default right.', $position: 2, }, mode: { @@ -531,6 +531,10 @@ export const ACTION_DESCRIPTIONS = { 'once = bounded eased nudge; continuous = until stop/manual input. rotate_globe defaults continuous; other motions default once.', $position: 2, }, + pitchDeg: { + description: 'Absolute; -45 default, -89 top-down.', + $position: 1, + }, }, }, }, diff --git a/src/cameraHandoff.test.mjs b/src/cameraHandoff.test.mjs index 2c0d356b2..398c51c34 100644 --- a/src/cameraHandoff.test.mjs +++ b/src/cameraHandoff.test.mjs @@ -223,11 +223,20 @@ test('validated voice camera destinations share the UI navigation authority faca 'move camera', ); ordered(move, [ - "if (!['orbit', 'pan', 'tilt', 'rotate', 'rotate_globe'].includes(motion))", + 'if (argsError) return argsError;', 'const start = () => {', /const preserveCameraFlight =\s*motion === 'orbit'/, 'runNavigation(start, { preserveCameraFlight })', ], 'move validation before handoff'); + const moveArgs = body( + cameraVerbs, + /export function moveCameraArgsError\(args = \{\}\) \{([\s\S]*?)\n\}/, + 'move camera argument checks', + ); + assert.ok( + moveArgs.includes("if (!['orbit', 'pan', 'tilt', 'rotate', 'rotate_globe'].includes(motion))"), + 'unknown motions are refused before handoff', + ); const route = body( cameraVerbs, diff --git a/src/cameraTilt.test.mjs b/src/cameraTilt.test.mjs new file mode 100644 index 000000000..8a3d1447f --- /dev/null +++ b/src/cameraTilt.test.mjs @@ -0,0 +1,773 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import * as Cesium from 'cesium'; +import { + adjustOrbitRange, + getActiveCameraMotion, + initCameraVerbs, + interruptCameraMotion, + moveCamera, +} from './cameraVerbs.js'; +import { createGevActionRunner } from './voice/gevActions.js'; + +const radians = Cesium.Math.toRadians; +const degrees = Cesium.Math.toDegrees; + +/** Real Cesium camera over London with no renderer: the voice fly-to frame. */ +function harness( + t, + { height = 4000, heading = 0, pitch = -90, pivot = true } = {}, +) { + const realPerformance = globalThis.performance; + let nowMs = 0; + globalThis.performance = { now: () => nowMs }; + const canvas = Object.assign(new EventTarget(), { + clientWidth: 1280, + clientHeight: 800, + width: 1280, + height: 800, + }); + const ellipsoid = Cesium.Ellipsoid.WGS84; + const scene = { + canvas, + drawingBufferWidth: 1280, + drawingBufferHeight: 800, + frameState: { mode: Cesium.SceneMode.SCENE3D }, + mapProjection: new Cesium.GeographicProjection(ellipsoid), + globe: { ellipsoid }, + mapMode2D: Cesium.MapMode2D.INFINITE_SCROLL, + mode: Cesium.SceneMode.SCENE3D, + pixelRatio: 1, + tweens: [], + requestRender() {}, + }; + const camera = new Cesium.Camera(scene); + scene.camera = camera; + camera.setView({ + destination: Cesium.Cartesian3.fromDegrees(-0.1276, 51.5072, height), + orientation: { heading: radians(heading), pitch: radians(pitch), roll: 0 }, + }); + const viewer = { + camera, + scene, + trackedEntity: undefined, + trackedEntityChanged: new Cesium.Event(), + clock: { onTick: new Cesium.Event() }, + }; + // The app's view target: the ground point under the screen centre. + const viewTarget = () => { + const ray = new Cesium.Ray(camera.positionWC, camera.directionWC); + const hit = Cesium.IntersectionTests.rayEllipsoid(ray, ellipsoid); + return hit ? Cesium.Ray.getPoint(ray, hit.start) : undefined; + }; + const target = viewTarget(); + initCameraVerbs(viewer, pivot ? viewTarget : () => undefined); + t.after(() => { + interruptCameraMotion('test-cleanup'); + globalThis.performance = realPerformance; + }); + const tick = (seconds = 1 / 30) => { + nowMs += seconds * 1000; + viewer.clock.onTick.raiseEvent(viewer.clock); + }; + const finish = () => { + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) tick(); + assert.equal(getActiveCameraMotion(), null, 'tilt must finish'); + }; + return { viewer, camera, target, tick, finish, viewTarget }; +} + +const pitchDeg = (camera) => degrees(camera.pitch); +/** Tick until done() holds; a regression fails here instead of hanging. */ +const tickUntil = (h, done, frames = 600) => { + for (let i = 0; i < frames && !done(); i += 1) h.tick(); + assert.ok(done(), `not reached in ${frames} frames`); +}; +const headingDeg = (camera) => + degrees(Cesium.Math.negativePiToPi(camera.heading)); + +/** Metres between the original centre and the ground now under the crosshair. */ +function centreDrift(h) { + return Cesium.Cartesian3.distance(h.viewTarget(), h.target); +} + +test('tilt up leaves a top-down view and keeps the place centred', (t) => { + const h = harness(t); + const result = moveCamera({ motion: 'tilt', direction: 'up' }); + assert.equal(result.ok, true); + h.finish(); + assert.ok( + Math.abs(pitchDeg(h.camera) - -75) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); + assert.ok(centreDrift(h) < 5, `centre drifted ${centreDrift(h)} m`); + assert.ok(Cesium.Matrix4.equals(h.camera.transform, Cesium.Matrix4.IDENTITY)); +}); + +test('continuous tilt up from top-down keeps moving', (t) => { + const h = harness(t); + assert.equal( + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }).ok, + true, + ); + for (let i = 0; i < 30; i += 1) h.tick(); + assert.equal(getActiveCameraMotion()?.kind, 'tilt'); + assert.ok(pitchDeg(h.camera) > -82, `pitch ${pitchDeg(h.camera)}`); +}); + +test('a requested angle tilts to that pitch', (t) => { + const h = harness(t); + const result = moveCamera({ motion: 'tilt', pitchDeg: -45 }); + assert.equal(result.ok, true); + assert.equal(result.direction, 'up'); + assert.equal(result.pitchDeg, -45); + h.finish(); + assert.ok( + Math.abs(pitchDeg(h.camera) - -45) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); + assert.ok(centreDrift(h) < 5, `centre drifted ${centreDrift(h)} m`); +}); + +test('a bare tilt settles on the default oblique angle', (t) => { + const h = harness(t); + const result = moveCamera({ motion: 'tilt' }); + assert.equal(result.ok, true); + assert.equal(result.pitchDeg, -45); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -45) < 0.5); +}); + +test('a requested angle can tilt back down toward top-down', (t) => { + const h = harness(t, { pitch: -30 }); + const result = moveCamera({ motion: 'tilt', pitchDeg: -80 }); + assert.equal(result.direction, 'down'); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -80) < 0.5); +}); + +test('requested angles clamp to the tilt limits', (t) => { + const h = harness(t); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -2 }).pitchDeg, -5); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -5) < 0.5); +}); + +test('a positive angle is read as the same angle below the horizon', (t) => { + const h = harness(t); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: 45 }).pitchDeg, -45); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -45) < 0.5); +}); + +for (const pitch of ['', null, false, ' ', 'steep']) + test(`a blank or non-numeric angle ${JSON.stringify(pitch)} is ignored`, (t) => { + harness(t); + const result = moveCamera({ motion: 'tilt', pitchDeg: pitch }); + // No angle and no direction: the default oblique view, never the horizon. + assert.equal(result.pitchDeg, -45); + }); + +test('a requested angle the camera already holds is a no-op', (t) => { + harness(t, { pitch: -45 }); + const result = moveCamera({ motion: 'tilt', pitchDeg: -45 }); + assert.equal(result.ok, true); + assert.equal(result.unchanged, true); + assert.equal(getActiveCameraMotion(), null); +}); + +for (const pitch of [-90, -89]) { + test(`a top-down view asked for ${pitch}° stays put`, (t) => { + // The tool description calls -89 top-down; the clamp must not tilt a + // -90° view up 1° and report it as a move up. + const h = harness(t); + const before = pitchDeg(h.camera); + const result = moveCamera({ motion: 'tilt', pitchDeg: pitch }); + assert.equal(result.ok, true); + assert.equal(result.unchanged, true); + assert.equal(result.pitchDeg, Math.round(before)); + assert.equal(getActiveCameraMotion(), null); + assert.equal(pitchDeg(h.camera), before); + }); +} + +test('tilt down from top-down still refuses honestly', (t) => { + harness(t); + const result = moveCamera({ motion: 'tilt', direction: 'down' }); + assert.equal(result.ok, false); + assert.match(result.error, /straight down/); +}); + +test('a tilt up nudge stops at the horizon limit', (t) => { + const h = harness(t, { pitch: -12 }); + assert.equal(moveCamera({ motion: 'tilt', direction: 'up' }).ok, true); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -5) < 0.5); +}); + +test('without a ground pivot a tilt still leaves top-down in place', (t) => { + const h = harness(t, { pivot: false }); + const before = Cesium.Cartesian3.clone(h.camera.positionWC); + assert.equal(moveCamera({ motion: 'tilt', direction: 'up' }).ok, true); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -75) < 0.5); + assert.ok(Cesium.Cartesian3.distance(h.camera.positionWC, before) < 1e-6); +}); + +for (const heading of [0, 30, -120]) + test(`tilt from top-down keeps the ${heading}° heading`, (t) => { + const h = harness(t, { heading, height: 67000 }); + // Top-down float noise must not pick a new heading for the pivot. + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -45 }).ok, true); + h.finish(); + assert.ok( + Math.abs(headingDeg(h.camera) - heading) < 1, + `heading ${headingDeg(h.camera)}`, + ); + assert.ok(centreDrift(h) < 50, `centre drifted ${centreDrift(h)} m`); + }); + +test('an oblique tilt keeps its heading toward the pivot', (t) => { + const h = harness(t, { heading: 75, pitch: -60 }); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -30 }).ok, true); + h.finish(); + assert.ok(Math.abs(headingDeg(h.camera) - 75) < 0.5); + assert.ok(centreDrift(h) < 5, `centre drifted ${centreDrift(h)} m`); +}); + +test('a pivoting tilt yields when tracking takes the camera', (t) => { + const h = harness(t); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -45 }).ok, true); + h.tick(); + const entity = { id: 'tracked' }; + h.viewer.trackedEntity = entity; + h.viewer.trackedEntityChanged.raiseEvent(entity); + assert.equal(getActiveCameraMotion(), null); +}); + +/** Metres between the camera and the point it pivots about. */ +const pivotRange = (h) => + Cesium.Cartesian3.distance(h.camera.positionWC, h.target); +const heightM = (camera) => + Cesium.Cartographic.fromCartesian(camera.positionWC).height; + +test('a zoom during a pivoting tilt changes the range it holds', (t) => { + const h = harness(t); + assert.equal( + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }).ok, + true, + ); + h.tick(); + const before = pivotRange(h); + assert.deepEqual( + adjustOrbitRange(0.5), + { kind: 'tilt', changed: true }, + 'the tilt must take the zoom', + ); + for (let i = 0; i < 5; i += 1) h.tick(); + // The next frame's lookAt must not put the old range back. + assert.ok( + Math.abs(pivotRange(h) - before / 2) < 1, + `range ${pivotRange(h)} m, wanted ${before / 2} m`, + ); +}); + +test('a zoom with no pivot to hold is left to the camera', (t) => { + harness(t, { pivot: false }); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + assert.equal(adjustOrbitRange(0.5), null); +}); + +test('a pivot tilt toward the horizon keeps its clearance above the ground', (t) => { + // A close top-down view: swinging 250 m of range to -5° would end ~22 m up. + const h = harness(t, { height: 250 }); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -5 }).ok, true); + let lowest = Infinity; + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + lowest = Math.min(lowest, heightM(h.camera)); + } + assert.equal(getActiveCameraMotion(), null, 'tilt must finish'); + assert.ok(lowest > 149, `camera dropped to ${lowest} m`); + assert.ok( + Math.abs(pitchDeg(h.camera) - -5) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); + assert.ok(Cesium.Matrix4.equals(h.camera.transform, Cesium.Matrix4.IDENTITY)); +}); + +test('a continuous tilt up from close in keeps its clearance', (t) => { + const h = harness(t, { height: 250 }); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + assert.ok(heightM(h.camera) > 149, `camera at ${heightM(h.camera)} m`); + } + assert.ok(Math.abs(pitchDeg(h.camera) - -5) < 0.5); +}); + +test('a city-scale pivot tilt keeps the place centred to the horizon limit', (t) => { + const h = harness(t); + moveCamera({ motion: 'tilt', pitchDeg: -5 }); + h.finish(); + // 4 km of range clears the floor at every pitch: all pivot, no drift. + assert.ok(centreDrift(h) < 5, `centre drifted ${centreDrift(h)} m`); +}); + +test('pitchDeg against a contradicting direction is refused', (t) => { + harness(t, { pitch: -45 }); + // "tilt down 10 degrees": -10° is an absolute angle, which is up from -45°. + const result = moveCamera({ + motion: 'tilt', + direction: 'down', + pitchDeg: 10, + }); + assert.equal(result.ok, false); + assert.match(result.error, /absolute angle/); + assert.equal(getActiveCameraMotion(), null); +}); + +test('pitchDeg with a direction that agrees still tilts', (t) => { + const h = harness(t, { pitch: -45 }); + const result = moveCamera({ + motion: 'tilt', + direction: 'down', + pitchDeg: -80, + }); + assert.equal(result.ok, true); + assert.equal(result.direction, 'down'); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -80) < 0.5); +}); + +test('the pitchDeg and direction check uses the camera pitch', (t) => { + // At regional range the Earth curves away and the pivot pitch is a degree + // or two shallower than the camera's (-30°); -29° is still up from it. + harness(t, { height: 150000, pitch: -30 }); + const down = moveCamera({ motion: 'tilt', direction: 'down', pitchDeg: -29 }); + assert.equal(down.ok, false); + assert.match(down.error, /tilts up, not down/); + const up = moveCamera({ motion: 'tilt', direction: 'up', pitchDeg: -29 }); + assert.equal(up.ok, true); + assert.equal(up.direction, 'up'); +}); + +test('after a zoom in, a pivoting tilt never lowers the camera further', (t) => { + const h = harness(t, { height: 1000 }); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + tickUntil(h, () => pitchDeg(h.camera) >= -30); + adjustOrbitRange(1 / 2.4); + adjustOrbitRange(1 / 2.4); + h.tick(); + const afterZoom = heightM(h.camera); + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + assert.ok( + heightM(h.camera) > afterZoom - 1, + `camera fell from ${afterZoom} m to ${heightM(h.camera)} m`, + ); + } + assert.ok(Math.abs(pitchDeg(h.camera) - -5) < 0.5); +}); + +test('a zoom in during a close pivot tilt never pushes the camera out', (t) => { + // 62 m of range, under the 80 m zoom floor. + const h = harness(t, { height: 40, pitch: -40 }); + moveCamera({ motion: 'tilt', direction: 'down', mode: 'continuous' }); + h.tick(); + const before = pivotRange(h); + assert.ok(before < 80, `range ${before} m`); + adjustOrbitRange(1 / 1.6); + h.tick(); + assert.ok(pivotRange(h) <= before + 0.01, `range grew to ${pivotRange(h)} m`); +}); + +test('a zoom out during a far orbit never pulls the camera in', (t) => { + // 15,000 km up, beyond the 5,000 km zoom cap. + const h = harness(t, { height: 15_000_000 }); + assert.equal(moveCamera({ motion: 'orbit', mode: 'continuous' }).ok, true); + h.tick(); + const before = pivotRange(h); + assert.ok(before > 5_000_000, `range ${before} m`); + assert.deepEqual(adjustOrbitRange(2.4), { kind: 'orbit', changed: false }); + h.tick(); + assert.ok(pivotRange(h) >= before - 1, `range fell to ${pivotRange(h)} m`); +}); + +test('an out-of-range pitchDeg is clamped to the tilt limits', (t) => { + // The schema carries no range; 120 reads as 120° below the horizon. + const h = harness(t, { pitch: -45 }); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: 120 }).ok, true); + h.finish(); + assert.ok( + Math.abs(pitchDeg(h.camera) - -89) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); +}); + +test('a zoom in during a pivot tilt stops at the clearance', (t) => { + // 400 m of range at -40°, then "zoom in a lot" twice: unfloored, that left + // the camera about 107 m up, then at the 80 m zoom floor. + const h = harness(t, { height: 400 }); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + tickUntil(h, () => pitchDeg(h.camera) >= -40); + assert.deepEqual(adjustOrbitRange(1 / 2.4), { kind: 'tilt', changed: true }); + // At -40° the floor is 150 / sin 40° ≈ 233 m, which the first zoom reached. + assert.deepEqual(adjustOrbitRange(1 / 2.4), { kind: 'tilt', changed: false }); + // The camera must take that range, not just report it. + h.tick(); + const floor = 150 / Math.sin(radians(40)); + assert.ok( + Math.abs(pivotRange(h) - floor) < floor * 0.02, + `range ${pivotRange(h)} m, wanted about ${floor} m`, + ); + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + assert.ok(heightM(h.camera) > 149, `camera at ${heightM(h.camera)} m`); + } + assert.ok(Math.abs(pitchDeg(h.camera) - -5) < 0.5); +}); + +test('a voice zoom during a pivot tilt says what it changed', async (t) => { + const h = harness(t, { height: 400 }); + const runner = createGevActionRunner({ + viewer: h.viewer, + styleManager: {}, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + h.tick(); + const zoom = { direction: 'in', amount: 'lot' }; + const first = await runner('adjust_camera_zoom', zoom); + assert.equal(first.ok, true); + assert.equal(first.tiltRangeAdjusted, true); + assert.equal('orbitRadiusAdjusted' in first, false); + // 400 m → 167 m → the 150 m clearance; a third zoom in moves nothing, and + // says so. + assert.equal((await runner('adjust_camera_zoom', zoom)).ok, true); + const third = await runner('adjust_camera_zoom', zoom); + assert.equal(third.ok, false); + assert.match(third.error, /tilt is already at its closest range/); +}); + +test('from globe range a tilt happens in place', (t) => { + // A pivot 2,000 km out would swing the camera thousands of km. + const h = harness(t, { height: 2_000_000 }); + const before = Cesium.Cartesian3.clone(h.camera.positionWC); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -45 }).ok, true); + h.finish(); + assert.ok(Math.abs(pitchDeg(h.camera) - -45) < 0.5); + assert.ok(Cesium.Cartesian3.distance(h.camera.positionWC, before) < 1e-3); +}); + +for (const height of [100_000, 400_000]) + test(`from ${height / 1000} km a tilt to -5° ends with the camera at -5°`, (t) => { + // The pivot pitch is shallower than the camera's by up to 4° at this + // range; the camera reached only -5.9° and -8.6° when the swing stopped + // at a pivot pitch of -5°. + const h = harness(t, { height }); + const result = moveCamera({ motion: 'tilt', pitchDeg: 5 }); + assert.equal(result.pitchDeg, -5); + h.finish(); + assert.ok( + Math.abs(pitchDeg(h.camera) - -5) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); + assert.ok(heightM(h.camera) > 149, `camera at ${heightM(h.camera)} m`); + const nudge = moveCamera({ motion: 'tilt', direction: 'up' }); + assert.equal(nudge.ok, false); + assert.match(nudge.error, /upper tilt limit/); + }); + +// From 550 km out the camera's -5° lies beyond the place's own horizon: the +// swing ran into its clearance, let go of the pivot and tilted the rest in +// place, which put the crosshair on the sky (28 to 63 km up), and a return +// to top-down then landed 600 to 900 km from the place. +for (const height of [600_000, 700_000, 900_000]) { + test(`from ${height / 1000} km a tilt to -5° keeps the place centred`, (t) => { + const h = harness(t, { height }); + const result = moveCamera({ motion: 'tilt', pitchDeg: 5 }); + assert.equal(result.ok, true); + h.finish(); + assert.ok(h.viewTarget(), 'the crosshair is on the sky'); + assert.ok(centreDrift(h) < 100, `centre drifted ${centreDrift(h)} m`); + assert.ok(heightM(h.camera) > 149, `camera at ${heightM(h.camera)} m`); + // The reply names the angle the camera reached, not the one asked for. + assert.ok(result.pitchDeg < -5, `reported ${result.pitchDeg}°`); + assert.ok( + Math.abs(pitchDeg(h.camera) - result.pitchDeg) < 0.5, + `camera at ${pitchDeg(h.camera)}°, reported ${result.pitchDeg}°`, + ); + assert.match(result.note, /horizon/); + const nudge = moveCamera({ motion: 'tilt', direction: 'up' }); + assert.equal(nudge.ok, false); + assert.match(nudge.error, /upper tilt limit/); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -89 }).ok, true); + h.finish(); + assert.ok(centreDrift(h) < 100, `centre drifted ${centreDrift(h)} m`); + // Back over the place at the range it began with (-89° is ~1° off nadir). + const range = Cesium.Cartesian3.distance(h.camera.positionWC, h.target); + assert.ok( + Math.abs(range - height) < height * 0.01, + `top-down ${range} m from the place`, + ); + }); + + test(`a continuous tilt up from ${height / 1000} km stops with the place centred`, (t) => { + const h = harness(t, { height }); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + for (let i = 0; i < 1200 && getActiveCameraMotion(); i += 1) { + h.tick(); + assert.ok(h.viewTarget(), `the crosshair is on the sky at frame ${i}`); + } + assert.equal(getActiveCameraMotion(), null, 'tilt must stop at its limit'); + assert.ok(centreDrift(h) < 100, `centre drifted ${centreDrift(h)} m`); + }); +} + +test('a pitchDeg tilt toward the horizon limit says it is already there', (t) => { + const h = harness(t, { height: 900_000 }); + moveCamera({ motion: 'tilt', pitchDeg: 5 }); + h.finish(); + const again = moveCamera({ motion: 'tilt', pitchDeg: 5 }); + assert.equal(again.ok, false); + assert.match(again.error, /upper tilt limit/); +}); + +test('a regional tilt-up nudge goes to the camera horizon limit', (t) => { + // 20 km up at -7°: the pivot 186 km out sits at -5.3°, but the camera can + // still tilt up 2°. + const h = harness(t, { height: 20000, pitch: -7 }); + assert.equal(moveCamera({ motion: 'tilt', direction: 'up' }).ok, true); + h.finish(); + assert.ok( + Math.abs(pitchDeg(h.camera) - -5) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); +}); + +test('a voice tilt from a tracked view judges its limit from the camera pitch', (t) => { + const h = harness(t, { height: 20000, pitch: -5.2 }); + h.viewer.trackedEntity = { id: 'tracked' }; + // Voice releases the follow camera before the motion starts. + const release = (start) => { + h.viewer.trackedEntity = undefined; + return start(); + }; + const result = moveCamera({ motion: 'tilt', direction: 'up' }, release); + assert.equal(result.ok, false); + assert.match(result.error, /upper tilt limit/); +}); + +test('a zoom out during a pivoting tilt stops at the pivot range limit', (t) => { + // 900 km top-down, "tilt to 10°", then "zoom out a lot" twice: capped at + // the orbit's 5,000 km, the swing reached 2,386 km up at a real -44°. + const h = harness(t, { height: 900_000 }); + assert.equal(moveCamera({ motion: 'tilt', pitchDeg: -10 }).ok, true); + h.tick(); + assert.deepEqual(adjustOrbitRange(2.4), { kind: 'tilt', changed: true }); + assert.deepEqual(adjustOrbitRange(2.4), { kind: 'tilt', changed: false }); + let farthest = 0; + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + farthest = Math.max(farthest, pivotRange(h)); + } + assert.equal(getActiveCameraMotion(), null, 'tilt must finish'); + assert.ok(farthest <= 1_000_001, `swung ${farthest} m from the pivot`); + assert.ok( + Math.abs(pitchDeg(h.camera) - -10) < 0.5, + `pitch ${pitchDeg(h.camera)}`, + ); +}); + +test('an unchanged angle still stops a running motion through navigation', (t) => { + const h = harness(t); + moveCamera({ motion: 'tilt', direction: 'up', mode: 'continuous' }); + tickUntil(h, () => pitchDeg(h.camera) >= -45.3); + let handoffs = 0; + const navigate = (start) => { + handoffs += 1; + return start(); + }; + const result = moveCamera({ motion: 'tilt', pitchDeg: -45 }, navigate); + assert.equal(result.ok, true); + assert.equal(result.unchanged, true); + assert.equal(handoffs, 1, 'the request must take navigation'); + assert.equal(getActiveCameraMotion(), null, 'the tilt must stop'); + const held = pitchDeg(h.camera); + for (let i = 0; i < 30; i += 1) h.tick(); + assert.equal(pitchDeg(h.camera), held); +}); + +for (const direction of ['left', 'right']) + test(`a tilt ${direction} is refused`, (t) => { + harness(t, { pitch: -45 }); + const result = moveCamera({ motion: 'tilt', direction }); + assert.equal(result.ok, false); + assert.match(result.error, /tilt needs up or down/); + const angled = moveCamera({ motion: 'tilt', direction, pitchDeg: -30 }); + assert.equal(angled.ok, false); + assert.match(angled.error, /pitchDeg alone/); + assert.equal(getActiveCameraMotion(), null); + }); + +// The 1,000 km pivot cutoff includes its boundary: at exactly 1,000 km the +// pivot range came back a hair over it, so the tilt ran in place and put the +// crosshair on the sky. +test('from exactly 1,000 km a tilt to -5° still pivots about the place', (t) => { + const h = harness(t, { height: 1_000_000 }); + const result = moveCamera({ motion: 'tilt', pitchDeg: 5 }); + assert.equal(result.ok, true); + h.finish(); + assert.ok(h.viewTarget(), 'the crosshair is on the sky'); + assert.ok(centreDrift(h) < 100, `centre drifted ${centreDrift(h)} m`); + assert.ok( + Math.abs(pitchDeg(h.camera) - result.pitchDeg) < 0.5, + `camera at ${pitchDeg(h.camera)}°, reported ${result.pitchDeg}°`, + ); + assert.match(result.note, /horizon/); +}); + +// A zoom mid-tilt moves the pivot's horizon limit: the zoom's reply names the +// angle the tilt now ends at, and the camera ends there. +test('a zoom out during a tilt to an angle reports the angle it now ends at', (t) => { + const h = harness(t, { height: 400_000 }); + const result = moveCamera({ motion: 'tilt', pitchDeg: -5 }); + assert.equal(result.pitchDeg, -5); + assert.equal(result.note, undefined); + for (let i = 0; i < 10; i += 1) h.tick(); + const zoom = adjustOrbitRange(2.5); + assert.equal(zoom.kind, 'tilt'); + assert.equal(zoom.changed, true); + h.finish(); + assert.ok(h.viewTarget(), 'the crosshair is on the sky'); + assert.ok(zoom.pitchDeg < -5, `zoom reported ${zoom.pitchDeg}°`); + assert.ok( + Math.abs(pitchDeg(h.camera) - zoom.pitchDeg) < 0.5, + `camera at ${pitchDeg(h.camera)}°, zoom reported ${zoom.pitchDeg}°`, + ); + assert.match(zoom.note, /horizon/); +}); + +test('a zoom in during a horizon-limited tilt reports the angle it now reaches', (t) => { + const h = harness(t, { height: 900_000 }); + const result = moveCamera({ motion: 'tilt', pitchDeg: -5 }); + assert.ok(result.pitchDeg < -5, `reported ${result.pitchDeg}°`); + for (let i = 0; i < 10; i += 1) h.tick(); + const zoom = adjustOrbitRange(1 / 2.4); + assert.equal(zoom.changed, true); + h.finish(); + assert.equal(zoom.pitchDeg, -5); + assert.ok( + Math.abs(pitchDeg(h.camera) - -5) < 0.5, + `camera at ${pitchDeg(h.camera)}°`, + ); +}); + +test('a voice zoom during a tilt to an angle passes on the new end angle', async (t) => { + const h = harness(t, { height: 400_000 }); + const runner = createGevActionRunner({ + viewer: h.viewer, + styleManager: {}, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + moveCamera({ motion: 'tilt', pitchDeg: -5 }); + for (let i = 0; i < 10; i += 1) h.tick(); + const zoom = await runner('adjust_camera_zoom', { + direction: 'out', + amount: 'lot', + }); + assert.equal(zoom.ok, true); + assert.equal(zoom.tiltRangeAdjusted, true); + h.finish(); + assert.ok( + Math.abs(pitchDeg(h.camera) - zoom.pitchDeg) < 0.5, + `camera at ${pitchDeg(h.camera)}°, zoom reported ${zoom.pitchDeg}°`, + ); + assert.match(zoom.note, /horizon/); +}); + +// Late in the tilt the camera is already past the angle the new range's +// horizon allows (from 400 km at -8°, a zoom out x2.5 reported -8°): past it +// the place is behind the Earth, so the camera steps down to that limit with +// the zoom, then holds it with no further jump, and the zoom's reply names +// the angle it ends at, not the one it stood at. +test('a late zoom out during a tilt steps to the new limit and reports it', (t) => { + const h = harness(t, { height: 400_000 }); + moveCamera({ motion: 'tilt', pitchDeg: -5 }); + for (let i = 0; i < 70; i += 1) h.tick(); + const zoom = adjustOrbitRange(2.5); + assert.equal(zoom.changed, true); + h.tick(); + assert.equal(zoom.pitchDeg, Math.round(pitchDeg(h.camera))); + let last = pitchDeg(h.camera); + let maxStep = 0; + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + maxStep = Math.max(maxStep, Math.abs(pitchDeg(h.camera) - last)); + last = pitchDeg(h.camera); + } + assert.equal(getActiveCameraMotion(), null, 'tilt must finish'); + assert.ok(maxStep < 0.1, `pitch jumped ${maxStep}° in one frame`); + assert.ok(h.viewTarget(), 'the crosshair is on the sky'); + // Replies give whole degrees: the reported angle is the reached one rounded. + assert.equal( + zoom.pitchDeg, + Math.round(pitchDeg(h.camera)), + `camera at ${pitchDeg(h.camera)}°, zoom reported ${zoom.pitchDeg}°`, + ); + assert.match(zoom.note, /horizon/); +}); + +// Easing from the camera's old pitch at the zoomed-out range put the camera +// below the place's horizon: the first frame's crosshair hit the ground +// 186 km from London, and London stayed hidden for ~11 frames. +test('a late zoom out during a tilt keeps the place under the crosshair', (t) => { + const h = harness(t, { height: 400_000 }); + moveCamera({ motion: 'tilt', pitchDeg: -5 }); + for (let i = 0; i < 70; i += 1) h.tick(); + const zoom = adjustOrbitRange(2.5); + assert.equal(zoom.changed, true); + for (let i = 0; i < 600 && getActiveCameraMotion(); i += 1) { + h.tick(); + const hit = h.viewTarget(); + assert.ok(hit, `frame ${i}: the crosshair is on the sky`); + const drift = centreDrift(h); + assert.ok( + drift < 100, + `frame ${i}: crosshair ${Math.round(drift)} m off the place`, + ); + } + assert.equal(getActiveCameraMotion(), null, 'tilt must finish'); + assert.ok( + Math.abs(pivotRange(h) - 1_000_000) < 1, + `range ${pivotRange(h)} m`, + ); + assert.equal( + zoom.pitchDeg, + Math.round(pitchDeg(h.camera)), + `camera at ${pitchDeg(h.camera)}°, zoom reported ${zoom.pitchDeg}°`, + ); + assert.match(zoom.note, /horizon/); +}); + +// The zoom out left the pivot pitch level with the place, so the zoom in's +// clearance floor came out infinite and it was refused at 1,000 km. +test('a zoom in right after a late zoom out brings the tilt back in', (t) => { + const h = harness(t, { height: 400_000 }); + moveCamera({ motion: 'tilt', pitchDeg: -5 }); + for (let i = 0; i < 70; i += 1) h.tick(); + const range = pivotRange(h); + assert.equal(adjustOrbitRange(2.5).changed, true); + const zoomIn = adjustOrbitRange(0.4); + assert.equal(zoomIn.changed, true, 'the zoom in must not be refused'); + h.finish(); + assert.ok( + Math.abs(pivotRange(h) - range) < 1, + `range ${pivotRange(h)} m, wanted ${range} m`, + ); + assert.ok( + centreDrift(h) < 100, + `crosshair ${centreDrift(h)} m off the place`, + ); +}); diff --git a/src/cameraVerbs.js b/src/cameraVerbs.js index e037472cb..d7127c2f7 100644 --- a/src/cameraVerbs.js +++ b/src/cameraVerbs.js @@ -48,6 +48,33 @@ const ONCE = { }; const PITCH_MIN = Cesium.Math.toRadians(-89); const PITCH_MAX = Cesium.Math.toRadians(-5); +/** A bare "tilt the camera" / "oblique view" settles here. */ +const TILT_DEFAULT_PITCH_DEG = -45; +/** °/s for a tilt to a requested angle; short ones keep the once duration. */ +const TILT_TO_DEG_S = 30; +/** + * A pivoting tilt keeps the range, so its height above the pivot is + * range × sin|pitch|. It swings no lower than this; the rest tilts in place. + */ +const TILT_MIN_CLEARANCE_M = 150; +/** + * Beyond this range a pivot would swing the camera thousands of km about the + * crosshair point, so the tilt happens in place instead. + */ +const TILT_PIVOT_MAX_RANGE_M = 1_000_000; +/** + * A pivot range rebuilt from the camera's position comes back a hair off the + * range it was set at: a range this close past the cutoff still counts as on + * it. + */ +const TILT_PIVOT_RANGE_SLACK_M = 1; +/** + * The shallowest a pivot swing goes when its clearance is not what stops it. + * From several hundred km the camera's own horizon limit lies past the + * place's horizon; tilting the rest in place there would lift the crosshair + * onto the sky, so the swing stops with the place this far above it. + */ +const TILT_MIN_PIVOT_ELEVATION = Cesium.Math.toRadians(0.5); /* ── Route dolly: cinematic tuning ────────────────────────────────────────── * Every knob an owner may want to retune lives in this block. The shaping is @@ -966,6 +993,17 @@ let _trackingRemover = null; let _inputRemovers = []; /** Monotonic id for the motion currently running; 0 means "no motion". */ let _motionSeq = 0; +/** Canvas grabs, touches and wheel turns since load. */ +let _manualInputCount = 0; + +/** + * Lets a caller holding a motion back (behind a fly-to) see whether the user + * took the camera meanwhile; Cesium does not cancel a flight on input. + * @returns {number} Manual camera inputs so far. + */ +export function manualCameraInputCount() { + return _manualInputCount; +} /** * Install a motion and stamp it with a fresh id. The id is what lets a caller @@ -1052,7 +1090,9 @@ export function interruptCameraMotion(reason = 'interrupt') { if ( wasActive && !tracking && - (_active.kind === 'orbit' || _active.kind === 'route') + (_active.kind === 'orbit' || + _active.kind === 'route' || + (_active.kind === 'tilt' && _active.hpr)) ) clearLookAt(); // Instant, not eased: the user grabbed the camera, or the flight is over. @@ -1104,7 +1144,7 @@ function onTick() { let scale = 1; if (m.mode === 'once') { m.elapsed = (m.elapsed || 0) + dt; - const t = Math.min(1, m.elapsed / ONCE.durationS); + const t = Math.min(1, m.elapsed / (m.durationS || ONCE.durationS)); scale = 1 - (1 - t) * (1 - t); // ease-out progress share if (t >= 1) { interruptCameraMotion('once-complete'); @@ -1173,18 +1213,13 @@ function onTick() { else if (m.direction === 'right') cam.moveRight(step); else if (m.direction === 'up') cam.moveUp(step); else cam.moveDown(step); - } else if ( - m.kind === 'tilt' || - m.kind === 'rotate' || - m.kind === 'rotate_globe' - ) { + } else if (m.kind === 'tilt') { + tickTilt(cam, m, scale, dt); + } else if (m.kind === 'rotate' || m.kind === 'rotate_globe') { const degS = TILT_DEG_S[m.speed]; const stepRad = m.mode === 'once' - ? Cesium.Math.toRadians( - m.kind === 'tilt' ? ONCE.tiltDeg : ONCE.rotateDeg, - ) * - (scale - (m.lastScale || 0)) + ? Cesium.Math.toRadians(ONCE.rotateDeg) * (scale - (m.lastScale || 0)) : Cesium.Math.toRadians(degS) * dt; m.lastScale = scale; if (m.kind === 'rotate_globe') { @@ -1218,16 +1253,6 @@ function onTick() { return; } cam.rotate(m.rotationAxis, angle); - } else if (m.kind === 'tilt') { - const up = m.direction === 'up'; - // Clamp pitch to [-89°, -5°]: never through the ground, never at the sky. - const next = cam.pitch + (up ? stepRad : -stepRad); - if (next > PITCH_MAX || next < PITCH_MIN) { - interruptCameraMotion('tilt-clamp'); - return; - } - if (up) cam.lookUp(stepRad); - else cam.lookDown(stepRad); } else if (m.direction === 'left') cam.lookLeft(stepRad); else cam.lookRight(stepRad); } else if (m.kind === 'route') { @@ -1252,6 +1277,186 @@ function onTick() { } } +/** + * One frame of tilt. With a ground pivot the camera swings about it (lookAt), + * so the place under the crosshair stays centred; without one it looks up or + * down in place. Every angle and limit is the camera's own pitch, which at + * regional range is several degrees steeper than the pivot's (the Earth + * curves away), so the swing solves for the pivot pitch that gives it. Pitch + * is clamped to [-89°, -5°] only in the direction of travel: a top-down view + * sits at -90°, below the limit, and must still be able to tilt up out of it. + * A pivot swing toward the horizon lowers the camera, so it stops at + * TILT_MIN_CLEARANCE_M and the rest tilts in place. Farther out it stops with + * the camera TILT_MIN_PIVOT_ELEVATION above the place's horizon instead, and + * keeps the place centred: tilting in place there would lift the crosshair + * onto the sky. + */ +function tickTilt(cam, m, scale, dt) { + const from = cam.pitch; + const want = + m.mode === 'once' + ? m.startPitch + (m.goalPitch - m.startPitch) * scale + : from + + Cesium.Math.toRadians(TILT_DEG_S[m.speed]) * + dt * + (m.direction === 'up' ? 1 : -1); + const up = want > from; + const limited = up + ? Math.min(want, Math.max(from, PITCH_MAX)) + : Math.max(want, Math.min(from, PITCH_MIN)); + let atLimit = false; + if (m.hpr) { + // Apply a range a zoom set since the last frame, even when the pitch + // then has nowhere to go (a zoom in to the clearance ends the pivot). + cam.lookAt(m.target, m.hpr); + // Up toward the horizon the swing lowers the camera: hold the clearance, + // and keep the place below the horizon. + const cap = up + ? Math.max(m.hpr.pitch, tiltPivotPitchMax(m.hpr.range)) + : Infinity; + for (let i = 0; i < 3 && Math.abs(limited - cam.pitch) > 1e-6; i += 1) { + const pitch = Math.min( + cap, + Math.max(-Cesium.Math.PI_OVER_TWO, m.hpr.pitch + limited - cam.pitch), + ); + if (pitch === m.hpr.pitch) break; + m.hpr.pitch = pitch; + cam.lookAt(m.target, m.hpr); + } + // A once motion that just completed released the lookAt frame before + // this last step re-took it. + if (_active !== m) clearLookAt(); + if (up && m.hpr.pitch >= cap && cam.pitch < limited - 1e-6) { + if (tiltClearanceBinds(m.hpr.range)) { + // Down to its clearance: let go of the pivot and tilt the rest in + // place, which keeps the height. + if (_active === m) clearLookAt(); + m.hpr = null; + m.target = null; + } else { + // The place is at its horizon limit: stop here, still centred. + atLimit = true; + } + } + } + if (!m.hpr) cam.lookUp(limited - cam.pitch); + if (m.mode !== 'once' && (limited !== want || atLimit)) + interruptCameraMotion('tilt-clamp'); +} + +/** The refusal for a tilt up the camera has no room left for. */ +function tiltUpLimitRefusal() { + return { + ok: false, + action: 'move_camera', + error: + 'Already at the upper tilt limit (near the horizon) — tilt down, or zoom/pan instead.', + }; +} + +/** Shallowest pivot pitch that keeps the clearance at this range. */ +function tiltPivotPitchMax(range) { + return Math.min( + -Math.asin(Math.min(1, TILT_MIN_CLEARANCE_M / range)), + -TILT_MIN_PIVOT_ELEVATION, + ); +} + +/** + * Whether the clearance, not the horizon, stops a swing at this range. Only + * then does the rest of the tilt happen in place: that close in (under ~17 + * km) the camera's -5° limit stays well below its own horizon. + */ +function tiltClearanceBinds(range) { + return TILT_MIN_CLEARANCE_M / range >= Math.sin(TILT_MIN_PIVOT_ELEVATION); +} + +/** + * The shallowest camera pitch a tilt starting from this frame can reach with + * the place under the crosshair still in view: PITCH_MAX, or less where a + * far pivot meets its horizon limit first. + */ +function tiltUpLimit({ target, pivot }) { + if (!pivot || tiltClearanceBinds(pivot.range)) return PITCH_MAX; + return Math.min( + PITCH_MAX, + pivotCameraPitch( + target, + pivot, + Math.max(pivot.pitch, tiltPivotPitchMax(pivot.range)), + ), + ); +} + +/** The camera pitch lookAt gives for this pivot at the given pivot pitch. */ +function pivotCameraPitch(target, { heading, range }, pitch) { + const offset = new Cesium.Cartesian3( + -range * Math.cos(pitch) * Math.sin(heading), + -range * Math.cos(pitch) * Math.cos(heading), + -range * Math.sin(pitch), + ); + const eye = Cesium.Matrix4.multiplyByPoint( + Cesium.Transforms.eastNorthUpToFixedFrame(target), + offset, + new Cesium.Cartesian3(), + ); + const look = Cesium.Cartesian3.subtract(target, eye, new Cesium.Cartesian3()); + Cesium.Cartesian3.normalize(look, look); + const ellipsoid = _viewer.scene.globe?.ellipsoid ?? Cesium.Ellipsoid.WGS84; + const upAtEye = ellipsoid.geodeticSurfaceNormal(eye, new Cesium.Cartesian3()); + return Math.asin( + Cesium.Math.clamp(Cesium.Cartesian3.dot(look, upAtEye), -1, 1), + ); +} + +/** + * Where a tilt starts. It swings about the ground under the crosshair so the + * place stays centred. It tilts in place with no ground there, from globe + * range, or while an entity is tracked (the follow camera owns that frame; + * voice releases it before moving, so a voice tilt still pivots). + */ +function tiltStartFrame(cam) { + const target = _viewer.trackedEntity ? null : _getTarget?.(_viewer); + let pivot = target ? tiltPivot(cam, target) : null; + if (pivot && !tiltPivotInReach(pivot.range)) pivot = null; + return { target, pivot, from: cam.pitch }; +} + +/** Whether a tilt may pivot at this range: up to and including the cutoff. */ +function tiltPivotInReach(range) { + return range <= TILT_PIVOT_MAX_RANGE_M + TILT_PIVOT_RANGE_SLACK_M; +} + +/** The reply's note for a tilt the place's horizon stops short of its angle. */ +function tiltHorizonNote(pitch) { + return `Stopped at ${Math.round(Cesium.Math.toDegrees(pitch))}°: from this range a shallower angle puts the place past the horizon. Zoom in to tilt further.`; +} + +/** Ground pivot for a tilt: heading, pitch and range in the target's own frame. */ +function tiltPivot(cam, target) { + const local = Cesium.Matrix4.multiplyByPoint( + Cesium.Matrix4.inverseTransformation( + Cesium.Transforms.eastNorthUpToFixedFrame(target), + new Cesium.Matrix4(), + ), + cam.positionWC, + new Cesium.Cartesian3(), + ); + const range = Cesium.Cartesian3.magnitude(local); + if (!(range > 1) || local.z <= 0) return null; + // Near straight overhead the offset's heading is float noise (it flipped a + // north-up London view to south); keep the camera's own. + const heading = + Math.hypot(local.x, local.y) > range * 0.01 + ? Math.atan2(-local.x, -local.y) + : cam.heading; + return new Cesium.HeadingPitchRange( + heading, + -Math.asin(Math.min(1, local.z / range)), + range, + ); +} + /** Rigid origin-centred rotation stays outside the actual ellipsoid's bounding sphere. */ function globeRotationHasClearance(position) { const ellipsoid = _viewer.scene.globe?.ellipsoid ?? Cesium.Ellipsoid.WGS84; @@ -1421,18 +1626,108 @@ export function initCameraVerbs( _trackingRemover = viewer.trackedEntityChanged?.addEventListener((entity) => { if (entity && _active?.kind === 'rotate_globe') interruptCameraMotion('globe-camera-owner-changed'); + // A pivoting tilt's lookAt would fight the follow camera every frame. + else if (entity && _active?.kind === 'tilt' && _active.hpr) + interruptCameraMotion('tilt-camera-owner-changed'); }); for (const rm of _inputRemovers) rm(); _inputRemovers = []; const canvas = viewer.scene.canvas; // ANY manual camera input reclaims control — the cancelFlight reflex. for (const evt of ['pointerdown', 'touchstart', 'wheel']) { - const h = () => interruptCameraMotion('manual-input'); + const h = () => { + _manualInputCount += 1; + interruptCameraMotion('manual-input'); + }; canvas.addEventListener(evt, h, { passive: true }); _inputRemovers.push(() => canvas.removeEventListener(evt, h)); } } +/** + * A tilt's angle: the one asked for, the default oblique view when a bare + * "tilt the camera" names neither angle nor direction, or null for a nudge. + */ +function requestedTiltPitch(args, direction) { + // Only a real number or numeric text: Number('') and Number(null) are 0, + // which would swing the camera to the horizon. + const raw = args.pitchDeg; + const requested = + typeof raw === 'number' || (typeof raw === 'string' && raw.trim() !== '') + ? Number(raw) + : NaN; + // The camera never looks above the horizon, so a positive angle can only + // mean the same angle below it ("tilt 45 degrees" sent as 45). + if (Number.isFinite(requested)) + return Cesium.Math.clamp( + Cesium.Math.toRadians(-Math.abs(requested)), + PITCH_MIN, + PITCH_MAX, + ); + return direction === null + ? Cesium.Math.toRadians(TILT_DEFAULT_PITCH_DEG) + : null; +} + +/** + * Argument errors move_camera refuses whatever the camera is doing, so a + * caller holding a motion back behind a flight can refuse it before waiting. + * @param {Object} args + * @returns {Object|null} The refusal, or null when the arguments are usable. + */ +export function moveCameraArgsError(args = {}) { + const motion = String(args.motion || '').toLowerCase(); + const direction = args.direction + ? String(args.direction).toLowerCase() + : null; + if (motion === 'stop') return null; + if (!['orbit', 'pan', 'tilt', 'rotate', 'rotate_globe'].includes(motion)) { + return { + ok: false, + action: 'move_camera', + error: `Unknown motion "${args.motion}" — use orbit, pan, tilt, rotate, rotate_globe, or stop.`, + }; + } + if ( + motion !== 'orbit' && + motion !== 'rotate_globe' && + (motion !== 'tilt' || requestedTiltPitch(args, direction) === null) && + !['left', 'right', 'up', 'down'].includes(direction) + ) { + return { + ok: false, + action: 'move_camera', + error: `${motion} needs a direction (${motion === 'tilt' ? 'up/down' : motion === 'rotate' ? 'left/right' : 'left/right/up/down'}).`, + }; + } + if (motion === 'tilt' && (direction === 'left' || direction === 'right')) { + return { + ok: false, + action: 'move_camera', + error: 'tilt needs up or down, or pitchDeg alone for an angle.', + }; + } + if (motion === 'rotate' && !['left', 'right'].includes(direction)) { + return { + ok: false, + action: 'move_camera', + error: 'rotate needs left or right.', + }; + } + if ( + motion === 'rotate_globe' && + direction !== null && + !['left', 'right'].includes(direction) + ) { + return { + ok: false, + action: 'move_camera', + error: 'rotate_globe needs left or right.', + }; + } + return null; +} + /** * move_camera implementation. Returns the house result shape; rejections are * plain-English `ok:false` errors. @@ -1465,54 +1760,66 @@ export function moveCamera(args = {}, runNavigation = null) { }; return typeof runNavigation === 'function' ? runNavigation(stop) : stop(); } - if (!['orbit', 'pan', 'tilt', 'rotate', 'rotate_globe'].includes(motion)) { - return { - ok: false, - action: 'move_camera', - error: `Unknown motion "${args.motion}" — use orbit, pan, tilt, rotate, rotate_globe, or stop.`, - }; - } - if ( - motion !== 'orbit' && - motion !== 'rotate_globe' && - !['left', 'right', 'up', 'down'].includes(direction) - ) { - return { - ok: false, - action: 'move_camera', - error: `${motion} needs a direction (left/right${motion !== 'rotate' ? '/up/down' : ''}).`, - }; - } - if (motion === 'rotate' && !['left', 'right'].includes(direction)) { - return { - ok: false, - action: 'move_camera', - error: 'rotate needs left or right.', - }; - } - if ( - motion === 'rotate_globe' && - direction !== null && - !['left', 'right'].includes(direction) - ) { - return { - ok: false, - action: 'move_camera', - error: 'rotate_globe needs left or right.', - }; - } - if (motion === 'tilt') { - // Honest limits: at the clamp a tilt is a no-op — say so instead of - // reporting success and letting the model gaslight itself (field test). - const pitch = _viewer.camera.pitch; - if (direction === 'up' && pitch >= PITCH_MAX - Cesium.Math.toRadians(0.5)) { + const argsError = moveCameraArgsError(args); + if (argsError) return argsError; + const tiltToPitch = + motion === 'tilt' ? requestedTiltPitch(args, direction) : null; + if (motion === 'tilt' && tiltToPitch !== null) { + // Angles and limits are the camera's own pitch, whatever the swing pivots + // about; releasing the follow camera does not change it. + const from = _viewer.camera.pitch; + // -89 or -90 means top-down, which a -90° view already is: the clamp + // must not tilt it up 1°. + const atTopDown = tiltToPitch <= PITCH_MIN && from <= PITCH_MIN; + if ( + atTopDown || + Math.abs(tiltToPitch - from) < Cesium.Math.toRadians(0.5) + ) { + // Still a camera request: it stops a running motion and releases the + // follow camera, as a tilt that moves would. + const hold = () => { + interruptCameraMotion('replaced'); + return { + ok: true, + action: 'move_camera', + motion, + pitchDeg: Math.round( + Cesium.Math.toDegrees(atTopDown ? from : tiltToPitch), + ), + unchanged: true, + }; + }; + return typeof runNavigation === 'function' ? runNavigation(hold) : hold(); + } + // pitchDeg is an absolute angle, not a step: "tilt down 10 degrees" sent + // as {direction:'down', pitchDeg:10} would tilt up from -45°. + const towards = tiltToPitch > from ? 'up' : 'down'; + if ((direction === 'up' || direction === 'down') && direction !== towards) { + const target = Math.round(Cesium.Math.toDegrees(tiltToPitch)); + const now = Math.round(Cesium.Math.toDegrees(from)); return { ok: false, action: 'move_camera', - error: - 'Already at the upper tilt limit (near the horizon) — tilt down, or zoom/pan instead.', + error: `pitchDeg is an absolute angle, not a step: ${target}° from ${now}° tilts ${towards}, not ${direction}. Send direction alone for a nudge, or pitchDeg alone for an angle.`, }; } + if ( + towards === 'up' && + from >= + tiltUpLimit(tiltStartFrame(_viewer.camera)) - Cesium.Math.toRadians(0.5) + ) + return tiltUpLimitRefusal(); + } else if (motion === 'tilt') { + // Honest limits: at the clamp a tilt is a no-op — say so instead of + // reporting success and letting the model gaslight itself (field test). + const pitch = _viewer.camera.pitch; + if ( + direction === 'up' && + pitch >= + tiltUpLimit(tiltStartFrame(_viewer.camera)) - Cesium.Math.toRadians(0.5) + ) { + return tiltUpLimitRefusal(); + } if ( direction === 'down' && pitch <= PITCH_MIN + Cesium.Math.toRadians(0.5) @@ -1607,6 +1914,7 @@ export function moveCamera(args = {}, runNavigation = null) { error: 'Globe rotation requires a clear ellipsoid bounding sphere.', }; } + let horizonLimited = null; const start = () => { interruptCameraMotion('replaced'); const state = { @@ -1615,6 +1923,43 @@ export function moveCamera(args = {}, runNavigation = null) { speed, mode, }; + if (motion === 'tilt') { + const frame = tiltStartFrame(_viewer.camera); + const { target, pivot, from } = frame; + if (pivot) { + state.target = target; + state.hpr = pivot; + } + state.startPitch = from; + const upLimit = tiltUpLimit(frame); + if (tiltToPitch !== null) { + // From far out the place reaches its horizon before the camera + // reaches the asked angle: stop there, and say so. + const goal = + tiltToPitch > from + ? Math.min(tiltToPitch, Math.max(from, upLimit)) + : tiltToPitch; + state.mode = 'once'; + state.direction = goal > from ? 'up' : 'down'; + state.goalPitch = goal; + // A zoom mid-tilt moves the horizon limit; it re-judges the goal + // against the angle asked for. + state.askedPitch = tiltToPitch; + state.durationS = Math.max( + ONCE.durationS, + Math.abs(Cesium.Math.toDegrees(goal - from)) / TILT_TO_DEG_S, + ); + if (goal !== tiltToPitch) horizonLimited = goal; + } else { + // A nudge stops at the limit it travels toward. + const step = + Cesium.Math.toRadians(ONCE.tiltDeg) * (direction === 'up' ? 1 : -1); + state.goalPitch = + step > 0 + ? Math.min(from + step, Math.max(from, upLimit)) + : Math.max(from + step, Math.min(from, PITCH_MIN)); + } + } if (motion === 'rotate_globe') { // Convert an inherited ground-target frame without changing the world // pose. Rotation must pivot about Earth, never the previous local target. @@ -1633,9 +1978,11 @@ export function moveCamera(args = {}, runNavigation = null) { } if (motion === 'orbit') { if (_viewer.scene.tweens.length > 0) { - // A camera flight is in progress ("fly to X and orbit it") — ALWAYS arm - // and capture at the destination; a target existing right now is - // meaningless (the globe is always under the crosshair). + // A camera flight is in progress — ALWAYS arm and capture at the + // destination; a target existing right now is meaningless (the globe + // is always under the crosshair). The voice runner already holds an + // orbit until its own fly-to lands; this covers any other + // flight. setActiveMotion({ kind: 'orbit', direction: direction || 'right', @@ -1699,7 +2046,13 @@ export function moveCamera(args = {}, runNavigation = null) { motion, direction: state.direction, speed, - mode, + mode: state.mode, + ...(tiltToPitch !== null && { + pitchDeg: Math.round( + Cesium.Math.toDegrees(horizonLimited ?? tiltToPitch), + ), + }), + ...(horizonLimited !== null && { note: tiltHorizonNote(horizonLimited) }), }; }; const preserveCameraFlight = @@ -1714,16 +2067,93 @@ export function moveCamera(args = {}, runNavigation = null) { /** * Zoom while orbiting: scale the orbit RADIUS instead of letting the zoom * fight the per-frame lookAt (field test: "zoom out did literally nothing"). + * A tilt swinging about its pivot holds a range the same way. * @param {number} factor >1 zooms out (radius grows), <1 zooms in. - * @returns {boolean} true when an orbit consumed the zoom. + * @returns {{kind: string, changed: boolean, pitchDeg?: number, + * note?: string}|null} the motion that consumed the zoom, and whether its + * range moved; null when none did. A tilt to an angle whose end the new + * range moves also names the angle it now ends at (pitchDeg), with a note. */ export function adjustOrbitRange(factor) { - if (!_active || _active.kind !== 'orbit' || !_active.hpr) return false; - _active.hpr.range = Math.max( - 80, - Math.min(5_000_000, _active.hpr.range * factor), + if (!_active || !['orbit', 'tilt'].includes(_active.kind) || !_active.hpr) + return null; + // A tilt's zoom in stops at the range that keeps its clearance. The limits + // never move a view the wrong way: a zoom in leaves a closer view in place, + // and a zoom out leaves a farther one. + const { kind, hpr } = _active; + const floor = + kind === 'tilt' + ? Math.max(80, TILT_MIN_CLEARANCE_M / Math.abs(Math.sin(hpr.pitch))) + : 80; + // A tilt pivots only within TILT_PIVOT_MAX_RANGE_M; farther out the swing + // would carry the camera thousands of km. + const ceiling = kind === 'tilt' ? TILT_PIVOT_MAX_RANGE_M : 5_000_000; + const range = hpr.range; + hpr.range = Math.max( + Math.min(floor, range), + Math.min(Math.max(ceiling, range), range * factor), ); - return true; + // Under half a metre counts as not moved, as for a plain zoom. + const changed = Math.abs(hpr.range - range) >= 0.5; + return { kind, changed, ...(changed && retargetTiltAfterZoom(_active)) }; +} + +/** + * A zoom changed the range a tilt to an angle holds, and with it the place's + * horizon limit: aim the tilt at the angle asked for or the new limit, + * whichever comes first, from where the camera is now. The pivot pitch is + * re-solved so the new range keeps the camera's pitch, and a limit the camera + * is already past is eased down to rather than jumped to. Returns the end + * angle and a note for the reply when that end moved, otherwise nothing. + */ +function retargetTiltAfterZoom(m) { + if (m.kind !== 'tilt' || m.mode !== 'once' || m.askedPitch === undefined) + return {}; + let from = _viewer.camera.pitch; + if (!(m.askedPitch > from)) return {}; + if (m.hpr) { + // At the same pivot pitch a longer range looks down more steeply (the + // Earth curves away): find the pivot pitch that holds this camera pitch. + // It stays below the place's horizon limit (or where the pivot already + // was): past it the place drops behind the Earth, so a camera the zoom + // leaves past that limit steps down to it with the zoom. + const cap = Math.max(m.hpr.pitch, tiltPivotPitchMax(m.hpr.range)); + for (let i = 0; i < 4; i += 1) { + const error = from - pivotCameraPitch(m.target, m.hpr, m.hpr.pitch); + if (Math.abs(error) < 1e-7) break; + const pitch = Cesium.Math.clamp( + m.hpr.pitch + error, + -Cesium.Math.PI_OVER_TWO, + cap, + ); + if (pitch === m.hpr.pitch) break; + m.hpr.pitch = pitch; + } + from = Math.min(from, pivotCameraPitch(m.target, m.hpr, m.hpr.pitch)); + } + const limit = + m.hpr && !tiltClearanceBinds(m.hpr.range) + ? Math.min( + PITCH_MAX, + pivotCameraPitch(m.target, m.hpr, tiltPivotPitchMax(m.hpr.range)), + ) + : PITCH_MAX; + const goal = Math.min(m.askedPitch, limit); + if (Math.abs(goal - m.goalPitch) < 1e-6) return {}; + m.goalPitch = goal; + m.startPitch = from; + m.elapsed = 0; + m.durationS = Math.max( + ONCE.durationS, + Math.abs(Cesium.Math.toDegrees(goal - from)) / TILT_TO_DEG_S, + ); + const limited = goal !== m.askedPitch; + return { + pitchDeg: Math.round(Cesium.Math.toDegrees(goal)), + note: limited + ? tiltHorizonNote(goal) + : `The tilt now reaches ${Math.round(Cesium.Math.toDegrees(goal))}°.`, + }; } /** diff --git a/src/firstRunExperience.test.mjs b/src/firstRunExperience.test.mjs index ddcdbbd53..8224cecbf 100644 --- a/src/firstRunExperience.test.mjs +++ b/src/firstRunExperience.test.mjs @@ -1020,6 +1020,14 @@ test('the voice TOOL SCHEMA matches the pinned contract — the mission mapping 'noir', ]); visualStyle.enum = [...visualStyle.enum, 'snow']; + // Tilt to an angle adds pitchDeg and rewords direction; restore + // the legacy camera fields to keep the pin. + const camera = legacyTools.find((tool) => tool.name === 'move_camera') + .parameters.properties; + assert.equal(camera.pitchDeg.type, 'number'); + delete camera.pitchDeg; + camera.direction.description = + 'Required for pan/tilt/rotate; orbit/rotate_globe default right.'; const block = JSON.stringify(legacyTools); // Re-derived for the voice layer manifest (generated layer enums, alias and // field hints), point-and-ask's pointer/referent arguments and the prompt diff --git a/src/radioMarkup.test.mjs b/src/radioMarkup.test.mjs index f3c3a98a6..324adceb5 100644 --- a/src/radioMarkup.test.mjs +++ b/src/radioMarkup.test.mjs @@ -232,10 +232,17 @@ test('no unchanged Realtime tool definition drifts silently', () => { camera.description, 'Direct the camera like a drone operator: orbit the current view target, pan, tilt, or rotate — one bounded nudge (mode=once) or continuous motion until stopped (mode=continuous). Continuous motion also stops on any manual camera input or when a navigation tool runs. Say the RESULTING state when confirming ("Orbiting slowly"). Globe rotation (rotate_globe) spins Earth right continuously by default; explicit direction, mode and speed take precedence. It checks fit and may refuse.', ); + // Tilt to an angle adds an absolute pitchDeg with its -45° default, and + // names tilt steps among the direction's uses; check them, then restore + // the legacy fields for the pin. assert.equal( camera.parameters.properties.direction.description, - 'Required for pan/tilt/rotate; orbit/rotate_globe default right.', + 'Pan/rotate/tilt steps; orbit/rotate_globe default right.', ); + assert.equal(camera.parameters.properties.pitchDeg.type, 'number'); + assert.match(camera.parameters.properties.pitchDeg.description, /^Absolute;/); + assert.match(camera.parameters.properties.pitchDeg.description, /-45 default/); + delete camera.parameters.properties.pitchDeg; assert.equal(camera.parameters.properties.mode.description, 'once = bounded eased nudge; continuous = until stop/manual input. rotate_globe defaults continuous; other motions default once.'); camera.parameters.properties.mode.description = 'once = bounded eased nudge (default); continuous = until stop/manual input.'; camera.parameters.properties.motion.enum = diff --git a/src/ui/shellFacade.js b/src/ui/shellFacade.js index 95bf791d4..678b69c79 100644 --- a/src/ui/shellFacade.js +++ b/src/ui/shellFacade.js @@ -6,7 +6,7 @@ export class ShellFacade { /** Observe camera ownership transfers through the navigation owner. */ subscribeCameraHandoff(listener) { - return this._navigation.subscribeCameraHandoff(listener); + return this._navigation?.subscribeCameraHandoff(listener); } get _models3dEnabled() { diff --git a/src/ui/shellFacade.test.mjs b/src/ui/shellFacade.test.mjs new file mode 100644 index 000000000..abb892b37 --- /dev/null +++ b/src/ui/shellFacade.test.mjs @@ -0,0 +1,33 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { ShellFacade } from './shellFacade.js'; + +// A shell assembled without its navigation owner (as unit tests build one +// from the prototype) has no handoffs to report, like the facade's other +// optional owners; the voice runner subscribes as it is created. +test('camera handoffs without a navigation owner subscribe to nothing', () => { + const shell = Object.create(ShellFacade.prototype); + assert.equal( + shell.subscribeCameraHandoff(() => {}), + undefined, + ); +}); + +test('camera handoffs reach listeners through the navigation owner', () => { + const listeners = new Set(); + const shell = Object.create(ShellFacade.prototype); + shell._navigation = { + subscribeCameraHandoff(listener) { + listeners.add(listener); + return () => listeners.delete(listener); + }, + }; + let handoffs = 0; + const unsubscribe = shell.subscribeCameraHandoff(() => { + handoffs += 1; + }); + for (const listener of listeners) listener(); + assert.equal(handoffs, 1); + unsubscribe(); + assert.equal(listeners.size, 0); +}); diff --git a/src/voice/actionSchemas.js b/src/voice/actionSchemas.js index 47f8fcdbe..ef122985f 100644 --- a/src/voice/actionSchemas.js +++ b/src/voice/actionSchemas.js @@ -753,6 +753,7 @@ const schemas = [ type: 'string', enum: ['once', 'continuous'], }, + pitchDeg: { type: 'number' }, }, required: ['motion'], }, diff --git a/src/voice/actionSchemas.test.mjs b/src/voice/actionSchemas.test.mjs index bbb1ed13b..296ae0251 100644 --- a/src/voice/actionSchemas.test.mjs +++ b/src/voice/actionSchemas.test.mjs @@ -56,7 +56,9 @@ test('the complete Realtime tool payload pins the manifest-generated layer relea // area scopes and the osm-places layer; natural-region candidates and the // tightened area tool wording follow review. // control_cockpit states the own-layer entry rule (Contacts not required). - '9522ff6f6d640bede81c51cea6b9601439b08becc5bc20b7355aec24feb17e8d', + // move_camera tilt takes an optional pitchDeg angle, described as + // absolute with its -45 default, with steps sent as a direction. + '6a0d154e68beb787c4e20f12d29a5c86577dc343f3e71d8b5651998ea45d8c3c', ); }); @@ -175,6 +177,8 @@ test('all legacy action arguments are byte-identical after removing the delibera 'stop', ]); motion.enum = motion.enum.filter((value) => value !== 'rotate_globe'); + // Tilt to an angle is additive. + delete property('move_camera').pitchDeg; const panels = property('set_panel_open').panelId; panels.enum = panels.enum.filter( (id) => @@ -320,3 +324,15 @@ test('direct voice timeout schema accepts None and exact integers, rejects malfo for (const minutes of [-1, 61, 1.5, '5']) assert.ok(validateArgs(tool.parameters, { minutes }).length > 0); }); + +test('tilt accepts an angle with no direction', () => { + const schema = GEV_ACTION_SCHEMAS.find( + (tool) => tool.name === 'move_camera', + ).parameters; + assert.deepEqual(validateArgs(schema, { motion: 'tilt', pitchDeg: -45 }), []); + assert.deepEqual(validateArgs(schema, { motion: 'tilt' }), []); + // No range in the schema (prompt budget): the runner clamps to -89..-5. + assert.ok( + validateArgs(schema, { motion: 'tilt', pitchDeg: 'steep' }).length > 0, + ); +}); diff --git a/src/voice/geminiSession.js b/src/voice/geminiSession.js index da93b4bb6..8996774b3 100644 --- a/src/voice/geminiSession.js +++ b/src/voice/geminiSession.js @@ -184,6 +184,9 @@ export class GeminiSessionController { this.model = null; this.epoch = 0; this.turn = 0; + // Identifies one turn's tool calls to the runner as one reply; a new + // object each turn, so no two sessions or turns share it. + this.turnReply = {}; this.globeToolsBlocked = false; this.globeResetFailed = false; this.inputGeneration = 0; @@ -971,6 +974,7 @@ export class GeminiSessionController { const tools = [...this.tools.values()]; this.turnAbort = new AbortController(); this.turn++; + this.turnReply = {}; this.typedIntent = null; this.radioStrongerCommit = null; this.pruneControls(); @@ -1544,6 +1548,7 @@ export class GeminiSessionController { ) throw new Error(msg('Gemini requested too many simultaneous actions')); const turn = this.turn; + const reply = this.turnReply; this.userTurnPending = false; this.modelComplete = false; // Admit each valid native call ID once per connection. @@ -1565,6 +1570,7 @@ export class GeminiSessionController { const action = new AbortController(); action.call = { id: call.id, name: call.name }; action.sequence = ++this.toolSequence; + action.reply = reply; this.tools.set(call.id, action); const priorTools = this.toolQueue; const args = @@ -1674,6 +1680,9 @@ export class GeminiSessionController { this.runAction(call.name, args, { signal: toolSignal, callId: call.id, + // Calls run one by one, so a motion can follow a fly-to of the + // same reply that already found nothing. + reply: action.reply, progress: (update) => { if (actionCurrent()) this.narration.progress(call.id, update); }, diff --git a/src/voice/geminiSession.test.mjs b/src/voice/geminiSession.test.mjs index 9ce8d351a..82f403ef7 100644 --- a/src/voice/geminiSession.test.mjs +++ b/src/voice/geminiSession.test.mjs @@ -1241,6 +1241,34 @@ test('successful stronger Radio stop still invalidates the prepared group', asyn h.c.stop(); }); +test('Gemini tools of one turn share a reply identity, and a new turn gets another', async () => { + const replies = []; + const h = harness({ + runAction: (name, _args, next) => { + replies.push([name, next.reply]); + return { ok: true, action: name }; + }, + }); + await h.c.start({ pushToTalk: true }); + h.c.sendTextCommand('Fly to Xyzzyville and pan left'); + const ws = h.sockets[0]; + requestTool(ws, 'fly', 'fly_to_location', { query: 'Xyzzyville' }); + requestTool(ws, 'pan', 'move_camera', { motion: 'pan', direction: 'left' }); + await h.c.toolQueue; + assert.equal(replies.length, 2); + assert.ok(replies[0][1], 'a sequentially run call carries its reply'); + assert.equal(replies[1][1], replies[0][1]); + drainRadioReply(h); + await tick(); + assert.equal(h.c.sendTextCommand('Pan left'), true); + requestTool(ws, 'pan-2', 'move_camera', { motion: 'pan', direction: 'left' }); + await h.c.toolQueue; + assert.equal(replies.length, 3); + assert.ok(replies[2][1]); + assert.notEqual(replies[2][1], replies[0][1]); + h.c.stop(); +}); + test('slow Gemini tools reuse shared progress timing without injecting a conversation turn', async () => { const pending = deferred(); let options; diff --git a/src/voice/gevActions.js b/src/voice/gevActions.js index 53208c538..9692edf94 100644 --- a/src/voice/gevActions.js +++ b/src/voice/gevActions.js @@ -47,6 +47,9 @@ import { flyRoute, interruptCameraMotion, adjustOrbitRange, + manualCameraInputCount, + moveCameraArgsError, + getActiveCameraMotion, } from '../cameraVerbs.js'; import * as defaultFloorServices from '../data/groundFloor.js'; import { isPickedWorldPosition } from '../data/scenePick.js'; @@ -286,6 +289,28 @@ export function createGevActionRunner({ ? { ...SPEECH_BUILDERS, ...speechBuilders } : SPEECH_BUILDERS; const _layerEnabledAt = new Map(); + // The latest voice flight. A tilt, orbit, pan or rotate in the same reply + // as "fly to X" waits for it to land: run at once, its navigation + // cancelled the flight, or the flight (still geocoding) then replaced the + // motion. The wait follows later fly-tos (voiceFlightTarget). + let voiceFlight = null; + // Waiters to tell when a newer voice flight begins. + const voiceFlightWatchers = new Set(); + // Camera handoffs (navigation stamps) seen, and how many of them voice + // fly-tos made. A search, share link or import that stamps during a flight + // leaves it flying; the motion queued behind it then yields to that one. + // The voice fly-tos of the latest reply that sent one: whether any moved + // the camera, and whether one found nothing. A motion of that reply run + // after its fly-to found nothing (Gemini runs a reply's calls one by one) + // has no flight to wait on, and must not move the old view instead. + let replyFlights = null; + let cameraHandoffs = 0; + let voiceFlightHandoffs = 0; + const otherHandoffs = () => cameraHandoffs - voiceFlightHandoffs; + const unsubscribeCameraHandoff = + styleManager?.subscribeCameraHandoff?.(() => { + cameraHandoffs += 1; + }) ?? null; let analystEngine; // Area handles live for this runner (the page): resolve_area, the // analyst's region/area/drawn scopes, find_imagery and osm_query all read @@ -427,13 +452,30 @@ export function createGevActionRunner({ const zoomOut = String(args.direction || '').toLowerCase() === 'out'; const amt = String(args.amount || 'medium').toLowerCase(); const factor = { little: 1.25, medium: 1.6, lot: 2.4 }[amt] || 1.6; - if (adjustOrbitRange(zoomOut ? factor : 1 / factor)) { + const held = adjustOrbitRange(zoomOut ? factor : 1 / factor); + if (held) { + const direction = zoomOut ? 'out' : 'in'; + if (!held.changed) { + return { + ok: false, + action: 'adjust_camera_zoom', + direction, + amount: amt, + error: `Not zoomed: the ${held.kind} is already at its ${zoomOut ? 'farthest' : 'closest'} range.`, + }; + } return { ok: true, action: 'adjust_camera_zoom', - direction: zoomOut ? 'out' : 'in', + direction, amount: amt, - orbitRadiusAdjusted: true, + [held.kind === 'tilt' ? 'tiltRangeAdjusted' : 'orbitRadiusAdjusted']: + true, + // A tilt to an angle names the angle the new range now ends it at. + ...(held.pitchDeg !== undefined && { + pitchDeg: held.pitchDeg, + note: held.note, + }), }; } } @@ -1125,34 +1167,104 @@ export function createGevActionRunner({ runOptions, ); } - return flyToRequestedLocation(viewer, args, { + // Chained fly-tos: a grab during an earlier one still flying counts. + // The flight replaced is the last one that moved, past lookups that + // found nothing. + const prior = voiceFlightTarget(voiceFlight); + const flight = trackVoiceFlight( + prior && !prior.done ? prior.inputCount : undefined, + prior, + ); + // Navigation by anything but a voice fly-to since this one began. + flight.handoffs = otherHandoffs(); + voiceFlight = flight; + const reply = voiceReplyOf(runOptions); + if (reply !== null && replyFlights?.reply !== reply) + replyFlights = { reply, moved: false, foundNothing: false }; + const replyRecord = reply !== null ? replyFlights : null; + for (const notify of [...voiceFlightWatchers]) notify(); + // A motion already running (one the model sent before this fly-to) is + // stopped when the flight takes navigation; the result says so. + let stoppedMotion = null; + const noteStop = + (hook) => + (...hookArgs) => { + const before = getActiveCameraMotion(); + const handoffsBefore = cameraHandoffs; + try { + return hook(...hookArgs); + } finally { + // A voice fly-to's own stamps are not another navigation, even + // for the flight it replaces (a later lookup may find nothing). + voiceFlightHandoffs += cameraHandoffs - handoffsBefore; + if ( + before && + !stoppedMotion && + getActiveCameraMotion()?.motionId !== before.motionId + ) + stoppedMotion = before.kind; + } + }; + const flown = flyToRequestedLocation(viewer, args, { + onFlightEnd: flight.settle, onProgress: reportProgress, placeSearch, searchNavigation, signal: runOptions.signal, runImmediate: typeof styleManager?.runImmediateLocationNavigation === 'function' - ? (navigate) => - styleManager.runImmediateLocationNavigation(navigate) + ? noteStop((navigate) => + styleManager.runImmediateLocationNavigation(navigate), + ) : null, beginDeferred: typeof styleManager?.beginDeferredLocationNavigation === 'function' - ? () => styleManager.beginDeferredLocationNavigation() + ? noteStop(() => styleManager.beginDeferredLocationNavigation()) : null, reassertDeferred: typeof styleManager?.reassertDeferredLocationNavigation === 'function' - ? (generation) => - styleManager.reassertDeferredLocationNavigation(generation) + ? noteStop((generation) => + styleManager.reassertDeferredLocationNavigation(generation), + ) : null, - onStart: () => { + onStart: noteStop(() => { if (typeof styleManager?.beginLocationNavigation === 'function') { styleManager.beginLocationNavigation(); return; } interruptCameraMotion('nav:fly_to_location'); if (viewer.trackedEntity) stopAllTracking(viewer, dataManager); - }, + }), }); + // No flight started: nothing to wait for. A lookup that was not found, + // threw, or was cancelled once a newer voice fly-to had begun (which + // takes navigation from it) moved nothing, so a queued motion looks + // past it to the flight it replaced. One cancelled while still the + // latest voice flight lost navigation to something else (a search, a + // share link, Cockpit): the motion then refuses. A flight its own hooks + // already settled (cancelled mid-air) keeps its status. + const notStarted = () => { + if (!flight.done) flight.movedNothing = true; + flight.settle('failed'); + }; + flown.then((result) => { + if (result?.ok === true) return; + if (result?.cancelled && voiceFlight === flight) + flight.settle('failed'); + else notStarted(); + }, notStarted); + // After the outcome above, which marks a lookup that moved nothing. + const noteReply = (result) => { + if (!replyRecord) return; + if (result?.ok === true) replyRecord.moved = true; + else if (flight.movedNothing) replyRecord.foundNothing = true; + }; + flown.then(noteReply, () => noteReply(null)); + return flown.then((result) => + stoppedMotion && result && typeof result === 'object' + ? { ...result, stoppedMotion } + : result, + ); } if (name === 'adjust_camera_zoom') { @@ -1254,6 +1366,85 @@ export function createGevActionRunner({ } if (name === 'move_camera') { + const motion = String(args.motion || '').toLowerCase(); + // A lookup that already found nothing does not hide the last flight + // that moved: running now would cancel that flight mid-air. + const pending = voiceFlightTarget(voiceFlight); + const reply = voiceReplyOf(runOptions); + if ( + VOICE_FLIGHT_WAITING_MOTIONS.has(motion) && + !(pending && !pending.done) && + reply !== null && + replyFlights?.reply === reply && + replyFlights.foundNothing && + !replyFlights.moved + ) + return { + ok: false, + action: 'move_camera', + motion, + error: `Not run: the fly-to in this reply found nothing, so the ${motion} was not run at the old view.`, + }; + if ( + VOICE_FLIGHT_WAITING_MOTIONS.has(motion) && + pending && + !pending.done + ) { + // Bad arguments are refused now, not after waiting on the flight. + const argsError = moveCameraArgsError(args); + if (argsError) return argsError; + const deadline = Date.now() + VOICE_FLIGHT_WAIT_MS; + const start = pending; + // Act on where the camera lands. Whatever a flight reports, a later + // fly-to replaces it, and the wait moves to that one as soon as it + // begins (a stalled geocode must not hold the motion past a newer + // flight that landed). A geocode that loses navigation reports + // failed, and a flight left flying while the next place is looked up + // can still land; running there would take navigation and cancel the + // newer one. A later lookup that found nothing moved nothing, so the + // motion looks past it to the last flight that did move the camera. + let flight = start; + let status; + for (;;) { + status = await waitForVoiceFlight( + flight, + runOptions.signal, + deadline, + voiceFlightWatchers, + ); + if (status === 'aborted' || status === 'timeout' || !current()) break; + const target = voiceFlightTarget(voiceFlight, start); + if (!target || (target === flight && status !== 'newer')) break; + flight = target; + } + if (!current()) + return { ok: false, action: 'move_camera', cancelled: true }; + if (status !== 'arrived' || flight.movedNothing) + return { + ok: false, + action: 'move_camera', + motion, + error: `Not run: the flight this ${motion} was waiting for did not arrive.`, + }; + // The flight lands even when the user grabs the globe mid-flight; + // the queued motion must not take the camera back from them. + if (manualCameraInputCount() !== flight.inputCount) + return { + ok: false, + action: 'move_camera', + motion, + error: `Not run: the user moved the camera during the flight, so the ${motion} was dropped.`, + }; + // A search, share link or import that took navigation during the + // flight leaves it flying; running now would cancel that navigation. + if (otherHandoffs() !== flight.handoffs) + return { + ok: false, + action: 'move_camera', + motion, + error: `Not run: another navigation took the camera during the flight, so the ${motion} was dropped.`, + }; + } return moveCamera(args, (navigate, releaseOptions) => runManagedVoiceNavigation( styleManager, @@ -1569,6 +1760,7 @@ export function createGevActionRunner({ * forget the analyst's "last answer". Area handles stay (page lifetime). */ runVoiceAction.resetConversation = () => { + replyFlights = null; conversationAbort.abort(); conversationAbort = new AbortController(); conversation++; @@ -1582,6 +1774,7 @@ export function createGevActionRunner({ if (disposed) return; runVoiceAction.resetConversation(); disposed = true; + unsubscribeCameraHandoff?.(); voiceAreas.dispose(); }; return runVoiceAction; @@ -2854,6 +3047,116 @@ async function frameOverhead(viewer, dataManager, styleManager, args = {}) { ); } +let voiceFlightCount = 0; + +/** + * The model reply a voice call belongs to: Gemini's turn (`reply`), or the + * OpenAI response its sibling calls share (`authorityGroup`). Null when the + * caller names none. + */ +function voiceReplyOf(options) { + return options?.reply ?? options?.authorityGroup ?? null; +} + +/** + * A voice flight's end: 'arrived', 'cancelled' or 'failed', settled once. + * `replaced` is the last flight that moved when this one began (past lookups + * that found nothing), landed or not. It is kept only while this one may turn + * out to have moved nothing (a lookup that found no place), so a queued + * motion can look past it to that flight and its outcome. + */ +function trackVoiceFlight( + inputCount = manualCameraInputCount(), + replaced = null, +) { + const flight = { + done: false, + seq: (voiceFlightCount += 1), + replaced, + movedNothing: false, + // Manual input after this means the user took the camera mid-flight. + inputCount, + status: null, + // Waiters to tell when this flight ends. + watchers: new Set(), + settle: (status) => { + if (flight.done) return; + flight.done = true; + flight.status = status; + // Kept only past a lookup that found nothing, and then straight to the + // flight that moved, so a run of such lookups holds no chain. + flight.replaced = flight.movedNothing + ? voiceFlightTarget(flight.replaced) + : null; + for (const notify of [...flight.watchers]) notify(status); + flight.watchers.clear(); + }, + }; + return flight; +} + +/** + * The flight a motion queued since `start` acts on: the latest one that + * actually started. A lookup that found nothing moved nothing, so look past + * it to the flight it replaced, but never to one older than `start` (a flight + * from an earlier reply). Null when every later lookup found nothing. With + * no `start`, the latest flight that started, back through lookups that + * found nothing. + */ +function voiceFlightTarget(latest, start = null) { + let flight = latest; + while (flight?.movedNothing) flight = flight.replaced; + return flight && (!start || flight.seq >= start.seq) ? flight : null; +} + +/** + * Longest a motion waits for its flight (geocode plus flight, which is about + * 2 s) before giving up. The wait also holds the tool output, and on Gemini + * the later tools in the turn. + */ +const VOICE_FLIGHT_WAIT_MS = 15_000; + +/** + * Motions that act on the place the camera ends up at, so one sent beside a + * fly-to waits for it. Stop is not one, and rotate_globe turns the + * whole Earth from Globe View rather than at a place. + */ +const VOICE_FLIGHT_WAITING_MOTIONS = new Set([ + 'tilt', + 'orbit', + 'pan', + 'rotate', +]); + +/** + * Wait for a voice flight to end, or for a newer one to begin ('newer'): + * 'timeout' or 'aborted' if neither happens. A flight that has ended answers + * at once. Every listener is removed when the wait ends, so a flight that + * never settles holds nothing. + */ +function waitForVoiceFlight(flight, signal, deadline, newerWatchers) { + if (signal?.aborted) return Promise.resolve('aborted'); + if (flight.done) return Promise.resolve(flight.status); + return new Promise((resolve) => { + const finish = (status) => { + clearTimeout(timer); + signal?.removeEventListener?.('abort', onAbort); + flight.watchers.delete(finish); + newerWatchers?.delete(onNewer); + resolve(status); + }; + const onAbort = () => finish('aborted'); + const onNewer = () => finish('newer'); + const timer = setTimeout( + () => finish('timeout'), + Math.max(0, deadline - Date.now()), + ); + signal?.addEventListener?.('abort', onAbort, { once: true }); + flight.watchers.add(finish); + newerWatchers?.add(onNewer); + }); +} + /** Run one validated voice camera mutation through the UI-owned authority seam. */ function runManagedVoiceNavigation( styleManager, @@ -3417,6 +3720,7 @@ async function flyToRequestedLocation( runImmediate = null, beginDeferred = null, reassertDeferred = null, + onFlightEnd = null, } = {}, ) { const requestedRangeM = Number(args.rangeM); @@ -3440,12 +3744,19 @@ async function flyToRequestedLocation( settleArrival = resolve; }) : null; - const arrivalHooks = arrival - ? { - onComplete: () => settleArrival?.('arrived'), - onCancel: () => settleArrival?.('cancelled'), - } - : {}; + const arrivalHooks = + arrival || onFlightEnd + ? { + onComplete: () => { + settleArrival?.('arrived'); + onFlightEnd?.('arrived'); + }, + onCancel: () => { + settleArrival?.('cancelled'); + onFlightEnd?.('cancelled'); + }, + } + : {}; const afterArrival = async (result, label) => { if (!arrival || result?.ok !== true) return result; onProgress('fly', label); diff --git a/src/voice/gevActions.test.mjs b/src/voice/gevActions.test.mjs index 44073d60b..f4be07148 100644 --- a/src/voice/gevActions.test.mjs +++ b/src/voice/gevActions.test.mjs @@ -143,6 +143,14 @@ test('track_entity runner narrates a callsign-less aircraft by its registration' function createVoiceNavigationHarness({ cockpitActive = false } = {}) { const order = []; let generation = 0; + // Camera handoffs, published on every navigation stamp as the shell does. + const handoffListeners = new Set(); + const stamp = (noun) => { + generation += 1; + order.push(`stamp:${noun}`); + for (const listener of [...handoffListeners]) listener({ generation }); + return generation; + }; const position = Cesium.Cartesian3.fromDegrees(-97.74, 30.26, 500); const viewer = { trackedEntity: { id: 'prior-aircraft' }, @@ -179,11 +187,7 @@ function createVoiceNavigationHarness({ cockpitActive = false } = {}) { return runExplicitNavigation({ cockpitActive, noun, - stamp: () => { - generation += 1; - order.push(`stamp:${noun}`); - return generation; - }, + stamp: () => stamp(noun), release: () => { order.push('release'); viewer.trackedEntity = undefined; @@ -195,12 +199,21 @@ function createVoiceNavigationHarness({ cockpitActive = false } = {}) { }); }, getDetectionState: () => ({ detectionMode: 'DENSE' }), + subscribeCameraHandoff(listener) { + handoffListeners.add(listener); + return () => handoffListeners.delete(listener); + }, + // A voice lookup stamps when it begins and leaves any flight flying. + beginDeferredLocationNavigation: () => stamp('location-lookup'), + reassertDeferredLocationNavigation: (stamped) => stamped === generation, }; return { order, viewer, styleManager, currentGeneration: () => generation, + // A location-panel search: a deferred stamp that leaves a flight flying. + panelSearch: () => stamp('panel-search'), }; } @@ -276,6 +289,841 @@ test('dependent voice navigation waits for the destination viewport to arrive', assert.equal(result.arrived, true); }); +// "fly to London and tilt the camera" arrives as two sibling calls. +// Run at once, the tilt's navigation cancelled the flight; with a geocoded +// place, the late flight replaced the tilt. +function tiltAfterFlightHarness() { + globalThis.window = globalThis.window || { clearTimeout, setTimeout, requestIdleCallback: null }; + const harness = createVoiceNavigationHarness(); + const { viewer, styleManager } = harness; + const flights = []; + viewer.camera.flyTo = (options) => { + harness.order.push('fly'); + flights.push(options); + }; + styleManager.runImmediateLocationNavigation = (navigate) => ( + styleManager.runImmediateNavigation('location', navigate) + ); + return { ...harness, flights }; +} + +const settledYet = (promise) => { + let settled = false; + promise.then(() => { settled = true; }); + return () => settled; +}; + +test('a tilt in the same reply as a fly-to waits for the flight to land', async () => { + const { viewer, styleManager, order, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + const fly = await runner('fly_to_location', { locationId: 'austin' }); + assert.equal(fly.ok, true); + const tilt = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + const tiltDone = settledYet(tilt); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(tiltDone(), false, 'the tilt must wait while the camera flies'); + assert.equal( + order.slice(order.indexOf('fly')).includes('cancel'), + false, + 'the tilt must not cancel the flight', + ); + flights[0].complete(); + const result = await tilt; + assert.equal(result.ok, true); + assert.equal(result.motion, 'tilt'); +}); + +test('a tilt waits through a geocoded fly-to search, then for its flight', async () => { + const { viewer, styleManager } = tiltAfterFlightHarness(); + let finishSearch; + let landFlight; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + finishSearch = () => { + landFlight = options.onComplete; + resolve({ label: query, navigationMode: 'city-overview' }); + }; + }), + }); + const fly = runner('fly_to_location', { query: 'Somewhere Far' }); + const tilt = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + const tiltDone = settledYet(tilt); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(tiltDone(), false, 'the tilt must wait for the place lookup'); + finishSearch(); + assert.equal((await fly).ok, true); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(tiltDone(), false, 'the tilt must wait for the flight too'); + landFlight(); + assert.equal((await tilt).ok, true); +}); + +test('a tilt waiting on a flight that never lands is not run', async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + await runner('fly_to_location', { locationId: 'austin' }); + const tilt = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + flights[0].cancel(); + const result = await tilt; + assert.equal(result.ok, false); + assert.match(result.error, /did not arrive/); +}); + +test('a tilt beside a fly-to that found nothing is not run', async () => { + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: async () => null, + }); + const fly = runner('fly_to_location', { query: 'Nowhere' }); + const tilt = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + assert.equal((await fly).ok, false); + // Nothing flew: the tilt answers at once and honestly, never at the old view. + const result = await tilt; + assert.equal(result.ok, false); + assert.match(result.error, /did not arrive/); + // A later, separate tilt is not held by the failed lookup. + assert.equal((await runner('move_camera', { motion: 'tilt', pitchDeg: -45 })).ok, true); +}); + +test('a tilt with no voice flight pending runs at once', async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + await runner('fly_to_location', { locationId: 'austin' }); + flights[0].complete(); + const result = await runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + assert.equal(result.ok, true); +}); + +// Orbit, pan and rotate also act at the place the camera ends up, so "fly to +// Paris and orbit it" holds the orbit the same way. +for (const motion of [ + { motion: 'orbit' }, + { motion: 'pan', direction: 'left' }, + { motion: 'rotate', direction: 'right' }, +]) { + test(`${motion.motion} in the same reply as a fly-to waits for the flight to land`, async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, order, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', motion); + const movedDone = settledYet(moved); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, `the ${motion.motion} must wait while the camera flies`); + assert.equal( + order.slice(order.indexOf('fly')).includes('cancel'), + false, + `the ${motion.motion} must not cancel the flight`, + ); + flights[0].complete(); + const result = await moved; + assert.equal(result.ok, true); + assert.equal(result.motion, motion.motion); + }); + + test(`${motion.motion} waits through a geocoded fly-to search, then for its flight`, async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager } = tiltAfterFlightHarness(); + let finishSearch; + let landFlight; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + finishSearch = () => { + landFlight = options.onComplete; + resolve({ label: query, navigationMode: 'city-overview' }); + }; + }), + }); + const fly = runner('fly_to_location', { query: 'Somewhere Far' }); + const moved = runner('move_camera', motion); + const movedDone = settledYet(moved); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, `the ${motion.motion} must wait for the place lookup`); + assert.equal(getActiveCameraMotion(), null, 'nothing moves before the flight'); + finishSearch(); + assert.equal((await fly).ok, true); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, `the ${motion.motion} must wait for the flight too`); + landFlight(); + assert.equal((await moved).ok, true); + assert.equal(getActiveCameraMotion()?.kind, motion.motion); + }); + + test(`${motion.motion} waiting on a flight that never lands is not run`, async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + await runner('fly_to_location', { locationId: 'austin' }); + const moved = runner('move_camera', motion); + flights[0].cancel(); + const result = await moved; + assert.equal(result.ok, false); + assert.equal(result.motion, motion.motion); + assert.match(result.error, new RegExp(`this ${motion.motion} was waiting for did not arrive`)); + assert.equal(getActiveCameraMotion(), null); + }); +} + +// Cesium lands a flight even when the user grabs the globe mid-flight, so the +// motion queued behind it must not then take the camera back. +for (const motion of [ + { motion: 'orbit' }, + { motion: 'tilt', pitchDeg: -45 }, +]) { + test(`${motion.motion} queued behind a fly-to is dropped when the user drags mid-flight`, async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const canvasListeners = new Map(); + viewer.scene.canvas.addEventListener = (type, listener) => { + canvasListeners.set(type, listener); + }; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', motion); + canvasListeners.get('pointerdown')(); + flights[0].complete(); + const result = await moved; + assert.equal(result.ok, false); + assert.equal(result.motion, motion.motion); + assert.match(result.error, /moved the camera during the flight/); + assert.equal(getActiveCameraMotion(), null); + }); +} + +test('a drag during the first of two chained fly-tos still drops the queued motion', async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const canvasListeners = new Map(); + viewer.scene.canvas.addEventListener = (type, listener) => { + canvasListeners.set(type, listener); + }; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + canvasListeners.get('touchstart')(); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }); + flights.at(-1).complete(); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /moved the camera during the flight/); +}); + +test('input before a fly-to does not drop the motion queued behind it', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const canvasListeners = new Map(); + viewer.scene.canvas.addEventListener = (type, listener) => { + canvasListeners.set(type, listener); + }; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + canvasListeners.get('wheel')(); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }); + flights[0].complete(); + assert.equal((await moved).ok, true); +}); + +test('a motion follows a later fly-to in the same reply that cancels its flight', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }); + const movedDone = settledYet(moved); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + // The second flight supersedes the first, as Cesium's flyTo does. + flights[0].cancel(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, 'the orbit must wait for the second flight'); + flights[1].complete(); + const result = await moved; + assert.equal(result.ok, true); + assert.equal(result.motion, 'orbit'); +}); + +test('a motion follows a later fly-to whose start cancels its geocode', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + const flyA = runner('fly_to_location', { query: 'Place A' }); + const moved = runner('move_camera', { motion: 'orbit' }); + const movedDone = settledYet(moved); + const flyB = runner('fly_to_location', { query: 'Place B' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + // B takes navigation authority, so A's lookup comes back cancelled. + searches[0].resolve({ cancelled: true }); + assert.equal((await flyA).cancelled, true); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, 'the orbit must follow B, not drop'); + searches[1].resolve({ label: 'Place B', navigationMode: 'city-overview' }); + assert.equal((await flyB).ok, true); + searches[1].options.onComplete(); + const result = await moved; + assert.equal(result.ok, true); + assert.equal(result.motion, 'orbit'); +}); + +test('a motion does not run at a flight that lands while the next place is looked up', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, order, flights } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + const movedDone = settledYet(moved); + const flyB = runner('fly_to_location', { query: 'Place B' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + // Deferred navigation leaves A flying while B is looked up, and A lands. + const before = order.length; + flights[0].complete(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, 'the tilt must wait for B'); + assert.equal( + order.slice(before).includes('cancel'), + false, + 'the tilt must not take navigation and cancel B', + ); + searches[0].resolve({ label: 'Place B', navigationMode: 'city-overview' }); + assert.equal((await flyB).ok, true); + searches[0].options.onComplete(); + const result = await moved; + assert.equal(result.ok, true); + assert.equal(result.motion, 'tilt'); +}); + +// A later fly-to that finds nothing moves nothing: the motion still runs where +// the earlier flight lands, whether the lookup fails before or after that. +for (const order of ['lands first', 'not found first']) { + test(`a motion runs at its flight when a later fly-to finds nothing (${order})`, async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const harness = tiltAfterFlightHarness(); + const { viewer, styleManager, flights } = harness; + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + const flyB = runner('fly_to_location', { query: 'Nowhere' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + if (order === 'lands first') flights[0].complete(); + searches[0].resolve(null); + assert.equal((await flyB).ok, false); + if (order !== 'lands first') flights[0].complete(); + const result = await moved; + assert.equal(result.ok, true, result.error); + assert.equal(result.motion, 'tilt'); + }); +} + +test('a motion beside a fly-to that finds nothing never acts at an earlier reply\'s flight', async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + // Reply 1: "fly to Austin", still flying. + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + // Reply 2: "fly to Nowhere and tilt". + const flyB = runner('fly_to_location', { query: 'Nowhere' }); + const moved = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + await new Promise((resolve) => setTimeout(resolve, 0)); + searches[0].resolve(null); + assert.equal((await flyB).ok, false); + flights[0].complete(); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /did not arrive/); +}); + +// Two later lookups that both find nothing, in either order, leave the motion +// acting where the first flight landed. +for (const first of [0, 1]) { + test(`a motion looks past two lookups that found nothing (lookup ${first + 1} fails first)`, async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }); + const flyB = runner('fly_to_location', { query: 'Nowhere B' }); + const flyC = runner('fly_to_location', { query: 'Nowhere C' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + searches[first].resolve(null); + searches[1 - first].resolve(null); + assert.equal((await flyB).ok, false); + assert.equal((await flyC).ok, false); + flights[0].complete(); + const result = await moved; + assert.equal(result.ok, true, result.error); + }); +} + +test('a lookup cancelled by a newer one that finds nothing does not hide the flight that landed', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + const flyB = runner('fly_to_location', { query: 'Place B' }); + const flyC = runner('fly_to_location', { query: 'Nowhere C' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + flights[0].complete(); + // C's start takes navigation from B's lookup; C then finds nothing. + searches[0].resolve({ cancelled: true }); + searches[1].resolve(null); + assert.equal((await flyB).cancelled, true); + assert.equal((await flyC).ok, false); + const result = await moved; + assert.equal(result.ok, true, result.error); +}); + +test('a motion sent after a fly-to that found nothing still waits for the flight before it', async (t) => { + // Serial dispatch: "fly to Austin, fly to Nowhere, tilt", one call at a time. + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, order, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: async () => null, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + assert.equal((await runner('fly_to_location', { query: 'Nowhere' })).ok, false); + const before = order.length; + const moved = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + const movedDone = settledYet(moved); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, 'the tilt must wait for the Austin flight'); + assert.equal(order.slice(before).includes('cancel'), false); + flights[0].complete(); + assert.equal((await moved).ok, true); +}); + +test('two lookups that found nothing in a row still leave the motion waiting for the flight', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, order, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: async () => null, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + assert.equal((await runner('fly_to_location', { query: 'Nowhere' })).ok, false); + assert.equal((await runner('fly_to_location', { query: 'Nowhere 2' })).ok, false); + const before = order.length; + const moved = runner('move_camera', { motion: 'orbit' }); + const movedDone = settledYet(moved); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), false, 'the orbit must wait for the Austin flight'); + assert.equal(order.slice(before).includes('cancel'), false); + flights[0].complete(); + assert.equal((await moved).ok, true); +}); + +// A flight that already settled is still what a later failed lookup replaced. +test('a motion acts at a landed flight that a later failed lookup replaced', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + const flyA = runner('fly_to_location', { query: 'Slow A' }); + const moved = runner('move_camera', { motion: 'orbit' }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + flights[0].complete(); + const flyC = runner('fly_to_location', { query: 'Nowhere C' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + // B's start made A's lookup stale; C then finds nothing. + searches[0].resolve({ cancelled: true }); + searches[1].resolve(null); + assert.equal((await flyA).cancelled, true); + assert.equal((await flyC).ok, false); + const result = await moved; + assert.equal(result.ok, true, result.error); +}); + +test('a motion refuses when the flight a later failed lookup replaced was cancelled', async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }); + const flyA = runner('fly_to_location', { query: 'Slow A' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + // The motion sees Austin land, then follows A's lookup. + flights[0].complete(); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + flights[1].cancel(); + const flyC = runner('fly_to_location', { query: 'Nowhere C' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + searches[0].resolve({ cancelled: true }); + searches[1].resolve(null); + assert.equal((await flyA).cancelled, true); + assert.equal((await flyC).ok, false); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /did not arrive/); +}); + +test('a stale motion does not follow a newer reply\'s fly-to', async () => { + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + let live = true; + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }, { isCurrent: () => live }); + // A newer reply takes over and flies elsewhere; this reply is stale. + live = false; + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + flights[0].cancel(); + // The newer flight never lands here: following it would hold this answer. + const result = await Promise.race([ + moved, + new Promise((resolve) => setTimeout(() => resolve('still waiting'), 50)), + ]); + assert.notEqual(result, 'still waiting'); + assert.equal(result.ok, false); + assert.equal(result.cancelled, true); +}); + +test('a motion gives up on a flight that has not landed in 15 s', async (t) => { + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + t.mock.timers.enable({ apis: ['setTimeout', 'Date'] }); + const moved = runner('move_camera', { motion: 'orbit' }); + const movedDone = settledYet(moved); + await Promise.resolve(); + t.mock.timers.tick(14_999); + await Promise.resolve(); + assert.equal(movedDone(), false); + t.mock.timers.tick(1); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /did not arrive/); +}); + +test('a motion follows a newer fly-to that lands while its own lookup stalls', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + // Place A's lookup never answers. + searchNavigation: () => new Promise(() => {}), + }); + t.mock.timers.enable({ apis: ['setTimeout', 'Date'] }); + runner('fly_to_location', { query: 'Place A' }); + const moved = runner('move_camera', { motion: 'orbit' }); + const movedDone = settledYet(moved); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + t.mock.timers.tick(2_000); + await Promise.resolve(); + assert.equal(movedDone(), false, 'the orbit must wait for the newer flight to land'); + flights[0].complete(); + for (let i = 0; i < 5 && !movedDone(); i += 1) await Promise.resolve(); + assert.equal(movedDone(), true, 'the orbit must not wait out A'); + const result = await moved; + assert.equal(result.ok, true); + assert.equal(result.motion, 'orbit'); +}); + +// "Fly to Paris and orbit it", then a location-panel search while the camera +// flies: the search stamps a deferred navigation without stopping the flight. +// Run where the flight landed, the orbit took navigation and the search's +// reassert failed, so the search was silently dropped. +// Gemini runs one reply's calls one after another: by the time "pan left" +// runs, "fly to Xyzzyville" has already found nothing, so there is no flight +// to wait on, and the pan moved the old view with ok: true. +test('a motion run after its reply\'s fly-to found nothing is not run', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: async () => null, + }); + const reply = { turn: 1 }; + const fly = await runner('fly_to_location', { query: 'Xyzzyville' }, { reply }); + assert.equal(fly.ok, false); + const pan = await runner( + 'move_camera', + { motion: 'pan', direction: 'left' }, + { reply }, + ); + assert.equal(pan.ok, false); + assert.match(pan.error, /Not run: the fly-to in this reply found nothing/); + assert.equal(getActiveCameraMotion(), null); + // The same holds for OpenAI's reply identity. + const group = 'response-1'; + await runner('fly_to_location', { query: 'Xyzzyville' }, { authorityGroup: group }); + const tilt = await runner( + 'move_camera', + { motion: 'tilt', pitchDeg: -45 }, + { authorityGroup: group }, + ); + assert.equal(tilt.ok, false); + assert.match(tilt.error, /found nothing/); +}); + +test('a motion in a later reply runs after a fly-to that found nothing', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: async () => null, + }); + await runner('fly_to_location', { query: 'Xyzzyville' }, { reply: { turn: 1 } }); + const pan = await runner( + 'move_camera', + { motion: 'pan', direction: 'left' }, + { reply: { turn: 2 } }, + ); + assert.equal(pan.ok, true, pan.error); +}); + +test('a motion runs at a flight of its reply that landed after another lookup found nothing', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: async () => null, + }); + const reply = { turn: 1 }; + await runner('fly_to_location', { query: 'Xyzzyville' }, { reply }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' }, { reply })).ok, true); + flights[0].complete(); + const pan = await runner( + 'move_camera', + { motion: 'pan', direction: 'left' }, + { reply }, + ); + assert.equal(pan.ok, true, pan.error); +}); + +test('a motion queued behind a fly-to yields to a search started during the flight', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights, panelSearch, currentGeneration } = + tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'orbit' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + const search = panelSearch(); + flights[0].complete(); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /Not run: another navigation took the camera/); + assert.equal(getActiveCameraMotion(), null); + assert.equal(currentGeneration(), search, 'the search still owns navigation'); +}); + +test('a motion behind a geocoded fly-to yields to a search started while it flies', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, panelSearch } = tiltAfterFlightHarness(); + const searches = []; + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + searchNavigation: (_viewer, query, options) => new Promise((resolve) => { + searches.push({ query, options, resolve }); + }), + }); + const fly = runner('fly_to_location', { query: 'Place A' }); + const moved = runner('move_camera', { motion: 'orbit' }); + await new Promise((resolve) => setTimeout(resolve, 0)); + searches[0].resolve({ label: 'Place A', navigationMode: 'city-overview' }); + assert.equal((await fly).ok, true); + panelSearch(); + searches[0].options.onComplete(); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /another navigation took the camera/); + assert.equal(getActiveCameraMotion(), null); +}); + +test('a search before the fly-to does not drop the motion queued behind it', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager, flights, panelSearch } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + panelSearch(); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'tilt', pitchDeg: -45 }); + await new Promise((resolve) => setTimeout(resolve, 0)); + flights[0].complete(); + const result = await moved; + assert.equal(result.ok, true); + assert.equal(result.motion, 'tilt'); +}); + +test('a fly-to that stops a running motion says so', async (t) => { + t.after(() => interruptCameraMotion('test-cleanup')); + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + // The model sent the orbit first, so it ran at once. + const orbit = await runner('move_camera', { motion: 'orbit', mode: 'continuous' }); + assert.equal(orbit.ok, true); + const fly = await runner('fly_to_location', { locationId: 'austin' }); + assert.equal(fly.ok, true); + assert.equal(fly.stoppedMotion, 'orbit'); + const again = await runner('fly_to_location', { locationId: 'austin' }); + assert.equal('stoppedMotion' in again, false); +}); + +test('bad move_camera arguments are refused without waiting for a flight', async () => { + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + assert.equal((await runner('fly_to_location', { locationId: 'austin' })).ok, true); + const moved = runner('move_camera', { motion: 'pan' }); + const movedDone = settledYet(moved); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(movedDone(), true, 'the refusal must not wait on the flight'); + const result = await moved; + assert.equal(result.ok, false); + assert.match(result.error, /needs a direction/); +}); + +test('stop does not wait for a voice flight', async () => { + const { viewer, styleManager } = tiltAfterFlightHarness(); + const runner = createGevActionRunner({ + viewer, + styleManager, + dataManager: { layers: new Map(), getAll: () => [] }, + }); + await runner('fly_to_location', { locationId: 'austin' }); + const result = await runner('move_camera', { motion: 'stop' }); + assert.equal(result.ok, true); + assert.equal(result.motion, 'stop'); +}); + test('nearest-aircraft voice action serializes layer enable, arrival, refresh, airborne query, and selection', async () => { globalThis.window = globalThis.window || { clearTimeout, diff --git a/src/voice/pointAndAsk.test.mjs b/src/voice/pointAndAsk.test.mjs index 00c062d9f..5d6d0fdd1 100644 --- a/src/voice/pointAndAsk.test.mjs +++ b/src/voice/pointAndAsk.test.mjs @@ -309,7 +309,10 @@ test('markup: the chip sits under the user caption with a screen-reader separato assert.equal(pointer.hidden, true); const separator = pointer.querySelector('.gev-voice-card-sr'); assert.equal(separator.textContent, ': '); - assert.equal(separator.previousElementSibling.id, 'gev-voice-card-pointer-kind'); + assert.equal( + separator.previousElementSibling.id, + 'gev-voice-card-pointer-kind', + ); }); test('deixis prompt lint: one short rule, and every sentinel it names exists in the schema', () => { @@ -347,15 +350,18 @@ test('deixis prompt lint: one short rule, and every sentinel it names exists in // the analyst area scopes) raised the ceiling to 55,500; tightening the // AREAS rule and the area tool descriptions left it at 55,103 as measured // below (every tool, osm_query included); main's Sea Marks and Ski Pistes - // layers bring it to 55,210, ~290 under. A new tool or rule has to fit - // under this ceiling. + // layers bring it to 55,210, and the tilt pitchDeg argument (with its -45 + // default) and main's cockpit entry rule to 55,339. integrationContract's + // Overpass prompt is the tightest, at 55,499 (1 under). A new tool or rule + // has to fit under this ceiling. const size = instructions.length + JSON.stringify(GEV_REALTIME_TOOLS).length; assert.ok(size <= 55_500, `prompt size ${size}`); }); - test('after-tool speech allows returned items to answer requested lists and rankings', () => { - const rule = realtimeInstructions().split('\n').find(line => line.startsWith('AFTER TOOLS:')); + const rule = realtimeInstructions() + .split('\n') + .find((line) => line.startsWith('AFTER TOOLS:')); assert.match(rule, /requested analyst list or ranking/); assert.match(rule, /returned items.*returned values/); assert.match(rule, /keep their order/);