Merge pull request #835 from kvnloo/feat/viewport-context-decision-upstream

feat(voice): reason-code viewport context escalation

changelog.d/835-viewport-escalation-reasons.md

@@ -0,0 +1,7 @@
+- The voice debug log now says why OpenAI Realtime did or did not get a
+  viewport screenshot or pointer crop. When an entity-context answer reaches
+  the image step, the log records a reason-coded decision and, when an image
+  was due, one outcome under the same id: sent, or why it was dropped (a stale
+  pointer, a camera move, a newer turn). What is captured, sent and kept is
+  unchanged, and no image data or user text is logged (Kevin Rajan, @kvnloo,
+  Manjunath, @manjunath22466, #835).

docs/CURRENT-STATE.md

@@ -6309,6 +6309,7 @@ and unreachable upstream (502/504) separately from road geometry.
 - **Reliability**: tool-call dedupe (2.5s window across call/item/args keys); response-create queueing that respects active responses and defers follow-ups when the user starts speaking; per-tool follow-up instructions so the agent confirms only what actually happened (zoom confirms only on `ok=true`).
 - **Aircraft identity honesty:** “What is this aircraft?” reads callsign, operator, registration, type, and route only from the selected contact context. Missing operator, route, or type enrichment is named explicitly rather than silently omitted or inferred from the callsign.
 - **Diagnostics**: every client/server event is posted to `/api/realtime/debug-log` and appended to `.gev-logs/realtime-conversations.jsonl` (gitignored) with secret/image redaction; last 30 errors persist in `localStorage` (`gev-realtime-errors`); `window.__gevVoiceCommands.getDiagnostics()` in the console.
+  - **Visual-context escalation** (OpenAI Realtime): when a response's last tool result is a `get_entity_context` answer, a tool output from that response was sent and the data channel is open, it logs one `viewport_context.decision` with `contextId`, `send` and `reason`, plus `viewScale` and, when `send` is true, `kind` (`viewport` or `crop`). Decision reasons: `structured-identity-sufficient` (structured context made an image unnecessary), `view-scale-not-local`, `structured-identity-missing` (requests a viewport), `pointer-crop-eligible` (a pointer answer with no identity; requests a crop, and the crop checks below decide whether one is sent), `pointer-identity-present` and `pointer-missing`. A send decision is followed by exactly one outcome with the same `contextId` and `kind`: `viewport_context.result` (`sent`, `data-channel-send-failed`), logged as soon as the send returns, or `viewport_context.skipped`. Pointer-crop skips are `pointer-not-fresh` (no snapshot, or a stale one), `pointer-not-ground`, `pointer-off-screen`, `pointer-scale-not-useful` (zoomed out past `local` or `city`), `pointer-already-requested` (the most recently requested snapshot again: already sent, still being captured, or an earlier attempt for it threw), `pointer-expired`, `pointer-pose-unknown` and `pose-changed`; dropped captures are `capture-unavailable`, `session-reset`, `superseded` (a newer request or turn), `channel-lost` and `frame-stale`; a step that throws before an outcome closes the pair with `error`, and the caller then logs `viewport_context.failed`. Direct calls to the crop or image senders log their skips with a null `contextId`. Decision and outcome are separate records, so the debug-log rate limit can drop one half of a pair. Payloads are static codes only, never image data, tool payloads or user text, and other tool results log nothing here. Gemini Live uses a different gate in `geminiSession.js` (no pointer crop, a 1 s throttle), which is not instrumented.
 - **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].total`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
   1. **Contacts ACTIVE** → "near / nearby / how many aircraft" means the **Contacts window** — the panel's numbers, spoken verbatim. Mechanism: `contactsWindow` (`{centeredOn, radiusKm, flights, military, vessels, complete}`), carried by both `analyst_query` and `get_current_view_state`, derived by `contactsWindowFromSnapshot()` from the same snapshot the panel renders so the two cannot drift. Voice filtering and follow-up memory use that immutable snapshot too. A cohort whose feed cannot answer reports `'unknown'`, never a confident zero; a cohort at the 20,000-row retention bound reports a lower bound.
   2. **Contacts OFF** → "nearby" means **in view**; "near \<place\>" means a radius around that place. A radius query with Contacts active and no explicit centre is centred on the **active contact**, not the camera.

scripts/format-scope.json

@@ -679,6 +679,7 @@
   "src/voice/control.js",
   "src/voice/realtimeBackend.js",
   "src/voice/realtimeBackend.test.mjs",
+  "src/voice/realtimeViewportPolicy.test.mjs",
   "src/agent/worldPacket.test.mjs",
   "src/voice/layerSummary.js",
   "src/app/sources.js",

src/voice/gevRealtime.test.mjs

@@ -3708,3 +3708,29 @@ test('voice runs app actions itself and other tools through the catalog', async
   assert.deepEqual(disposals, [['removed']]);
   assert.equal(withToolCatalog(runner, undefined), runner);
 });
+
+test('the controller forwards viewport escalation diagnostics to its debug log', async () => {
+  const records = [];
+  const controller = new GevRealtimeController({
+    ui: createTestVoiceUi(),
+    runner: async () => ({ ok: true }),
+    radioLayer: { subscribePlaybackControls: () => () => {}, setVoiceDucked() {} },
+  });
+  controller.debugLog = (event, detail) => records.push({ event, detail });
+  controller.dc = { readyState: 'open', send() {}, close() {} };
+
+  assert.equal(
+    await controller.sendVisualContextIfUseful({
+      action: 'get_entity_context',
+      scene: { basemap: { viewScale: 'regional' } },
+    }),
+    false,
+  );
+  assert.deepEqual(
+    records.filter((record) => record.event.startsWith('viewport_context.')),
+    [{
+      event: 'viewport_context.decision',
+      detail: { contextId: 1, send: false, reason: 'view-scale-not-local', viewScale: 'regional' },
+    }],
+  );
+});

src/voice/realtimeController.js

@@ -101,6 +101,7 @@ export class GevRealtimeController extends RealtimeFacade {
 
       operations: {
         sendRealtimeEvent: (...args) => this.sendRealtimeEvent(...args),
+        debugLog: (...args) => this.debugLog(...args),
       },
     });
     this._diagnostics = new RealtimeDiagnostics({

src/voice/realtimeViewport.js

@@ -72,6 +72,43 @@ const POINTER_CROP_MAX_AGE_MS = 20000;
 /** View scales where a crop shows buildings and terrain, not a smear. */
 const POINTER_CROP_SCALES = new Set(['local', 'city']);
 
+/**
+ * Reason-coded explanation of the existing visual escalation policy, for an
+ * entity-context answer on an open channel (the caller checks both).
+ * Pointer crops remain a separate path from full-frame screenshots.
+ * These reasons are static diagnostics, never model-generated claims.
+ */
+export function viewportContextDecision(result) {
+  const viewScale = result?.scene?.basemap?.viewScale ?? null;
+  if (result?.scope === 'pointer') {
+    if (result.selected)
+      return { send: false, reason: 'pointer-identity-present', viewScale };
+    if (!result.pointer)
+      return { send: false, reason: 'pointer-missing', viewScale };
+    // The crop path checks ground, freshness and scale itself, and logs why.
+    return {
+      send: true,
+      kind: 'crop',
+      reason: 'pointer-crop-eligible',
+      viewScale,
+    };
+  }
+  if (!shouldSendViewportImage(viewScale))
+    return { send: false, reason: 'view-scale-not-local', viewScale };
+  if (hasStructuredViewIdentity(result))
+    return {
+      send: false,
+      reason: 'structured-identity-sufficient',
+      viewScale,
+    };
+  return {
+    send: true,
+    kind: 'viewport',
+    reason: 'structured-identity-missing',
+    viewScale,
+  };
+}
+
 /**
  * Square source rectangle around a canvas point, clamped inside the canvas,
  * in drawing-buffer pixels, plus where the point lands in the output image.
@@ -299,6 +336,16 @@ export function isBenignViewportDeleteError(payload, pendingDeleteIds = null) {
   return code === 'item_not_found';
 }
 
+/** Pairs one visual-context request's outcome with its decision. */
+function diagnosticsRecord(contextId = null) {
+  return { contextId, logged: false };
+}
+
+function reportOutcome(owner, diag, event, kind, reason) {
+  diag.logged = true;
+  owner.debugLog?.(event, { contextId: diag.contextId, kind, reason });
+}
+
 /** Own the one retained viewport image and bounded pending deletion identities. */
 export class RealtimeViewport {
   constructor({
@@ -319,6 +366,8 @@ export class RealtimeViewport {
     // Bumped by every capture request and every new user turn: only the
     // newest request of the current turn may publish.
     this.captureSeq = 0;
+    // Diagnostics only: pairs each visual-context decision with its outcome.
+    this.contextSeq = 0;
     this.lastPointerCrop = null;
     // What the retained image shows: `{kind:'viewport'|'crop', rect, cameraKey}`.
     // Pixel targeting (screenX/screenY) is read against it.
@@ -330,28 +379,51 @@ export class RealtimeViewport {
     return this.readChannel();
   }
 
+  /**
+   * Escalate an entity-context answer to an image when the gate allows it.
+   * Logs one `viewport_context.decision`, then, for a send decision, exactly
+   * one `.skipped` or `.result` outcome, even when a step throws. Every
+   * event carries the same `contextId` and the image `kind`, so overlapping
+   * requests pair up.
+   */
   async sendVisualContextIfUseful(result) {
+    // Runs after every tool output. Only entity context on an open channel
+    // can escalate, so anything else logs nothing and leaves the shared
+    // debug-log budget alone.
     if (
       result?.action !== 'get_entity_context' ||
       !this.dc ||
       this.dc.readyState !== 'open'
     )
       return false;
-    const viewScale = result.scene?.basemap?.viewScale;
-    // Point-and-ask on bare ground: the crop around the pointer replaces the
-    // whole-frame screenshot, and is pulled only when nothing was named.
-    if (result.scope === 'pointer') {
-      if (result.selected || !result.pointer) return false;
-      return this.sendPointerCrop(this.readPointer(), { viewScale });
+    const contextId = ++this.contextSeq;
+    const decision = viewportContextDecision(result);
+    this.debugLog?.('viewport_context.decision', { contextId, ...decision });
+    if (!decision.send) return false;
+    const diag = diagnosticsRecord(contextId);
+    try {
+      // #837's pointer crop is separate from the full viewport: preserve it.
+      if (decision.kind === 'crop')
+        return await this.sendPointerCrop(this.readPointer(), {
+          viewScale: decision.viewScale,
+          diag,
+        });
+      return await this.sendRetainedImage(
+        () => this.capture(),
+        "Current God's Eye View viewport screenshot. Read any clearly visible street, building, and place labels in the image and combine them with the structured nearbyPlaces, streetLabels, and scene context. Do not invent labels that are not legible.",
+        'client.viewport_context',
+        { kind: 'viewport', diag },
+      );
+    } catch (error) {
+      // The caller logs the error itself; close the pair, then rethrow.
+      if (!diag.logged)
+        this.debugLog?.('viewport_context.skipped', {
+          contextId,
+          kind: decision.kind,
+          reason: 'error',
+        });
+      throw error;
     }
-    if (!shouldSendViewportImage(viewScale)) return false;
-    if (hasStructuredViewIdentity(result)) return false;
-    return this.sendRetainedImage(
-      () => this.capture(),
-      "Current God's Eye View viewport screenshot. Read any clearly visible street, building, and place labels in the image and combine them with the structured nearbyPlaces, streetLabels, and scene context. Do not invent labels that are not legible.",
-      'client.viewport_context',
-      { kind: 'viewport' },
-    );
   }
 
   /** A new user turn: captures still in flight from the last one are dropped. */
@@ -365,27 +437,39 @@ export class RealtimeViewport {
    * @param {object|null} snapshot Fresh pointer snapshot on bare ground.
    * @returns {Promise<boolean>}
    */
-  async sendPointerCrop(snapshot, { viewScale } = {}) {
-    if (!snapshot?.fresh || snapshot.target !== 'ground' || !snapshot.screenPx)
+  async sendPointerCrop(
+    snapshot,
+    { viewScale, diag = diagnosticsRecord() } = {},
+  ) {
+    const skip = (reason) => {
+      reportOutcome(this, diag, 'viewport_context.skipped', 'crop', reason);
       return false;
+    };
+    if (!snapshot?.fresh) return skip('pointer-not-fresh');
+    if (snapshot.target !== 'ground') return skip('pointer-not-ground');
+    if (!snapshot.screenPx) return skip('pointer-off-screen');
     // Too far out, the crop is a smear: every path shares this guard.
-    if (!POINTER_CROP_SCALES.has(viewScale)) return false;
-    if (snapshot === this.lastPointerCrop) return false;
+    if (!POINTER_CROP_SCALES.has(viewScale))
+      return skip('pointer-scale-not-useful');
+    // Set before the capture finishes, so this also covers one in flight.
+    if (snapshot === this.lastPointerCrop)
+      return skip('pointer-already-requested');
     const capturedAt = Number(snapshot.capturedAt) || 0;
-    if (this.now() - capturedAt > POINTER_CROP_MAX_AGE_MS) return false;
+    if (this.now() - capturedAt > POINTER_CROP_MAX_AGE_MS)
+      return skip('pointer-expired');
     // The snapshot's pixels only name its own camera pose.
     const snapshotCameraKey = snapshot.cameraKey;
-    if (!snapshotCameraKey) return false;
+    if (!snapshotCameraKey) return skip('pointer-pose-unknown');
     const samePose = () =>
       snapshotCameraKey === this.readCameraKey() &&
       this.now() - capturedAt <= POINTER_CROP_MAX_AGE_MS;
-    if (!samePose()) return false;
+    if (!samePose()) return skip('pose-changed');
     this.lastPointerCrop = snapshot;
     const sent = await this.sendRetainedImage(
       () => this.capturePointer(snapshot.screenPx),
       "Crop of the map around the user's pointer; the ring marks the exact spot they mean by this or here. Describe only what is clearly visible.",
       'client.pointer_crop',
-      { kind: 'crop', valid: samePose },
+      { kind: 'crop', valid: samePose, diag },
     );
     if (!sent && this.lastPointerCrop === snapshot) this.lastPointerCrop = null;
     return sent;
@@ -393,22 +477,40 @@ export class RealtimeViewport {
 
   /** Capture one image and make it the single retained image in context. */
   async sendRetainedImage(captureImage, text, label, frame = {}) {
-    if (!this.dc || this.dc.readyState !== 'open') return false;
+    const kind = frame.kind || 'viewport';
+    const diag = frame.diag || diagnosticsRecord();
+    const report = (event, reason) =>
+      reportOutcome(this, diag, event, kind, reason);
+    if (!this.dc || this.dc.readyState !== 'open') {
+      report('viewport_context.skipped', 'channel-lost');
+      return false;
+    }
     const generation = this.generation;
     const request = ++this.captureSeq;
     const channel = this.dc;
     const captured = await captureImage();
     const imageUrl =
       typeof captured === 'string' ? captured : captured?.dataUrl || null;
-    if (
-      !imageUrl ||
-      generation !== this.generation ||
-      request !== this.captureSeq ||
-      this.dc !== channel ||
-      channel.readyState !== 'open' ||
-      (typeof frame.valid === 'function' && !frame.valid())
-    )
+    if (!imageUrl) {
+      report('viewport_context.skipped', 'capture-unavailable');
       return false;
+    }
+    // Same guards as before, checked in order so each drop names its cause:
+    // reset() bumps both counters, so a session reset is tested first.
+    const invalidated =
+      generation !== this.generation
+        ? 'session-reset'
+        : request !== this.captureSeq
+          ? 'superseded'
+          : this.dc !== channel || channel.readyState !== 'open'
+            ? 'channel-lost'
+            : typeof frame.valid === 'function' && !frame.valid()
+              ? 'frame-stale'
+              : null;
+    if (invalidated) {
+      report('viewport_context.skipped', invalidated);
+      return false;
+    }
 
     // Keep at most one viewport screenshot in context. Images are the single
     // most expensive item (re-billed every turn they linger), so we proactively
@@ -461,10 +563,16 @@ export class RealtimeViewport {
     // the item we just deleted as null and fall through so the caller still
     // issues queueResponseCreate WITHOUT the image, instead of stranding the turn.
     const sent = this.sendRealtimeEvent(contextEvent, label);
+    // Report before the bookkeeping below, so a later throw cannot turn a
+    // real send into a logged "error".
+    report(
+      'viewport_context.result',
+      sent ? 'sent' : 'data-channel-send-failed',
+    );
     this.lastViewportItemId = sent ? newItemId : null;
     this.retainedFrame = sent
       ? {
-          kind: frame.kind || 'viewport',
+          kind,
           rect: captured?.rect || null,
           cameraKey: this.readCameraKey(),
         }

src/voice/realtimeViewportPolicy.test.mjs

@@ -0,0 +1,615 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+
+import {
+  RealtimeViewport,
+  viewportContextDecision,
+} from './realtimeViewport.js';
+
+const localContext = (overrides = {}) => ({
+  action: 'get_entity_context',
+  scene: { basemap: { viewScale: 'local' } },
+  ...overrides,
+});
+
+test('viewport context reasons preserve the existing escalation gate', () => {
+  assert.deepEqual(
+    viewportContextDecision({
+      action: 'get_entity_context',
+      scene: { basemap: { viewScale: 'regional' } },
+    }),
+    {
+      send: false,
+      reason: 'view-scale-not-local',
+      viewScale: 'regional',
+    },
+  );
+  assert.deepEqual(
+    viewportContextDecision(
+      localContext({
+        selected: {
+          id: 'earthquake:1',
+          layerId: 'earthquakes',
+        },
+      }),
+    ),
+    {
+      send: false,
+      reason: 'structured-identity-sufficient',
+      viewScale: 'local',
+    },
+  );
+  assert.deepEqual(viewportContextDecision(localContext()), {
+    send: true,
+    kind: 'viewport',
+    reason: 'structured-identity-missing',
+    viewScale: 'local',
+  });
+});
+
+test('nearby structured place identity avoids visual escalation too', () => {
+  assert.deepEqual(
+    viewportContextDecision(
+      localContext({
+        scene: {
+          basemap: {
+            viewScale: 'local',
+            nearbyPlaces: [{ name: 'Austin' }],
+          },
+        },
+      }),
+    ),
+    {
+      send: false,
+      reason: 'structured-identity-sufficient',
+      viewScale: 'local',
+    },
+  );
+});
+
+test('pointer ground selects the #837 crop path, not a full viewport', () => {
+  const pointerContext = localContext({
+    scope: 'pointer',
+    pointer: { target: 'ground' },
+    scene: { basemap: { viewScale: 'city' } },
+  });
+  assert.deepEqual(viewportContextDecision(pointerContext), {
+    send: true,
+    kind: 'crop',
+    reason: 'pointer-crop-eligible',
+    viewScale: 'city',
+  });
+  assert.deepEqual(
+    viewportContextDecision({
+      ...pointerContext,
+      scene: { basemap: { viewScale: 'global' } },
+    }),
+    {
+      send: true,
+      kind: 'crop',
+      reason: 'pointer-crop-eligible',
+      viewScale: 'global',
+    },
+  );
+  assert.deepEqual(
+    viewportContextDecision({
+      ...pointerContext,
+      selected: { id: 'building' },
+    }),
+    {
+      send: false,
+      reason: 'pointer-identity-present',
+      viewScale: 'city',
+    },
+  );
+  assert.deepEqual(
+    viewportContextDecision({ ...pointerContext, pointer: null }),
+    { send: false, reason: 'pointer-missing', viewScale: 'city' },
+  );
+});
+
+test('runtime dispatch preserves pointer crop and full image separation', async () => {
+  const channel = { readyState: 'open' };
+  const pointer = { fresh: true, target: 'ground' };
+  const calls = [];
+  const viewport = new RealtimeViewport({
+    readChannel: () => channel,
+    readPointer: () => pointer,
+    operations: {
+      debugLog: (event, detail) => calls.push({ event, detail }),
+    },
+    capture: async () => 'unused',
+  });
+  viewport.sendPointerCrop = async (snapshot, options) => {
+    calls.push({ kind: 'crop', snapshot, options });
+    return true;
+  };
+  viewport.sendRetainedImage = async () => {
+    calls.push({ kind: 'viewport' });
+    return true;
+  };
+
+  assert.equal(
+    await viewport.sendVisualContextIfUseful(
+      localContext({
+        scope: 'pointer',
+        pointer: { target: 'ground' },
+        scene: { basemap: { viewScale: 'city' } },
+      }),
+    ),
+    true,
+  );
+  assert.equal(calls.filter((call) => call.kind === 'crop').length, 1);
+  assert.deepEqual(
+    calls.find((call) => call.kind === 'crop').snapshot,
+    pointer,
+  );
+  assert.equal(
+    calls.find((call) => call.kind === 'crop').options.viewScale,
+    'city',
+  );
+  assert.equal(calls.filter((call) => call.kind === 'viewport').length, 0);
+
+  assert.equal(await viewport.sendVisualContextIfUseful(localContext()), true);
+  assert.equal(calls.filter((call) => call.kind === 'viewport').length, 1);
+  assert.equal(
+    await viewport.sendVisualContextIfUseful(
+      localContext({ selected: { id: 'existing' } }),
+    ),
+    false,
+  );
+  assert.equal(calls.filter((call) => call.kind === 'viewport').length, 1);
+  assert.ok(
+    calls.some(
+      (call) =>
+        call.event === 'viewport_context.decision' &&
+        call.detail.reason === 'pointer-crop-eligible',
+    ),
+  );
+});
+
+test('closed channel never captures pointer or full viewport', async () => {
+  const viewport = new RealtimeViewport({
+    readChannel: () => ({ readyState: 'closed' }),
+    readPointer: () => {
+      throw new Error('should not inspect pointer when channel is closed');
+    },
+    operations: {},
+  });
+  assert.equal(
+    await viewport.sendVisualContextIfUseful(
+      localContext({ scope: 'pointer', pointer: { target: 'ground' } }),
+    ),
+    false,
+  );
+  assert.equal(await viewport.sendVisualContextIfUseful(localContext()), false);
+});
+
+test('capture failures and invalidated turns are reason-coded without sending', async () => {
+  const channel = { readyState: 'open' };
+  const diagnostics = [];
+  const viewport = new RealtimeViewport({
+    readChannel: () => channel,
+    operations: {
+      sendRealtimeEvent: () => {
+        throw new Error('should not send unavailable image');
+      },
+      debugLog: (event, detail) => diagnostics.push({ event, detail }),
+    },
+    capture: async () => null,
+  });
+  assert.equal(await viewport.sendVisualContextIfUseful(localContext()), false);
+  assert.ok(
+    diagnostics.some(
+      (item) =>
+        item.event === 'viewport_context.skipped' &&
+        item.detail.reason === 'capture-unavailable',
+    ),
+  );
+  let release;
+  viewport.capture = () =>
+    new Promise((resolve) => {
+      release = resolve;
+    });
+  const pending = viewport.sendVisualContextIfUseful(localContext());
+  viewport.reset();
+  release('data:image/jpeg;base64,AA==');
+  assert.equal(await pending, false);
+  assert.ok(
+    diagnostics.some(
+      (item) =>
+        item.event === 'viewport_context.skipped' &&
+        item.detail.reason === 'session-reset',
+    ),
+  );
+});
+
+test('successful retained send is logged without recording image contents', async () => {
+  const channel = { readyState: 'open' };
+  const diagnostics = [];
+  const viewport = new RealtimeViewport({
+    readChannel: () => channel,
+    readCameraKey: () => 'camera:fixture',
+    operations: {
+      sendRealtimeEvent: () => true,
+      debugLog: (event, detail) => diagnostics.push({ event, detail }),
+    },
+    capture: async () => 'data:image/jpeg;base64,AA==',
+  });
+  assert.equal(await viewport.sendVisualContextIfUseful(localContext()), true);
+  assert.ok(
+    diagnostics.some(
+      (item) =>
+        item.event === 'viewport_context.result' &&
+        item.detail.reason === 'sent',
+    ),
+  );
+  assert.equal(JSON.stringify(diagnostics).includes('data:image/jpeg'), false);
+});
+
+const outcomeEvents = new Set([
+  'viewport_context.skipped',
+  'viewport_context.result',
+]);
+
+function recordingViewport(options = {}) {
+  const events = [];
+  const channel = options.channel ?? { readyState: 'open' };
+  const viewport = new RealtimeViewport({
+    readChannel: () => channel,
+    readCameraKey: () => 'camera:fixture',
+    now: () => 1000,
+    capturePointer: async () => ({
+      dataUrl: 'data:image/jpeg;base64,AA==',
+      rect: { x: 0, y: 0, width: 1, height: 1 },
+    }),
+    capture: async () => 'data:image/jpeg;base64,AA==',
+    ...options,
+    operations: {
+      sendRealtimeEvent: () => true,
+      debugLog: (event, detail) => events.push({ event, detail }),
+      ...options.operations,
+    },
+  });
+  return { viewport, events };
+}
+
+const pointerGround = (overrides = {}) =>
+  localContext({
+    scope: 'pointer',
+    pointer: { target: 'ground' },
+    scene: { basemap: { viewScale: 'city' } },
+    ...overrides,
+  });
+
+const groundSnapshot = (overrides = {}) => ({
+  fresh: true,
+  target: 'ground',
+  screenPx: { x: 10, y: 10 },
+  capturedAt: 900,
+  cameraKey: 'camera:fixture',
+  ...overrides,
+});
+
+test('tool results other than entity context log nothing', async () => {
+  const { viewport, events } = recordingViewport();
+  assert.equal(
+    await viewport.sendVisualContextIfUseful({ action: 'fly_to' }),
+    false,
+  );
+  assert.equal(await viewport.sendVisualContextIfUseful(null), false);
+  assert.deepEqual(events, []);
+});
+
+test('a crop decision that cannot send logs why, under the same id', async () => {
+  const cases = [
+    [{ fresh: false }, 'pointer-not-fresh'],
+    [{ target: 'building' }, 'pointer-not-ground'],
+    [{ screenPx: null }, 'pointer-off-screen'],
+    [{ capturedAt: -50000 }, 'pointer-expired'],
+    [{ cameraKey: null }, 'pointer-pose-unknown'],
+    [{ cameraKey: 'camera:moved' }, 'pose-changed'],
+  ];
+  for (const [override, reason] of cases) {
+    const snapshot = groundSnapshot(override);
+    const { viewport, events } = recordingViewport({
+      readPointer: () => snapshot,
+    });
+    assert.equal(
+      await viewport.sendVisualContextIfUseful(pointerGround()),
+      false,
+      reason,
+    );
+    assert.equal(events.length, 2, reason);
+    const [decision, outcome] = events;
+    assert.equal(decision.event, 'viewport_context.decision');
+    assert.equal(decision.detail.reason, 'pointer-crop-eligible');
+    assert.deepEqual(outcome, {
+      event: 'viewport_context.skipped',
+      detail: { contextId: decision.detail.contextId, kind: 'crop', reason },
+    });
+  }
+});
+
+test('a repeated pointer snapshot is logged as already requested', async () => {
+  const snapshot = groundSnapshot();
+  const { viewport, events } = recordingViewport({
+    readPointer: () => snapshot,
+  });
+  assert.equal(await viewport.sendVisualContextIfUseful(pointerGround()), true);
+  assert.equal(
+    await viewport.sendVisualContextIfUseful(pointerGround()),
+    false,
+  );
+  const outcomes = events.filter((item) => outcomeEvents.has(item.event));
+  assert.deepEqual(
+    outcomes.map((item) => [item.detail.kind, item.detail.reason]),
+    [
+      ['crop', 'sent'],
+      ['crop', 'pointer-already-requested'],
+    ],
+  );
+});
+
+test('overlapping requests pair each outcome with its own decision', async () => {
+  const releases = [];
+  const { viewport, events } = recordingViewport({
+    capture: () => new Promise((resolve) => releases.push(resolve)),
+  });
+  const first = viewport.sendVisualContextIfUseful(localContext());
+  const second = viewport.sendVisualContextIfUseful(localContext());
+  await Promise.resolve();
+  releases[1]('data:image/jpeg;base64,AA==');
+  releases[0]('data:image/jpeg;base64,AA==');
+  assert.equal(await first, false);
+  assert.equal(await second, true);
+  const [firstId, secondId] = events
+    .filter((item) => item.event === 'viewport_context.decision')
+    .map((item) => item.detail.contextId);
+  assert.notEqual(firstId, secondId);
+  const outcomeFor = (id) =>
+    events.filter(
+      (item) => outcomeEvents.has(item.event) && item.detail.contextId === id,
+    );
+  assert.deepEqual(
+    outcomeFor(firstId).map((item) => item.detail),
+    [{ contextId: firstId, kind: 'viewport', reason: 'superseded' }],
+  );
+  assert.deepEqual(
+    outcomeFor(secondId).map((item) => item.detail),
+    [{ contextId: secondId, kind: 'viewport', reason: 'sent' }],
+  );
+});
+
+test('dropped captures name their cause', async () => {
+  const reasonFor = async (context, interrupt, extra = {}) => {
+    let release;
+    const pending = () => new Promise((resolve) => (release = resolve));
+    const { viewport, events } = recordingViewport({
+      capture: pending,
+      capturePointer: pending,
+      ...extra,
+    });
+    const sent = viewport.sendVisualContextIfUseful(context);
+    await Promise.resolve();
+    interrupt(viewport);
+    release('data:image/jpeg;base64,AA==');
+    assert.equal(await sent, false);
+    return events
+      .filter((item) => outcomeEvents.has(item.event))
+      .map((item) => item.detail.reason);
+  };
+
+  assert.deepEqual(
+    await reasonFor(localContext(), (viewport) => viewport.beginTurn()),
+    ['superseded'],
+  );
+  assert.deepEqual(
+    await reasonFor(localContext(), (viewport) => viewport.reset()),
+    ['session-reset'],
+  );
+  const channel = { readyState: 'open' };
+  assert.deepEqual(
+    await reasonFor(localContext(), () => (channel.readyState = 'closed'), {
+      channel,
+    }),
+    ['channel-lost'],
+  );
+  let cameraKey = 'camera:fixture';
+  const snapshot = groundSnapshot();
+  assert.deepEqual(
+    await reasonFor(pointerGround(), () => (cameraKey = 'camera:moved'), {
+      readPointer: () => snapshot,
+      readCameraKey: () => cameraKey,
+    }),
+    ['frame-stale'],
+  );
+});
+
+test('every send decision is followed by exactly one outcome', async () => {
+  const snapshots = [
+    null,
+    groundSnapshot(),
+    groundSnapshot({ fresh: false }),
+    groundSnapshot({ cameraKey: 'camera:moved' }),
+    groundSnapshot({ capturedAt: -50000 }),
+  ];
+  const contexts = [
+    localContext(),
+    localContext({ selected: { id: 'x', layerId: 'y' } }),
+    pointerGround(),
+    pointerGround({ scene: { basemap: { viewScale: 'global' } } }),
+    pointerGround({ pointer: null }),
+  ];
+  const captures = [
+    async () => 'data:image/jpeg;base64,AA==',
+    async () => null,
+  ];
+  for (const snapshot of snapshots)
+    for (const context of contexts)
+      for (const capture of captures)
+        for (const sendResult of [true, false]) {
+          const { viewport, events } = recordingViewport({
+            readPointer: () => snapshot,
+            capture,
+            capturePointer: capture,
+            operations: { sendRealtimeEvent: () => sendResult },
+          });
+          await viewport.sendVisualContextIfUseful(context);
+          const [decision, ...rest] = events;
+          assert.equal(decision.event, 'viewport_context.decision');
+          const outcomes = rest.filter((item) => outcomeEvents.has(item.event));
+          assert.equal(outcomes.length, decision.detail.send ? 1 : 0);
+          for (const outcome of outcomes) {
+            assert.equal(outcome.detail.contextId, decision.detail.contextId);
+            assert.equal(outcome.detail.kind, decision.detail.kind);
+            if (outcome.event === 'viewport_context.result')
+              assert.equal(
+                outcome.detail.reason,
+                sendResult ? 'sent' : 'data-channel-send-failed',
+              );
+          }
+        }
+});
+
+test('a failed data-channel send is never logged as sent', async () => {
+  for (const [sendResult, reason] of [
+    [true, 'sent'],
+    [false, 'data-channel-send-failed'],
+  ]) {
+    const { viewport, events } = recordingViewport({
+      operations: { sendRealtimeEvent: () => sendResult },
+    });
+    assert.equal(
+      await viewport.sendVisualContextIfUseful(localContext()),
+      sendResult,
+    );
+    const outcomes = events.filter((item) => outcomeEvents.has(item.event));
+    assert.deepEqual(
+      outcomes.map((item) => [item.event, item.detail.reason]),
+      [['viewport_context.result', reason]],
+    );
+  }
+});
+
+test('a step that throws still closes its decision, then rethrows', async () => {
+  for (const [context, extra] of [
+    [
+      localContext(),
+      {
+        capture: async () => {
+          throw new Error('capture boom');
+        },
+      },
+    ],
+    [
+      pointerGround(),
+      {
+        readPointer: () => {
+          throw new Error('pointer boom');
+        },
+      },
+    ],
+    [
+      pointerGround(),
+      {
+        readPointer: () => groundSnapshot(),
+        capturePointer: async () => {
+          throw new Error('crop boom');
+        },
+      },
+    ],
+  ]) {
+    const { viewport, events } = recordingViewport(extra);
+    await assert.rejects(viewport.sendVisualContextIfUseful(context), /boom/);
+    const [decision, ...rest] = events;
+    assert.deepEqual(rest, [
+      {
+        event: 'viewport_context.skipped',
+        detail: {
+          contextId: decision.detail.contextId,
+          kind: decision.detail.kind,
+          reason: 'error',
+        },
+      },
+    ]);
+  }
+});
+
+test('direct calls log their defensive skips without a decision id', async () => {
+  const open = recordingViewport();
+  assert.equal(
+    await open.viewport.sendPointerCrop(groundSnapshot(), {
+      viewScale: 'global',
+    }),
+    false,
+  );
+  const closed = recordingViewport({ channel: { readyState: 'closed' } });
+  assert.equal(
+    await closed.viewport.sendPointerCrop(groundSnapshot(), {
+      viewScale: 'city',
+    }),
+    false,
+  );
+  assert.equal(
+    await closed.viewport.sendRetainedImage(
+      async () => 'data:image/jpeg;base64,AA==',
+      'text',
+      'client.viewport_context',
+    ),
+    false,
+  );
+  assert.deepEqual(
+    [...open.events, ...closed.events].map((item) => item.detail),
+    [
+      { contextId: null, kind: 'crop', reason: 'pointer-scale-not-useful' },
+      { contextId: null, kind: 'crop', reason: 'channel-lost' },
+      { contextId: null, kind: 'viewport', reason: 'channel-lost' },
+    ],
+  );
+});
+
+test('a throw after a real send still logs it as sent', async () => {
+  let reads = 0;
+  const { viewport, events } = recordingViewport({
+    // The first read happens only after the image was handed to the channel.
+    readCameraKey: () => {
+      reads += 1;
+      throw new Error('camera gone');
+    },
+  });
+  await assert.rejects(
+    viewport.sendVisualContextIfUseful(localContext()),
+    /camera gone/,
+  );
+  assert.equal(reads, 1);
+  const outcomes = events.filter((item) => outcomeEvents.has(item.event));
+  assert.deepEqual(
+    outcomes.map((item) => [item.event, item.detail.reason]),
+    [['viewport_context.result', 'sent']],
+  );
+});
+
+test('a second request for a snapshot still being captured is not sent twice', async () => {
+  const releases = [];
+  const snapshot = groundSnapshot();
+  const { viewport, events } = recordingViewport({
+    readPointer: () => snapshot,
+    capturePointer: () => new Promise((resolve) => releases.push(resolve)),
+  });
+  const first = viewport.sendVisualContextIfUseful(pointerGround());
+  const second = viewport.sendVisualContextIfUseful(pointerGround());
+  await Promise.resolve();
+  for (const release of releases)
+    release({ dataUrl: 'data:image/jpeg;base64,AA==', rect: {} });
+  assert.deepEqual([await first, await second], [true, false]);
+  assert.equal(releases.length, 1);
+  const outcomes = events.filter((item) => outcomeEvents.has(item.event));
+  assert.deepEqual(
+    outcomes.map((item) => [item.detail.contextId, item.detail.reason]),
+    [
+      [2, 'pointer-already-requested'],
+      [1, 'sent'],
+    ],
+  );
+});