mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-05 02:44:02 +00:00
Merge remote-tracking branch 'upstream/main' into feat/cctv-a22-autobrennero
# Conflicts: # CHANGELOG.md
This commit is contained in:
commit
1490f37e1b
33 files changed
+2500
-47
No files matched your search
+12
-4
@@ -396,7 +396,8 @@ the speaker (`mayVoiceClaimSpeaker()`). Progress lines stay outside the reply
|
||||
lifecycle and Radio handoff. `turn.span` debug records carry generation and
|
||||
playback latency, silence, preamble and word-count figures. Caption
|
||||
transcription (`OPENAI_REALTIME_TRANSCRIBE_MODEL`, default on) is metered into
|
||||
the session cost and cap. The debug log omits transcripts, text, tool arguments
|
||||
the session cost and cap in API-key sessions; ChatGPT OAuth sessions count
|
||||
captions without pricing them. The debug log omits transcripts, text, tool arguments
|
||||
and complete tool-result bodies (including serialized function outputs) unless
|
||||
`GEV_VOICE_LOG_CONTENT=1`.
|
||||
Tool names, call IDs, protocol event metadata and usage remain available.
|
||||
@@ -4035,7 +4036,7 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
|
||||
`GEV MIC` button (bottom UI) starts an OpenAI Realtime session over WebRTC:
|
||||
|
||||
- **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware holds `OPENAI_API_KEY` and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token.
|
||||
- **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware uses `OPENAI_API_KEY` by default, or the signed-in local ChatGPT/Codex OAuth access token when OAuth was explicitly selected in Provider Settings, and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. Both long-lived credentials stay server-side; OAuth minting is loopback-only. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token.
|
||||
- **Session defaults** (env-tunable): model `gpt-realtime-2` (or `gpt-realtime-2.1-mini` when the MINI tier is selected — see the model-tier entry below), voice `marin`, reasoning effort `low`, semantic VAD with low eagerness, no response interruption, context window truncated to ~3,000 post-instruction tokens with 0.5 retention ratio — the conversational window stays short because map state is fetched live per turn.
|
||||
- **Thirty tools** (argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `next_iss_pass`, and `next_satellite_pass`.
|
||||
|
||||
@@ -4081,7 +4082,10 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
2. **Contacts OFF** → "nearby" means **in view**; "near \<place\>" means a radius around that place. A radius query with Contacts active and no explicit centre is centred on the **active contact**, not the camera.
|
||||
3. **Every count names its scope in words** — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number. `analyst_query` returns `scopeLabel` so this is mechanical. Two different numbers with named scopes are not a contradiction.
|
||||
4. **The loaded-data caveat is stated once when relevant**: counts cover loaded data, and the flights layer loads where you look (appended to `coverage.note` for radius/view scopes over viewport-loaded layers).
|
||||
- **Degradation**: without `OPENAI_API_KEY`, `/api/realtime/token` returns 503 and the mic button surfaces the error; the rest of the app is unaffected.
|
||||
- **Degradation**: API-key mode without `OPENAI_API_KEY` returns 503 as before. Selecting OAuth without a usable local ChatGPT/Codex sign-in starts the local `codex login` browser flow, polls for the resulting local token, and selects OAuth when sign-in completes. If the Codex login flow cannot start or does not complete, the rest of the app is unaffected.
|
||||
- **Login lifecycle and storage**: one pending Codex login is shared by the login and status routes. Process failure, success without readable credentials, and a two-minute timeout each report a clear failure; timeout and server shutdown cancel the owned process. The launcher resolves `CODEX_HOME` before changing directories and refuses a mismatched `CODEX_AUTH_JSON`. GEV reads file credentials only, never writes or refreshes them, and does not require the vendor refresh token. Expired access tokens report `CODEX_OAUTH_REAUTH_REQUIRED`. OAuth status and login routes also use the same-site request gate.
|
||||
- **Auth button focus**: changing voice auth updates the existing controls in place. Enter, Space, and mouse selection keep focus on the auth button; Tab then follows the normal control order. Pending sign-in blocks repeated activation without removing the button from the focus order. Completion does not take focus from another control or discard text in the key fields. `scripts/qa-voice-auth-focus.mjs` checks these paths in Chromium with fixture sign-in responses.
|
||||
- **Support boundary**: the local OAuth option is experimental. OpenAI's documented Sign in with ChatGPT plan-sharing inference endpoint is `/v1/responses`; support for third-party reuse of Codex credentials at `/v1/realtime/client_secrets` remains unconfirmed. Technical success does not establish a supported auth or billing contract.
|
||||
|
||||
### AI HUD Summary (June 2026)
|
||||
|
||||
@@ -4301,7 +4305,7 @@ are omitted rather than framing the wrong part of the globe.
|
||||
- GBFS proxy refuses upstream redirects (`redirect: 'manual'`; any 3xx becomes a 502 and the redirect target is logged server-side only) and enforces its 5 MB response cap while the body streams, cancelling the upstream read past the cap; CCTV health map is bounded.
|
||||
- Proxy error payloads are sanitized (no internal error details returned to clients).
|
||||
- That holds for the OpenAI and CCTV media paths too: `/api/openai/hud-summary` never relays OpenAI's own `error.message`, `/api/realtime/token` passes successful ephemeral-token responses through but answers with a fixed error when minting fails or upstream rejects the request, and a failed CCTV media fetch stores a fixed camera health `message` — `GET /api/cctv/health` serializes that field and the CCTV panel renders it as a status label, so it is a client surface as much as the response body is.
|
||||
- `OPENAI_API_KEY` is server-side only; the browser receives ephemeral Realtime client secrets from `/api/realtime/token`.
|
||||
- `OPENAI_API_KEY` remains server-side only and remains the default voice credential. The optional OAuth mode reads the local ChatGPT/Codex access token server-side on loopback only; the browser receives only ephemeral Realtime client secrets from `/api/realtime/token`.
|
||||
- `AISSTREAM_API_KEY` is server-side only; the browser reads the same-origin `/api/vessels` cache.
|
||||
- `/api/google/nearby-places` keeps the Google key out of Places requests issued for voice scene context.
|
||||
- `/api/google/text-search` keeps the Google key server-side for view-biased Places recovery used by annotation resolution.
|
||||
@@ -4704,6 +4708,10 @@ easier to meet (detection is now on more often), but does not create it.
|
||||
shown dot against the allowed source classes and records phase timings.
|
||||
Positions and segment distances are precomputed, with no new animation-loop
|
||||
allocation. Visible-globe roads use cached terrain without offscreen mesh picks. The road-source label names the geometry being drawn, with partial/unavailable states shown plainly.
|
||||
- Vector tile sources accept tiles only from an allowed origin, which defaults
|
||||
to the configured `tileJsonUrl` origin; an explicit `allowedOrigin` overrides
|
||||
it. Repointing only `tileJsonUrl` loads tiles from that host or fails with an
|
||||
origin error rather than silently falling back to OpenFreeMap.
|
||||
- TileJSON caches successful metadata. Transient failures retry after a
|
||||
five-second cooldown; invalid metadata/origins stay unavailable until the
|
||||
source is cleared. Clear resets metadata and cancels pending requests.
|
||||
|
||||
Reference in new issue
Block a user