mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 10:24:02 +00:00
fix(voice): address review of typed AI chat
Typed chat sessions no longer end on the voice inactivity timeout, and a failed or stopped reply keeps the conversation: Stop cancels only the reply in progress, and map actions that already ran stay in the model's context. Each upstream request and map action gets its own deadline, old tool results shrink instead of the history overflowing, and step limits, a missing or rejected key, the OAuth lock and throttling each get their own message. - Server: OpenAI chat is off under GEV_PREFER_CODEX_OAUTH; chat throttles default to the provider's throttle; replies carry API_KEY_MISSING, API_KEY_REJECTED, CODEX_OAUTH_REQUIRED and RATE_LIMITED codes; upstream Retry-After passes through, and a rejected key (Gemini's invalid-key 400 included) cools down for a minute - Pinokio forwards the chat throttle settings - UI: the reply reveal publishes only the message list; the border orbit runs only while busy; layers attach with a button or Enter; the log is aria-busy while a reply types out; auto-scroll respects a reader who scrolled up; Escape closes the attach menu; the chat gear says Settings; a note says chat bills the OpenAI key when voice uses ChatGPT sign-in - Layout does less work while panel height animates; unused blurs removed - Gemini voice start ends chat; the voice preview clears on every new session - Tests for the route gates, client limits, attachments and the fixes
This commit is contained in:
1 parent
11cd0142ac
commit
da8b0dd8cd
49 files changed
+2121
-160
No files matched your search
+16
-2
@@ -699,6 +699,14 @@ files, 10 KB each and bounded together, sent only with the message; a layer
|
||||
reference names the layer without exporting its records. The conversation and
|
||||
recent commands stay in memory for the page.
|
||||
|
||||
The chat routes answer a missing key with `503` and code `API_KEY_MISSING`.
|
||||
OpenAI chat can only bill `OPENAI_API_KEY`, so while `GEV_PREFER_CODEX_OAUTH`
|
||||
pins OpenAI to ChatGPT OAuth it answers `503` with code `CODEX_OAUTH_REQUIRED`
|
||||
instead; Gemini chat is unaffected. An upstream `429` or `503` keeps its status
|
||||
and passes on a `Retry-After` given in seconds, capped at an hour. After an
|
||||
upstream `401` or `403`, the route answers `502` without calling the provider
|
||||
for a minute, unless the configured key changes.
|
||||
|
||||
Chat and the voice panel sit above the dock or dock into the right rail's lane,
|
||||
which hides the rail until the panel minimizes or closes, or a rail panel opens.
|
||||
Minimized panels always sit as a bar centred above the dock; restoring returns a
|
||||
@@ -856,7 +864,7 @@ Source factories have dedicated `layers/<family>/source` exports. ALPR and earth
|
||||
|
||||
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/progress/result, interruption, completion and turn-metrics events; stopping or replacing a session cancels pending actions. An active session's normal Stop publishes one idle transition, including adapters that emit their own terminal state; a restart triggered by that transition keeps ownership of its new startup state. Alternate adapters can use the same controls and action runner.
|
||||
|
||||
The standalone app offers OpenAI and Gemini Live through an explicit provider chooser. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter.
|
||||
The standalone app offers OpenAI and Gemini Live through the **AI Provider** setting, shared by voice and typed chat. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter.
|
||||
|
||||
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
|
||||
|
||||
@@ -1745,6 +1753,12 @@ unaffected. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` ov
|
||||
those; exactly `0` disables the limiter, while a value that cannot be read as a
|
||||
number falls back to the default rather than to unlimited. Over-limit requests
|
||||
receive a sanitized `429` with `Retry-After: 5` and never reach the provider.
|
||||
Typed chat (`/api/openai/chat`, `/api/gemini/chat`) counts in its own window,
|
||||
set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`.
|
||||
Left unset, each takes its provider's resolved cap — `GEV_RATELIMIT_OPENAI_PER_MIN`
|
||||
or `GEV_RATELIMIT_GEMINI_PER_MIN`, both 30 by default — so lowering a provider
|
||||
throttle lowers chat too; an explicit chat value, including `0`, wins. Its
|
||||
over-limit reply carries `Retry-After: 60`.
|
||||
The client key is the socket peer address only — a forwarded-for header is not
|
||||
trusted, so a proxied deployment shares one bucket per upstream hop.
|
||||
|
||||
@@ -5832,7 +5846,7 @@ apply to `public/` assets or built-preview serving.
|
||||
- Panel POSITION keys are versioned `v8` (`godsEyeView.v8.panelPos.<id>`); collapsed-state keys remain `v6`. The one-time position reset clears stale DISPLAY placements that could overlap the Context rail.
|
||||
- Map Source lives in the bottom Visual Presets tray. The left accordion contains no MAP STACK panel, and the `k` panel token that addressed it is gone from the share registry, so legacy `ui=k...` state takes the ordinary unknown-token skip.
|
||||
- A dock popover (Visual Presets, Location) auto-dismisses on mouse-away unless pinned. Focus inside the tray defers that dismissal only when the browser reports `:focus-visible` — keyboard focus and typed-into fields hold the tray open; a mouse-clicked tile does not, because Chromium focuses a `<button>` on press.
|
||||
- GEV MIC control is a glass capsule (var(--glass-bg), blur(24px) saturate(1.4), 999px radius; panel radius in error state).
|
||||
- The voice and chat panels share the left panels' glass surface (`--glass-bg`, `--glass-border`, blur(24px) saturate(1.4); `src/ui/styles/ai-chat.css`). The docked voice control draws its own blur on a layer behind it, so the panels inside it still blur the map. Cyber keeps its own surfaces.
|
||||
- The desktop right rail (`#right-context-rail`) owns `DISPLAY`, `CCTV`, its active parameter controls, and `GLOBAL CONTEXT` as one fixed responsive stack in that order. Its compact buttons use the same 176 px width as the left accordion and one consistent 50 px height, share the left stack's 52 px edge inset and measured top baseline across HUD variants, then constrain themselves against visible HUD/chrome rectangles and the remaining vertical corridor. `DISPLAY` is no longer draggable and legacy saved coordinates are ignored.
|
||||
- The right rail is labeled **DISPLAY** (formerly "MOVE") and groups, in order, HUD, DETECT, Bloom, Sharpen, 3D, Clean-UI (HUD + DETECT promoted to the top). Its expanded controls retain the same compact 176 px width as the right-side tabs instead of growing to the wider Context detail-card width. It starts expanded on first run and respects the user's later `v6` collapse choice. Collapses/expands with directional chevrons (`◀` collapsed, `▶` expanded).
|
||||
- Display and Context use matching 330 px expanded widths and matching compact tab dimensions. The parameter panel is part of Display's expanded content. One right-rail panel is open at a time in every HUD layout and collapsed launchers are hidden; the panel rail beside the stack switches between them (see Panel rails). The most recently opened right-rail panel owns the lane even when it appears later in DOM order; passive restoration and automatic disclosure do not replace that explicit owner. When the open panel exceeds the measured corridor, it scrolls internally.
|
||||
|
||||
Reference in new issue
Block a user