feat(voice): add typed AI chat beside voice

Chat sits next to the mic in the command dock. Replies are silent and never
request the microphone; the first message connects with the selected provider
and runs the same map actions as voice through each provider's text API
(OpenAI OPENAI_CHAT_MODEL, Gemini GEMINI_CHAT_MODEL via generateContent, which
never starts Live). Only one mode runs at a time.

- Chat panel rendered by the voice control presenter: conversation, recent and
  quick-command suggestions, Up/Down history, attachments (map-layer
  references or small UTF-8 text files, sent only with the message)
- Chat and voice panels sit above the dock or dock into the right rail's
  column; minimized bars sit centred above the dock, and height animates on
  swaps
- A centred panel steps down to its bar for the Location and Visual Presets
  trays without moving focus, rests above an open tray, and docks right while
  expanded over one; it returns to the centre once the trays close
- Minimized voice previews the last sentence from each side; the empty Voice
  Preview suggests things to say
- Settings opened from chat keep chat's frame and show the shared AI provider
  and quick commands; voice Settings show Back to voice
- Closing chat ends a typed session; the voice panel's close turns voice off
- The dock widens its voice control for the chat button and restyles its
  wings, which show icons only at 900px and below; assistant panels share the
  left panels' glass, and Cyber keeps its own surfaces
- Server text-chat handlers with body, time and per-IP limits
  (GEV_RATELIMIT_*_CHAT_PER_MIN), documented in SECURITY.md; tests and all
  eleven locales

Co-authored-by: Darren Dignam <[email protected]>
This commit is contained in:
manjunath22466andDarren Dignam committed 2026-11-02 20:56:39 +05:30
1 parent 720260af58
commit af461febae
57 files changed
+6500 -63

No files matched your search

+35
View File
@@ -30,6 +30,8 @@
"src/ui/views/panels/LocalePicker.vue",
"src/ui/views/panels/SceneDialog.vue",
"src/ui/views/panels/VoiceControl.vue",
"src/ui/views/panels/ChatPanel.vue",
"src/ui/views/panels/PanelLayoutButtons.vue",
"src/ui/views/panels/VoiceCard.vue",
"src/ui/views/panels/VoiceVisualizer.vue",
"src/ui/localePicker.js",
@@ -474,6 +476,15 @@
"server/providers/common/rate-limit.js",
"server/providers/common/request.js",
"server/providers/common/source-root.js",
"server/providers/common/textChat.js",
"server/providers/common/textInstructions.js",
"server/providers/common/textTools.js",
"server/providers/openai/chat.js",
"server/providers/gemini/request.js",
"build/allowedHosts.js",
"src/tools/schema.js",
"src/sources/requestDeadline.js",
"src/sources/httpBody.js",
"server/providers/openai.js",
"server/providers/openai/codex-auth.js",
"server/providers/openai/constants.js",
@@ -2116,6 +2127,14 @@
"src/voice/realtimeBackend.js",
"src/voice/cloudVoiceAuth.js",
"src/voice/control.js",
"src/voice/chatControl.js",
"src/voice/geminiTextChat.js",
"src/voice/textChat.js",
"src/voice/textProtocol.js",
"src/voice/chatAttachments.js",
"src/voice/chatPreferences.js",
"src/voice/panelLayout.js",
"src/voice/voiceMiniPreview.js",
"src/voice/layerSummary.js",
"src/voice/voiceCost.js",
"src/voice/commands.js",
@@ -2796,6 +2815,14 @@
"src/voice/panel/nativeActivity.js",
"src/voice/panel/preferences.js",
"src/voice/control.js",
"src/voice/chatControl.js",
"src/voice/geminiTextChat.js",
"src/voice/textChat.js",
"src/voice/textProtocol.js",
"src/voice/chatAttachments.js",
"src/voice/chatPreferences.js",
"src/voice/panelLayout.js",
"src/voice/voiceMiniPreview.js",
"src/voice/gevActions.js",
"src/agent/worldPacket.js",
"src/voice/speech.js",
@@ -4124,6 +4151,14 @@
"src/voice/sessionCommands.js",
"src/voice/session.js",
"src/voice/control.js",
"src/voice/chatControl.js",
"src/voice/geminiTextChat.js",
"src/voice/textChat.js",
"src/voice/textProtocol.js",
"src/voice/chatAttachments.js",
"src/voice/chatPreferences.js",
"src/voice/panelLayout.js",
"src/voice/voiceMiniPreview.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",