fix(voice): address review of typed AI chat

Typed chat sessions no longer end on the voice inactivity timeout, and a
failed or stopped reply keeps the conversation: Stop cancels only the
reply in progress, and map actions that already ran stay in the model's
context. Each upstream request and map action gets its own deadline, old
tool results shrink instead of the history overflowing, and step limits,
a missing or rejected key, the OAuth lock and throttling each get their
own message.

- Server: OpenAI chat is off under GEV_PREFER_CODEX_OAUTH; chat throttles
  default to the provider's throttle; replies carry API_KEY_MISSING,
  API_KEY_REJECTED, CODEX_OAUTH_REQUIRED and RATE_LIMITED codes; upstream
  Retry-After passes through, and a rejected key (Gemini's invalid-key 400
  included) cools down for a minute
- Pinokio forwards the chat throttle settings
- UI: the reply reveal publishes only the message list; the border orbit
  runs only while busy; layers attach with a button or Enter; the log is
  aria-busy while a reply types out; auto-scroll respects a reader who
  scrolled up; Escape closes the attach menu; the chat gear says Settings;
  a note says chat bills the OpenAI key when voice uses ChatGPT sign-in
- Layout does less work while panel height animates; unused blurs removed
- Gemini voice start ends chat; the voice preview clears on every new
  session
- Tests for the route gates, client limits, attachments and the fixes
This commit is contained in:
manjunath22466 committed 2026-11-03 22:01:08 +05:30
1 parent 11cd0142ac
commit da8b0dd8cd
49 files changed
+2121 -160

No files matched your search

+2 -2
View File
@@ -85,7 +85,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur
Allowing `maps.googleapis.com` in `script-src` also trusts its JSONP endpoints, which can bypass the script allowlist if an HTML-injection bug exists; this is an accepted limitation of loading the Maps JavaScript API, alongside the existing `'unsafe-eval'` allowance.
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
- **Typed chat spends the provider keys.** `/api/openai/chat` and `/api/gemini/chat` send the conversation to OpenAI Chat Completions or Gemini `generateContent` with the server's `OPENAI_API_KEY` or `GEMINI_API_KEY`. They require a same-site `Origin` (the Gemini request check), cap request bodies at 512 KB, refuse upstream redirects, cut the upstream call off after 60 s and return only sanitized errors. Chat always bills `OPENAI_API_KEY`; `GEV_PREFER_CODEX_OAUTH` governs voice only.
- **Typed chat spends the provider keys.** `/api/openai/chat` and `/api/gemini/chat` send the conversation to OpenAI Chat Completions or Gemini `generateContent` with the server's `OPENAI_API_KEY` or `GEMINI_API_KEY`. They require a same-site `Origin` (the Gemini request check), cap request bodies at 512 KB, refuse upstream redirects, cut the upstream call off after 60 s and return only sanitized errors. OpenAI chat can only bill `OPENAI_API_KEY`, so while `GEV_PREFER_CODEX_OAUTH` pins OpenAI to ChatGPT OAuth it refuses with `503` (`CODEX_OAUTH_REQUIRED`) rather than spend the key. After an upstream `401` or `403`, a route stops sending that key upstream for a minute, or until the key changes.
- **ChatGPT OAuth voice reuses the local Codex sign-in.** `/api/realtime/oauth-status`, `/api/realtime/oauth-login` and `/api/realtime/token?auth=oauth` answer only loopback connections, behind the same cross-site gate as the token endpoint; the login route also refuses a foreign `Origin`. None of them returns the access token. The login route runs `codex login` (or the executable named by `CODEX_BIN`) with an allowlisted environment that leaves out GEV's provider keys, keeps one attempt pending at a time and cancels it after two minutes or when the server stops. **Limits:** any program on this machine that can reach the server can mint Realtime sessions on the signed-in ChatGPT account, as it can with the API key; a local reverse proxy that hides the client's address defeats the loopback check; and OpenAI has not documented this use of Codex credentials for Realtime, so it may stop working or fall outside the account's terms.
## Network exposure — the operator threat model
@@ -94,7 +94,7 @@ The dev server is a **key broker**: every server-side key above is spendable by
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Typed chat has its own throttles, `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN` (default `60` each); lowering the voice throttles does not limit chat. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Typed chat keeps its own per-IP window, set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`; left unset, each takes its provider's resolved cap (`GEV_RATELIMIT_OPENAI_PER_MIN`, or `GEV_RATELIMIT_GEMINI_PER_MIN`, default `30`), so lowering a provider throttle lowers chat too, while an explicit chat value, including `0`, wins. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **Docker trusts one address, and only because the port is on loopback.**
Docker delivers the host's own browser to the container from the container
network's gateway, never from `127.0.0.1`, so a loopback-only Provider