mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 02:14:01 +00:00
fix(voice): address review of typed AI chat
Typed chat sessions no longer end on the voice inactivity timeout, and a failed or stopped reply keeps the conversation: Stop cancels only the reply in progress, and map actions that already ran stay in the model's context. Each upstream request and map action gets its own deadline, old tool results shrink instead of the history overflowing, and step limits, a missing or rejected key, the OAuth lock and throttling each get their own message. - Server: OpenAI chat is off under GEV_PREFER_CODEX_OAUTH; chat throttles default to the provider's throttle; replies carry API_KEY_MISSING, API_KEY_REJECTED, CODEX_OAUTH_REQUIRED and RATE_LIMITED codes; upstream Retry-After passes through, and a rejected key (Gemini's invalid-key 400 included) cools down for a minute - Pinokio forwards the chat throttle settings - UI: the reply reveal publishes only the message list; the border orbit runs only while busy; layers attach with a button or Enter; the log is aria-busy while a reply types out; auto-scroll respects a reader who scrolled up; Escape closes the attach menu; the chat gear says Settings; a note says chat bills the OpenAI key when voice uses ChatGPT sign-in - Layout does less work while panel height animates; unused blurs removed - Gemini voice start ends chat; the voice preview clears on every new session - Tests for the route gates, client limits, attachments and the fixes
This commit is contained in:
1 parent
11cd0142ac
commit
da8b0dd8cd
49 files changed
+2121
-160
No files matched your search
+11
-9
@@ -61,14 +61,15 @@ CESIUM_ION_TOKEN=
|
|||||||
# ChatGPT/Codex OAuth session; the browser still receives only a short-lived
|
# ChatGPT/Codex OAuth session; the browser still receives only a short-lived
|
||||||
# Realtime client secret. Do not prefix with VITE_.
|
# Realtime client secret. Do not prefix with VITE_.
|
||||||
OPENAI_API_KEY=
|
OPENAI_API_KEY=
|
||||||
# Typed chat uses Chat Completions, independently of the voice model. It always
|
# Typed chat uses Chat Completions, independently of the voice model. It can
|
||||||
# bills OPENAI_API_KEY; GEV_PREFER_CODEX_OAUTH below applies to voice only.
|
# only bill OPENAI_API_KEY, so GEV_PREFER_CODEX_OAUTH below turns it off.
|
||||||
OPENAI_CHAT_MODEL=gpt-4.1-mini
|
OPENAI_CHAT_MODEL=gpt-4.1-mini
|
||||||
# Typed chat requests (/api/openai/chat) per IP per minute, separate from the
|
# Typed chat requests (/api/openai/chat) per IP per minute, counted apart from
|
||||||
# voice throttle; 0 disables it. Not a billing cap.
|
# voice. Unset, it takes GEV_RATELIMIT_OPENAI_PER_MIN's cap; 0 disables it.
|
||||||
# GEV_RATELIMIT_OPENAI_CHAT_PER_MIN=60 # the default
|
# Not a billing cap.
|
||||||
# Optional: forbid API-key voice sessions. Select ChatGPT OAuth in Provider
|
# GEV_RATELIMIT_OPENAI_CHAT_PER_MIN=30
|
||||||
# Settings; missing or expired Codex sign-in then fails closed.
|
# Optional: forbid API-key voice sessions and OpenAI typed chat. Select ChatGPT
|
||||||
|
# OAuth in Provider Settings; missing or expired Codex sign-in then fails closed.
|
||||||
# GEV_PREFER_CODEX_OAUTH=true
|
# GEV_PREFER_CODEX_OAUTH=true
|
||||||
OPENAI_REALTIME_MODEL=gpt-realtime-2
|
OPENAI_REALTIME_MODEL=gpt-realtime-2
|
||||||
# Cheaper voice tier, used when the GEV MIC panel's MINI toggle is on.
|
# Cheaper voice tier, used when the GEV MIC panel's MINI toggle is on.
|
||||||
@@ -304,5 +305,6 @@ OVERPASS_UPSTREAMS=
|
|||||||
|
|
||||||
# Gemini typed chat uses generateContent; voice continues to use Gemini Live.
|
# Gemini typed chat uses generateContent; voice continues to use Gemini Live.
|
||||||
GEMINI_CHAT_MODEL=gemini-3.5-flash-lite
|
GEMINI_CHAT_MODEL=gemini-3.5-flash-lite
|
||||||
# Typed chat requests (/api/gemini/chat) per IP per minute; 0 disables it.
|
# Typed chat requests (/api/gemini/chat) per IP per minute. Unset, it takes
|
||||||
# GEV_RATELIMIT_GEMINI_CHAT_PER_MIN=60 # the default
|
# GEV_RATELIMIT_GEMINI_PER_MIN's cap (default 30); 0 disables it.
|
||||||
|
# GEV_RATELIMIT_GEMINI_CHAT_PER_MIN=30
|
||||||
@@ -362,11 +362,11 @@ _Why cockpit mode exists: you're riding a real aircraft over real terrain — an
|
|||||||
|
|
||||||
## 🎙️ Talk to It
|
## 🎙️ Talk to It
|
||||||
|
|
||||||
The standalone app offers **OpenAI** and **Gemini** in the GEV MIC provider chooser. OpenAI is the initial default; your choice is saved in this browser. To use Gemini, add `GEMINI_API_KEY` in Provider Settings or your local `.env` file, then select Gemini. `GEMINI_LIVE_MODEL` defaults to `gemini-3.8-live`. The permanent key stays on the local server, which issues a constrained, single-use session credential.
|
The standalone app offers **OpenAI** and **Gemini** under **AI Provider** in Settings, one choice shared by voice and chat. OpenAI is the initial default; your choice is saved in this browser. To use Gemini, add `GEMINI_API_KEY` in Provider Settings or your local `.env` file, then select Gemini. `GEMINI_LIVE_MODEL` defaults to `gemini-3.8-live`. The permanent key stays on the local server, which issues a constrained, single-use session credential.
|
||||||
|
|
||||||
Changing provider ends the current conversation. If voice is on, the new provider starts with the same open-mic or push-to-talk mode; a held Space key is not carried over. Gemini uses the same map actions and captions, but has no OpenAI cost meter or MINI tier. When Gemini reports that its session has expired, start voice again for a new conversation. The provider chooser does not change the OpenAI-backed AI HUD summary.
|
Changing provider ends the current conversation. If voice is on, the new provider starts with the same open-mic or push-to-talk mode; a held Space key is not carried over. Gemini uses the same map actions and captions, but has no OpenAI cost meter or MINI tier. When Gemini reports that its session has expired, start voice again for a new conversation. The AI Provider setting does not change the OpenAI-backed AI HUD summary.
|
||||||
|
|
||||||
Next to the mic, **CHAT** opens a typed chat with silent replies. It uses the selected provider and the same map actions, and never asks for the microphone. OpenAI chat needs an API key (`OPENAI_CHAT_MODEL`); Gemini chat uses a separate text model (`GEMINI_CHAT_MODEL`). Chat and voice panels sit above the dock or dock to the right.
|
Next to the mic, **CHAT** opens a typed chat with silent replies. It uses the selected provider and the same map actions, and never asks for the microphone. OpenAI chat needs an API key (`OPENAI_CHAT_MODEL`), so it is off while `GEV_PREFER_CODEX_OAUTH` pins OpenAI to ChatGPT OAuth; Gemini chat uses a separate text model (`GEMINI_CHAT_MODEL`). Chat and voice panels share the left panels' glass surface and sit above the dock or dock to the right.
|
||||||
|
|
||||||
> Voice can use an **OpenAI API key** or, on the same machine, an opt-in signed-in **ChatGPT/Codex OAuth** session. The API-key path remains the default. Selecting ChatGPT OAuth in Provider Settings reuses an existing local sign-in or starts the Codex browser sign-in flow and selects OAuth when it completes. The OpenAI key still drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move.
|
> Voice can use an **OpenAI API key** or, on the same machine, an opt-in signed-in **ChatGPT/Codex OAuth** session. The API-key path remains the default. Selecting ChatGPT OAuth in Provider Settings reuses an existing local sign-in or starts the Codex browser sign-in flow and selects OAuth when it completes. The OpenAI key still drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move.
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -85,7 +85,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur
|
|||||||
|
|
||||||
Allowing `maps.googleapis.com` in `script-src` also trusts its JSONP endpoints, which can bypass the script allowlist if an HTML-injection bug exists; this is an accepted limitation of loading the Maps JavaScript API, alongside the existing `'unsafe-eval'` allowance.
|
Allowing `maps.googleapis.com` in `script-src` also trusts its JSONP endpoints, which can bypass the script allowlist if an HTML-injection bug exists; this is an accepted limitation of loading the Maps JavaScript API, alongside the existing `'unsafe-eval'` allowance.
|
||||||
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
|
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
|
||||||
- **Typed chat spends the provider keys.** `/api/openai/chat` and `/api/gemini/chat` send the conversation to OpenAI Chat Completions or Gemini `generateContent` with the server's `OPENAI_API_KEY` or `GEMINI_API_KEY`. They require a same-site `Origin` (the Gemini request check), cap request bodies at 512 KB, refuse upstream redirects, cut the upstream call off after 60 s and return only sanitized errors. Chat always bills `OPENAI_API_KEY`; `GEV_PREFER_CODEX_OAUTH` governs voice only.
|
- **Typed chat spends the provider keys.** `/api/openai/chat` and `/api/gemini/chat` send the conversation to OpenAI Chat Completions or Gemini `generateContent` with the server's `OPENAI_API_KEY` or `GEMINI_API_KEY`. They require a same-site `Origin` (the Gemini request check), cap request bodies at 512 KB, refuse upstream redirects, cut the upstream call off after 60 s and return only sanitized errors. OpenAI chat can only bill `OPENAI_API_KEY`, so while `GEV_PREFER_CODEX_OAUTH` pins OpenAI to ChatGPT OAuth it refuses with `503` (`CODEX_OAUTH_REQUIRED`) rather than spend the key. After an upstream `401` or `403`, a route stops sending that key upstream for a minute, or until the key changes.
|
||||||
- **ChatGPT OAuth voice reuses the local Codex sign-in.** `/api/realtime/oauth-status`, `/api/realtime/oauth-login` and `/api/realtime/token?auth=oauth` answer only loopback connections, behind the same cross-site gate as the token endpoint; the login route also refuses a foreign `Origin`. None of them returns the access token. The login route runs `codex login` (or the executable named by `CODEX_BIN`) with an allowlisted environment that leaves out GEV's provider keys, keeps one attempt pending at a time and cancels it after two minutes or when the server stops. **Limits:** any program on this machine that can reach the server can mint Realtime sessions on the signed-in ChatGPT account, as it can with the API key; a local reverse proxy that hides the client's address defeats the loopback check; and OpenAI has not documented this use of Codex credentials for Realtime, so it may stop working or fall outside the account's terms.
|
- **ChatGPT OAuth voice reuses the local Codex sign-in.** `/api/realtime/oauth-status`, `/api/realtime/oauth-login` and `/api/realtime/token?auth=oauth` answer only loopback connections, behind the same cross-site gate as the token endpoint; the login route also refuses a foreign `Origin`. None of them returns the access token. The login route runs `codex login` (or the executable named by `CODEX_BIN`) with an allowlisted environment that leaves out GEV's provider keys, keeps one attempt pending at a time and cancels it after two minutes or when the server stops. **Limits:** any program on this machine that can reach the server can mint Realtime sessions on the signed-in ChatGPT account, as it can with the API key; a local reverse proxy that hides the client's address defeats the loopback check; and OpenAI has not documented this use of Codex credentials for Realtime, so it may stop working or fall outside the account's terms.
|
||||||
|
|
||||||
## Network exposure — the operator threat model
|
## Network exposure — the operator threat model
|
||||||
@@ -94,7 +94,7 @@ The dev server is a **key broker**: every server-side key above is spendable by
|
|||||||
|
|
||||||
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
|
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
|
||||||
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
|
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
|
||||||
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Typed chat has its own throttles, `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN` (default `60` each); lowering the voice throttles does not limit chat. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
|
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Typed chat keeps its own per-IP window, set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`; left unset, each takes its provider's resolved cap (`GEV_RATELIMIT_OPENAI_PER_MIN`, or `GEV_RATELIMIT_GEMINI_PER_MIN`, default `30`), so lowering a provider throttle lowers chat too, while an explicit chat value, including `0`, wins. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
|
||||||
- **Docker trusts one address, and only because the port is on loopback.**
|
- **Docker trusts one address, and only because the port is on loopback.**
|
||||||
Docker delivers the host's own browser to the container from the container
|
Docker delivers the host's own browser to the container from the container
|
||||||
network's gateway, never from `127.0.0.1`, so a loopback-only Provider
|
network's gateway, never from `127.0.0.1`, so a loopback-only Provider
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
- Add a typed AI chat beside the voice mic in the command dock. Replies are
|
- Add a typed AI chat beside the voice mic in the command dock. Replies are
|
||||||
silent and never use the microphone; messages run the same map actions as
|
silent and never use the microphone; messages run the same map actions as
|
||||||
voice through OpenAI Chat Completions (`OPENAI_CHAT_MODEL`) or Gemini
|
voice through OpenAI Chat Completions (`OPENAI_CHAT_MODEL`) or Gemini
|
||||||
`generateContent` (`GEMINI_CHAT_MODEL`), each throttled per IP by
|
`generateContent` (`GEMINI_CHAT_MODEL`). Chat is throttled per IP by
|
||||||
`GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` or `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`
|
`GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` or `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`,
|
||||||
(default 60). The chat and voice panels minimize to a bar or dock beside the
|
which default to that provider's voice throttle, and `GEV_PREFER_CODEX_OAUTH`
|
||||||
right rail, and step aside for the Location and Visual Presets trays
|
turns OpenAI chat off. The chat and voice panels minimize to a bar or dock
|
||||||
(Manjunath, @manjunath22466, with Darren Dignam, @darrendignam, #1507).
|
beside the right rail, and step aside for the Location and Visual Presets
|
||||||
|
trays (Manjunath, @manjunath22466, with Darren Dignam, @darrendignam, #1507).
|
||||||
+16
-2
@@ -699,6 +699,14 @@ files, 10 KB each and bounded together, sent only with the message; a layer
|
|||||||
reference names the layer without exporting its records. The conversation and
|
reference names the layer without exporting its records. The conversation and
|
||||||
recent commands stay in memory for the page.
|
recent commands stay in memory for the page.
|
||||||
|
|
||||||
|
The chat routes answer a missing key with `503` and code `API_KEY_MISSING`.
|
||||||
|
OpenAI chat can only bill `OPENAI_API_KEY`, so while `GEV_PREFER_CODEX_OAUTH`
|
||||||
|
pins OpenAI to ChatGPT OAuth it answers `503` with code `CODEX_OAUTH_REQUIRED`
|
||||||
|
instead; Gemini chat is unaffected. An upstream `429` or `503` keeps its status
|
||||||
|
and passes on a `Retry-After` given in seconds, capped at an hour. After an
|
||||||
|
upstream `401` or `403`, the route answers `502` without calling the provider
|
||||||
|
for a minute, unless the configured key changes.
|
||||||
|
|
||||||
Chat and the voice panel sit above the dock or dock into the right rail's lane,
|
Chat and the voice panel sit above the dock or dock into the right rail's lane,
|
||||||
which hides the rail until the panel minimizes or closes, or a rail panel opens.
|
which hides the rail until the panel minimizes or closes, or a rail panel opens.
|
||||||
Minimized panels always sit as a bar centred above the dock; restoring returns a
|
Minimized panels always sit as a bar centred above the dock; restoring returns a
|
||||||
@@ -856,7 +864,7 @@ Source factories have dedicated `layers/<family>/source` exports. ALPR and earth
|
|||||||
|
|
||||||
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/progress/result, interruption, completion and turn-metrics events; stopping or replacing a session cancels pending actions. An active session's normal Stop publishes one idle transition, including adapters that emit their own terminal state; a restart triggered by that transition keeps ownership of its new startup state. Alternate adapters can use the same controls and action runner.
|
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/progress/result, interruption, completion and turn-metrics events; stopping or replacing a session cancels pending actions. An active session's normal Stop publishes one idle transition, including adapters that emit their own terminal state; a restart triggered by that transition keeps ownership of its new startup state. Alternate adapters can use the same controls and action runner.
|
||||||
|
|
||||||
The standalone app offers OpenAI and Gemini Live through an explicit provider chooser. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter.
|
The standalone app offers OpenAI and Gemini Live through the **AI Provider** setting, shared by voice and typed chat. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter.
|
||||||
|
|
||||||
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
|
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
|
||||||
|
|
||||||
@@ -1745,6 +1753,12 @@ unaffected. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` ov
|
|||||||
those; exactly `0` disables the limiter, while a value that cannot be read as a
|
those; exactly `0` disables the limiter, while a value that cannot be read as a
|
||||||
number falls back to the default rather than to unlimited. Over-limit requests
|
number falls back to the default rather than to unlimited. Over-limit requests
|
||||||
receive a sanitized `429` with `Retry-After: 5` and never reach the provider.
|
receive a sanitized `429` with `Retry-After: 5` and never reach the provider.
|
||||||
|
Typed chat (`/api/openai/chat`, `/api/gemini/chat`) counts in its own window,
|
||||||
|
set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`.
|
||||||
|
Left unset, each takes its provider's resolved cap — `GEV_RATELIMIT_OPENAI_PER_MIN`
|
||||||
|
or `GEV_RATELIMIT_GEMINI_PER_MIN`, both 30 by default — so lowering a provider
|
||||||
|
throttle lowers chat too; an explicit chat value, including `0`, wins. Its
|
||||||
|
over-limit reply carries `Retry-After: 60`.
|
||||||
The client key is the socket peer address only — a forwarded-for header is not
|
The client key is the socket peer address only — a forwarded-for header is not
|
||||||
trusted, so a proxied deployment shares one bucket per upstream hop.
|
trusted, so a proxied deployment shares one bucket per upstream hop.
|
||||||
|
|
||||||
@@ -5832,7 +5846,7 @@ apply to `public/` assets or built-preview serving.
|
|||||||
- Panel POSITION keys are versioned `v8` (`godsEyeView.v8.panelPos.<id>`); collapsed-state keys remain `v6`. The one-time position reset clears stale DISPLAY placements that could overlap the Context rail.
|
- Panel POSITION keys are versioned `v8` (`godsEyeView.v8.panelPos.<id>`); collapsed-state keys remain `v6`. The one-time position reset clears stale DISPLAY placements that could overlap the Context rail.
|
||||||
- Map Source lives in the bottom Visual Presets tray. The left accordion contains no MAP STACK panel, and the `k` panel token that addressed it is gone from the share registry, so legacy `ui=k...` state takes the ordinary unknown-token skip.
|
- Map Source lives in the bottom Visual Presets tray. The left accordion contains no MAP STACK panel, and the `k` panel token that addressed it is gone from the share registry, so legacy `ui=k...` state takes the ordinary unknown-token skip.
|
||||||
- A dock popover (Visual Presets, Location) auto-dismisses on mouse-away unless pinned. Focus inside the tray defers that dismissal only when the browser reports `:focus-visible` — keyboard focus and typed-into fields hold the tray open; a mouse-clicked tile does not, because Chromium focuses a `<button>` on press.
|
- A dock popover (Visual Presets, Location) auto-dismisses on mouse-away unless pinned. Focus inside the tray defers that dismissal only when the browser reports `:focus-visible` — keyboard focus and typed-into fields hold the tray open; a mouse-clicked tile does not, because Chromium focuses a `<button>` on press.
|
||||||
- GEV MIC control is a glass capsule (var(--glass-bg), blur(24px) saturate(1.4), 999px radius; panel radius in error state).
|
- The voice and chat panels share the left panels' glass surface (`--glass-bg`, `--glass-border`, blur(24px) saturate(1.4); `src/ui/styles/ai-chat.css`). The docked voice control draws its own blur on a layer behind it, so the panels inside it still blur the map. Cyber keeps its own surfaces.
|
||||||
- The desktop right rail (`#right-context-rail`) owns `DISPLAY`, `CCTV`, its active parameter controls, and `GLOBAL CONTEXT` as one fixed responsive stack in that order. Its compact buttons use the same 176 px width as the left accordion and one consistent 50 px height, share the left stack's 52 px edge inset and measured top baseline across HUD variants, then constrain themselves against visible HUD/chrome rectangles and the remaining vertical corridor. `DISPLAY` is no longer draggable and legacy saved coordinates are ignored.
|
- The desktop right rail (`#right-context-rail`) owns `DISPLAY`, `CCTV`, its active parameter controls, and `GLOBAL CONTEXT` as one fixed responsive stack in that order. Its compact buttons use the same 176 px width as the left accordion and one consistent 50 px height, share the left stack's 52 px edge inset and measured top baseline across HUD variants, then constrain themselves against visible HUD/chrome rectangles and the remaining vertical corridor. `DISPLAY` is no longer draggable and legacy saved coordinates are ignored.
|
||||||
- The right rail is labeled **DISPLAY** (formerly "MOVE") and groups, in order, HUD, DETECT, Bloom, Sharpen, 3D, Clean-UI (HUD + DETECT promoted to the top). Its expanded controls retain the same compact 176 px width as the right-side tabs instead of growing to the wider Context detail-card width. It starts expanded on first run and respects the user's later `v6` collapse choice. Collapses/expands with directional chevrons (`◀` collapsed, `▶` expanded).
|
- The right rail is labeled **DISPLAY** (formerly "MOVE") and groups, in order, HUD, DETECT, Bloom, Sharpen, 3D, Clean-UI (HUD + DETECT promoted to the top). Its expanded controls retain the same compact 176 px width as the right-side tabs instead of growing to the wider Context detail-card width. It starts expanded on first run and respects the user's later `v6` collapse choice. Collapses/expands with directional chevrons (`◀` collapsed, `▶` expanded).
|
||||||
- Display and Context use matching 330 px expanded widths and matching compact tab dimensions. The parameter panel is part of Display's expanded content. One right-rail panel is open at a time in every HUD layout and collapsed launchers are hidden; the panel rail beside the stack switches between them (see Panel rails). The most recently opened right-rail panel owns the lane even when it appears later in DOM order; passive restoration and automatic disclosure do not replace that explicit owner. When the open panel exceeds the measured corridor, it scrolls internally.
|
- Display and Context use matching 330 px expanded widths and matching compact tab dimensions. The parameter panel is part of Display's expanded content. One right-rail panel is open at a time in every HUD layout and collapsed launchers are hidden; the panel rail beside the stack switches between them (see Panel rails). The most recently opened right-rail panel owns the lane even when it appears later in DOM order; passive restoration and automatic disclosure do not replace that explicit owner. When the open panel exceeds the measured corridor, it scrolls internally.
|
||||||
|
|||||||
@@ -30,3 +30,7 @@ PINOKIO_SHARE_VAR=__gev_sharing_disabled__
|
|||||||
# App-level guards, not billing caps. Provider-side budgets remain authoritative.
|
# App-level guards, not billing caps. Provider-side budgets remain authoritative.
|
||||||
GEV_RATELIMIT_OPENAI_PER_MIN=30
|
GEV_RATELIMIT_OPENAI_PER_MIN=30
|
||||||
GEV_RATELIMIT_GOOGLE_PER_MIN=120
|
GEV_RATELIMIT_GOOGLE_PER_MIN=120
|
||||||
|
# Typed chat follows the OpenAI cap above, or Gemini voice's default of 30,
|
||||||
|
# unless set here; 0 disables it.
|
||||||
|
# GEV_RATELIMIT_OPENAI_CHAT_PER_MIN=
|
||||||
|
# GEV_RATELIMIT_GEMINI_CHAT_PER_MIN=
|
||||||
@@ -28,6 +28,8 @@ module.exports = {
|
|||||||
LL2_API_TOKEN: '{{env.LL2_API_TOKEN || ""}}',
|
LL2_API_TOKEN: '{{env.LL2_API_TOKEN || ""}}',
|
||||||
GEV_RATELIMIT_OPENAI_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_PER_MIN || ""}}',
|
GEV_RATELIMIT_OPENAI_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_PER_MIN || ""}}',
|
||||||
GEV_RATELIMIT_GOOGLE_PER_MIN: '{{env.GEV_RATELIMIT_GOOGLE_PER_MIN || ""}}',
|
GEV_RATELIMIT_GOOGLE_PER_MIN: '{{env.GEV_RATELIMIT_GOOGLE_PER_MIN || ""}}',
|
||||||
|
GEV_RATELIMIT_OPENAI_CHAT_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_CHAT_PER_MIN || ""}}',
|
||||||
|
GEV_RATELIMIT_GEMINI_CHAT_PER_MIN: '{{env.GEV_RATELIMIT_GEMINI_CHAT_PER_MIN || ""}}',
|
||||||
},
|
},
|
||||||
message: 'node scripts/pinokio-install.mjs',
|
message: 'node scripts/pinokio-install.mjs',
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -23,6 +23,8 @@ module.exports = {
|
|||||||
PINOKIO_SHARE_VAR: '{{env.PINOKIO_SHARE_VAR || "__gev_sharing_disabled__"}}',
|
PINOKIO_SHARE_VAR: '{{env.PINOKIO_SHARE_VAR || "__gev_sharing_disabled__"}}',
|
||||||
GEV_RATELIMIT_OPENAI_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_PER_MIN || ""}}',
|
GEV_RATELIMIT_OPENAI_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_PER_MIN || ""}}',
|
||||||
GEV_RATELIMIT_GOOGLE_PER_MIN: '{{env.GEV_RATELIMIT_GOOGLE_PER_MIN || ""}}',
|
GEV_RATELIMIT_GOOGLE_PER_MIN: '{{env.GEV_RATELIMIT_GOOGLE_PER_MIN || ""}}',
|
||||||
|
GEV_RATELIMIT_OPENAI_CHAT_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_CHAT_PER_MIN || ""}}',
|
||||||
|
GEV_RATELIMIT_GEMINI_CHAT_PER_MIN: '{{env.GEV_RATELIMIT_GEMINI_CHAT_PER_MIN || ""}}',
|
||||||
},
|
},
|
||||||
message: 'node scripts/pinokio-start.mjs',
|
message: 'node scripts/pinokio-start.mjs',
|
||||||
on: [{
|
on: [{
|
||||||
|
|||||||
@@ -19,6 +19,8 @@ module.exports = {
|
|||||||
LL2_API_TOKEN: '{{env.LL2_API_TOKEN || ""}}',
|
LL2_API_TOKEN: '{{env.LL2_API_TOKEN || ""}}',
|
||||||
GEV_RATELIMIT_OPENAI_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_PER_MIN || ""}}',
|
GEV_RATELIMIT_OPENAI_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_PER_MIN || ""}}',
|
||||||
GEV_RATELIMIT_GOOGLE_PER_MIN: '{{env.GEV_RATELIMIT_GOOGLE_PER_MIN || ""}}',
|
GEV_RATELIMIT_GOOGLE_PER_MIN: '{{env.GEV_RATELIMIT_GOOGLE_PER_MIN || ""}}',
|
||||||
|
GEV_RATELIMIT_OPENAI_CHAT_PER_MIN: '{{env.GEV_RATELIMIT_OPENAI_CHAT_PER_MIN || ""}}',
|
||||||
|
GEV_RATELIMIT_GEMINI_CHAT_PER_MIN: '{{env.GEV_RATELIMIT_GEMINI_CHAT_PER_MIN || ""}}',
|
||||||
},
|
},
|
||||||
message: 'node scripts/pinokio-update.mjs',
|
message: 'node scripts/pinokio-update.mjs',
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1210,6 +1210,9 @@
|
|||||||
"src/voice/voiceMiniPreview.js",
|
"src/voice/voiceMiniPreview.js",
|
||||||
"src/voice/voiceMiniPreview.test.mjs",
|
"src/voice/voiceMiniPreview.test.mjs",
|
||||||
"src/voice/chatPreferences.test.mjs",
|
"src/voice/chatPreferences.test.mjs",
|
||||||
|
"src/voice/textChat.test.mjs",
|
||||||
|
"src/voice/chatAttachments.test.mjs",
|
||||||
|
"src/voice/chatControlView.test.mjs",
|
||||||
"src/textChatServer.test.mjs",
|
"src/textChatServer.test.mjs",
|
||||||
"src/geminiChatServer.test.mjs",
|
"src/geminiChatServer.test.mjs",
|
||||||
"src/voice/geminiTextChat.test.mjs",
|
"src/voice/geminiTextChat.test.mjs",
|
||||||
|
|||||||
@@ -4202,6 +4202,7 @@
|
|||||||
"src/voice/session.js",
|
"src/voice/session.js",
|
||||||
"src/voice/control.js",
|
"src/voice/control.js",
|
||||||
"src/voice/chatControl.js",
|
"src/voice/chatControl.js",
|
||||||
|
"src/voice/cloudVoiceAuth.js",
|
||||||
"src/voice/geminiTextChat.js",
|
"src/voice/geminiTextChat.js",
|
||||||
"src/voice/textChat.js",
|
"src/voice/textChat.js",
|
||||||
"src/voice/textProtocol.js",
|
"src/voice/textProtocol.js",
|
||||||
|
|||||||
@@ -19,6 +19,8 @@ export const PINOKIO_CONFIG_FIELDS = Object.freeze([
|
|||||||
'LL2_API_TOKEN',
|
'LL2_API_TOKEN',
|
||||||
'GEV_RATELIMIT_OPENAI_PER_MIN',
|
'GEV_RATELIMIT_OPENAI_PER_MIN',
|
||||||
'GEV_RATELIMIT_GOOGLE_PER_MIN',
|
'GEV_RATELIMIT_GOOGLE_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_OPENAI_CHAT_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_GEMINI_CHAT_PER_MIN',
|
||||||
'PINOKIO_SHARE_CLOUDFLARE',
|
'PINOKIO_SHARE_CLOUDFLARE',
|
||||||
'PINOKIO_SHARE_LOCAL',
|
'PINOKIO_SHARE_LOCAL',
|
||||||
'PINOKIO_SHARE_VAR',
|
'PINOKIO_SHARE_VAR',
|
||||||
|
|||||||
@@ -1,5 +1,9 @@
|
|||||||
import { readRequestBody } from './request.js';
|
import { readRequestBody } from './request.js';
|
||||||
import { clientKey, makeCostRateLimiter } from './rate-limit.js';
|
import {
|
||||||
|
clientKey,
|
||||||
|
makeCostRateLimiter,
|
||||||
|
resolvePerMinuteCap,
|
||||||
|
} from './rate-limit.js';
|
||||||
import { waitForSignal } from '../../../src/sources/requestDeadline.js';
|
import { waitForSignal } from '../../../src/sources/requestDeadline.js';
|
||||||
import { readResponseJsonCapped } from '../../../src/sources/httpBody.js';
|
import { readResponseJsonCapped } from '../../../src/sources/httpBody.js';
|
||||||
import { isGeminiRequestOriginAllowed } from '../gemini/request.js';
|
import { isGeminiRequestOriginAllowed } from '../gemini/request.js';
|
||||||
@@ -7,6 +11,29 @@ import { parseUsage } from './textTools.js';
|
|||||||
|
|
||||||
const MAX_BYTES = 512 * 1024;
|
const MAX_BYTES = 512 * 1024;
|
||||||
const FAILURE = 'Could not connect. Try again.';
|
const FAILURE = 'Could not connect. Try again.';
|
||||||
|
// A rejected key is not retried upstream for this long, unless the key changes.
|
||||||
|
const REJECTED_KEY_COOLDOWN_MS = 60_000;
|
||||||
|
|
||||||
|
// Gemini answers an invalid key with 400 and this reason, not 401 or 403.
|
||||||
|
const KEY_REJECTED_REASON = 'API_KEY_INVALID';
|
||||||
|
async function rejectsKey(response, signal) {
|
||||||
|
if ([401, 403].includes(response.status)) return true;
|
||||||
|
if (response.status !== 400) return false;
|
||||||
|
try {
|
||||||
|
const body = await readResponseJsonCapped(response, 16 * 1024, signal);
|
||||||
|
return JSON.stringify(body).includes(KEY_REJECTED_REASON);
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pass through only a delay in whole seconds, capped at an hour.
|
||||||
|
function upstreamRetryAfter(response) {
|
||||||
|
const value = String(response.headers?.get?.('retry-after') ?? '').trim();
|
||||||
|
if (!/^\d{1,6}$/.test(value)) return null;
|
||||||
|
return String(Math.min(Math.max(Number(value), 1), 3600));
|
||||||
|
}
|
||||||
|
|
||||||
export function createTextChatHandler({
|
export function createTextChatHandler({
|
||||||
provider,
|
provider,
|
||||||
prepare,
|
prepare,
|
||||||
@@ -16,21 +43,32 @@ export function createTextChatHandler({
|
|||||||
resolveHost = () => process.env.HOST,
|
resolveHost = () => process.env.HOST,
|
||||||
resolveAllowedHosts = () => process.env.GEV_ALLOWED_HOSTS,
|
resolveAllowedHosts = () => process.env.GEV_ALLOWED_HOSTS,
|
||||||
resolveRateLimit = () =>
|
resolveRateLimit = () =>
|
||||||
process.env[`GEV_RATELIMIT_${provider.toUpperCase()}_CHAT_PER_MIN`] ?? '60',
|
process.env[`GEV_RATELIMIT_${provider.toUpperCase()}_CHAT_PER_MIN`],
|
||||||
|
// An unset chat throttle follows the provider's own cost throttle.
|
||||||
|
resolveProviderRateLimit = () => undefined,
|
||||||
|
providerPerMin = 30,
|
||||||
|
// Returns { error, code } when the route must refuse before any key is used.
|
||||||
|
refuse = () => null,
|
||||||
|
now = Date.now,
|
||||||
timeoutMs = 60000,
|
timeoutMs = 60000,
|
||||||
} = {}) {
|
} = {}) {
|
||||||
let limiter;
|
let limiter;
|
||||||
|
let rejected = null;
|
||||||
return async (req, res) => {
|
return async (req, res) => {
|
||||||
if (req.aborted || res.destroyed || res.writableEnded) return;
|
if (req.aborted || res.destroyed || res.writableEnded) return;
|
||||||
res.setHeader('Cache-Control', 'no-store');
|
res.setHeader('Cache-Control', 'no-store');
|
||||||
res.setHeader('Content-Type', 'application/json; charset=utf-8');
|
res.setHeader('Content-Type', 'application/json; charset=utf-8');
|
||||||
res.setHeader('X-Content-Type-Options', 'nosniff');
|
res.setHeader('X-Content-Type-Options', 'nosniff');
|
||||||
const reply = (status, value) => {
|
const reply = (status, value, code) => {
|
||||||
if (req.aborted || res.destroyed || res.writableEnded) return;
|
if (req.aborted || res.destroyed || res.writableEnded) return;
|
||||||
res.statusCode = status;
|
res.statusCode = status;
|
||||||
if (status >= 400) res.setHeader('Connection', 'close');
|
if (status >= 400) res.setHeader('Connection', 'close');
|
||||||
res.end(
|
res.end(
|
||||||
JSON.stringify(typeof value === 'string' ? { error: value } : value),
|
JSON.stringify(
|
||||||
|
typeof value === 'string'
|
||||||
|
? { error: value, ...(code ? { code } : {}) }
|
||||||
|
: value,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
};
|
};
|
||||||
if (req.method !== 'POST') {
|
if (req.method !== 'POST') {
|
||||||
@@ -52,13 +90,32 @@ export function createTextChatHandler({
|
|||||||
!isGeminiRequestOriginAllowed(req, resolveHost(), resolveAllowedHosts())
|
!isGeminiRequestOriginAllowed(req, resolveHost(), resolveAllowedHosts())
|
||||||
)
|
)
|
||||||
return reply(403, FAILURE);
|
return reply(403, FAILURE);
|
||||||
|
const refusal = refuse();
|
||||||
|
if (refusal) return reply(503, refusal.error, refusal.code);
|
||||||
const apiKey = String(resolveApiKey() || '').trim();
|
const apiKey = String(resolveApiKey() || '').trim();
|
||||||
if (!apiKey) return reply(503, 'Add an API key in Provider Settings');
|
if (!apiKey)
|
||||||
|
return reply(
|
||||||
|
503,
|
||||||
|
'Add an API key in Provider Settings',
|
||||||
|
'API_KEY_MISSING',
|
||||||
|
);
|
||||||
|
if (rejected && (rejected.key !== apiKey || now() >= rejected.until))
|
||||||
|
rejected = null;
|
||||||
|
if (rejected) {
|
||||||
|
res.setHeader(
|
||||||
|
'Retry-After',
|
||||||
|
String(Math.max(1, Math.ceil((rejected.until - now()) / 1000))),
|
||||||
|
);
|
||||||
|
return reply(502, FAILURE, 'API_KEY_REJECTED');
|
||||||
|
}
|
||||||
if (limiter === undefined)
|
if (limiter === undefined)
|
||||||
limiter = makeCostRateLimiter(resolveRateLimit(), 60);
|
limiter = makeCostRateLimiter(
|
||||||
|
resolveRateLimit(),
|
||||||
|
resolvePerMinuteCap(resolveProviderRateLimit(), providerPerMin),
|
||||||
|
);
|
||||||
if (limiter && !limiter(clientKey(req))) {
|
if (limiter && !limiter(clientKey(req))) {
|
||||||
res.setHeader('Retry-After', '60');
|
res.setHeader('Retry-After', '60');
|
||||||
return reply(429, FAILURE);
|
return reply(429, FAILURE, 'RATE_LIMITED');
|
||||||
}
|
}
|
||||||
if (
|
if (
|
||||||
!/^application\/json(?:\s*;|$)/i.test(req.headers['content-type'] || '')
|
!/^application\/json(?:\s*;|$)/i.test(req.headers['content-type'] || '')
|
||||||
@@ -99,10 +156,19 @@ export function createTextChatHandler({
|
|||||||
if (response.redirected || (response.url && response.url !== endpoint))
|
if (response.redirected || (response.url && response.url !== endpoint))
|
||||||
throw new Error('Unexpected destination');
|
throw new Error('Unexpected destination');
|
||||||
if (!response.ok) {
|
if (!response.ok) {
|
||||||
|
if (await rejectsKey(response, lifetime.signal)) {
|
||||||
|
void response.body?.cancel().catch(() => {});
|
||||||
|
rejected = { key: apiKey, until: now() + REJECTED_KEY_COOLDOWN_MS };
|
||||||
|
return reply(502, FAILURE, 'API_KEY_REJECTED');
|
||||||
|
}
|
||||||
void response.body?.cancel().catch(() => {});
|
void response.body?.cancel().catch(() => {});
|
||||||
|
if (![429, 503].includes(response.status)) return reply(502, FAILURE);
|
||||||
|
const retryAfter = upstreamRetryAfter(response);
|
||||||
|
if (retryAfter) res.setHeader('Retry-After', retryAfter);
|
||||||
return reply(
|
return reply(
|
||||||
[429, 503].includes(response.status) ? response.status : 502,
|
response.status,
|
||||||
FAILURE,
|
FAILURE,
|
||||||
|
response.status === 429 ? 'RATE_LIMITED' : undefined,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
const data = await readResponseJsonCapped(
|
const data = await readResponseJsonCapped(
|
||||||
|
|||||||
@@ -4,6 +4,9 @@ import { textInstructions } from '../common/textInstructions.js';
|
|||||||
import { GEV_REALTIME_TOOLS } from '../openai/tools.js';
|
import { GEV_REALTIME_TOOLS } from '../openai/tools.js';
|
||||||
import { normalizeGeminiModel } from './config.js';
|
import { normalizeGeminiModel } from './config.js';
|
||||||
|
|
||||||
|
// The Gemini token route's throttle default (GEV_RATELIMIT_GEMINI_PER_MIN).
|
||||||
|
export const GEMINI_DEFAULT_PER_MIN = 30;
|
||||||
|
|
||||||
const object = (value) =>
|
const object = (value) =>
|
||||||
value && typeof value === 'object' && !Array.isArray(value);
|
value && typeof value === 'object' && !Array.isArray(value);
|
||||||
const keys = (value, allowed) =>
|
const keys = (value, allowed) =>
|
||||||
@@ -80,12 +83,15 @@ export function createGeminiChatHandler({
|
|||||||
resolveChatModel = () =>
|
resolveChatModel = () =>
|
||||||
process.env.GEMINI_CHAT_MODEL || 'gemini-3.5-flash-lite',
|
process.env.GEMINI_CHAT_MODEL || 'gemini-3.5-flash-lite',
|
||||||
resolveApiKey = () => process.env.GEMINI_API_KEY,
|
resolveApiKey = () => process.env.GEMINI_API_KEY,
|
||||||
|
resolveProviderRateLimit = () => process.env.GEV_RATELIMIT_GEMINI_PER_MIN,
|
||||||
...options
|
...options
|
||||||
} = {}) {
|
} = {}) {
|
||||||
return createTextChatHandler({
|
return createTextChatHandler({
|
||||||
...options,
|
...options,
|
||||||
provider: 'gemini',
|
provider: 'gemini',
|
||||||
resolveApiKey,
|
resolveApiKey,
|
||||||
|
resolveProviderRateLimit,
|
||||||
|
providerPerMin: GEMINI_DEFAULT_PER_MIN,
|
||||||
prepare(input) {
|
prepare(input) {
|
||||||
const model = normalizeGeminiModel(resolveChatModel());
|
const model = normalizeGeminiModel(resolveChatModel());
|
||||||
if (!model || /live|audio|tts/i.test(model)) return { errorStatus: 503 };
|
if (!model || /live|audio|tts/i.test(model)) return { errorStatus: 503 };
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
import { createTextChatHandler } from '../common/textChat.js';
|
import { createTextChatHandler } from '../common/textChat.js';
|
||||||
|
import { preferCodexOAuth } from './codex-auth.js';
|
||||||
|
import { OPENAI_DEFAULT_PER_MIN } from './rate-limit.js';
|
||||||
import { invalidCalls } from '../common/textTools.js';
|
import { invalidCalls } from '../common/textTools.js';
|
||||||
import { textInstructions } from '../common/textInstructions.js';
|
import { textInstructions } from '../common/textInstructions.js';
|
||||||
import { GEV_REALTIME_TOOLS } from './tools.js';
|
import { GEV_REALTIME_TOOLS } from './tools.js';
|
||||||
@@ -66,12 +68,27 @@ export function createOpenAiChatHandler({
|
|||||||
tools = GEV_REALTIME_TOOLS,
|
tools = GEV_REALTIME_TOOLS,
|
||||||
resolveApiKey = () => process.env.OPENAI_API_KEY,
|
resolveApiKey = () => process.env.OPENAI_API_KEY,
|
||||||
resolveChatModel = () => process.env.OPENAI_CHAT_MODEL || 'gpt-4.1-mini',
|
resolveChatModel = () => process.env.OPENAI_CHAT_MODEL || 'gpt-4.1-mini',
|
||||||
|
resolveProviderRateLimit = () => process.env.GEV_RATELIMIT_OPENAI_PER_MIN,
|
||||||
|
environment = process.env,
|
||||||
...options
|
...options
|
||||||
} = {}) {
|
} = {}) {
|
||||||
return createTextChatHandler({
|
return createTextChatHandler({
|
||||||
...options,
|
...options,
|
||||||
provider: 'openai',
|
provider: 'openai',
|
||||||
resolveApiKey,
|
resolveApiKey,
|
||||||
|
resolveProviderRateLimit,
|
||||||
|
providerPerMin: OPENAI_DEFAULT_PER_MIN,
|
||||||
|
// Chat can only bill the API key, which this operator lock forbids.
|
||||||
|
refuse() {
|
||||||
|
let error =
|
||||||
|
'ChatGPT OAuth is required by GEV_PREFER_CODEX_OAUTH, and typed OpenAI chat needs an API key.';
|
||||||
|
try {
|
||||||
|
if (!preferCodexOAuth(environment)) return null;
|
||||||
|
} catch (invalid) {
|
||||||
|
error = invalid.message;
|
||||||
|
}
|
||||||
|
return { error, code: 'CODEX_OAUTH_REQUIRED' };
|
||||||
|
},
|
||||||
prepare(input) {
|
prepare(input) {
|
||||||
const model = String(resolveChatModel()).trim();
|
const model = String(resolveChatModel()).trim();
|
||||||
if (
|
if (
|
||||||
|
|||||||
@@ -2,9 +2,13 @@ import assert from 'node:assert/strict';
|
|||||||
import test from 'node:test';
|
import test from 'node:test';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import { readFileSync } from 'node:fs';
|
||||||
|
import { EventEmitter } from 'node:events';
|
||||||
import { Readable } from 'node:stream';
|
import { Readable } from 'node:stream';
|
||||||
import { readPinokioEnvironment } from '../scripts/pinokio-environment.mjs';
|
import { readPinokioEnvironment } from '../scripts/pinokio-environment.mjs';
|
||||||
import { openAiRealtimeProxy } from 'gods-eye-view/server/providers/openai';
|
import { openAiRealtimeProxy } from 'gods-eye-view/server/providers/openai';
|
||||||
|
import { geminiLiveProxy } from '../server/providers/gemini.js';
|
||||||
|
import { GEMINI_DEFAULT_PER_MIN } from '../server/providers/gemini/chat.js';
|
||||||
import {
|
import {
|
||||||
makeCostRateLimiter,
|
makeCostRateLimiter,
|
||||||
resolvePerMinuteCap,
|
resolvePerMinuteCap,
|
||||||
@@ -168,6 +172,92 @@ test('the realtime token route throttles by default, and says so in a 429', asyn
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
function chatRequest(handler, input) {
|
||||||
|
const req = Readable.from([Buffer.from(JSON.stringify(input))]);
|
||||||
|
Object.assign(req, {
|
||||||
|
method: 'POST',
|
||||||
|
url: '/',
|
||||||
|
headers: {
|
||||||
|
host: 'localhost:4173',
|
||||||
|
origin: 'http://localhost:4173',
|
||||||
|
'content-type': 'application/json',
|
||||||
|
},
|
||||||
|
socket: { remoteAddress: '127.0.0.1' },
|
||||||
|
});
|
||||||
|
const headers = {};
|
||||||
|
const res = Object.assign(new EventEmitter(), {
|
||||||
|
statusCode: 200,
|
||||||
|
setHeader(name, value) {
|
||||||
|
headers[name.toLowerCase()] = value;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
res.end = (body = '') => {
|
||||||
|
res.writableEnded = true;
|
||||||
|
resolve({ status: res.statusCode, headers, body: String(body) });
|
||||||
|
};
|
||||||
|
Promise.resolve(handler(req, res)).catch(reject);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
test('typed chat follows each provider throttle by default, before upstream', async (t) => {
|
||||||
|
for (const name of [
|
||||||
|
'GEV_RATELIMIT_OPENAI_CHAT_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_OPENAI_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_GEMINI_CHAT_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_GEMINI_PER_MIN',
|
||||||
|
'GEV_PREFER_CODEX_OAUTH',
|
||||||
|
])
|
||||||
|
env(t, name, undefined);
|
||||||
|
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
|
||||||
|
env(t, 'GEMINI_API_KEY', 'fixture-upstream-secret');
|
||||||
|
let upstreamCalls = 0;
|
||||||
|
t.mock.method(globalThis, 'fetch', async (url) => {
|
||||||
|
upstreamCalls += 1;
|
||||||
|
return Response.json(
|
||||||
|
String(url).includes('openai')
|
||||||
|
? { choices: [{ message: { role: 'assistant', content: 'Hi' } }] }
|
||||||
|
: {
|
||||||
|
candidates: [
|
||||||
|
{ content: { role: 'model', parts: [{ text: 'Hi' }] } },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
// The chat fallback must track the Gemini token route's own default.
|
||||||
|
const tokenDefault = readFileSync(
|
||||||
|
new URL('../server/providers/gemini/realtime.js', import.meta.url),
|
||||||
|
'utf8',
|
||||||
|
).match(/GEV_RATELIMIT_GEMINI_PER_MIN \?\? '(\d+)'/)?.[1];
|
||||||
|
assert.equal(Number(tokenDefault), GEMINI_DEFAULT_PER_MIN);
|
||||||
|
|
||||||
|
for (const [route, handler, cap, input] of [
|
||||||
|
[
|
||||||
|
'/api/openai/chat',
|
||||||
|
install(openAiRealtimeProxy({})).get('/api/openai/chat'),
|
||||||
|
OPENAI_DEFAULT_PER_MIN,
|
||||||
|
{ messages: [{ role: 'user', content: 'Hello' }] },
|
||||||
|
],
|
||||||
|
[
|
||||||
|
'/api/gemini/chat',
|
||||||
|
install(geminiLiveProxy({})).get('/api/gemini/chat'),
|
||||||
|
GEMINI_DEFAULT_PER_MIN,
|
||||||
|
{ contents: [{ role: 'user', parts: [{ text: 'Hello' }] }] },
|
||||||
|
],
|
||||||
|
]) {
|
||||||
|
upstreamCalls = 0;
|
||||||
|
const statuses = [];
|
||||||
|
for (let i = 0; i < cap; i += 1)
|
||||||
|
statuses.push((await chatRequest(handler, input)).status);
|
||||||
|
assert.deepEqual(statuses, Array(cap).fill(200), route);
|
||||||
|
const overLimit = await chatRequest(handler, input);
|
||||||
|
assert.equal(overLimit.status, 429, route);
|
||||||
|
assert.equal(overLimit.headers['retry-after'], '60');
|
||||||
|
assert.equal(overLimit.body.includes('fixture-upstream-secret'), false);
|
||||||
|
assert.equal(upstreamCalls, cap, `${route} refuses before upstream`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
test('the HUD summary route throttles the paid path, never the free one', async (t) => {
|
test('the HUD summary route throttles the paid path, never the free one', async (t) => {
|
||||||
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
|
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
|
||||||
let upstreamCalls = 0;
|
let upstreamCalls = 0;
|
||||||
|
|||||||
@@ -3286,5 +3286,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "مزوّد الذكاء الاصطناعي المختار مشترك بين الصوت والدردشة.",
|
"The selected AI provider is shared by voice and chat.": "مزوّد الذكاء الاصطناعي المختار مشترك بين الصوت والدردشة.",
|
||||||
"Show quick commands": "إظهار الأوامر السريعة",
|
"Show quick commands": "إظهار الأوامر السريعة",
|
||||||
"Chat replies are silent. No microphone access is needed.": "ردود الدردشة صامتة. لا حاجة إلى الوصول إلى الميكروفون.",
|
"Chat replies are silent. No microphone access is needed.": "ردود الدردشة صامتة. لا حاجة إلى الوصول إلى الميكروفون.",
|
||||||
"Chat": "الدردشة"
|
"Chat": "الدردشة",
|
||||||
|
"Attach layer": "إرفاق الطبقة",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "تستخدم الدردشة الكتابية مفتاح OpenAI API الخاص بك. تسجيل الدخول إلى ChatGPT يشمل الصوت فقط.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "الدردشة الكتابية غير متاحة بينما يفرض GEV_PREFER_CODEX_OAUTH استخدام ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "تطلّب هذا الطلب خطوات كثيرة جدًا. تبقى التغييرات التي أُجريت على الخريطة كما هي.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "تطلّب هذا الطلب خطوات كثيرة جدًا. جرّب طلبًا أبسط.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "رفض المزوّد مفتاح API. تحقّق منه في إعدادات المزوّد.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "طلبات دردشة كثيرة جدًا. حاول مجددًا بعد {seconds} ث."
|
||||||
}
|
}
|
||||||
@@ -3006,5 +3006,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "Der gewählte KI-Anbieter gilt für Sprache und Chat.",
|
"The selected AI provider is shared by voice and chat.": "Der gewählte KI-Anbieter gilt für Sprache und Chat.",
|
||||||
"Show quick commands": "Schnellbefehle anzeigen",
|
"Show quick commands": "Schnellbefehle anzeigen",
|
||||||
"Chat replies are silent. No microphone access is needed.": "Chat-Antworten sind stumm. Kein Mikrofonzugriff nötig.",
|
"Chat replies are silent. No microphone access is needed.": "Chat-Antworten sind stumm. Kein Mikrofonzugriff nötig.",
|
||||||
"Chat": "Chat"
|
"Chat": "Chat",
|
||||||
|
"Attach layer": "Ebene anhängen",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "Der Text-Chat nutzt deinen OpenAI-API-Schlüssel. Die ChatGPT-Anmeldung gilt nur für Sprache.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "Der Text-Chat ist nicht verfügbar, solange GEV_PREFER_CODEX_OAUTH ChatGPT OAuth vorschreibt.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Diese Anfrage brauchte zu viele Schritte. Bereits vorgenommene Kartenänderungen bleiben erhalten.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Diese Anfrage brauchte zu viele Schritte. Versuche eine einfachere Anfrage.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "Der Anbieter hat den API-Schlüssel abgelehnt. Prüfe ihn in den Anbieter-Einstellungen.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Zu viele Chat-Anfragen. In {seconds} s erneut versuchen."
|
||||||
}
|
}
|
||||||
@@ -3076,5 +3076,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "El proveedor de IA seleccionado se usa tanto para voz como para chat.",
|
"The selected AI provider is shared by voice and chat.": "El proveedor de IA seleccionado se usa tanto para voz como para chat.",
|
||||||
"Show quick commands": "Mostrar comandos rápidos",
|
"Show quick commands": "Mostrar comandos rápidos",
|
||||||
"Chat replies are silent. No microphone access is needed.": "Las respuestas del chat son silenciosas. No se necesita acceso al micrófono.",
|
"Chat replies are silent. No microphone access is needed.": "Las respuestas del chat son silenciosas. No se necesita acceso al micrófono.",
|
||||||
"Chat": "Chat"
|
"Chat": "Chat",
|
||||||
|
"Attach layer": "Adjuntar capa",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "El chat escrito usa tu clave de la API de OpenAI. El inicio de sesión de ChatGPT solo cubre la voz.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "El chat escrito no está disponible mientras GEV_PREFER_CODEX_OAUTH exija ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Esa solicitud necesitó demasiados pasos. Los cambios ya hechos en el mapa se mantienen.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Esa solicitud necesitó demasiados pasos. Prueba una solicitud más sencilla.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "El proveedor rechazó la clave de API. Revísala en Ajustes del proveedor.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Demasiadas solicitudes de chat. Vuelve a intentarlo en {seconds} s."
|
||||||
}
|
}
|
||||||
@@ -3076,5 +3076,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "Le fournisseur d’IA choisi est partagé par la voix et la discussion.",
|
"The selected AI provider is shared by voice and chat.": "Le fournisseur d’IA choisi est partagé par la voix et la discussion.",
|
||||||
"Show quick commands": "Afficher les commandes rapides",
|
"Show quick commands": "Afficher les commandes rapides",
|
||||||
"Chat replies are silent. No microphone access is needed.": "Les réponses de la discussion sont silencieuses. Aucun accès au micro n’est nécessaire.",
|
"Chat replies are silent. No microphone access is needed.": "Les réponses de la discussion sont silencieuses. Aucun accès au micro n’est nécessaire.",
|
||||||
"Chat": "Discussion"
|
"Chat": "Discussion",
|
||||||
|
"Attach layer": "Joindre la couche",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "Le chat écrit utilise votre clé d’API OpenAI. La connexion ChatGPT ne couvre que la voix.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "Le chat écrit est indisponible tant que GEV_PREFER_CODEX_OAUTH impose ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Cette demande a nécessité trop d’étapes. Les modifications déjà apportées à la carte sont conservées.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Cette demande a nécessité trop d’étapes. Essayez une demande plus simple.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "Le fournisseur a refusé la clé d’API. Vérifiez-la dans les paramètres du fournisseur.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Trop de requêtes de chat. Réessayez dans {seconds} s."
|
||||||
}
|
}
|
||||||
@@ -3076,5 +3076,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "Il fornitore di IA scelto è condiviso da voce e chat.",
|
"The selected AI provider is shared by voice and chat.": "Il fornitore di IA scelto è condiviso da voce e chat.",
|
||||||
"Show quick commands": "Mostra comandi rapidi",
|
"Show quick commands": "Mostra comandi rapidi",
|
||||||
"Chat replies are silent. No microphone access is needed.": "Le risposte della chat sono silenziose. Non serve l’accesso al microfono.",
|
"Chat replies are silent. No microphone access is needed.": "Le risposte della chat sono silenziose. Non serve l’accesso al microfono.",
|
||||||
"Chat": "Chat"
|
"Chat": "Chat",
|
||||||
|
"Attach layer": "Allega livello",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "La chat scritta usa la tua chiave API OpenAI. L’accesso a ChatGPT copre solo la voce.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "La chat scritta non è disponibile finché GEV_PREFER_CODEX_OAUTH richiede ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Questa richiesta ha richiesto troppi passaggi. Le modifiche già fatte alla mappa restano.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Questa richiesta ha richiesto troppi passaggi. Prova una richiesta più semplice.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "Il provider ha rifiutato la chiave API. Controllala nelle impostazioni del provider.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Troppe richieste di chat. Riprova tra {seconds} s."
|
||||||
}
|
}
|
||||||
@@ -2936,5 +2936,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "選んだ AI プロバイダーは音声とチャットで共通です。",
|
"The selected AI provider is shared by voice and chat.": "選んだ AI プロバイダーは音声とチャットで共通です。",
|
||||||
"Show quick commands": "クイックコマンドを表示",
|
"Show quick commands": "クイックコマンドを表示",
|
||||||
"Chat replies are silent. No microphone access is needed.": "チャットの応答は音声なしです。マイクへのアクセスは不要です。",
|
"Chat replies are silent. No microphone access is needed.": "チャットの応答は音声なしです。マイクへのアクセスは不要です。",
|
||||||
"Chat": "チャット"
|
"Chat": "チャット",
|
||||||
|
"Attach layer": "レイヤーを添付",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "テキストチャットは OpenAI API キーを使用します。ChatGPT のサインインは音声のみが対象です。",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "GEV_PREFER_CODEX_OAUTH が ChatGPT OAuth を必須にしている間、テキストチャットは使えません。",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "このリクエストは手順が多すぎました。地図に加えた変更はそのまま残ります。",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "このリクエストは手順が多すぎました。より簡単なリクエストを試してください。",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "プロバイダーが API キーを拒否しました。プロバイダー設定で確認してください。",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "チャットのリクエストが多すぎます。{seconds} 秒後にもう一度お試しください。"
|
||||||
}
|
}
|
||||||
@@ -3076,5 +3076,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "O provedor de IA escolhido é compartilhado por voz e chat.",
|
"The selected AI provider is shared by voice and chat.": "O provedor de IA escolhido é compartilhado por voz e chat.",
|
||||||
"Show quick commands": "Mostrar comandos rápidos",
|
"Show quick commands": "Mostrar comandos rápidos",
|
||||||
"Chat replies are silent. No microphone access is needed.": "As respostas do chat são silenciosas. Não é preciso acesso ao microfone.",
|
"Chat replies are silent. No microphone access is needed.": "As respostas do chat são silenciosas. Não é preciso acesso ao microfone.",
|
||||||
"Chat": "Chat"
|
"Chat": "Chat",
|
||||||
|
"Attach layer": "Anexar camada",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "O chat por texto usa sua chave da API da OpenAI. O login do ChatGPT cobre apenas a voz.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "O chat por texto fica indisponível enquanto GEV_PREFER_CODEX_OAUTH exigir o ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Esse pedido precisou de etapas demais. As alterações já feitas no mapa continuam.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Esse pedido precisou de etapas demais. Tente um pedido mais simples.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "O provedor recusou a chave de API. Verifique-a nas configurações do provedor.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Pedidos de chat demais. Tente de novo em {seconds} s."
|
||||||
}
|
}
|
||||||
@@ -3146,5 +3146,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "Выбранный провайдер ИИ используется для голоса и чата.",
|
"The selected AI provider is shared by voice and chat.": "Выбранный провайдер ИИ используется для голоса и чата.",
|
||||||
"Show quick commands": "Показывать быстрые команды",
|
"Show quick commands": "Показывать быстрые команды",
|
||||||
"Chat replies are silent. No microphone access is needed.": "Ответы в чате беззвучны. Доступ к микрофону не требуется.",
|
"Chat replies are silent. No microphone access is needed.": "Ответы в чате беззвучны. Доступ к микрофону не требуется.",
|
||||||
"Chat": "Чат"
|
"Chat": "Чат",
|
||||||
|
"Attach layer": "Прикрепить слой",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "Текстовый чат использует ваш ключ API OpenAI. Вход в ChatGPT действует только для голоса.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "Текстовый чат недоступен, пока GEV_PREFER_CODEX_OAUTH требует ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Этот запрос потребовал слишком много шагов. Уже внесённые изменения карты сохранятся.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Этот запрос потребовал слишком много шагов. Попробуйте запрос попроще.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "Провайдер отклонил ключ API. Проверьте его в настройках провайдера.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Слишком много запросов чата. Повторите через {seconds} с."
|
||||||
}
|
}
|
||||||
@@ -3146,5 +3146,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "Вибраний постачальник ШІ спільний для голосу й чату.",
|
"The selected AI provider is shared by voice and chat.": "Вибраний постачальник ШІ спільний для голосу й чату.",
|
||||||
"Show quick commands": "Показувати швидкі команди",
|
"Show quick commands": "Показувати швидкі команди",
|
||||||
"Chat replies are silent. No microphone access is needed.": "Відповіді чату беззвучні. Доступ до мікрофона не потрібен.",
|
"Chat replies are silent. No microphone access is needed.": "Відповіді чату беззвучні. Доступ до мікрофона не потрібен.",
|
||||||
"Chat": "Чат"
|
"Chat": "Чат",
|
||||||
|
"Attach layer": "Прикріпити шар",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "Текстовий чат використовує ваш ключ API OpenAI. Вхід у ChatGPT діє лише для голосу.",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "Текстовий чат недоступний, поки GEV_PREFER_CODEX_OAUTH вимагає ChatGPT OAuth.",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "Цей запит потребував забагато кроків. Уже внесені зміни на мапі залишаються.",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "Цей запит потребував забагато кроків. Спробуйте простіший запит.",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "Провайдер відхилив ключ API. Перевірте його в налаштуваннях провайдера.",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "Забагато запитів до чату. Спробуйте ще раз через {seconds} с."
|
||||||
}
|
}
|
||||||
@@ -2936,5 +2936,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "语音和聊天共用所选的 AI 服务商。",
|
"The selected AI provider is shared by voice and chat.": "语音和聊天共用所选的 AI 服务商。",
|
||||||
"Show quick commands": "显示快捷命令",
|
"Show quick commands": "显示快捷命令",
|
||||||
"Chat replies are silent. No microphone access is needed.": "聊天回复不播放声音,无需访问麦克风。",
|
"Chat replies are silent. No microphone access is needed.": "聊天回复不播放声音,无需访问麦克风。",
|
||||||
"Chat": "聊天"
|
"Chat": "聊天",
|
||||||
|
"Attach layer": "附加图层",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "文字聊天使用你的 OpenAI API 密钥。ChatGPT 登录仅适用于语音。",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "GEV_PREFER_CODEX_OAUTH 要求使用 ChatGPT OAuth 时,文字聊天不可用。",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "此请求所需步骤过多。已对地图做出的更改会保留。",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "此请求所需步骤过多。请尝试更简单的请求。",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "提供商拒绝了 API 密钥。请在提供商设置中检查。",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "聊天请求过多。请在 {seconds} 秒后重试。"
|
||||||
}
|
}
|
||||||
@@ -2936,5 +2936,12 @@
|
|||||||
"The selected AI provider is shared by voice and chat.": "所選的 AI 供應商由語音與聊天共用。",
|
"The selected AI provider is shared by voice and chat.": "所選的 AI 供應商由語音與聊天共用。",
|
||||||
"Show quick commands": "顯示快速指令",
|
"Show quick commands": "顯示快速指令",
|
||||||
"Chat replies are silent. No microphone access is needed.": "聊天回覆不會發出聲音,不需要麥克風權限。",
|
"Chat replies are silent. No microphone access is needed.": "聊天回覆不會發出聲音,不需要麥克風權限。",
|
||||||
"Chat": "聊天"
|
"Chat": "聊天",
|
||||||
|
"Attach layer": "附加圖層",
|
||||||
|
"Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.": "文字聊天使用你的 OpenAI API 金鑰。ChatGPT 登入僅適用於語音。",
|
||||||
|
"Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.": "GEV_PREFER_CODEX_OAUTH 要求使用 ChatGPT OAuth 時,文字聊天無法使用。",
|
||||||
|
"That request needed too many steps. Changes already made to the map stay in place.": "此請求所需步驟過多。已對地圖做出的變更會保留。",
|
||||||
|
"That request needed too many steps. Try a simpler request.": "此請求所需步驟過多。請嘗試更簡單的請求。",
|
||||||
|
"The provider rejected the API key. Check it in Provider Settings.": "供應商拒絕了 API 金鑰。請在供應商設定中檢查。",
|
||||||
|
"Too many chat requests. Try again in {seconds} s.": "聊天請求過多。請在 {seconds} 秒後重試。"
|
||||||
}
|
}
|
||||||
@@ -25,6 +25,8 @@ const PROVIDER_FIELDS = [
|
|||||||
const RATE_LIMIT_FIELDS = [
|
const RATE_LIMIT_FIELDS = [
|
||||||
'GEV_RATELIMIT_OPENAI_PER_MIN',
|
'GEV_RATELIMIT_OPENAI_PER_MIN',
|
||||||
'GEV_RATELIMIT_GOOGLE_PER_MIN',
|
'GEV_RATELIMIT_GOOGLE_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_OPENAI_CHAT_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_GEMINI_CHAT_PER_MIN',
|
||||||
];
|
];
|
||||||
const APP_VALUE_FIELDS = [...PROVIDER_FIELDS, ...RATE_LIMIT_FIELDS];
|
const APP_VALUE_FIELDS = [...PROVIDER_FIELDS, ...RATE_LIMIT_FIELDS];
|
||||||
const SHARING_FIELDS = [
|
const SHARING_FIELDS = [
|
||||||
|
|||||||
+362
-33
@@ -83,7 +83,7 @@ const providers = {
|
|||||||
}),
|
}),
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
async function fixture(t, provider, fetchImpl) {
|
async function fixture(t, provider, fetchImpl, overrides = {}) {
|
||||||
const calls = [];
|
const calls = [];
|
||||||
const handler = provider.create({
|
const handler = provider.create({
|
||||||
tools,
|
tools,
|
||||||
@@ -91,12 +91,14 @@ async function fixture(t, provider, fetchImpl) {
|
|||||||
resolveRateLimit: () => '0',
|
resolveRateLimit: () => '0',
|
||||||
resolveHost: () => '',
|
resolveHost: () => '',
|
||||||
resolveAllowedHosts: () => '',
|
resolveAllowedHosts: () => '',
|
||||||
|
environment: {},
|
||||||
fetchImpl:
|
fetchImpl:
|
||||||
fetchImpl ||
|
fetchImpl ||
|
||||||
(async (url, options) => {
|
(async (url, options) => {
|
||||||
calls.push({ url, body: JSON.parse(options.body) });
|
calls.push({ url, body: JSON.parse(options.body), options });
|
||||||
return Response.json(provider.reply(true));
|
return Response.json(provider.reply(true));
|
||||||
}),
|
}),
|
||||||
|
...overrides,
|
||||||
});
|
});
|
||||||
const server = http.createServer((req, res) => void handler(req, res));
|
const server = http.createServer((req, res) => void handler(req, res));
|
||||||
server.listen(0, '127.0.0.1');
|
server.listen(0, '127.0.0.1');
|
||||||
@@ -109,6 +111,24 @@ async function fixture(t, provider, fetchImpl) {
|
|||||||
return {
|
return {
|
||||||
calls,
|
calls,
|
||||||
origin,
|
origin,
|
||||||
|
raw: (options, body) =>
|
||||||
|
new Promise((resolve, reject) => {
|
||||||
|
const request = http.request(origin, options, (response) => {
|
||||||
|
let text = '';
|
||||||
|
response.setEncoding('utf8');
|
||||||
|
response.on('data', (chunk) => (text += chunk));
|
||||||
|
response.on('end', () =>
|
||||||
|
resolve({
|
||||||
|
status: response.statusCode,
|
||||||
|
headers: response.headers,
|
||||||
|
body: text ? JSON.parse(text) : null,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
request.on('error', reject);
|
||||||
|
if (body === undefined) request.end();
|
||||||
|
else request.end(body);
|
||||||
|
}),
|
||||||
send: (input) =>
|
send: (input) =>
|
||||||
fetch(origin, {
|
fetch(origin, {
|
||||||
method: 'POST',
|
method: 'POST',
|
||||||
@@ -190,38 +210,50 @@ for (const [name, provider] of Object.entries(providers)) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
test(`${name} client disconnect aborts upstream and cancels a late body`, async (t) => {
|
test(
|
||||||
let finish,
|
`${name} client disconnect aborts upstream and cancels a late body`,
|
||||||
signal,
|
{
|
||||||
cancelled = 0,
|
timeout: 5000,
|
||||||
ready;
|
},
|
||||||
const acquired = new Promise((resolve) => (ready = resolve));
|
async (t) => {
|
||||||
const f = await fixture(t, provider, (_url, options) => {
|
let finish,
|
||||||
signal = options.signal;
|
signal,
|
||||||
ready();
|
cancelled = 0,
|
||||||
return new Promise((resolve) => (finish = resolve));
|
ready;
|
||||||
});
|
const acquired = new Promise((resolve) => (ready = resolve));
|
||||||
const request = http.request(f.origin, {
|
// A deadline this far out cannot be what aborts the upstream call.
|
||||||
method: 'POST',
|
const f = await fixture(
|
||||||
headers: { Origin: f.origin, 'Content-Type': 'application/json' },
|
t,
|
||||||
});
|
provider,
|
||||||
request.on('error', () => {});
|
(_url, options) => {
|
||||||
request.end(JSON.stringify(provider.input));
|
signal = options.signal;
|
||||||
await acquired;
|
ready();
|
||||||
const aborted = once(signal, 'abort');
|
return new Promise((resolve) => (finish = resolve));
|
||||||
request.destroy();
|
|
||||||
await aborted;
|
|
||||||
assert.equal(signal.aborted, true);
|
|
||||||
finish({
|
|
||||||
body: {
|
|
||||||
cancel: async () => {
|
|
||||||
cancelled++;
|
|
||||||
},
|
},
|
||||||
},
|
{ timeoutMs: 1e9 },
|
||||||
});
|
);
|
||||||
await new Promise((resolve) => setImmediate(resolve));
|
const request = http.request(f.origin, {
|
||||||
assert.ok(cancelled > 0);
|
method: 'POST',
|
||||||
});
|
headers: { Origin: f.origin, 'Content-Type': 'application/json' },
|
||||||
|
});
|
||||||
|
request.on('error', () => {});
|
||||||
|
request.end(JSON.stringify(provider.input));
|
||||||
|
await acquired;
|
||||||
|
const aborted = once(signal, 'abort');
|
||||||
|
request.destroy();
|
||||||
|
await aborted;
|
||||||
|
assert.equal(signal.aborted, true);
|
||||||
|
finish({
|
||||||
|
body: {
|
||||||
|
cancel: async () => {
|
||||||
|
cancelled++;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await new Promise((resolve) => setImmediate(resolve));
|
||||||
|
assert.ok(cancelled > 0);
|
||||||
|
},
|
||||||
|
);
|
||||||
test(`${name} invalid upstream content still returns reported usage`, async (t) => {
|
test(`${name} invalid upstream content still returns reported usage`, async (t) => {
|
||||||
const f = await fixture(t, provider, async () =>
|
const f = await fixture(t, provider, async () =>
|
||||||
Response.json({
|
Response.json({
|
||||||
@@ -233,6 +265,246 @@ for (const [name, provider] of Object.entries(providers)) {
|
|||||||
assert.equal(response.status, 502);
|
assert.equal(response.status, 502);
|
||||||
assert.equal((await response.json()).usage.totalTokens, 10);
|
assert.equal((await response.json()).usage.totalTokens, 10);
|
||||||
});
|
});
|
||||||
|
test(`${name} route refuses bad methods, media types and bodies before upstream`, async (t) => {
|
||||||
|
const f = await fixture(t, provider, undefined, { timeoutMs: 2000 });
|
||||||
|
const json = { Origin: f.origin, 'Content-Type': 'application/json' };
|
||||||
|
const get = await f.raw({ method: 'GET', headers: { Origin: f.origin } });
|
||||||
|
assert.equal(get.status, 405);
|
||||||
|
assert.equal(get.headers.allow, 'POST');
|
||||||
|
const text = await f.raw(
|
||||||
|
{
|
||||||
|
method: 'POST',
|
||||||
|
headers: { Origin: f.origin, 'Content-Type': 'text/plain' },
|
||||||
|
},
|
||||||
|
JSON.stringify(provider.input),
|
||||||
|
);
|
||||||
|
assert.equal(text.status, 415);
|
||||||
|
// The declared length alone refuses; the short body never completes it.
|
||||||
|
const declared = await f.raw(
|
||||||
|
{ method: 'POST', headers: { ...json, 'Content-Length': 600 * 1024 } },
|
||||||
|
'{}',
|
||||||
|
);
|
||||||
|
assert.equal(declared.status, 413);
|
||||||
|
const streamed = await f.raw(
|
||||||
|
{ method: 'POST', headers: { ...json, 'Transfer-Encoding': 'chunked' } },
|
||||||
|
JSON.stringify({ ...provider.input, padding: 'x'.repeat(600 * 1024) }),
|
||||||
|
);
|
||||||
|
assert.equal(streamed.status, 413);
|
||||||
|
assert.equal(f.calls.length, 0);
|
||||||
|
});
|
||||||
|
test(`${name} route names a missing key without calling upstream`, async (t) => {
|
||||||
|
const f = await fixture(t, provider, undefined, {
|
||||||
|
resolveApiKey: () => ' ',
|
||||||
|
});
|
||||||
|
const response = await f.send();
|
||||||
|
assert.equal(response.status, 503);
|
||||||
|
assert.deepEqual(await response.json(), {
|
||||||
|
error: 'Add an API key in Provider Settings',
|
||||||
|
code: 'API_KEY_MISSING',
|
||||||
|
});
|
||||||
|
assert.equal(f.calls.length, 0);
|
||||||
|
});
|
||||||
|
test(
|
||||||
|
`${name} upstream deadline answers 504 and aborts the call`,
|
||||||
|
{
|
||||||
|
timeout: 5000,
|
||||||
|
},
|
||||||
|
async (t) => {
|
||||||
|
let signal;
|
||||||
|
const f = await fixture(
|
||||||
|
t,
|
||||||
|
provider,
|
||||||
|
(_url, options) => {
|
||||||
|
signal = options.signal;
|
||||||
|
return new Promise(() => {});
|
||||||
|
},
|
||||||
|
{ timeoutMs: 50 },
|
||||||
|
);
|
||||||
|
const response = await f.send();
|
||||||
|
assert.equal(response.status, 504);
|
||||||
|
assert.equal(signal.aborted, true);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
test(`${name} route refuses redirects and unexpected destinations`, async (t) => {
|
||||||
|
const f = await fixture(t, provider);
|
||||||
|
assert.equal((await f.send()).status, 200);
|
||||||
|
assert.equal(f.calls[0].options.redirect, 'error');
|
||||||
|
const moved = await fixture(t, provider, async () => {
|
||||||
|
const response = Response.json(provider.reply());
|
||||||
|
Object.defineProperty(response, 'url', {
|
||||||
|
value: 'https://example.test/',
|
||||||
|
});
|
||||||
|
return response;
|
||||||
|
});
|
||||||
|
assert.equal((await moved.send()).status, 502);
|
||||||
|
const redirected = await fixture(t, provider, async () => {
|
||||||
|
const response = Response.json(provider.reply());
|
||||||
|
Object.defineProperty(response, 'redirected', { value: true });
|
||||||
|
return response;
|
||||||
|
});
|
||||||
|
assert.equal((await redirected.send()).status, 502);
|
||||||
|
});
|
||||||
|
test(`${name} route caps the upstream response and never echoes the key`, async (t) => {
|
||||||
|
const large = await fixture(t, provider, async () =>
|
||||||
|
Response.json({ ...provider.reply(), padding: 'x'.repeat(600 * 1024) }),
|
||||||
|
);
|
||||||
|
assert.equal((await large.send()).status, 502);
|
||||||
|
const echo = provider.reply();
|
||||||
|
if (name === 'gemini')
|
||||||
|
echo.candidates[0].content.parts = [{ text: 'key fixture-only' }];
|
||||||
|
else
|
||||||
|
echo.choices[0].message = {
|
||||||
|
role: 'assistant',
|
||||||
|
content: 'key fixture-only',
|
||||||
|
};
|
||||||
|
const echoed = await fixture(t, provider, async () => Response.json(echo));
|
||||||
|
const response = await echoed.send();
|
||||||
|
assert.equal(response.status, 502);
|
||||||
|
assert.ok(!(await response.text()).includes('fixture-only'));
|
||||||
|
});
|
||||||
|
test(`${name} upstream 429 and 503 keep their status and a sane Retry-After`, async (t) => {
|
||||||
|
for (const [status, retryAfter, expected] of [
|
||||||
|
[429, '7', '7'],
|
||||||
|
[503, '7200', '3600'],
|
||||||
|
[429, 'Wed, 21 Oct 2015 07:28:00 GMT', undefined],
|
||||||
|
[500, '7', undefined],
|
||||||
|
]) {
|
||||||
|
const f = await fixture(t, provider, async () =>
|
||||||
|
Response.json({}, { status, headers: { 'Retry-After': retryAfter } }),
|
||||||
|
);
|
||||||
|
const response = await f.send();
|
||||||
|
assert.equal(response.status, status === 500 ? 502 : status);
|
||||||
|
assert.equal(response.headers.get('retry-after') ?? undefined, expected);
|
||||||
|
assert.deepEqual(await response.json(), {
|
||||||
|
error: 'Could not connect. Try again.',
|
||||||
|
...(status === 429 ? { code: 'RATE_LIMITED' } : {}),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test(`${name} rejected key cools down until it changes or the window ends`, async (t) => {
|
||||||
|
for (const status of [401, 403]) {
|
||||||
|
let clock = 1_000_000;
|
||||||
|
let key = 'fixture-only';
|
||||||
|
let upstream = 0;
|
||||||
|
const f = await fixture(
|
||||||
|
t,
|
||||||
|
provider,
|
||||||
|
async () => {
|
||||||
|
upstream++;
|
||||||
|
return Response.json({}, { status });
|
||||||
|
},
|
||||||
|
{ now: () => clock, resolveApiKey: () => key },
|
||||||
|
);
|
||||||
|
assert.equal((await f.send()).status, 502);
|
||||||
|
const cooling = await f.send();
|
||||||
|
assert.equal(cooling.status, 502);
|
||||||
|
assert.equal(cooling.headers.get('retry-after'), '60');
|
||||||
|
assert.equal(upstream, 1, 'a rejected key must not reach upstream again');
|
||||||
|
key = 'replacement-key';
|
||||||
|
assert.equal((await f.send()).status, 502);
|
||||||
|
assert.equal(upstream, 2, 'a new key is tried at once');
|
||||||
|
clock += 59_000;
|
||||||
|
assert.equal((await f.send()).status, 502);
|
||||||
|
assert.equal(upstream, 2);
|
||||||
|
clock += 1_000;
|
||||||
|
assert.equal((await f.send()).status, 502);
|
||||||
|
assert.equal(upstream, 3, 'the cooldown ends after a minute');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
test(`${name} an invalid-key 400 cools down like 401, and every refusal names the key`, async (t) => {
|
||||||
|
let upstream = 0;
|
||||||
|
let body = {
|
||||||
|
error: {
|
||||||
|
code: 400,
|
||||||
|
status: 'INVALID_ARGUMENT',
|
||||||
|
details: [{ reason: 'API_KEY_INVALID' }],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const f = await fixture(t, provider, async () => {
|
||||||
|
upstream++;
|
||||||
|
return Response.json(body, { status: 400 });
|
||||||
|
});
|
||||||
|
const first = await f.send();
|
||||||
|
assert.equal(first.status, 502);
|
||||||
|
assert.equal((await first.json()).code, 'API_KEY_REJECTED');
|
||||||
|
const cooling = await f.send();
|
||||||
|
assert.equal((await cooling.json()).code, 'API_KEY_REJECTED');
|
||||||
|
assert.equal(upstream, 1, 'the rejected key is not sent again');
|
||||||
|
// Any other 400 is a plain failure and does not cool the key down.
|
||||||
|
body = { error: { code: 400, status: 'INVALID_ARGUMENT' } };
|
||||||
|
const plain = await fixture(t, provider, async () => {
|
||||||
|
upstream++;
|
||||||
|
return Response.json(body, { status: 400 });
|
||||||
|
});
|
||||||
|
const other = await plain.send();
|
||||||
|
assert.equal(other.status, 502);
|
||||||
|
assert.equal((await other.json()).code, undefined);
|
||||||
|
await plain.send();
|
||||||
|
assert.equal(upstream, 3);
|
||||||
|
});
|
||||||
|
test(`${name} throttled replies say they are rate limited`, async (t) => {
|
||||||
|
const local = await fixture(t, provider, undefined, {
|
||||||
|
resolveRateLimit: () => '1',
|
||||||
|
});
|
||||||
|
await local.send();
|
||||||
|
const limited = await local.send();
|
||||||
|
assert.equal(limited.status, 429);
|
||||||
|
assert.equal((await limited.json()).code, 'RATE_LIMITED');
|
||||||
|
const upstream = await fixture(t, provider, async () =>
|
||||||
|
Response.json({}, { status: 429, headers: { 'Retry-After': '7' } }),
|
||||||
|
);
|
||||||
|
const busy = await upstream.send();
|
||||||
|
assert.equal(busy.status, 429);
|
||||||
|
assert.equal(busy.headers.get('retry-after'), '7');
|
||||||
|
assert.equal((await busy.json()).code, 'RATE_LIMITED');
|
||||||
|
});
|
||||||
|
test(`${name} chat throttle follows the provider throttle unless set`, async (t) => {
|
||||||
|
const burst = async (overrides, count) => {
|
||||||
|
const f = await fixture(t, provider, undefined, overrides);
|
||||||
|
const statuses = [];
|
||||||
|
for (let i = 0; i < count; i++) statuses.push((await f.send()).status);
|
||||||
|
return { statuses, calls: f.calls.length, last: f };
|
||||||
|
};
|
||||||
|
const unset = await burst(
|
||||||
|
{
|
||||||
|
resolveRateLimit: () => undefined,
|
||||||
|
resolveProviderRateLimit: () => undefined,
|
||||||
|
},
|
||||||
|
31,
|
||||||
|
);
|
||||||
|
assert.equal(
|
||||||
|
unset.calls,
|
||||||
|
30,
|
||||||
|
'the default cap is the provider default, 30',
|
||||||
|
);
|
||||||
|
assert.equal(unset.statuses.at(-1), 429);
|
||||||
|
const follows = await burst(
|
||||||
|
{ resolveRateLimit: () => '', resolveProviderRateLimit: () => '2' },
|
||||||
|
3,
|
||||||
|
);
|
||||||
|
assert.deepEqual(follows.statuses, [200, 200, 429]);
|
||||||
|
const own = await burst(
|
||||||
|
{ resolveRateLimit: () => '3', resolveProviderRateLimit: () => '2' },
|
||||||
|
4,
|
||||||
|
);
|
||||||
|
assert.deepEqual(own.statuses, [200, 200, 200, 429]);
|
||||||
|
const off = await burst(
|
||||||
|
{ resolveRateLimit: () => '0', resolveProviderRateLimit: () => '2' },
|
||||||
|
4,
|
||||||
|
);
|
||||||
|
assert.deepEqual(off.statuses, [200, 200, 200, 200]);
|
||||||
|
const capped = await fixture(t, provider, undefined, {
|
||||||
|
resolveRateLimit: () => '1',
|
||||||
|
});
|
||||||
|
assert.equal((await capped.send()).status, 200);
|
||||||
|
const refused = await capped.send();
|
||||||
|
assert.equal(refused.status, 429);
|
||||||
|
assert.equal(refused.headers.get('retry-after'), '60');
|
||||||
|
assert.deepEqual(await refused.json(), {
|
||||||
|
error: 'Could not connect. Try again.',
|
||||||
|
code: 'RATE_LIMITED',
|
||||||
|
});
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
test('text adapts speech wording while retaining shared Radio and annotation policies', () => {
|
test('text adapts speech wording while retaining shared Radio and annotation policies', () => {
|
||||||
@@ -259,6 +531,63 @@ test('every spoken phrase chat rewrites still occurs in the voice directives', (
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('chat throttles read their own variables and fall back per provider', async (t) => {
|
||||||
|
const names = [
|
||||||
|
'GEV_RATELIMIT_OPENAI_CHAT_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_OPENAI_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_GEMINI_CHAT_PER_MIN',
|
||||||
|
'GEV_RATELIMIT_GEMINI_PER_MIN',
|
||||||
|
];
|
||||||
|
const saved = Object.fromEntries(names.map((key) => [key, process.env[key]]));
|
||||||
|
t.after(() => {
|
||||||
|
for (const [key, value] of Object.entries(saved))
|
||||||
|
if (value === undefined) delete process.env[key];
|
||||||
|
else process.env[key] = value;
|
||||||
|
});
|
||||||
|
const allowed = async (provider, count) => {
|
||||||
|
const f = await fixture(t, provider, undefined, {
|
||||||
|
resolveRateLimit: undefined,
|
||||||
|
});
|
||||||
|
let ok = 0;
|
||||||
|
for (let i = 0; i < count; i++) if ((await f.send()).status === 200) ok++;
|
||||||
|
return ok;
|
||||||
|
};
|
||||||
|
for (const key of names) delete process.env[key];
|
||||||
|
process.env.GEV_RATELIMIT_OPENAI_PER_MIN = '2';
|
||||||
|
process.env.GEV_RATELIMIT_GEMINI_PER_MIN = '3';
|
||||||
|
assert.equal(await allowed(providers.openai, 5), 2);
|
||||||
|
assert.equal(await allowed(providers.gemini, 5), 3);
|
||||||
|
process.env.GEV_RATELIMIT_OPENAI_CHAT_PER_MIN = '4';
|
||||||
|
process.env.GEV_RATELIMIT_GEMINI_CHAT_PER_MIN = '0';
|
||||||
|
assert.equal(await allowed(providers.openai, 6), 4);
|
||||||
|
assert.equal(await allowed(providers.gemini, 6), 6);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the ChatGPT OAuth lock refuses OpenAI chat but not Gemini chat', async (t) => {
|
||||||
|
for (const [value, code] of [
|
||||||
|
['true', 'CODEX_OAUTH_REQUIRED'],
|
||||||
|
['maybe', 'CODEX_OAUTH_REQUIRED'],
|
||||||
|
['false', undefined],
|
||||||
|
]) {
|
||||||
|
const environment = { GEV_PREFER_CODEX_OAUTH: value };
|
||||||
|
const openai = await fixture(t, providers.openai, undefined, {
|
||||||
|
environment,
|
||||||
|
});
|
||||||
|
const response = await openai.send();
|
||||||
|
const body = await response.json();
|
||||||
|
assert.equal(response.status, code ? 503 : 200);
|
||||||
|
assert.equal(body.code, code);
|
||||||
|
if (code) {
|
||||||
|
assert.equal(typeof body.error, 'string');
|
||||||
|
assert.equal(openai.calls.length, 0, 'the API key must not be billed');
|
||||||
|
}
|
||||||
|
const gemini = await fixture(t, providers.gemini, undefined, {
|
||||||
|
environment,
|
||||||
|
});
|
||||||
|
assert.equal((await gemini.send()).status, 200);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
test('OpenAI empty tool arrays are removed before history is returned', async (t) => {
|
test('OpenAI empty tool arrays are removed before history is returned', async (t) => {
|
||||||
const f = await fixture(t, providers.openai, async () =>
|
const f = await fixture(t, providers.openai, async () =>
|
||||||
Response.json({
|
Response.json({
|
||||||
|
|||||||
@@ -85,7 +85,6 @@
|
|||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
color: var(--text-primary);
|
color: var(--text-primary);
|
||||||
background: rgba(7, 22, 31, 0.97);
|
background: rgba(7, 22, 31, 0.97);
|
||||||
backdrop-filter: blur(24px);
|
|
||||||
border: 1px solid rgba(54, 220, 255, 0.3);
|
border: 1px solid rgba(54, 220, 255, 0.3);
|
||||||
border-radius: 0.9rem;
|
border-radius: 0.9rem;
|
||||||
box-shadow: 0 1rem 3rem #0008;
|
box-shadow: 0 1rem 3rem #0008;
|
||||||
@@ -181,6 +180,12 @@
|
|||||||
#gev-chat-empty p {
|
#gev-chat-empty p {
|
||||||
margin: 0 0 1rem;
|
margin: 0 0 1rem;
|
||||||
}
|
}
|
||||||
|
#gev-chat-auth-note {
|
||||||
|
margin: 0;
|
||||||
|
padding: 0 1rem 0.3rem;
|
||||||
|
color: var(--text-secondary);
|
||||||
|
font-size: 0.7rem;
|
||||||
|
}
|
||||||
#gev-chat-empty button {
|
#gev-chat-empty button {
|
||||||
border: 1px solid var(--glass-border);
|
border: 1px solid var(--glass-border);
|
||||||
border-radius: 1rem;
|
border-radius: 1rem;
|
||||||
@@ -469,6 +474,19 @@
|
|||||||
background: #10232d;
|
background: #10232d;
|
||||||
font: inherit;
|
font: inherit;
|
||||||
}
|
}
|
||||||
|
#gev-chat-layer-row {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.4rem;
|
||||||
|
margin-bottom: 0.4rem;
|
||||||
|
}
|
||||||
|
#gev-chat-layer-row select {
|
||||||
|
flex: 1 1 auto;
|
||||||
|
min-width: 0;
|
||||||
|
}
|
||||||
|
#gev-chat-attach-menu button:disabled {
|
||||||
|
opacity: 0.5;
|
||||||
|
cursor: default;
|
||||||
|
}
|
||||||
#gev-chat-attach-menu p {
|
#gev-chat-attach-menu p {
|
||||||
margin-bottom: 0;
|
margin-bottom: 0;
|
||||||
font-size: 0.65rem;
|
font-size: 0.65rem;
|
||||||
@@ -1052,7 +1070,19 @@
|
|||||||
linear-gradient(#fff 0 0) content-box,
|
linear-gradient(#fff 0 0) content-box,
|
||||||
linear-gradient(#fff 0 0);
|
linear-gradient(#fff 0 0);
|
||||||
mask-composite: exclude;
|
mask-composite: exclude;
|
||||||
animation: gev-border-orbit 4s linear infinite;
|
}
|
||||||
|
/* The edge orbits only while voice is live or a chat reply is on its way;
|
||||||
|
an idle or minimized chat keeps it still. Touch screens and reduced
|
||||||
|
motion keep it still too. */
|
||||||
|
@media (prefers-reduced-motion: no-preference) and (pointer: fine) {
|
||||||
|
#command-dock
|
||||||
|
> #gev-voice-control:is(
|
||||||
|
[data-assistant-mode='voice'],
|
||||||
|
[data-assistant-mode='chat'][data-chat-busy='true']
|
||||||
|
)
|
||||||
|
> .gev-active-border {
|
||||||
|
animation: gev-border-orbit 4s linear infinite;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@keyframes gev-border-orbit {
|
@keyframes gev-border-orbit {
|
||||||
to {
|
to {
|
||||||
@@ -1433,6 +1463,13 @@ body[data-assistant-docked='true'] #right-context-rail {
|
|||||||
background: var(--assistant-glass);
|
background: var(--assistant-glass);
|
||||||
border-color: var(--assistant-glass-border);
|
border-color: var(--assistant-glass-border);
|
||||||
box-shadow: var(--assistant-glass-shadow);
|
box-shadow: var(--assistant-glass-shadow);
|
||||||
|
}
|
||||||
|
/* The mini reply sits inside the chat panel, whose own blur makes it the
|
||||||
|
backdrop root: a blur on the reply could not reach the map. */
|
||||||
|
:root:not([data-ui-theme='cyber'])
|
||||||
|
#command-dock
|
||||||
|
#gev-voice-control
|
||||||
|
:is(#gev-chat-panel, #gev-voice-panel, #gev-voice-panel[data-minimized]) {
|
||||||
-webkit-backdrop-filter: blur(24px) saturate(1.4);
|
-webkit-backdrop-filter: blur(24px) saturate(1.4);
|
||||||
backdrop-filter: blur(24px) saturate(1.4);
|
backdrop-filter: blur(24px) saturate(1.4);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,6 +14,8 @@ const store = useVoiceStore();
|
|||||||
const { controls, chat: messages } = storeToRefs(store);
|
const { controls, chat: messages } = storeToRefs(store);
|
||||||
const emit = (name, ...args) => controls.value?.emit(name, ...args);
|
const emit = (name, ...args) => controls.value?.emit(name, ...args);
|
||||||
const list = ref(null);
|
const list = ref(null);
|
||||||
|
const layerSelect = ref(null);
|
||||||
|
const layerChoice = ref('');
|
||||||
|
|
||||||
// The input holds what the person types until its owner writes it (history
|
// The input holds what the person types until its owner writes it (history
|
||||||
// recall, a suggestion, clearing after send): only then is it replaced.
|
// recall, a suggestion, clearing after send): only then is it replaced.
|
||||||
@@ -27,12 +29,31 @@ const vOwnedDraft = {
|
|||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
// Keep the newest message in view as the conversation grows.
|
// Keep the newest message in view as the conversation grows, while the
|
||||||
watch(messages, () =>
|
// reader is at the bottom: one who scrolled up to read stays there until
|
||||||
nextTick(() => {
|
// they send. The position is read on scroll, not on every reveal frame.
|
||||||
|
let stuck = true;
|
||||||
|
const onListScroll = () => {
|
||||||
|
const el = list.value;
|
||||||
|
if (el) stuck = el.scrollHeight - el.scrollTop - el.clientHeight < 24;
|
||||||
|
};
|
||||||
|
watch(messages, (next, previous) => {
|
||||||
|
const sent =
|
||||||
|
next.length > (previous?.length ?? 0) && next.at(-1)?.role === 'user';
|
||||||
|
if (!stuck && !sent) return;
|
||||||
|
void nextTick(() => {
|
||||||
if (list.value) list.value.scrollTop = list.value.scrollHeight;
|
if (list.value) list.value.scrollTop = list.value.scrollHeight;
|
||||||
}),
|
});
|
||||||
);
|
});
|
||||||
|
|
||||||
|
// Arrow keys only move the layer choice (a closed select changes on each
|
||||||
|
// press on Windows and Linux); the Attach button or Enter attaches it.
|
||||||
|
function attachLayer() {
|
||||||
|
const value = layerChoice.value;
|
||||||
|
layerChoice.value = '';
|
||||||
|
if (layerSelect.value) layerSelect.value.value = '';
|
||||||
|
if (value) emit('chat-layer', value);
|
||||||
|
}
|
||||||
|
|
||||||
const quickPrompts = () => [t('Fly to Tokyo'), t('Show live flights')];
|
const quickPrompts = () => [t('Fly to Tokyo'), t('Show live flights')];
|
||||||
</script>
|
</script>
|
||||||
@@ -43,11 +64,11 @@ const quickPrompts = () => [t('Fly to Tokyo'), t('Show live flights')];
|
|||||||
<div><strong>{{ t('GEV CHAT') }}</strong><p>{{ t('Ask about the map. Tell it where to go.') }}</p></div>
|
<div><strong>{{ t('GEV CHAT') }}</strong><p>{{ t('Ask about the map. Tell it where to go.') }}</p></div>
|
||||||
<div id="gev-chat-header-actions">
|
<div id="gev-chat-header-actions">
|
||||||
<PanelLayoutButtons :docked="props.view.layout.docked" :minimized="props.view.chat.minimized" @dock="emit('layout-dock')" @minimize="emit('chat-minimize')" />
|
<PanelLayoutButtons :docked="props.view.layout.docked" :minimized="props.view.chat.minimized" @dock="emit('layout-dock')" @minimize="emit('chat-minimize')" />
|
||||||
<button v-if="props.view.panel" id="gev-voice-settings-button" class="gev-voice-settings-button" type="button" :aria-label="t('Voice Settings')" aria-controls="gev-voice-panel" :aria-expanded="String(props.view.voicePanel.settingsExpanded)" @click="emit('settings-button')"><span class="material-symbols-outlined gev-voice-settings-icon" aria-hidden="true">settings</span></button>
|
<button v-if="props.view.panel" id="gev-voice-settings-button" class="gev-voice-settings-button" type="button" :aria-label="t('Settings')" :title="t('Settings')" aria-controls="gev-voice-panel" :aria-expanded="String(props.view.voicePanel.settingsExpanded)" @click="emit('settings-button')"><span class="material-symbols-outlined gev-voice-settings-icon" aria-hidden="true">settings</span></button>
|
||||||
<button id="gev-chat-close" type="button" :aria-label="t('Close chat')" @click="emit('chat-close')">×</button>
|
<button id="gev-chat-close" type="button" :aria-label="t('Close chat')" @click="emit('chat-close')">×</button>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
<div id="gev-chat-messages" ref="list" role="log" :aria-label="t('Conversation')" aria-live="polite">
|
<div id="gev-chat-messages" ref="list" role="log" :aria-label="t('Conversation')" aria-live="polite" :aria-busy="props.view.chat.busy || props.view.chat.revealing ? 'true' : undefined" @scroll.passive="onListScroll">
|
||||||
<div id="gev-chat-empty" :hidden="!props.view.chat.empty">
|
<div id="gev-chat-empty" :hidden="!props.view.chat.empty">
|
||||||
<svg viewBox="0 0 24 24" width="32" height="32" fill="none" stroke="currentColor" stroke-width="1" aria-hidden="true"><circle cx="12" cy="12" r="10"></circle><ellipse cx="12" cy="12" rx="4" ry="10"></ellipse><path d="M2 12h20"></path></svg>
|
<svg viewBox="0 0 24 24" width="32" height="32" fill="none" stroke="currentColor" stroke-width="1" aria-hidden="true"><circle cx="12" cy="12" r="10"></circle><ellipse cx="12" cy="12" rx="4" ry="10"></ellipse><path d="M2 12h20"></path></svg>
|
||||||
<h3>{{ t('Explore with a question') }}</h3>
|
<h3>{{ t('Explore with a question') }}</h3>
|
||||||
@@ -59,15 +80,20 @@ const quickPrompts = () => [t('Fly to Tokyo'), t('Show live flights')];
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<p id="gev-chat-status" role="status">{{ props.view.chat.status }}</p>
|
<p id="gev-chat-status" role="status">{{ props.view.chat.status }}</p>
|
||||||
|
<!-- Stays under the status for the whole conversation. -->
|
||||||
|
<p v-if="props.view.chat.authNote" id="gev-chat-auth-note">{{ props.view.chat.authNote }}</p>
|
||||||
<div id="gev-chat-attachments" :aria-label="t('Attachments')" :hidden="!props.view.chat.attachments.length">
|
<div id="gev-chat-attachments" :aria-label="t('Attachments')" :hidden="!props.view.chat.attachments.length">
|
||||||
<button v-for="item in props.view.chat.attachments" :key="item.key" type="button" :disabled="props.view.chat.locked" :aria-label="`${t('Remove attachment')}: ${item.name}`" @click="emit('chat-attachment-remove', item.key)">{{ item.name }} ×</button>
|
<button v-for="item in props.view.chat.attachments" :key="item.key" type="button" :disabled="props.view.chat.locked" :aria-label="`${t('Remove attachment')}: ${item.name}`" @click="emit('chat-attachment-remove', item.key)">{{ item.name }} ×</button>
|
||||||
</div>
|
</div>
|
||||||
<div id="gev-chat-attach-menu" :hidden="!props.view.chat.attachMenuOpen">
|
<div id="gev-chat-attach-menu" :hidden="!props.view.chat.attachMenuOpen" @keydown="emit('chat-attach-keydown', $event)">
|
||||||
<label for="gev-chat-layer">{{ t('Attach a map layer') }}</label>
|
<label for="gev-chat-layer">{{ t('Attach a map layer') }}</label>
|
||||||
<select id="gev-chat-layer" :disabled="props.view.chat.locked" @change="emit('chat-layer', $event.target.value); $event.target.value = ''">
|
<div id="gev-chat-layer-row">
|
||||||
<option value="">{{ t('Choose a layer…') }}</option>
|
<select id="gev-chat-layer" ref="layerSelect" :disabled="props.view.chat.locked" @change="layerChoice = $event.target.value" @keydown.enter.prevent="attachLayer()">
|
||||||
<option v-for="layer in props.view.chat.layers" :key="layer.value" :value="layer.value">{{ layer.label }}</option>
|
<option value="">{{ t('Choose a layer…') }}</option>
|
||||||
</select>
|
<option v-for="layer in props.view.chat.layers" :key="layer.value" :value="layer.value">{{ layer.label }}</option>
|
||||||
|
</select>
|
||||||
|
<button id="gev-chat-layer-attach" type="button" :disabled="props.view.chat.locked || !layerChoice" @click="attachLayer()">{{ t('Attach layer') }}</button>
|
||||||
|
</div>
|
||||||
<button id="gev-chat-file-button" type="button" :disabled="props.view.chat.locked" @click="emit('chat-file-button')">{{ t('Attach a file') }}</button>
|
<button id="gev-chat-file-button" type="button" :disabled="props.view.chat.locked" @click="emit('chat-file-button')">{{ t('Attach a file') }}</button>
|
||||||
<p>{{ t('TXT, Markdown, CSV, JSON or GeoJSON · up to 10 KB each. Attachments are sent to your AI provider with your message.') }}</p>
|
<p>{{ t('TXT, Markdown, CSV, JSON or GeoJSON · up to 10 KB each. Attachments are sent to your AI provider with your message.') }}</p>
|
||||||
<input id="gev-chat-file" type="file" accept=".txt,.md,.csv,.json,.geojson" multiple hidden @change="emit('chat-files', $event.target)" />
|
<input id="gev-chat-file" type="file" accept=".txt,.md,.csv,.json,.geojson" multiple hidden @change="emit('chat-files', $event.target)" />
|
||||||
@@ -84,7 +110,7 @@ const quickPrompts = () => [t('Fly to Tokyo'), t('Show live flights')];
|
|||||||
</div>
|
</div>
|
||||||
<form id="gev-chat-form" @submit.prevent="emit('chat-submit')">
|
<form id="gev-chat-form" @submit.prevent="emit('chat-submit')">
|
||||||
<button id="gev-chat-restore" type="button" :aria-label="t('Expand chat')" :title="t('Expand chat')" @click="emit('chat-restore')"><svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="M5 4h14a2 2 0 0 1 2 2v10a2 2 0 0 1-2 2H9l-6 4V6a2 2 0 0 1 2-2Z"></path></svg></button>
|
<button id="gev-chat-restore" type="button" :aria-label="t('Expand chat')" :title="t('Expand chat')" @click="emit('chat-restore')"><svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="M5 4h14a2 2 0 0 1 2 2v10a2 2 0 0 1-2 2H9l-6 4V6a2 2 0 0 1 2-2Z"></path></svg></button>
|
||||||
<button id="gev-chat-attach" type="button" :disabled="props.view.chat.busy" :aria-label="t('Add attachment')" :aria-expanded="String(props.view.chat.attachMenuOpen)" aria-controls="gev-chat-attach-menu" @click="emit('chat-attach-toggle')"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="m8 12 6-6a3 3 0 0 1 4 4l-8 8a5 5 0 0 1-7-7l9-9"></path></svg></button>
|
<button id="gev-chat-attach" type="button" :disabled="props.view.chat.busy" :aria-label="t('Add attachment')" :aria-expanded="String(props.view.chat.attachMenuOpen)" aria-controls="gev-chat-attach-menu" @click="emit('chat-attach-toggle')" @keydown="emit('chat-attach-keydown', $event)"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true"><path d="m8 12 6-6a3 3 0 0 1 4 4l-8 8a5 5 0 0 1-7-7l9-9"></path></svg></button>
|
||||||
<textarea id="gev-chat-input" v-owned-draft="{ text: props.view.chat.draft, revision: props.view.chat.draftRevision }" :rows="props.view.chat.rows" maxlength="4000" :aria-label="t('Message GEV')" :placeholder="t('Ask GEV anything about the map…')" @input="emit('chat-input', $event.target.value)" @keydown="emit('chat-keydown', $event)"></textarea>
|
<textarea id="gev-chat-input" v-owned-draft="{ text: props.view.chat.draft, revision: props.view.chat.draftRevision }" :rows="props.view.chat.rows" maxlength="4000" :aria-label="t('Message GEV')" :placeholder="t('Ask GEV anything about the map…')" @input="emit('chat-input', $event.target.value)" @keydown="emit('chat-keydown', $event)"></textarea>
|
||||||
<button id="gev-chat-send" :type="props.view.chat.busy ? 'button' : 'submit'" :disabled="props.view.chat.sendDisabled" :data-busy="String(props.view.chat.busy)" :aria-label="props.view.chat.busy ? t('Stop reply') : t('Send message')" :title="props.view.chat.busy ? t('Stop reply') : t('Send message')" @click="props.view.chat.busy && emit('chat-stop')"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" aria-hidden="true"><path class="gev-chat-enter-icon" d="M19 5v8H5m5-5-5 5 5 5"></path><rect class="gev-chat-stop-icon" x="6" y="6" width="12" height="12" rx="1.5" fill="currentColor" stroke="none"></rect></svg></button>
|
<button id="gev-chat-send" :type="props.view.chat.busy ? 'button' : 'submit'" :disabled="props.view.chat.sendDisabled" :data-busy="String(props.view.chat.busy)" :aria-label="props.view.chat.busy ? t('Stop reply') : t('Send message')" :title="props.view.chat.busy ? t('Stop reply') : t('Send message')" @click="props.view.chat.busy && emit('chat-stop')"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" aria-hidden="true"><path class="gev-chat-enter-icon" d="M19 5v8H5m5-5-5 5 5 5"></path><rect class="gev-chat-stop-icon" x="6" y="6" width="12" height="12" rx="1.5" fill="currentColor" stroke="none"></rect></svg></button>
|
||||||
</form>
|
</form>
|
||||||
|
|||||||
@@ -88,7 +88,8 @@ const frameStyle = (panel) => ({
|
|||||||
<ChatPanel v-if="view.panel" :view="view" />
|
<ChatPanel v-if="view.panel" :view="view" />
|
||||||
<VoiceVisualizer />
|
<VoiceVisualizer />
|
||||||
<div class="gev-chat-thinking" aria-hidden="true"><span></span><span></span><span></span></div>
|
<div class="gev-chat-thinking" aria-hidden="true"><span></span><span></span><span></span></div>
|
||||||
<div id="gev-assistant-phase" role="status" aria-live="polite">{{ view.phase.text }}</div>
|
<!-- Chat announces its phase in its own status line (#gev-chat-status); in chat mode this one stays silent. -->
|
||||||
|
<div id="gev-assistant-phase" role="status" :aria-live="view.root.assistantMode === 'chat' ? 'off' : 'polite'" :aria-hidden="view.root.assistantMode === 'chat' ? 'true' : undefined">{{ view.phase.text }}</div>
|
||||||
<div class="gev-voice-readout">
|
<div class="gev-voice-readout">
|
||||||
<div id="gev-voice-detail" :title="optional(view.detail.title)">{{ view.detail.text }}</div>
|
<div id="gev-voice-detail" :title="optional(view.detail.title)">{{ view.detail.text }}</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -66,6 +66,14 @@ export function bindChatAttachments({ ui, dataManager, onChange, setStatus }) {
|
|||||||
setStatus(error.message);
|
setStatus(error.message);
|
||||||
}
|
}
|
||||||
}),
|
}),
|
||||||
|
// Escape closes the menu and returns to its Attach button.
|
||||||
|
ui.on('chat-attach-keydown', (event) => {
|
||||||
|
if (event.key !== 'Escape' || !ui.state.chat.attachMenuOpen) return;
|
||||||
|
event.preventDefault();
|
||||||
|
event.stopPropagation();
|
||||||
|
close();
|
||||||
|
ui.element('chatPanel')?.querySelector?.('#gev-chat-attach')?.focus?.();
|
||||||
|
}),
|
||||||
ui.on('chat-file-button', () => {
|
ui.on('chat-file-button', () => {
|
||||||
if (!locked) ui.element('chatFile')?.click();
|
if (!locked) ui.element('chatFile')?.click();
|
||||||
}),
|
}),
|
||||||
@@ -82,10 +90,14 @@ export function bindChatAttachments({ ui, dataManager, onChange, setStatus }) {
|
|||||||
);
|
);
|
||||||
if (file.size > 10000)
|
if (file.size > 10000)
|
||||||
throw new Error(t('Each file must be 10 KB or smaller.'));
|
throw new Error(t('Each file must be 10 KB or smaller.'));
|
||||||
const content = new TextDecoder('utf-8', { fatal: true }).decode(
|
const bytes = await file.arrayBuffer();
|
||||||
await file.arrayBuffer(),
|
|
||||||
);
|
|
||||||
if (disposed) return;
|
if (disposed) return;
|
||||||
|
let content = '';
|
||||||
|
try {
|
||||||
|
content = new TextDecoder('utf-8', { fatal: true }).decode(bytes);
|
||||||
|
} catch {
|
||||||
|
// Not UTF-8: the decoder's own message is not the person's words.
|
||||||
|
}
|
||||||
if (!content.trim() || /[\u0000-\u0008\u000e-\u001f]/.test(content))
|
if (!content.trim() || /[\u0000-\u0008\u000e-\u001f]/.test(content))
|
||||||
throw new Error(t('Choose a non-empty UTF-8 text file.'));
|
throw new Error(t('Choose a non-empty UTF-8 text file.'));
|
||||||
add(`file:${file.name}`, { kind: 'file', name: file.name, content });
|
add(`file:${file.name}`, { kind: 'file', name: file.name, content });
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
// src/voice/chatAttachments.test.mjs — the chat's draft attachments: what
|
||||||
|
// they accept and refuse, and how their menu closes.
|
||||||
|
import '../testSupport/registerDom.mjs';
|
||||||
|
import test from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { bindChatAttachments } from './chatAttachments.js';
|
||||||
|
import { createTestVoiceUi } from '../testSupport/voiceUi.mjs';
|
||||||
|
import { mountVoiceView } from '../testSupport/voiceView.mjs';
|
||||||
|
|
||||||
|
const $ = (selector) => document.querySelector(selector);
|
||||||
|
|
||||||
|
function layers(count) {
|
||||||
|
return new Map(
|
||||||
|
Array.from({ length: count }, (_, index) => [
|
||||||
|
`layer-${index + 1}`,
|
||||||
|
{ module: { name: `Layer ${index + 1}` } },
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function attachments(t, { ui = createTestVoiceUi({ panel: true }) } = {}) {
|
||||||
|
const statuses = [];
|
||||||
|
const bound = bindChatAttachments({
|
||||||
|
ui,
|
||||||
|
dataManager: { layers: layers(6), isEnabled: () => true },
|
||||||
|
onChange() {},
|
||||||
|
setStatus: (text) => statuses.push(text),
|
||||||
|
});
|
||||||
|
t.after(() => bound.destroy());
|
||||||
|
const names = () => ui.state.chat.attachments.map((item) => item.name);
|
||||||
|
// The file input as the view hands it over; the reading is asynchronous.
|
||||||
|
const choose = async (...files) => {
|
||||||
|
const input = { files, value: 'C:\\fakepath\\file' };
|
||||||
|
ui.emit('chat-files', input);
|
||||||
|
for (let i = 0; i < 5; i++) await new Promise((r) => setTimeout(r, 0));
|
||||||
|
return input;
|
||||||
|
};
|
||||||
|
return { ui, bound, statuses, names, choose };
|
||||||
|
}
|
||||||
|
|
||||||
|
const file = (name, content) => new File([content], name);
|
||||||
|
|
||||||
|
test('a fifth attachment is refused: the limit is four', (t) => {
|
||||||
|
const { ui, statuses, names } = attachments(t);
|
||||||
|
for (let i = 1; i <= 4; i++) ui.emit('chat-layer', `layer-${i}`);
|
||||||
|
assert.deepEqual(names(), ['Layer 1', 'Layer 2', 'Layer 3', 'Layer 4']);
|
||||||
|
assert.deepEqual(statuses, []);
|
||||||
|
ui.emit('chat-layer', 'layer-5');
|
||||||
|
assert.equal(names().length, 4);
|
||||||
|
assert.deepEqual(statuses, ['You can attach up to four items.']);
|
||||||
|
// Choosing an attached layer again replaces it and is not a fifth.
|
||||||
|
ui.emit('chat-layer', 'layer-2');
|
||||||
|
assert.equal(names().length, 4);
|
||||||
|
assert.equal(statuses.length, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a 10,001-byte file is refused: each file may be up to 10 KB', async (t) => {
|
||||||
|
const { statuses, names, choose } = attachments(t);
|
||||||
|
const input = await choose(file('big.txt', 'a'.repeat(10001)));
|
||||||
|
assert.deepEqual(names(), []);
|
||||||
|
assert.deepEqual(statuses, ['Each file must be 10 KB or smaller.']);
|
||||||
|
// The input clears so the same file can be chosen again.
|
||||||
|
assert.equal(input.value, '');
|
||||||
|
await choose(file('notes.txt', 'a'.repeat(9000)));
|
||||||
|
assert.deepEqual(names(), ['notes.txt']);
|
||||||
|
assert.equal(statuses.length, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('only text file types are accepted', async (t) => {
|
||||||
|
const { statuses, names, choose } = attachments(t);
|
||||||
|
await choose(file('setup.exe', 'MZ'));
|
||||||
|
assert.deepEqual(names(), []);
|
||||||
|
assert.deepEqual(statuses, [
|
||||||
|
'Choose a TXT, Markdown, CSV, JSON or GeoJSON file.',
|
||||||
|
]);
|
||||||
|
await choose(file('places.GeoJSON', '{"type":"FeatureCollection"}'));
|
||||||
|
assert.deepEqual(names(), ['places.GeoJSON']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a file that is not UTF-8 is refused in the person’s words', async (t) => {
|
||||||
|
const { statuses, names, choose } = attachments(t);
|
||||||
|
await choose(file('latin1.csv', new Uint8Array([0x63, 0x61, 0x66, 0xe9])));
|
||||||
|
assert.deepEqual(names(), []);
|
||||||
|
assert.deepEqual(statuses, ['Choose a non-empty UTF-8 text file.']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('control characters and empty files are refused; tabs and line breaks are text', async (t) => {
|
||||||
|
const { statuses, names, choose } = attachments(t);
|
||||||
|
await choose(file('binary.txt', 'name\u0001value'));
|
||||||
|
await choose(file('blank.md', ' \n\t '));
|
||||||
|
assert.deepEqual(names(), []);
|
||||||
|
assert.deepEqual(statuses, [
|
||||||
|
'Choose a non-empty UTF-8 text file.',
|
||||||
|
'Choose a non-empty UTF-8 text file.',
|
||||||
|
]);
|
||||||
|
await choose(file('table.csv', 'a\tb\r\nc\td\n'));
|
||||||
|
assert.deepEqual(names(), ['table.csv']);
|
||||||
|
assert.equal(statuses.length, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('attachments over 10 KB together are refused', async (t) => {
|
||||||
|
const { statuses, names, choose } = attachments(t);
|
||||||
|
await choose(file('one.txt', 'a'.repeat(6000)));
|
||||||
|
await choose(file('two.txt', 'b'.repeat(6000)));
|
||||||
|
assert.deepEqual(names(), ['one.txt']);
|
||||||
|
assert.deepEqual(statuses, [
|
||||||
|
'Attachments are too large together. Use smaller files.',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Escape closes the attach menu and returns to the Attach button', async (t) => {
|
||||||
|
const { pinia, settle } = mountVoiceView(t);
|
||||||
|
const ui = createTestVoiceUi({ panel: true, pinia });
|
||||||
|
attachments(t, { ui });
|
||||||
|
ui.update('chat', { open: true });
|
||||||
|
await settle();
|
||||||
|
$('#gev-chat-attach').click();
|
||||||
|
await settle();
|
||||||
|
assert.equal($('#gev-chat-attach-menu').hidden, false);
|
||||||
|
assert.equal(document.activeElement?.id, 'gev-chat-layer');
|
||||||
|
const escape = new KeyboardEvent('keydown', {
|
||||||
|
key: 'Escape',
|
||||||
|
bubbles: true,
|
||||||
|
cancelable: true,
|
||||||
|
});
|
||||||
|
$('#gev-chat-layer').dispatchEvent(escape);
|
||||||
|
await settle();
|
||||||
|
assert.equal(escape.defaultPrevented, true);
|
||||||
|
assert.equal($('#gev-chat-attach-menu').hidden, true);
|
||||||
|
assert.equal(document.activeElement?.id, 'gev-chat-attach');
|
||||||
|
// With the menu closed, Escape is left to whatever else handles it.
|
||||||
|
const again = new KeyboardEvent('keydown', {
|
||||||
|
key: 'Escape',
|
||||||
|
bubbles: true,
|
||||||
|
cancelable: true,
|
||||||
|
});
|
||||||
|
$('#gev-chat-attach').dispatchEvent(again);
|
||||||
|
assert.equal(again.defaultPrevented, false);
|
||||||
|
});
|
||||||
+64
-15
@@ -1,5 +1,6 @@
|
|||||||
import { bindChatAttachments } from './chatAttachments.js';
|
import { bindChatAttachments } from './chatAttachments.js';
|
||||||
import { bindChatPreferences } from './chatPreferences.js';
|
import { bindChatPreferences } from './chatPreferences.js';
|
||||||
|
import { readStoredCloudVoiceAuthMode } from './cloudVoiceAuth.js';
|
||||||
import { t } from '../i18n/index.js';
|
import { t } from '../i18n/index.js';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -21,6 +22,8 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
let latestReply;
|
let latestReply;
|
||||||
let typingFrame = 0;
|
let typingFrame = 0;
|
||||||
let lastTypingTime = 0;
|
let lastTypingTime = 0;
|
||||||
|
// Set by Stop until the cancelled turn reports its end or the next send.
|
||||||
|
let stopping = false;
|
||||||
const isTyped = () => Boolean(session.adapter.getStartOptions?.().textOnly);
|
const isTyped = () => Boolean(session.adapter.getStartOptions?.().textOnly);
|
||||||
const chat = () => ui.state.chat;
|
const chat = () => ui.state.chat;
|
||||||
// Chat Settings are still chat: the dock keeps its chat state.
|
// Chat Settings are still chat: the dock keeps its chat state.
|
||||||
@@ -81,11 +84,19 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
}
|
}
|
||||||
// Layout moves (yielding to a dock tray, say) pass `focus: false`, so they
|
// Layout moves (yielding to a dock tray, say) pass `focus: false`, so they
|
||||||
// never take focus from what the user is typing in.
|
// never take focus from what the user is typing in.
|
||||||
|
// ChatGPT sign-in covers OpenAI voice only; typed chat still uses the key.
|
||||||
|
const authNote = () =>
|
||||||
|
ui.state.root.provider === 'openai' &&
|
||||||
|
readStoredCloudVoiceAuthMode() === 'oauth'
|
||||||
|
? t(
|
||||||
|
'Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.',
|
||||||
|
)
|
||||||
|
: '';
|
||||||
function open({ focus = true } = {}) {
|
function open({ focus = true } = {}) {
|
||||||
leaveChatSettings();
|
leaveChatSettings();
|
||||||
if (!isTyped() && session.isActive()) session.stop();
|
if (!isTyped() && session.isActive()) session.stop();
|
||||||
onOpen?.();
|
onOpen?.();
|
||||||
ui.update('chat', { open: true, minimized: false, rows: 2 });
|
ui.update('chat', { open: true, minimized: false, rows: 2, authNote });
|
||||||
sync();
|
sync();
|
||||||
if (focus) void ui.focus(['chatInput']);
|
if (focus) void ui.focus(['chatInput']);
|
||||||
}
|
}
|
||||||
@@ -110,6 +121,20 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
return entry;
|
return entry;
|
||||||
}
|
}
|
||||||
const preview = (text) => ui.update('chat', { miniReply: text });
|
const preview = (text) => ui.update('chat', { miniReply: text });
|
||||||
|
// The reveal writes only the message list each frame. The minimized bar's
|
||||||
|
// reply and the log's busy state change once, as the reveal starts and
|
||||||
|
// ends, so the frames do not republish the whole control.
|
||||||
|
function startReveal() {
|
||||||
|
if (typingFrame) return;
|
||||||
|
ui.update('chat', { revealing: true });
|
||||||
|
typingFrame = requestAnimationFrame(animateReply);
|
||||||
|
}
|
||||||
|
function endReveal() {
|
||||||
|
ui.update('chat', {
|
||||||
|
revealing: false,
|
||||||
|
...(latestReply && { miniReply: latestReply.entry.text }),
|
||||||
|
});
|
||||||
|
}
|
||||||
function animateReply(time) {
|
function animateReply(time) {
|
||||||
if (disposed) return;
|
if (disposed) return;
|
||||||
const elapsed = Math.min(100, time - (lastTypingTime || time - 16));
|
const elapsed = Math.min(100, time - (lastTypingTime || time - 16));
|
||||||
@@ -122,26 +147,35 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
Math.ceil(remaining / 40),
|
Math.ceil(remaining / 40),
|
||||||
);
|
);
|
||||||
reply.entry.text = reply.text.slice(0, reply.entry.text.length + step);
|
reply.entry.text = reply.text.slice(0, reply.entry.text.length + step);
|
||||||
if (reply === latestReply) preview(reply.entry.text);
|
|
||||||
if (reply.entry.text === reply.text) typing.delete(reply);
|
if (reply.entry.text === reply.text) typing.delete(reply);
|
||||||
}
|
}
|
||||||
publishMessages();
|
publishMessages();
|
||||||
typingFrame = typing.size ? requestAnimationFrame(animateReply) : 0;
|
typingFrame = typing.size ? requestAnimationFrame(animateReply) : 0;
|
||||||
if (!typingFrame) lastTypingTime = 0;
|
if (!typingFrame) {
|
||||||
|
lastTypingTime = 0;
|
||||||
|
endReveal();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
function finishTyping() {
|
function finishTyping() {
|
||||||
|
const revealing = Boolean(typingFrame);
|
||||||
if (typingFrame) cancelAnimationFrame(typingFrame);
|
if (typingFrame) cancelAnimationFrame(typingFrame);
|
||||||
typingFrame = 0;
|
typingFrame = 0;
|
||||||
lastTypingTime = 0;
|
lastTypingTime = 0;
|
||||||
for (const reply of typing) reply.entry.text = reply.text;
|
for (const reply of typing) reply.entry.text = reply.text;
|
||||||
if (typing.size) publishMessages();
|
if (typing.size) publishMessages();
|
||||||
typing.clear();
|
typing.clear();
|
||||||
|
if (revealing) endReveal();
|
||||||
}
|
}
|
||||||
const unsubscribe = session.subscribe((event) => {
|
const unsubscribe = session.subscribe((event) => {
|
||||||
if (disposed) return;
|
if (disposed) return;
|
||||||
const typed = isTyped();
|
const typed = isTyped();
|
||||||
if (event.type === 'state') sync();
|
if (event.type === 'state') sync();
|
||||||
if (event.type === 'transcript' && event.role === 'assistant' && typed) {
|
if (
|
||||||
|
event.type === 'transcript' &&
|
||||||
|
event.role === 'assistant' &&
|
||||||
|
typed &&
|
||||||
|
!stopping
|
||||||
|
) {
|
||||||
const id = event.itemId || event.responseId || 'current';
|
const id = event.itemId || event.responseId || 'current';
|
||||||
let reply = replies.get(id);
|
let reply = replies.get(id);
|
||||||
if (!reply) {
|
if (!reply) {
|
||||||
@@ -152,12 +186,12 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
reply.text = event.final ? event.text : reply.text + event.text;
|
reply.text = event.final ? event.text : reply.text + event.text;
|
||||||
if (globalThis.matchMedia?.('(prefers-reduced-motion: reduce)').matches) {
|
if (globalThis.matchMedia?.('(prefers-reduced-motion: reduce)').matches) {
|
||||||
reply.entry.text = reply.text;
|
reply.entry.text = reply.text;
|
||||||
preview(reply.text);
|
if (event.final) preview(reply.text);
|
||||||
typing.delete(reply);
|
typing.delete(reply);
|
||||||
publishMessages();
|
publishMessages();
|
||||||
} else {
|
} else {
|
||||||
typing.add(reply);
|
typing.add(reply);
|
||||||
if (!typingFrame) typingFrame = requestAnimationFrame(animateReply);
|
startReveal();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (event.type === 'action-call' && busy) {
|
if (event.type === 'action-call' && busy) {
|
||||||
@@ -167,14 +201,19 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
if (typed && event.type === 'completion' && !event.intermediate) {
|
if (typed && event.type === 'completion' && !event.intermediate) {
|
||||||
busy = false;
|
busy = false;
|
||||||
const completed = event.status === 'completed';
|
const completed = event.status === 'completed';
|
||||||
setStatus(() =>
|
// A turn the person stopped keeps its "Reply stopped." status.
|
||||||
completed
|
if (stopping) stopping = false;
|
||||||
? t('Silent chat · microphone off')
|
else {
|
||||||
: t('Reply interrupted. You can try again.'),
|
setStatus(() =>
|
||||||
);
|
completed
|
||||||
if (!completed) {
|
? t('Silent chat · microphone off')
|
||||||
finishTyping();
|
: t('Reply interrupted. You can try again.'),
|
||||||
preview(t('Reply interrupted. You can try again.'));
|
);
|
||||||
|
if (!completed) {
|
||||||
|
finishTyping();
|
||||||
|
preview(t('Reply interrupted. You can try again.'));
|
||||||
|
} else if (latestReply && !typing.has(latestReply))
|
||||||
|
preview(latestReply.entry.text);
|
||||||
}
|
}
|
||||||
sync();
|
sync();
|
||||||
}
|
}
|
||||||
@@ -242,8 +281,10 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
const attempt = ++revision;
|
const attempt = ++revision;
|
||||||
busy = true;
|
busy = true;
|
||||||
connecting = true;
|
connecting = true;
|
||||||
|
stopping = false;
|
||||||
finishTyping();
|
finishTyping();
|
||||||
replies.clear();
|
replies.clear();
|
||||||
|
latestReply = undefined;
|
||||||
sync();
|
sync();
|
||||||
setStatus(() => t('Connecting…'));
|
setStatus(() => t('Connecting…'));
|
||||||
preview(t('Connecting…'));
|
preview(t('Connecting…'));
|
||||||
@@ -312,13 +353,21 @@ export function bindChatControl({ ui, session, onOpen, dataManager }) {
|
|||||||
}
|
}
|
||||||
}),
|
}),
|
||||||
ui.on('chat-submit', () => void submit()),
|
ui.on('chat-submit', () => void submit()),
|
||||||
|
// Stop ends only the reply in flight: the session and conversation stay.
|
||||||
|
// An adapter without turn cancellation ends the session instead.
|
||||||
ui.on('chat-stop', () => {
|
ui.on('chat-stop', () => {
|
||||||
if (!busy) return;
|
if (!busy) return;
|
||||||
finishTyping();
|
finishTyping();
|
||||||
session.stop();
|
const cancelTurn = session.adapter.cancelTurn?.bind(session.adapter);
|
||||||
|
if (cancelTurn && !connecting) {
|
||||||
|
stopping = true;
|
||||||
|
busy = false;
|
||||||
|
cancelTurn();
|
||||||
|
} else session.stop();
|
||||||
if (disposed) return;
|
if (disposed) return;
|
||||||
setStatus(() => t('Reply stopped.'));
|
setStatus(() => t('Reply stopped.'));
|
||||||
preview(t('Reply stopped.'));
|
preview(t('Reply stopped.'));
|
||||||
|
sync();
|
||||||
}),
|
}),
|
||||||
ui.on('chat-prompt', (text) => {
|
ui.on('chat-prompt', (text) => {
|
||||||
setDraft(text);
|
setDraft(text);
|
||||||
|
|||||||
@@ -0,0 +1,331 @@
|
|||||||
|
// src/voice/chatControlView.test.mjs — the typed chat's owner and its view:
|
||||||
|
// the reply reveal, Stop, the attach menu's layer choice, the conversation's
|
||||||
|
// scrolling and what screen readers are told.
|
||||||
|
import '../testSupport/registerDom.mjs';
|
||||||
|
import test from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { bindChatControl } from './chatControl.js';
|
||||||
|
import { CLOUD_VOICE_AUTH_STORAGE_KEY } from './cloudVoiceAuth.js';
|
||||||
|
import { createTestVoiceUi } from '../testSupport/voiceUi.mjs';
|
||||||
|
import { mountVoiceView } from '../testSupport/voiceView.mjs';
|
||||||
|
|
||||||
|
const $ = (selector) => document.querySelector(selector);
|
||||||
|
const flush = async () => {
|
||||||
|
for (let i = 0; i < 5; i++) await new Promise((r) => setTimeout(r, 0));
|
||||||
|
};
|
||||||
|
|
||||||
|
// A typed session that connects at once and sends what it is given.
|
||||||
|
function typedSession({ cancelTurn = false } = {}) {
|
||||||
|
const listeners = new Set();
|
||||||
|
const session = {
|
||||||
|
state: 'idle',
|
||||||
|
sent: [],
|
||||||
|
stops: 0,
|
||||||
|
cancels: 0,
|
||||||
|
adapter: {
|
||||||
|
getStartOptions: () => ({ textOnly: true }),
|
||||||
|
...(cancelTurn && {
|
||||||
|
cancelTurn() {
|
||||||
|
session.cancels++;
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
subscribe(fn) {
|
||||||
|
listeners.add(fn);
|
||||||
|
return () => listeners.delete(fn);
|
||||||
|
},
|
||||||
|
emit(event) {
|
||||||
|
for (const fn of [...listeners]) fn(event);
|
||||||
|
},
|
||||||
|
setState(state) {
|
||||||
|
session.state = state;
|
||||||
|
session.emit({ type: 'state', state });
|
||||||
|
},
|
||||||
|
isActive: () => !['idle', 'error'].includes(session.state),
|
||||||
|
start() {
|
||||||
|
session.setState('listening');
|
||||||
|
},
|
||||||
|
stop() {
|
||||||
|
session.stops++;
|
||||||
|
session.setState('idle');
|
||||||
|
},
|
||||||
|
sendText(text) {
|
||||||
|
session.sent.push(text);
|
||||||
|
return true;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return session;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Animation frames run only when the test steps them.
|
||||||
|
function manualFrames(t) {
|
||||||
|
const original = {
|
||||||
|
request: globalThis.requestAnimationFrame,
|
||||||
|
cancel: globalThis.cancelAnimationFrame,
|
||||||
|
};
|
||||||
|
const queue = new Map();
|
||||||
|
let next = 0;
|
||||||
|
let time = 0;
|
||||||
|
globalThis.requestAnimationFrame = (fn) => {
|
||||||
|
queue.set(++next, fn);
|
||||||
|
return next;
|
||||||
|
};
|
||||||
|
globalThis.cancelAnimationFrame = (id) => queue.delete(id);
|
||||||
|
t.after(() => {
|
||||||
|
globalThis.requestAnimationFrame = original.request;
|
||||||
|
globalThis.cancelAnimationFrame = original.cancel;
|
||||||
|
});
|
||||||
|
return {
|
||||||
|
get pending() {
|
||||||
|
return queue.size;
|
||||||
|
},
|
||||||
|
step() {
|
||||||
|
time += 16;
|
||||||
|
const due = [...queue.values()];
|
||||||
|
queue.clear();
|
||||||
|
for (const fn of due) fn(time);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function chat(t, { pinia, session = typedSession(), dataManager } = {}) {
|
||||||
|
const ui = createTestVoiceUi({ panel: true, pinia });
|
||||||
|
const control = bindChatControl({ ui, session, dataManager });
|
||||||
|
t.after(() => control.destroy());
|
||||||
|
const send = async (text) => {
|
||||||
|
ui.emit('chat-input', text);
|
||||||
|
ui.emit('chat-submit');
|
||||||
|
await flush();
|
||||||
|
};
|
||||||
|
const reply = (text, extra = {}) =>
|
||||||
|
session.emit({
|
||||||
|
type: 'transcript',
|
||||||
|
role: 'assistant',
|
||||||
|
itemId: 'reply-1',
|
||||||
|
text,
|
||||||
|
...extra,
|
||||||
|
});
|
||||||
|
const texts = () => ui.chatMessages.map((m) => `${m.role}:${m.text}`);
|
||||||
|
return { ui, session, control, send, reply, texts };
|
||||||
|
}
|
||||||
|
|
||||||
|
test('a revealed reply writes only the message list each frame', async (t) => {
|
||||||
|
const frames = manualFrames(t);
|
||||||
|
const { ui, send, reply, texts } = chat(t);
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await send('Fly to Tokyo');
|
||||||
|
assert.equal(ui.state.chat.miniReply, 'Thinking…');
|
||||||
|
const text = 'Flying to Tokyo now. '.repeat(8).trim();
|
||||||
|
reply(text, { final: true });
|
||||||
|
assert.equal(ui.state.chat.revealing, true);
|
||||||
|
const control = ui.state;
|
||||||
|
const seen = new Set();
|
||||||
|
for (let guard = 0; frames.pending; guard++) {
|
||||||
|
if (guard > 200) assert.fail('the reveal did not finish');
|
||||||
|
frames.step();
|
||||||
|
if (!frames.pending) break;
|
||||||
|
seen.add(texts().at(-1));
|
||||||
|
// Mid-reveal the control itself is not republished.
|
||||||
|
assert.equal(ui.state, control);
|
||||||
|
}
|
||||||
|
assert.ok(seen.size > 1, 'the reply revealed over several frames');
|
||||||
|
assert.equal(texts().at(-1), `assistant:${text}`);
|
||||||
|
// The minimized bar's reply and the log's busy state change once, at the end.
|
||||||
|
assert.equal(ui.state.chat.revealing, false);
|
||||||
|
assert.equal(ui.state.chat.miniReply, text);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Stop cancels only the reply in flight and keeps the conversation', async (t) => {
|
||||||
|
manualFrames(t);
|
||||||
|
const session = typedSession({ cancelTurn: true });
|
||||||
|
const { ui, send, reply, texts } = chat(t, { session });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await send('Fly to Tokyo');
|
||||||
|
reply('Flying');
|
||||||
|
assert.equal(ui.state.chat.busy, true);
|
||||||
|
ui.emit('chat-stop');
|
||||||
|
assert.equal(session.cancels, 1);
|
||||||
|
assert.equal(session.stops, 0);
|
||||||
|
assert.equal(session.isActive(), true);
|
||||||
|
assert.equal(ui.state.chat.busy, false);
|
||||||
|
assert.equal(ui.state.chat.status, 'Reply stopped.');
|
||||||
|
assert.equal(ui.state.chat.miniReply, 'Reply stopped.');
|
||||||
|
assert.deepEqual(texts(), ['user:Fly to Tokyo', 'assistant:Flying']);
|
||||||
|
// The cancelled turn's own end and late text keep the stop as it was.
|
||||||
|
reply(' to Tokyo.');
|
||||||
|
session.emit({ type: 'completion', status: 'cancelled' });
|
||||||
|
assert.equal(ui.state.chat.status, 'Reply stopped.');
|
||||||
|
assert.deepEqual(texts(), ['user:Fly to Tokyo', 'assistant:Flying']);
|
||||||
|
// The same session takes the next message.
|
||||||
|
await send('Show live flights');
|
||||||
|
assert.equal(session.sent.length, 2);
|
||||||
|
assert.equal(ui.state.chat.busy, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Stop ends the session when its adapter cannot cancel a turn', async (t) => {
|
||||||
|
manualFrames(t);
|
||||||
|
const { ui, session, send } = chat(t);
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await send('Fly to Tokyo');
|
||||||
|
ui.emit('chat-stop');
|
||||||
|
assert.equal(session.stops, 1);
|
||||||
|
assert.equal(ui.state.chat.busy, false);
|
||||||
|
assert.equal(ui.state.chat.status, 'Reply stopped.');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('with ChatGPT sign-in chosen for OpenAI voice, chat says it uses the API key', (t) => {
|
||||||
|
t.after(() => localStorage.removeItem(CLOUD_VOICE_AUTH_STORAGE_KEY));
|
||||||
|
const { ui } = chat(t);
|
||||||
|
const note =
|
||||||
|
'Typed chat uses your OpenAI API key. ChatGPT sign-in covers voice only.';
|
||||||
|
ui.update('root', { provider: 'openai' });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
assert.equal(ui.state.chat.authNote, '');
|
||||||
|
ui.emit('chat-close');
|
||||||
|
localStorage.setItem(CLOUD_VOICE_AUTH_STORAGE_KEY, 'oauth');
|
||||||
|
ui.emit('chat-button');
|
||||||
|
assert.equal(ui.state.chat.authNote, note);
|
||||||
|
ui.emit('chat-close');
|
||||||
|
ui.update('root', { provider: 'gemini' });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
assert.equal(ui.state.chat.authNote, '');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the conversation is busy for screen readers while a reply types out', async (t) => {
|
||||||
|
const frames = manualFrames(t);
|
||||||
|
const { pinia, settle } = mountVoiceView(t);
|
||||||
|
const { ui, session, send, reply } = chat(t, { pinia });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await settle();
|
||||||
|
const log = $('#gev-chat-messages');
|
||||||
|
const phase = $('#gev-assistant-phase');
|
||||||
|
assert.equal(log.getAttribute('aria-busy'), null);
|
||||||
|
// In chat, the chat's own status line is the one status region.
|
||||||
|
assert.equal(phase.getAttribute('aria-hidden'), 'true');
|
||||||
|
assert.equal(phase.getAttribute('aria-live'), 'off');
|
||||||
|
await send('Fly to Tokyo');
|
||||||
|
await settle();
|
||||||
|
assert.equal(log.getAttribute('aria-busy'), 'true');
|
||||||
|
reply('Flying to Tokyo now.', { final: true });
|
||||||
|
session.emit({ type: 'completion', status: 'completed' });
|
||||||
|
frames.step();
|
||||||
|
await settle();
|
||||||
|
// Done thinking, still typing out.
|
||||||
|
assert.equal(ui.state.chat.busy, false);
|
||||||
|
assert.equal(log.getAttribute('aria-busy'), 'true');
|
||||||
|
while (frames.pending) frames.step();
|
||||||
|
await settle();
|
||||||
|
assert.equal(log.getAttribute('aria-busy'), null);
|
||||||
|
ui.emit('chat-close');
|
||||||
|
await settle();
|
||||||
|
assert.equal(phase.getAttribute('aria-hidden'), null);
|
||||||
|
assert.equal(phase.getAttribute('aria-live'), 'polite');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a layer attaches with the Attach button or Enter, not as the choice moves', async (t) => {
|
||||||
|
const { pinia, settle } = mountVoiceView(t);
|
||||||
|
const dataManager = {
|
||||||
|
layers: new Map([
|
||||||
|
['flights', { module: { name: 'Live flights' } }],
|
||||||
|
['quakes', { module: { name: 'Earthquakes' } }],
|
||||||
|
]),
|
||||||
|
isEnabled: () => true,
|
||||||
|
};
|
||||||
|
const { ui } = chat(t, { pinia, dataManager });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await settle();
|
||||||
|
$('#gev-chat-attach').click();
|
||||||
|
await settle();
|
||||||
|
const select = $('#gev-chat-layer');
|
||||||
|
const attach = $('#gev-chat-layer-attach');
|
||||||
|
assert.equal(attach.disabled, true);
|
||||||
|
// Arrow keys on a closed select change it once per press.
|
||||||
|
for (const value of ['flights', 'quakes']) {
|
||||||
|
select.value = value;
|
||||||
|
select.dispatchEvent(new Event('change', { bubbles: true }));
|
||||||
|
await settle();
|
||||||
|
}
|
||||||
|
assert.deepEqual(ui.state.chat.attachments, []);
|
||||||
|
assert.equal(attach.disabled, false);
|
||||||
|
attach.click();
|
||||||
|
await settle();
|
||||||
|
assert.deepEqual(
|
||||||
|
ui.state.chat.attachments.map((item) => item.name),
|
||||||
|
['Earthquakes'],
|
||||||
|
);
|
||||||
|
$('#gev-chat-attach').click();
|
||||||
|
await settle();
|
||||||
|
select.value = 'flights';
|
||||||
|
select.dispatchEvent(new Event('change', { bubbles: true }));
|
||||||
|
const enter = new KeyboardEvent('keydown', {
|
||||||
|
key: 'Enter',
|
||||||
|
bubbles: true,
|
||||||
|
cancelable: true,
|
||||||
|
});
|
||||||
|
select.dispatchEvent(enter);
|
||||||
|
await settle();
|
||||||
|
assert.equal(enter.defaultPrevented, true);
|
||||||
|
assert.deepEqual(
|
||||||
|
ui.state.chat.attachments.map((item) => item.name),
|
||||||
|
['Earthquakes', 'Live flights'],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the conversation follows new messages only while read at the bottom', async (t) => {
|
||||||
|
const { pinia, settle } = mountVoiceView(t);
|
||||||
|
const { ui } = chat(t, { pinia });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await settle();
|
||||||
|
const list = $('#gev-chat-messages');
|
||||||
|
let top = 0;
|
||||||
|
Object.defineProperties(list, {
|
||||||
|
scrollHeight: { configurable: true, get: () => 1000 },
|
||||||
|
clientHeight: { configurable: true, get: () => 200 },
|
||||||
|
scrollTop: {
|
||||||
|
configurable: true,
|
||||||
|
get: () => top,
|
||||||
|
set: (value) => {
|
||||||
|
top = Math.min(value, 800);
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const show = async (...messages) => {
|
||||||
|
ui.setChatMessages(
|
||||||
|
messages.map(([role, text], index) => ({ id: index + 1, role, text })),
|
||||||
|
);
|
||||||
|
await settle();
|
||||||
|
};
|
||||||
|
await show(['user', 'Fly to Tokyo'], ['assistant', 'Fly']);
|
||||||
|
assert.equal(top, 800);
|
||||||
|
// The reader scrolls up; the reply keeps typing out below them.
|
||||||
|
top = 300;
|
||||||
|
list.dispatchEvent(new Event('scroll'));
|
||||||
|
await show(['user', 'Fly to Tokyo'], ['assistant', 'Flying to Tokyo.']);
|
||||||
|
assert.equal(top, 300);
|
||||||
|
// Sending brings them back to the newest message.
|
||||||
|
await show(
|
||||||
|
['user', 'Fly to Tokyo'],
|
||||||
|
['assistant', 'Flying to Tokyo.'],
|
||||||
|
['user', 'Show live flights'],
|
||||||
|
);
|
||||||
|
assert.equal(top, 800);
|
||||||
|
list.dispatchEvent(new Event('scroll'));
|
||||||
|
await show(
|
||||||
|
['user', 'Fly to Tokyo'],
|
||||||
|
['assistant', 'Flying to Tokyo.'],
|
||||||
|
['user', 'Show live flights'],
|
||||||
|
['assistant', 'Showing'],
|
||||||
|
);
|
||||||
|
assert.equal(top, 800);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the chat gear is named for the settings it opens', async (t) => {
|
||||||
|
const { pinia, settle } = mountVoiceView(t);
|
||||||
|
const { ui } = chat(t, { pinia });
|
||||||
|
ui.emit('chat-button');
|
||||||
|
await settle();
|
||||||
|
assert.equal(
|
||||||
|
$('#gev-voice-settings-button').getAttribute('aria-label'),
|
||||||
|
'Settings',
|
||||||
|
);
|
||||||
|
});
|
||||||
@@ -134,6 +134,10 @@ function initialState(panel) {
|
|||||||
quickChecked: true,
|
quickChecked: true,
|
||||||
// The latest reply, shown above the minimized bar.
|
// The latest reply, shown above the minimized bar.
|
||||||
miniReply: '',
|
miniReply: '',
|
||||||
|
// A reply is typing out; the conversation is busy for screen readers.
|
||||||
|
revealing: false,
|
||||||
|
// Says chat bills the OpenAI key while voice uses ChatGPT sign-in.
|
||||||
|
authNote: '',
|
||||||
suggestions: [],
|
suggestions: [],
|
||||||
attachments: [],
|
attachments: [],
|
||||||
attachMenuOpen: false,
|
attachMenuOpen: false,
|
||||||
|
|||||||
@@ -2545,6 +2545,7 @@ export function createGeminiSession(options) {
|
|||||||
});
|
});
|
||||||
let textOnly = false;
|
let textOnly = false;
|
||||||
const startVoice = (settings) => {
|
const startVoice = (settings) => {
|
||||||
|
chat.stop({ preserveStatus: true });
|
||||||
textOnly = false;
|
textOnly = false;
|
||||||
return controller.start(settings);
|
return controller.start(settings);
|
||||||
};
|
};
|
||||||
@@ -2571,6 +2572,8 @@ export function createGeminiSession(options) {
|
|||||||
chat.stop({ ...settings, preserveStatus: true });
|
chat.stop({ ...settings, preserveStatus: true });
|
||||||
controller.stop(settings);
|
controller.stop(settings);
|
||||||
},
|
},
|
||||||
|
// Chat Stop: abort the reply in progress, keep the conversation.
|
||||||
|
cancelTurn: () => textOnly && chat.cancelTurn(),
|
||||||
sendText: (text) =>
|
sendText: (text) =>
|
||||||
textOnly ? chat.sendText(text) : controller.sendTextCommand(text),
|
textOnly ? chat.sendText(text) : controller.sendTextCommand(text),
|
||||||
sendMapEvent: (event) =>
|
sendMapEvent: (event) =>
|
||||||
|
|||||||
@@ -177,6 +177,58 @@ test('composite chat never starts Live and the next keyboard start resets its mo
|
|||||||
adapter.stop({ removeUi: true });
|
adapter.stop({ removeUi: true });
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('Gemini composite: chat Stop keeps the conversation and voice start ends chat', async () => {
|
||||||
|
const { createGeminiSession } = await import('./geminiSession.js');
|
||||||
|
const requests = [];
|
||||||
|
const signals = [];
|
||||||
|
const adapter = createGeminiSession({
|
||||||
|
emit: () => {},
|
||||||
|
runAction: (_name, _args, options) => {
|
||||||
|
signals.push(options.signal);
|
||||||
|
return new Promise(() => {});
|
||||||
|
},
|
||||||
|
ui: createTestVoiceUi(),
|
||||||
|
fetchImpl: async (_url, options) => {
|
||||||
|
requests.push(JSON.parse(options.body));
|
||||||
|
return requests.length === 1
|
||||||
|
? final()
|
||||||
|
: Response.json({
|
||||||
|
content: {
|
||||||
|
role: 'model',
|
||||||
|
parts: [{ functionCall: { name: 'fly_to', args: {} } }],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
},
|
||||||
|
getUserMedia: () => {
|
||||||
|
throw Error('No microphone');
|
||||||
|
},
|
||||||
|
createAudio: () => {
|
||||||
|
throw Error('No audio');
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await adapter.start({ textOnly: true });
|
||||||
|
adapter.sendText('First');
|
||||||
|
for (let i = 0; i < 5; i++) await tick();
|
||||||
|
adapter.sendText('Second');
|
||||||
|
for (let i = 0; i < 5 && !signals[0]; i++) await tick();
|
||||||
|
assert.equal(adapter.cancelTurn(), true);
|
||||||
|
assert.equal(signals[0].aborted, true);
|
||||||
|
assert.equal(adapter.sendText('Third'), true);
|
||||||
|
for (let i = 0; i < 5 && !signals[1]; i++) await tick();
|
||||||
|
assert.deepEqual(
|
||||||
|
requests[2].contents
|
||||||
|
.filter((item) => item.parts[0].text)
|
||||||
|
.map((item) => item.parts[0].text),
|
||||||
|
['First', 'Done', 'Third'],
|
||||||
|
);
|
||||||
|
// Starting voice retires the typed turn in progress.
|
||||||
|
adapter.controller.start = async () => {};
|
||||||
|
await adapter.controller.input.start({ pushToTalk: true });
|
||||||
|
assert.equal(signals[1].aborted, true);
|
||||||
|
assert.equal(adapter.cancelTurn(), false);
|
||||||
|
adapter.stop({ removeUi: true });
|
||||||
|
});
|
||||||
|
|
||||||
test('failed Radio startup cancels its stream before releasing the mute', async () => {
|
test('failed Radio startup cancels its stream before releasing the mute', async () => {
|
||||||
const actions = [];
|
const actions = [];
|
||||||
let count = 0;
|
let count = 0;
|
||||||
@@ -307,7 +359,8 @@ for (const intent of ['pause', 'off'])
|
|||||||
assert.equal(plays, 0);
|
assert.equal(plays, 0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('a tool deadline reports one error, aborts work and admits a successor', async () => {
|
test('a tool deadline reports one error, aborts work and admits a successor', async (t) => {
|
||||||
|
t.mock.timers.enable({ apis: ['setTimeout'] });
|
||||||
const events = [];
|
const events = [];
|
||||||
let actionSignal,
|
let actionSignal,
|
||||||
requests = 0;
|
requests = 0;
|
||||||
@@ -331,7 +384,9 @@ test('a tool deadline reports one error, aborts work and admits a successor', as
|
|||||||
});
|
});
|
||||||
chat.start();
|
chat.start();
|
||||||
chat.sendText('Go');
|
chat.sendText('Go');
|
||||||
await new Promise((resolve) => setTimeout(resolve, 70));
|
for (let i = 0; i < 5 && !actionSignal; i++) await tick();
|
||||||
|
assert.equal(actionSignal.aborted, false);
|
||||||
|
t.mock.timers.tick(30);
|
||||||
assert.equal(actionSignal.aborted, true);
|
assert.equal(actionSignal.aborted, true);
|
||||||
assert.equal(events.filter((e) => e.state === 'error').length, 1);
|
assert.equal(events.filter((e) => e.state === 'error').length, 1);
|
||||||
assert.equal(requests, 1);
|
assert.equal(requests, 1);
|
||||||
|
|||||||
@@ -289,7 +289,9 @@ export function createVoicePanelHost({
|
|||||||
signal: session.signal,
|
signal: session.signal,
|
||||||
...clock,
|
...clock,
|
||||||
onExpire: () => {
|
onExpire: () => {
|
||||||
if (!owns()) return;
|
// Typed chat has no microphone or audio to time out, and stopping it
|
||||||
|
// would drop the conversation the chat log still shows.
|
||||||
|
if (!owns() || session.adapter?.getStartOptions?.().textOnly) return;
|
||||||
const ticket = revision;
|
const ticket = revision;
|
||||||
session.stop();
|
session.stop();
|
||||||
closeAfterStop(ticket);
|
closeAfterStop(ticket);
|
||||||
|
|||||||
@@ -472,6 +472,25 @@ test('native connecting phases retain output observation; idle starts before fir
|
|||||||
assert.equal(adapter.stops, 1);
|
assert.equal(adapter.stops, 1);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('the voice inactivity timeout never ends a typed chat session', async (t) => {
|
||||||
|
const f = mounted(t, { native: true });
|
||||||
|
const adapter = f.instances[0];
|
||||||
|
adapter.getStartOptions = () => ({ textOnly: true });
|
||||||
|
await f.commands.session.start({ textOnly: true });
|
||||||
|
adapter.controller.handleMessage({});
|
||||||
|
f.clock.advance(600000);
|
||||||
|
assert.equal(adapter.stops, 0);
|
||||||
|
assert.equal(f.commands.session.isActive(), true);
|
||||||
|
// A quiet voice session started afterwards still times out.
|
||||||
|
f.commands.session.stop();
|
||||||
|
const stops = adapter.stops;
|
||||||
|
adapter.getStartOptions = () => ({});
|
||||||
|
await f.commands.session.start();
|
||||||
|
adapter.controller.handleMessage({});
|
||||||
|
f.clock.advance(600000);
|
||||||
|
assert.equal(adapter.stops, stops + 1);
|
||||||
|
});
|
||||||
|
|
||||||
async function directActions(
|
async function directActions(
|
||||||
t,
|
t,
|
||||||
{ storage = null, delayExit = false, exposePanel = true } = {},
|
{ storage = null, delayExit = false, exposePanel = true } = {},
|
||||||
@@ -1134,6 +1153,17 @@ test('closing chat Settings ends a typed session', async (t) => {
|
|||||||
assert.equal($('#gev-chat-panel').hidden, true);
|
assert.equal($('#gev-chat-panel').hidden, true);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('opening chat stops a live voice session', async (t) => {
|
||||||
|
const f = mounted(t, { view: true });
|
||||||
|
await f.commands.session.start();
|
||||||
|
await f.settle();
|
||||||
|
assert.equal(f.commands.session.isActive(), true);
|
||||||
|
$('#gev-chat-button').click();
|
||||||
|
await f.settle();
|
||||||
|
assert.equal(f.commands.session.isActive(), false);
|
||||||
|
assert.equal($('#gev-chat-panel').hidden, false);
|
||||||
|
});
|
||||||
|
|
||||||
test('the mic button closes chat and minimize collapses voice to a bar', async (t) => {
|
test('the mic button closes chat and minimize collapses voice to a bar', async (t) => {
|
||||||
const f = mounted(t, { view: true });
|
const f = mounted(t, { view: true });
|
||||||
await f.settle();
|
await f.settle();
|
||||||
|
|||||||
+54
-11
@@ -145,6 +145,8 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
let lastHeight = 0;
|
let lastHeight = 0;
|
||||||
let animateNext = false;
|
let animateNext = false;
|
||||||
let heightAnimation = null;
|
let heightAnimation = null;
|
||||||
|
// What the last placement depended on, short of measuring.
|
||||||
|
let placedInputs = '';
|
||||||
const animateHeight = (from, to) => {
|
const animateHeight = (from, to) => {
|
||||||
if (view.matchMedia?.('(prefers-reduced-motion: reduce)').matches) return;
|
if (view.matchMedia?.('(prefers-reduced-motion: reduce)').matches) return;
|
||||||
heightAnimation?.cancel();
|
heightAnimation?.cancel();
|
||||||
@@ -155,12 +157,35 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
],
|
],
|
||||||
{ duration: 240, easing: 'cubic-bezier(0.2, 0.8, 0.2, 1)' },
|
{ duration: 240, easing: 'cubic-bezier(0.2, 0.8, 0.2, 1)' },
|
||||||
);
|
);
|
||||||
|
// Frames skipped while it runs are placed once it ends.
|
||||||
|
if (heightAnimation) heightAnimation.onfinish = schedule;
|
||||||
};
|
};
|
||||||
const place = () => {
|
const place = () => {
|
||||||
const minimized = Boolean(panel.dataset.minimized);
|
const minimized = Boolean(panel.dataset.minimized);
|
||||||
// Minimized bars always sit centred above the dock; only an expanded
|
// Minimized bars always sit centred above the dock; only an expanded
|
||||||
// docked panel takes the right lane.
|
// docked panel takes the right lane.
|
||||||
const inLane = isDocked() && !minimized && !isMobile();
|
const mobile = isMobile();
|
||||||
|
const inLane = isDocked() && !minimized && !mobile;
|
||||||
|
// The height animation resizes the panel every frame. While it runs and
|
||||||
|
// nothing it was placed for has changed, leave the placement as it is;
|
||||||
|
// it is placed again when the animation finishes.
|
||||||
|
const inputs = [
|
||||||
|
minimized,
|
||||||
|
isDocked(),
|
||||||
|
mobile,
|
||||||
|
panel.hidden,
|
||||||
|
root.dataset?.chatOpen,
|
||||||
|
root.dataset?.assistantMode,
|
||||||
|
view.innerWidth,
|
||||||
|
view.innerHeight,
|
||||||
|
].join('|');
|
||||||
|
if (
|
||||||
|
heightAnimation?.playState === 'running' &&
|
||||||
|
!animateNext &&
|
||||||
|
inputs === placedInputs
|
||||||
|
)
|
||||||
|
return;
|
||||||
|
placedInputs = inputs;
|
||||||
panel.dataset.placement = inLane ? 'docked' : 'compact';
|
panel.dataset.placement = inLane ? 'docked' : 'compact';
|
||||||
syncRail();
|
syncRail();
|
||||||
if (lifetime.signal.aborted) return;
|
if (lifetime.signal.aborted) return;
|
||||||
@@ -174,7 +199,7 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
const expanding = !appearing && wasMinimized && !minimized;
|
const expanding = !appearing && wasMinimized && !minimized;
|
||||||
seen = true;
|
seen = true;
|
||||||
wasMinimized = minimized;
|
wasMinimized = minimized;
|
||||||
if (!isDocked() && !isMobile()) {
|
if (!isDocked() && !mobile) {
|
||||||
// A panel opened or switched to while the trays hold the centre
|
// A panel opened or switched to while the trays hold the centre
|
||||||
// starts as its bar.
|
// starts as its bar.
|
||||||
if (appearing && !minimized && (ui.state.layout.yielded || trayOpen())) {
|
if (appearing && !minimized && (ui.state.layout.yielded || trayOpen())) {
|
||||||
@@ -198,6 +223,7 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
top: box.top + (root.clientTop || 0),
|
top: box.top + (root.clientTop || 0),
|
||||||
};
|
};
|
||||||
const anchorBottom = anchor.top + (root.clientHeight || box.height);
|
const anchorBottom = anchor.top + (root.clientHeight || box.height);
|
||||||
|
let measured = null;
|
||||||
const w = view.innerWidth,
|
const w = view.innerWidth,
|
||||||
h = view.innerHeight;
|
h = view.innerHeight;
|
||||||
if (inLane) {
|
if (inLane) {
|
||||||
@@ -227,7 +253,7 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
.getElementById?.('command-dock')
|
.getElementById?.('command-dock')
|
||||||
?.getBoundingClientRect();
|
?.getBoundingClientRect();
|
||||||
panel.style.removeProperty('height');
|
panel.style.removeProperty('height');
|
||||||
if (isMobile() && !minimized) panel.style.width = `${w - 16}px`;
|
if (mobile && !minimized) panel.style.width = `${w - 16}px`;
|
||||||
else panel.style.removeProperty('width');
|
else panel.style.removeProperty('width');
|
||||||
const rect = panel.getBoundingClientRect();
|
const rect = panel.getBoundingClientRect();
|
||||||
const centre = dockRect?.width
|
const centre = dockRect?.width
|
||||||
@@ -253,13 +279,16 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
panel.style.left = `${x - anchor.left}px`;
|
panel.style.left = `${x - anchor.left}px`;
|
||||||
panel.style.top = 'auto';
|
panel.style.top = 'auto';
|
||||||
panel.style.bottom = `${anchorBottom - (y + rect.height)}px`;
|
panel.style.bottom = `${anchorBottom - (y + rect.height)}px`;
|
||||||
|
measured = rect.height;
|
||||||
}
|
}
|
||||||
panel.style.transform = 'none';
|
panel.style.transform = 'none';
|
||||||
// Grow from the panel shown before (chat and voice swap), or from this
|
// Grow from the panel shown before (chat and voice swap), or from this
|
||||||
// panel's last height when its view changes. A running animation is
|
// panel's last height when its view changes. A running animation is
|
||||||
// mid-flight, so its frames are not recorded as the panel's height.
|
// mid-flight, so its frames are not recorded as the panel's height.
|
||||||
if (heightAnimation?.playState === 'running' && !animateNext) return;
|
if (heightAnimation?.playState === 'running' && !animateNext) return;
|
||||||
const height = panel.getBoundingClientRect().height;
|
// A centred panel was measured above; placing it moves it but keeps its
|
||||||
|
// size. The lane's height is set here, so it is measured again.
|
||||||
|
const height = measured ?? panel.getBoundingClientRect().height;
|
||||||
const previous = handoffs.get(root);
|
const previous = handoffs.get(root);
|
||||||
const from = minimized
|
const from = minimized
|
||||||
? 0
|
? 0
|
||||||
@@ -314,7 +343,23 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
.map((item) => item.id),
|
.map((item) => item.id),
|
||||||
);
|
);
|
||||||
let railOpen = expandedRailPanels();
|
let railOpen = expandedRailPanels();
|
||||||
const railChanges = new MutationObserver(() => {
|
// Rail panels open and close through their own class, so only the rail's
|
||||||
|
// direct children are watched, each as it joins the rail.
|
||||||
|
const watchedRailPanels = new WeakSet();
|
||||||
|
const watchRailPanels = () => {
|
||||||
|
for (const item of rail?.children ?? []) {
|
||||||
|
if (watchedRailPanels.has(item)) continue;
|
||||||
|
watchedRailPanels.add(item);
|
||||||
|
railChanges.observe(item, {
|
||||||
|
attributes: true,
|
||||||
|
attributeFilter: ['class'],
|
||||||
|
});
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const railChanges = new MutationObserver((records) => {
|
||||||
|
if (records.some((record) => record.type === 'childList'))
|
||||||
|
watchRailPanels();
|
||||||
|
if (!records.some((record) => record.type === 'attributes')) return;
|
||||||
const next = expandedRailPanels();
|
const next = expandedRailPanels();
|
||||||
const opened = [...next].some((id) => !railOpen.has(id));
|
const opened = [...next].some((id) => !railOpen.has(id));
|
||||||
railOpen = next;
|
railOpen = next;
|
||||||
@@ -326,12 +371,10 @@ function layOut({ ui, name, minimize, restore }) {
|
|||||||
)
|
)
|
||||||
minimize();
|
minimize();
|
||||||
});
|
});
|
||||||
if (rail)
|
if (rail) {
|
||||||
railChanges.observe(rail, {
|
railChanges.observe(rail, { childList: true });
|
||||||
attributes: true,
|
watchRailPanels();
|
||||||
attributeFilter: ['class'],
|
}
|
||||||
subtree: true,
|
|
||||||
});
|
|
||||||
// Reaching for a wing's button steps a centred panel down to its bar,
|
// Reaching for a wing's button steps a centred panel down to its bar,
|
||||||
// which then rests above the tray.
|
// which then rests above the tray.
|
||||||
const reachWing = ({ target }) => {
|
const reachWing = ({ target }) => {
|
||||||
|
|||||||
@@ -47,6 +47,8 @@ export function createRealtimeSession({
|
|||||||
chat.stop({ ...settings, preserveStatus: true });
|
chat.stop({ ...settings, preserveStatus: true });
|
||||||
controller.stop(settings);
|
controller.stop(settings);
|
||||||
},
|
},
|
||||||
|
// Chat Stop: abort the reply in progress, keep the conversation.
|
||||||
|
cancelTurn: () => textOnly && chat.cancelTurn(),
|
||||||
sendText: (text) =>
|
sendText: (text) =>
|
||||||
textOnly ? chat.sendText(text) : controller.sendTextCommand(text),
|
textOnly ? chat.sendText(text) : controller.sendTextCommand(text),
|
||||||
sendMapEvent: (event) =>
|
sendMapEvent: (event) =>
|
||||||
|
|||||||
@@ -650,3 +650,53 @@ test('OpenAI composite starts HTTP chat without Realtime and restores voice keyb
|
|||||||
assert.equal(adapter.sendText('Hello'), 'voice');
|
assert.equal(adapter.sendText('Hello'), 'voice');
|
||||||
adapter.stop();
|
adapter.stop();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('OpenAI composite chat Stop cancels only the reply in progress', async () => {
|
||||||
|
const requests = [];
|
||||||
|
let pendingSignal;
|
||||||
|
const controller = {
|
||||||
|
_input: {},
|
||||||
|
_radio: {},
|
||||||
|
isActive: () => true,
|
||||||
|
start: async () => {},
|
||||||
|
stop: () => {},
|
||||||
|
sendTextCommand: () => 'voice',
|
||||||
|
notifyMapEvent: () => 'voice-event',
|
||||||
|
};
|
||||||
|
const adapter = createRealtimeSession({
|
||||||
|
emit: () => {},
|
||||||
|
runAction: async () => ({ ok: true }),
|
||||||
|
createController: () => controller,
|
||||||
|
fetchImpl: (_url, options) => {
|
||||||
|
requests.push(JSON.parse(options.body));
|
||||||
|
if (requests.length !== 2)
|
||||||
|
return Response.json({
|
||||||
|
content: { role: 'assistant', content: 'Hello' },
|
||||||
|
finishReason: 'stop',
|
||||||
|
});
|
||||||
|
pendingSignal = options.signal;
|
||||||
|
return new Promise(() => {});
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const settle = async () => {
|
||||||
|
for (let i = 0; i < 5; i++)
|
||||||
|
await new Promise((resolve) => setImmediate(resolve));
|
||||||
|
};
|
||||||
|
await adapter.start({ textOnly: true });
|
||||||
|
adapter.sendText('First');
|
||||||
|
await settle();
|
||||||
|
adapter.sendText('Second');
|
||||||
|
await settle();
|
||||||
|
assert.equal(adapter.cancelTurn(), true);
|
||||||
|
assert.equal(pendingSignal.aborted, true);
|
||||||
|
assert.equal(adapter.sendText('Third'), true);
|
||||||
|
await settle();
|
||||||
|
assert.deepEqual(
|
||||||
|
requests[2].messages.map((m) => m.content),
|
||||||
|
['First', 'Hello', 'Third'],
|
||||||
|
);
|
||||||
|
adapter.stop();
|
||||||
|
await controller._input.start({ pushToTalk: true });
|
||||||
|
assert.equal(adapter.cancelTurn(), false);
|
||||||
|
adapter.stop();
|
||||||
|
});
|
||||||
+149
-36
@@ -12,6 +12,17 @@ import {
|
|||||||
responseInstructionForToolResult,
|
responseInstructionForToolResult,
|
||||||
} from './realtimeProtocol.js';
|
} from './realtimeProtocol.js';
|
||||||
|
|
||||||
|
const encoder = new TextEncoder();
|
||||||
|
const byteSize = (value) =>
|
||||||
|
encoder.encode(typeof value === 'string' ? value : JSON.stringify(value))
|
||||||
|
.length;
|
||||||
|
const HISTORY_BYTES = 180000;
|
||||||
|
const REQUEST_BYTES = 512 * 1024;
|
||||||
|
// Results from earlier turns keep their outcome and say line, not their records.
|
||||||
|
const COMPACT_RESULT_BYTES = 1500;
|
||||||
|
const PARTIAL_NOTE =
|
||||||
|
'This reply stopped before it finished. The map actions above already ran.';
|
||||||
|
|
||||||
/** Text-only HTTP turns: no Live token, socket, microphone or audio context. */
|
/** Text-only HTTP turns: no Live token, socket, microphone or audio context. */
|
||||||
export function createTextChat({
|
export function createTextChat({
|
||||||
provider,
|
provider,
|
||||||
@@ -28,6 +39,8 @@ export function createTextChat({
|
|||||||
const sessionUsage = emptyTextUsage();
|
const sessionUsage = emptyTextUsage();
|
||||||
let active = false;
|
let active = false;
|
||||||
let turn = null;
|
let turn = null;
|
||||||
|
let turnUsage = null;
|
||||||
|
let partial = null;
|
||||||
let sequence = 0;
|
let sequence = 0;
|
||||||
let history = [];
|
let history = [];
|
||||||
let mapEvents = [];
|
let mapEvents = [];
|
||||||
@@ -43,6 +56,37 @@ export function createTextChat({
|
|||||||
radioLayer?.setVoiceDucked?.(false);
|
radioLayer?.setVoiceDucked?.(false);
|
||||||
};
|
};
|
||||||
const state = (value) => emit({ type: 'state', state: value });
|
const state = (value) => emit({ type: 'state', state: value });
|
||||||
|
// Every call keeps its paired result; only the result body shrinks.
|
||||||
|
const compact = (items) =>
|
||||||
|
items.map((item) => protocol.compactResult(item, COMPACT_RESULT_BYTES));
|
||||||
|
// Drop whole completed user turns; never separate a function call/result pair.
|
||||||
|
const trim = (items, maxBytes) => {
|
||||||
|
while (items.length && (items.length > 64 || byteSize(items) > maxBytes)) {
|
||||||
|
const next = items.findIndex(
|
||||||
|
(item, index) => index > 0 && protocol.isUser(item),
|
||||||
|
);
|
||||||
|
items = next < 0 ? [] : items.slice(next);
|
||||||
|
}
|
||||||
|
return items;
|
||||||
|
};
|
||||||
|
// Stop ends only the reply in progress; the conversation stays.
|
||||||
|
const cancelTurn = () => {
|
||||||
|
const owner = turn;
|
||||||
|
if (!active || !owner || signal?.aborted) return false;
|
||||||
|
if (partial?.owner === owner) partial.keep();
|
||||||
|
turn = null;
|
||||||
|
owner.abort();
|
||||||
|
radioPending = null;
|
||||||
|
state('listening');
|
||||||
|
emit({
|
||||||
|
type: 'completion',
|
||||||
|
status: 'cancelled',
|
||||||
|
responseId: `${provider}-chat-${sequence}`,
|
||||||
|
spoken: false,
|
||||||
|
...(turnUsage ? { usage: { ...turnUsage } } : {}),
|
||||||
|
});
|
||||||
|
return true;
|
||||||
|
};
|
||||||
const stop = ({ preserveStatus = false, removeUi = false } = {}) => {
|
const stop = ({ preserveStatus = false, removeUi = false } = {}) => {
|
||||||
active = false;
|
active = false;
|
||||||
if (radioHandoff) releaseRadio(radioHandoff, true);
|
if (radioHandoff) releaseRadio(radioHandoff, true);
|
||||||
@@ -79,46 +123,78 @@ export function createTextChat({
|
|||||||
addTextUsage(sessionUsage, value);
|
addTextUsage(sessionUsage, value);
|
||||||
reportUsage();
|
reportUsage();
|
||||||
};
|
};
|
||||||
|
turnUsage = usage;
|
||||||
|
// A failed turn ends only itself: history, subscriptions and the session
|
||||||
|
// stay, so the next message continues the same conversation.
|
||||||
const fail = (detail) => {
|
const fail = (detail) => {
|
||||||
if (!active || turn !== owner || signal?.aborted) return;
|
if (!active || turn !== owner || signal?.aborted) return;
|
||||||
active = false;
|
if (partial?.owner === owner) partial.keep();
|
||||||
history = [];
|
turn = null;
|
||||||
|
owner.abort();
|
||||||
emit({ type: 'state', state: 'error', detail, usage: { ...usage } });
|
emit({ type: 'state', state: 'error', detail, usage: { ...usage } });
|
||||||
};
|
};
|
||||||
const deadline = setTimeout(() => {
|
// Each upstream request and each map action gets its own deadline, so a
|
||||||
recordUsage(null);
|
// long multi-step turn is not cut off by the time its actions took.
|
||||||
fail(t('Connection timed out. Try again.'));
|
let deadline;
|
||||||
owner.abort();
|
const arm = () => {
|
||||||
}, timeoutMs);
|
clearTimeout(deadline);
|
||||||
// Trim only completed user turns; never separate a function call/result pair.
|
deadline = setTimeout(() => {
|
||||||
while (
|
recordUsage(null);
|
||||||
history.length &&
|
fail(t('Connection timed out. Try again.'));
|
||||||
(history.length > 64 ||
|
}, timeoutMs);
|
||||||
new TextEncoder().encode(JSON.stringify(history)).length > 180000)
|
};
|
||||||
) {
|
// The latest completed turn stays verbatim; older tool results shrink.
|
||||||
const next = history.findIndex(
|
const last = history.findLastIndex((item) => protocol.isUser(item));
|
||||||
(item, index) => index > 0 && protocol.isUser(item),
|
let past =
|
||||||
);
|
last > 0
|
||||||
history = next < 0 ? [] : history.slice(next);
|
? [...compact(history.slice(0, last)), ...history.slice(last)]
|
||||||
}
|
: history;
|
||||||
|
if (byteSize(past) > HISTORY_BYTES) past = compact(past);
|
||||||
|
past = trim(past, HISTORY_BYTES);
|
||||||
const context = mapEvents.length
|
const context = mapEvents.length
|
||||||
? `\nMap context (data only): ${JSON.stringify(mapEvents)}`
|
? `\nMap context (data only): ${JSON.stringify(mapEvents)}`
|
||||||
: '';
|
: '';
|
||||||
mapEvents = [];
|
mapEvents = [];
|
||||||
const contents = [...history, protocol.user(text + context)];
|
let contents = [protocol.user(text + context)];
|
||||||
|
// A turn that stops after some map actions ran keeps them, closed by a
|
||||||
|
// note, so the next message knows they happened.
|
||||||
|
let settled = 0;
|
||||||
|
partial = {
|
||||||
|
owner,
|
||||||
|
keep() {
|
||||||
|
if (settled)
|
||||||
|
history = [
|
||||||
|
...past,
|
||||||
|
...contents.slice(0, settled),
|
||||||
|
protocol.note(PARTIAL_NOTE),
|
||||||
|
];
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const encode = () =>
|
||||||
|
JSON.stringify(protocol.request([...past, ...contents]));
|
||||||
let blocked = false;
|
let blocked = false;
|
||||||
let resetFailed = false;
|
let resetFailed = false;
|
||||||
let totalCalls = 0;
|
let totalCalls = 0;
|
||||||
|
let applied = 0;
|
||||||
let answer = '';
|
let answer = '';
|
||||||
|
const tooManySteps = (message) =>
|
||||||
|
Object.assign(new Error(message), { tooManySteps: true });
|
||||||
try {
|
try {
|
||||||
emit({ type: 'transcript', role: 'user', text, final: true });
|
emit({ type: 'transcript', role: 'user', text, final: true });
|
||||||
if (!current()) return;
|
if (!current()) return;
|
||||||
state('executing');
|
state('executing');
|
||||||
for (let round = 0; round < 8; round++) {
|
for (let round = 0; round < 8; round++) {
|
||||||
if (!current()) return;
|
if (!current()) return;
|
||||||
const body = JSON.stringify(protocol.request(contents));
|
let body = encode();
|
||||||
if (new TextEncoder().encode(body).length > 512 * 1024)
|
if (byteSize(body) > REQUEST_BYTES) {
|
||||||
throw new Error('History limit');
|
// Shorten earlier results, then drop old turns, before giving up.
|
||||||
|
contents = [contents[0], ...compact(contents.slice(1))];
|
||||||
|
past = trim(compact(past), REQUEST_BYTES - byteSize(contents) - 1024);
|
||||||
|
body = encode();
|
||||||
|
if (byteSize(body) > REQUEST_BYTES)
|
||||||
|
throw tooManySteps('Request limit');
|
||||||
|
}
|
||||||
|
arm();
|
||||||
requestPending = true;
|
requestPending = true;
|
||||||
const acquisition = Promise.resolve(
|
const acquisition = Promise.resolve(
|
||||||
fetchImpl(chatEndpoint || protocol.endpoint, {
|
fetchImpl(chatEndpoint || protocol.endpoint, {
|
||||||
@@ -141,15 +217,23 @@ export function createTextChat({
|
|||||||
void response.body?.cancel().catch(() => {});
|
void response.body?.cancel().catch(() => {});
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const data = await readResponseJsonCapped(
|
let data;
|
||||||
response,
|
try {
|
||||||
512 * 1024,
|
data = await readResponseJsonCapped(
|
||||||
owner.signal,
|
response,
|
||||||
);
|
512 * 1024,
|
||||||
|
owner.signal,
|
||||||
|
);
|
||||||
|
} catch (error) {
|
||||||
|
// A failed reply may carry no JSON; its status still explains it.
|
||||||
|
if (response.ok || owner.signal.aborted) throw error;
|
||||||
|
}
|
||||||
recordUsage(data?.usage);
|
recordUsage(data?.usage);
|
||||||
if (!response.ok)
|
if (!response.ok)
|
||||||
throw Object.assign(new Error('Request failed'), {
|
throw Object.assign(new Error('Request failed'), {
|
||||||
status: response.status,
|
status: response.status,
|
||||||
|
code: typeof data?.code === 'string' ? data.code : undefined,
|
||||||
|
retryAfter: Number(response.headers?.get?.('retry-after')) || 0,
|
||||||
});
|
});
|
||||||
if (!current()) return;
|
if (!current()) return;
|
||||||
const {
|
const {
|
||||||
@@ -174,7 +258,7 @@ export function createTextChat({
|
|||||||
throw Object.assign(new Error('Incomplete response'), {
|
throw Object.assign(new Error('Incomplete response'), {
|
||||||
incomplete: true,
|
incomplete: true,
|
||||||
});
|
});
|
||||||
history = contents;
|
history = [...past, ...contents];
|
||||||
turn = null;
|
turn = null;
|
||||||
state('listening');
|
state('listening');
|
||||||
if (
|
if (
|
||||||
@@ -193,7 +277,7 @@ export function createTextChat({
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
totalCalls += calls.length;
|
totalCalls += calls.length;
|
||||||
if (totalCalls > 16) throw new Error('Tool limit');
|
if (totalCalls > 16) throw tooManySteps('Tool limit');
|
||||||
const pairs = [];
|
const pairs = [];
|
||||||
const invalid = new Map(
|
const invalid = new Map(
|
||||||
(data.invalidCalls || []).map((item) => [item.index, item.error]),
|
(data.invalidCalls || []).map((item) => [item.index, item.error]),
|
||||||
@@ -226,6 +310,7 @@ export function createTextChat({
|
|||||||
: 'globe-view-required',
|
: 'globe-view-required',
|
||||||
};
|
};
|
||||||
} else {
|
} else {
|
||||||
|
arm();
|
||||||
try {
|
try {
|
||||||
if (
|
if (
|
||||||
call.name === 'control_radio' ||
|
call.name === 'control_radio' ||
|
||||||
@@ -284,6 +369,7 @@ export function createTextChat({
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (!current()) return;
|
if (!current()) return;
|
||||||
|
if (result?.ok === true) applied++;
|
||||||
if (!invalid.size && isGlobeViewRefusal(result)) blocked = true;
|
if (!invalid.size && isGlobeViewRefusal(result)) blocked = true;
|
||||||
if (
|
if (
|
||||||
!invalid.size &&
|
!invalid.size &&
|
||||||
@@ -300,21 +386,47 @@ export function createTextChat({
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
contents.push(...protocol.results(pairs));
|
contents.push(...protocol.results(pairs));
|
||||||
|
settled = contents.length;
|
||||||
}
|
}
|
||||||
throw new Error('Round limit');
|
throw tooManySteps('Round limit');
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
recordUsage(null);
|
recordUsage(null);
|
||||||
fail(
|
fail(
|
||||||
error?.status === 504 || owner.signal.aborted
|
error?.code === 'API_KEY_MISSING'
|
||||||
? t('Connection timed out. Try again.')
|
? t('Needs {envVars} — add it in Provider Settings', {
|
||||||
: error?.incomplete
|
envVars: protocol.keyEnv,
|
||||||
? t('Reply interrupted. You can try again.')
|
})
|
||||||
: t('Could not connect. Try again.'),
|
: error?.code === 'CODEX_OAUTH_REQUIRED'
|
||||||
|
? t(
|
||||||
|
'Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.',
|
||||||
|
)
|
||||||
|
: error?.code === 'API_KEY_REJECTED'
|
||||||
|
? t(
|
||||||
|
'The provider rejected the API key. Check it in Provider Settings.',
|
||||||
|
)
|
||||||
|
: error?.code === 'RATE_LIMITED' || error?.status === 429
|
||||||
|
? t('Too many chat requests. Try again in {seconds} s.', {
|
||||||
|
seconds: Math.min(error.retryAfter || 60, 3600),
|
||||||
|
})
|
||||||
|
: error?.status === 504 || owner.signal.aborted
|
||||||
|
? t('Connection timed out. Try again.')
|
||||||
|
: error?.incomplete
|
||||||
|
? t('Reply interrupted. You can try again.')
|
||||||
|
: error?.tooManySteps
|
||||||
|
? applied
|
||||||
|
? t(
|
||||||
|
'That request needed too many steps. Changes already made to the map stay in place.',
|
||||||
|
)
|
||||||
|
: t(
|
||||||
|
'That request needed too many steps. Try a simpler request.',
|
||||||
|
)
|
||||||
|
: t('Could not connect. Try again.'),
|
||||||
);
|
);
|
||||||
} finally {
|
} finally {
|
||||||
recordUsage(null);
|
recordUsage(null);
|
||||||
clearTimeout(deadline);
|
clearTimeout(deadline);
|
||||||
if (turn === owner) turn = null;
|
if (turn === owner) turn = null;
|
||||||
|
if (turnUsage === usage) turnUsage = null;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
@@ -322,7 +434,7 @@ export function createTextChat({
|
|||||||
if (signal?.aborted) return;
|
if (signal?.aborted) return;
|
||||||
active = true;
|
active = true;
|
||||||
const cancelRadioTurn = () => {
|
const cancelRadioTurn = () => {
|
||||||
if (radioPending && radioPending === turn) stop();
|
if (radioPending && radioPending === turn) cancelTurn();
|
||||||
};
|
};
|
||||||
unsubscribeControls?.();
|
unsubscribeControls?.();
|
||||||
unsubscribeVisibility?.();
|
unsubscribeVisibility?.();
|
||||||
@@ -352,6 +464,7 @@ export function createTextChat({
|
|||||||
state('listening');
|
state('listening');
|
||||||
},
|
},
|
||||||
stop,
|
stop,
|
||||||
|
cancelTurn,
|
||||||
sendText(text) {
|
sendText(text) {
|
||||||
if (
|
if (
|
||||||
!active ||
|
!active ||
|
||||||
|
|||||||
@@ -0,0 +1,392 @@
|
|||||||
|
import test from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { createTextChat } from './textChat.js';
|
||||||
|
import { t } from '../i18n/index.js';
|
||||||
|
|
||||||
|
const tick = () => new Promise((resolve) => setImmediate(resolve));
|
||||||
|
const settle = async (count = 8) => {
|
||||||
|
for (let i = 0; i < count; i++) await tick();
|
||||||
|
};
|
||||||
|
const bytes = (value) =>
|
||||||
|
new TextEncoder().encode(
|
||||||
|
typeof value === 'string' ? value : JSON.stringify(value),
|
||||||
|
).length;
|
||||||
|
|
||||||
|
// The wire shapes differ; the turn loop under test is shared.
|
||||||
|
const wire = {
|
||||||
|
openai: {
|
||||||
|
final: (text) => ({
|
||||||
|
content: { role: 'assistant', content: text },
|
||||||
|
finishReason: 'stop',
|
||||||
|
}),
|
||||||
|
calls: (count, prefix = 'call') => ({
|
||||||
|
content: {
|
||||||
|
role: 'assistant',
|
||||||
|
content: null,
|
||||||
|
tool_calls: Array.from({ length: count }, (_, index) => ({
|
||||||
|
id: `${prefix}-${index}`,
|
||||||
|
type: 'function',
|
||||||
|
function: { name: 'fly_to', arguments: '{}' },
|
||||||
|
})),
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
items: (body) => body.messages,
|
||||||
|
userTexts: (items) =>
|
||||||
|
items.filter((item) => item.role === 'user').map((item) => item.content),
|
||||||
|
keyEnv: 'OPENAI_API_KEY',
|
||||||
|
},
|
||||||
|
gemini: {
|
||||||
|
final: (text) => ({
|
||||||
|
content: { role: 'model', parts: [{ text }] },
|
||||||
|
finishReason: 'STOP',
|
||||||
|
}),
|
||||||
|
calls: (count, prefix = 'call') => ({
|
||||||
|
content: {
|
||||||
|
role: 'model',
|
||||||
|
parts: Array.from({ length: count }, (_, index) => ({
|
||||||
|
functionCall: { name: 'fly_to', args: {}, id: `${prefix}-${index}` },
|
||||||
|
})),
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
items: (body) => body.contents,
|
||||||
|
userTexts: (items) =>
|
||||||
|
items
|
||||||
|
.filter((item) => item.role === 'user' && item.parts[0].text)
|
||||||
|
.map((item) => item.parts[0].text),
|
||||||
|
keyEnv: 'GEMINI_API_KEY',
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
function harness(provider, replies, options = {}) {
|
||||||
|
const events = [];
|
||||||
|
const requests = [];
|
||||||
|
const actions = [];
|
||||||
|
const chat = createTextChat({
|
||||||
|
provider,
|
||||||
|
emit: (event) => events.push(event),
|
||||||
|
runAction: async (name, args, context) => {
|
||||||
|
actions.push(name);
|
||||||
|
return options.runAction
|
||||||
|
? options.runAction(name, args, context)
|
||||||
|
: { ok: true };
|
||||||
|
},
|
||||||
|
fetchImpl: async (_url, init) => {
|
||||||
|
const body = JSON.parse(init.body);
|
||||||
|
requests.push(body);
|
||||||
|
const reply = replies(requests.length, body, init);
|
||||||
|
return reply instanceof Response ? reply : Response.json(reply);
|
||||||
|
},
|
||||||
|
...options.chat,
|
||||||
|
});
|
||||||
|
const errors = () => events.filter((event) => event.state === 'error');
|
||||||
|
const completions = (status = 'completed') =>
|
||||||
|
events.filter(
|
||||||
|
(event) => event.type === 'completion' && event.status === status,
|
||||||
|
);
|
||||||
|
return { chat, events, requests, actions, errors, completions };
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const provider of ['openai', 'gemini']) {
|
||||||
|
const w = wire[provider];
|
||||||
|
|
||||||
|
test(`${provider}: a failed turn keeps the conversation for the next message`, async () => {
|
||||||
|
const h = harness(provider, (n) =>
|
||||||
|
n === 2
|
||||||
|
? Response.json({ error: 'unavailable' }, { status: 503 })
|
||||||
|
: w.final(`Reply ${n}`),
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('First');
|
||||||
|
await settle();
|
||||||
|
h.chat.sendText('Second');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.errors().length, 1);
|
||||||
|
assert.equal(h.errors()[0].detail, t('Could not connect. Try again.'));
|
||||||
|
// The session stays usable: no restart is needed before the next send.
|
||||||
|
assert.equal(h.chat.sendText('Third'), true);
|
||||||
|
await settle();
|
||||||
|
assert.deepEqual(w.userTexts(w.items(h.requests[2])), ['First', 'Third']);
|
||||||
|
assert.equal(h.completions().length, 2);
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test(`${provider}: a turn that fails after map actions keeps them for the next message`, async () => {
|
||||||
|
const h = harness(provider, (n) =>
|
||||||
|
n === 1
|
||||||
|
? w.calls(1)
|
||||||
|
: n === 2
|
||||||
|
? Response.json({ error: 'unavailable' }, { status: 503 })
|
||||||
|
: w.final('Done'),
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Fly there');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.actions.length, 1);
|
||||||
|
assert.equal(h.errors().length, 1);
|
||||||
|
h.chat.sendText('Continue');
|
||||||
|
await settle();
|
||||||
|
const items = w.items(h.requests[2]);
|
||||||
|
assert.deepEqual(w.userTexts(items), ['Fly there', 'Continue']);
|
||||||
|
// Call, result, then a note that closes the stopped reply.
|
||||||
|
const note = items.at(-2);
|
||||||
|
const noteText = provider === 'openai' ? note.content : note.parts[0].text;
|
||||||
|
assert.equal(note.role, provider === 'openai' ? 'assistant' : 'model');
|
||||||
|
assert.match(noteText, /map actions above already ran/);
|
||||||
|
assert.equal(items.length, 5);
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test(`${provider}: cancelTurn aborts only the reply in progress`, async () => {
|
||||||
|
let toolSignal;
|
||||||
|
const h = harness(
|
||||||
|
provider,
|
||||||
|
(n) => (n === 2 ? w.calls(1) : w.final(`Reply ${n}`)),
|
||||||
|
{
|
||||||
|
runAction: (_name, _args, context) => {
|
||||||
|
toolSignal = context.signal;
|
||||||
|
return new Promise(() => {});
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('First');
|
||||||
|
await settle();
|
||||||
|
h.chat.sendText('Second');
|
||||||
|
await settle();
|
||||||
|
assert.ok(toolSignal);
|
||||||
|
assert.equal(h.chat.cancelTurn(), true);
|
||||||
|
assert.equal(toolSignal.aborted, true);
|
||||||
|
assert.equal(h.chat.cancelTurn(), false);
|
||||||
|
assert.equal(h.completions('cancelled').length, 1);
|
||||||
|
assert.equal(h.events.at(-2).state, 'listening');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.errors().length, 0);
|
||||||
|
assert.equal(h.chat.sendText('Third'), true);
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.requests.length, 3);
|
||||||
|
assert.deepEqual(w.userTexts(w.items(h.requests[2])), ['First', 'Third']);
|
||||||
|
// Ending the session still starts the next one fresh.
|
||||||
|
h.chat.stop();
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Fresh');
|
||||||
|
await settle();
|
||||||
|
assert.deepEqual(w.userTexts(w.items(h.requests.at(-1))), ['Fresh']);
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
for (const [code, expected] of [
|
||||||
|
[
|
||||||
|
'API_KEY_MISSING',
|
||||||
|
() =>
|
||||||
|
t('Needs {envVars} — add it in Provider Settings', {
|
||||||
|
envVars: w.keyEnv,
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
[
|
||||||
|
'CODEX_OAUTH_REQUIRED',
|
||||||
|
() =>
|
||||||
|
t(
|
||||||
|
'Typed chat is unavailable while GEV_PREFER_CODEX_OAUTH requires ChatGPT OAuth.',
|
||||||
|
),
|
||||||
|
],
|
||||||
|
])
|
||||||
|
test(`${provider}: a 503 ${code} reply names the fix`, async () => {
|
||||||
|
const h = harness(provider, () =>
|
||||||
|
Response.json({ error: 'Provider refused', code }, { status: 503 }),
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Hello');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.errors().length, 1);
|
||||||
|
assert.equal(h.errors()[0].detail, expected());
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
for (const [status, body, headers, expected] of [
|
||||||
|
[
|
||||||
|
502,
|
||||||
|
{ error: 'Refused', code: 'API_KEY_REJECTED' },
|
||||||
|
{},
|
||||||
|
() =>
|
||||||
|
t('The provider rejected the API key. Check it in Provider Settings.'),
|
||||||
|
],
|
||||||
|
[
|
||||||
|
429,
|
||||||
|
{ error: 'Refused', code: 'RATE_LIMITED' },
|
||||||
|
{ 'Retry-After': '42' },
|
||||||
|
() =>
|
||||||
|
t('Too many chat requests. Try again in {seconds} s.', { seconds: 42 }),
|
||||||
|
],
|
||||||
|
[
|
||||||
|
429,
|
||||||
|
{ error: 'Refused' },
|
||||||
|
{},
|
||||||
|
() =>
|
||||||
|
t('Too many chat requests. Try again in {seconds} s.', { seconds: 60 }),
|
||||||
|
],
|
||||||
|
])
|
||||||
|
test(`${provider}: a ${status} ${body.code ?? 'throttle'} reply says what to do`, async () => {
|
||||||
|
const h = harness(provider, () =>
|
||||||
|
Response.json(body, { status, headers }),
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Hello');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.errors()[0].detail, expected());
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test(`${provider}: a gateway timeout without a JSON body still reads as a timeout`, async () => {
|
||||||
|
const h = harness(
|
||||||
|
provider,
|
||||||
|
() => new Response('<html>Gateway Timeout</html>', { status: 504 }),
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Hello');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.errors()[0].detail, t('Connection timed out. Try again.'));
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test(`${provider}: a round with 17 calls hits the call cap before any action`, async () => {
|
||||||
|
const h = harness(provider, () => w.calls(17));
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Do everything');
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.requests.length, 1);
|
||||||
|
assert.deepEqual(h.actions, []);
|
||||||
|
assert.equal(
|
||||||
|
h.errors()[0].detail,
|
||||||
|
t('That request needed too many steps. Try a simpler request.'),
|
||||||
|
);
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test(`${provider}: the round limit says applied map changes stay`, async () => {
|
||||||
|
const h = harness(provider, (n) => w.calls(1, `round-${n}`));
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Keep going');
|
||||||
|
await settle(30);
|
||||||
|
assert.equal(h.requests.length, 8);
|
||||||
|
assert.equal(h.actions.length, 8);
|
||||||
|
assert.equal(
|
||||||
|
h.errors()[0].detail,
|
||||||
|
t(
|
||||||
|
'That request needed too many steps. Changes already made to the map stay in place.',
|
||||||
|
),
|
||||||
|
);
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test(`${provider}: the deadline covers each request, not the whole turn`, async (context) => {
|
||||||
|
context.mock.timers.enable({ apis: ['setTimeout'] });
|
||||||
|
let pending = 0;
|
||||||
|
const h = harness(
|
||||||
|
provider,
|
||||||
|
(n) => (n < 3 ? w.calls(1, `round-${n}`) : w.final('Done')),
|
||||||
|
{
|
||||||
|
chat: { timeoutMs: 50 },
|
||||||
|
runAction: () =>
|
||||||
|
new Promise((resolve) => {
|
||||||
|
pending++;
|
||||||
|
setTimeout(() => resolve({ ok: true }), 40);
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
h.chat.sendText('Two slow steps');
|
||||||
|
for (const expected of [1, 2]) {
|
||||||
|
await settle();
|
||||||
|
assert.equal(pending, expected);
|
||||||
|
context.mock.timers.tick(40);
|
||||||
|
}
|
||||||
|
await settle();
|
||||||
|
assert.equal(h.errors().length, 0);
|
||||||
|
assert.equal(h.completions().length, 1);
|
||||||
|
h.chat.stop();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
test('long history shortens old tool results and keeps every call paired', async () => {
|
||||||
|
const w = wire.openai;
|
||||||
|
let turn = 0;
|
||||||
|
const h = harness(
|
||||||
|
'openai',
|
||||||
|
(n, body) => {
|
||||||
|
const last = body.messages.at(-1);
|
||||||
|
return last.role === 'user'
|
||||||
|
? w.calls(1, `turn-${++turn}`)
|
||||||
|
: w.final('Ok');
|
||||||
|
},
|
||||||
|
{
|
||||||
|
runAction: async () => ({
|
||||||
|
ok: true,
|
||||||
|
say: 'Found',
|
||||||
|
records: ['x'.repeat(11000)],
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
h.chat.start();
|
||||||
|
for (let i = 1; i <= 20; i++) {
|
||||||
|
assert.equal(h.chat.sendText(`Turn ${i}`), true);
|
||||||
|
await settle();
|
||||||
|
}
|
||||||
|
assert.equal(h.completions().length, 20);
|
||||||
|
const messages = h.requests.at(-1).messages;
|
||||||
|
assert.equal(messages[0].role, 'user');
|
||||||
|
assert.ok(messages.length <= 64 + 3);
|
||||||
|
const calls = new Set();
|
||||||
|
for (const message of messages) {
|
||||||
|
for (const call of message.tool_calls || []) calls.add(call.id);
|
||||||
|
if (message.role === 'tool') {
|
||||||
|
assert.ok(calls.delete(message.tool_call_id), message.tool_call_id);
|
||||||
|
const result = JSON.parse(message.content);
|
||||||
|
assert.equal(result.ok, true);
|
||||||
|
assert.equal(result.say, 'Found');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert.deepEqual([...calls], []);
|
||||||
|
const results = messages.filter((message) => message.role === 'tool');
|
||||||
|
// Earlier turns are compact; the latest turns keep their full result.
|
||||||
|
assert.ok(results.slice(0, -2).every((item) => bytes(item.content) <= 1500));
|
||||||
|
assert.ok(results.slice(-2).every((item) => bytes(item.content) > 10000));
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a request over the size cap drops old turns instead of failing', async () => {
|
||||||
|
const w = wire.gemini;
|
||||||
|
const big = (char, size) => char.repeat(size);
|
||||||
|
let turn = 0;
|
||||||
|
const h = harness('gemini', (n, body) => {
|
||||||
|
const last = body.contents.at(-1);
|
||||||
|
const text = last.parts[0].text;
|
||||||
|
if (text && turn < 4) {
|
||||||
|
turn++;
|
||||||
|
return w.final(big('a', 40000));
|
||||||
|
}
|
||||||
|
if (text)
|
||||||
|
return {
|
||||||
|
content: {
|
||||||
|
role: 'model',
|
||||||
|
parts: [
|
||||||
|
{ text: big('b', 380000) },
|
||||||
|
{ functionCall: { name: 'fly_to', args: {}, id: 'large' } },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return w.final('Done');
|
||||||
|
});
|
||||||
|
h.chat.start();
|
||||||
|
for (let i = 1; i <= 5; i++) {
|
||||||
|
h.chat.sendText(`Turn ${i}`);
|
||||||
|
await settle();
|
||||||
|
}
|
||||||
|
assert.equal(h.errors().length, 0);
|
||||||
|
assert.equal(h.completions().length, 5);
|
||||||
|
const request = h.requests.at(-1);
|
||||||
|
assert.ok(bytes(request) <= 512 * 1024);
|
||||||
|
const texts = w.userTexts(request.contents);
|
||||||
|
assert.equal(texts.at(-1), 'Turn 5');
|
||||||
|
assert.ok(!texts.includes('Turn 1'));
|
||||||
|
assert.equal(request.contents[0].role, 'user');
|
||||||
|
assert.ok(request.contents[0].parts[0].text);
|
||||||
|
});
|
||||||
@@ -2,7 +2,9 @@
|
|||||||
export const textProtocols = {
|
export const textProtocols = {
|
||||||
gemini: {
|
gemini: {
|
||||||
endpoint: '/api/gemini/chat',
|
endpoint: '/api/gemini/chat',
|
||||||
|
keyEnv: 'GEMINI_API_KEY',
|
||||||
user: (text) => ({ role: 'user', parts: [{ text }] }),
|
user: (text) => ({ role: 'user', parts: [{ text }] }),
|
||||||
|
note: (text) => ({ role: 'model', parts: [{ text }] }),
|
||||||
isUser: (item) =>
|
isUser: (item) =>
|
||||||
item.role === 'user' &&
|
item.role === 'user' &&
|
||||||
item.parts.some((part) => typeof part.text === 'string'),
|
item.parts.some((part) => typeof part.text === 'string'),
|
||||||
@@ -40,10 +42,32 @@ export const textProtocols = {
|
|||||||
})),
|
})),
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
|
compactResult: (item, maxBytes) =>
|
||||||
|
item.role === 'user' && item.parts.some((part) => part.functionResponse)
|
||||||
|
? {
|
||||||
|
...item,
|
||||||
|
parts: item.parts.map((part) =>
|
||||||
|
part.functionResponse
|
||||||
|
? {
|
||||||
|
...part,
|
||||||
|
functionResponse: {
|
||||||
|
...part.functionResponse,
|
||||||
|
response: boundedToolResult(
|
||||||
|
part.functionResponse.response,
|
||||||
|
maxBytes,
|
||||||
|
),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
: part,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
: item,
|
||||||
},
|
},
|
||||||
openai: {
|
openai: {
|
||||||
endpoint: '/api/openai/chat',
|
endpoint: '/api/openai/chat',
|
||||||
|
keyEnv: 'OPENAI_API_KEY',
|
||||||
user: (content) => ({ role: 'user', content }),
|
user: (content) => ({ role: 'user', content }),
|
||||||
|
note: (content) => ({ role: 'assistant', content }),
|
||||||
isUser: (item) => item.role === 'user',
|
isUser: (item) => item.role === 'user',
|
||||||
request: (messages) => ({ messages }),
|
request: (messages) => ({ messages }),
|
||||||
decode(data) {
|
decode(data) {
|
||||||
@@ -75,6 +99,19 @@ export const textProtocols = {
|
|||||||
tool_call_id: call.id,
|
tool_call_id: call.id,
|
||||||
content: JSON.stringify(result),
|
content: JSON.stringify(result),
|
||||||
})),
|
})),
|
||||||
|
compactResult(item, maxBytes) {
|
||||||
|
if (item.role !== 'tool') return item;
|
||||||
|
let result;
|
||||||
|
try {
|
||||||
|
result = JSON.parse(item.content);
|
||||||
|
} catch {
|
||||||
|
result = { ok: false, error: 'Tool result could not be read' };
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
...item,
|
||||||
|
content: JSON.stringify(boundedToolResult(result, maxBytes)),
|
||||||
|
};
|
||||||
|
},
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -52,8 +52,10 @@ export function bindVoiceMiniPreview({ ui, session }) {
|
|||||||
};
|
};
|
||||||
clear();
|
clear();
|
||||||
return session.subscribe((event) => {
|
return session.subscribe((event) => {
|
||||||
if (session.adapter?.getStartOptions?.().textOnly) return;
|
// Any new session starts empty, even one whose start is still reported
|
||||||
|
// as typed chat while voice takes over.
|
||||||
if (event.type === 'state' && event.state === 'connecting') clear();
|
if (event.type === 'state' && event.state === 'connecting') clear();
|
||||||
|
if (session.adapter?.getStartOptions?.().textOnly) return;
|
||||||
if (event.type !== 'transcript' || !event.text) return;
|
if (event.type !== 'transcript' || !event.text) return;
|
||||||
if (event.role === 'user') show('user', event.text);
|
if (event.role === 'user') show('user', event.text);
|
||||||
if (event.role !== 'assistant') return;
|
if (event.role !== 'assistant') return;
|
||||||
|
|||||||
@@ -23,8 +23,9 @@ test('lastSentence keeps only the final sentence across scripts', () => {
|
|||||||
function fixture({ textOnly = false } = {}) {
|
function fixture({ textOnly = false } = {}) {
|
||||||
const ui = createTestVoiceUi({ panel: true });
|
const ui = createTestVoiceUi({ panel: true });
|
||||||
const listeners = new Set();
|
const listeners = new Set();
|
||||||
|
const mode = { textOnly };
|
||||||
const session = {
|
const session = {
|
||||||
adapter: { getStartOptions: () => ({ textOnly }) },
|
adapter: { getStartOptions: () => ({ textOnly: mode.textOnly }) },
|
||||||
subscribe(fn) {
|
subscribe(fn) {
|
||||||
listeners.add(fn);
|
listeners.add(fn);
|
||||||
return () => listeners.delete(fn);
|
return () => listeners.delete(fn);
|
||||||
@@ -32,7 +33,7 @@ function fixture({ textOnly = false } = {}) {
|
|||||||
};
|
};
|
||||||
const emit = (event) => listeners.forEach((fn) => fn(event));
|
const emit = (event) => listeners.forEach((fn) => fn(event));
|
||||||
const unbind = bindVoiceMiniPreview({ ui, session });
|
const unbind = bindVoiceMiniPreview({ ui, session });
|
||||||
return { ui, emit, unbind, listeners };
|
return { ui, emit, unbind, listeners, mode };
|
||||||
}
|
}
|
||||||
|
|
||||||
test('voice preview shows the last user and assistant sentence', () => {
|
test('voice preview shows the last user and assistant sentence', () => {
|
||||||
@@ -90,3 +91,15 @@ test('typed chat turns do not reach the voice preview', () => {
|
|||||||
assert.deepEqual(f.ui.state.mini, { user: '', assistant: '' });
|
assert.deepEqual(f.ui.state.mini, { user: '', assistant: '' });
|
||||||
assert.equal(f.ui.state.root.voiceExchange, false);
|
assert.equal(f.ui.state.root.voiceExchange, false);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('a new session clears the preview even while the adapter still reports chat', () => {
|
||||||
|
const f = fixture();
|
||||||
|
f.emit({ type: 'transcript', role: 'user', text: 'Show live flights.' });
|
||||||
|
assert.equal(f.ui.state.mini.user, 'Show live flights.');
|
||||||
|
// Voice -> chat -> voice: the next start's connecting can arrive before
|
||||||
|
// the adapter has left typed mode.
|
||||||
|
f.mode.textOnly = true;
|
||||||
|
f.emit({ type: 'state', state: 'connecting' });
|
||||||
|
assert.deepEqual(f.ui.state.mini, { user: '', assistant: '' });
|
||||||
|
assert.equal(f.ui.state.root.voiceExchange, false);
|
||||||
|
});
|
||||||
Reference in new issue
Block a user