mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 10:24:02 +00:00
Merge remote-tracking branch 'upstream/main' into fix/tomtom-cross-site-gate
This commit is contained in:
Sunil committed
commit
e32d2eb9ed
33 files changed
+2492
-39
No files matched your search
+7
-1
@@ -55,8 +55,14 @@ GOOGLE_MAPS_SERVER_API_KEY=
|
||||
# CLIENT-EXPOSED — use a public assets:read token with URL restrictions.
|
||||
CESIUM_ION_TOKEN=
|
||||
|
||||
# Optional: OpenAI Realtime voice control. Do not prefix with VITE_.
|
||||
# Optional: OpenAI Realtime voice control. API key is the default cloud auth.
|
||||
# Provider Settings can instead opt voice into the signed-in local
|
||||
# ChatGPT/Codex OAuth session; the browser still receives only a short-lived
|
||||
# Realtime client secret. Do not prefix with VITE_.
|
||||
OPENAI_API_KEY=
|
||||
# Optional: forbid API-key voice sessions. Select ChatGPT OAuth in Provider
|
||||
# Settings; missing or expired Codex sign-in then fails closed.
|
||||
# GEV_PREFER_CODEX_OAUTH=true
|
||||
OPENAI_REALTIME_MODEL=gpt-realtime-2
|
||||
# Cheaper voice tier, used when the GEV MIC panel's MINI toggle is on.
|
||||
# Model ids and prices are external facts that drift — if OpenAI moves this id,
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Changelog
|
||||
|
||||
- Vector tile sources take their allowed tile origin from the configured
|
||||
`tileJsonUrl`, so repointing only that URL at a mirror loads tiles from the
|
||||
mirror or fails visibly instead of silently using OpenFreeMap. Thanks to
|
||||
[daikaginza](https://github.com/daikaginza) (#935).
|
||||
|
||||
- MCP setup examples use the app's default port, `4173`. Thanks to
|
||||
[Huanyi Xie](https://github.com/xiehuanyi) (#934) and
|
||||
[shirunjies8-png](https://github.com/shirunjies8-png) (#959) for the same fix.
|
||||
@@ -63,6 +68,18 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- Add an experimental ChatGPT/Codex OAuth option for cloud voice in Provider
|
||||
Settings. API-key voice remains the default. Local sign-in can start from
|
||||
the auth button and reports completion, failure, timeout, or expired
|
||||
credentials. OAuth credentials stay on the server; local routes require
|
||||
loopback access and same-site requests. OAuth sessions retain usage data
|
||||
and show unknown cost; caption transcriptions are counted rather than
|
||||
priced into the API-key cap. The auth button keeps keyboard focus after a
|
||||
mode change, and pending sign-in does not discard text in the key fields.
|
||||
Thanks to [Shayan Khan](https://github.com/devv-shayan) for #621 and
|
||||
[TheSmokeDev](https://github.com/TheSmokeDev) for the credential handling
|
||||
from #653.
|
||||
|
||||
- Stdio servers from one install share a panel key so a page read from one
|
||||
process can make requests through another. Concurrent malformed-key repairs
|
||||
now select one winner; unavailable storage or a busy repair retains the
|
||||
|
||||
@@ -235,7 +235,13 @@ _Why cockpit mode exists: you're riding a real aircraft over real terrain — an
|
||||
|
||||
## 🎙️ Talk to It
|
||||
|
||||
> Voice needs an **OpenAI key**. Without one the entire app still runs — the mic button just reports voice is unavailable. The same key drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move.
|
||||
> Voice can use an **OpenAI API key** or, on the same machine, an opt-in signed-in **ChatGPT/Codex OAuth** session. The API-key path remains the default. Selecting ChatGPT OAuth in Provider Settings reuses an existing local sign-in or starts the Codex browser sign-in flow and selects OAuth when it completes. The OpenAI key still drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move.
|
||||
|
||||
The OAuth option is experimental. OpenAI documents ChatGPT plan usage for the [Responses API](https://developers.openai.com/siwc/token-sharing-open-source/models-and-inference); a supported third-party Realtime contract for reusing Codex tokens has not been confirmed. A successful token mint does not establish billing or support terms.
|
||||
|
||||
GEV reads Codex's file credentials but does not write or refresh them. It checks token shape and expiry, and reports expired sign-in as requiring re-authentication. The login button runs `codex login`, permits one pending attempt, and reports exit failures or a two-minute timeout. `CODEX_HOME` must match the directory that Codex uses. If `CODEX_AUTH_JSON` points elsewhere, sign in separately or fix that mismatch before using the button. Codex credential storage must use `auth.json`; Keychain-only and in-memory storage are not readable by GEV. On Windows, the login button may not be able to start an npm-installed Codex; run `codex login` yourself, then select OAuth. See [Codex credential storage](https://learn.chatgpt.com/docs/auth#credential-storage).
|
||||
|
||||
For a live voice check, run `node scripts/qa-voice-auth.mjs http://127.0.0.1:4189 oauth microphone.wav` against your local dev server. Use a WAV with three seconds of leading silence followed by a short spoken reply request. Replace `oauth` with `api-key` to check the API-key path. The check verifies audio input, a completed spoken reply, usage, resource cleanup, and reconnect. It saves screenshots and credential-free results under `.gev-logs/voice-auth/`.
|
||||
|
||||
Click **GEV MIC**, grant the microphone, and just talk. This is more than a voice-controlled remote:
|
||||
|
||||
@@ -243,7 +249,7 @@ Click **GEV MIC**, grant the microphone, and just talk. This is more than a voic
|
||||
- **🎯 Entity Q&A.** Click any plane, ship, or datacenter and ask _"what's this?"_ It answers using the object's live telemetry.
|
||||
- **👁️ Visual grounding.** At street level, it reads a viewport screenshot to identify legible signage and building names, and is instructed never to hallucinate labels.
|
||||
- **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director.
|
||||
- **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token.
|
||||
- **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token. If you choose **USE CHATGPT OAUTH** in Provider Settings, the local server uses that sign-in only to mint the same kind of short-lived Realtime credential, and refuses the OAuth path for non-loopback clients.
|
||||
|
||||
Thirty tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook:
|
||||
|
||||
@@ -429,7 +435,7 @@ Seven keys. Five have a free tier, and the two 🔴 ones are metered:
|
||||
| --- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 🟡 | **Cesium ion** | 🗺️ Google Photorealistic 3D, world terrain, and additional ion-hosted imagery stacks. The free Community plan is for eligible individual, personal/non-commercial use and has quotas | [cesium.com/ion](https://cesium.com/ion) — use a public `assets:read` token and check current [pricing/eligibility](https://cesium.com/platform/cesium-ion/pricing/) |
|
||||
| 🔴 | **Google Maps** | Direct Google Photorealistic 3D + Google place search ([Map Tiles API](https://developers.google.com/maps/documentation/tile)) | [Google Cloud Console](https://console.cloud.google.com/) — URL-restrict it |
|
||||
| 🔴 | **OpenAI** | 🎙️ The voice experience + AI HUD summary. The mini model works; the standard model is noticeably smarter. Want Gemini or another provider behind the mic? PRs welcome | [platform.openai.com](https://platform.openai.com) — metered, see costs below |
|
||||
| 🔴 | **OpenAI** | 🎙️ Voice + AI HUD summary through an API key. Provider Settings also offers **USE CHATGPT OAUTH** for cloud voice without removing the API-key option | [platform.openai.com](https://platform.openai.com) — API-key usage is metered, see costs below |
|
||||
| 🟡 | **AISStream** | 🚢 Live global ships | [aisstream.io](https://aisstream.io) — free signup |
|
||||
| 🟡 | **NASA FIRMS** | 🔥 Live active fires | [firms.modaps.eosdis.nasa.gov](https://firms.modaps.eosdis.nasa.gov/api/map_key/) — free |
|
||||
| 🟡 | **TomTom** | 🚦 Live flow speeds and congestion colors for the simulated traffic layer | [my.tomtom.com](https://my.tomtom.com/keys) — free tier available |
|
||||
@@ -508,7 +514,7 @@ Honest numbers, roughly, as of mid-2026 — always check the provider pricing pa
|
||||
| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, transit feeds, OSRM routing, NOAA and ECMWF weather, OpenStreetMap ALPR mapping, bundled datasets. |
|
||||
| **🟡 The free-key tier** | **$0 with a signup.** AISStream, FIRMS, TomTom, OpenSky, plus Cesium ion for eligible personal/non-commercial use. Provider quotas and eligibility still apply. |
|
||||
| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for Google place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. |
|
||||
| **🔴 OpenAI voice** | **The one that costs real money — so the app meters it for you.** Realtime audio runs a few cents per active minute; an evening of heavy use is single-digit dollars. A live session-spend readout sits next to the mic, with an STD/MINI model toggle, a $2 warning, and a **$5 hard cap that ends the session**. The voice context window is kept deliberately short too. |
|
||||
| **🔴 OpenAI voice** | API-key sessions keep the live spend readout, STD/MINI model toggle, $2 warning, and **$5 hard cap**. ChatGPT OAuth is an explicit choice in Provider Settings. Its usage is tracked without an API-dollar estimate or spend cap. The voice context window stays deliberately short. |
|
||||
|
||||
Google's direct 3D route is surprisingly generous: the first 1,000 Photorealistic
|
||||
3D Tiles sessions each month are currently free, and one root request supports
|
||||
|
||||
@@ -18,6 +18,7 @@ The golden rule: **secret-bearing API keys stay on the server side.** The dev/pr
|
||||
| Key | Where it lives | How the browser uses it |
|
||||
|-----|----------------|--------------------------|
|
||||
| `OPENAI_API_KEY` | Server only | Browser fetches a short-lived **ephemeral** Realtime session token from `/api/realtime/token`; the real key never ships |
|
||||
| ChatGPT sign-in through Codex (optional, experimental) | Codex's own `auth.json` (`CODEX_HOME`, default `~/.codex`); GEV reads it server-side and never writes or refreshes it | Only after **USE CHATGPT OAUTH** is selected in Provider Settings: the server mints the same ephemeral Realtime token with the ChatGPT access token, for loopback clients only |
|
||||
| `AISSTREAM_API_KEY` | Server only | Server holds the AISStream websocket; browser polls the same-origin `/api/vessels` cache |
|
||||
| OpenSky OAuth (`OPENSKY_CLIENT_ID/SECRET`) | Server only | Server mints + refreshes the token behind `/api/flights` |
|
||||
| `GOOGLE_MAPS_SERVER_API_KEY` (optional, #33) | Server only | Server calls Places (`/api/google/nearby-places`, `/api/google/text-search`) and the Street View fallback with this key; falls back to `GOOGLE_MAPS_API_KEY` when unset |
|
||||
@@ -74,6 +75,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur
|
||||
- **Redacted debug logging.** The voice debug log (`.gev-logs/`, gitignored) strips API keys, bearer tokens, client secrets, and image data URLs before writing.
|
||||
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`), quota-bearing (`/api/mapillary/*`, with a per-IP limit on tiles) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control.
|
||||
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
|
||||
- **ChatGPT OAuth voice reuses the local Codex sign-in.** `/api/realtime/oauth-status`, `/api/realtime/oauth-login` and `/api/realtime/token?auth=oauth` answer only loopback connections, behind the same cross-site gate as the token endpoint; the login route also refuses a foreign `Origin`. None of them returns the access token. The login route runs `codex login` (or the executable named by `CODEX_BIN`) with an allowlisted environment that leaves out GEV's provider keys, keeps one attempt pending at a time and cancels it after two minutes or when the server stops. **Limits:** any program on this machine that can reach the server can mint Realtime sessions on the signed-in ChatGPT account, as it can with the API key; a local reverse proxy that hides the client's address defeats the loopback check; and OpenAI has not documented this use of Codex credentials for Realtime, so it may stop working or fall outside the account's terms.
|
||||
|
||||
## Network exposure — the operator threat model
|
||||
|
||||
|
||||
+12
-4
@@ -396,7 +396,8 @@ the speaker (`mayVoiceClaimSpeaker()`). Progress lines stay outside the reply
|
||||
lifecycle and Radio handoff. `turn.span` debug records carry generation and
|
||||
playback latency, silence, preamble and word-count figures. Caption
|
||||
transcription (`OPENAI_REALTIME_TRANSCRIBE_MODEL`, default on) is metered into
|
||||
the session cost and cap. The debug log omits transcripts, text, tool arguments
|
||||
the session cost and cap in API-key sessions; ChatGPT OAuth sessions count
|
||||
captions without pricing them. The debug log omits transcripts, text, tool arguments
|
||||
and complete tool-result bodies (including serialized function outputs) unless
|
||||
`GEV_VOICE_LOG_CONTENT=1`.
|
||||
Tool names, call IDs, protocol event metadata and usage remain available.
|
||||
@@ -4035,7 +4036,7 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
|
||||
`GEV MIC` button (bottom UI) starts an OpenAI Realtime session over WebRTC:
|
||||
|
||||
- **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware holds `OPENAI_API_KEY` and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token.
|
||||
- **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware uses `OPENAI_API_KEY` by default, or the signed-in local ChatGPT/Codex OAuth access token when OAuth was explicitly selected in Provider Settings, and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. Both long-lived credentials stay server-side; OAuth minting is loopback-only. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token.
|
||||
- **Session defaults** (env-tunable): model `gpt-realtime-2` (or `gpt-realtime-2.1-mini` when the MINI tier is selected — see the model-tier entry below), voice `marin`, reasoning effort `low`, semantic VAD with low eagerness, no response interruption, context window truncated to ~3,000 post-instruction tokens with 0.5 retention ratio — the conversational window stays short because map state is fetched live per turn.
|
||||
- **Thirty tools** (argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `next_iss_pass`, and `next_satellite_pass`.
|
||||
|
||||
@@ -4081,7 +4082,10 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
2. **Contacts OFF** → "nearby" means **in view**; "near \<place\>" means a radius around that place. A radius query with Contacts active and no explicit centre is centred on the **active contact**, not the camera.
|
||||
3. **Every count names its scope in words** — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number. `analyst_query` returns `scopeLabel` so this is mechanical. Two different numbers with named scopes are not a contradiction.
|
||||
4. **The loaded-data caveat is stated once when relevant**: counts cover loaded data, and the flights layer loads where you look (appended to `coverage.note` for radius/view scopes over viewport-loaded layers).
|
||||
- **Degradation**: without `OPENAI_API_KEY`, `/api/realtime/token` returns 503 and the mic button surfaces the error; the rest of the app is unaffected.
|
||||
- **Degradation**: API-key mode without `OPENAI_API_KEY` returns 503 as before. Selecting OAuth without a usable local ChatGPT/Codex sign-in starts the local `codex login` browser flow, polls for the resulting local token, and selects OAuth when sign-in completes. If the Codex login flow cannot start or does not complete, the rest of the app is unaffected.
|
||||
- **Login lifecycle and storage**: one pending Codex login is shared by the login and status routes. Process failure, success without readable credentials, and a two-minute timeout each report a clear failure; timeout and server shutdown cancel the owned process. The launcher resolves `CODEX_HOME` before changing directories and refuses a mismatched `CODEX_AUTH_JSON`. GEV reads file credentials only, never writes or refreshes them, and does not require the vendor refresh token. Expired access tokens report `CODEX_OAUTH_REAUTH_REQUIRED`. OAuth status and login routes also use the same-site request gate.
|
||||
- **Auth button focus**: changing voice auth updates the existing controls in place. Enter, Space, and mouse selection keep focus on the auth button; Tab then follows the normal control order. Pending sign-in blocks repeated activation without removing the button from the focus order. Completion does not take focus from another control or discard text in the key fields. `scripts/qa-voice-auth-focus.mjs` checks these paths in Chromium with fixture sign-in responses.
|
||||
- **Support boundary**: the local OAuth option is experimental. OpenAI's documented Sign in with ChatGPT plan-sharing inference endpoint is `/v1/responses`; support for third-party reuse of Codex credentials at `/v1/realtime/client_secrets` remains unconfirmed. Technical success does not establish a supported auth or billing contract.
|
||||
|
||||
### AI HUD Summary (June 2026)
|
||||
|
||||
@@ -4301,7 +4305,7 @@ are omitted rather than framing the wrong part of the globe.
|
||||
- GBFS proxy refuses upstream redirects (`redirect: 'manual'`; any 3xx becomes a 502 and the redirect target is logged server-side only) and enforces its 5 MB response cap while the body streams, cancelling the upstream read past the cap; CCTV health map is bounded.
|
||||
- Proxy error payloads are sanitized (no internal error details returned to clients).
|
||||
- That holds for the OpenAI and CCTV media paths too: `/api/openai/hud-summary` never relays OpenAI's own `error.message`, `/api/realtime/token` passes successful ephemeral-token responses through but answers with a fixed error when minting fails or upstream rejects the request, and a failed CCTV media fetch stores a fixed camera health `message` — `GET /api/cctv/health` serializes that field and the CCTV panel renders it as a status label, so it is a client surface as much as the response body is.
|
||||
- `OPENAI_API_KEY` is server-side only; the browser receives ephemeral Realtime client secrets from `/api/realtime/token`.
|
||||
- `OPENAI_API_KEY` remains server-side only and remains the default voice credential. The optional OAuth mode reads the local ChatGPT/Codex access token server-side on loopback only; the browser receives only ephemeral Realtime client secrets from `/api/realtime/token`.
|
||||
- `AISSTREAM_API_KEY` is server-side only; the browser reads the same-origin `/api/vessels` cache.
|
||||
- `/api/google/nearby-places` keeps the Google key out of Places requests issued for voice scene context.
|
||||
- `/api/google/text-search` keeps the Google key server-side for view-biased Places recovery used by annotation resolution.
|
||||
@@ -4704,6 +4708,10 @@ easier to meet (detection is now on more often), but does not create it.
|
||||
shown dot against the allowed source classes and records phase timings.
|
||||
Positions and segment distances are precomputed, with no new animation-loop
|
||||
allocation. Visible-globe roads use cached terrain without offscreen mesh picks. The road-source label names the geometry being drawn, with partial/unavailable states shown plainly.
|
||||
- Vector tile sources accept tiles only from an allowed origin, which defaults
|
||||
to the configured `tileJsonUrl` origin; an explicit `allowedOrigin` overrides
|
||||
it. Repointing only `tileJsonUrl` loads tiles from that host or fails with an
|
||||
origin error rather than silently falling back to OpenFreeMap.
|
||||
- TileJSON caches successful metadata. Transient failures retry after a
|
||||
five-second cooldown; invalid metadata/origins stay unavailable until the
|
||||
source is cleared. Clear resets metadata and cancels pending requests.
|
||||
|
||||
@@ -347,7 +347,7 @@ case "${OPENSKY_AUTH_MODE}" in
|
||||
echo "OpenSky auth: disabled (anonymous mode)"
|
||||
;;
|
||||
esac
|
||||
[[ -n "${OPENAI_API_KEY}" ]] && echo "OpenAI key (voice + HUD summary): configured" || echo "OpenAI key (voice + HUD summary): not set — GEV MIC disabled"
|
||||
[[ -n "${OPENAI_API_KEY}" ]] && echo "OpenAI key (voice + HUD summary): configured" || echo "OpenAI key (voice + HUD summary): not set — GEV MIC off unless a Codex ChatGPT sign-in is selected in Provider Settings (experimental)"
|
||||
[[ -n "${AISSTREAM_API_KEY}" ]] && echo "AISStream key (live vessels): configured" || echo "AISStream key (live vessels): not set — ships layer empty"
|
||||
if [[ -n "${GOOGLE_MAPS_API_KEY}" ]]; then
|
||||
echo "Startup map: Google Photorealistic 3D Tiles (direct)"
|
||||
|
||||
@@ -36,6 +36,10 @@
|
||||
"src/main.js",
|
||||
"docs/APPLICATION.md",
|
||||
"scripts/qa-application.mjs",
|
||||
"scripts/qa-voice-auth.mjs",
|
||||
"scripts/qa-voice-auth-focus.mjs",
|
||||
"src/codexOauthRealtime.test.mjs",
|
||||
"src/voice/cloudVoiceAuth.test.mjs",
|
||||
"scripts/qa-panel-resize.mjs",
|
||||
"scripts/qa-panelDrag.mjs",
|
||||
"src/tooling/panelResizeQa.test.mjs",
|
||||
|
||||
@@ -300,6 +300,7 @@
|
||||
"server/providers/common/request.js",
|
||||
"server/providers/common/source-root.js",
|
||||
"server/providers/openai.js",
|
||||
"server/providers/openai/codex-auth.js",
|
||||
"server/providers/openai/constants.js",
|
||||
"server/providers/openai/debug-log.js",
|
||||
"server/providers/openai/hud-summary.js",
|
||||
@@ -1360,6 +1361,7 @@
|
||||
"modules": [
|
||||
"src/voice/realtimeController.js",
|
||||
"src/voice/realtimeBackend.js",
|
||||
"src/voice/cloudVoiceAuth.js",
|
||||
"src/voice/control.js",
|
||||
"src/voice/layerSummary.js",
|
||||
"src/voice/voiceCost.js",
|
||||
@@ -1975,6 +1977,7 @@
|
||||
"src/voice/resultDisplay.js",
|
||||
"src/voice/gevRealtime.js",
|
||||
"src/voice/layerSummary.js",
|
||||
"src/voice/cloudVoiceAuth.js",
|
||||
"src/voice/realtimeBackend.js",
|
||||
"src/voice/realtimeController.js",
|
||||
"src/voice/voiceCost.js",
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Check auth-mode keyboard focus with the real settings template and controls.
|
||||
* Run: node --test --test-reporter=spec scripts/qa-voice-auth-focus.mjs
|
||||
* Sign-in responses are fixtures. No credentials or live login are used.
|
||||
*/
|
||||
import assert from 'node:assert/strict';
|
||||
import { existsSync } from 'node:fs';
|
||||
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
||||
import { createServer } from 'node:http';
|
||||
import { after, before, test } from 'node:test';
|
||||
import puppeteer from 'puppeteer';
|
||||
|
||||
const root = new URL('../', import.meta.url);
|
||||
const template = await readFile(
|
||||
new URL('src/ui/templates/provider-settings.html', root),
|
||||
'utf8',
|
||||
);
|
||||
const assets = new Set([
|
||||
'/src/keySetup.js',
|
||||
'/src/voice/cloudVoiceAuth.js',
|
||||
'/src/ui/surfaceKeyboard.js',
|
||||
'/src/ui/styles/provider-settings.css',
|
||||
]);
|
||||
const observations = [];
|
||||
let server;
|
||||
let browser;
|
||||
let origin;
|
||||
|
||||
before(async () => {
|
||||
server = createServer(async (req, res) => {
|
||||
const pathname = new URL(req.url, 'http://localhost').pathname;
|
||||
if (pathname === '/') {
|
||||
res.setHeader('Content-Type', 'text/html; charset=utf-8');
|
||||
res.end(`<!doctype html><html><head><meta charset="utf-8">
|
||||
<title>Voice auth keyboard focus test</title>
|
||||
<link rel="stylesheet" href="/src/ui/styles/provider-settings.css">
|
||||
</head><body>${template}</body></html>`);
|
||||
return;
|
||||
}
|
||||
if (!assets.has(pathname)) {
|
||||
res.statusCode = 404;
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
res.setHeader(
|
||||
'Content-Type',
|
||||
pathname.endsWith('.css') ? 'text/css' : 'text/javascript',
|
||||
);
|
||||
res.end(await readFile(new URL(pathname.slice(1), root)));
|
||||
} catch {
|
||||
res.statusCode = 500;
|
||||
res.end('Could not load the test asset');
|
||||
}
|
||||
});
|
||||
await new Promise((resolve, reject) => {
|
||||
server.once('error', reject);
|
||||
server.listen(0, '127.0.0.1', resolve);
|
||||
});
|
||||
origin = `http://127.0.0.1:${server.address().port}`;
|
||||
const chrome = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
|
||||
browser = await puppeteer.launch({
|
||||
headless: true,
|
||||
executablePath:
|
||||
process.env.PUPPETEER_EXECUTABLE_PATH ||
|
||||
(existsSync(chrome) ? chrome : await puppeteer.executablePath()),
|
||||
});
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
await browser?.close();
|
||||
if (server?.listening) await new Promise((resolve) => server.close(resolve));
|
||||
const output = new URL('.gev-logs/voice-auth-focus/', root);
|
||||
await mkdir(output, { recursive: true });
|
||||
await writeFile(
|
||||
new URL('results.json', output),
|
||||
`${JSON.stringify(observations, null, 2)}\n`,
|
||||
);
|
||||
});
|
||||
|
||||
const scenarios = [
|
||||
{
|
||||
name: 'OAuth to API key',
|
||||
initial: 'oauth',
|
||||
expected: 'api-key',
|
||||
flow: 'unused',
|
||||
},
|
||||
{
|
||||
name: 'API key to existing OAuth',
|
||||
initial: 'api-key',
|
||||
expected: 'oauth',
|
||||
flow: 'existing',
|
||||
},
|
||||
{
|
||||
name: 'API key to OAuth ready at login launch',
|
||||
initial: 'api-key',
|
||||
expected: 'oauth',
|
||||
flow: 'login-ready',
|
||||
},
|
||||
{
|
||||
name: 'API key to OAuth ready after polling',
|
||||
initial: 'api-key',
|
||||
expected: 'oauth',
|
||||
flow: 'poll-ready',
|
||||
},
|
||||
];
|
||||
|
||||
async function mountSettings(page, scenario) {
|
||||
await page.goto(`${origin}/?setup=1`, {
|
||||
waitUntil: 'domcontentloaded',
|
||||
});
|
||||
await page.evaluate(async ({ initial, flow }) => {
|
||||
localStorage.clear();
|
||||
localStorage.setItem('godsEyeView.voice.cloudAuth', initial);
|
||||
window.__focusTestCalls = [];
|
||||
let statusReads = 0;
|
||||
const fetchImpl = async (url) => {
|
||||
window.__focusTestCalls.push(url);
|
||||
let payload;
|
||||
let status = 200;
|
||||
if (url === '/api/setup/status') {
|
||||
payload = {
|
||||
setCount: 0,
|
||||
total: 1,
|
||||
keys: [
|
||||
{
|
||||
id: 'openai',
|
||||
title: 'OPENAI',
|
||||
tier: 'metered',
|
||||
set: false,
|
||||
unlocks: 'Voice control',
|
||||
envVars: ['OPENAI_API_KEY'],
|
||||
getUrl: 'https://example.invalid/',
|
||||
},
|
||||
],
|
||||
};
|
||||
} else if (url === '/api/realtime/oauth-status') {
|
||||
statusReads += 1;
|
||||
if (flow === 'deferred') {
|
||||
return await new Promise((resolve) => {
|
||||
window.__resolveOAuthStatus = (available) =>
|
||||
resolve(
|
||||
new Response(
|
||||
JSON.stringify({ available, error: 'Fixture failure' }),
|
||||
{
|
||||
status: available ? 200 : 503,
|
||||
},
|
||||
),
|
||||
);
|
||||
});
|
||||
}
|
||||
payload = {
|
||||
available:
|
||||
flow === 'existing' || (flow === 'poll-ready' && statusReads > 1),
|
||||
};
|
||||
} else if (url === '/api/realtime/oauth-login') {
|
||||
const available = flow === 'login-ready';
|
||||
status = available ? 200 : 202;
|
||||
payload = { available, pending: !available, started: !available };
|
||||
} else {
|
||||
throw new Error(`Unexpected request: ${url}`);
|
||||
}
|
||||
return new Response(JSON.stringify(payload), { status });
|
||||
};
|
||||
const { initKeySetup } = await import('/src/keySetup.js');
|
||||
window.__focusTestSettings = await initKeySetup({ fetchImpl });
|
||||
}, scenario);
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
document.getElementById('key-setup').classList.contains('visible') &&
|
||||
document.activeElement.matches('input[data-env-var]'),
|
||||
);
|
||||
}
|
||||
|
||||
for (const scenario of scenarios) {
|
||||
for (const key of ['Enter', 'Space', 'Mouse click']) {
|
||||
test(`${scenario.name} keeps keyboard focus after ${key}`, async (t) => {
|
||||
const page = await browser.newPage();
|
||||
try {
|
||||
await mountSettings(page, scenario);
|
||||
|
||||
if (key !== 'Mouse click') {
|
||||
// Tab from the initial key field to the real auth button.
|
||||
await page.keyboard.press('Tab');
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
document.activeElement.matches('[data-cloud-voice-auth-toggle]'),
|
||||
),
|
||||
true,
|
||||
'Keyboard navigation must reach the auth toggle before activation',
|
||||
);
|
||||
}
|
||||
await page.evaluate(() => {
|
||||
const toggle = document.querySelector(
|
||||
'[data-cloud-voice-auth-toggle]',
|
||||
);
|
||||
window.__originalFocusToggle = toggle;
|
||||
window.__focusAtActivation = null;
|
||||
toggle.addEventListener(
|
||||
'click',
|
||||
() => {
|
||||
window.__focusAtActivation = document.activeElement === toggle;
|
||||
},
|
||||
{ capture: true, once: true },
|
||||
);
|
||||
});
|
||||
if (key === 'Mouse click') {
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
getComputedStyle(document.getElementById('key-setup')).opacity ===
|
||||
'1',
|
||||
);
|
||||
await page.click('[data-cloud-voice-auth-toggle]');
|
||||
} else {
|
||||
await page.keyboard.press(key);
|
||||
}
|
||||
await page.waitForFunction(
|
||||
(expected) =>
|
||||
localStorage.getItem('godsEyeView.voice.cloudAuth') === expected &&
|
||||
!document.querySelector('[data-cloud-voice-auth-toggle]')
|
||||
.disabled &&
|
||||
document
|
||||
.querySelector('[data-cloud-voice-auth-toggle]')
|
||||
.getAttribute('aria-disabled') !== 'true',
|
||||
{},
|
||||
scenario.expected,
|
||||
);
|
||||
const selection = await page.evaluate(() => ({
|
||||
focusOnToggleAtActivation: window.__focusAtActivation,
|
||||
focusOnToggle:
|
||||
document.activeElement ===
|
||||
document.querySelector('[data-cloud-voice-auth-toggle]'),
|
||||
activeTag: document.activeElement.tagName,
|
||||
originalToggleConnected: window.__originalFocusToggle.isConnected,
|
||||
requests: window.__focusTestCalls,
|
||||
}));
|
||||
await page.keyboard.press('Tab');
|
||||
const nextTab = await page.evaluate(() => ({
|
||||
focusOnSave: document.activeElement.matches('[data-key-setup-apply]'),
|
||||
activeLabel:
|
||||
document.activeElement.getAttribute('aria-label') ||
|
||||
document.activeElement.textContent.trim(),
|
||||
}));
|
||||
const observed = { scenario: scenario.name, key, selection, nextTab };
|
||||
observations.push(observed);
|
||||
t.diagnostic(JSON.stringify(observed));
|
||||
assert.equal(
|
||||
selection.focusOnToggle,
|
||||
true,
|
||||
'Auth selection must keep focus on its usable toggle',
|
||||
);
|
||||
assert.equal(
|
||||
nextTab.focusOnSave,
|
||||
true,
|
||||
'Tab must continue to SAVE KEYS rather than restart at Close',
|
||||
);
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
for (const available of [true, false]) {
|
||||
test(`pending OAuth ${available ? 'success' : 'failure'} blocks repeats without moving focus or losing a key draft`, async () => {
|
||||
const page = await browser.newPage();
|
||||
try {
|
||||
await mountSettings(page, { initial: 'api-key', flow: 'deferred' });
|
||||
await page.type('input[data-env-var]', 'test-draft-only');
|
||||
await page.keyboard.press('Tab');
|
||||
await page.keyboard.press('Enter');
|
||||
await page.waitForFunction(
|
||||
() => typeof window.__resolveOAuthStatus === 'function',
|
||||
);
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
document.activeElement.matches('[data-cloud-voice-auth-toggle]'),
|
||||
),
|
||||
true,
|
||||
);
|
||||
await page.keyboard.press('Enter');
|
||||
await page.keyboard.press('Space');
|
||||
assert.equal(
|
||||
await page.evaluate(
|
||||
() =>
|
||||
window.__focusTestCalls.filter(
|
||||
(url) => url === '/api/realtime/oauth-status',
|
||||
).length,
|
||||
),
|
||||
1,
|
||||
);
|
||||
await page.keyboard.press('Tab');
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
document.activeElement.matches('[data-key-setup-apply]'),
|
||||
),
|
||||
true,
|
||||
);
|
||||
await page.evaluate(
|
||||
(available) => window.__resolveOAuthStatus(available),
|
||||
available,
|
||||
);
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
document
|
||||
.querySelector('[data-cloud-voice-auth-toggle]')
|
||||
.getAttribute('aria-disabled') === 'false',
|
||||
);
|
||||
const observed = await page.evaluate(() => ({
|
||||
mode: localStorage.getItem('godsEyeView.voice.cloudAuth'),
|
||||
focusOnSave: document.activeElement.matches('[data-key-setup-apply]'),
|
||||
draft: document.querySelector('input[data-env-var]').value,
|
||||
requests: window.__focusTestCalls,
|
||||
}));
|
||||
observations.push({ scenario: 'pending OAuth', available, observed });
|
||||
assert.equal(observed.mode, available ? 'oauth' : 'api-key');
|
||||
assert.equal(
|
||||
observed.focusOnSave,
|
||||
true,
|
||||
'Completing a request must not take focus from another control',
|
||||
);
|
||||
assert.equal(observed.draft, 'test-draft-only');
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Live Realtime auth acceptance with a prerecorded Chromium microphone.
|
||||
* Run against the local dev server after configuring the selected credential:
|
||||
* node scripts/qa-voice-auth.mjs http://127.0.0.1:4189 oauth microphone.wav
|
||||
* Use a short spoken reply request with three seconds of leading silence.
|
||||
* Saves only usage, transport metadata, and screenshots; never credentials.
|
||||
*/
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import puppeteer from 'puppeteer';
|
||||
|
||||
const root = fileURLToPath(new URL('../', import.meta.url));
|
||||
const url = new URL(process.argv[2] || 'http://127.0.0.1:4189');
|
||||
const auth = process.argv[3] || 'oauth';
|
||||
assert.ok(['oauth', 'api-key'].includes(auth), 'Choose oauth or api-key');
|
||||
assert.ok(
|
||||
['127.0.0.1', 'localhost', '[::1]'].includes(url.hostname),
|
||||
'Use a local server',
|
||||
);
|
||||
assert.ok(process.argv[4], 'Provide a prerecorded microphone WAV');
|
||||
const wav = path.resolve(process.argv[4]);
|
||||
assert.ok(fs.existsSync(wav), 'Microphone WAV must exist');
|
||||
url.searchParams.set('welcome', '0');
|
||||
url.searchParams.set('setup', '1');
|
||||
const output = path.join(root, '.gev-logs', 'voice-auth');
|
||||
fs.mkdirSync(output, { recursive: true });
|
||||
const executablePath =
|
||||
process.env.PUPPETEER_EXECUTABLE_PATH ||
|
||||
(fs.existsSync('/Applications/Google Chrome.app/Contents/MacOS/Google Chrome')
|
||||
? '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
|
||||
: puppeteer.executablePath());
|
||||
const browser = await puppeteer.launch({
|
||||
headless: true,
|
||||
executablePath,
|
||||
args: [
|
||||
'--no-sandbox',
|
||||
'--disable-setuid-sandbox',
|
||||
'--use-fake-device-for-media-stream',
|
||||
`--use-file-for-fake-audio-capture=${wav}%noloop`,
|
||||
'--autoplay-policy=no-user-gesture-required',
|
||||
'--disable-background-timer-throttling',
|
||||
'--window-size=1440,900',
|
||||
],
|
||||
});
|
||||
const result = {
|
||||
ok: false,
|
||||
auth,
|
||||
input: 'prerecorded microphone',
|
||||
mints: [],
|
||||
negotiations: [],
|
||||
};
|
||||
let page;
|
||||
const readSession = () =>
|
||||
page.evaluate(async () => {
|
||||
const voice = window.__godsEyeView.voiceCommands;
|
||||
const stats = voice.pc ? [...(await voice.pc.getStats()).values()] : [];
|
||||
const input = stats.find(
|
||||
(entry) => entry.type === 'outbound-rtp' && entry.kind === 'audio',
|
||||
);
|
||||
const source = stats.find(
|
||||
(entry) => entry.type === 'media-source' && entry.kind === 'audio',
|
||||
);
|
||||
const audio = stats.find(
|
||||
(entry) => entry.type === 'inbound-rtp' && entry.kind === 'audio',
|
||||
);
|
||||
const meter = document.getElementById('gev-voice-cost-value');
|
||||
return {
|
||||
...window.__voiceAuthEvidence,
|
||||
connection: voice.pc?.connectionState,
|
||||
microphoneEnabled: voice.stream
|
||||
?.getAudioTracks()
|
||||
.every((track) => track.enabled),
|
||||
inputBytes: input?.bytesSent || 0,
|
||||
inputAudioEnergy: source?.totalAudioEnergy ?? null,
|
||||
outputBytes: audio?.bytesReceived || 0,
|
||||
outputAudioEnergy: audio?.totalAudioEnergy || 0,
|
||||
playbackTime: voice.audioEl?.currentTime || 0,
|
||||
meter: {
|
||||
text: meter?.textContent,
|
||||
usage: meter?.title,
|
||||
visible: Boolean(meter?.getClientRects().length),
|
||||
},
|
||||
};
|
||||
});
|
||||
try {
|
||||
await browser
|
||||
.defaultBrowserContext()
|
||||
.overridePermissions(url.origin, ['microphone']);
|
||||
page = await browser.newPage();
|
||||
await page.setViewport({ width: 1440, height: 900 });
|
||||
page.on('response', (response) => {
|
||||
const route = new URL(response.url()).pathname;
|
||||
if (route === '/api/realtime/token')
|
||||
result.mints.push({
|
||||
status: response.status(),
|
||||
auth: response.headers()['x-gev-voice-auth'],
|
||||
model: response.headers()['x-gev-voice-model'],
|
||||
});
|
||||
if (
|
||||
route === '/v1/realtime/calls' &&
|
||||
response.request().method() === 'POST'
|
||||
)
|
||||
result.negotiations.push({ status: response.status() });
|
||||
});
|
||||
// Observe the application's real transport. Do not replace fetch or replies.
|
||||
await page.evaluateOnNewDocument(() => {
|
||||
window.__voiceAuthEvidence = {
|
||||
speechStopped: 0,
|
||||
transcripts: [],
|
||||
replies: [],
|
||||
responses: [],
|
||||
errors: [],
|
||||
eventCounts: {},
|
||||
};
|
||||
const create = RTCPeerConnection.prototype.createDataChannel;
|
||||
RTCPeerConnection.prototype.createDataChannel = function (...args) {
|
||||
const channel = create.apply(this, args);
|
||||
channel.addEventListener('message', ({ data }) => {
|
||||
let event;
|
||||
try {
|
||||
event = JSON.parse(data);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const evidence = window.__voiceAuthEvidence;
|
||||
evidence.eventCounts[event.type] =
|
||||
(evidence.eventCounts[event.type] || 0) + 1;
|
||||
if (event.type === 'input_audio_buffer.speech_stopped')
|
||||
evidence.speechStopped += 1;
|
||||
if (
|
||||
event.type === 'conversation.item.input_audio_transcription.completed'
|
||||
)
|
||||
evidence.transcripts.push(event.transcript);
|
||||
if (event.type === 'response.output_audio_transcript.done')
|
||||
evidence.replies.push(event.transcript);
|
||||
if (event.type === 'response.done')
|
||||
evidence.responses.push({
|
||||
status: event.response?.status,
|
||||
inputTokens: event.response?.usage?.input_tokens,
|
||||
outputTokens: event.response?.usage?.output_tokens,
|
||||
});
|
||||
if (event.type === 'error')
|
||||
evidence.errors.push({ code: event.error?.code });
|
||||
});
|
||||
return channel;
|
||||
};
|
||||
});
|
||||
await page.goto(url.href, { waitUntil: 'domcontentloaded', timeout: 30_000 });
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__godsEyeView?.voiceCommands &&
|
||||
document.getElementById('loading-screen')?.classList.contains('hidden'),
|
||||
{ timeout: 45_000 },
|
||||
);
|
||||
await page.waitForSelector('[data-cloud-voice-auth-toggle]', {
|
||||
visible: true,
|
||||
});
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
getComputedStyle(document.getElementById('key-setup')).opacity === '1',
|
||||
);
|
||||
const selected = await page.evaluate(
|
||||
() => localStorage.getItem('godsEyeView.voice.cloudAuth') || 'api-key',
|
||||
);
|
||||
if (selected !== auth) {
|
||||
await page.click('[data-cloud-voice-auth-toggle]');
|
||||
await page.waitForFunction(
|
||||
(mode) => localStorage.getItem('godsEyeView.voice.cloudAuth') === mode,
|
||||
{ timeout: 125_000 },
|
||||
auth,
|
||||
);
|
||||
}
|
||||
await page.click('[data-key-setup-close]');
|
||||
await page.click('#gev-voice-button');
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__godsEyeView.voiceCommands.dc?.readyState === 'open' ||
|
||||
window.__godsEyeView.voiceCommands.status === 'error',
|
||||
{ timeout: 35_000 },
|
||||
);
|
||||
assert.equal(
|
||||
await page.evaluate(
|
||||
() => window.__godsEyeView.voiceCommands.dc?.readyState,
|
||||
),
|
||||
'open',
|
||||
'Realtime channel must open',
|
||||
);
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__voiceAuthEvidence.speechStopped > 0 &&
|
||||
window.__voiceAuthEvidence.responses.some(
|
||||
(response) =>
|
||||
response.status === 'completed' && response.outputTokens > 0,
|
||||
),
|
||||
{ timeout: 45_000 },
|
||||
);
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__voiceAuthEvidence.eventCounts['output_audio_buffer.stopped'] > 0,
|
||||
{ timeout: 30_000 },
|
||||
);
|
||||
result.session = await readSession();
|
||||
assert.ok(
|
||||
result.session.inputBytes > 0 && result.session.outputBytes > 0,
|
||||
'Audio must flow in both directions',
|
||||
);
|
||||
assert.ok(
|
||||
result.session.outputAudioEnergy > 0 && result.session.playbackTime > 0,
|
||||
'Reply audio must play',
|
||||
);
|
||||
assert.equal(
|
||||
result.session.meter.text === 'COST UNKNOWN',
|
||||
auth === 'oauth',
|
||||
'Use the correct accounting display',
|
||||
);
|
||||
await page.screenshot({ path: path.join(output, `${auth}-usage.png`) });
|
||||
await page.click('#key-setup-chip');
|
||||
await page.waitForSelector('[data-cloud-voice-auth-toggle]', {
|
||||
visible: true,
|
||||
});
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
getComputedStyle(document.getElementById('key-setup')).opacity === '1',
|
||||
);
|
||||
await page.screenshot({ path: path.join(output, `${auth}-settings.png`) });
|
||||
await page.click('[data-key-setup-close]');
|
||||
result.stopped = await page.evaluate(() => {
|
||||
const voice = window.__godsEyeView.voiceCommands;
|
||||
const pc = voice.pc;
|
||||
const tracks = voice.stream?.getTracks() || [];
|
||||
voice.session.stop();
|
||||
return (
|
||||
pc.connectionState === 'closed' &&
|
||||
tracks.every((track) => track.readyState === 'ended')
|
||||
);
|
||||
});
|
||||
assert.ok(result.stopped, 'Stop must close the peer and microphone tracks');
|
||||
await page.click('#gev-voice-button');
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__godsEyeView.voiceCommands.dc?.readyState === 'open' ||
|
||||
window.__godsEyeView.voiceCommands.status === 'error',
|
||||
{ timeout: 35_000 },
|
||||
);
|
||||
assert.equal(
|
||||
await page.evaluate(
|
||||
() => window.__godsEyeView.voiceCommands.dc?.readyState,
|
||||
),
|
||||
'open',
|
||||
'Reconnect must work',
|
||||
);
|
||||
assert.equal(
|
||||
result.mints.length,
|
||||
2,
|
||||
'Reconnect must request a fresh client secret',
|
||||
);
|
||||
assert.ok(
|
||||
result.mints.every(
|
||||
(mint) =>
|
||||
mint.status === 200 &&
|
||||
mint.auth === (auth === 'oauth' ? 'codex-oauth' : 'env'),
|
||||
),
|
||||
);
|
||||
assert.ok(result.negotiations.every((entry) => entry.status === 201));
|
||||
result.ok = true;
|
||||
} catch (error) {
|
||||
result.error = error.message;
|
||||
if (page) {
|
||||
result.session = await readSession().catch(() => null);
|
||||
result.detail = await page
|
||||
.$eval('#gev-voice-detail', (element) => element.textContent)
|
||||
.catch(() => null);
|
||||
await page
|
||||
.screenshot({ path: path.join(output, `${auth}-failure.png`) })
|
||||
.catch(() => {});
|
||||
}
|
||||
} finally {
|
||||
await page
|
||||
?.evaluate(() => window.__godsEyeView?.voiceCommands?.session?.stop())
|
||||
.catch(() => {});
|
||||
await browser.close();
|
||||
fs.writeFileSync(
|
||||
path.join(output, `${auth}-result.json`),
|
||||
`${JSON.stringify(result, null, 2)}\n`,
|
||||
);
|
||||
}
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
if (!result.ok) process.exitCode = 1;
|
||||
@@ -176,7 +176,9 @@ export function buildCapabilitySummary(
|
||||
? 'Google Photorealistic 3D Tiles through Cesium ion; Bing and world-terrain stacks available'
|
||||
: 'Esri World Imagery (keyless satellite basemap) with keyless terrain',
|
||||
flights,
|
||||
voice: configured('OPENAI_API_KEY') ? 'available' : 'off until an OpenAI key is added',
|
||||
voice: configured('OPENAI_API_KEY')
|
||||
? 'available'
|
||||
: 'off until an OpenAI key is added, or a Codex ChatGPT sign-in is selected in Provider Settings (experimental)',
|
||||
vessels: configured('AISSTREAM_API_KEY') ? 'live AISStream feed' : 'off until an AISStream key is added',
|
||||
fires: configured('FIRMS_MAP_KEY') ? 'live NASA FIRMS feed' : 'off until a FIRMS key is added',
|
||||
traffic: configured('TOMTOM_API_KEY') ? 'live TomTom flow' : 'built-in traffic simulation',
|
||||
|
||||
@@ -1,20 +1,26 @@
|
||||
import { defaultSourceRoot } from './common/source-root.js';
|
||||
import { handleHudSummary } from './openai/hud-summary.js';
|
||||
import { createDebugLogHandler } from './openai/debug-log.js';
|
||||
import { createRealtimeTokenHandler } from './openai/realtime.js';
|
||||
import {
|
||||
createRealtimeOAuthLoginHandler,
|
||||
createRealtimeOAuthStatusHandler,
|
||||
createRealtimeTokenHandler,
|
||||
} from './openai/realtime.js';
|
||||
import { sameSiteGated } from './common/same-site.js';
|
||||
import { createCodexOAuthLogin } from './openai/codex-auth.js';
|
||||
|
||||
/**
|
||||
* Vite plugin: OpenAI Realtime ephemeral client secret.
|
||||
*
|
||||
* Keeps OPENAI_API_KEY server-side while the browser connects to the
|
||||
* Realtime API over WebRTC with a short-lived secret.
|
||||
* Keeps the selected cloud credential server-side while the browser connects
|
||||
* to the Realtime API over WebRTC with a short-lived secret.
|
||||
*/
|
||||
function openAiRealtimeProxy({
|
||||
sourceRoot = defaultSourceRoot,
|
||||
annotationGuidance,
|
||||
realtime = {},
|
||||
} = {}) {
|
||||
const oauthLogin = createCodexOAuthLogin(realtime);
|
||||
function install(middlewares) {
|
||||
// Cost-bearing and log endpoints refuse cross-site browser requests
|
||||
// (see server/providers/common/same-site.js and SECURITY.md).
|
||||
@@ -31,16 +37,29 @@ function openAiRealtimeProxy({
|
||||
createRealtimeTokenHandler({ ...realtime, annotationGuidance }),
|
||||
),
|
||||
);
|
||||
|
||||
middlewares.use(
|
||||
'/api/realtime/oauth-status',
|
||||
sameSiteGated(createRealtimeOAuthStatusHandler({ oauthLogin })),
|
||||
);
|
||||
|
||||
middlewares.use(
|
||||
'/api/realtime/oauth-login',
|
||||
sameSiteGated(createRealtimeOAuthLoginHandler({ oauthLogin })),
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
name: 'openai-realtime-proxy',
|
||||
configureServer(server) {
|
||||
install(server.middlewares);
|
||||
server.httpServer?.once('close', oauthLogin.dispose);
|
||||
},
|
||||
configurePreviewServer(server) {
|
||||
install(server.middlewares);
|
||||
server.httpServer?.once('close', oauthLogin.dispose);
|
||||
},
|
||||
closeBundle: oauthLogin.dispose,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,360 @@
|
||||
import { spawn } from 'node:child_process';
|
||||
import {
|
||||
accessSync,
|
||||
constants as fsConstants,
|
||||
readFileSync,
|
||||
statSync,
|
||||
} from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
export function codexAuthPath({
|
||||
environment = process.env,
|
||||
home = os.homedir(),
|
||||
} = {}) {
|
||||
const explicit = String(environment.CODEX_AUTH_JSON || '').trim();
|
||||
if (explicit) return path.resolve(explicit);
|
||||
const codexHome = String(environment.CODEX_HOME || '').trim();
|
||||
return path.join(
|
||||
codexHome ? path.resolve(codexHome) : path.join(home, '.codex'),
|
||||
'auth.json',
|
||||
);
|
||||
}
|
||||
|
||||
/** An optional operator lock that refuses the metered API-key voice lane. */
|
||||
export function preferCodexOAuth(environment = process.env) {
|
||||
if (!Object.hasOwn(environment, 'GEV_PREFER_CODEX_OAUTH')) return false;
|
||||
const value = String(environment.GEV_PREFER_CODEX_OAUTH || '')
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
if (['1', 'true', 'yes', 'on'].includes(value)) return true;
|
||||
if (['0', 'false', 'no', 'off'].includes(value)) return false;
|
||||
throw new Error(
|
||||
'GEV_PREFER_CODEX_OAUTH must be true or false. Refusing to choose a voice auth mode.',
|
||||
);
|
||||
}
|
||||
|
||||
function codexOAuthError(message, code = 'CODEX_OAUTH_UNAVAILABLE') {
|
||||
const error = new Error(message);
|
||||
error.code = code;
|
||||
return error;
|
||||
}
|
||||
|
||||
export function readCodexOAuthAccessToken({
|
||||
authPath = codexAuthPath(),
|
||||
readFile = readFileSync,
|
||||
statFile = statSync,
|
||||
nowMs = Date.now(),
|
||||
} = {}) {
|
||||
const unavailable =
|
||||
'ChatGPT sign-in is unavailable. Sign in to Codex or the ChatGPT desktop app and try again.';
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(readFile(authPath, 'utf8'));
|
||||
} catch {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
const mode = String(parsed.auth_mode || '')
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
const chatGptMode = mode
|
||||
? mode === 'chatgpt' || mode === 'chatgptauthtokens'
|
||||
: typeof parsed.OPENAI_API_KEY !== 'string';
|
||||
const token = parsed.tokens?.access_token;
|
||||
if (!chatGptMode || typeof token !== 'string' || !token.trim()) {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
|
||||
// This only checks token shape and expiry. OpenAI verifies the signature when
|
||||
// minting the short-lived Realtime credential; we never modify auth.json.
|
||||
const parts = token.split('.');
|
||||
if (
|
||||
parts.length !== 3 ||
|
||||
parts.some((part) => !/^[A-Za-z0-9_-]+$/.test(part))
|
||||
) {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
let payload;
|
||||
try {
|
||||
payload = JSON.parse(Buffer.from(parts[1], 'base64url').toString('utf8'));
|
||||
} catch {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
let expiresAtMs;
|
||||
if (Object.hasOwn(payload, 'exp')) {
|
||||
if (typeof payload.exp !== 'number' || !Number.isFinite(payload.exp)) {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
expiresAtMs = payload.exp * 1000;
|
||||
} else {
|
||||
try {
|
||||
expiresAtMs = statFile(authPath).mtimeMs + 60 * 60 * 1000;
|
||||
} catch {
|
||||
throw codexOAuthError(unavailable);
|
||||
}
|
||||
}
|
||||
if (!Number.isFinite(expiresAtMs) || expiresAtMs <= nowMs + 60_000) {
|
||||
throw codexOAuthError(
|
||||
'ChatGPT sign-in has expired. Re-authenticate in Provider Settings or run codex login and try again.',
|
||||
'CODEX_OAUTH_REAUTH_REQUIRED',
|
||||
);
|
||||
}
|
||||
return token;
|
||||
}
|
||||
|
||||
export function resolveCodexExecutable({
|
||||
environment = process.env,
|
||||
home = os.homedir(),
|
||||
access = accessSync,
|
||||
} = {}) {
|
||||
const explicit = String(environment.CODEX_BIN || '').trim();
|
||||
if (explicit) return explicit;
|
||||
|
||||
for (const candidate of [
|
||||
path.join(home, '.local', 'bin', 'codex'),
|
||||
path.join(
|
||||
home,
|
||||
'.codex',
|
||||
'packages',
|
||||
'standalone',
|
||||
'current',
|
||||
'bin',
|
||||
'codex',
|
||||
),
|
||||
]) {
|
||||
try {
|
||||
access(candidate, fsConstants.X_OK);
|
||||
return candidate;
|
||||
} catch {
|
||||
// Fall through to the next known location, then PATH.
|
||||
}
|
||||
}
|
||||
return 'codex';
|
||||
}
|
||||
|
||||
// What `codex login` needs to find its home, open a browser and reach OpenAI.
|
||||
// The server's own environment also holds every provider secret, so the
|
||||
// child receives only these names.
|
||||
const CODEX_LOGIN_ENV =
|
||||
/^(?:PATH|PATHEXT|HOME|USER|USERNAME|LOGNAME|SHELL|LANG|LANGUAGE|LC_[A-Z_]+|TERM|TZ|TMPDIR|TEMP|TMP|XDG_[A-Z_]+|DISPLAY|WAYLAND_DISPLAY|DBUS_SESSION_BUS_ADDRESS|BROWSER|WSL_[A-Z_]+|WSLENV|(?:HTTPS?|ALL|NO)_PROXY|SSL_CERT_(?:FILE|DIR)|NODE_EXTRA_CA_CERTS|SYSTEMROOT|WINDIR|COMSPEC|USERPROFILE|APPDATA|LOCALAPPDATA|PROGRAMDATA|HOMEDRIVE|HOMEPATH|CODEX_[A-Z_]+)$/i;
|
||||
|
||||
/** The environment `codex login` runs with: the allowlist, then `CODEX_HOME`. */
|
||||
export function codexLoginEnvironment(environment, codexHome) {
|
||||
const env = {};
|
||||
for (const [name, value] of Object.entries(environment || {}))
|
||||
if (CODEX_LOGIN_ENV.test(name) && value !== undefined) env[name] = value;
|
||||
env.CODEX_HOME = codexHome;
|
||||
return env;
|
||||
}
|
||||
|
||||
export function startCodexChatGptLogin({
|
||||
spawnImpl = spawn,
|
||||
environment = process.env,
|
||||
home = os.homedir(),
|
||||
executable = resolveCodexExecutable({ environment, home }),
|
||||
} = {}) {
|
||||
// Codex owns its credential storage. An explicit read path must agree with
|
||||
// the CLI's write path; never silently sign in to another credential store.
|
||||
const configuredHome = String(environment.CODEX_HOME || '').trim();
|
||||
const loginHome = configuredHome
|
||||
? path.resolve(configuredHome)
|
||||
: path.join(home, '.codex');
|
||||
if (
|
||||
codexAuthPath({ environment, home }) !== path.join(loginHome, 'auth.json')
|
||||
) {
|
||||
return Promise.reject(
|
||||
codexOAuthError(
|
||||
'CODEX_AUTH_JSON does not match the Codex login storage path. Set CODEX_HOME to the matching directory or sign in to Codex separately.',
|
||||
'CODEX_OAUTH_STORAGE_MISMATCH',
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
let child;
|
||||
let spawned = false;
|
||||
let finish;
|
||||
const completion = new Promise((done) => {
|
||||
finish = done;
|
||||
});
|
||||
const fail = () => {
|
||||
reject(
|
||||
codexOAuthError(
|
||||
'Could not start ChatGPT sign-in. Check that Codex is installed, or set CODEX_BIN to its executable.',
|
||||
'CODEX_OAUTH_LOGIN_FAILED',
|
||||
),
|
||||
);
|
||||
finish({ exitCode: null });
|
||||
};
|
||||
try {
|
||||
child = spawnImpl(executable, ['login'], {
|
||||
cwd: home,
|
||||
detached: true,
|
||||
stdio: 'ignore',
|
||||
env: codexLoginEnvironment(environment, loginHome),
|
||||
});
|
||||
} catch {
|
||||
fail();
|
||||
return;
|
||||
}
|
||||
|
||||
child.once('error', () => {
|
||||
if (!spawned) fail();
|
||||
else finish({ exitCode: null });
|
||||
});
|
||||
child.once('exit', (exitCode) => finish({ exitCode }));
|
||||
child.once('spawn', () => {
|
||||
spawned = true;
|
||||
child.unref?.();
|
||||
resolve({
|
||||
completion,
|
||||
cancel() {
|
||||
try {
|
||||
child.kill?.('SIGTERM');
|
||||
} catch {
|
||||
/* Already stopped. */
|
||||
}
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** One local login owner shared by the launch and status endpoints. */
|
||||
export function createCodexOAuthLogin({
|
||||
resolveOAuthAccessToken = () => readCodexOAuthAccessToken(),
|
||||
startOAuthLogin = () => startCodexChatGptLogin(),
|
||||
loginTimeoutMs = 120_000,
|
||||
setTimer = setTimeout,
|
||||
clearTimer = clearTimeout,
|
||||
} = {}) {
|
||||
let active = null;
|
||||
let failure = null;
|
||||
let disposed = false;
|
||||
|
||||
function credentials() {
|
||||
try {
|
||||
if (resolveOAuthAccessToken()) return { available: true };
|
||||
} catch (error) {
|
||||
const code = error?.code || 'CODEX_OAUTH_UNAVAILABLE';
|
||||
return {
|
||||
available: false,
|
||||
error: error?.message || 'ChatGPT sign-in is unavailable',
|
||||
code,
|
||||
reauthRequired: code === 'CODEX_OAUTH_REAUTH_REQUIRED',
|
||||
};
|
||||
}
|
||||
return {
|
||||
available: false,
|
||||
error: 'ChatGPT sign-in is unavailable',
|
||||
code: 'CODEX_OAUTH_UNAVAILABLE',
|
||||
reauthRequired: false,
|
||||
};
|
||||
}
|
||||
|
||||
function status() {
|
||||
const current = credentials();
|
||||
if (current.available) return current;
|
||||
if (failure) return { available: false, ...failure, loginFailed: true };
|
||||
return active ? { ...current, pending: true } : current;
|
||||
}
|
||||
|
||||
function finish(attempt, error) {
|
||||
if (active !== attempt) return;
|
||||
clearTimer(attempt.timer);
|
||||
active = null;
|
||||
failure = error;
|
||||
}
|
||||
|
||||
async function start() {
|
||||
if (disposed)
|
||||
throw codexOAuthError(
|
||||
'ChatGPT sign-in is unavailable because the server has stopped.',
|
||||
'CODEX_OAUTH_LOGIN_FAILED',
|
||||
);
|
||||
if (credentials().available)
|
||||
return { available: true, started: false, pending: false };
|
||||
if (active) return { available: false, started: false, pending: true };
|
||||
|
||||
// Reserve the attempt before awaiting spawn so concurrent POSTs share it.
|
||||
const attempt = { process: null, timer: null };
|
||||
active = attempt;
|
||||
failure = null;
|
||||
attempt.timer = setTimer(() => {
|
||||
finish(attempt, {
|
||||
code: 'CODEX_OAUTH_LOGIN_TIMEOUT',
|
||||
error:
|
||||
'ChatGPT sign-in timed out. Click USE CHATGPT OAUTH to try again.',
|
||||
});
|
||||
attempt.process?.cancel();
|
||||
}, loginTimeoutMs);
|
||||
attempt.timer?.unref?.();
|
||||
|
||||
try {
|
||||
const login = await startOAuthLogin();
|
||||
if (!login?.completion || typeof login.cancel !== 'function')
|
||||
throw new Error('Invalid login process');
|
||||
attempt.process = login;
|
||||
if (active !== attempt) {
|
||||
login.cancel();
|
||||
return { ...status(), started: false, pending: false };
|
||||
}
|
||||
Promise.resolve(login.completion).then(
|
||||
(result) => {
|
||||
if (active !== attempt) return;
|
||||
if (result?.exitCode !== 0) {
|
||||
finish(attempt, {
|
||||
code: 'CODEX_OAUTH_LOGIN_FAILED',
|
||||
error:
|
||||
'ChatGPT sign-in did not complete. Try again in Provider Settings, or run codex login separately.',
|
||||
});
|
||||
} else if (!credentials().available) {
|
||||
finish(attempt, {
|
||||
code: 'CODEX_OAUTH_STORAGE_UNAVAILABLE',
|
||||
error:
|
||||
'Codex sign-in finished, but its configured auth.json has no usable ChatGPT token. Check CODEX_HOME and Codex credential storage. GEV can read file storage only.',
|
||||
});
|
||||
} else finish(attempt, null);
|
||||
},
|
||||
() =>
|
||||
finish(attempt, {
|
||||
code: 'CODEX_OAUTH_LOGIN_FAILED',
|
||||
error:
|
||||
'ChatGPT sign-in did not complete. Try again in Provider Settings.',
|
||||
}),
|
||||
);
|
||||
return { available: false, started: true, pending: true };
|
||||
} catch (error) {
|
||||
const code =
|
||||
error?.code === 'CODEX_OAUTH_STORAGE_MISMATCH'
|
||||
? error.code
|
||||
: 'CODEX_OAUTH_LOGIN_FAILED';
|
||||
finish(attempt, {
|
||||
code,
|
||||
error:
|
||||
code === 'CODEX_OAUTH_STORAGE_MISMATCH'
|
||||
? error.message
|
||||
: 'Could not start ChatGPT sign-in. Check that Codex is installed, or set CODEX_BIN to its executable.',
|
||||
});
|
||||
return { ...status(), started: false, pending: false };
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
status,
|
||||
start,
|
||||
dispose() {
|
||||
disposed = true;
|
||||
if (!active) return;
|
||||
const attempt = active;
|
||||
finish(attempt, null);
|
||||
attempt.process?.cancel();
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -14,12 +14,126 @@ import {
|
||||
} from './constants.js';
|
||||
import { realtimeInstructions } from './instructions.js';
|
||||
import { GEV_REALTIME_TOOLS } from './tools.js';
|
||||
import {
|
||||
preferCodexOAuth,
|
||||
readCodexOAuthAccessToken,
|
||||
createCodexOAuthLogin,
|
||||
} from './codex-auth.js';
|
||||
|
||||
const LOOPBACK_ADDRESSES = new Set(['127.0.0.1', '::1', '::ffff:127.0.0.1']);
|
||||
|
||||
function isLoopbackRequest(req) {
|
||||
return LOOPBACK_ADDRESSES.has(String(req.socket?.remoteAddress || ''));
|
||||
}
|
||||
|
||||
function isSameOriginRequest(req) {
|
||||
const origin = String(req.headers?.origin || '').trim();
|
||||
if (!origin) return true;
|
||||
const host = String(req.headers?.host || '').trim();
|
||||
if (!host) return false;
|
||||
const protocol = req.socket?.encrypted ? 'https:' : 'http:';
|
||||
try {
|
||||
return new URL(origin).origin === new URL(`${protocol}//${host}`).origin;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function requestQuery(req) {
|
||||
try {
|
||||
return new URL(req.url || '', 'http://localhost').searchParams;
|
||||
} catch {
|
||||
return new URLSearchParams();
|
||||
}
|
||||
}
|
||||
|
||||
function resolveRealtimeAuthMode(query) {
|
||||
const requested = String(query.get('auth') || 'api-key').toLowerCase();
|
||||
if (requested === 'api-key' || requested === 'oauth') return requested;
|
||||
return null;
|
||||
}
|
||||
|
||||
function createRealtimeOAuthStatusHandler({
|
||||
oauthLogin = createCodexOAuthLogin(),
|
||||
} = {}) {
|
||||
return (req, res) => {
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
if (req.method !== 'GET') {
|
||||
res.statusCode = 405;
|
||||
res.end(JSON.stringify({ error: 'Method not allowed' }));
|
||||
return;
|
||||
}
|
||||
if (!isLoopbackRequest(req)) {
|
||||
res.statusCode = 403;
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
available: false,
|
||||
error: 'ChatGPT OAuth is available only from this machine',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
res.statusCode = 200;
|
||||
res.end(JSON.stringify(oauthLogin.status()));
|
||||
};
|
||||
}
|
||||
|
||||
function createRealtimeOAuthLoginHandler({
|
||||
oauthLogin = createCodexOAuthLogin(),
|
||||
} = {}) {
|
||||
return async (req, res) => {
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
if (req.method !== 'POST') {
|
||||
res.statusCode = 405;
|
||||
res.end(JSON.stringify({ error: 'Method not allowed' }));
|
||||
return;
|
||||
}
|
||||
if (!isLoopbackRequest(req)) {
|
||||
res.statusCode = 403;
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
available: false,
|
||||
error: 'ChatGPT OAuth sign-in is available only from this machine',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (!isSameOriginRequest(req)) {
|
||||
res.statusCode = 403;
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
available: false,
|
||||
error: 'Cross-origin ChatGPT OAuth sign-in is not allowed',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await oauthLogin.start();
|
||||
res.statusCode = result.available ? 200 : result.loginFailed ? 503 : 202;
|
||||
res.end(JSON.stringify(result));
|
||||
} catch {
|
||||
res.statusCode = 503;
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
available: false,
|
||||
error: 'Could not start ChatGPT sign-in',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
function createRealtimeTokenHandler({
|
||||
annotationGuidance,
|
||||
endpoint = 'https://api.openai.com/v1/realtime/client_secrets',
|
||||
fetchImpl = (...args) => fetch(...args),
|
||||
resolveApiKey = () => process.env.OPENAI_API_KEY,
|
||||
resolveOAuthAccessToken = () => readCodexOAuthAccessToken(),
|
||||
environment = process.env,
|
||||
models = {},
|
||||
tools = GEV_REALTIME_TOOLS,
|
||||
} = {}) {
|
||||
@@ -32,15 +146,84 @@ function createRealtimeTokenHandler({
|
||||
return;
|
||||
}
|
||||
|
||||
const query = requestQuery(req);
|
||||
const authMode = resolveRealtimeAuthMode(query);
|
||||
if (!authMode) {
|
||||
res.statusCode = 400;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(JSON.stringify({ error: 'Unsupported cloud voice auth mode' }));
|
||||
return;
|
||||
}
|
||||
|
||||
let oauthPinned;
|
||||
try {
|
||||
oauthPinned = preferCodexOAuth(environment);
|
||||
} catch (error) {
|
||||
res.statusCode = 503;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(JSON.stringify({ error: error.message }));
|
||||
return;
|
||||
}
|
||||
if (oauthPinned && authMode !== 'oauth') {
|
||||
res.statusCode = 503;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error:
|
||||
'ChatGPT OAuth is required by GEV_PREFER_CODEX_OAUTH. Select USE CHATGPT OAUTH in Provider Settings.',
|
||||
code: 'CODEX_OAUTH_REQUIRED',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
// Per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). On by default; 0 disables.
|
||||
if (!enforceRateLimit(openAiRateLimiter(), req, res)) return;
|
||||
|
||||
const apiKey = resolveApiKey();
|
||||
if (!apiKey) {
|
||||
res.statusCode = 503;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(JSON.stringify({ error: 'OPENAI_API_KEY is not set' }));
|
||||
return;
|
||||
let bearer;
|
||||
if (authMode === 'oauth') {
|
||||
if (!isLoopbackRequest(req)) {
|
||||
res.statusCode = 403;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: 'ChatGPT OAuth voice is available only from this machine',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
bearer = resolveOAuthAccessToken();
|
||||
} catch (error) {
|
||||
res.statusCode = 503;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: error?.message || 'ChatGPT sign-in is unavailable',
|
||||
code: error?.code || 'CODEX_OAUTH_UNAVAILABLE',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (!bearer) {
|
||||
res.statusCode = 503;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: 'ChatGPT sign-in is unavailable',
|
||||
code: 'CODEX_OAUTH_UNAVAILABLE',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
} else {
|
||||
bearer = resolveApiKey();
|
||||
if (!bearer) {
|
||||
res.statusCode = 503;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(JSON.stringify({ error: 'OPENAI_API_KEY is not set' }));
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Voice model tier, requested by the client as ?tier=standard|mini.
|
||||
@@ -49,15 +232,7 @@ function createRealtimeTokenHandler({
|
||||
// bad querystring degrades to a normal session rather than a dead mic.
|
||||
// The env overrides stay authoritative per tier (see .env.example) —
|
||||
// a wrong upstream model id is then a config fix, not a code change.
|
||||
const requestedTier = (() => {
|
||||
try {
|
||||
return new URL(req.url || '', 'http://localhost').searchParams.get(
|
||||
'tier',
|
||||
);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
})();
|
||||
const requestedTier = query.get('tier');
|
||||
const tier = resolveVoiceModel(requestedTier).tier;
|
||||
const model =
|
||||
tier === 'mini'
|
||||
@@ -135,7 +310,7 @@ function createRealtimeTokenHandler({
|
||||
redirect: 'error',
|
||||
signal: AbortSignal.timeout(30_000),
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
Authorization: `Bearer ${bearer}`,
|
||||
'Content-Type': 'application/json',
|
||||
'OpenAI-Safety-Identifier': 'gev-local-dev',
|
||||
},
|
||||
@@ -149,6 +324,10 @@ function createRealtimeTokenHandler({
|
||||
// case where a bogus ?tier= was silently downgraded to standard.
|
||||
res.setHeader('X-GEV-Voice-Tier', tier);
|
||||
res.setHeader('X-GEV-Voice-Model', model);
|
||||
res.setHeader(
|
||||
'X-GEV-Voice-Auth',
|
||||
authMode === 'oauth' ? 'codex-oauth' : 'env',
|
||||
);
|
||||
// Captions are billed separately; the client meters them with this id.
|
||||
res.setHeader(
|
||||
'X-GEV-Voice-Transcribe-Model',
|
||||
@@ -183,4 +362,8 @@ function createRealtimeTokenHandler({
|
||||
};
|
||||
}
|
||||
|
||||
export { createRealtimeTokenHandler };
|
||||
export {
|
||||
createRealtimeOAuthLoginHandler,
|
||||
createRealtimeOAuthStatusHandler,
|
||||
createRealtimeTokenHandler,
|
||||
};
|
||||
@@ -0,0 +1,273 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import test from 'node:test';
|
||||
import path from 'node:path';
|
||||
import {
|
||||
codexAuthPath,
|
||||
preferCodexOAuth,
|
||||
readCodexOAuthAccessToken,
|
||||
resolveCodexExecutable,
|
||||
startCodexChatGptLogin,
|
||||
} from '../server/providers/openai/codex-auth.js';
|
||||
|
||||
const nowMs = Date.UTC(2026, 8, 27);
|
||||
const jwt = (payload) =>
|
||||
`${Buffer.from('{}').toString('base64url')}.${Buffer.from(JSON.stringify(payload)).toString('base64url')}.signature`;
|
||||
const auth = (token, extras = {}) =>
|
||||
JSON.stringify({
|
||||
auth_mode: 'chatgpt',
|
||||
tokens: { access_token: token, refresh_token: 'refresh-fixture' },
|
||||
...extras,
|
||||
});
|
||||
|
||||
test('optional Codex OAuth pin fails closed on ambiguous values', () => {
|
||||
assert.equal(preferCodexOAuth({}), false);
|
||||
assert.equal(preferCodexOAuth({ GEV_PREFER_CODEX_OAUTH: 'true' }), true);
|
||||
assert.equal(preferCodexOAuth({ GEV_PREFER_CODEX_OAUTH: 'off' }), false);
|
||||
assert.throws(
|
||||
() => preferCodexOAuth({ GEV_PREFER_CODEX_OAUTH: 'maybe' }),
|
||||
/must be true or false/,
|
||||
);
|
||||
});
|
||||
|
||||
test('Codex auth path follows explicit auth file, CODEX_HOME, then home directory', () => {
|
||||
assert.equal(
|
||||
codexAuthPath({
|
||||
environment: { CODEX_AUTH_JSON: '/tmp/custom-auth.json' },
|
||||
home: '/home/fixture',
|
||||
}),
|
||||
path.resolve('/tmp/custom-auth.json'),
|
||||
);
|
||||
assert.equal(
|
||||
codexAuthPath({
|
||||
environment: { CODEX_HOME: '/tmp/codex-home' },
|
||||
home: '/home/fixture',
|
||||
}),
|
||||
path.join(path.resolve('/tmp/codex-home'), 'auth.json'),
|
||||
);
|
||||
assert.equal(
|
||||
codexAuthPath({ environment: {}, home: '/home/fixture' }),
|
||||
path.join('/home/fixture', '.codex', 'auth.json'),
|
||||
);
|
||||
});
|
||||
|
||||
test('Codex OAuth reader accepts a current ChatGPT token and rejects unusable credentials', () => {
|
||||
const current = jwt({ exp: nowMs / 1000 + 3600 });
|
||||
assert.equal(
|
||||
readCodexOAuthAccessToken({
|
||||
authPath: '/unused',
|
||||
nowMs,
|
||||
readFile: () => auth(current),
|
||||
}),
|
||||
current,
|
||||
);
|
||||
assert.equal(
|
||||
readCodexOAuthAccessToken({
|
||||
authPath: '/unused',
|
||||
nowMs,
|
||||
readFile: () =>
|
||||
JSON.stringify({
|
||||
auth_mode: 'chatgpt',
|
||||
tokens: { access_token: current },
|
||||
}),
|
||||
}),
|
||||
current,
|
||||
'GEV does not require or consume the vendor refresh token',
|
||||
);
|
||||
for (const readFile of [
|
||||
() => '{bad json',
|
||||
() => JSON.stringify({ tokens: {} }),
|
||||
() => auth(current, { auth_mode: 'apikey' }),
|
||||
() =>
|
||||
JSON.stringify({
|
||||
OPENAI_API_KEY: 'sk-fixture',
|
||||
tokens: { access_token: current, refresh_token: 'refresh-fixture' },
|
||||
}),
|
||||
() => auth('opaque-token'),
|
||||
() => auth('header.not-json.signature'),
|
||||
() => auth(jwt({ exp: 'tomorrow' })),
|
||||
() => {
|
||||
throw new Error('missing');
|
||||
},
|
||||
]) {
|
||||
assert.throws(
|
||||
() => readCodexOAuthAccessToken({ authPath: '/unused', readFile, nowMs }),
|
||||
/ChatGPT sign-in is unavailable/,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('Codex OAuth reader enforces expiry margin and bounds tokens without exp by file age', () => {
|
||||
for (const exp of [nowMs / 1000 - 1, nowMs / 1000 + 59]) {
|
||||
assert.throws(
|
||||
() => {
|
||||
readCodexOAuthAccessToken({
|
||||
authPath: '/unused',
|
||||
nowMs,
|
||||
readFile: () => auth(jwt({ exp })),
|
||||
});
|
||||
},
|
||||
(error) => {
|
||||
assert.match(error.message, /sign-in has expired/);
|
||||
assert.equal(error.code, 'CODEX_OAUTH_REAUTH_REQUIRED');
|
||||
return true;
|
||||
},
|
||||
);
|
||||
}
|
||||
const noExp = jwt({ sub: 'fixture' });
|
||||
const base = { authPath: '/unused', nowMs, readFile: () => auth(noExp) };
|
||||
assert.equal(
|
||||
readCodexOAuthAccessToken({
|
||||
...base,
|
||||
statFile: () => ({ mtimeMs: nowMs }),
|
||||
}),
|
||||
noExp,
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
readCodexOAuthAccessToken({
|
||||
...base,
|
||||
statFile: () => ({ mtimeMs: nowMs - 3600_000 }),
|
||||
}),
|
||||
/sign-in has expired/,
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
readCodexOAuthAccessToken({
|
||||
...base,
|
||||
statFile: () => {
|
||||
throw new Error('missing');
|
||||
},
|
||||
}),
|
||||
/sign-in is unavailable/,
|
||||
);
|
||||
});
|
||||
|
||||
test('Codex login resolves an installed executable before PATH and starts browser auth detached', async () => {
|
||||
const checked = [];
|
||||
assert.equal(
|
||||
resolveCodexExecutable({
|
||||
environment: {},
|
||||
home: '/home/fixture',
|
||||
access(candidate) {
|
||||
checked.push(candidate);
|
||||
if (candidate.endsWith('/.local/bin/codex')) return;
|
||||
throw new Error('missing');
|
||||
},
|
||||
}),
|
||||
path.join('/home/fixture', '.local', 'bin', 'codex'),
|
||||
);
|
||||
assert.equal(checked.length, 1);
|
||||
|
||||
const child = new EventEmitter();
|
||||
let unref = 0;
|
||||
child.unref = () => {
|
||||
unref += 1;
|
||||
};
|
||||
const calls = [];
|
||||
const pending = startCodexChatGptLogin({
|
||||
executable: '/fixture/codex',
|
||||
home: '/home/fixture',
|
||||
environment: {
|
||||
PATH: '/fixture',
|
||||
https_proxy: 'http://proxy.fixture:8080',
|
||||
OPENAI_API_KEY: 'fixture-api-key',
|
||||
GOOGLE_MAPS_SERVER_API_KEY: 'fixture-google-key',
|
||||
OPENSKY_CLIENT_SECRET: 'fixture-opensky-secret',
|
||||
},
|
||||
spawnImpl(command, args, options) {
|
||||
calls.push({ command, args, options });
|
||||
queueMicrotask(() => child.emit('spawn'));
|
||||
return child;
|
||||
},
|
||||
});
|
||||
const login = await pending;
|
||||
assert.equal(unref, 1);
|
||||
assert.equal(calls[0].command, '/fixture/codex');
|
||||
assert.deepEqual(calls[0].args, ['login']);
|
||||
assert.equal(calls[0].options.detached, true);
|
||||
assert.equal(calls[0].options.stdio, 'ignore');
|
||||
assert.deepEqual(
|
||||
calls[0].options.env,
|
||||
{
|
||||
PATH: '/fixture',
|
||||
https_proxy: 'http://proxy.fixture:8080',
|
||||
CODEX_HOME: path.join('/home/fixture', '.codex'),
|
||||
},
|
||||
'provider secrets stay out of the login process',
|
||||
);
|
||||
child.emit('exit', 1);
|
||||
assert.deepEqual(await login.completion, { exitCode: 1 });
|
||||
});
|
||||
|
||||
test('Codex login refuses a read/write storage mismatch before spawning', async () => {
|
||||
await assert.rejects(
|
||||
startCodexChatGptLogin({
|
||||
environment: { CODEX_AUTH_JSON: '/tmp/another-auth.json' },
|
||||
home: '/home/fixture',
|
||||
executable: 'codex',
|
||||
spawnImpl: () => assert.fail('mismatched storage must not launch login'),
|
||||
}),
|
||||
{ code: 'CODEX_OAUTH_STORAGE_MISMATCH' },
|
||||
);
|
||||
});
|
||||
|
||||
test('Codex login resolves relative CODEX_HOME before changing the child directory', async () => {
|
||||
const child = new EventEmitter();
|
||||
const environment = { CODEX_HOME: './fixture-codex' };
|
||||
const login = await startCodexChatGptLogin({
|
||||
environment,
|
||||
executable: 'codex',
|
||||
spawnImpl(_command, _args, options) {
|
||||
assert.equal(
|
||||
options.env.CODEX_HOME,
|
||||
path.resolve(environment.CODEX_HOME),
|
||||
);
|
||||
queueMicrotask(() => child.emit('spawn'));
|
||||
return child;
|
||||
},
|
||||
});
|
||||
assert.equal(
|
||||
environment.CODEX_HOME,
|
||||
'./fixture-codex',
|
||||
'caller environment is unchanged',
|
||||
);
|
||||
child.emit('exit', 0);
|
||||
assert.deepEqual(await login.completion, { exitCode: 0 });
|
||||
});
|
||||
|
||||
test('Codex login hides spawn errors and can cancel only its owned child', async () => {
|
||||
const child = new EventEmitter();
|
||||
await assert.rejects(
|
||||
startCodexChatGptLogin({
|
||||
executable: 'codex',
|
||||
spawnImpl() {
|
||||
queueMicrotask(() =>
|
||||
child.emit('error', new Error('private-token-fixture')),
|
||||
);
|
||||
return child;
|
||||
},
|
||||
}),
|
||||
(error) =>
|
||||
error.code === 'CODEX_OAUTH_LOGIN_FAILED' &&
|
||||
!error.message.includes('private-token-fixture'),
|
||||
);
|
||||
|
||||
const running = new EventEmitter();
|
||||
let cancelled = 0;
|
||||
running.kill = (signal) => {
|
||||
assert.equal(signal, 'SIGTERM');
|
||||
cancelled += 1;
|
||||
running.emit('exit', null);
|
||||
};
|
||||
const login = await startCodexChatGptLogin({
|
||||
executable: 'codex',
|
||||
spawnImpl() {
|
||||
queueMicrotask(() => running.emit('spawn'));
|
||||
return running;
|
||||
},
|
||||
});
|
||||
login.cancel();
|
||||
assert.equal(cancelled, 1);
|
||||
assert.deepEqual(await login.completion, { exitCode: null });
|
||||
});
|
||||
+179
@@ -1,4 +1,8 @@
|
||||
import { createSurfaceKeyboard } from './ui/surfaceKeyboard.js';
|
||||
import {
|
||||
readStoredCloudVoiceAuthMode,
|
||||
writeStoredCloudVoiceAuthMode,
|
||||
} from './voice/cloudVoiceAuth.js';
|
||||
|
||||
/**
|
||||
* The POWER UP surface — paste a key, get a power.
|
||||
@@ -38,6 +42,52 @@ export function collectKeyUpdates(fields) {
|
||||
return updates;
|
||||
}
|
||||
|
||||
function sleepWithSignal(ms, signal) {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (signal?.aborted) {
|
||||
reject(signal.reason || new DOMException('Aborted', 'AbortError'));
|
||||
return;
|
||||
}
|
||||
const onAbort = () => {
|
||||
clearTimeout(timer);
|
||||
reject(signal.reason || new DOMException('Aborted', 'AbortError'));
|
||||
};
|
||||
const timer = setTimeout(() => {
|
||||
signal?.removeEventListener('abort', onAbort);
|
||||
resolve();
|
||||
}, ms);
|
||||
signal?.addEventListener('abort', onAbort, { once: true });
|
||||
});
|
||||
}
|
||||
|
||||
export async function waitForChatGptOAuth({
|
||||
fetchImpl,
|
||||
signal,
|
||||
timeoutMs = 120_000,
|
||||
pollMs = 1_000,
|
||||
now = () => Date.now(),
|
||||
sleep = sleepWithSignal,
|
||||
} = {}) {
|
||||
const deadline = now() + Math.max(0, timeoutMs);
|
||||
while (!signal?.aborted && now() <= deadline) {
|
||||
const response = await fetchImpl('/api/realtime/oauth-status', {
|
||||
cache: 'no-store',
|
||||
signal,
|
||||
});
|
||||
const payload = await response.json().catch(() => ({}));
|
||||
if (signal?.aborted) return false;
|
||||
if (response.ok && payload.available) return true;
|
||||
if (!response.ok || payload.loginFailed) {
|
||||
throw new Error(
|
||||
payload.error || 'Could not check ChatGPT sign-in. Try again.',
|
||||
);
|
||||
}
|
||||
if (now() >= deadline) break;
|
||||
await sleep(Math.max(0, pollMs), signal);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* After the FIRST Google key lands, the restart's reload should boot the
|
||||
* photoreal default — not faithfully restore the auto-selected keyless OSM
|
||||
@@ -61,6 +111,21 @@ export function stripKeylessBasemapFromHash(hash) {
|
||||
|
||||
const TIER_DOTS = Object.freeze({ metered: '🔴', free: '🟡' });
|
||||
|
||||
function updateCloudVoiceAuthControl(auth, busy = false) {
|
||||
const mode = readStoredCloudVoiceAuthMode();
|
||||
const state = auth.querySelector('.key-setup-cloud-auth-state');
|
||||
const toggle = auth.querySelector('[data-cloud-voice-auth-toggle]');
|
||||
state.textContent =
|
||||
mode === 'oauth' ? 'VOICE AUTH · CHATGPT OAUTH' : 'VOICE AUTH · API KEY';
|
||||
toggle.textContent = mode === 'oauth' ? 'USE API KEY' : 'USE CHATGPT OAUTH';
|
||||
toggle.title =
|
||||
mode === 'oauth'
|
||||
? 'Use OPENAI_API_KEY for the next cloud voice session'
|
||||
: 'Use the signed-in local ChatGPT/Codex OAuth session for the next cloud voice session';
|
||||
// Keep the control focusable while the delegated handler blocks repeats.
|
||||
toggle.setAttribute('aria-disabled', String(busy));
|
||||
}
|
||||
|
||||
/** Build one key row. All content is our own registry text, set via textContent. */
|
||||
function buildRow(documentRef, key) {
|
||||
const row = documentRef.createElement('section');
|
||||
@@ -144,6 +209,19 @@ function buildRow(documentRef, key) {
|
||||
}
|
||||
row.append(fields);
|
||||
}
|
||||
if (key.id === 'openai') {
|
||||
const auth = documentRef.createElement('div');
|
||||
auth.className = 'key-setup-cloud-auth';
|
||||
const state = documentRef.createElement('span');
|
||||
state.className = 'key-setup-cloud-auth-state';
|
||||
const toggle = documentRef.createElement('button');
|
||||
toggle.type = 'button';
|
||||
toggle.className = 'key-setup-cloud-auth-toggle';
|
||||
toggle.dataset.cloudVoiceAuthToggle = 'true';
|
||||
auth.append(state, toggle);
|
||||
updateCloudVoiceAuthControl(auth);
|
||||
row.append(auth);
|
||||
}
|
||||
return row;
|
||||
}
|
||||
|
||||
@@ -224,8 +302,14 @@ export async function initKeySetup({
|
||||
const statusLine = root.querySelector('[data-key-setup-status]');
|
||||
const defaultStatusText = statusLine?.textContent || '';
|
||||
let busy = false;
|
||||
let oauthBusy = false;
|
||||
let open = false;
|
||||
|
||||
const syncCloudVoiceAuth = () => {
|
||||
for (const auth of rowsHost.querySelectorAll('.key-setup-cloud-auth'))
|
||||
updateCloudVoiceAuthControl(auth, oauthBusy);
|
||||
};
|
||||
|
||||
const render = (nextStatus) => {
|
||||
if (disposed) return;
|
||||
status = nextStatus;
|
||||
@@ -239,6 +323,7 @@ export async function initKeySetup({
|
||||
rowsHost.textContent = '';
|
||||
for (const key of status.keys || [])
|
||||
rowsHost.append(buildRow(documentRef, key));
|
||||
syncCloudVoiceAuth();
|
||||
};
|
||||
|
||||
const visible = () =>
|
||||
@@ -367,6 +452,100 @@ export async function initKeySetup({
|
||||
applyButton?.addEventListener('click', onApply);
|
||||
// Remove buttons are rendered per row; delegate so re-renders stay wired.
|
||||
rowsHost?.addEventListener('click', (event) => {
|
||||
const authButton = event.target?.closest?.(
|
||||
'[data-cloud-voice-auth-toggle]',
|
||||
);
|
||||
if (authButton && !disposed) {
|
||||
if (oauthBusy) return;
|
||||
void (async () => {
|
||||
const current = readStoredCloudVoiceAuthMode();
|
||||
if (current === 'oauth') {
|
||||
writeStoredCloudVoiceAuthMode('api-key');
|
||||
syncCloudVoiceAuth();
|
||||
say('Cloud voice will use OPENAI_API_KEY on the next session.');
|
||||
return;
|
||||
}
|
||||
oauthBusy = true;
|
||||
syncCloudVoiceAuth();
|
||||
say('Checking local ChatGPT OAuth sign-in…');
|
||||
try {
|
||||
const response = await doFetch('/api/realtime/oauth-status', {
|
||||
cache: 'no-store',
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
const payload = await response.json().catch(() => ({}));
|
||||
if (lifetime.signal.aborted) return;
|
||||
if (!response.ok)
|
||||
throw new Error(
|
||||
payload.error || 'Could not check ChatGPT sign-in.',
|
||||
);
|
||||
if (payload.available) {
|
||||
writeStoredCloudVoiceAuthMode('oauth');
|
||||
syncCloudVoiceAuth();
|
||||
say(
|
||||
'ChatGPT OAuth selected for cloud voice. Your API key stays saved and available.',
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
say(
|
||||
payload.code === 'CODEX_OAUTH_REAUTH_REQUIRED'
|
||||
? 'ChatGPT sign-in expired. Opening sign-in again…'
|
||||
: 'Opening ChatGPT sign-in in your browser…',
|
||||
);
|
||||
const loginResponse = await doFetch('/api/realtime/oauth-login', {
|
||||
method: 'POST',
|
||||
cache: 'no-store',
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
const loginPayload = await loginResponse.json().catch(() => ({}));
|
||||
if (lifetime.signal.aborted) return;
|
||||
if (!loginResponse.ok) {
|
||||
say(
|
||||
loginPayload.error ||
|
||||
payload.error ||
|
||||
'Could not start ChatGPT sign-in on this machine.',
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (loginPayload.available) {
|
||||
writeStoredCloudVoiceAuthMode('oauth');
|
||||
syncCloudVoiceAuth();
|
||||
say(
|
||||
'ChatGPT OAuth selected for cloud voice. Your API key stays saved and available.',
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
say(
|
||||
'Finish ChatGPT sign-in in the browser. Waiting for it to complete…',
|
||||
);
|
||||
const available = await waitForChatGptOAuth({
|
||||
fetchImpl: doFetch,
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
if (lifetime.signal.aborted) return;
|
||||
if (!available) {
|
||||
say(
|
||||
'ChatGPT sign-in timed out. Click USE CHATGPT OAUTH to try again.',
|
||||
);
|
||||
return;
|
||||
}
|
||||
writeStoredCloudVoiceAuthMode('oauth');
|
||||
syncCloudVoiceAuth();
|
||||
say(
|
||||
'ChatGPT sign-in complete. OAuth will be used for the next cloud voice session.',
|
||||
);
|
||||
} catch (error) {
|
||||
if (lifetime.signal.aborted) return;
|
||||
say(`OAuth check failed: ${error?.message || error}`);
|
||||
} finally {
|
||||
oauthBusy = false;
|
||||
if (!disposed) syncCloudVoiceAuth();
|
||||
}
|
||||
})();
|
||||
return;
|
||||
}
|
||||
const button = event.target?.closest?.('[data-key-setup-remove]');
|
||||
if (disposed || !button || busy) return;
|
||||
let envVars = [];
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
keySetupChipLabel,
|
||||
stripKeylessBasemapFromHash,
|
||||
bindKeySetupPlacement,
|
||||
waitForChatGptOAuth,
|
||||
} from './keySetup.js';
|
||||
|
||||
test('setup placement moves the same button only in Cyber and disconnects on teardown', () => {
|
||||
@@ -61,6 +62,64 @@ test('collectKeyUpdates keeps only non-empty trimmed values', () => {
|
||||
assert.deepEqual(collectKeyUpdates(null), {});
|
||||
});
|
||||
|
||||
test('OAuth login polling stops as soon as local ChatGPT auth becomes available', async () => {
|
||||
let checks = 0;
|
||||
let clock = 0;
|
||||
const available = await waitForChatGptOAuth({
|
||||
fetchImpl: async () => {
|
||||
checks += 1;
|
||||
return {
|
||||
ok: true,
|
||||
json: async () => ({ available: checks >= 3 }),
|
||||
};
|
||||
},
|
||||
timeoutMs: 10_000,
|
||||
pollMs: 100,
|
||||
now: () => clock,
|
||||
sleep: async (ms) => {
|
||||
clock += ms;
|
||||
},
|
||||
});
|
||||
assert.equal(available, true);
|
||||
assert.equal(checks, 3);
|
||||
});
|
||||
|
||||
test('OAuth login polling stops on login or HTTP failure instead of waiting for timeout', async () => {
|
||||
for (const result of [
|
||||
{ ok: true, payload: { available: false, loginFailed: true, error: 'Sign-in failed.' } },
|
||||
{ ok: false, payload: { error: 'Sign-in is local only.' } },
|
||||
]) {
|
||||
let checks = 0;
|
||||
await assert.rejects(waitForChatGptOAuth({
|
||||
fetchImpl: async () => { checks += 1; return { ok: result.ok, json: async () => result.payload }; },
|
||||
sleep: () => assert.fail('failed login must stop polling'),
|
||||
}), { message: result.payload.error });
|
||||
assert.equal(checks, 1);
|
||||
}
|
||||
});
|
||||
|
||||
test('OAuth login polling respects timeout and an aborted setup surface', async () => {
|
||||
let clock = 0;
|
||||
let checks = 0;
|
||||
const poll = { fetchImpl: async () => { checks += 1; return { ok: true, json: async () => ({ available: false, pending: true }) }; } };
|
||||
assert.equal(await waitForChatGptOAuth({ ...poll, timeoutMs: 100, pollMs: 100, now: () => clock, sleep: async (ms) => { clock += ms; } }), false);
|
||||
assert.equal(checks, 2);
|
||||
const controller = new AbortController();
|
||||
controller.abort();
|
||||
assert.equal(await waitForChatGptOAuth({ ...poll, signal: controller.signal }), false);
|
||||
assert.equal(checks, 2, 'aborted polling does not fetch');
|
||||
});
|
||||
|
||||
test('OAuth polling ignores a ready response received after the surface was aborted', async () => {
|
||||
const controller = new AbortController();
|
||||
const available = await waitForChatGptOAuth({
|
||||
signal: controller.signal,
|
||||
fetchImpl: async () => ({ ok: true, json: async () => { controller.abort(); return { available: true }; } }),
|
||||
sleep: () => assert.fail('aborted polling must not continue'),
|
||||
});
|
||||
assert.equal(available, false);
|
||||
});
|
||||
|
||||
test('the first Google key strips ONLY the keyless OSM basemap from the share hash', () => {
|
||||
const stripped = stripKeylessBasemapFromHash('lat=30.2&lon=-97.7&map=osm&style=normal');
|
||||
assert.ok(stripped !== null);
|
||||
|
||||
@@ -55,7 +55,7 @@ export const KEY_SETUP_KEYS = Object.freeze([
|
||||
Object.freeze({
|
||||
id: 'openai',
|
||||
title: 'OPENAI',
|
||||
unlocks: 'Voice control — talk to the planet',
|
||||
unlocks: 'Voice control — API key or ChatGPT OAuth',
|
||||
getUrl: 'https://platform.openai.com/api-keys',
|
||||
envVars: Object.freeze(['OPENAI_API_KEY']),
|
||||
tier: 'metered',
|
||||
|
||||
@@ -274,6 +274,13 @@ test('doctor describes the credential ladder without exposing values', () => {
|
||||
);
|
||||
assert.match(capabilities.map, /Bing and world-terrain stacks/);
|
||||
assert.equal(capabilities.voice, 'available');
|
||||
assert.equal(
|
||||
buildCapabilitySummary({
|
||||
...credentials,
|
||||
OPENAI_API_KEY: { configured: false },
|
||||
}).voice,
|
||||
'off until an OpenAI key is added, or a Codex ChatGPT sign-in is selected in Provider Settings (experimental)',
|
||||
);
|
||||
assert.match(capabilities.missions, /token allowance/);
|
||||
assert.equal(
|
||||
capabilities.flights,
|
||||
|
||||
@@ -214,11 +214,18 @@ function decodeLayers(bytes, z, x, y, layerNames) {
|
||||
return { roads, military };
|
||||
}
|
||||
|
||||
/** Construct an immutable-version road/military tile source without starting I/O. */
|
||||
/**
|
||||
* Construct an immutable-version road/military tile source without starting I/O.
|
||||
*
|
||||
* `allowedOrigin` is deliberately not set here: `createVectorTileSource`
|
||||
* derives it from whichever `tileJsonUrl` is in effect, so a caller that
|
||||
* repoints only `tileJsonUrl` gets tiles from that host or a named error —
|
||||
* never a silent fall back to the default host (#933). Passing
|
||||
* `allowedOrigin` explicitly still overrides both.
|
||||
*/
|
||||
export function createOpenFreeMapSource(options = {}) {
|
||||
return createVectorTileSource({
|
||||
tileJsonUrl: 'https://tiles.openfreemap.org/planet',
|
||||
allowedOrigin: 'https://tiles.openfreemap.org',
|
||||
decode: decodeOpenFreeMapTile,
|
||||
maxEntries: 192,
|
||||
maxCacheBytes: 64 * 1024 * 1024,
|
||||
|
||||
@@ -320,3 +320,41 @@ test('Capitol, greenbelt and commercial property fixtures exclude non-public dri
|
||||
assert.ok(records.some((r) => r.area === area && !r.drivable));
|
||||
}
|
||||
});
|
||||
|
||||
test('repointing tileJsonUrl at a mirror never falls back to the default tile host', async () => {
|
||||
const urls = [];
|
||||
const mirror = 'http://127.0.0.1:12348';
|
||||
const source = createOpenFreeMapSource({
|
||||
fetchImpl: async (url) => {
|
||||
urls.push(url);
|
||||
return url.endsWith('/planet')
|
||||
? Response.json({ tiles: [`${mirror}/planet/v/{z}/{x}/{y}.pbf`] })
|
||||
: new Response(fixture('ofm-austin-14-3743-6745.pbf'));
|
||||
},
|
||||
tileJsonUrl: `${mirror}/planet`,
|
||||
});
|
||||
await source.fetchBounds(
|
||||
{ south: 30.264, north: 30.271, west: -97.75, east: -97.74 },
|
||||
{ zoom: 14 },
|
||||
);
|
||||
assert.ok(urls.length > 1, 'tiles were fetched');
|
||||
assert.deepEqual(
|
||||
urls.filter((url) => !url.startsWith(mirror)),
|
||||
[],
|
||||
'a repointed source must not fetch from tiles.openfreemap.org',
|
||||
);
|
||||
});
|
||||
|
||||
test('a mirror that serves the default host TileJSON verbatim fails loudly', async () => {
|
||||
// A transparent caching proxy returns upstream's TileJSON unchanged, whose
|
||||
// tiles[0] is an absolute upstream URL. That must be an error, not 100% of
|
||||
// tile traffic quietly leaving the configured mirror.
|
||||
const source = createOpenFreeMapSource({
|
||||
fetchImpl: async () =>
|
||||
Response.json({
|
||||
tiles: ['https://tiles.openfreemap.org/planet/v/{z}/{x}/{y}.pbf'],
|
||||
}),
|
||||
tileJsonUrl: 'http://127.0.0.1:12348/planet',
|
||||
});
|
||||
await assert.rejects(source.getMetadata(), { retryable: false });
|
||||
});
|
||||
@@ -16,11 +16,27 @@ export function validTileBounds(box) {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The origin a configured TileJSON is served from, or undefined when there is
|
||||
* no absolute URL to read one off. Deriving `allowedOrigin` from it means
|
||||
* repointing `tileJsonUrl` at a mirror cannot silently keep loading tiles from
|
||||
* the old host: the origin check below then names the mismatch instead of
|
||||
* passing it (#933).
|
||||
*/
|
||||
function tileJsonOrigin(tileJsonUrl) {
|
||||
if (typeof tileJsonUrl !== 'string') return undefined;
|
||||
try {
|
||||
return new URL(tileJsonUrl).origin;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/** A decoded XYZ cache with bounded workers, reads, retained bytes and request lifetimes. */
|
||||
export function createVectorTileSource({
|
||||
tileJsonUrl,
|
||||
template,
|
||||
allowedOrigin,
|
||||
allowedOrigin = tileJsonOrigin(tileJsonUrl),
|
||||
decode,
|
||||
fetchImpl = (...args) => globalThis.fetch(...args),
|
||||
maxTiles = 16,
|
||||
|
||||
@@ -203,3 +203,45 @@ test('polygon-selected tiles keep nearest-first order and cached revisits issue
|
||||
'over-budget selection is rejected before fetching',
|
||||
);
|
||||
});
|
||||
|
||||
test('a configured tileJsonUrl sets the tile origin, and a foreign tiles[0] is still refused', async () => {
|
||||
// Without an explicit allowedOrigin the mirror's own TileJSON is honoured…
|
||||
const mirrored = createVectorTileSource({
|
||||
tileJsonUrl: 'http://127.0.0.1:12348/planet',
|
||||
decode: () => [],
|
||||
fetchImpl: async () =>
|
||||
Response.json({ tiles: ['http://127.0.0.1:12348/p/{z}/{x}/{y}.pbf'] }),
|
||||
});
|
||||
assert.equal(
|
||||
(await mirrored.getMetadata()).template,
|
||||
'http://127.0.0.1:12348/p/{z}/{x}/{y}.pbf',
|
||||
);
|
||||
|
||||
// …while a TileJSON that names a different host is refused rather than
|
||||
// silently fetched, which is what the origin check is for.
|
||||
const foreign = createVectorTileSource({
|
||||
tileJsonUrl: 'http://127.0.0.1:12348/planet',
|
||||
decode: () => [],
|
||||
fetchImpl: async () =>
|
||||
Response.json({
|
||||
tiles: ['https://tiles.example/planet/{z}/{x}/{y}.pbf'],
|
||||
}),
|
||||
});
|
||||
await assert.rejects(foreign.getMetadata(), { retryable: false });
|
||||
});
|
||||
|
||||
test('an explicit allowedOrigin still wins over the one derived from tileJsonUrl', async () => {
|
||||
const source = createVectorTileSource({
|
||||
tileJsonUrl: 'http://127.0.0.1:12348/planet',
|
||||
allowedOrigin: 'https://tiles.example',
|
||||
decode: () => [],
|
||||
fetchImpl: async () =>
|
||||
Response.json({
|
||||
tiles: ['https://tiles.example/planet/{z}/{x}/{y}.pbf'],
|
||||
}),
|
||||
});
|
||||
assert.equal(
|
||||
(await source.getMetadata()).template,
|
||||
'https://tiles.example/planet/{z}/{x}/{y}.pbf',
|
||||
);
|
||||
});
|
||||
@@ -41,6 +41,7 @@ function request(
|
||||
url = '/',
|
||||
body = '',
|
||||
origin = 'http://localhost:4173',
|
||||
remoteAddress = '127.0.0.1',
|
||||
} = {},
|
||||
) {
|
||||
return new Promise((resolve, reject) => {
|
||||
@@ -53,7 +54,7 @@ function request(
|
||||
origin,
|
||||
'content-type': 'application/json',
|
||||
},
|
||||
socket: { remoteAddress: '127.0.0.1' },
|
||||
socket: { remoteAddress },
|
||||
});
|
||||
const headers = {};
|
||||
const res = {
|
||||
@@ -199,6 +200,386 @@ test('Realtime handler preserves tools and default instructions, isolates suppli
|
||||
assert.equal(sent[0].session.instructions, sent[2].session.instructions);
|
||||
});
|
||||
|
||||
test('Realtime handler keeps API key as default and uses OAuth only when requested', async (t) => {
|
||||
env(t, 'GEV_RATELIMIT_OPENAI_PER_MIN', undefined);
|
||||
let oauthReads = 0;
|
||||
const authorizations = [];
|
||||
const routes = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveApiKey: () => 'fixture-api-key',
|
||||
resolveOAuthAccessToken: () => {
|
||||
oauthReads += 1;
|
||||
return 'fixture-oauth-token';
|
||||
},
|
||||
fetchImpl: async (_url, options) => {
|
||||
authorizations.push(options.headers.Authorization);
|
||||
return Response.json({ value: 'fixture-ephemeral' });
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const token = routes.get('/api/realtime/token');
|
||||
|
||||
const apiResponse = await request(token);
|
||||
assert.equal(apiResponse.status, 200);
|
||||
assert.equal(apiResponse.headers['x-gev-voice-auth'], 'env');
|
||||
assert.equal(oauthReads, 0, 'default cloud voice does not read OAuth');
|
||||
assert.equal(authorizations.at(-1), 'Bearer fixture-api-key');
|
||||
|
||||
const oauthResponse = await request(token, { url: '/?auth=oauth' });
|
||||
assert.equal(oauthResponse.status, 200);
|
||||
assert.equal(oauthResponse.headers['x-gev-voice-auth'], 'codex-oauth');
|
||||
assert.equal(oauthReads, 1);
|
||||
assert.equal(authorizations.at(-1), 'Bearer fixture-oauth-token');
|
||||
});
|
||||
|
||||
test('a Codex OAuth pin refuses API-key minting without silently switching modes', async (t) => {
|
||||
env(t, 'GEV_RATELIMIT_OPENAI_PER_MIN', undefined);
|
||||
const authorizations = [];
|
||||
const routes = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: {
|
||||
environment: { GEV_PREFER_CODEX_OAUTH: 'true' },
|
||||
resolveApiKey: () => assert.fail('pinned OAuth cannot read an API key'),
|
||||
resolveOAuthAccessToken: () => 'fixture-oauth-token',
|
||||
fetchImpl: async (_url, options) => {
|
||||
authorizations.push(options.headers.Authorization);
|
||||
return Response.json({ value: 'fixture-ephemeral' });
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const token = routes.get('/api/realtime/token');
|
||||
const refused = await request(token);
|
||||
assert.equal(refused.status, 503);
|
||||
assert.equal(refused.json().code, 'CODEX_OAUTH_REQUIRED');
|
||||
assert.equal(authorizations.length, 0);
|
||||
assert.equal((await request(token, { url: '/?auth=oauth' })).status, 200);
|
||||
assert.deepEqual(authorizations, ['Bearer fixture-oauth-token']);
|
||||
});
|
||||
|
||||
test('OAuth status and token minting never expose local OAuth to remote clients', async () => {
|
||||
let oauthReads = 0;
|
||||
const routes = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: () => {
|
||||
oauthReads += 1;
|
||||
return 'must-not-be-read-remotely';
|
||||
},
|
||||
fetchImpl: () =>
|
||||
assert.fail('remote OAuth request must not reach OpenAI'),
|
||||
},
|
||||
}),
|
||||
);
|
||||
const remote = { remoteAddress: '192.168.1.20' };
|
||||
assert.equal(
|
||||
(await request(routes.get('/api/realtime/oauth-status'), remote)).status,
|
||||
403,
|
||||
);
|
||||
assert.equal(
|
||||
(
|
||||
await request(routes.get('/api/realtime/token'), {
|
||||
...remote,
|
||||
url: '/?auth=oauth',
|
||||
})
|
||||
).status,
|
||||
403,
|
||||
);
|
||||
assert.equal(oauthReads, 0);
|
||||
});
|
||||
|
||||
test('OAuth status reports local sign-in availability without returning the token', async () => {
|
||||
const routes = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: { resolveOAuthAccessToken: () => 'fixture-oauth-secret' },
|
||||
}),
|
||||
);
|
||||
const response = await request(routes.get('/api/realtime/oauth-status'));
|
||||
assert.equal(response.status, 200);
|
||||
assert.deepEqual(response.json(), { available: true });
|
||||
assert.equal(response.body.includes('fixture-oauth-secret'), false);
|
||||
});
|
||||
|
||||
test('OAuth status and token minting report expired sign-in as re-auth required', async () => {
|
||||
const expired = () => {
|
||||
const error = new Error('ChatGPT sign-in has expired.');
|
||||
error.code = 'CODEX_OAUTH_REAUTH_REQUIRED';
|
||||
throw error;
|
||||
};
|
||||
const routes = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: expired,
|
||||
fetchImpl: () => assert.fail('expired OAuth must not reach OpenAI'),
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const status = await request(routes.get('/api/realtime/oauth-status'));
|
||||
assert.equal(status.status, 200);
|
||||
assert.deepEqual(status.json(), {
|
||||
available: false,
|
||||
error: 'ChatGPT sign-in has expired.',
|
||||
code: 'CODEX_OAUTH_REAUTH_REQUIRED',
|
||||
reauthRequired: true,
|
||||
});
|
||||
|
||||
const token = await request(routes.get('/api/realtime/token'), {
|
||||
url: '/?auth=oauth',
|
||||
});
|
||||
assert.equal(token.status, 503);
|
||||
assert.equal(token.json().code, 'CODEX_OAUTH_REAUTH_REQUIRED');
|
||||
});
|
||||
|
||||
test('OAuth login endpoint shares pending attempts and never exposes credentials', async (t) => {
|
||||
t.mock.timers.enable({ apis: ['setTimeout'] });
|
||||
let starts = 0;
|
||||
let spawnReady;
|
||||
let exit;
|
||||
const plugin = openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: () => {
|
||||
throw new Error('not signed in');
|
||||
},
|
||||
startOAuthLogin: async () => {
|
||||
starts += 1;
|
||||
await new Promise((resolve) => {
|
||||
spawnReady = resolve;
|
||||
});
|
||||
return {
|
||||
completion: new Promise((resolve) => {
|
||||
exit = resolve;
|
||||
}),
|
||||
cancel() {},
|
||||
};
|
||||
},
|
||||
},
|
||||
});
|
||||
t.after(plugin.closeBundle);
|
||||
const routes = install(plugin);
|
||||
const handler = routes.get('/api/realtime/oauth-login');
|
||||
|
||||
const firstRequest = request(handler, { method: 'POST' });
|
||||
const concurrent = await request(handler, { method: 'POST' });
|
||||
assert.equal(concurrent.status, 202);
|
||||
assert.equal(concurrent.json().started, false);
|
||||
assert.equal(starts, 1, 'reserve the attempt before awaiting spawn');
|
||||
spawnReady();
|
||||
const started = await firstRequest;
|
||||
assert.equal(started.status, 202);
|
||||
assert.deepEqual(started.json(), {
|
||||
available: false,
|
||||
started: true,
|
||||
pending: true,
|
||||
});
|
||||
assert.equal(starts, 1);
|
||||
|
||||
t.mock.timers.tick(31_000);
|
||||
const duplicate = await request(handler, { method: 'POST' });
|
||||
assert.equal(duplicate.status, 202);
|
||||
assert.equal(duplicate.json().pending, true);
|
||||
assert.equal(starts, 1, 'rapid retries must not spawn another login');
|
||||
|
||||
const remote = await request(handler, {
|
||||
method: 'POST',
|
||||
remoteAddress: '192.168.1.20',
|
||||
});
|
||||
assert.equal(remote.status, 403);
|
||||
assert.equal(starts, 1);
|
||||
|
||||
const crossOrigin = await request(handler, {
|
||||
method: 'POST',
|
||||
origin: 'https://example.com',
|
||||
});
|
||||
assert.equal(crossOrigin.status, 403);
|
||||
assert.equal(starts, 1);
|
||||
|
||||
exit({ exitCode: 1 });
|
||||
await Promise.resolve();
|
||||
const failed = await request(routes.get('/api/realtime/oauth-status'));
|
||||
assert.equal(failed.json().loginFailed, true);
|
||||
assert.equal(failed.json().code, 'CODEX_OAUTH_LOGIN_FAILED');
|
||||
});
|
||||
|
||||
test('OAuth status follows process completion, permits retry, and checks credential storage', async (t) => {
|
||||
let token = null;
|
||||
let exit;
|
||||
let starts = 0;
|
||||
const plugin = openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: () => token,
|
||||
startOAuthLogin: async () => {
|
||||
starts += 1;
|
||||
return {
|
||||
completion: new Promise((resolve) => {
|
||||
exit = resolve;
|
||||
}),
|
||||
cancel() {},
|
||||
};
|
||||
},
|
||||
},
|
||||
});
|
||||
t.after(plugin.closeBundle);
|
||||
const routes = install(plugin);
|
||||
const launch = () =>
|
||||
request(routes.get('/api/realtime/oauth-login'), { method: 'POST' });
|
||||
const status = () => request(routes.get('/api/realtime/oauth-status'));
|
||||
await launch();
|
||||
exit({ exitCode: 0 });
|
||||
await Promise.resolve();
|
||||
assert.equal(
|
||||
(await status()).json().code,
|
||||
'CODEX_OAUTH_STORAGE_UNAVAILABLE',
|
||||
'exit success alone is not usable sign-in',
|
||||
);
|
||||
await launch();
|
||||
assert.equal(starts, 2, 'failed attempts can be retried immediately');
|
||||
token = 'fixture-private-token';
|
||||
exit({ exitCode: 0 });
|
||||
await Promise.resolve();
|
||||
const ready = await status();
|
||||
assert.deepEqual(ready.json(), { available: true });
|
||||
assert.equal(ready.body.includes(token), false);
|
||||
});
|
||||
|
||||
test('OAuth login timeout and server shutdown cancel the owned process', async () => {
|
||||
let timeout;
|
||||
let cancelled = 0;
|
||||
let cleared = 0;
|
||||
const plugin = openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: () => null,
|
||||
startOAuthLogin: async () => ({
|
||||
completion: new Promise(() => {}),
|
||||
cancel() {
|
||||
cancelled += 1;
|
||||
},
|
||||
}),
|
||||
setTimer(callback, duration) {
|
||||
timeout = callback;
|
||||
assert.equal(duration, 120_000);
|
||||
return 1;
|
||||
},
|
||||
clearTimer() {
|
||||
cleared += 1;
|
||||
},
|
||||
},
|
||||
});
|
||||
const routes = install(plugin);
|
||||
const launch = () =>
|
||||
request(routes.get('/api/realtime/oauth-login'), { method: 'POST' });
|
||||
await launch();
|
||||
timeout();
|
||||
assert.equal(cancelled, 1);
|
||||
const status = await request(routes.get('/api/realtime/oauth-status'));
|
||||
assert.equal(status.json().code, 'CODEX_OAUTH_LOGIN_TIMEOUT');
|
||||
await launch();
|
||||
plugin.closeBundle();
|
||||
plugin.closeBundle();
|
||||
assert.equal(cancelled, 2, 'shutdown is idempotent');
|
||||
assert.equal(cleared, 2);
|
||||
});
|
||||
|
||||
test('OAuth launch errors are safe and cross-origin status requests cannot read credentials', async () => {
|
||||
let reads = 0;
|
||||
const routes = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken() {
|
||||
reads += 1;
|
||||
return null;
|
||||
},
|
||||
startOAuthLogin: async () => {
|
||||
throw new Error('private-token-fixture');
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const remote = await request(routes.get('/api/realtime/oauth-status'), {
|
||||
origin: 'https://example.com',
|
||||
});
|
||||
assert.equal(remote.status, 403);
|
||||
assert.equal(reads, 0);
|
||||
const failed = await request(routes.get('/api/realtime/oauth-login'), {
|
||||
method: 'POST',
|
||||
});
|
||||
assert.equal(failed.status, 503);
|
||||
assert.equal(failed.json().code, 'CODEX_OAUTH_LOGIN_FAILED');
|
||||
assert.equal(failed.body.includes('private-token-fixture'), false);
|
||||
});
|
||||
|
||||
test('a login that spawns after timeout is cancelled and cannot replace a retry', async (t) => {
|
||||
let timeout;
|
||||
let releaseSpawn;
|
||||
let cancelled = 0;
|
||||
let launches = 0;
|
||||
const plugin = openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: () => null,
|
||||
startOAuthLogin: async () => {
|
||||
launches += 1;
|
||||
if (launches === 1)
|
||||
await new Promise((resolve) => {
|
||||
releaseSpawn = resolve;
|
||||
});
|
||||
return {
|
||||
completion: new Promise(() => {}),
|
||||
cancel() {
|
||||
cancelled += 1;
|
||||
},
|
||||
};
|
||||
},
|
||||
setTimer(callback) {
|
||||
timeout = callback;
|
||||
return 1;
|
||||
},
|
||||
clearTimer() {},
|
||||
},
|
||||
});
|
||||
t.after(plugin.closeBundle);
|
||||
const routes = install(plugin);
|
||||
const launch = () =>
|
||||
request(routes.get('/api/realtime/oauth-login'), { method: 'POST' });
|
||||
const first = launch();
|
||||
timeout();
|
||||
await launch();
|
||||
releaseSpawn();
|
||||
await first;
|
||||
assert.equal(launches, 2);
|
||||
assert.equal(cancelled, 1, 'late process must not become an orphan');
|
||||
const status = (
|
||||
await request(routes.get('/api/realtime/oauth-status'))
|
||||
).json();
|
||||
assert.equal(status.pending, true);
|
||||
assert.equal(
|
||||
status.loginFailed,
|
||||
undefined,
|
||||
'the older attempt cannot fail the retry',
|
||||
);
|
||||
});
|
||||
|
||||
test('OAuth login endpoint selects an existing local sign-in without launching Codex', async () => {
|
||||
const handler = install(
|
||||
openAiRealtimeProxy({
|
||||
realtime: {
|
||||
resolveOAuthAccessToken: () => 'fixture-existing-secret',
|
||||
startOAuthLogin: () =>
|
||||
assert.fail('existing auth must not launch login'),
|
||||
},
|
||||
}),
|
||||
).get('/api/realtime/oauth-login');
|
||||
const response = await request(handler, { method: 'POST' });
|
||||
assert.equal(response.status, 200);
|
||||
assert.deepEqual(response.json(), {
|
||||
available: true,
|
||||
started: false,
|
||||
pending: false,
|
||||
});
|
||||
assert.equal(response.body.includes('fixture-existing-secret'), false);
|
||||
});
|
||||
|
||||
test('caption transcription is on by default, reported to the client meter, and can be turned off', async (t) => {
|
||||
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
|
||||
env(t, 'GEV_RATELIMIT_OPENAI_PER_MIN', undefined);
|
||||
|
||||
@@ -300,6 +300,43 @@
|
||||
color: rgba(148, 178, 188, 0.42);
|
||||
}
|
||||
|
||||
.key-setup-cloud-auth {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.key-setup-cloud-auth-state {
|
||||
color: rgba(190, 214, 222, 0.7);
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.56rem;
|
||||
letter-spacing: 0.05rem;
|
||||
}
|
||||
|
||||
.key-setup-cloud-auth-toggle {
|
||||
margin-left: auto;
|
||||
padding: 0.28rem 0.55rem;
|
||||
color: rgba(54, 220, 255, 0.9);
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.55rem;
|
||||
letter-spacing: 0.06rem;
|
||||
background: rgba(54, 220, 255, 0.08);
|
||||
border: 1px solid rgba(54, 220, 255, 0.28);
|
||||
border-radius: 0.32rem;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.key-setup-cloud-auth-toggle:hover,
|
||||
.key-setup-cloud-auth-toggle:focus-visible {
|
||||
border-color: rgba(54, 220, 255, 0.65);
|
||||
outline: none;
|
||||
}
|
||||
|
||||
.key-setup-cloud-auth-toggle[aria-disabled='true'] {
|
||||
cursor: wait;
|
||||
opacity: 0.5;
|
||||
}
|
||||
|
||||
.key-setup-footer {
|
||||
margin-top: 0.85rem;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
export const CLOUD_VOICE_AUTH_MODES = Object.freeze(['api-key', 'oauth']);
|
||||
export const DEFAULT_CLOUD_VOICE_AUTH_MODE = 'api-key';
|
||||
export const CLOUD_VOICE_AUTH_STORAGE_KEY = 'godsEyeView.voice.cloudAuth';
|
||||
|
||||
function storageRef(storage) {
|
||||
if (storage) return storage;
|
||||
try {
|
||||
return typeof localStorage === 'undefined' ? null : localStorage;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function normalizeCloudVoiceAuthMode(mode) {
|
||||
const normalized = String(mode || '')
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
return CLOUD_VOICE_AUTH_MODES.includes(normalized)
|
||||
? normalized
|
||||
: DEFAULT_CLOUD_VOICE_AUTH_MODE;
|
||||
}
|
||||
|
||||
export function readStoredCloudVoiceAuthMode(storage) {
|
||||
try {
|
||||
return normalizeCloudVoiceAuthMode(
|
||||
storageRef(storage)?.getItem(CLOUD_VOICE_AUTH_STORAGE_KEY),
|
||||
);
|
||||
} catch {
|
||||
return DEFAULT_CLOUD_VOICE_AUTH_MODE;
|
||||
}
|
||||
}
|
||||
|
||||
export function writeStoredCloudVoiceAuthMode(mode, storage) {
|
||||
const resolved = normalizeCloudVoiceAuthMode(mode);
|
||||
try {
|
||||
storageRef(storage)?.setItem(CLOUD_VOICE_AUTH_STORAGE_KEY, resolved);
|
||||
} catch {
|
||||
/* best effort */
|
||||
}
|
||||
return resolved;
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
CLOUD_VOICE_AUTH_STORAGE_KEY,
|
||||
readStoredCloudVoiceAuthMode,
|
||||
writeStoredCloudVoiceAuthMode,
|
||||
} from './cloudVoiceAuth.js';
|
||||
|
||||
function memoryStorage(seed = {}) {
|
||||
const values = new Map(Object.entries(seed));
|
||||
return {
|
||||
getItem: (key) => values.get(key) ?? null,
|
||||
setItem: (key, value) => values.set(key, value),
|
||||
};
|
||||
}
|
||||
|
||||
test('cloud voice auth defaults to API key and persists OAuth independently', () => {
|
||||
const storage = memoryStorage();
|
||||
assert.equal(readStoredCloudVoiceAuthMode(storage), 'api-key');
|
||||
assert.equal(writeStoredCloudVoiceAuthMode('oauth', storage), 'oauth');
|
||||
assert.equal(readStoredCloudVoiceAuthMode(storage), 'oauth');
|
||||
assert.equal(writeStoredCloudVoiceAuthMode('unknown', storage), 'api-key');
|
||||
assert.equal(
|
||||
readStoredCloudVoiceAuthMode(
|
||||
memoryStorage({ [CLOUD_VOICE_AUTH_STORAGE_KEY]: 'oauth' }),
|
||||
),
|
||||
'oauth',
|
||||
);
|
||||
});
|
||||
@@ -3195,6 +3195,54 @@ test('F1: the toggle still records the next-session preference while live', () =
|
||||
assert.match(ui.tierButton.title, /this session stays on/i);
|
||||
});
|
||||
|
||||
test('OAuth cloud sessions report unknown cost and usage without tripping the API spend guard', () => {
|
||||
const { controller, ui } = costControllerHarness();
|
||||
controller.status = 'listening';
|
||||
controller._cost.cloudVoiceAuth = 'oauth';
|
||||
controller._cost.sessionCloudVoiceAuth = 'oauth';
|
||||
controller.syncCostUi();
|
||||
const before = controller.costTracker.state();
|
||||
assert.equal(ui.costValue.hidden, false);
|
||||
assert.equal(ui.costValue.textContent, 'COST UNKNOWN');
|
||||
assert.deepEqual(controller.recordUsage(usdUsage(100)), {
|
||||
responses: 1,
|
||||
input: 0,
|
||||
output: 1562500,
|
||||
captions: 0,
|
||||
costKnown: false,
|
||||
});
|
||||
const after = controller.costTracker.state();
|
||||
assert.equal(after.totalUsd, before.totalUsd);
|
||||
assert.equal(controller.costCapStopped, false);
|
||||
assert.match(ui.costValue.title, /1 response\(s\).*1562500 output tokens/);
|
||||
controller.dc = {};
|
||||
controller.responseActive = true;
|
||||
controller.stop();
|
||||
assert.match(ui.costValue.title, /Usage is incomplete/);
|
||||
});
|
||||
|
||||
test('OAuth cloud sessions count caption transcriptions without pricing them into the API cap', () => {
|
||||
const { controller, ui } = costControllerHarness();
|
||||
controller.status = 'listening';
|
||||
controller._cost.cloudVoiceAuth = 'oauth';
|
||||
controller._cost.sessionCloudVoiceAuth = 'oauth';
|
||||
controller._cost.bindTranscriptionModel('gpt-4o-mini-transcribe');
|
||||
const before = controller.costTracker.state();
|
||||
// Priced, this much caption audio would cross the $5 cap.
|
||||
const result = controller.recordTranscriptionUsage({
|
||||
type: 'duration',
|
||||
seconds: 3_600_000,
|
||||
});
|
||||
assert.equal(result.captions, 1);
|
||||
assert.equal(result.costKnown, false);
|
||||
const after = controller.costTracker.state();
|
||||
assert.equal(after.totalUsd, before.totalUsd);
|
||||
assert.equal(after.capReached, false);
|
||||
assert.equal(controller.costCapStopped, false);
|
||||
assert.equal(ui.costValue.textContent, 'COST UNKNOWN');
|
||||
assert.match(ui.costValue.title, /1 caption transcription\(s\)/);
|
||||
});
|
||||
|
||||
test('F1: when idle, toggling does re-price the preview meter', () => {
|
||||
const { controller } = costControllerHarness();
|
||||
controller.status = 'idle';
|
||||
|
||||
@@ -15,11 +15,19 @@ export function createRealtimeBackend({
|
||||
);
|
||||
return Object.freeze({
|
||||
protocol: 'openai-realtime',
|
||||
async requestToken({ tier = DEFAULT_VOICE_TIER, signal } = {}) {
|
||||
async requestToken({
|
||||
tier = DEFAULT_VOICE_TIER,
|
||||
authMode = 'api-key',
|
||||
signal,
|
||||
} = {}) {
|
||||
signal = scoped(signal);
|
||||
signal.throwIfAborted();
|
||||
const separator = tokenEndpoint.includes('?') ? '&' : '?';
|
||||
const url = `${tokenEndpoint}${separator}tier=${encodeURIComponent(resolveVoiceModel(tier).tier)}`;
|
||||
const query = new URLSearchParams({
|
||||
tier: resolveVoiceModel(tier).tier,
|
||||
});
|
||||
if (authMode === 'oauth') query.set('auth', 'oauth');
|
||||
const url = `${tokenEndpoint}${separator}${query}`;
|
||||
const response = await tokenTransport(url, {
|
||||
signal,
|
||||
cache: 'no-store',
|
||||
|
||||
@@ -41,6 +41,22 @@ test('separate configured token and SDP transports preserve model metadata and c
|
||||
]);
|
||||
});
|
||||
|
||||
test('OAuth is an additive token query while API key stays the default', async () => {
|
||||
const calls = [];
|
||||
const backend = createRealtimeBackend({
|
||||
tokenTransport: async (input) => {
|
||||
calls.push(input);
|
||||
return tokenReply();
|
||||
},
|
||||
});
|
||||
await backend.requestToken();
|
||||
await backend.requestToken({ authMode: 'oauth' });
|
||||
assert.deepEqual(calls, [
|
||||
'/api/realtime/token?tier=standard',
|
||||
'/api/realtime/token?tier=standard&auth=oauth',
|
||||
]);
|
||||
});
|
||||
|
||||
test('denied tokens never negotiate or retry; reconnect requests a fresh token', async () => {
|
||||
let requests = 0;
|
||||
const backend = createRealtimeBackend({
|
||||
|
||||
@@ -82,6 +82,7 @@ export class RealtimeConnection {
|
||||
this.debugLog('session.starting', {
|
||||
epoch,
|
||||
tier: this.cost.voiceTier,
|
||||
authMode: this.cost.sessionCloudVoiceAuth,
|
||||
connection: this.connectionDiagnostics(),
|
||||
});
|
||||
let localStream = null;
|
||||
@@ -89,6 +90,7 @@ export class RealtimeConnection {
|
||||
try {
|
||||
const minted = await this.backend.requestToken({
|
||||
tier: this.cost.voiceTier,
|
||||
authMode: this.cost.sessionCloudVoiceAuth,
|
||||
signal,
|
||||
});
|
||||
const token = minted.token;
|
||||
|
||||
@@ -374,7 +374,10 @@ export class GevRealtimeController extends RealtimeFacade {
|
||||
status: this.status,
|
||||
connection: this.connectionDiagnostics(),
|
||||
});
|
||||
if (this.dc && this.responseActive) this.costTracker.markIncomplete();
|
||||
if (this.dc && this.responseActive) {
|
||||
this.costTracker.markIncomplete();
|
||||
this.syncCostUi();
|
||||
}
|
||||
this._metrics.flush('stop');
|
||||
this._connection.closeTransport();
|
||||
this.stopVoiceVisualizer();
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
writeStoredVoiceTier,
|
||||
writeStoredVoiceLimits,
|
||||
} from './realtimePreferences.js';
|
||||
import { readStoredCloudVoiceAuthMode } from './cloudVoiceAuth.js';
|
||||
import {
|
||||
createVoiceCostTracker,
|
||||
resolveVoiceModel,
|
||||
@@ -17,6 +18,9 @@ export class RealtimeCost {
|
||||
Object.assign(this, { readUi, readStatus }, operations);
|
||||
this.voiceTier = readStoredVoiceTier();
|
||||
this.voiceLimits = readStoredVoiceLimits();
|
||||
this.cloudVoiceAuth = readStoredCloudVoiceAuthMode();
|
||||
this.sessionCloudVoiceAuth = null;
|
||||
this.oauthUsage = { responses: 0, input: 0, output: 0, captions: 0 };
|
||||
this.transcribeModel = null;
|
||||
this.costTracker = createVoiceCostTracker({
|
||||
tier: this.voiceTier,
|
||||
@@ -58,6 +62,21 @@ export class RealtimeCost {
|
||||
}; applies next session`;
|
||||
}
|
||||
if (this.ui?.costValue) {
|
||||
const liveAuth = this.sessionCloudVoiceAuth || this.cloudVoiceAuth;
|
||||
if (liveAuth === 'oauth') {
|
||||
const { responses, input, output, captions } = this.oauthUsage;
|
||||
this.ui.costValue.hidden = false;
|
||||
this.ui.costValue.textContent = 'COST UNKNOWN';
|
||||
this.ui.costValue.dataset.level = 'unknown';
|
||||
this.ui.costValue.title =
|
||||
`ChatGPT OAuth session: ${responses} response(s), ${input} input and ${output} output tokens, ${captions} caption transcription(s) reported. ` +
|
||||
'USD cost and the API spend cap are unavailable for this auth mode.' +
|
||||
(state.incomplete
|
||||
? ' Usage is incomplete because a response was still in flight when the session ended.'
|
||||
: '');
|
||||
return;
|
||||
}
|
||||
this.ui.costValue.hidden = false;
|
||||
this.ui.costValue.textContent = state.display;
|
||||
this.ui.costValue.dataset.level = state.level;
|
||||
this.ui.costValue.title =
|
||||
@@ -138,6 +157,15 @@ export class RealtimeCost {
|
||||
*/
|
||||
recordUsage(usage) {
|
||||
if (!usage) return null;
|
||||
if (this.sessionCloudVoiceAuth === 'oauth') {
|
||||
const count = (value) =>
|
||||
Number.isFinite(value) && value > 0 ? value : 0;
|
||||
this.oauthUsage.responses += 1;
|
||||
this.oauthUsage.input += count(usage.input_tokens);
|
||||
this.oauthUsage.output += count(usage.output_tokens);
|
||||
this.syncCostUi();
|
||||
return { ...this.oauthUsage, costKnown: false };
|
||||
}
|
||||
return this.applyCost(this.costTracker.record(usage));
|
||||
}
|
||||
|
||||
@@ -148,6 +176,13 @@ export class RealtimeCost {
|
||||
*/
|
||||
recordTranscriptionUsage(usage) {
|
||||
if (!usage) return null;
|
||||
// An OAuth session has no API-dollar meter, so captions are counted
|
||||
// rather than priced into the API-key warning and cap.
|
||||
if (this.sessionCloudVoiceAuth === 'oauth') {
|
||||
this.oauthUsage.captions += 1;
|
||||
this.syncCostUi();
|
||||
return { ...this.oauthUsage, costKnown: false };
|
||||
}
|
||||
return this.applyCost(
|
||||
this.costTracker.recordUsd(
|
||||
estimateTranscriptionCostUsd(usage, this.transcribeModel),
|
||||
@@ -213,6 +248,9 @@ export class RealtimeCost {
|
||||
prepareSession() {
|
||||
this.voiceTier = readStoredVoiceTier();
|
||||
this.voiceLimits = readStoredVoiceLimits();
|
||||
this.cloudVoiceAuth = readStoredCloudVoiceAuthMode();
|
||||
this.sessionCloudVoiceAuth = this.cloudVoiceAuth;
|
||||
this.oauthUsage = { responses: 0, input: 0, output: 0, captions: 0 };
|
||||
this.costCapStopped = false;
|
||||
// Provisional meter (tier-priced) so the readout shows $0.00 while
|
||||
// connecting. It is REPLACED below with one bound to the model the server
|
||||
|
||||
Reference in new issue
Block a user