diff --git a/README.md b/README.md index 37a0b0553..16a2ebf7e 100644 --- a/README.md +++ b/README.md @@ -237,6 +237,12 @@ _Why cockpit mode exists: you're riding a real aircraft over real terrain — an > Voice can use an **OpenAI API key** or, on the same machine, an opt-in signed-in **ChatGPT/Codex OAuth** session. The API-key path remains the default. Selecting ChatGPT OAuth in Provider Settings reuses an existing local sign-in or starts the Codex browser sign-in flow and selects OAuth when it completes. The OpenAI key still drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move. +The OAuth option is experimental. OpenAI documents ChatGPT plan usage for the [Responses API](https://developers.openai.com/siwc/token-sharing-open-source/models-and-inference); a supported third-party Realtime contract for reusing Codex tokens has not been confirmed. A successful token mint does not establish billing or support terms. + +GEV reads Codex's file credentials but does not write or refresh them. It checks token shape and expiry, and reports expired sign-in as requiring re-authentication. The login button runs `codex login`, permits one pending attempt, and reports exit failures or a two-minute timeout. `CODEX_HOME` must match the directory that Codex uses. If `CODEX_AUTH_JSON` points elsewhere, sign in separately or fix that mismatch before using the button. Codex credential storage must use `auth.json`; Keychain-only and in-memory storage are not readable by GEV. See [Codex credential storage](https://learn.chatgpt.com/docs/auth#credential-storage). + +For a live voice check, run `node scripts/qa-voice-auth.mjs http://127.0.0.1:4189 oauth microphone.wav` against your local dev server. Use a WAV with three seconds of leading silence followed by a short spoken reply request. Replace `oauth` with `api-key` to check the API-key path. The check verifies audio input, a completed spoken reply, usage, resource cleanup, and reconnect. It saves screenshots and credential-free results under `.gev-logs/voice-auth/`. + Click **GEV MIC**, grant the microphone, and just talk. This is more than a voice-controlled remote: - **🧠 It knows what it's looking at.** The agent pulls live scene context before answering — including coordinates, street names, active layers, and view scale. Ask _"what city is this?"_ mid-flight and it knows. diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index e768be94e..2834a5b1c 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -3924,6 +3924,8 @@ and unreachable upstream (502/504) separately from road geometry. 3. **Every count names its scope in words** — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number. `analyst_query` returns `scopeLabel` so this is mechanical. Two different numbers with named scopes are not a contradiction. 4. **The loaded-data caveat is stated once when relevant**: counts cover loaded data, and the flights layer loads where you look (appended to `coverage.note` for radius/view scopes over viewport-loaded layers). - **Degradation**: API-key mode without `OPENAI_API_KEY` returns 503 as before. Selecting OAuth without a usable local ChatGPT/Codex sign-in starts the local `codex login` browser flow, polls for the resulting local token, and selects OAuth when sign-in completes. If the Codex login flow cannot start or does not complete, the rest of the app is unaffected. +- **Login lifecycle and storage**: one pending Codex login is shared by the login and status routes. Process failure, success without readable credentials, and a two-minute timeout each report a clear failure; timeout and server shutdown cancel the owned process. The launcher resolves `CODEX_HOME` before changing directories and refuses a mismatched `CODEX_AUTH_JSON`. GEV reads file credentials only, never writes or refreshes them, and does not require the vendor refresh token. Expired access tokens report `CODEX_OAUTH_REAUTH_REQUIRED`. OAuth status and login routes also use the same-site request gate. +- **Support boundary**: the local OAuth option is experimental. OpenAI's documented Sign in with ChatGPT plan-sharing inference endpoint is `/v1/responses`; support for third-party reuse of Codex credentials at `/v1/realtime/client_secrets` remains unconfirmed. Technical success does not establish a supported auth or billing contract. ### AI HUD Summary (June 2026) diff --git a/scripts/qa-voice-auth.mjs b/scripts/qa-voice-auth.mjs new file mode 100644 index 000000000..9c8cf0c1e --- /dev/null +++ b/scripts/qa-voice-auth.mjs @@ -0,0 +1,291 @@ +#!/usr/bin/env node +/** + * Live Realtime auth acceptance with a prerecorded Chromium microphone. + * Run against the local dev server after configuring the selected credential: + * node scripts/qa-voice-auth.mjs http://127.0.0.1:4189 oauth microphone.wav + * Use a short spoken reply request with three seconds of leading silence. + * Saves only usage, transport metadata, and screenshots; never credentials. + */ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import puppeteer from 'puppeteer'; + +const root = fileURLToPath(new URL('../', import.meta.url)); +const url = new URL(process.argv[2] || 'http://127.0.0.1:4189'); +const auth = process.argv[3] || 'oauth'; +assert.ok(['oauth', 'api-key'].includes(auth), 'Choose oauth or api-key'); +assert.ok( + ['127.0.0.1', 'localhost', '[::1]'].includes(url.hostname), + 'Use a local server', +); +assert.ok(process.argv[4], 'Provide a prerecorded microphone WAV'); +const wav = path.resolve(process.argv[4]); +assert.ok(fs.existsSync(wav), 'Microphone WAV must exist'); +url.searchParams.set('welcome', '0'); +url.searchParams.set('setup', '1'); +const output = path.join(root, '.gev-logs', 'voice-auth'); +fs.mkdirSync(output, { recursive: true }); +const executablePath = + process.env.PUPPETEER_EXECUTABLE_PATH || + (fs.existsSync('/Applications/Google Chrome.app/Contents/MacOS/Google Chrome') + ? '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome' + : puppeteer.executablePath()); +const browser = await puppeteer.launch({ + headless: true, + executablePath, + args: [ + '--no-sandbox', + '--disable-setuid-sandbox', + '--use-fake-device-for-media-stream', + `--use-file-for-fake-audio-capture=${wav}%noloop`, + '--autoplay-policy=no-user-gesture-required', + '--disable-background-timer-throttling', + '--window-size=1440,900', + ], +}); +const result = { + ok: false, + auth, + input: 'prerecorded microphone', + mints: [], + negotiations: [], +}; +let page; +const readSession = () => + page.evaluate(async () => { + const voice = window.__godsEyeView.voiceCommands; + const stats = voice.pc ? [...(await voice.pc.getStats()).values()] : []; + const input = stats.find( + (entry) => entry.type === 'outbound-rtp' && entry.kind === 'audio', + ); + const source = stats.find( + (entry) => entry.type === 'media-source' && entry.kind === 'audio', + ); + const audio = stats.find( + (entry) => entry.type === 'inbound-rtp' && entry.kind === 'audio', + ); + const meter = document.getElementById('gev-voice-cost-value'); + return { + ...window.__voiceAuthEvidence, + connection: voice.pc?.connectionState, + microphoneEnabled: voice.stream + ?.getAudioTracks() + .every((track) => track.enabled), + inputBytes: input?.bytesSent || 0, + inputAudioEnergy: source?.totalAudioEnergy ?? null, + outputBytes: audio?.bytesReceived || 0, + outputAudioEnergy: audio?.totalAudioEnergy || 0, + playbackTime: voice.audioEl?.currentTime || 0, + meter: { + text: meter?.textContent, + usage: meter?.title, + visible: Boolean(meter?.getClientRects().length), + }, + }; + }); +try { + await browser + .defaultBrowserContext() + .overridePermissions(url.origin, ['microphone']); + page = await browser.newPage(); + await page.setViewport({ width: 1440, height: 900 }); + page.on('response', (response) => { + const route = new URL(response.url()).pathname; + if (route === '/api/realtime/token') + result.mints.push({ + status: response.status(), + auth: response.headers()['x-gev-voice-auth'], + model: response.headers()['x-gev-voice-model'], + }); + if ( + route === '/v1/realtime/calls' && + response.request().method() === 'POST' + ) + result.negotiations.push({ status: response.status() }); + }); + // Observe the application's real transport. Do not replace fetch or replies. + await page.evaluateOnNewDocument(() => { + window.__voiceAuthEvidence = { + speechStopped: 0, + transcripts: [], + replies: [], + responses: [], + errors: [], + eventCounts: {}, + }; + const create = RTCPeerConnection.prototype.createDataChannel; + RTCPeerConnection.prototype.createDataChannel = function (...args) { + const channel = create.apply(this, args); + channel.addEventListener('message', ({ data }) => { + let event; + try { + event = JSON.parse(data); + } catch { + return; + } + const evidence = window.__voiceAuthEvidence; + evidence.eventCounts[event.type] = + (evidence.eventCounts[event.type] || 0) + 1; + if (event.type === 'input_audio_buffer.speech_stopped') + evidence.speechStopped += 1; + if ( + event.type === 'conversation.item.input_audio_transcription.completed' + ) + evidence.transcripts.push(event.transcript); + if (event.type === 'response.output_audio_transcript.done') + evidence.replies.push(event.transcript); + if (event.type === 'response.done') + evidence.responses.push({ + status: event.response?.status, + inputTokens: event.response?.usage?.input_tokens, + outputTokens: event.response?.usage?.output_tokens, + }); + if (event.type === 'error') + evidence.errors.push({ code: event.error?.code }); + }); + return channel; + }; + }); + await page.goto(url.href, { waitUntil: 'domcontentloaded', timeout: 30_000 }); + await page.waitForFunction( + () => + window.__godsEyeView?.voiceCommands && + document.getElementById('loading-screen')?.classList.contains('hidden'), + { timeout: 45_000 }, + ); + await page.waitForSelector('[data-cloud-voice-auth-toggle]', { + visible: true, + }); + await page.waitForFunction( + () => + getComputedStyle(document.getElementById('key-setup')).opacity === '1', + ); + const selected = await page.evaluate( + () => localStorage.getItem('godsEyeView.voice.cloudAuth') || 'api-key', + ); + if (selected !== auth) { + await page.click('[data-cloud-voice-auth-toggle]'); + await page.waitForFunction( + (mode) => localStorage.getItem('godsEyeView.voice.cloudAuth') === mode, + { timeout: 125_000 }, + auth, + ); + } + await page.click('[data-key-setup-close]'); + await page.click('#gev-voice-button'); + await page.waitForFunction( + () => + window.__godsEyeView.voiceCommands.dc?.readyState === 'open' || + window.__godsEyeView.voiceCommands.status === 'error', + { timeout: 35_000 }, + ); + assert.equal( + await page.evaluate( + () => window.__godsEyeView.voiceCommands.dc?.readyState, + ), + 'open', + 'Realtime channel must open', + ); + await page.waitForFunction( + () => + window.__voiceAuthEvidence.speechStopped > 0 && + window.__voiceAuthEvidence.responses.some( + (response) => + response.status === 'completed' && response.outputTokens > 0, + ), + { timeout: 45_000 }, + ); + await page.waitForFunction( + () => + window.__voiceAuthEvidence.eventCounts['output_audio_buffer.stopped'] > 0, + { timeout: 30_000 }, + ); + result.session = await readSession(); + assert.ok( + result.session.inputBytes > 0 && result.session.outputBytes > 0, + 'Audio must flow in both directions', + ); + assert.ok( + result.session.outputAudioEnergy > 0 && result.session.playbackTime > 0, + 'Reply audio must play', + ); + assert.equal( + result.session.meter.text === 'COST UNKNOWN', + auth === 'oauth', + 'Use the correct accounting display', + ); + await page.screenshot({ path: path.join(output, `${auth}-usage.png`) }); + await page.click('#key-setup-chip'); + await page.waitForSelector('[data-cloud-voice-auth-toggle]', { + visible: true, + }); + await page.waitForFunction( + () => + getComputedStyle(document.getElementById('key-setup')).opacity === '1', + ); + await page.screenshot({ path: path.join(output, `${auth}-settings.png`) }); + await page.click('[data-key-setup-close]'); + result.stopped = await page.evaluate(() => { + const voice = window.__godsEyeView.voiceCommands; + const pc = voice.pc; + const tracks = voice.stream?.getTracks() || []; + voice.session.stop(); + return ( + pc.connectionState === 'closed' && + tracks.every((track) => track.readyState === 'ended') + ); + }); + assert.ok(result.stopped, 'Stop must close the peer and microphone tracks'); + await page.click('#gev-voice-button'); + await page.waitForFunction( + () => + window.__godsEyeView.voiceCommands.dc?.readyState === 'open' || + window.__godsEyeView.voiceCommands.status === 'error', + { timeout: 35_000 }, + ); + assert.equal( + await page.evaluate( + () => window.__godsEyeView.voiceCommands.dc?.readyState, + ), + 'open', + 'Reconnect must work', + ); + assert.equal( + result.mints.length, + 2, + 'Reconnect must request a fresh client secret', + ); + assert.ok( + result.mints.every( + (mint) => + mint.status === 200 && + mint.auth === (auth === 'oauth' ? 'codex-oauth' : 'api-key'), + ), + ); + assert.ok(result.negotiations.every((entry) => entry.status === 201)); + result.ok = true; +} catch (error) { + result.error = error.message; + if (page) { + result.session = await readSession().catch(() => null); + result.detail = await page + .$eval('#gev-voice-detail', (element) => element.textContent) + .catch(() => null); + await page + .screenshot({ path: path.join(output, `${auth}-failure.png`) }) + .catch(() => {}); + } +} finally { + await page + ?.evaluate(() => window.__godsEyeView?.voiceCommands?.session?.stop()) + .catch(() => {}); + await browser.close(); + fs.writeFileSync( + path.join(output, `${auth}-result.json`), + `${JSON.stringify(result, null, 2)}\n`, + ); +} +console.log(JSON.stringify(result, null, 2)); +if (!result.ok) process.exitCode = 1; diff --git a/server/providers/openai.js b/server/providers/openai.js index 6e8c65281..ad2161d6e 100644 --- a/server/providers/openai.js +++ b/server/providers/openai.js @@ -7,6 +7,7 @@ import { createRealtimeTokenHandler, } from './openai/realtime.js'; import { sameSiteGated } from './common/same-site.js'; +import { createCodexOAuthLogin } from './openai/codex-auth.js'; /** * Vite plugin: OpenAI Realtime ephemeral client secret. @@ -19,6 +20,7 @@ function openAiRealtimeProxy({ annotationGuidance, realtime = {}, } = {}) { + const oauthLogin = createCodexOAuthLogin(realtime); function install(middlewares) { // Cost-bearing and log endpoints refuse cross-site browser requests // (see server/providers/common/same-site.js and SECURITY.md). @@ -38,12 +40,12 @@ function openAiRealtimeProxy({ middlewares.use( '/api/realtime/oauth-status', - createRealtimeOAuthStatusHandler(realtime), + sameSiteGated(createRealtimeOAuthStatusHandler({ oauthLogin })), ); middlewares.use( '/api/realtime/oauth-login', - createRealtimeOAuthLoginHandler(realtime), + sameSiteGated(createRealtimeOAuthLoginHandler({ oauthLogin })), ); } @@ -51,10 +53,13 @@ function openAiRealtimeProxy({ name: 'openai-realtime-proxy', configureServer(server) { install(server.middlewares); + server.httpServer?.once('close', oauthLogin.dispose); }, configurePreviewServer(server) { install(server.middlewares); + server.httpServer?.once('close', oauthLogin.dispose); }, + closeBundle: oauthLogin.dispose, }; } diff --git a/server/providers/openai/codex-auth.js b/server/providers/openai/codex-auth.js index 68c20e929..376830cf4 100644 --- a/server/providers/openai/codex-auth.js +++ b/server/providers/openai/codex-auth.js @@ -34,6 +34,12 @@ export function preferCodexOAuth(environment = process.env) { ); } +function codexOAuthError(message, code = 'CODEX_OAUTH_UNAVAILABLE') { + const error = new Error(message); + error.code = code; + return error; +} + export function readCodexOAuthAccessToken({ authPath = codexAuthPath(), readFile = readFileSync, @@ -46,10 +52,10 @@ export function readCodexOAuthAccessToken({ try { parsed = JSON.parse(readFile(authPath, 'utf8')); } catch { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } const mode = String(parsed.auth_mode || '') .trim() @@ -58,15 +64,8 @@ export function readCodexOAuthAccessToken({ ? mode === 'chatgpt' || mode === 'chatgptauthtokens' : typeof parsed.OPENAI_API_KEY !== 'string'; const token = parsed.tokens?.access_token; - const refresh = parsed.tokens?.refresh_token; - if ( - !chatGptMode || - typeof token !== 'string' || - !token.trim() || - typeof refresh !== 'string' || - !refresh.trim() - ) { - throw new Error(unavailable); + if (!chatGptMode || typeof token !== 'string' || !token.trim()) { + throw codexOAuthError(unavailable); } // This only checks token shape and expiry. OpenAI verifies the signature when @@ -76,33 +75,34 @@ export function readCodexOAuthAccessToken({ parts.length !== 3 || parts.some((part) => !/^[A-Za-z0-9_-]+$/.test(part)) ) { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } let payload; try { payload = JSON.parse(Buffer.from(parts[1], 'base64url').toString('utf8')); } catch { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } if (!payload || typeof payload !== 'object' || Array.isArray(payload)) { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } let expiresAtMs; if (Object.hasOwn(payload, 'exp')) { if (typeof payload.exp !== 'number' || !Number.isFinite(payload.exp)) { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } expiresAtMs = payload.exp * 1000; } else { try { expiresAtMs = statFile(authPath).mtimeMs + 60 * 60 * 1000; } catch { - throw new Error(unavailable); + throw codexOAuthError(unavailable); } } if (!Number.isFinite(expiresAtMs) || expiresAtMs <= nowMs + 60_000) { - throw new Error( - 'ChatGPT sign-in has expired. Run codex login and try again.', + throw codexOAuthError( + 'ChatGPT sign-in has expired. Re-authenticate in Provider Settings or run codex login and try again.', + 'CODEX_OAUTH_REAUTH_REQUIRED', ); } return token; @@ -144,28 +144,202 @@ export function startCodexChatGptLogin({ home = os.homedir(), executable = resolveCodexExecutable({ environment, home }), } = {}) { + // Codex owns its credential storage. An explicit read path must agree with + // the CLI's write path; never silently sign in to another credential store. + const configuredHome = String(environment.CODEX_HOME || '').trim(); + const loginHome = configuredHome + ? path.resolve(configuredHome) + : path.join(home, '.codex'); + if ( + codexAuthPath({ environment, home }) !== path.join(loginHome, 'auth.json') + ) { + return Promise.reject( + codexOAuthError( + 'CODEX_AUTH_JSON does not match the Codex login storage path. Set CODEX_HOME to the matching directory or sign in to Codex separately.', + 'CODEX_OAUTH_STORAGE_MISMATCH', + ), + ); + } + return new Promise((resolve, reject) => { let child; + let spawned = false; + let finish; + const completion = new Promise((done) => { + finish = done; + }); + const fail = () => { + reject( + codexOAuthError( + 'Could not start ChatGPT sign-in. Check that Codex is installed, or set CODEX_BIN to its executable.', + 'CODEX_OAUTH_LOGIN_FAILED', + ), + ); + finish({ exitCode: null }); + }; try { child = spawnImpl(executable, ['login'], { cwd: home, detached: true, stdio: 'ignore', - env: environment, + env: { ...environment, CODEX_HOME: loginHome }, }); - } catch (error) { - reject(error); + } catch { + fail(); return; } - child.once('error', reject); + child.once('error', () => { + if (!spawned) fail(); + else finish({ exitCode: null }); + }); + child.once('exit', (exitCode) => finish({ exitCode })); child.once('spawn', () => { + spawned = true; child.unref?.(); - resolve({ executable }); + resolve({ + completion, + cancel() { + try { + child.kill?.('SIGTERM'); + } catch { + /* Already stopped. */ + } + }, + }); }); - }).catch((error) => { - throw new Error( - `Could not start ChatGPT sign-in with Codex: ${error?.message || error}`, - ); }); } + +/** One local login owner shared by the launch and status endpoints. */ +export function createCodexOAuthLogin({ + resolveOAuthAccessToken = () => readCodexOAuthAccessToken(), + startOAuthLogin = () => startCodexChatGptLogin(), + loginTimeoutMs = 120_000, + setTimer = setTimeout, + clearTimer = clearTimeout, +} = {}) { + let active = null; + let failure = null; + let disposed = false; + + function credentials() { + try { + if (resolveOAuthAccessToken()) return { available: true }; + } catch (error) { + const code = error?.code || 'CODEX_OAUTH_UNAVAILABLE'; + return { + available: false, + error: error?.message || 'ChatGPT sign-in is unavailable', + code, + reauthRequired: code === 'CODEX_OAUTH_REAUTH_REQUIRED', + }; + } + return { + available: false, + error: 'ChatGPT sign-in is unavailable', + code: 'CODEX_OAUTH_UNAVAILABLE', + reauthRequired: false, + }; + } + + function status() { + const current = credentials(); + if (current.available) return current; + if (failure) return { available: false, ...failure, loginFailed: true }; + return active ? { ...current, pending: true } : current; + } + + function finish(attempt, error) { + if (active !== attempt) return; + clearTimer(attempt.timer); + active = null; + failure = error; + } + + async function start() { + if (disposed) + throw codexOAuthError( + 'ChatGPT sign-in is unavailable because the server has stopped.', + 'CODEX_OAUTH_LOGIN_FAILED', + ); + if (credentials().available) + return { available: true, started: false, pending: false }; + if (active) return { available: false, started: false, pending: true }; + + // Reserve the attempt before awaiting spawn so concurrent POSTs share it. + const attempt = { process: null, timer: null }; + active = attempt; + failure = null; + attempt.timer = setTimer(() => { + finish(attempt, { + code: 'CODEX_OAUTH_LOGIN_TIMEOUT', + error: + 'ChatGPT sign-in timed out. Click USE CHATGPT OAUTH to try again.', + }); + attempt.process?.cancel(); + }, loginTimeoutMs); + attempt.timer?.unref?.(); + + try { + const login = await startOAuthLogin(); + if (!login?.completion || typeof login.cancel !== 'function') + throw new Error('Invalid login process'); + attempt.process = login; + if (active !== attempt) { + login.cancel(); + return { ...status(), started: false, pending: false }; + } + Promise.resolve(login.completion).then( + (result) => { + if (active !== attempt) return; + if (result?.exitCode !== 0) { + finish(attempt, { + code: 'CODEX_OAUTH_LOGIN_FAILED', + error: + 'ChatGPT sign-in did not complete. Try again in Provider Settings, or run codex login separately.', + }); + } else if (!credentials().available) { + finish(attempt, { + code: 'CODEX_OAUTH_STORAGE_UNAVAILABLE', + error: + 'Codex sign-in finished, but its configured auth.json has no usable ChatGPT token. Check CODEX_HOME and Codex credential storage. GEV can read file storage only.', + }); + } else finish(attempt, null); + }, + () => + finish(attempt, { + code: 'CODEX_OAUTH_LOGIN_FAILED', + error: + 'ChatGPT sign-in did not complete. Try again in Provider Settings.', + }), + ); + return { available: false, started: true, pending: true }; + } catch (error) { + const code = + error?.code === 'CODEX_OAUTH_STORAGE_MISMATCH' + ? error.code + : 'CODEX_OAUTH_LOGIN_FAILED'; + finish(attempt, { + code, + error: + code === 'CODEX_OAUTH_STORAGE_MISMATCH' + ? error.message + : 'Could not start ChatGPT sign-in. Check that Codex is installed, or set CODEX_BIN to its executable.', + }); + return { ...status(), started: false, pending: false }; + } + } + + return { + status, + start, + dispose() { + disposed = true; + if (!active) return; + const attempt = active; + finish(attempt, null); + attempt.process?.cancel(); + }, + }; +} diff --git a/server/providers/openai/realtime.js b/server/providers/openai/realtime.js index 44f0f9529..588f43c77 100644 --- a/server/providers/openai/realtime.js +++ b/server/providers/openai/realtime.js @@ -16,7 +16,7 @@ import { GEV_REALTIME_TOOLS } from './tools.js'; import { preferCodexOAuth, readCodexOAuthAccessToken, - startCodexChatGptLogin, + createCodexOAuthLogin, } from './codex-auth.js'; const LOOPBACK_ADDRESSES = new Set(['127.0.0.1', '::1', '::ffff:127.0.0.1']); @@ -53,7 +53,7 @@ function resolveRealtimeAuthMode(query) { } function createRealtimeOAuthStatusHandler({ - resolveOAuthAccessToken = () => readCodexOAuthAccessToken(), + oauthLogin = createCodexOAuthLogin(), } = {}) { return (req, res) => { res.setHeader('Cache-Control', 'no-store'); @@ -73,30 +73,14 @@ function createRealtimeOAuthStatusHandler({ ); return; } - try { - const token = resolveOAuthAccessToken(); - if (!token) throw new Error('ChatGPT sign-in is unavailable'); - res.statusCode = 200; - res.end(JSON.stringify({ available: true })); - } catch (error) { - res.statusCode = 200; - res.end( - JSON.stringify({ - available: false, - error: error?.message || 'ChatGPT sign-in is unavailable', - }), - ); - } + res.statusCode = 200; + res.end(JSON.stringify(oauthLogin.status())); }; } function createRealtimeOAuthLoginHandler({ - resolveOAuthAccessToken = () => readCodexOAuthAccessToken(), - startOAuthLogin = () => startCodexChatGptLogin(), - now = () => Date.now(), - restartCooldownMs = 30_000, + oauthLogin = createCodexOAuthLogin(), } = {}) { - let lastStartedAt = 0; return async (req, res) => { res.setHeader('Cache-Control', 'no-store'); res.setHeader('Content-Type', 'application/json'); @@ -127,42 +111,15 @@ function createRealtimeOAuthLoginHandler({ } try { - if (resolveOAuthAccessToken()) { - res.statusCode = 200; - res.end( - JSON.stringify({ available: true, started: false, pending: false }), - ); - return; - } + const result = await oauthLogin.start(); + res.statusCode = result.available ? 200 : result.loginFailed ? 503 : 202; + res.end(JSON.stringify(result)); } catch { - // No usable local token yet; launch the supported Codex sign-in flow. - } - - const currentTime = now(); - if ( - lastStartedAt > 0 && - currentTime - lastStartedAt < Math.max(0, restartCooldownMs) - ) { - res.statusCode = 202; - res.end( - JSON.stringify({ available: false, started: false, pending: true }), - ); - return; - } - - try { - await startOAuthLogin(); - lastStartedAt = currentTime; - res.statusCode = 202; - res.end( - JSON.stringify({ available: false, started: true, pending: true }), - ); - } catch (error) { res.statusCode = 503; res.end( JSON.stringify({ available: false, - error: error?.message || 'Could not start ChatGPT sign-in', + error: 'Could not start ChatGPT sign-in', }), ); } @@ -242,7 +199,7 @@ function createRealtimeTokenHandler({ res.end( JSON.stringify({ error: error?.message || 'ChatGPT sign-in is unavailable', - code: 'CODEX_OAUTH_UNAVAILABLE', + code: error?.code || 'CODEX_OAUTH_UNAVAILABLE', }), ); return; diff --git a/src/codexOauthRealtime.test.mjs b/src/codexOauthRealtime.test.mjs index ffd2b169f..391082ff4 100644 --- a/src/codexOauthRealtime.test.mjs +++ b/src/codexOauthRealtime.test.mjs @@ -61,6 +61,19 @@ test('Codex OAuth reader accepts a current ChatGPT token and rejects unusable cr }), current, ); + assert.equal( + readCodexOAuthAccessToken({ + authPath: '/unused', + nowMs, + readFile: () => + JSON.stringify({ + auth_mode: 'chatgpt', + tokens: { access_token: current }, + }), + }), + current, + 'GEV does not require or consume the vendor refresh token', + ); for (const readFile of [ () => '{bad json', () => JSON.stringify({ tokens: {} }), @@ -70,11 +83,6 @@ test('Codex OAuth reader accepts a current ChatGPT token and rejects unusable cr OPENAI_API_KEY: 'sk-fixture', tokens: { access_token: current, refresh_token: 'refresh-fixture' }, }), - () => - JSON.stringify({ - auth_mode: 'chatgpt', - tokens: { access_token: current }, - }), () => auth('opaque-token'), () => auth('header.not-json.signature'), () => auth(jwt({ exp: 'tomorrow' })), @@ -92,13 +100,18 @@ test('Codex OAuth reader accepts a current ChatGPT token and rejects unusable cr test('Codex OAuth reader enforces expiry margin and bounds tokens without exp by file age', () => { for (const exp of [nowMs / 1000 - 1, nowMs / 1000 + 59]) { assert.throws( - () => + () => { readCodexOAuthAccessToken({ authPath: '/unused', nowMs, readFile: () => auth(jwt({ exp })), - }), - /sign-in has expired/, + }); + }, + (error) => { + assert.match(error.message, /sign-in has expired/); + assert.equal(error.code, 'CODEX_OAUTH_REAUTH_REQUIRED'); + return true; + }, ); } const noExp = jwt({ sub: 'fixture' }); @@ -162,10 +175,88 @@ test('Codex login resolves an installed executable before PATH and starts browse return child; }, }); - assert.deepEqual(await pending, { executable: '/fixture/codex' }); + const login = await pending; assert.equal(unref, 1); assert.equal(calls[0].command, '/fixture/codex'); assert.deepEqual(calls[0].args, ['login']); assert.equal(calls[0].options.detached, true); assert.equal(calls[0].options.stdio, 'ignore'); + assert.equal( + calls[0].options.env.CODEX_HOME, + path.join('/home/fixture', '.codex'), + ); + child.emit('exit', 1); + assert.deepEqual(await login.completion, { exitCode: 1 }); +}); + +test('Codex login refuses a read/write storage mismatch before spawning', async () => { + await assert.rejects( + startCodexChatGptLogin({ + environment: { CODEX_AUTH_JSON: '/tmp/another-auth.json' }, + home: '/home/fixture', + executable: 'codex', + spawnImpl: () => assert.fail('mismatched storage must not launch login'), + }), + { code: 'CODEX_OAUTH_STORAGE_MISMATCH' }, + ); +}); + +test('Codex login resolves relative CODEX_HOME before changing the child directory', async () => { + const child = new EventEmitter(); + const environment = { CODEX_HOME: './fixture-codex' }; + const login = await startCodexChatGptLogin({ + environment, + executable: 'codex', + spawnImpl(_command, _args, options) { + assert.equal( + options.env.CODEX_HOME, + path.resolve(environment.CODEX_HOME), + ); + queueMicrotask(() => child.emit('spawn')); + return child; + }, + }); + assert.equal( + environment.CODEX_HOME, + './fixture-codex', + 'caller environment is unchanged', + ); + child.emit('exit', 0); + assert.deepEqual(await login.completion, { exitCode: 0 }); +}); + +test('Codex login hides spawn errors and can cancel only its owned child', async () => { + const child = new EventEmitter(); + await assert.rejects( + startCodexChatGptLogin({ + executable: 'codex', + spawnImpl() { + queueMicrotask(() => + child.emit('error', new Error('private-token-fixture')), + ); + return child; + }, + }), + (error) => + error.code === 'CODEX_OAUTH_LOGIN_FAILED' && + !error.message.includes('private-token-fixture'), + ); + + const running = new EventEmitter(); + let cancelled = 0; + running.kill = (signal) => { + assert.equal(signal, 'SIGTERM'); + cancelled += 1; + running.emit('exit', null); + }; + const login = await startCodexChatGptLogin({ + executable: 'codex', + spawnImpl() { + queueMicrotask(() => running.emit('spawn')); + return running; + }, + }); + login.cancel(); + assert.equal(cancelled, 1); + assert.deepEqual(await login.completion, { exitCode: null }); }); diff --git a/src/keySetup.js b/src/keySetup.js index 5208c9063..ec8c1aa94 100644 --- a/src/keySetup.js +++ b/src/keySetup.js @@ -75,7 +75,13 @@ export async function waitForChatGptOAuth({ signal, }); const payload = await response.json().catch(() => ({})); + if (signal?.aborted) return false; if (response.ok && payload.available) return true; + if (!response.ok || payload.loginFailed) { + throw new Error( + payload.error || 'Could not check ChatGPT sign-in. Try again.', + ); + } if (now() >= deadline) break; await sleep(Math.max(0, pollMs), signal); } @@ -288,6 +294,7 @@ export async function initKeySetup({ const statusLine = root.querySelector('[data-key-setup-status]'); const defaultStatusText = statusLine?.textContent || ''; let busy = false; + let oauthBusy = false; let open = false; const render = (nextStatus) => { @@ -303,6 +310,10 @@ export async function initKeySetup({ rowsHost.textContent = ''; for (const key of status.keys || []) rowsHost.append(buildRow(documentRef, key)); + for (const button of rowsHost.querySelectorAll( + '[data-cloud-voice-auth-toggle]', + )) + button.disabled = oauthBusy; }; const visible = () => @@ -435,6 +446,7 @@ export async function initKeySetup({ '[data-cloud-voice-auth-toggle]', ); if (authButton && !disposed) { + if (oauthBusy) return; void (async () => { const current = readStoredCloudVoiceAuthMode(); if (current === 'oauth') { @@ -444,6 +456,7 @@ export async function initKeySetup({ return; } authButton.disabled = true; + oauthBusy = true; say('Checking local ChatGPT OAuth sign-in…'); try { const response = await doFetch('/api/realtime/oauth-status', { @@ -451,6 +464,11 @@ export async function initKeySetup({ signal: lifetime.signal, }); const payload = await response.json().catch(() => ({})); + if (lifetime.signal.aborted) return; + if (!response.ok) + throw new Error( + payload.error || 'Could not check ChatGPT sign-in.', + ); if (response.ok && payload.available) { writeStoredCloudVoiceAuthMode('oauth'); render(status); @@ -460,13 +478,18 @@ export async function initKeySetup({ return; } - say('Opening ChatGPT sign-in in your browser…'); + say( + payload.code === 'CODEX_OAUTH_REAUTH_REQUIRED' + ? 'ChatGPT sign-in expired. Opening sign-in again…' + : 'Opening ChatGPT sign-in in your browser…', + ); const loginResponse = await doFetch('/api/realtime/oauth-login', { method: 'POST', cache: 'no-store', signal: lifetime.signal, }); const loginPayload = await loginResponse.json().catch(() => ({})); + if (lifetime.signal.aborted) return; if (!loginResponse.ok && loginResponse.status !== 202) { say( loginPayload.error || @@ -491,9 +514,10 @@ export async function initKeySetup({ fetchImpl: doFetch, signal: lifetime.signal, }); + if (lifetime.signal.aborted) return; if (!available) { say( - 'ChatGPT sign-in is still pending. Finish it in the browser, then click USE CHATGPT OAUTH again.', + 'ChatGPT sign-in timed out. Click USE CHATGPT OAUTH to try again.', ); return; } @@ -506,7 +530,11 @@ export async function initKeySetup({ if (lifetime.signal.aborted) return; say(`OAuth check failed: ${error?.message || error}`); } finally { - authButton.disabled = false; + oauthBusy = false; + for (const button of rowsHost.querySelectorAll( + '[data-cloud-voice-auth-toggle]', + )) + button.disabled = false; } })(); return; diff --git a/src/keySetup.test.mjs b/src/keySetup.test.mjs index a03f2d556..4cdba1765 100644 --- a/src/keySetup.test.mjs +++ b/src/keySetup.test.mjs @@ -84,6 +84,42 @@ test('OAuth login polling stops as soon as local ChatGPT auth becomes available' assert.equal(checks, 3); }); +test('OAuth login polling stops on login or HTTP failure instead of waiting for timeout', async () => { + for (const result of [ + { ok: true, payload: { available: false, loginFailed: true, error: 'Sign-in failed.' } }, + { ok: false, payload: { error: 'Sign-in is local only.' } }, + ]) { + let checks = 0; + await assert.rejects(waitForChatGptOAuth({ + fetchImpl: async () => { checks += 1; return { ok: result.ok, json: async () => result.payload }; }, + sleep: () => assert.fail('failed login must stop polling'), + }), { message: result.payload.error }); + assert.equal(checks, 1); + } +}); + +test('OAuth login polling respects timeout and an aborted setup surface', async () => { + let clock = 0; + let checks = 0; + const poll = { fetchImpl: async () => { checks += 1; return { ok: true, json: async () => ({ available: false, pending: true }) }; } }; + assert.equal(await waitForChatGptOAuth({ ...poll, timeoutMs: 100, pollMs: 100, now: () => clock, sleep: async (ms) => { clock += ms; } }), false); + assert.equal(checks, 2); + const controller = new AbortController(); + controller.abort(); + assert.equal(await waitForChatGptOAuth({ ...poll, signal: controller.signal }), false); + assert.equal(checks, 2, 'aborted polling does not fetch'); +}); + +test('OAuth polling ignores a ready response received after the surface was aborted', async () => { + const controller = new AbortController(); + const available = await waitForChatGptOAuth({ + signal: controller.signal, + fetchImpl: async () => ({ ok: true, json: async () => { controller.abort(); return { available: true }; } }), + sleep: () => assert.fail('aborted polling must not continue'), + }); + assert.equal(available, false); +}); + test('the first Google key strips ONLY the keyless OSM basemap from the share hash', () => { const stripped = stripKeylessBasemapFromHash('lat=30.2&lon=-97.7&map=osm&style=normal'); assert.ok(stripped !== null); diff --git a/src/tooling/localServices.test.mjs b/src/tooling/localServices.test.mjs index 05296ec7e..f6f78f6b6 100644 --- a/src/tooling/localServices.test.mjs +++ b/src/tooling/localServices.test.mjs @@ -299,25 +299,72 @@ test('OAuth status reports local sign-in availability without returning the toke assert.equal(response.body.includes('fixture-oauth-secret'), false); }); -test('OAuth login endpoint launches local Codex sign-in once and never exposes credentials', async () => { - let starts = 0; - let clock = 1_000; +test('OAuth status and token minting report expired sign-in as re-auth required', async () => { + const expired = () => { + const error = new Error('ChatGPT sign-in has expired.'); + error.code = 'CODEX_OAUTH_REAUTH_REQUIRED'; + throw error; + }; const routes = install( openAiRealtimeProxy({ realtime: { - resolveOAuthAccessToken: () => { - throw new Error('not signed in'); - }, - startOAuthLogin: async () => { - starts += 1; - }, - now: () => clock, + resolveOAuthAccessToken: expired, + fetchImpl: () => assert.fail('expired OAuth must not reach OpenAI'), }, }), ); + + const status = await request(routes.get('/api/realtime/oauth-status')); + assert.equal(status.status, 200); + assert.deepEqual(status.json(), { + available: false, + error: 'ChatGPT sign-in has expired.', + code: 'CODEX_OAUTH_REAUTH_REQUIRED', + reauthRequired: true, + }); + + const token = await request(routes.get('/api/realtime/token'), { + url: '/?auth=oauth', + }); + assert.equal(token.status, 503); + assert.equal(token.json().code, 'CODEX_OAUTH_REAUTH_REQUIRED'); +}); + +test('OAuth login endpoint shares pending attempts and never exposes credentials', async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + let starts = 0; + let spawnReady; + let exit; + const plugin = openAiRealtimeProxy({ + realtime: { + resolveOAuthAccessToken: () => { + throw new Error('not signed in'); + }, + startOAuthLogin: async () => { + starts += 1; + await new Promise((resolve) => { + spawnReady = resolve; + }); + return { + completion: new Promise((resolve) => { + exit = resolve; + }), + cancel() {}, + }; + }, + }, + }); + t.after(plugin.closeBundle); + const routes = install(plugin); const handler = routes.get('/api/realtime/oauth-login'); - const started = await request(handler, { method: 'POST' }); + const firstRequest = request(handler, { method: 'POST' }); + const concurrent = await request(handler, { method: 'POST' }); + assert.equal(concurrent.status, 202); + assert.equal(concurrent.json().started, false); + assert.equal(starts, 1, 'reserve the attempt before awaiting spawn'); + spawnReady(); + const started = await firstRequest; assert.equal(started.status, 202); assert.deepEqual(started.json(), { available: false, @@ -326,7 +373,7 @@ test('OAuth login endpoint launches local Codex sign-in once and never exposes c }); assert.equal(starts, 1); - clock += 1_000; + t.mock.timers.tick(31_000); const duplicate = await request(handler, { method: 'POST' }); assert.equal(duplicate.status, 202); assert.equal(duplicate.json().pending, true); @@ -345,6 +392,169 @@ test('OAuth login endpoint launches local Codex sign-in once and never exposes c }); assert.equal(crossOrigin.status, 403); assert.equal(starts, 1); + + exit({ exitCode: 1 }); + await Promise.resolve(); + const failed = await request(routes.get('/api/realtime/oauth-status')); + assert.equal(failed.json().loginFailed, true); + assert.equal(failed.json().code, 'CODEX_OAUTH_LOGIN_FAILED'); +}); + +test('OAuth status follows process completion, permits retry, and checks credential storage', async (t) => { + let token = null; + let exit; + let starts = 0; + const plugin = openAiRealtimeProxy({ + realtime: { + resolveOAuthAccessToken: () => token, + startOAuthLogin: async () => { + starts += 1; + return { + completion: new Promise((resolve) => { + exit = resolve; + }), + cancel() {}, + }; + }, + }, + }); + t.after(plugin.closeBundle); + const routes = install(plugin); + const launch = () => + request(routes.get('/api/realtime/oauth-login'), { method: 'POST' }); + const status = () => request(routes.get('/api/realtime/oauth-status')); + await launch(); + exit({ exitCode: 0 }); + await Promise.resolve(); + assert.equal( + (await status()).json().code, + 'CODEX_OAUTH_STORAGE_UNAVAILABLE', + 'exit success alone is not usable sign-in', + ); + await launch(); + assert.equal(starts, 2, 'failed attempts can be retried immediately'); + token = 'fixture-private-token'; + exit({ exitCode: 0 }); + await Promise.resolve(); + const ready = await status(); + assert.deepEqual(ready.json(), { available: true }); + assert.equal(ready.body.includes(token), false); +}); + +test('OAuth login timeout and server shutdown cancel the owned process', async () => { + let timeout; + let cancelled = 0; + let cleared = 0; + const plugin = openAiRealtimeProxy({ + realtime: { + resolveOAuthAccessToken: () => null, + startOAuthLogin: async () => ({ + completion: new Promise(() => {}), + cancel() { + cancelled += 1; + }, + }), + setTimer(callback, duration) { + timeout = callback; + assert.equal(duration, 120_000); + return 1; + }, + clearTimer() { + cleared += 1; + }, + }, + }); + const routes = install(plugin); + const launch = () => + request(routes.get('/api/realtime/oauth-login'), { method: 'POST' }); + await launch(); + timeout(); + assert.equal(cancelled, 1); + const status = await request(routes.get('/api/realtime/oauth-status')); + assert.equal(status.json().code, 'CODEX_OAUTH_LOGIN_TIMEOUT'); + await launch(); + plugin.closeBundle(); + plugin.closeBundle(); + assert.equal(cancelled, 2, 'shutdown is idempotent'); + assert.equal(cleared, 2); +}); + +test('OAuth launch errors are safe and cross-origin status requests cannot read credentials', async () => { + let reads = 0; + const routes = install( + openAiRealtimeProxy({ + realtime: { + resolveOAuthAccessToken() { + reads += 1; + return null; + }, + startOAuthLogin: async () => { + throw new Error('private-token-fixture'); + }, + }, + }), + ); + const remote = await request(routes.get('/api/realtime/oauth-status'), { + origin: 'https://example.com', + }); + assert.equal(remote.status, 403); + assert.equal(reads, 0); + const failed = await request(routes.get('/api/realtime/oauth-login'), { + method: 'POST', + }); + assert.equal(failed.status, 503); + assert.equal(failed.json().code, 'CODEX_OAUTH_LOGIN_FAILED'); + assert.equal(failed.body.includes('private-token-fixture'), false); +}); + +test('a login that spawns after timeout is cancelled and cannot replace a retry', async (t) => { + let timeout; + let releaseSpawn; + let cancelled = 0; + let launches = 0; + const plugin = openAiRealtimeProxy({ + realtime: { + resolveOAuthAccessToken: () => null, + startOAuthLogin: async () => { + launches += 1; + if (launches === 1) + await new Promise((resolve) => { + releaseSpawn = resolve; + }); + return { + completion: new Promise(() => {}), + cancel() { + cancelled += 1; + }, + }; + }, + setTimer(callback) { + timeout = callback; + return 1; + }, + clearTimer() {}, + }, + }); + t.after(plugin.closeBundle); + const routes = install(plugin); + const launch = () => + request(routes.get('/api/realtime/oauth-login'), { method: 'POST' }); + const first = launch(); + timeout(); + await launch(); + releaseSpawn(); + await first; + assert.equal(launches, 2); + assert.equal(cancelled, 1, 'late process must not become an orphan'); + const status = ( + await request(routes.get('/api/realtime/oauth-status')) + ).json(); + assert.equal(status.pending, true); + assert.equal( + status.loginFailed, + undefined, + 'the older attempt cannot fail the retry', + ); }); test('OAuth login endpoint selects an existing local sign-in without launching Codex', async () => {