diff --git a/CHANGELOG.md b/CHANGELOG.md index 05fe4fdb8..90abb0716 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -254,6 +254,15 @@ before playback. The id now falls back to a v4 UUID built from `crypto.getRandomValues`, which the server's lease check accepts. +- Move Google geocoding behind the local server. Forward and reverse lookups now + go through `/api/google/geocode` and `/api/google/reverse-geocode` with the + server key, joining Places and Street View. Google's Geocoding web service + refuses referrer-restricted keys, so the previous browser call left + `SECURITY.md`'s advice and working search mutually exclusive: the bundled + browser key had to be unrestricted for search to work at all. That key now + needs only the Map Tiles API and the Maps JavaScript API (Street View). Keyless sessions and the Photon/Nominatim + fallback order are unchanged. + ## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features) - Ship each bundled data pack once. The region, marine, admin-boundary, diff --git a/SECURITY.md b/SECURITY.md index 664a3e6d5..bf88cafb7 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -21,19 +21,19 @@ The golden rule: **secret-bearing API keys stay on the server side.** The dev/pr | ChatGPT sign-in through Codex (optional, experimental) | Codex's own `auth.json` (`CODEX_HOME`, default `~/.codex`); GEV reads it server-side and never writes or refreshes it | Only after **USE CHATGPT OAUTH** is selected in Provider Settings: the server mints the same ephemeral Realtime token with the ChatGPT access token, for loopback clients only | | `AISSTREAM_API_KEY` | Server only | Server holds the AISStream websocket; browser polls the same-origin `/api/vessels` cache | | OpenSky OAuth (`OPENSKY_CLIENT_ID/SECRET`) | Server only | Server mints + refreshes the token behind `/api/flights` | -| `GOOGLE_MAPS_SERVER_API_KEY` (optional, #33) | Server only | Server calls Places (`/api/google/nearby-places`, `/api/google/text-search`) and the Street View fallback with this key; falls back to `GOOGLE_MAPS_API_KEY` when unset | +| `GOOGLE_MAPS_SERVER_API_KEY` (optional, #33) | Server only | Server calls Places (`/api/google/nearby-places`, `/api/google/text-search`), Geocoding (`/api/google/geocode`, `/api/google/reverse-geocode`) and the Street View fallback with this key; falls back to `GOOGLE_MAPS_API_KEY` when unset | ### Three deliberately client-side keys — restrict them These are designed to be used directly in the browser (like a Mapbox public token). They are injected into the client bundle via Vite's `define`, so they **will** be visible in browser devtools. Scope and restrict them rather than trying to hide them: -1. **Google Maps API key** — loads Photorealistic 3D Tiles directly and powers GEV place search and Street View (Maps JavaScript API; add it to the key's API restrictions). **Restrict it** (HTTP referrer + API restriction to the required Google APIs) in the Google Cloud Console. An unrestricted key in a public deployment can be abused and billed to you. +1. **Google Maps API key** — loads Photorealistic 3D Tiles directly and powers Street View (Maps JavaScript API; add it to the key's API restrictions). **Restrict it** (HTTP referrer + API restriction to the required Google APIs) in the Google Cloud Console. An unrestricted key in a public deployment can be abused and billed to you. Restrict it to the **Map Tiles API and Maps JavaScript API only**: a referrer restriction applies to the Maps JavaScript API and the client SDKs, not to the web services, so a key carrying Geocoding under one is refused with `API keys with referer restrictions cannot be used with this API`. Search does not need it — geocoding goes through the server proxies above (#363). 2. **Cesium ion token** (`CESIUM_ION_TOKEN`, optional — for ion-hosted Google Photorealistic 3D Tiles, Bing world imagery, and world terrain) — used as `Cesium.Ion.defaultAccessToken` client-side. Use a public **`assets:read`** token with **URL restrictions** for any hosted deployment. The Community plan has eligibility and usage limits; a public token is not a secret, but it can still consume the account's quota. 3. **Mapillary client token** (`MAPILLARY_CLIENT_TOKEN`, optional, Street Level) — used by the MapillaryJS viewer and Graph API lookups in the browser, and by the server's coverage-tile proxy. Mapillary client tokens are meant for browser use. > The explicit browser `define` block in `build/vite.js` controls exactly what reaches the client: only these three keys. Everything else stays server-side. -**Places and Street View never needed to be on that list** (#33): they're called from the server-side proxies in the table above, which use `GOOGLE_MAPS_SERVER_API_KEY` when it's set. Splitting it from the browser-exposed key lets each key's Google Cloud restriction actually match what it does — the browser key referrer-restricted to the APIs the client loads, the server key IP-restricted (never a referrer, since it never leaves your server) to Places + Street View Static — instead of one key that has to be either over-permissioned or broken for one of its two jobs. A single shared `GOOGLE_MAPS_API_KEY` still works if you don't split them; it just has to cover every API both sides use. +**Places, Geocoding and the Street View Static fallback never needed to be on that list** (#33, #363): they're called from the server-side proxies in the table above, which use `GOOGLE_MAPS_SERVER_API_KEY` when it's set. Splitting it from the browser-exposed key lets each key's Google Cloud restriction actually match what it does — the browser key referrer-restricted to Map Tiles + Maps JavaScript API, the server key IP-restricted (never a referrer, since it never leaves your server) to Places + Geocoding + Street View Static — instead of one key that has to be either over-permissioned or broken for one of its jobs. A single shared `GOOGLE_MAPS_API_KEY` still works if you don't split them, but then it cannot carry a referrer restriction at all: the web services refuse one, and search would be the half that breaks. Never commit real keys. `.env` is gitignored; only `.env.example` (placeholder names) is tracked. On macOS `dev-fresh.sh` can read keys from the Keychain; plain Vite uses env vars or a local `.env`, and Pinokio uses its ignored app `ENVIRONMENT` file. diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 72e0c1ae1..c72cb93e3 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -4445,7 +4445,24 @@ Providers normalize coordinates, canonical name, label, place types and optional bounds. Camera framing, nearby landmark recovery and footprint selection remain in their consumers, including the Capitol identity/containment safeguards. Radio keeps localized country names in labels rather than station filters. -Reverse geocoding and nearby/text-search endpoints retain their existing behavior. +Nearby/text-search endpoints retain their existing behavior. + +**Google geocoding is a server route, not a browser call** (#363). Forward and +reverse lookups go to `/api/google/geocode` and `/api/google/reverse-geocode`, +which carry `GOOGLE_MAPS_SERVER_API_KEY` (falling back to `GOOGLE_MAPS_API_KEY`) +and share the Places routes' opt-in `GEV_RATELIMIT_GOOGLE_PER_MIN` throttle. +Google's Geocoding **web service** refuses referrer-restricted keys, so the +previous browser call forced the bundled browser key — visible in devtools by +design — to be left unrestricted; it now needs only the Map Tiles API and the +Maps JavaScript API (Street View). Both routes answer in Google's own +`status`/`results` shape rather than the Places routes' `places: []` contract, +because the browser adapters normalize that shape directly. The browser key still gates the provider client-side, so a keyless +session spends no request to learn it has no Google: the route's own keyless +answer reads as an unanswered provider, which would stop the chain caching the +negative Photon or Nominatim then produces. Its query is capped at 200 +characters, its reverse coordinates are validated against WGS84 bounds before +the request is paid for, and a malformed viewport bias is dropped rather than +refused, since a bias only ranks results. Only valid answers and definitive misses enter bounded caches; malformed replies, HTTP refusals and outages remain retryable. Searches share a 12-second total diff --git a/scripts/track-regression.mjs b/scripts/track-regression.mjs index 84aa2494e..756daccfc 100644 --- a/scripts/track-regression.mjs +++ b/scripts/track-regression.mjs @@ -443,7 +443,10 @@ async function main() { // for nearby names. Answer both with empty results: the entity-context // assertions are about the SELECTED contact, and a live lookup would // make the run non-hermetic (and bill the owner's Google quota). - if (url.hostname === 'maps.googleapis.com' && url.pathname.startsWith('/maps/api/geocode')) { + if ( + isAppRequest && + (url.pathname === '/api/google/reverse-geocode' || url.pathname === '/api/google/geocode') + ) { return Promise.resolve(jsonResponse({ status: 'ZERO_RESULTS', results: [] })); } if (isAppRequest && url.pathname === '/api/google/nearby-places') { diff --git a/server/providers/local.js b/server/providers/local.js index 9fb6ae9c0..854b77f33 100644 --- a/server/providers/local.js +++ b/server/providers/local.js @@ -83,6 +83,9 @@ export { LL2_CACHE_TTL_MS, launchLibraryRequestHeaders } from './space.js'; export { googlePlacesContextProxy } from './places.js'; export { googleServerApiKey } from './places.js'; export { keylessGooglePlacesResponse } from './places.js'; +export { keylessGoogleGeocodeResponse } from './places.js'; +export { validateGeocodeAddress } from './places.js'; +export { geocodeBounds } from './places.js'; export { adsbLolFallbackAnchor } from './aircraft/opensky.js'; export { readResponseTextCapped } from './common/http.js'; export { readResponseJsonCapped } from './common/http.js'; diff --git a/server/providers/places.js b/server/providers/places.js index 12f0688c6..9c9e88645 100644 --- a/server/providers/places.js +++ b/server/providers/places.js @@ -1,8 +1,13 @@ /** Place-search and route middleware for Node development servers. */ -export { googlePlacesContextProxy } from './places/google.js'; +export { + googlePlacesContextProxy, + validateGeocodeAddress, + geocodeBounds, +} from './places/google.js'; export { googleServerApiKey, keylessGooglePlacesResponse, + keylessGoogleGeocodeResponse, } from './places/google-key.js'; export { installRouteMiddleware } from './places/routes.js'; export { diff --git a/server/providers/places/google-key.js b/server/providers/places/google-key.js index 353bc9e24..ff5fe6356 100644 --- a/server/providers/places/google-key.js +++ b/server/providers/places/google-key.js @@ -14,12 +14,34 @@ export function keylessGooglePlacesResponse(apiKey) { } /** - * Google API key for the SERVER-SIDE calls (Places nearby/text search, the - * CCTV Street View fallback). These never reach the browser, so this key can - * be restricted by server IP and scoped to Places API + Street View Static - * API — while GOOGLE_MAPS_API_KEY stays referrer-restricted to Map Tiles + - * Geocoding for the browser (#33). Splitting them is opt-in: unset, this - * falls back to the shared browser key and nothing changes. + * Geocoding speaks Google's own response shape rather than the `places: []` + * contract its two siblings use, because the browser adapters read `status` + * and `results` directly. The keyless capability reads the same way: a status + * that is not `OK` and no results, which the adapters already treat as "this + * provider did not answer" and pass to the next geocoder in the chain. + */ +export function keylessGoogleGeocodeResponse(apiKey) { + if (String(apiKey ?? '').trim()) return null; + return { + statusCode: 200, + payload: { + configured: false, + status: 'REQUEST_DENIED', + results: [], + error: null, + }, + }; +} + +/** + * Google API key for the SERVER-SIDE calls (Places nearby/text search, + * geocoding, the CCTV Street View fallback). These never reach the browser, so + * this key can be restricted by server IP and scoped to Places API + Geocoding + * API + Street View Static API — while GOOGLE_MAPS_API_KEY stays + * referrer-restricted to Map Tiles + Maps JavaScript API for the browser + * (#33, #363). Splitting them is opt-in: unset, this falls back to the shared + * browser key, which then has to carry every API both sides use and cannot + * take a referrer restriction. */ export function googleServerApiKey() { return resolveGoogleServerKey(process.env); diff --git a/server/providers/places/google.js b/server/providers/places/google.js index a2225c606..1ddaa9593 100644 --- a/server/providers/places/google.js +++ b/server/providers/places/google.js @@ -1,9 +1,11 @@ import { googleServerApiKey, keylessGooglePlacesResponse, + keylessGoogleGeocodeResponse, } from './google-key.js'; import { makeCostRateLimiter, clientKey } from '../common/rate-limit.js'; import { admitSameSite } from '../common/same-site.js'; +import { readResponseJsonCapped } from '../common/http.js'; import { projectNearbyPlaces, projectTextSearchPlaces, @@ -25,7 +27,7 @@ let _googleRateLimiter; */ export const GOOGLE_DEFAULT_PER_MIN = 120; -/** Google cost endpoints (nearby-places + text-search). Null only when set to 0. */ +/** Google cost endpoints (nearby-places, text-search, geocoding). Null only when set to 0. */ function googleRateLimiter() { if (_googleRateLimiter === undefined) _googleRateLimiter = makeCostRateLimiter( @@ -56,6 +58,51 @@ export function validatePlacesCoordinates(searchParams) { return { ok: true, latitude, longitude }; } +/** Longest accepted geocoder query. Real place names are far shorter. */ +const GEOCODE_MAX_QUERY = 200; + +/** + * Upstream deadline and body ceiling for the geocode routes. The deadline stays + * armed through body consumption, so a stalled upstream cannot hold a handler + * open, and the cap bounds a chunked or length-omitted reply. A geocode answer + * is a handful of results; 256 KiB is far above any honest one. + */ +const GEOCODE_UPSTREAM_TIMEOUT_MS = 10_000; + +const GEOCODE_MAX_RESPONSE_BYTES = 256 * 1024; + +/** Validate a free-text geocoder query before consuming request quota. */ +export function validateGeocodeAddress(searchParams) { + const address = String(searchParams.get('address') || '').trim(); + if (!address) return { ok: false, error: 'address is required' }; + if (address.length > GEOCODE_MAX_QUERY) + return { ok: false, error: 'address is too long' }; + return { ok: true, address }; +} + +/** + * Validate the optional viewport bias, which Google reads as + * `sw_lat,sw_lng|ne_lat,ne_lng`. An unparsed value is dropped rather than + * refused: the bias only ranks results, so a malformed one costs relevance, + * not an answer. + */ +export function geocodeBounds(searchParams) { + const raw = String(searchParams.get('bounds') || '').trim(); + if (!raw) return null; + const corners = raw.split('|'); + if (corners.length !== 2) return null; + const parts = corners.flatMap((corner) => corner.split(',')); + // A blank component must be refused before Number() turns it into 0 and a + // half-written corner like `30.4,` reads as the equator. + if (parts.length !== 4 || parts.some((part) => !part.trim())) return null; + const numbers = parts.map(Number); + if (!numbers.every(Number.isFinite)) return null; + const [swLat, swLon, neLat, neLon] = numbers; + if (Math.abs(swLat) > 90 || Math.abs(neLat) > 90) return null; + if (Math.abs(swLon) > 180 || Math.abs(neLon) > 180) return null; + return raw; +} + /** Nearby place labels and view-biased text search, with request-time key resolution. */ export function googlePlacesContextProxy({ resolveApiKey = googleServerApiKey, @@ -293,6 +340,149 @@ export function googlePlacesContextProxy({ ); } }); + + /** + * Geocoding belongs on the server, not in the page. Google's Geocoding web + * service refuses referrer-restricted keys, so calling it from the browser + * forces GOOGLE_MAPS_API_KEY — which ships in the bundle by design — to be + * left unrestricted (#363). Here the request carries the server key, and + * the browser key no longer needs the Geocoding API at all. + * + * Unlike the two Places routes above, these answer in Google's own shape + * (`status` + `results`) rather than the `places: []` contract: the browser + * normalizers read that shape directly, and keeping it means the proxy adds + * a hop without adding a translation nobody asked for. + */ + function installGeocodeRoute(path, readQuery) { + middlewares.use(path, async (req, res) => { + const refuse = (statusCode, error) => { + res.statusCode = statusCode; + res.setHeader('Content-Type', 'application/json'); + res.end( + JSON.stringify({ status: 'REQUEST_DENIED', results: [], error }), + ); + }; + + // Gate first, like the Places routes above: a cross-site caller learns + // nothing about this endpoint's method surface. + if (admitSameSite(req, res)) return; + if (req.method !== 'GET') { + refuse(405, 'Method not allowed'); + return; + } + + // A keyless geocode has no provider cost, so it resolves before the + // paid-endpoint limiter can consume or exhaust quota (mirrors the + // Places routes above). + const apiKey = resolveApiKey(); + const keyless = keylessGoogleGeocodeResponse(apiKey); + if (keyless) { + res.statusCode = keyless.statusCode; + res.setHeader('Content-Type', 'application/json'); + res.setHeader('Cache-Control', 'no-store'); + res.end(JSON.stringify(keyless.payload)); + return; + } + + const requestUrl = new URL(req.url || '', 'http://localhost'); + const query = readQuery(requestUrl.searchParams); + if (!query.ok) { + refuse(400, query.error); + return; + } + + // Per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN), shared with the + // Places routes because both spend the same Google budget. + const _grl = googleRateLimiter(); + if (_grl && !_grl(clientKey(req))) { + res.statusCode = 429; + res.setHeader('Content-Type', 'application/json'); + res.setHeader('Retry-After', '5'); + res.end( + JSON.stringify({ + status: 'OVER_QUERY_LIMIT', + results: [], + error: 'Rate limit exceeded', + }), + ); + return; + } + + try { + const upstream = new URL( + endpoints.geocode || + 'https://maps.googleapis.com/maps/api/geocode/json', + ); + for (const [name, value] of Object.entries(query.params)) + upstream.searchParams.set(name, value); + upstream.searchParams.set('key', apiKey); + const deadline = AbortSignal.timeout(GEOCODE_UPSTREAM_TIMEOUT_MS); + const response = await fetchImpl(upstream.toString(), { + redirect: 'error', + signal: deadline, + }); + // An unreadable, oversized or timed-out body is not an answer: it + // falls through to the same shaped failure as a refusal below. + const data = await readResponseJsonCapped( + response, + GEOCODE_MAX_RESPONSE_BYTES, + deadline, + ).catch(() => null); + + // A body we could not read is an upstream failure, not an answer with + // a status of its own, so it does not borrow the upstream's code. + if (!data) { + res.statusCode = 502; + res.setHeader('Content-Type', 'application/json; charset=utf-8'); + res.end( + JSON.stringify({ + status: 'UNKNOWN_ERROR', + results: [], + error: 'Google Geocoding request failed', + }), + ); + return; + } + + res.statusCode = response.ok ? 200 : response.status; + res.setHeader('Content-Type', 'application/json; charset=utf-8'); + res.setHeader('Cache-Control', 'private, max-age=300'); + res.end(JSON.stringify(data)); + } catch { + // Fixed public text: a transport, URL or runtime message would + // describe the server to the browser (SECURITY.md, sanitized errors). + res.statusCode = 502; + res.setHeader('Content-Type', 'application/json; charset=utf-8'); + res.end( + JSON.stringify({ + status: 'UNKNOWN_ERROR', + results: [], + error: 'Google Geocoding request failed', + }), + ); + } + }); + } + + installGeocodeRoute('/api/google/geocode', (searchParams) => { + const address = validateGeocodeAddress(searchParams); + if (!address.ok) return address; + const bounds = geocodeBounds(searchParams); + return { + ok: true, + params: { + address: address.address, + ...(bounds ? { bounds } : {}), + }, + }; + }); + + installGeocodeRoute('/api/google/reverse-geocode', (searchParams) => { + const coordinates = validatePlacesCoordinates(searchParams); + if (!coordinates.ok) return coordinates; + const { latitude, longitude } = coordinates; + return { ok: true, params: { latlng: `${latitude},${longitude}` } }; + }); } return { diff --git a/src/annotations/annotationEngine.test.mjs b/src/annotations/annotationEngine.test.mjs index 3c4b9165b..5f39a077a 100644 --- a/src/annotations/annotationEngine.test.mjs +++ b/src/annotations/annotationEngine.test.mjs @@ -404,7 +404,7 @@ test('outline upgrade updates the rendered element in place without remove/add', }; let overpassCall = 0; globalThis.fetch = async (url) => { - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { return { json: async () => ({ status: 'OK', results: [{ diff --git a/src/annotations/annotationResolver.test.mjs b/src/annotations/annotationResolver.test.mjs index 452d6997c..2c51213c1 100644 --- a/src/annotations/annotationResolver.test.mjs +++ b/src/annotations/annotationResolver.test.mjs @@ -200,7 +200,7 @@ test('ask-side admin bypass: "the Texas Capitol" recovers near-view despite a fa const calls = []; installGoogleMocks(t, async (url) => { calls.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { return { json: async () => geocodePayload({ lat: 31.0000, lon: -99.0000, @@ -235,7 +235,7 @@ test('ask-side admin bypass: explicit "state of Texas" skips recovery and proxim const calls = []; installGoogleMocks(t, async (url) => { calls.push(String(url)); - assert.match(String(url), /^https:\/\/maps\.googleapis\.com\/maps\/api\/geocode/); + assert.match(String(url), /^\/api\/google\/geocode\?/); return { json: async () => geocodePayload({ lat: 31.0000, lon: -99.0000, @@ -273,7 +273,7 @@ for (const fixture of [ const calls = []; installGoogleMocks(t, async (url) => { calls.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { return { json: async () => geocodePayload({ lat: fixture.lat, lon: fixture.lon, @@ -299,7 +299,7 @@ test('ask-side admin bypass: bare "Texas" remains on the guarded recovery path', const calls = []; installGoogleMocks(t, async (url) => { calls.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { return { json: async () => geocodePayload({ lat: 31.0000, lon: -99.0000, @@ -328,7 +328,7 @@ test('ask-side admin bypass: admin level 2/3 result types never grant a township const calls = []; installGoogleMocks(t, async (url) => { calls.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { const query = new URL(String(url)).searchParams.get('address'); return { json: async () => geocodePayload({ lat: 39.7817, @@ -380,7 +380,7 @@ function installCapitolMocks(t, elements, components = [ { long_name: 'Capitol Hill', types: ['neighborhood', 'political'] }, ]) { installGoogleMocks(t, async (url) => { - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { return { json: async () => ({ status: 'OK', results: [{ formatted_address: 'Washington, DC 20004, USA', types: ['establishment', 'landmark', 'point_of_interest', 'tourist_attraction'], @@ -458,7 +458,7 @@ test('keyless: a key that geocodes to nothing still anchors the annotation', asy const requests = []; installGoogleMocks(t, async (url) => { requests.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) return GOOGLE_FOUND_NOTHING; + if (String(url).startsWith('/api/google/geocode')) return GOOGLE_FOUND_NOTHING; return { ok: true, json: async () => ({ @@ -495,7 +495,7 @@ test('keyless: a key that geocodes to nothing still anchors the annotation', asy high: { latitude: 30.28, longitude: -97.68 }, }); // Google is asked first and exactly once; one Photon call answers it. - assert.equal(requests.filter((url) => url.includes('maps.googleapis.com')).length, 1); + assert.equal(requests.filter((url) => url.includes('/api/google/geocode')).length, 1); assert.equal(requests.filter((url) => url.includes('photon.komoot.io')).length, 1); }); @@ -512,7 +512,7 @@ test('keyless: OSM matching uses the feature\'s canonical name, not the user\'s installGoogleMocks(t, async (url, init) => { const href = String(url); - if (href.startsWith('https://maps.googleapis.com/')) return GOOGLE_FOUND_NOTHING; + if (href.startsWith('/api/google/geocode')) return GOOGLE_FOUND_NOTHING; if (href.startsWith('/api/google/text-search')) return { ok: true, json: async () => ({ places: [] }) }; if (href.startsWith('https://photon.komoot.io/')) { return { @@ -575,7 +575,7 @@ test('keyless: a Photon outage is not remembered as "no such place"', async (t) const requests = []; installGoogleMocks(t, async (url) => { requests.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) return GOOGLE_FOUND_NOTHING; + if (String(url).startsWith('/api/google/geocode')) return GOOGLE_FOUND_NOTHING; if (!photonReachable) throw new Error('network down'); return photonUp; }); diff --git a/src/googleGeocodeProxy.test.mjs b/src/googleGeocodeProxy.test.mjs new file mode 100644 index 000000000..98ffd876e --- /dev/null +++ b/src/googleGeocodeProxy.test.mjs @@ -0,0 +1,204 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { + googlePlacesContextProxy, + validateGeocodeAddress, + geocodeBounds, +} from '../vite.config.js'; + +const UPSTREAM = 'https://geocode.test/json'; + +function installRoutes({ apiKey = 'server-secret', fetchImpl } = {}) { + const routes = new Map(); + googlePlacesContextProxy({ + resolveApiKey: () => apiKey, + fetchImpl, + endpoints: { geocode: UPSTREAM }, + }).configureServer({ + middlewares: { use: (name, handler) => routes.set(name, handler) }, + }); + return routes; +} + +function invokeRoute(handler, { method = 'GET', url = '/' } = {}) { + return new Promise((resolve, reject) => { + const headers = new Map(); + const res = { + statusCode: 200, + setHeader(name, value) { + headers.set(String(name).toLowerCase(), String(value)); + }, + end(body = '') { + resolve({ + statusCode: this.statusCode, + headers: Object.fromEntries(headers), + body: body ? JSON.parse(String(body)) : null, + }); + }, + }; + Promise.resolve( + handler( + { method, url, headers: {}, socket: { remoteAddress: '127.0.0.1' } }, + res, + ), + ).catch(reject); + }); +} + +test('geocoder query and viewport bias are validated before Google is paid', () => { + assert.deepEqual(validateGeocodeAddress(new URLSearchParams()), { + ok: false, + error: 'address is required', + }); + assert.deepEqual(validateGeocodeAddress(new URLSearchParams({ address: ' ' })), { + ok: false, + error: 'address is required', + }); + assert.deepEqual( + validateGeocodeAddress(new URLSearchParams({ address: 'x'.repeat(201) })), + { ok: false, error: 'address is too long' }, + ); + assert.deepEqual( + validateGeocodeAddress(new URLSearchParams({ address: ' Austin ' })), + { ok: true, address: 'Austin' }, + ); + + // A bias only ranks results, so an unusable one is dropped, not refused. + const bias = '30.1,-97.9|30.4,-97.6'; + assert.equal(geocodeBounds(new URLSearchParams({ bounds: bias })), bias); + for (const bounds of [ + '', + '30.1,-97.9', + '30.1,-97.9|30.4', + 'north,-97.9|30.4,-97.6', + '91,-97.9|30.4,-97.6', + '30.1,-181|30.4,-97.6', + // A blank component must not read as 0 — that is the equator, not a corner. + '30.1,-97.9|30.4,', + '30.1,|30.4,-97.6', + '30.1,-97.9| ,-97.6', + ]) + assert.equal(geocodeBounds(new URLSearchParams({ bounds })), null); +}); + +test('the geocode routes keep the key server-side and answer in Google shape', async () => { + const upstream = []; + const routes = installRoutes({ + fetchImpl: async (url) => { + upstream.push(new URL(String(url))); + return Response.json({ status: 'OK', results: [{ place_id: 'fixture' }] }); + }, + }); + const forward = routes.get('/api/google/geocode'); + const reverse = routes.get('/api/google/reverse-geocode'); + assert.equal(typeof forward, 'function'); + assert.equal(typeof reverse, 'function'); + + const forwarded = await invokeRoute(forward, { + url: '/?address=Austin&bounds=30.1,-97.9|30.4,-97.6', + }); + assert.equal(forwarded.statusCode, 200); + assert.deepEqual(forwarded.body, { + status: 'OK', + results: [{ place_id: 'fixture' }], + }); + assert.equal(upstream.at(-1).searchParams.get('address'), 'Austin'); + assert.equal(upstream.at(-1).searchParams.get('bounds'), '30.1,-97.9|30.4,-97.6'); + assert.equal(upstream.at(-1).searchParams.get('key'), 'server-secret'); + + const reversed = await invokeRoute(reverse, { url: '/?lat=30.27&lon=-97.74' }); + assert.equal(reversed.statusCode, 200); + assert.equal(upstream.at(-1).searchParams.get('latlng'), '30.27,-97.74'); + assert.equal(upstream.at(-1).searchParams.get('key'), 'server-secret'); + + // The key travels to Google, never back to the page that asked. + assert.ok(!JSON.stringify(forwarded.body).includes('server-secret')); + assert.ok(!JSON.stringify(reversed.body).includes('server-secret')); + + // A malformed bias is dropped rather than forwarded. + await invokeRoute(forward, { url: '/?address=Austin&bounds=north' }); + assert.equal(upstream.at(-1).searchParams.has('bounds'), false); +}); + +test('a bad or keyless geocode request never reaches Google', async () => { + const upstream = []; + const fetchImpl = async (url) => { + upstream.push(String(url)); + return Response.json({ status: 'OK', results: [] }); + }; + + const routes = installRoutes({ fetchImpl }); + for (const [path, url] of [ + ['/api/google/geocode', '/?address='], + ['/api/google/reverse-geocode', '/?lat=91&lon=-97.74'], + ]) { + const response = await invokeRoute(routes.get(path), { url }); + assert.equal(response.statusCode, 400); + assert.deepEqual(response.body.results, []); + assert.equal(response.body.status, 'REQUEST_DENIED'); + } + + for (const path of ['/api/google/geocode', '/api/google/reverse-geocode']) { + const rejected = await invokeRoute(routes.get(path), { + method: 'POST', + url: '/?address=Austin&lat=30&lon=-97', + }); + assert.equal(rejected.statusCode, 405); + } + + // Keyless is an empty capability, not an outage: a 200 the browser adapters + // read as "this provider did not answer", so the chain moves on. It resolves + // ahead of validation, like the Places routes: with nothing configured there + // is no request to validate and no quota to protect. + const keyless = installRoutes({ apiKey: ' ', fetchImpl }); + for (const path of ['/api/google/geocode', '/api/google/reverse-geocode']) + for (const url of ['/?address=Austin&lat=30&lon=-97', '/?address=&lat=91']) { + const response = await invokeRoute(keyless.get(path), { url }); + assert.equal(response.statusCode, 200); + assert.equal(response.headers['cache-control'], 'no-store'); + assert.deepEqual(response.body, { + configured: false, + status: 'REQUEST_DENIED', + results: [], + error: null, + }); + } + + assert.deepEqual(upstream, []); +}); + +test('an upstream failure reaches the browser as fixed public text', async () => { + const routes = installRoutes({ + fetchImpl: async () => { + throw new Error('connect ECONNREFUSED 10.0.0.7:443'); + }, + }); + for (const path of ['/api/google/geocode', '/api/google/reverse-geocode']) { + const response = await invokeRoute(routes.get(path), { + url: '/?address=Austin&lat=30&lon=-97', + }); + assert.equal(response.statusCode, 502); + assert.deepEqual(response.body, { + status: 'UNKNOWN_ERROR', + results: [], + error: 'Google Geocoding request failed', + }); + } +}); + +test('an oversized upstream body is dropped rather than buffered', async () => { + const routes = installRoutes({ + // Two hundred times the 256 KiB ceiling, declared honestly. + fetchImpl: async () => + Response.json({ status: 'OK', results: [{ pad: 'x'.repeat(52_428_800) }] }), + }); + const response = await invokeRoute(routes.get('/api/google/geocode'), { + url: '/?address=Austin', + }); + assert.equal(response.statusCode, 502); + assert.deepEqual(response.body, { + status: 'UNKNOWN_ERROR', + results: [], + error: 'Google Geocoding request failed', + }); +}); diff --git a/src/googleServerKey.test.mjs b/src/googleServerKey.test.mjs index 0851afcd5..2789219f5 100644 --- a/src/googleServerKey.test.mjs +++ b/src/googleServerKey.test.mjs @@ -67,11 +67,19 @@ test('the Street View tool resolves per-variable overrides before preferring the } }); -test('both Places routes select the intended key and keep it out of responses', async (t) => { +test('every Google route selects the intended key and keeps it out of responses', async (t) => { const original = { server: process.env.GOOGLE_MAPS_SERVER_API_KEY, browser: process.env.GOOGLE_MAPS_API_KEY, limit: process.env.GEV_RATELIMIT_GOOGLE_PER_MIN }; const calls = []; t.mock.method(globalThis, 'fetch', async (url, options) => { - calls.push({ url, key: options.headers['X-Goog-Api-Key'] }); + // Places sends the key as a header; Geocoding's web service takes it as a + // query parameter. Either way it must be the server key, and never the + // browser's copy of it. + calls.push({ + url, + key: + options?.headers?.['X-Goog-Api-Key'] ?? + new URL(String(url)).searchParams.get('key'), + }); return Response.json({ places: [] }); }); try { @@ -88,12 +96,12 @@ test('both Places routes select the intended key and keep it out of responses', for (const install of ['configureServer', 'configurePreviewServer']) { const routes = new Map(); googlePlacesContextProxy()[install]({ middlewares: { use: (name, handler) => routes.set(name, handler) } }); - assert.equal(routes.size, 2); + assert.equal(routes.size, 4); for (const handler of routes.values()) { const before = calls.length; let body; const res = { setHeader() {}, end(value) { body = value; } }; - await handler({ method: 'GET', url: '/?lat=30&lon=-97&q=capitol', headers: {}, socket: { remoteAddress: '127.0.0.1' } }, res); + await handler({ method: 'GET', url: '/?lat=30&lon=-97&q=capitol&address=capitol', headers: {}, socket: { remoteAddress: '127.0.0.1' } }, res); assert.equal(res.statusCode, 200); assert.ok(!body.includes('server-secret')); if (expected) { diff --git a/src/search/defaults.js b/src/search/defaults.js index f440ca3ad..e4ca0596c 100644 --- a/src/search/defaults.js +++ b/src/search/defaults.js @@ -42,18 +42,23 @@ export function createDefaultPlaceSearch({ ...(selected ? [selected] : [ + // Google answers through the local proxy, which holds the server + // key: its Geocoding web service refuses referrer-restricted keys, + // so calling it from here would force the bundled browser key to be + // left unrestricted (#363). That key still gates the provider, + // because a keyless session must not spend a request to learn it + // has no Google: the proxy's "not configured" reads as an + // unanswered provider, which stops the chain caching the negative + // Photon or Nominatim then produces. createGoogleGeocoder({ request(query, { bias, signal }) { - const key = resolveApiKey?.(); - if (!key) return null; - const url = new URL( - endpoints.geocode || - 'https://maps.googleapis.com/maps/api/geocode/json', + if (!resolveApiKey?.()) return null; + const params = new URLSearchParams({ address: query }); + if (bias) params.set('bounds', bias); + return fetchImpl( + `${endpoints.geocode || '/api/google/geocode'}?${params}`, + { signal }, ); - url.searchParams.set('address', query); - url.searchParams.set('key', key); - if (bias) url.searchParams.set('bounds', bias); - return fetchImpl(url.toString(), { signal }); }, }), createPhotonGeocoder({ fetchImpl, endpoint: endpoints.photon }), diff --git a/src/search/http.js b/src/search/http.js index 5b1642c67..5afa35b75 100644 --- a/src/search/http.js +++ b/src/search/http.js @@ -53,7 +53,9 @@ export function createHttpGeospatialProvider({ endpoints = {}, } = {}) { const urls = { - reverse: 'https://maps.googleapis.com/maps/api/geocode/json', + // Same-origin like its siblings below: Google's Geocoding web service + // refuses referrer-restricted keys, so the key stays on the server (#363). + reverse: '/api/google/reverse-geocode', textSearch: '/api/google/text-search', nearby: '/api/google/nearby-places', route: '/api/route', @@ -88,14 +90,11 @@ export function createHttpGeospatialProvider({ }, routeProfiles: ['foot', 'car', 'bike'], async reverseGeocode(latitude, longitude, options) { - const key = resolveApiKey?.(); - if (resolveApiKey && !key) return null; + // The browser key gates the provider without travelling with the + // request: the proxy supplies the key Google actually sees. + if (resolveApiKey && !resolveApiKey()) return null; return normalizeGoogleReverse( - await json( - urls.reverse, - { latlng: `${latitude},${longitude}`, ...(key ? { key } : {}) }, - options, - ), + await json(urls.reverse, { lat: latitude, lon: longitude }, options), ); }, async textSearch(query, { latitude, longitude, radiusM = 6000 }, options) { diff --git a/src/search/placeSearch.test.mjs b/src/search/placeSearch.test.mjs index e609f9da9..05e1aaea0 100644 --- a/src/search/placeSearch.test.mjs +++ b/src/search/placeSearch.test.mjs @@ -20,7 +20,7 @@ test('standalone geocoding falls back after Google connection, JSON and refusal for (const fail of [() => { throw new Error('offline'); }, () => new Response('invalid json'), () => Response.json({ status: 'REQUEST_DENIED' })]) { const urls = []; const service = createStandalonePlaceSearch({ resolveApiKey: () => 'fixture', fetchImpl: async (url) => { - urls.push(url); return url.includes('maps.googleapis.com') ? fail() : hit(); + urls.push(url); return url.includes('/api/google/geocode') ? fail() : hit(); } }); const result = await service.geocode('Hanoi'); assert.equal(result.place.name, 'Hà Nội'); diff --git a/src/voice/gevActions.test.mjs b/src/voice/gevActions.test.mjs index 6edc604ef..16b00be4a 100644 --- a/src/voice/gevActions.test.mjs +++ b/src/voice/gevActions.test.mjs @@ -3180,7 +3180,7 @@ test('voice Radio: a key that geocodes to nothing still places the station, keyl const requests = []; installKeyedFetch(t, async (url) => { requests.push(String(url)); - if (String(url).startsWith('https://maps.googleapis.com/')) { + if (String(url).startsWith('/api/google/geocode')) { return { ok: true, json: async () => ({ status: 'ZERO_RESULTS', results: [] }) }; } assert.match(String(url), /^https:\/\/photon\.komoot\.io\/api\/\?/); @@ -3205,7 +3205,7 @@ test('voice Radio: a key that geocodes to nothing still places the station, keyl assert.ok(Math.abs(calls[0].criteria.anchor.lat - 20.9101) < 1e-9); assert.ok(Math.abs(calls[0].criteria.anchor.lon - 107.1839) < 1e-9); // Google is asked first and exactly once; Photon answers unbiased, in one call. - assert.equal(requests.filter((url) => url.includes('maps.googleapis.com')).length, 1); + assert.equal(requests.filter((url) => url.includes('/api/google/geocode')).length, 1); assert.equal(requests.filter((url) => url.includes('photon.komoot.io')).length, 1); assert.match(requests.at(-1), /[?&]q=H%E1%BA%A1\+Long\+Bay/); assert.doesNotMatch(requests.at(-1), /[?&](lat|lon|bbox)=/, 'a named radio location is not viewport-biased'); @@ -3218,7 +3218,7 @@ test('voice Radio: the keyless path applies no country filter the keyed path wou // "No Radio station matched" for exactly the places it just resolved, while a // keyed install placed a station. The label may carry it; the filter may not. const { calls, dataManager } = radioSelectionHarness(); - installKeyedFetch(t, async (url) => (String(url).startsWith('https://maps.googleapis.com/') + installKeyedFetch(t, async (url) => (String(url).startsWith('/api/google/geocode') ? { ok: true, json: async () => ({ status: 'ZERO_RESULTS', results: [] }) } : { ok: true,