CCTV Street View fallback: the browser's pose at the catalog position

The fallback keeps the catalog position but takes heading, field of view
and pitch from the request again, as before, clamped as before. A catalog
camera with no pose (every world-cameras row on main) otherwise asked
Google for heading 0, pitch 0, FOV 80 while the browser draws it at the
id-hash heading, pitch -17 and FOV 74, and a calibrated camera lost its
calibration. Cost and the per-IP budget are unchanged.

- The Street View budget is built per proxy, so tests no longer depend on
  GEV_RATELIMIT_GOOGLE_PER_MIN in the shell; the fixture pins it.
- New tests: a pose-less catalog camera, and the per-IP budget (no call
  without a key, the cap, a separate budget per address).
- The client no longer sends lat/lon on /frame.
- CURRENT-STATE, the changelog fragment and SECURITY.md say the fallback
  has its own per-client limit, and that unlisted (seed) cameras get the
  placeholder.
This commit is contained in:
German Khokhlov committed 2026-11-19 19:06:53 +04:00
1 parent f579b23f52
commit 11e1c9d26d
8 files changed
+235 -36

No files matched your search

+2 -2
View File
@@ -68,7 +68,7 @@ browser-key fallback for existing single-key setups.
The data proxies under `server/providers/` are written so the browser cannot turn the server into an open relay:
- **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). Other proxies target fixed upstream hosts.
- **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). The key-bearing Street View fallback likewise answers only for a registered camera, always at its catalog position: a request sets only the view direction, field of view (20–120°) and pitch (−40–20°), so it cannot buy Street View of a place of its choosing. Other proxies target fixed upstream hosts.
- **Live HLS is bounded and origin-pinned.** Registered HTTP(S) HLS sources use a Node puller that rejects redirects and off-origin playlist references, bounds playlists and segment bodies while streaming, and aborts timed-out or released sessions. At most two sessions retain 12 segments/24 MiB each, plus bounded download buffers. Stores are memory-only and idle sessions expire after 15 seconds. RTMP/ffmpeg execution is not enabled. Locally configured source URLs remain an operator trust boundary.
- **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio.
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
@@ -95,7 +95,7 @@ The dev server is a **key broker**: every server-side key above is spendable by
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Typed chat keeps its own per-IP window, set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`; left unset, each takes its provider's resolved cap (`GEV_RATELIMIT_OPENAI_PER_MIN`, or `GEV_RATELIMIT_GEMINI_PER_MIN`, default `30`), so lowering a provider throttle lowers chat too, while an explicit chat value, including `0`, wins. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). The CCTV Street View fallback has its own per-IP window on `GEV_RATELIMIT_GOOGLE_PER_MIN`, separate from the Places and Geocoding one; over it, a frame gets the placeholder instead of a `429`. Typed chat keeps its own per-IP window, set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`; left unset, each takes its provider's resolved cap (`GEV_RATELIMIT_OPENAI_PER_MIN`, or `GEV_RATELIMIT_GEMINI_PER_MIN`, default `30`), so lowering a provider throttle lowers chat too, while an explicit chat value, including `0`, wins. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **Docker trusts one address, and only because the port is on loopback.**
Docker delivers the host's own browser to the container from the container
network's gateway, never from `127.0.0.1`, so a loopback-only Provider
@@ -1,5 +1,5 @@
- The CCTV Street View fallback serves only catalog cameras, at their catalog
position, heading, field of view and pitch, and ignores coordinates a
request supplies. It spends from the per-client Google budget
(`GEV_RATELIMIT_GOOGLE_PER_MIN`, default 120), so the Google key can no
longer be used as an unmetered Street View service (asturksever, #1602).
- The CCTV Street View fallback serves only catalog cameras, always at their
catalog position, and ignores coordinates a request supplies. It has its own
per-client limit on the Google setting (`GEV_RATELIMIT_GOOGLE_PER_MIN`,
default 120), so the Google key can no longer be used as an unmetered Street
View service (asturksever, #1602).
+8 -3
View File
@@ -1277,9 +1277,14 @@ redirects on those hosts, are pinned by parsed URL to the agency's image
directory. Every `/api/cctv` route refuses cross-site browser requests but
accepts forwarding headers, so cameras work behind a reverse proxy; a forwarded
request skips the key-bearing Street View fallback and gets the placeholder. The
Street View fallback serves only catalog cameras, at the catalog position,
heading, field of view and pitch, never coordinates a request supplies, and it
shares the per-client Google budget (`GEV_RATELIMIT_GOOGLE_PER_MIN`, default 120).
Street View fallback serves only catalog cameras, always at the catalog
position; coordinates a request supplies are ignored. Heading, field of view
and pitch come from the request, the pose the browser draws with calibration
included, and fall back to the catalog pose. A camera the server does not list,
such as a bundled seed camera the browser shows when the server catalog is
empty, gets the placeholder. The fallback has its own per-client limit on the
Google setting (`GEV_RATELIMIT_GOOGLE_PER_MIN`, default 120, `0` disables it);
a request over it gets the placeholder.
The CCTV layer draws only the cameras on or near the screen. Zoomed out, it
keeps one camera per ground cell sized to the view, preferring a healthy feed,
+31 -20
View File
@@ -21,19 +21,6 @@ import { createHlsPuller } from './cctv/stream.js';
import { googleServerApiKey } from './places/google-key.js';
import { clientKey, makeCostRateLimiter } from './common/rate-limit.js';
export { CCTV_FRAME_FETCH_TIMEOUT_MS, fetchCctvImageFromUpstream };
let _streetViewRateLimiter;
/**
* Per-IP budget for the Street View fallback, on the same setting as the
* other Google routes (GEV_RATELIMIT_GOOGLE_PER_MIN; 0 disables it).
*/
function streetViewRateLimiter() {
if (_streetViewRateLimiter === undefined)
_streetViewRateLimiter = makeCostRateLimiter(
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
120,
);
return _streetViewRateLimiter;
}
/**
* Vite plugin: CCTV camera proxy with source registry, frame/media serving,
@@ -96,6 +83,22 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
};
};
/**
* Per-IP budget for the Street View fallback, one per proxy, on the same
* setting as the other Google routes (GEV_RATELIMIT_GOOGLE_PER_MIN; 0
* disables it). Built on first use, after the standalone env has loaded.
* undefined = not built yet; null = the explicit 0 opt-out.
*/
let streetViewBudget;
const streetViewRateLimiter = () => {
if (streetViewBudget === undefined)
streetViewBudget = makeCostRateLimiter(
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
120,
);
return streetViewBudget;
};
/**
* Fetch a Google Street View static image as a fallback frame. Server-side
* call, never reaches the browser — prefers GOOGLE_MAPS_SERVER_API_KEY
@@ -524,6 +527,13 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
const source = sourceById.get(cameraId);
const label = url.searchParams.get('label') || source?.name || cameraId;
const city = url.searchParams.get('city') || source?.city || '';
// The view follows the pose the browser draws (calibration included);
// the position never comes from the query (see the Street View fallback).
const heading = Number(
url.searchParams.get('heading') || source?.headingDeg,
);
const fov = Number(url.searchParams.get('fov') || source?.fovDeg);
const pitch = Number(url.searchParams.get('pitch') || source?.pitchDeg);
// Only use server-registered upstream URLs — never accept client-supplied URLs
// (prevents SSRF via ?upstream= query parameter)
@@ -557,8 +567,9 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
// request (whose client this server cannot see) gets the placeholder,
// as if no key were configured. Any forwarding header counts, even an
// empty one.
// It is also only for catalog cameras, framed from the catalog (never
// a spot the query names), within the per-IP Google budget.
// It is also only for catalog cameras, at the catalog position (never a
// spot the query names; the query sets only the view), within the per-IP
// Google budget.
const forwarded = PROXY_SIGNALS.some(
(name) => req.headers?.[name] !== undefined,
);
@@ -571,11 +582,11 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
(googleBudget && !googleBudget(clientKey(req)))
? null
: await streetViewFallback({
lat: Number(source.lat),
lon: Number(source.lon),
heading: Number(source.headingDeg),
fov: Number(source.fovDeg),
pitch: Number(source.pitchDeg ?? -10),
lat: source.lat,
lon: source.lon,
heading,
fov,
pitch,
});
if (sv?.ok) {
setHealth(cameraId, {
+46 -2
View File
@@ -111,6 +111,7 @@ function mount(t, respond) {
austin: process.env.CCTV_FORCE_AUSTIN,
serverKey: process.env.GOOGLE_MAPS_SERVER_API_KEY,
browserKey: process.env.GOOGLE_MAPS_API_KEY,
rateLimit: process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
};
process.env.CCTV_SOURCES_JSON = JSON.stringify([
STILL,
@@ -125,6 +126,8 @@ function mount(t, respond) {
// than a network call this file does not model.
delete process.env.GOOGLE_MAPS_SERVER_API_KEY;
delete process.env.GOOGLE_MAPS_API_KEY;
// The default Google budget, whatever the shell sets.
delete process.env.GEV_RATELIMIT_GOOGLE_PER_MIN;
const nativeFetch = globalThis.fetch;
const requests = [];
globalThis.fetch = async (url, init = {}) => {
@@ -139,6 +142,7 @@ function mount(t, respond) {
['CCTV_FORCE_AUSTIN', before.austin],
['GOOGLE_MAPS_SERVER_API_KEY', before.serverKey],
['GOOGLE_MAPS_API_KEY', before.browserKey],
['GEV_RATELIMIT_GOOGLE_PER_MIN', before.rateLimit],
]) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
@@ -155,9 +159,9 @@ function mount(t, respond) {
});
assert.ok(handler, 'the CCTV provider must mount a middleware');
const call = async (path, headers = {}) => {
const call = async (path, headers = {}, socket) => {
const { res, done } = recordingResponse();
await handler({ url: path, headers, method: 'GET', on() {} }, res);
await handler({ url: path, headers, method: 'GET', on() {}, socket }, res);
await done;
return res;
};
@@ -554,4 +558,44 @@ test('the Street View fallback frames catalog cameras only, from the catalog pos
assert.equal(steered.headers['X-CCTV-Source'], 'streetview');
const location = new URL(streetView()[0].url).searchParams.get('location');
assert.equal(location, `${STILL.lat},${STILL.lon}`);
assert.equal(new URL(streetView()[0].url).searchParams.get('heading'), '90');
});
test('the Street View fallback spends a per-IP Google budget, and only with a key', async (t) => {
const app = mount(t, ({ url }) =>
url.startsWith('https://maps.googleapis.com/')
? trackedUpstream({
headers: { 'content-type': 'image/jpeg' },
body: 'street-view',
}).response
: upstreamResponse({ status: 503 }),
);
// Read when the first keyed fallback builds the limiter, so after mount().
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN = '2';
const streetViewCalls = () =>
app.requests.filter(({ url }) =>
url.startsWith('https://maps.googleapis.com/'),
).length;
const first = { remoteAddress: '203.0.113.7' };
const frameSource = async (socket) =>
(await app.call(`/frame/${STILL.id}`, {}, socket)).headers['X-CCTV-Source'];
// Without a key nothing is spent.
for (let i = 0; i < 3; i++)
assert.equal(await frameSource(first), 'synthetic');
assert.equal(streetViewCalls(), 0);
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
assert.deepEqual(
[await frameSource(first), await frameSource(first), await frameSource(first)],
['streetview', 'streetview', 'synthetic'],
);
assert.equal(streetViewCalls(), 2, 'no Google call over the budget');
// Another address has a budget of its own.
assert.equal(
await frameSource({ remoteAddress: '203.0.113.8' }),
'streetview',
);
assert.equal(streetViewCalls(), 3);
});
+141
View File
@@ -0,0 +1,141 @@
// When a catalog camera has no pose, the browser sends /frame a heading derived
// from the camera ID, a pitch of -17°, and a field of view of 74°. Street View
// uses that pose at the camera's catalog coordinates.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { cctvProxy } from '../../server/providers/cctv.js';
// Same shape as a world-cameras row: an image URL, position, and no
// headingDeg / pitchDeg / fovDeg.
const POSELESS = {
id: 'world-or-fixture',
name: 'Pose-less fixture',
lat: 44.05,
lon: -123.09,
feedType: 'image',
url: 'https://cams.upstream.invalid/still.jpg',
};
/** The browser's id-hash heading (src/layers/cctv/model.js headingFromId). */
function browserHeadingFromId(id) {
let acc = 0;
for (let i = 0; i < id.length; i++) acc = (acc * 33 + id.charCodeAt(i)) >>> 0;
return ((((acc % 16) * 22.5) % 360) + 360) % 360;
}
function response({ status = 200, headers = {}, body = '' }) {
return new Response(status === 204 ? null : String(body), {
status,
headers,
});
}
function recording() {
const chunks = [];
let finished;
const done = new Promise((resolve) => (finished = resolve));
const res = {
statusCode: 0,
headers: {},
writableEnded: false,
writeHead(status, headers) {
res.statusCode = status;
res.headers = { ...res.headers, ...headers };
},
setHeader(name, value) {
res.headers[name] = value;
},
write(chunk) {
chunks.push(Buffer.from(chunk));
return true;
},
end(chunk) {
if (chunk) chunks.push(Buffer.from(chunk));
res.writableEnded = true;
res.body = Buffer.concat(chunks).toString('utf8');
finished();
},
on() {},
once() {},
emit() {},
destroy() {},
};
return { res, done };
}
test('the Street View fallback frames a pose-less catalog camera the way the browser draws it', async (t) => {
const saved = Object.fromEntries(
[
'CCTV_SOURCES_JSON',
'CCTV_SOURCES_FILE',
'CCTV_FORCE_AUSTIN',
'GOOGLE_MAPS_SERVER_API_KEY',
'GOOGLE_MAPS_API_KEY',
'GEV_RATELIMIT_GOOGLE_PER_MIN',
].map((name) => [name, process.env[name]]),
);
process.env.CCTV_SOURCES_JSON = JSON.stringify([POSELESS]);
process.env.CCTV_SOURCES_FILE = 'absent-source-file.json';
process.env.CCTV_FORCE_AUSTIN = '0';
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
delete process.env.GOOGLE_MAPS_API_KEY;
delete process.env.GEV_RATELIMIT_GOOGLE_PER_MIN;
const nativeFetch = globalThis.fetch;
const streetView = [];
globalThis.fetch = async (url) => {
const href = String(url);
if (href.startsWith('https://maps.googleapis.com/')) {
streetView.push(new URL(href));
return response({
headers: { 'content-type': 'image/jpeg' },
body: 'street-view',
});
}
return response({ status: 503 }); // the camera's upstream is down
};
t.after(() => {
globalThis.fetch = nativeFetch;
for (const [name, value] of Object.entries(saved)) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
}
});
let handler;
cctvProxy().configureServer({
middlewares: { use: (_route, fn) => (handler = fn) },
});
// What src/layers/cctv/source.js sends for this camera with no calibration.
const heading = Math.round(browserHeadingFromId(POSELESS.id));
const params = new URLSearchParams({
label: POSELESS.name,
city: '',
lat: POSELESS.lat.toFixed(6),
lon: POSELESS.lon.toFixed(6),
heading: String(heading),
fov: '74',
pitch: '-17',
ts: '1',
});
const { res, done } = recording();
await handler(
{
url: `/frame/${POSELESS.id}?${params}`,
headers: {},
method: 'GET',
on() {},
},
res,
);
await done;
assert.equal(res.headers['X-CCTV-Source'], 'streetview');
assert.equal(streetView.length, 1);
const sv = streetView[0].searchParams;
assert.equal(sv.get('location'), `${POSELESS.lat},${POSELESS.lon}`);
assert.equal(Number(sv.get('heading')), heading, 'heading the browser draws');
assert.equal(Number(sv.get('fov')), 74, 'FOV the browser draws');
assert.equal(Number(sv.get('pitch')), -17, 'pitch the browser draws');
});
-2
View File
@@ -22,8 +22,6 @@ function frameUrlFor(camera, refreshMs = ACTIVE_FRAME_REFRESH_MS) {
const params = new URLSearchParams({
label: camera.name,
city: camera.city,
lat: camera.lat.toFixed(6),
lon: camera.lon.toFixed(6),
heading: String(Math.round(camera.headingDeg)),
fov: String(Math.round(camera.fovDeg)),
pitch: String(Math.round(camera.pitchDeg || -10)),
+2 -2
View File
@@ -81,8 +81,8 @@ test('frame and media URLs preserve registered camera identity and encoded metad
);
assert.equal(frame.searchParams.get('label'), camera.name);
assert.equal(frame.searchParams.get('city'), camera.city);
assert.equal(frame.searchParams.get('lat'), '30.267000');
assert.equal(frame.searchParams.get('lon'), '-97.744000');
assert.equal(frame.searchParams.has('lat'), false);
assert.equal(frame.searchParams.has('lon'), false);
assert.equal(frame.searchParams.get('heading'), '45');
assert.equal(frame.searchParams.get('pitch'), '-12');
assert.deepEqual([...media.searchParams.keys()], ['ts']);