Merge pull request #1317 from bilawalsidhu/fix/provider-response-hardening

Harden provider response handling and camera source validation
This commit is contained in:
Sameh Khamis authored and GitHub committed 2026-10-21 16:18:50 -07:00
commit 7be5937fc3
16 files changed
+1010 -78

No files matched your search

+3 -1
View File
@@ -69,11 +69,13 @@ The data proxies under `server/providers/` are written so the browser cannot tur
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
- **Transit fetches registered feeds only.** `/api/transit/vehicles/<id>` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator.
- **Local receiver feeds are operator-configured and local-only.** `/api/local-receivers/aircraft` reads only the `aircraft.json` URLs in the server's `LOCAL_RECEIVER_FEEDS`; the browser never supplies an address. Each host must pass the shared tap address rule (`src/data/tapAddress.js`: loopback, RFC1918, `localhost`, `*.local`; no link-local, IPv6 or other names), the scheme must be http(s), the path must end in `aircraft.json`, and credentials, queries and fragments are refused. Invalid entries are logged at startup and never fetched. Reads use a 2 s timeout, refuse redirects, cap bodies at 2 MB and share one read per second; responses name each feed by band only and never carry upstream error text.
- **Validated response formats.** Proxied bodies are relayed only in the format the route serves. CCTV frames and the Street View fallback accept JPEG, PNG, GIF, WebP and AVIF only, sent under their canonical type; any other type falls through to the next fallback. Live CCTV media must match its feed kind (`video/mp4`, `video/webm`, or `multipart/x-mixed-replace` with a boundary) before it is relayed or the camera counts as healthy, and its redirects stay on the registered origin. GBFS bodies must parse as JSON and are re-serialized as `application/json; charset=utf-8`. Camera catalog URLs are pinned by parsed URL — scheme, exact host, no credentials, default port, and a normalized path inside the agency's image directory — and a redirect on a pinned host must stay inside the same directory. Every `/api/cctv` route uses the keyless cross-site checks described below, so cameras stay reachable behind a reverse proxy; a request carrying forwarding headers never reaches the key-bearing Street View fallback and gets the placeholder frame instead, as when no key is configured.
- **Consistent API response headers.** Every `/api` response, including errors, partial content, Host refusals and unknown routes, carries `Content-Security-Policy: default-src 'none'; sandbox` and `X-Content-Type-Options: nosniff` (`apiResponseHeadersPlugin` in `build/vite.js`, installed ahead of the Host check). A route that sets its own policy adds it as a further policy; it never replaces this one.
- **Response-size caps and timeouts** on proxied responses.
- **Sanitized errors** — internal error details are not echoed back to clients.
- **Coalesced OAuth refresh** and cached successful responses only (OpenSky).
- **Redacted debug logging.** The voice debug log (`.gev-logs/`, gitignored) strips API keys, bearer tokens, client secrets, and image data URLs before writing.
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`), quota-bearing (`/api/mapillary/*`, with a per-IP limit on tiles, and `/api/tomtom/*`, backed by its persistent daily tile budget) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control. The keyless terrain route (`/api/terrain/heights`) uses the same cross-site checks, minus the forwarding-header refusal: it guards only the per-client quota and point cache, not a key or a bill, and the map needs it to render behind a reverse proxy.
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`), quota-bearing (`/api/mapillary/*`, with a per-IP limit on tiles, and `/api/tomtom/*`, backed by its persistent daily tile budget) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control. The keyless terrain route (`/api/terrain/heights`) uses the same cross-site checks, minus the forwarding-header refusal: it guards only the per-client quota and point cache, not a key or a bill, and the map needs it to render behind a reverse proxy. The CCTV routes (`/api/cctv/*`) use the same keyless checks; their one key-bearing step, the Street View fallback, is skipped for any request with forwarding headers.
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads and the Maps JavaScript API that Street View runs on, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
Allowing `maps.googleapis.com` in `script-src` also trusts its JSONP endpoints, which can bypass the script allowlist if an HTML-injection bug exists; this is an accepted limitation of loading the Maps JavaScript API, alongside the existing `'unsafe-eval'` allowance.
+72 -1
View File
@@ -47,6 +47,75 @@ export const BROWSER_HEADERS = Object.freeze({
'X-Content-Type-Options': 'nosniff',
});
/**
* Content-Security-Policy for every `/api` response. API routes return data,
* never a document, so a response opened directly in a tab gets no origin and
* can load nothing.
*/
export const API_CSP = "default-src 'none'; sandbox";
/** Response headers every `/api` response carries. */
export const API_HEADERS = Object.freeze({
'Content-Security-Policy': API_CSP,
'X-Content-Type-Options': 'nosniff',
});
/**
* Whether a request reaches an `/api` mount, by the router's own rule: the
* prefix compared without case and followed by the end, `/` or `.`. The router
* reads an absolute-form target's path without resolving dot segments, so the
* path is taken here the same way (scheme and authority dropped, backslashes as
* slashes), alongside the raw and the fully parsed path; any match counts.
*/
function isApiPath(url) {
const raw = String(url || '/');
const target = raw
.trim()
.split(/[?#]/, 1)[0]
.replace(/\\/g, '/')
.replace(/^[a-z][a-z0-9+.-]*:(?:\/\/[^/]*)?/i, '');
let parsed = '';
try {
parsed = new URL(raw, 'http://localhost').pathname;
} catch {
/* the unparsed paths still decide */
}
return [raw.split(/[?#]/, 1)[0], target, parsed].some((path) =>
/^\/api(?:$|[/.])/i.test(path),
);
}
/**
* Vite plugin that sets API_HEADERS on every API response before any other
* middleware runs, so success, error, partial, refused and unknown-route
* answers all carry them. Vite's `server.headers` and `preview.headers` only
* reach the documents and assets Vite serves itself. A route may still add
* headers; a route's own Content-Security-Policy is sent as an additional
* policy, which can only restrict further, never replace this one.
*/
export function apiResponseHeadersPlugin() {
const install = (server) => {
server.middlewares.use((req, res, next) => {
if (!isApiPath(req.url)) return next();
for (const [name, value] of Object.entries(API_HEADERS))
res.setHeader(name, value);
const setHeader = res.setHeader.bind(res);
res.setHeader = (name, value) =>
String(name).toLowerCase() === 'content-security-policy' &&
String(value) !== API_CSP
? setHeader(name, `${API_CSP}, ${value}`)
: setHeader(name, value);
next();
});
};
return {
name: 'api-response-headers',
enforce: 'pre',
configureServer: install,
configurePreviewServer: install,
};
}
/** Build browser assets with explicit inputs; never load environment or providers. */
export function createBrowserViteConfig({
plugins = [],
@@ -61,7 +130,9 @@ export function createBrowserViteConfig({
} = {}) {
return {
plugins: [
// First, so no provider route answers a Host the server does not allow.
// First, so every API response, including a Host refusal, carries them.
apiResponseHeadersPlugin(),
// Next, so no provider route answers a Host the server does not allow.
hostCheckPlugin(),
cesium(),
applicationHtmlPlugin(),
@@ -0,0 +1 @@
- Validate provider response formats, strengthen camera source checks, and apply consistent API response protections (samehkhamis, #1317).
+19
View File
@@ -30,6 +30,16 @@ time. This clears obsolete contacts and settles startup without manufacturing
position timestamps. Stale, incomplete and malformed empty results retain the
existing recovery behavior.
Every `/api` response on the dev and preview servers carries
`Content-Security-Policy: default-src 'none'; sandbox` and
`X-Content-Type-Options: nosniff`. `apiResponseHeadersPlugin` sets them ahead of
the Host check and matches `/api` the way the router mounts it (any letter case, a
`/` or `.` boundary, absolute-form targets), so errors, partial responses, Host
refusals and unknown routes are covered too; a route's own policy is sent as an
additional policy, never in place of it. The app document's policy is unchanged.
The GBFS proxy relays only bodies that parse as JSON, re-serialized as
`application/json; charset=utf-8`; anything else answers 502.
Catalog source contracts and their consuming layers are declared in
`src/app/sourceComposition.js`. Omitted or null acquisition sources preserve layer
registration and serialization, report unavailable, and are refused by the existing
@@ -812,6 +822,15 @@ standalone entry supplies application-owned scene, ground and activation service
Each layer owns its state and visibility listener; destruction cancels source
reads and pending initialization. Malformed health responses retain prior health.
Existing catalog fallback, camera poses, frame pacing and coverage controls remain.
CCTV frames and the Street View fallback accept only JPEG, PNG, GIF, WebP and
AVIF, sent under the canonical type; any other type falls through to the next
fallback. Live media must match its feed kind (`video/mp4`, `video/webm`, or
`multipart/x-mixed-replace` with a boundary) before health reports it connected,
and follows redirects only on the camera's origin. Catalog image URLs, and
redirects on those hosts, are pinned by parsed URL to the agency's image
directory. Every `/api/cctv` route refuses cross-site browser requests but
accepts forwarding headers, so cameras work behind a reverse proxy; a forwarded
request skips the key-bearing Street View fallback and gets the placeholder.
Traffic owns its rendered-surface observer only while enabled, removes it on disable/destroy, and caches surface identity without allocating on unchanged frames.
+1
View File
@@ -121,6 +121,7 @@
"server/providers/cctv/normalize.js",
"src/data/cctvMissingNumbers.test.mjs",
"server/providers/cctv/sources.js",
"src/data/cctvSourcePins.test.mjs",
"server/providers/radio.js",
"server/providers/radio/catalog.js",
"server/providers/radio/constants.js",
+2
View File
@@ -194,6 +194,8 @@
"exports": ["./server/providers/cctv"],
"modules": [
"server/providers/cctv.js",
"server/providers/common/same-site.js",
"src/localRequestGate.mjs",
"server/providers/cctv/cap.js",
"server/providers/cctv/groundHeights.js",
"server/providers/cctv/catalog.js",
+63 -29
View File
@@ -6,6 +6,8 @@ import {
} from './cctv/normalize.js';
import {
buildSyntheticCctvSvg,
cctvLiveMediaType,
cctvRasterImageType,
proxyMediaResponse,
fetchCctvImageFromUpstream,
fetchTxdotSnapshot,
@@ -17,6 +19,8 @@ import {
CCTV_MAX_SOURCES_CEILING,
} from './cctv/constants.js';
import { sanitizeCctvRangeHeader } from './cctv/range.js';
import { admitSameSiteKeyless } from './common/same-site.js';
import { PROXY_SIGNALS } from '../../src/localRequestGate.mjs';
import { createHlsPuller } from './cctv/stream.js';
import { googleServerApiKey } from './places/google-key.js';
export { CCTV_FRAME_FETCH_TIMEOUT_MS, fetchCctvImageFromUpstream };
@@ -116,8 +120,15 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
headers: { 'User-Agent': 'gods-eye-view-cctv-proxy/1.0' },
signal: AbortSignal.timeout(CCTV_FRAME_FETCH_TIMEOUT_MS),
});
const svType = svResp.headers.get('content-type') || '';
if (!svResp.ok || !svType.startsWith('image/')) return null;
const svType = cctvRasterImageType(svResp.headers.get('content-type'));
if (!svResp.ok || !svType) {
try {
await svResp.body?.cancel();
} catch {
/* no-op */
}
return null;
}
return {
ok: true,
@@ -135,11 +146,15 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
});
server.middlewares.use('/api/cctv', async (req, res) => {
try {
const url = new URL(req.url || '/', 'http://localhost');
// Refuse cross-site requests before any catalog or upstream work. The
// keyless gate keeps the routes reachable behind a reverse proxy; the
// key-bearing Street View fallback below has its own check.
if (admitSameSiteKeyless(req, res)) return;
const sources = await getCctvSources();
const sourceById = new Map(
sources.map((source) => [source.id, source]),
);
const url = new URL(req.url || '/', 'http://localhost');
if (url.pathname === '/sources') {
const body = {
@@ -217,8 +232,8 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
const mediaUrl = source?.url || '';
const feedType = normalizeFeedType(source?.feedType || 'image');
// Live feeds only. A still camera is served by /frame, whose fetch
// keeps redirects on the registered origin, accepts only image/* and
// caps the body; relaying it here would bypass all three.
// keeps redirects on the registered origin, accepts only raster
// images and caps the body; relaying it here would bypass all three.
if (!isVideoFeedType(feedType) && feedType !== 'mjpeg') {
res.writeHead(404, {
'Content-Type': 'application/json',
@@ -382,7 +397,6 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
}
return;
}
const contentType = upstream.headers.get('content-type') || '';
if (!upstream.ok) {
try {
await upstream.body?.cancel();
@@ -407,31 +421,42 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
return;
}
if (
isVideoFeedType(feedType) &&
!(
contentType.startsWith('video/') ||
contentType.includes('mpegurl')
)
) {
// The relayed type is the feed kind's own, checked before the
// camera counts as healthy and before any byte is relayed.
const contentType = cctvLiveMediaType(
feedType,
upstream.headers.get('content-type'),
);
if (!contentType) {
try {
await upstream.body?.cancel();
} catch {
/* already closed */
}
setHealth(cameraId, {
status: 'degraded',
sourceKind: 'upstream',
label: source?.provider || 'Configured source',
message: `Unexpected media type ${contentType || 'unknown'}`,
message: 'Unexpected media type',
});
} else {
setHealth(cameraId, {
status: 'ok',
sourceKind: isVideoFeedType(feedType) ? 'live' : 'snapshot',
label: source?.provider || 'Configured source',
message: isVideoFeedType(feedType)
? 'Live stream connected'
: 'Snapshot feed connected',
res.writeHead(502, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify({ error: 'Media proxy failed' }));
return;
}
setHealth(cameraId, {
status: 'ok',
sourceKind: isVideoFeedType(feedType) ? 'live' : 'snapshot',
label: source?.provider || 'Configured source',
message: isVideoFeedType(feedType)
? 'Live stream connected'
: 'Snapshot feed connected',
});
await proxyMediaResponse(res, upstream, {
contentType,
sourceHeader: isVideoFeedType(feedType)
? 'live-media'
: 'upstream-image',
@@ -524,13 +549,22 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
return;
}
const sv = await streetViewFallback({
lat,
lon,
heading,
fov,
pitch,
});
// The Street View fallback spends the Google key, so a forwarded
// request (whose client this server cannot see) gets the placeholder,
// as if no key were configured. Any forwarding header counts, even an
// empty one.
const forwarded = PROXY_SIGNALS.some(
(name) => req.headers?.[name] !== undefined,
);
const sv = forwarded
? null
: await streetViewFallback({
lat,
lon,
heading,
fov,
pitch,
});
if (sv?.ok) {
setHealth(cameraId, {
status: 'degraded',
+130 -9
View File
@@ -1,13 +1,21 @@
import { Readable } from 'node:stream';
import { hashSeed, escapeXml } from './normalize.js';
import { hashSeed, escapeXml, pinnedCameraUrl } from './normalize.js';
import {
CALGARY_IMAGE_ORIGIN,
CCTV_FRAME_FETCH_TIMEOUT_MS,
CCTV_FRAME_MAX_BODY_BYTES,
CCTV_MEDIA_FETCH_TIMEOUT_MS,
CCTV_MEDIA_IDLE_TIMEOUT_MS,
CCTV_MEDIA_MAX_BODY_BYTES,
HK_IMAGE_ORIGIN,
KING_COUNTY_IMAGE_ORIGIN,
NSW_IMAGE_ORIGIN,
NSW_IMAGE_USER_AGENT,
TALLINN_IMAGE_ORIGIN,
TARKTEE_IMAGE_ORIGIN,
TFL_IMAGE_ORIGIN,
WARENDORF_IMAGE_ORIGINS,
WSDOT_IMAGE_ORIGIN,
} from './constants.js';
/**
* Generate a synthetic SVG billboard image for a CCTV camera placeholder.
@@ -93,6 +101,73 @@ export function toReadable(body) {
return null;
}
/**
* Split a Content-Type value into its lower-case essence and parameters.
*
* @param {string|null|undefined} value
* @returns {{essence:string,params:Map<string,string>}|null} Null when the
* value is missing or is not a `type/subtype` media type.
*/
export function parseMediaType(value) {
const [head, ...rest] = String(value ?? '').split(';');
const essence = head.trim().toLowerCase();
if (!/^[a-z0-9!#$&^_.+-]+\/[a-z0-9!#$&^_.+-]+$/.test(essence)) return null;
const params = new Map();
for (const part of rest) {
const eq = part.indexOf('=');
if (eq <= 0) continue;
const name = part.slice(0, eq).trim().toLowerCase();
let paramValue = part.slice(eq + 1).trim();
if (/^".*"$/.test(paramValue)) paramValue = paramValue.slice(1, -1);
if (name && !params.has(name)) params.set(name, paramValue);
}
return { essence, params };
}
/** Still-image types a camera frame may be served as. */
const CCTV_RASTER_IMAGE_TYPES = new Set([
'image/jpeg',
'image/png',
'image/gif',
'image/webp',
'image/avif',
]);
/**
* The canonical type of a raster camera frame, or null for any other type.
*
* @param {string|null|undefined} contentType
* @returns {string|null}
*/
export function cctvRasterImageType(contentType) {
const essence = parseMediaType(contentType)?.essence;
return CCTV_RASTER_IMAGE_TYPES.has(essence) ? essence : null;
}
/**
* The type to relay a live feed as, or null when the upstream type does not
* match the feed kind. MJPEG keeps its part boundary, without which the body
* cannot be split into frames.
*
* @param {string} feedType - Normalized feed type (mp4, webm or mjpeg).
* @param {string|null|undefined} contentType
* @returns {string|null}
*/
export function cctvLiveMediaType(feedType, contentType) {
const parsed = parseMediaType(contentType);
if (!parsed) return null;
if (feedType === 'mp4' || feedType === 'webm')
return parsed.essence === `video/${feedType}` ? parsed.essence : null;
if (feedType !== 'mjpeg' || parsed.essence !== 'multipart/x-mixed-replace')
return null;
const boundary = parsed.params.get('boundary') || '';
if (!/^[0-9A-Za-z'()+_,./:=? -]{0,69}[0-9A-Za-z'()+_,./:=?-]$/.test(boundary))
return null;
return /^[0-9A-Za-z'+_.-]+$/.test(boundary)
? `multipart/x-mixed-replace; boundary=${boundary}`
: `multipart/x-mixed-replace; boundary="${boundary}"`;
}
/**
* Pipe an upstream fetch Response (image or video) to the client HTTP response.
*
@@ -104,6 +179,8 @@ export function toReadable(body) {
* @param {Response} upstream - fetch() Response object.
* @param {object} [opts]
* @param {string} [opts.sourceHeader='upstream'] - Value for X-CCTV-Source header.
* @param {string} [opts.contentType] - Validated type to send instead of the
* upstream's own.
* @param {number} [opts.idleTimeoutMs=CCTV_MEDIA_IDLE_TIMEOUT_MS] - Silence
* allowed between upstream chunks before the stream is released. Injectable
* only to keep the deadline unit-testable.
@@ -114,10 +191,10 @@ export async function proxyMediaResponse(
{
sourceHeader = 'upstream',
idleTimeoutMs = CCTV_MEDIA_IDLE_TIMEOUT_MS,
contentType = upstream.headers.get('content-type') ||
'application/octet-stream',
} = {},
) {
const contentType =
upstream.headers.get('content-type') || 'application/octet-stream';
const cacheControl = upstream.headers.get('cache-control') || 'no-store';
const contentLength = upstream.headers.get('content-length');
const contentRange = upstream.headers.get('content-range');
@@ -340,7 +417,14 @@ export async function fetchCctvMediaUpstream(
if (downstream?.aborted) controller.abort();
else downstream?.addEventListener?.('abort', onDownstreamAbort);
try {
return await fetchImpl(url, { headers, signal: controller.signal });
// Redirects stay on the registered origin, as for frames.
const upstream = await fetchWithinHost(
url,
{ headers, signal: controller.signal },
fetchImpl,
);
if (!upstream) throw new Error('Media redirect refused');
return upstream;
} finally {
clearTimeout(timeoutId);
downstream?.removeEventListener?.('abort', onDownstreamAbort);
@@ -428,9 +512,40 @@ export async function fetchTxdotSnapshot(
/** Redirect hops the frame path will follow, and only within the same host. */
const MAX_SAME_HOST_REDIRECTS = 2;
/**
* Directories camera catalogs are pinned to, by host. A redirect on one of
* these hosts must stay inside the pinned directory too, so a same-origin
* redirect cannot leave it (another bucket on a shared storage host).
*/
const CCTV_PINNED_PREFIXES_BY_HOST = new Map();
for (const prefix of [
TFL_IMAGE_ORIGIN,
TALLINN_IMAGE_ORIGIN,
TARKTEE_IMAGE_ORIGIN,
...WARENDORF_IMAGE_ORIGINS,
NSW_IMAGE_ORIGIN,
CALGARY_IMAGE_ORIGIN,
HK_IMAGE_ORIGIN,
KING_COUNTY_IMAGE_ORIGIN,
WSDOT_IMAGE_ORIGIN,
]) {
const host = new URL(prefix).hostname;
CCTV_PINNED_PREFIXES_BY_HOST.set(host, [
...(CCTV_PINNED_PREFIXES_BY_HOST.get(host) || []),
prefix,
]);
}
/** Whether a redirect target stays inside its host's pinned directory. */
function withinPinnedPrefix(url) {
const prefixes = CCTV_PINNED_PREFIXES_BY_HOST.get(url.hostname);
return !prefixes || prefixes.some((p) => pinnedCameraUrl(url.href, p));
}
/**
* Fetch a registered frame URL following redirects ONLY within the original
* origin (scheme, host and port; at most MAX_SAME_HOST_REDIRECTS hops).
* origin (scheme, host and port; at most MAX_SAME_HOST_REDIRECTS hops) and,
* on a pinned camera host, within its pinned directory.
* Default redirect-following would let an upstream steer a host-pinned
* request, and its host-specific headers, to any origin, another port, or a
* plaintext downgrade.
@@ -471,7 +586,7 @@ export async function fetchWithinHost(url, init, fetchImpl = fetch) {
} catch {
return null;
}
if (next.origin !== origin) return null;
if (next.origin !== origin || !withinPinnedPrefix(next)) return null;
current = next;
}
return null;
@@ -543,9 +658,15 @@ export async function fetchCctvImageFromUpstream(
fetchImpl,
);
if (!upstream) return null;
const contentType = upstream.headers.get('content-type') || '';
if (!upstream.ok || !contentType.startsWith('image/')) {
controller.abort();
const contentType = cctvRasterImageType(
upstream.headers.get('content-type'),
);
if (!upstream.ok || !contentType) {
try {
await upstream.body?.cancel();
} catch {
/* no-op */
}
return null;
}
const body = await readCappedResponseBytes(upstream, maxBytes);
+35
View File
@@ -659,3 +659,38 @@ export function isLikelyTorontoCoordinate(lat, lon) {
lon <= -79.1
);
}
/**
* A camera URL under a pinned directory, by parsed URL rather than string
* prefix: the same scheme and exact host as `prefix`, no credentials, the
* default port, and a normalized path inside the prefix's directory. Dot
* segments (including encoded ones) and backslashes are resolved before the
* path is compared, and an encoded slash or backslash is refused.
*
* @param {string} raw - Camera URL from a catalog or redirect.
* @param {string} prefix - Pinned origin and directory, ending in `/`.
* @returns {string|null} The normalized URL, or null when it is not pinned.
*/
export function pinnedCameraUrl(raw, prefix) {
let url;
let pin;
try {
url = new URL(String(raw ?? '').trim());
pin = new URL(prefix);
} catch {
return null;
}
if (
url.protocol !== pin.protocol ||
url.hostname !== pin.hostname ||
url.username ||
url.password ||
url.port ||
!url.pathname.startsWith(
pin.pathname.endsWith('/') ? pin.pathname : `${pin.pathname}/`,
) ||
/%2f|%5c/i.test(url.pathname)
)
return null;
return url.href;
}
+40 -28
View File
@@ -149,6 +149,7 @@ import {
cameraDisplayCode,
rowArrayToObject,
prioritizeSources,
pinnedCameraUrl,
} from './normalize.js';
import { directionToHeading } from '../../../src/data/directionText.js';
import {
@@ -310,9 +311,12 @@ export async function loadCaltransSourcesFromOpenData() {
const lon = toFiniteNumber(loc.longitude);
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
const imageUrl = String(cctv.imageData?.static?.currentImageURL || '');
// Official-host pin (see JSDoc). Also drops records with no still image.
if (!imageUrl.startsWith('https://cwwp2.dot.ca.gov/')) continue;
const imageUrl = pinnedCameraUrl(
cctv.imageData?.static?.currentImageURL,
'https://cwwp2.dot.ca.gov/',
);
if (!imageUrl) continue;
const locationName = String(loc.locationName || '').trim();
// Leading token of locationName is the stable camera code ("TV102 -- I-580 : …").
@@ -417,8 +421,8 @@ export async function loadTflSourcesFromOpenData() {
const lat = toFiniteNumber(place?.lat);
const lon = toFiniteNumber(place?.lon);
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
const imageUrl = String(props.imageUrl || '');
if (!imageUrl.startsWith(TFL_IMAGE_ORIGIN)) continue; // official-bucket pin
const imageUrl = pinnedCameraUrl(props.imageUrl, TFL_IMAGE_ORIGIN);
if (!imageUrl) continue; // official-bucket pin
// "JamCams_00002.00865" → "tfl-00002.00865" (provider-stable id).
const rawId = String(place?.id || '').replace(/^JamCams_/, '');
@@ -1127,13 +1131,15 @@ export function loadTallinnSourcesFromCatalog({
// Rough Estonia/Tallinn metro sanity (allows nearby suburbs already in the pack).
if (lat < 59.2 || lat > 59.7 || lon < 24.3 || lon > 25.4) continue;
const imageUrl =
const imageUrl = pinnedCameraUrl(
typeof item.url === 'string'
? item.url.trim()
? item.url
: typeof item.snapshotUrl === 'string'
? item.snapshotUrl.trim()
: '';
if (!imageUrl.startsWith(TALLINN_IMAGE_ORIGIN)) continue;
? item.snapshotUrl
: '',
TALLINN_IMAGE_ORIGIN,
);
if (!imageUrl) continue;
const extractedHeading = toFiniteNumber(item.headingDeg, NaN);
const hasHeading = Number.isFinite(extractedHeading);
@@ -1235,8 +1241,8 @@ export function parseTarkteeDatexImages(xml) {
body,
);
if (!refMatch || !urlMatch) continue;
const url = urlMatch[1].trim();
if (!url.startsWith(TARKTEE_IMAGE_ORIGIN)) continue;
const url = pinnedCameraUrl(urlMatch[1], TARKTEE_IMAGE_ORIGIN);
if (!url) continue;
out.set(refMatch[1], url);
}
return out;
@@ -1422,14 +1428,16 @@ export function loadWarendorfSourcesFromCatalog({
for (const item of rows) {
if (!item || typeof item !== 'object') continue;
const id = typeof item.id === 'string' ? item.id.trim() : '';
const url =
const raw =
typeof item.url === 'string'
? item.url.trim()
? item.url
: typeof item.snapshotUrl === 'string'
? item.snapshotUrl.trim()
? item.snapshotUrl
: '';
if (!id || !WARENDORF_IMAGE_ORIGINS.some((o) => url.startsWith(o)))
continue;
const url = WARENDORF_IMAGE_ORIGINS.map((o) =>
pinnedCameraUrl(raw, o),
).find(Boolean);
if (!id || !url) continue;
const lat = typeof item.lat === 'number' ? item.lat : NaN;
const lon = typeof item.lon === 'number' ? item.lon : NaN;
if (!isPlausibleLatLon(lat, lon)) continue;
@@ -1483,8 +1491,8 @@ export function nswCameraToSource(feature) {
const lat = typeof coords?.[1] === 'number' ? coords[1] : NaN;
if (!isLikelyNswCoordinate(lat, lon)) return null;
const props = feature?.properties || {};
const url = String(props.href || '').trim();
if (!url.startsWith(NSW_IMAGE_ORIGIN)) return null;
const url = pinnedCameraUrl(props.href, NSW_IMAGE_ORIGIN);
if (!url) return null;
// "N-E" -> "NE" for the compass lookup.
const direction = String(props.direction || '')
.trim()
@@ -1578,8 +1586,7 @@ export function normalizeCalgaryImageUrl(raw) {
}
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return null;
parsed.protocol = 'https:';
const upgraded = parsed.toString();
return upgraded.startsWith(CALGARY_IMAGE_ORIGIN) ? upgraded : null;
return pinnedCameraUrl(parsed.toString(), CALGARY_IMAGE_ORIGIN);
}
/**
@@ -2085,8 +2092,7 @@ export function normalizeHkImageUrl(raw) {
}
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return null;
parsed.protocol = 'https:';
const upgraded = parsed.toString();
return upgraded.startsWith(HK_IMAGE_ORIGIN) ? upgraded : null;
return pinnedCameraUrl(parsed.toString(), HK_IMAGE_ORIGIN);
}
/**
@@ -2423,10 +2429,16 @@ export async function loadKingCountySourcesFromOpenData() {
// county image origin, so they are skipped.
if (owner === 'WSDOT') continue;
// The handler is served over HTTPS; the feed still writes http://.
const imageUrl = String(attrs.ImageURL || '')
.trim()
.replace(/^http:\/\/info\.kingcounty\.gov\//, KING_COUNTY_IMAGE_ORIGIN);
if (!imageUrl.startsWith(KING_COUNTY_IMAGE_ORIGIN)) continue;
const imageUrl = pinnedCameraUrl(
String(attrs.ImageURL || '')
.trim()
.replace(
/^http:\/\/info\.kingcounty\.gov\//,
KING_COUNTY_IMAGE_ORIGIN,
),
KING_COUNTY_IMAGE_ORIGIN,
);
if (!imageUrl) continue;
const [lon, lat] = Array.isArray(row?.geometry?.coordinates)
? row.geometry.coordinates
: [];
@@ -2586,8 +2598,8 @@ export async function loadWsdotSourcesFromOpenData() {
const attrs = row?.attributes;
const rawId = attrs?.CameraID;
if (!Number.isSafeInteger(rawId) || rawId <= 0) continue;
const imageUrl = String(attrs.ImageURL || '').trim();
if (!imageUrl.startsWith(WSDOT_IMAGE_ORIGIN)) continue;
const imageUrl = pinnedCameraUrl(attrs.ImageURL, WSDOT_IMAGE_ORIGIN);
if (!imageUrl) continue;
const x = row?.geometry?.x;
const y = row?.geometry?.y;
if (!Number.isFinite(x) || !Number.isFinite(y)) continue;
+15 -2
View File
@@ -236,13 +236,26 @@ export function gbfsProxy({ ratePerMinute = GBFS_RATE_PER_MIN } = {}) {
}
const upstream = await fetchGbfsUpstream(upstreamUrl.toString());
// Only JSON is relayed, re-serialized under a fixed type: neither the
// upstream's text nor its declared type reaches the browser.
let payload;
try {
payload = JSON.parse(upstream.body);
} catch {
res.writeHead(502, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify({ error: 'GBFS upstream returned non-JSON' }));
return;
}
res.writeHead(upstream.status, {
'Content-Type': upstream.contentType,
'Content-Type': 'application/json; charset=utf-8',
'Cache-Control': gbfsCacheControl(upstreamUrl.pathname),
'X-GBFS-Upstream': upstreamUrl.hostname,
'X-GBFS-Cache': 'MISS',
});
res.end(upstream.body);
res.end(JSON.stringify(payload));
} catch (error) {
if (error?.name === 'AbortError') {
res.writeHead(504, {
+263 -3
View File
@@ -33,6 +33,14 @@ const VIDEO = {
feedType: 'video',
url: 'https://cams.upstream.invalid/live.mp4',
};
const WEBM = {
id: 'webm-fixture',
name: 'WebM fixture',
lat: 30.27,
lon: -97.74,
feedType: 'webm',
url: 'https://cams.upstream.invalid/live.webm',
};
const MJPEG = {
id: 'mjpeg-fixture',
name: 'MJPEG fixture',
@@ -69,7 +77,10 @@ function recordingResponse() {
body: '',
writeHead(status, headers) {
res.statusCode = status;
res.headers = headers || {};
res.headers = { ...res.headers, ...headers };
},
setHeader(name, value) {
res.headers[name] = value;
},
write(chunk) {
chunks.push(Buffer.from(chunk));
@@ -105,6 +116,7 @@ function mount(t, respond) {
STILL,
STILL_ALIAS,
VIDEO,
WEBM,
MJPEG,
]);
process.env.CCTV_SOURCES_FILE = 'absent-source-file.json';
@@ -143,9 +155,9 @@ function mount(t, respond) {
});
assert.ok(handler, 'the CCTV provider must mount a middleware');
const call = async (path) => {
const call = async (path, headers = {}) => {
const { res, done } = recordingResponse();
await handler({ url: path, headers: {}, method: 'GET', on() {} }, res);
await handler({ url: path, headers, method: 'GET', on() {} }, res);
await done;
return res;
};
@@ -268,3 +280,251 @@ test('/frame does not follow a still camera off its origin', async (t) => {
[STILL.url],
);
});
/** An upstream Response whose body records whether it was cancelled. */
function trackedUpstream({ status = 200, headers = {}, body = 'payload' }) {
const state = { cancelled: false };
const response = new Response(
new ReadableStream(
{
pull(controller) {
controller.enqueue(new TextEncoder().encode(body));
controller.close();
},
cancel() {
state.cancelled = true;
},
},
{ highWaterMark: 0 },
),
{ status, headers },
);
return { response, state };
}
test('/media refuses an upstream type that does not match the feed kind', async (t) => {
let current;
const app = mount(t, () => current.response);
for (const [camera, contentType, body] of [
[VIDEO, 'text/html', '<!doctype html><p>x</p>'],
[VIDEO, 'image/svg+xml', '<svg xmlns="http://www.w3.org/2000/svg"/>'],
[VIDEO, 'application/vnd.apple.mpegurl', '#EXTM3U'],
[VIDEO, 'video/webm', 'video'],
[VIDEO, undefined, 'video'],
[WEBM, 'video/mp4', 'video'],
[MJPEG, 'multipart/x-mixed-replace', '--frame'],
[MJPEG, 'multipart/x-mixed-replace; boundary=', '--frame'],
[MJPEG, 'image/jpeg', 'jpeg'],
[MJPEG, 'text/html; boundary=frame', '<!doctype html><p>x</p>'],
]) {
current = trackedUpstream({
headers: contentType === undefined ? {} : { 'content-type': contentType },
body,
});
const label = `${camera.id}: ${contentType}`;
const res = await app.call(`/media/${camera.id}`);
assert.equal(res.statusCode, 502, label);
assert.equal(res.headers['Content-Type'], 'application/json', label);
assert.deepEqual(JSON.parse(res.body), { error: 'Media proxy failed' });
assert.equal(current.state.cancelled, true, `${label}: upstream cancelled`);
const health = await app.health(camera.id);
assert.equal(health.status, 'degraded', label);
assert.equal(health.message, 'Unexpected media type', label);
}
});
test('/media relays each feed kind under its canonical type, including a partial response', async (t) => {
let current;
const app = mount(t, () => current);
for (const [camera, upstream, expected] of [
[VIDEO, { 'content-type': 'Video/MP4; codecs="avc1.42E01E"' }, 'video/mp4'],
[
VIDEO,
{
'content-type': 'video/mp4',
'content-range': 'bytes 0-4/100',
'content-length': '5',
'accept-ranges': 'bytes',
},
'video/mp4',
],
[WEBM, { 'content-type': 'video/webm' }, 'video/webm'],
[
MJPEG,
{ 'content-type': 'multipart/x-mixed-replace;boundary="frame"' },
'multipart/x-mixed-replace; boundary=frame',
],
[
MJPEG,
{ 'content-type': 'multipart/x-mixed-replace; boundary="a b"' },
'multipart/x-mixed-replace; boundary="a b"',
],
]) {
const status = upstream['content-range'] ? 206 : 200;
current = upstreamResponse({ status, headers: upstream, body: 'bytes' });
const res = await app.call(`/media/${camera.id}`);
assert.equal(res.statusCode, status, camera.id);
assert.equal(res.headers['Content-Type'], expected);
assert.equal(res.body, 'bytes');
if (status === 206)
assert.equal(res.headers['Content-Range'], 'bytes 0-4/100');
assert.equal((await app.health(camera.id)).status, 'ok');
}
});
test('/media does not follow a live camera off its origin', async (t) => {
const app = mount(t, ({ url }) =>
url === VIDEO.url
? upstreamResponse({
status: 302,
headers: { location: 'https://elsewhere.invalid/live.mp4' },
body: '',
})
: upstreamResponse({
headers: { 'content-type': 'video/mp4' },
body: 'elsewhere',
}),
);
const res = await app.call(`/media/${VIDEO.id}`);
assert.equal(res.statusCode, 502);
assert.doesNotMatch(res.body, /elsewhere/);
assert.deepEqual(app.requests, [{ url: VIDEO.url, redirect: 'manual' }]);
});
test('every CCTV route refuses cross-site requests and admits the app itself', async (t) => {
const app = mount(t, ({ url }) =>
upstreamResponse({
headers: {
'content-type': url === VIDEO.url ? 'video/mp4' : 'image/jpeg',
},
body: 'bytes',
}),
);
const host = 'localhost:5173';
const paths = [
'/sources',
'/health',
`/stream/${STILL.id}`,
`/frame/${STILL.id}`,
`/media/${VIDEO.id}`,
];
for (const headers of [
{ host, 'sec-fetch-site': 'cross-site' },
{ host, 'sec-fetch-site': 'same-site' },
{ host, origin: 'https://elsewhere.example' },
{ host, origin: 'null' },
])
for (const path of paths) {
const res = await app.call(path, headers);
assert.equal(res.statusCode, 403, `${path} ${JSON.stringify(headers)}`);
}
assert.deepEqual(app.requests, [], 'a refused request reached an upstream');
for (const headers of [
{ host, 'sec-fetch-site': 'same-origin' },
{ host, origin: `http://${host}` },
{},
]) {
const frame = await app.call(`/frame/${STILL.id}`, headers);
assert.equal(frame.statusCode, 200);
assert.equal(frame.headers['X-CCTV-Source'], 'upstream-image');
const media = await app.call(`/media/${VIDEO.id}`, headers);
assert.equal(media.statusCode, 200);
assert.equal(media.headers['Content-Type'], 'video/mp4');
}
});
test('/frame falls back past a non-raster upstream and Street View frame', async (t) => {
let streetViewType;
const app = mount(t, ({ url }) =>
url.startsWith('https://maps.googleapis.com/')
? trackedUpstream({
headers:
streetViewType === undefined
? {}
: { 'content-type': streetViewType },
body: 'street-view',
}).response
: upstreamResponse({
headers: { 'content-type': 'image/svg+xml' },
body: '<svg xmlns="http://www.w3.org/2000/svg"/>',
}),
);
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
for (const type of [
'image/svg+xml',
'text/html',
'application/xml',
undefined,
]) {
streetViewType = type;
const res = await app.call(`/frame/${STILL.id}`);
assert.equal(res.statusCode, 200);
assert.equal(res.headers['X-CCTV-Source'], 'synthetic', String(type));
assert.doesNotMatch(res.body, /street-view/);
}
streetViewType = 'image/JPEG; charset=x';
const res = await app.call(`/frame/${STILL.id}`);
assert.equal(res.headers['X-CCTV-Source'], 'streetview');
assert.equal(res.headers['Content-Type'], 'image/jpeg');
assert.equal(res.body, 'street-view');
});
test('a forwarded same-origin request reaches every CCTV route but not the Street View fallback', async (t) => {
const app = mount(t, ({ url }) => {
if (url.startsWith('https://maps.googleapis.com/'))
return trackedUpstream({
headers: { 'content-type': 'image/jpeg' },
body: 'street-view',
}).response;
return url === VIDEO.url
? upstreamResponse({ headers: { 'content-type': 'video/mp4' } })
: upstreamResponse({ status: 503, body: '' });
});
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
const host = 'localhost:5173';
const streetViewCalls = () =>
app.requests.filter(({ url }) =>
url.startsWith('https://maps.googleapis.com/'),
).length;
for (const forwarded of [
{ 'x-forwarded-for': '203.0.113.9' },
{ via: '1.1 proxy' },
{ forwarded: 'for=203.0.113.9' },
{ 'x-real-ip': '203.0.113.9' },
{ 'x-forwarded-for': '' },
{ 'x-forwarded-for': ' ' },
{ 'cf-connecting-ip': '' },
]) {
const headers = { host, 'sec-fetch-site': 'same-origin', ...forwarded };
const label = JSON.stringify(forwarded);
assert.equal((await app.call('/sources', headers)).statusCode, 200, label);
const media = await app.call(`/media/${VIDEO.id}`, headers);
assert.equal(media.statusCode, 200, label);
assert.equal(media.headers['Content-Type'], 'video/mp4', label);
const frame = await app.call(`/frame/${STILL.id}`, headers);
assert.equal(frame.statusCode, 200, label);
assert.equal(frame.headers['X-CCTV-Source'], 'synthetic', label);
assert.equal(frame.headers['Content-Type'], 'image/svg+xml', label);
// Cross-site stays refused whatever the forwarding headers say.
for (const path of ['/sources', `/frame/${STILL.id}`, `/media/${VIDEO.id}`])
assert.equal(
(
await app.call(path, {
...forwarded,
host,
'sec-fetch-site': 'cross-site',
})
).statusCode,
403,
`${label} ${path}`,
);
}
assert.equal(streetViewCalls(), 0, 'a forwarded request spent the key');
const direct = await app.call(`/frame/${STILL.id}`, {
host,
'sec-fetch-site': 'same-origin',
});
assert.equal(direct.headers['X-CCTV-Source'], 'streetview');
assert.equal(direct.body, 'street-view');
assert.equal(streetViewCalls(), 1);
});
+54
View File
@@ -259,3 +259,57 @@ test('rejected snapshot responses abort the upstream download', async () => {
assert.equal(signal.aborted, true);
}
});
test('CCTV upstream frame fetch accepts only raster image types and sends their canonical type', async () => {
const frame = (contentType, body = 'frame') => {
const state = { cancelled: false };
const headers = contentType === null ? {} : { 'Content-Type': contentType };
const response = new Response(new ReadableStream({
pull(controller) {
controller.enqueue(new TextEncoder().encode(body));
controller.close();
},
cancel() {
state.cancelled = true;
},
}, { highWaterMark: 0 }), { status: 200, headers });
return { response, state };
};
for (const [contentType, canonical] of [
['image/jpeg', 'image/jpeg'],
['image/JPEG; charset=x', 'image/jpeg'],
[' Image/PNG ', 'image/png'],
['image/gif', 'image/gif'],
['image/webp;q=1', 'image/webp'],
['image/avif', 'image/avif'],
]) {
const { response } = frame(contentType);
const result = await fetchCctvImageFromUpstream('https://example.com/frame.jpg', {
fetchImpl: async () => response,
});
assert.equal(result?.contentType, canonical, contentType);
assert.equal(result.body.toString(), 'frame');
}
for (const [contentType, body] of [
['image/svg+xml', '<svg xmlns="http://www.w3.org/2000/svg"/>'],
['Image/SVG+XML; charset=utf-8', '<svg xmlns="http://www.w3.org/2000/svg"/>'],
['text/html', '<!doctype html><p>x</p>'],
['text/html; charset=utf-8', '<!doctype html><p>x</p>'],
['application/xml', '<x/>'],
['text/xml', '<x/>'],
[null, 'frame'],
['', 'frame'],
['image', 'frame'],
['image/', 'frame'],
['image/jpeg/x', 'frame'],
['image/jpeg, text/html', 'frame'],
['application/octet-stream', 'frame'],
]) {
const { response, state } = frame(contentType, body);
const result = await fetchCctvImageFromUpstream('https://example.com/frame.jpg', {
fetchImpl: async () => response,
});
assert.equal(result, null, String(contentType));
assert.equal(state.cancelled, true, `${contentType}: body cancelled`);
}
});
+114
View File
@@ -0,0 +1,114 @@
// Camera catalogs are pinned to their agency's image directory. The pin is a
// parsed-URL check, not a string prefix, so a path that only starts with the
// directory before dot segments, an encoded separator, credentials or another
// port are resolved never counts as inside it, and a redirect on a pinned host
// cannot leave the directory either.
//
// Run with: npm test (node --test)
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { loadTflSourcesFromOpenData } from '../../server/providers/cctv/sources.js';
import { fetchWithinHost } from '../../server/providers/cctv/media.js';
import { pinnedCameraUrl } from '../../server/providers/cctv/normalize.js';
const BUCKET = 'https://s3-eu-west-1.amazonaws.com/jamcams.tfl.gov.uk/';
/** One available JamCam whose frame is at `imageUrl`. */
const jamCam = (n, imageUrl) => ({
id: `JamCams_00001.0000${n}`,
commonName: `JamCam ${n}`,
lat: 51.5072,
lon: -0.1276,
additionalProperties: [
{ key: 'available', value: 'true' },
{ key: 'imageUrl', value: imageUrl },
],
});
test('TfL JamCams keep only frames inside the official bucket, by parsed URL', async (t) => {
t.mock.method(console, 'log', () => {});
t.mock.method(console, 'warn', () => {});
const accepted = [
[`${BUCKET}00001.00001.jpg`, `${BUCKET}00001.00001.jpg`],
[`${BUCKET}00001.00002.jpg?t=1`, `${BUCKET}00001.00002.jpg?t=1`],
[
'https://s3-eu-west-1.amazonaws.com:443/jamcams.tfl.gov.uk/00001.00003.jpg',
`${BUCKET}00001.00003.jpg`,
],
];
const rejected = [
`${BUCKET}../other/x.jpg`,
`${BUCKET}%2e%2e/other/x.jpg`,
`${BUCKET}%2E%2E/other/x.jpg`,
`${BUCKET}.%2e/other/x.jpg`,
`${BUCKET}..%2fother/x.jpg`,
`${BUCKET}..%5Cother/x.jpg`,
`${BUCKET}..\\other\\x.jpg`,
'https://s3-eu-west-1.amazonaws.com/jamcams.tfl.gov.uk.other/x.jpg',
'https://s3-eu-west-1.amazonaws.com/jamcams.tfl.gov.uk-other/x.jpg',
'https://s3-eu-west-1.amazonaws.com/other/jamcams.tfl.gov.uk/x.jpg',
'https://user:[email protected]/jamcams.tfl.gov.uk/x.jpg',
'https://s3-eu-west-1.amazonaws.com:8443/jamcams.tfl.gov.uk/x.jpg',
'http://s3-eu-west-1.amazonaws.com/jamcams.tfl.gov.uk/x.jpg',
'https://s3-eu-west-1.amazonaws.com.other.example/jamcams.tfl.gov.uk/x.jpg',
'not a url',
];
t.mock.method(globalThis, 'fetch', async () =>
Response.json(
[...accepted.map(([url]) => url), ...rejected].map((url, i) =>
jamCam(i + 1, url),
),
),
);
const cameras = await loadTflSourcesFromOpenData();
assert.deepEqual(
cameras.map((camera) => camera.url),
accepted.map(([, normalized]) => normalized),
);
});
test('a redirect on a pinned camera host stays inside the pinned directory', async () => {
const redirectTo = (location) => async (url) =>
url === `${BUCKET}00001.00001.jpg`
? new Response(null, { status: 302, headers: { location } })
: new Response('frame', { headers: { 'content-type': 'image/jpeg' } });
for (const location of [
'/jamcams.tfl.gov.uk.other/x.jpg',
'/other/x.jpg',
'/jamcams.tfl.gov.uk/../other/x.jpg',
'/jamcams.tfl.gov.uk/%2e%2e/other/x.jpg',
'/jamcams.tfl.gov.uk/..%2fother/x.jpg',
])
assert.equal(
await fetchWithinHost(
`${BUCKET}00001.00001.jpg`,
{},
redirectTo(location),
),
null,
location,
);
const followed = await fetchWithinHost(
`${BUCKET}00001.00001.jpg`,
{},
redirectTo('/jamcams.tfl.gov.uk/00001.00002.jpg'),
);
assert.equal(await followed.text(), 'frame');
// A host with no pinned directory keeps the same-origin rule alone.
const unpinned = await fetchWithinHost(
'https://cams.example/a/frame.jpg',
{},
async (url) =>
url === 'https://cams.example/a/frame.jpg'
? new Response(null, { status: 302, headers: { location: '/b/x.jpg' } })
: new Response('frame'),
);
assert.equal(await unpinned.text(), 'frame');
});
test('a pin prefix without a trailing slash still names a directory', () => {
assert.equal(
pinnedCameraUrl('https://h.example/camsX/a.jpg', 'https://h.example/cams'),
null,
);
});
+58
View File
@@ -304,3 +304,61 @@ test('GBFS rejects redirects and caps bodies through both server hooks', async (
globalThis.fetch = originalFetch;
}
});
test('GBFS relays only JSON, re-serialized under a fixed type, through both server hooks', async () => {
const { gbfsProxy } = await import('../../server/providers/gbfs.js');
const originalFetch = globalThis.fetch;
try {
for (const hook of ['configureServer', 'configurePreviewServer']) {
let handler;
gbfsProxy()[hook]({
middlewares: {
use(_path, callback) {
handler = callback;
},
},
});
const request = async (response) => {
globalThis.fetch = async () => response;
const res = {};
await handler(
{ method: 'GET', url: '/' + encodeURIComponent(STATION_URL) },
{
writeHead(code, headers) {
Object.assign(res, { status: code, headers });
},
end(value) {
res.body = value;
},
},
);
return res;
};
for (const [contentType, body] of [
['text/html', '<!doctype html><p>x</p>'],
['application/json', '<!doctype html><p>x</p>'],
['image/svg+xml', '<svg xmlns="http://www.w3.org/2000/svg"/>'],
['application/json', ''],
]) {
const res = await request(
new Response(body, { headers: { 'Content-Type': contentType } }),
);
assert.equal(res.status, 502, `${hook}: ${contentType} ${body}`);
assert.equal(res.headers['Content-Type'], 'application/json');
assert.equal(typeof JSON.parse(res.body).error, 'string');
assert.doesNotMatch(res.body, /doctype|svg/);
}
const res = await request(
new Response(' { "data" : { "stations" : [] } } ', {
headers: { 'Content-Type': 'text/html; charset=iso-8859-1' },
}),
);
assert.equal(res.status, 200);
assert.equal(res.headers['Content-Type'], 'application/json; charset=utf-8');
assert.equal(res.headers['Cache-Control'], 'no-store');
assert.equal(res.body, STATION_BODY);
}
} finally {
globalThis.fetch = originalFetch;
}
});
+140 -5
View File
@@ -1,9 +1,15 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { rm, writeFile } from 'node:fs/promises';
import { request } from 'node:http';
import path from 'node:path';
import { build, createServer, preview } from 'vite';
import { BROWSER_CSP, createBrowserViteConfig } from '../../build/vite.js';
import standaloneConfig, * as compatibility from '../../vite.config.js';
import * as providers from '../../server/providers/local.js';
import { apiNotFoundPlugin } from '../../server/standalone/api-not-found.js';
import { makeFixtureRoot } from './fixtureRoot.mjs';
test('explicit build inputs preserve browser-only defines, plugin order and loopback protections', () => {
const plugin = { name: 'fixture-provider' };
@@ -12,10 +18,13 @@ test('explicit build inputs preserve browser-only defines, plugin order and loop
googleApiKey: 'browser-fixture',
cesiumToken: 'ion-fixture',
});
// The Host check runs before every other plugin's middleware.
assert.equal(config.plugins[0].name, 'host-check');
// API response headers, then the Host check, run before every other
// plugin's middleware.
assert.equal(config.plugins[0].name, 'api-response-headers');
assert.equal(config.plugins[0].enforce, 'pre');
assert.equal(config.plugins[3], plugin);
assert.equal(config.plugins[1].name, 'host-check');
assert.equal(config.plugins[1].enforce, 'pre');
assert.equal(config.plugins[4], plugin);
assert.equal(config.server.host, 'localhost');
assert.equal(config.server.port, 4173);
assert.deepEqual(config.server.allowedHosts, ['localhost', '127.0.0.1']);
@@ -75,7 +84,7 @@ test('build helper does not discover environment values or construct local provi
undefined,
);
assert.deepEqual(
config.plugins.slice(3).map((plugin) => plugin.name),
config.plugins.slice(4).map((plugin) => plugin.name),
['embed-framing', 'panel-build'],
);
} finally {
@@ -89,7 +98,7 @@ test('root config retains existing named exports and standalone provider order',
assert.equal(compatibility[name], value, name);
const config = standaloneConfig({ mode: 'test' });
const standaloneProviders = config.plugins
.slice(3, -5)
.slice(4, -5)
.map((plugin) => plugin.name);
assert.equal(
standaloneProviders.filter((name) => name === 'gemini-live-proxy').length,
@@ -257,3 +266,129 @@ test('the CSP admits the event media embeds and the Maps JavaScript API, and no
// X posts load in X's own frames.
assert.ok(frames.includes('https://platform.twitter.com'));
});
/** A raw GET, so the Host header and an absolute-form target are sent as given. */
function rawGet(port, target, headers = {}) {
return new Promise((resolve, reject) => {
const req = request(
{
host: '127.0.0.1',
port,
path: target,
headers: { host: `localhost:${port}`, ...headers },
},
(res) => {
res.resume();
res.on('end', () =>
resolve({ status: res.statusCode, headers: res.headers }),
);
},
);
req.on('error', reject);
req.end();
});
}
test('every API response carries the API policy and nosniff on real dev and preview servers', async (t) => {
const policy = "default-src 'none'; sandbox";
const root = await makeFixtureRoot('gev-api-headers-');
t.after(() => rm(root, { recursive: true, force: true }));
await writeFile(
path.join(root, 'index.html'),
'<!doctype html><title>Fixture</title><p>x</p>',
);
const install = (server) => {
server.middlewares.use('/api/fixture', (req, res) => {
const route = new URL(req.url, 'http://localhost').pathname;
if (route === '/ok') {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end('{}');
} else if (route === '/error') {
res.statusCode = 500;
res.setHeader('Content-Type', 'application/json');
res.end('{"error":"fixture"}');
} else if (route === '/partial') {
res.writeHead(206, { 'Content-Range': 'bytes 0-0/2' });
res.end('x');
} else if (route === '/own-policy') {
res.setHeader('Content-Security-Policy', "frame-ancestors 'none'");
res.end('{}');
} else {
res.writeHead(200, { 'Content-Security-Policy': 'default-src *' });
res.end('{}');
}
});
};
const fixture = {
name: 'fixture-provider',
configureServer: install,
configurePreviewServer: install,
};
const built = createBrowserViteConfig({ plugins: [fixture] });
const base = {
root,
configFile: false,
envFile: false,
publicDir: false,
logLevel: 'silent',
};
await build(base);
for (const isPreview of [false, true]) {
const server = await (isPreview ? preview : createServer)({
...base,
plugins: [
...built.plugins.filter(
(plugin) =>
plugin === fixture ||
['api-response-headers', 'host-check'].includes(plugin.name),
),
apiNotFoundPlugin(),
],
server: { host: '127.0.0.1', port: 0, hmr: false },
preview: { host: '127.0.0.1', port: 0 },
});
if (!isPreview) await server.listen();
const { port } = server.httpServer.address();
const mode = isPreview ? 'preview' : 'dev';
try {
for (const [target, status, csp = policy, headers] of [
['/api/fixture/ok', 200],
['/api/fixture/error', 500],
['/api/fixture/partial', 206],
['/api/nope', 404],
['/api', 404],
['/api/fixture/ok?x=1', 200],
// Spellings the router mounts as the same routes.
['/API/fixture/ok', 200],
['/Api/fixture/error', 500],
['/aPi/nope', 404],
['/API', 404],
['/api.x', 404],
[`http://localhost:${port}/api/fixture/ok`, 200],
[`http://localhost:${port}/API/nope`, 404],
// The router keeps dot segments in an absolute-form target's path.
['http://elsewhere.example/api/../x', 404],
['http://elsewhere.example/API/%2e%2e/x', 404],
['/api/fixture/ok', 403, policy, { host: 'elsewhere.example' }],
// A route's own policy is added to the API policy, never replaces it.
['/api/fixture/own-policy', 200, `${policy}, frame-ancestors 'none'`],
['/api/fixture/own-policy-head', 200, `${policy}, default-src *`],
]) {
const res = await rawGet(port, target, headers);
const label = `${mode} ${target} ${JSON.stringify(headers || {})}`;
assert.equal(res.status, status, label);
assert.equal(res.headers['content-security-policy'], csp, label);
assert.equal(res.headers['x-content-type-options'], 'nosniff', label);
}
// The app document and other paths keep the server's own headers.
for (const target of ['/', '/apiary', '/src/api/x.js'])
assert.notEqual(
(await rawGet(port, target)).headers['content-security-policy'],
policy,
`${mode} ${target}`,
);
} finally {
await server.close();
}
}
});