fix(cctv): harden the Catalonia catalog fetch and frame-link allowlist

Address review feedback on #595:

- Reject frame links carrying userinfo (user:pass@) in
  normalizeCataloniaImageUrl, so catalog data cannot inject credentials
  into the server-side frame request on an allowlisted host.
- Fetch cameres.xml over https (verified: same 200 body, no redirect),
  refuse redirects with the unread body released, and read the body
  through readResponseTextCapped with a 1 MiB CATALONIA_MAX_CATALOG_BYTES
  cap instead of an unbounded resp.text().
- Add regression tests for userinfo links on every allowed host, the
  refused redirect, and declared and streamed oversize bodies.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

CHANGELOG.md

@@ -295,7 +295,9 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
   hotlinked Barcelona and Terrassa municipal cameras, each partner named as a
   credit beside the shared provider. Ambient-card refresh paced to 3 minutes,
   matching the empirically observed upstream cadence. `CCTV_CATALONIA_MAX_SOURCES`
-  sets the cap and `CCTV_CATALONIA_ENABLED=0` turns the pack off.
+  sets the cap and `CCTV_CATALONIA_ENABLED=0` turns the pack off. The catalog
+  is fetched over https with redirects refused and a 1 MiB body cap, and frame
+  links carrying userinfo are rejected even on an allowlisted host.
 
 - Distinguish PARTIAL vessel snapshots from STALE data in the layer panel, with
   accepted-record counts and unchanged retention, freshness and outage safeguards.

DATA_SOURCES.md

@@ -90,7 +90,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
 - **Warendorf webcam.** The Stadt Warendorf Marktplatz webcam, registered from the curated `config/cctv_sources.warendorf.json` (the Kreis Warendorf registration-office cameras from the contributing PR are indoor waiting rooms and were left out); only the municipal image host is proxied, frames are fetched at request time and never stored or redistributed. Disable with `CCTV_WARENDORF_ENABLED=0`.
 - **Live Traffic NSW cameras.** The camera list is the keyless public `data.livetraffic.com/cameras/traffic-cam.json` feed (Transport for NSW open data, CC BY 4.0); frames are stills on `webcams.transport.nsw.gov.au`, and only that host is registered. That image host answers non-browser clients with an HTML placeholder, so the proxy identifies as a browser for that one host and as itself everywhere else. Frames are fetched at request time and never stored or redistributed. Disable with `CCTV_NSW_ENABLED=0`.
 - **Open Calgary traffic cameras.** Keyless. The camera list is the public Socrata dataset `k7p9-kppz` on `data.calgary.ca`; frames are stills on the city's own `trafficcam.calgary.ca` host, and only that host is registered. Most rows publish an `http://` frame URL and the host redirects those to HTTPS, so URLs are upgraded and pinned to that origin before registration. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `quadrant` field and the quadrant suffix on each camera name are Calgary's address grid, not a bearing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. The licence requires the attribution statement above. Disable with `CCTV_CALGARY_ENABLED=0`.
-- **Catalonia (Servei Català de Trànsit) cameras.** The camera list is the keyless `gencat.cat/transit/opendata/cameres.xml` WFS/GML feed. It carries a `font` field naming the real publisher per camera; this pack keeps SCT's own highway cameras plus hotlinked Barcelona (IMI) and Terrassa municipal cameras (~135 total), each confirmed to return a real frame, so frame URLs are checked against a small host allowlist (`mct.gencat.cat`, `www.bcn.cat`, `emap.terrassa.cat`) rather than one pinned origin, and every non-SCT camera carries its real owner as a `credit` shown beside the shared `provider`. The feed carries no compass heading, so every camera loads at low heading confidence (same as TfL and Fintraffic). SCT's own feed link (`RenderService`) needed a workaround: it always 302s to a second endpoint (`TransitCamera`) on a Location the upstream hardcodes as plain http even when requested over https — refused by `fetchWithinHost`'s same-origin (scheme-included) redirect policy — and separately, `mct.gencat.cat`'s TLS setup fails the handshake with Node's fetch outright (`ERR_SSL_WRONG_SIGNATURE_TYPE`) even though other HTTP clients complete it. The loader therefore builds the `TransitCamera` URL itself, over plain http, from `RenderService`'s `sctidcam` id, rather than registering the (broken) upstream link — the only transport this one host actually serves to this runtime. Frames are fetched at request time and never stored or redistributed. Disable with `CCTV_CATALONIA_ENABLED=0`.
+- **Catalonia (Servei Català de Trànsit) cameras.** The camera list is the keyless `https://www.gencat.cat/transit/opendata/cameres.xml` WFS/GML feed, fetched over https with redirects refused and the body read capped at 1 MiB, since the list decides ids, coordinates and which allowlisted frame path is proxied. It carries a `font` field naming the real publisher per camera; this pack keeps SCT's own highway cameras plus hotlinked Barcelona (IMI) and Terrassa municipal cameras (~135 total), each confirmed to return a real frame, so frame URLs are checked against a small host allowlist (`mct.gencat.cat`, `www.bcn.cat`, `emap.terrassa.cat`) rather than one pinned origin (a link carrying `user:pass@` userinfo is rejected even on an allowed host), and every non-SCT camera carries its real owner as a `credit` shown beside the shared `provider`. The feed carries no compass heading, so every camera loads at low heading confidence (same as TfL and Fintraffic). SCT's own feed link (`RenderService`) needed a workaround: it always 302s to a second endpoint (`TransitCamera`) on a Location the upstream hardcodes as plain http even when requested over https — refused by `fetchWithinHost`'s same-origin (scheme-included) redirect policy — and separately, `mct.gencat.cat`'s TLS setup fails the handshake with Node's fetch outright (`ERR_SSL_WRONG_SIGNATURE_TYPE`) even though other HTTP clients complete it. The loader therefore builds the `TransitCamera` URL itself, over plain http, from `RenderService`'s `sctidcam` id, rather than registering the (broken) upstream link — the only transport this one host actually serves to this runtime. Frames are fetched at request time and never stored or redistributed. Disable with `CCTV_CATALONIA_ENABLED=0`.
 - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals.
 - **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All seven registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth.
 - **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply.

server/providers/cctv/constants.js

@@ -238,7 +238,8 @@ export const CALGARY_MAX_CATALOG_BYTES = 4 * 1024 * 1024;
 
 /**
  * Catalonia traffic cameras (Servei Català de Trànsit): one keyless WFS/GML
- * XML list (verified 2026-09-15, hourly catalog refresh). The feed carries a
+ * XML list (verified 2026-09-15, hourly catalog refresh), fetched over https
+ * (verified 2026-09-27: same 200 body, no redirect). The feed carries a
  * `font` field naming the actual publisher per camera — SCT's own highway
  * cameras plus hotlinked Barcelona (IMI) and Terrassa municipal cameras are
  * kept, each confirmed to return a real frame. Frame URLs are checked
@@ -246,7 +247,10 @@ export const CALGARY_MAX_CATALOG_BYTES = 4 * 1024 * 1024;
  * publisher serves from its own host.
  */
 export const CATALONIA_CAMERAS_URL =
-  'http://www.gencat.cat/transit/opendata/cameres.xml';
+  'https://www.gencat.cat/transit/opendata/cameres.xml';
+/** Byte ceiling for the Catalonia catalog read (~90 KB observed 2026-09-27),
+ * so a runaway body cannot be buffered without limit. */
+export const CATALONIA_MAX_CATALOG_BYTES = 1024 * 1024;
 /** Allowed image hosts, one per kept `font` publisher in the feed. */
 export const CATALONIA_IMAGE_HOSTS = Object.freeze(
   new Set([

server/providers/cctv/sources.js

@@ -56,6 +56,7 @@ import {
   CALGARY_DOWNTOWN,
   CALGARY_MAX_CATALOG_BYTES,
   CATALONIA_CAMERAS_URL,
+  CATALONIA_MAX_CATALOG_BYTES,
   CATALONIA_IMAGE_HOSTS,
   CATALONIA_FONT_CREDIT,
   CATALONIA_FONT_ELEVATION_M,
@@ -89,7 +90,10 @@ import {
   prioritizeSources,
 } from './normalize.js';
 import { directionToHeading } from '../../../src/data/directionText.js';
-import { readResponseJsonCapped } from '../common/http.js';
+import {
+  readResponseJsonCapped,
+  readResponseTextCapped,
+} from '../common/http.js';
 /**
  * Fetch and parse Austin traffic camera records from the city Open Data portal.
  *
@@ -1649,7 +1653,10 @@ export function parseCataloniaXml(xml) {
 /**
  * Validate and normalize one Catalonia feed `link` against the publisher
  * host allowlist (CATALONIA_IMAGE_HOSTS), upgrading to https. Rejects
- * anything else, including a host that merely resembles an allowed one.
+ * anything else, including a host that merely resembles an allowed one, and
+ * any URL carrying userinfo (`https://user:[email protected]/...`): the
+ * hostname check alone would pass it, letting catalog data inject
+ * credentials into the server-side frame request.
  *
  * SCT's own `RenderService` frame link is a special case, for two stacked
  * reasons (both verified 2026-09-15):
@@ -1679,6 +1686,7 @@ export function normalizeCataloniaImageUrl(rawUrl) {
     return '';
   }
   if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return '';
+  if (parsed.username || parsed.password) return '';
   if (!CATALONIA_IMAGE_HOSTS.has(parsed.hostname)) return '';
 
   if (parsed.hostname === 'mct.gencat.cat') {
@@ -1725,19 +1733,41 @@ export function cataloniaCameraName(carretera, municipi) {
  * shared `provider`. The feed carries no heading, so every camera takes the
  * id-hash fallback, low-confidence pose (same as TfL and Fintraffic).
  *
+ * The list decides ids, coordinates, labels and which allowlisted frame path
+ * is proxied, so it is fetched over https, redirects are refused (the list
+ * host cannot be steered or downgraded) and the body read is capped at
+ * CATALONIA_MAX_CATALOG_BYTES.
+ *
  * @returns {Promise<Array<object>>} Normalized camera source objects.
  */
 export async function loadCataloniaSourcesFromOpenData() {
   try {
     const resp = await fetch(CATALONIA_CAMERAS_URL, {
       headers: { Accept: 'application/xml,text/xml,*/*' },
+      redirect: 'manual',
       signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
     });
+    // A response this loader will not read still owns its transport until the
+    // body is released, so every rejection path cancels before returning.
+    const discard = async () => {
+      try {
+        await resp.body?.cancel();
+      } catch {
+        /* no-op */
+      }
+      return [];
+    };
+    if (resp.status >= 300 && resp.status < 400) {
+      console.warn(
+        '[CCTV] Catalonia catalog redirected; redirects are not followed',
+      );
+      return discard();
+    }
     if (!resp.ok) {
       console.warn('[CCTV] Catalonia camera download failed:', resp.status);
-      return [];
+      return discard();
     }
-    const xml = await resp.text();
+    const xml = await readResponseTextCapped(resp, CATALONIA_MAX_CATALOG_BYTES);
     const rows = parseCataloniaXml(xml);
     if (!rows.length) return [];
 

src/data/cctvCataloniaSource.test.mjs

@@ -4,7 +4,10 @@ import fs from 'node:fs';
 import os from 'node:os';
 import path from 'node:path';
 import { createCctvCatalog } from '../../server/providers/cctv/catalog.js';
-import { CATALONIA_CAMERAS_URL } from '../../server/providers/cctv/constants.js';
+import {
+  CATALONIA_CAMERAS_URL,
+  CATALONIA_MAX_CATALOG_BYTES,
+} from '../../server/providers/cctv/constants.js';
 import {
   parseCataloniaXml,
   normalizeCataloniaImageUrl,
@@ -31,6 +34,24 @@ function quiet(t) {
   t.mock.method(console, 'warn', () => {});
 }
 
+/**
+ * A response whose body is a live stream, plus a flag that flips when the
+ * stream is cancelled. A rejection path that returns without cancelling holds
+ * the transport open, so the flag is what the refusal tests actually assert.
+ */
+const streamingResponse = (init = {}) => {
+  const state = { cancelled: false };
+  const body = new ReadableStream({
+    start(controller) {
+      controller.enqueue(new TextEncoder().encode('<?xml'));
+    },
+    cancel() {
+      state.cancelled = true;
+    },
+  });
+  return { response: new Response(body, init), state };
+};
+
 /** One `<gml:featureMember>` block matching the real feed's shape. */
 function cataloniaFeature({
   fid,
@@ -143,6 +164,19 @@ test('normalizeCataloniaImageUrl accepts the other registered hosts, upgraded to
   assert.equal(normalizeCataloniaImageUrl('not a url'), '');
 });
 
+test('normalizeCataloniaImageUrl rejects credential-bearing URLs on every allowed host', () => {
+  // The hostname matches the allowlist, so only an explicit userinfo check
+  // stops catalog data injecting credentials into the frame request.
+  for (const link of [
+    'https://user:[email protected]/transit/imatges/a.gif',
+    'http://[email protected]/transit/imatges/a.gif',
+    'https://:[email protected]/it_terrassa/cam02.jpeg',
+    'http://user:[email protected]/mct2bo/RenderService?sctidcam=nc87.gif',
+  ]) {
+    assert.equal(normalizeCataloniaImageUrl(link), '', link);
+  }
+});
+
 test('cataloniaCameraName joins road and municipality, skipping a redundant municipality', () => {
   assert.equal(
     cataloniaCameraName('C-58', 'Nus Trinitat'),
@@ -164,8 +198,8 @@ test('Catalonia loader keeps SCT, Barcelona and Terrassa cameras with per-publis
   quiet(t);
   withEnv(t, { CCTV_CATALONIA_MAX_SOURCES: undefined });
   const requested = [];
-  t.mock.method(globalThis, 'fetch', async (url) => {
-    requested.push(String(url));
+  t.mock.method(globalThis, 'fetch', async (url, init) => {
+    requested.push([String(url), init.redirect]);
     const xml = cataloniaFeed([
       cataloniaFeature({
         fid: 'cameres.fid-1',
@@ -220,7 +254,11 @@ test('Catalonia loader keeps SCT, Barcelona and Terrassa cameras with per-publis
 
   const cameras = await loadCataloniaSourcesFromOpenData();
 
-  assert.deepEqual(requested, [CATALONIA_CAMERAS_URL]);
+  assert.deepEqual(requested, [[CATALONIA_CAMERAS_URL, 'manual']]);
+  assert.ok(
+    CATALONIA_CAMERAS_URL.startsWith('https://'),
+    'the catalog is fetched over https',
+  );
   assert.deepEqual(
     cameras.map((camera) => camera.id).sort(),
     ['cat-1', 'cat-2', 'cat-3'],
@@ -264,6 +302,81 @@ test('Catalonia loader fails soft on HTTP errors, empty payloads and network err
   }
 });
 
+test('the Catalonia catalog fetch refuses redirects and oversized bodies', async (t) => {
+  t.mock.method(console, 'warn', () => {});
+  // A redirect is never followed: the list host cannot be steered or
+  // downgraded. Its body is a live stream, so the test also proves the
+  // refusal releases the transport.
+  const seen = [];
+  const redirected = streamingResponse({
+    status: 302,
+    headers: { location: 'http://www.gencat.cat/transit/opendata/cameres.xml' },
+  });
+  t.mock.method(globalThis, 'fetch', async (url, init) => {
+    seen.push([String(url), init.redirect]);
+    return redirected.response;
+  });
+  assert.deepEqual(await loadCataloniaSourcesFromOpenData(), []);
+  assert.deepEqual(seen, [[CATALONIA_CAMERAS_URL, 'manual']]);
+  assert.equal(
+    redirected.state.cancelled,
+    true,
+    'the redirect body is cancelled',
+  );
+
+  // An HTTP error body is released too.
+  t.mock.restoreAll();
+  t.mock.method(console, 'warn', () => {});
+  const failed = streamingResponse({ status: 503 });
+  t.mock.method(globalThis, 'fetch', async () => failed.response);
+  assert.deepEqual(await loadCataloniaSourcesFromOpenData(), []);
+  assert.equal(failed.state.cancelled, true, 'the error body is cancelled');
+
+  const validFeed = cataloniaFeed([
+    cataloniaFeature({
+      fid: 'cameres.fid-1',
+      lon: '2.1849528',
+      lat: '41.45989301',
+      carretera: 'C-58',
+      link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=nc87.gif',
+      font: 'SCT',
+    }),
+  ]);
+
+  // A body that declares a size over the cap is refused rather than buffered,
+  // even though its actual content would parse.
+  t.mock.restoreAll();
+  t.mock.method(console, 'warn', () => {});
+  t.mock.method(globalThis, 'fetch', async () => {
+    return new Response(validFeed, {
+      headers: {
+        'Content-Type': 'application/xml',
+        'content-length': String(CATALONIA_MAX_CATALOG_BYTES + 1),
+      },
+    });
+  });
+  assert.deepEqual(await loadCataloniaSourcesFromOpenData(), []);
+
+  // So is a body with no declared size that only turns out too long while
+  // streaming — a valid feed padded past the cap.
+  t.mock.restoreAll();
+  t.mock.method(console, 'warn', () => {});
+  t.mock.method(globalThis, 'fetch', async () => {
+    const oversized =
+      validFeed + ' '.repeat(CATALONIA_MAX_CATALOG_BYTES + 1024);
+    return new Response(
+      new ReadableStream({
+        start(controller) {
+          controller.enqueue(new TextEncoder().encode(oversized));
+          controller.close();
+        },
+      }),
+      { headers: { 'Content-Type': 'application/xml' } },
+    );
+  });
+  assert.deepEqual(await loadCataloniaSourcesFromOpenData(), []);
+});
+
 test('CCTV catalog merges Catalonia cameras and CCTV_CATALONIA_ENABLED=0 skips the request', async (t) => {
   quiet(t);
   const sourceRoot = fs.mkdtempSync(
@@ -282,8 +395,8 @@ test('CCTV catalog merges Catalonia cameras and CCTV_CATALONIA_ENABLED=0 skips t
     CCTV_CATALONIA_MAX_SOURCES: undefined,
   });
   const requested = [];
-  t.mock.method(globalThis, 'fetch', async (url) => {
-    requested.push(String(url));
+  t.mock.method(globalThis, 'fetch', async (url, init) => {
+    requested.push([String(url), init.redirect]);
     if (String(url) !== CATALONIA_CAMERAS_URL) {
       return new Response('unavailable', { status: 503 });
     }