fix(cctv): give Catalonia cameras stable ids and pin their frame paths

Ids were built from the feed's fid, which embeds an upstream server-start stamp, so every upstream restart renamed all cameras (it already had since the PR fixture). Ids now come from the validated frame URL (cat-sct-<stem>, or host and path for partner cameras), which also collapses the five duplicate rows. Frame links must sit directly in each host's frame directory with no explicit port, and sctidcam must be a bare image filename. Numeric entity decoding handles hex references and leaves invalid code points as written, so one bad name no longer empties the pack. Counts in the docs are now ~130.
2026-11-04 02:14:01 +00:00 · 2026-10-15 15:21:34 -07:00
1 parent e8b6ffd196
commit 3f239f1173
@@ -220,7 +220,7 @@ AISSTREAM_API_KEY=
 # CCTV_CALGARY_MAX_SOURCES=220                         # cap cameras loaded from the Open Calgary catalog
 # CCTV_CALGARY_ROWS_URL=                               # override the Open Calgary catalog URL
 # CCTV_CATALONIA_ENABLED=1                             # set 0 to disable the Servei Català de Trànsit cameras
-# CCTV_CATALONIA_MAX_SOURCES=150                       # cap cameras loaded from Catalonia (~135 exist today)
+# CCTV_CATALONIA_MAX_SOURCES=150                       # cap cameras loaded from Catalonia (~130 exist today)
 # CCTV_VEGVESEN_ENABLED=1                              # set 0 to disable the Statens vegvesen (Norway) road cameras
 # CCTV_VEGVESEN_MAX_SOURCES=900                        # cap cameras loaded from Statens vegvesen (default loads all ~850 live; max 1000)
 # CCTV_VEGVESEN_VIDEO=1                                # set 0 to show stills instead of live HLS for Statens vegvesen video cameras
@@ -351,14 +351,17 @@
  largest cities) and `CCTV_VEGVESEN_VIDEO=0` keeps stills only. Attributed
  under NLOD 2.0.

- Add Catalonia (Servei Català de Trànsit) as a keyless CCTV source pack: ~135
+- Add Catalonia (Servei Català de Trànsit) as a keyless CCTV source pack: ~130
  cameras from a WFS/GML feed that aggregates SCT's own highway cameras with
  hotlinked Barcelona and Terrassa municipal cameras, each partner named as a
  credit beside the shared provider. Ambient-card refresh paced to 3 minutes,
  matching the empirically observed upstream cadence. `CCTV_CATALONIA_MAX_SOURCES`
  sets the cap and `CCTV_CATALONIA_ENABLED=0` turns the pack off. The catalog
  is fetched over https with redirects refused and a 1 MiB body cap, and frame
-  links carrying userinfo are rejected even on an allowlisted host.
+  links carrying userinfo or an explicit port, or outside the publisher's frame
+  directory, are rejected even on an allowlisted host. Camera ids come from
+  each frame's SCT camera file or image path rather than the feed's
+  restart-dependent feature id, and rows the feed repeats collapse to one.

 - Import your own GeoJSON, KML, KMZ and GPX files from DISPLAY or by dropping
  them onto the app. Show, hide, fly to or remove each file for the session.
@@ -143,7 +143,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
 - **Warendorf webcam.** The Stadt Warendorf Marktplatz webcam, registered from the curated `config/cctv_sources.warendorf.json` (the Kreis Warendorf registration-office cameras from the contributing PR are indoor waiting rooms and were left out); only the municipal image host is proxied, frames are fetched at request time and never stored or redistributed. Disable with `CCTV_WARENDORF_ENABLED=0`.
 - **Live Traffic NSW cameras.** The camera list is the keyless public `data.livetraffic.com/cameras/traffic-cam.json` feed (Transport for NSW open data, CC BY 4.0); frames are stills on `webcams.transport.nsw.gov.au`, and only that host is registered. That image host answers non-browser clients with an HTML placeholder, so the proxy identifies as a browser for that one host and as itself everywhere else. Frames are fetched at request time and never stored or redistributed. Disable with `CCTV_NSW_ENABLED=0`.
 - **Open Calgary traffic cameras.** Keyless. The camera list is the public Socrata dataset `k7p9-kppz` on `data.calgary.ca`; frames are stills on the city's own `trafficcam.calgary.ca` host, and only that host is registered. Most rows publish an `http://` frame URL and the host redirects those to HTTPS, so URLs are upgraded and pinned to that origin before registration. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `quadrant` field and the quadrant suffix on each camera name are Calgary's address grid, not a bearing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. The licence requires the attribution statement above. Disable with `CCTV_CALGARY_ENABLED=0`.
- **Catalonia (Servei Català de Trànsit) cameras.** The camera list is the keyless `https://www.gencat.cat/transit/opendata/cameres.xml` WFS/GML feed, fetched over https with redirects refused and the body read capped at 1 MiB, since the list decides ids, coordinates and which allowlisted frame path is proxied. It carries a `font` field naming the real publisher per camera; this pack keeps SCT's own highway cameras plus hotlinked Barcelona (IMI) and Terrassa municipal cameras (~135 total), each confirmed to return a real frame, so frame URLs are checked against a small host allowlist (`mct.gencat.cat`, `www.bcn.cat`, `emap.terrassa.cat`) rather than one pinned origin (a link carrying `user:pass@` userinfo is rejected even on an allowed host), and every non-SCT camera carries its real owner as a `credit` shown beside the shared `provider`. The feed carries no compass heading, so every camera loads at low heading confidence (same as TfL and Fintraffic). SCT's own feed link (`RenderService`) needed a workaround: it always 302s to a second endpoint (`TransitCamera`) on a Location the upstream hardcodes as plain http even when requested over https — refused by `fetchWithinHost`'s same-origin (scheme-included) redirect policy — and separately, `mct.gencat.cat`'s TLS setup fails the handshake with Node's fetch outright (`ERR_SSL_WRONG_SIGNATURE_TYPE`) even though other HTTP clients complete it. The loader therefore builds the `TransitCamera` URL itself, over plain http, from `RenderService`'s `sctidcam` id, rather than registering the (broken) upstream link — the only transport this one host actually serves to this runtime. The feed carries no camera status field, so a camera its publisher has taken offline still registers and shows the publisher's own "image unavailable" still (SCT and Barcelona both serve one with HTTP 200), the same as other packs without a status field. Frames are fetched at request time and never stored or redistributed. Disable with `CCTV_CATALONIA_ENABLED=0`.
+- **Catalonia (Servei Català de Trànsit) cameras.** The camera list is the keyless `https://www.gencat.cat/transit/opendata/cameres.xml` WFS/GML feed, fetched over https with redirects refused and the body read capped at 1 MiB, since the list decides ids, coordinates and which allowlisted frame path is proxied. It carries a `font` field naming the real publisher per camera; this pack keeps SCT's own highway cameras plus hotlinked Barcelona (IMI) and Terrassa municipal cameras (~130 total), each confirmed to return a real frame, so frame URLs are checked against a small host allowlist (`mct.gencat.cat`, `www.bcn.cat`, `emap.terrassa.cat`) rather than one pinned origin (a link carrying `user:pass@` userinfo or an explicit port, or pointing outside that host's frame directory, is rejected even on an allowed host; an SCT `sctidcam` must be a bare `.gif`/`.jpg` filename), and every non-SCT camera carries its real owner as a `credit` shown beside the shared `provider`. The feed carries no compass heading, so every camera loads at low heading confidence (same as TfL and Fintraffic). SCT's own feed link (`RenderService`) needed a workaround: it always 302s to a second endpoint (`TransitCamera`) on a Location the upstream hardcodes as plain http even when requested over https — refused by `fetchWithinHost`'s same-origin (scheme-included) redirect policy — and separately, `mct.gencat.cat`'s TLS setup fails the handshake with Node's fetch outright (`ERR_SSL_WRONG_SIGNATURE_TYPE`) even though other HTTP clients complete it. The loader therefore builds the `TransitCamera` URL itself, over plain http, from `RenderService`'s `sctidcam` id, rather than registering the (broken) upstream link — the only transport this one host actually serves to this runtime. Camera ids are built from the SCT camera file or the partner image path, not the feed's `fid` (which embeds an upstream server-start stamp and changes on every upstream restart), and rows the feed lists more than once collapse to one camera. The feed carries no camera status field, so a camera its publisher has taken offline still registers and shows the publisher's own "image unavailable" still (SCT and Barcelona both serve one with HTTP 200), the same as other packs without a status field. Frames are fetched at request time and never stored or redistributed. The feed itself states no licence; the pack relies on the Generalitat de Catalunya open-data portal's terms for this dataset (the licence and attribution above). Disable with `CCTV_CATALONIA_ENABLED=0`.
 - **Statens vegvesen road cameras.** Keyless. The camera list is one GeoJSON request to the OGC API Features view of the official DATEX 3.1 CCTV site table (`ogckart-sn1.atlas.vegvesen.no/ogc/features/v1/collections/datex_3_1:CctvSimple/items`), so no DATEX account is needed; frames are stills on `kamera.atlas.vegvesen.no/api/images/<cameraId>`, and only that exact path per camera is registered. Cameras the feed reports as faulty are dropped. About 135 working cameras also publish live HLS (`kamera.vegvesen.no/public/<cameraId>/manifest.m3u8`, MPEG-TS); those play as live video through the bounded in-memory HLS relay, with the still as the snapshot fallback, and only that exact manifest path per camera is registered. `CCTV_VEGVESEN_VIDEO=0` keeps every camera on stills. `orientationDescription` names the place a camera faces, not a bearing, so it goes into the label and headings use the id-hash fallback at low confidence. By default every live camera loads (~850); `CCTV_VEGVESEN_MAX_SOURCES` (8–1000) lowers the cap, keeping the cameras nearest Oslo, Bergen, Stavanger, Trondheim, Kristiansand, Bodø and Tromsø and `CCTV_VEGVESEN_ENABLED=0` turns the pack off. Frames are fetched at request time and never stored or redistributed. The NLOD attribution is registered in the Data attribution popover.
 - **NYC DOT Traffic Management Center cameras.** Off by default. NYC DOT's [developer data feeds page](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) asks developers who want the camera feed to contact `[email protected]` and sign a data-sharing agreement. This project does not hold one that covers downstream installations, so the pack loads only when an operator who holds their own agreement sets `CCTV_NYC_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to NYC DOT. The endpoint needs no key. The camera list is the public `webcams.nyctmc.org/api/cameras` catalog (~980 cameras across the five boroughs); frames are stills at `webcams.nyctmc.org/api/cameras/<uuid>/image`, synthesized from a validated UUID rather than taken from the payload, and only that host is registered. Cameras the feed reports as offline are dropped. The catalog publishes no camera facing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. Once enabled, the 250 cameras nearest the borough cores load; `CCTV_NYC_MAX_SOURCES` (8–400) changes the cap. Frames are fetched at request time and never stored or redistributed. NYC DOT asks every application that uses its data to carry its disclaimer; it is registered verbatim in the Data attribution popover.
 - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals.
@@ -377,7 +377,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi
 | 🌍 **Earthquakes**          | Global seismic activity, last 24h                                                                                                                                                                                                                                                                                                                                                   | USGS                                    | 🟢                                                                                                  |
 | ◍ **Air quality**           | Live AQHI on the published 1–10+ health-risk scale, with a Weather-panel card showing the band legend and the highest readings in view. **Canada only** — ECCC is the first registered provider | ECCC MSC GeoMet                          | 🟢                                                                                                  |
 | 🚗 **Traffic**              | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations                                                                                                                                                                                                       | TomTom + OSM                            | 🟢 simulation · 🟡 live flow speeds                                                                 |
-| 📹 **CCTV Mesh**            | ~3,900 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Catalonia (Servei Català de Trànsit), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs                               | 🟢                                                                                                  |
+| 📹 **CCTV Mesh**            | Up to 4,000 public cameras (the shared catalog cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Catalonia (Servei Català de Trànsit), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs                               | 🟢                                                                                                  |
 | 📷 **Mapped ALPR Cameras**  | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video                                                                                                                                                                                           | OpenStreetMap (incl. DeFlock mapping)   | 🟢                                                                                                  |
 | 📷 **Street Level**         | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer and **FOLLOW**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key |
 | 📻 **Radio**                | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster                                                                                                                                                                                                                                                 | Radio Browser / broadcasters            | 🟢                                                                                                  |
@@ -251,14 +251,19 @@ export const CATALONIA_CAMERAS_URL =
 /** Byte ceiling for the Catalonia catalog read (~90 KB observed 2026-09-27),
 * so a runaway body cannot be buffered without limit. */
 export const CATALONIA_MAX_CATALOG_BYTES = 1024 * 1024;
-/** Allowed image hosts, one per kept `font` publisher in the feed. */
+/** Allowed image hosts, one per kept `font` publisher in the feed, each
+ * mapped to the one directory its frame links live in (every link in the
+ * 2026-10 feed shares it); a link must name a single file directly inside. */
 export const CATALONIA_IMAGE_HOSTS = Object.freeze(
-  new Set([
-    'mct.gencat.cat', // SCT
-    'www.bcn.cat', // IMI (Ajuntament de Barcelona)
-    'emap.terrassa.cat', // Ajuntament de Terrassa
+  new Map([
+    ['mct.gencat.cat', '/mct2bo/'], // SCT
+    ['www.bcn.cat', '/transit/imatges/'], // IMI (Ajuntament de Barcelona)
+    ['emap.terrassa.cat', '/it_terrassa/'], // Ajuntament de Terrassa
  ]),
 );
+/** SCT `sctidcam` values: a bare image filename, which also keys the camera
+ * id (the feed's `fid` embeds an upstream server-start stamp). */
+export const CATALONIA_SCT_CAMERA_FILE_RE = /^[a-z0-9]+\.(gif|jpe?g)$/i;
 /** Per-`font` partner credit, shown beside the provider in the CCTV panel
 * (same pattern as DriveBC's partner-supplied cameras). SCT is the feed's
 * own publisher and carries no separate credit. */
@@ -406,16 +406,24 @@ export function isLikelyCataloniaCoordinate(lat, lon) {
 }

 /**
- * Decode decimal numeric HTML character references ("&#237;" -> "í"), the
- * entity form the Catalonia feed uses for accented place names, plus the
- * handful of named entities XML/HTML text commonly carries.
+ * Decode numeric HTML character references, decimal ("&#237;") and hex
+ * ("&#xED;") alike, the entity form the Catalonia feed uses for accented
+ * place names, plus the handful of named entities XML/HTML text commonly
+ * carries. A reference that names no valid character (NUL, a surrogate, or
+ * anything past U+10FFFF) is left as written rather than thrown on, so one
+ * bad field cannot fail a whole catalog.
 *
 * @param {string} text
 * @returns {string}
 */
 export function decodeNumericEntities(text) {
  return String(text || '')
-    .replace(/&#(\d+);/g, (_, code) => String.fromCodePoint(Number(code)))
+    .replace(/&#(?:x([0-9a-f]+)|(\d+));/gi, (match, hex, dec) => {
+      const code = hex ? Number.parseInt(hex, 16) : Number(dec);
+      const valid =
+        code > 0 && code <= 0x10ffff && (code < 0xd800 || code > 0xdfff);
+      return valid ? String.fromCodePoint(code) : match;
+    })
    .replace(/&amp;/g, '&')
    .replace(/&quot;/g, '"')
    .replace(/&#39;/g, "'")
@@ -58,6 +58,7 @@ import {
  CATALONIA_CAMERAS_URL,
  CATALONIA_MAX_CATALOG_BYTES,
  CATALONIA_IMAGE_HOSTS,
+  CATALONIA_SCT_CAMERA_FILE_RE,
  CATALONIA_FONT_CREDIT,
  CATALONIA_FONT_ELEVATION_M,
  DEFAULT_CATALONIA_MAX_SOURCES,
@@ -1668,10 +1669,11 @@ export function parseCataloniaXml(xml) {
 /**
 * Validate and normalize one Catalonia feed `link` against the publisher
 * host allowlist (CATALONIA_IMAGE_HOSTS), upgrading to https. Rejects
- * anything else, including a host that merely resembles an allowed one, and
- * any URL carrying userinfo (`https://user:[email protected]/...`): the
- * hostname check alone would pass it, letting catalog data inject
- * credentials into the server-side frame request.
+ * anything else, including a host that merely resembles an allowed one, any
+ * URL carrying userinfo (`https://user:[email protected]/...`) or an explicit
+ * port — the hostname check alone would pass either, letting catalog data
+ * inject credentials or steer the server-side frame request to another
+ * service on that host — and any path outside the host's frame directory.
 *
 * SCT's own `RenderService` frame link is a special case, for two stacked
 * reasons (both verified 2026-09-15):
@@ -1688,7 +1690,8 @@ export function parseCataloniaXml(xml) {
 * TransitCamera answers the frame directly over **http**, no redirect, and
 * that is the only transport this host actually serves to this runtime — so
 * this builds that URL itself from RenderService's `sctidcam` id instead of
- * registering the (https, then-redirecting) RenderService link.
+ * registering the (https, then-redirecting) RenderService link. The
+ * `sctidcam` id must be a bare image filename (CATALONIA_SCT_CAMERA_FILE_RE).
 *
 * @param {string} rawUrl
 * @returns {string} Normalized URL, or '' if not accepted.
@@ -1701,12 +1704,16 @@ export function normalizeCataloniaImageUrl(rawUrl) {
    return '';
  }
  if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return '';
-  if (parsed.username || parsed.password) return '';
-  if (!CATALONIA_IMAGE_HOSTS.has(parsed.hostname)) return '';
+  if (parsed.username || parsed.password || parsed.port) return '';
+  const dir = CATALONIA_IMAGE_HOSTS.get(parsed.hostname);
+  if (!dir || !parsed.pathname.startsWith(dir)) return '';
+  const file = parsed.pathname.slice(dir.length);
+  if (!file || file.includes('/')) return '';

  if (parsed.hostname === 'mct.gencat.cat') {
    const camId = parsed.searchParams.get('sctidcam');
-    if (!camId) return '';
+    if (file !== 'RenderService') return '';
+    if (!camId || !CATALONIA_SCT_CAMERA_FILE_RE.test(camId)) return '';
    const direct = new URL('http://mct.gencat.cat/mct2bo/TransitCamera');
    direct.searchParams.set('nom', camId);
    direct.searchParams.set('visualitzacio', 'imatge');
@@ -1717,6 +1724,30 @@ export function normalizeCataloniaImageUrl(rawUrl) {
  return parsed.toString();
 }

+/**
+ * Stable camera id for one normalized Catalonia frame URL (the output of
+ * normalizeCataloniaImageUrl). The feed's `fid` is not used: it embeds an
+ * upstream server-start stamp and changes on every upstream restart. SCT
+ * cameras key on the `sctidcam` filename stem (`cat-sct-nc87`), partner
+ * cameras on the frame host and path (`cat-www-bcn-cat-transit-imatges-…`).
+ *
+ * @param {string} imageUrl - A URL normalizeCataloniaImageUrl accepted.
+ * @returns {string}
+ */
+export function cataloniaCameraId(imageUrl) {
+  const parsed = new URL(imageUrl);
+  const slug = (text) =>
+    text
+      .toLowerCase()
+      .replace(/[^a-z0-9]+/g, '-')
+      .replace(/^-+|-+$/g, '');
+  if (parsed.hostname === 'mct.gencat.cat') {
+    const file = parsed.searchParams.get('nom') || '';
+    return `cat-sct-${slug(file.replace(/\.[^.]*$/, ''))}`;
+  }
+  return `cat-${slug(parsed.hostname)}-${slug(parsed.pathname)}`;
+}
+
 /**
 * Human label for one Catalonia camera: "<road/place> (<municipality>)",
 * skipping the parenthetical when the municipality is already named in the
@@ -1751,6 +1782,10 @@ export function cataloniaCameraName(carretera, municipi) {
 * shared `provider`. The feed carries no heading, so every camera takes the
 * id-hash fallback, low-confidence pose (same as TfL and Fintraffic).
 *
+ * Ids come from the validated frame URL (cataloniaCameraId), not the feed's
+ * restart-dependent `fid`, and rows sharing one are collapsed before the cap
+ * (the live feed repeats a few SCT cameras).
+ *
 * The list decides ids, coordinates, labels and which allowlisted frame path
 * is proxied, so it is fetched over https, redirects are refused (the list
 * host cannot be steered or downgraded) and the body read is capped at
@@ -1791,15 +1826,12 @@ export async function loadCataloniaSourcesFromOpenData() {

    const cameras = [];
    for (const row of rows) {
-      if (!row.fid) continue;
      if (!isLikelyCataloniaCoordinate(row.lat, row.lon)) continue;
      const imageUrl = normalizeCataloniaImageUrl(row.link);
      if (!imageUrl) continue;

      const font = row.font.trim();
-      const cameraId = `cat-${row.fid
-        .replace(/^cameres\.fid-/, '')
-        .replace(/[^a-zA-Z0-9]/g, '')}`;
+      const cameraId = cataloniaCameraId(imageUrl);

      cameras.push({
        id: cameraId,
@@ -8,6 +8,7 @@ import {
  CATALONIA_CAMERAS_URL,
  CATALONIA_MAX_CATALOG_BYTES,
 } from '../../server/providers/cctv/constants.js';
+import { decodeNumericEntities } from '../../server/providers/cctv/normalize.js';
 import {
  parseCataloniaXml,
  normalizeCataloniaImageUrl,
@@ -177,6 +178,62 @@ test('normalizeCataloniaImageUrl rejects credential-bearing URLs on every allowe
  }
 });

+test("normalizeCataloniaImageUrl rejects explicit ports and paths outside each host's frame directory", () => {
+  // An allowlisted hostname alone would let catalog data aim the frame
+  // request at another service or path on that host.
+  for (const link of [
+    'http://www.bcn.cat:8080/transit/imatges/a.gif',
+    'https://emap.terrassa.cat:8443/it_terrassa/cam02.jpeg',
+    'http://mct.gencat.cat:8080/mct2bo/RenderService?sctidcam=nc87.gif',
+    'http://www.bcn.cat/admin/a.gif',
+    'http://www.bcn.cat/transit/imatges/',
+    'http://www.bcn.cat/transit/imatges/sub/a.gif',
+    'http://www.bcn.cat/transit/imatges/../../admin/a.gif',
+    'https://emap.terrassa.cat/other/cam02.jpeg',
+    'http://mct.gencat.cat/mct2bo/OtherService?sctidcam=nc87.gif',
+  ]) {
+    assert.equal(normalizeCataloniaImageUrl(link), '', link);
+  }
+});
+
+test('normalizeCataloniaImageUrl accepts only a bare image filename as the SCT sctidcam', () => {
+  for (const camId of [
+    '../../etc/passwd',
+    '..%2F..%2Fetc%2Fpasswd',
+    'nc87.png',
+    'nc87',
+    'nc87.gif/../x.gif',
+    'nc87.gif&nom=other.gif',
+    'nc-87.gif',
+  ]) {
+    const link = `http://mct.gencat.cat/mct2bo/RenderService?sctidcam=${encodeURIComponent(camId)}`;
+    assert.equal(normalizeCataloniaImageUrl(link), '', camId);
+  }
+  assert.equal(
+    normalizeCataloniaImageUrl(
+      'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=CB10.JPEG',
+    ),
+    'http://mct.gencat.cat/mct2bo/TransitCamera?nom=CB10.JPEG&visualitzacio=imatge',
+  );
+});
+
+test('decodeNumericEntities decodes decimal and hex references and leaves invalid ones as written', () => {
+  assert.equal(decodeNumericEntities('Llu&#237;s'), 'Lluís');
+  assert.equal(decodeNumericEntities('Llu&#xED;s &#x1F6A6;'), 'Lluís 🚦');
+  assert.equal(decodeNumericEntities('Llu&#XeD;s'), 'Lluís');
+  for (const raw of [
+    'a&#99999999999999999999999;b',
+    'a&#1114112;b',
+    'a&#x110000;b',
+    'a&#xFFFFFFFFFFFFFFFFFFFF;b',
+    'a&#55296;b',
+    'a&#xDFFF;b',
+    'a&#0;b',
+  ]) {
+    assert.equal(decodeNumericEntities(raw), raw, raw);
+  }
+});
+
 test('cataloniaCameraName joins road and municipality, skipping a redundant municipality', () => {
  assert.equal(
    cataloniaCameraName('C-58', 'Nus Trinitat'),
@@ -261,11 +318,15 @@ test('Catalonia loader keeps SCT, Barcelona and Terrassa cameras with per-publis
  );
  assert.deepEqual(
    cameras.map((camera) => camera.id).sort(),
-    ['cat-1', 'cat-2', 'cat-3'],
+    [
+      'cat-emap-terrassa-cat-it-terrassa-cam02-jpeg',
+      'cat-sct-nc87',
+      'cat-www-bcn-cat-transit-imatges-plurquinaona-gif',
+    ],
    'the off-allowlist link (fid-4) and the out-of-bbox row (fid-5) are excluded',
  );

-  const sct = cameras.find((c) => c.id === 'cat-1');
+  const sct = cameras.find((c) => c.id === 'cat-sct-nc87');
  assert.equal(sct.name, 'C-58 (Nus Trinitat)');
  assert.equal(sct.provider, 'Servei Català de Trànsit');
  assert.equal(sct.credit, '', 'SCT is the publisher, no separate credit');
@@ -277,12 +338,16 @@ test('Catalonia loader keeps SCT, Barcelona and Terrassa cameras with per-publis
  assert.equal(sct.groundElevationM, 80);
  assert.equal(sct.headingConfidence, 'low', 'the feed carries no heading');

-  const bcn = cameras.find((c) => c.id === 'cat-2');
+  const bcn = cameras.find(
+    (c) => c.id === 'cat-www-bcn-cat-transit-imatges-plurquinaona-gif',
+  );
  assert.equal(bcn.credit, 'Ajuntament de Barcelona');
  assert.equal(bcn.city, 'Barcelona');
  assert.equal(bcn.groundElevationM, 20);

-  const terrassa = cameras.find((c) => c.id === 'cat-3');
+  const terrassa = cameras.find(
+    (c) => c.id === 'cat-emap-terrassa-cat-it-terrassa-cam02-jpeg',
+  );
  assert.equal(terrassa.credit, 'Ajuntament de Terrassa');
  assert.equal(terrassa.groundElevationM, 270);
 });
@@ -415,7 +480,7 @@ test('CCTV catalog merges Catalonia cameras and CCTV_CATALONIA_ENABLED=0 skips t
  const sources = await createCctvCatalog({ sourceRoot })();
  assert.deepEqual(
    sources.map((source) => source.id),
-    ['cat-1'],
+    ['cat-sct-nc87'],
  );

  process.env.CCTV_CATALONIA_ENABLED = '0';
@@ -423,8 +488,196 @@ test('CCTV catalog merges Catalonia cameras and CCTV_CATALONIA_ENABLED=0 skips t
  assert.deepEqual(await createCctvCatalog({ sourceRoot })(), []);
  assert.ok(requested.length > 0, 'the other live packs still load');
  assert.equal(
-    requested.includes(CATALONIA_CAMERAS_URL),
+    requested.some(([url]) => url === CATALONIA_CAMERAS_URL),
    false,
    'a disabled pack makes no request',
  );
 });
+
+/** The two SCT, one Barcelona and one Terrassa rows of a real feed snapshot,
+ * under one upstream run's `fid` prefix. */
+function snapshotFeed(fidPrefix) {
+  return cataloniaFeed([
+    cataloniaFeature({
+      fid: `cameres.fid-${fidPrefix}_386f`,
+      lon: '2.1849528',
+      lat: '41.45989301',
+      carretera: 'C-58',
+      link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=nc87.gif',
+      font: 'SCT',
+    }),
+    cataloniaFeature({
+      fid: `cameres.fid-${fidPrefix}_b48`,
+      lon: '2.18',
+      lat: '41.46',
+      carretera: 'C-31',
+      link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=nc85.gif',
+      font: 'SCT',
+    }),
+    cataloniaFeature({
+      fid: `cameres.fid-${fidPrefix}_1c2`,
+      lon: '2.17228212',
+      lat: '41.38890733',
+      carretera: 'Plaça Urquinaona',
+      municipi: 'Barcelona',
+      link: 'http://www.bcn.cat/transit/imatges/PlUrquinaona.gif?a=1',
+      font: 'IMI',
+    }),
+    cataloniaFeature({
+      fid: `cameres.fid-${fidPrefix}_1c3`,
+      lon: '2.0111',
+      lat: '41.5636',
+      municipi: 'Terrassa',
+      link: 'https://emap.terrassa.cat/it_terrassa/cam01.jpeg?a=1 &#13;',
+      font: 'Terrassa',
+    }),
+  ]);
+}
+
+async function loadFromFeed(t, xml) {
+  t.mock.method(globalThis, 'fetch', async () => {
+    return new Response(xml, {
+      status: 200,
+      headers: { 'Content-Type': 'application/xml' },
+    });
+  });
+  try {
+    return await loadCataloniaSourcesFromOpenData();
+  } finally {
+    t.mock.restoreAll();
+  }
+}
+
+test('Catalonia camera ids survive an upstream restart that changes every fid', async (t) => {
+  quiet(t);
+  withEnv(t, { CCTV_CATALONIA_MAX_SOURCES: undefined });
+  // The fid embeds the upstream server-start stamp: the PR fixture's
+  // 1a0a06bea1c run and a later 1a140311953 run name the same cameras.
+  const before = await loadFromFeed(t, snapshotFeed('52bf6eca_1a0a06bea1c'));
+  const after = await loadFromFeed(t, snapshotFeed('52bf6eca_1a140311953'));
+  const ids = (cameras) => cameras.map((camera) => camera.id).sort();
+  assert.deepEqual(ids(before), [
+    'cat-emap-terrassa-cat-it-terrassa-cam01-jpeg',
+    'cat-sct-nc85',
+    'cat-sct-nc87',
+    'cat-www-bcn-cat-transit-imatges-plurquinaona-gif',
+  ]);
+  assert.deepEqual(ids(after), ids(before));
+});
+
+test('Catalonia loader collapses rows the feed repeats for one camera', async (t) => {
+  quiet(t);
+  withEnv(t, { CCTV_CATALONIA_MAX_SOURCES: undefined });
+  // The live feed lists nc85, cb5, cb10, cb25 and cd17 twice each, under
+  // different fids.
+  const row = (fid, camId) =>
+    cataloniaFeature({
+      fid,
+      lon: '2.18',
+      lat: '41.46',
+      carretera: 'C-31',
+      link: `http://mct.gencat.cat/mct2bo/RenderService?sctidcam=${camId}`,
+      font: 'SCT',
+    });
+  const cameras = await loadFromFeed(
+    t,
+    cataloniaFeed([
+      row('cameres.fid-x_1', 'nc85.gif'),
+      row('cameres.fid-x_2', 'cb5.gif'),
+      row('cameres.fid-x_3', 'nc85.gif'),
+      row('cameres.fid-x_4', 'cb5.gif'),
+      row('cameres.fid-x_5', 'cb10.gif'),
+    ]),
+  );
+  assert.deepEqual(cameras.map((camera) => camera.id).sort(), [
+    'cat-sct-cb10',
+    'cat-sct-cb5',
+    'cat-sct-nc85',
+  ]);
+});
+
+test('Catalonia loader drops ported and bad-sctidcam links but keeps the rest', async (t) => {
+  quiet(t);
+  withEnv(t, { CCTV_CATALONIA_MAX_SOURCES: undefined });
+  const cameras = await loadFromFeed(
+    t,
+    cataloniaFeed([
+      cataloniaFeature({
+        fid: 'cameres.fid-x_1',
+        lon: '2.17',
+        lat: '41.39',
+        link: 'http://www.bcn.cat:8080/transit/imatges/PlUrquinaona.gif?a=1',
+        font: 'IMI',
+      }),
+      cataloniaFeature({
+        fid: 'cameres.fid-x_2',
+        lon: '2.0111',
+        lat: '41.5636',
+        link: 'https://emap.terrassa.cat:8443/it_terrassa/cam02.jpeg?a=1',
+        font: 'Terrassa',
+      }),
+      cataloniaFeature({
+        fid: 'cameres.fid-x_3',
+        lon: '2.18',
+        lat: '41.46',
+        link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=../../etc/passwd',
+        font: 'SCT',
+      }),
+      cataloniaFeature({
+        fid: 'cameres.fid-x_4',
+        lon: '2.18',
+        lat: '41.46',
+        link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=nc87.gif',
+        font: 'SCT',
+      }),
+    ]),
+  );
+  assert.deepEqual(
+    cameras.map((camera) => [camera.id, camera.url]),
+    [
+      [
+        'cat-sct-nc87',
+        'http://mct.gencat.cat/mct2bo/TransitCamera?nom=nc87.gif&visualitzacio=imatge',
+      ],
+    ],
+  );
+});
+
+test('a malformed numeric entity in one row neither throws nor empties the Catalonia pack', async (t) => {
+  quiet(t);
+  withEnv(t, { CCTV_CATALONIA_MAX_SOURCES: undefined });
+  const cameras = await loadFromFeed(
+    t,
+    cataloniaFeed([
+      cataloniaFeature({
+        fid: 'cameres.fid-x_1',
+        lon: '2.18',
+        lat: '41.46',
+        carretera: 'C-31 &#99999999999999999999;',
+        municipi: 'Bad&#xD800;',
+        link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=nc85.gif',
+        font: 'SCT',
+      }),
+      cataloniaFeature({
+        fid: 'cameres.fid-x_2',
+        lon: '2.18',
+        lat: '41.46',
+        carretera: 'Pla&#xE7;a Catalunya',
+        municipi: 'Llinars del Vall&#232;s',
+        link: 'http://mct.gencat.cat/mct2bo/RenderService?sctidcam=nc87.gif',
+        font: 'SCT',
+      }),
+    ]),
+  );
+  const byId = new Map(cameras.map((camera) => [camera.id, camera]));
+  assert.equal(cameras.length, 2);
+  assert.equal(
+    byId.get('cat-sct-nc85').name,
+    'C-31 &#99999999999999999999; (Bad&#xD800;)',
+    'an invalid reference is kept as written',
+  );
+  assert.equal(
+    byId.get('cat-sct-nc87').name,
+    'Plaça Catalunya (Llinars del Vallès)',
+  );
+});