diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index baf154774..613a1d918 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -69,6 +69,7 @@ How to read this: | **King County Road Services** (`services.arcgis.com`, King County GIS) | CCTV camera catalog + frames, unincorporated King County WA road network | [KCGIS Center terms](https://kingcounty.gov/en/dept/kcit/data-information-services/gis-center/about/terms-conditions-copyrights) — attribution required; public ArcGIS layer | "King County Road Services" (partner-city cameras name their owner) | | **WSDOT** (`data.wsdot.wa.gov`) | CCTV camera catalog + frames, Washington State highways | [WSDOT travel information disclaimer](https://wsdot.wa.gov/about/policies/travel-information-disclaimer) — public "low volume" feed; catalog cached, frames on WSDOT's own image host only | "Washington State Department of Transportation (Travel Center)" (courtesy) | | **Hong Kong Transport Department / DATA.GOV.HK** | CCTV camera catalog + frames, Hong Kong traffic snapshot cameras | [DATA.GOV.HK Terms of Use](https://data.gov.hk/en/terms-and-conditions) — attribution REQUIRED | "Hong Kong Transport Department / DATA.GOV.HK" | +| **Webcam Romania — Giroc** (`webcamromania.ro`, `w4.webcamromania.ro`) | Optional Giroc camera metadata, JPEG snapshot and live HLS video | [Provider terms](https://webcamromania.ro/termeni-si-conditii/): video belongs to the original camera operator; no open video redistribution licence asserted. The [camera page](https://webcamromania.ro/webcam-orase/webcam-giroc/) offers an embed. Local opt-in via `CCTV_SOURCES_FILE`; media is fetched at runtime, never bundled. | "Webcam Romania · webcamromania.ro"; source link and rights notice in the Data attribution popover | | **Statens vegvesen** (catalog `ogckart-sn1.atlas.vegvesen.no`, frames `kamera.atlas.vegvesen.no`, HLS `kamera.vegvesen.no`) | CCTV camera catalog + frames + live HLS video, Norway road cameras | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) distributed by Statens vegvesen" | | **Kartverket** (`ws.geonorge.no/hoydedata/v1/punkt`) | Terrain heights (NN2000) used as the ground prior for Statens vegvesen cameras | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — attribution REQUIRED | "© Kartverket", modified (heights over water set to 0 m, rounded to 0.1 m) | | **NYC DOT Traffic Management Center** (`webcams.nyctmc.org`) | CCTV camera catalog + frames, New York City five boroughs | NYC DOT requires developers to sign a [data-sharing agreement](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) for the camera feed. **Off by default**; enable only under your own agreement | "NYC DOT Traffic Management Center" + NYC DOT's required application disclaimer (in the Data attribution popover) | diff --git a/README.md b/README.md index ef2eb0767..00f65bf3b 100644 --- a/README.md +++ b/README.md @@ -639,7 +639,7 @@ and panel. DelDOT uses its official HTTPS HLS catalog links. Disable that pack with `CCTV_DELDOT_ENABLED=0`. The server allows two concurrent sessions. Each retains at most 12 segments -and 24 MiB in memory; individual downloads are capped at 4 MiB with a ten +and 24 MiB in memory; individual downloads are capped at 8 MiB with a ten second deadline. There are no segment files or ffmpeg processes. Redirects, off-origin references, encrypted playlists and non-MPEG-TS segments are refused. Each decoder has its own client lease (at most eight per session), including diff --git a/SECURITY.md b/SECURITY.md index d1552028f..6e3625f71 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -69,7 +69,7 @@ browser-key fallback for existing single-key setups. The data proxies under `server/providers/` are written so the browser cannot turn the server into an open relay: - **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). Other proxies target fixed upstream hosts. -- **Live HLS is bounded and origin-pinned.** Registered HTTP(S) HLS sources use a Node puller that rejects redirects and off-origin playlist references, bounds playlists and segment bodies while streaming, and aborts timed-out or released sessions. At most two sessions retain 12 segments/24 MiB each, plus bounded download buffers. Stores are memory-only and idle sessions expire after 15 seconds. RTMP/ffmpeg execution is not enabled. Locally configured source URLs remain an operator trust boundary. +- **Live HLS is bounded and origin-pinned.** Registered HTTP(S) HLS sources use a Node puller that rejects redirects and off-origin playlist references, bounds playlists to 256 KiB and segment bodies to 8 MiB while streaming, and aborts timed-out or released sessions. At most two sessions retain 12 segments/24 MiB each, plus bounded download buffers. Stores are memory-only and idle sessions expire after 15 seconds. RTMP/ffmpeg execution is not enabled. Locally configured source URLs remain an operator trust boundary. - **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio. - **Québec 511 video plays in the browser, opt-in and after Play.** With `CCTV_QUEBEC_VIDEO_ENABLED=1`, `/api/cctv/sources` lists, for each MTQ camera, one URL built server-side from its numeric id on `https://www.quebec511.info/Carte/Fenetres/camera.ashx`; the overlay accepts no other host, path or format. The browser loads that clip only after the viewer presses Play for the selected camera, so Québec 511 sees the viewer's IP address, and the panel says so before Play. Selecting a camera by click, auto-hop, a scene or voice never loads it, and switching cameras stops playback. GEV never proxies, caches or reads the video's pixels. With the flag unset (the default) the browser makes no request to Québec 511. - **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body. diff --git a/changelog.d/1619-giroc-camera-hls-segments.md b/changelog.d/1619-giroc-camera-hls-segments.md new file mode 100644 index 000000000..f47718e16 --- /dev/null +++ b/changelog.d/1619-giroc-camera-hls-segments.md @@ -0,0 +1,5 @@ +- Add an optional Giroc / Webcam Romania CCTV source pack with corrected + coordinates, live HLS, a snapshot fallback and provider attribution. Allow + HLS segments up to 8 MiB so high-resolution cameras whose segments exceed + 4 MiB can play, while retaining the 24 MiB per-session cache and existing + origin and session limits (NeoXrated23, #1619). diff --git a/config/cctv_sources.romania.json b/config/cctv_sources.romania.json new file mode 100644 index 000000000..11b3f9696 --- /dev/null +++ b/config/cctv_sources.romania.json @@ -0,0 +1,23 @@ +[ + { + "id": "ro-giroc-webcamromania", + "name": "Giroc — Webcam Romania", + "city": "Giroc, Timiș", + "cityId": "giroc", + "provider": "Webcam Romania · webcamromania.ro", + "sourceKind": "configured", + "feedType": "hls", + "url": "https://w4.webcamromania.ro/giroc/index.m3u8", + "snapshotUrl": "https://webcamromania.ro/media/snapshots/01a0267e-d893-7203-a7eb-7ce3c8c99703/latest.jpg", + "lat": 45.70028947266781, + "lon": 21.233675914004227, + "headingDeg": 0, + "headingConfidence": "low", + "pitchDeg": -17, + "fovDeg": 74, + "rangeM": 700, + "mountHeightM": 24, + "groundElevationM": 87, + "license": "Public camera: https://webcamromania.ro/webcam-orase/webcam-giroc/ . Provider terms: https://webcamromania.ro/termeni-si-conditii/ . Video rights remain with the camera operator. Coordinates corrected by the contributor; direction, height, elevation and field of view are uncalibrated estimates." + } +] diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 3f7e96e44..3bf6b1051 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1558,6 +1558,32 @@ owners. Tuner calculations have a pure entry, with existing layer exports preserved. Destruction removes listeners and state subscriptions before ending tuning; a delayed Enable result cannot reveal or refocus removed controls. +## Giroc / Webcam Romania CCTV source pack + +`config/cctv_sources.romania.json` is an optional single-camera pack for Giroc, +Timiș, Romania. To use it, set `CCTV_SOURCES_FILE=config/cctv_sources.romania.json` +in the local `.env` and restart the server. Set `CCTV_FORCE_AUSTIN=1` as well to +retain the built-in live packs; despite its historical name, this flag applies +to all live packs. The default catalog is unchanged. + +The camera uses Webcam Romania's public HLS stream and JPEG snapshot, fetched +at runtime without an API key. Its position, 45.70028947266781, 21.233675914004227, +was corrected by the contributor rather than copied from the directory's +inaccurate pin. Heading, pitch, field of view, mounting height and ground +elevation are uncalibrated priors; the camera retains the RAW PRIOR badge. +See `DATA_SOURCES.md` for attribution and provider terms. Webcam Romania's +terms grant no reuse licence for the video and name no camera operator; the +provider's sanctioned route is its iframe embed. Keep this pack to local use +and off hosted deployments unless the operator grants permission. + +The existing HLS puller resolves same-origin relative master/media references +and serves local leased playlists and segments. It downloads complete segments +without forwarding a browser Range header. Its per-segment streamed-byte cap +is 8 MiB: Giroc's roughly ten-second 2702×1520 segments can exceed the former +4 MiB cap (one measured 6,437,120 bytes). Retained data remains capped at +24 MiB and 12 segments per session, with two sessions maximum. Redirect +refusal, origin pinning, timeouts and lease expiry are unchanged. + ## NYC DOT TMC CCTV source pack The CCTV catalog can add New York City traffic cameras from the NYC DOT diff --git a/server/providers/cctv/stream.js b/server/providers/cctv/stream.js index da811f5f9..4e37cda20 100644 --- a/server/providers/cctv/stream.js +++ b/server/providers/cctv/stream.js @@ -5,7 +5,9 @@ export const HLS_LIMITS = Object.freeze({ sessions: 2, leasesPerSession: 8, playlistBytes: 256 * 1024, - segmentBytes: 4 * 1024 * 1024, + // High-resolution cameras can exceed 4 MiB in a single ten-second TS segment. + // Retained data still shares the independent 24 MiB session budget below. + segmentBytes: 8 * 1024 * 1024, sessionBytes: 24 * 1024 * 1024, segments: 12, pollMs: 2000, diff --git a/src/data/cctvHlsStream.test.mjs b/src/data/cctvHlsStream.test.mjs index a087eebdf..4a5f4afbc 100644 --- a/src/data/cctvHlsStream.test.mjs +++ b/src/data/cctvHlsStream.test.mjs @@ -10,6 +10,77 @@ const playlist = '#EXTM3U\n#EXT-X-MEDIA-SEQUENCE:12\n#EXTINF:2,\na.ts\n#EXTINF:2,\nb.ts\n#EXTINF:2,\nc.ts\n'; const base = 'https://camera.example/live/list.m3u8'; +test('high-resolution HLS segments play while the retained session cache stays bounded', async (t) => { + // Giroc publishes roughly ten-second TS segments; one measured 6,437,120 + // bytes. The former 4 MiB download cap rejected it before playback began. + const segmentBytes = 6_437_120; + const master = + '#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=4570000\ntracks-v1a1/mono.m3u8\n'; + const media = + '#EXTM3U\n#EXT-X-MEDIA-SEQUENCE:100\n' + + [0, 1, 2, 3, 4].map((i) => `#EXTINF:10.272,\n${i}.ts\n`).join(''); + const manager = createHlsPuller({ + limits: { ...HLS_LIMITS, pollMs: 100000 }, + fetchImpl: async (url) => { + if (url === base) return new Response(master); + if (url === 'https://camera.example/live/tracks-v1a1/mono.m3u8') + return new Response(media); + const match = + /^https:\/\/camera\.example\/live\/tracks-v1a1\/([0-4])\.ts$/.exec(url); + if (!match) throw new Error('Unexpected playlist reference'); + // No Content-Length: the actual streamed bytes must also be accepted. + return new Response(Buffer.alloc(segmentBytes, Number(match[1]))); + }, + }); + t.after(() => manager.shutdown()); + const entry = await manager.ensure('giroc', base); + await entry.polling; + const result = await manager.buildPlaylist(entry, 'giroc'); + assert.ok(result, 'large valid segments should produce a playable playlist'); + assert.match(result, /#EXT-X-MEDIA-SEQUENCE:2\n/); + assert.equal(manager.getSegment('giroc', entry.token, 0), null); + const newest = manager.getSegment('giroc', entry.token, 4); + assert.equal(newest.length, segmentBytes); + assert.equal(newest[0], 4); + assert.deepEqual(manager.stats(), { sessions: 1, bytes: 19_311_360 }); +}); + +test('HLS downloads still reject declared and chunked bodies above 8 MiB', async () => { + for (const declared of [true, false]) { + let chunks = 0; + let cancelled = false; + await assert.rejects( + fetchHlsBytes(base, { + maxBytes: HLS_LIMITS.segmentBytes, + fetchImpl: async () => + new Response( + declared + ? 'small body' + : new ReadableStream({ + pull(controller) { + // Each chunk fits on its own; only their cumulative size + // crosses the cap. Leave it open to verify cancellation. + controller.enqueue( + Buffer.alloc(4 * 1024 * 1024 + (++chunks === 2 ? 1 : 0)), + ); + }, + cancel() { + cancelled = true; + }, + }), + { + headers: declared + ? { 'content-length': String(8 * 1024 * 1024 + 1) } + : {}, + }, + ), + }), + /HLS response too large/, + ); + if (!declared) assert.equal(cancelled, true); + } +}); + test('playlist parser uses sequence, refuses escaping and unsupported references', () => { assert.deepEqual( parseHlsMedia(playlist, base).map((s) => s.seq), diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index 8ffbf3aaf..82184b6dd 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -386,6 +386,14 @@ export const DATA_CREDITS = [ 'Transport Department / DATA.GOV.HK ' + '(Terms of Use — attribution required)', }, + { + key: 'webcamromania-cctv', + html: + 'Giroc camera (when configured): ' + + 'Webcam Romania. ' + + 'Video rights remain with the camera operator; ' + + 'provider terms apply.', + }, { key: 'vegvesen-cctv', html: