Commit Graph
22 Commits
Author SHA1 Message Date
Said Turksever db8afbae81 fix(cctv): key-gate Ottawa, opt-in Québec video, keep shared slots
Review fixes for #1506.

- Ottawa: use the documented camera list and the city's Traffic Camera
  Images API. The pack is skipped with no request unless
  OTTAWA_CAMERA_CERTIFICATE is set; the certificate is added only at
  fetch time inside the pinned /opendata/ path, and each camera is
  fetched at most once a minute through the shared frame cache.
- Québec 511 video is opt-in (CCTV_QUEBEC_VIDEO_ENABLED=1) and loads
  only after the viewer presses Play; selection, auto-hop, scenes and
  voice no longer contact Québec 511. Credit splits locations (MTQ,
  CC BY 4.0) from video (Québec 511). Interrupted play() is no longer
  reported as an unavailable source; the per-frame listener runs only
  while a Québec camera is selected and the canvas rect is cached.
- Catalog cap: reserved Québec and World slots come only from capacity
  above the 4,000 shared slots, so a lower CCTV_MAX_SOURCES thins every
  pack instead of starving the existing ones. dev-fresh and
  .env.example default to 7,516.
- Throttled frames (World, Ottawa): inside the refresh floor with no
  cached frame the proxy serves the placeholder, never Street View; a
  failed fetch backs off for a minute; the cache is bounded by bytes.
- MJPEG first-frame extraction runs only for sources registered as
  MJPEG (Taiwan Highway Bureau), caps a frame at 2 MB, reads in linear
  time, accepts a CRLF preamble and a boundary that already starts with
  "--", and ends a part without Content-Length at the next delimiter.
  THB hosts are registered in the redirect pin list.
- Tests: World loader policy, Québec loader and opt-in, proxy cooldown
  without Street View, Ottawa signing, cap reservations at 7,516, 4,000
  and 48, overlay Play gate, refresh, visibility and teardown. Guard
  tests check the 4,000 shared slots again.
- Remove the unused standalone Québec preview module and public page.
- Docs: DATA_SOURCES, SECURITY, CURRENT-STATE, CCTV_INTEGRATION, README,
  .env.example, changelog fragment renamed to its full scope.
2026-11-03 15:31:08 +00:00
InoukT fa4ce4b2a5 feat(cctv): add Quebec, Ottawa, and World feeds 2026-11-02 22:57:22 -05:00
Sameh Khamis f60b9ef7c8 chore: remove unused imports and local helpers
Drop module-local helpers that nothing calls (linkAbort, summarize,
stringOption, summarizeEntity and the polygonCenter, propertyObject and
unwrapProperties helpers only it used, CALLOUT_DOT_SIZE_PX) and unused
import specifiers in runtime modules, tests, QA scripts and tools.

None of the removed bindings is exported. Re-exports such as the ALPR
policy constants and every package export are unchanged.
2026-10-22 18:09:37 -07:00
Sameh Khamis b75db874c6 Read API paths and forwarded requests as the router does
The router reads an absolute-form target's path without resolving dot
segments, so a target such as http://host/api/../x still reaches /api
middleware. Take that path the same way when deciding whether the API
response headers apply.

The Street View fallback now treats any forwarding header as forwarded, even
an empty one, before it spends the key.
2026-10-21 15:43:29 -07:00
Sameh Khamis b1ed77c26c Keep camera routes reachable behind a reverse proxy
The CCTV routes now use the keyless cross-site checks, which refuse cross-site
browser requests but accept forwarding headers, so cameras work behind the
documented reverse proxy. The Street View fallback is the one step that spends
a key; a request with forwarding headers skips it and gets the placeholder
frame, as when no key is configured.
2026-10-21 15:33:14 -07:00
Sameh Khamis 0ad75095a7 Validate provider response formats
CCTV frames and the Street View fallback accept only JPEG, PNG, GIF, WebP and
AVIF, parsed from the media type essence and sent under the canonical type;
any other type is cancelled and falls through to the next fallback as a
failed upstream does. Live media must match its feed kind (video/mp4,
video/webm, or multipart/x-mixed-replace with a boundary) before the camera
is reported healthy or any byte is relayed, and its redirects now stay on the
registered origin like frames. Every /api/cctv route uses the same-site gate.

The GBFS proxy relays only bodies that parse as JSON, re-serialized under a
fixed application/json type.
2026-10-21 15:07:20 -07:00
Said TurkseverandClaude Opus 5.5 b66d02df7c fix(cctv): space out failed background refreshes; test the stale paths
- A background catalog refresh that rejects now keeps the old catalog for
  another TTL, as an empty refresh already did, so a refresh that keeps
  failing is retried once per TTL instead of on every request.
- Trim notes are keyed by the source list they describe, and /sources asks
  for the list it served, so a request answered during a refresh never pairs
  the old cameras with the new list's notes.
- Tests: a failed background refresh (one warning, no unhandled rejection,
  no retry before the TTL, trim notes stay with their list) and an empty
  background refresh (old catalog kept, no upstream fetch before the TTL).
  The race assertion's message now names the failure; its timer is cleared.
- The Tarktee test no longer imports the new constant, so on main it fails
  on its own assertion instead of taking the whole file down.
- precompute-cctv-heights waits 60 s for a cold catalog, past Tarktee's 30 s.
- CURRENT-STATE notes the 30 s bound and the retry spacing.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 17:02:06 +01:00
Sameh Khamis a800d28ddf Merge branch 'main' into feature/catalonia-cctv-cameras 2026-10-15 16:05:20 -07:00
Sameh Khamis 248a658713 fix(cctv): serve only live feeds on /media
/api/cctv/media relayed still-image cameras with an ordinary fetch: it followed
redirects to any host, passed through any content type and had no body cap.
Stills are served by /api/cctv/frame, which keeps redirects on the registered
origin, accepts only image/* and caps the body, and no client requests a still
through /media. /media now answers 404 for anything but HLS, MP4/WebM and
MJPEG, before any upstream request.
2026-10-15 15:53:09 -07:00
bassem chagra 054f1cc198 Merge remote-tracking branch 'origin/main' into feat/king-county-camera-pack
# Conflicts:
#	.env.example
#	CHANGELOG.md
#	DATA_SOURCES.md
#	README.md
#	docs/CURRENT-STATE.md
#	server/providers/cctv/catalog.js
#	server/providers/cctv/sources.js
#	src/data/dataCredits.js
2026-10-08 15:52:38 +02:00
Sameh Khamis 66b07418bd fix: say when the camera catalog serves only part of a region
Camera packs keep their nearest cameras and the catalog cap thins them
further, so London answered 250 cameras while TfL offers many more. The
catalog now records each pack's full count, tags cameras with their pack,
and serves the packs it trimmed; find_cctv_cameras names them.
2026-10-02 15:21:36 -07:00
bassem chagra c3f197823e fix(cctv): serve King County model/manufacturer through /api/cctv/sources
The loader and normalizer carried the hardware model, but the /sources
response projection and the browser catalog both whitelist fields and
dropped it, so no client consumer ever saw it. Project model and
manufacturer (manufacturer disambiguates bare model strings like "3950")
at both boundaries, and add a test that drives the real middleware and
browser catalog end to end.
2026-09-27 12:09:27 +00:00
0251c74109 feat(weather): add globe wind, observed weather and cyclone context (#617)
Wind (NOAA GFS and ECMWF IFS: animated particles, colour field, a reading at map centre), observed weather (NOAA MRMS radar, GOES and NESDIS infrared clouds, lightning density) on one shared history timeline, NHC and CPHC cyclone advisories with tracks and cones, and a WEATHER panel in the right rail. On 3D Tiles the observed products render as raised shells with a sharper window around the view.

Co-authored-by: Gustavo Beneduzi <[email protected]>
Co-authored-by: Daniel Slay <[email protected]>
Co-authored-by: Rehaan Delmotra <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
2026-09-22 17:45:39 -05:00
Pedro Lobato 3308569edd docs(cctv): the media-fetch log line drops the error text on purpose
The comment claimed "Keep the detail in the log" while the line it introduces
carries none, so the next reader would have read the omission as an oversight
and added the errno back.

Detail-free is the deliberate choice: a media-fetch failure names the camera's
upstream host, and these lines get pasted into issues. Say that, and say why
the catch at the bottom of this file may still log its error text.
2026-09-15 12:06:11 -05:00
Pedro Lobato 9e71081a5a fix(server): stop relaying upstream and JS error text to the client
Ported onto the split provider layout: the routes now live in
server/providers/openai/{hud-summary,realtime}.js and the camera health
entry in server/providers/cctv.js.

The routes #263 names — celestrak, terrain, adsbdb — were sanitized by
#289. The same defect had moved:

- /api/openai/hud-summary relayed `data.error.message` verbatim on a
  non-ok upstream. That is OpenAI's own error text, carrying request ids,
  organization hints and quota wording. Its catch echoed the JS error too.
- /api/realtime/token echoed the JS error when minting failed, which for a
  network fault is a resolver message naming the upstream host.
- cctvProxy stored the raw error as a camera's health `message`. This is
  the least obvious of the three and the reason the PR is worth its size:
  GET /api/cctv/health serializes `message`, and src/layers/cctv/frames.js
  renders it as the camera's status label, so the detail reached the
  screen through a different door than the sanitized 502 beside it.

Logs name the failure and the upstream status without the text, matching
the Launch Library and terrain paths; these lines get pasted into issues.

Coverage extends the existing harnesses rather than adding files —
localServices.test.mjs drives both OpenAI routes over a mocked upstream,
mediaProviders.test.mjs drives a failing media fetch and then reads
/health. All three assertions fail on the previous implementation.

Refs #263. Does not touch /api/realtime/debug-log's identical echo; that
route is the subject of the #23 PR, so the two do not overlap a line.
2026-09-14 21:24:05 -05:00
Bilawal SidhuandMaher Kurdi da700be77f fix(cctv): bound the Range header and release abandoned camera streams (#512)
Supersedes #253 by @Maher-Reven, ported onto the current component layout with fixes from review.

Verified before merge: format, package boundaries, unit suite, build, and the browser tracking suite; a browser harness for the change where one applies; adversarial review applied.

Co-authored-by: Maher Kurdi <[email protected]>
2026-09-14 19:17:52 -05:00
12790b36c6 feat(cctv): seven camera packs, semantic labels, terrain-clear monitor planes (#433)
Seven contributed CCTV camera packs (Ontario 511, Fintraffic Finland, DriveBC, TxDOT Texas, Estonia Tallinn + Tarktee, Warendorf, Live Traffic NSW) on a fair per-pack catalog cap; semantic camera labels; monitor planes lifted clear of the terrain from a precomputed, bundled ground-height dataset. Default catalog 800 → 3,446 cameras. Full context in PR #433.

Co-authored-by: sendmebits <[email protected]>
Co-authored-by: avocadoAgent <[email protected]>
Co-authored-by: Alex Harvey <[email protected]>
Co-authored-by: Jason Meredith <[email protected]>
Co-authored-by: Daniel Smith <[email protected]>
Co-authored-by: Vladimir Sobolev <[email protected]>
Co-authored-by: Matthias Schneider <[email protected]>
Co-authored-by: Cyber Knight <[email protected]>
Co-authored-by: NovaSoft Australia <[email protected]>
2026-09-13 21:48:33 -05:00
Bilawal Sidhu 3d956177ab fix(providers): validate before limiting and tighten capped image reads
Apply shared Places validation, owned image chunks, and response-header deadline tests to the provider modules.
2026-09-12 16:35:11 -07:00
Prajwal Raj c78ea4194c fix: bound CCTV media upstream connection time
Adapt the response-header timeout contributed in public PR #53 to the CCTV provider.
2026-09-12 16:33:51 -07:00
Jejugo a0f07b752f fix: register data providers for local preview
Adapt the preview middleware contribution from PR #324 to the extracted provider modules, including the existing OpenSky route.
2026-09-12 12:43:54 -07:00
Sameh Khamis c69416ff6b style: adopt formatting for CCTV and radio provider modules 2026-09-12 12:09:04 -07:00
Sameh Khamis 79b4929b88 refactor: extract CCTV media and radio directory providers 2026-09-12 12:09:02 -07:00