Merge main into denis/iss-live-telemetry

This commit is contained in:
Denis Groshev committed 2026-11-20 13:12:36 +01:00
commit 050e0a3a7a
38 files changed
+1431 -251

No files matched your search

+2
View File
@@ -37,6 +37,8 @@ GOOGLE_MAPS_API_KEY=
GOOGLE_MAPS_SERVER_API_KEY=
# Per-IP rate limit for the Google Places cost endpoints
# (/api/google/nearby-places + /api/google/text-search), in requests per minute.
# The CCTV Street View fallback counts in its own window with the same cap;
# over it, a frame gets the placeholder image instead of a 429.
# DEFAULTS TO 120, the value the Pinokio build already ships. Raise or lower it
# with a positive integer; 60 is a sensible tightening whenever HOST is not
# localhost. Set it to exactly 0 to run unthrottled. An unreadable value (a
+2 -2
View File
@@ -72,7 +72,7 @@ How to read this:
| **Caltrans (cwwp2.dot.ca.gov)** | CCTV camera catalogs + frames, California districts | Public Caltrans traffic camera data | "Caltrans — cwwp2.dot.ca.gov" (courtesy) |
| **DelDOT (video.deldot.gov)** | CCTV camera catalog (`tmc.deldot.gov/json/videocamera.json`, cached for 15 minutes) + live HTTPS HLS video, Delaware. Keyless | Public DelDOT traffic camera streams via the [DelDOT traffic map](https://deldot.gov/map/) | "DelDOT — Delaware Department of Transportation" (courtesy) |
| **TfL Open Data (JamCams)** | CCTV camera catalog + frames, London | [TfL Open Data terms](https://tfl.gov.uk/info-for/open-data-users/) — attribution REQUIRED | "Powered by TfL Open Data. Contains OS data © Crown copyright and database rights" |
| **Ontario 511** | CCTV camera catalog + frames, Ontario highways including Kitchener-area routes | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) | "Ontario 511" plus Open Government Licence - Ontario |
| **Ontario 511** | CCTV camera catalog + frames, Ontario highways including Kitchener-area routes | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) | "Ontario 511. Contains information licensed under the Open Government Licence – Ontario" |
| **Fintraffic / Digitraffic (weathercams)** | CCTV camera catalog + frames, Finland | [CC BY 4.0](https://www.digitraffic.fi/en/terms-of-service/) — attribution REQUIRED | "Fintraffic / digitraffic.fi, license CC BY 4.0" |
| **DriveBC** (Government of British Columbia) | CCTV camera catalog + frames, British Columbia highways | [Open Government Licence – British Columbia](https://www2.gov.bc.ca/gov/content/data/open-data/open-government-licence-bc) — attribution REQUIRED | "DriveBC. Contains information licensed under the Open Government Licence – British Columbia" |
| **Open Calgary** (`data.calgary.ca`) | CCTV camera catalog + frames, Calgary | [Open Government Licence – City of Calgary](https://data.calgary.ca/stories/s/Open-Calgary-Terms-of-Use/u45n-7awa) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – City of Calgary" |
@@ -251,7 +251,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
- **Open Vessel Photos.** When you select a vessel with a nine-digit MMSI, the browser asks `https://openvesselphotos.org/v1/mmsi/{mmsi}` directly (the API is keyless and answers cross-origin, so no key and no server proxy are involved; account credentials are never sent) for that vessel only, once per MMSI per session, with an 8-second timeout. A miss (a 404 or a vessel without photos) is remembered for the session; a failed request may be asked again after five minutes. The first photo whose image is an `https://openvesselphotos.org` URL and which names a licence is shown, at the 800 px size, in the CONTEXT panel's CONTACTS view under the vessel's name and MMSI (`src/ui/views/panels/AwarenessPanel.vue`), and nowhere else (`src/data/openVesselPhoto.js`, `src/layers/vessels/photos.js`). The image is hotlinked from the URL the API gives (never copied or re-hosted); the photographer and licence are visible text next to it ("Photo: {photographer} · {licence}", or "Photo · {licence}" when the answer names no one), and the photo and credit link to the vessel's Open Vessel Photos page with `target="_blank" rel="noopener"`. If the image fails to load, the photo and its credit are hidden.
- **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state.
- **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover.
- **Ontario 511 CCTV.** The camera list comes from the `511on.ca/api/v2/get/cameras` endpoint, which now requires a free developer key (`ONTARIO_511_API_KEY`: sign up at 511on.ca/my511/register, then request it while logged in at 511on.ca/developers/doc; throttled to 10 calls a minute). Without the key the pack is skipped. Frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence - Ontario. Access is governed by Ontario 511's Developer's Access Agreement (511on.ca/developers/resources): the data stays under the Open Government Licence - Ontario unless otherwise specified; key requests are reviewed case by case and ask for the intended use and any third parties that will receive the data; attribution by link to 511on.ca is encouraged, and the Ontario 511 logo may not be altered or used to imply endorsement (this app links and shows no logo); and the data is provided "as is", a disclaimer distributors should pass on, which the credit does.
- **Ontario 511 CCTV.** The camera list comes from the `511on.ca/api/v2/get/cameras` endpoint, which now requires a free developer key (`ONTARIO_511_API_KEY`: sign up at 511on.ca/my511/register, then request it while logged in at 511on.ca/developers/doc; throttled to 10 calls a minute). Without the key the pack is skipped. Frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence – Ontario. Access is governed by Ontario 511's Developer's Access Agreement (511on.ca/developers/resources): the data stays under the Open Government Licence – Ontario unless otherwise specified; key requests are reviewed case by case and ask for the intended use and any third parties that will receive the data; attribution by link to 511on.ca is encouraged, and the Ontario 511 logo may not be altered or used to imply endorsement (this app links and shows no logo); and the data is provided "as is", a disclaimer distributors should pass on, which the credit does.
- **DelDOT live video cameras.** The keyless `tmc.deldot.gov/json/videocamera.json` catalog supplies active cameras and official `https://video.deldot.gov/live/…/playlist.m3u8` links. The local server transiently buffers bounded MPEG-TS segments in memory and deletes them on session release or idle expiry. No recordings are retained. `CCTV_DELDOT_ENABLED=0` disables the pack. Public availability is not a blanket redistribution license; deployment beyond personal local viewing needs confirmation of applicable DelDOT terms.
- **Fintraffic weathercams.** The camera list is one keyless GeoJSON request to `tie.digitraffic.fi/api/weathercam/v1/stations` (809 stations / 2,275 presets), sent with the `Digitraffic-User: gods-eye-view` header the service asks clients to identify themselves with and with gzip; frames are stills on `weathercam.digitraffic.fi`, refreshed on each station's 600 s collection interval, which is also the client's ambient-still refresh cadence for this pack (the active camera keeps the usual 10-second refresh). One preset (a fixed camera view) is one camera; stations not `GATHERING` and presets not `inCollection` are dropped. Attribution is required by the CC BY 4.0 licence and is registered in the Data attribution popover. `CCTV_FINTRAFFIC_ENABLED=0` is the kill switch.
- **DriveBC highway cameras.** The camera list comes from the keyless `https://www.drivebc.ca/api/webcams/` endpoint served by the DriveBC.ca site ([bcgov/DriveBC.ca](https://github.com/bcgov/DriveBC.ca)); frames come from `https://www.drivebc.ca/images/<id>.jpg`, built from the numeric camera id. DataBC's [DriveBC HighwayCams](https://catalogue.data.gov.bc.ca/dataset/bc-highwaycams) catalogue entry publishes the same cameras under the Open Government Licence – British Columbia, but its CSV still lists the retired `images.drivebc.ca` frame URLs, which now return a placeholder. The licence's attribution statement is required and is registered in the Data attribution popover. By default the 250 cameras nearest Vancouver and Victoria load; `CCTV_DRIVEBC_MAX_SOURCES` changes the cap and `CCTV_DRIVEBC_ENABLED=0` turns the pack off. Some cameras in the feed are supplied by partners (TransLink, the City of Vancouver, the City of Surrey, Parks Canada / Alberta Motor Association and others); the feed's per-camera credit is carried onto the camera record and shown beside the provider in the CCTV panel.
+2 -2
View File
@@ -68,7 +68,7 @@ browser-key fallback for existing single-key setups.
The data proxies under `server/providers/` are written so the browser cannot turn the server into an open relay:
- **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). Other proxies target fixed upstream hosts.
- **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). The key-bearing Street View fallback likewise answers only for a registered camera, at its catalog position; a request sets only the heading, field of view and pitch, never the location. Other proxies target fixed upstream hosts.
- **Live HLS is bounded and origin-pinned.** Registered HTTP(S) HLS sources use a Node puller that rejects redirects and off-origin playlist references, bounds playlists to 256 KiB and segment bodies to 8 MiB while streaming, and aborts timed-out or released sessions. At most two sessions retain 12 segments/24 MiB each, plus bounded download buffers. Stores are memory-only and idle sessions expire after 15 seconds. RTMP/ffmpeg execution is not enabled. Locally configured source URLs remain an operator trust boundary.
- **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio.
- **Québec 511 video plays in the browser, opt-in and after Play.** With `CCTV_QUEBEC_VIDEO_ENABLED=1`, `/api/cctv/sources` lists, for each MTQ camera, one URL built server-side from its numeric id on `https://www.quebec511.info/Carte/Fenetres/camera.ashx`; the overlay accepts no other host, path or format. The browser loads that clip only after the viewer presses Play for the selected camera, so Québec 511 sees the viewer's IP address, and the panel says so before Play. Selecting a camera by click, auto-hop, a scene or voice never loads it, and switching cameras stops playback. GEV never proxies, caches or reads the video's pixels. With the flag unset (the default) the browser makes no request to Québec 511.
@@ -100,7 +100,7 @@ The dev server is a **key broker**: every server-side key above is spendable by
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Typed chat keeps its own per-IP window, set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`; left unset, each takes its provider's resolved cap (`GEV_RATELIMIT_OPENAI_PER_MIN`, or `GEV_RATELIMIT_GEMINI_PER_MIN`, default `30`), so lowering a provider throttle lowers chat too, while an explicit chat value, including `0`, wins. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). The CCTV Street View fallback has its own per-IP window on `GEV_RATELIMIT_GOOGLE_PER_MIN`, separate from the Places and Geocoding one; over it, a frame gets the placeholder instead of a `429`. Typed chat keeps its own per-IP window, set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`; left unset, each takes its provider's resolved cap (`GEV_RATELIMIT_OPENAI_PER_MIN`, or `GEV_RATELIMIT_GEMINI_PER_MIN`, default `30`), so lowering a provider throttle lowers chat too, while an explicit chat value, including `0`, wins. Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **Docker trusts one address, and only because the port is on loopback.**
Docker delivers the host's own browser to the container from the container
network's gateway, never from `127.0.0.1`, so a loopback-only Provider
+13
View File
@@ -0,0 +1,13 @@
- Refuse Ontario 511 catalog redirects and keep the key-bearing request URL
and upstream error text out of failure logs, which now name only the error
class and system code; release the body of an error response. Credit
Ontario 511 with the Open Government Licence – Ontario attribution statement
(asturksever, #1266).
- Abort the other Tarktee (Estonia) DATEX request as soon as one fails, before
its headers or while its body is read, instead of waiting out the shared
30 s deadline; a response that arrives after the failure is released too.
The Lithuania pack reads its two catalogs through the same helper
(asturksever, #1266).
- Frame the calibration gizmo before the CCTV QA drag regression drags it;
default autofocus could leave the arrow shorter than the 10 px grab
threshold (asturksever, #1266).
@@ -0,0 +1,2 @@
- Stop the browser caching a refused Google Places or Geocoding answer, so a
fixed key works at once (Rudy Celekli, @rudycelekli, #1570).
@@ -0,0 +1,5 @@
- The CCTV Street View fallback answers only for cameras the server lists, at
their catalog position; a request sets only the view. Bundled seed cameras
get the placeholder. A per-IP limit on `GEV_RATELIMIT_GOOGLE_PER_MIN`
(default 120) keeps the Google key from serving as an unmetered Street View
service (asturksever, #1602).
@@ -0,0 +1,5 @@
- Keep the CCTV camera picked with NEAREST, FOCUS or by a scene when it is
the camera CCTV switched to after the view moved. The pick did not stick:
NEAREST's own flight, or the next pan, switched to another camera, so the
panel showed a camera nobody had picked. To keep the camera the list
already shows, press FOCUS (gekh, #1702).
+38 -8
View File
@@ -2025,7 +2025,13 @@ and follows redirects only on the camera's origin. Catalog image URLs, and
redirects on those hosts, are pinned by parsed URL to the agency's image
directory. Every `/api/cctv` route refuses cross-site browser requests but
accepts forwarding headers, so cameras work behind a reverse proxy; a forwarded
request skips the key-bearing Street View fallback and gets the placeholder.
request skips the key-bearing Street View fallback and gets the placeholder. The
fallback answers only for cameras the server lists, at their catalog position,
so position calibration never reaches it. Heading, field of view (clamped to
20–120°) and pitch (clamped to −40–20°) come from the request, else from the
catalog pose. A bundled seed camera, shown when the server catalog is empty,
gets the placeholder. The fallback counts in its own per-IP Google window (see
the cost-bearing proxies below).
The CCTV layer draws only the cameras on or near the screen. Zoomed out, a
ground cell with two or more cameras shows one marker at their mean position. A
@@ -2055,6 +2061,19 @@ in view finishes loading. The camera picker lists the active camera first, then
one camera per visible cell, preferring healthy feeds. It ends with the count of
catalog cameras missing from the list.
Turning CCTV on opens on the camera nearest the view. Until someone picks a
camera, each settled move, such as a shared view's flight landing, moves it to
the camera then nearest the view. Automatic selection ends at the first pick:
NEAREST, PREV or NEXT, FOCUS, a camera chosen in the picker, a click on a
camera, a click on empty ground that deselects, an auto-hop step, or a call to
`selectCamera`, `focusNearest`, `focusNearestToPoint` or
`setParams({ selectedCameraId })`, which voice and scenes use. Picking the
camera that is already active counts. The picker shows the automatic camera
as its own option, and a native list sends no change for the option it
already shows, so the list cannot pick that camera: FOCUS does, and so does
NEAREST when that camera is the nearest. Closing the list without a choice
keeps automatic selection.
Traffic owns its rendered-surface observer only while enabled, removes it on disable/destroy, and rebuilds no cached heights on frames where the surface identity is unchanged. Its dots and heat-lines are two renderer draw sets, it animates on the frame port's preRender, and it follows the camera through the camera port at its own 5% threshold rather than the shared `percentageChanged` ledger. The surface identity is the globe port's `surfaceVersion()` and tile readiness its `isSettled('surface')`, so the layer no longer reaches Cesium. Heat-lines are always asked for draped: where the Cesium renderer cannot drape (no depth textures) its draw port draws them (and every other draped line) over the surface, in their colour where the surface hides them and hidden past the horizon, recolouring a line in place when only its colour changes, rather than the layer dropping them.
Traffic and bikeshare expose factories through `./layers/traffic` and
@@ -3793,6 +3812,9 @@ unaffected. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` ov
those; exactly `0` disables the limiter, while a value that cannot be read as a
number falls back to the default rather than to unlimited. Over-limit requests
receive a sanitized `429` with `Retry-After: 5` and never reach the provider.
The CCTV Street View fallback counts in a separate window with the Google cap.
An IPv6 /64 counts as one client. Over the cap, a frame gets the placeholder
image instead of a `429`.
Typed chat (`/api/openai/chat`, `/api/gemini/chat`) counts in its own window,
set by `GEV_RATELIMIT_OPENAI_CHAT_PER_MIN` and `GEV_RATELIMIT_GEMINI_CHAT_PER_MIN`.
Left unset, each takes its provider's resolved cap — `GEV_RATELIMIT_OPENAI_PER_MIN`
@@ -7818,7 +7840,7 @@ silently demoting every later lookup for the session.
requires a developer key, so the pack needs `ONTARIO_511_API_KEY` and is skipped without one
(#1228). All sources are RAW PRIOR
poses, and the layer is stills-first.
sources. Fintraffic Finland road weather cameras (2026-09-13; cap 300, `CCTV_FINTRAFFIC_MAX_SOURCES`,
Fintraffic Finland road weather cameras (2026-09-13; cap 300, `CCTV_FINTRAFFIC_MAX_SOURCES`,
kill switch `CCTV_FINTRAFFIC_ENABLED=0`) are the fourth pack: one keyless GeoJSON station list
covering the whole country, where one _preset_ (a station's fixed view) is one camera — 806
GATHERING stations carry 2,256 in-collection presets, prioritized to 300 against seven anchors
@@ -7833,11 +7855,17 @@ silently demoting every later lookup for the session.
empty keeps the old catalog and is retried after another 15 minutes, not on the next
request. Estonia's Tarktee DATEX feeds, which answer in 2 to 16.6 s, get 30 s
(`TARKTEE_FETCH_TIMEOUT_MS`) where other packs get 15 s.
Both Tarktee XML catalogs refuse redirects and have a 4 MiB per-document
limit, enforced against declared length and streamed bytes. A rejected fetch,
unsuccessful response, or body-read failure releases both available response
bodies; read failures also cancel an active sibling reader. Cameras sharing
an image number are deduplicated before the Tarktee pack cap is applied.
Ontario catalog requests refuse redirects and log only the error class and
system code for transport/JSON failures, never the request URL or key.
Both Tarktee XML catalogs share that 30 s deadline, refuse redirects and
have a 4 MiB per-document limit, enforced against declared length and
streamed bytes. A failed request, unsuccessful response or body read aborts
the other feed at once and releases every response body, including one that
arrives after the failure. The Lithuania pack reads its two catalogs
through the same helper (`readCatalogsTogether`,
`server/providers/cctv/catalogPair.js`) with its 15 s deadline. An Ontario
error response has its body released. Cameras sharing an image number are
deduplicated before the Tarktee pack cap is applied.
- **CCTV v3 UX — viewshed + calibration gizmo** (built 2026-07-05 and field
validated 2026-07-21): the COVERAGE toggle is a
tri-state cycle `OFF → ON → VIEWSHED`; viewshed mode renders each visible camera's frustum
@@ -8625,7 +8653,8 @@ Providers normalize coordinates, canonical name, label, place types and optional
bounds. Camera framing, nearby landmark recovery and footprint selection remain
in their consumers, including the Capitol identity/containment safeguards.
Radio keeps localized country names in labels rather than station filters.
Nearby/text-search endpoints retain their existing behavior.
Nearby/text-search cache and error rules are listed under Proxy/Security
Baseline below.
**Google geocoding is a server route, not a browser call** (#363). Forward and
reverse lookups go to `/api/google/geocode` and `/api/google/reverse-geocode`,
@@ -8945,6 +8974,7 @@ apply to `public/` assets or built-preview serving.
- `AISSTREAM_API_KEY` is server-side only; the browser reads the same-origin `/api/vessels` cache.
- `/api/google/nearby-places` keeps the Google key out of Places requests issued for voice scene context.
- `/api/google/text-search` keeps the Google key server-side for view-biased Places recovery used by annotation resolution.
- The Places and geocode routes cache only an answer (`private, max-age=300`). For geocode, that means an HTTP success with `OK` or `ZERO_RESULTS`, because Google sends `REQUEST_DENIED` and `OVER_QUERY_LIMIT` as HTTP 200. Every other reply, including an unreadable Places 200 (now a 502), is `no-store`.
- `/api/overpass` is bounded by body/response caps, per-client/global rate limits, concurrency limits, operator-configured upstreams, in-flight dedupe, cache bounds, and static validation that every selector is spatially bounded.
- `/api/military-installations` uses an independent limiter with the same 90-per-client/300-global one-minute bounds, so viewport installation refreshes never consume `/api/overpass` annotation capacity.
- `/api/route` proxies bounded OSRM route requests for annotation routes, with profile allowlisting, distance caps, response caps, caching, and sanitized "no route found" errors; routing outages return 502.
+1
View File
@@ -148,6 +148,7 @@
"server/providers/cctv/groundHeights.js",
"server/providers/cctv/catalog.js",
"server/providers/cctv/lithuania.js",
"server/providers/cctv/catalogPair.js",
"src/data/cctvLithuania.test.mjs",
"server/providers/cctv/taiwan.js",
"src/data/cctvTaiwan.test.mjs",
+4 -1
View File
@@ -571,6 +571,7 @@
"server/providers/cctv/groundHeights.js",
"server/providers/cctv/catalog.js",
"server/providers/cctv/lithuania.js",
"server/providers/cctv/catalogPair.js",
"server/providers/cctv/taiwan.js",
"server/providers/cctv/quebec.js",
"server/providers/cctv/ottawa.js",
@@ -589,7 +590,9 @@
"src/data/cctvFootprint.js",
"src/sources/cctvTypes.js",
"src/sources/httpBody.js",
"src/i18n/marker.js"
"src/i18n/marker.js",
"server/providers/common/rate-limit.js",
"src/sources/rateLimit.js"
],
"external": []
},
+37
View File
@@ -1790,6 +1790,43 @@ async function main() {
gizmoStates.map((s, i) => `${GIZMO_PARTS[i]}=${s}`).join(' '),
);
// Frame the actual gizmo for the drag fixture. Default autofocus can leave
// its shaft below the 10px interaction threshold; that distant view is not
// the drag-usability scenario this regression exercises. The gizmo is drawn
// through the renderer ports, so its arrow is read from the layer's drawn
// state (plain {x,y,z} points), not from viewer.entities.
const dragFraming = await page.evaluate(async () => {
const viewer = window.__godsEyeView.viewer;
const mod = window.__godsEyeView.dataManager.layers.get('cctv').module;
const arrow = () =>
mod._cctvDrawnForTest().gizmo['cctv-gizmo-move-east']?.positions;
if (!arrow()?.length) return null;
const Cartesian3 = viewer.camera.positionWC.constructor;
const measure = () => {
const pts = arrow();
if (!pts?.length) return { shaftPx: null, rangeM: null };
const a = viewer.scene.cartesianToCanvasCoordinates(pts[0]);
const b = viewer.scene.cartesianToCanvasCoordinates(pts[pts.length - 1]);
return {
shaftPx: a && b ? Math.hypot(b.x - a.x, b.y - a.y) : null,
rangeM: Cartesian3.distance(viewer.camera.positionWC, pts[0]),
};
};
const before = measure();
const target = Cartesian3.clone(arrow()[0]);
viewer.camera.cancelFlight();
viewer.camera.lookAt(target, new Cartesian3(0, -200, 180));
viewer.camera.lookAtTransform(viewer.camera.transform.constructor.IDENTITY);
viewer.scene.requestRender();
const frames = await window.__qaRenderFrames(3, 10000);
return { before, after: measure(), frames };
});
record(
'drag fixture frames the east arrow above the 10px grab threshold',
dragFraming?.after?.shaftPx >= 10,
JSON.stringify(dragFraming),
);
// Synthetic CDP drag on the EAST arrow — the owner's field-test
// regression. Sample effective elevation after every mouse move: it must
// stay frozen until release, with no transient sampleHeight calls.
+36 -13
View File
@@ -21,8 +21,13 @@ import { PROXY_SIGNALS } from '../../src/localRequestGate.mjs';
import { createHlsPuller } from './cctv/stream.js';
import { isMlitRiverAdjustmentImage } from './cctv/worldCameras.js';
import { ottawaAuthorizedFrameUrl } from './cctv/ottawa.js';
import { googleServerApiKey } from './places/google-key.js';
import {
GOOGLE_DEFAULT_PER_MIN,
googleServerApiKey,
} from './places/google-key.js';
import { clientNetworkKey, makeCostRateLimiter } from './common/rate-limit.js';
export { CCTV_FRAME_FETCH_TIMEOUT_MS, fetchCctvImageFromUpstream };
/**
* Vite plugin: CCTV camera proxy with source registry, frame/media serving,
* fallback chain (upstream -> Street View -> synthetic SVG), and health tracking.
@@ -198,17 +203,31 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
};
};
let streetViewBudget; // undefined = not built yet; null = the explicit 0 opt-out
/**
* Fetch a Google Street View static image as a fallback frame. Server-side
* call, never reaches the browser — prefers GOOGLE_MAPS_SERVER_API_KEY
* (#33: a key scoped to Street View Static/Places, restricted by server IP
* rather than HTTP referrer) and falls back to the browser-exposed
* GOOGLE_MAPS_API_KEY for setups that haven't split the two yet.
* Each IP (or IPv6 /64) has its own GEV_RATELIMIT_GOOGLE_PER_MIN window,
* separate from Places. Setting it to 0 disables the limit.
*/
const streetViewFallback = async ({ lat, lon, heading, fov, pitch }) => {
const streetViewFallback = async (
client,
{ lat, lon, heading, fov, pitch },
) => {
const streetViewKey = googleServerApiKey();
if (!streetViewKey || !Number.isFinite(lat) || !Number.isFinite(lon))
return null;
// Built on first use, after the standalone env has loaded.
if (streetViewBudget === undefined)
streetViewBudget = makeCostRateLimiter(
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
GOOGLE_DEFAULT_PER_MIN,
);
if (streetViewBudget && !streetViewBudget(client)) return null;
try {
const sv = new URL('https://maps.googleapis.com/maps/api/streetview');
sv.searchParams.set('size', '960x540');
@@ -264,6 +283,8 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
// keyless gate keeps the routes reachable behind a reverse proxy; the
// key-bearing Street View fallback below has its own check.
if (admitSameSiteKeyless(req, res)) return;
// Read before any await: a socket the client has closed has no address.
const client = clientNetworkKey(req);
const sources = await getCctvSources();
const sourceById = new Map(
sources.map((source) => [source.id, source]),
@@ -635,8 +656,7 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
const source = sourceById.get(cameraId);
const label = url.searchParams.get('label') || source?.name || cameraId;
const city = url.searchParams.get('city') || source?.city || '';
const lat = Number(url.searchParams.get('lat') || source?.lat);
const lon = Number(url.searchParams.get('lon') || source?.lon);
// The query sets only the view, the pose the browser draws.
const heading = Number(
url.searchParams.get('heading') || source?.headingDeg,
);
@@ -731,18 +751,21 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
// request (whose client this server cannot see) gets the placeholder,
// as if no key were configured. Any forwarding header counts, even an
// empty one.
// It also answers only for catalog cameras, at the catalog position;
// the query never sets the location.
const forwarded = PROXY_SIGNALS.some(
(name) => req.headers?.[name] !== undefined,
);
const sv = forwarded
? null
: await streetViewFallback({
lat,
lon,
heading,
fov,
pitch,
});
const sv =
forwarded || !source
? null
: await streetViewFallback(client, {
lat: source.lat,
lon: source.lon,
heading,
fov,
pitch,
});
if (sv?.ok) {
setHealth(cameraId, {
status: 'degraded',
+53
View File
@@ -0,0 +1,53 @@
import { readResponseTextCapped } from '../common/http.js';
/**
* Read catalogs that are only useful together, because the pack joins them.
*
* The requests share one deadline and refuse redirects. Whichever fails first
* (before its headers, on its status, or while its capped body is read or
* parsed) aborts the others at once, so a stalled feed is not waited on until
* the deadline. Every response body is released, including one that arrives
* after another feed has already failed.
*
* @param {Array<{url: string, label: string}>} feeds `label` names the feed
* in the error ("locations download failed: 503").
* @param {{accept: string, maxBytes: number, timeoutMs: number,
* parse?: (text: string) => unknown}} options
* @returns {Promise<unknown[]>} The parsed bodies (texts without `parse`), in
* feed order.
*/
export async function readCatalogsTogether(
feeds,
{ accept, maxBytes, timeoutMs, parse = (text) => text },
) {
const sibling = new AbortController();
const signal = AbortSignal.any([
sibling.signal,
AbortSignal.timeout(timeoutMs),
]);
return Promise.all(
feeds.map(async ({ url, label }) => {
let response;
try {
response = await fetch(url, {
headers: { Accept: accept },
redirect: 'manual',
signal,
});
signal.throwIfAborted();
if (!response.ok) {
throw new Error(
response.status >= 300 && response.status < 400
? `${label} catalog redirected; redirects are not followed`
: `${label} download failed: ${response.status}`,
);
}
return parse(await readResponseTextCapped(response, maxBytes, signal));
} catch (error) {
sibling.abort();
await response?.body?.cancel().catch(() => {});
throw error;
}
}),
);
}
+13 -38
View File
@@ -4,7 +4,7 @@ import {
fallbackHeadingFromId,
prioritizeSources,
} from './normalize.js';
import { readResponseJsonCapped } from '../common/http.js';
import { readCatalogsTogether } from './catalogPair.js';
const BASE = 'https://eismoinfo.lt/eismoinfo-backend';
export const LITHUANIA_CAMERAS_URL = `${BASE}/camera-info-table`;
@@ -81,46 +81,21 @@ export function lithuaniaCameraToSource(row, feature) {
};
}
async function readCatalog(url, sibling) {
const signal = AbortSignal.any([
AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
sibling.signal,
]);
try {
const response = await fetch(url, {
headers: { Accept: 'application/json' },
redirect: 'manual',
signal,
});
if (!response.ok) {
await response.body?.cancel().catch(() => {});
throw new Error(
response.status >= 300 && response.status < 400
? 'catalog redirected; redirects are not followed'
: `Catalog HTTP ${response.status}`,
);
}
return await readResponseJsonCapped(
response,
LITHUANIA_MAX_CATALOG_BYTES,
signal,
);
} catch (error) {
// The join needs both catalogs; release the other request at once
// rather than leaving its body open until the timeout.
sibling.abort();
throw error;
}
}
/** Two bounded requests per shared catalog refresh, no API key required. */
export async function loadLithuaniaSources() {
try {
const sibling = new AbortController();
const [rows, layers] = await Promise.all([
readCatalog(LITHUANIA_CAMERAS_URL, sibling),
readCatalog(LITHUANIA_LOCATIONS_URL, sibling),
]);
const [rows, layers] = await readCatalogsTogether(
[
{ url: LITHUANIA_CAMERAS_URL, label: 'cameras' },
{ url: LITHUANIA_LOCATIONS_URL, label: 'locations' },
],
{
accept: 'application/json',
maxBytes: LITHUANIA_MAX_CATALOG_BYTES,
timeoutMs: CCTV_SOURCE_FETCH_TIMEOUT_MS,
parse: JSON.parse,
},
);
if (!Array.isArray(rows) || !Array.isArray(layers)) return [];
const features = layers.find((layer) => layer?.layer === 'VKR')?.features;
if (!Array.isArray(features)) return [];
+92
View File
@@ -127,3 +127,95 @@ for (const status of [429, 500, 503]) {
assert.doesNotMatch(lines[0], /secret-key-value/);
});
}
for (const failure of ['transport', 'invalid-json']) {
test(`Ontario ${failure} failures cannot log reflected keys or request URLs`, async (t) => {
const lines = captureConsole(t);
const key = 'synthetic secret&/=%value';
t.mock.method(globalThis, 'fetch', async (url) => {
if (failure === 'transport')
throw new Error(`request failed: ${url} ${key}`);
return new Response(`not JSON: ${url} ${key}`);
});
assert.deepEqual(
await loadOntarioSourcesFromOpenData({}, { ONTARIO_511_API_KEY: key }),
[],
);
assert.equal(lines.length, 1);
assert.match(lines[0], /Ontario 511 camera download error/);
assert.equal(lines[0].includes(key), false);
assert.equal(lines[0].includes('synthetic'), false);
assert.equal(lines[0].includes('https://'), false);
});
}
test('Ontario encodes reserved key characters and refuses credential-bearing redirects', async (t) => {
const lines = captureConsole(t);
const key = 'synthetic &/=% value';
let request;
t.mock.method(globalThis, 'fetch', async (url, init) => {
request = { url: new URL(url), init };
return Response.json([ROW]);
});
const cameras = await loadOntarioSourcesFromOpenData(
{},
{ ONTARIO_511_API_KEY: key },
);
assert.equal(request.url.searchParams.get('key'), key);
assert.equal(request.init.redirect, 'error');
assert.equal(cameras.length, 1);
assert.equal(JSON.stringify(cameras).includes('synthetic'), false);
assert.equal(lines.join(' ').includes('synthetic'), false);
});
test('Ontario names a refused redirect and a system code without the request text', async (t) => {
const lines = captureConsole(t);
const key = 'synthetic-secret';
const failures = [
// The shape Node's fetch gives a refused redirect and a DNS failure.
new TypeError('fetch failed', {
cause: new Error('unexpected redirect'),
}),
new TypeError('fetch failed', {
cause: Object.assign(new Error(`getaddrinfo ENOTFOUND ${key}`), {
code: 'ENOTFOUND',
}),
}),
];
t.mock.method(globalThis, 'fetch', async () => {
throw failures.shift();
});
for (let i = 0; i < 2; i++) {
assert.deepEqual(
await loadOntarioSourcesFromOpenData({}, { ONTARIO_511_API_KEY: key }),
[],
);
}
assert.deepEqual(lines, [
'[CCTV] Ontario 511 camera download error (TypeError, redirect refused)',
'[CCTV] Ontario 511 camera download error (TypeError ENOTFOUND)',
]);
});
test('Ontario releases the body of an error response', async (t) => {
captureConsole(t);
let cancelled = false;
t.mock.method(
globalThis,
'fetch',
async () =>
new Response(
new ReadableStream({
cancel() {
cancelled = true;
},
}),
{ status: 503 },
),
);
assert.deepEqual(
await loadOntarioSourcesFromOpenData({}, { ONTARIO_511_API_KEY: 'k' }),
[],
);
assert.equal(cancelled, true);
});
+34 -70
View File
@@ -43,6 +43,7 @@ import {
TARKTEE_IMAGE_ORIGIN,
DEFAULT_TARKTEE_MAX_SOURCES,
TARKTEE_MAX_CATALOG_BYTES,
TARKTEE_FETCH_TIMEOUT_MS,
TARKTEE_ANCHORS,
DEFAULT_WARENDORF_SOURCE_FILE,
WARENDORF_IMAGE_ORIGINS,
@@ -82,7 +83,6 @@ import {
HK_ANCHORS,
HK_MAX_CATALOG_BYTES,
CCTV_SOURCE_FETCH_TIMEOUT_MS,
TARKTEE_FETCH_TIMEOUT_MS,
DELDOT_CCTV_URL,
DEFAULT_DELDOT_MAX_SOURCES,
DELDOT_ANCHORS,
@@ -172,6 +172,7 @@ import {
pinnedCameraUrl,
} from './normalize.js';
import { directionToHeading } from '../../../src/data/directionText.js';
import { readCatalogsTogether } from './catalogPair.js';
import {
readResponseBytesCapped,
readResponseJsonCapped,
@@ -561,6 +562,22 @@ export function _resetOntarioKeyWarningForTest() {
ontarioKeyMissingLogged = false;
}
/**
* What failed in an Ontario download, without upstream text: the error class,
* plus the system code (`ENOTFOUND`) or a refused redirect when there is one.
* @param {unknown} error
* @returns {string}
*/
function ontarioFailureKind(error) {
const name = /^[A-Za-z]{1,40}$/.test(error?.name) ? error.name : 'Error';
if (error?.cause?.message === 'unexpected redirect')
return `${name}, redirect refused`;
const code = error?.cause?.code;
return typeof code === 'string' && /^[A-Z][A-Z0-9_]{1,40}$/.test(code)
? `${name} ${code}`
: name;
}
/**
* Fetch Ontario 511 CCTV cameras. The camera list comes from the 511 API,
* which now requires a free developer key (511on.ca/developers/doc, "Requires a
@@ -593,6 +610,7 @@ export async function loadOntarioSourcesFromOpenData(
url.searchParams.set('key', key);
const resp = await fetch(url, {
headers: { Accept: 'application/json' },
redirect: 'error',
signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
});
if (!resp.ok) {
@@ -604,6 +622,7 @@ export async function loadOntarioSourcesFromOpenData(
console.warn(
`[CCTV] Ontario 511 camera download failed: ${resp.status}${keyHint}`,
);
await resp.body?.cancel().catch(() => {});
return [];
}
const rows = await resp.json();
@@ -658,7 +677,7 @@ export async function loadOntarioSourcesFromOpenData(
url: view.url,
snapshotUrl: view.url,
sourceKind: 'ontario-511-open-data',
license: 'Open Government Licence - Ontario',
license: 'Open Government Licence – Ontario',
});
}
@@ -677,9 +696,10 @@ export async function loadOntarioSourcesFromOpenData(
);
return prioritized;
} catch (error) {
// Transport and JSON errors may reflect the credential-bearing request,
// so only the error class and a system code are logged, never the text.
console.warn(
'[CCTV] Ontario 511 camera download error:',
error?.message || error,
`[CCTV] Ontario 511 camera download error (${ontarioFailureKind(error)})`,
);
return [];
}
@@ -1279,73 +1299,17 @@ export function parseTarkteeDatexImages(xml) {
*/
export async function loadTarkteeSourcesFromDatex() {
try {
const responses = await Promise.allSettled([
fetch(TARKTEE_LOCATIONS_URL, {
headers: { Accept: 'application/xml,text/xml,*/*' },
redirect: 'manual',
signal: AbortSignal.timeout(TARKTEE_FETCH_TIMEOUT_MS),
}),
fetch(TARKTEE_IMAGES_URL, {
headers: { Accept: 'application/xml,text/xml,*/*' },
redirect: 'manual',
signal: AbortSignal.timeout(TARKTEE_FETCH_TIMEOUT_MS),
}),
]);
const [locResp, imgResp] = responses.map((result) =>
result.status === 'fulfilled' ? result.value : undefined,
const [locXml, imgXml] = await readCatalogsTogether(
[
{ url: TARKTEE_LOCATIONS_URL, label: 'locations' },
{ url: TARKTEE_IMAGES_URL, label: 'images' },
],
{
accept: 'application/xml,text/xml,*/*',
maxBytes: TARKTEE_MAX_CATALOG_BYTES,
timeoutMs: TARKTEE_FETCH_TIMEOUT_MS,
},
);
// Both responses are fetched in parallel; if either is rejected or overflows
// its byte cap, both bodies must be released so neither transport leaks.
const discardBoth = async () => {
await Promise.allSettled([
locResp?.body?.cancel(),
imgResp?.body?.cancel(),
]);
return [];
};
const rejected = responses.find((result) => result.status === 'rejected');
if (rejected) {
await discardBoth();
throw rejected.reason;
}
if (
(locResp.status >= 300 && locResp.status < 400) ||
(imgResp.status >= 300 && imgResp.status < 400)
) {
console.warn(
'[CCTV] Tarktee DATEX catalog redirected; redirects are not followed',
);
return discardBoth();
}
if (!locResp.ok) {
console.warn('[CCTV] Tarktee locations download failed:', locResp.status);
return discardBoth();
}
if (!imgResp.ok) {
console.warn('[CCTV] Tarktee images download failed:', imgResp.status);
return discardBoth();
}
let locXml;
let imgXml;
const readAbort = new AbortController();
try {
[locXml, imgXml] = await Promise.all([
readResponseTextCapped(
locResp,
TARKTEE_MAX_CATALOG_BYTES,
readAbort.signal,
),
readResponseTextCapped(
imgResp,
TARKTEE_MAX_CATALOG_BYTES,
readAbort.signal,
),
]);
} catch (error) {
readAbort.abort();
await discardBoth();
throw error;
}
const locations = parseTarkteeDatexLocations(locXml);
const images = parseTarkteeDatexImages(imgXml);
if (!locations.size || !images.size) {
+8
View File
@@ -1,5 +1,13 @@
import { resolveGoogleServerKey } from '../../../scripts/google-server-key.mjs';
/**
* Default GEV_RATELIMIT_GOOGLE_PER_MIN for the Places and Geocoding window
* and the separate CCTV Street View window, in requests per minute per IP.
* The Pinokio build sets the same default in pinokio/_ENVIRONMENT.
* `.env.example` suggests 60 for hosts other than localhost.
*/
export const GOOGLE_DEFAULT_PER_MIN = 120;
/**
* Optional Google place context is an empty capability when no key is present,
* not a server outage. Returning 200 keeps a deliberately keyless session out
+29 -25
View File
@@ -1,4 +1,5 @@
import {
GOOGLE_DEFAULT_PER_MIN,
googleServerApiKey,
keylessGooglePlacesResponse,
keylessGoogleGeocodeResponse,
@@ -15,17 +16,7 @@ import {
// undefined = not built yet; null = the explicit 0 opt-out; fn = active limiter
let _googleRateLimiter;
/**
* Requests/min/IP applied when GEV_RATELIMIT_GOOGLE_PER_MIN is unset. This is
* the value the Pinokio build already ships (pinokio/_ENVIRONMENT), so the
* packaged app keeps behaving exactly as it does today and only an
* unconfigured server changes — from unlimited to what the product already
* runs with. Places calls are user-driven (a search, a nearby lookup, an
* installation probe) rather than polled, so this sits far above what the app
* generates and bites only a caller enumerating the endpoint. `.env.example`
* suggests a tighter 60 when the host is not localhost; that still applies.
*/
export const GOOGLE_DEFAULT_PER_MIN = 120;
export { GOOGLE_DEFAULT_PER_MIN };
/** Google cost endpoints (nearby-places, text-search, geocoding). Null only when set to 0. */
function googleRateLimiter() {
@@ -71,6 +62,8 @@ const GEOCODE_UPSTREAM_TIMEOUT_MS = 10_000;
const GEOCODE_MAX_RESPONSE_BYTES = 256 * 1024;
const GEOCODE_ANSWER_STATUSES = new Set(['OK', 'ZERO_RESULTS']);
/** Validate a free-text geocoder query before consuming request quota. */
export function validateGeocodeAddress(searchParams) {
const address = String(searchParams.get('address') || '').trim();
@@ -114,6 +107,10 @@ export function googlePlacesContextProxy({
// Gate first, like the OpenAI routes: a cross-site caller learns nothing
// about this endpoint's method surface.
if (admitSameSite(req, res)) return;
// Default to no-store. The browser caches only the 200 answer below.
// A cached refusal (bad key, API off, quota) would outlive the key fix
// by five minutes.
res.setHeader('Cache-Control', 'no-store');
if (req.method !== 'GET') {
res.statusCode = 405;
res.setHeader('Content-Type', 'application/json');
@@ -129,7 +126,6 @@ export function googlePlacesContextProxy({
if (keyless) {
res.statusCode = keyless.statusCode;
res.setHeader('Content-Type', 'application/json');
res.setHeader('Cache-Control', 'no-store');
res.end(JSON.stringify(keyless.payload));
return;
}
@@ -194,18 +190,21 @@ export function googlePlacesContextProxy({
}),
},
);
const data = await response.json().catch(() => ({}));
const places = projectNearbyPlaces(data, latitude, longitude);
const data = await response.json().catch(() => null);
// A 200 we could not read is an upstream failure, not an empty answer.
if (response.ok && !data)
throw new Error('Google Places request failed');
const places = projectNearbyPlaces(data || {}, latitude, longitude);
res.statusCode = response.ok ? 200 : response.status;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'private, max-age=300');
if (response.ok) res.setHeader('Cache-Control', 'private, max-age=300');
res.end(
JSON.stringify({
places,
error: response.ok
? null
: data.error?.message || 'Google Places request failed',
: data?.error?.message || 'Google Places request failed',
}),
);
} catch (error) {
@@ -222,11 +221,12 @@ export function googlePlacesContextProxy({
// Text Search: resolve a named landmark/POI to a real coordinate, biased to
// the view. Geocoding scatters obscure monument/POI names across the city;
// a view-biased Text Search lands on the actual feature. Same key, field
// mask, throttle, and `places: []` error contract as nearby-places above.
// a view-biased Text Search lands on the actual feature. Same key, throttle,
// cache rule, and `places: []` error contract as nearby-places above.
middlewares.use('/api/google/text-search', async (req, res) => {
// Paid like nearby-places, so gated the same way, first.
if (admitSameSite(req, res)) return;
res.setHeader('Cache-Control', 'no-store');
if (req.method !== 'GET') {
res.statusCode = 405;
res.setHeader('Content-Type', 'application/json');
@@ -242,7 +242,6 @@ export function googlePlacesContextProxy({
if (keyless) {
res.statusCode = keyless.statusCode;
res.setHeader('Content-Type', 'application/json');
res.setHeader('Cache-Control', 'no-store');
res.end(JSON.stringify(keyless.payload));
return;
}
@@ -315,18 +314,20 @@ export function googlePlacesContextProxy({
}),
},
);
const data = await response.json().catch(() => ({}));
const places = projectTextSearchPlaces(data, latitude, longitude);
const data = await response.json().catch(() => null);
if (response.ok && !data)
throw new Error('Google Places request failed');
const places = projectTextSearchPlaces(data || {}, latitude, longitude);
res.statusCode = response.ok ? 200 : response.status;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'private, max-age=300');
if (response.ok) res.setHeader('Cache-Control', 'private, max-age=300');
res.end(
JSON.stringify({
places,
error: response.ok
? null
: data.error?.message || 'Google Places request failed',
: data?.error?.message || 'Google Places request failed',
}),
);
} catch (error) {
@@ -366,6 +367,7 @@ export function googlePlacesContextProxy({
// Gate first, like the Places routes above: a cross-site caller learns
// nothing about this endpoint's method surface.
if (admitSameSite(req, res)) return;
res.setHeader('Cache-Control', 'no-store');
if (req.method !== 'GET') {
refuse(405, 'Method not allowed');
return;
@@ -379,7 +381,6 @@ export function googlePlacesContextProxy({
if (keyless) {
res.statusCode = keyless.statusCode;
res.setHeader('Content-Type', 'application/json');
res.setHeader('Cache-Control', 'no-store');
res.end(JSON.stringify(keyless.payload));
return;
}
@@ -446,7 +447,10 @@ export function googlePlacesContextProxy({
res.statusCode = response.ok ? 200 : response.status;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'private, max-age=300');
// Geocoding reports a denied key or spent quota as HTTP 200 with a
// status of its own; only an answer is worth keeping.
if (response.ok && GEOCODE_ANSWER_STATUSES.has(data.status))
res.setHeader('Cache-Control', 'private, max-age=300');
res.end(JSON.stringify(data));
} catch {
// Fixed public text: a transport, URL or runtime message would
+1
View File
@@ -335,6 +335,7 @@ test('the Google places route throttles by default and keeps its places[] contra
const overLimit = await request(handler, { url, method: 'GET' });
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
assert.equal(overLimit.headers['cache-control'], 'no-store');
// Every error response on this route carries places[]; the client reads it
// unconditionally, so a throttled reply must not be the one shape that omits it.
assert.deepEqual(JSON.parse(overLimit.body).places, []);
+155 -54
View File
@@ -12,6 +12,14 @@ import {
parseTarkteeDatexLocations,
} from '../../server/providers/cctv/sources.js';
/**
* Disarm the 30 s deadline, so a loader that only stops at that deadline
* fails at the test timeout instead of passing late.
*/
function withoutDeadline(t) {
t.mock.method(AbortSignal, 'timeout', () => new AbortController().signal);
}
/** Silence the loaders' progress and failure logging. */
function quiet(t) {
t.mock.method(console, 'log', () => {});
@@ -97,6 +105,29 @@ test('Tarktee DATEX image parser rejects non-official hosts', () => {
assert.equal(images.size, 0);
});
test('Tarktee DATEX requests wait 30 s, past the slow answers measured live', async (t) => {
const timeouts = [];
const timeout = AbortSignal.timeout;
t.mock.method(AbortSignal, 'timeout', (ms) => {
timeouts.push(ms);
return timeout.call(AbortSignal, ms);
});
t.mock.method(console, 'log', () => {});
t.mock.method(
globalThis,
'fetch',
async (url) =>
new Response(
String(url).includes('roadCameraLocations')
? SAMPLE_LOCATIONS_XML
: SAMPLE_IMAGES_XML,
),
);
const cameras = await loadTarkteeSourcesFromDatex();
assert.deepEqual(timeouts, [30_000]);
assert.equal(cameras.length, 1);
});
test('Tarktee loader deduplicates cameras sharing an image number before the cap', async (t) => {
quiet(t);
const dupLocationsXml = `<?xml version="1.0" encoding="UTF-8"?>
@@ -168,46 +199,6 @@ test('Tarktee loader refuses a redirect and cancels both response bodies', async
assert.equal(cancelled.images, true, 'sibling 200 body is also cancelled');
});
test('Tarktee loader cancels fulfilled bodies when either fetch rejects', async (t) => {
for (const failedUrl of [TARKTEE_LOCATIONS_URL, TARKTEE_IMAGES_URL]) {
for (const delayedSibling of [false, true]) {
await t.test(
`${failedUrl}, delayed sibling: ${delayedSibling}`,
async (t) => {
quiet(t);
let cancelled = false;
const response = new Response(
new ReadableStream({
start(controller) {
controller.enqueue(new TextEncoder().encode('<xml/>'));
},
cancel() {
cancelled = true;
},
}),
);
t.mock.method(globalThis, 'fetch', async (url) => {
if (String(url) === failedUrl) throw new TypeError('fetch failed');
if (delayedSibling) await new Promise(setImmediate);
return response;
});
try {
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.equal(
cancelled,
true,
'fulfilled sibling body is cancelled',
);
} finally {
await response.body.cancel();
}
},
);
}
}
});
test('Tarktee loader cancels the sibling body when one feed fails or overflows the byte cap', async (t) => {
quiet(t);
let siblingCancelled = false;
@@ -264,25 +255,135 @@ test('Tarktee loader cancels the sibling body when one feed fails or overflows t
);
});
test('Tarktee DATEX requests wait 30 s, past the slow answers measured live', async (t) => {
const timeouts = [];
const timeout = AbortSignal.timeout;
t.mock.method(AbortSignal, 'timeout', (ms) => {
timeouts.push(ms);
return timeout.call(AbortSignal, ms);
});
t.mock.method(console, 'log', () => {});
for (const failingUrl of [TARKTEE_LOCATIONS_URL, TARKTEE_IMAGES_URL]) {
for (const lateSibling of [false, true]) {
test(
`Tarktee cancels a ${lateSibling ? 'late' : 'received'} sibling when ${failingUrl} rejects before headers`,
{ timeout: 2000 },
async (t) => {
quiet(t);
withoutDeadline(t);
let cancelled = false;
let siblingSignal;
let release;
const gate = new Promise((resolve) => {
release = resolve;
});
t.mock.method(globalThis, 'fetch', async (url, init) => {
if (String(url) === failingUrl)
throw new Error('headers unavailable');
siblingSignal = init.signal;
// A late transport answers once the sibling is aborted, or on release.
if (lateSibling)
await Promise.race([
gate,
new Promise((resolve) =>
init.signal.addEventListener('abort', resolve, { once: true }),
),
]);
return new Response(
new ReadableStream({
start(controller) {
controller.enqueue(new TextEncoder().encode('<xml/>'));
},
cancel() {
cancelled = true;
},
}),
);
});
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
// Also exercise a response arriving after abort, as a queued transport
// completion can race the rejection owner even when fetch honors abort.
if (lateSibling) {
release();
await new Promise((resolve) => setImmediate(resolve));
}
assert.equal(cancelled, true, 'every obtained body must be cancelled');
assert.equal(
siblingSignal.aborted,
true,
'a pending transport is aborted immediately',
);
},
);
}
test(
`Tarktee aborts a pending sibling when ${failingUrl} rejects`,
{ timeout: 2000 },
async (t) => {
quiet(t);
withoutDeadline(t);
let aborted = false;
t.mock.method(globalThis, 'fetch', async (url, init) => {
if (String(url) === failingUrl) throw new Error('headers unavailable');
return new Promise((resolve, reject) => {
init.signal.addEventListener(
'abort',
() => {
aborted = true;
reject(init.signal.reason);
},
{ once: true },
);
});
});
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.equal(aborted, true, 'do not wait for the 30-second deadline');
},
);
}
test('Tarktee cancels both lengthless streams when one exceeds the byte cap', async (t) => {
quiet(t);
const cancelled = [];
t.mock.method(
globalThis,
'fetch',
async (url) =>
new Response(
String(url).includes('roadCameraLocations')
? SAMPLE_LOCATIONS_XML
: SAMPLE_IMAGES_XML,
new ReadableStream({
start(controller) {
const oversized = String(url) === TARKTEE_LOCATIONS_URL;
controller.enqueue(
new Uint8Array(oversized ? TARKTEE_MAX_CATALOG_BYTES + 1 : 1),
);
},
cancel() {
cancelled.push(String(url));
},
}),
),
);
const cameras = await loadTarkteeSourcesFromDatex();
assert.deepEqual(timeouts, [30_000, 30_000]);
assert.equal(cameras.length, 1);
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.deepEqual(
cancelled.sort(),
[TARKTEE_LOCATIONS_URL, TARKTEE_IMAGES_URL].sort(),
);
});
test(
'Tarktee aborts pending headers as soon as its sibling returns an HTTP error',
{ timeout: 2000 },
async (t) => {
quiet(t);
withoutDeadline(t);
let aborted = false;
t.mock.method(globalThis, 'fetch', async (url, init) => {
if (String(url) === TARKTEE_LOCATIONS_URL)
return new Response('unavailable', { status: 503 });
return new Promise((resolve, reject) => {
init.signal.addEventListener(
'abort',
() => {
aborted = true;
reject(init.signal.reason);
},
{ once: true },
);
});
});
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.equal(aborted, true);
},
);
+112
View File
@@ -0,0 +1,112 @@
// A body-phase failure of one Tarktee DATEX feed must abort the sibling that is
// still waiting for headers instead of leaving it until the 30 s deadline.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import http from 'node:http';
import {
TARKTEE_IMAGES_URL,
TARKTEE_LOCATIONS_URL,
TARKTEE_MAX_CATALOG_BYTES,
} from '../../server/providers/cctv/constants.js';
import { loadTarkteeSourcesFromDatex } from '../../server/providers/cctv/sources.js';
/** The sibling sends its headers this late; the loader must not wait for it. */
const SIBLING_HEADER_DELAY_MS = 2000;
/** Generous bound for "at once" on a loaded CI machine. */
const FAIL_FAST_MS = 1000;
function startServer(handlers) {
const server = http.createServer((req, res) => handlers[req.url](req, res));
return new Promise((resolve) => {
server.listen(0, '127.0.0.1', () => resolve(server));
});
}
/** Send the two DATEX URLs to the local server through the real fetch. */
function routeFetchTo(t, port) {
const realFetch = globalThis.fetch;
t.mock.method(globalThis, 'fetch', (url, init) => {
const path =
String(url) === TARKTEE_LOCATIONS_URL
? '/loc'
: String(url) === TARKTEE_IMAGES_URL
? '/img'
: null;
assert.ok(path, `unexpected url ${url}`);
return realFetch(`http://127.0.0.1:${port}${path}`, init);
});
}
/** The sibling answers late and reports whether it was cut off before headers. */
function slowSibling() {
const arrived = Promise.withResolvers();
const closed = Promise.withResolvers();
return {
arrived: arrived.promise,
closed: closed.promise,
handler(req, res) {
arrived.resolve();
const timer = setTimeout(() => {
res.writeHead(200, { 'content-type': 'application/xml' });
res.end('<d2LogicalModel/>');
}, SIBLING_HEADER_DELAY_MS);
res.on('close', () => {
clearTimeout(timer);
closed.resolve({ beforeHeaders: !res.headersSent });
});
},
};
}
const failingBodies = {
'declares a body over the byte cap': (req, res) => {
res.writeHead(200, {
'content-type': 'application/xml',
'content-length': String(TARKTEE_MAX_CATALOG_BYTES + 1),
});
res.write('<d2LogicalModel>');
},
'resets the connection mid-body': (req, res) => {
res.writeHead(200, { 'content-type': 'application/xml' });
res.write('<d2LogicalModel><payloadPublication>');
setTimeout(() => res.socket.destroy(), 20);
},
};
for (const [label, failingFeed] of Object.entries(failingBodies)) {
for (const [failing, pending] of [
['/loc', '/img'],
['/img', '/loc'],
]) {
test(`Tarktee aborts ${pending} pending headers when ${failing} ${label}`, { timeout: 10_000 }, async (t) => {
t.mock.method(console, 'log', () => {});
t.mock.method(console, 'warn', () => {});
const sibling = slowSibling();
const server = await startServer({
[failing]: (req, res) =>
sibling.arrived.then(() => failingFeed(req, res)),
[pending]: sibling.handler,
});
t.after(() => {
server.closeAllConnections();
server.close();
});
routeFetchTo(t, server.address().port);
const started = Date.now();
const cameras = await loadTarkteeSourcesFromDatex();
const elapsed = Date.now() - started;
assert.deepEqual(cameras, []);
assert.ok(
elapsed < FAIL_FAST_MS,
`loader waited ${elapsed} ms for the sibling's headers instead of aborting it`,
);
assert.deepEqual(
await sibling.closed,
{ beforeHeaders: true },
`${pending} was not aborted`,
);
});
}
}
@@ -0,0 +1,138 @@
// An explicit pick of the camera that a settled move just re-opened must end
// the opening pick. That pick goes through setActiveCamera's UNCHANGED path
// (same camera, already activated); if that path keeps the provisional flag,
// the next settled move (a pan, or the landing of NEAREST's own flight)
// switches the camera away from the one the user chose.
import assert from 'node:assert/strict';
import test from 'node:test';
import * as Cesium from 'cesium';
import cctvLayer, {
_setCctvCoverageStateForTest,
_setCctvOverlayHostForTest,
reopenNearestCamera,
} from './cctv.js';
import { bindFakeDraw } from '../testSupport/fakeDrawPort.mjs';
const BASE = {
headingDeg: 41,
pitchDeg: -5,
fovDeg: 56,
rangeM: 210,
mountHeightM: 100,
groundElevationM: 0,
city: 'London',
};
function record(id, lat, lon) {
return {
camera: { ...BASE, id, name: `Camera ${id}`, lat, lon },
coverageEntities: [],
projection: null,
activationDone: true,
groundResolved: {},
groundSamples: {},
groundPrior: 0,
frustumGeometry: null,
frustumPositions: null,
probeClampRangeM: null,
position: Cesium.Cartesian3.fromDegrees(lon, lat, 100),
};
}
function viewerAt(lat, lon) {
const viewer = {
isDestroyed: () => false,
eye: { lat, lon, height: 300 },
scene: {
canvas: { clientWidth: 1200, clientHeight: 800 },
cartesianToCanvasCoordinates: () => undefined,
primitives: { add: (p) => p, remove: () => true },
requestRender() {},
},
};
// The layer reads the eye and flies through the renderer's camera port.
viewer.draw = bindFakeDraw(viewer, {
camera: {
pose: () => ({ ...viewer.eye }),
on: () => () => {},
followed: null,
flyToBounds() {},
},
}).draw;
return viewer;
}
function moveViewer(viewer, lat, lon) {
viewer.eye = { lat, lon, height: 300 };
}
function withReopenedLayer(fn) {
_setCctvOverlayHostForTest({
setEntries() {},
setVisible() {},
clearSource() {},
});
// Activation builds the projection canvas and frame image.
const originalDocument = globalThis.document;
const originalImage = globalThis.Image;
globalThis.document = { createElement: () => ({ getContext: () => null }) };
globalThis.Image = class {};
try {
const trafalgar = record('a-trafalgar', 51.508, -0.128);
const cambridge = record('b-cambridge', 51.513, -0.129);
// enable() assigned the opening camera without activating it.
trafalgar.activationDone = false;
const viewer = viewerAt(51.5081, -0.1281);
_setCctvCoverageStateForTest({
viewer,
records: [trafalgar, cambridge],
activeCameraId: 'a-trafalgar',
enabled: true,
openingCameraProvisional: true,
});
// A settled move next to Cambridge Circus re-opens there, still as the
// opening pick.
moveViewer(viewer, 51.5131, -0.1291);
assert.equal(reopenNearestCamera(), true);
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
fn(viewer);
} finally {
globalThis.document = originalDocument;
globalThis.Image = originalImage;
_setCctvOverlayHostForTest();
_setCctvCoverageStateForTest({ enabled: false });
}
}
for (const [name, pick] of [
['NEAREST (focusNearest)', () => cctvLayer.focusNearest({ focus: false })],
['selectCamera', () => cctvLayer.selectCamera('b-cambridge')],
[
'setParams({ selectedCameraId })',
() => cctvLayer.setParams({ selectedCameraId: 'b-cambridge' }),
],
[
'focusNearestToPoint',
() => cctvLayer.focusNearestToPoint(51.513, -0.129, { focus: false }),
],
]) {
test(`an explicit ${name} pick of the re-opened camera ends the opening pick`, () => {
withReopenedLayer((viewer) => {
pick();
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
// A later settled move (a pan, or the landing of the pick's own flight)
// leaves the explicit choice alone.
moveViewer(viewer, 51.5081, -0.1281);
assert.equal(reopenNearestCamera(), false);
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
});
});
}
test('without a pick, the next settled move still follows the view', () => {
withReopenedLayer((viewer) => {
moveViewer(viewer, 51.5081, -0.1281);
assert.equal(reopenNearestCamera(), true);
assert.equal(cctvLayer.getUIState().activeCameraId, 'a-trafalgar');
});
});
+20 -17
View File
@@ -140,25 +140,28 @@ for (const [label, flag] of [
});
}
test('a failed catalog cancels the sibling request instead of waiting for the timeout', async (t) => {
t.mock.method(console, 'warn', () => {});
let siblingSignal;
t.mock.method(globalThis, 'fetch', async (url, options) => {
if (url === LITHUANIA_CAMERAS_URL) {
return new Response('', { status: 503 });
}
siblingSignal = options.signal;
// Never settles on its own; only an abort ends it.
return new Promise((_, reject) => {
options.signal.addEventListener('abort', () =>
reject(options.signal.reason),
);
for (const [failure, camerasResponse] of [
['failed', () => new Response('', { status: 503 })],
['malformed', () => new Response('<html>error page</html>')],
]) {
test(`a ${failure} catalog cancels the sibling request instead of waiting for the timeout`, async (t) => {
t.mock.method(console, 'warn', () => {});
let siblingSignal;
t.mock.method(globalThis, 'fetch', async (url, options) => {
if (url === LITHUANIA_CAMERAS_URL) return camerasResponse();
siblingSignal = options.signal;
// Never settles on its own; only an abort ends it.
return new Promise((_, reject) => {
options.signal.addEventListener('abort', () =>
reject(options.signal.reason),
);
});
});
assert.deepEqual(await loadLithuaniaSources(), []);
assert.equal(siblingSignal?.aborted, true);
assert.notEqual(siblingSignal.reason?.name, 'TimeoutError');
});
assert.deepEqual(await loadLithuaniaSources(), []);
assert.equal(siblingSignal?.aborted, true);
assert.notEqual(siblingSignal.reason?.name, 'TimeoutError');
});
}
test('a redirected catalog is reported as not followed', async (t) => {
const warnings = [];
+103 -2
View File
@@ -111,6 +111,7 @@ function mount(t, respond) {
austin: process.env.CCTV_FORCE_AUSTIN,
serverKey: process.env.GOOGLE_MAPS_SERVER_API_KEY,
browserKey: process.env.GOOGLE_MAPS_API_KEY,
rateLimit: process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
};
process.env.CCTV_SOURCES_JSON = JSON.stringify([
STILL,
@@ -125,6 +126,8 @@ function mount(t, respond) {
// than a network call this file does not model.
delete process.env.GOOGLE_MAPS_SERVER_API_KEY;
delete process.env.GOOGLE_MAPS_API_KEY;
// The default Google budget, whatever the shell sets.
delete process.env.GEV_RATELIMIT_GOOGLE_PER_MIN;
const nativeFetch = globalThis.fetch;
const requests = [];
globalThis.fetch = async (url, init = {}) => {
@@ -139,6 +142,7 @@ function mount(t, respond) {
['CCTV_FORCE_AUSTIN', before.austin],
['GOOGLE_MAPS_SERVER_API_KEY', before.serverKey],
['GOOGLE_MAPS_API_KEY', before.browserKey],
['GEV_RATELIMIT_GOOGLE_PER_MIN', before.rateLimit],
]) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
@@ -155,9 +159,9 @@ function mount(t, respond) {
});
assert.ok(handler, 'the CCTV provider must mount a middleware');
const call = async (path, headers = {}) => {
const call = async (path, headers = {}, socket) => {
const { res, done } = recordingResponse();
await handler({ url: path, headers, method: 'GET', on() {} }, res);
await handler({ url: path, headers, method: 'GET', on() {}, socket }, res);
await done;
return res;
};
@@ -528,3 +532,100 @@ test('a forwarded same-origin request reaches every CCTV route but not the Stree
assert.equal(direct.body, 'street-view');
assert.equal(streetViewCalls(), 1);
});
test('the Street View fallback frames catalog cameras only, from the catalog position', async (t) => {
const app = mount(t, ({ url }) =>
url.startsWith('https://maps.googleapis.com/')
? trackedUpstream({
headers: { 'content-type': 'image/jpeg' },
body: 'street-view',
}).response
: upstreamResponse({ status: 503 }),
);
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
const streetView = () =>
app.requests.filter(({ url }) =>
url.startsWith('https://maps.googleapis.com/'),
);
const unknown = await app.call('/frame/not-a-camera?lat=48.85&lon=2.29');
assert.equal(unknown.headers['X-CCTV-Source'], 'synthetic');
assert.deepEqual(streetView(), [], 'no Google call for an unknown id');
const steered = await app.call(
`/frame/${STILL.id}?lat=48.85&lon=2.29&heading=90&fov=74&pitch=-17`,
);
assert.equal(steered.headers['X-CCTV-Source'], 'streetview');
const sv = new URL(streetView()[0].url).searchParams;
assert.equal(sv.get('location'), `${STILL.lat},${STILL.lon}`);
assert.equal(sv.get('heading'), '90');
assert.equal(sv.get('fov'), '74');
assert.equal(sv.get('pitch'), '-17');
});
test('the Street View fallback spends a per-IP Google budget, and only with a key', async (t) => {
const app = mount(t, ({ url }) =>
url.startsWith('https://maps.googleapis.com/')
? trackedUpstream({
headers: { 'content-type': 'image/jpeg' },
body: 'street-view',
}).response
: upstreamResponse({ status: 503 }),
);
// Read when the first keyed fallback builds the limiter, so after mount().
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN = '2';
const streetViewCalls = () =>
app.requests.filter(({ url }) =>
url.startsWith('https://maps.googleapis.com/'),
).length;
const first = { remoteAddress: '203.0.113.7' };
const frameSource = async (socket) =>
(await app.call(`/frame/${STILL.id}`, {}, socket)).headers['X-CCTV-Source'];
// Without a key nothing is spent.
for (let i = 0; i < 3; i++)
assert.equal(await frameSource(first), 'synthetic');
assert.equal(streetViewCalls(), 0);
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
// Neither an unknown id nor a forwarded request spends the window.
assert.equal(
(await app.call('/frame/not-a-camera', {}, first)).headers['X-CCTV-Source'],
'synthetic',
);
assert.equal(
(
await app.call(
`/frame/${STILL.id}`,
{ 'x-forwarded-for': '203.0.113.9' },
first,
)
).headers['X-CCTV-Source'],
'synthetic',
);
assert.deepEqual(
[
await frameSource(first),
await frameSource(first),
await frameSource(first),
],
['streetview', 'streetview', 'synthetic'],
);
assert.equal(streetViewCalls(), 2, 'no Google call over the budget');
// Another address has a budget of its own.
assert.equal(
await frameSource({ remoteAddress: '203.0.113.8' }),
'streetview',
);
assert.equal(streetViewCalls(), 3);
// Addresses within one IPv6 /64 share a budget.
const v6 = (remoteAddress) => frameSource({ remoteAddress });
assert.deepEqual(
[await v6('2001:db8::1'), await v6('2001:db8::2'), await v6('2001:db8::3')],
['streetview', 'streetview', 'synthetic'],
);
assert.equal(await v6('2001:db8:0:1::1'), 'streetview', 'another /64');
assert.equal(streetViewCalls(), 6);
});
@@ -0,0 +1,87 @@
// Real sockets: a client that closes its connection while the camera upstream
// is still pending must not get a Street View call outside its own budget.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import http from 'node:http';
import { cctvProxy } from '../../server/providers/cctv.js';
const CAM = {
id: 'still-fixture',
name: 'Still fixture',
lat: 30.27,
lon: -97.74,
feedType: 'image',
url: 'https://cams.upstream.invalid/still.jpg',
};
test('a client that disconnects mid-fetch stays inside its own Street View budget', async (t) => {
const saved = { ...process.env };
process.env.CCTV_SOURCES_JSON = JSON.stringify([CAM]);
process.env.CCTV_SOURCES_FILE = 'absent-source-file.json';
process.env.CCTV_FORCE_AUSTIN = '0';
process.env.GOOGLE_MAPS_SERVER_API_KEY = 'fixture-key';
delete process.env.GOOGLE_MAPS_API_KEY;
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN = '1';
const nativeFetch = globalThis.fetch;
let streetViewCalls = 0;
globalThis.fetch = async (url) => {
if (String(url).startsWith('https://maps.googleapis.com/')) {
streetViewCalls += 1;
return new Response('street-view', {
headers: { 'content-type': 'image/jpeg' },
});
}
await new Promise((r) => setTimeout(r, 400));
return new Response(null, { status: 503 });
};
let handler;
cctvProxy().configureServer({
middlewares: { use: (_r, fn) => (handler = fn) },
});
const server = http.createServer((req, res) => {
req.url = req.url.replace(/^\/api\/cctv/, '');
handler(req, res, () => res.end());
});
await new Promise((r) => server.listen(0, '127.0.0.1', r));
t.after(() => {
server.close();
globalThis.fetch = nativeFetch;
for (const k of Object.keys(process.env))
if (!(k in saved)) delete process.env[k];
Object.assign(process.env, saved);
});
const { port } = server.address();
const get = (abortAfterMs) =>
new Promise((resolve) => {
const req = http.get(
{
host: '127.0.0.1',
port,
path: `/api/cctv/frame/${CAM.id}`,
headers: { host: `localhost:${port}` },
agent: false,
},
(res) => {
res.resume();
res.on('end', () => resolve(res.headers['x-cctv-source']));
},
);
req.on('error', () => resolve('aborted'));
if (abortAfterMs)
setTimeout(() => {
req.destroy();
resolve('aborted');
}, abortAfterMs);
});
assert.equal(await get(), 'streetview');
assert.equal(await get(), 'synthetic', 'the cap of one is spent');
assert.equal(streetViewCalls, 1);
await get(50);
await new Promise((r) => setTimeout(r, 800));
assert.equal(
streetViewCalls,
1,
'an aborted request spent the key outside its own budget',
);
});
+3 -3
View File
@@ -327,9 +327,9 @@ export const DATA_CREDITS = [
key: 'ontario-511-cctv',
html:
'CCTV cameras &amp; frames (Ontario): ' +
'<a href="https://511on.ca/" target="_blank" rel="noopener">Ontario 511</a> ' +
'(<a href="https://www.ontario.ca/page/open-government-licence-ontario" target="_blank" rel="noopener">Open Government Licence - Ontario</a>) ' +
'· provided as is, without warranty; actual road conditions may differ',
'<a href="https://511on.ca/" target="_blank" rel="noopener">Ontario 511</a>. ' +
'Contains information licensed under the <a href="https://www.ontario.ca/page/open-government-licence-ontario" target="_blank" rel="noopener">Open Government Licence – Ontario</a>. ' +
'Provided as is, without warranty; actual road conditions may differ.',
},
{
key: 'fintraffic-cctv',
+12
View File
@@ -173,6 +173,18 @@ test('the NYC camera credit carries NYC DOT’s requested disclaimer', () => {
assert.match(credit.html, /observe posted signage/);
});
test('the Ontario 511 camera credit carries the licence attribution statement', () => {
const credit = DATA_CREDITS.find((entry) => entry.key === 'ontario-511-cctv');
assert.ok(credit, 'Ontario 511 camera frames must be credited');
assert.ok(
credit.html.includes(
'Contains information licensed under the <a href="https://www.ontario.ca/page/open-government-licence-ontario"',
),
);
assert.match(credit.html, />Open Government Licence – Ontario<\/a>/);
assert.match(credit.html, /Provided as is, without warranty/);
});
test("the Houston camera credit names TranStar's terms and the permission they require", () => {
const credit = DATA_CREDITS.find((entry) => entry.key === 'houston-cctv');
assert.ok(credit, 'Houston TranStar camera frames must be credited');
+30
View File
@@ -134,6 +134,7 @@ test('a bad or keyless geocode request never reaches Google', async () => {
]) {
const response = await invokeRoute(routes.get(path), { url });
assert.equal(response.statusCode, 400);
assert.equal(response.headers['cache-control'], 'no-store');
assert.deepEqual(response.body.results, []);
assert.equal(response.body.status, 'REQUEST_DENIED');
}
@@ -144,6 +145,7 @@ test('a bad or keyless geocode request never reaches Google', async () => {
url: '/?address=Austin&lat=30&lon=-97',
});
assert.equal(rejected.statusCode, 405);
assert.equal(rejected.headers['cache-control'], 'no-store');
}
// Keyless is an empty capability, not an outage: a 200 the browser adapters
@@ -167,6 +169,33 @@ test('a bad or keyless geocode request never reaches Google', async () => {
assert.deepEqual(upstream, []);
});
test('only a Geocoding answer is kept by the browser', async () => {
// Google can report a denied key or exhausted quota with HTTP 200.
for (const [httpStatus, status, cacheControl] of [
[200, 'OK', 'private, max-age=300'],
[200, 'ZERO_RESULTS', 'private, max-age=300'],
[200, 'REQUEST_DENIED', 'no-store'],
[200, 'OVER_QUERY_LIMIT', 'no-store'],
[200, 'OVER_DAILY_LIMIT', 'no-store'],
[200, 'UNKNOWN_ERROR', 'no-store'],
[503, 'ZERO_RESULTS', 'no-store'],
]) {
const routes = installRoutes({
fetchImpl: async () =>
Response.json({ status, results: [] }, { status: httpStatus }),
});
for (const path of ['/api/google/geocode', '/api/google/reverse-geocode']) {
const label = `${path} ${httpStatus} ${status}`;
const response = await invokeRoute(routes.get(path), {
url: '/?address=Austin&lat=30&lon=-97',
});
assert.equal(response.statusCode, httpStatus, label);
assert.equal(response.body.status, status, label);
assert.equal(response.headers['cache-control'], cacheControl, label);
}
}
});
test('an upstream failure reaches the browser as fixed public text', async () => {
const routes = installRoutes({
fetchImpl: async () => {
@@ -178,6 +207,7 @@ test('an upstream failure reaches the browser as fixed public text', async () =>
url: '/?address=Austin&lat=30&lon=-97',
});
assert.equal(response.statusCode, 502);
assert.equal(response.headers['cache-control'], 'no-store');
assert.deepEqual(response.body, {
status: 'UNKNOWN_ERROR',
results: [],
+31
View File
@@ -139,3 +139,34 @@ test('same pixels at a new aspect redraw at the new fit', () => {
]);
assert.equal(runtime.canvasStamp, 2);
});
test('same pixels at a new resolution with the same fit skip the redraw', () => {
const { runtime, load } = stillHarness();
load(640, 480);
assert.deepEqual(load(1280, 960), []);
assert.equal(runtime.canvasStamp, 1);
assert.equal(runtime.drawnImageStamp, 2);
});
test('changed pixels redraw even when the fitted rectangle is unchanged', () => {
const { runtime, load } = stillHarness();
load(640, 480);
const changedPixels = new Uint8ClampedArray(
FRAME_SIGNATURE_W * FRAME_SIGNATURE_H * 4,
).fill(90);
runtime.signatureCtx.getImageData = () => ({ data: changedPixels });
const calls = load(640, 480);
assert.equal(calls[0][0], 'fillRect');
assert.deepEqual(calls[1], ['drawImage', 240, 0, 1440, 1080]);
assert.equal(runtime.canvasStamp, 2);
});
test('a failed pixel signature redraws rather than assuming equal content', () => {
const { runtime, load } = stillHarness();
load(640, 480);
runtime.signatureCtx.getImageData = () => {
throw new Error('tainted canvas');
};
assert.deepEqual(load(640, 480).at(-1), ['drawImage', 240, 0, 1440, 1080]);
assert.equal(runtime.canvasStamp, 2);
});
+2 -3
View File
@@ -81,9 +81,8 @@ export function createNavigation({
return false;
const nearest = nearestCameraIdToViewer();
if (!nearest || nearest === layerState._activeCameraId) return false;
parts.selection.setActiveCamera(nearest);
// setActiveCamera ends the opening pick; this is still that pick.
layerState._openingCameraProvisional = true;
// Still the opening pick: later settled moves may replace it.
parts.selection.setActiveCamera(nearest, { opening: true });
return true;
}
+74
View File
@@ -428,6 +428,80 @@ test('a pose with no perspective field of view is projected by the renderer', ()
}
});
test('focus release restores video hue and alpha after the style throttle', () => {
const viewer = {};
const camera = fakeCameraPort({
pose: { position: { x: 6378137 + 1000, y: 0, z: 0 } },
});
const { unbind } = bindFakeDraw(viewer, {
camera: camera.port,
query: { project: () => ({ x: 10, y: 20 }) },
});
let target = {};
const records = ['mp4', 'webm'].map((feedType) => ({
camera: { id: feedType, feedType },
position: { x: 6378137, y: 0, z: 0 },
}));
const state = {
_enabled: true,
_viewer: viewer,
_records: records,
_activeCameraId: null,
_activeFocusStyleCount: 0,
_lastFocusStyleAt: -Infinity,
};
const focus = {
focusNowMs: (nowMs) => nowMs,
getFocusTarget: () => target,
focusPassIsNeeded: (value, count) => !!value || count > 0,
advanceSpriteFocus: () => ({
factor: target ? FOCUS_FACTOR : 1,
active: !!target,
transitioning: false,
}),
focusAlphaNeedsWrite: (before, after) => before !== after,
};
const { createCameraIcons, addCameraIcon, refreshCctvFocusStyles } =
createRendering({
state,
services: { focus },
parts: {
model: {
isVideoFeedCamera: (camera) => isVideoFeedType(camera.feedType),
},
},
source: {},
});
try {
createCameraIcons(viewer);
for (const record of records) addCameraIcon(record);
const icons = state._icons;
refreshCctvFocusStyles(1000);
for (const record of records) {
assertKeepsColor(
icons,
record.camera.id,
LIVE_CAMERA_COLOR,
record.camera.id,
);
}
target = null;
refreshCctvFocusStyles(1050);
for (const record of records) {
assert.equal(icons.focus(record.camera.id), FOCUS_FACTOR, 'throttled');
}
refreshCctvFocusStyles(1100);
for (const record of records) {
assert.deepEqual(icons.item(record.camera.id).color, LIVE_CAMERA_COLOR);
assert.equal(icons.focus(record.camera.id), 1);
}
assert.equal(state._activeFocusStyleCount, 0);
} finally {
unbind();
}
});
test('hover picks the underlying camera only outside a painted count', () => {
const viewer = {};
const icon = { set: 'icons', id: 'camera' };
+8 -2
View File
@@ -108,10 +108,13 @@ export function createSelection({
* Sets the active camera by ID, initializes its projection runtime, refreshes
* its frame, and updates styles.
* @param {string} cameraId - ID of the camera to activate.
* @param {Object} [options]
* @param {boolean} [options.opening=false] - The opening camera following
* the view, not someone's pick: settled moves may still replace it.
* @returns {'activated'|'unchanged'|'not-found'} Discriminated activation result.
*/
function setActiveCamera(cameraId) {
function setActiveCamera(cameraId, { opening = false } = {}) {
if (!cameraId || !layerState._recordById.has(cameraId))
return CCTV_ACTIVATION_RESULT.NOT_FOUND;
const record = layerState._recordById.get(cameraId);
@@ -129,6 +132,9 @@ export function createSelection({
record,
)
) {
// Picking the camera a settled move re-opened is still a pick: later
// settled moves must leave it alone.
if (!opening) layerState._openingCameraProvisional = false;
// Browser-only feeds have a separate player; a reselect retries it
// without re-running activation or touching the projection geometry.
if (parts.model.isBrowserOverlayCamera(record.camera)) {
@@ -139,7 +145,7 @@ export function createSelection({
}
layerState._activeCameraId = cameraId;
layerState._autoHopSuspended = false;
layerState._openingCameraProvisional = false;
layerState._openingCameraProvisional = opening;
// A real activation creates projection work — wake the self-stopping loop.
parts.projection.startProjectionLoop();
if (previousActiveRecord && previousActiveRecord !== record) {
+13
View File
@@ -215,3 +215,16 @@ test('a count click is the card on top for every layer, unless the count yields'
picking.unregisterPickOwner('flights');
}
});
test('same-camera selection ends the opening pick without notifying snapshot feeds', () => {
const fixture = makeSelection('tripcheck-odot');
fixture.state._openingCameraProvisional = true;
assert.equal(
fixture.selection.setActiveCamera('same-camera'),
CCTV_ACTIVATION_RESULT.UNCHANGED,
);
assert.equal(fixture.state._openingCameraProvisional, false);
assert.equal(fixture.state._activeCameraReselection, 0);
assert.equal(fixture.notifications(), 0);
});
-2
View File
@@ -22,8 +22,6 @@ function frameUrlFor(camera, refreshMs = ACTIVE_FRAME_REFRESH_MS) {
const params = new URLSearchParams({
label: camera.name,
city: camera.city,
lat: camera.lat.toFixed(6),
lon: camera.lon.toFixed(6),
heading: String(Math.round(camera.headingDeg)),
fov: String(Math.round(camera.fovDeg)),
pitch: String(Math.round(camera.pitchDeg || -10)),
+2 -2
View File
@@ -82,8 +82,8 @@ test('frame and media URLs preserve registered camera identity and encoded metad
);
assert.equal(frame.searchParams.get('label'), camera.name);
assert.equal(frame.searchParams.get('city'), camera.city);
assert.equal(frame.searchParams.get('lat'), '30.267000');
assert.equal(frame.searchParams.get('lon'), '-97.744000');
assert.equal(frame.searchParams.has('lat'), false);
assert.equal(frame.searchParams.has('lon'), false);
assert.equal(frame.searchParams.get('heading'), '45');
assert.equal(frame.searchParams.get('pitch'), '-12');
assert.deepEqual([...media.searchParams.keys()], ['ts']);
+1
View File
@@ -108,6 +108,7 @@ export function createTesting({ state: layerState, services, parts, source }) {
} = {}) {
layerState._viewer = viewer;
layerState._records = Array.isArray(records) ? records : [];
layerState._count = layerState._records.length;
layerState._recordById = new Map(
layerState._records
.filter((record) => record?.camera?.id)
+66 -7
View File
@@ -52,6 +52,11 @@ function install(register) {
};
}
const ROUTES = [
['/api/google/nearby-places', '?lat=30&lon=-97'],
['/api/google/text-search', '?q=museum&lat=30&lon=-97'],
];
test('nearby labels rank landmarks, deduplicate names/addresses and bound the projection', () => {
const place = (name, types, longitude = 0) => ({
displayName: { text: name },
@@ -176,16 +181,70 @@ for (const preview of [false, true]) {
JSON.parse(calls[1].options.body).locationBias.circle.radius,
50,
);
assert.equal((await request(search, '?lat=30&lon=-97')).statusCode, 400);
assert.equal((await request(nearby, '?lat=bad&lon=-97')).statusCode, 400);
assert.equal((await request(nearby, '', 'POST')).statusCode, 405);
for (const [route, query, method, statusCode] of [
[search, '?lat=30&lon=-97', 'GET', 400],
[nearby, '?lat=bad&lon=-97', 'GET', 400],
[nearby, '', 'POST', 405],
]) {
const refused = await request(route, query, method);
assert.equal(refused.statusCode, statusCode, `${method} ${route}`);
assert.equal(
refused.headers['cache-control'],
'no-store',
`${method} ${route}`,
);
}
t.mock.method(globalThis, 'fetch', async () =>
Response.json({ error: { message: 'Denied' } }, { status: 403 }),
);
assert.deepEqual((await request(search, '?q=museum&lat=30&lon=-97')).body, {
places: [],
error: 'Denied',
});
for (const [route, query] of ROUTES) {
const refused = await request(route, query);
assert.equal(refused.statusCode, 403, route);
assert.equal(refused.headers['cache-control'], 'no-store', route);
assert.deepEqual(refused.body, { places: [], error: 'Denied' }, route);
}
});
}
for (const [name, upstream, statusCode, cacheControl] of [
[
'unreadable 200',
() => new Response('<html>not json</html>', { status: 200 }),
502,
'no-store',
],
[
'unreadable 429',
() => new Response('<html>Too Many Requests</html>', { status: 429 }),
429,
'no-store',
],
[
'transport failure',
() => {
throw new TypeError('fetch failed');
},
502,
'no-store',
],
// Google returns {} for a successful search with no places.
['empty answer ({})', () => Response.json({}), 200, 'private, max-age=300'],
]) {
test(`a Places ${name} is a ${statusCode} with Cache-Control ${cacheControl}`, async () => {
const request = install((middlewares) =>
googlePlacesContextProxy({
resolveApiKey: () => 'fixture-server-key',
fetchImpl: async () => upstream(),
}).configureServer({ middlewares }),
);
for (const [route, query] of ROUTES) {
const res = await request(route, query);
assert.equal(res.statusCode, statusCode, route);
assert.equal(res.headers['cache-control'], cacheControl, route);
assert.deepEqual(res.body.places, [], route);
if (statusCode === 200) assert.equal(res.body.error, null, route);
else assert.ok(res.body.error, route);
}
});
}
+197
View File
@@ -0,0 +1,197 @@
// src/ui/cctvPickerOpening.test.mjs — the picker and the opening camera.
// The opening camera follows the view until someone picks one. The picker
// shows it as its own option; a native select fires no change for the
// option it already shows, so FOCUS keeps that camera, and leaving the list
// without a choice lets it follow the view.
import '../testSupport/registerDom.mjs';
import test from 'node:test';
import assert from 'node:assert/strict';
import * as Cesium from 'cesium';
import { createPinia } from 'pinia';
import { createApp, nextTick } from 'vue';
import cctvLayer, {
_setCctvCoverageStateForTest,
_setCctvOverlayHostForTest,
reopenNearestCamera,
} from '../data/cctv.js';
import { bindFakeDraw } from '../testSupport/fakeDrawPort.mjs';
import { CctvControls } from './cctvControls.js';
import { useCctvStore } from './stores/cctv.js';
import CctvPanel from './views/panels/CctvPanel.vue';
const TRAFALGAR = [51.508, -0.128];
const CAMBRIDGE = [51.513, -0.129];
function record(id, [lat, lon]) {
return {
camera: {
id,
name: `Camera ${id}`,
city: 'London',
lat,
lon,
headingDeg: 41,
pitchDeg: -5,
fovDeg: 56,
rangeM: 210,
mountHeightM: 100,
groundElevationM: 0,
},
coverageEntities: [],
projection: null,
activationDone: true,
groundResolved: {},
groundSamples: {},
groundPrior: 0,
frustumGeometry: null,
frustumPositions: null,
probeClampRangeM: null,
position: Cesium.Cartesian3.fromDegrees(lon, lat, 100),
};
}
function viewerAt([lat, lon]) {
const viewer = {
isDestroyed: () => false,
eye: { lat, lon, height: 300 },
scene: {
canvas: { clientWidth: 1200, clientHeight: 800 },
cartesianToCanvasCoordinates: () => undefined,
primitives: { add: (p) => p, remove: () => true },
requestRender() {},
},
};
// The layer reads the eye and flies through the renderer's camera port.
viewer.draw = bindFakeDraw(viewer, {
camera: {
pose: () => ({ ...viewer.eye }),
on: () => () => {},
followed: null,
flyToBounds() {},
},
}).draw;
return viewer;
}
/** A settled move: the view is now over `[lat, lon]`. */
function settleOver(viewer, [lat, lon]) {
viewer.eye = { lat, lon, height: 300 };
return reopenNearestCamera();
}
/**
* CCTV opened at Trafalgar Square, then a settled move re-opened it at
* Cambridge Circus; the panel is connected to the layer.
*/
async function reopenedAtCambridge(t) {
_setCctvOverlayHostForTest({
setEntries() {},
setVisible() {},
clearSource() {},
});
// Activation builds the frame image and asks for ground heights; this test
// has no server. happy-dom's request timeout would hold the run open 30 s.
const originalImage = globalThis.Image;
const originalFetch = globalThis.fetch;
const originalTimeout = AbortSignal.timeout;
globalThis.Image = class {};
globalThis.fetch = async () => {
throw new TypeError('no server in this test');
};
AbortSignal.timeout = () => new AbortController().signal;
const trafalgar = record('a-trafalgar', TRAFALGAR);
const cambridge = record('b-cambridge', CAMBRIDGE);
// enable() assigned the opening camera without activating it.
trafalgar.activationDone = false;
const viewer = viewerAt(TRAFALGAR);
_setCctvCoverageStateForTest({
viewer,
records: [trafalgar, cambridge],
activeCameraId: 'a-trafalgar',
enabled: true,
openingCameraProvisional: true,
});
document.body.replaceChildren();
const host = document.body.appendChild(document.createElement('div'));
const pinia = createPinia();
const app = createApp(CctvPanel).use(pinia);
app.mount(host);
const select = document.getElementById('cctv-camera-select');
const controls = new CctvControls({
elements: {
_cctvPanel: document.getElementById('cctv-panel'),
_cctvSelect: select,
_cctvFocusBtn: document.getElementById('cctv-focus-btn'),
},
cctv: cctvLayer,
actions: {
isEnabled: () => true,
toggleEnabled: async () => true,
runExplicitFocus(select, focus) {
const cameraId = select();
if (cameraId) focus(cameraId);
},
setParams: (params) => cctvLayer.setParams(params),
setPanelCollapsed() {},
syncViewport() {},
},
store: useCctvStore(pinia),
});
controls.connect();
t.after(() => {
controls.destroy();
app.unmount();
globalThis.Image = originalImage;
globalThis.fetch = originalFetch;
AbortSignal.timeout = originalTimeout;
_setCctvOverlayHostForTest();
_setCctvCoverageStateForTest({ enabled: false });
});
assert.equal(settleOver(viewer, CAMBRIDGE), true);
await nextTick();
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
return { viewer, select };
}
test('the opening camera shows as its own option', async (t) => {
const { select } = await reopenedAtCambridge(t);
assert.equal(select.value, 'b-cambridge');
const shown = select.options[select.selectedIndex];
assert.equal(shown.hidden, false);
assert.equal(shown.disabled, false);
assert.equal(
[...select.options].filter((option) => option.value === 'b-cambridge')
.length,
1,
'the list shows the camera once',
);
assert.ok([...select.options].every((option) => !option.hidden));
});
test('FOCUS on the shown camera keeps it after a pan', async (t) => {
const { viewer, select } = await reopenedAtCambridge(t);
const focus = document.getElementById('cctv-focus-btn');
assert.equal(focus.disabled, false);
focus.click();
// The click handler waits for the layer switch before it picks.
await new Promise((resolve) => setTimeout(resolve, 0));
await nextTick();
assert.equal(settleOver(viewer, TRAFALGAR), false);
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
assert.equal(select.value, 'b-cambridge');
});
test('leaving the picker without a pick lets the camera follow a pan', async (t) => {
const { viewer, select } = await reopenedAtCambridge(t);
select.focus();
select.blur();
await nextTick();
assert.equal(settleOver(viewer, TRAFALGAR), true);
await nextTick();
assert.equal(cctvLayer.getUIState().activeCameraId, 'a-trafalgar');
assert.equal(select.value, 'a-trafalgar');
});