Merge pull request #1266 from bilawalsidhu/codex/cctv-shared-batch-20261019

Complete the first CCTV shared-fix batch
This commit is contained in:
German Khokhlov authored and GitHub committed 2026-11-20 14:35:58 +04:00
commit 80b727ebaa
17 files changed
+665 -190

No files matched your search

+2 -2
View File
@@ -68,7 +68,7 @@ How to read this:
| **Caltrans (cwwp2.dot.ca.gov)** | CCTV camera catalogs + frames, California districts | Public Caltrans traffic camera data | "Caltrans — cwwp2.dot.ca.gov" (courtesy) |
| **DelDOT (video.deldot.gov)** | CCTV camera catalog (`tmc.deldot.gov/json/videocamera.json`, cached for 15 minutes) + live HTTPS HLS video, Delaware. Keyless | Public DelDOT traffic camera streams via the [DelDOT traffic map](https://deldot.gov/map/) | "DelDOT — Delaware Department of Transportation" (courtesy) |
| **TfL Open Data (JamCams)** | CCTV camera catalog + frames, London | [TfL Open Data terms](https://tfl.gov.uk/info-for/open-data-users/) — attribution REQUIRED | "Powered by TfL Open Data. Contains OS data © Crown copyright and database rights" |
| **Ontario 511** | CCTV camera catalog + frames, Ontario highways including Kitchener-area routes | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) | "Ontario 511" plus Open Government Licence - Ontario |
| **Ontario 511** | CCTV camera catalog + frames, Ontario highways including Kitchener-area routes | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) | "Ontario 511. Contains information licensed under the Open Government Licence – Ontario" |
| **Fintraffic / Digitraffic (weathercams)** | CCTV camera catalog + frames, Finland | [CC BY 4.0](https://www.digitraffic.fi/en/terms-of-service/) — attribution REQUIRED | "Fintraffic / digitraffic.fi, license CC BY 4.0" |
| **DriveBC** (Government of British Columbia) | CCTV camera catalog + frames, British Columbia highways | [Open Government Licence – British Columbia](https://www2.gov.bc.ca/gov/content/data/open-data/open-government-licence-bc) — attribution REQUIRED | "DriveBC. Contains information licensed under the Open Government Licence – British Columbia" |
| **Open Calgary** (`data.calgary.ca`) | CCTV camera catalog + frames, Calgary | [Open Government Licence – City of Calgary](https://data.calgary.ca/stories/s/Open-Calgary-Terms-of-Use/u45n-7awa) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – City of Calgary" |
@@ -247,7 +247,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
- **Open Vessel Photos.** When you select a vessel with a nine-digit MMSI, the browser asks `https://openvesselphotos.org/v1/mmsi/{mmsi}` directly (the API is keyless and answers cross-origin, so no key and no server proxy are involved; account credentials are never sent) for that vessel only, once per MMSI per session, with an 8-second timeout. A miss (a 404 or a vessel without photos) is remembered for the session; a failed request may be asked again after five minutes. The first photo whose image is an `https://openvesselphotos.org` URL and which names a licence is shown, at the 800 px size, in the CONTEXT panel's CONTACTS view under the vessel's name and MMSI (`src/ui/views/panels/AwarenessPanel.vue`), and nowhere else (`src/data/openVesselPhoto.js`, `src/layers/vessels/photos.js`). The image is hotlinked from the URL the API gives (never copied or re-hosted); the photographer and licence are visible text next to it ("Photo: {photographer} · {licence}", or "Photo · {licence}" when the answer names no one), and the photo and credit link to the vessel's Open Vessel Photos page with `target="_blank" rel="noopener"`. If the image fails to load, the photo and its credit are hidden.
- **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state.
- **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover.
- **Ontario 511 CCTV.** The camera list comes from the `511on.ca/api/v2/get/cameras` endpoint, which now requires a free developer key (`ONTARIO_511_API_KEY`: sign up at 511on.ca/my511/register, then request it while logged in at 511on.ca/developers/doc; throttled to 10 calls a minute). Without the key the pack is skipped. Frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence - Ontario. Access is governed by Ontario 511's Developer's Access Agreement (511on.ca/developers/resources): the data stays under the Open Government Licence - Ontario unless otherwise specified; key requests are reviewed case by case and ask for the intended use and any third parties that will receive the data; attribution by link to 511on.ca is encouraged, and the Ontario 511 logo may not be altered or used to imply endorsement (this app links and shows no logo); and the data is provided "as is", a disclaimer distributors should pass on, which the credit does.
- **Ontario 511 CCTV.** The camera list comes from the `511on.ca/api/v2/get/cameras` endpoint, which now requires a free developer key (`ONTARIO_511_API_KEY`: sign up at 511on.ca/my511/register, then request it while logged in at 511on.ca/developers/doc; throttled to 10 calls a minute). Without the key the pack is skipped. Frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence – Ontario. Access is governed by Ontario 511's Developer's Access Agreement (511on.ca/developers/resources): the data stays under the Open Government Licence – Ontario unless otherwise specified; key requests are reviewed case by case and ask for the intended use and any third parties that will receive the data; attribution by link to 511on.ca is encouraged, and the Ontario 511 logo may not be altered or used to imply endorsement (this app links and shows no logo); and the data is provided "as is", a disclaimer distributors should pass on, which the credit does.
- **DelDOT live video cameras.** The keyless `tmc.deldot.gov/json/videocamera.json` catalog supplies active cameras and official `https://video.deldot.gov/live/…/playlist.m3u8` links. The local server transiently buffers bounded MPEG-TS segments in memory and deletes them on session release or idle expiry. No recordings are retained. `CCTV_DELDOT_ENABLED=0` disables the pack. Public availability is not a blanket redistribution license; deployment beyond personal local viewing needs confirmation of applicable DelDOT terms.
- **Fintraffic weathercams.** The camera list is one keyless GeoJSON request to `tie.digitraffic.fi/api/weathercam/v1/stations` (809 stations / 2,275 presets), sent with the `Digitraffic-User: gods-eye-view` header the service asks clients to identify themselves with and with gzip; frames are stills on `weathercam.digitraffic.fi`, refreshed on each station's 600 s collection interval, which is also the client's ambient-still refresh cadence for this pack (the active camera keeps the usual 10-second refresh). One preset (a fixed camera view) is one camera; stations not `GATHERING` and presets not `inCollection` are dropped. Attribution is required by the CC BY 4.0 licence and is registered in the Data attribution popover. `CCTV_FINTRAFFIC_ENABLED=0` is the kill switch.
- **DriveBC highway cameras.** The camera list comes from the keyless `https://www.drivebc.ca/api/webcams/` endpoint served by the DriveBC.ca site ([bcgov/DriveBC.ca](https://github.com/bcgov/DriveBC.ca)); frames come from `https://www.drivebc.ca/images/<id>.jpg`, built from the numeric camera id. DataBC's [DriveBC HighwayCams](https://catalogue.data.gov.bc.ca/dataset/bc-highwaycams) catalogue entry publishes the same cameras under the Open Government Licence – British Columbia, but its CSV still lists the retired `images.drivebc.ca` frame URLs, which now return a placeholder. The licence's attribution statement is required and is registered in the Data attribution popover. By default the 250 cameras nearest Vancouver and Victoria load; `CCTV_DRIVEBC_MAX_SOURCES` changes the cap and `CCTV_DRIVEBC_ENABLED=0` turns the pack off. Some cameras in the feed are supplied by partners (TransLink, the City of Vancouver, the City of Surrey, Parks Canada / Alberta Motor Association and others); the feed's per-camera credit is carried onto the camera record and shown beside the provider in the CCTV panel.
+13
View File
@@ -0,0 +1,13 @@
- Refuse Ontario 511 catalog redirects and keep the key-bearing request URL
and upstream error text out of failure logs, which now name only the error
class and system code; release the body of an error response. Credit
Ontario 511 with the Open Government Licence – Ontario attribution statement
(asturksever, #1266).
- Abort the other Tarktee (Estonia) DATEX request as soon as one fails, before
its headers or while its body is read, instead of waiting out the shared
30 s deadline; a response that arrives after the failure is released too.
The Lithuania pack reads its two catalogs through the same helper
(asturksever, #1266).
- Frame the calibration gizmo before the CCTV QA drag regression drags it;
default autofocus could leave the arrow shorter than the 10 px grab
threshold (asturksever, #1266).
+12 -6
View File
@@ -7626,7 +7626,7 @@ silently demoting every later lookup for the session.
requires a developer key, so the pack needs `ONTARIO_511_API_KEY` and is skipped without one
(#1228). All sources are RAW PRIOR
poses, and the layer is stills-first.
sources. Fintraffic Finland road weather cameras (2026-09-13; cap 300, `CCTV_FINTRAFFIC_MAX_SOURCES`,
Fintraffic Finland road weather cameras (2026-09-13; cap 300, `CCTV_FINTRAFFIC_MAX_SOURCES`,
kill switch `CCTV_FINTRAFFIC_ENABLED=0`) are the fourth pack: one keyless GeoJSON station list
covering the whole country, where one _preset_ (a station's fixed view) is one camera — 806
GATHERING stations carry 2,256 in-collection presets, prioritized to 300 against seven anchors
@@ -7641,11 +7641,17 @@ silently demoting every later lookup for the session.
empty keeps the old catalog and is retried after another 15 minutes, not on the next
request. Estonia's Tarktee DATEX feeds, which answer in 2 to 16.6 s, get 30 s
(`TARKTEE_FETCH_TIMEOUT_MS`) where other packs get 15 s.
Both Tarktee XML catalogs refuse redirects and have a 4 MiB per-document
limit, enforced against declared length and streamed bytes. A rejected fetch,
unsuccessful response, or body-read failure releases both available response
bodies; read failures also cancel an active sibling reader. Cameras sharing
an image number are deduplicated before the Tarktee pack cap is applied.
Ontario catalog requests refuse redirects and log only the error class and
system code for transport/JSON failures, never the request URL or key.
Both Tarktee XML catalogs share that 30 s deadline, refuse redirects and
have a 4 MiB per-document limit, enforced against declared length and
streamed bytes. A failed request, unsuccessful response or body read aborts
the other feed at once and releases every response body, including one that
arrives after the failure. The Lithuania pack reads its two catalogs
through the same helper (`readCatalogsTogether`,
`server/providers/cctv/catalogPair.js`) with its 15 s deadline. An Ontario
error response has its body released. Cameras sharing an image number are
deduplicated before the Tarktee pack cap is applied.
- **CCTV v3 UX — viewshed + calibration gizmo** (built 2026-07-05 and field
validated 2026-07-21): the COVERAGE toggle is a
tri-state cycle `OFF → ON → VIEWSHED`; viewshed mode renders each visible camera's frustum
+1
View File
@@ -148,6 +148,7 @@
"server/providers/cctv/groundHeights.js",
"server/providers/cctv/catalog.js",
"server/providers/cctv/lithuania.js",
"server/providers/cctv/catalogPair.js",
"src/data/cctvLithuania.test.mjs",
"server/providers/cctv/taiwan.js",
"src/data/cctvTaiwan.test.mjs",
+1
View File
@@ -565,6 +565,7 @@
"server/providers/cctv/groundHeights.js",
"server/providers/cctv/catalog.js",
"server/providers/cctv/lithuania.js",
"server/providers/cctv/catalogPair.js",
"server/providers/cctv/taiwan.js",
"server/providers/cctv/quebec.js",
"server/providers/cctv/ottawa.js",
+37
View File
@@ -1790,6 +1790,43 @@ async function main() {
gizmoStates.map((s, i) => `${GIZMO_PARTS[i]}=${s}`).join(' '),
);
// Frame the actual gizmo for the drag fixture. Default autofocus can leave
// its shaft below the 10px interaction threshold; that distant view is not
// the drag-usability scenario this regression exercises. The gizmo is drawn
// through the renderer ports, so its arrow is read from the layer's drawn
// state (plain {x,y,z} points), not from viewer.entities.
const dragFraming = await page.evaluate(async () => {
const viewer = window.__godsEyeView.viewer;
const mod = window.__godsEyeView.dataManager.layers.get('cctv').module;
const arrow = () =>
mod._cctvDrawnForTest().gizmo['cctv-gizmo-move-east']?.positions;
if (!arrow()?.length) return null;
const Cartesian3 = viewer.camera.positionWC.constructor;
const measure = () => {
const pts = arrow();
if (!pts?.length) return { shaftPx: null, rangeM: null };
const a = viewer.scene.cartesianToCanvasCoordinates(pts[0]);
const b = viewer.scene.cartesianToCanvasCoordinates(pts[pts.length - 1]);
return {
shaftPx: a && b ? Math.hypot(b.x - a.x, b.y - a.y) : null,
rangeM: Cartesian3.distance(viewer.camera.positionWC, pts[0]),
};
};
const before = measure();
const target = Cartesian3.clone(arrow()[0]);
viewer.camera.cancelFlight();
viewer.camera.lookAt(target, new Cartesian3(0, -200, 180));
viewer.camera.lookAtTransform(viewer.camera.transform.constructor.IDENTITY);
viewer.scene.requestRender();
const frames = await window.__qaRenderFrames(3, 10000);
return { before, after: measure(), frames };
});
record(
'drag fixture frames the east arrow above the 10px grab threshold',
dragFraming?.after?.shaftPx >= 10,
JSON.stringify(dragFraming),
);
// Synthetic CDP drag on the EAST arrow — the owner's field-test
// regression. Sample effective elevation after every mouse move: it must
// stay frozen until release, with no transient sampleHeight calls.
+53
View File
@@ -0,0 +1,53 @@
import { readResponseTextCapped } from '../common/http.js';
/**
* Read catalogs that are only useful together, because the pack joins them.
*
* The requests share one deadline and refuse redirects. Whichever fails first
* (before its headers, on its status, or while its capped body is read or
* parsed) aborts the others at once, so a stalled feed is not waited on until
* the deadline. Every response body is released, including one that arrives
* after another feed has already failed.
*
* @param {Array<{url: string, label: string}>} feeds `label` names the feed
* in the error ("locations download failed: 503").
* @param {{accept: string, maxBytes: number, timeoutMs: number,
* parse?: (text: string) => unknown}} options
* @returns {Promise<unknown[]>} The parsed bodies (texts without `parse`), in
* feed order.
*/
export async function readCatalogsTogether(
feeds,
{ accept, maxBytes, timeoutMs, parse = (text) => text },
) {
const sibling = new AbortController();
const signal = AbortSignal.any([
sibling.signal,
AbortSignal.timeout(timeoutMs),
]);
return Promise.all(
feeds.map(async ({ url, label }) => {
let response;
try {
response = await fetch(url, {
headers: { Accept: accept },
redirect: 'manual',
signal,
});
signal.throwIfAborted();
if (!response.ok) {
throw new Error(
response.status >= 300 && response.status < 400
? `${label} catalog redirected; redirects are not followed`
: `${label} download failed: ${response.status}`,
);
}
return parse(await readResponseTextCapped(response, maxBytes, signal));
} catch (error) {
sibling.abort();
await response?.body?.cancel().catch(() => {});
throw error;
}
}),
);
}
+13 -38
View File
@@ -4,7 +4,7 @@ import {
fallbackHeadingFromId,
prioritizeSources,
} from './normalize.js';
import { readResponseJsonCapped } from '../common/http.js';
import { readCatalogsTogether } from './catalogPair.js';
const BASE = 'https://eismoinfo.lt/eismoinfo-backend';
export const LITHUANIA_CAMERAS_URL = `${BASE}/camera-info-table`;
@@ -81,46 +81,21 @@ export function lithuaniaCameraToSource(row, feature) {
};
}
async function readCatalog(url, sibling) {
const signal = AbortSignal.any([
AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
sibling.signal,
]);
try {
const response = await fetch(url, {
headers: { Accept: 'application/json' },
redirect: 'manual',
signal,
});
if (!response.ok) {
await response.body?.cancel().catch(() => {});
throw new Error(
response.status >= 300 && response.status < 400
? 'catalog redirected; redirects are not followed'
: `Catalog HTTP ${response.status}`,
);
}
return await readResponseJsonCapped(
response,
LITHUANIA_MAX_CATALOG_BYTES,
signal,
);
} catch (error) {
// The join needs both catalogs; release the other request at once
// rather than leaving its body open until the timeout.
sibling.abort();
throw error;
}
}
/** Two bounded requests per shared catalog refresh, no API key required. */
export async function loadLithuaniaSources() {
try {
const sibling = new AbortController();
const [rows, layers] = await Promise.all([
readCatalog(LITHUANIA_CAMERAS_URL, sibling),
readCatalog(LITHUANIA_LOCATIONS_URL, sibling),
]);
const [rows, layers] = await readCatalogsTogether(
[
{ url: LITHUANIA_CAMERAS_URL, label: 'cameras' },
{ url: LITHUANIA_LOCATIONS_URL, label: 'locations' },
],
{
accept: 'application/json',
maxBytes: LITHUANIA_MAX_CATALOG_BYTES,
timeoutMs: CCTV_SOURCE_FETCH_TIMEOUT_MS,
parse: JSON.parse,
},
);
if (!Array.isArray(rows) || !Array.isArray(layers)) return [];
const features = layers.find((layer) => layer?.layer === 'VKR')?.features;
if (!Array.isArray(features)) return [];
+92
View File
@@ -127,3 +127,95 @@ for (const status of [429, 500, 503]) {
assert.doesNotMatch(lines[0], /secret-key-value/);
});
}
for (const failure of ['transport', 'invalid-json']) {
test(`Ontario ${failure} failures cannot log reflected keys or request URLs`, async (t) => {
const lines = captureConsole(t);
const key = 'synthetic secret&/=%value';
t.mock.method(globalThis, 'fetch', async (url) => {
if (failure === 'transport')
throw new Error(`request failed: ${url} ${key}`);
return new Response(`not JSON: ${url} ${key}`);
});
assert.deepEqual(
await loadOntarioSourcesFromOpenData({}, { ONTARIO_511_API_KEY: key }),
[],
);
assert.equal(lines.length, 1);
assert.match(lines[0], /Ontario 511 camera download error/);
assert.equal(lines[0].includes(key), false);
assert.equal(lines[0].includes('synthetic'), false);
assert.equal(lines[0].includes('https://'), false);
});
}
test('Ontario encodes reserved key characters and refuses credential-bearing redirects', async (t) => {
const lines = captureConsole(t);
const key = 'synthetic &/=% value';
let request;
t.mock.method(globalThis, 'fetch', async (url, init) => {
request = { url: new URL(url), init };
return Response.json([ROW]);
});
const cameras = await loadOntarioSourcesFromOpenData(
{},
{ ONTARIO_511_API_KEY: key },
);
assert.equal(request.url.searchParams.get('key'), key);
assert.equal(request.init.redirect, 'error');
assert.equal(cameras.length, 1);
assert.equal(JSON.stringify(cameras).includes('synthetic'), false);
assert.equal(lines.join(' ').includes('synthetic'), false);
});
test('Ontario names a refused redirect and a system code without the request text', async (t) => {
const lines = captureConsole(t);
const key = 'synthetic-secret';
const failures = [
// The shape Node's fetch gives a refused redirect and a DNS failure.
new TypeError('fetch failed', {
cause: new Error('unexpected redirect'),
}),
new TypeError('fetch failed', {
cause: Object.assign(new Error(`getaddrinfo ENOTFOUND ${key}`), {
code: 'ENOTFOUND',
}),
}),
];
t.mock.method(globalThis, 'fetch', async () => {
throw failures.shift();
});
for (let i = 0; i < 2; i++) {
assert.deepEqual(
await loadOntarioSourcesFromOpenData({}, { ONTARIO_511_API_KEY: key }),
[],
);
}
assert.deepEqual(lines, [
'[CCTV] Ontario 511 camera download error (TypeError, redirect refused)',
'[CCTV] Ontario 511 camera download error (TypeError ENOTFOUND)',
]);
});
test('Ontario releases the body of an error response', async (t) => {
captureConsole(t);
let cancelled = false;
t.mock.method(
globalThis,
'fetch',
async () =>
new Response(
new ReadableStream({
cancel() {
cancelled = true;
},
}),
{ status: 503 },
),
);
assert.deepEqual(
await loadOntarioSourcesFromOpenData({}, { ONTARIO_511_API_KEY: 'k' }),
[],
);
assert.equal(cancelled, true);
});
+34 -70
View File
@@ -43,6 +43,7 @@ import {
TARKTEE_IMAGE_ORIGIN,
DEFAULT_TARKTEE_MAX_SOURCES,
TARKTEE_MAX_CATALOG_BYTES,
TARKTEE_FETCH_TIMEOUT_MS,
TARKTEE_ANCHORS,
DEFAULT_WARENDORF_SOURCE_FILE,
WARENDORF_IMAGE_ORIGINS,
@@ -82,7 +83,6 @@ import {
HK_ANCHORS,
HK_MAX_CATALOG_BYTES,
CCTV_SOURCE_FETCH_TIMEOUT_MS,
TARKTEE_FETCH_TIMEOUT_MS,
DELDOT_CCTV_URL,
DEFAULT_DELDOT_MAX_SOURCES,
DELDOT_ANCHORS,
@@ -172,6 +172,7 @@ import {
pinnedCameraUrl,
} from './normalize.js';
import { directionToHeading } from '../../../src/data/directionText.js';
import { readCatalogsTogether } from './catalogPair.js';
import {
readResponseBytesCapped,
readResponseJsonCapped,
@@ -561,6 +562,22 @@ export function _resetOntarioKeyWarningForTest() {
ontarioKeyMissingLogged = false;
}
/**
* What failed in an Ontario download, without upstream text: the error class,
* plus the system code (`ENOTFOUND`) or a refused redirect when there is one.
* @param {unknown} error
* @returns {string}
*/
function ontarioFailureKind(error) {
const name = /^[A-Za-z]{1,40}$/.test(error?.name) ? error.name : 'Error';
if (error?.cause?.message === 'unexpected redirect')
return `${name}, redirect refused`;
const code = error?.cause?.code;
return typeof code === 'string' && /^[A-Z][A-Z0-9_]{1,40}$/.test(code)
? `${name} ${code}`
: name;
}
/**
* Fetch Ontario 511 CCTV cameras. The camera list comes from the 511 API,
* which now requires a free developer key (511on.ca/developers/doc, "Requires a
@@ -593,6 +610,7 @@ export async function loadOntarioSourcesFromOpenData(
url.searchParams.set('key', key);
const resp = await fetch(url, {
headers: { Accept: 'application/json' },
redirect: 'error',
signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
});
if (!resp.ok) {
@@ -604,6 +622,7 @@ export async function loadOntarioSourcesFromOpenData(
console.warn(
`[CCTV] Ontario 511 camera download failed: ${resp.status}${keyHint}`,
);
await resp.body?.cancel().catch(() => {});
return [];
}
const rows = await resp.json();
@@ -658,7 +677,7 @@ export async function loadOntarioSourcesFromOpenData(
url: view.url,
snapshotUrl: view.url,
sourceKind: 'ontario-511-open-data',
license: 'Open Government Licence - Ontario',
license: 'Open Government Licence – Ontario',
});
}
@@ -677,9 +696,10 @@ export async function loadOntarioSourcesFromOpenData(
);
return prioritized;
} catch (error) {
// Transport and JSON errors may reflect the credential-bearing request,
// so only the error class and a system code are logged, never the text.
console.warn(
'[CCTV] Ontario 511 camera download error:',
error?.message || error,
`[CCTV] Ontario 511 camera download error (${ontarioFailureKind(error)})`,
);
return [];
}
@@ -1279,73 +1299,17 @@ export function parseTarkteeDatexImages(xml) {
*/
export async function loadTarkteeSourcesFromDatex() {
try {
const responses = await Promise.allSettled([
fetch(TARKTEE_LOCATIONS_URL, {
headers: { Accept: 'application/xml,text/xml,*/*' },
redirect: 'manual',
signal: AbortSignal.timeout(TARKTEE_FETCH_TIMEOUT_MS),
}),
fetch(TARKTEE_IMAGES_URL, {
headers: { Accept: 'application/xml,text/xml,*/*' },
redirect: 'manual',
signal: AbortSignal.timeout(TARKTEE_FETCH_TIMEOUT_MS),
}),
]);
const [locResp, imgResp] = responses.map((result) =>
result.status === 'fulfilled' ? result.value : undefined,
const [locXml, imgXml] = await readCatalogsTogether(
[
{ url: TARKTEE_LOCATIONS_URL, label: 'locations' },
{ url: TARKTEE_IMAGES_URL, label: 'images' },
],
{
accept: 'application/xml,text/xml,*/*',
maxBytes: TARKTEE_MAX_CATALOG_BYTES,
timeoutMs: TARKTEE_FETCH_TIMEOUT_MS,
},
);
// Both responses are fetched in parallel; if either is rejected or overflows
// its byte cap, both bodies must be released so neither transport leaks.
const discardBoth = async () => {
await Promise.allSettled([
locResp?.body?.cancel(),
imgResp?.body?.cancel(),
]);
return [];
};
const rejected = responses.find((result) => result.status === 'rejected');
if (rejected) {
await discardBoth();
throw rejected.reason;
}
if (
(locResp.status >= 300 && locResp.status < 400) ||
(imgResp.status >= 300 && imgResp.status < 400)
) {
console.warn(
'[CCTV] Tarktee DATEX catalog redirected; redirects are not followed',
);
return discardBoth();
}
if (!locResp.ok) {
console.warn('[CCTV] Tarktee locations download failed:', locResp.status);
return discardBoth();
}
if (!imgResp.ok) {
console.warn('[CCTV] Tarktee images download failed:', imgResp.status);
return discardBoth();
}
let locXml;
let imgXml;
const readAbort = new AbortController();
try {
[locXml, imgXml] = await Promise.all([
readResponseTextCapped(
locResp,
TARKTEE_MAX_CATALOG_BYTES,
readAbort.signal,
),
readResponseTextCapped(
imgResp,
TARKTEE_MAX_CATALOG_BYTES,
readAbort.signal,
),
]);
} catch (error) {
readAbort.abort();
await discardBoth();
throw error;
}
const locations = parseTarkteeDatexLocations(locXml);
const images = parseTarkteeDatexImages(imgXml);
if (!locations.size || !images.size) {
+155 -54
View File
@@ -12,6 +12,14 @@ import {
parseTarkteeDatexLocations,
} from '../../server/providers/cctv/sources.js';
/**
* Disarm the 30 s deadline, so a loader that only stops at that deadline
* fails at the test timeout instead of passing late.
*/
function withoutDeadline(t) {
t.mock.method(AbortSignal, 'timeout', () => new AbortController().signal);
}
/** Silence the loaders' progress and failure logging. */
function quiet(t) {
t.mock.method(console, 'log', () => {});
@@ -97,6 +105,29 @@ test('Tarktee DATEX image parser rejects non-official hosts', () => {
assert.equal(images.size, 0);
});
test('Tarktee DATEX requests wait 30 s, past the slow answers measured live', async (t) => {
const timeouts = [];
const timeout = AbortSignal.timeout;
t.mock.method(AbortSignal, 'timeout', (ms) => {
timeouts.push(ms);
return timeout.call(AbortSignal, ms);
});
t.mock.method(console, 'log', () => {});
t.mock.method(
globalThis,
'fetch',
async (url) =>
new Response(
String(url).includes('roadCameraLocations')
? SAMPLE_LOCATIONS_XML
: SAMPLE_IMAGES_XML,
),
);
const cameras = await loadTarkteeSourcesFromDatex();
assert.deepEqual(timeouts, [30_000]);
assert.equal(cameras.length, 1);
});
test('Tarktee loader deduplicates cameras sharing an image number before the cap', async (t) => {
quiet(t);
const dupLocationsXml = `<?xml version="1.0" encoding="UTF-8"?>
@@ -168,46 +199,6 @@ test('Tarktee loader refuses a redirect and cancels both response bodies', async
assert.equal(cancelled.images, true, 'sibling 200 body is also cancelled');
});
test('Tarktee loader cancels fulfilled bodies when either fetch rejects', async (t) => {
for (const failedUrl of [TARKTEE_LOCATIONS_URL, TARKTEE_IMAGES_URL]) {
for (const delayedSibling of [false, true]) {
await t.test(
`${failedUrl}, delayed sibling: ${delayedSibling}`,
async (t) => {
quiet(t);
let cancelled = false;
const response = new Response(
new ReadableStream({
start(controller) {
controller.enqueue(new TextEncoder().encode('<xml/>'));
},
cancel() {
cancelled = true;
},
}),
);
t.mock.method(globalThis, 'fetch', async (url) => {
if (String(url) === failedUrl) throw new TypeError('fetch failed');
if (delayedSibling) await new Promise(setImmediate);
return response;
});
try {
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.equal(
cancelled,
true,
'fulfilled sibling body is cancelled',
);
} finally {
await response.body.cancel();
}
},
);
}
}
});
test('Tarktee loader cancels the sibling body when one feed fails or overflows the byte cap', async (t) => {
quiet(t);
let siblingCancelled = false;
@@ -264,25 +255,135 @@ test('Tarktee loader cancels the sibling body when one feed fails or overflows t
);
});
test('Tarktee DATEX requests wait 30 s, past the slow answers measured live', async (t) => {
const timeouts = [];
const timeout = AbortSignal.timeout;
t.mock.method(AbortSignal, 'timeout', (ms) => {
timeouts.push(ms);
return timeout.call(AbortSignal, ms);
});
t.mock.method(console, 'log', () => {});
for (const failingUrl of [TARKTEE_LOCATIONS_URL, TARKTEE_IMAGES_URL]) {
for (const lateSibling of [false, true]) {
test(
`Tarktee cancels a ${lateSibling ? 'late' : 'received'} sibling when ${failingUrl} rejects before headers`,
{ timeout: 2000 },
async (t) => {
quiet(t);
withoutDeadline(t);
let cancelled = false;
let siblingSignal;
let release;
const gate = new Promise((resolve) => {
release = resolve;
});
t.mock.method(globalThis, 'fetch', async (url, init) => {
if (String(url) === failingUrl)
throw new Error('headers unavailable');
siblingSignal = init.signal;
// A late transport answers once the sibling is aborted, or on release.
if (lateSibling)
await Promise.race([
gate,
new Promise((resolve) =>
init.signal.addEventListener('abort', resolve, { once: true }),
),
]);
return new Response(
new ReadableStream({
start(controller) {
controller.enqueue(new TextEncoder().encode('<xml/>'));
},
cancel() {
cancelled = true;
},
}),
);
});
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
// Also exercise a response arriving after abort, as a queued transport
// completion can race the rejection owner even when fetch honors abort.
if (lateSibling) {
release();
await new Promise((resolve) => setImmediate(resolve));
}
assert.equal(cancelled, true, 'every obtained body must be cancelled');
assert.equal(
siblingSignal.aborted,
true,
'a pending transport is aborted immediately',
);
},
);
}
test(
`Tarktee aborts a pending sibling when ${failingUrl} rejects`,
{ timeout: 2000 },
async (t) => {
quiet(t);
withoutDeadline(t);
let aborted = false;
t.mock.method(globalThis, 'fetch', async (url, init) => {
if (String(url) === failingUrl) throw new Error('headers unavailable');
return new Promise((resolve, reject) => {
init.signal.addEventListener(
'abort',
() => {
aborted = true;
reject(init.signal.reason);
},
{ once: true },
);
});
});
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.equal(aborted, true, 'do not wait for the 30-second deadline');
},
);
}
test('Tarktee cancels both lengthless streams when one exceeds the byte cap', async (t) => {
quiet(t);
const cancelled = [];
t.mock.method(
globalThis,
'fetch',
async (url) =>
new Response(
String(url).includes('roadCameraLocations')
? SAMPLE_LOCATIONS_XML
: SAMPLE_IMAGES_XML,
new ReadableStream({
start(controller) {
const oversized = String(url) === TARKTEE_LOCATIONS_URL;
controller.enqueue(
new Uint8Array(oversized ? TARKTEE_MAX_CATALOG_BYTES + 1 : 1),
);
},
cancel() {
cancelled.push(String(url));
},
}),
),
);
const cameras = await loadTarkteeSourcesFromDatex();
assert.deepEqual(timeouts, [30_000, 30_000]);
assert.equal(cameras.length, 1);
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.deepEqual(
cancelled.sort(),
[TARKTEE_LOCATIONS_URL, TARKTEE_IMAGES_URL].sort(),
);
});
test(
'Tarktee aborts pending headers as soon as its sibling returns an HTTP error',
{ timeout: 2000 },
async (t) => {
quiet(t);
withoutDeadline(t);
let aborted = false;
t.mock.method(globalThis, 'fetch', async (url, init) => {
if (String(url) === TARKTEE_LOCATIONS_URL)
return new Response('unavailable', { status: 503 });
return new Promise((resolve, reject) => {
init.signal.addEventListener(
'abort',
() => {
aborted = true;
reject(init.signal.reason);
},
{ once: true },
);
});
});
assert.deepEqual(await loadTarkteeSourcesFromDatex(), []);
assert.equal(aborted, true);
},
);
+112
View File
@@ -0,0 +1,112 @@
// A body-phase failure of one Tarktee DATEX feed must abort the sibling that is
// still waiting for headers instead of leaving it until the 30 s deadline.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import http from 'node:http';
import {
TARKTEE_IMAGES_URL,
TARKTEE_LOCATIONS_URL,
TARKTEE_MAX_CATALOG_BYTES,
} from '../../server/providers/cctv/constants.js';
import { loadTarkteeSourcesFromDatex } from '../../server/providers/cctv/sources.js';
/** The sibling sends its headers this late; the loader must not wait for it. */
const SIBLING_HEADER_DELAY_MS = 2000;
/** Generous bound for "at once" on a loaded CI machine. */
const FAIL_FAST_MS = 1000;
function startServer(handlers) {
const server = http.createServer((req, res) => handlers[req.url](req, res));
return new Promise((resolve) => {
server.listen(0, '127.0.0.1', () => resolve(server));
});
}
/** Send the two DATEX URLs to the local server through the real fetch. */
function routeFetchTo(t, port) {
const realFetch = globalThis.fetch;
t.mock.method(globalThis, 'fetch', (url, init) => {
const path =
String(url) === TARKTEE_LOCATIONS_URL
? '/loc'
: String(url) === TARKTEE_IMAGES_URL
? '/img'
: null;
assert.ok(path, `unexpected url ${url}`);
return realFetch(`http://127.0.0.1:${port}${path}`, init);
});
}
/** The sibling answers late and reports whether it was cut off before headers. */
function slowSibling() {
const arrived = Promise.withResolvers();
const closed = Promise.withResolvers();
return {
arrived: arrived.promise,
closed: closed.promise,
handler(req, res) {
arrived.resolve();
const timer = setTimeout(() => {
res.writeHead(200, { 'content-type': 'application/xml' });
res.end('<d2LogicalModel/>');
}, SIBLING_HEADER_DELAY_MS);
res.on('close', () => {
clearTimeout(timer);
closed.resolve({ beforeHeaders: !res.headersSent });
});
},
};
}
const failingBodies = {
'declares a body over the byte cap': (req, res) => {
res.writeHead(200, {
'content-type': 'application/xml',
'content-length': String(TARKTEE_MAX_CATALOG_BYTES + 1),
});
res.write('<d2LogicalModel>');
},
'resets the connection mid-body': (req, res) => {
res.writeHead(200, { 'content-type': 'application/xml' });
res.write('<d2LogicalModel><payloadPublication>');
setTimeout(() => res.socket.destroy(), 20);
},
};
for (const [label, failingFeed] of Object.entries(failingBodies)) {
for (const [failing, pending] of [
['/loc', '/img'],
['/img', '/loc'],
]) {
test(`Tarktee aborts ${pending} pending headers when ${failing} ${label}`, { timeout: 10_000 }, async (t) => {
t.mock.method(console, 'log', () => {});
t.mock.method(console, 'warn', () => {});
const sibling = slowSibling();
const server = await startServer({
[failing]: (req, res) =>
sibling.arrived.then(() => failingFeed(req, res)),
[pending]: sibling.handler,
});
t.after(() => {
server.closeAllConnections();
server.close();
});
routeFetchTo(t, server.address().port);
const started = Date.now();
const cameras = await loadTarkteeSourcesFromDatex();
const elapsed = Date.now() - started;
assert.deepEqual(cameras, []);
assert.ok(
elapsed < FAIL_FAST_MS,
`loader waited ${elapsed} ms for the sibling's headers instead of aborting it`,
);
assert.deepEqual(
await sibling.closed,
{ beforeHeaders: true },
`${pending} was not aborted`,
);
});
}
}
+20 -17
View File
@@ -140,25 +140,28 @@ for (const [label, flag] of [
});
}
test('a failed catalog cancels the sibling request instead of waiting for the timeout', async (t) => {
t.mock.method(console, 'warn', () => {});
let siblingSignal;
t.mock.method(globalThis, 'fetch', async (url, options) => {
if (url === LITHUANIA_CAMERAS_URL) {
return new Response('', { status: 503 });
}
siblingSignal = options.signal;
// Never settles on its own; only an abort ends it.
return new Promise((_, reject) => {
options.signal.addEventListener('abort', () =>
reject(options.signal.reason),
);
for (const [failure, camerasResponse] of [
['failed', () => new Response('', { status: 503 })],
['malformed', () => new Response('<html>error page</html>')],
]) {
test(`a ${failure} catalog cancels the sibling request instead of waiting for the timeout`, async (t) => {
t.mock.method(console, 'warn', () => {});
let siblingSignal;
t.mock.method(globalThis, 'fetch', async (url, options) => {
if (url === LITHUANIA_CAMERAS_URL) return camerasResponse();
siblingSignal = options.signal;
// Never settles on its own; only an abort ends it.
return new Promise((_, reject) => {
options.signal.addEventListener('abort', () =>
reject(options.signal.reason),
);
});
});
assert.deepEqual(await loadLithuaniaSources(), []);
assert.equal(siblingSignal?.aborted, true);
assert.notEqual(siblingSignal.reason?.name, 'TimeoutError');
});
assert.deepEqual(await loadLithuaniaSources(), []);
assert.equal(siblingSignal?.aborted, true);
assert.notEqual(siblingSignal.reason?.name, 'TimeoutError');
});
}
test('a redirected catalog is reported as not followed', async (t) => {
const warnings = [];
+3 -3
View File
@@ -303,9 +303,9 @@ export const DATA_CREDITS = [
key: 'ontario-511-cctv',
html:
'CCTV cameras &amp; frames (Ontario): ' +
'<a href="https://511on.ca/" target="_blank" rel="noopener">Ontario 511</a> ' +
'(<a href="https://www.ontario.ca/page/open-government-licence-ontario" target="_blank" rel="noopener">Open Government Licence - Ontario</a>) ' +
'· provided as is, without warranty; actual road conditions may differ',
'<a href="https://511on.ca/" target="_blank" rel="noopener">Ontario 511</a>. ' +
'Contains information licensed under the <a href="https://www.ontario.ca/page/open-government-licence-ontario" target="_blank" rel="noopener">Open Government Licence – Ontario</a>. ' +
'Provided as is, without warranty; actual road conditions may differ.',
},
{
key: 'fintraffic-cctv',
+12
View File
@@ -173,6 +173,18 @@ test('the NYC camera credit carries NYC DOT’s requested disclaimer', () => {
assert.match(credit.html, /observe posted signage/);
});
test('the Ontario 511 camera credit carries the licence attribution statement', () => {
const credit = DATA_CREDITS.find((entry) => entry.key === 'ontario-511-cctv');
assert.ok(credit, 'Ontario 511 camera frames must be credited');
assert.ok(
credit.html.includes(
'Contains information licensed under the <a href="https://www.ontario.ca/page/open-government-licence-ontario"',
),
);
assert.match(credit.html, />Open Government Licence – Ontario<\/a>/);
assert.match(credit.html, /Provided as is, without warranty/);
});
test("the Houston camera credit names TranStar's terms and the permission they require", () => {
const credit = DATA_CREDITS.find((entry) => entry.key === 'houston-cctv');
assert.ok(credit, 'Houston TranStar camera frames must be credited');
+31
View File
@@ -139,3 +139,34 @@ test('same pixels at a new aspect redraw at the new fit', () => {
]);
assert.equal(runtime.canvasStamp, 2);
});
test('same pixels at a new resolution with the same fit skip the redraw', () => {
const { runtime, load } = stillHarness();
load(640, 480);
assert.deepEqual(load(1280, 960), []);
assert.equal(runtime.canvasStamp, 1);
assert.equal(runtime.drawnImageStamp, 2);
});
test('changed pixels redraw even when the fitted rectangle is unchanged', () => {
const { runtime, load } = stillHarness();
load(640, 480);
const changedPixels = new Uint8ClampedArray(
FRAME_SIGNATURE_W * FRAME_SIGNATURE_H * 4,
).fill(90);
runtime.signatureCtx.getImageData = () => ({ data: changedPixels });
const calls = load(640, 480);
assert.equal(calls[0][0], 'fillRect');
assert.deepEqual(calls[1], ['drawImage', 240, 0, 1440, 1080]);
assert.equal(runtime.canvasStamp, 2);
});
test('a failed pixel signature redraws rather than assuming equal content', () => {
const { runtime, load } = stillHarness();
load(640, 480);
runtime.signatureCtx.getImageData = () => {
throw new Error('tainted canvas');
};
assert.deepEqual(load(640, 480).at(-1), ['drawImage', 240, 0, 1440, 1080]);
assert.equal(runtime.canvasStamp, 2);
});
+74
View File
@@ -428,6 +428,80 @@ test('a pose with no perspective field of view is projected by the renderer', ()
}
});
test('focus release restores video hue and alpha after the style throttle', () => {
const viewer = {};
const camera = fakeCameraPort({
pose: { position: { x: 6378137 + 1000, y: 0, z: 0 } },
});
const { unbind } = bindFakeDraw(viewer, {
camera: camera.port,
query: { project: () => ({ x: 10, y: 20 }) },
});
let target = {};
const records = ['mp4', 'webm'].map((feedType) => ({
camera: { id: feedType, feedType },
position: { x: 6378137, y: 0, z: 0 },
}));
const state = {
_enabled: true,
_viewer: viewer,
_records: records,
_activeCameraId: null,
_activeFocusStyleCount: 0,
_lastFocusStyleAt: -Infinity,
};
const focus = {
focusNowMs: (nowMs) => nowMs,
getFocusTarget: () => target,
focusPassIsNeeded: (value, count) => !!value || count > 0,
advanceSpriteFocus: () => ({
factor: target ? FOCUS_FACTOR : 1,
active: !!target,
transitioning: false,
}),
focusAlphaNeedsWrite: (before, after) => before !== after,
};
const { createCameraIcons, addCameraIcon, refreshCctvFocusStyles } =
createRendering({
state,
services: { focus },
parts: {
model: {
isVideoFeedCamera: (camera) => isVideoFeedType(camera.feedType),
},
},
source: {},
});
try {
createCameraIcons(viewer);
for (const record of records) addCameraIcon(record);
const icons = state._icons;
refreshCctvFocusStyles(1000);
for (const record of records) {
assertKeepsColor(
icons,
record.camera.id,
LIVE_CAMERA_COLOR,
record.camera.id,
);
}
target = null;
refreshCctvFocusStyles(1050);
for (const record of records) {
assert.equal(icons.focus(record.camera.id), FOCUS_FACTOR, 'throttled');
}
refreshCctvFocusStyles(1100);
for (const record of records) {
assert.deepEqual(icons.item(record.camera.id).color, LIVE_CAMERA_COLOR);
assert.equal(icons.focus(record.camera.id), 1);
}
assert.equal(state._activeFocusStyleCount, 0);
} finally {
unbind();
}
});
test('hover picks the underlying camera only outside a painted count', () => {
const viewer = {};
const icon = { set: 'icons', id: 'camera' };