mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 02:14:01 +00:00
fix(cctv): back off the Taiwan catalog when unreachable; review fixes
- taiwan.js: a failed download serves the last good table (up to 24 h) and is not retried for an hour. The table host does not answer from some networks outside Taiwan, and each attempt held the shared catalog refresh for the ~10 s connect timeout. - Own server/providers/cctv/taiwan.js in package-boundaries.json (check:boundaries failed). - Decode numeric character references; name falls back to the CCTVID when a row has no road details. - Credit names the dataset and links the OGDL Taiwan licence. - Document CCTV_TAIWAN_URL; list Taiwan in the CURRENT-STATE source table. - Tests: isolate catalog env, reset loader state, cover the byte cap, failed fetch, backoff, last-good fallback, cap clamping, redirect with a valid body, name fallback, numeric entities and UUID camera ids. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
0868bb97f6
commit
ce5f69b586
7 files changed
+237
-34
No files matched your search
@@ -249,6 +249,7 @@ AISSTREAM_API_KEY=
|
|||||||
# CCTV_FLORIDA_MAX_SOURCES=500 # cap cameras loaded from the FL511 catalog (nearest regional anchors first)
|
# CCTV_FLORIDA_MAX_SOURCES=500 # cap cameras loaded from the FL511 catalog (nearest regional anchors first)
|
||||||
# CCTV_TAIWAN_ENABLED=1 # set 0 to skip Taiwan Freeway Bureau (MOTC) road cameras
|
# CCTV_TAIWAN_ENABLED=1 # set 0 to skip Taiwan Freeway Bureau (MOTC) road cameras
|
||||||
# CCTV_TAIWAN_MAX_SOURCES=600 # cap Taiwan freeway cameras (1-2000); no API key required
|
# CCTV_TAIWAN_MAX_SOURCES=600 # cap Taiwan freeway cameras (1-2000); no API key required
|
||||||
|
# CCTV_TAIWAN_URL= # override the Freeway Bureau MOTC 2.0 CCTV.xml URL
|
||||||
# CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs
|
# CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs
|
||||||
# CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry
|
# CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry
|
||||||
# CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured
|
# CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured
|
||||||
|
|||||||
+1
-1
@@ -112,7 +112,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
|
|||||||
**Quebec is thinly represented**, and that is upstream rather than a filter here: the province runs its own Info-Smog program, so only a handful of Quebec stations appear in ECCC's national feed. The reporting station count also varies hour to hour as stations report or fall silent.
|
**Quebec is thinly represented**, and that is upstream rather than a filter here: the province runs its own Info-Smog program, so only a handful of Quebec stations appear in ECCC's national feed. The reporting station count also varies hour to hour as stations report or fall silent.
|
||||||
|
|
||||||
- **Via Lietuva / Eismoinfo (Lithuania).** Public road-camera catalogs and periodically updated still images from [Eismoinfo](https://eismoinfo.lt/), credited to Via Lietuva / eismoinfo.lt in the app. **Off by default.** The camera endpoints are the eismoinfo.lt website backend and are not among the services listed in Via Lietuva's [open-data statement](https://eismoinfo.lt/eismoinfo-backend/content?id=44), whose conditions are attribution and reuse on the same terms. Until the statement is confirmed to cover the camera photos, the pack loads only when an operator sets `CCTV_LITHUANIA_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to Eismoinfo. `eismoinfo-backend/camera-info-table` is joined by ID to `layer-static-features/VKR`. Without `lks=true`, the latter returns WGS84 `[latitude, longitude]`; the table's `x/y` instead use LKS-94 metres. Only the official HTTPS `/eismoinfo-backend/image-provider/camera/last?id=…` image endpoint is registered. Catalogs use the shared 15-minute cache; frames are fetched on demand and never bundled or archived. These are periodic snapshots, not continuous video. Camera headings are low-confidence estimates. No API key is required. `CCTV_LITHUANIA_MAX_SOURCES` defaults to 400.
|
- **Via Lietuva / Eismoinfo (Lithuania).** Public road-camera catalogs and periodically updated still images from [Eismoinfo](https://eismoinfo.lt/), credited to Via Lietuva / eismoinfo.lt in the app. **Off by default.** The camera endpoints are the eismoinfo.lt website backend and are not among the services listed in Via Lietuva's [open-data statement](https://eismoinfo.lt/eismoinfo-backend/content?id=44), whose conditions are attribution and reuse on the same terms. Until the statement is confirmed to cover the camera photos, the pack loads only when an operator sets `CCTV_LITHUANIA_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to Eismoinfo. `eismoinfo-backend/camera-info-table` is joined by ID to `layer-static-features/VKR`. Without `lks=true`, the latter returns WGS84 `[latitude, longitude]`; the table's `x/y` instead use LKS-94 metres. Only the official HTTPS `/eismoinfo-backend/image-provider/camera/last?id=…` image endpoint is registered. Catalogs use the shared 15-minute cache; frames are fetched on demand and never bundled or archived. These are periodic snapshots, not continuous video. Camera headings are low-confidence estimates. No API key is required. `CCTV_LITHUANIA_MAX_SOURCES` defaults to 400.
|
||||||
- **Freeway Bureau, MOTC (Taiwan).** National freeway cameras from the Freeway Bureau's keyless [CCTV static table](https://tisvcloud.freeway.gov.tw/history/motc20/CCTV.xml) in the MOTC 2.0 standard, refreshed daily and fetched at runtime (never bundled). Freeway Bureau traffic data is published under the [Open Government Data License, Taiwan (Version 1.0)](https://data.gov.tw/license), which permits reuse with attribution; the app credits the Freeway Bureau, MOTC. The table publishes each camera's MJPEG stream on one of four official camera servers (`cctvn`, `cctvn5`, `cctvc`, `cctvs` under `freeway.gov.tw`). The frame proxy relays single images, so each published stream URL is mapped to the same server's single-JPEG path for the same camera number (`/abs2mjpg/bmjpg` → `/abs2mjpg/jpg`, `/live-view/mjpg/video.cgi` → `/live-view/jpg/image.cgi`); only those exact HTTPS pairs are registered, and the still path is not itself listed in the table. These are on-demand snapshots, not continuous video. Camera headings are low-confidence estimates. No API key is required. On by default; `CCTV_TAIWAN_ENABLED=0` skips the pack, and `CCTV_TAIWAN_MAX_SOURCES` defaults to 600 (bounded to 1–2000).
|
- **Freeway Bureau, MOTC (Taiwan).** National freeway cameras from the Freeway Bureau's keyless [CCTV static table](https://tisvcloud.freeway.gov.tw/history/motc20/CCTV.xml) in the MOTC 2.0 standard, refreshed daily and fetched at runtime (never bundled). Freeway Bureau traffic data is published under the [Open Government Data License, Taiwan (Version 1.0)](https://data.gov.tw/license), which permits reuse, including commercial reuse, with attribution; the app credits the Freeway Bureau, MOTC, names the dataset and links the licence, as its attribution exhibit asks. The table publishes each camera's MJPEG stream on one of four official camera servers (`cctvn`, `cctvn5`, `cctvc`, `cctvs` under `freeway.gov.tw`). The frame proxy relays single images, so each published stream URL is mapped to the same server's single-JPEG path for the same camera number (`/abs2mjpg/bmjpg` → `/abs2mjpg/jpg`, `/live-view/mjpg/video.cgi` → `/live-view/jpg/image.cgi`); only those exact HTTPS pairs are registered, and the still path is not itself listed in the table. These are on-demand snapshots, not continuous video. Camera headings are low-confidence estimates. No API key is required. The table host does not answer from some networks outside Taiwan (the camera servers do); there the pack is empty, logs one warning and retries hourly. On by default; `CCTV_TAIWAN_ENABLED=0` skips the pack, and `CCTV_TAIWAN_MAX_SOURCES` defaults to 600 (bounded to 1–2000).
|
||||||
|
|
||||||
- **Google Maps Platform.** You supply your own API key and are bound by [Google's ToS](https://cloud.google.com/maps-platform/terms). Google Maps Content (tiles, geocodes, places) **may not be cached, stored, rehosted, or committed** — this app only ever uses it live, which is the compliant pattern. The "Google" attribution is displayed on the globe and must stay visible. Restrict your key (see [SECURITY.md](SECURITY.md)).
|
- **Google Maps Platform.** You supply your own API key and are bound by [Google's ToS](https://cloud.google.com/maps-platform/terms). Google Maps Content (tiles, geocodes, places) **may not be cached, stored, rehosted, or committed** — this app only ever uses it live, which is the compliant pattern. The "Google" attribution is displayed on the globe and must stay visible. Restrict your key (see [SECURITY.md](SECURITY.md)).
|
||||||
- **OpenSky Network.** Its license is **non-commercial**, and operational use of the REST API in a live product can require a prior written agreement with OpenSky — even for non-profit/government use. If you deploy this commercially, contact OpenSky for your own terms. The flights layer is a toggle and runs anonymously by default.
|
- **OpenSky Network.** Its license is **non-commercial**, and operational use of the REST API in a live product can require a prior written agreement with OpenSky — even for non-profit/government use. If you deploy this commercially, contact OpenSky for your own terms. The flights layer is a toggle and runs anonymously by default.
|
||||||
|
|||||||
@@ -1102,14 +1102,18 @@ The CCTV catalog includes Taiwan national freeway cameras from the Freeway
|
|||||||
Bureau, MOTC. One keyless request reads the daily MOTC 2.0 `CCTV.xml` table
|
Bureau, MOTC. One keyless request reads the daily MOTC 2.0 `CCTV.xml` table
|
||||||
(~1,800 cameras); the request refuses redirects, has a 15-second deadline and a
|
(~1,800 cameras); the request refuses redirects, has a 15-second deadline and a
|
||||||
6 MiB response cap, and uses the shared 15-minute cache and fair catalog
|
6 MiB response cap, and uses the shared 15-minute cache and fair catalog
|
||||||
allocation. An unavailable source contributes an empty pack. The table
|
allocation. A failed download serves the last good table (up to a day old),
|
||||||
|
or an empty pack, and is not retried for an hour: the catalog host does not
|
||||||
|
answer from some networks outside Taiwan, and without the backoff every refresh
|
||||||
|
there would wait out the connect timeout. The table
|
||||||
publishes MJPEG stream URLs; each is mapped to the same camera server's
|
publishes MJPEG stream URLs; each is mapped to the same camera server's
|
||||||
single-JPEG path for the same camera number, and only those exact HTTPS pairs
|
single-JPEG path for the same camera number, and only those exact HTTPS pairs
|
||||||
on the four `freeway.gov.tw` camera servers are registered with the frame
|
on the four `freeway.gov.tw` camera servers are registered with the frame
|
||||||
proxy. Camera keys come from the server and camera number, because published
|
proxy. Camera keys come from the server and camera number, because published
|
||||||
CCTV IDs contain Chinese place names and a few repeat across cameras. Poses use
|
CCTV IDs contain Chinese place names and a few repeat across cameras. Poses use
|
||||||
the low-confidence heading prior. `CCTV_TAIWAN_ENABLED=0` disables the pack;
|
the low-confidence heading prior. `CCTV_TAIWAN_ENABLED=0` disables the pack;
|
||||||
`CCTV_TAIWAN_MAX_SOURCES` sets its cap (default 600, bounded to 1–2000).
|
`CCTV_TAIWAN_MAX_SOURCES` sets its cap (default 600, bounded to 1–2000), and
|
||||||
|
`CCTV_TAIWAN_URL` overrides the table URL.
|
||||||
Gate: `src/data/cctvTaiwan.test.mjs`.
|
Gate: `src/data/cctvTaiwan.test.mjs`.
|
||||||
|
|
||||||
## DriveBC CCTV source pack
|
## DriveBC CCTV source pack
|
||||||
@@ -3474,7 +3478,7 @@ its criteria cannot be silently ignored.
|
|||||||
| Satellites | CelesTrak | `src/data/satellites.js` | `/api/celestrak` | 120s |
|
| Satellites | CelesTrak | `src/data/satellites.js` | `/api/celestrak` | 120s |
|
||||||
| Space Missions (30d) | Launch Library 2 + CelesTrak | `src/data/rocketLaunches.js` | `/api/launches` + `/api/celestrak/active` | 5 min |
|
| Space Missions (30d) | Launch Library 2 + CelesTrak | `src/data/rocketLaunches.js` | `/api/launches` + `/api/celestrak/active` | 5 min |
|
||||||
| Traffic | Selectable TomTom / OpenStreetMap / Hybrid roads; optional TomTom flow (BYOK) | `src/data/traffic.js` | browser-direct OpenFreeMap tiles; `/api/tomtom` for flow | viewport-driven; capped tile cache |
|
| Traffic | Selectable TomTom / OpenStreetMap / Hybrid roads; optional TomTom flow (BYOK) | `src/data/traffic.js` | browser-direct OpenFreeMap tiles; `/api/tomtom` for flow | viewport-driven; capped tile cache |
|
||||||
| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS, Kartverket ground heights) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + FL511 (FL, opt-in) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) |
|
| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS, Kartverket ground heights) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + FL511 (FL, opt-in) + Freeway Bureau, MOTC (Taiwan) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) |
|
||||||
| Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh |
|
| Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh |
|
||||||
| Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) |
|
| Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) |
|
||||||
| Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto, Bike Itaú (Brazil), Bicing (Barcelona) and seven other Spanish cities + Urban Sharing, Entur, JCDecaux (Lillestrøm, Valencia, Seville) + nextbike (AMBici)) | `src/data/bikeshare.js` | `/api/gbfs` | 60s |
|
| Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto, Bike Itaú (Brazil), Bicing (Barcelona) and seven other Spanish cities + Urban Sharing, Entur, JCDecaux (Lillestrøm, Valencia, Seville) + nextbike (AMBici)) | `src/data/bikeshare.js` | `/api/gbfs` | 60s |
|
||||||
|
|||||||
@@ -191,6 +191,7 @@
|
|||||||
"server/providers/cctv/groundHeights.js",
|
"server/providers/cctv/groundHeights.js",
|
||||||
"server/providers/cctv/catalog.js",
|
"server/providers/cctv/catalog.js",
|
||||||
"server/providers/cctv/lithuania.js",
|
"server/providers/cctv/lithuania.js",
|
||||||
|
"server/providers/cctv/taiwan.js",
|
||||||
"server/providers/cctv/constants.js",
|
"server/providers/cctv/constants.js",
|
||||||
"server/providers/cctv/media.js",
|
"server/providers/cctv/media.js",
|
||||||
"server/providers/cctv/normalize.js",
|
"server/providers/cctv/normalize.js",
|
||||||
|
|||||||
@@ -13,6 +13,13 @@ export const TAIWAN_FREEWAY_CCTV_URL =
|
|||||||
/** The whole network is ~1.2 MB. */
|
/** The whole network is ~1.2 MB. */
|
||||||
export const TAIWAN_MAX_CATALOG_BYTES = 6 * 1024 * 1024;
|
export const TAIWAN_MAX_CATALOG_BYTES = 6 * 1024 * 1024;
|
||||||
export const DEFAULT_TAIWAN_MAX_SOURCES = 600;
|
export const DEFAULT_TAIWAN_MAX_SOURCES = 600;
|
||||||
|
/** After a failed download, wait this long before asking again. The catalog
|
||||||
|
* host does not answer from some networks outside Taiwan, and every attempt
|
||||||
|
* there holds the shared catalog refresh until the connect timeout. */
|
||||||
|
export const TAIWAN_RETRY_AFTER_FAILURE_MS = 60 * 60 * 1000;
|
||||||
|
/** The table is republished daily; a good copy stands in for a failed
|
||||||
|
* download for up to this long. */
|
||||||
|
export const TAIWAN_LAST_GOOD_MAX_AGE_MS = 24 * 60 * 60 * 1000;
|
||||||
|
|
||||||
// The table publishes each camera's MJPEG stream. The same camera servers
|
// The table publishes each camera's MJPEG stream. The same camera servers
|
||||||
// answer a single JPEG for the same camera ID on a sibling path, which is what
|
// answer a single JPEG for the same camera ID on a sibling path, which is what
|
||||||
@@ -62,9 +69,20 @@ const XML_ENTITIES = {
|
|||||||
apos: "'",
|
apos: "'",
|
||||||
};
|
};
|
||||||
|
|
||||||
|
function codePoint(value, radix) {
|
||||||
|
const code = parseInt(value, radix);
|
||||||
|
return Number.isInteger(code) && code > 0 && code <= 0x10ffff
|
||||||
|
? String.fromCodePoint(code)
|
||||||
|
: '';
|
||||||
|
}
|
||||||
|
|
||||||
function decodeXml(text) {
|
function decodeXml(text) {
|
||||||
return String(text ?? '')
|
return String(text ?? '')
|
||||||
.replace(/&(amp|lt|gt|quot|apos);/g, (_, name) => XML_ENTITIES[name])
|
.replace(
|
||||||
|
/&(?:(amp|lt|gt|quot|apos)|#(\d{1,7})|#x([0-9a-fA-F]{1,6}));/g,
|
||||||
|
(_, name, dec, hex) =>
|
||||||
|
name ? XML_ENTITIES[name] : codePoint(dec ?? hex, dec ? 10 : 16),
|
||||||
|
)
|
||||||
.trim();
|
.trim();
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -156,9 +174,11 @@ export function taiwanCameraToSource(row) {
|
|||||||
.map((part) => String(part ?? '').trim())
|
.map((part) => String(part ?? '').trim())
|
||||||
.filter(Boolean)
|
.filter(Boolean)
|
||||||
.join('→');
|
.join('→');
|
||||||
|
// A bare carriageway letter is not a name; fall back to the published ID.
|
||||||
const name =
|
const name =
|
||||||
[road, direction, mile].filter(Boolean).join(' ') +
|
(road || mile || section
|
||||||
(section ? ` ${section}` : '') ||
|
? [road, direction, mile, section].filter(Boolean).join(' ')
|
||||||
|
: '') ||
|
||||||
rawId ||
|
rawId ||
|
||||||
`Freeway camera ${frame.camera}`;
|
`Freeway camera ${frame.camera}`;
|
||||||
|
|
||||||
@@ -191,13 +211,42 @@ export function taiwanCameraToSource(row) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
let lastGood = null;
|
||||||
|
let lastGoodAt = 0;
|
||||||
|
let retryAfter = 0;
|
||||||
|
|
||||||
|
/** Forget the last good table and any failure backoff (tests only). */
|
||||||
|
export function resetTaiwanSourcesCache() {
|
||||||
|
lastGood = null;
|
||||||
|
lastGoodAt = 0;
|
||||||
|
retryAfter = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
function fallbackSources() {
|
||||||
|
return lastGood && Date.now() - lastGoodAt <= TAIWAN_LAST_GOOD_MAX_AGE_MS
|
||||||
|
? lastGood
|
||||||
|
: [];
|
||||||
|
}
|
||||||
|
|
||||||
|
function taiwanSourceCap() {
|
||||||
|
const rawCap = Number(
|
||||||
|
process.env.CCTV_TAIWAN_MAX_SOURCES || DEFAULT_TAIWAN_MAX_SOURCES,
|
||||||
|
);
|
||||||
|
return Number.isFinite(rawCap)
|
||||||
|
? Math.max(1, Math.min(2000, Math.floor(rawCap)))
|
||||||
|
: DEFAULT_TAIWAN_MAX_SOURCES;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Fetch Taiwan Freeway Bureau cameras from the keyless MOTC 2.0 XML table.
|
* Fetch Taiwan Freeway Bureau cameras from the keyless MOTC 2.0 XML table.
|
||||||
* One bounded request per shared catalog refresh.
|
* One bounded request per shared catalog refresh. A failed download serves the
|
||||||
|
* last good table (up to a day old) and is not retried for an hour, so a host
|
||||||
|
* that cannot reach the Freeway Bureau does not wait on it every refresh.
|
||||||
*
|
*
|
||||||
* @returns {Promise<Array<object>>}
|
* @returns {Promise<Array<object>>}
|
||||||
*/
|
*/
|
||||||
export async function loadTaiwanSources() {
|
export async function loadTaiwanSources() {
|
||||||
|
if (Date.now() < retryAfter) return fallbackSources();
|
||||||
try {
|
try {
|
||||||
const signal = AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS);
|
const signal = AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS);
|
||||||
const response = await fetch(
|
const response = await fetch(
|
||||||
@@ -210,13 +259,11 @@ export async function loadTaiwanSources() {
|
|||||||
);
|
);
|
||||||
if (!response.ok) {
|
if (!response.ok) {
|
||||||
await response.body?.cancel().catch(() => {});
|
await response.body?.cancel().catch(() => {});
|
||||||
console.warn(
|
throw new Error(
|
||||||
'[CCTV] Taiwan camera download failed:',
|
|
||||||
response.status >= 300 && response.status < 400
|
response.status >= 300 && response.status < 400
|
||||||
? 'catalog redirected; redirects are not followed'
|
? 'catalog redirected; redirects are not followed'
|
||||||
: response.status,
|
: `HTTP ${response.status}`,
|
||||||
);
|
);
|
||||||
return [];
|
|
||||||
}
|
}
|
||||||
const xml = await readResponseTextCapped(
|
const xml = await readResponseTextCapped(
|
||||||
response,
|
response,
|
||||||
@@ -228,22 +275,29 @@ export async function loadTaiwanSources() {
|
|||||||
const camera = taiwanCameraToSource(row);
|
const camera = taiwanCameraToSource(row);
|
||||||
if (camera && !cameras.has(camera.id)) cameras.set(camera.id, camera);
|
if (camera && !cameras.has(camera.id)) cameras.set(camera.id, camera);
|
||||||
}
|
}
|
||||||
const rawCap = Number(
|
if (!cameras.size) throw new Error('no usable cameras in the table');
|
||||||
process.env.CCTV_TAIWAN_MAX_SOURCES || DEFAULT_TAIWAN_MAX_SOURCES,
|
const sources = prioritizeSources(
|
||||||
|
[...cameras.values()],
|
||||||
|
taiwanSourceCap(),
|
||||||
|
ANCHORS,
|
||||||
);
|
);
|
||||||
const cap = Number.isFinite(rawCap)
|
|
||||||
? Math.max(1, Math.min(2000, Math.floor(rawCap)))
|
|
||||||
: DEFAULT_TAIWAN_MAX_SOURCES;
|
|
||||||
const sources = prioritizeSources([...cameras.values()], cap, ANCHORS);
|
|
||||||
console.log(
|
console.log(
|
||||||
`[CCTV] Loaded Taiwan camera sources: ${cameras.size} (using ${sources.length})`,
|
`[CCTV] Loaded Taiwan camera sources: ${cameras.size} (using ${sources.length})`,
|
||||||
);
|
);
|
||||||
|
lastGood = sources;
|
||||||
|
lastGoodAt = Date.now();
|
||||||
|
retryAfter = 0;
|
||||||
return sources;
|
return sources;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
retryAfter = Date.now() + TAIWAN_RETRY_AFTER_FAILURE_MS;
|
||||||
|
const fallback = fallbackSources();
|
||||||
console.warn(
|
console.warn(
|
||||||
'[CCTV] Taiwan camera download error:',
|
`[CCTV] Taiwan camera download failed (${error?.message || error}); ` +
|
||||||
error?.message || error,
|
(fallback.length
|
||||||
|
? `serving the last good ${fallback.length} cameras`
|
||||||
|
: 'no Taiwan cameras') +
|
||||||
|
', next attempt in 1 h',
|
||||||
);
|
);
|
||||||
return [];
|
return fallback;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
+154
-12
@@ -1,14 +1,45 @@
|
|||||||
import { test } from 'node:test';
|
import { afterEach, beforeEach, test } from 'node:test';
|
||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import {
|
import {
|
||||||
loadTaiwanSources,
|
loadTaiwanSources,
|
||||||
parseTaiwanCctvXml,
|
parseTaiwanCctvXml,
|
||||||
|
resetTaiwanSourcesCache,
|
||||||
taiwanCameraToSource,
|
taiwanCameraToSource,
|
||||||
taiwanFrame,
|
taiwanFrame,
|
||||||
TAIWAN_FREEWAY_CCTV_URL,
|
TAIWAN_FREEWAY_CCTV_URL,
|
||||||
|
TAIWAN_MAX_CATALOG_BYTES,
|
||||||
|
TAIWAN_RETRY_AFTER_FAILURE_MS,
|
||||||
} from '../../server/providers/cctv/taiwan.js';
|
} from '../../server/providers/cctv/taiwan.js';
|
||||||
import { createCctvCatalog } from '../../server/providers/cctv/catalog.js';
|
import { createCctvCatalog } from '../../server/providers/cctv/catalog.js';
|
||||||
|
|
||||||
|
// Settings that would steer the loader or the catalog away from the mocks.
|
||||||
|
const ISOLATED_ENV = [
|
||||||
|
'CCTV_SOURCES_FILE',
|
||||||
|
'CCTV_SOURCES_JSON',
|
||||||
|
'CCTV_PREFER_AUSTIN',
|
||||||
|
'CCTV_FORCE_AUSTIN',
|
||||||
|
'CCTV_TAIWAN_ENABLED',
|
||||||
|
'CCTV_TAIWAN_URL',
|
||||||
|
'CCTV_TAIWAN_MAX_SOURCES',
|
||||||
|
];
|
||||||
|
|
||||||
|
let savedEnv = {};
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
resetTaiwanSourcesCache();
|
||||||
|
savedEnv = Object.fromEntries(
|
||||||
|
ISOLATED_ENV.map((key) => [key, process.env[key]]),
|
||||||
|
);
|
||||||
|
for (const key of ISOLATED_ENV) delete process.env[key];
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
for (const [key, value] of Object.entries(savedEnv)) {
|
||||||
|
if (value === undefined) delete process.env[key];
|
||||||
|
else process.env[key] = value;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
function cctvXml(...blocks) {
|
function cctvXml(...blocks) {
|
||||||
return `<?xml version='1.0' encoding='utf-8'?>
|
return `<?xml version='1.0' encoding='utf-8'?>
|
||||||
<CCTVList xmlns="http://traffic.transportdata.tw/standard/traffic/schema/">
|
<CCTVList xmlns="http://traffic.transportdata.tw/standard/traffic/schema/">
|
||||||
@@ -158,27 +189,21 @@ test('a redirected or failed catalog contributes an empty pack', async (t) => {
|
|||||||
globalThis,
|
globalThis,
|
||||||
'fetch',
|
'fetch',
|
||||||
async () =>
|
async () =>
|
||||||
new Response('', {
|
new Response(cctvXml(block()), {
|
||||||
status,
|
status,
|
||||||
headers: status === 302 ? { location: 'https://evil.test/' } : {},
|
headers: status === 302 ? { location: 'https://evil.test/' } : {},
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
|
resetTaiwanSourcesCache();
|
||||||
assert.deepEqual(await loadTaiwanSources(), []);
|
assert.deepEqual(await loadTaiwanSources(), []);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
async function runCatalogWithTaiwanFlag(t, flag) {
|
async function runCatalogWithTaiwanFlag(t, flag) {
|
||||||
const requested = mockCatalog(t, cctvXml(block()));
|
const requested = mockCatalog(t, cctvXml(block()));
|
||||||
const saved = process.env.CCTV_TAIWAN_ENABLED;
|
if (flag !== undefined) process.env.CCTV_TAIWAN_ENABLED = flag;
|
||||||
try {
|
const sources = await createCctvCatalog({ sourceRoot: '/nonexistent' })();
|
||||||
if (flag === undefined) delete process.env.CCTV_TAIWAN_ENABLED;
|
return { requested, sources };
|
||||||
else process.env.CCTV_TAIWAN_ENABLED = flag;
|
|
||||||
const sources = await createCctvCatalog({ sourceRoot: '/nonexistent' })();
|
|
||||||
return { requested, sources };
|
|
||||||
} finally {
|
|
||||||
if (saved === undefined) delete process.env.CCTV_TAIWAN_ENABLED;
|
|
||||||
else process.env.CCTV_TAIWAN_ENABLED = saved;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
test('the Taiwan pack is in the catalog by default', async (t) => {
|
test('the Taiwan pack is in the catalog by default', async (t) => {
|
||||||
@@ -198,3 +223,120 @@ test('CCTV_TAIWAN_ENABLED=0 makes no Freeway Bureau request', async (t) => {
|
|||||||
);
|
);
|
||||||
assert.ok(!sources.some((c) => c.cityId === 'taiwan'));
|
assert.ok(!sources.some((c) => c.cityId === 'taiwan'));
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('a catalog over the byte cap or a failed request contributes an empty pack', async (t) => {
|
||||||
|
t.mock.method(console, 'warn', () => {});
|
||||||
|
const padding = `<!--${'x'.repeat(TAIWAN_MAX_CATALOG_BYTES)}-->`;
|
||||||
|
t.mock.method(
|
||||||
|
globalThis,
|
||||||
|
'fetch',
|
||||||
|
async () => new Response(cctvXml(block()) + padding),
|
||||||
|
);
|
||||||
|
assert.deepEqual(await loadTaiwanSources(), []);
|
||||||
|
resetTaiwanSourcesCache();
|
||||||
|
t.mock.method(globalThis, 'fetch', async () => {
|
||||||
|
throw new TypeError('fetch failed');
|
||||||
|
});
|
||||||
|
assert.deepEqual(await loadTaiwanSources(), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a failed download is not retried until the backoff ends', async (t) => {
|
||||||
|
t.mock.method(console, 'warn', () => {});
|
||||||
|
let calls = 0;
|
||||||
|
t.mock.method(globalThis, 'fetch', async () => {
|
||||||
|
calls += 1;
|
||||||
|
throw new TypeError('fetch failed');
|
||||||
|
});
|
||||||
|
t.mock.method(Date, 'now', () => 1_000_000);
|
||||||
|
assert.deepEqual(await loadTaiwanSources(), []);
|
||||||
|
assert.deepEqual(await loadTaiwanSources(), []);
|
||||||
|
assert.equal(calls, 1, 'the second refresh made no request');
|
||||||
|
Date.now.mock.mockImplementation(
|
||||||
|
() => 1_000_000 + TAIWAN_RETRY_AFTER_FAILURE_MS + 1,
|
||||||
|
);
|
||||||
|
await loadTaiwanSources();
|
||||||
|
assert.equal(calls, 2, 'the backoff ends after an hour');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a failed download serves the last good table', async (t) => {
|
||||||
|
t.mock.method(console, 'log', () => {});
|
||||||
|
t.mock.method(console, 'warn', () => {});
|
||||||
|
const fetchMock = t.mock.method(
|
||||||
|
globalThis,
|
||||||
|
'fetch',
|
||||||
|
async () => new Response(cctvXml(block())),
|
||||||
|
);
|
||||||
|
const first = await loadTaiwanSources();
|
||||||
|
assert.deepEqual(
|
||||||
|
first.map((c) => c.id),
|
||||||
|
['tw-freeway-cctvn-10000'],
|
||||||
|
);
|
||||||
|
fetchMock.mock.mockImplementation(
|
||||||
|
async () => new Response('', { status: 503 }),
|
||||||
|
);
|
||||||
|
assert.deepEqual(
|
||||||
|
(await loadTaiwanSources()).map((c) => c.id),
|
||||||
|
['tw-freeway-cctvn-10000'],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('CCTV_TAIWAN_MAX_SOURCES is clamped to 1-2000', async (t) => {
|
||||||
|
t.mock.method(console, 'log', () => {});
|
||||||
|
const rows = Array.from({ length: 3 }, (_, i) =>
|
||||||
|
block({
|
||||||
|
stream: `https://cctvn.freeway.gov.tw/abs2mjpg/bmjpg?camera=${i + 1}`,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
t.mock.method(
|
||||||
|
globalThis,
|
||||||
|
'fetch',
|
||||||
|
async () => new Response(cctvXml(...rows)),
|
||||||
|
);
|
||||||
|
for (const [value, expected] of [
|
||||||
|
['0', 1],
|
||||||
|
['2', 2],
|
||||||
|
['5000', 3],
|
||||||
|
['abc', 3],
|
||||||
|
]) {
|
||||||
|
resetTaiwanSourcesCache();
|
||||||
|
process.env.CCTV_TAIWAN_MAX_SOURCES = value;
|
||||||
|
const sources = await loadTaiwanSources();
|
||||||
|
assert.equal(sources.length, expected, value);
|
||||||
|
assert.equal(sources.available, 3, value);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('names fall back to the published ID when the row has no road details', () => {
|
||||||
|
const xml = cctvXml(block({ id: 'CCTV-N3-N-100-M' }))
|
||||||
|
.replace('<RoadName>國道1號</RoadName>', '<RoadName></RoadName>')
|
||||||
|
.replace('<LocationMile>0K+000</LocationMile>', '')
|
||||||
|
.replace('<Start>基隆端</Start><End>基隆交流道</End>', '');
|
||||||
|
const camera = taiwanCameraToSource(parseTaiwanCctvXml(xml)[0]);
|
||||||
|
assert.equal(camera.name, 'CCTV-N3-N-100-M');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('numeric character references in the table are decoded', () => {
|
||||||
|
const xml = cctvXml(block()).replace(
|
||||||
|
'<RoadName>國道1號</RoadName>',
|
||||||
|
'<RoadName>國道1號</RoadName>',
|
||||||
|
);
|
||||||
|
assert.equal(parseTaiwanCctvXml(xml)[0].roadName, '國道1號');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('camera numbers that are UUIDs keep their case in the frame URL only', () => {
|
||||||
|
const camera = taiwanCameraToSource(
|
||||||
|
parseTaiwanCctvXml(
|
||||||
|
cctvXml(
|
||||||
|
block({
|
||||||
|
stream:
|
||||||
|
'https://cctvn5.freeway.gov.tw/abs2mjpg/bmjpg?camera=C3A580C9-0232-46A1-BFC6-927590758AB4',
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
)[0],
|
||||||
|
);
|
||||||
|
assert.equal(
|
||||||
|
camera.id,
|
||||||
|
'tw-freeway-cctvn5-c3a580c9-0232-46a1-bfc6-927590758ab4',
|
||||||
|
);
|
||||||
|
assert.match(camera.url, /camera=C3A580C9-0232-46A1-BFC6-927590758AB4$/);
|
||||||
|
});
|
||||||
@@ -263,7 +263,8 @@ export const DATA_CREDITS = [
|
|||||||
html:
|
html:
|
||||||
'Freeway camera stills (Taiwan): ' +
|
'Freeway camera stills (Taiwan): ' +
|
||||||
'<a href="https://www.freeway.gov.tw/" target="_blank" rel="noopener">Freeway Bureau, MOTC</a>, ' +
|
'<a href="https://www.freeway.gov.tw/" target="_blank" rel="noopener">Freeway Bureau, MOTC</a>, ' +
|
||||||
'Open Government Data License, Taiwan (Version 1.0)',
|
'CCTV static data (MOTC 2.0), released under the ' +
|
||||||
|
'<a href="https://data.gov.tw/license" target="_blank" rel="noopener">Open Government Data License, Taiwan (Version 1.0)</a>',
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
key: 'calgary-cctv',
|
key: 'calgary-cctv',
|
||||||
|
|||||||
Reference in new issue
Block a user