mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 10:24:02 +00:00
Merge branch 'main' into panel-chrome-consistency
This commit is contained in:
17 files changed
+810
-10
No files matched your search
@@ -256,6 +256,8 @@ AISSTREAM_API_KEY=
|
||||
# CCTV_TAIWAN_ENABLED=1 # set 0 to skip Taiwan Freeway Bureau (MOTC) road cameras
|
||||
# CCTV_TAIWAN_MAX_SOURCES=600 # cap Taiwan freeway cameras (1-2000); no API key required
|
||||
# CCTV_TAIWAN_URL= # override the Freeway Bureau MOTC 2.0 CCTV.xml URL
|
||||
# CCTV_A22_ENABLED=1 # set 0 to disable the Autostrada del Brennero (A22, Italy) webcams
|
||||
# CCTV_A22_URL= # override the Open Data Hub A22 webcam catalog URL
|
||||
# CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs
|
||||
# CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry
|
||||
# CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured
|
||||
|
||||
@@ -69,6 +69,7 @@ How to read this:
|
||||
| **Kartverket** (`ws.geonorge.no/hoydedata/v1/punkt`) | Terrain heights (NN2000) used as the ground prior for Statens vegvesen cameras | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — attribution REQUIRED | "© Kartverket", modified (heights over water set to 0 m, rounded to 0.1 m) |
|
||||
| **NYC DOT Traffic Management Center** (`webcams.nyctmc.org`) | CCTV camera catalog + frames, New York City five boroughs | NYC DOT requires developers to sign a [data-sharing agreement](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) for the camera feed. **Off by default**; enable only under your own agreement | "NYC DOT Traffic Management Center" + NYC DOT's required application disclaimer (in the Data attribution popover) |
|
||||
| **City of Toronto Open Data** (`opendata.toronto.ca`) | CCTV camera catalog + frames, Toronto city streets | [Open Government Licence – Toronto](https://www.toronto.ca/city-government/data-research-maps/open-data/open-data-licence/) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – Toronto" |
|
||||
| **Autostrada del Brennero (A22)** (catalog `tourism.api.opendatahub.com`, frames `www.autobrennero.it/WebCamImg/`) | CCTV camera catalog + frames, A22 Brenner–Modena motorway, Italy | Public Autobrennero webcam images; camera list republished by the Open Data Hub, whose records are tagged CC0 (`ClosedData: false`, licence holder Autobrennero); Autobrennero publishes no terms of its own for the images | "Webcam images © Autostrada del Brennero S.p.A., via the Open Data Hub (NOI Techpark)" (courtesy) |
|
||||
| **GBFS (Lyft / BCycle / Social Bicycles / PBSC)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` |
|
||||
| **Bike Share Toronto** (`toronto.publicbikesystem.net`) | Bikeshare station availability, Toronto | Operator GBFS feed. No licence is published: the City's open-data package declares none and the feed has no `license_url`. Attribution is given to the operator. | "Bike Share Toronto (Toronto Parking Authority)" |
|
||||
| **Bike Itaú / Tembici** (`*.publicbikesystem.net`, seven Brazilian cities, Santiago and Bogotá) | Bikeshare station availability: São Paulo, Rio de Janeiro, Salvador, Recife, Porto Alegre, Brasília, Curitiba, Santiago de Chile and Bogotá | Operator GBFS feeds. No licence is published: the feeds' `system_information.json` carries no `license_url`, operator or attribution fields. Attribution is given to the operator. | "Bike Itaú (Tembici)"; "Tembici Bogotá" |
|
||||
@@ -176,6 +177,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
|
||||
- **Kartverket ground heights for Statens vegvesen cameras.** The camera feed's points are 2D, so the loader asks Kartverket's keyless point-height API (`ws.geonorge.no/hoydedata/v1/punkt`, CC BY 4.0) for the terrain height at each camera. Requests carry 20 points: the API accepts 50, but above ~20 it answers the extra points from contour lines (`hoydekurver`, metres to ~20 m off) or not at all. Eight requests run at a time, about 36 for the whole country: a cold lookup took 6–13 s in testing (single requests 0.6–9.5 s). Answers are matched by the coordinates the API echoes and recorded only for points that request asked about. Over water the API returns the sea-floor depth (`dybdekurver` or `Havflate`), which is replaced by 0 m, the water surface; for the ~14 bridge cameras that is below the deck, but much closer than the sea floor or the flat prior. Heights are cached in memory by position for the life of the server, and points with no height (a few ferry quays and bridges, or a point a 200 answer leaves out) for 6 h, so the 15-minute catalog refresh only asks about new or moved cameras. The first failing request (a 408, 429 or 5xx, a network error, a request over 10 s, or an unreadable or oversized body) stops the lookup, with no retries, and no new lookup starts until the backoff ends (Retry-After when sent, else 30 min, longer than the refresh; bounded to 1 min–1 h). Any other 4xx marks that request's points as having no height and the lookup carries on. The whole lookup has a 30 s deadline; a lookup cut short by it resumes on the next refresh without a backoff. The lane waits at most 12 s, counted from the start of the loader, and only when more than 10% of the cameras have no answer yet; a few new cameras or expired misses are looked up in the background. A slower lookup keeps filling the cache, and the cameras it had not reached keep the flat prior until the next 15-minute refresh (about 190 of 865 on one slow first run). Only coordinates are sent, with the same User-Agent as the other CCTV requests. `CCTV_VEGVESEN_HEIGHTS=0` turns the lookup off. The Kartverket credit, with a note that heights over water are set to 0 m and heights are rounded to 0.1 m, is shown beside the Statens vegvesen entry in the Data attribution popover.
|
||||
- **NYC DOT Traffic Management Center cameras.** Off by default. NYC DOT's [developer data feeds page](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) asks developers who want the camera feed to contact `[email protected]` and sign a data-sharing agreement. This project does not hold one that covers downstream installations, so the pack loads only when an operator who holds their own agreement sets `CCTV_NYC_ENABLED=1`; with the flag unset, empty or any other value, the server makes no request to NYC DOT. The endpoint needs no key. The camera list is the public `webcams.nyctmc.org/api/cameras` catalog (~980 cameras across the five boroughs); frames are stills at `webcams.nyctmc.org/api/cameras/<uuid>/image`, synthesized from a validated UUID rather than taken from the payload, and only that host is registered. Cameras the feed reports as offline are dropped. The catalog publishes no camera facing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. Once enabled, the 250 cameras nearest the borough cores load; `CCTV_NYC_MAX_SOURCES` (8–400) changes the cap. Frames are fetched at request time and never stored or redistributed. NYC DOT asks every application that uses its data to carry its disclaimer; it is registered verbatim in the Data attribution popover.
|
||||
- **City of Toronto RESCU cameras.** Keyless. The camera list is the City's public `tmcearthcameras.json` on `opendata.toronto.ca` (dataset "Traffic Cameras", Transportation Services), served as JSONP and unwrapped with an anchored pattern. This is the municipal network the Ontario 511 pack does not carry: downtown arterials plus the Gardiner, DVP, Allen and Lake Shore. The list publishes no frame URL, so frame URLs are BUILT from the documented `CameraImages/loc####.jpg` convention and a strictly-digits camera number, pinning the frame proxy to the portal origin by construction; the list fetch refuses redirects and bounds the body it buffers. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `D1`-`D4` fields are the directions of the *comparison* images (static reference stills captured in 2016 under a different path), not a bearing for the live pan-tilt-zoom frame, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. A camera that is temporarily out of service still answers 200 with a byte-identical "Image Unavailable" placeholder rather than an error (about 6% of the network at any moment), so such a camera reads as healthy; the frame it shows is the agency's, not a synthesized one. The licence requires the attribution statement above. Disable with `CCTV_TORONTO_ENABLED=0`.
|
||||
- **Autostrada del Brennero (A22) webcams.** Keyless. The camera list is one request to the Open Data Hub tourism API (`tourism.api.opendatahub.com/v1/WebcamInfo?source=a22`), run by NOI Techpark in South Tyrol, which republishes Autobrennero's own list of 13 motorway webcams with coordinates and kilometre markers. Frames are the operator's stills on `www.autobrennero.it/WebCamImg/km<N>.jpg`, refreshed about once a minute, and only the exact image for each camera's kilometre is registered. The feed names no bearing, so headings use the id-hash fallback at low confidence. The road climbs from about 20 m in the Po valley to about 1,370 m at the Brenner Pass, so each camera's ground elevation comes from a per-kilometre table sampled from the Copernicus GLO-90 DEM (via Open-Meteo's elevation API); a camera not in the table uses a 200 m prior. The Open Data Hub tags every record CC0 (`ClosedData: false`, Autobrennero as licence holder), but that tag is the aggregator's metadata and Autobrennero publishes no terms of its own for the images, so they are credited to the operator as a courtesy and are not redistributed. Frames are fetched at request time and never stored. Disable with `CCTV_A22_ENABLED=0`.
|
||||
- **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals.
|
||||
- **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All sixteen registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. The TTC feed is BusTime, the surface system: it carries buses and streetcars and no subway at all. About four in ten of its vehicles report no route at all (unassigned or deadheading), and those are shown with the feed default rather than a route-established mode. It sends no ETag or Last-Modified, so its conditional requests fall through to the ordinary snapshot cache. Three Golden Horseshoe neighbours run alongside it — MiWay, Hamilton HSR and Durham Region Transit — each bus-only and each polled only near its own region. Durham Region Transit publishes no bearing on any vehicle, so those sprites take their heading from observed movement like any other feed that omits it. Guelph, Kingston and Barrie extend the same pattern eastward and north. Hamilton, Guelph, Burlington and YRT publish route ids that are not the number on the vehicle, so those entries carry a route-name table taken from each operator’s own published GTFS; any id missing from a table renders unchanged rather than blank. A label is never invented: YRT runs a route `9101` that appears nowhere in its published routes, so it stays as the feed sends it. Burlington’s is the one that cannot be shortcut: `351` is route 1 while `3510` is route 10, so the table is a real join rather than a trimmed prefix. GO Transit / Metrolinx is deliberately absent: its REALTIME feed requires an API key issued by manual review, which the keyless admission rule excludes (its static schedule download is keyless, but carries no vehicle positions). Barrie and YRT/Viva each invite acceptance of their licence through a form, and each licence also states that using the data is itself acceptance; this deployment relies on that clause, which is recorded in both entries. Grand River Transit in Waterloo is absent for a different reason: it answers, but its feed header was months stale and it publishes no per-vehicle timestamps, so it would render vehicles that look live and are not. Ottawa’s OC Transpo falls to the same keyless admission rule as GO Transit — its realtime feed requires an API key. London Transit and Thunder Bay Transit publish vehicle positions over plain HTTP with no HTTPS endpoint available. Every other Ontario system — Brampton, Oakville, Niagara, Milton, Windsor, Greater Sudbury, UP Express and a dozen smaller operators — publishes static GTFS schedules only, with no realtime vehicle positions to register. Between the nine included and these exclusions, every Ontario realtime vehicle-position feed catalogued in the Mobility Database is accounted for.
|
||||
- **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and non-array or empty successful payloads are rejected atomically. Individual structurally malformed rows are skipped rather than failing the whole query; the response carries their bounded `rejectedStationCount` (at most 10,000), which the client adds to its own rejected rows and reports as degraded coverage without discarding the accepted generation. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply.
|
||||
|
||||
@@ -399,7 +399,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi
|
||||
| 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 |
|
||||
| ◍ **Air quality** | Live AQHI on the published 1–10+ health-risk scale, with a Weather-panel card showing the band legend and the highest readings in view. **Canada only** — ECCC is the first registered provider | ECCC MSC GeoMet | 🟢 |
|
||||
| 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds |
|
||||
| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511, free developer key) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit) · Hong Kong (Transport Department) · Taiwan (Freeway Bureau, MOTC), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement), opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed) and opt-in Florida (FL511, individual personal use only; other use needs FDOT's written consent). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
|
||||
| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511, free developer key) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit) · Hong Kong (Transport Department) · Taiwan (Freeway Bureau, MOTC) · Italy (A22 Brennero motorway), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement), opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed) and opt-in Florida (FL511, individual personal use only; other use needs FDOT's written consent). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
|
||||
| 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 |
|
||||
| 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer with full-screen **EXPAND**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key |
|
||||
| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 |
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
- FIRMS fire data loads on hosts where IPv6 is unreachable and IPv4 is slow to
|
||||
connect: a request whose every address fails to connect retries once over
|
||||
IPv4, and FIRMS warnings name the underlying connection errors instead of
|
||||
only `fetch failed` (jeffblakely, #1078).
|
||||
@@ -0,0 +1,8 @@
|
||||
- Add the Autostrada del Brennero (A22, Brenner–Modena) motorway webcams as a
|
||||
CCTV pack, the first Italian cameras. The 13-camera list is one keyless
|
||||
request to the Open Data Hub (NOI Techpark) tourism API, which republishes
|
||||
Autobrennero's own list with coordinates; frames are the operator's stills,
|
||||
refreshed about once a minute and pinned to each camera's own `km<N>.jpg`.
|
||||
Each camera's ground elevation comes from a per-kilometre DEM table, since
|
||||
the road climbs from the Po valley to the Brenner Pass. `CCTV_A22_ENABLED=0`
|
||||
disables the pack (Jibran Tahir, @jibraaan, #956). Closes #155.
|
||||
+14
-1
@@ -2114,6 +2114,19 @@ throws, that source reports failure without a contradictory success entry.
|
||||
The existing per-record append continues to support large feeds; sequential
|
||||
fetching, trailing-24-hour filtering and partial-success caching are unchanged.
|
||||
|
||||
FIRMS requests retry once over IPv4 when every resolved address fails to
|
||||
connect. Node's network-family auto-selection gives each address a short
|
||||
attempt timeout, so on hosts where IPv6 is unreachable a slow IPv4 handshake
|
||||
was abandoned and the request reported only `fetch failed` (#68). The retry
|
||||
makes a single IPv4 connection within the same deadline and the same body
|
||||
cap, and does not follow redirects. HTTP errors, an expired request deadline
|
||||
and DNS failures are not retried; a per-address connect timeout, which Node 26
|
||||
reports as `ConnectTimeoutError`, is. A retry answered with a null-body status
|
||||
(204, 205, 304) or a status above 599 fails that source instead of crashing
|
||||
the server, and a non-OK response releases its connection at once. FIRMS
|
||||
warnings include the per-address connection errors and never the request URL,
|
||||
which embeds the MAP_KEY.
|
||||
|
||||
## Installations and map-source guidance
|
||||
|
||||
- Keyless mapped installations use OpenFreeMap `landuse` military polygons,
|
||||
@@ -3872,7 +3885,7 @@ its criteria cannot be silently ignored.
|
||||
| Satellites | CelesTrak | `src/data/satellites.js` | `/api/celestrak` | 120s |
|
||||
| Space Missions (30d) | Launch Library 2 + CelesTrak | `src/data/rocketLaunches.js` | `/api/launches` + `/api/celestrak/active` | 5 min |
|
||||
| Traffic | Selectable TomTom / OpenStreetMap / Hybrid roads; optional TomTom flow (BYOK) | `src/data/traffic.js` | browser-direct OpenFreeMap tiles; `/api/tomtom` for flow | viewport-driven; capped tile cache |
|
||||
| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS, Kartverket ground heights) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Hong Kong Transport Department + FL511 (FL, opt-in) + Freeway Bureau, MOTC (Taiwan) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) |
|
||||
| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS, Kartverket ground heights) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Hong Kong Transport Department + FL511 (FL, opt-in) + Freeway Bureau, MOTC (Taiwan) + Autostrada del Brennero (IT, A22 stills) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) |
|
||||
| Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh |
|
||||
| Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) |
|
||||
| Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto, Bike Itaú (Brazil), Bicing (Barcelona) and seven other Spanish cities + Urban Sharing, Entur, JCDecaux (Lillestrøm, Valencia, Seville) + nextbike (AMBici)) | `src/data/bikeshare.js` | `/api/gbfs` | 60s |
|
||||
|
||||
@@ -561,6 +561,7 @@
|
||||
"src/data/cctvCataloniaSource.test.mjs",
|
||||
"src/data/cctvHk.test.mjs",
|
||||
"src/data/cctvVegvesen.test.mjs",
|
||||
"src/data/cctvA22.test.mjs",
|
||||
"src/data/cctvNyc.test.mjs",
|
||||
"src/data/cctvNswSource.test.mjs",
|
||||
"src/data/cctvTxdotSource.test.mjs",
|
||||
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
loadNycSourcesFromOpenData,
|
||||
loadTorontoSourcesFromOpenData,
|
||||
loadFloridaSourcesFromOpenData,
|
||||
loadA22SourcesFromOpenDataHub,
|
||||
} from './sources.js';
|
||||
|
||||
/** Env kill switch: unset or anything but "0" means enabled. */
|
||||
@@ -165,6 +166,11 @@ const LIVE_PACKS = [
|
||||
enabled: () => envEnabled('CCTV_TAIWAN_ENABLED'),
|
||||
load: loadTaiwanSources,
|
||||
},
|
||||
{
|
||||
name: 'a22',
|
||||
enabled: () => envEnabled('CCTV_A22_ENABLED'),
|
||||
load: loadA22SourcesFromOpenDataHub,
|
||||
},
|
||||
];
|
||||
/**
|
||||
* Load CCTV sources from a local JSON file (CCTV_SOURCES_FILE env or default).
|
||||
|
||||
@@ -443,6 +443,39 @@ export const NYC_ANCHORS = [
|
||||
{ lat: 40.5795, lon: -74.1502 }, // Staten Island
|
||||
];
|
||||
|
||||
/** Autostrada del Brennero (A22, Brenner–Modena) webcams: the keyless Open
|
||||
* Data Hub (NOI Techpark, South Tyrol) tourism API republishes Autobrennero's
|
||||
* own camera list with coordinates, one request for the whole motorway
|
||||
* (13 cameras). Frames are stills on the operator's host, refreshed about
|
||||
* once a minute. */
|
||||
export const DEFAULT_A22_CCTV_URL =
|
||||
'https://tourism.api.opendatahub.com/v1/WebcamInfo?source=a22&pagesize=200&removenullvalues=true';
|
||||
/** The only origin A22 camera frames may come from: `km<N>.jpg` per camera. */
|
||||
export const A22_IMAGE_ORIGIN = 'https://www.autobrennero.it/WebCamImg/';
|
||||
/** Hard ceiling on the catalog body. The whole motorway is ~45 KB. */
|
||||
export const A22_MAX_CATALOG_BYTES = 2 * 1024 * 1024;
|
||||
/** Orthometric ground elevation (m) at each camera, keyed by its motorway
|
||||
* kilometre, from the Copernicus GLO-90 DEM via Open-Meteo's elevation API.
|
||||
* The feed's coordinates are 2D and the road climbs from the Po valley (~20 m)
|
||||
* to the Brenner Pass (~1,370 m), so one flat prior would bury the Alpine
|
||||
* cameras. A camera not listed here uses A22_DEFAULT_GROUND_ELEVATION_M. */
|
||||
export const A22_GROUND_ELEVATION_BY_KM = Object.freeze({
|
||||
1: 1365,
|
||||
68: 375,
|
||||
77: 301,
|
||||
100: 219,
|
||||
129: 206,
|
||||
138: 204,
|
||||
159: 175,
|
||||
205: 188,
|
||||
228: 72,
|
||||
240: 38,
|
||||
272: 16,
|
||||
285: 19,
|
||||
313: 40,
|
||||
});
|
||||
export const A22_DEFAULT_GROUND_ELEVATION_M = 200;
|
||||
|
||||
/** DelDOT CCTV: one keyless statewide JSON catalog; live video via RTMP-over-HTTP (rtmpt:80). */
|
||||
export const DELDOT_CCTV_URL = 'https://tmc.deldot.gov/json/videocamera.json';
|
||||
export const DEFAULT_DELDOT_MAX_SOURCES = 300;
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Readable } from 'node:stream';
|
||||
import { hashSeed, escapeXml, pinnedCameraUrl } from './normalize.js';
|
||||
import {
|
||||
A22_IMAGE_ORIGIN,
|
||||
CALGARY_IMAGE_ORIGIN,
|
||||
CCTV_FRAME_FETCH_TIMEOUT_MS,
|
||||
CCTV_FRAME_MAX_BODY_BYTES,
|
||||
@@ -528,6 +529,7 @@ for (const prefix of [
|
||||
HK_IMAGE_ORIGIN,
|
||||
KING_COUNTY_IMAGE_ORIGIN,
|
||||
WSDOT_IMAGE_ORIGIN,
|
||||
A22_IMAGE_ORIGIN,
|
||||
]) {
|
||||
const host = new URL(prefix).hostname;
|
||||
CCTV_PINNED_PREFIXES_BY_HOST.set(host, [
|
||||
|
||||
@@ -421,6 +421,17 @@ export function isLikelyNorwayCoordinate(lat, lon) {
|
||||
);
|
||||
}
|
||||
|
||||
/** The A22 corridor, Brenner Pass to Modena, with slack either side. */
|
||||
export function isLikelyA22Coordinate(lat, lon) {
|
||||
return (
|
||||
isPlausibleLatLon(lat, lon) &&
|
||||
lat >= 44.5 &&
|
||||
lat <= 47.1 &&
|
||||
lon >= 10.5 &&
|
||||
lon <= 11.8
|
||||
);
|
||||
}
|
||||
|
||||
export function isLikelyFinlandCoordinate(lat, lon) {
|
||||
if (!Number.isFinite(lat) || !Number.isFinite(lon)) return false;
|
||||
return lat >= 59.5 && lat <= 70.5 && lon >= 19 && lon <= 32;
|
||||
|
||||
@@ -122,6 +122,11 @@ import {
|
||||
NYC_MAX_CATALOG_BYTES,
|
||||
NYC_GROUND_ELEVATION_M,
|
||||
NYC_ANCHORS,
|
||||
DEFAULT_A22_CCTV_URL,
|
||||
A22_IMAGE_ORIGIN,
|
||||
A22_MAX_CATALOG_BYTES,
|
||||
A22_GROUND_ELEVATION_BY_KM,
|
||||
A22_DEFAULT_GROUND_ELEVATION_M,
|
||||
} from './constants.js';
|
||||
import {
|
||||
toFiniteNumber,
|
||||
@@ -145,6 +150,7 @@ import {
|
||||
isLikelyHongKongCoordinate,
|
||||
isLikelyNorwayCoordinate,
|
||||
isLikelyTorontoCoordinate,
|
||||
isLikelyA22Coordinate,
|
||||
cameraDisplayCode,
|
||||
rowArrayToObject,
|
||||
prioritizeSources,
|
||||
@@ -3645,3 +3651,123 @@ export async function loadFloridaSourcesFromOpenData() {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* One Open Data Hub `WebcamInfo` item (source `a22`) -> one catalog source,
|
||||
* or null.
|
||||
*
|
||||
* The frame must be the operator's own `WebCamImg/km<N>.jpg` for the
|
||||
* camera's motorway kilometre. The feed names the place and the kilometre but
|
||||
* no bearing, so the heading uses the shared id-hash fallback at low
|
||||
* confidence, like Calgary and Vegvesen; the ground elevation comes from the
|
||||
* per-kilometre DEM table because the road spans ~1,350 m of altitude.
|
||||
*
|
||||
* @param {object} item - One element of the API's `Items` array.
|
||||
* @returns {?object}
|
||||
*/
|
||||
export function a22WebcamToSource(item) {
|
||||
if (!item || typeof item !== 'object') return null;
|
||||
if (item.Active === false) return null;
|
||||
if (String(item.Source ?? '').toLowerCase() !== 'a22') return null;
|
||||
|
||||
const gps = Array.isArray(item.GpsInfo) ? item.GpsInfo[0] : null;
|
||||
const lat = toFiniteNumber(gps?.Latitude);
|
||||
const lon = toFiniteNumber(gps?.Longitude);
|
||||
if (!isLikelyA22Coordinate(lat, lon)) return null;
|
||||
|
||||
const km = String(item.Mapping?.a22?.km ?? '').trim();
|
||||
if (!/^\d{1,3}$/.test(km)) return null;
|
||||
const imageUrl = `${A22_IMAGE_ORIGIN}km${Number(km)}.jpg`;
|
||||
let image;
|
||||
try {
|
||||
image = new URL(String(item.Webcamurl ?? ''));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (image.username || image.password || image.href !== imageUrl) return null;
|
||||
|
||||
const place =
|
||||
String(item.Shortname ?? item.Webcamname?.it ?? '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim() || `Km ${Number(km)}`;
|
||||
const cameraId = `it-a22-km${Number(km)}`;
|
||||
const ground = A22_GROUND_ELEVATION_BY_KM[Number(km)];
|
||||
|
||||
return {
|
||||
id: cameraId,
|
||||
name: `A22 ${place} (km ${Number(km)})`,
|
||||
// One motorway-wide category in the camera picker; the place is already
|
||||
// in the name.
|
||||
city: 'A22 Brennero',
|
||||
cityId: 'a22',
|
||||
provider: 'Autostrada del Brennero',
|
||||
lat,
|
||||
lon,
|
||||
headingDeg: fallbackHeadingFromId(cameraId),
|
||||
headingConfidence: 'low',
|
||||
pitchDeg: -15,
|
||||
fovDeg: 50,
|
||||
rangeM: 160,
|
||||
mountHeightM: 8,
|
||||
groundElevationM: Number.isFinite(ground)
|
||||
? ground
|
||||
: A22_DEFAULT_GROUND_ELEVATION_M,
|
||||
feedType: 'image',
|
||||
url: imageUrl,
|
||||
snapshotUrl: imageUrl,
|
||||
sourceKind: 'a22-opendatahub',
|
||||
license:
|
||||
'Webcam images © Autostrada del Brennero S.p.A., via the Open Data Hub (NOI Techpark)',
|
||||
code: cameraDisplayCode(place.toUpperCase()),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch the Autostrada del Brennero (A22) webcams from the keyless Open Data
|
||||
* Hub tourism API. Frames are stills on www.autobrennero.it.
|
||||
*
|
||||
* @returns {Promise<Array<object>>} Normalized camera source objects.
|
||||
*/
|
||||
export async function loadA22SourcesFromOpenDataHub() {
|
||||
try {
|
||||
const endpoint = process.env.CCTV_A22_URL || DEFAULT_A22_CCTV_URL;
|
||||
const resp = await fetch(endpoint, {
|
||||
headers: { Accept: 'application/json' },
|
||||
redirect: 'manual',
|
||||
signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
|
||||
});
|
||||
const discard = async () => {
|
||||
try {
|
||||
await resp.body?.cancel();
|
||||
} catch {
|
||||
/* no-op */
|
||||
}
|
||||
return [];
|
||||
};
|
||||
if (resp.status >= 300 && resp.status < 400) {
|
||||
console.warn('[CCTV] A22 catalog redirected; redirects are not followed');
|
||||
return discard();
|
||||
}
|
||||
if (!resp.ok) {
|
||||
console.warn('[CCTV] A22 camera download failed:', resp.status);
|
||||
return discard();
|
||||
}
|
||||
const payload = await readResponseJsonCapped(resp, A22_MAX_CATALOG_BYTES);
|
||||
const items = Array.isArray(payload?.Items) ? payload.Items : [];
|
||||
const cameras = [];
|
||||
const seen = new Set();
|
||||
for (const item of items) {
|
||||
const camera = a22WebcamToSource(item);
|
||||
if (!camera || seen.has(camera.id)) continue;
|
||||
seen.add(camera.id);
|
||||
cameras.push(camera);
|
||||
}
|
||||
// North to south along the motorway, so the picker reads in road order.
|
||||
cameras.sort((a, b) => b.lat - a.lat);
|
||||
console.log(`[CCTV] Loaded A22 camera sources: ${cameras.length}`);
|
||||
return cameras;
|
||||
} catch (error) {
|
||||
console.warn('[CCTV] A22 camera download error:', error?.message || error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
+138
-6
@@ -1,9 +1,132 @@
|
||||
import path from 'node:path';
|
||||
import https from 'node:https';
|
||||
import { promises as fsp } from 'node:fs';
|
||||
import { Readable } from 'node:stream';
|
||||
|
||||
import { filterTrailing24h, parseFirmsCsv } from '../../src/data/firmsCsv.js';
|
||||
import { readResponseTextCapped } from './common/http.js';
|
||||
|
||||
const NULL_BODY_STATUSES = new Set([204, 205, 304]);
|
||||
|
||||
/**
|
||||
* The per-address connection errors behind a fetch failure, or null. Node 24
|
||||
* puts the AggregateError directly on `err.cause`; Node 26's fetch wraps it in
|
||||
* a ConnectTimeoutError one level deeper.
|
||||
* @param {unknown} err
|
||||
* @returns {?Array<unknown>}
|
||||
*/
|
||||
function connectAttemptErrors(err) {
|
||||
let cause = err?.cause;
|
||||
for (let depth = 0; cause && depth < 3; depth++) {
|
||||
if (Array.isArray(cause.errors)) return cause.errors;
|
||||
cause = cause.cause;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Describe a fetch failure with its socket-level cause. Node reports every
|
||||
* connection failure as `fetch failed`; the cause carries the error code and
|
||||
* the address that failed (never the URL, which embeds the MAP_KEY).
|
||||
* @param {unknown} err
|
||||
* @returns {string}
|
||||
*/
|
||||
export function describeFetchError(err) {
|
||||
const message = err?.message || String(err);
|
||||
const cause = err?.cause;
|
||||
if (!cause) return message;
|
||||
const attempts = connectAttemptErrors(err);
|
||||
const parts = attempts?.length
|
||||
? attempts.map((inner) => inner?.message || inner?.code || String(inner))
|
||||
: [cause.message || cause.code || String(cause)];
|
||||
return `${message} (${parts.join('; ')})`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a fetch failed because every address it tried failed to connect.
|
||||
* Node's network-family auto-selection gives each address a short attempt
|
||||
* timeout, so a slow IPv4 handshake is abandoned in favour of an unreachable
|
||||
* IPv6 address and the whole request fails (#68).
|
||||
* @param {unknown} err
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function isAddressSelectionFailure(err) {
|
||||
return err instanceof TypeError && connectAttemptErrors(err)?.length > 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* GET over IPv4 only, with one connection attempt bounded by the caller's
|
||||
* signal rather than by the auto-selection attempt timeout.
|
||||
* @param {string} url
|
||||
* @param {{signal?: AbortSignal}} [options]
|
||||
* @returns {Promise<Response>}
|
||||
*/
|
||||
export function fetchIpv4(url, { signal } = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const request = https.request(
|
||||
url,
|
||||
{
|
||||
method: 'GET',
|
||||
signal,
|
||||
family: 4,
|
||||
autoSelectFamily: false,
|
||||
agent: false,
|
||||
},
|
||||
(response) => {
|
||||
// Nothing thrown here may escape: this callback runs outside the
|
||||
// promise, so an exception would be uncaught and crash the server.
|
||||
try {
|
||||
const status = response.statusCode || 500;
|
||||
// A `Response` only represents statuses up to 599; Node never hands
|
||||
// this callback a 1xx.
|
||||
if (status > 599)
|
||||
throw new Error(`FIRMS returned unsupported status ${status}`);
|
||||
const headers = new Headers();
|
||||
for (const [name, value] of Object.entries(response.headers)) {
|
||||
if (Array.isArray(value))
|
||||
value.forEach((item) => headers.append(name, item));
|
||||
else if (value !== undefined) headers.set(name, String(value));
|
||||
}
|
||||
const nullBody = NULL_BODY_STATUSES.has(status);
|
||||
const upstream = new Response(
|
||||
nullBody ? null : Readable.toWeb(response),
|
||||
{ status, statusText: response.statusMessage || '', headers },
|
||||
);
|
||||
if (nullBody) response.resume();
|
||||
resolve(upstream);
|
||||
} catch (error) {
|
||||
response.destroy();
|
||||
reject(error);
|
||||
}
|
||||
},
|
||||
);
|
||||
request.on('error', reject);
|
||||
request.end();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a FIRMS URL, retrying once over IPv4 when address selection fails.
|
||||
* Other failures (HTTP status, timeout, DNS) are returned or thrown as is.
|
||||
* @param {string} url
|
||||
* @param {{signal?: AbortSignal}} options
|
||||
* @param {{fetchImpl?: typeof fetch, fetchIpv4Impl?: typeof fetchIpv4, warn?: Function}} [deps]
|
||||
* @returns {Promise<Response>}
|
||||
*/
|
||||
export async function fetchFirms(
|
||||
url,
|
||||
options,
|
||||
{ fetchImpl = fetch, fetchIpv4Impl = fetchIpv4, warn = console.warn } = {},
|
||||
) {
|
||||
try {
|
||||
return await fetchImpl(url, options);
|
||||
} catch (err) {
|
||||
if (!isAddressSelectionFailure(err) || options?.signal?.aborted) throw err;
|
||||
warn(`[firms-proxy] ${describeFetchError(err)} — retrying over IPv4`);
|
||||
return fetchIpv4Impl(url, options);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* NASA FIRMS live active-fire proxy with a memory + disk cache.
|
||||
* Upstream: https://firms.modaps.eosdis.nasa.gov/api/area/csv/{KEY}/{SOURCE}/world/2
|
||||
@@ -91,8 +214,12 @@ export function firmsProxy() {
|
||||
async function fetchSource(key, source) {
|
||||
const url = `https://firms.modaps.eosdis.nasa.gov/api/area/csv/${encodeURIComponent(key)}/${source}/world/2`;
|
||||
const signal = AbortSignal.timeout(60_000);
|
||||
const res = await fetch(url, { signal });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const res = await fetchFirms(url, { signal });
|
||||
if (!res.ok) {
|
||||
// Release the connection now rather than when the deadline fires.
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new Error(`HTTP ${res.status}`);
|
||||
}
|
||||
const records = parseFirmsCsv(
|
||||
await readResponseTextCapped(res, FIRMS_MAX_RESPONSE_BYTES, signal),
|
||||
);
|
||||
@@ -121,7 +248,7 @@ export function firmsProxy() {
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`[firms-proxy] ${source} fetch failed:`,
|
||||
err?.message || err,
|
||||
describeFetchError(err),
|
||||
);
|
||||
sources.push({ source, count: 0, ok: false });
|
||||
}
|
||||
@@ -156,8 +283,13 @@ export function firmsProxy() {
|
||||
statusInflight = (async () => {
|
||||
try {
|
||||
const url = `https://firms.modaps.eosdis.nasa.gov/mapserver/mapkey_status/?MAP_KEY=${encodeURIComponent(key)}`;
|
||||
const res = await fetch(url, { signal: AbortSignal.timeout(10_000) });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const res = await fetchFirms(url, {
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
});
|
||||
if (!res.ok) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new Error(`HTTP ${res.status}`);
|
||||
}
|
||||
const body = await res.json();
|
||||
const used = Number(body?.current_transactions);
|
||||
const limit = Number(body?.transaction_limit);
|
||||
@@ -167,7 +299,7 @@ export function firmsProxy() {
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
'[firms-proxy] mapkey status failed:',
|
||||
err?.message || err,
|
||||
describeFetchError(err),
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,226 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
import {
|
||||
a22WebcamToSource,
|
||||
loadA22SourcesFromOpenDataHub,
|
||||
} from '../../server/providers/cctv/sources.js';
|
||||
import {
|
||||
A22_DEFAULT_GROUND_ELEVATION_M,
|
||||
A22_IMAGE_ORIGIN,
|
||||
DEFAULT_A22_CCTV_URL,
|
||||
} from '../../server/providers/cctv/constants.js';
|
||||
import { createCctvCatalog } from '../../server/providers/cctv/catalog.js';
|
||||
import { fetchWithinHost } from '../../server/providers/cctv/media.js';
|
||||
import { staticFrameRefreshMs } from './cctvLod.js';
|
||||
|
||||
/** One Open Data Hub `WebcamInfo` item, shaped like the live a22 payload. */
|
||||
const item = (
|
||||
patch = {},
|
||||
{ km = '68', lat = 46.53591, lon = 11.49772 } = {},
|
||||
) => ({
|
||||
Id: `A22_${km}`,
|
||||
Active: true,
|
||||
Source: 'a22',
|
||||
Shortname: 'ADS Sciliar',
|
||||
Webcamname: { it: 'ADS Sciliar' },
|
||||
Webcamurl: `https://www.autobrennero.it/WebCamImg/km${km}.jpg`,
|
||||
Mapping: { a22: { km } },
|
||||
GpsInfo: [{ Gpstype: 'position', Latitude: lat, Longitude: lon }],
|
||||
LicenseInfo: {
|
||||
ClosedData: false,
|
||||
LicenseHolder: 'https://www.autobrennero.it/',
|
||||
},
|
||||
...patch,
|
||||
});
|
||||
|
||||
test('an A22 webcam maps to a source on the pinned operator frame', () => {
|
||||
const source = a22WebcamToSource(item());
|
||||
assert.equal(source.id, 'it-a22-km68');
|
||||
assert.equal(source.name, 'A22 ADS Sciliar (km 68)');
|
||||
assert.equal(source.city, 'A22 Brennero');
|
||||
assert.equal(source.cityId, 'a22');
|
||||
assert.equal(source.provider, 'Autostrada del Brennero');
|
||||
assert.equal(source.lat, 46.53591);
|
||||
assert.equal(source.lon, 11.49772);
|
||||
assert.equal(source.headingConfidence, 'low');
|
||||
assert.equal(source.feedType, 'image');
|
||||
assert.equal(source.sourceKind, 'a22-opendatahub');
|
||||
assert.equal(source.code, 'ADS SCILIAR');
|
||||
assert.equal(source.url, `${A22_IMAGE_ORIGIN}km68.jpg`);
|
||||
assert.equal(source.snapshotUrl, source.url);
|
||||
});
|
||||
|
||||
test('ground elevation follows the motorway from the Brenner Pass to the Po', () => {
|
||||
const brenner = a22WebcamToSource(
|
||||
item({ Shortname: 'Brennero' }, { km: '1', lat: 46.99682, lon: 11.50218 }),
|
||||
);
|
||||
const ponte = a22WebcamToSource(
|
||||
item(
|
||||
{ Shortname: 'Ponte Po' },
|
||||
{ km: '272', lat: 45.03145, lon: 10.85369 },
|
||||
),
|
||||
);
|
||||
assert.ok(brenner.groundElevationM > 1300);
|
||||
assert.ok(ponte.groundElevationM < 50);
|
||||
// A kilometre the table does not know falls back to the corridor prior.
|
||||
const unknown = a22WebcamToSource(
|
||||
item({}, { km: '150', lat: 45.95, lon: 11.05 }),
|
||||
);
|
||||
assert.equal(unknown.groundElevationM, A22_DEFAULT_GROUND_ELEVATION_M);
|
||||
});
|
||||
|
||||
test('inactive cameras, off-host frames and bad geometry are dropped', () => {
|
||||
assert.equal(a22WebcamToSource(item({ Active: false })), null);
|
||||
assert.equal(a22WebcamToSource(item({ Source: 'other' })), null);
|
||||
assert.equal(
|
||||
a22WebcamToSource(
|
||||
item({ Webcamurl: 'https://evil.test/WebCamImg/km68.jpg' }),
|
||||
),
|
||||
null,
|
||||
);
|
||||
// The frame must be exactly the camera's own kilometre image.
|
||||
assert.equal(
|
||||
a22WebcamToSource(
|
||||
item({ Webcamurl: 'https://www.autobrennero.it/WebCamImg/km1.jpg' }),
|
||||
),
|
||||
null,
|
||||
);
|
||||
assert.equal(
|
||||
a22WebcamToSource(item({ Mapping: { a22: { km: '../x' } } })),
|
||||
null,
|
||||
);
|
||||
// Milan: plausible, but off the Brenner–Modena corridor.
|
||||
assert.equal(
|
||||
a22WebcamToSource(item({}, { km: '68', lat: 45.46, lon: 9.19 })),
|
||||
null,
|
||||
);
|
||||
assert.equal(a22WebcamToSource(item({ GpsInfo: [] })), null);
|
||||
assert.equal(a22WebcamToSource(null), null);
|
||||
});
|
||||
|
||||
test('the loader dedupes, orders north to south, and degrades to empty', async (t) => {
|
||||
t.mock.method(console, 'log', () => {});
|
||||
t.mock.method(console, 'warn', () => {});
|
||||
const requested = [];
|
||||
const fetchMock = t.mock.method(globalThis, 'fetch', async (url) => {
|
||||
requested.push(String(url));
|
||||
return Response.json({
|
||||
TotalResults: 3,
|
||||
Items: [
|
||||
item(
|
||||
{ Shortname: 'Ponte Po' },
|
||||
{ km: '272', lat: 45.03145, lon: 10.85369 },
|
||||
),
|
||||
item(),
|
||||
item(),
|
||||
item(
|
||||
{ Shortname: 'Brennero' },
|
||||
{ km: '1', lat: 46.99682, lon: 11.50218 },
|
||||
),
|
||||
],
|
||||
});
|
||||
});
|
||||
const cameras = await loadA22SourcesFromOpenDataHub();
|
||||
assert.deepEqual(requested, [DEFAULT_A22_CCTV_URL]);
|
||||
assert.deepEqual(
|
||||
cameras.map((c) => c.id),
|
||||
['it-a22-km1', 'it-a22-km68', 'it-a22-km272'],
|
||||
);
|
||||
|
||||
fetchMock.mock.mockImplementation(
|
||||
async () => new Response('nope', { status: 503 }),
|
||||
);
|
||||
assert.deepEqual(await loadA22SourcesFromOpenDataHub(), []);
|
||||
fetchMock.mock.mockImplementation(
|
||||
async () =>
|
||||
new Response(null, {
|
||||
status: 302,
|
||||
headers: { Location: 'https://evil.test/' },
|
||||
}),
|
||||
);
|
||||
assert.deepEqual(await loadA22SourcesFromOpenDataHub(), []);
|
||||
fetchMock.mock.mockImplementation(async () => {
|
||||
throw new Error('offline');
|
||||
});
|
||||
assert.deepEqual(await loadA22SourcesFromOpenDataHub(), []);
|
||||
});
|
||||
|
||||
const runCatalog = async (t) => {
|
||||
const requested = [];
|
||||
t.mock.method(console, 'log', () => {});
|
||||
t.mock.method(console, 'warn', () => {});
|
||||
t.mock.method(globalThis, 'fetch', async (url) => {
|
||||
const href = String(url);
|
||||
requested.push(href);
|
||||
if (href === DEFAULT_A22_CCTV_URL) {
|
||||
return Response.json({ Items: [item()] });
|
||||
}
|
||||
return Response.json([]);
|
||||
});
|
||||
const sources = await createCctvCatalog({ sourceRoot: '/nonexistent' })();
|
||||
return { requested, sources };
|
||||
};
|
||||
|
||||
const withEnv = async (patch, fn) => {
|
||||
const saved = { ...process.env };
|
||||
try {
|
||||
delete process.env.CCTV_SOURCES_FILE;
|
||||
delete process.env.CCTV_SOURCES_JSON;
|
||||
delete process.env.CCTV_A22_ENABLED;
|
||||
Object.assign(process.env, patch);
|
||||
await fn();
|
||||
} finally {
|
||||
for (const key of Object.keys(process.env)) {
|
||||
if (!(key in saved)) delete process.env[key];
|
||||
}
|
||||
Object.assign(process.env, saved);
|
||||
}
|
||||
};
|
||||
|
||||
test('the A22 lane is wired into the catalog', async (t) => {
|
||||
await withEnv({}, async () => {
|
||||
const { requested, sources } = await runCatalog(t);
|
||||
assert.ok(requested.includes(DEFAULT_A22_CCTV_URL));
|
||||
assert.deepEqual(
|
||||
sources.filter((s) => s.cityId === 'a22').map((s) => s.id),
|
||||
['it-a22-km68'],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('CCTV_A22_ENABLED=0 keeps the lane from being loaded', async (t) => {
|
||||
await withEnv({ CCTV_A22_ENABLED: '0' }, async () => {
|
||||
const { requested, sources } = await runCatalog(t);
|
||||
assert.equal(requested.includes(DEFAULT_A22_CCTV_URL), false);
|
||||
assert.deepEqual(
|
||||
sources.filter((s) => s.cityId === 'a22'),
|
||||
[],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('a redirect on the A22 frame host stays inside WebCamImg/', async () => {
|
||||
const frame = `${A22_IMAGE_ORIGIN}km68.jpg`;
|
||||
const redirectTo = (location) => async (url) =>
|
||||
url === frame
|
||||
? new Response(null, { status: 302, headers: { location } })
|
||||
: new Response('frame', { headers: { 'content-type': 'image/jpeg' } });
|
||||
for (const location of ['/it/area-riservata/', '/WebCamImg/../x.jpg'])
|
||||
assert.equal(
|
||||
await fetchWithinHost(frame, {}, redirectTo(location)),
|
||||
null,
|
||||
location,
|
||||
);
|
||||
const followed = await fetchWithinHost(
|
||||
frame,
|
||||
{},
|
||||
redirectTo('/WebCamImg/km77.jpg'),
|
||||
);
|
||||
assert.equal(await followed.text(), 'frame');
|
||||
});
|
||||
|
||||
test('A22 ambient cards refresh on the highway cadence, not the default', () => {
|
||||
const source = a22WebcamToSource(item());
|
||||
assert.equal(staticFrameRefreshMs(source), 3 * 60 * 1000);
|
||||
});
|
||||
@@ -42,6 +42,9 @@ const PROVIDER_STATIC_REFRESH_MS = Object.freeze({
|
||||
// every 20s showed no change inside 64s but a new frame by ~90-150s later —
|
||||
// matches the other highway packs' 3-minute cadence.
|
||||
'servei català de trànsit': 3 * 60 * 1000,
|
||||
// Autobrennero (A22) refreshes its stills about once a minute (Last-Modified
|
||||
// tracks the minute); 3 min matches the other highway packs.
|
||||
'autostrada del brennero': 3 * 60 * 1000,
|
||||
});
|
||||
|
||||
/**
|
||||
|
||||
@@ -343,6 +343,13 @@ export const DATA_CREDITS = [
|
||||
key: 'panoramax',
|
||||
html: 'Street Level: imagery © <a href="https://panoramax.fr" target="_blank" rel="noopener">Panoramax</a> contributors, licence per picture',
|
||||
},
|
||||
{
|
||||
key: 'a22-cctv',
|
||||
html:
|
||||
'Motorway webcams (A22 Brennero–Modena): images © ' +
|
||||
'<a href="https://www.autobrennero.it/" target="_blank" rel="noopener">Autostrada del Brennero S.p.A.</a>, ' +
|
||||
'camera list via the <a href="https://opendatahub.com/" target="_blank" rel="noopener">Open Data Hub</a> (NOI Techpark)',
|
||||
},
|
||||
{
|
||||
key: 'gbfs',
|
||||
html:
|
||||
|
||||
@@ -1,7 +1,18 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import test from 'node:test';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import dns from 'node:dns/promises';
|
||||
import https from 'node:https';
|
||||
import net from 'node:net';
|
||||
import { Readable } from 'node:stream';
|
||||
import { filterTrailing24h, parseFirmsCsv } from './firmsCsv.js';
|
||||
import {
|
||||
describeFetchError,
|
||||
fetchFirms,
|
||||
fetchIpv4,
|
||||
isAddressSelectionFailure,
|
||||
} from '../../server/providers/firms.js';
|
||||
|
||||
const config = fs.readFileSync(new URL('../../server/providers/firms.js', import.meta.url), 'utf8');
|
||||
const start = config.indexOf(' async function refreshUpstream(key) {');
|
||||
@@ -17,8 +28,8 @@ const recent = { acqDate: '2026-09-11', acqTime: '1100' };
|
||||
// Inject only its upstream, clock and filter dependencies; keep its aggregation
|
||||
// and source-status code intact, including failures while consuming records.
|
||||
function createRefresh(fetchSource, filter = filterTrailing24h) {
|
||||
return new Function('SOURCES', 'fetchSource', 'filterTrailing24h', 'Date', 'console',
|
||||
`return (${refreshSource});`)(SOURCES, fetchSource, filter, { now: () => NOW }, { warn() {} });
|
||||
return new Function('SOURCES', 'fetchSource', 'filterTrailing24h', 'describeFetchError', 'Date', 'console',
|
||||
`return (${refreshSource});`)(SOURCES, fetchSource, filter, describeFetchError, { now: () => NOW }, { warn() {} });
|
||||
}
|
||||
|
||||
test('FIRMS retains large sources in order and filters expired rows', async () => {
|
||||
@@ -78,3 +89,216 @@ test('FIRMS distinguishes all-source failure from successful empty sources', asy
|
||||
assert.deepEqual(result.fires, []);
|
||||
assert.deepEqual(result.sources, SOURCES.map(source => ({ source, count: 0, ok: true })));
|
||||
});
|
||||
|
||||
// #68: Node's network-family auto-selection gives each address a short attempt
|
||||
// timeout, abandons a slow IPv4 handshake for an unreachable IPv6 address, and
|
||||
// reports only `fetch failed`. This is the shape Node 24 throws for it.
|
||||
function addressSelectionFailure() {
|
||||
const v4 = Object.assign(new Error('connect ETIMEDOUT 198.118.194.34:443'), { code: 'ETIMEDOUT' });
|
||||
const v6 = Object.assign(new Error('connect ENETUNREACH 2001:4d0:241a:40c0::34:443'), { code: 'ENETUNREACH' });
|
||||
const cause = Object.assign(new AggregateError([v4, v6]), { code: 'ETIMEDOUT' });
|
||||
return new TypeError('fetch failed', { cause });
|
||||
}
|
||||
// Node 26's fetch wraps the same AggregateError in a ConnectTimeoutError.
|
||||
function wrappedAddressSelectionFailure() {
|
||||
const inner = addressSelectionFailure().cause;
|
||||
const cause = Object.assign(new Error('Connect Timeout Error (attempted addresses: 198.118.194.34:443, 2001:4d0:241a:40c0::34:443, timeout: 10000ms)'),
|
||||
{ name: 'ConnectTimeoutError', code: 'UND_ERR_CONNECT_TIMEOUT', cause: inner });
|
||||
return new TypeError('fetch failed', { cause });
|
||||
}
|
||||
const URL_WITH_KEY = 'https://firms.modaps.eosdis.nasa.gov/api/area/csv/SECRET/VIIRS_SNPP_NRT/world/2';
|
||||
|
||||
test('FIRMS failures log the socket cause, never the keyed URL', () => {
|
||||
const described = describeFetchError(addressSelectionFailure());
|
||||
assert.equal(described,
|
||||
'fetch failed (connect ETIMEDOUT 198.118.194.34:443; connect ENETUNREACH 2001:4d0:241a:40c0::34:443)');
|
||||
assert.ok(!described.includes('SECRET'));
|
||||
assert.equal(describeFetchError(wrappedAddressSelectionFailure()), described, 'Node 26 reports the same attempts');
|
||||
const single = new TypeError('fetch failed', { cause: Object.assign(new Error('getaddrinfo ENOTFOUND x'), { code: 'ENOTFOUND' }) });
|
||||
assert.equal(describeFetchError(single), 'fetch failed (getaddrinfo ENOTFOUND x)');
|
||||
assert.equal(describeFetchError(new Error('HTTP 503')), 'HTTP 503');
|
||||
});
|
||||
|
||||
test('FIRMS retries once over IPv4 when every address fails to connect', async () => {
|
||||
for (const shape of [addressSelectionFailure, wrappedAddressSelectionFailure]) {
|
||||
const retried = [];
|
||||
await fetchFirms(URL_WITH_KEY, {}, {
|
||||
fetchImpl: async () => { throw shape(); },
|
||||
fetchIpv4Impl: async (url) => { retried.push(url); return new Response('csv'); },
|
||||
warn() {},
|
||||
});
|
||||
assert.deepEqual(retried, [URL_WITH_KEY], `${shape.name} is retried`);
|
||||
}
|
||||
const signal = new AbortController().signal;
|
||||
const ok = new Response('csv');
|
||||
const retried = [];
|
||||
const warnings = [];
|
||||
const response = await fetchFirms(URL_WITH_KEY, { signal }, {
|
||||
fetchImpl: async () => { throw addressSelectionFailure(); },
|
||||
fetchIpv4Impl: async (url, options) => { retried.push([url, options]); return ok; },
|
||||
warn: (message) => warnings.push(message),
|
||||
});
|
||||
assert.equal(response, ok);
|
||||
assert.deepEqual(retried, [[URL_WITH_KEY, { signal }]], 'one retry, same URL and deadline');
|
||||
assert.equal(warnings.length, 1);
|
||||
assert.match(warnings[0], /ETIMEDOUT 198\.118\.194\.34:443; .*ENETUNREACH .* retrying over IPv4/);
|
||||
assert.ok(!warnings[0].includes('SECRET'));
|
||||
|
||||
const failure = new Error('connect ETIMEDOUT 198.118.194.34:443');
|
||||
await assert.rejects(fetchFirms(URL_WITH_KEY, {}, {
|
||||
fetchImpl: async () => { throw addressSelectionFailure(); },
|
||||
fetchIpv4Impl: async () => { throw failure; },
|
||||
warn() {},
|
||||
}), (err) => err === failure, 'a failed retry reports its own error');
|
||||
});
|
||||
|
||||
test('FIRMS does not retry anything but an address-selection failure', async () => {
|
||||
const deps = (fetchImpl) => ({
|
||||
fetchImpl,
|
||||
fetchIpv4Impl: async () => assert.fail('must not retry'),
|
||||
warn: () => assert.fail('must not warn'),
|
||||
});
|
||||
const serverError = new Response('down', { status: 503 });
|
||||
assert.equal(await fetchFirms(URL_WITH_KEY, {}, deps(async () => serverError)), serverError);
|
||||
for (const thrown of [
|
||||
new TypeError('fetch failed', { cause: Object.assign(new Error('getaddrinfo ENOTFOUND x'), { code: 'ENOTFOUND' }) }),
|
||||
new TypeError('fetch failed', { cause: new AggregateError([]) }),
|
||||
new TypeError('fetch failed'),
|
||||
new DOMException('The operation was aborted due to timeout', 'TimeoutError'),
|
||||
]) {
|
||||
await assert.rejects(fetchFirms(URL_WITH_KEY, {}, deps(async () => { throw thrown; })), (err) => err === thrown);
|
||||
}
|
||||
const controller = new AbortController();
|
||||
controller.abort();
|
||||
const aborted = addressSelectionFailure();
|
||||
await assert.rejects(fetchFirms(URL_WITH_KEY, { signal: controller.signal },
|
||||
deps(async () => { throw aborted; })), (err) => err === aborted, 'an expired deadline is not retried');
|
||||
});
|
||||
|
||||
test('the IPv4 retry connects to an IPv4 address only', async (t) => {
|
||||
// `localhost` resolves to both ::1 and 127.0.0.1. Listen on both with the
|
||||
// same port and record which one the retry reaches; the TLS handshake then
|
||||
// fails against a plain socket, which is fine — only the connection matters.
|
||||
const reached = [];
|
||||
const listen = (host, port) => new Promise((resolve, reject) => {
|
||||
const server = net.createServer((socket) => { reached.push(host); socket.destroy(); });
|
||||
server.once('error', reject).listen(port, host, () => resolve(server));
|
||||
});
|
||||
const v4 = await listen('127.0.0.1', 0);
|
||||
t.after(() => v4.close());
|
||||
const { port } = v4.address();
|
||||
let v6;
|
||||
try {
|
||||
v6 = await listen('::1', port);
|
||||
} catch {
|
||||
t.skip('no IPv6 loopback on this host, so IPv4-only cannot be told apart');
|
||||
return;
|
||||
}
|
||||
t.after(() => v6.close());
|
||||
const order = await dns.lookup('localhost', { all: true });
|
||||
if (order[0]?.family !== 6) {
|
||||
t.skip('localhost resolves IPv4 first here, so IPv4-only cannot be told apart');
|
||||
return;
|
||||
}
|
||||
await assert.rejects(fetchIpv4(`https://localhost:${port}/`, { signal: AbortSignal.timeout(5_000) }));
|
||||
assert.deepEqual(reached, ['127.0.0.1']);
|
||||
});
|
||||
|
||||
// Answer the IPv4 retry with `status`, invoking the response callback
|
||||
// asynchronously as `https.request` does, outside the caller's promise.
|
||||
// The returned promise rejects on an uncaught exception, so a regression fails
|
||||
// the race at once instead of crashing the test runner.
|
||||
function mockIpv4Status(t, status, statusMessage = '') {
|
||||
let crash;
|
||||
const crashed = new Promise((_resolve, reject) => (crash = reject));
|
||||
const onUncaught = (error) =>
|
||||
crash(new Error(`uncaught exception: ${error.message}`));
|
||||
process.on('uncaughtException', onUncaught);
|
||||
t.after(() => process.off('uncaughtException', onUncaught));
|
||||
t.mock.method(https, 'request', (_url, _options, callback) => {
|
||||
const request = new EventEmitter();
|
||||
request.end = () =>
|
||||
setImmediate(() =>
|
||||
callback(
|
||||
Object.assign(Readable.from([]), {
|
||||
statusCode: status,
|
||||
statusMessage,
|
||||
headers: { 'content-type': 'text/csv' },
|
||||
}),
|
||||
),
|
||||
);
|
||||
return request;
|
||||
});
|
||||
return crashed;
|
||||
}
|
||||
|
||||
for (const status of [204, 205, 304]) {
|
||||
test(`the IPv4 retry answering null-body status ${status} resolves instead of crashing`, async (t) => {
|
||||
const crashed = mockIpv4Status(t, status);
|
||||
const response = await Promise.race([fetchIpv4(URL_WITH_KEY), crashed]);
|
||||
assert.equal(response.status, status);
|
||||
assert.equal(response.body, null);
|
||||
});
|
||||
}
|
||||
|
||||
for (const status of [600, 999]) {
|
||||
test(`the IPv4 retry answering out-of-range status ${status} rejects instead of crashing`, async (t) => {
|
||||
const crashed = mockIpv4Status(t, status);
|
||||
await assert.rejects(
|
||||
Promise.race([fetchIpv4(URL_WITH_KEY), crashed]),
|
||||
new RegExp(`unsupported status ${status}`),
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
test('the IPv4 retry answering an invalid reason phrase rejects instead of crashing', async (t) => {
|
||||
const crashed = mockIpv4Status(t, 200, 'OK\x7f');
|
||||
await assert.rejects(Promise.race([fetchIpv4(URL_WITH_KEY), crashed]), TypeError);
|
||||
});
|
||||
|
||||
// The fixtures above copy Node's error shape; check it against a real fetch so
|
||||
// a Node or undici change that stops matching fails here instead of silently
|
||||
// disabling the retry. A closed port on `localhost` refuses both addresses.
|
||||
test('a real all-addresses connect failure is recognised as address selection', async (t) => {
|
||||
const probe = net.createServer();
|
||||
try {
|
||||
await new Promise((resolve, reject) => probe.once('error', reject).listen(0, '::1', resolve));
|
||||
} catch {
|
||||
t.skip('no IPv6 loopback on this host');
|
||||
return;
|
||||
}
|
||||
const { port } = probe.address();
|
||||
await new Promise((resolve) => probe.close(resolve));
|
||||
const v4 = net.createServer();
|
||||
await new Promise((resolve, reject) => v4.once('error', reject).listen(port, '127.0.0.1', resolve));
|
||||
await new Promise((resolve) => v4.close(resolve));
|
||||
const err = await fetch(`http://localhost:${port}/`).then(() => null, (error) => error);
|
||||
assert.ok(err, 'a closed port refuses the connection');
|
||||
assert.equal(isAddressSelectionFailure(err), true, describeFetchError(err));
|
||||
assert.match(describeFetchError(err), /ECONNREFUSED .*; .*ECONNREFUSED/);
|
||||
});
|
||||
|
||||
// #68 only stays fixed while both upstream calls go through fetchFirms; the
|
||||
// refresh harness injects fetchSource, so guard the call sites directly.
|
||||
test('every FIRMS upstream request goes through the IPv4 fallback', async () => {
|
||||
assert.doesNotMatch(config, /await fetch\(/, 'no FIRMS request bypasses fetchFirms');
|
||||
const fetchStart = config.indexOf(' async function fetchSource(key, source) {');
|
||||
assert.notEqual(fetchStart, -1, 'FIRMS fetchSource exists');
|
||||
const fetchSourceText = config.slice(fetchStart, config.indexOf('\n }', fetchStart) + 4);
|
||||
const calls = [];
|
||||
const response = new Response('latitude,longitude\n');
|
||||
const fetchSource = new Function('fetchFirms', 'readResponseTextCapped', 'parseFirmsCsv',
|
||||
'FIRMS_MAX_RESPONSE_BYTES', 'AbortSignal', 'fetch', `return (${fetchSourceText});`)(
|
||||
async (url, options) => { calls.push(['fetchFirms', url, options.signal]); return response; },
|
||||
async (res, cap, signal) => { calls.push(['read', res, signal]); return 'csv'; },
|
||||
() => [],
|
||||
1024,
|
||||
AbortSignal,
|
||||
() => assert.fail('fetchSource must call fetchFirms, not fetch'),
|
||||
);
|
||||
assert.deepEqual(await fetchSource('SECRET', 'MODIS_NRT'), []);
|
||||
assert.equal(calls.length, 2);
|
||||
assert.equal(calls[0][0], 'fetchFirms');
|
||||
assert.ok(calls[0][2] instanceof AbortSignal);
|
||||
assert.deepEqual(calls[1], ['read', response, calls[0][2]], 'the body read shares the request deadline');
|
||||
});
|
||||
Reference in new issue
Block a user