- State FL511's terms the same way everywhere: individual, personal use
only, and any other use (a public deployment included) needs FDOT's
express written consent. .env.example, README, dataCredits, the source
licence string, the catalog comment and the changelog fragment said only
"individual, non-commercial use".
- Page the ArcGIS catalog on exceededTransferLimit alone and advance the
offset by the rows each page returned, so a lower server-side max
record count no longer cuts the catalog short at the first short page.
- Order pages by OBJECTID_1 (this layer's object id; a plain OBJECTID is
rejected with HTTP 400) so offset paging is stable.
- Raise the page ceiling to 40 and warn when it is hit.
- Tests: short pages with exceededTransferLimit keep paging, orderByFields
is sent, and a service that ignores resultOffset stops at the ceiling.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Ids were built from the feed's fid, which embeds an upstream server-start stamp,
so every upstream restart renamed all cameras (it already had since the PR
fixture). Ids now come from the validated frame URL (cat-sct-<stem>, or host
and path for partner cameras), which also collapses the five duplicate rows.
Frame links must sit directly in each host's frame directory with no explicit
port, and sctidcam must be a bare image filename. Numeric entity decoding
handles hex references and leaves invalid code points as written, so one bad
name no longer empties the pack. Counts in the docs are now ~130.
The catalog fetch now refuses redirects, cancels rejected bodies and reads
through the shared capped reader before the windows-1252 decode. The feed sends
cardinal codes plus B (both), O and null, not BW or diagonals; tests, comments
and docs now say so, with unchanged id-hash fallbacks. Documents the 8-1,200
per-pack range, drops the unverified low-volume compliance claim for facts and
a link to WSDOT's terms, and stops claiming the per-pack defaults fit under
the 4,000 cap: it is filled round-robin.
The catalog fetch now refuses redirects, cancels rejected bodies and reads
through the shared capped JSON reader. Duplicate ids are dropped before
prioritization so they cannot use up the per-pack cap. Vendor-generic model
strings (PTZ, PTZ IP, BOSCH/PTZ) are treated as no model. Comments and credits
say why WSDOT rows are skipped (their frames are WSDOT-hosted, outside the
pinned origin) and name Newcastle as the partner city that appears.
Adds the Toronto settings to .env.example and a current-state section for the
pack, restores an approximate camera count in the README, and corrects the
claim that adding a pack never trims another region: with the shared
4,000-camera cap, the largest packs give way round-robin. Fixes two stale
comments about the list's refresh and the ground-height sidecar.
Keeps main's 4,000 catalog-wide camera cap, which main pins in tests; WSDOT
shares it round-robin with the other packs. The branch's earlier cap raise and
its removal of main's cap tests are reverted in this merge.
Keep both: Catalonia CCTV alongside the NYC DOT TMC pack in the source
imports, README layer table and CURRENT-STATE data table, and both
CHANGELOG entries.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The pack shipped a flat 90 m. Sampling a DEM at all 336 camera sites puts
the network between 76 m (Queens Quay) and 209 m (Keele & Steeles), mean
116 m: a median error of 14 m against the old value, 88 m at the 90th
percentile and 119 m at worst, with the Allen Road cameras — 170-197 m —
the furthest underground.
No constant fits a city with that much relief; the precomputed ground
heights are what place a camera properly, and until this pack has them the
prior is the fallback for a sidecar miss or an edited pose. The mean is at
least wrong in both directions rather than only one.
This value is not part of the served pose, so it changes no pose hash and
invalidates no precomputed entry.
The province's 511 pack covers Ontario's highways; this adds the municipal
network underneath it — roughly 336 cameras across downtown arterials plus
the Gardiner, DVP, Allen and Lake Shore, from the City's own open data.
The list is keyless and carries no frame URL, so frame URLs are built from
the convention the dataset documents (`CameraImages/loc####.jpg`) and a
camera number validated as plain digits. The frame proxy is therefore
pinned to the portal origin by construction rather than by trusting a
field. The list fetch refuses redirects, bounds the body it buffers, and
releases the transport on every rejection path.
The dataset publishes no camera facing. `D1`-`D4` look like a compass and
map trivially onto one, but they describe the *comparison* images — static
reference stills captured in 2016 and served from a different path — not
the live pan-tilt-zoom frame. A heading derived from them would be
confidently wrong for every camera in the city, so every camera takes the
shared id-hash fallback at low confidence, as headingless Austin, TfL,
Fintraffic and Calgary cameras do. A test pins that down so it is not
"fixed" later.
Names stay verbatim: the feed's labels are upper-case, and title-casing
would mangle MCCOWAN RD and ST HELEN'S AVE more often than it helped.
Carries the Open Government Licence – Toronto attribution the licence
requires. Disable with `CCTV_TORONTO_ENABLED=0`.
Register the keyless webcams.nyctmc.org catalog as a live CCTV source,
synthesize frame URLs from validated camera UUIDs, and keep the nearest
cameras to the five borough cores under the existing catalog ceiling.
Resolve the source-pack conflicts with Statens vegvesen (keep both packs).
The Norway pack's 900 default pushed the per-pack defaults to 5,204, past
the 4,500 catalog cap this PR enforces, so raise DEFAULT_CCTV_MAX_SOURCES
to 5,500 and CCTV_MAX_SOURCES_CEILING to 6,000.
- Lower the Vegvesen default cap to 900, still above the ~850 working
cameras, so the per-pack defaults stay closer to the catalog default.
- Name the flat ground-elevation prior as a known limitation: the feed
is 2D, and mountain-pass cameras can sit below terrain where neither
the point-height prior nor the ground snap resolves.
- README: ~3,900 public cameras.
- Use one HLS figure (~135 working) in CHANGELOG and DATA_SOURCES.
- DATA_SOURCES: list the frame and HLS hosts alongside the catalog host.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Norway's ~850 working road cameras load from one keyless GeoJSON request
to the agency's OGC API view of its DATEX 3.1 CCTV table, so no DATEX
account is needed. Faulty cameras are dropped, every camera sits under a
"Norway" category, and the ~135 that publish HLS play as live video
through the existing in-memory relay with the still as fallback. Frames
and manifests are pinned to each camera's own path on the agency hosts.
CCTV_VEGVESEN_ENABLED=0 disables the pack, CCTV_VEGVESEN_MAX_SOURCES
caps it (nearest the largest cities first) and CCTV_VEGVESEN_VIDEO=0
keeps stills only. Attributed under NLOD 2.0.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The per-pack defaults already summed to 4,054 before WSDOT (4,304 with
it), past the documented 4,000 cap, so a default install was trimming.
The existing sum check in the DriveBC test listed packs by hand and
missed Calgary and DelDOT. Replace it with a check in
cctvCatalogCap.test.mjs that sums every exported DEFAULT_*_MAX_SOURCES,
and drop the Calgary test that pinned the cap at exactly 4,000.
Address review feedback on #595:
- Reject frame links carrying userinfo (user:pass@) in
normalizeCataloniaImageUrl, so catalog data cannot inject credentials
into the server-side frame request on an allowlisted host.
- Fetch cameres.xml over https (verified: same 200 body, no redirect),
refuse redirects with the unread body released, and read the body
through readResponseTextCapped with a 1 MiB CATALONIA_MAX_CATALOG_BYTES
cap instead of an unbounded resp.text().
- Add regression tests for userinfo links on every allowed host, the
refused redirect, and declared and streamed oversize bodies.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Adds King County DOT Road Services' cameras as a keyless source pack, from
the county's public ArcGIS FeatureServer layer ("King County and WSDOT
Traffic Cameras"). 59 live cameras against the current feed.
- The `outFields` list is explicit on purpose: the layer also carries
asset-administration columns (including staff names) that must never be
fetched.
- Only live, active cameras with frames on the county's own
info.kingcounty.gov image handler are registered; the feed writes http://
and the handler serves HTTPS, so URLs are upgraded and pinned before
registration. WSDOT-owned rows are skipped so a camera never appears twice
when a statewide WSDOT pack is present; the few city-owned cameras the
county hosts (Redmond, Sammamish, Newcastle, Snoqualmie) carry their owner
as the per-camera credit, DriveBC-style.
- The layer publishes no facing: headings use the shared id-hash fallback at
low confidence, correctable with the calibration gizmo.
- Uniquely among the packs, the county publishes each camera's hardware
Manufacturer/Model (Cohu 3950/3965, Bosch AUTODOME IP 5000i, Axis
Q6135-LE, Siqura PD910). The model string is whitespace-collapsed (the
feed carries stray newlines) and carried on the source as `model`, with an
additive passthrough in normalizeSourceItem (same pattern as poseSource)
so identified-hardware consumers can use it. All 59 live cameras carry a
model value.
- Attribution per the KCGIS Center terms, registered in the attribution
popover and documented in DATA_SOURCES.md.
- Defaults: the whole catalog loads (125 cap); CCTV_KINGCOUNTY_MAX_SOURCES
changes the cap, CCTV_KINGCOUNTY_ENABLED=0 turns the pack off.
- Tests: filtering (owner/live/status/host/geometry/bbox), HTTPS upgrade,
model whitespace-collapse and passthrough, partner-city credit, env cap,
upstream-failure fallback, and normalizeSourceItem model semantics.
Adds Washington State's highway cameras as a keyless source pack, from the
Esri JSON layer WSDOT's own Travel Center map consumes
(data.wsdot.wa.gov/travelcenter/Cameras.json, ~1,700 point features).
Closes#624.
- Geometry arrives in EPSG:3857; converted to WGS84 on load and bounded to
a Washington bounding box.
- Only frames on WSDOT's own images.wsdot.wa.gov host are registered — the
catalog also lists ~70 partner cameras (Oregon DOT, City of Seattle,
lodges, airports) whose own terms apply; those are skipped rather than
proxied. 1,630 cameras survive the filters against the live feed.
- CompassDirection letters (N/E/S/W) give a high-confidence heading; the
majority `B` ("both directions") rows carry no single facing and use the
shared id-hash fallback at low confidence, with the usual two pose
personalities.
- The feed body carries Windows-1252 bytes (en-dashes in titles) with no
charset declaration, so it is decoded explicitly before parsing.
- WSDOT's layer metadata marks the feed "low volume": the 15-minute catalog
cache and per-request frame fetches respect that, frames are never stored
or redistributed, and the WSDOT credit is registered in the attribution
popover. A keyed WSDOT Traveler Information API serves the same population
for higher-volume use and is deliberately not required.
- Defaults: 250 cameras nearest Seattle and Spokane; CCTV_WSDOT_MAX_SOURCES
changes the cap, CCTV_WSDOT_ENABLED=0 turns the pack off.
- Tests: Mercator conversion against a real layer feature, official-host and
malformed-row filtering, compass mapping and both pose personalities,
cp1252 decoding, env cap, upstream-failure fallback.
Keeps PR #595's Catalonia CCTV pack in sync with upstream's DelDOT
CCTV pack (#699) and the wind/weather doc updates from #617. The
sources.js conflict was a false positive from git's line-based diff
interleaving two structurally similar but independent loader
functions (loadCataloniaSourcesFromOpenData / loadDelDOTSourcesFromOpenData);
resolved by keeping both as complete, separate functions.
Full unit suite passes (4743/4743, 1 pre-existing Node-version skip).
Wind (NOAA GFS and ECMWF IFS: animated particles, colour field, a reading at map centre), observed weather (NOAA MRMS radar, GOES and NESDIS infrared clouds, lightning density) on one shared history timeline, NHC and CPHC cyclone advisories with tracks and cones, and a WEATHER panel in the right rail. On 3D Tiles the observed products render as raised shells with a sharper window around the view.
Co-authored-by: Gustavo Beneduzi <[email protected]>
Co-authored-by: Daniel Slay <[email protected]>
Co-authored-by: Rehaan Delmotra <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
The media relay only deadlined the wait for upstream response headers. Once
headers arrived the timer stopped, so an upstream that answered and then sent
nothing held both the client response and the upstream socket open for as long
as the camera host allowed. The declared Content-Length ceiling was the only
body bound, and a chunked or length-less body had none.
A 30-second idle deadline now measures the gap between upstream chunks and
tears the stream down through the existing release path. Because a paused pipe
also stops upstream bytes arriving, the deadline reschedules itself whenever
the response is still waiting to drain, so a viewer on a slow link is not read
as a dead camera. Live feeds that keep delivering frames are unaffected.
One WFS/GML XML feed (~135 cameras) aggregating three publishers under a
`font` field: SCT's own highway cameras plus hotlinked Barcelona (IMI) and
Terrassa municipal cameras. Frame URLs are checked against a small host
allowlist rather than one pinned origin, and every non-SCT camera carries
its real owner as a `credit` shown beside the shared provider. The feed
carries no compass heading, so every camera loads at low heading confidence
(same as TfL and Fintraffic), and ground elevation is primed per publisher
rather than one flat prior.
SCT's own frame link needed a workaround: RenderService always 302s to a
second endpoint (TransitCamera) on a Location the upstream hardcodes as
plain http even when requested over https — refused by fetchWithinHost's
same-origin (scheme-included) redirect policy, by design (stops an upstream
steering a host-pinned request into a plaintext downgrade). Separately,
mct.gencat.cat's TLS setup fails the handshake with Node's fetch outright
(ERR_SSL_WRONG_SIGNATURE_TYPE) even though other HTTP clients complete it.
The loader builds the TransitCamera URL itself, over plain http, from
RenderService's sctidcam id, instead of registering the broken upstream
link — the only transport this host actually serves to this runtime.
Verified against the live feed and the production fetch path
(fetchCctvImageFromUpstream): all 135 cameras return real image bytes, 0
false positives. Ambient-card refresh cadence added to src/data/cctvLod.js
(3 minutes), matching the upstream update rate measured empirically (no
change inside 64s, changed within ~90-150s) — the same cadence already used
for Caltrans/TfL/Ontario/TxDOT.
CCTV_CATALONIA_MAX_SOURCES sets the cap and CCTV_CATALONIA_ENABLED=0 turns
the pack off. No overlap with any existing pack (checked catalog.js and the
git history for Spain/Catalonia/Barcelona coverage: none exists).
Verified before this commit: format, check:boundaries, unit suite (3692
tests), and build all green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Supersedes #438 by @rileygramlich, ported onto the current component layout with fixes from review.
Verified before merge: format, package boundaries, unit suite, build, and the browser tracking suite; a browser harness for the change where one applies; adversarial review applied.
Co-authored-by: rileygramlich <[email protected]>
