- A background catalog refresh that rejects now keeps the old catalog for
another TTL, as an empty refresh already did, so a refresh that keeps
failing is retried once per TTL instead of on every request.
- Trim notes are keyed by the source list they describe, and /sources asks
for the list it served, so a request answered during a refresh never pairs
the old cameras with the new list's notes.
- Tests: a failed background refresh (one warning, no unhandled rejection,
no retry before the TTL, trim notes stay with their list) and an empty
background refresh (old catalog kept, no upstream fetch before the TTL).
The race assertion's message now names the failure; its timer is cleared.
- The Tarktee test no longer imports the new constant, so on main it fails
on its own assertion instead of taking the whole file down.
- precompute-cctv-heights waits 60 s for a cold catalog, past Tarktee's 30 s.
- CURRENT-STATE notes the 30 s bound and the retry spacing.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- State FL511's terms the same way everywhere: individual, personal use
only, and any other use (a public deployment included) needs FDOT's
express written consent. .env.example, README, dataCredits, the source
licence string, the catalog comment and the changelog fragment said only
"individual, non-commercial use".
- Page the ArcGIS catalog on exceededTransferLimit alone and advance the
offset by the rows each page returned, so a lower server-side max
record count no longer cuts the catalog short at the first short page.
- Order pages by OBJECTID_1 (this layer's object id; a plain OBJECTID is
rejected with HTTP 400) so offset paging is stable.
- Raise the page ceiling to 40 and warn when it is hit.
- Tests: short pages with exceededTransferLimit keep paging, orderByFields
is sent, and a service that ignores resultOffset stops at the ceiling.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Attach the background-refresh warning where the refresh starts, so a
failed refresh logs once instead of once per request served meanwhile.
The test now checks that requests during a refresh share it, and turns
the hang off before releasing the fetches so a retrying loader can't
stall it.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
An expired catalog made every /api/cctv request (camera list and frames)
wait for the full refresh, which lasts as long as the slowest of the live
packs: Tarktee's DATEX feeds took 2 to 16.6 s across ten requests. With
Tarktee's timeout raised to 30 s, that wait could reach 30 s at each
15-minute refresh. The cached list is now returned while a single-flight
refresh replaces it in the background; only a first load, with nothing
to serve, waits. A background refresh that throws is logged.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The Estonian Transpordiamet feeds answered in 2 to 16.6 s across ten
requests (120 and 160 KB), and both must arrive for any camera to load,
so the shared 15 s catalog timeout dropped the whole pack on about a
third of loads ("Tarktee DATEX download error: The operation was aborted
due to timeout"), for the 15 minutes until the next catalog refresh.
TARKTEE_FETCH_TIMEOUT_MS gives the two requests 30 s; other packs keep
15 s.
Live: 175 cameras in 3.3 s.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review follow-up on #975. The shared same-site gate refuses any forwarding header, which is right for routes that spend a key, but terrain is keyless and needed for rendering: behind the reverse proxy the README documents, every /api/terrain/heights chunk got 403 and fell back to the geoid. Terrain now uses admitSameSiteKeyless, which keeps the Sec-Fetch-Site and exact-Origin checks (comparing Origin against a single X-Forwarded-Proto scheme) but does not refuse forwarding headers. The shared admitSameSite gate is unchanged for every other route. The changelog entry moves to changelog.d/975-terrain-cross-site-gate.md.
Co-authored-by: daikaginza <[email protected]>
overpassLooksRuntimeError and overpassLooksRateLimited searched the whole
200 body, element tags included. A result containing a place named
"Timed Out Escape Rooms" (or any tag with "runtime error", "out of
memory", "too many requests" or "rate_limited") was refused as an
upstream failure, and the endpoint was put on cooldown for every other
query.
Parse the body once. A data reply (one with an elements array) reports
failure in its remark, so the phrase checks run on the remark. Any other
body, an error page or JSON without elements, is matched whole as
before.
Conflicts:
- CCTV HUD: main's datasheet FOV, capability range and spec summary,
with their labels translated; specFovCapabilityToken takes a format
so the HUD can pass a translated one. Datasheet resolution labels are
camera data (i18n-ignore).
- Transit: main's route labels and operator mode words, with the
generic words still translated; an operator's own word stays a name.
Catalogs: es/ru 2566 messages each, complete.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
main now rejects out-of-range coordinates at the proxy itself (19fcc00b), the
same check this branch added. main's version is kept, and the changelog entry
no longer claims that part; the cache cap, file ceiling and rate limit remain.
Replacement buses on 5xx streetcar routes were drawn as streetcars, and
Flexity streetcars reporting no route were drawn as buses. A feed can now
supply vehicleMode(id); TTC maps 4400-4699 to streetcars and every other
fleet number to buses, falling back to the route otherwise.
Number() reads null, blank strings and booleans as 0 (or 1), and
/api/cctv/sources sends an unknown number as JSON null. A source-pack
camera that left out its pose therefore reached the browser as zeros
clamped to each field's floor (north, pitch -2, FOV 20, range 120 m,
mount 6 m, ground 0 m) instead of taking the defaults, and a null or
blank position put the camera at 0,0.
toFiniteNumber (server) and safeNumber (browser) now return the fallback
for absent, blank and boolean values, as validateBox already does for
Recent Imagery. The same conversion fixes the per-pack paths: a Tallinn
row with a null heading no longer reads as a high-confidence north-facing
camera, a Caltrans camera with a blank elevation keeps its 150 m ground
prior, and TfL or Catalonia cameras with a null or blank coordinate are
dropped instead of kept at 0,0 or on the zero meridian.
Resolve the CHANGELOG.md conflict by keeping both sides: this PR's cyclone
entry stays above main's newer entries; no text changed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
/api/cctv/media relayed still-image cameras with an ordinary fetch: it followed
redirects to any host, passed through any content type and had no body cap.
Stills are served by /api/cctv/frame, which keeps redirects on the registered
origin, accepts only image/* and caps the body, and no client requests a still
through /media. /media now answers 404 for anything but HLS, MP4/WebM and
MJPEG, before any upstream request.
Ids were built from the feed's fid, which embeds an upstream server-start stamp,
so every upstream restart renamed all cameras (it already had since the PR
fixture). Ids now come from the validated frame URL (cat-sct-<stem>, or host
and path for partner cameras), which also collapses the five duplicate rows.
Frame links must sit directly in each host's frame directory with no explicit
port, and sctidcam must be a bare image filename. Numeric entity decoding
handles hex references and leaves invalid code points as written, so one bad
name no longer empties the pack. Counts in the docs are now ~130.
The catalog fetch now refuses redirects, cancels rejected bodies and reads
through the shared capped reader before the windows-1252 decode. The feed sends
cardinal codes plus B (both), O and null, not BW or diagonals; tests, comments
and docs now say so, with unchanged id-hash fallbacks. Documents the 8-1,200
per-pack range, drops the unverified low-volume compliance claim for facts and
a link to WSDOT's terms, and stops claiming the per-pack defaults fit under
the 4,000 cap: it is filled round-robin.
The camera endpoints are the eismoinfo.lt website backend, which the Via
Lietuva open-data statement does not list, so the pack now follows the NYC
opt-in pattern (CCTV_LITHUANIA_ENABLED=1) and the docs say why. A failed
catalog request now aborts its sibling instead of leaving it open until the
timeout, and a redirect is logged as not followed.
The catalog fetch now refuses redirects, cancels rejected bodies and reads
through the shared capped JSON reader. Duplicate ids are dropped before
prioritization so they cannot use up the per-pack cap. Vendor-generic model
strings (PTZ, PTZ IP, BOSCH/PTZ) are treated as no model. Comments and credits
say why WSDOT rows are skipped (their frames are WSDOT-hosted, outside the
pinned origin) and name Newcastle as the partner city that appears.
Adds the Toronto settings to .env.example and a current-state section for the
pack, restores an approximate camera count in the README, and corrects the
claim that adding a pack never trims another region: with the shared
4,000-camera cap, the largest packs give way round-robin. Fixes two stale
comments about the list's refresh and the ground-height sidecar.
Keeps main's 4,000 catalog-wide camera cap, which main pins in tests; WSDOT
shares it round-robin with the other packs. The branch's earlier cap raise and
its removal of main's cap tests are reverted in this merge.
Upstream's #641 (bikeshare Toronto, city failures scoped to range with
partial outages read DEGRADED/STALE) replaced the layer-level GBFS
error strings this branch had msg()-marked, in the two paths that
conflicted.
- ingestion.js/viewport.js taken as upstream's per-city
_failedCityIds tracking; the layer-level error assignments are gone.
- 'GBFS fetch error' still reaches the panel via stats.error from
controls.js, so its mark moved there (translateFeedError keeps
translating it).
- 'GBFS status update failed' no longer exists anywhere: pruned from
es/ru and from the layerStatusText SITES table, which now pins the
controls.js site instead of the removed ones.
Also brings #581 (upstream-read caps, OpenSky 429 cooldown) - no i18n
surface.
npm run i18n: complete (2553), no unmarked strings. npm test: only the
pre-existing environmental OPENAI_API_KEY leak in devFreshDotenv fails.
Build passes.
A 429 now starts the credit cooldown from its headers and drops the body
unread, so a stalled or oversized 429 cannot skip the governor; an oversized
snapshot is not retried. A 429 without retry-after now waits the intended
2 minutes (headers.get() returns null, which Number() read as 0 s). Adds
oversized-stream tests for OpenSky, adsb.lol, CelesTrak and TomTom that fail
on main, names the six reads in the changelog, and corrects the docs on
what was already bounded.
Conflicts:
- KartaView viewer moved into the shared photoViewer.js on main; its
labels now take the provider name ("{name} street-level photo") and
are translated there.
- Weather: main's cloud filter strength slider, with its labels in t().
- Street Level caption: translated creator line plus main's licence.
Panoramax status and error texts are marked like KartaView's, and
"Panoramax answered HTTP {status}" gets a template in the panel and the
layer row. Catalogs: es/ru 2553 messages each, complete.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
quanticsoul4772