Commit Graph
162 Commits
Author SHA1 Message Date
Said Turksever d6c60f94b6 fix(cctv): pin A22 frame redirects to WebCamImg/ and refresh on the highway cadence
- Add A22_IMAGE_ORIGIN to the pinned frame directories, so a same-host
  redirect from www.autobrennero.it/WebCamImg/ cannot leave that directory,
  as for the other directory-pinned packs.
- Give Autostrada del Brennero a 3-minute ambient refresh in cctvLod.js,
  like the other highway packs, instead of the 5-minute default.
- DATA_SOURCES: the Open Data Hub now tags every A22 record CC0; note that
  this is the aggregator's metadata, not Autobrennero's own terms.
- Tests for the redirect pin and the refresh cadence.
2026-10-30 14:50:53 +00:00
Said Turksever 38302a27a4 Merge upstream/main into feat/cctv-a22-autobrennero
Resolve conflicts with the NYC, Toronto, Lithuania, Florida, Taiwan and
Kartverket work on main. The CHANGELOG entry moves to
changelog.d/956-a22-brennero-webcams.md, as main now asks.
2026-10-30 14:50:41 +00:00
Mazeyar Moeini Feizabadi 4da002bd42 Merge current main into S2 GhostMaps and preserve updated import and voice behavior 2026-10-26 16:25:49 +01:00
Said TurkseverandClaude Opus 5.5 f93982f55e Carry server structural rejections into Radio diagnostics
The proxy now skips malformed upstream rows individually but counts
them, capped at 10,000, in the catalog metadata and the /stations
envelope (rejectedStationCount). The count never fails health policy,
so an otherwise healthy generation is still accepted. The client
validates the field, adds its own rejected rows after admission, and
reports degraded coverage when either side skipped rows. Adds an
end-to-end proxy-to-client test and aligns DATA_SOURCES.md and
CURRENT-STATE.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 12:39:56 +01:00
Mazeyar Moeini Feizabadi d5c49d35e8 Merge latest main into S2 GhostMaps contribution 2026-10-22 02:41:26 +02:00
German KhokhlovandGPT-6 27d4c0ba9c fix(sky): preserve dark model lighting and address review
Merge origin/main and retain both format-scope additions.
Give StarrySky a separate black environment cube map and release it on destroy.
Guard anime updates when StarrySky is absent, require a visible Normal sky
before anime QA, and compare a lit airplane against black and white controls.
Document the sky background contract and its verification.

Co-Authored-By: GPT-6 (Codex) <[email protected]>
2026-10-21 14:21:14 +04:00
GPTmadeitandClaude Opus 5.5 1f77d2290c feat: add S2 GhostMaps live intel layer
Adds an "S2 GhostMaps" layer (Events group) that renders S2 Underground's
GhostMaps KMZ exports (Common Intelligence Picture, Kinetic 30d, Border
Crisis, Iran CIP/Kinetic, Yemen FLOT, latest incident drop).

- server/providers/ghostMaps.js: watcher polls the repo's git smart-HTTP
  ref advertisement (default every 15s, GHOSTMAPS_POLL_SECONDS), fetches
  the tree only on a new head, caches commit-pinned files by blob sha and
  pushes catalog updates to browsers over SSE (/api/ghostmaps/events).
- src/layers/ghostMaps: KML reader, ArcGIS popup-table parsing, per-product
  data sources, product chips, horizon-culled pins, report cards.
- Unit tests, QA gate (scripts/qa-ghostmaps.mjs), credits (CC BY-NC-SA 4.0),
  i18n, voice manifest, layer-state token 5.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-20 16:24:53 -04:00
Said Turksever 1ec21c59ce Merge remote-tracking branch 'upstream/main' into fix-1237
# Conflicts:
#	README.md
#	docs/CURRENT-STATE.md
2026-10-20 14:54:57 +01:00
Said TurkseverandClaude Opus 5.5 ce5f69b586 fix(cctv): back off the Taiwan catalog when unreachable; review fixes
- taiwan.js: a failed download serves the last good table (up to 24 h)
  and is not retried for an hour. The table host does not answer from
  some networks outside Taiwan, and each attempt held the shared catalog
  refresh for the ~10 s connect timeout.
- Own server/providers/cctv/taiwan.js in package-boundaries.json
  (check:boundaries failed).
- Decode numeric character references; name falls back to the CCTVID
  when a row has no road details.
- Credit names the dataset and links the OGDL Taiwan licence.
- Document CCTV_TAIWAN_URL; list Taiwan in the CURRENT-STATE source table.
- Tests: isolate catalog env, reset loader state, cover the byte cap,
  failed fetch, backoff, last-good fallback, cap clamping, redirect with
  a valid body, name fallback, numeric entities and UUID camera ids.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-20 14:51:48 +01:00
Said Turksever 0868bb97f6 Merge remote-tracking branch 'upstream/main' into fix-1237
# Conflicts:
#	.env.example
#	README.md
#	server/providers/cctv/catalog.js
2026-10-20 14:42:08 +01:00
JCY2515andCursor 22c22d1876 feat: add Hong Kong Transport Department CCTV pack
Fetch keyless DATA.GOV.HK camera locations at runtime and pin stills to
tdcctv.data.one.gov.hk with required attribution. Default cap 1,100 covers
the ~1,013-camera TD catalog, prioritized across Hong Kong Island, Kowloon
and the New Territories.

Co-authored-by: Cursor <[email protected]>
2026-10-20 14:40:01 +01:00
Said Turksever 72983215fc Merge branch 'main' into upstream/ontario-511-key 2026-10-20 14:21:35 +01:00
Said Turksever 24ff5c7ea7 Merge branch 'main' into german/street-level-slim 2026-10-20 13:06:56 +01:00
German KhokhlovandClaude Opus 5.5 b00c7c12c8 refactor(sky): cut what the sky does not need
- qa-anime-sky keeps only the Milky Way case; the satellites and keyless
  cases checked an anime shader this branch no longer changes.
- Drop qa-sky-sun-moon.mjs: the viewer-options unit test covers it.
- VisualEffects sets the sky's visibility directly: the viewer always
  carries a StarrySky and never outlives the effects.
- Drop the unused star intensity knob, the malformed-row tolerance for
  our own pack, the test-only Transforms parameter and a private-name test.
- DATA_SOURCES points to sky/README.md instead of repeating it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-20 13:45:36 +04:00
German Khokhlov 7d4da26905 Merge remote-tracking branch 'origin/main' into claude/hal-627-starfield-cb6fa2
# Conflicts:
#	scripts/format-scope.json
2026-10-19 20:09:55 +04:00
Said TurkseverandClaude Opus 5.5 c808bfe609 fix(cctv): harden the Kartverket height lookup for Vegvesen cameras
- Default backoff is 30 min, longer than the 15-minute catalog refresh,
  so a failing Kartverket costs one attempt per backoff, not per refresh.
- A point a 200 answer leaves out is cached as no height (6 h TTL)
  instead of being asked again, and waited on, every refresh.
- Only 408, 429 and 5xx stop the lookup; any other 4xx marks that
  request's points as no height and the lookup carries on.
- Each request has its own 10 s timeout inside the shared 30 s deadline;
  a lookup cut short only by the deadline resumes without a backoff.
- The lane waits only on a cold cache (>10% of cameras missing); a few
  new cameras or expired misses are looked up in the background.
- applyVegvesenGroundHeights skips cameras without finite coordinates,
  so it really never throws.
- clearKartverketHeightCache bumps a generation, so a lookup started
  before a clear cannot write to the cache, backoff or pending slot.
- Credit and DATA_SOURCES note the modification (water set to 0 m,
  rounded to 0.1 m) as CC BY 4.0 asks; docs match the stop rules.
- Tests for the deadline being shared, per-request timeouts, 503,
  oversized and non-JSON bodies, 4xx, left-out points, Retry-After end
  to end, overlapping refreshes, the warm-cache wait, bad coordinates,
  the cache-clear generation and the loader's wait budget.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 17:03:50 +01:00
Said Turksever e9a234094b Merge remote-tracking branch 'upstream/main' into pr-1228-fix
# Conflicts:
#	README.md
2026-10-19 16:56:57 +01:00
German KhokhlovandClaude Opus 5.5 a536e7879c Merge origin/main into german/street-level-slim
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 19:56:09 +04:00
Trygve Hemminghytt 4999f6dc60 Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	docs/CURRENT-STATE.md
2026-10-19 16:16:37 +02:00
Said Turksever 326c41df9c Merge branch 'main' into cctv-florida-stills 2026-10-19 14:56:33 +01:00
Said TurkseverandClaude Opus 5.5 c174742e2c fix(cctv): FL511 written-consent wording and robust catalog paging
- State FL511's terms the same way everywhere: individual, personal use
  only, and any other use (a public deployment included) needs FDOT's
  express written consent. .env.example, README, dataCredits, the source
  licence string, the catalog comment and the changelog fragment said only
  "individual, non-commercial use".
- Page the ArcGIS catalog on exceededTransferLimit alone and advance the
  offset by the rows each page returned, so a lower server-side max
  record count no longer cuts the catalog short at the first short page.
- Order pages by OBJECTID_1 (this layer's object id; a plain OBJECTID is
  rejected with HTTP 400) so offset paging is stable.
- Raise the page ceiling to 40 and warn when it is hit.
- Tests: short pages with exceededTransferLimit keep paging, orderByFields
  is sent, and a service that ignores resultOffset stops at the ceiling.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 14:32:58 +01:00
Jerry LaiandClaude Opus 5.5 73336b297e feat(cctv): add Taiwan Freeway Bureau camera pack
Read ~1,800 national freeway cameras at runtime from the Freeway Bureau's
keyless MOTC 2.0 CCTV.xml table. Each published MJPEG stream maps to the
same camera server's single-JPEG path; only those exact HTTPS pairs on the
four freeway.gov.tw camera servers are registered. On by default,
CCTV_TAIWAN_ENABLED=0 skips it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 17:52:47 +08:00
German KhokhlovandClaude Opus 5.5 6390a3346f Merge origin/main: Google Street View, KartaView and Panoramax on the slim Street Level
Main gained three imagery providers (Google Street View, KartaView,
Panoramax), shared line-coverage, cone and photo-viewer modules, one source
lit at a time and the translated interface. Their shared modules still
carried what this branch removes, so the merge ports them onto it:

- lineCoverage.js is this branch's coverage.js made provider-neutral
  (draped lines near the ground only, the selection drawn over them, key
  and rate-limit blocking); sequenceCones.js is sequences.js the same way.
  Mapillary, KartaView and Panoramax all build on them; Mapillary moves
  back to providers/mapillary/ beside the others.
- No ground casting, mesh sampling, surface modes, FOLLOW, legend, prewarm,
  orbit overview points (Panoramax's density dots included), stale-tile
  swaps or water-filled point budget come back.
- Kept from main because the new providers need them: requiresMapStack and
  attachMapStackController (Street View on Google 3D only), ground clicks,
  openNearest with an optional provider nearestImage, openImage {frame},
  reportError, the view-cone marker (now an SVG glyph), filters /
  coverageLabel / sinceBasis, OPEN STREET VIEW, keyAlternatives and i18n.
- photoViewer.js treats the viewer as expanded when its wrap is full screen
  or the iPhone popover, as EXPAND now is.
- The fixture browser gate keeps its 8 steps and adds one each for
  KartaView and Panoramax; catalogs, docs, boundaries and format scope
  follow.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 13:51:45 +04:00
German KhokhlovandClaude Opus 5.5 6549344946 feat(sky): replace the blocky default sky box with catalogue stars
Cesium's default Tycho-2 sky box is six 1024² JPEGs stretched 3–6× around
the whole-globe view, so space read as an over-compressed JPEG (HAL-627).

The scene now draws its own sky in the Scene#skyBox slot: the 9,096 stars
of the Yale Bright Star Catalogue as point sprites sized in CSS pixels
(magnitude → width and opacity, B−V → colour), over a soft, star-free
Milky Way cube map built from ESO/S. Brunier's panorama (CC BY 4.0). Both
layers share one ICRF→Earth-fixed rotation at the scene clock, with the
TEME rotation filling in until Cesium's IAU 2006 tables load.

Assets total 482 KB, built by scripts/build-bright-stars.mjs and
scripts/build-milky-way.mjs. Frame time and the idle render governor are
unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 11:08:46 +04:00
quanticsoul4772andClaude Opus 5.5 83b9940733 docs(cctv): Ontario 511 docs match the key requirement and the developer agreement
Review fixes for #1228: CURRENT-STATE.md and the constants comment no
longer call Ontario 511 keyless. DATA_SOURCES.md notes Ontario 511's
Developer's Access Agreement (511on.ca/developers/resources): the data
stays under the Open Government Licence - Ontario, key requests are
reviewed case by case, attribution by link is encouraged and the logo
rules don't apply because the app shows none, and the "as is" disclaimer
should be passed on, which the credit now does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 16:48:02 -07:00
quanticsoul4772andClaude Opus 5.5 fa085cbd64 docs(cctv): point to the 511on.ca pages that exist for the Ontario key
511on.ca/developers is a 404. Sign up at 511on.ca/my511/register, then
request the key while logged in at 511on.ca/developers/doc (both checked).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 15:40:22 -07:00
quanticsoul4772andClaude Opus 5.5 48c8a64c97 fix(cctv): Ontario 511 cameras take the developer key the API now requires
The Ontario 511 camera list answered 400 "Invalid Key" on every catalog load
("[CCTV] Ontario 511 camera download failed: 400"), so the pack loaded
nothing. 511on.ca's developer docs now say the API "Requires a developer
key" (free, from 511on.ca/developers; 10 calls a minute). The loader reads
ONTARIO_511_API_KEY and sends it as `key`; without it the pack is skipped
with one log line instead of a failed request on every load. The key is
never logged. .env.example and DATA_SOURCES no longer call the endpoint
keyless.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-18 15:40:22 -07:00
Trygve Hemminghytt 59db4ac227 docs(cctv): state the measured cold lookup time and what a slow first run leaves 2026-10-18 20:22:21 +02:00
Trygve Hemminghytt 80a0954030 Merge remote-tracking branch 'origin/main' into HEAD 2026-10-18 20:02:40 +02:00
ulihhhhandClaude Opus 5.5 5508caa174 docs: credit and document the Spanish city bike systems
- dataCredits `gbfs`: names the seven PBSC operators, with BiciMAD's
  required "Powered by EMT de Madrid" linking to emtmadrid.es.
- dataCredits `jcdecaux-gbfs`: adds Valenbisi and Sevici.
- DATA_SOURCES.md: a BiciMAD row (Madrid's open-data catalogue publishes
  the feed under EMT's terms, attribution required), one row for the other
  six PBSC feeds (no licence published; attribution to the operator), and
  the JCDecaux row widened to Valencia and Seville.
- CURRENT-STATE.md: the Bikeshare row, and the underscore and
  geofenced-capacity rules in "GBFS upstream bounds".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 22:33:57 +01:00
Daniel Slay c09865e582 feat(cctv): FL511 Florida camera pack (stills, opt-in) 2026-10-17 13:50:33 -04:00
Said Turksever c5feec26dc Merge remote-tracking branch 'upstream/main' into feat/bikeshare-bogota-santiago
# Conflicts:
#	CHANGELOG.md
#	DATA_SOURCES.md
#	src/data/dataCredits.js
#	src/layers/bikeshare/registry.js
2026-10-16 11:41:25 +01:00
ulihhhhandClaude Opus 5.5 194595e1b6 feat(bikeshare): add Barcelona's Bicing and AMBici
- Bicing (543 stations, PBSC) and AMBici (237 e-bike stations across the
  metropolitan area, nextbike) join the GBFS registry from the operators'
  keyless GBFS 2.3 feeds, both listed in MobilityData's systems catalogue.
- gbfs.nextbike.net joins the proxy allowlist as an exact host.
- Credits, DATA_SOURCES, CURRENT-STATE and CHANGELOG updated; a test pins
  each system's proxy reachability and the range check for its outermost
  stations.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-16 10:54:58 +02:00
Trygve Hemminghytt 2bec3d521b Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	docs/CURRENT-STATE.md
#	server/providers/cctv/sources.js
2026-10-16 07:21:30 +02:00
Sameh Khamis 4adeae0afe Merge branch 'main' into feat/transit-ontario
# Conflicts:
#	CHANGELOG.md
2026-10-15 19:14:21 -07:00
Sameh Khamis ed00f7af30 Merge branch 'main' into feat/transit-ontario 2026-10-15 19:09:40 -07:00
Sameh Khamis a4888d7846 Merge branch 'main' into feat/cctv-model-spec-enrichment
# Conflicts:
#	CHANGELOG.md
2026-10-15 19:08:55 -07:00
Sameh Khamis 265c8a1cc0 Tighten datasheet model matching and keep given fields of view
Codec, product and resolution words never identify a model; generation suffixes such as Mk II no longer fall back to the base model; a mismatched manufacturer never matches, and brand-only numeric names need the brand. The datasheet FOV only fills when the source and seed give none and the pose is not curated, and DATASHEET is shown only while no calibration offset applies. Adds the CC0 extract to DATA_SOURCES.
2026-10-15 17:27:55 -07:00
Sameh Khamis 687a94c9df Document Bike Itaú (Tembici) and test the Brazilian cities
The seven feeds publish no licence, operator or attribution fields, so DATA_SOURCES records that and credits the operator. Adds a registry test per city: allowed host and path, provider, and activation over its outermost live station.
2026-10-15 17:27:55 -07:00
Cliff Flood 9ab26bbcb0 docs: record Hamilton's route table and the feeds left out
Two gaps. The Hamilton entry never said its route ids are internal keys,
which the Guelph, Burlington and YRT entries all say of theirs; the
route-table sentence in the transit notes named those three and not
Hamilton, though it now carries the largest table of the four. Both were
stale the moment Hamilton got a table.

The exclusions were half-recorded: GO Transit and Grand River had their
reasons written down, but Ottawa, London, Thunder Bay and the static-only
majority did not, so a later reader would have re-investigated feeds that
were already surveyed and rejected. All of them are now named with the
rule each one falls to, and the survey is stated as complete against the
Mobility Database's Ontario catalogue.

Also records that a route label is never invented, which is why YRT's
unpublished 9101 renders as the feed sends it.
2026-10-15 20:18:21 -04:00
Cliff Flood a7fc5ea7c5 feat(transit): add Burlington, switch Barrie and YRT on
Burlington's route ids are internal keys with a trap in them: `351` is
route 1 while `3510` is route 10, so no prefix trim is correct and the
table is a real join against the City's own published GTFS. Its licence is
the same 2011 municipal template as Mississauga — revocable, and the terms
must travel with the data, so licenseUrl is the terms document itself.
Plain http:// has no listener at all, so https is mandatory, not preferred.

Barrie and YRT ship on. Both operators invite acceptance of their licence
through a form, and both licences also state that use is itself acceptance
("By using Barrie Transit Data, you agree to be bound by these terms";
"Your Use of Information indicates Your acceptance"). This deployment
relies on that clause, and both entries record it so the reasoning does not
have to be re-derived.

Nine Ontario feeds now, all verified live against their upstreams.
2026-10-15 20:18:15 -04:00
Cliff Flood 8ccb0cd895 feat(transit): register YRT/Viva off, named by Viva colour
York Region's feed is first-party infrastructure under an open licence that
permits commercial use and does not even require attribution. It is the one
feed that fills the gap north of Toronto, and it is switched off anyway.

The operator asks that its licence be accepted through a form collecting a
name, company, phone and email. No credential is issued and nothing gates
the endpoint — the licence body says use is itself acceptance — but a form
asking for contact details is close enough to the registration this registry
refuses that a person should decide it, not a poller. Barrie is off for the
same reason. Both are one line from on.

Viva route ids get labels: YRT publishes `601` where every station sign says
Viva Blue. The numeric local routes already read correctly and carry no
entry, so the table is six lines, not 125.

No vehicle in this feed carries a bearing; heading comes from observed
movement, as with Durham.
2026-10-15 20:18:15 -04:00
Cliff Flood 11dd914c2f feat(transit): add Guelph and Kingston, register Barrie off
Guelph's feed publishes the internal database key of a route (`2991`)
where the rider sees the number on the bus (`1`), which would have put
"Route 2991" on the label a person reads. So a feed may now carry its own
`routeLabel`, the same way it may carry its own `modeWords`. Guelph's table
is its 33 routes, taken from the City's own published GTFS; an id missing
from it renders unchanged, because a raw id you can compare against the bus
beats a blank. Every other feed is identity and carries no table.

Kingston is catalogued by the City against this exact URL. Its licence PDF
link is pinned to the canonical location rather than the legacy path that
currently redirects there — Durham's entry records what happens when such a
redirect is eventually dropped.

Barrie ships REGISTERED AND OFF. The City offers its licence for acceptance
behind a click-through gate, and nobody on this project has accepted it.
Whether to accept is a person's decision, not an assumption a poller should
make, so the feed is visible, documented, and one line from enabled.
2026-10-15 20:18:15 -04:00
Cliff Flood 849cb80375 feat(transit): add MiWay, Hamilton HSR and Durham Region Transit
Three keyless GTFS-Realtime feeds around Toronto, each verified live: all
vehicles carry real coordinates, and each is polled only near its own
region rather than across the Golden Horseshoe.

The licences differ in ways the registry now records rather than flattens:
Hamilton PRESCRIBES its attribution wording, so the credit is that exact
sentence; Durham makes credit optional and its catalogued licence PDF is
dead after a site relaunch, so licenseUrl points at the live page where the
v.1.0 text was actually read; Mississauga's licence is revocable and asks
that the terms travel with the data, so licenseUrl is the terms themselves.

All three are bus-only and set no route hint, so their modes default rather
than resolve. Durham publishes no bearing on any vehicle — heading comes
from observed movement, as with any feed that omits it.

GO Transit / Metrolinx stays out: its feed requires an API key.
2026-10-15 20:18:15 -04:00
Cliff Flood c5f7fe285b feat(transit): add the TTC Toronto surface feed
TTC BusTime publishes keyless GTFS-Realtime VehiclePositions with real
coordinates for the surface network: ~1,180 buses and streetcars with
bearing and per-vehicle timestamps. Registering it is one registry row.

The subway is deliberately absent. BusTime covers surface vehicles only
and the subway publishes no positions, so Lines 1/2/4 are left out rather
than inferred from schedules — the same rule that keeps NYCT out.

Route numbers establish the mode: the three-digit 500-series is streetcar,
as are the four Blue Night routes on streetcar trackage. A leading 5 alone
would misclassify bus routes 50-59, so the rule is anchored and tested.

Licensing is the portal-wide Open Government Licence - Toronto; the CKAN
record for the BusTime NVAS dataset leaves its own licence field blank,
and the terms note says so rather than overstating the grant.

Refs #70
2026-10-15 20:18:15 -04:00
Sameh Khamis 477eb86d64 Merge branch 'main' into feat/brazil-gbfs-cities 2026-10-15 16:28:05 -07:00
Sameh Khamis 7366a46eba Merge branch 'main' into feat/wsdot-camera-pack 2026-10-15 16:17:22 -07:00
Sameh Khamis 05d8529edc Merge branch 'main' into feat/wsdot-camera-pack 2026-10-15 16:10:59 -07:00
Sameh Khamis ee10b2ce1c Merge branch 'main' into feature/catalonia-cctv-cameras 2026-10-15 16:10:47 -07:00
Sameh Khamis 52d26b0577 Merge branch 'main' into feat/wsdot-camera-pack 2026-10-15 16:05:47 -07:00