Compare commits

..
Author SHA1 Message Date
German Khokhlov b65a7aaed8 Keep an explicit CCTV pick of the camera a move re-opened
Until someone picks a camera, a settled move re-opens CCTV on the camera
nearest the view and keeps the opening pick provisional. setActiveCamera
ends that opening pick, but for a camera that is already active and
activated it returns UNCHANGED before clearing the flag. So pressing
NEAREST on the camera a move had just re-opened (or selecting it from the
list, or through setParams) left the pick provisional, and the landing of
NEAREST's own flight, or the next pan, switched to another camera.

Clear the provisional flag on the UNCHANGED path too. reopenNearestCamera
only activates a different camera, so it is unaffected, and the same-camera
re-select still skips the activation work.
2026-11-13 18:47:30 +04:00
German Khokhlov 42540ad85e Merge pull request #1698 from bilawalsidhu/german/i18n-unreviewed-strings-fixes
i18n: review and fix the strings added since the last translation review
2026-11-13 18:13:17 +04:00
German Khokhlov bc8aadf55f Changelog fragment for #1698 2026-11-13 17:50:58 +04:00
German Khokhlov 7314dac1ce i18n: review and fix the strings added since the last translation review
Reviews the 474 strings that 27 pull requests added since the last
review, in all twelve catalogs, and fixes what the review confirmed:
wrong agreement when a placeholder takes a neutral value, plural-less
counts that read "1 sightings", terms that differ from the glossaries,
and typography the language guides set. Also applies most of the
findings earlier reviews left open.

zh-Hant is converted from zh-CN again. The new strings had been
translated separately with Taiwan vocabulary. The zh-Hant guide lists
two more characters OpenCC picks wrongly.

Three tests expected the old German and Spanish wording.
2026-11-13 17:49:54 +04:00
Sameh Khamis 188f0ccd9b Merge pull request #1696 from bilawalsidhu/sameh/i18n-entry
feat(i18n): an ./i18n entry for an application with its own language choice
2026-11-12 22:21:25 -08:00
Sameh Khamis ad9bd0f4f9 docs(changelog): 1696-i18n-entry 2026-11-12 22:07:30 -08:00
Sameh Khamis 5f96009b67 feat(i18n): an ./i18n entry for an application with its own language choice
startPageLocale, followPageLocale and switchLocale start, follow and switch
the page's language as the standalone application and the globe picker do,
for an application that offers the language elsewhere and passes
localePicker: false. The standalone application uses them.
2026-11-12 22:07:13 -08:00
Sameh Khamis feab48397f Merge pull request #1695 from bilawalsidhu/sameh/vessel-photos
feat(vessels): a selected vessel shows its Open Vessel Photos photo and MMSI
2026-11-12 20:42:37 -08:00
Sameh Khamis e4832c589c Merge branch 'main' into sameh/vessel-photos 2026-11-12 20:32:03 -08:00
Sameh Khamis e3069612b5 Merge pull request #1694 from bilawalsidhu/sameh/cctv-globe-plane-clamp
fix(cctv): on a globe map the camera plane ends where the view meets the ground
2026-11-12 20:31:59 -08:00
Sameh Khamis a930677c83 Merge branch 'main' into sameh/cctv-globe-plane-clamp 2026-11-12 20:22:04 -08:00
Sameh Khamis 3da5ffe71f Merge pull request #1693 from bilawalsidhu/sameh/cctv-frame-no-flash
fix(cctv): a new camera's frame shows the moment it is decoded, without a fade
2026-11-12 20:21:44 -08:00
Sameh Khamis 2bc3c1bed1 docs(changelog): 1695-vessel-photos 2026-11-12 20:11:20 -08:00
Sameh Khamis d5b030e7fa docs(changelog): 1694-cctv-globe-plane-clamp 2026-11-12 20:11:19 -08:00
Sameh Khamis 23aa1027fc docs(changelog): 1693-cctv-frame-no-flash 2026-11-12 20:11:17 -08:00
Sameh Khamis 347e77d1f6 feat(vessels): a selected vessel shows its Open Vessel Photos photo and MMSI
Selecting a vessel asks Open Vessel Photos (a free, keyless API) for its
photo by MMSI, once per MMSI. The CONTACTS view shows it under the vessel's
name and MMSI with the photographer, the photo's open licence and a link to
the vessel's page; only images on Open Vessel Photos' own host are shown, and
an answer about another MMSI is no photo. The vessel source exposes the lookup
as getEnrichment({kind: 'photo'}) and fetchOpenVesselPhoto for applications
with their own vessel source.
2026-11-12 20:10:30 -08:00
Sameh Khamis 2445af349d fix(cctv): on a globe map the camera plane ends where the view meets the ground
The activation probe raycast only the photoreal tiles. On a globe map the ray
missed, or ran on through the Earth and met a primitive thousands of
kilometres away, and the monitor plane stood at the camera's full range:
about a kilometre wide and hundreds of metres up. A hit at or beyond the range
now counts as none, and on a globe map the plane also clamps short of the
globe's terrain along the axis, or flat ground at the mount's ground while
that terrain is still loading, whichever is nearer.
2026-11-12 20:10:02 -08:00
Sameh Khamis 6e4df4c287 fix(cctv): a new camera's frame shows the moment it is decoded, without a fade
The preview settled when the image had loaded, so the panel's <img> took the
src while it was still decoding and then faded in over 180 ms: every camera
click lingered on an empty box. The preview now settles once the frame is
decoded (a rejected decode still shows the loaded frame, and a decode that
never settles shows it after one second), and the fade is gone.
2026-11-12 20:10:02 -08:00
Sameh Khamis c02ede3bb4 Merge pull request #1691 from bilawalsidhu/said/shortcuts-ignore-modifier-chords
fix(shortcuts): a shortcut key held with Cmd, Ctrl or Alt runs no shortcut
2026-11-12 18:19:30 -08:00
Sameh Khamis 16aae73622 Merge branch 'main' into said/shortcuts-ignore-modifier-chords 2026-11-12 18:11:19 -08:00
Sameh Khamis 01bb68a9a2 Merge pull request #1604 from bilawalsidhu/feat/data-library
Add the Data Library: live regional datasets, tiles and packs
2026-11-12 18:11:08 -08:00
Sameh Khamis 1d476c87ee fix(location): a POI key held with Cmd, Ctrl or Alt runs no POI
The points-of-interest keys (Q/W/E/R/T while a city is expanded) are a
second bare-key document listener, and it did not look at modifiers
either: Alt+R or a delivered Ctrl+R picked the fourth POI and flew the
camera there. It now leaves Cmd, Ctrl and Alt chords to the browser like
the application shortcuts; Shift still counts as the key. The docs and
the changelog entry say so.
2026-11-12 18:02:41 -08:00
Sameh Khamis d1966b9751 Clear a view-bounded layer's error when the camera returns to the drawn view
A failed request for the view in between left its error set, and the
early return for the already drawn view never cleared it, so the drawn
features showed with an error state.
2026-11-12 17:57:31 -08:00
Sameh Khamis 92d6188045 Stop a moved-on view from cancelling the GeoParquet index the next view needs
The browser shared one partition-index build per entry but ran it under
the first view's signal. A camera move cancels the old view before the
next one starts, so the next view was handed the same build, already
cancelled, and drew nothing (the view layer takes an AbortError for its
own cancellation) until the camera moved again. EPC hit this on its
first view after any quick move. The build now belongs to no view: each
view stops waiting on its own signal, and the build runs on for the next
view and every view after it.
2026-11-12 17:56:17 -08:00
Sameh Khamis 8cca78bcd5 Stop sharing a Data Library entry as on once its failed Remove switched it off
When Remove switched a layer off and then its destroy failed, the
manager published no visibility change, so the selection still listed
the entry as on: the link shared it as on and a reload restored it on
while its row read off. A failed Remove now drops the entry from what is
on and remembered whenever its layer ended up off.
2026-11-12 17:49:38 -08:00
Sameh Khamis 837eff7328 Let a share fit a full Data Library and show entries it cannot serve
A share's entries joined the library after the stored ones, and the
selection kept its first 64, so a link opened into a full library never
showed what it shared and the next hash write dropped its lib=. Now the
oldest entries the share leaves off make room for it, and their layers
are removed with them.

Browse counted an entry as added only while its layer was registered,
but an entry this instance cannot serve (switched off by the operator,
or its server unreachable) stayed in the selection and held one of the
64 places. A library full of them looked empty and refused every Add.
Such entries now show as added, so they can be removed one by one or
with Remove all.
2026-11-12 17:49:01 -08:00
Sameh Khamis 4fe8796e89 Swap refreshed Data Library features in place, on every tick
The live refresh had three faults. Its data age counted from when a
fetch finished, so with the manager ticking every cacheSeconds the next
tick found the data just short of due and data refreshed every other
tick. It switched the layer off and on to redraw, which removed the old
features before the new ones were built, closed an open feature card and,
if the rebuild failed, left the layer on but empty. And the tick the
manager runs right after enable refetched after a failed first load, so
a second failure switched the layer off instead of showing UNAVAILABLE.

The local GeoJSON layer gains replaceFeatures(): it builds the new set
while the old one stays drawn, removes the old source only once the new
one is in the scene, keeps a selected feature that is still in the new
set (same id, or same place and label), and on a failed rebuild puts the
old features, records and selection back and rejects. The library's
refresh uses it, counts age from the request, allows a little slack on
the tick, and does nothing until a first load has succeeded.
2026-11-12 17:47:29 -08:00
Said Turksever 8ac0d95f1b docs(changelog): shortcut keys ignore Cmd, Ctrl and Alt chords (#1691) 2026-11-13 01:13:42 +00:00
Said Turksever 2c4d865ef9 fix(shortcuts): a shortcut key held with Cmd, Ctrl or Alt runs no shortcut
The application shortcut listener read the key alone, so copying with
Cmd+C or Ctrl+C toggled the CCTV layer and opened the nearest camera,
Cmd+F opened the layers panel and Cmd+1 to 6 changed the visual style.
Chords are now left to the browser. Shift is not a chord, a held key
still repeats, and Escape dismisses search either way.
2026-11-13 01:12:58 +00:00
Sameh Khamis 1a657e2a64 Keep the library layer namespace in the layer lifecycle package
The core layer lifecycle imported the whole Data Library manifest
contract only to check the lib- prefix, so the layer-lifecycle package
owned a feature module. The namespace rule (prefix, entry id pattern,
the two id helpers) now lives in src/data/libraryLayerIds.js, which the
lifecycle owns; the manifest contract imports and re-exports it, so
library callers are unchanged.
2026-11-12 16:04:55 -08:00
Sameh Khamis 9e57fd84c8 Draw Data Library coverage outlines from the bundled Natural Earth countries
Imagery outlines fetched Natural Earth's 1:50m countries (about 3 MB)
from the natural-earth-vector repository's master branch on GitHub at
run time: a moving, rate-limited source whose failures silently removed
the outlines. Core already ships the pinned 1:10m countries pack, so
the outlines now read that, once, the first time an outline needs it.
2026-11-12 16:02:07 -08:00
Sameh Khamis 8d90cac7dc Keep lib= in the URL before the library starts and for entries it cannot serve
The share encoder wrote only registered entries. Until availability
loaded nothing was registered, so any hash write in the meantime (a
camera move during start-up) deleted lib= from the address bar, and an
entry this instance could not serve (its proxy unreachable) was dropped
for good: copying the link or reloading lost those entries. The URL now
carries what the page opened with until the library starts, and every
entry that is on afterwards, served here or not.
2026-11-12 15:59:14 -08:00
Sameh Khamis 2d5db9f63d Never let a share rewrite the Data Library entries remembered as on
Every restore wrote the selection it applied to browser storage, so
opening a link that switched no library entry on stored every saved
entry as off, and the next bare visit restored them off. Core layer
state never writes a recipient's preferences from a share. The library
now keeps what is remembered as on apart from what a share switches on
for the session: a share's entries still join the library, but only a
local restore and the viewer's own choices change what is remembered.
2026-11-12 15:58:22 -08:00
Sameh Khamis 53f0a32a1b Keep a Data Library entry whose layer fails to tear down
Remove dropped an entry from the selection and reported success even
when the manager could not disable or destroy its layer, leaving a
registered, possibly visible layer that Browse, share links and saved
state no longer knew. Remove now keeps the entry and reports failure,
so its row and Remove all still reach it and a later Remove can retry.
2026-11-12 15:57:02 -08:00
Sameh Khamis f33a6ada52 Refuse a Data Library entry past the 64-entry cap before it registers
Add registered and switched on an entry before the selection was cut to
its first 64, so a 65th was dropped from the selection but stayed on:
Browse showed it as not added, share links left it out and Remove all
missed it. Start-up registered a share's entries the same way past the
cap. Add now refuses an entry a full library cannot keep, and start-up
registers only what the selection keeps.
2026-11-12 15:53:29 -08:00
Sameh Khamis e25cce874d Read an ArcGIS load stopped at maxFeatures as truncated
Paging stopped once maxFeatures were read before it noted that the
upstream had more, so a page holding exactly the cap with
exceededTransferLimit set read as the whole dataset. FEMA USA
Structures (2,000 features, one page) never showed PARTIAL in a dense
view. Stopping at the cap or the last page with more upstream now marks
the load truncated.
2026-11-12 15:52:31 -08:00
Sameh Khamis 134239fa5a Read a view over an incomplete GeoParquet index as PARTIAL
The browser's partition index kept whether it was complete only to
schedule a rebuild and handed the partitions on without it, so a view
counted unread partitions from an index that might be missing some and
could read as complete, even empty, over an area it had never indexed.
A view over an incomplete index now reads PARTIAL until a rebuild
completes.
2026-11-12 15:51:52 -08:00
Sameh Khamis e05a9566e4 Retry a GeoParquet footer whose body fails mid-stream
Only a failed fetch() marked a footer read as worth retrying. A body cut
off after its headers arrived (a reset, the proxy's 20-second timeout)
was taken for an unparseable file: the partition was left out and the
index still read as complete, so the server cached it without that
partition for the entry's cacheSeconds (24 hours for EPC). Such a read
is now retried like any fetch failure; an oversized body is still final.
2026-11-12 15:50:46 -08:00
Sameh Khamis 7d91a1875b Cancel a view-bounded load when the camera returns to the drawn view
Moving from view A to B started B's request; moving back to A before it
finished returned early because A was already drawn, so B's request ran
on and its features replaced A's while the camera was at A. Returning to
the drawn view now cancels the request in between.
2026-11-12 15:49:56 -08:00
Sameh Khamis 0f13c0a1c2 Let a newer share switch off a library layer still switching on
A share restore skipped every entry whose settled state already matched
the share, but a layer still switching on for an earlier share reads as
off until it settles. Opening a second share in that window left the
first share's layer to come on anyway. The restore now compares with the
manager's effective visibility (the newest request), so the newer share
supersedes the earlier one.
2026-11-12 15:49:11 -08:00
Sameh Khamis d58ba23fba Refresh live Data Library datasets once older than cacheSeconds
A dataset layer loaded its features once and kept them until it was
removed: the parsed set survives disable/enable, so NWS alerts, GDACS and
EONET showed what they held when first switched on for the whole session.

A library dataset now asks the manager for a refresh every cacheSeconds.
Once its data is that old it loads again, and swaps the new set in only
when it is ready, so the old features stay drawn while the request runs
and stay if it fails. An enable after cacheSeconds also loads afresh.
Disable and destroy cancel a refresh in flight.
2026-11-12 15:48:14 -08:00
Said Turksever 1a2589d4ef Port the Data Library Browse dialog to Vue and Pinia
The dialog is now DataLibraryDialog.vue (with DataLibraryCard.vue for each
entry), placed with the Browse button by DataLibraryBrowse.vue in
LayerPanelsView and teleported to the body. It renders what the
gev-data-library store holds and sends clicks and typing to the owner.

src/ui/libraryBrowser.js is now the framework-free owner: it presents the
open state, filters, the rows they show, the stat summary and Remove all
through `publish`, and its controls answer the view. Its text helpers are
what the view renders with t(), so every label follows the locale without
rebuilding anything. Only showModal/close and focus moves stay imperative;
focus returns to Browse also when the browser closes the dialog on Escape
itself.

Component tests mount the view over the real owner (open, filters, Add to
Remove keeping focus, Zoom, Remove all, a language switch, clean view).
2026-11-12 23:01:00 +00:00
Said Turksever 24ccdd4dc3 Fix Data Library share restore, dense tile layers and EPC partition reads
- Share links: the compact /share/<code> format carries the switched-on
  library entries (field 29), so Copy Link keeps them; a share opened in the
  same tab restores them; any share decides what is on, so one with every
  library entry off no longer switches on the recipient's stored entries.
- A tile entry too dense at its minimum zoom (Norway cadastre z14, depth z11)
  is hidden below that scale via minimumTerrainLevel instead of allocating
  millions of tiles when zoomed out.
- EPC partitions keep the full extent for exclusion and the 98 % extent for
  ranking only; the old trimmed-only extents are retired.
- Partition index footers that fail transiently are retried; an incomplete
  index is never cached as complete, a complete prior index is kept, and it
  is rebuilt after five minutes.
- Views that leave touching partitions unread are partial; listing and
  footer bodies are capped and ranges validated.
- The Browse dialog relabels after a language switch.
2026-11-12 22:37:34 +00:00
Said Turksever 871b7f0fcc Remove Data Library entries whose licence or provider links are broken
A link check of every manifest found 404 licence or provider pages for
OpenSeaMap seamarks, EMODnet bathymetry and the PDOK, basemap.at, Geneva
SITG, ICGC Catalunya, Luxembourg and Vlaanderen aerial photos, and no
answer from the SEDAC GPW, Minnesota and Bangor hosts. Drop those 11
entries, renumber the unpublished share-token ledger, point the tests at
remaining entries and regenerate DATA_SOURCES.md.
2026-11-12 17:05:21 +00:00
Said TurkseverandMaher Kurdi 11706a68cb Add Amsterdam traffic and ANPR camera positions to the Data Library
The City of Amsterdam's verkeersinformatiesystemen register lists where
its ~500 traffic and ANPR cameras stand, with type and asset number, but
publishes no pictures. That fits a Data Library dataset better than a
CCTV pack: the register answers GeoJSON in WGS84 with CORS headers, so
the entry fetches it browser-direct with no proxy code.

The register names no licence ("public unless stated otherwise"), so the
entry stays operator opt-in, as #251 had it. The dataset and the
position-only camera idea come from @Maher-Reven's #251, which this
replaces; the entry lists them as maintainer and in its attribution.

Co-authored-by: Maher Kurdi <[email protected]>
2026-11-12 12:26:45 +00:00
Said Turksever db80d82b8d Add the Data Library: live regional datasets, tiles and packs
DATA LAYERS keeps its curated core layers; regional and niche sources
live in the Data Library, opened from Browse data library at the foot
of the panel. Each entry is a JSON manifest in src/library/entries/
pointing at a live upstream (query API, provider-hosted file or tile
endpoint), so nothing is bundled and a self-hosted instance hosts no
data. Sources without CORS go through one allowlisted /api/library
route. Kinds: dataset (GeoJSON, CSV, JSON records, ArcGIS, partitioned
GeoParquet), tiles (XYZ, WMS/ArcGIS bbox export, terrarium relief) and
pack. Selections persist and travel in share links as lib=, separate
from the core layer tokens. Library rows get Zoom and Remove icon
chips that work while the layer is off, rendered through RailChips.
Strings are translated in every locale.
2026-11-12 12:26:45 +00:00
Sameh Khamis 10a788db78 Merge pull request #1683 from bilawalsidhu/sameh/news-cards-every-height
fix(news): an application's picture cards stay at every height
2026-11-11 19:52:44 -08:00
Sameh Khamis a105e819e0 fix(news): an application's picture cards stay at every height
Zoomed out, a hotspot's reports give way to its beacon, and the cards an application draws through pictureCards went with them; they came back only once the camera had stopped after zooming in. The cards now come from every report the close-up view would mark, at any height; the layer's own markers and beacons are unchanged.
2026-11-11 19:43:06 -08:00
Sameh Khamis 202cb8c3dc Merge pull request #1677 from ZloyS/denis/news-world-clock
feat(news): the News layer reads its time from the world clock
2026-11-11 15:28:35 -08:00
Sameh Khamis 2b1784d7ed Merge remote-tracking branch 'upstream/main' into denis/news-world-clock 2026-11-11 15:17:44 -08:00
Sameh Khamis beeb29f0fa Merge pull request #1675 from ZloyS/denis/rail-layer
feat(rail): add a Rail layer with keyless live trains
2026-11-11 15:07:54 -08:00
Sameh Khamis b7015f8819 Merge remote-tracking branch 'upstream/main' into denis/news-world-clock
# Conflicts:
#	docs/CURRENT-STATE.md
#	src/app/constructCatalog.js
2026-11-11 15:03:45 -08:00
Sameh Khamis 283890cee0 fix(rail): trip-update delays are as old as the server's read
The browser stamped every trip-updates answer as just read. While an
operator's feed fails, the rail proxy replays its last good copy for up to
ten minutes, and a cached copy can be a TTL old, so those delays never
reached RAIL_UPDATES_MAX_AGE_MS and a selected train's card showed them as
current. Delays now take their age from the snapshot's fetchedAt (capped
at now), as positions do in ingestion.js, so they stop applying 75 s after
the server last read the operator.
2026-11-11 14:57:02 -08:00
Sameh Khamis ace2160002 fix(rail): Montpellier tram speeds are converted from km/h
TaM publishes km/h where GTFS-Realtime asks m/s, and the Transit layer's
entry for the same feed already sets speedUnit: 'km/h'. The Rail entry did
not, so the proxy passed the readings through and the card multiplied them
by 3.6 again: a tram at 30 km/h read about 108 km/h. A registry test now
requires a Rail feed that Transit also reads to declare the same unit.
2026-11-11 14:54:01 -08:00
Sameh Khamis 0f18c03041 chore(news): credit the #1677 changelog fragment 2026-11-11 14:48:05 -08:00
Denis Groshev 9b3ca481ce Merge upstream main into denis/rail-layer 2026-11-11 23:23:58 +01:00
Sameh Khamis 632da1cebd Merge pull request #1680 from bilawalsidhu/sameh/news-host-cards
feat(news): an application may draw the News picture cards itself
2026-11-11 14:01:22 -08:00
Sameh Khamis 99a1e06b61 feat(news): an application may draw the News picture cards itself
createNewsLayer takes a pictureCards(cards, { selectedId }) function; createApplicationCatalog passes it through newsOptions. With it, the layer paints no picture cards of its own and hands the function one card per pictured event on the map, most severe then newest first, with the selected report's id; it hands no cards when the layer turns off.
2026-11-11 13:53:10 -08:00
German Khokhlov c19495644d Merge pull request #1509 from quanticsoul4772/upstream/route-outage-unavailable
fix(routes): report an upstream routing outage as unavailable
2026-11-11 23:14:47 +04:00
Kevin Rajan ca58e113c0 test(routes): cover HTML 502 outage in MCP plan_route (#1509)
Preserve quanticsoul4772's routing-outage fix and add a focused voice-facing regression for a non-JSON gateway response.
2026-11-11 09:44:24 -08:00
German Khokhlov 94ff78be3b refactor(routes): inline the single-use 502 responder
The outage branch is the only caller, so it writes its 502 directly.
Same status, headers and body.
2026-11-11 21:23:35 +04:00
German Khokhlov 82e2b716bc docs(routes): tighten the outage comments and notes
Shorter comments, changelog and CURRENT-STATE text for the 502 outage
path; the CURRENT-STATE paragraph now also names the 429 exception and
which budget expiry answers busy and which answers 502. Text only.
2026-11-11 21:21:07 +04:00
German Khokhlov a29611ca17 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-11 21:16:50 +04:00
Denis Groshev 72c24a86d1 chore(news): name the changelog fragment for #1677 2026-11-11 18:06:15 +01:00
Denis Groshev 13818d0b9f Merge upstream main into denis/rail-layer 2026-11-11 18:02:52 +01:00
Denis Groshev 44d2c0695c feat(news): the News layer reads its time from the world clock
Core's catalog passes worldClock.timeFor('news') as the layer's now, so a host that follows the clock for News can show a past time's reports with their ages, window and counts as they were then. Core does not follow the clock for the layer, so it keeps reading the wall clock.
2026-11-11 18:00:24 +01:00
German Khokhlov aa95c2c25f Merge pull request #1531 from quanticsoul4772/upstream/firms-age-rounding
fix(fires): show a 47.6-hour-old detection as 2d, not 48h
2026-11-11 20:24:53 +04:00
German Khokhlov 1c08482fa8 Merge branch 'main' into upstream/firms-age-rounding 2026-11-11 20:14:57 +04:00
German Khokhlov 98b642869e Merge pull request #1113 from asturksever/pr/street-level-show-action
feat(street-level): "show me X on Street View" — assistant action, view field and share link
2026-11-11 19:13:30 +04:00
Denis Groshev d793406aef docs(changelog): add the #1675 fragment 2026-11-11 16:08:24 +01:00
Denis Groshev 830c080803 feat(rail): add a Rail layer with keyless live trains
Live trains from 30 keyless operator feeds: GPS positions from 29 and
Irish Rail's station reports, shown as the operators publish them. 28
work out of the box; Renfe long and medium distance and KTMB appear once
their pack is built, because their vehicles name no route.

Packs are opt-in (scripts/build-rail-paths.mjs): the operator's own
route lines, stop names, next stops and trip-update delays for live
trains. Nothing downloads when the app starts, no train is placed from a
timetable, and no request carries a deployer's key.
2026-11-11 16:06:22 +01:00
German Khokhlov 71797427d6 Street Level: a posted view whose camera was taken stands down whole again
Applying the rest of a posted view after the camera was taken during its
terrain wait still let it change the camera: stop_tracking dropped a follow
the user had just started, and annotate_map framed the view's marks. A
camera taken during the wait now ends the view again, as before. The 2 s
terrain timeout and the per-view catch on the queue stay.
2026-11-11 19:01:58 +04:00
German Khokhlov 0ebc09079d Street Level: polar look-at, a bounded terrain wait, takeovers during the flight
- The look-at longitude wraps fully, so a tilted camera at a pole gives a
  point on the globe. The strict off-globe check on street_level is undone:
  points are clamped again, as before.
- A posted view that fails is answered, and the views behind it still apply.
- A posted view waits at most 2 s for the terrain under its camera, then
  keeps its own estimate.
- A camera taken during that wait stays where it is: the rest of the view
  applies, and only its photos stand down.
- show_street_level keeps a search or a press on the globe made during its
  own flight: it adopts only its own navigation stamps on landing, and a
  gesture is never forgiven.
2026-11-11 18:02:15 +04:00
Said Turksever ae66d07575 i18n(sv): the PR's 27 Street Level strings
Main added a Swedish catalog in #1630, so the catalog test failed on the
strings this PR adds. They use the catalog's terms (Gatunivå, nyckel,
bilder på gatunivå, vyns mitt).
2026-11-11 12:16:58 +00:00
Said Turksever 43893697aa Merge upstream main into pr/street-level-show-action 2026-11-11 12:15:18 +00:00
German Khokhlov 4fd36c44e0 Street Level: a replacement keeps its own state, and a posted view stands down
- Releasing a held viewer frees the old source's adapter without resetting
  the panel, so a replacement's error, outcome and armed OPEN survive; a
  cross-source replacement drops the old photo's details and marker.
- The action holds the open photo only for its own switch-off: a source
  switched by hand during the load stops it and reports cancelled.
- A posted view waiting on terrain watches the camera and stands down after a
  navigation or a globe press; a flat or straight-down camera asks no terrain.
- An arming link opens the panel without persisting the layout, and a posted
  view's street-level step runs as programmatic.
- An off-globe street_level point throws TypeError.
- zh-Hant uses the file's wording for the new strings; README and
  CURRENT-STATE count 35 tools.
2026-11-11 16:05:26 +04:00
German Khokhlov 45a33b54df Street Level: docs for the OPEN exception, panel freshness and what stops a request
- SECURITY.md: OPEN also takes the link's point when the screen centre
  cannot be picked at all.
- TOOLS.md: the panel's own link opens Street View only for a result issued
  in the last five minutes; an older one only arms.
- Changelog: the hiding modes stop a request, collapsing the panel does not;
  failure cards are in the interface language.
- CURRENT-STATE: a posted view without a point grounds its look-at too.
2026-11-11 13:46:35 +04:00
German Khokhlov f612fc2180 Street Level: a posted view without a photo point grounds its look-at too
show_in_gods_eye_view measured a view's altitude from the terrain before
taking the point the camera looks at; a view posted straight into the
embedded app used the ellipsoid altitude, so a camera over high ground
opened photos kilometres ahead. groundedLookAt moves to the view module,
and the embedded app uses it with the app's terrain heights for a posted
street_level that names no point.
2026-11-11 13:46:10 +04:00
German Khokhlov a8970c1f18 Street Level: a geocoder that throws fails show_street_level at its location stage
A provider that threw while geocoding (offline, an upstream down) escaped
the action as a bare error. show_street_level now reports it as its
location stage ("Could not arrive at …", the cause in detail), and nothing
is switched or looked up; fly_to_location itself is unchanged.
2026-11-11 13:46:10 +04:00
German Khokhlov 8334262109 Street Level: the action's failure cards in the interface language
The voice card showed English for show_street_level failures the catalog
did not know: a missing key, a map or source that could not be switched on,
the layer, invalid coordinates, an imagery fallback and the filter clause.
They now translate like the other action errors (exact messages and
patterns in speech.js), in all twelve languages; the spoken line stays
English, and the filter clause names the filters as their chips do.
2026-11-11 13:46:10 +04:00
Said Turksever e0fedc13dd Street Level: a link whose source has no working key says why 2026-11-11 09:21:06 +00:00
Said Turksever 5b7e917f94 Street Level: Hebrew strings for the show_street_level action 2026-11-11 08:55:55 +00:00
Said Turksever f878efe22d Merge commit '0ceee8df40b97c6c693e86a4443794cd4ed1ff35' into HEAD 2026-11-11 08:54:28 +00:00
German Khokhlov c76832ca69 Street Level: drop the unused coverageLines diagnostics
Nothing reads the drawn coverage lines: the browser gate aims elsewhere, so
lineCoverage's drawnLines, the providers' coverageLines and the layer's
coverageLines go, with their tests and the doc mention.
2026-11-10 23:58:03 +04:00
German Khokhlov e34b5103c6 Street Level: shorter comments and docs
- Comments in the action, the layer, embed, the Mapillary policy and the
  controls say why in fewer words; restatements of the code are gone.
- CURRENT-STATE's "Show me X on Street View" section is a list by topic and
  no longer says the browser gate aims at coverageLines.
- TOOLS.md's street-level view paragraph is shorter.
2026-11-10 23:54:16 +04:00
German Khokhlov f3996da0af Street Level: a landed link, a replaced photo and a retired lookup keep their state
- A link's photo step compares with the navigation token its own restore
  flight owned, so a navigation after the flight landed (while the layers
  still load) keeps the camera.
- A request that switches the source keeps the open photo up until it is
  replaced (holdViewer), so EXPAND stays across sources.
- A lookup given up while it looked leaves the open photo and the panel as
  they were, also when it found nothing or failed.
- A failed replacement shows its error inside the expanded viewer.
- A replacing lookup stops a playing Mapillary sequence before the found
  photo opens.
- The action refuses coordinates outside the globe, keeps a name given with
  coordinates as its label, and fly: false keeps the camera only for
  coordinates (its test-only entity and query branches are gone).
- Numbered entities carry their label; a view's street-level point is a pair.
- Mapillary's nearest search always asks for 50 m and 100 images.
- Tests: the lifecycle reproductions, PLAY on replacement, out-of-range
  coordinates, half a point, and the named label; three layer tests share
  enabledLayer; the second payload digest is gone.
- Docs: the tool count, the panel freshness rule, armed: false, the photo
  replacement, the browser-gate comment and a shorter changelog fragment.
2026-11-10 23:48:57 +04:00
Said Turksever 9c73939425 Street Level: leave an interrupted link and an overtaken view alone (gekh review)
- A pasted or start-up link opens its photos only when its restore's camera
  landed: a drag or search during the restore flight keeps the camera.
- A posted view whose photo step was cancelled or superseded no longer closes
  the viewer or flies; one that found a photo it could not load keeps the error.
- openNearest gains `replace` (keep the open photo while it looks, close it
  only when nothing opens, so EXPAND stays) and `proceed()` (asked before a
  found photo loads); the action uses both, so a navigation, a gesture or a
  hiding mode during the lookup loads no billed panorama.
- Mapillary's timeout message is a msg() string with all eleven translations.
2026-11-10 15:12:50 +00:00
Said Turksever 5370b8a268 Merge upstream main into pr/street-level-show-action
gevActions.test.mjs: keep the PR's street-level tests and main's analyst display tests.
2026-11-10 14:59:39 +00:00
quanticsoul4772 409dbf1660 Merge branch 'main' into upstream/firms-age-rounding 2026-11-09 13:44:05 -08:00
quanticsoul4772andClaude Opus 5.5 fdd153b73d ci: re-run checks after a failed Chrome download on the runner
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-09 08:38:03 -08:00
quanticsoul4772 f7b0367d4b Merge remote-tracking branch 'origin/main' into upstream/route-outage-unavailable
# Conflicts:
#	docs/CURRENT-STATE.md
2026-11-09 08:14:35 -08:00
quanticsoul4772andClaude Opus 5.5 fc6944951b refactor(fires): read both age formatters' unit from one helper
formatAge and formatAgeLabel repeated the validity guard, the <1h case,
the rounding and the 48-hour switch, which is why the rounding fix had to
land twice. Both now read { unit, count } from a private ageParts, so
they cannot pick different units again. Output is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-09 05:35:23 -08:00
quanticsoul4772 a42b38c4b3 Merge branch 'main' into upstream/firms-age-rounding 2026-11-09 04:48:51 -08:00
Said Turksever 7efcf37c62 Merge upstream main into pr/street-level-show-action
Locale conflicts: main's catalogs plus the PR's 10 street-level strings.
2026-11-09 00:22:26 +00:00
Said Turksever b7531e655d Merge upstream main into pr/street-level-show-action
Locale conflicts: main's catalogs plus the PR's 10 street-level strings.
2026-11-09 00:11:38 +00:00
Said Turksever efaeece117 Merge upstream main into pr/street-level-show-action
Locale conflicts: main's catalogs plus the PR's 10 street-level strings.
2026-11-08 20:30:04 +00:00
Said Turksever e5aee68e1d Merge upstream main into pr/street-level-show-action
Locale conflicts: main's catalogs plus the PR's 10 street-level strings.
2026-11-07 18:45:57 +00:00
quanticsoul4772 3dc65aca64 Merge branch 'main' into upstream/firms-age-rounding 2026-11-07 09:38:29 -08:00
quanticsoul4772 a073c58e50 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-07 09:38:09 -08:00
Said Turksever 489156366a Merge upstream main into pr/street-level-show-action
Street Level: keep the PR's billedPerImage contract note beside main's keyReady, and both sides' layer tests (armed OPEN, failing source) and (keyless default lighting).
2026-11-06 17:52:55 +00:00
Said Turksever 4e3f6bf505 Merge upstream main into pr/street-level-show-action
Locale conflicts: main's catalogs plus the PR's 10 street-level strings.
2026-11-06 16:56:54 +00:00
Said Turksever ce72e12743 Merge upstream main into pr/street-level-show-action
Locale conflicts: main's catalogs plus the PR's 10 street-level strings.
2026-11-06 01:41:37 +00:00
Said Turksever 78773d2ccb Merge upstream main into pr/street-level-show-action
Re-pin the Realtime payload digests for main's move_camera pitchDeg
argument; the prompt measures 56,276 under the 56,500 ceiling.
2026-11-06 01:15:36 +00:00
Said Turksever 30e6cd39c7 fix(street-level): re-check after every await; close the old photo; fresh panel views
Address samehkhamis's review on #1113:

- show_street_level checks the turn, the camera's owner (navigation stamp,
  re-taken where its own flight lands, or a press/wheel on the globe) and
  the hiding modes after the flight, the map switch, the layer switch-on and
  the lookup; a photo that opened after either is closed again. The hidden
  panel is also refused before the flight.
- An open photo closes before a new request arms or looks up, so a place
  with no imagery shows its own error.
- The panel posts every view with `fresh`; the inline app opens a billed
  photo only for a fresh one.
- A passed-through view's street_level point moves with a new camera
  position; revealPanel does nothing above 720px, scrolling included.
- Document that the app's own links do not carry `sl`.
2026-11-06 00:40:20 +00:00
Said Turksever 14f7326dbf Merge upstream main into pr/street-level-show-action
Re-pin the Realtime payload digests for main's control_cockpit wording.
2026-11-06 00:23:42 +00:00
Said Turksever 451f3e41cf fix(street-level): links keep the recipient's preferences; aim under the screen centre
Review fixes for gekh's #1113 round:
- a link runs show_street_level with origin 'programmatic', so its source
  switch and layer are not saved as the recipient's preferences;
- with no place named, the action looks under the screen centre
  (aim 'screen-centre', armed false), keeps the sky refusal and leaves a
  link's armed point alone;
- an armed OPEN aimed at the sky opens nothing instead of the far link point;
- an imagery-stage failure is spoken as an error, missing imagery as missing
  (street.nearestOutcome); the card says "the view centre" in the interface
  language;
- every photo the action opens reveals its panel, and revealPanel scrolls the
  panel into view in the phone sheet;
- embed: EXPAND survives embed's own clean view, the lifted photo hides in
  cockpit, recording and scene playback, and a failed photo step closes the
  earlier photo before the view flies;
- links pasted into an open tab run their street-level step after the
  restore settles; a superseded start link does not;
- Mapillary's nearest search gives up after 12 s, body included.
2026-11-05 18:25:21 +00:00
quanticsoul4772 c0a74be7f6 Merge branch 'main' into upstream/firms-age-rounding 2026-11-05 10:23:31 -08:00
quanticsoul4772 f08a7a2d72 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-05 10:23:06 -08:00
quanticsoul4772andClaude Opus 5.5 ceff3eb6c9 test(fires): pin the 47.5-hour flip to days and name where the age shows
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 05:43:40 -08:00
quanticsoul4772 5c45469dca Merge branch 'main' into upstream/route-outage-unavailable 2026-11-04 12:47:05 -08:00
quanticsoul4772 bf1c44a9d3 Merge branch 'main' into upstream/firms-age-rounding 2026-11-04 12:46:52 -08:00
quanticsoul4772 ffbbb970f9 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-04 07:57:49 -08:00
quanticsoul4772 bb4a7050d3 Merge branch 'main' into upstream/firms-age-rounding 2026-11-04 07:57:35 -08:00
Said Turksever 3f55077c2c Merge upstream main (typed chat translations) into pr/street-level-show-action 2026-11-04 14:24:04 +00:00
quanticsoul4772 52aea49faa Merge branch 'main' into upstream/route-outage-unavailable 2026-11-03 20:30:43 -08:00
quanticsoul4772 483807530b Merge branch 'main' into upstream/firms-age-rounding 2026-11-03 20:30:20 -08:00
Said Turksever 77ac2894d4 Merge upstream main (typed AI chat) into pr/street-level-show-action 2026-11-04 00:19:05 +00:00
Said Turksever 3b9aebe9f1 Merge upstream main (restricted airspace) into pr/street-level-show-action 2026-11-03 22:57:59 +00:00
quanticsoul4772andClaude Opus 5.5 a4bd028bd2 changelog: FIRMS age rounding (#1531)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 11:47:41 -08:00
quanticsoul4772andClaude Opus 5.5 9d27366b3e fix(fires): show a 47.6-hour-old detection as 2d, not 48h
formatAge and formatAgeLabel picked the hour unit on the raw value and
rounded afterwards, so a detection between 47.5 and 48 hours old read
48h, while one exactly 48 hours old read 2d. Round to whole hours first,
then choose the unit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 11:43:10 -08:00
Said Turksever fed397a764 test(voice): raise the Realtime size budget to 56,500 for show_street_level 2026-11-03 18:07:59 +00:00
Said Turksever 34f694c6f2 Merge upstream main (voice geometry: resolve_area, find_imagery, osm_query) into pr/street-level-show-action 2026-11-03 18:05:11 +00:00
Said Turksever e5ed90ff17 Merge commit '2ebde97fb3af19402eab83fca2bfbec9ddfd166a' into easy-wins-20261103-pr1509 2026-11-03 16:17:32 +00:00
Said Turksever 7c1d5ff576 fix: resolve review findings and refresh PR #1509 against main 2026-11-03 16:17:24 +00:00
Said Turksever 7703090d90 fix(street-level): retire lookups and preserve embedded photos
Clear cancelled lookup loading over an existing photo, release caller abort listeners on internal retirement, surface embedded errors, and permit embedded photos over the clean globe. Add seven regression cases and update runtime documentation. Preserve the published localization and other review fixes.
2026-11-03 15:40:46 +00:00
Said Turksever 0ec57057fa fix(street-level): review fixes for show_street_level and sl= links
P1: on a phone the Street Level panel sits in the Cameras tab sheet, so the
action and sl= links opened photos nobody could see and still reported
success. A photo that opens, and a link that arms, now open that sheet
(revealPanel). In cockpit, clean view, recording or scene playback the action
stops at a new `view` stage before anything switches or loads.

P2s:
- Mapillary's nearest search asks for 100 photos, the API's most. It cuts
  the radius's photos at the limit in no particular order, so 8 left the
  distance sort and the imagery filter a few arbitrary photos.
- The MCP globe panel opens a billed photo only for a result issued in the
  last five minutes (show_in_gods_eye_view stamps issued_at); a host that
  mounts the panel again hands back the old result, which only arms.
- OPEN uses an armed link's point only while the view centre is within 1 km
  of it; a user who moved on opens what the view shows.
- show_in_gods_eye_view measures a camera's altitude from the terrain under
  it before taking its look-at point, so high cities no longer put the
  photos kilometres too far ahead.
- The armed-link toast is translated, and the new view-stage message too,
  in all 11 catalogs.
- Tests: a turn superseded during the geocode, flight or map switch changes
  nothing; an unknown key state lets a cold sl= link through; a wide tilted
  area still opens photos at its centre.
2026-11-03 15:30:48 +00:00
Said Turksever 8ce01cf48e Merge upstream main (OSM sea-mark and ski-piste overlays) into pr/street-level-show-action
Keeps main's locale catalogs and re-adds this PR's eight Street Level strings; re-pins the Realtime payload digests for main's two new overlay layers, where the unchanged tools match main's pin, so show_street_level is still the whole difference.
2026-11-03 15:12:11 +00:00
Said Turksever fcf240e789 test(voice): raise the Realtime prompt budget to 53,000 for show_street_level
Main's prompt sits 28 characters under the 52,000 ceiling, and the
street-level tool needs 337 even with no descriptions. Kept as its own
commit so the budget decision can be reverted on its own.
2026-11-03 14:32:03 +00:00
Said Turksever ed2703d345 feat(street-level): restore Mapillary's nearest-image lookup for show_street_level and sl= links
The slim Street Level core (#974) removed nearestImage from the Mapillary
provider. show_street_level and sl=mapillary links need it to open the
photo closest to a point, so this restores it on the slim core: Graph API
/images within 50 m, sorted by distance, first one that passes the imagery
filter. Kept as its own commit so it can be dropped if #974's removal
should stand.
2026-11-03 14:32:03 +00:00
Said Turksever 98de41c94a feat(street-level): "show me X on Street View" action, street_level view field and sl= links
show_street_level (voice and typed) flies to a place and opens its nearest
street-level photo from Google Street View, Mapillary, KartaView or
Panoramax; Street View switches to Google 3D first. The view model gains a
street_level field, share links carry sl=<source>,<lat>,<lon>, and
show_in_gods_eye_view can open a photo. A plain sl=google link only arms the
point and waits for OPEN, so it never bills a key without a click.

Embed pages show an open photo in an overlay that Esc closes. On-screen
strings are translated into es, ru, zh-CN and zh-Hant. Covered by unit tests
and three new steps in the Street Level browser suite.
2026-11-03 14:32:02 +00:00
quanticsoul4772 562a2ea544 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-03 06:19:14 -08:00
quanticsoul4772 fa2e1a9ec7 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-02 09:52:07 -08:00
quanticsoul4772 55d5896cb3 Merge branch 'main' into upstream/route-outage-unavailable 2026-11-02 09:25:34 -08:00
quanticsoul4772andClaude Opus 5.5 1bff1961b4 fix(routes): keep the 12 s budget from arrival and name the outage to voice
Starting the upstream timeout after the outbound queue let a request run
up to about 19 s, past the Directions layer's 15 s; the budget again runs
from arrival, queue wait included, and a budget spent before the request
leaves the queue answers 429 "routing busy" rather than an outage.

plan_route threw "Routing HTTP 502" for an outage, and voice passes only
the message on; it now throws "Routing service unavailable (HTTP 502)".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-02 09:21:08 -08:00
quanticsoul4772andClaude Opus 5.5 536f371dde docs(changelog): credit #1509 in the fragment
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-02 08:52:35 -08:00
quanticsoul4772andClaude Opus 5.5 f8795ff426 fix(routes): report an upstream routing outage as unavailable
When the OSRM routing service failed (a 5xx, a 403 block, a 404), answered
with a body that was not JSON, or did not answer at all (timeout, network
error), /api/route returned HTTP 200 with 'no route found' or 'route proxy
error'. plan_route turned that into invalid_arguments ("No walk route from
...: no route found"), blaming the caller for an outage, and the directions
layer reported "No route found between A and B".

The proxy now answers all of these with 502 {ok:false, error:'routing
service unavailable'}: a rejected upstream fetch, any failure status but
400, a non-JSON content type, and a body read that fails or does not parse.
An exception in the proxy's own code after that point is still a 200 'route
proxy error'. createRouteService throws for a 5xx before reading the body's
ok:false, so plan_route reports unavailable, and the directions source
treats any 5xx as "Routing unavailable (HTTP n)", also when the failure
page in front of the proxy is not JSON. OSRM's own refusal (a 400 such as
NoRoute, or a 200 without a route) still reads "no route found".

The 12 s upstream timeout now starts when the request leaves rather than
before it waits for an outbound slot, so a wait in the proxy's own queue
(up to about 8 s at the 1 s spacing) is not reported as an outage.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-02 08:42:57 -08:00
395 changed files with 54476 additions and 1574 deletions

No files matched your search

+13
View File
@@ -280,6 +280,11 @@ AISSTREAM_API_KEY=
# CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry
# CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured
# Optional: Data Library (DATA LAYERS ▸ Browse data library). Every entry is
# fetched live from its provider; nothing is hosted here. See docs/DATA-LIBRARY.md.
# GEV_LIBRARY_ENABLE= # comma-separated ids of operator opt-in entries to serve (packs use their host's own switch, e.g. CCTV_NYC_ENABLED)
# GEV_LIBRARY_DISABLE= # comma-separated ids to remove from this instance entirely
# Optional: comma-separated Overpass URLs; use only instances you run or pay for.
# Also enables voice place search (osm_query, "hospitals in Kathmandu").
OVERPASS_UPSTREAMS=
@@ -352,3 +357,11 @@ GEMINI_CHAT_MODEL=gemini-3.5-flash-lite
# the free plan). After that Local says the daily limit is reached.
# Raise it on a paid plan.
# WORLDNEWSAPI_DAILY_READS=30
# Rail layer: every feed is keyless, so nothing needs setting. Packs are an
# opt-in extra for live trains (the operator's lines, stop names, next stops):
# scripts/build-rail-paths.mjs (--feed <id> ... or --all) writes path and
# schedule packs, and the server reads them from this directory
# (default .cache/rail-paths). The build reads it from its own environment,
# not from this file: export it, or pass the same folder as --out.
# RAIL_PATHS_DIR=
+4
View File
@@ -53,6 +53,10 @@ jobs:
if: github.event_name == 'pull_request'
run: npm run layer-token:check -- --base-ref origin/main
- name: Check Data Library manifests and share tokens
if: github.event_name == 'pull_request'
run: npm run library:check -- --base-ref origin/main
- name: Run unit tests
run: npm test
+2
View File
@@ -14,6 +14,8 @@ qa-shots/
output/
.gev-logs/
.gev-cache/
# Local build output, e.g. rail path packs (scripts/build-rail-paths.mjs).
.cache/
# Browser QA gate evidence (screenshots, console and network dumps).
qa-artifacts/
# Playwright MCP session logs: browser consoles can carry tokens in URLs.
+2 -1
View File
@@ -129,6 +129,7 @@ prints anything; never fix that by renaming or deleting PR A's published token.
The highest-leverage places to jump in:
- **🌆 Add a CCTV source pack.** Austin is the reference camera source. Follow the [CCTV integration workflow](docs/CCTV_INTEGRATION.md) for source review, provider registration, proxy security, tests, and attribution. The proxy only fetches server-registered URLs — never client-supplied ones; see [SECURITY.md](SECURITY.md).
- **📚 Add a Data Library entry.** A regional or niche source that is not a core layer belongs in the Data Library: one JSON manifest pointing at a live API, provider-hosted file or tile endpoint, with its licence and attribution, and no JavaScript. See [docs/DATA-LIBRARY.md](docs/DATA-LIBRARY.md). Library entries use their own share-token ledger, not the layer tokens below.
- **🛰️ Add or improve a data layer.** Layer factories live in `src/layers/<family>/`, with source, record, controller and renderer owners implementing the layer interface (`init/enable/disable/update/destroy/getStats`, optional `getDetectableObjects`/`getStats`). Use an existing layer as a template, and see [Adding a new layer](#adding-a-new-layer) for what the PR needs.
- **🎙️ Extend voice control.** Voice arguments are defined in `src/voice/actionSchemas.js`, with server-side descriptions in `server/providers/openai/tools.js` and client-side execution in `src/voice/gevActions.js`. Keep the tool surface tight and the responses honest (confirm only what actually happened).
- **🎨 Add a visual style.** Styles are GLSL post-process shaders in `src/styles/`.
@@ -228,7 +229,7 @@ ownership and adoption process.
1. Branch off `main`.
2. Keep `npm run build`, `npm test`, and `npm run test:track` green and avoid new console errors, plus the [feature gate](#feature-regression-gates) for the area you touched.
3. If you change runtime behavior, update `docs/CURRENT-STATE.md` in the same PR, and add your changelog entry as `changelog.d/<pr-number>-<slug>.md` instead of editing `CHANGELOG.md` — see [changelog.d/README.md](changelog.d/README.md). Distinct filenames don't conflict; a release assembles the directory into `CHANGELOG.md`. A documentation claim that needs a regression test should assert against `docs/CURRENT-STATE.md` or `README.md`, which a PR still edits directly.
4. If you add or change a data source, update [DATA_SOURCES.md](DATA_SOURCES.md) with its license and attribution. **Don't add data you don't have the right to redistribute** — fetch it at runtime instead.
4. If you add or change a data source, update [DATA_SOURCES.md](DATA_SOURCES.md) with its license and attribution (for a Data Library entry, `npm run library:docs` writes its row). **Don't add data you don't have the right to redistribute** — fetch it at runtime instead.
5. Describe what you changed and how you verified it (screenshots welcome for anything visual).
6. If you add a new layer, fill in the new-layer section of the PR template. See [Adding a new layer](#adding-a-new-layer).
+144 -3
View File
@@ -31,6 +31,7 @@ How to read this:
| **adsb.lol** | Military flights + aircraft traces | ODbL 1.0 | "adsb.lol" (ODbL) |
| **adsbdb** (`api.adsbdb.com`) | Aircraft type, model name and registration by ICAO hex for enriched flights; airline and origin/destination airports for the tracked flight's callsign | No licence or rate limit is published for the service. Its own credits: aircraft data from PlaneBase, ICAO-to-N-number conversion by Guillaume Michel, and flight route data that "is the work of David Taylor, Edinburgh and Jim Mason, Glasgow, and may not be copied, published, or incorporated into other databases without the explicit permission of David J Taylor, Edinburgh" | "adsbdb — api.adsbdb.com" (courtesy); route data credited to David Taylor, Edinburgh, and Jim Mason, Glasgow |
| **Planespotters.net** (`api.planespotters.net/pub/photos/hex/{hex}`) | One thumbnail photo of the tracked aircraft, by ICAO hex, in the CONTEXT panel's CONTACTS view under the tracked flight's name | [Planespotters photo API terms](https://www.planespotters.net/photo/api); each photo remains the copyright of its photographer. Thumbnails may be hotlinked on the condition that the photographer is named next to the photo and the photo links to its Planespotters page with a followable link | "Photo © {photographer}" beside every photo that names one, linked to its Planespotters page; "Planespotters.net" in Data attribution |
| **Open Vessel Photos** (`openvesselphotos.org/v1/mmsi/{mmsi}`) | One photo of the selected vessel, by MMSI, in the CONTEXT panel's CONTACTS view under the vessel's name and MMSI | Free, keyless public API ([README](https://github.com/louptheron/openvesselphotos)); each photo keeps the open licence its author chose (CC BY, CC BY-SA, CC0, or public domain), given with it in the API | "Photo: {photographer} · {licence}" beside every photo, linked to the vessel's Open Vessel Photos page; "Open Vessel Photos" in Data attribution |
| **AISStream.io** | Live vessels (AIS) | Free, beta, no formal ToS; AIS is a public broadcast | "AISStream.io" (courtesy) |
| **CelesTrak** | Satellite TLEs (SGP4) and SATCAT records (object type, operational status, owner) that file each satellite under a class | US-government-origin data, no license; citation requested | "CelesTrak (celestrak.org), Dr. T.S. Kelso" |
| **The Space Devs — Launch Library 2 v2.3** | Recent launch, payload, stage, and recovery metadata for Space Missions (30d) | [The Space Devs terms of use](https://github.com/TheSpaceDevs/Tutorials/blob/main/faqs/faq_TSD.md#terms-of-use): data may be used and shared in any form; avoid forwarding it without added value; attribution is encouraged (not mandatory). [Official API limits](https://ll.thespacedevs.com/docs/): 15 unauthenticated calls/hour; optional token | "Launch Library 2 — The Space Devs" (courtesy attribution) |
@@ -110,6 +111,36 @@ How to read this:
| **YRT/Viva** (rtu.york.ca) | Transit layer live vehicles, York Region ON | York Region Transit realtime data, first-party York Region infrastructure under the YRT Open Data Licence v2.0, which permits commercial use and makes attribution optional. Viva route ids are named by colour from York Region’s published GTFS. | "Contains public transit Information made available under YRT’s Open Data Licence" — the licence’s own wording, given though not required |
| **Burlington Transit** (opendata.burlington.ca) | Transit layer live vehicles, Burlington ON | Burlington Transit realtime data under the City of Burlington Open Data Terms of Use, which permit any lawful purpose. The terms are revocable and must travel with the data, so the credit links them. Route ids are internal keys and are joined to the City’s published GTFS. | Credit to the City of Burlington plus a link to its Open Data Terms of Use, which the licence requires |
| **SMTR Rio de Janeiro** (dados.mobilidade.rio) | Transit layer live vehicles, Rio de Janeiro: municipal buses (SPPO) and the Mobi-Rio BRT, as two feeds | CC BY 4.0, stated on both data.rio catalog pages. Keyless GTFS-Realtime published by the Secretaria Municipal de Transportes, labelled beta, with a gateway limit of 5 requests a second and 60 a minute. Line names come from SMTR’s GTFS `routes.txt` (dados.mobilidade.rio/gtfs/schedule), under the same licence. | Attribution to SMTR / Prefeitura da Cidade do Rio de Janeiro under CC BY 4.0 |
| **Metro-North** (api-endpoint.mta.info, rrgtfsfeeds.s3.amazonaws.com) | Rail layer trains, New York, NY: Metro-North Railroad (MTA). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (api-endpoint.mta.info) add delays at its stops. | [MTA Data Feed Terms and Conditions](https://www.mta.info/developers/terms-and-conditions): "Developers must download and store the MTA data feed on a non-MTA server and redistribute the data to users from that server." This proxy is that server: browsers never contact MTA. The feed stamps every vehicle with the header time, so a fix is aged from the snapshot, not from the train. | "Trains (MTA): [MTA Data Feed Terms and Conditions](https://www.mta.info/developers/terms-and-conditions)" |
| **LIRR** (api-endpoint.mta.info, rrgtfsfeeds.s3.amazonaws.com) | Rail layer trains, New York, NY: Long Island Rail Road (MTA). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (api-endpoint.mta.info) add delays at its stops. | [MTA Data Feed Terms and Conditions](https://www.mta.info/developers/terms-and-conditions): "Developers must download and store the MTA data feed on a non-MTA server and redistribute the data to users from that server." The feed was seen frozen for two days on 2026-11-06; the layer ages trains by their own report time, so a frozen feed empties instead of showing old trains as live. | "Trains (MTA): [MTA Data Feed Terms and Conditions](https://www.mta.info/developers/terms-and-conditions)" |
| **MBTA rail** (cdn.mbta.com) | Rail layer trains, Boston, MA: Massachusetts Bay Transportation Authority. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (cdn.mbta.com) add delays at its stops. | [MassDOT Developers License Agreement](https://cdn.mbta.com/sites/default/files/2023-08/mbta-massdot-develop-license-agreement.pdf): "MassDOT ... hereby grants You (Licensee) non-exclusive, limited, and revocable rights to use, reproduce, and redistribute the Data. ... Clearly acknowledge MassDOT as the provider of the Data." The same feed the Transit layer polls; this layer keeps only Commuter Rail, rapid transit and light rail. | "Trains (MBTA / MassDOT): [MassDOT Developers License Agreement](https://cdn.mbta.com/sites/default/files/2023-08/mbta-massdot-develop-license-agreement.pdf)" |
| **SEPTA Regional Rail** (www3.septa.org, github.com) | Rail layer trains, Philadelphia, PA: Southeastern Pennsylvania Transportation Authority. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (www3.septa.org) add delays at its stops. | [SEPTA Trip Planning Data Agreement](https://www3.septa.org/developer/): "non-exclusive, limited and revocable right to use, reproduce and redistribute the Trip Planning Data". The agreement names the static data; SEPTA publishes the realtime feeds on the same developer page with no separate terms. Credit is text only: SEPTA trademarks may not be used. | "Trains (SEPTA): [SEPTA Trip Planning Data Agreement](https://www3.septa.org/developer/)" |
| **SEPTA trolleys** (www3.septa.org, github.com) | Rail layer trains, Philadelphia, PA: Southeastern Pennsylvania Transportation Authority. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (www3.septa.org) add delays at its stops. | [SEPTA Trip Planning Data Agreement](https://www3.septa.org/developer/): "non-exclusive, limited and revocable right to use, reproduce and redistribute the Trip Planning Data". The bus feed, filtered to the trolley routes and the Norristown High Speed Line. The Market-Frankford and Broad Street lines are in no SEPTA feed. Routes T1–T5, G1, D1 and D2 (trams) and M1, the Norristown High Speed Line (metro). | "Trains (SEPTA): [SEPTA Trip Planning Data Agreement](https://www3.septa.org/developer/)" |
| **MARC** (mdotmta-gtfs-rt.s3.amazonaws.com, feeds.mta.maryland.gov) | Rail layer trains, Baltimore–Washington: Maryland Transit Administration. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (mdotmta-gtfs-rt.s3.amazonaws.com) add delays at its stops. | [MDOT MTA developer feeds (no endorsement implied)](https://www.mta.maryland.gov/developer-resources): "MTA does not guarantee the accuracy of the data nor does it warrant or endorse any application that uses this data." Published for open fetch with this disclaimer and no licence; the site terms cover only the website and apps. Shown with attribution and the no-endorsement wording, and switched off if MDOT MTA publishes restrictive terms. | "Trains (Maryland Transit Administration): [MDOT MTA developer feeds (no endorsement implied)](https://www.mta.maryland.gov/developer-resources)" |
| **Tri-Rail** (gtfsr.tri-rail.com, gtfs.tri-rail.com) | Rail layer trains, South Florida: South Florida Regional Transportation Authority. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfsr.tri-rail.com) add delays at its stops. | [No licence stated (open feed pages)](https://gtfsr.tri-rail.com/): "Download Realtime Feeds Below:". Used without a published licence (project owner decision, 2026-11-10). gtfsr.tri-rail.com (the page quoted above, read 2026-11-10) links the GTFS-RT trip updates, alerts and position updates with no terms, and the static GTFS (gtfs.tri-rail.com/gtfs.zip, publisher ETA Transit Systems) states none either; no SFRTA data terms were found. The realtime vehicles carry the train number as their trip id and no route. | "Trains (South Florida Regional Transportation Authority (Tri-Rail)): [No licence stated (open feed pages)](https://gtfsr.tri-rail.com/)" |
| **RTD** (open-data.rtd-denver.com, www.rtd-denver.com) | Rail layer trains, Denver, CO: Regional Transportation District. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (open-data.rtd-denver.com) add delays at its stops. | [RTD GTFS-Realtime License Agreement](https://www.rtd-denver.com/business-center/open-spatial-data/gtfs-realtime-license-agreement): "non-exclusive, limited and revocable rights to use, reproduce, and redistribute the Data". RTD marks may not be used with the data, so the credit is text only. The published `www` URL redirects off its own origin, which the proxy refuses, so the final host is the one registered. | "Trains (RTD): [RTD GTFS-Realtime License Agreement](https://www.rtd-denver.com/business-center/open-spatial-data/gtfs-realtime-license-agreement)" |
| **CapMetro MetroRail** (data.texas.gov) | Rail layer trains, Austin, TX: Capital Metropolitan Transportation Authority. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. | [CapMetro Open Data Terms](https://www.capmetro.org/metrolabs/): "non-exclusive, limited and revocable rights to use, reproduce, and redistribute CMTA Data". The bus feed the Transit layer polls; MetroRail is route 550. | "Trains (CapMetro): [CapMetro Open Data Terms](https://www.capmetro.org/metrolabs/)" |
| **METRO light rail** (svc.metrotransit.org) | Rail layer trains, Minneapolis–St Paul, MN: Metro Transit (Metropolitan Council). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. | [No licence published](https://svc.metrotransit.org/). Publishes no licence or terms: the data feeds page linked above (read 2026-11-10) lists the GTFS and GTFS-realtime files and shows only "© 2020 Metro Transit", so there is nothing to quote. Used without a published licence, on the same basis as the Transit layer's Metro Transit feed, which polls the same file (DATA_SOURCES.md). Blue Line 901 and Green Line 902 only; Northstar ended on 2026-01-04 and route 888 is now a bus. | "Trains (Metro Transit / Metropolitan Council): [No licence published](https://svc.metrotransit.org/)" |
| **LYNX** (gtfsrealtime.ridetransit.org) | Rail layer trains, Charlotte, NC: Charlotte Area Transit System. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfsrealtime.ridetransit.org) add delays at its stops. | [No licence published](https://www.charlottenc.gov/cats). Publishes no terms; used without a published licence (project owner decision, 2026-11-10). The feed host (gtfsrealtime.ridetransit.org) serves the files with no page and no terms, so there is nothing to quote (2026-11-10: 126 vehicles, 13 of them trains; 205 trip updates, 11 on the two lines; every realtime trip id is a static trip id). No CATS page with terms for them was found: charlottenc.gov answered 403 to automated requests, and the CATS Terms and Conditions found there by search govern the CATS mobile app. Routes 501 LYNX Blue Line (light rail) and 510 CityLYNX Gold Line (streetcar), both route_type 0: transit, tram. | "Trains (Charlotte Area Transit System): [No licence published](https://www.charlottenc.gov/cats)" |
| **GRT ION** (webapps.regionofwaterloo.ca) | Rail layer trains, Waterloo Region, ON: Grand River Transit (Region of Waterloo). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. | [Region of Waterloo Open Data Licence v2.0](https://www.regionofwaterloo.ca/government-and-council/transparency-and-accountability/open-data/): "You are free to: Copy, publish, distribute and transmit the Information ... Use the Information commercially". The registered URL is the `/2` light-rail feed; the suffix-less URL serves a file frozen since February 2026 and is not used. | "Trains (Contains information licensed under the Open Data Licence – Region of Waterloo): [Region of Waterloo Open Data Licence v2.0](https://www.regionofwaterloo.ca/government-and-council/transparency-and-accountability/open-data/)" |
| **Queensland Rail Citytrain** (gtfsrt.api.translink.com.au) | Rail layer trains, South East Queensland, Australia: Translink (Queensland Government). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfsrt.api.translink.com.au) add delays at its stops. | [CC BY 4.0](https://translink.com.au/about-translink/open-data): "Our data is licensed under a Creative Commons Attribution 4.0 International License." The terms add: "You must not represent that the State in any way endorses your Application." | "Trains (Translink / Queensland Government): [CC BY 4.0](https://translink.com.au/about-translink/open-data)" |
| **G:link** (gtfsrt.api.translink.com.au) | Rail layer trains, Gold Coast, Australia: Translink (Queensland Government). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfsrt.api.translink.com.au) add delays at its stops. | [CC BY 4.0](https://translink.com.au/about-translink/open-data): "Our data is licensed under a Creative Commons Attribution 4.0 International License." | "Trains (Translink / Queensland Government): [CC BY 4.0](https://translink.com.au/about-translink/open-data)" |
| **Adelaide Metro** (gtfs.adelaidemetro.com.au) | Rail layer trains, Adelaide, Australia: Adelaide Metro (Rail Commissioner). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfs.adelaidemetro.com.au) add delays at its stops. | [CC BY 4.0](https://data.sa.gov.au/data/dataset/https-gtfs-adelaidemetro-com-au): "Creative Commons Attribution 4.0". Licence as catalogued on data.sa.gov.au; the operator developer page sits behind a bot check. The feed is labelled beta. Its gateway answers base64 text unless the request sends `Accept: application/x-google-protobuf`, so the proxy sends that header for this feed. | "Trains (Adelaide Metro / Government of South Australia): [CC BY 4.0](https://data.sa.gov.au/data/dataset/https-gtfs-adelaidemetro-com-au)" |
| **Irish Rail** (api.irishrail.ie, www.transportforireland.ie) | Rail layer trains, Ireland: Iarnród Éireann / Irish Rail. Placed at the station or timing point last reported, not by GPS. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, the delay each train's report states moves its stops. | [No licence published (positions); CC BY 4.0 (NTA timetable)](https://developer.nationaltransport.ie/usagepolicy). The train positions publish no licence or terms: the Irish Rail realtime API page (api.irishrail.ie/realtime/, read 2026-11-10) documents the calls and says only that "Irish Rail provides this information as is" (quoted under `limits`), so there is nothing to quote. They are used without a published licence, credited "Iarnród Éireann / Irish Rail". The NTA's CC BY 4.0 covers only the NTA's GTFS data (the timetable), not these positions. Positions are the station or timing point the signalling system last reported, not GPS. | "Trains (Iarnród Éireann / Irish Rail; timetable: National Transport Authority, provided as is): [No licence published (positions); CC BY 4.0 (NTA timetable)](https://developer.nationaltransport.ie/usagepolicy)" |
| **VR (Finland)** (rata.digitraffic.fi) | Rail layer trains, Finland: VR, via Fintraffic Digitraffic. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (rata.digitraffic.fi) add delays at its stops. | [CC BY 4.0](https://www.digitraffic.fi/en/terms-of-service/): "Fintraffic's open data is licensed under the Creative Commons 4.0 By license. It gives the right to distribute, remix, tweak, and build upon our data, even commercially, as long as you credit the source for the original creation." The Digitraffic-User header names the application, never a person. | "Trains (Source: Fintraffic / digitraffic.fi, license CC 4.0 BY): [CC BY 4.0](https://www.digitraffic.fi/en/terms-of-service/)" |
| **Renfe long and medium distance** (gtfsrt.renfe.com, ssl.renfe.com) | Rail layer trains, Spain: Renfe (AVE, Avlo, Alvia, Euromed, Intercity, Media Distancia). Live GPS positions. Its vehicles name no route, so it is listed only once its path and schedule packs are built from its static GTFS; trip updates (gtfsrt.renfe.com) then add delays at its stops. | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/): "CC-BY-4.0". Licence field of the data.renfe.com datasets (posicion-vehiculos-av-ld-md and the static GTFS). "Esta información se actualiza cada 15 segundos". Iryo and OUIGO España are not in Renfe’s feeds. | "Trains (Renfe): [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)" |
| **Renfe Cercanías** (gtfsrt.renfe.com, ssl.renfe.com) | Rail layer trains, Spain: Renfe Cercanías and Rodalies. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfsrt.renfe.com) add delays at its stops. | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/): "CC-BY-4.0". Licence field of the data.renfe.com Cercanías datasets. Every vehicle in this feed is a Cercanías or Rodalies train. | "Trains (Renfe): [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)" |
| **Vy** (api.entur.io, storage.googleapis.com) | Rail layer trains, Norway: Vy (Norway), via Entur. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. | [NLOD 2.0](https://data.norge.no/nlod/en/2.0): "Entur's open data is published under the Norwegian Licence for Open Government Data (NLOD). NLOD permits free reuse for any purpose, including commercial use, subject to attribution requirements". | "Trains (Inneholder data under Norsk lisens for offentlige data (NLOD) tilgjengeliggjort av Entur AS): [NLOD 2.0](https://data.norge.no/nlod/en/2.0)" |
| **Go-Ahead** (api.entur.io, storage.googleapis.com) | Rail layer trains, Norway: Go-Ahead Norge, via Entur. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (api.entur.io) add delays at its stops. | [NLOD 2.0](https://data.norge.no/nlod/en/2.0): "Entur's open data is published under the Norwegian Licence for Open Government Data (NLOD). NLOD permits free reuse for any purpose, including commercial use, subject to attribution requirements". | "Trains (Inneholder data under Norsk lisens for offentlige data (NLOD) tilgjengeliggjort av Entur AS): [NLOD 2.0](https://data.norge.no/nlod/en/2.0)" |
| **Flytoget** (api.entur.io, storage.googleapis.com) | Rail layer trains, Norway: Flytoget, via Entur. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (api.entur.io) add delays at its stops. | [NLOD 2.0](https://data.norge.no/nlod/en/2.0): "Entur's open data is published under the Norwegian Licence for Open Government Data (NLOD). NLOD permits free reuse for any purpose, including commercial use, subject to attribution requirements". | "Trains (Inneholder data under Norsk lisens for offentlige data (NLOD) tilgjengeliggjort av Entur AS): [NLOD 2.0](https://data.norge.no/nlod/en/2.0)" |
| **SJ Nord** (api.entur.io, storage.googleapis.com) | Rail layer trains, Norway: SJ Nord, via Entur. Live GPS positions, read from SIRI VM. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, the delay each train's report states moves its stops. | [NLOD 2.0](https://data.norge.no/nlod/en/2.0): "Entur's open data is published under the Norwegian Licence for Open Government Data (NLOD). NLOD permits free reuse for any purpose, including commercial use, subject to attribution requirements". SJ Nord trains are missing from Entur’s GTFS-RT vehicle positions (0 vehicles on 2026-11-06); they are read from SIRI VM instead (13 trains on 2026-11-09). Its journey refs are DatedServiceJourneys, not the GTFS trip ids, so a train is matched to its run by line and position. | "Trains (Inneholder data under Norsk lisens for offentlige data (NLOD) tilgjengeliggjort av Entur AS): [NLOD 2.0](https://data.norge.no/nlod/en/2.0)" |
| **KTMB** (api.data.gov.my) | Rail layer trains, Peninsular Malaysia: Keretapi Tanah Melayu (ETS, Intercity, KTM Komuter, Shuttle Tebrau). Live GPS positions. Its vehicles name no route, so it is listed only once its path and schedule packs are built from its static GTFS. | [CC BY 4.0](https://developer.data.gov.my/realtime-api/gtfs-realtime): "All GTFS Realtime feeds are licensed under CC BY 4.0". developer.data.gov.my, read 2026-11-09; its GTFS Static page says the same of "All GTFS Static feeds". The static GTFS has no shapes. | "Trains (KTMB, via data.gov.my (Government of Malaysia)): [CC BY 4.0](https://developer.data.gov.my/realtime-api/gtfs-realtime)" |
| **Seattle Streetcar** (s3.amazonaws.com, metro.kingcounty.gov) | Rail layer trains, Seattle, WA: City of Seattle (South Lake Union and First Hill lines), via King County Metro. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (s3.amazonaws.com) add delays at its stops. | [King County Transit Data Terms of Use](https://kingcounty.gov/en/dept/metro/rider-tools/mobile-and-web-apps): "King County grants you a limited, revocable license to use, reproduce, and redistribute the Data in accordance with these Terms." Transit Data Terms of Use on the page linked above (undated; read 2026-11-10), which links the GTFS and GTFS-RT used here and covers "schedule, geographic, real-time, or any other data posted by King County". King County also permits users "to copy, distribute, store (electronically or otherwise), create derivative works based on and perform and display (publicly or otherwise) such Data, and make such Data available to end users", and asks for the legend the credit carries; no King County service marks beyond it. Commercial use is not mentioned. The feed bundles King County Metro, Sound Transit and City of Seattle routes; only the City of Seattle streetcars (agency 23) are kept, `100340` South Lake Union and `102638` First Hill (transit, tram). Link light rail is not in this GTFS. | "Trains (Transit scheduling, geographic, and real-time data provided by permission of King County): [King County Transit Data Terms of Use](https://kingcounty.gov/en/dept/metro/rider-tools/mobile-and-web-apps)" |
| **Connector streetcar** (tmgtfsprd.sorttrpcloud.com, www.go-metro.com) | Rail layer trains, Cincinnati, OH: Southwest Ohio Regional Transit Authority (SORTA/Metro). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (tmgtfsprd.sorttrpcloud.com) add delays at its stops. | [SORTA/Metro developer data Terms](https://www.go-metro.com/about/developer-data/): "The Southwest Ohio Regional Transit Authority (SORTA/Metro) hereby grants you (Licensee) non-exclusive, limited and revocable rights to use, reproduce, and redistribute SORTA/Metro’s transit route schedules, pricing, and other associated data (“Data”) subject to the following Terms:". The developer data page linked above (read 2026-11-10), which links the GTFS and these GTFS-RT files. SORTA/Metro trademarks "may not be used in association with Data unless approved in writing by SORTA/Metro", so the credit is text only; "Any modification, alteration, combination or arrangement of the Data is solely at your own risk." Commercial use is not mentioned. Seen 2026-11-10: 2 streetcars among 147 vehicles, 6 streetcar trips among 265 updates, trip ids those of the static file. The file of 2026-10-01 ends on 2026-12-05. One rail route, `100` Streetcar - OTR - Banks (route_type 0): transit, tram. | "Trains (Southwest Ohio Regional Transit Authority (SORTA/Metro)): [SORTA/Metro developer data Terms](https://www.go-metro.com/about/developer-data/)" |
| **Manyōsen** (gtfs-rt-files.buscatch.jp, api.gtfs-data.jp) | Rail layer trains, Toyama, Japan: 万葉線 Manyōsen (Takaoka–Kōshinokata tram). Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfs-rt-files.buscatch.jp) add delays at its stops. | [CC0 1.0](https://api.gtfs-data.jp/v2/organizations/manyosen/feeds/manyosen): ""feed_license": "CC0 1.0"". GTFS Data Repository record of the feed "万葉線", read 2026-11-10: the record lists the GTFS-RT VehiclePositions and TripUpdates (buscatch.jp, keyless) with the CC0 feed. Tram: transit. | "Trains (万葉線株式会社, via the GTFS Data Repository (gtfs-data.jp)): [CC0 1.0](https://api.gtfs-data.jp/v2/organizations/manyosen/feeds/manyosen)" |
| **Toyama tram** (gtfs-rt-files.buscatch.jp) | Rail layer trains, Toyama, Japan: 富山地方鉄道 Toyama Chihō Railway city tram (city lines, Toyamakō Line, loop line). Live GPS positions. No static GTFS is registered, so there are no path or schedule packs, and its trip updates are not used. | [CC0 1.0](https://api.gtfs-data.jp/v2/organizations/chitetsu/feeds/chitetsushinaidensha): ""feed_license": "CC0 1.0"". GTFS Data Repository record of the feed "富山地方鉄道市内電車", read 2026-11-10: the record lists the GTFS-RT VehiclePositions and TripUpdates (buscatch.jp, keyless) with the CC0 feed. The tram-only feed (the company's buses are a separate feed). Tram: transit. | "Trains (富山地方鉄道株式会社, via the GTFS Data Repository (gtfs-data.jp)): [CC0 1.0](https://api.gtfs-data.jp/v2/organizations/chitetsu/feeds/chitetsushinaidensha)" |
| **Bordeaux tram** (bdx.mecatran.com) | Rail layer trains, Bordeaux, France: TBM (Bordeaux Métropole): trams A to F. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (bdx.mecatran.com) add delays at its stops. | [Licence Ouverte / Open Licence 1.0](https://transport.data.gouv.fr/datasets/offres-de-services-bus-tram-et-scolaire-au-format-gtfs-netex-gtfs-rt-siri-lite): "fr-lo". Licence field of the transport.data.gouv.fr dataset "Réseau urbain et scolaire TBM" (publisher Bordeaux Métropole; read through its API on 2026-11-10), covering its GTFS and its GTFS-RT VehiclePositions and TripUpdates; Licence Ouverte 1.0 (Etalab) allows reuse, commercial included, with attribution. Tram: transit, tram. | "Trains (Bordeaux Métropole (TBM)): [Licence Ouverte / Open Licence 1.0](https://transport.data.gouv.fr/datasets/offres-de-services-bus-tram-et-scolaire-au-format-gtfs-netex-gtfs-rt-siri-lite)" |
| **Montpellier tram** (gtfsproxy.e-tam.fr) | Rail layer trains, Montpellier, France: TaM (Montpellier Méditerranée Métropole): trams 1 to 5. Live GPS positions. Optional path and schedule packs built from its static GTFS add its lines, stop names and next stops. With a built schedule pack, trip updates (gtfsproxy.e-tam.fr) add delays at its stops. | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/): "odc-odbl". Licence field of the transport.data.gouv.fr dataset "Réseau urbain TaM" (publisher Montpellier Méditerranée Métropole; read through its API on 2026-11-10), covering its GTFS and its GTFS-RT VehiclePositions and TripUpdates. ODbL is share-alike, with the access point's special conditions (doc.transport.data.gouv.fr "Licence ODbL", read 2026-11-10): the share-alike clause covers information "de même nature, de même granularité, de mêmes conditions temporelles et de même emprise géographique", and "Les bases de données ainsi repartagées doivent être publiées sur le Point d'accès national transport.data.gouv.fr, dans le format d'origine." The path and schedule packs built from this GTFS are such a database once shared: offered under ODbL with the attribution kept. Tram: transit, tram. | "Trains (TaM / Montpellier Méditerranée Métropole): [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/)" |
| **ZDiTM Szczecin** (zditm.szczecin.pl) | Transit layer live vehicles, Szczecin, Poland: buses and trams as one feed | CC0 1.0, stated on the ZDiTM developer page: “The data are provided free of charge under the CC0 1.0 licence, for use in any context, with no prior notification required.” Keyless GTFS-Realtime; the page says one update a minute, measured at about every 20 seconds. The feed carries no route id, so every vehicle takes the default bus mode. | None required (CC0); credited to ZDiTM Szczecin as a courtesy |
| **Carris Metropolitana** (api.carrismetropolitana.pt) | Transit layer live vehicles, Lisbon Metropolitan Area, Portugal | Keyless GTFS-Realtime documented in the operator’s public API repository (github.com/carrismetropolitana/api). No licence is stated for the realtime endpoint; the operator’s static GTFS is CC BY 4.0. Included on the same basis as the Metro Transit and OVapi feeds and credited as a courtesy; it can be switched off if Carris objects. Speeds are published in km/h and converted to m/s by the proxy. Route ids are read as the line number per the operator’s `/v2/routes`. | Courtesy credit to Carris Metropolitana |
| **Prague PID** (api.golemio.cz) | Transit layer live vehicles, Prague and Central Bohemia, Czechia | CC BY (https://pid.cz/o-systemu/opendata/). Stated: “Data, která lze stáhnout přímo zde z webu, jsou stejně jako celý obsah webu opatřena licencí CC-BY” The PID open-data page lists this realtime feed and licenses the data it publishes under CC BY, crediting the author and any changes. The same page says data served from other sources follows that source’s licence; the feed is served by Golemio, the city’s data platform, which states none of its own. Keyless since the catalogue moved it off the X-Access-Token endpoint. Speeds are m/s, checked against movement on 2026-11-07. Route ids are `L` plus a number whose range gives the mode and the line (metro `L991` is A, train `L1004` is S4, ferry `L1801` is P1), read from PID’s GTFS routes.txt. | Pražská integrovaná doprava (PID) / Golemio |
@@ -208,7 +239,8 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
- **adsbdb enrichment.** `/api/adsbdb` resolves an ICAO hex to aircraft type, model name and registration, and the tracked flight's callsign to an airline and an origin/destination pair. Fan-out is deliberately small because adsbdb is a free community API that publishes no quota: at most 4 requests in flight, dispatches dripped to roughly 5/s, and each key asked for at most once per session (for the flights layer's enrichment, a lookup that failed temporarily, such as an outage, a timeout, a rate limit or an authorization refusal that may clear, may be asked again after a 60-second cool-down; an answer, found or not, and a permanent refusal such as a 404 are final; Local ADS-B lookups are asked once), with route lookups reserved for the tracked aircraft. The dev-server proxy caches each key for 24 hours in `.gev-cache/adsbdb.json`, negative answers included, so fresh entries survive server restarts. That gitignored cache remains local to the deployment and is not bundled with this repo. adsbdb publishes no general service license; its README says the route data may not be copied, published, or incorporated into other databases without the explicit permission of David J Taylor, Edinburgh. Deployers are responsible for obtaining any permission their use requires. Enrichment is best-effort and fail-silent — a flight that gets no answer keeps what the flight feed already gave it.
- **Planespotters photos.** When you track an aircraft, the browser asks `https://api.planespotters.net/pub/photos/hex/{hex}` directly (the API answers cross-origin, so no key and no server proxy are involved) for that aircraft only, once per hex per session, through the same enrichment queue as adsbdb, with an 8-second timeout. A miss is remembered for the session; a request that failed temporarily (an outage, the timeout or a rate limit) may be asked again after a 60-second cool-down while the aircraft stays tracked, as for the flights layer's other enrichment. Either way no photo shows meanwhile. The first photo whose image is an `https://` URL on `*.plnspttrs.net`, and whose page is an `https://www.planespotters.net` URL is shown in the CONTEXT panel's CONTACTS view, under the tracked flight's name (`src/ui/views/panels/AwarenessPanel.vue`), and nowhere else; a vessel, an installation, a military aircraft or a converted TR-3B has none, and a military aircraft or TR-3B is never looked up. Anything else is dropped, so the app never shows or links outside Planespotters (`src/data/planespottersPhoto.js`, also exported as `gods-eye-view/sources/planespotters` for servers that cache the lookup). The photo follows the attribution rules: the thumbnail is hotlinked from the URL the API gives (never copied or re-hosted), the photographer's name, when the answer gives one, is visible text next to it ("Photo © {photographer}"; a photo whose answer names no one shows no credit line), and the photo and credit are a plain link to the photo's Planespotters page with `target="_blank" rel="noopener"` and never `rel="nofollow"`. If the image fails to load, the photo and its credit are hidden.
- **Planespotters photos.** When you track an aircraft, the browser asks `https://api.planespotters.net/pub/photos/hex/{hex}` directly (the API answers cross-origin, so no key and no server proxy are involved) for that aircraft only, once per hex per session, through the same enrichment queue as adsbdb, with an 8-second timeout. A miss is remembered for the session; a request that failed temporarily (an outage, the timeout or a rate limit) may be asked again after a 60-second cool-down while the aircraft stays tracked, as for the flights layer's other enrichment. Either way no photo shows meanwhile. The first photo whose image is an `https://` URL on `*.plnspttrs.net`, and whose page is an `https://www.planespotters.net` URL is shown in the CONTEXT panel's CONTACTS view, under the tracked flight's name (`src/ui/views/panels/AwarenessPanel.vue`), and nowhere else; a vessel (see Open Vessel Photos below), an installation, a military aircraft or a converted TR-3B has none, and a military aircraft or TR-3B is never looked up. Anything else is dropped, so the app never shows or links outside Planespotters (`src/data/planespottersPhoto.js`, also exported as `gods-eye-view/sources/planespotters` for servers that cache the lookup). The photo follows the attribution rules: the thumbnail is hotlinked from the URL the API gives (never copied or re-hosted), the photographer's name, when the answer gives one, is visible text next to it ("Photo © {photographer}"; a photo whose answer names no one shows no credit line), and the photo and credit are a plain link to the photo's Planespotters page with `target="_blank" rel="noopener"` and never `rel="nofollow"`. If the image fails to load, the photo and its credit are hidden.
- **Open Vessel Photos.** When you select a vessel with a nine-digit MMSI, the browser asks `https://openvesselphotos.org/v1/mmsi/{mmsi}` directly (the API is keyless and answers cross-origin, so no key and no server proxy are involved; account credentials are never sent) for that vessel only, once per MMSI per session, with an 8-second timeout. A miss (a 404 or a vessel without photos) is remembered for the session; a failed request may be asked again after five minutes. The first photo whose image is an `https://openvesselphotos.org` URL and which names a licence is shown, at the 800 px size, in the CONTEXT panel's CONTACTS view under the vessel's name and MMSI (`src/ui/views/panels/AwarenessPanel.vue`), and nowhere else (`src/data/openVesselPhoto.js`, `src/layers/vessels/photos.js`). The image is hotlinked from the URL the API gives (never copied or re-hosted); the photographer and licence are visible text next to it ("Photo: {photographer} · {licence}", or "Photo · {licence}" when the answer names no one), and the photo and credit link to the vessel's Open Vessel Photos page with `target="_blank" rel="noopener"`. If the image fails to load, the photo and its credit are hidden.
- **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state.
- **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover.
- **Ontario 511 CCTV.** The camera list comes from the `511on.ca/api/v2/get/cameras` endpoint, which now requires a free developer key (`ONTARIO_511_API_KEY`: sign up at 511on.ca/my511/register, then request it while logged in at 511on.ca/developers/doc; throttled to 10 calls a minute). Without the key the pack is skipped. Frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence - Ontario. Access is governed by Ontario 511's Developer's Access Agreement (511on.ca/developers/resources): the data stays under the Open Government Licence - Ontario unless otherwise specified; key requests are reviewed case by case and ask for the intended use and any third parties that will receive the data; attribution by link to 511on.ca is encouraged, and the Ontario 511 logo may not be altered or used to imply endorsement (this app links and shows no logo); and the data is provided "as is", a disclaimer distributors should pass on, which the credit does.
@@ -237,6 +269,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
- **Autostrada del Brennero (A22) webcams.** Keyless. The camera list is one request to the Open Data Hub tourism API (`tourism.api.opendatahub.com/v1/WebcamInfo?source=a22`), run by NOI Techpark in South Tyrol, which republishes Autobrennero's own list of 13 motorway webcams with coordinates and kilometre markers. Frames are the operator's stills on `www.autobrennero.it/WebCamImg/km<N>.jpg`, refreshed about once a minute, and only the exact image for each camera's kilometre is registered. The feed names no bearing, so headings use the id-hash fallback at low confidence. The road climbs from about 20 m in the Po valley to about 1,370 m at the Brenner Pass, so each camera's ground elevation comes from a per-kilometre table sampled from the Copernicus GLO-90 DEM (via Open-Meteo's elevation API); a camera not in the table uses a 200 m prior. The Open Data Hub tags every record CC0 (`ClosedData: false`, Autobrennero as licence holder), but that tag is the aggregator's metadata and Autobrennero publishes no terms of its own for the images, so they are credited to the operator as a courtesy and are not redistributed. Frames are fetched at request time and never stored. Disable with `CCTV_A22_ENABLED=0`.
- **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals.
- **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All fifty-two registered feeds, covering forty-nine regions, are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. The TTC feed is BusTime, the surface system: it carries buses and streetcars and no subway at all. About four in ten of its vehicles report no route at all (unassigned or deadheading), and those are shown with the feed default rather than a route-established mode. It sends no ETag or Last-Modified, so its conditional requests fall through to the ordinary snapshot cache. Three Golden Horseshoe neighbours run alongside it — MiWay, Hamilton HSR and Durham Region Transit — each bus-only and each polled only near its own region. Durham Region Transit publishes no bearing on any vehicle, so those sprites take their heading from observed movement like any other feed that omits it. Guelph, Kingston and Barrie extend the same pattern eastward and north. Hamilton, Guelph, Burlington, YRT and Rio’s SMTR publish route ids that are not the number on the vehicle, so those entries carry a route-name table taken from each operator’s own published GTFS; any id missing from a table renders unchanged rather than blank. A label is never invented: YRT runs a route `9101` that appears nowhere in its published routes, so it stays as the feed sends it. Burlington’s is the one that cannot be shortcut: `351` is route 1 while `3510` is route 10, so the table is a real join rather than a trimmed prefix. GO Transit / Metrolinx is deliberately absent: its REALTIME feed requires an API key issued by manual review, which the keyless admission rule excludes (its static schedule download is keyless, but carries no vehicle positions). Barrie and YRT/Viva each invite acceptance of their licence through a form, and each licence also states that using the data is itself acceptance; this deployment relies on that clause, which is recorded in both entries. Grand River Transit in Waterloo is absent for a different reason: it answers, but its feed header was months stale and it publishes no per-vehicle timestamps, so it would render vehicles that look live and are not. Ottawa’s OC Transpo falls to the same keyless admission rule as GO Transit — its realtime feed requires an API key. London Transit and Thunder Bay Transit publish vehicle positions over plain HTTP with no HTTPS endpoint available. Every other Ontario system — Brampton, Oakville, Niagara, Milton, Windsor, Greater Sudbury, UP Express and a dozen smaller operators — publishes static GTFS schedules only, with no realtime vehicle positions to register. Between the nine included and these exclusions, every Ontario realtime vehicle-position feed catalogued in the Mobility Database is accounted for.
- **Rail (GTFS-Realtime trains).** Each feed in `src/data/railFeeds.js` records its licence, its credit and the passage of its terms the decision rests on, and a feed is switched on only when those terms were read and cover this use, or, for those that publish no licence at all, on the basis its entry records. Such a basis is never a licence, and the credit does not present it as one: its label says there is no licence ("No licence published", "No licence stated") or names the publication the use rests on. Five enabled feeds publish no licence for their train positions. MARC is used on what its publication says (open publication with a no-endorsement disclaimer), and METRO light rail on the same basis as the Transit layer's Metro Transit feed. Tri-Rail and Charlotte's LYNX are used without a published licence by project owner decision of 2026-11-10. The fifth is Irish Rail, whose train positions publish no licence while its NTA timetable is CC BY 4.0. The registry lists 30 feeds, all switched on, and every one is keyless: its realtime feed and its static timetable are published for anyone to fetch, so no feed needs a key, an account or a registration, and the server holds no rail credential. No rail request carries a credential of the deployer's; one feed's published URL includes a fixed public value its publisher prints for everyone (Bordeaux's `apiKey`). `/api/rail/feeds` lists the offered feeds, `/api/rail/trains/<feedId>` returns one feed's train snapshot, `/api/rail/updates/<feedId>` its trip updates, and `/api/rail/paths/<feedId>` and `/api/rail/schedule/<feedId>` its path and schedule packs, when built. The proxy requests only registered feeds and reuses the Transit proxy's request policy (a short snapshot cache with single-flight refresh, serve-stale, admission limits, the failure cooldown ladder, redirects followed hop by hop only on the feed's own https origin, and an 8 MiB body cap unless a feed sets its own, at most 64 MiB). Many operators publish trains in the same feed as their buses, so each feed has a classifier: the proxy keeps only vehicles it recognises as trains, tags each with a category (`transit`, `local`, `regional`, `national` or `international`) and a mode (`rail`, `metro` or `tram`), and drops everything else server-side, counted in the snapshot as `dropped`; the browser never receives a bus. Every train shown is live, where its operator last reported it: 29 feeds publish live GPS positions (SJ Nord's through SIRI VM), and Irish Rail reports the station or timing point a train last passed. No train is placed or moved from a timetable; a feed whose live positions fail shows as unavailable, and its trains age out as any quiet train does. 22 feeds register trip updates, polled while one of their trains is selected and that feed's schedule pack is loaded. 28 feeds work out of the box. Packs are an opt-in extra for live trains: nothing is downloaded when the app starts, and `node scripts/build-rail-paths.mjs --feed <id>` (repeatable) or `--all` downloads a feed's static GTFS (29 of the 30 have one) and writes a path pack (the operator's track shapes) and a schedule pack (the stop times for a window of days) to `.cache/rail-paths/` (or to `RAIL_PATHS_DIR` when set, the directory the server then serves). With them, a live train is drawn on its operator's route line and its card names its stops, its next stop and, from the trip updates, their delays. Renfe long and medium distance and KTMB, whose vehicles name no route, are classified from their pack's routes, so `/api/rail/feeds` lists them only once that pack is built. Lines are the operator's own shapes as published: nothing is routed or redrawn from other map data, and no line is drawn where an operator publishes no shape. **Share-alike sources bind the deployer.** A pack built from a ShareAlike or ODbL source is a derived database of it, so a deployer who serves it publicly makes it subject to the same licence and must offer it under that licence with the attribution kept. That covers the Montpellier packs (ODbL). For the ODbL datasets of France's national access point, its special conditions also require that "Les bases de données ainsi repartagées doivent être publiées sur le Point d'accès national transport.data.gouv.fr, dans le format d'origine" ([Licence ODbL](https://doc.transport.data.gouv.fr/le-point-d-acces-national/cadre-juridique/conditions-dutilisation-des-donnees/licence-odbl), read 2026-11-10): a deployer who serves the Montpellier packs publicly takes on republishing that derived data there, as GTFS. The build downloads each static GTFS itself, over https and with no credential. A feed's `staticRouteTypes` lists the route types it keeps when its classifier accepts every route of a mode-specific feed. A frequency-based trip becomes one run per departure on its template's stops and run times. Packs are build output, not part of the repository, and cover a fixed window of days, so they are rebuilt before it ends, anLine truncated
- **Radio country coverage (optional).** Set `RADIO_PRIORITY_COUNTRIES=IN,PL` in the local `.env` and restart to include stations beyond the global click charts. The operator may choose up to eight unique ISO alpha-2 country codes; no key is needed. Each country adds one bounded query (at most 200 rows) and reserves up to 40 unique, usable stations inside the existing 750-station catalog. Country seeds precede specialist-tag seeds and global popularity fill; unused slots return to the shared pool. This is a coverage preference, not an exhaustive national directory or a guarantee of stations in every city. Countries with no eligible HTTPS MP3/AAC geolocated stations remain empty. An unset list preserves global selection. Country requests share the directory cache and three-worker pool; failed country queries are reported in coverage without blocking a healthy global refresh. Failed/degraded refreshes back off for one minute, with at most three mirrors per query.
- **Radio Browser.** In every installation, including when `RADIO_PRIORITY_COUNTRIES` is unset, each request tries at most three mirror candidates (later fallback candidates are skipped), and failed or partial catalog refreshes back off for 60 seconds. A cold-start failure therefore continues returning 503 during that delay even if upstream recovers. `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; broad/specialist responses whose rows all fail the product's health policy do not count as successful queries. Optional priority-country outcomes are advisory: valid, in-limit responses with no eligible stations succeed as empty results, while failed or malformed country queries report unsuccessful coverage without degrading the global catalog. A usable partial cold catalog is explicitly `DEGRADED`, and non-array or empty broad/specialist payloads are rejected atomically. Individual structurally malformed rows are skipped rather than failing the whole query; the response carries their bounded `rejectedStationCount` (at most 10,000), which the client adds to its own rejected rows and reports as degraded coverage without discarding the accepted generation. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply.
@@ -257,6 +290,113 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
---
<!-- data-library:start (generated by npm run library:docs) -->
## Data Library entries (fetched at runtime — not stored in this repo)
Each Data Library entry is a manifest in `src/library/entries/` pointing at a live upstream its provider hosts; see [docs/DATA-LIBRARY.md](docs/DATA-LIBRARY.md). An entry appears on the map only after a viewer adds it from **DATA LAYERS ▸ Browse data library**, and its attribution is registered in the Data attribution popover while it is in use. This table is generated from the manifests: edit the manifest, then run `npm run library:docs`.
| Entry | How it is served | Licence / terms | Attribution |
| --- | --- | --- | --- |
| **Alaska Statewide Imagery (2023)** (`ak-statewide-imagery`, `apps.geo.fpac.usda.gov`) | tiles, bbox, browser-direct | [Public Domain (U.S. Government work)](https://www.usgs.gov/information-policies-and-instructions/copyrights-and-credits) | "USDA / Alaska Geospatial Council" |
| **Amsterdam traffic and ANPR camera positions** (`amsterdam-traffic-cameras`, `api.data.amsterdam.nl`) | dataset, geojson, browser-direct; operator adds `amsterdam-traffic-cameras` to `GEV_LIBRARY_ENABLE` | [Gemeente Amsterdam open data: public unless stated otherwise; no named licence](https://schemas.data.amsterdam.nl/datasets/verkeersinformatiesystemen/dataset) | "Gemeente Amsterdam, verkeersinformatiesystemen register (entry contributed by @Maher-Reven)" |
| **Arkansas Statewide Imagery (2023)** (`ar-statewide-imagery`, `gis.arkansas.gov`) | tiles, bbox, browser-direct | [Public domain — Arkansas GIS Office open data](https://gis.arkansas.gov/) | "Arkansas GIS Office" |
| **Arizona Statewide Imagery (2017)** (`az-statewide-imagery`, `azgeo.az.gov`) | tiles, xyz, browser-direct; operator adds `az-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://azgeo.az.gov/) | "AZGEO" |
| **Digitales Orthophoto Bayern (DOP 40 cm, open)** (`bayern-dop40`, `geoservices.bayern.de`) | tiles, bbox, browser-direct | [CC BY 4.0](https://geodaten.bayern.de/opengeodata/) | "© Bayerische Vermessungsverwaltung" |
| **Bismarck–Mandan, North Dakota Aerial Photography** (`bismarck-mandan-nd`, `ndgishub.nd.gov`) | tiles, bbox, browser-direct; operator adds `bismarck-mandan-nd` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.gis.nd.gov/) | "95West Aerial Mapping" |
| **TopPlusOpen basemap (BKG)** (`bkg-topplus-open`, `sgx.geodatenzentrum.de`) | tiles, xyz, browser-direct | [Datenlizenz Deutschland – Namensnennung 2.0](https://www.govdata.de/dl-de/by-2-0) | "© BKG, dl-de/by-2-0" |
| **Florida cameras (FL511)** (`cctv-florida-fl511`, `fl511.com`) | CCTV pack; operator sets `CCTV_FLORIDA_ENABLED=1` | [FL511 content: individual personal use only](https://fl511.com/privacy#disclaimer) | "Florida DOT — FL511.com" |
| **Lithuania road cameras (Via Lietuva)** (`cctv-lithuania`, `eismoinfo.lt`) | CCTV pack; operator sets `CCTV_LITHUANIA_ENABLED=1` | [Via Lietuva open-data statement; camera photo coverage unconfirmed](https://eismoinfo.lt/eismoinfo-backend/content?id=44) | "Via Lietuva / eismoinfo.lt" |
| **New York City cameras (NYC DOT TMC)** (`cctv-nyc-dot`, `webcams.nyctmc.org`) | CCTV pack; operator sets `CCTV_NYC_ENABLED=1` | [NYC DOT camera feed: data-sharing agreement required](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml) | "NYC DOT Traffic Management Center" |
| **Connecticut Statewide Imagery (2019)** (`ct-statewide-imagery`, `cteco.uconn.edu`) | tiles, bbox, browser-direct; operator adds `ct-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://ct-deep-gis-open-data-website-ctdeep.hub.arcgis.com/) | "CT ECO — UConn CLEAR / CT DEEP" |
| **Dakota County, Minnesota Air Photo** (`dakota-county-mn`, `gisimg.co.dakota.mn.us`) | tiles, bbox, browser-direct; operator adds `dakota-county-mn` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://gis.co.dakota.mn.us/) | "Dakota County GIS" |
| **District of Columbia Statewide Imagery (2025)** (`dc-statewide-imagery`, `imagery.dcgis.dc.gov`) | tiles, bbox, browser-direct | [Open data — DC public domain terms](https://opendata.dc.gov/pages/terms-of-use) | "Office of the Chief Technology Officer" |
| **Delaware Statewide Imagery (2024)** (`de-statewide-imagery`, `imagery.firstmap.delaware.gov`) | tiles, bbox, browser-direct; operator adds `de-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://firstmap.delaware.gov/) | "This data is meant for use by requested parties. This data cannot be used without express consent.This data is meant for use by requested parties. This data cannot be used without express consent." |
| **Eesti ortofoto (Maa-amet)** (`ee-maa-amet-ortofoto`, `tiles.maaamet.ee`) | tiles, xyz, browser-direct | [Maa-amet open data licence (attribution)](https://geoportaal.maaamet.ee/eng/Spatial-Data/Orthophotos-p309.html) | "© Maa-amet (Estonian Land Board)" |
| **US power plants, 100 MW and up (EIA)** (`eia-power-plants`, `services2.arcgis.com`) | dataset, arcgis, browser-direct | [U.S. Energy Information Administration data: public domain](https://www.eia.gov/about/copyrights_reuse.php) | "U.S. Energy Information Administration (EIA-860, EIA-860M, EIA-923)" |
| **Superfund sites (US EPA)** (`epa-superfund-sites`, `geopub.epa.gov`) | dataset, arcgis, browser-direct | [U.S. EPA data: public domain](https://www.epa.gov/web-policies-and-procedures/epa-disclaimers) | "U.S. Environmental Protection Agency — Facility Registry Service (EnviroMapper)" |
| **Toxic release facilities (US EPA TRI)** (`epa-toxic-release-facilities`, `geopub.epa.gov`) | dataset, arcgis, browser-direct | [U.S. EPA data: public domain](https://www.epa.gov/web-policies-and-procedures/epa-disclaimers) | "U.S. Environmental Protection Agency — Toxics Release Inventory via the Facility Registry Service" |
| **Home energy ratings, England and Wales (EPC)** (`epc-england-wales`, `s3.us-west-2.amazonaws.com`) | dataset, geoparquet, browser-direct | [Open Government Licence v3.0 (EPC data); OS Open UPRN under OGL](https://epc.opendatacommunities.org/docs/copyright) | "Energy Performance of Buildings Data, DLUHC; geocoded by Addresscloud with OS Open UPRN — contains OS data © Crown copyright and database right; via Source Cooperative" |
| **US building footprints (FEMA USA Structures)** (`fema-usa-structures`, `services2.arcgis.com`) | dataset, arcgis, browser-direct | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | "FEMA Response Geospatial Office and Oak Ridge National Laboratory — USA Structures" |
| **Biodiversity records (GBIF)** (`gbif-occurrence-density`, `api.gbif.org`) | tiles, xyz, browser-direct | [GBIF data use agreement (records under CC0, CC BY or CC BY-NC)](https://www.gbif.org/terms) | "GBIF.org — Global Biodiversity Information Facility" |
| **Disaster alerts (GDACS)** (`gdacs-disaster-alerts`, `www.gdacs.org`) | dataset, geojson, browser-direct | [GDACS terms of use (free, attribution required)](https://www.gdacs.org/About/termofuse.aspx) | "GDACS — Global Disaster Alert and Coordination System (UN, European Commission)" |
| **Grand Forks, North Dakota Aerial Photography** (`grand-forks-nd`, `ndgishub.nd.gov`) | tiles, bbox, browser-direct; operator adds `grand-forks-nd` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.gis.nd.gov/) | "Grand Forks/East Grand Forks Metropolitan Planning Organization (MPO)" |
| **Japan standard map (GSI)** (`gsi-japan-map`, `cyberjapandata.gsi.go.jp`) | tiles, xyz, browser-direct | [GSI Tiles terms (free use with attribution)](https://maps.gsi.go.jp/development/ichiran.html) | "Geospatial Information Authority of Japan (GSI Tiles)" |
| **Japan aerial photos (GSI)** (`gsi-japan-photo`, `cyberjapandata.gsi.go.jp`) | tiles, xyz, browser-direct | [GSI Tiles terms (free use with attribution)](https://maps.gsi.go.jp/development/ichiran.html) | "Geospatial Information Authority of Japan (GSI Tiles)" |
| **Holocene volcanoes (Smithsonian GVP)** (`gvp-holocene-volcanoes`, `webservices.volcano.si.edu`) | dataset, geojson, proxied | [Smithsonian Terms of Use (personal, educational, non-commercial use; cite the Global Volcanism Program)](https://www.si.edu/termsofuse) | "Global Volcanism Program, Smithsonian Institution — Volcanoes of the World" |
| **Iowa Statewide Imagery (2023)** (`ia-statewide-imagery`, `ortho.gis.iastate.edu`) | tiles, bbox, browser-direct; operator adds `ia-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://geodata.iowa.gov/) | "USDA NAIP, Iowa State University GIS Facility" |
| **Idaho Statewide Imagery (2023)** (`id-statewide-imagery`, `insideidaho.org`) | tiles, bbox, browser-direct | [Public domain / open — INSIDE Idaho terms](https://insideidaho.org/) | "United States Department of Agriculture, Farm Production and Conservation Business Center, Geospatial Enterprise Operations (FPAC-BC-GEO)" |
| **France aerial photos (IGN)** (`ign-orthophotos-france`, `data.geopf.fr`) | tiles, xyz, browser-direct | [Licence Ouverte / Etalab 2.0](https://www.etalab.gouv.fr/licence-ouverte-open-licence/) | "© IGN — Géoplateforme" |
| **Illinois Statewide Imagery** (`il-statewide-imagery`, `data.isgs.illinois.edu`) | tiles, bbox, browser-direct; operator adds `il-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://clearinghouse.isgs.illinois.edu/) | "Illinois State Geological Survey / IDOT" |
| **Maritime chokepoints (IMF PortWatch)** (`imf-portwatch-chokepoints`, `services9.arcgis.com`) | dataset, arcgis, browser-direct; operator adds `imf-portwatch-chokepoints` to `GEV_LIBRARY_ENABLE` | [IMF PortWatch terms; attribution required (reuse terms under review)](https://www.imf.org/external/terms.htm) | "IMF PortWatch (portwatch.imf.org)" |
| **Port disruptions (IMF PortWatch)** (`imf-portwatch-disruptions`, `services9.arcgis.com`) | dataset, arcgis, browser-direct; operator adds `imf-portwatch-disruptions` to `GEV_LIBRARY_ENABLE` | [IMF PortWatch terms; attribution required (reuse terms under review)](https://www.imf.org/external/terms.htm) | "IMF PortWatch (portwatch.imf.org), with GDACS event data" |
| **Indiana Statewide Imagery (2016)** (`in-statewide-imagery`, `di-ingov.img.arcgis.com`) | tiles, bbox, browser-direct | [CC0 1.0 Universal (public domain dedication)](https://creativecommons.org/publicdomain/zero/1.0/legalcode) | "Indiana Geographic Information Office (CC0)" |
| **Surface water occurrence (JRC)** (`jrc-surface-water`, `storage.googleapis.com`) | tiles, xyz, browser-direct | [Copernicus programme: free, without restriction of use](https://global-surface-water.appspot.com/download) | "Source: EC JRC/Google — Pekel et al. (2016), Global Surface Water" |
| **Norway counties and municipalities (Kartverket)** (`kartverket-admin-units`, `wms.geonorge.no`) | tiles, bbox, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway cadastral map (Kartverket)** (`kartverket-cadastre`, `wms.geonorge.no`) | tiles, bbox, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway sea depths (Kartverket)** (`kartverket-depth`, `wms.geonorge.no`) | tiles, bbox, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway hillshade (Kartverket)** (`kartverket-hillshade`, `wms.geonorge.no`) | tiles, bbox, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway nautical charts (Kartverket)** (`kartverket-nautical-charts`, `cache.kartverket.no`) | tiles, xyz, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway terrain relief (Kartverket DTM)** (`kartverket-terrain-relief`, `wms.geonorge.no`) | tiles, bbox, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway topographic map (Kartverket)** (`kartverket-topo`, `cache.kartverket.no`) | tiles, xyz, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway topographic map, greyscale (Kartverket)** (`kartverket-topo-grey`, `cache.kartverket.no`) | tiles, xyz, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Norway topographic raster map (Kartverket)** (`kartverket-toporaster`, `cache.kartverket.no`) | tiles, xyz, browser-direct | [CC BY 4.0 (Kartverket free products)](https://www.kartverket.no/api-og-data/vilkar-for-bruk) | "© Kartverket" |
| **Kansas Statewide Imagery (2021)** (`ks-statewide-imagery`, `services.kgs.ku.edu`) | tiles, bbox, browser-direct; operator adds `ks-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://www.kansasgis.org/) | "FSA, USDA," |
| **Kentucky Statewide Imagery** (`ky-statewide-imagery`, `kyraster.ky.gov`) | tiles, bbox, browser-direct | [Public domain with attribution](https://kygeonet.ky.gov/) | "KyFromAbove" |
| **Lake County, Indiana Aerial Imagery** (`lake-county-in`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `lake-county-in` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.lakecountyin.org/) | "Lake County Surveyor's Office" |
| **Lawrence, Kansas Aerial Photos** (`lawrence-ks`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `lawrence-ks` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://gis-lawrenceks.opendata.arcgis.com/) | "City of Lawrence GIS" |
| **Lee County, Florida Aerial Imagery** (`lee-county-fl`, `gisimageserver.leegov.com`) | tiles, bbox, browser-direct; operator adds `lee-county-fl` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.leegov.com/gis) | "Eagle View, Lee County Property Appraiser, Lee County GIS" |
| **Loudoun County, Virginia Imagery** (`loudoun-county-va`, `logis.loudoun.gov`) | tiles, bbox, browser-direct; operator adds `loudoun-county-va` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://geohub-loudoungis.opendata.arcgis.com/) | "Loudoun County GIS" |
| **Louisville / LOJIC, Kentucky Imagery** (`louisville-lojic-ky`, `services.lojic.org`) | tiles, bbox, browser-direct; operator adds `louisville-lojic-ky` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.lojic.org/) | "LOJIC, USDA-NAIP" |
| **Massachusetts Statewide Imagery** (`ma-statewide-imagery`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `ma-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://gis.data.mass.gov/) | "MassGIS, Commonwealth of Massachusetts" |
| **Geologic map (Macrostrat)** (`macrostrat-geology`, `tiles.macrostrat.org`) | tiles, xyz, browser-direct | [CC BY 4.0 (Macrostrat; source maps credited by their publishers)](https://macrostrat.org/about) | "Macrostrat — geologic maps from their original publishers" |
| **Global terrain shaded relief (Mapterhorn)** (`mapterhorn-terrain`, `tiles.mapterhorn.com`) | tiles, xyz, browser-direct | [Open data, per source (mostly CC BY 4.0 and public domain)](https://mapterhorn.com/attribution) | "© Mapterhorn and its open elevation sources" |
| **MARIS Imagery, Mississippi** (`maris-ms`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `maris-ms` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.gis.ms.gov/) | "MARIS" |
| **Maryland Statewide Imagery** (`md-statewide-imagery`, `mdgeodata.md.gov`) | tiles, bbox, browser-direct; operator adds `md-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://imap.maryland.gov/) | "Maryland iMAP" |
| **Maine Statewide Imagery** (`me-statewide-imagery`, `gis.maine.gov`) | tiles, bbox, browser-direct; operator adds `me-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://www.maine.gov/geolib/) | "USDA Aerial Photography Field Office" |
| **Michigan Statewide Imagery (2024)** (`mi-statewide-imagery`, `imagery.michigan.gov`) | tiles, bbox, browser-direct; operator adds `mi-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://gis-michigan.opendata.arcgis.com/) | "State of Michigan" |
| **Mountain Village, Colorado Orthoimagery** (`mountain-village-co`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `mountain-village-co` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.townofmountainvillage.com/) | "Town of Mountain Village" |
| **Mississippi Statewide Imagery** (`ms-statewide-imagery`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `ms-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://www.gis.ms.gov/) | "USDA-FSA-APFO" |
| **Montana Statewide Imagery (2023)** (`mt-statewide-imagery`, `gisservice.mt.gov`) | tiles, bbox, browser-direct | [Public domain — Montana State Library open data](https://msl.mt.gov/geoinfo/) | "Montana State Library" |
| **NAIP — National Agriculture Imagery Program** (`naip`, `imagery.nationalmap.gov`) | tiles, bbox, browser-direct | [Public Domain (U.S. Government work)](https://naip-usdaonline.hub.arcgis.com/) | "USDA NAIP" |
| **Earth at night (NASA Black Marble 2016)** (`nasa-black-marble`, `gibs.earthdata.nasa.gov`) | tiles, xyz, browser-direct | [NASA data: U.S. public domain; no endorsement implied](https://www.earthdata.nasa.gov/engage/open-data-services-software-policies) | "NASA Earth Observatory / GIBS — VIIRS Black Marble" |
| **NASA EONET open natural events** (`nasa-eonet-events`, `eonet.gsfc.nasa.gov`) | dataset, geojson, browser-direct | [NASA data: U.S. public domain; no endorsement implied](https://www.nasa.gov/nasa-brand-center/images-and-media/) | "NASA Earth Observatory Natural Event Tracker (EONET)" |
| **North Carolina Statewide Imagery (2023)** (`nc-statewide-imagery`, `services.nconemap.gov`) | tiles, bbox, browser-direct | [Public domain — NC OneMap terms of use](https://www.nconemap.gov/pages/terms) | "NC911, NCCGIA" |
| **North Central Texas (NCTCOG) Ortho** (`nctcog-tx`, `mapitimage.tarrantcounty.com`) | tiles, bbox, browser-direct; operator adds `nctcog-tx` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.nctcog.org/) | "North Central Texas Council of Governments" |
| **North Dakota Statewide Imagery (2013)** (`nd-statewide-imagery`, `ndgishub.nd.gov`) | tiles, xyz, browser-direct; operator adds `nd-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open state geodata — attribution requested](https://www.gis.nd.gov/) | "ND Dept. of Transportation, ND Dept. of Emergency Services, and the ND GIS Hub" |
| **New Orleans Regional Planning Commission Imagery** (`new-orleans-rpc-la`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `new-orleans-rpc-la` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.norpc.org/) | "Regional Planning Commission for Jefferson, Orleans, Plaquemines, St. Bernard, St. Charles, St. John the Baptist, St. Tammany, and Tangipahoa Parishes (RPC)" |
| **New Hampshire Statewide Imagery (2022)** (`nh-statewide-imagery`, `nhgeodata.unh.edu`) | tiles, bbox, browser-direct; operator adds `nh-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open — NH GRANIT terms](https://www.granit.unh.edu/) | "NH GRANIT, University of New Hampshire" |
| **New Jersey Statewide Imagery (2015)** (`nj-statewide-imagery`, `maps.nj.gov`) | tiles, xyz, browser-direct; operator adds `nj-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open — State of New Jersey provides data 'as is'](https://njgin.nj.gov/) | "Project definition, funding requests, and data distribution by NJ Office of Information Technology, Office of Geographic Information Systems (NJOIT - OGIS). Funding for the imagery project provided by" |
| **US tide stations (NOAA CO-OPS)** (`noaa-tide-stations`, `api.tidesandcurrents.noaa.gov`) | dataset, json, browser-direct | [NOAA data: U.S. public domain](https://tidesandcurrents.noaa.gov/disclaimers.html) | "NOAA Center for Operational Oceanographic Products and Services (CO-OPS)" |
| **Digitale Orthophotos NRW (DOP 10 cm)** (`nrw-dop`, `www.wms.nrw.de`) | tiles, bbox, browser-direct | [Datenlizenz Deutschland – Zero (public-domain-like)](https://www.govdata.de/dl-de/zero-2-0) | "Geobasis NRW (dl-de/zero-2-0)" |
| **NSW Imagery (Six Maps)** (`nsw-imagery`, `maps.six.nsw.gov.au`) | tiles, xyz, browser-direct | [CC BY 4.0](https://www.spatial.nsw.gov.au/products_and_services/web_services) | "© Spatial Services NSW" |
| **US bridges and their condition (NBI)** (`ntad-national-bridges`, `services.arcgis.com`) | dataset, arcgis, browser-direct | [U.S. government work: unrestricted public use](https://www.bts.gov/ntad) | "USDOT Bureau of Transportation Statistics — National Transportation Atlas Database, National Bridge Inventory" |
| **Northwest Arkansas (Benton & Washington County) Imagery** (`nwa-benton-washington-ar`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `nwa-benton-washington-ar` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://www.nwarpc.org/) | "NWARPC; Washington County; Benton County" |
| **US weather alerts with drawn areas (NWS)** (`nws-active-alerts`, `api.weather.gov`) | dataset, geojson, browser-direct | [NWS data: U.S. public domain](https://www.weather.gov/disclaimer) | "NOAA National Weather Service" |
| **New York Statewide Imagery (2019)** (`ny-statewide-imagery`, `orthos.its.ny.gov`) | tiles, xyz, browser-direct | [Public domain — NYS open data](https://data.gis.ny.gov/) | "USDA; NYS ITS Geospatial Services" |
| **Oakland County, Michigan Ortho Imagery** (`oakland-county-mi`, `gisservices.oakgov.com`) | tiles, bbox, browser-direct; operator adds `oakland-county-mi` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://oakgov.maps.arcgis.com/) | "Oakland County GIS" |
| **Ohio Statewide Imagery** (`oh-statewide-imagery`, `maps.ohio.gov`) | tiles, xyz, browser-direct | [Public domain — Ohio open data](https://maps.ohio.gov/) | "Ohio Geographically Referenced Information Program (OGRIP)" |
| **Oregon Statewide Imagery (2000)** (`or-statewide-imagery`, `imagery.oregonexplorer.info`) | tiles, bbox, browser-direct | [Public domain — Oregon open data](https://www.oregon.gov/geo/) | "Oregon, GEO, DAS" |
| **Pennsylvania Statewide Imagery** (`pa-statewide-imagery`, `imagery.pasda.psu.edu`) | tiles, bbox, browser-direct | [Public domain — PASDA open data](https://www.pasda.psu.edu/) | "PASDA / PAMAP, Commonwealth of Pennsylvania" |
| **Ortofotomapa Polski (GUGiK)** (`pl-ortofotomapa`, `mapy.geoportal.gov.pl`) | tiles, bbox, browser-direct | [Polish open geodata (free reuse, Prawo geodezyjne art. 40a)](https://www.geoportal.gov.pl/) | "© GUGiK / geoportal.gov.pl" |
| **PNOA Máxima Actualidad** (`pnoa-ma`, `www.ign.es`) | tiles, bbox, browser-direct | [CC BY 4.0 (compatible) — Orden FOM/2807/2015](https://www.scne.es/productos.html) | "PNOA © Instituto Geográfico Nacional de España" |
| **Portland Metro (RLIS), Oregon Aerial Photos** (`portland-metro-or`, `utility.arcgis.com`) | tiles, xyz, browser-direct; operator adds `portland-metro-or` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://rlisdiscovery.oregonmetro.gov/) | "Oregon Metro RLIS" |
| **Ortofotos de Portugal Continental (DGT)** (`pt-dgt-ortos`, `ortos.dgterritorio.gov.pt`) | tiles, bbox, browser-direct | [CC BY 4.0](https://www.dgterritorio.gov.pt/) | "© Direção-Geral do Território" |
| **Rhode Island Statewide Imagery (2019)** (`ri-statewide-imagery`, `tiles.arcgis.com`) | tiles, xyz, browser-direct; operator adds `ri-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open — RIGIS terms](https://www.rigis.org/) | "URI EDC, RIGIS" |
| **Switzerland aerial photos (SWISSIMAGE 10 cm)** (`swisstopo-swissimage`, `wmts.geo.admin.ch`) | tiles, xyz, browser-direct | [swisstopo free geodata (OGD): free use, including commercial, with source credit](https://www.swisstopo.admin.ch/en/terms-of-use-free-geodata-and-geoservices) | "©swisstopo — SWISSIMAGE 10 cm" |
| **Texas Statewide Imagery (2022)** (`tx-statewide-imagery`, `imagery.geographic.texas.gov`) | tiles, bbox, browser-direct | [Public domain — TxGIO open data](https://geographic.texas.gov/) | "Texas Geographic Information Office (TxGIO)" |
| **Union County, Ohio Aerial Imagery** (`union-county-oh`, `www7.co.union.oh.us`) | tiles, bbox, browser-direct; operator adds `union-county-oh` to `GEV_LIBRARY_ENABLE` | [Local government open data — see publisher terms](https://union-county-auditor-unionco.hub.arcgis.com/) | "Union County / City of Marysville GIS" |
| **US rivers and lakes (USGS)** (`usgs-hydrography`, `basemap.nationalmap.gov`) | tiles, xyz, browser-direct | [USGS The National Map: U.S. public domain](https://www.usgs.gov/information-policies-and-instructions/copyrights-and-credits) | "USGS The National Map — National Hydrography Dataset" |
| **US topographic map (USGS)** (`usgs-topo`, `basemap.nationalmap.gov`) | tiles, xyz, browser-direct | [USGS The National Map: U.S. public domain](https://www.usgs.gov/information-policies-and-instructions/copyrights-and-credits) | "USGS The National Map" |
| **Virginia Statewide Imagery** (`va-statewide-imagery`, `vginmaps.vdem.virginia.gov`) | tiles, xyz, browser-direct; operator adds `va-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open — VGIN VBMP terms](https://vgin.vdem.virginia.gov/) | "Virginia Geographic Information Network (VGIN)" |
| **Vermont Statewide Imagery** (`vt-statewide-imagery`, `maps.vcgi.vermont.gov`) | tiles, bbox, browser-direct | [Public domain — Vermont open data](https://vcgi.vermont.gov/) | "Vermont Center for Geographic Information (VCGI)" |
| **Orthophotos de Wallonie** (`wallonie-ortho`, `geoservices.wallonie.be`) | tiles, bbox, browser-direct | [Conditions ouvertes SPW (attribution)](https://geoportail.wallonie.be/catalogue-donnees) | "© Service public de Wallonie" |
| **Wisconsin Statewide Imagery (2020)** (`wi-statewide-imagery`, `di-wisdot.img.arcgis.com`) | tiles, bbox, browser-direct; operator adds `wi-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open — WisconsinView / state open data](https://www.wisconsinview.org/) | "WisDOT - NAIP 2020" |
| **Global power plants (WRI)** (`wri-global-power-plants`, `raw.githubusercontent.com`) | dataset, csv, browser-direct | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | "World Resources Institute, Global Power Plant Database (2021)" |
| **West Virginia Statewide Imagery** (`wv-statewide-imagery`, `services.wvgis.wvu.edu`) | tiles, bbox, browser-direct; operator adds `wv-statewide-imagery` to `GEV_LIBRARY_ENABLE` | [Open — WV GIS Technical Center terms](https://wvgis.wvu.edu/) | "WV GIS Technical Center, West Virginia University" |
<!-- data-library:end -->
## Bundled snapshots
Static datasets shipped in the repo for an out-of-the-box experience. **None are MIT** — each keeps its own license (see the carve-out in [LICENSE](LICENSE)). Each folder also has its own provenance README.
@@ -268,6 +408,7 @@ The [Bhote Koshi event pack](public/events/bhote-koshi-2026/README.md), under `p
| **Datacenters** (4,352) | `datacenters/` | **ODbL 1.0** (OpenStreetMap extract) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) |
| **Dams** (704) | `dams/` | **ODbL 1.0** (OpenInfraMap / OSM extract) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + Open Infrastructure Map |
| **Military area names** (36,466) | `osm_military_names/` | **ODbL 1.0** (OpenStreetMap, via Overture Maps) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + Overture Maps Foundation |
| **Ukraine outline** (Rail layer no-positions rule; 1 ring, 7,801 points) | `src/data/railNoPositions.js` | **ODbL 1.0** (OpenStreetMap, via geoBoundaries gbOpen UKR ADM0, commit 9469f09; simplified to 0.0005°) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + geoBoundaries |
| **Québec CCTV source catalog** (1,120 locations) | `public/quebec-cameras.json` | Mixed: MTQ locations CC BY 4.0; Montréal camera nodes ODbL 1.0; scenic records retain per-record provenance. No video files are bundled or cached. | Location data is per-record; video remains hosted by Québec 511 and its platform terms apply; scenic source pages may have separate terms | MTQ / Données Québec CC BY 4.0; © OpenStreetMap contributors (ODbL); scenic operator and position source shown in the selected-camera panel |
| **World camera image subset** (2,230 locations) | `config/cctv_sources.world-image-cameras.json` | Mixed: ODOT TripCheck public images (ODOT permits use with credit) and MLIT site content under terms compatible with CC BY 4.0. Frames are not bundled or persisted; the server holds a bounded in-memory cache. | Per-source terms apply; NZTA Journey Planner web-cache rows are excluded; Vegagerðin rows are served by the Iceland pack instead | Oregon Department of Transportation (Camera courtesy of ODOT) and Japan Ministry of Land, Infrastructure, Transport and Tourism (出典:国土交通省 川の防災情報) |
| **TeleGeography Submarine Cable Map** (712 cables + 1,917 landing points) | `telegeography_submarine_cables/` | **CC BY-NC-SA 3.0** | ❌ **NonCommercial — remove for commercial use** | "© TeleGeography — submarinecablemap.com" |
@@ -326,7 +467,7 @@ credits it anyway as `SPECS: CCTV-DATABASE.COM` beside any spec line it renders.
### Bundled OpenStreetMap data
Bundled OSM-derived data is distributed under [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/), separately from the MIT code: [datacenters](src/data/local_data/datacenters/), [dams](src/data/local_data/dams/) (via Open Infrastructure Map), [military names](src/data/local_data/osm_military_names/), and [OSM test fixtures](src/data/fixtures/). Keep © OpenStreetMap contributors and offer modified databases under ODbL.
Bundled OSM-derived data is distributed under [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/), separately from the MIT code: [datacenters](src/data/local_data/datacenters/), [dams](src/data/local_data/dams/) (via Open Infrastructure Map), [military names](src/data/local_data/osm_military_names/), the Rail layer's Ukraine outline in [`railNoPositions.js`](src/data/railNoPositions.js) (OpenStreetMap via geoBoundaries gbOpen, Runfola et al. 2020, simplified; the module's code stays MIT), and [OSM test fixtures](src/data/fixtures/). Keep © OpenStreetMap contributors and offer modified databases under ODbL.
### NASA FIRMS acknowledgement
@@ -547,4 +688,4 @@ NOAA density imagery with the raw GLM product.
## In-app attribution
The required Google Maps / Cesium credit renders on the on-globe credit line (`#cesium-credits`, bottom-left) and must stay visible — including in clean-view and recording modes (the whole line, logo + "Google Maps" + the "Data attribution" link, stays on screen; only the GEV panels/HUD fade). The layer-specific credits (adsb.lol, adsbdb, Planespotters.net, TeleGeography, OSM datacenters/dams/roads, NASA FIRMS, CelesTrak, USGS, City of Austin, Fintraffic, GBFS, Radio Browser, OpenSky, AISStream) are registered into the expandable **"Data attribution"** popover on that credit line via `viewer.creditDisplay.addStaticCredit(new Cesium.Credit(html, /* showOnScreen */ false))` — see `src/data/dataCredits.js`. When you add a new data source, add its license and attribution to this file **and** append an entry to `DATA_CREDITS` in `src/data/dataCredits.js` so it surfaces in the app.
The required Google Maps / Cesium credit renders on the on-globe credit line (`#cesium-credits`, bottom-left) and must stay visible — including in clean-view and recording modes (the whole line, logo + "Google Maps" + the "Data attribution" link, stays on screen; only the GEV panels/HUD fade). The layer-specific credits (adsb.lol, adsbdb, Planespotters.net, Open Vessel Photos, TeleGeography, OSM datacenters/dams/roads, NASA FIRMS, CelesTrak, USGS, City of Austin, Fintraffic, GBFS, Radio Browser, OpenSky, AISStream) are registered into the expandable **"Data attribution"** popover on that credit line via `viewer.creditDisplay.addStaticCredit(new Cesium.Credit(html, /* showOnScreen */ false))` — see `src/data/dataCredits.js`. When you add a new data source, add its license and attribution to this file **and** append an entry to `DATA_CREDITS` in `src/data/dataCredits.js` so it surfaces in the app.
+2 -1
View File
@@ -384,7 +384,7 @@ Click **GEV MIC**, grant the microphone, and just talk. This is more than a voic
- **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director.
- **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token. If you choose **USE CHATGPT OAUTH** in Provider Settings, the local server uses that sign-in only to mint the same kind of short-lived Realtime credential, and refuses the OAuth path for non-loopback clients.
Thirty tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook:
Thirty-five tools (thirty-four unless an Overpass server is set), four jobs — the commands below come straight from the product's voice test suite and tool playbook:
**🎥 Direct it** — drone-operator camera verbs:
@@ -446,6 +446,7 @@ Twenty-three layers and map sources. **Twenty-one have a keyless path.** Some of
| 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer with full-screen **EXPAND**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key |
| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 |
| 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland, Rio de Janeiro (buses and BRT), Szczecin, Lisbon (Carris Metropolitana and the city’s Carris), Prague, Warsaw, Kraków, Poznań, Gdańsk, Rzeszów, Vilnius, Bucharest, Cluj-Napoca, Iași, Rome, Turin, Venice (buses and waterbuses), Bordeaux, Montpellier, Rouen, Toulon, Le Havre, Tours, Málaga, Sendai, Johor Bahru, Kota Bharu, Adelaide, Atlanta, Denver, Edmonton, Halifax, Victoria, and nine in Ontario: Toronto (surface vehicles; the subway publishes no positions), Mississauga, Hamilton, Durham Region, York Region, Guelph, Burlington, Barrie and Kingston | Operator GTFS-Realtime feeds | 🟢 |
| 🚆 **Rail** | Live trains across the US, Canada, Australia, Ireland, Finland, Spain, Norway, Malaysia, Japan and France, from 30 keyless operator feeds: live GPS positions from 29 of them and Irish Rail’s station reports, 28 live out of the box, and Renfe long and medium distance and KTMB once their pack is built. No train is placed from a timetable. Opt-in packs (`node scripts/build-rail-paths.mjs --feed <id>` or `--all`) add the operator’s own route lines, stop names, next stops and trip-update delays. Coloured by service category (urban, local, regional, national, international) with a filter chip and legend for each; trains are drawn once when several feeds carry them, and open a card per train. Buses in mixed feeds are dropped by the server | Operator GTFS-Realtime feeds (static GTFS for opt-in packs) | 🟢 |
| 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 |
| 🧭 **Directions** | Click A and B on the globe for a street-following drive, walk or cycle route draped on the terrain with turn-by-turn steps — then FLY the camera along it. No key, no geocoder, no mic | OSRM on FOSSGIS servers (OpenStreetMap) | 🟢 |
| ⌖ **OSM Places** | Voice place search ("hospitals in Kathmandu") from OpenStreetMap, drawn as numbered points that follow-up questions can filter. Needs your own Overpass server (`OVERPASS_UPSTREAMS`); off otherwise |
+3
View File
@@ -73,7 +73,9 @@ The data proxies under `server/providers/` are written so the browser cannot tur
- **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio.
- **Québec 511 video plays in the browser, opt-in and after Play.** With `CCTV_QUEBEC_VIDEO_ENABLED=1`, `/api/cctv/sources` lists, for each MTQ camera, one URL built server-side from its numeric id on `https://www.quebec511.info/Carte/Fenetres/camera.ashx`; the overlay accepts no other host, path or format. The browser loads that clip only after the viewer presses Play for the selected camera, so Québec 511 sees the viewer's IP address, and the panel says so before Play. Selecting a camera by click, auto-hop, a scene or voice never loads it, and switching cameras stops playback. GEV never proxies, caches or reads the video's pixels. With the flag unset (the default) the browser makes no request to Québec 511.
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
- **The Data Library proxy fetches manifest upstreams only.** `/api/library/data/<id>`, `/api/library/tile/<id>/<z>/<x>/<y>` and `/api/library/index/<id>` (a partitioned GeoParquet entry's file index, built from an S3 ListObjectsV2 listing and one ranged footer read per file, cached in memory for the entry's `cacheSeconds`) resolve the id against the validated manifests in `src/library/entries/`; the browser never supplies a URL, and every derived request must stay on the manifest's https host. Redirects are refused, dataset bodies are capped while streaming at the manifest's `maxBytes` (at most 32 MiB) and tiles at 2 MiB of `image/png`, `image/jpeg` or `image/webp`, responses are cached in memory only (at most 512 tiles), requests are rate-limited per client, and upstream error text is never returned. A keyed entry must be proxied; its key is read from this server's environment through the key registry and its routes refuse cross-site and forwarded requests like the other cost-bearing routes. Browser-direct entries are fetched by the browser with no cookies or referrer.
- **Transit fetches registered feeds only.** `/api/transit/vehicles/<id>` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator.
- **Rail fetches registered feeds only, and none needs a credential.** `/api/rail/feeds` lists the offered feeds; `/api/rail/trains/<id>`, `/api/rail/updates/<id>`, `/api/rail/paths/<id>` and `/api/rail/schedule/<id>` resolve the id against `src/data/railFeeds.js` (`RAIL_FEED_ID_PATTERN`: lowercase letters, digits and hyphens). The browser never supplies a URL, and an id that is not registered, or a trip-updates route for a feed that registers none, does not resolve at all; any method but GET answers 405 (`Allow: GET`, never cached). Every registered feed and every static timetable is open: no rail request carries a credential of the deployer's; one feed's published URL includes a fixed public value its publisher prints for everyone (Bordeaux's `apiKey`), and the only rail setting the server reads is `RAIL_PATHS_DIR`. Upstream requests follow the Transit proxy's policy: https only, each redirect hop validated against the feed's own https origin before it is requested (three hops at most), a request timeout, bodies capped at 8 MiB unless the registry sets a feed's own cap (never above 64 MiB), and a per-feed admission limiter and the failure cooldown ladder bound what the process asks of any operator. Responses are decoded server-side: GTFS-Realtime under entity-count and string-length ceilings, a differential feed is refused, and an empty or header-less GTFS-Realtime reply (an error page served as 200, say) is refused as unreadable, so the last good snapshot is served stale or the request fails with 502. The Irish Rail and SIRI VM XML is read by small scanners, not an XML library: a document that declares a DTD or an entity is refused, and the whole text, the number of trains and every field are capped. Only vehicles the feed's classifier recognises as trains leave the server, and no train position inside Ukraine is ever sent (`src/data/railNoPositions.js`). Snapshots and trip updates live in memory for the feed's own freshness window, with serve-stale after a failure; there is no disk cache and no history. Path and schedule packs are optional build output, read from disk (`.cache/rail-paths/` or `RAIL_PATHS_DIR`) by a reader that re-checks the id against the same pattern, so a name never leaves that directory. A pack over 24 MiB is refused before it is read and answers 503, never a partial file, and so does a pack that cannot be read: only a missing pack is 404; the rest pass through as bytes, through a 64 MiB in-memory read cache keyed by each file's modification time and size, and are served as JSON holding timetables and track shapes only, `no-cache` with an ETag from that time and size (304 when unchanged). A schedule pack of another format version or another feed, or whose window has ended, is not used, and a feed that cannot be read without it answers 503 rather than an empty list. The pack build (`scripts/build-rail-paths.mjs`) runs only when a person runs it, never on app start: it downloads over https only (a redirect to plain http is refused), caps each download in size and time, sends no credential, and refuses an `--out` folder (or a `RAIL_PATHS_DIR` it writes to) inside `src/` or `public/`.
- **Local receiver feeds are operator-configured and local-only.** `/api/local-receivers/aircraft` reads only the `aircraft.json` URLs in the server's `LOCAL_RECEIVER_FEEDS`; the browser never supplies an address. Each host must pass the shared tap address rule (`src/data/tapAddress.js`: loopback, RFC1918, `localhost`, `*.local`; no link-local, IPv6 or other names), the scheme must be http(s), the path must end in `aircraft.json`, and credentials, queries and fragments are refused. Invalid entries are logged at startup and never fetched. Reads use a 2 s timeout, refuse redirects, cap bodies at 2 MB and share one read per second; responses name each feed by band only and never carry upstream error text.
- **Voice place search runs only on the operator's Overpass.** `/api/osm/features` answers only when `OVERPASS_UPSTREAMS` is set and never falls back to a public instance. The client sends a curated preset id, a box, a mode and a limit; the server writes and quotes the Overpass query itself, so no client or model text reaches it. Boxes are capped (36 deg² for a count, 4 deg² for a listing), as are the feature limit, the Overpass timeout and memory, and the response bytes read. Each client (IPv6 clients keyed by /64, as on `/api/overpass`) runs one search at a time and two run at once in all, under a per-client rate limit (12/min) and a global one (30/min); a search gets 22 s across every configured endpoint in turn (Overpass itself is told 20 s), which ends before the voice answer stops waiting at 25 s. The route sits behind the cross-site gate below. Answers are cached in memory and on disk (`.gev-cache/osm-features/`, at most 400 entries and 256 MB, kept a day), keyed to the configured instance so a repointed `OVERPASS_UPSTREAMS` never serves the old one's answers; a malformed answer is never cached.
- **Validated response formats.** Proxied bodies are relayed only in the format the route serves. CCTV frames and the Street View fallback accept JPEG, PNG, GIF, WebP and AVIF only, sent under their canonical type; any other type falls through to the next fallback. The one exception is a source registered as an MJPEG still (the Taiwan Highway Bureau feeds): its `multipart/x-mixed-replace` answer is read only up to its first JPEG part, capped at 2 MB, and sent as `image/jpeg`; a multipart answer from any other camera falls through. A camera whose API needs an operator credential (the City of Ottawa certificate, `OTTAWA_CAMERA_CERTIFICATE`) has a keyless catalog URL; the frame proxy adds the credential only at fetch time, only for a URL inside that camera's pinned path, so `/api/cctv/sources` and `/api/cctv/stream` never carry it. Live CCTV media must match its feed kind (`video/mp4`, `video/webm`, or `multipart/x-mixed-replace` with a boundary) before it is relayed or the camera counts as healthy, and its redirects stay on the registered origin. GBFS bodies must parse as JSON and are re-serialized as `application/json; charset=utf-8`. Camera catalog URLs are pinned by parsed URL — scheme, exact host, no credentials, default port, and a normalized path inside the agency's image directory — and a redirect on a pinned host must stay inside the same directory. Every `/api/cctv` route uses the keyless cross-site checks described below, so cameras stay reachable behind a reverse proxy; a request carrying forwarding headers never reaches the key-bearing Street View fallback and gets the placeholder frame instead, as when no key is configured.
@@ -132,6 +134,7 @@ God's Eye View's tools are also served to MCP clients: over stdio (`npm run mcp`
- **What the panel's requests can reach.** In clients that display MCP Apps, the panel loads the app through the `panel_request` tool, which requests paths on the app's server. It is marked for the panel only and requires a key that each MCP server puts in its panel page, which keeps it from clients that list tools to the model without loading the panel. That key is not access control: any MCP client can read the panel page, key included, and then reach the app's files and data routes through `panel_request`, much as its other tools reach the data. Provider Settings, credential and model endpoints, `/mcp` and the development server's internal routes are refused to every caller.
- **The panel carries the two browser keys.** The panel build (`npm run build:panel`) includes the same Google Maps key and Cesium ion token as the app, and they run inside the client's panel page, on the client's site. A key restricted by HTTP referrer must also allow that page, or the panel cannot load tiles. To keep the app's own keys narrower, build the panel with separately restricted keys in the environment of `npm run build:panel`.
- **Embed mode is not framable by default.** `?embed=1` documents keep `X-Frame-Options: DENY` and `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names the pages that may frame them; a framing page can change what the app shows. Other documents can never be framed.
- **A plain link never loads a billed Street View panorama without a click.** A share link with `sl=google,<lat>,<lon>` flies there, switches to Google 3D and lights Street View, then stops: the panel is armed and the photo loads only when the visitor presses OPEN GOOGLE STREET VIEW (one lookup and one panorama load on the browser Maps key, no retries, no geocoding or server call). A view posted into the embedded app (the God's Eye View panel, or a page `GEV_EMBED_FRAME_ANCESTORS` allows), and the panel page's own opening link for a tool result issued in the last five minutes, are that client's own request and open the photo; freshness is judged only by the result's `issued_at`, so a result older than five minutes only arms, whether the panel was mounted again (reopening a conversation) or got it later on the same mount, while one re-sent or remounted within five minutes of issue opens the photo again; adding `?embed=1` to a published link does not lift the gate (in a frame the link only arms; the panel that holds OPEN is hidden there, so a posted view is the way to open a photo). The OPEN press uses the link's point only while the ground under the screen centre is within 1 km of it, or when the screen centre cannot be picked at all (no ground picking, or a globe with no size yet); with the sky in the middle it opens nothing. The armed lookup dies when its source is switched off, when its map leaves, or when any other photo opens or closes. A link pasted into an open tab follows the same rule, and a link's source switch and layer are never saved as the recipient's own preferences. A link that follows an entity ignores `sl`. Mapillary, KartaView and Panoramax links open their photo at once; none of them bills per image.
## Scope & expectations
+17 -2
View File
@@ -1,8 +1,8 @@
# Third-party software notices
The project depends on third-party npm packages whose own licenses apply. This
file records the packages added for the browser-local SDR feature and the
Street Level viewer; the complete resolved dependency inventory remains in
file records the packages added for the browser-local SDR feature, the
Street Level viewer and the Data Library's GeoParquet reader; the complete resolved dependency inventory remains in
`package-lock.json`.
## Web RTL-SDR
@@ -45,3 +45,18 @@ Street Level viewer; the complete resolved dependency inventory remains in
- Source: <https://github.com/mrdoob/three.js>
- License: MIT
- License text: <https://github.com/mrdoob/three.js/blob/dev/LICENSE>
## hyparquet
- Package: `hyparquet` 1.31.2 (loaded on demand when a Data Library
GeoParquet entry is switched on)
- Author: Hyperparam
- Source: <https://github.com/hyparam/hyparquet>
- License: MIT
## fzstd (hyparquet ZSTD decompressor)
- Package: `fzstd` 0.1.1
- Author: Copyright (c) 2020 Arjun Barrett
- Source: <https://github.com/101arrowz/fzstd>
- License: MIT
+18
View File
@@ -0,0 +1,18 @@
- "Show me X on Street View" (or on Mapillary, KartaView or Panoramax): a
`show_street_level` voice and typed action flies to the place, switches to
Google 3D when the source needs it, lights that source in Street Level,
turns the layer on and opens the nearest photo, saying which step failed.
With no place named it looks under the screen centre, as OPEN does. Views
and share links carry the same request (`street_level`, `sl=`). A plain
`sl=google` link never loads a billed panorama by itself: it arms Street
View and leaves the load to OPEN GOOGLE STREET VIEW; embedded views open it,
and so does the God's Eye View panel for a result issued in the last five
minutes; a link whose source has no working key says so. A request stops
when the user navigates, or enters cockpit, clean view, recording or scene
playback, before its photo loads (collapsing the panel does not stop it: the
photo reveals it); a new photo replaces the open one in place, so EXPAND
stays. When nothing is found and a 360°, flat or SINCE filter is on, the
answer names the filter. Failure cards are in the interface language. An
unavailable map stack now reports the
controller's own reason instead of always asking for a Cesium ion token
(Said Turksever, @asturksever, #1113).
-4
View File
@@ -1,4 +0,0 @@
- Stop screen annotations sampling surface height for marks that are hidden,
behind the camera or over the horizon. Visible marks still waiting for
terrain share 32 samples per projection pass and retry once a second, so a
large board no longer stalls the frame (asturksever, #1277).
@@ -0,0 +1,3 @@
- Routing outages report "Routing service unavailable" in voice route planning
and "Routing unavailable" in Directions instead of "no route found"
(quanticsoul4772, #1509).
+5
View File
@@ -0,0 +1,5 @@
- FIRMS fire ages round to whole hours before choosing a unit, so an age of
47.5–48 hours reads "2d", as exactly 48 hours does, not "48h". Detections
are filtered to the trailing 24 hours, so this shows mostly in the
`STALE · cached {age}` stats line after about two days without a successful
refresh (@quanticsoul4772, #1531).
+29
View File
@@ -0,0 +1,29 @@
- Add the Data Library. New regional and niche sources no longer join the
Data Layers panel: **Browse data library** at its foot lists live entries
(a manifest each in `src/library/entries/`, fetched from the provider, never
bundled) with their licence, coverage and what this instance needs to serve
them, and **Add** puts one under a Library heading. Datasets (GeoJSON, CSV,
ArcGIS), raster tiles and host-layer packs are supported; selections persist
and travel in share links as `lib=`, separate from the layer tokens; one
generic `/api/library` proxy serves entries without CORS. Seeded with NASA
EONET events, EIA and WRI power plants, NWS alerts, NOAA tide stations,
Smithsonian volcanoes, NASA Black Marble, and the
opt-in IMF PortWatch and CCTV packs, plus the City of Amsterdam's
traffic and ANPR camera positions (opt-in; contributed by @Maher-Reven in
#251). FEMA USA Structures and the
National Bridge Inventory load what is in view once you zoom in, and
Addresscloud's EPC ratings stream from Source Cooperative's GeoParquet on
S3, coloured A–G with live legend counts. Open tile overlays (geology,
biodiversity, surface water, national aerial photos and maps)
join GDACS alerts and EPA Superfund and TRI facilities. 57 aerial photo
layers from the Open Imagery Register (national programmes across Europe,
NSW, NAIP and US state, county and city services, WMS and ArcGIS export
included) show their resolution, true-ortho flag, years flown and refresh
cycle in Browse; aerial photos stay above every other drape, are clipped to
their coverage, and the providers' blank fill no longer whitens the globe.
Kartverket's open Norwegian maps (topographic, nautical charts, hillshade,
terrain relief, sea depths, boundaries and the cadastre) and Mapterhorn's
global terrain, drawn in the browser as shaded relief with an elevation
legend, are in the library too. Stat tiles head the dialog. A library layer's row has
Zoom to layer and Remove icons, and the dialog has Zoom and a confirmed
Remove all (asturksever, #1604).
+24
View File
@@ -0,0 +1,24 @@
- Add Rail in the Movement group, off by default: live trains from 30 keyless
operator feeds in the United States, Canada, Australia, Ireland, Finland,
Spain, Norway, Malaysia, Japan and France, from commuter and long-distance
rail to the streetcars and trams of Seattle, Cincinnati, Toyama, Bordeaux
and Montpellier. A new `/api/rail` proxy fetches registered feeds only,
needs no key and drops buses from mixed feeds. Every train shown is live:
29 feeds publish GPS positions and Irish Rail its station reports, and 28
of them show with no setup. No train is placed or moved from a timetable;
a feed whose live positions fail shows as unavailable. Packs are an opt-in
extra: `node scripts/build-rail-paths.mjs --feed <id>` (or `--all`) builds
a feed's path and schedule packs into `.cache/rail-paths/`, which add the
operator's own route lines, stop names, the next stop and trip-update
delays to its live trains; Renfe long and medium distance and KTMB, whose
vehicles name no route, appear once their pack is built. Nothing is
downloaded when the app starts, and lines are only ever the operator's own
shapes. Trains are coloured by category (urban, local, regional, national,
international), each with a chip and legend entry shared via token `02`,
glide between reports with Transit's delayed playback, are drawn once when
several feeds carry them, and open a card on click. A selected train that
is moving is followed by the camera, as an aircraft is, and any other is
flown to once; the selected train and its line stay on the map from orbit.
No train is placed in Ukraine. Feeds that publish no licence are used on
the basis each one's entry records, and their credit never presents that
as a licence (Denis Groshev, @ZloyS, #1675).
+4
View File
@@ -0,0 +1,4 @@
- The News layer reads its time from the world clock, so a host that follows
the clock for News can show a past time's reports with their ages, window and
counts as they were then. Nothing changes while the timeline is live
(Denis Groshev, @ZloyS, #1677).
@@ -0,0 +1,5 @@
- Leave browser and system chords alone: a shortcut key held with Cmd, Ctrl or
Alt no longer runs its single-key shortcut. Copying with Cmd+C or Ctrl+C used
to turn the CCTV layer on and open the nearest camera, Cmd+F opened the layers
panel and Cmd+1 to 6 changed the visual style. The Q/W/E/R/T keys of an
expanded city's points of interest follow the same rule (asturksever, #1691).
+2
View File
@@ -0,0 +1,2 @@
- Show a new CCTV camera's frame the moment it is decoded, with no fade, so
switching cameras no longer lingers on an empty preview (samehkhamis, #1693).
@@ -0,0 +1,3 @@
- End a CCTV camera's monitor plane where its view meets the ground on globe
maps too, instead of a kilometre-wide plane hundreds of metres up
(samehkhamis, #1694).
+3
View File
@@ -0,0 +1,3 @@
- Show a selected vessel's photo from Open Vessel Photos, credited with its
photographer and open licence, and its MMSI beside its name in the CONTACTS
view (samehkhamis, #1695).
+3
View File
@@ -0,0 +1,3 @@
- Add an `./i18n` entry so an application with its own language menu can
start, follow and switch the interface language as the globe picker does
(samehkhamis, #1696).
@@ -0,0 +1,5 @@
- Fix the translations of the strings added since the last review (News,
Data Library, Wildlife Sightings, Shipwrecks, Rail, the vessel and orbital
cockpits) in all twelve languages: neutral side labels and counts no longer
break the sentence, terms follow each glossary, and Traditional Chinese
follows Simplified Chinese again (German, @gekh, #1698).
+3 -1
View File
@@ -224,7 +224,9 @@ leaving out `command-dock` the voice control and its pointer reticle;
leaving out `context` the Bhote Koshi reconstruction panel; and leaving out
`intel-hud` the Intel HUD overlay (`#intel-hud`), whose readouts the HUD
implementation publishes to the `gev-hud` store. An unknown
view name is an error. A host that does not initialize Core's locale passes
view name is an error. An application that offers its own language choice
starts and switches the language with the `./i18n` entry (`startPageLocale`,
`followPageLocale`, `switchLocale`; see [I18N.md](./I18N.md)) and passes
`localePicker: false` to `createApplicationControls` instead of editing the
markup. Node tests that import application modules register the component
loader with `node --import=<core package>/build/register-vue`.
+644 -19
View File
@@ -145,16 +145,6 @@ voice follow-ups match on them and bikeshare uses the raw station name as its
ID. The text is still untrusted: normalization does not prevent prompt
injection.
Screen annotations skip hidden records and, before surface sampling, reject an
unsettled anchor only when its whole accepted height range (-430 to 9000 m) is
behind the camera or below the globe horizon; a settled height is tested as the
actual point. Anchors outside the conservative frustum still project at their
cached or zero height, so a partly visible arrow keeps its connector, but do
not sample. Unresolved visible heights share a 32-sample projection-pass budget,
served oldest-refused first, retry at most once per second per anchor on
rendered frames, and retain no extra timer or listener. Validated heights remain
settled in the bounded cache.
Terrain height requests accept at most 64 points, matching the browser service's
existing batch size and one bounded upstream chunk. Larger requests return HTTP
413 before disk-cache loading or upstream work; malformed coordinates remain 400.
@@ -394,6 +384,229 @@ provider registration rejects compositions that cannot fit rather than truncatin
identities. A malformed or duplicate entry invalidates the complete custom-switch
field. Existing whole-link size limits still apply.
## Data Library — October 21, 2026
DATA LAYERS keeps its curated core layers; everything else lives in the Data
Library, opened from **Browse data library** at the foot of the panel. An entry
is a manifest in `src/library/entries/` that points at a live upstream its
provider hosts — a query API, a provider-hosted static file or a tile endpoint.
Nothing an entry describes is bundled, so a self-hosted instance serves every
entry without hosting data. Manifests are validated strictly when
`src/library/catalog.js` loads (unknown fields are errors), so an invalid entry
fails the unit suite. Authoring guide: [DATA-LIBRARY.md](DATA-LIBRARY.md).
Three kinds ship. A `dataset` (GeoJSON, GeoJSON Lines, CSV with lat/lon
columns, a JSON record list read from `source.records` with named or guessed
lat/lon fields, or a paged ArcGIS FeatureServer query) renders through the same
GeoJSON layer as Datacenters and Dams, with the manifest's label, up to two
card fields and an optional `rank` field that orders a capped set and feeds the
layer's level-of-detail priority (0–600, below the named/unnamed gap). A capped
dataset reads PARTIAL. A dataset with `source.view` is view-bounded,
for layers too large to load whole (FEMA USA Structures: ~135 million
footprints): below `view.maxAltitudeM` it queries ArcGIS for what intersects a
window around the point the camera looks at (half the camera height each way,
330 m – 5.5 km, clipped to the visible rectangle), one page per view after the
camera settles, replacing the previous set only when the new one is ready and
aborting a superseded request; above it the row reads "zoom in to load this
layer" and nothing is fetched. Footprints are ground-clamped polygons (draped on
Google 3D tiles too), without outlines. Zoom to layer lands straight above the
manifest's `view.sample` area at 60 % of the altitude limit. View-bounded
entries are ArcGIS or GeoParquet, and browser-direct, in this build.
`geoparquet` datasets are read in the browser with `hyparquet` (with `fzstd`
for ZSTD), imported lazily on first use; only the columns an entry uses are
decoded, and WKB geometry arrives as GeoJSON. A partitioned set
(`source.partitions`: an S3 ListObjectsV2 `listUrl` and a `keyBase`, both on the
source host) must be view-bounded. Its index — each file's GeoParquet footer
bbox, read with one range request per file — comes from `/api/library/index/<id>`
(built once per `cacheSeconds` and shared), or is built in the browser and kept
in `gev:library:index:<id>` on a static build. Footer bboxes can be stretched
by stray geocodes, so candidates are ranked by how much of each file's bbox
the view covers, read in parallel batches of four (up to eight per view)
until a batch has features in view, and each file read once teaches its real
extent (the box holding 98 % of its points, `gev:library:extent:<id>`), so
later views skip files that do not cover them. The last four decoded files
stay in memory. `display.colorBy` colours each feature by a field's value (a
dotted path such as `categories.0.title` is flattened into the feature's
`__gevColorKey`): view-bounded layers draw 6 px dots or filled polygons, other
datasets keep the GeoJSON layer's stems, cards and level of detail with a
per-feature colour (`featureColor` in `localGeojsonCore.js`). Values sharing a
`colorBy.labels` label form one legend group (coal, gas and oil are "Fossil");
values the manifest does not colour draw in `display.color` and count as
Other. The row legend lists each group with its count (in view, or in the
loaded set), one item per line as Fire Perimeters' does. Status-like fields
(GDACS and PortWatch alert levels, NWS severity, bridge condition) use the
fixed status palette (good #0ca30c, warning #fab219, serious #ec835a,
critical #d03b3b); categorical ones (fuel, EONET category, building
occupancy) use at most four hues that pass all-pairs colour-vision checks on
the dark globe, plus a neutral Other. Tile layers carry a static
`display.legend` copied from the provider's own scale (GBIF classic.poly,
JRC occurrence, Macrostrat's ICS period colours). Aerial photo layers (IGN, GSI
photos, swisstopo SWISSIMAGE) set `display.outline`: their covered countries'
borders draw as a thin white ground-clamped line, so the imagery's edge is
plain, with a "Coverage outline" legend item. Borders come from Natural
Earth's 1:50m admin-0 countries (public domain, fetched live once per page and
credited); `countries` draws every part (Japan's islands, Liechtenstein beside
Switzerland), `coverage` only the parts inside `coverage.bbox` (metropolitan
France). A failed border fetch costs only the
outline. The same layers set `display.aerial`: they drape above every other
imagery layer and raise themselves again whenever another drape is added, so a
basemap or weather overlay switched on later goes underneath. Each is
`source.bounded` (tiles requested inside `coverage.bbox` only) and names its
provider's no-imagery fill in `display.noDataColor`, drawn transparent through
Cesium's colour-to-alpha (max channel distance 0.02): measured fills are
swisstopo white, IGN's flat ocean blue (#0a3b59) and GSI's ocean grey
(#343434). Without both, a white fill covered the whole globe. A 4xx tile
answer counts as "no imagery here", never as a tile error. IGN's overseas
departments are outside its single bbox and are not shown. The aerial set now holds 60
entries: 57 imported from the Open Imagery Register (national programmes in
Belgium (Wallonia), Germany, Estonia, Spain, Poland, Portugal, Australia's
NSW, and 49 US services: NAIP plus state, county and city programmes) plus
IGN, GSI and SWISSIMAGE, each with an `imagery`
block (type, published resolution, true ortho, years flown, refresh) that
Browse shows as a spec list. Tiles take `xyz` (with `{reverseY}` for TMS) or
`bbox` templates (`{bbox-epsg-3857}`: WMS GetMap and ArcGIS export, filled
by Cesium in the browser and by the proxy from z/x/y). Sub-national entries
draw a `bbox` rectangle outline without fetching the country borders. 35
entries whose licence the register gives only as "see publisher terms" are
operator opt-in. Norway comes from Kartverket's open
services (CC BY 4.0, "© Kartverket"; its zoom 12–20 Geovekst data may be shown
in live services but not copied): the topographic, greyscale, raster-topo and
nautical-chart caches (`xyz`), and WMS hillshade, DTM colour relief, sea depths
(from zoom 11, Kartverket's 16-class depth legend), county and municipality
boundaries, and the cadastre (from zoom 14). Norge i bilder needs
authentication and is not included. `mapterhorn-terrain` is the first
`source.encoding: terrarium` entry: Mapterhorn's global 512 px terrain-RGB
tiles are decoded per tile in the browser (`src/library/relief.js`) into
shaded relief with a six-class elevation legend in metres. A `tiles` entry (`{z}/{x}/{y}` template) drapes on the
active imagery host — the globe, or the Google 3D tileset — and follows map
stack switches; with no host it shows the existing "choose a globe map"
guidance, and its row reads DRAPED rather than a count. A `pack` describes an
existing host-layer source pack (today the opt-in CCTV packs): it is
informational in Browse, names the host's own switch (`CCTV_NYC_ENABLED=1` and
so on) and offers no Add. The `provider` kind from the RFC is refused by the
validator until it ships.
Entries register their layer only when a viewer adds them, after the core
catalog is sealed: `LayerLifecycle.registerLibraryLayer` accepts only ids in
the `lib-<entry id>` namespace and only after `finalizeRegistrations`, and
`unregisterLibraryLayer` destroys them. Library layers never enter
`LAYER_STATE_REGISTRY` or the core token ledger. The panel groups them under a
**Library** heading after every core group and rebuilds its rows on a
`registrations` activity. Add switches the layer on; Remove destroys it.
A library layer's row carries two icon chips, on or off (the panel shows
row controls for an off layer only for Data Library rows, whose chips are
marked `whileOff`): **Zoom to layer**
(`zoom_in_map`) flies to the extent of the loaded features, or a tiles entry's
declared coverage, taking the shorter way round the antimeridian and showing the
whole globe for world-scale sets; **Remove from Data Layers** (`delete`)
removes the entry. Zoom on a layer that is off switches it on first (a
dataset only knows its extent once loaded). Zoom runs as an immediate
navigation (`layer`), so Cockpit
refuses it with its own message and the current follow owner is released first;
reduced motion flies instantly. Icon chips (`chipGroup.js` `icon`/`action`) use
their label as accessible name and tooltip and carry no `aria-pressed`.
This browser's selection persists in `gev:library:v1` (`{ added, on }`), written
only for explicit (user, voice, tool) changes, like core layer state, so a
Context session's temporary changes are not remembered. Share links carry the
switched-on entries as `lib=<t>.<t>` with the permanent two-character tokens in
`src/library/ledger.json` (append-only; packs take none). A link's entries join
the viewer's library and, as for core layers, the link alone decides what is
on. Unknown tokens are skipped. The live hash follows the existing
initial-restore suppression.
Browser-direct entries are fetched without cookies or a referrer, redirects
refused, bodies capped while streaming. Proxied entries go through
`/api/library` (`server/providers/library.js`): `/catalog` reports each entry's
availability and reason (`operator-opt-in`, `operator-disabled`, `key-missing`)
without secrets; `/data/<id>` and `/tile/<id>/<z>/<x>/<y>` reach only the
manifest's fixed upstream (the browser names an id, never a URL), refuse
redirects and off-manifest hosts, cap dataset bodies at the manifest's
`maxBytes` (≤ 32 MiB) and tiles at 2 MiB of `image/png|jpeg|webp`, cache per the
manifest's `cacheSeconds` (60 s – 24 h; at most 512 tiles), serve the last good
dataset as stale on failure, rate-limit per client, and never return upstream
text. A keyed entry must be proxied; its key comes from the server environment
through the key registry and its routes use the strict cross-site gate, while
keyless routes use the keyless gate so they work behind a reverse proxy.
Operators opt entries in with `GEV_LIBRARY_ENABLE` and remove them with
`GEV_LIBRARY_DISABLE`. A static build with no server still serves every
browser-direct, non-opt-in entry; proxied ones say they need the app's server.
The Browse dialog is a `dialog[data-panel-surface]`: modal, labelled, search
focused on open, Escape and backdrop dismiss and return focus to Browse, and it
closes when clean view, recording or Cockpit hides the panels. Its surface is
frosted glass (a lighter `--glass-bg` than the rails, a 22 px backdrop blur
clipped to the theme's frame, a barely dimmed backdrop), so the globe shows
through without the HUD text behind it competing. An added entry that is on
offers ZOOM, which closes the dialog and flies to it. Above the intro, four
stat tiles summarise the library: Datasets (every entry, broken down by kind),
Themes (how many distinct themes the entries cover), Added (and how many are on
the map), and Map features (features loaded by dataset layers that are on;
tiles have no count).
They refresh with the dialog and once a second while it is open, so counts
land as datasets finish loading. The line beside **REMOVE ALL** answers the
filter ("Showing 2 of 8"). **REMOVE ALL** removes every added entry; its first
press arms a confirm for four seconds. It lists
addable entries first, then entries this instance would need to switch on, then
packs, with kind, status, opt-in or key badges, provider, coverage and a licence
link, and filters by text, theme and "only what this instance serves".
Archived entries are hidden unless added. An entry's attribution and licence are
registered in the Data attribution popover while it is in use.
Seeds: NASA EONET open events (GeoJSON, browser-direct), EIA US power plants
≥ 100 MW (ArcGIS, browser-direct, ranked by capacity), Smithsonian GVP Holocene
volcanoes (WFS GeoJSON, proxied — the upstream sends no CORS headers), NASA
Black Marble 2016 night lights (tiles), and the NYC,
Lithuania and FL511 CCTV packs. Added from the Ask dataset registry (its live,
mappable sources that are not core layers): the WRI Global Power Plant Database
(provider-hosted CSV, the 10,000 largest of ~35,000 by capacity, so it reads
PARTIAL), NWS active alerts that carry their own polygon (zone-wide alerts have
no geometry and are dropped), NOAA CO-OPS tide stations (JSON records), and IMF
PortWatch chokepoints and port disruptions (ArcGIS, operator opt-in until the
reuse terms are confirmed, as Ask holds them in hosted mode). FEMA/ORNL USA
Structures (CC BY 4.0) is the first view-bounded entry; its service refuses a
query page smaller than its own 2,000 records, so the probe asks for a full
page. Addresscloud's EPC ratings for England and Wales (Source Cooperative,
read from `s3.us-west-2.amazonaws.com/us-west-2.opendata.source.coop`, 344
local-authority GeoParquet files, OGL v3.0) are the first partitioned
GeoParquet entry, coloured by band A–G with the official EPC colours. From an
open/freemium registry review (commercial sources excluded): GDACS disaster
alerts; the National Bridge Inventory (view-bounded, coloured Good/Fair/Poor);
EPA Superfund sites and Toxics Release Inventory facilities (view-bounded); and
tile overlays for Macrostrat geology, GBIF biodiversity density, USGS
topographic and hydrography maps, IGN (France) aerial photos, BKG
TopPlusOpen, GSI Japan map and photos and JRC surface water occurrence.
Entries whose licence or provider pages no longer resolve (404 or unreachable
host, checked 2026-11-12) were removed: OpenSeaMap seamarks, EMODnet
bathymetry, SEDAC GPW population density, PDOK, basemap.at, Geneva SITG,
ICGC Catalunya, Luxembourg, Vlaanderen, Minnesota and Bangor aerial photos. `npm run library:check` validates manifests,
the ledger (append-only with `--base-ref`, run in CI on pull requests) and the
generated DATA_SOURCES.md section; `npm run library:probe` hits every upstream
once; `npm run qa:data-library` is the browser gate (fixture-backed).
Shares: the compact `/share/<code>` link carries the switched-on entries in
field 29 (docs/SHARE-LINK-FORMAT.md), so Copy Link keeps them, and a share
opened later in the same tab (pasted address, Back/Forward) restores them too.
Any share decides what is on, as for core layers: one with no library entry
on switches the recipient's stored ones off and leaves them listed. A tile
entry whose coverage needs more than 256 tiles at its minimum zoom (Norway's
cadastre from z14, sea depths from z11) is hidden below that scale through
Cesium's `minimumTerrainLevel`, so zoomed out it requests nothing. EPC
partitions learn two extents (`gev:library:extent:v2:`): every point's box
rules a partition out, the 98 % box only ranks it, so homes at a partition's
edge are found on later visits; the v1 trimmed-only key is dropped. A
partition index whose footers keep failing after two retries is
`complete: false`: neither the server nor the browser keeps it as whole, a
complete earlier index is served instead, and it is rebuilt after five
minutes. A view that leaves partitions touching it unread is PARTIAL. Listing
pages and footer reads are size-capped and ranges are validated. The Browse dialog is a Vue view (src/ui/views/panels/DataLibraryDialog.vue
and DataLibraryCard.vue, teleported to the body from LayerPanelsView) over the
`gev-data-library` Pinia store; src/ui/libraryBrowser.js is its framework-free
owner (open state, filters, the rows they show, Remove all) and the text
helpers the view renders, so every label follows the locale. Only the modal
open/close and focus moves are imperative.
## Utilities panel — October 29, 2026
UTILITIES is its own left-side panel, between Data Layers and Scenes, with
@@ -1147,7 +1360,7 @@ Voice controls bind to a protocol-independent session factory. The default WebRT
The standalone app offers OpenAI and Gemini Live through the **AI Provider** setting, shared by voice and typed chat. Gemini keeps its permanent API key on the local server and mints a short-lived, single-use credential with server-owned instructions and the standalone action catalog. With no Gemini key the token route answers `503` with code `API_KEY_MISSING`, and the voice control says `Add a Gemini API key in Provider Settings` without the microphone-permission hint. Voice status and error messages keep the message as given on the session's `state` event (`sourceDetail`, English catalog text when it is the app's own, with `sourceParams` for its placeholders), beside the worded `detail`, so the voice control words them again after a language change. Typed-chat errors and the chat panel are worded once. Its native WebSocket adapter supports open microphone and push-to-talk input, streaming audio, captions and the existing action runner. In push-to-talk, a tool call that arrives with no input, reply or tool result outstanding is answered with a failure and never runs, and neither does any other call in the reply it starts; open-microphone turns start on Gemini's server-side activity detection, which reports no boundary to check. Switching providers ends the old conversation and releases its microphone and connection; only the input mode carries over. Gemini uses bounded shutdown draining and explicit manual restart, without automatic reconnection or a promise of uninterrupted session duration. Hosts that do not opt into the provider facade retain their existing default or custom adapter.
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 33 tools using separate description-only metadata; a session is offered 32 of them unless `OVERPASS_UPSTREAMS` is set, which adds `osm_query` and its one prompt sentence (`server/providers/openai/tools.js`). Description customization cannot replace argument types, enum values or required fields.
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 35 tools using separate description-only metadata; a session is offered 34 of them unless `OVERPASS_UPSTREAMS` is set, which adds `osm_query` and its one prompt sentence (`server/providers/openai/tools.js`). Description customization cannot replace argument types, enum values or required fields.
Voice areas live in a store owned by the action runner for the page (`src/data/areaStore.js`); the model gets an `areaId`, a rounded bounding box and a summary, never geometry. `resolve_area` (`src/voice/areaActions.js`, `src/data/areaResolver.js`) checks the store, bundled Natural Earth regions, then bundled countries and states by name — a name several prominent units share returns up to four candidates unless the view is inside exactly one — and then the annotation outline ladder (`resolveAnnotationTarget`: bundled counties, Census places and neighborhoods, the guarded `/api/geocode/outline`, and Overpass only when configured). Administrative geometry is stored only when its source identity agrees with the requested unit, by name or by a form the bundled matcher reads as that unit at its level ("State of Texas", "Province of Bagmati", "USA", "UK"; not "Great Britain" or "Korea"); stale, mismatched, synthesized or otherwise unverified administrative boundaries are refused rather than relabelled. Clarification IDs use the bounded `ne:<kind>:<slug>` grammar and are accepted only from the current resolver's candidate memory. A lookup past 8 s answers `AREA_TIMEOUT` and keeps filling the caches. `around=true` asks the same ladder for the grounds enclosing a landmark (Nominatim, then the OpenFreeMap area holding it) and otherwise stores a 250 m buffer marked approximate. `analyst_query` scopes add `area`, `drawn` (newest draw-tool outline) and `annotation` (newest or named outline); region scopes use the same resolver within 3 s. Drawn outlines are a simplified copy (≤1,200 vertices); counting reads the stored geometry (`src/data/areaGeometry.js`). `find_imagery` (`src/voice/imageryActions.js`) sets the Recent Imagery box from an area, the view or the turn's pointer snapshot and pins the best acquisition; any hand change to the panel or explicit layer OFF while it searches ends the request (`DISPLACED`) without pinning or success narration. `osm_query` asks `/api/osm/features` for one curated preset in a bounded box; the route answers `OVERPASS_NOT_CONFIGURED` without an operator Overpass. Results go to the session-only `osm-places` layer and become the analyst's last answer; a list capped at its limit stays incomplete in follow-ups, which keep its "listed the first N" caveat and report counts as floors. An explicit OSM Places OFF while the provider is pending aborts the work and prevents re-enable, result, credit and follow-up-memory commits. The voice answer deadline also aborts the client provider request, so the caller does not leave a timed-out request without a cancellation owner. The server's whole search (22 s across every configured endpoint, with Overpass told 20 s) ends before that 25 s deadline, because a browser that stops waiting cancels the upstream query and leaves nothing cached for a retry.
@@ -2011,7 +2224,9 @@ Frame loading, calibration editing and status display have separate modules;
providers, camera placement and navigation policy retain their existing owners.
Changing cameras invalidates old image callbacks and cancels an unfinished
calibration edit. Failed refreshes preserve settled pixels for the same camera,
while a newly selected camera never shows the prior camera's image. Disposal
while a newly selected camera never shows the prior camera's image. A frame is
shown without a fade once it is decoded (or after one second if its decode never
settles), so a new camera's preview is blank only while its frame loads. Disposal
releases listeners, subscriptions, image handlers, summary timers and chip-hide timers.
## UI disposal
@@ -2335,8 +2550,13 @@ Browser acceptance: `QA_BASE_URL=http://127.0.0.1:4173 node scripts/qa-segmented
`ui/input` supplies the bubbling application shortcut listener and generated
shader-parameter rows. Number/style keys, H/O/V/F/D/C actions, native form-control
typing and Escape behavior retain their existing mappings. Capture-phase
surfaces continue to arbitrate their own keyboard events first.
typing and Escape behavior retain their existing mappings. A shortcut key held
with Cmd, Ctrl or Alt runs no shortcut, so the browser's own chord (Cmd+C to
copy, Cmd+F to find, Cmd+1 to switch tab) does not also toggle CCTV, the layers
panel or a style; Shift does not count, and Escape dismisses search either way.
The Q/W/E/R/T keys of an expanded city's points of interest follow the same
rule.
Capture-phase surfaces continue to arbitrate their own keyboard events first.
The UI facade retains shader values, share-restore authority, render requests,
search dismissal, visibility and Cockpit portal policy. Parameter rows preserve
@@ -2981,6 +3201,15 @@ stays in view or tracked; an answer, found or not, and a permanent refusal
stay final for the session. A route belongs to the callsign it was looked up
for, so a callsign change drops the old route. See `DATA_SOURCES.md`.
A selected vessel gets a photo the same way. The browser asks Open Vessel
Photos for it by MMSI (`fetchOpenVesselPhoto` in
`src/sources/live/standalone.js`, read by `src/data/openVesselPhoto.js`), once
per MMSI (`src/layers/vessels/photos.js`), and the CONTACTS view shows it under
the vessel's name and MMSI ("NAME · MMSI 366999709") with the photographer,
the photo's open licence and a link to the vessel's page. A vessel without an
MMSI is never looked up; a miss stays final for the session, and a failed
lookup is asked again after five minutes.
`gods-eye-view/server/providers/live` is a Node-only entry for the existing
plugins and shared request helpers. Importing it starts no sockets or timers.
The existing aircraft normalizer is separately available through the portable
@@ -3063,6 +3292,7 @@ to a set time, naming its end as `at` (epoch ms): the row then reads
`window_end`. The dev proxy never sets `at`.
The FIRMS CSV parser rejects impossible acquisition calendar dates before the proxy filters records to the trailing 24 hours, while retaining valid UTC acquisition times.
FIRMS ages round to whole hours before choosing a unit, so a 47.6-hour age reads "2d", not "48h"; with detections filtered to 24 hours, this shows mostly in the `STALE · cached {age}` stats line.
The `get_active_fires` tool reports acquisition times as UTC ISO timestamps,
including zero-padded hours and minutes; missing or invalid dates and times
remain `null`, including impossible calendar dates from a custom source.
@@ -4850,6 +5080,10 @@ Shipped locales: **en** (source and fallback), **ar**, **de**, **es**, **fr**, *
views re-render, code that published translated text words it again through
`onLocaleChange`, and `<html lang>`/`dir` and the page's `data-i18n` markup
follow. Nothing is lost; a toast already on screen keeps its words.
An application that offers the language elsewhere (a menu) uses the
`./i18n` entry (`src/i18n/page.js`): `startPageLocale` at boot,
`followPageLocale` for the markup and `switchLocale`, which does what a
picker choice does, with `localePicker: false`.
- **Right to left.** Arabic (`ar`) and Hebrew (`he`) are in `RTL_LOCALES`, so
`<html dir="rtl">`: the title, style indicator, both panel stacks and their
icon rails swap sides through logical properties (`inset-inline-*`,
@@ -4921,6 +5155,7 @@ its criteria cannot be silently ignored.
| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS, Kartverket ground heights) + King County (WA) Open Data + WSDOT (WA) Open Data + NYC DOT TMC (opt-in) + City of Toronto RESCU + Catalonia (Servei Català de Trànsit) + Iceland (Vegagerðin / IRCA) + Hong Kong Transport Department + FL511 (FL, opt-in) + Freeway Bureau, MOTC (Taiwan) + Houston TranStar (opt-in) + Autostrada del Brennero (IT, A22 stills) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) |
| Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh |
| Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) |
| Rail 🚆 | Live operator train positions (GPS, or station reports for Irish Rail) and trip updates, trains only, with opt-in route packs built from static GTFS (30 keyless feeds, all on, 28 out of the box; `src/data/railFeeds.js`) | `src/layers/rail/` via `src/app/layers/rail.js` | `/api/rail` | 15s (poll + delayed playback) |
| Bikeshare 🚲 | GBFS (Lyft + BCycle + Social Bicycles + PBSC incl. Bike Share Toronto, Bike Itaú (Brazil), Bicing (Barcelona) and seven other Spanish cities + Urban Sharing, Entur, JCDecaux (Lillestrøm, Valencia, Seville) + nextbike (AMBici)) | `src/data/bikeshare.js` | `/api/gbfs` | 60s |
| Directions 🧭 | OSRM on FOSSGIS servers (OpenStreetMap) | `src/data/directions.js` | `/api/route` (`steps=1`) | on placement / mode change |
| Datacenters ▣ | OSM extract (bundled) | `src/data/infrastructure.js` | — | static |
@@ -5267,6 +5502,111 @@ core's lookups alone. The MCP globe panel allows the meta-catalogue and the
three biggest instances; a picture on another instance shows only in the
app.
**"Show me X on Street View."** `show_street_level` (voice and typed;
`src/voice/gevActions.js`) is one call whose stages fail separately
(`location`, `view`, `map`, `provider`, `layer`, `key`, `imagery`):
- **Key first.** A keyless source explains itself and changes nothing, as a
chip does.
- **Place.** A name, coordinates, a pointed spot or a numbered item: it flies
there at street framing and waits for arrival. With no place it looks at the
ground under the screen centre, as OPEN does (`aim: 'screen-centre'`,
`armed: false`: sky refuses, and a link's armed point is not used and is kept
until a photo opens). Coordinates off the globe are refused.
- **Map and source.** It switches to the map the source needs (Google Street
View: Google 3D), lights the source through `setLayerParams`, one at a time
and kept like a chip click (origin `voice`; a link runs the action with
origin `programmatic`, so it never writes the recipient's saved layer state),
turns the layer on and calls `openNearest`. `provider` is `google` for
"Street View", `mapillary`, `kartaview` or `panoramax` by name, else the lit
source.
- **Hidden panel.** Cockpit, clean view, recording or scene playback end it at
`view`, before the flight and after every await; embed mode's own clean view
is exempt, since the photo is lifted over the globe.
- **Superseded.** After every await (flight, map switch, layer on, lookup) a
newer turn, a shell navigation stamp (`_navigationGeneration`, re-taken where
its own flight lands) or a press or wheel on the globe stops it
(`superseded: true`). The same checks run before a found photo loads
(`proceed`), so nothing billed loads for a request that was given up. A
lookup given up leaves the panel as it was when the request kept the photo's
source; one that had switched the source off closes that source's photo with
it. A photo that opened after either is closed again.
- **Replacing a photo.** An open photo stays up while the new request looks
(`openNearest` `replace: true`; the viewer is replaced, not closed, so EXPAND
stays, also across sources through `holdViewer`, which holds only the action's
own switch-off of the photo's source: a source switched by hand meanwhile
still stops the load) and closes if nothing opens, so a place with no imagery
shows its own error; a link that only arms closes it at once. A replacement
from another source drops the old photo's details, and the request's error or
armed OPEN outlives the old viewer. A replacement stops a playing Mapillary
sequence.
- **Outcome.** `noImagery` marks an empty result, `imageryError` a source or
viewer failure and `aim: 'sky'` a sky centre (from `street.nearestOutcome`);
an empty result names a narrowing 360°/flat or SINCE filter the source
honours, as `filter`. Every photo the action opens reveals the panel, also
one the user had collapsed.
- **Search.** A source without a coverage layer searches up to 250 m
(`NEAREST_RADIUS_MAX_M`; Google honours it); clicks keep 50 m. Mapillary
asks for 100 photos within 50 m, its API's most (the API returns them
unordered and cuts at the limit), and gives up after 12 s, body included
(`NEAREST_TIMEOUT_MS`).
- **Phone.** The panel sits in the Cameras tab sheet: a photo that opens, or a
link that arms, opens the sheet and scrolls the panel into view
(`revealPanel`, which does nothing above 720px).
The layer also exposes `requiredMapStack(id)` and `providerStatus(id)` for
callers that decide before the panel lists a source, and `holdViewer()`. A
caller's abort is honoured through the viewer's mount and
load, and only the lookup that still owns the panel changes its spinner or
error.
Views carry the request as `street_level` (`src/view/index.js`): a source and
a point, which defaults to where the camera looks (below 20 km, at a pitch
steeper than -5° and within 10 km ahead; else the camera's own point); half a
point is not mixed with the camera's: both coordinates come from that default. A `google` source brings `map: photoreal` and refuses
another map; `street_level` does not combine with `follow`.
`show_in_gods_eye_view`, and the embedded app for a posted view with no
point, measure the view's altitude from the terrain under the camera
(`groundedLookAt` with the terrain heights service) before taking the look-at
point, and keep the view's own estimate without a terrain answer (the embedded
app waits 2 s for one). Links carry
`sl=<source>,<lat>,<lon>`; the globe panel applies views through the action,
and a plain link runs it once restored (one shot: the rewritten hash drops
`sl`). A link that follows an entity ignores `sl`.
- **Billing gate.** A plain link never loads a photo from a source that bills
every load (`billedPerImage`; Google): the action runs with
`autoOpen: false` and arms the layer (`armNearest`: the link's point and
radius, for the next OPEN press only, and only while the ground under the
screen centre is within 1 km of it (`ARMED_REACH_M`) or cannot be picked; a
sky centre refuses; any other lookup, open, close, disable, the source
switched off or its map leaving drops it), opens the panel and shows a toast
pointing at OPEN GOOGLE STREET VIEW. A link whose source has no working
key here changes nothing and says so in a toast (not in a frame).
- **Who may open.** Posted views, and the panel page's own link (inline embed)
for a tool result issued in the last 5 minutes, open the photo:
`show_in_gods_eye_view` stamps a street-level result `issued_at`, every
`gev:view` the panel posts carries `fresh`, and the inline app applies a
view without it with `autoOpen: false` (a framing page's posted views still
open). `?embed=1` on a published link only arms, without the panel nudge or
toast.
- **Who owns the camera.** A link's photo step is skipped when a newer link
superseded its restore, when its restore flight never landed
(`share.camera !== 'applied'`: the visitor dragged or searched), or after
any navigation since the generation that flight owned (`navigationToken` on
the settled result), also while its layers still load. A link pasted into
the open tab runs its step after its restore settles
(`gev:share-navigation-restore-settled`).
- **Embed.** The rail is hidden, so an open photo moves to an overlay on the
globe (`sl-viewer-wrap-embed`) and back when it closes; Esc closes it (in
full screen, Esc leaves full screen first). EXPAND stays under embed mode's
clean view; cockpit, recording and scene playback hide the overlay. Loading
errors render inside the overlay, and inside the viewer while EXPAND is on.
A posted view whose photo step fails before the imagery stage closes the
earlier photo before its camera flies; one the user overtook (a click on the
map, a newer request) leaves their photo and camera alone; one that found a
photo it could not load keeps that error.
Street Level has two browser gates. `npm run qa:street-level -- --url <server>`
runs against real Mapillary and needs `MAPILLARY_CLIENT_TOKEN`. `npm run
qa:street-level:fixtures -- --url <server>` answers every Mapillary request
@@ -5351,7 +5691,14 @@ service's usage policy states; past a bounded queue the answer is a 429 with a
`Retry-After` rather than a queue that grows until everything times out. The
upstream host is pinned and a redirect is refused rather than followed, and a
rate limit from the routing service is reported as one instead of as "no route
found". `src/data/routeSteps.js` turns OSRM maneuver type / modifier / exit /
found". Upstream failure statuses except 400 and 429, network failures, and
unreadable, oversized or non-JSON bodies return `502 routing service unavailable`.
Voice hears `Routing service unavailable (HTTP 502)`; Directions shows
`Routing unavailable (HTTP 502)`. OSRM 400 responses or successful responses
without a usable route return "no route found". The 12 s upstream budget includes
queue time: expiry before dispatch returns 429 "routing busy" when the queue slot
opens; expiry during the upstream request returns 502.
`src/data/routeSteps.js` turns OSRM maneuver type / modifier / exit /
road name into one plain-English sentence per decision, folding
`exit roundabout` steps into the roundabout they leave, and caps one route at
200 maneuvers — a route with more is reported as cut off, in the row summary
@@ -5373,7 +5720,7 @@ answered yet opens nothing rather than a card hanging at sea level. The cells
are re-read for up to 90 seconds, which outlasts a cold terrain round trip;
reroute, CLEAR and disable cancel that.
Transit is off by default, and share links use token `j`. The movement panel order
places Transit directly after Street Traffic and before Bike Share. Cameras and
places Transit directly after Street Traffic and before Rail. Cameras and
utilities retain their own groups. The layer polls
registered feeds near the camera every 15 seconds, below a 3,000 km altitude
gate. Forty-nine keyless regions are available: Boston, Austin, Minneapolis–St Paul,
@@ -5598,6 +5945,279 @@ refreshes. A 304 confirms contact without appending history. Missing vehicles
are removed after two missed polls; reports older than ten minutes and feeds
silent for five minutes are removed. Credits appear when the feed is used.
**Rail.** Rail is off by default, and share links use token `02`, with five
boolean options for its categories: `t` (urban), `l` (local), `r`
(regional), `n` (national) and `i` (international), all on by default. It sits
in the Movement panel directly after Transit and before Bike Share. The layer
lives in `src/layers/rail/` and is built by `src/app/layers/rail.js`; feeds are
registered in `src/data/railFeeds.js`.
Every train shown is live: it is where its operator last reported it, by GPS
or, for Irish Rail, by the station or timing point the signalling last
reported. No train is placed or moved from a timetable, and a feed whose live
positions fail shows as unavailable while its trains age out as any quiet
train does. Below a 3,000 km altitude gate (with 150 km hysteresis), the layer
polls every 15 seconds each listed feed whose coverage circle contains the
centre of the view; a feed already polled stays active 40 km past its edge. A
feed's trip updates are polled every 30 seconds while one of its trains is
selected and its schedule pack is loaded. The registry lists 30 feeds, all
switched on and all keyless: every realtime feed and every static timetable
is published for anyone to fetch, so no feed needs a key, an account or a
registration, and no rail request carries a credential.
**Opt-in packs.** Live feeds work out of the box, and nothing is downloaded
when the app starts. A path pack and a schedule pack, built only when someone
runs `scripts/build-rail-paths.mjs` for a feed (Packs below), are an extra for
its live trains: the operator's own route lines, stop names, the next stop
and, from the trip updates, the delays at each stop. Renfe long and medium
distance and KTMB, whose vehicles name no route, are classified from the
pack's routes (`railFeedNeedsSchedulePack`), so `/api/rail/feeds` lists them
only once `<id>.schedule.json` is in the pack directory (`publicRailCatalog`).
The server checks that the file exists, never reading it for this, and keeps
each answer for 60 seconds; the catalog itself is sent with `max-age=300`. So
a host that has built nothing shows 28 feeds, in 22 regions, and the other two
appear as their packs are built. The browser reads the catalog once per layer
instance; until it answers, or while it fails (asked again after 30 s,
doubling to 10 minutes), only the feeds that need no pack are polled. A feed
the catalog lists without live positions is never polled.
Five enabled feeds publish no licence for their train positions and are used
on the basis their entries record, which is never a licence and never
credited as one: MARC on what its publication says, and METRO light rail on
the basis the Transit layer uses for the same file. Tri-Rail and Charlotte's
LYNX are used without a published licence by project owner decision of
2026-11-10. The fifth is Irish Rail, whose train positions publish no licence
while its NTA timetable is CC BY 4.0. The
Montpellier tram's GTFS is ODbL, so the packs built from it are ODbL too: a
deployer who serves them publicly makes them subject to the same licence,
offered under it with the attribution kept, and France's national access
point adds that a re-shared derived database must be published on
transport.data.gouv.fr in its original format.
By position source, 29 feeds publish live GPS. 13 are in the US: Metro-North,
LIRR, MBTA, SEPTA Regional Rail and trolleys, MARC, Tri-Rail, RTD, CapMetro,
METRO, Charlotte's LYNX, the Seattle Streetcar and Cincinnati's Connector. Three are in Australia: Citytrain, G:link and Adelaide Metro. The
remaining 13 are GRT ION in Canada, VR in Finland, Renfe long and medium
distance and Cercanías in Spain, the four Entur operators in Norway (SJ Nord
through SIRI VM), KTMB in Malaysia, the Bordeaux and Montpellier trams in
France, and the Manyōsen and Toyama trams in Japan. Irish Rail publishes the
station or timing point the signalling last reported.
22 feeds register trip updates; on a feed whose pack is built their delays
reach the selected train's stops. Irish Rail and SJ Nord publish none, but
each train's report states its delay: with the pack built, that delay moves
the train's stops and the run its report is matched to, in the browser and,
for Irish Rail's train codes, on the server, so a late train is not shown
past a stop it has not reached. `DATA_SOURCES.md` lists every feed, one row
each, with its licence, terms and attribution.
**Proxy.** `GET /api/rail/feeds` returns the public catalog of listed feeds,
`GET /api/rail/trains/<feedId>` one feed's snapshot,
`GET /api/rail/updates/<feedId>` its trip updates, and
`GET /api/rail/paths/<feedId>` and `GET /api/rail/schedule/<feedId>` its path
and schedule packs once built (`src/sources/railService.js`,
`src/data/railProxy.js`, served in development and preview by
`server/providers/rail.js`, which reads the packs from `.cache/rail-paths/` or
`RAIL_PATHS_DIR`). The request policy is the Transit proxy's: a memory cache
with single-flight refresh (15 s, or the feed's own poll interval; 30 s for
trip updates, or the feed's own), serve-stale for up to 10 minutes after a
failure (an empty or header-less GTFS-Realtime reply is refused as unreadable
and counts as one: stale, or 502 without a snapshot), per-feed and global
admission limits, the failure cooldown ladder, redirects followed hop by hop
only on the feed's own https origin, and an 8 MiB body cap unless a feed sets
its own, at most 64 MiB. A feed whose positions or trip updates can be read
only with its schedule pack (routes taken from the pack) answers
503 "Rail schedule unavailable" (`X-Rail-Backoff: schedule`, never cached)
whenever every record it sent was dropped, rather than an empty list: the
pack is missing or refused, or names none of the trips the feed runs now (a
stale pack: Renfe's trip ids carry their date). The server refuses a schedule
pack, as it does one it cannot read, when it is of another format version or
another feed, or once the last run it holds has arrived (its window has
ended), and reads it again after five minutes. A feed that needs
its pack to be listed is served only once that pack is built: until then
`/trains` and `/updates` answer 503 "Rail schedule unavailable"
(`X-Rail-Backoff: schedule`) without contacting the operator. Without a
`hasSchedulePack` probe, the server asks `schedulePackVersion` or
`readSchedulePack` (null: not built); with neither, no such feed is listed.
Only a missing pack is 404: a pack that cannot be read answers 503 with
`Retry-After: 60`. A pack is sent `no-cache` with an ETag from its file's
modification time and size, and a request whose If-None-Match names it is
answered 304 without reading the file: a feed's path and schedule packs pair
by `builtAt`, so a browser never keeps one of them past a rebuild. The browser asks again for a path or schedule pack after
30 s, doubling to 10 minutes. LIRR and Metro-North, whose positions and trip
updates are one document, are fetched once per freshness window for both.
Each feed's classifier runs on every decoded vehicle; only trains leave the
server, each tagged with a `category` (`transit`, `local`, `regional`,
`national`, `international`) and a `mode` (`rail`, `metro`, `tram`).
Everything else, buses in a mixed feed included, is dropped and counted as
`dropped`. There is no disk cache and no history.
**Packs.** `scripts/build-rail-paths.mjs` builds, for each feed it is given
that has a static GTFS (29 of the 30), a path pack (`<id>.json`: the
operator's track shapes) and a schedule pack (`<id>.schedule.json`: routes,
stops, stop patterns, run-time profiles and the service dates of a window) into `--out`, else `RAIL_PATHS_DIR` when the build's own environment sets it (the folder the server reads), else `.cache/rail-paths/`, which is gitignored. A
build names its feeds (`--feed <id>`, repeatable) or passes `--all` for every
enabled feed (many downloads and a long run); a bare run exits 1, and
`--help` prints the usage. A build in which any feed fails exits 2 once the
others are built (1 stays for a build that cannot start: a bad argument, or
another build holding the lock). `--out` may not be `src/` or `public/` or a
folder inside them. The window is four service dates by default (yesterday,
today and the next two days), 1 to 14 with `--days`, starting at `--from`.
`--local-gtfs <id>=<zip>` builds a feed from a zip already on disk instead of
downloading it, a copy of the GTFS the feed publishes; the zip is only read,
and the packs record its file name, not its path. The server serves a pack of
up to 24 MiB, passed through as bytes from a 64 MiB in-memory read cache keyed
by the file's modification time and size (`server/providers/rail.js`), and
answers a larger one with 503 "Rail paths too large" or "Rail schedule too
large". The build refuses to write a pack over that cap (the feed counts as
failed and its previous packs stay), and a pack whose trips fall under a fifth
of the pack it would replace, or to none, which reads as an upstream fault,
unless `--force` is given. A feed's two packs are written together: to
temporary names, then renamed into place, the path pack first, only once both
are written; with `--no-schedule` the feed's old schedule pack, whose patterns
index the old path pack, is deleted, and a feed named only through its
schedule pack (Renfe long and medium distance, KTMB) is refused and counts
as failed. A lock file in the `--out` folder
(`.build-rail-paths.lock`) makes a second build there refuse to start. A
schedule pack records the `builtAt` of the path pack its shape indices refer
to (`pathsBuiltAt`); where the two no longer match, its shape indices are not
used. Every download is open and made over https only (a redirect to plain
http is refused), capped in size and time, with the script's User-Agent and
any identifying header the operator asks for (`staticHeaders`). Shapes are
taken at face value: the operator's `shapes.txt` as published; nothing is
redrawn from other map data, and no line is drawn where the operator
publishes none. A shape with a gap of more than 5 km between two points is a
chord, not track, and is dropped; a trip without a usable shape gets none, and
its train moves between reports in straight lines. A feed's
`staticRouteTypes` lists the route types it keeps when its classifier accepts
every route of a mode-specific feed. A frequency-based trip
(`frequencies.txt`) becomes one run per departure, `<trip_id>@<HH:MM:SS>`, on
its template's stops and run times. A stop the GTFS gives no coordinates,
which some feeds write as (0, 0), is left out of the run with its times, so no
line is drawn to it; a run needs two located stops. The Toyama tram has no
static GTFS registered, so it has no packs and its trains move in straight
lines between reports.
**Duplicates across feeds.** Where loaded feeds carry the same train,
`src/layers/rail/dedupe.js` draws it once. Copies from different feeds match
by train number within 100 km of each other. A train number is three or more
digits, or a shorter number after a long-distance product (EC, ECE, EN, ICE,
NJ, RJ, RJX, TGV; not IC, where a short number names a line); a trip id that
is an operator prefix and a number, or a headsign that is a bare number, is
read too, and numbers that differ are different trains. A short line name
(S1, RB 1, U6) names a line many trains run: copies on different lines are
different trains, and copies on one line match only in the same category
within 15 km, the nearest pairs first. The copy drawn is scored by its
position source (live GPS, then station reports), then by its line (bound to
its operator's shape in a built pack, then without one). A drawn copy stays
until another scores strictly higher, and the comparison runs once per poll,
never per frame.
**Ukraine.** No train is placed in Ukraine, a safety rule
(`src/data/railNoPositions.js`: its recognised borders as mapped in
OpenStreetMap, via geoBoundaries, simplified to about 50 m, and stop ids that
name the country). The path build leaves out every run with a stop there: a
stop whose id names Ukraine or whose position is inside its borders, either
being enough, so an id naming another country does not keep a stop that
lies inside. The proxy drops every live position inside it, for every feed.
**Categories and markers.** Each train is a disc in its category colour (urban
`#FF7A59`, local `#F2D04B`, regional `#7BE495`, national `#5AB8FF`,
international `#C58BFF`) with a white glyph for its mode: a train front, a tram
with a pantograph, or an "M" for metro. The layer row shows one chip per
category, which hides or shows that category, and a colour legend with live
counts for the categories present. On photorealistic tiles a train's height
is sampled from the rendered mesh at points on its own line
(about every 55 m, for trains within 8 km of a camera at most 5 km above the
ground there, and only once the tiles in view have loaded), where the draped
line is the track bed or the deck. A reading more than 25 m under or 80 m over
the floor there is a coarse tile and is dropped; a kept one is read again
after 30 s, and the new reading replaces it when the camera is no higher than
it was, or once the old one is five minutes old. Elsewhere it stands on the
shared ground floor that Transit and aircraft use, and on a floor cell not
sampled yet it keeps a neighbouring cell's floor or its last one. Only a train
with no floor yet, or a stop of the selected trip, stands on the ellipsoid,
and only until its cell is warm: the stops are read again every second until
they stand on a floor. Rail warms those floor cells on one shared budget that
refills at 1,200 cells a minute and holds at most 320, so no minute spends
more than 1,520 cells (about 24 of the 90 requests of 64 points that
`/api/terrain/heights` allows a client each minute), and only under trains
within 30 km of the camera and at the selected trip's stops, the selected
train and its stops first and then the nearest, in one batch a second, so a
view full of trains cannot use up the limit every layer shares. Its drawn
height changes no faster than 5% of the distance it moves (plus 0.5 m/s while
it stands), so a roof or overpass sampled under it is a bump, not a jump, and
a followed train rides smoothly; it snaps when a better height replaces a
fallback (a real floor, or any floor replacing the ellipsoid) and after a jump
of more than 2 km.
**Playback, lines and cards.** Trains use the Transit layer's delayed
playback between reports, so a train glides at 1× behind real time rather
than jumping. Between reports they move along the line in the feed's path
pack where there is one, else in straight lines. A loaded feed's faint
network is drawn in full below 150 km and not at all above 200 km. A feed
out of use for ten minutes gives back its network, path pack and schedule
pack; they load again when it is next used. The selected train's line, its
operator's shape from the path pack, is drawn ahead of it and, fainter,
behind it. Clicking a train opens a card titled with its route or headsign,
with at most six lines under it: category, mode, feed and region; displayed
motion and position source ("Live GPS" or "Estimated from station reports");
the next and previous stops with their times and delays where the schedule
pack gives them; report age (marked "feed time" when the feed stamps the
header rather than the train) and display delay; reported speed and heading;
the last reported stop status where no stops are known; and the vehicle id.
Metro trains are shown at street level, as in Transit, and the card says so.
**Camera.** Selecting a moving train gives it the camera as selecting an
aircraft does: Cesium's tracked entity rides on the train's marker, framed
2.5 km out and 35° down along the camera's heading (`WORLD_FOCUS_FRAMING.train`),
and a drag or a zoom orbits or zooms around the train. A train counts as
moving while it glides between reports, waits at a stop with its departure
already reported, or waits for its next report after its last two moved it
at least 15 m. Any other train (parked, known from one report, or not heard
from for 90 seconds) gets one fly-to with the same framing, as a vessel does;
in the Cockpit that fly-to is refused with "Exit cockpit to fly to a train".
The follow lets go, leaving the camera where it is, when the train is
deselected (a click on anything else, Escape, its category hidden, the train
leaving its feed, the layer turned off) or another train is selected, which
takes the camera in turn. An aircraft or satellite tracked by another layer
clears the selection too; a navigation that only releases the follow camera
(a search, a fly-to) keeps the train selected. When the duplicate filter moves
the selection to another feed's copy of the train, the follow stays and
glides to the new copy over 1.2 seconds.
**The selected train from far out.** While a train is selected its feed stays
polled at any altitude and wherever the camera looks, so the train, its line
and its stops stay on the map from orbit. While nothing else would keep that
feed loaded, it carries the selected train alone: its other trains and its
faint network go as they would with the feed. On the far side of the globe
the selected train and its stop dots are hidden. Deselecting returns the feed
to the usual rules.
**Limits.** A train a feed stops listing for two polls is removed, as is any
report older than ten minutes; a train without its own timestamp is aged from
the snapshot's fetch time, so a frozen feed empties rather than showing old
trains as live. A feed whose operator has not answered for 90 seconds reads as
stale, and its trains are removed after five minutes. The layer holds at most
5,000 trains drawn (a copy hidden as another feed's does not count); past that
the layer row reads PARTIAL. Rail is not voice-controllable yet: it is listed in
`VOICE_OFF_LAYERS` for prompt budget, and the "trains" alias still names
Transit. Credits appear when a feed's trains are shown.
**Upkeep.**
- Packs cover a fixed window of service dates, so they are rebuilt before it
runs out: daily at the default four days, or weekly with `--all --days 7`
from a `--from` date that starts the coming week. The window and build time
of a pack are in its `window` and `builtAt` fields.
- After a rebuild, the server picks up the new pack within a minute: it
checks each schedule pack's file time and size once a minute and rebuilds
what it reads from a changed pack (a host with its own `readSchedulePack`
and no `schedulePackVersion` keeps it up to an hour). A feed whose
first pack was just built is listed within a minute; a browser that has
read the catalog sees it after a reload, once its five-minute cache has
passed.
`src/data/militaryAwareness.js` remains registered internally as the Contacts
coordinator, but it is not a user-visible Data Layers entry. Its visible entry
point is the right-side `CONTEXT` chooser's `CONTACTS` mode.
@@ -5965,6 +6585,10 @@ silently demoting every later lookup for the session.
far cap, retargeting the existing video/canvas texture pipeline. Stills are drawn at their source aspect, centered on the placeholder dark (`#01070c`), instead of being stretched to the 16:9 canvas. Manual calibration only —
auto-calibration and the drape mesh pipeline are deleted. A one-shot activation obstruction
probe (`pickFromRay` on camera activation, clamping the plane short of the first hit) remains;
a hit at or beyond the camera's range counts as a miss, and on a globe map the plane also clamps
short of the globe's terrain along the axis (flat ground at the mount's ground when terrain is
missed or met beyond the range), whichever is nearer, so it never stands at the full range there
(about a kilometre wide and hundreds of metres up);
ground placement is superseded by the shared-floor v3 behavior below. Calibration persists to
`godsEyeView.cctv.calibration.v2` (wiped clean, no v1 import); a panel-only CAL badge shows
`CALIBRATED`/`CURATED`/`RAW PRIOR` (no in-world tint). Panel is titled "CCTV" (not "CCTV
@@ -6322,7 +6946,7 @@ and unreachable upstream (502/504) separately from road geometry.
- **Token flow**: browser fetches a short-lived client secret from `/api/realtime/token`; the Vite middleware uses `OPENAI_API_KEY` by default, or the signed-in local ChatGPT/Codex OAuth access token when OAuth was explicitly selected in Provider Settings, and posts the full session config (instructions, tool schemas, VAD, truncation) to `api.openai.com/v1/realtime/client_secrets`. Both long-lived credentials stay server-side; OAuth minting is loopback-only. SDP exchange goes directly to `api.openai.com/v1/realtime/calls` with the ephemeral token. With no OpenAI key the token route answers `503` with code `API_KEY_MISSING`, and the voice control shows `Needs OPENAI_API_KEY — add it in Provider Settings` in the interface language, reworded on a language change, without the microphone-permission hint.
- **Session defaults** (env-tunable): model `gpt-realtime-2` (or `gpt-realtime-2.1-mini` when the MINI tier is selected — see the model-tier entry below), voice `marin`, reasoning effort `low`, semantic VAD with low eagerness, no response interruption, context window truncated to ~3,000 post-instruction tokens with 0.5 retention ratio — the conversational window stays short because map state is fetched live per turn.
- **Thirty tools** (argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `next_iss_pass`, and `next_satellite_pass`.
- **Thirty-five tools** (thirty-four in a session without `OVERPASS_UPSTREAMS`, which adds `osm_query`; argument schemas in `src/voice/actionSchemas.js`, served with their descriptions by `server/providers/openai/tools.js`, executed client-side in `src/voice/gevActions.js`): `fly_to_location`, `select_nearest_aircraft`, `show_street_level`, `adjust_camera_zoom`, `zoom_to_globe`, `set_layer_visibility`, `show_data_layers_menu`, `set_panel_open`, `set_voice_inactivity_timeout`, `set_visual_style`, `get_entity_context`, `get_current_view_state`, `set_hud`, `set_cyber_sonar`, `set_detection`, `set_map_stack`, `set_post_processing`, `control_scene`, `control_cctv`, `set_context_mode`, `control_cockpit`, `control_radio`, `track_entity`, `stop_tracking`, `frame_overhead`, `annotate_map`, `clear_annotations`, `move_camera`, `fly_route`, `analyst_query`, `resolve_area`, `find_imagery`, `osm_query`, `next_iss_pass`, and `next_satellite_pass`.
> **Reading `npm test` totals:** the count depends on the Node major. The two
> GC-bracketed allocation microbenchmarks (`src/data/focusAllocations.test.mjs`
@@ -6595,7 +7219,8 @@ request viewbox and continues searching without that geographic preference.
### News (November 2026)
- **What it shows.** Events ▸ **News**: the last 24 hours only, or the fewer hours a server names. Zoomed out, one beacon per hotspot with its count for the window and its trend, and a breathing beacon per developing situation (still under reduced motion). Below 2,500 km (7,500 km with a hotspot chosen), the hotspot outlines with disputed areas dotted and named, and every placed report as a ground-clamped marker: fill is severity (0 grey no physical event, 1 yellow minor, 2 orange significant, 3 red major), the icon is the category, the rim is attestation (solid confirmed, dashed claim, thin single), size grows with sources and the marker fades over the day. Colour never marks a side. Up to ten picture cards (physical events with two or more sources, major or pressing) stand over their markers without overlapping. A selected report's card reads "Source: host ↗" and opens the article.
- **What it shows.** Events ▸ **News**: the last 24 hours only, or the fewer hours a server names. Zoomed out, one beacon per hotspot with its count for the window and its trend, and a breathing beacon per developing situation (still under reduced motion). Below 2,500 km (7,500 km with a hotspot chosen), the hotspot outlines with disputed areas dotted and named, and every placed report as a ground-clamped marker: fill is severity (0 grey no physical event, 1 yellow minor, 2 orange significant, 3 red major), the icon is the category, the rim is attestation (solid confirmed, dashed claim, thin single), size grows with sources and the marker fades over the day. Colour never marks a side. Up to ten picture cards (physical events with two or more sources, major or pressing) stand over their markers without overlapping. An app built on Core may draw the picture cards itself: `createApplicationCatalog({ newsOptions: { pictureCards } })` calls `pictureCards(cards, { selectedId })` on each redraw with one card per event on the map that has a picture, single source or not, at every height (zoomed out, where a hotspot stands as one beacon, its reports' pictures still come), most severe then newest first (`{ id, title, lat, lon, publishedAt, severity, color, image, video, credit }`, at the marker), and the selected report's id or null (its own card still stands at its marker); with no cards when the layer turns off; the layer then paints none of its own, and a click on one is `setParams({ reportId, focus: true })`. A selected report's card reads "Source: host ↗" and opens the article.
- **Time.** The layer reads its time from the world clock (`worldClock.timeFor('news')`, passed as `now` in `src/app/constructCatalog.js`): report ages, the 24-hour window, "today", fading and the pressing and card rules all count from it. Core does not follow the clock for the layer, so this is the wall clock and nothing changes. A host whose news source can answer a past time may follow it for `news`; the layer then shows that time's news as it was then, with ages counted from the timeline.
- **Panel.** The right-rail NEWS panel (`src/ui/newsPanel.js`, `src/ui/news/`) has a time window at the top (the server's 24 h, or 12, 6, 3 or 1 h; the hotspot counts are read for the chosen window) and two sections. **Conflicts**: hotspot chips with counts, a severity floor (significant and above by default), a category filter, the chosen event, and a continuous feed, newest first. **Local**, while the camera is below 3,000 km: every report in the area at the view centre, newest first, named at the top ("News from …") and outlined on the globe, which then shows those reports instead of the hotspots' news; at sea, reports within 500 km. A disputed area of the bundled list (11, see `src/data/local_data/natural_earth/README.md`), or a territory Natural Earth gives no country code (Somaliland, Northern Cyprus and others), is named as itself and read from no country's outlets; elsewhere Natural Earth's default country lines apply. Each card shows the publisher's picture when the event has one (hotlinked, no referrer, publisher credited), the headline (a model's English headline marked as such), a mono meta line (severity marker, category, place, sources, age), who reported it, attestation and media chips, and the service it came through ("via GDELT Project"). Open, a card plays the post's clip on request. A click flies the camera straight down from 150 km and selects the report.
- **Notices.** A section whose key the server lacks says "{provider} key required" and names the variable; a refused key says "{provider} refused its key"; a spent daily allowance says "{provider} limit reached for today". The other section keeps working.
- **Source.** `sources.news` (`createNewsSource`, export `./layers/news/source`) reads same-origin `/api/news/*` with a 15 s timeout, a 4 MiB cap and record-by-record validation; `newsQueryAllowed` lets a host admit exactly the client's requests. Core's local server (`server/providers/news.js`) forwards only those GET requests to `GEV_NEWS_ORIGIN` when it is set.
@@ -6761,7 +7386,7 @@ apply to `public/` assets or built-preview serving.
- `/api/google/text-search` keeps the Google key server-side for view-biased Places recovery used by annotation resolution.
- `/api/overpass` is bounded by body/response caps, per-client/global rate limits, concurrency limits, operator-configured upstreams, in-flight dedupe, cache bounds, and static validation that every selector is spatially bounded.
- `/api/military-installations` uses an independent limiter with the same 90-per-client/300-global one-minute bounds, so viewport installation refreshes never consume `/api/overpass` annotation capacity.
- `/api/route` proxies bounded OSRM route requests for annotation routes, with profile allowlisting, distance caps, response caps, caching, and sanitized "no route found" errors.
- `/api/route` proxies bounded OSRM route requests for annotation routes, with profile allowlisting, distance caps, response caps, caching, and sanitized "no route found" errors; routing outages return 502.
- `/api/flights`, `/api/military`, and `/api/route` answer `405 Method Not Allowed` with `Allow: GET` to anything but `GET`, before rate limiting and any upstream call.
- `/api/flights` negotiates gzip. The cached global snapshot and the regional adsb.lol fallback are each gzipped once per refresh (about 860 KB becomes about 250 KB). Clients that offer gzip get that frame; other clients, and bodies under 1 KiB or that gzip cannot shrink, get identity bytes. Both answers carry `Vary: Accept-Encoding`. Uncached bodies, such as auth failures, are never sent under a cached frame's encoding.
- Track endpoints: `/api/vessels/track?mmsi=` (server-accumulated ring buffers; sub-route handled before the rows snapshot), `/api/flights/track?icao24=` (OAuth, 60s cache, sanitized errors, independent OpenSky credit bucket), `/api/military/track?hex=` (400 for an invalid hex, otherwise `503 history_unavailable` with `Retry-After: 1800` and `Cache-Control: no-store`, no upstream request; military tracking uses its locally accumulated trail).
+183
View File
@@ -0,0 +1,183 @@
# Data Library
DATA LAYERS is the curated set of core layers. The Data Library holds
everything else: regional sources, niche themes and community contributions.
A viewer opens it from **DATA LAYERS ▸ Browse data library**, adds what they
want, and the entry appears in the panel under **Library**. Nothing in the
library is on until someone adds it.
Runtime behaviour is described in [CURRENT-STATE.md](CURRENT-STATE.md#data-library--october-21-2026).
This page is the authoring guide.
## The one rule: live sources only
A library entry points at data its provider hosts. That can be a query API, a
static file at a stable URL the provider controls, or a tile endpoint. The
data is never copied into this repository. If the upstream goes away, the
entry goes stale; this project never becomes the host of record. That is what
lets every self-hosted instance serve the whole library with no data of its
own.
It also keeps the existing rule in [CONTRIBUTING.md](../CONTRIBUTING.md): do
not add data you have no right to redistribute. A live source still needs a
licence that allows this use, and the manifest must say what it is.
## Kinds
| Kind | What it is | Renders through |
| --- | --- | --- |
| `dataset` | Vector features: `geojson`, `geojsonl`, `csv` (lat/lon columns), `json` (a record list with lat/lon fields), `arcgis` (a FeatureServer or MapServer layer, paged) or `geoparquet` (one file, or a partitioned set on S3) | The GeoJSON layer Datacenters and Dams use; view-bounded and GeoParquet sets draw as ground-clamped dots and polygons |
| `tiles` | Raster tiles: `xyz` with a `{z}/{x}/{y}` template (`{reverseY}` for TMS rows), or `bbox` with a `{bbox-epsg-3857}` template (WMS GetMap, ArcGIS `export`/`exportImage`) | The active map's imagery host (globe or Google 3D tiles) |
| `pack` | A source pack an existing host layer already loads (today: CCTV) | The host layer, switched on by its own environment variable |
A `dataset` or `tiles` entry needs no JavaScript. A `pack` documents a host
layer's opt-in source so viewers can see it and operators know the switch; the
host layer owns its code. Provider entries (code implementing a family
contract, such as a Street Level provider) are not supported yet and are
refused by the validator.
## Add an entry
1. Create `src/library/entries/<id>.json`. The id is 3–48 lowercase letters,
digits or dashes and is permanent.
2. Add one import line for it in `src/library/catalog.js` (keep the list in
file-name order).
3. For a `dataset` or `tiles` entry, run `npm run library:next -- <id>` and
append `["<id>", "<token>"]` to `src/library/ledger.json`. Tokens are
permanent: never change or reuse a row, even when an entry is removed.
Packs take no token.
4. Run `npm run library:docs` to regenerate the DATA_SOURCES.md table.
5. Run `npm run library:check -- --base-ref origin/main`,
`npm run library:probe`, `npm test` and `npm run qa:data-library`.
6. Open a pull request with a `changelog.d/` fragment.
Start new entries at `"status": "community"`. Maintainers move an entry to
`verified` after reviewing its licence and seeing it stay healthy; `stale` and
`archived` mark entries whose upstream has stopped answering. Archived entries
are hidden from Browse unless a viewer already added them.
## Manifest reference
```json
{
"id": "eia-power-plants",
"kind": "dataset",
"family": "infrastructure",
"title": "US power plants, 100 MW and up (EIA)",
"provider": "EIA",
"description": "Operating US power plants with at least 100 MW of capacity.",
"icon": "ϟ",
"maintainer": "github:your-handle",
"coverage": { "countries": ["US"], "bbox": [-180, 13, -64, 72] },
"source": {
"format": "arcgis",
"url": "https://services2.arcgis.com/…/Power_Plants_in_the_US/FeatureServer/0",
"query": { "where": "Total_MW >= 100", "outFields": "Plant_Name,PrimSource,Total_MW" },
"fetch": "browser-direct",
"cacheSeconds": 86400,
"maxFeatures": 4000,
"maxPages": 3
},
"display": {
"color": "#ffc23d",
"label": "Plant_Name",
"fields": ["PrimSource", "Total_MW"],
"rank": { "field": "Total_MW", "order": "desc" }
},
"licence": { "name": "EIA data: public domain", "url": "https://www.eia.gov/about/copyrights_reuse.php" },
"attribution": { "text": "U.S. Energy Information Administration", "url": "https://atlas.eia.gov/" },
"status": "community",
"reviewed": "2026-10-21"
}
```
Unknown fields are errors, so a typo cannot silently drop a cap or a licence.
| Field | Required | Notes |
| --- | --- | --- |
| `id`, `kind`, `family`, `title` (≤ 60), `provider` (≤ 32), `description` (≤ 240), `maintainer`, `status` | yes | `family` is one of movement, cameras, infrastructure, events, weather, environment, imagery. `provider` is the short name the panel row shows. |
| `coverage` | yes | `global: true`, ISO 3166-1 alpha-2 `countries`, and/or a `bbox` `[west, south, east, north]`. |
| `source.url` | yes | https, a real hostname, no credentials or fragment. |
| `source.format` | yes | Datasets: `geojson`, `geojsonl`, `csv`, `json`, `arcgis`, `geoparquet`. Tiles: `xyz` (`{z}`, `{x}` and `{y}` or `{reverseY}`) or `bbox` (one `{bbox-epsg-3857}`, filled with the tile's Web Mercator bounds in metres, as MapLibre does). Packs: `host`. |
| `source.records` | no | JSON only: the property path to the record list, such as `stations` for `{ "stations": [ … ] }`. Omit when the response is the list itself. |
| `source.fetch` | datasets, tiles | `browser-direct` when the upstream sends CORS headers and needs no key; otherwise `proxied`. |
| `source.query` | no | Query parameters added to the URL. For `arcgis`, `where` and `outFields`; geometry is always WGS84 GeoJSON. |
| `source.columns` | no | CSV or JSON: `{ "lat": "…", "lon": "…" }` when the fields are not lat/latitude/y and lon/lng/longitude/x. |
| `source.cacheSeconds` | no | 60–86,400, default 3,600. How long the proxy keeps a response. |
| `source.maxBytes` | no | Body cap, default 16 MiB, at most 32 MiB. |
| `source.maxFeatures` | no | Feature cap, default 10,000, at most 20,000. A capped dataset reads PARTIAL. |
| `source.maxPages` | no | ArcGIS pages to follow, default 5, at most 10. |
| `source.view` | no | `{ "maxAltitudeM": 300–50000, "sample": [w, s, e, n] }`. Makes a huge ArcGIS layer view-bounded: below that camera height it loads what is in view, one page at a time; above it the row asks the viewer to zoom in. Needs `arcgis` or `geoparquet`, and `browser-direct`. `sample` is where Zoom to layer lands and what `library:probe` queries. |
| `source.partitions` | no | GeoParquet only: `{ "listUrl": "<S3 ListObjectsV2 URL with prefix>", "keyBase": "<URL keys are relative to>" }`, both on the source host. Needs `source.view`. The library indexes each file by its footer bbox and reads only the files in view. |
| `source.minZoom`, `source.maxZoom` | no | Tiles only, 0–22. |
| `source.encoding` | no | Tiles (`xyz`): `terrarium` when the pixels are elevation as terrain-RGB (height = R × 256 + G + B / 256 − 32768 m). The browser draws them as shaded relief: one orange ramp in six elevation classes (legend in metres), lit from the north-west; sea (exactly 0 m) is transparent. |
| `source.tileSize` | no | Tiles: 256 (default) or 512 px. |
| `source.bounded` | no | Tiles with `coverage.bbox`: `true` requests tiles inside the bbox only. Required for aerial imagery, whose servers paint a fill (white, a flat ocean, a coarse world mosaic) everywhere they have no photos. A 4xx tile answer is treated as "no imagery here", never retried or counted as an error. |
| `source.key` | no | `{ "id": "<key registry id>", "in": "query" \| "header", "name": "…" }`. The id must exist in `src/keySetupCore.mjs`. A keyed entry must be `proxied`, so the key never reaches the browser. |
| `display.color`, `display.label`, `display.fields` (≤ 4, two shown) | no | Datasets only, except `opacity`. |
| `display.rank` | no | `{ "field": "…", "order": "desc" \| "asc" }`. Keeps the most important features when capped and lets the map prefer them at global zoom. Numbers or ISO dates. Without it, a zoomed-out view shows whichever features come first. |
| `display.colorBy` | no | View-bounded datasets: `{ "field": "…", "colors": { "<value>": "#rrggbb" }, "labels": { "<value>": "…" } }`, up to 12 values. Colours each feature by value; the row legend lists each label with the count in view. |
| `display.legend` | no | `[{ "label": "…", "color": "#rrggbb" }]`, up to 12: a static legend, such as a tile layer's colour scale. Take the colours from the provider's own legend or colour map. |
| `display.outline` | no | Tiles: `countries` outlines every part of each covered country, `coverage` only the parts inside `coverage.bbox`, `bbox` the bbox rectangle itself (a state, county or city inside a country). Use it for aerial imagery so its edge shows. |
| `display.aerial` | no | Tiles: `true` for aerial or satellite photography. It drapes above every other imagery layer, including core overlays, and stays there when another layer is switched on. Pair it with `source.bounded`. |
| `display.noDataColor` | no | Tiles: `#rrggbb` of the provider's no-imagery fill, drawn transparent (within a small tolerance for JPEG noise along its edge). Measure it from a tile outside the coverage; do not guess. Globe maps only: Google 3D tiles ignore it. |
| `display.opacity` | no | Tiles: (0, 1]. |
| `imagery` | no | Tiles: what a photo layer is. `type` (`ortho`, `satellite-mosaic`, `oblique`, `historic`), `resolution` as the provider publishes it ("10 cm (plains) / 25 cm (Alps)"), `resolutionM` (the finest, in metres), `trueOrtho`, and optional `vintage` ("2021–present"), `updateCycle`, `orthoQuality` and `registry` (an https link to where the metadata came from). Browse shows it as a spec list on the card. |
| `pack` | packs | `{ "host": "cctv", "env": "CCTV_EXAMPLE_ENABLED" }`. Packs are always `licence.operatorOptIn: true`. |
| `licence` | yes | `name`, an https `url`, and `operatorOptIn: true` when the terms need the operator's own agreement or consent. |
| `attribution` | yes | `text` and an optional https `url`; shown in the Data attribution popover while the entry is in use. |
| `regionsServed` | no | ISO codes, when the upstream only answers from some countries. |
| `reviewed` | no | `YYYY-MM-DD` of the last licence and endpoint review. |
## Aerial imagery from the Open Imagery Register
The aerial photo entries came from the [Open Imagery Register](https://imagery-registry.netlify.app/)
on 2026-10-23: every register source with a map preview, less the ones that
fail the library's rules. Each entry keeps the register's resolution, product
type, true-ortho flag, years flown and refresh cycle in `imagery`. Before an
entry was written, a tile inside its coverage was fetched in a browser from
the app's origin: it had to be a real photo (not a blank fill), and the
provider had to send CORS headers (or the entry would be `proxied`). The
fill a provider paints at its coverage corners became `display.noDataColor`.
Left out, with the reason:
| Register source | Why |
| --- | --- |
| NASA GIBS true colour | Already in core (Recent imagery). |
| Sentinel-2 cloudless (EOX) | CC BY-NC-SA: non-commercial. |
| SWISSIMAGE Bern and Marion County PMTiles | Extracts hosted by the register, not the provider. |
| Cabarrus County NC, Hawaii, Nashville, Sioux Falls | Credit a commercial imagery vendor (EagleView, Maxar WorldView, Pictometry, Nearmap). |
| DVRPC, Missouri, Washington | The preview serves a blank white tile across the coverage centre. |
| Sarasota County FL | The service now requires a token. |
Entries whose register licence reads "see publisher terms" or "verify portal
terms" are `licence.operatorOptIn: true`: an operator reads the publisher's
terms and lists the id in `GEV_LIBRARY_ENABLE`. Re-check an entry's terms and
drop the opt-in when its publisher states an open licence.
## Choosing a fetch mode
Use `browser-direct` when the upstream answers a browser on another origin
(`Access-Control-Allow-Origin`) and needs no key. Tile servers must send CORS
headers for WebGL to use them; without them, use `proxied`.
Use `proxied` otherwise. The proxy is generic: there is no per-entry server
code. It reaches only the manifest's host, refuses redirects, caps bodies,
caches per `cacheSeconds` and never returns upstream error text. The browser
names an entry id, never a URL, so the proxy cannot be pointed anywhere else.
## Operator switches
| Variable | Effect |
| --- | --- |
| `GEV_LIBRARY_ENABLE` | Comma-separated ids of `operatorOptIn` datasets and tiles to serve. |
| `GEV_LIBRARY_DISABLE` | Comma-separated ids to remove from this instance. |
| A pack's own variable | For example `CCTV_NYC_ENABLED=1`; Browse then shows the pack as switched on. |
## When an entry belongs in core instead
A library entry can be proposed for DATA LAYERS when it has global or
near-global coverage, is keyless or uses a key the registry already has, has
stayed `verified` with no stale period for about six months, and a core
maintainer will own it. Promotion and demotion are maintainer decisions.
+25
View File
@@ -59,6 +59,30 @@ The language picker switches in place, without a reload. A view that calls `t()`
A toast already on screen keeps its words.
## An application with its own language choice
An application that composes Core and offers the language somewhere else (a menu, a settings page) imports the `./i18n` entry (`src/i18n/page.js`) and passes `localePicker: false` to `createApplicationControls`:
```js
import {
LOCALES,
LOCALE_NAMES,
followPageLocale,
getLocale,
onLocaleChange,
startPageLocale,
switchLocale,
} from '<core package>/i18n';
startPageLocale(document); // before any view renders: ?lang=, stored choice, browser, English; translates data-i18n markup
const stopFollowing = followPageLocale(document); // <html lang>/dir and data-i18n markup follow each switch
for (const code of LOCALES) addMenuItem(LOCALE_NAMES[code], code === getLocale());
menu.onChoose = (code) => switchLocale(code); // switched in place and stored (in ?lang when storage is blocked)
```
`switchLocale` does what a choice in the globe picker does, and returns `'unchanged'` for the active language. `onLocaleChange` tells the application's own menu to mark the new language.
## Rules
- **Pass literals.** `t('Close')`, not `t(label)` (unless `label` holds a `msg()` value) and never a template literal with `${…}`.
@@ -109,6 +133,7 @@ Icons (`aria-hidden="true"`, Material Symbols), comments, CSS values and lowerca
| `src/ui/i18n.js` | the page's vue-i18n instance: the active locale and the catalogs |
| `src/i18n/marker.js` | `msg` alone, for modules the server or tools import (no catalogs) |
| `src/i18n/locale.js` | shipped languages, their names, choosing the language at boot |
| `src/i18n/page.js` | the `./i18n` entry: start, follow and switch the page's language |
| `src/i18n/locales/<code>.json` | one catalog per language, keyed by English text |
| `src/i18n/same-as-english.json` | per language, the entries that stay as in English on purpose |
| `scripts/i18n.mjs` | `npm run i18n`: extracts messages, checks the catalogs, finds unmarked UI text |
+47 -31
View File
@@ -68,6 +68,16 @@ cannot name still makes a place view. Opening a vessel thing view with a
world time waits for the vessel layer's snapshot at that time and selects the
vessel once it has it.
## Street-level photos (`sl`)
Views built by `show_in_gods_eye_view` or `src/view/index.js` can add
`sl=<source>,<lat>,<lon>` to the long form. Once the link has restored, the
app opens the nearest photo there, or, for a source that bills every load,
arms the panel so OPEN loads it (see `docs/TOOLS.md`). The app's own links do
not carry it: the address bar's long form drops `sl` when the camera next
moves, and the short form has no field for it. A reload or the app's Copy
Link keeps the lit source, the layer and the map, but not the photo request.
## Payload
The code is unpadded base64url (`A–Z a–z 0–9 - _`). The decoded bytes are:
@@ -107,36 +117,37 @@ probability of about 255 in 256 (many of them already fail as base64url).
## Field registry (top level)
| # | State key | Type | Default | Encoding |
| --- | ---------------------------- | ----- | ----------- | ----------------------------------------------------------------- |
| 1 | `view.kind` | enum | see below | 0 place, 1 thing |
| 2 | `lat` | coord | 0 | zigzag of degrees × 10⁴ |
| 3 | `lon` | coord | 0 | zigzag of degrees × 10⁴ |
| 4 | `alt` | sint | 800 | whole metres |
| 5 | `heading` | angle | 0 | whole degrees, see angles |
| 6 | `pitch` | sint | −35 | whole degrees |
| 7 | `roll` | angle | 0 | whole degrees |
| 8 | `sharedAtMs` | epoch | none | zigzag seconds since 2026-01-01T00:00:00Z |
| 9 | booleans | flags | 0 | bit set when the value differs from default |
| 10 | `layerState.enabledLayerIds` | LEN | none | layer bitset, see layers |
| 11 | `layerState.options` | LEN | none | packed option triples, see layers |
| 12 | `panelState` | LEN | none | packed panel entries, see panels |
| 13 | thing | LEN | none | sub-message, see thing views |
| 14 | `style` | enum | `normal` | normal, retro, surveillance, thermal, anime, noir, snow (retired) |
| 15 | `mapStack` | enum | `photoreal` | photoreal, bing-aerial, bing-labels, esri-imagery, osm |
| 16 | `hudVariant` | enum | style | tactical, operator (retired), minimal, cyber |
| 17 | `bloomIntensity` | uint | style | |
| 18 | `bloomVersion` | uint | 2 | |
| 19 | `sharpenIntensity` | uint | style | |
| 20 | `detectionMode` | enum | style | OFF, SPARSE, BALANCED, DENSE |
| 21 | `detectionDensity` | uint | style | one of 0, 25, 50, 75, 100 |
| 22 | `detectionAllocation` | enum | style | ELASTIC, WEIGHTED |
| 23 | `detectionFadePct` | uint | style | 0–40 |
| 24 | `detectionOutsideOpacityPct` | uint | style | 0–100 |
| 25 | `scopeFeatherPct` | uint | 100 | 0–200 |
| 26 | `scopeTerminusPct` | uint | none | 94–100; absent is the adaptive terminus |
| 27 | `styleParams` | LEN | style | packed `(token, value × 100)` pairs |
| 28 | `worldTime` | epoch | none (live) | zigzag seconds since 2026-01-01T00:00:00Z |
| # | State key | Type | Default | Encoding |
| --- | ---------------------------- | ----- | ----------- | --------------------------------------------------------------------- |
| 1 | `view.kind` | enum | see below | 0 place, 1 thing |
| 2 | `lat` | coord | 0 | zigzag of degrees × 10⁴ |
| 3 | `lon` | coord | 0 | zigzag of degrees × 10⁴ |
| 4 | `alt` | sint | 800 | whole metres |
| 5 | `heading` | angle | 0 | whole degrees, see angles |
| 6 | `pitch` | sint | −35 | whole degrees |
| 7 | `roll` | angle | 0 | whole degrees |
| 8 | `sharedAtMs` | epoch | none | zigzag seconds since 2026-01-01T00:00:00Z |
| 9 | booleans | flags | 0 | bit set when the value differs from default |
| 10 | `layerState.enabledLayerIds` | LEN | none | layer bitset, see layers |
| 11 | `layerState.options` | LEN | none | packed option triples, see layers |
| 12 | `panelState` | LEN | none | packed panel entries, see panels |
| 13 | thing | LEN | none | sub-message, see thing views |
| 14 | `style` | enum | `normal` | normal, retro, surveillance, thermal, anime, noir, snow (retired) |
| 15 | `mapStack` | enum | `photoreal` | photoreal, bing-aerial, bing-labels, esri-imagery, osm |
| 16 | `hudVariant` | enum | style | tactical, operator (retired), minimal, cyber |
| 17 | `bloomIntensity` | uint | style | |
| 18 | `bloomVersion` | uint | 2 | |
| 19 | `sharpenIntensity` | uint | style | |
| 20 | `detectionMode` | enum | style | OFF, SPARSE, BALANCED, DENSE |
| 21 | `detectionDensity` | uint | style | one of 0, 25, 50, 75, 100 |
| 22 | `detectionAllocation` | enum | style | ELASTIC, WEIGHTED |
| 23 | `detectionFadePct` | uint | style | 0–40 |
| 24 | `detectionOutsideOpacityPct` | uint | style | 0–100 |
| 25 | `scopeFeatherPct` | uint | 100 | 0–200 |
| 26 | `scopeTerminusPct` | uint | none | 94–100; absent is the adaptive terminus |
| 27 | `styleParams` | LEN | style | packed `(token, value × 100)` pairs |
| 28 | `worldTime` | epoch | none (live) | zigzag seconds since 2026-01-01T00:00:00Z |
| 29 | `library` | LEN | none | Data Library ledger tokens, two base-36 characters each, concatenated |
Field 1 absent means `thing` when field 13 is present and `place` otherwise,
so v1 never writes it; a later view kind writes its code.
@@ -154,6 +165,11 @@ Field 28 is the world time the link was made at, written only when it was away
from live. A link without it opens live. Unlike field 8 it may lie ahead of the
link's creation (a forecast moment), and it opens paused at that time.
Field 29 lists the Data Library entries switched on, by their permanent
tokens in `src/library/ledger.json` (the long form's `lib=ab.cd`). A share
without it switches no library entry on; the recipient's added entries stay
in their panel, off. Tokens this build does not know are skipped.
Flag bits (field 9): 0 `bloom`, 1 `sharpen`, 2 `hudVisible`, 3
`celestialRing` (each defaulting by style), 4 `scopeEnabled` (default on).
@@ -381,7 +397,7 @@ the app that can open it.
offers keeps its code: `snow` (6) is retired, so a link that names it
opens as Normal and the parameters written for it are dropped.
5. **Add, don't change.** A new setting takes the next free field number
(28 onwards; fields 1–15 have one-byte tags and are all taken). A new
(30 onwards; fields 1–15 have one-byte tags and are all taken). A new
boolean takes the next flag bit instead, which costs nothing when it is at
its default.
6. **New view kinds and entity kinds** get the next code. A new entity kind's
+23 -5
View File
@@ -102,11 +102,29 @@ action draws (pins, highlights, areas, arrows, routes and labels at a place
name or coordinates). Links carry them in the `an` parameter, and the app
draws them once the link has been restored.
A view can open street-level photos too: `street_level` names a source
(`google` for Street View, `mapillary`, `kartaview`, `panoramax`, or none for
the lit one) and a point, by default the spot the camera looks at. Street View
shows only on the Google 3D map, so a `google` source sets `map` to
`photoreal` and refuses another; a view cannot both follow an entity and open
photos. Links carry it as `sl=<source>,<lat>,<lon>`: once restored, the app
runs `show_street_level`, which flies there and opens the nearest photo. A
plain link with a `google` source only arms the panel for OPEN GOOGLE STREET
VIEW; a view posted into the embedded app opens it, and so does the panel
page's own opening link, but only for a tool result issued in the last five
minutes (an older one, such as a host remounting the panel, only arms);
`?embed=1` on a published link does not. Mapillary,
KartaView and Panoramax, which do not bill per image, open at once. A link
that follows an entity ignores `sl`.
Tools take a view as `VIEW_ARGUMENTS`: an `area` to frame from above, or a
`camera`, plus `layers`, `style`, `map`, `follow` and `annotations`. A
tilted camera over an area looks at its center from behind; camera fields
given with an area override its framing. `resolveViewArguments` turns them
into a view.
`camera`, plus `layers`, `style`, `map`, `follow`, `annotations` and
`street_level`. A tilted camera over an area looks at its center from behind;
camera fields given with an area override its framing. A passed-through
view's `street_level` point moves with a new `area` or camera position (a
turned camera keeps it). `resolveViewArguments` turns them into a view. The
app's own links (the address bar, Copy Link, `/share/<code>`) do not carry
`sl`; see `docs/SHARE-LINK-FORMAT.md`.
### The God's Eye View panel
@@ -299,5 +317,5 @@ and the development server's internal routes. See SECURITY.md.
| `situation_brief` | `weather` | Weather, earthquakes, fires, aircraft, ships and cyclones for an area, by section |
| `military_awareness` | `military` | Military and other aircraft, ships and military installations within 250 km of a point, by section |
| `get_hud_caption` | `weather`, `summary` | The app's heads-up display caption for an area |
| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks and something to follow |
| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks, something to follow, or street-level photos (Street View or Mapillary) at a place |
| `panel_request` | `app` | Panel only: loads a path from the app's server for the God's Eye View panel |
+1 -1
View File
@@ -4,7 +4,7 @@ Catalog: `src/i18n/locales/zh-Hant.json`. General rules: [TRANSLATORS.md](../TRA
`zh-Hant` is the Simplified Chinese catalog (`zh-CN`, credited to abiyeenzo, #914, and EDIT-Aeterna, #979) written in Traditional characters with OpenCC character conversion. Wording and terminology are the same as in `zh-CN`: change a term in [zh-CN.md](./zh-CN.md) and the `zh-CN` catalog first, then convert. Only the script differs.
After `s2tw` conversion, fix the characters OpenCC picks wrongly: 年曆史 → 年歷史, 下遊 → 下游, 並道 → 併道, 發佈 → 發布, 公佈 → 公布.
After `s2tw` conversion, fix the characters OpenCC picks wrongly: 年曆史 → 年歷史, 下遊 → 下游, 並道 → 併道, 發佈 → 發布, 公佈 → 公布, 控制檯 → 控制台, and 周 → 週 where it means a week (1 週/秒).
## Style
+1 -1
View File
@@ -26,7 +26,7 @@
with the sources: src/materialSymbolsSubset.test.mjs fails when a source
names a glyph this list is missing, because a missing glyph does not draw
a box — the ligature never forms and the element renders the literal word. -->
<link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20,400,0,0&icon_names=adjust,arrow_drop_down,arrow_forward,arrow_left,arrow_right,bolt,bug_report,category,chevron_left,chevron_right,close,close_fullscreen,dark_mode,directions_boat,draw,east,eco,egg_alt,flare,flight,handyman,layers,layers_clear,light_mode,local_fire_department,location_on,more_horiz,movie,my_location,navigation,normal,on,open_in_full,partly_cloudy_day,pause,pentagon,pest_control,pets,play_arrow,public,question_mark,radar,radio,raven,right_panel_close,right_panel_open,rocket_launch,satellite_alt,search,set_meal,settings,skip_next,skip_previous,streetview,timeline,traffic,tune,umbrella,upload_file,videocam,view_in_ar,visibility,visibility_off,water_drop" rel="stylesheet" />
<link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20,400,0,0&icon_names=adjust,arrow_drop_down,arrow_forward,arrow_left,arrow_right,bolt,bug_report,category,chevron_left,chevron_right,close,close_fullscreen,dark_mode,delete,directions_boat,draw,east,eco,egg_alt,flare,flight,handyman,layers,layers_clear,light_mode,local_fire_department,location_on,more_horiz,movie,my_location,navigation,normal,on,open_in_full,partly_cloudy_day,pause,pentagon,pest_control,pets,play_arrow,public,question_mark,radar,radio,raven,right_panel_close,right_panel_open,rocket_launch,satellite_alt,search,set_meal,settings,skip_next,skip_previous,streetview,timeline,traffic,tune,umbrella,upload_file,videocam,view_in_ar,visibility,visibility_off,water_drop,zoom_in_map" rel="stylesheet" />
</head>
<body>
<div class="gev-view" data-gev-view="scene-chrome"></div>
+14
View File
@@ -15,7 +15,9 @@
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"fzstd": "0.1.1",
"hls.js": "^1.7.3",
"hyparquet": "1.31.2",
"mapillary-js": "^4.1.2",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
@@ -2460,6 +2462,12 @@
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/fzstd": {
"version": "0.1.1",
"resolved": "https://registry.npmjs.org/fzstd/-/fzstd-0.1.1.tgz",
"integrity": "sha512-dkuVSOKKwh3eas5VkJy1AW1vFpet8TA/fGmVA5krThl8YcOVE/8ZIoEA1+U1vEn5ckxxhLirSdY837azmbaNHA==",
"license": "MIT"
},
"node_modules/get-caller-file": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
@@ -2570,6 +2578,12 @@
"url": "https://opencollective.com/express"
}
},
"node_modules/hyparquet": {
"version": "1.31.2",
"resolved": "https://registry.npmjs.org/hyparquet/-/hyparquet-1.31.2.tgz",
"integrity": "sha512-dpVTCGkPPIdfDExa7YZGaJpuBTmLFDKGeZ6meHUj1mz/Q3qmaQHGLLqCyPBkQjfxrYNb08YTax682qeGO+zkVA==",
"license": "MIT"
},
"node_modules/ieee754": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
+13
View File
@@ -38,6 +38,10 @@
"i18n": "node scripts/i18n.mjs",
"layer-token:next": "node scripts/next-layer-state-token.mjs",
"layer-token:check": "node scripts/check-layer-state-tokens.mjs",
"library:check": "node scripts/library.mjs check",
"library:next": "node scripts/library.mjs next",
"library:docs": "node scripts/library.mjs docs",
"library:probe": "node scripts/library.mjs probe",
"changelog": "node scripts/assemble-changelog.mjs --write",
"changelog:check": "node scripts/assemble-changelog.mjs --check",
"dev": "vite",
@@ -54,6 +58,7 @@
"qa:street-level:fixtures": "node scripts/qa-street-level.mjs --fixtures",
"check:boundaries": "node scripts/check-import-directions.mjs && node scripts/check-package-boundaries.mjs",
"qa:transit": "node scripts/qa-transit.mjs",
"qa:data-library": "node scripts/qa-data-library.mjs",
"mcp": "node server/mcp/stdio.js"
},
"dependencies": {
@@ -63,7 +68,9 @@
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"fzstd": "0.1.1",
"hls.js": "^1.7.3",
"hyparquet": "1.31.2",
"mapillary-js": "^4.1.2",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
@@ -92,6 +99,7 @@
"./application": "./src/app/application.js",
"./application/viewer": "./src/app/viewer.js",
"./time": "./src/time/worldClock.js",
"./i18n": "./src/i18n/page.js",
"./build/vite": {
"node": "./build/vite.js"
},
@@ -285,6 +293,11 @@
"./layers/transit/standalone": "./src/data/transit.js",
"./layers/transit/source": "./src/layers/transit/source.js",
"./sources/transit-service": "./src/sources/transitService.js",
"./server/providers/rail": {
"node": "./server/providers/rail.js"
},
"./sources/rail-service": "./src/sources/railService.js",
"./layers/rail/source": "./src/layers/rail/source.js",
"./sources/map-features": "./src/sources/overpassFeatures.js",
"./server/providers/mapillary": {
"node": "./server/providers/mapillary.js"
File diff suppressed because it is too large. Load diff
@@ -181,6 +181,18 @@ export function answerMapillaryRequest({ method, url, images }) {
: [];
return { ...json(200, { data }), known: true };
}
if (q.has('lat') && q.has('lng')) {
const at = { lat: Number(q.get('lat')), lon: Number(q.get('lng')) };
const radius = Math.min(50, Number(q.get('radius')) || 50);
const limit = Number(q.get('limit')) || 10;
// In no particular order, as the real API answers.
const data = photos
.filter((image) => metresApart(at, image) <= radius)
.reverse()
.slice(0, limit)
.map((image) => withFields(imageRecord(image), fields));
return { ...json(200, { data }), known: true };
}
if (q.has('s2')) return { ...json(200, { data: [] }), known: true };
return {
...json(400, { error: 'unsupported images query' }),
+45
View File
@@ -385,6 +385,8 @@
"src/layers/vessels/index.js",
"src/layers/vessels/ingestion.js",
"src/layers/vessels/lifecycle.js",
"src/layers/vessels/photos.js",
"src/layers/vessels/photos.test.mjs",
"src/layers/vessels/policy.js",
"src/layers/vessels/queries.js",
"src/layers/vessels/rendering.js",
@@ -847,6 +849,45 @@
"scripts/qa-transit-recovery.mjs",
"scripts/format-runtime.json",
"src/tooling/transitService.test.mjs",
"src/tooling/railService.test.mjs",
"src/data/railFeeds.test.mjs",
"src/data/railProxy.test.mjs",
"src/layers/rail/policy.test.mjs",
"src/layers/rail/source.test.mjs",
"src/layers/rail/pathBinding.test.mjs",
"src/layers/rail/routeLines.test.mjs",
"src/layers/rail/stopLabels.test.mjs",
"src/layers/rail/schedules.test.mjs",
"src/data/railPaths.test.mjs",
"src/data/railSchedule.test.mjs",
"src/data/railTripUpdates.test.mjs",
"src/data/irishRail.test.mjs",
"src/tooling/buildRailPaths.test.mjs",
"src/tooling/buildRailPathsRoutes.test.mjs",
"scripts/build-rail-paths.mjs",
"src/tooling/buildRailPathsUnlocated.test.mjs",
"src/layers/rail/follow.test.mjs",
"src/layers/rail/groundHeight.test.mjs",
"src/layers/rail/liveOnly.test.mjs",
"src/testSupport/railLayerHarness.mjs",
"src/layers/rail/dedupe.test.mjs",
"src/data/railIreland.test.mjs",
"src/data/railNoPositions.test.mjs",
"src/data/railFeedsNational.test.mjs",
"src/data/railSiriVm.test.mjs",
"src/tooling/buildRailPathsTripEnds.test.mjs",
"src/data/railFeedsUs.test.mjs",
"src/data/railFeedsAsia.test.mjs",
"src/data/railFeedsFrance.test.mjs",
"src/data/railFeedsUsNoTerms.test.mjs",
"src/data/railFeedsUsUrban.test.mjs",
"src/tooling/buildRailPathsLocal.test.mjs",
"src/tooling/buildRailPathsWrite.test.mjs",
"src/tooling/railServiceFaults.test.mjs",
"src/tooling/railCatalogPacks.test.mjs",
"src/tooling/railServicePacks.test.mjs",
"src/tooling/buildRailPathsCli.test.mjs",
"src/data/railFeedsDocs.test.mjs",
"src/sources/overpassFeatures.test.mjs",
"src/data/cctvHlsStream.test.mjs",
"src/layers/cctv/videoPlayback.test.mjs",
@@ -1296,9 +1337,13 @@
"src/app/sourceComposition.js",
"src/app/sourceComposition.test.mjs",
"src/data/dataCredits.js",
"src/data/openVesselPhoto.js",
"src/data/openVesselPhoto.test.mjs",
"src/data/layerState.js",
"src/data/layerState.test.mjs",
"src/data/spriteOrder.js",
"src/i18n/page.js",
"src/i18n/page.test.mjs",
"src/i18n/locales/ar.json",
"src/i18n/locales/de.json",
"src/i18n/locales/es.json",
+321
View File
@@ -0,0 +1,321 @@
#!/usr/bin/env node
/**
* Data Library maintenance. See docs/DATA-LIBRARY.md.
*
* npm run library:check [-- --base-ref origin/main]
* Validate every manifest, the share-token ledger and the generated
* DATA_SOURCES.md section. With --base-ref, also prove the ledger only
* appended rows since that ref (published tokens never change).
* npm run library:next -- <entry-id>
* Print the next free share token for a new switchable entry.
* npm run library:docs
* Rewrite the generated Data Library section of DATA_SOURCES.md.
* npm run library:probe [-- --json] [-- --strict]
* Hit every dataset and tile upstream once (bounded) and report
* status, latency and feature count. --strict exits 1 on a failure.
*/
import { execFileSync } from 'node:child_process';
import { readFileSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { LIBRARY_ENTRIES, LIBRARY_LEDGER } from '../src/library/catalog.js';
import { libraryTileUrl } from '../src/library/manifest.js';
import {
arcgisPageUrl,
datasetRequestUrl,
parseLibraryBody,
} from '../src/library/formats.js';
import {
nextLibraryToken,
readLibraryLedger,
} from '../src/library/selection.js';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const DATA_SOURCES = path.join(root, 'DATA_SOURCES.md');
const START = '<!-- data-library:start (generated by npm run library:docs) -->';
const END = '<!-- data-library:end -->';
const LEDGER_PATH = 'src/library/ledger.json';
const cell = (text) => String(text).replace(/\|/g, '\\|').replace(/\s+/g, ' ');
/** The generated DATA_SOURCES.md section, one row per entry. */
export function libraryDocsSection(entries = LIBRARY_ENTRIES) {
const rows = [...entries]
.sort((a, b) => a.id.localeCompare(b.id))
.map((entry) => {
const host = (() => {
try {
return new URL(entry.source.url.replace(/\{[xyz]\}/g, '0')).host;
} catch {
return '';
}
})();
const how =
entry.kind === 'pack'
? `CCTV pack; operator sets \`${entry.pack.env}=1\``
: `${entry.kind}, ${entry.source.format}, ${entry.source.fetch}` +
(entry.licence.operatorOptIn
? `; operator adds \`${entry.id}\` to \`GEV_LIBRARY_ENABLE\``
: '') +
(entry.source.key
? `; needs the \`${entry.source.key.id}\` key`
: '');
return `| **${cell(entry.title)}** (\`${entry.id}\`, \`${host}\`) | ${cell(how)} | [${cell(entry.licence.name)}](${entry.licence.url}) | "${cell(entry.attribution.text)}" |`;
});
return [
START,
'',
'## Data Library entries (fetched at runtime — not stored in this repo)',
'',
'Each Data Library entry is a manifest in `src/library/entries/` pointing at a live upstream its provider hosts; see [docs/DATA-LIBRARY.md](docs/DATA-LIBRARY.md). An entry appears on the map only after a viewer adds it from **DATA LAYERS ▸ Browse data library**, and its attribution is registered in the Data attribution popover while it is in use. This table is generated from the manifests: edit the manifest, then run `npm run library:docs`.',
'',
'| Entry | How it is served | Licence / terms | Attribution |',
'| --- | --- | --- | --- |',
...rows,
'',
END,
].join('\n');
}
function replaceSection(text, section) {
const start = text.indexOf(START);
const end = text.indexOf(END);
if (start < 0 || end < start) {
const anchor = text.indexOf('\n## Bundled snapshots');
if (anchor < 0)
throw new Error('DATA_SOURCES.md has no Bundled snapshots heading');
return `${text.slice(0, anchor)}\n${section}\n${text.slice(anchor)}`;
}
return text.slice(0, start) + section + text.slice(end + END.length);
}
function check(baseRef) {
const errors = [];
const ledger = readLibraryLedger(LIBRARY_LEDGER);
errors.push(...ledger.errors.map((error) => `ledger ${error}`));
for (const entry of LIBRARY_ENTRIES) {
const switchable = entry.kind !== 'pack';
if (switchable && !ledger.tokenById.has(entry.id))
errors.push(
`${entry.id}: no share token (npm run library:next -- ${entry.id})`,
);
if (!switchable && ledger.tokenById.has(entry.id))
errors.push(
`${entry.id}: packs are switched by their host and take no token`,
);
}
if (baseRef) {
let published = [];
try {
published = JSON.parse(
execFileSync('git', ['show', `${baseRef}:${LEDGER_PATH}`], {
cwd: root,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
}),
);
} catch {
published = [];
}
published.forEach(([id, token], index) => {
const row = LIBRARY_LEDGER[index];
if (!row || row[0] !== id || row[1] !== token)
errors.push(
`ledger row ${index} (${id} → ${token}) was published on ${baseRef} and must not change`,
);
});
LIBRARY_LEDGER.slice(published.length).forEach(([id, token], offset) => {
const expected = nextLibraryToken(
LIBRARY_LEDGER.slice(0, published.length + offset),
);
if (token !== expected)
errors.push(
`ledger row for ${id} should take ${expected}, not ${token}`,
);
});
}
const text = readFileSync(DATA_SOURCES, 'utf8');
if (!text.includes(libraryDocsSection()))
errors.push(
'DATA_SOURCES.md Data Library section is out of date (npm run library:docs)',
);
if (errors.length) {
for (const error of errors) console.error(`✗ ${error}`);
process.exit(1);
}
console.log(
`Data Library valid: ${LIBRARY_ENTRIES.length} entries, ${LIBRARY_LEDGER.length} share tokens${baseRef ? `, ledger append-only against ${baseRef}` : ''}.`,
);
}
/** Points inside a coverage bbox to try, centre first, as [fx, fy] shares. */
const PROBE_POINTS = [
[0.5, 0.5],
[0.4, 0.45],
[0.6, 0.55],
[0.25, 0.25],
[0.75, 0.75],
];
/**
* A Web Mercator tile to probe. Regional entries try points inside their
* coverage bbox (a regional server answers 404 or a blank fill outside its
* photos, and a bbox centre can be sea); global entries probe a mid-world
* tile.
*/
function probeTile(entry, z, [fx, fy] = PROBE_POINTS[0]) {
const n = 2 ** z;
const bbox = entry.coverage?.bbox;
if (!bbox) return { x: Math.floor(n / 2), y: Math.floor(n / 3) };
const lon = bbox[0] + (bbox[2] - bbox[0]) * fx;
const lat = Math.max(-85, Math.min(85, bbox[1] + (bbox[3] - bbox[1]) * fy));
const rad = (lat * Math.PI) / 180;
const clamp = (value) => Math.min(n - 1, Math.max(0, Math.floor(value)));
return {
x: clamp(((lon + 180) / 360) * n),
y: clamp(
((1 - Math.log(Math.tan(rad) + 1 / Math.cos(rad)) / Math.PI) / 2) * n,
),
};
}
async function probeEntry(entry) {
const started = Date.now();
const signal = AbortSignal.timeout(30_000);
try {
if (entry.kind === 'tiles') {
// Regional imagery is probed at z13, the Open Imagery Register's
// validation zoom (some caches start there); global layers at their
// coarsest level.
const minZoom = entry.source.minZoom ?? 0;
const z = entry.coverage.bbox
? Math.min(entry.source.maxZoom ?? 18, Math.max(minZoom, 13))
: minZoom;
const points = entry.coverage.bbox ? PROBE_POINTS : [PROBE_POINTS[0]];
let status = null;
let lastError = null;
for (const point of points) {
const { x, y } = probeTile(entry, z, point);
const url = libraryTileUrl(entry.source.url, z, x, y);
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(20_000),
redirect: 'error',
});
await response.arrayBuffer();
status = response.status;
if (
response.ok &&
/^image\//.test(response.headers.get('content-type') || '')
)
return { id: entry.id, ok: true, status, ms: Date.now() - started };
} catch (error) {
lastError = error;
}
}
if (status === null && lastError) throw lastError;
return { id: entry.id, ok: false, status, ms: Date.now() - started };
}
if (entry.source.format === 'geoparquet') {
// List the partitions and read one footer by range request: proof the
// bucket answers and the files are GeoParquet, without reading data.
const { listPartitions, rangeBuffer, geoParquetInfo } =
await import('../src/library/parquet.js');
const { parquetMetadataAsync } = await import('hyparquet');
const fetchImpl = (target, options = {}) =>
fetch(target, { ...options, signal });
const objects = entry.source.partitions
? await listPartitions(entry, { fetchImpl, signal })
: [{ key: entry.source.url, size: null }];
const first = objects[0];
const url = entry.source.partitions
? new URL(first.key, entry.source.partitions.keyBase).href
: entry.source.url;
const size =
first.size ??
Number(
(await fetch(url, { method: 'HEAD', signal })).headers.get(
'content-length',
),
);
const metadata = await parquetMetadataAsync(
rangeBuffer(url, size, { fetchImpl, signal }),
{ initialFetchSize: 16 * 1024 },
);
return {
id: entry.id,
ok: Boolean(geoParquetInfo(metadata).column),
status: 200,
ms: Date.now() - started,
features: Number(metadata.num_rows),
partitions: objects.length,
};
}
const url =
entry.source.format === 'arcgis'
? arcgisPageUrl(
entry.source.url,
entry.source.query,
0,
// Some huge view services refuse a page smaller than their own.
entry.source.view ? 2000 : 100,
// A view-bounded layer is probed over its sample area.
entry.source.view?.sample ?? null,
)
: datasetRequestUrl(entry.source);
const response = await fetch(url, { signal, redirect: 'error' });
const text = await response.text();
const parsed = response.ok
? parseLibraryBody(entry.source.format, text, entry.source)
: null;
return {
id: entry.id,
ok: Boolean(parsed?.candidates.length),
status: response.status,
ms: Date.now() - started,
features: parsed?.candidates.length ?? 0,
};
} catch (error) {
return {
id: entry.id,
ok: false,
status: null,
ms: Date.now() - started,
error: error.message,
};
}
}
const [command = 'check', ...rest] = process.argv.slice(2);
const flag = (name) => rest.includes(name);
const option = (name) => {
const index = rest.indexOf(name);
return index >= 0 ? rest[index + 1] : null;
};
if (command === 'check') check(option('--base-ref'));
else if (command === 'next') {
const id = rest.find((arg) => !arg.startsWith('--'));
if (!id) throw new Error('usage: npm run library:next -- <entry-id>');
if (LIBRARY_LEDGER.some(([existing]) => existing === id))
throw new Error(`${id} already owns a token`);
console.log(nextLibraryToken(LIBRARY_LEDGER));
} else if (command === 'docs') {
const text = readFileSync(DATA_SOURCES, 'utf8');
writeFileSync(DATA_SOURCES, replaceSection(text, libraryDocsSection()));
console.log('DATA_SOURCES.md Data Library section updated.');
} else if (command === 'probe') {
const results = [];
for (const entry of LIBRARY_ENTRIES.filter(({ kind }) => kind !== 'pack'))
results.push(await probeEntry(entry));
if (flag('--json')) console.log(JSON.stringify(results, null, 2));
else
for (const result of results)
console.log(
`${result.ok ? 'ok ' : 'FAIL'} ${result.id} · ${result.status ?? '—'} · ${result.ms} ms${result.features !== undefined ? ` · ${result.features} features` : ''}${result.partitions !== undefined ? ` in the first of ${result.partitions} partitions` : ''}${result.error ? ` · ${result.error}` : ''}`,
);
if (flag('--strict') && results.some(({ ok }) => !ok)) process.exit(1);
} else {
console.error(`unknown command: ${command}`);
process.exit(2);
}
+283 -6
View File
@@ -105,6 +105,11 @@
"src/ui/views/panels/DrawSwatches.vue",
"src/annotations/drawMarks.js",
"src/ui/stores/display.js",
"src/ui/stores/dataLibrary.js",
"src/ui/views/panels/DataLibraryCard.vue",
"src/ui/views/panels/DataLibraryDialog.vue",
"src/ui/views/panels/DataLibraryBrowse.vue",
"src/ui/libraryBrowser.js",
"src/ui/localSdrPresentation.js",
"src/ui/unitLabels.js",
"src/layers/localAdsb/feedNames.js",
@@ -170,6 +175,28 @@
],
"external": ["vue", "pinia", "vue-i18n"]
},
"i18n": {
"exports": ["./i18n"],
"modules": [
"src/i18n/page.js",
"src/i18n/index.js",
"src/i18n/locale.js",
"src/i18n/locales/ar.json",
"src/i18n/locales/de.json",
"src/i18n/locales/es.json",
"src/i18n/locales/fr.json",
"src/i18n/locales/he.json",
"src/i18n/locales/it.json",
"src/i18n/locales/ja.json",
"src/i18n/locales/pt.json",
"src/i18n/locales/ru.json",
"src/i18n/locales/sv.json",
"src/i18n/locales/uk.json",
"src/i18n/locales/zh-CN.json",
"src/i18n/locales/zh-Hant.json"
],
"external": []
},
"time": {
"exports": ["./time"],
"modules": ["src/time/worldClock.js"],
@@ -1469,7 +1496,8 @@
"src/sources/live/index.js",
"src/sources/live/standalone.js",
"src/sources/live/vessels.js",
"src/data/planespottersPhoto.js"
"src/data/planespottersPhoto.js",
"src/data/openVesselPhoto.js"
],
"external": []
},
@@ -1691,7 +1719,9 @@
"src/i18n/locales/sv.json",
"src/i18n/locales/uk.json",
"src/i18n/locales/zh-CN.json",
"src/i18n/locales/zh-Hant.json"
"src/i18n/locales/zh-Hant.json",
"src/layers/vessels/photos.js",
"src/data/openVesselPhoto.js"
],
"external": ["cesium"]
},
@@ -2192,6 +2222,7 @@
"scripts/qa-street-level.mjs",
"src/i18n/index.js",
"src/i18n/locale.js",
"src/i18n/marker.js",
"src/i18n/locales/ar.json",
"src/i18n/locales/de.json",
"src/i18n/locales/es.json",
@@ -3249,6 +3280,27 @@
"src/i18n/marker.js",
"src/sources/nominatim.js",
"src/app/layers/transit.js",
"src/app/layers/rail.js",
"src/data/railFeeds.js",
"src/layers/rail/index.js",
"src/layers/rail/source.js",
"src/layers/rail/policy.js",
"src/layers/rail/ingestion.js",
"src/layers/rail/dedupe.js",
"src/layers/rail/rendering.js",
"src/layers/rail/selection.js",
"src/layers/rail/follow.js",
"src/layers/rail/viewport.js",
"src/layers/rail/controls.js",
"src/layers/rail/lifecycle.js",
"src/layers/rail/pathBinding.js",
"src/layers/rail/routeDisplay.js",
"src/layers/rail/routeLines.js",
"src/layers/rail/schedules.js",
"src/layers/rail/stopLabels.js",
"src/data/railPaths.js",
"src/data/railSchedule.js",
"src/data/railTimeZones.js",
"src/data/transit.js",
"src/data/transitFeeds.js",
"src/data/rioRouteNames.js",
@@ -3471,6 +3523,141 @@
"src/voice/geminiSession.js",
"src/voice/geminiAudio.js",
"src/voice/geminiCaptureWorklet.js",
"src/ui/libraryBrowser.js",
"src/library/manifest.js",
"src/data/libraryLayerIds.js",
"src/library/selection.js",
"src/library/ledger.json",
"src/library/entries/ak-statewide-imagery.json",
"src/library/entries/amsterdam-traffic-cameras.json",
"src/library/entries/ar-statewide-imagery.json",
"src/library/entries/az-statewide-imagery.json",
"src/library/entries/bayern-dop40.json",
"src/library/entries/bismarck-mandan-nd.json",
"src/library/entries/bkg-topplus-open.json",
"src/library/entries/cctv-florida-fl511.json",
"src/library/entries/cctv-lithuania.json",
"src/library/entries/cctv-nyc-dot.json",
"src/library/entries/ct-statewide-imagery.json",
"src/library/entries/dakota-county-mn.json",
"src/library/entries/dc-statewide-imagery.json",
"src/library/entries/de-statewide-imagery.json",
"src/library/entries/ee-maa-amet-ortofoto.json",
"src/library/entries/eia-power-plants.json",
"src/library/entries/epa-superfund-sites.json",
"src/library/entries/epa-toxic-release-facilities.json",
"src/library/entries/epc-england-wales.json",
"src/library/entries/fema-usa-structures.json",
"src/library/entries/gbif-occurrence-density.json",
"src/library/entries/gdacs-disaster-alerts.json",
"src/library/entries/grand-forks-nd.json",
"src/library/entries/gsi-japan-map.json",
"src/library/entries/gsi-japan-photo.json",
"src/library/entries/gvp-holocene-volcanoes.json",
"src/library/entries/ia-statewide-imagery.json",
"src/library/entries/id-statewide-imagery.json",
"src/library/entries/ign-orthophotos-france.json",
"src/library/entries/il-statewide-imagery.json",
"src/library/entries/imf-portwatch-chokepoints.json",
"src/library/entries/imf-portwatch-disruptions.json",
"src/library/entries/in-statewide-imagery.json",
"src/library/entries/jrc-surface-water.json",
"src/library/entries/kartverket-admin-units.json",
"src/library/entries/kartverket-cadastre.json",
"src/library/entries/kartverket-depth.json",
"src/library/entries/kartverket-hillshade.json",
"src/library/entries/kartverket-nautical-charts.json",
"src/library/entries/kartverket-terrain-relief.json",
"src/library/entries/kartverket-topo-grey.json",
"src/library/entries/kartverket-topo.json",
"src/library/entries/kartverket-toporaster.json",
"src/library/entries/ks-statewide-imagery.json",
"src/library/entries/ky-statewide-imagery.json",
"src/library/entries/lake-county-in.json",
"src/library/entries/lawrence-ks.json",
"src/library/entries/lee-county-fl.json",
"src/library/entries/loudoun-county-va.json",
"src/library/entries/louisville-lojic-ky.json",
"src/library/entries/ma-statewide-imagery.json",
"src/library/entries/macrostrat-geology.json",
"src/library/entries/mapterhorn-terrain.json",
"src/library/entries/maris-ms.json",
"src/library/entries/md-statewide-imagery.json",
"src/library/entries/me-statewide-imagery.json",
"src/library/entries/mi-statewide-imagery.json",
"src/library/entries/mountain-village-co.json",
"src/library/entries/ms-statewide-imagery.json",
"src/library/entries/mt-statewide-imagery.json",
"src/library/entries/naip.json",
"src/library/entries/nasa-black-marble.json",
"src/library/entries/nasa-eonet-events.json",
"src/library/entries/nc-statewide-imagery.json",
"src/library/entries/nctcog-tx.json",
"src/library/entries/nd-statewide-imagery.json",
"src/library/entries/new-orleans-rpc-la.json",
"src/library/entries/nh-statewide-imagery.json",
"src/library/entries/nj-statewide-imagery.json",
"src/library/entries/noaa-tide-stations.json",
"src/library/entries/nrw-dop.json",
"src/library/entries/nsw-imagery.json",
"src/library/entries/ntad-national-bridges.json",
"src/library/entries/nwa-benton-washington-ar.json",
"src/library/entries/nws-active-alerts.json",
"src/library/entries/ny-statewide-imagery.json",
"src/library/entries/oakland-county-mi.json",
"src/library/entries/oh-statewide-imagery.json",
"src/library/entries/or-statewide-imagery.json",
"src/library/entries/pa-statewide-imagery.json",
"src/library/entries/pl-ortofotomapa.json",
"src/library/entries/pnoa-ma.json",
"src/library/entries/portland-metro-or.json",
"src/library/entries/pt-dgt-ortos.json",
"src/library/entries/ri-statewide-imagery.json",
"src/library/entries/swisstopo-swissimage.json",
"src/library/entries/tx-statewide-imagery.json",
"src/library/entries/union-county-oh.json",
"src/library/entries/usgs-hydrography.json",
"src/library/entries/usgs-topo.json",
"src/library/entries/va-statewide-imagery.json",
"src/library/entries/vt-statewide-imagery.json",
"src/library/entries/wallonie-ortho.json",
"src/library/entries/wi-statewide-imagery.json",
"src/library/entries/wri-global-power-plants.json",
"src/library/entries/wv-statewide-imagery.json",
"src/library/formats.js",
"src/library/entries/gvp-holocene-volcanoes.json",
"src/library/entries/nasa-black-marble.json",
"src/library/entries/nasa-eonet-events.json",
"src/library/catalog.js",
"src/library/source.js",
"src/library/layers.js",
"src/library/controller.js",
"src/library/extent.js",
"src/library/camera.js",
"src/library/entries/imf-portwatch-disruptions.json",
"src/library/entries/imf-portwatch-chokepoints.json",
"src/library/entries/noaa-tide-stations.json",
"src/library/entries/nws-active-alerts.json",
"src/library/entries/wri-global-power-plants.json",
"src/library/entries/fema-usa-structures.json",
"src/library/entries/epc-england-wales.json",
"src/library/parquet.js",
"src/library/entries/bkg-topplus-open.json",
"src/library/entries/epa-superfund-sites.json",
"src/library/entries/epa-toxic-release-facilities.json",
"src/library/entries/gbif-occurrence-density.json",
"src/library/entries/gdacs-disaster-alerts.json",
"src/library/entries/gsi-japan-map.json",
"src/library/entries/ign-orthophotos-france.json",
"src/library/entries/gsi-japan-photo.json",
"src/library/entries/jrc-surface-water.json",
"src/library/entries/ntad-national-bridges.json",
"src/library/entries/usgs-hydrography.json",
"src/library/entries/macrostrat-geology.json",
"src/library/entries/swisstopo-swissimage.json",
"src/library/entries/usgs-topo.json",
"src/library/outlines.js",
"src/library/relief.js",
"src/sceneCommandRelease.js",
"src/scopeMaskPolicy.js",
"src/visualStylePolicy.js",
@@ -3594,7 +3781,10 @@
"src/layers/satellites/sensors.js",
"src/data/ucsSatellites.js",
"src/data/imagingSensors.js",
"src/data/planespottersPhoto.js"
"src/data/planespottersPhoto.js",
"src/ui/stores/dataLibrary.js",
"src/data/openVesselPhoto.js",
"src/layers/vessels/photos.js"
],
"external": [
"hls.js",
@@ -3606,6 +3796,8 @@
"satellite.js",
"mapillary-js",
"@jtarrio/webrtlsdr",
"hyparquet",
"fzstd",
"vue",
"pinia",
"vue-i18n"
@@ -4072,6 +4264,27 @@
"src/i18n/marker.js",
"src/sources/nominatim.js",
"src/app/layers/transit.js",
"src/app/layers/rail.js",
"src/data/railFeeds.js",
"src/layers/rail/index.js",
"src/layers/rail/source.js",
"src/layers/rail/policy.js",
"src/layers/rail/ingestion.js",
"src/layers/rail/dedupe.js",
"src/layers/rail/rendering.js",
"src/layers/rail/selection.js",
"src/layers/rail/follow.js",
"src/layers/rail/viewport.js",
"src/layers/rail/controls.js",
"src/layers/rail/lifecycle.js",
"src/layers/rail/pathBinding.js",
"src/layers/rail/routeDisplay.js",
"src/layers/rail/routeLines.js",
"src/layers/rail/schedules.js",
"src/layers/rail/stopLabels.js",
"src/data/railPaths.js",
"src/data/railSchedule.js",
"src/data/railTimeZones.js",
"src/data/transitFeeds.js",
"src/data/rioRouteNames.js",
"src/data/transitIcons.js",
@@ -4281,7 +4494,9 @@
"src/layers/satellites/sensors.js",
"src/data/ucsSatellites.js",
"src/data/imagingSensors.js",
"src/data/planespottersPhoto.js"
"src/data/planespottersPhoto.js",
"src/layers/vessels/photos.js",
"src/data/openVesselPhoto.js"
],
"external": [
"hls.js",
@@ -4383,7 +4598,11 @@
},
"layer-lifecycle": {
"exports": ["./data/lifecycle"],
"modules": ["src/data/lifecycle.js", "src/data/cameraArrival.js"],
"modules": [
"src/data/lifecycle.js",
"src/data/cameraArrival.js",
"src/data/libraryLayerIds.js"
],
"external": []
},
"ui-navigation": {
@@ -5090,6 +5309,63 @@
],
"external": []
},
"rail-provider": {
"runtime": "node",
"exports": ["./server/providers/rail"],
"modules": [
"server/providers/rail.js",
"src/data/railFeeds.js",
"src/data/railProxy.js",
"src/data/irishRail.js",
"src/data/railSiriVm.js",
"src/data/railTripUpdates.js",
"src/data/railScheduleIndex.js",
"src/data/railSchedule.js",
"src/data/railTimeZones.js",
"src/data/railNoPositions.js",
"src/data/transitFeeds.js",
"src/data/rioRouteNames.js",
"src/geoDistance.js",
"src/data/transitProxy.js",
"src/data/gtfsRealtime.js",
"src/sources/httpBody.js",
"src/i18n/marker.js",
"src/sources/transitHistory.js",
"src/sources/railService.js",
"src/sources/rateLimit.js"
],
"external": ["pbf"]
},
"rail-source-factory": {
"exports": ["./layers/rail/source"],
"modules": ["src/layers/rail/source.js"],
"external": []
},
"rail-service": {
"exports": ["./sources/rail-service"],
"modules": [
"src/data/railFeeds.js",
"src/data/railProxy.js",
"src/data/irishRail.js",
"src/data/railSiriVm.js",
"src/data/railTripUpdates.js",
"src/data/railScheduleIndex.js",
"src/data/railSchedule.js",
"src/data/railTimeZones.js",
"src/data/railNoPositions.js",
"src/data/transitFeeds.js",
"src/data/rioRouteNames.js",
"src/geoDistance.js",
"src/data/transitProxy.js",
"src/data/gtfsRealtime.js",
"src/sources/httpBody.js",
"src/i18n/marker.js",
"src/sources/transitHistory.js",
"src/sources/railService.js",
"src/sources/rateLimit.js"
],
"external": ["pbf"]
},
"transit-service": {
"exports": ["./sources/transit-service"],
"modules": [
@@ -5472,7 +5748,8 @@
"src/layers/satellites/sensors.js",
"src/data/ucsSatellites.js",
"src/data/imagingSensors.js",
"src/data/planespottersPhoto.js"
"src/data/planespottersPhoto.js",
"src/data/openVesselPhoto.js"
],
"external": ["@mapbox/vector-tile", "pbf"]
},
+737
View File
@@ -0,0 +1,737 @@
#!/usr/bin/env node
/**
* Data Library live gate. Run against this candidate's dev or preview server:
*
* npm run qa:data-library -- --url http://localhost:4173
*
* Every upstream a library entry names is answered from fixtures in the
* browser (EONET, the EIA ArcGIS pages, the GIBS tiles and the proxied
* volcano route), so the gate is deterministic and offline apart from the
* app itself. It drives the real panel and dialog: Browse opens a modal,
* Add registers a layer under the Library heading and switches it on, the
* share link carries `lib=`, a reload restores the selection, Escape and
* clean view close the dialog, and Remove destroys the layer.
*/
import assert from 'node:assert/strict';
const { default: puppeteer } = await import('puppeteer');
const args = process.argv.slice(2);
const urlIndex = args.indexOf('--url');
const base = urlIndex >= 0 ? args[urlIndex + 1] : 'http://localhost:4173';
const point = (lon, lat, properties) => ({
type: 'Feature',
geometry: { type: 'Point', coordinates: [lon, lat] },
properties,
});
const collection = (features, extra = {}) =>
JSON.stringify({ type: 'FeatureCollection', features, ...extra });
// 1×1 transparent PNG.
const PNG = Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==',
'base64',
);
const seen = {
eonet: 0,
arcgisPages: [],
tiles: 0,
proxied: 0,
structureBoxes: [],
};
function fixtureFor(url) {
const cors = { 'Access-Control-Allow-Origin': '*' };
if (url.hostname === 'eonet.gsfc.nasa.gov') {
seen.eonet += 1;
return {
status: 200,
headers: cors,
contentType: 'application/json',
body: collection([
point(-120, 38, {
title: 'Fixture Fire',
date: '2026-10-20T00:00:00Z',
}),
point(140, 35, {
title: 'Fixture Storm',
date: '2026-10-19T00:00:00Z',
}),
]),
};
}
if (
url.hostname === 'services2.arcgis.com' &&
url.pathname.includes('USA_Structures')
) {
seen.structureBoxes.push(url.searchParams.get('geometry'));
const square = (lon, lat) => ({
type: 'Feature',
geometry: {
type: 'Polygon',
coordinates: [
[
[lon, lat],
[lon + 0.0004, lat],
[lon + 0.0004, lat + 0.0003],
[lon, lat + 0.0003],
[lon, lat],
],
],
},
properties: { OCC_CLS: 'Government', HEIGHT: 30 },
});
return {
status: 200,
headers: cors,
contentType: 'application/json',
body: collection([
square(-77.035, 38.892),
square(-77.03, 38.893),
square(-77.025, 38.894),
]),
};
}
if (url.hostname === 'services2.arcgis.com') {
const offset = Number(url.searchParams.get('resultOffset'));
seen.arcgisPages.push(offset);
const features =
offset === 0
? [
point(-118.9, 47.9, { Plant_Name: 'Fixture Dam', Total_MW: 6809 }),
point(-112.9, 33.4, {
Plant_Name: 'Fixture Nuclear',
Total_MW: 3937,
}),
]
: [point(-97.6, 28.8, { Plant_Name: 'Fixture Coast', Total_MW: 2560 })];
return {
status: 200,
headers: cors,
contentType: 'application/json',
body: collection(features, {
properties: { exceededTransferLimit: offset === 0 },
}),
};
}
if (url.hostname === 'gibs.earthdata.nasa.gov') {
seen.tiles += 1;
return { status: 200, headers: cors, contentType: 'image/png', body: PNG };
}
if (url.pathname === '/api/library/data/gvp-holocene-volcanoes') {
seen.proxied += 1;
return {
status: 200,
contentType: 'application/json',
body: collection([
point(-121.76, 46.85, { name: 'Fixture Volcano', __gevDetails: [] }),
]),
};
}
return null;
}
const browser = await puppeteer.launch({
headless: true,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH || (await puppeteer.executablePath()),
defaultViewport: { width: 1440, height: 900 },
protocolTimeout: 300_000,
args: ['--no-sandbox', '--use-gl=angle', '--use-angle=swiftshader'],
});
const errors = [];
let count = 0;
const step = async (label, fn) => {
await fn();
console.log(`ok ${++count} ${label}`);
};
async function open(page, hash = '') {
await page.goto(`${base}/${hash}`, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(
() =>
window.__gevDataLibrary?.library.ready &&
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 180_000 },
);
}
const libraryRows = (page) =>
page.evaluate(() =>
[...document.querySelectorAll('#data-toggles [data-library-entry]')].map(
(row) => ({
id: row.dataset.libraryEntry,
on: row.querySelector('.data-toggle-btn')?.classList.contains('active'),
count: row.querySelector('.data-count')?.textContent,
}),
),
);
try {
const page = await browser.newPage();
page.on('console', (message) => {
if (message.type() === 'error' && /Library|library/.test(message.text()))
errors.push(message.text());
});
page.on('pageerror', (error) => errors.push(String(error)));
await page.setRequestInterception(true);
page.on('request', (request) => {
const fixture = fixtureFor(new URL(request.url()));
if (fixture) request.respond(fixture);
else request.continue();
});
await page.evaluateOnNewDocument(() => {
try {
// Start empty once; the reload step needs what this run stored.
if (!sessionStorage.getItem('qa-data-library')) {
localStorage.removeItem('gev:library:v1');
sessionStorage.setItem('qa-data-library', '1');
}
// Skip the first-run chooser (src/firstRunExperience.js).
localStorage.setItem('gev:first-run-mission:v1', 'suppressed');
} catch {
/* storage may be blocked; the gate still runs */
}
});
await open(page);
await step(
'the panel offers Browse and lists no library layer by default',
async () => {
assert.deepEqual(await libraryRows(page), []);
const hidden = await page.$eval(
'#data-library-browse',
(node) => node.hidden,
);
assert.equal(hidden, false);
},
);
await step(
'Browse opens a labelled modal with addable entries first and packs last',
async () => {
await page.evaluate(() => {
document.getElementById('data-panel')?.classList.remove('collapsed');
document.getElementById('data-library-browse').click();
});
const dialog = await page.$eval('#data-library', (node) => ({
open: node.open,
modal: node.matches(':modal'),
title: document.getElementById(node.getAttribute('aria-labelledby'))
?.textContent,
focus: document.activeElement?.type,
order: [...node.querySelectorAll('.data-library-item')].map(
(item) => item.dataset.libraryId,
),
}));
assert.equal(dialog.open, true);
assert.equal(dialog.modal, true);
assert.equal(dialog.title, 'DATA LIBRARY');
assert.equal(dialog.focus, 'search');
assert.ok(dialog.order.at(-1).startsWith('cctv-'), dialog.order.join());
assert.ok(!dialog.order[0].startsWith('cctv-'));
},
);
await step(
'a pack explains the operator switch instead of offering Add',
async () => {
const pack = await page.$eval(
'#data-library .data-library-item[data-library-id="cctv-nyc-dot"]',
(item) => ({
action: Boolean(item.querySelector('.data-library-action')),
reason: item.querySelector('.data-library-reason')?.textContent,
}),
);
assert.equal(pack.action, false);
assert.match(pack.reason, /CCTV_NYC_ENABLED=1/);
},
);
for (const [id, expected] of [
['nasa-eonet-events', '2'],
['eia-power-plants', '3'],
['gvp-holocene-volcanoes', '1'],
['nasa-black-marble', 'DRAPED'],
])
await step(
`Add ${id} registers it under Library and switches it on`,
async () => {
await page.click(
`#data-library button[data-library-id="${id}"][data-library-action="add"]`,
);
await page.waitForFunction(
(layerId, text) => {
const row = document.querySelector(
`#data-toggles [data-layer-id="${layerId}"]`,
);
return (
row
?.querySelector('.data-toggle-btn')
?.classList.contains('active') &&
row.querySelector('.data-count')?.textContent === text
);
},
{ timeout: 60_000 },
`lib-${id}`,
expected,
);
},
);
await step(
'the fixtures were reached the way each manifest says',
async () => {
assert.ok(seen.eonet >= 1);
assert.deepEqual(seen.arcgisPages.slice(0, 2), [0, 2]);
assert.ok(seen.proxied >= 1);
const draped = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('lib-nasa-black-marble')
.module.getDiagnostics().draped,
);
assert.equal(draped, true);
// Imagery requests tiles on the following frames, not on enable.
for (let waited = 0; seen.tiles === 0 && waited < 30_000; waited += 250)
await new Promise((resolve) => setTimeout(resolve, 250));
assert.ok(seen.tiles >= 1, 'the drape requested its tiles');
const headings = await page.$$eval(
'#data-toggles .data-layer-group-heading',
(nodes) => nodes.map((node) => node.textContent),
);
assert.equal(headings.at(-1), 'Library');
},
);
await step(
'the stats strip sits above the intro and counts what is on the map',
async () => {
await page.waitForFunction(
() =>
document.querySelector(
'[data-stat="features"] .data-library-stat-value',
)?.textContent === '6',
{ timeout: 15_000 },
);
const strip = await page.evaluate(() => {
const stats = document.querySelector(
'#data-library .data-library-stats',
);
const read = (id) =>
[...document.querySelectorAll(`[data-stat="${id}"] dd`)].map(
(node) => node.textContent,
);
return {
beforeIntro:
stats?.nextElementSibling?.classList.contains('data-library-intro'),
label: stats?.getAttribute('aria-label'),
entries: read('entries'),
added: read('added'),
features: read('features'),
};
});
assert.equal(strip.beforeIntro, true);
assert.equal(strip.label, 'Data Library at a glance');
// Counted from the shipped catalog, so a new entry does not break this.
const kinds = await page.evaluate(() => {
const counts = { dataset: 0, tiles: 0, pack: 0 };
for (const { entry } of window.__gevDataLibrary.library.list())
counts[entry.kind] += 1;
return counts;
});
assert.deepEqual(strip.entries, [
String(kinds.dataset + kinds.tiles + kinds.pack),
`${kinds.dataset} datasets · ${kinds.tiles} tile layers · ${kinds.pack} packs`,
]);
assert.deepEqual(strip.added, ['4', '4 on the map']);
// Fixtures: 2 EONET events, 3 power plants, 1 volcano.
assert.deepEqual(strip.features, ['6', 'from 3 datasets']);
},
);
await step('the share link carries lib= with the ledger tokens', async () => {
await page.waitForFunction(
() => /(?:^|&)lib=00\.02\.01\.03(?:&|$)/.test(location.hash.slice(1)),
{ timeout: 15_000 },
);
});
await step(
'Escape closes the dialog and returns focus to Browse',
async () => {
await page.focus('#data-library input[type="search"]');
await page.keyboard.press('Escape');
const state = await page.evaluate(() => ({
open: document.getElementById('data-library').open,
focus: document.activeElement?.id,
}));
assert.deepEqual(state, { open: false, focus: 'data-library-browse' });
},
);
await step('clean view closes an open dialog', async () => {
const open = await page.evaluate(async () => {
window.__gevDataLibrary.browser.open();
document.body.classList.add('ui-clean-view');
await new Promise((resolve) => setTimeout(resolve, 50));
const result = document.getElementById('data-library').open;
document.body.classList.remove('ui-clean-view');
return result;
});
assert.equal(open, false);
});
await step(
'a reload restores the added entries and what was on',
async () => {
await page.evaluate(() => {
document
.querySelector(
'#data-toggles [data-layer-id="lib-gvp-holocene-volcanoes"] .data-toggle-btn',
)
.click();
});
await page.waitForFunction(
() =>
JSON.parse(localStorage.getItem('gev:library:v1')).on.length === 3,
);
await open(page, '');
await page.waitForFunction(
() =>
document.querySelectorAll('#data-toggles [data-library-entry]')
.length === 4,
{ timeout: 60_000 },
);
const rows = await libraryRows(page);
assert.deepEqual(rows.map(({ id, on }) => [id, on]).sort(), [
['eia-power-plants', true],
['gvp-holocene-volcanoes', false],
['nasa-black-marble', true],
['nasa-eonet-events', true],
]);
},
);
const cameraCenter = () =>
page.evaluate(() => {
const { camera } = window.__godsEyeView.viewer;
const carto = camera.positionCartographic;
return {
lon: (carto.longitude * 180) / Math.PI,
lat: (carto.latitude * 180) / Math.PI,
height: carto.height,
};
});
const waitForFlight = () =>
page.waitForFunction(
() => !window.__godsEyeView.viewer.camera._currentFlight,
{ timeout: 30_000, polling: 200 },
);
await step(
'Zoom to layer on the row works while off: it switches on and flies',
async () => {
const chip =
'#data-toggles [data-layer-id="lib-eia-power-plants"] [data-chip-id="zoom"]';
await page.evaluate(() =>
document.getElementById('data-panel')?.classList.remove('collapsed'),
);
// The icons work while the layer is off; Zoom switches it back on.
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'lib-eia-power-plants',
false,
{ origin: 'user' },
),
);
await page.waitForSelector(chip, { visible: true, timeout: 30_000 });
await page.evaluate(
(selector) => document.querySelector(selector).click(),
chip,
);
await page.waitForFunction(
() =>
window.__godsEyeView.dataManager.isEnabled('lib-eia-power-plants'),
{ timeout: 30_000 },
);
await new Promise((resolve) => setTimeout(resolve, 300));
await waitForFlight();
const center = await cameraCenter();
// Fixture plants span -118.9…-97.6 °E and 28.8…47.9 °N.
assert.ok(center.lon > -125 && center.lon < -90, JSON.stringify(center));
assert.ok(center.lat > 22 && center.lat < 55, JSON.stringify(center));
},
);
await step(
'ZOOM in the dialog closes it and flies to that entry',
async () => {
await page.evaluate(() => window.__gevDataLibrary.browser.open());
const zoom =
'#data-library button[data-library-id="gvp-holocene-volcanoes"][data-library-action="zoom"]';
assert.ok(await page.$(zoom), 'Zoom is offered for an entry that is off');
await page.click(
'#data-library button[data-library-id="nasa-eonet-events"][data-library-action="zoom"]',
);
const open = await page.$eval('#data-library', (node) => node.open);
assert.equal(open, false);
await new Promise((resolve) => setTimeout(resolve, 300));
await waitForFlight();
const center = await cameraCenter();
// Fixture events at (-120, 38) and (140, 35): the frame sits between them.
assert.ok(center.lat > 20 && center.lat < 55, JSON.stringify(center));
},
);
await step('Remove destroys the layer and forgets it', async () => {
await page.evaluate(() => window.__gevDataLibrary.browser.open());
await page.click(
'#data-library button[data-library-id="nasa-black-marble"][data-library-action="remove"]',
);
await page.waitForFunction(
() =>
!document.querySelector(
'#data-toggles [data-layer-id="lib-nasa-black-marble"]',
) &&
!window.__godsEyeView.dataManager.layers.has('lib-nasa-black-marble'),
{ timeout: 30_000 },
);
const stored = await page.evaluate(() =>
JSON.parse(localStorage.getItem('gev:library:v1')),
);
assert.equal(stored.added.includes('nasa-black-marble'), false);
});
await step(
'the row delete icon removes that entry, even while it is off',
async () => {
await page.evaluate(() => {
window.__gevDataLibrary.browser.close();
document.getElementById('data-panel')?.classList.remove('collapsed');
});
const chip =
'#data-toggles [data-layer-id="lib-eia-power-plants"] [data-chip-id="remove"]';
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'lib-eia-power-plants',
false,
{ origin: 'user' },
),
);
await page.waitForSelector(chip, { visible: true, timeout: 30_000 });
const label = await page.$eval(chip, (node) => ({
name: node.getAttribute('aria-label'),
glyph: node.textContent,
pressed: node.getAttribute('aria-pressed'),
}));
assert.deepEqual(label, {
name: 'Remove from Data Layers',
glyph: 'delete',
pressed: null,
});
await page.evaluate(
(selector) => document.querySelector(selector).click(),
chip,
);
await page.waitForFunction(
() =>
!window.__godsEyeView.dataManager.layers.has(
'lib-eia-power-plants',
) &&
!document.querySelector(
'#data-toggles [data-layer-id="lib-eia-power-plants"]',
),
{ timeout: 30_000 },
);
},
);
await step(
'a view-bounded layer asks to zoom in, then loads only what is in view',
async () => {
await page.evaluate(() =>
window.__gevDataLibrary.library.add('fema-usa-structures'),
);
const row = '#data-toggles [data-layer-id="lib-fema-usa-structures"]';
await page.waitForFunction(
(selector) =>
/zoom in to load this layer/.test(
document.querySelector(`${selector} .data-toggle-meta`)
?.textContent || '',
),
{ timeout: 30_000 },
row,
);
assert.equal(seen.structureBoxes.length, 0, 'nothing loads from orbit');
await page.evaluate(() =>
window.__gevDataLibrary.library.zoom('fema-usa-structures'),
);
await page.waitForFunction(
() =>
window.__godsEyeView.dataManager.layers
.get('lib-fema-usa-structures')
.module.getDiagnostics().count === 3,
{ timeout: 60_000, polling: 250 },
);
const altitude = (await cameraCenter()).height;
assert.ok(altitude <= 4000, `landed at ${Math.round(altitude)} m`);
const [west, south, east, north] = seen.structureBoxes
.at(-1)
.split(',')
.map(Number);
assert.ok(east - west < 0.2 && north - south < 0.2, 'a view-sized box');
assert.ok(
west < -77.03 && east > -77.03 && south < 38.893 && north > 38.893,
);
// Climbing back out clears the buildings and shows the guidance again.
await page.evaluate(() => {
const { viewer, requestRender } = window.__godsEyeView;
viewer.camera.zoomOut(60_000);
requestRender?.('qa-data-library');
});
await page.waitForFunction(
() =>
window.__godsEyeView.dataManager.layers
.get('lib-fema-usa-structures')
.module.getDiagnostics().tooHigh,
{ timeout: 30_000, polling: 250 },
);
},
);
await step('Remove all asks twice, then clears the library', async () => {
await page.evaluate(() => window.__gevDataLibrary.browser.open());
await page.click('#data-library .data-library-reset');
const armed = await page.$eval(
'#data-library .data-library-reset',
(node) => ({
text: node.textContent,
armed: node.classList.contains('is-armed'),
}),
);
assert.equal(armed.armed, true);
assert.match(armed.text, /CONFIRM/);
assert.ok(
(await page.$$('#data-toggles [data-library-entry]')).length > 0,
'one press removes nothing',
);
await page.click('#data-library .data-library-reset');
await page.waitForFunction(
() =>
!document.querySelector('#data-toggles [data-library-entry]') &&
document.querySelector('#data-library .data-library-reset').hidden,
{ timeout: 30_000 },
);
const stored = await page.evaluate(() =>
JSON.parse(localStorage.getItem('gev:library:v1')),
);
assert.deepEqual(stored, { added: [], on: [] });
});
const onIds = (target) =>
target.evaluate(() =>
window.__gevDataLibrary.library
.list()
.filter((row) => row.on)
.map((row) => row.entry.id)
.sort(),
);
await step(
'a copied /share link opens the same library layers in a clean browser',
async () => {
await page.evaluate(async () => {
const { library } = window.__gevDataLibrary;
await library.add('nasa-eonet-events');
await library.add('nasa-black-marble');
});
await page.waitForFunction(
() => /(?:^|&)lib=00\.03(?:&|$)/.test(location.hash.slice(1)),
{ timeout: 15_000 },
);
const href = await page.evaluate(() =>
window.__gevDataLibrary.buildShareLink(),
);
assert.match(new URL(href).pathname, /\/share\//);
// A separate profile: nothing stored, nothing added.
const context = await browser.createBrowserContext();
try {
const recipient = await context.newPage();
await recipient.setRequestInterception(true);
recipient.on('request', (request) => {
const fixture = fixtureFor(new URL(request.url()));
if (fixture) request.respond(fixture);
else request.continue();
});
await recipient.evaluateOnNewDocument(() => {
try {
localStorage.setItem('gev:first-run-mission:v1', 'suppressed');
} catch {
/* the gate still runs */
}
});
await recipient.goto(href, { waitUntil: 'domcontentloaded' });
await recipient.waitForFunction(
() => window.__gevDataLibrary?.library.ready,
{ timeout: 180_000 },
);
await recipient.waitForFunction(
() =>
window.__gevDataLibrary.library.list().filter((row) => row.on)
.length === 2,
{ timeout: 60_000 },
);
assert.deepEqual(await onIds(recipient), [
'nasa-black-marble',
'nasa-eonet-events',
]);
} finally {
await context.close();
}
},
);
await step(
'a share opened in the same tab switches library layers on and off',
async () => {
const navigate = (lib) =>
page.evaluate((value) => {
const params = new URLSearchParams(location.hash.slice(1));
params.set('lat', String(Number(params.get('lat')) + 1));
if (value === null) params.delete('lib');
else params.set('lib', value);
location.hash = params.toString();
}, lib);
await navigate('00');
await page.waitForFunction(
() => {
const on = window.__gevDataLibrary.library
.list()
.filter((row) => row.on)
.map((row) => row.entry.id);
return on.length === 1 && on[0] === 'nasa-eonet-events';
},
{ timeout: 30_000 },
);
// A share with every library layer off: the added ones stay listed, off.
await navigate(null);
await page.waitForFunction(
() => window.__gevDataLibrary.library.list().every((row) => !row.on),
{ timeout: 30_000 },
);
assert.deepEqual(
(await libraryRows(page)).map(({ id, on }) => [id, on]).sort(),
[
['nasa-black-marble', false],
['nasa-eonet-events', false],
],
);
},
);
assert.deepEqual(errors, [], 'no library errors on the page');
console.log(`# ${count} passed`);
} finally {
await browser.close();
}
+244
View File
@@ -46,6 +46,15 @@ const VIEWPORT = { width: 1440, height: 900 };
/** Over downtown Sacramento, above the fixture photo line, looking down. */
const PARK = { lon: PHOTO_LINE.lon, lat: 38.5816, height: 900, pitch: -1.3 };
const DAY_MS = 86_400_000;
/** A Sacramento street with Street View, for the sl=google link step. */
const STREET_VIEW_STREET = Object.freeze({ lon: -121.49372, lat: 38.57986 });
/**
* What a Street View lookup and load cost: the Maps JS API's
* SingleImageSearch (the nearest lookup), GetMetadata and the panorama tiles.
* Script loads are not counted.
*/
const STREET_VIEW_REQUEST =
/MapsJsInternalService\/(SingleImageSearch|GetMetadata)|GeoPhotoService|streetviewpixels-pa\.googleapis\.com/;
/**
* Coverage for any street-zoom tile: a grid of 360°/flat, recent/old
@@ -234,6 +243,28 @@ async function boot(page, url) {
});
}
/**
* Open a share link as a fresh document. Changing only the hash of a live
* page and reloading races the app's share-link writer, which can put its own
* state back in the hash before the reload reads it. The page's fixtures (a
* CDP session on the page) carry over.
*/
async function bootLink(page, link) {
await page.goto('about:blank');
await boot(page, link);
}
/** The camera's position in degrees and metres. */
const cameraAt = (page) =>
page.evaluate(() => {
const c = window.__godsEyeView.viewer.camera.positionCartographic;
return {
lon: (c.longitude * 180) / Math.PI,
lat: (c.latitude * 180) / Math.PI,
height: c.height,
};
});
/** Run `test(uiState, arg)` in the page until it is true. */
const uiUntil = (page, test, arg = null, timeout = 30_000) =>
page.waitForFunction(
@@ -780,6 +811,219 @@ async function main() {
await uiUntil(page, litAlone, 'mapillary');
await park(page);
const base = url.split('#')[0];
await fixtureStep(
'a link to street-level photos (sl=) opens the photo at its point once the link restores',
async () => {
const photo = fixture.images[0];
// The camera sits ~330 m north of the photo: the lookup must use
// the link's point, not the view centre.
const cameraLat = Number((photo.lat + 0.003).toFixed(6));
await bootLink(
page,
`${base}#v=2&lat=${cameraLat}&lon=${photo.lon}&alt=600&heading=0&pitch=-60&roll=0&sl=mapillary,${photo.lat},${photo.lon}`,
);
await uiUntil(
page,
(u) => (u.street.imageId && !u.street.loading) || u.street.error,
null,
90_000,
);
const { street } = await getUI(page);
assert.equal(street.error, null, `viewer error: ${street.error}`);
assert.equal(street.providerId, 'mapillary');
assert.equal(
street.imageId,
photo.id,
`the photo at the link's point, not the one nearest the camera: ${street.imageId}`,
);
assert.equal(
await page.evaluate(() => location.hash.includes('sl=')),
false,
'the rewritten link no longer carries the one-shot request',
);
},
);
await fixtureStep(
'the show_street_level action opens the nearest photo at a point without a pointer',
async () => {
await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('street-level')
.module.closeViewer(),
);
const photo = fixture.images[0];
const result = await page.evaluate(
(at) =>
window.__godsEyeView.voiceCommands.runner('show_street_level', {
latitude: at.lat,
longitude: at.lon,
provider: 'mapillary',
fly: false,
}),
{ lat: photo.lat, lon: photo.lon },
);
assert.equal(
result.ok,
true,
`action failed: ${result.stage} ${result.error}`,
);
assert.equal(result.provider, 'mapillary');
assert.ok(
fixture.images.some((image) => image.id === result.imageId),
`a fixture photo opened: ${result.imageId}`,
);
assert.match(result.say || '', /^Mapillary at /);
},
);
await step(
'an sl=google link never loads a photo by itself: keyless it stops at the key, keyed it arms and waits for OPEN',
async ({ skipped }) => {
const point = STREET_VIEW_STREET;
// ~330 m north of the street: a flight would move the camera, and a
// press that ignored the armed point would open something else.
const cameraLat = Number((point.lat + 0.003).toFixed(6));
const googleRequests = [];
const onRequest = (request) => {
if (STREET_VIEW_REQUEST.test(request.url()))
googleRequests.push(request.url());
};
page.on('request', onRequest);
try {
await bootLink(
page,
`${base}#v=2&lat=${cameraLat}&lon=${point.lon}&alt=600&heading=0&pitch=-60&roll=0&map=osm&sl=google,${point.lat},${point.lon}`,
);
await page.waitForFunction(
(want) => {
const c =
window.__godsEyeView.viewer.camera.positionCartographic;
return Math.abs((c.latitude * 180) / Math.PI - want) < 0.0005;
},
{ timeout: 60_000 },
cameraLat,
);
const googleKey = await page.evaluate(() =>
Boolean(window.__GOOGLE_MAPS_API_KEY__),
);
if (!googleKey) {
// Without a key the action stops at the key, before any flight,
// map switch or source change, and the direct call says why.
const direct = await page.evaluate(
(at) =>
window.__godsEyeView.voiceCommands.runner(
'show_street_level',
{
latitude: at.lat,
longitude: at.lon,
provider: 'google',
fly: false,
autoOpen: false,
},
),
point,
);
assert.equal(direct.ok, false);
assert.equal(
direct.stage,
'key',
`${direct.stage} ${direct.error}`,
);
const after = await page.evaluate(() => {
const m =
window.__godsEyeView.dataManager.layers.get(
'street-level',
).module;
const u = m.getUIState();
return {
stack: window.__godsEyeView.mapStackController.getActiveId(),
google: m.getParams().google === true,
armed: u.street.armed,
open: u.street.open,
};
});
assert.deepEqual(
after,
{ stack: 'osm', google: false, armed: null, open: false },
'a keyless Street View link changes nothing',
);
const still = await cameraAt(page);
assert.ok(
Math.abs(still.lat - cameraLat) < 0.0005,
`the camera did not fly: ${still.lat}`,
);
assert.deepEqual(googleRequests, [], 'no Street View request');
return;
}
const photoreal = await page.evaluate(() =>
window.__godsEyeView.mapStackController.isStackAvailable(
'photoreal',
),
);
if (!photoreal) return skipped('no Google 3D');
await uiUntil(
page,
(u) =>
u.street.armed !== null &&
u.providers.some((p) => p.id === 'google' && p.on),
null,
90_000,
);
const beforePress = googleRequests.length;
const waiting = await getUI(page);
assert.equal(
waiting.street.open,
false,
'nothing open without a click',
);
assert.equal(waiting.street.imageId, null);
assert.deepEqual(
waiting.street.armed.point,
{ lat: point.lat, lon: point.lon },
'armed at the link point',
);
assert.equal(
beforePress,
0,
'no Street View request before the press',
);
await expandPanel(page);
await press(page, '#sl-nearest-btn');
await uiUntil(
page,
(u) => (u.street.imageId && !u.street.loading) || u.street.error,
null,
90_000,
);
const { street } = await getUI(page);
assert.equal(street.error, null, `viewer error: ${street.error}`);
assert.equal(street.providerId, 'google');
assert.ok(
googleRequests.length >= 1,
'the press made the Street View request',
);
const metres = Math.hypot(
(street.position.lat - point.lat) * 111_320,
(street.position.lon - point.lon) *
111_320 *
Math.cos((point.lat * Math.PI) / 180),
);
assert.ok(
metres <= 250,
`within 250 m of the link point: ${Math.round(metres)} m`,
);
// Back to Mapillary: the keyless page restores the saved layer
// and expects its KEY REQUIRED.
await press(page, '#sl-provider-chips [data-chip-id="mapillary"]');
await uiUntil(page, litAlone, 'mapillary');
} finally {
page.off('request', onRequest);
}
},
);
await fixtureStep(
// The fixture bundle carries a token, so this covers a keyless
// server behind a keyed bundle; the keyless default is checked above.
+1
View File
@@ -904,6 +904,7 @@ const SLOW_TOOLS = new Set([
'annotate_map',
'fly_to_location',
'select_nearest_aircraft',
'show_street_level',
'analyst_query',
'fly_route',
'frame_overhead',
+8
View File
@@ -35,6 +35,14 @@ export const CORE_PHRASES = Object.freeze([
{ phrase: 'Show me the whole earth', expect: 'zoom_to_globe' },
{ phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead', typedArgs: [where('frame_overhead', (a) => a.target === 'flights')] },
// — street-level photos: one composite action, never fly + basemap —
{ phrase: 'Show me the Eiffel Tower on Street View', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'google' && /eiffel/i.test(a.query || ''))] },
{ phrase: 'Street view of 10th Street in Sacramento', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'google')] },
{ phrase: 'Show me Times Square on Mapillary', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'mapillary')] },
{ phrase: 'Switch to Google Street View', expect: 'show_street_level', typedArgs: [where('show_street_level', (a) => a.provider === 'google' && !a.query)] },
{ phrase: 'Show me the Eiffel Tower', expect: 'fly_to_location' },
{ phrase: 'Turn on street level', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'street-level' && a.enabled === true)] },
// — the satellites trap: data layer, never basemap —
{ phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } },
{ phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'satellites' && a.enabled === false)] },
+392
View File
@@ -0,0 +1,392 @@
import { LIBRARY_ENTRIES } from '../../src/library/catalog.js';
import { loadLibraryDataset } from '../../src/library/formats.js';
import { buildPartitionIndex } from '../../src/library/parquet.js';
import {
libraryUpstreamUrl,
parseLibraryIdList,
libraryTileUrl,
} from '../../src/library/manifest.js';
import { KEY_SETUP_KEYS } from '../../src/keySetupCore.mjs';
import {
readResponseBytesCapped,
readResponseTextCapped,
coalesceProxyRequest,
} from './common/http.js';
import { makeRateLimiter, clientKey } from './common/rate-limit.js';
import { admitSameSite, admitSameSiteKeyless } from './common/same-site.js';
const DATASET_TIMEOUT_MS = 30_000;
const TILE_TIMEOUT_MS = 10_000;
/** Each listing page or footer read while building a partition index. */
const INDEX_TIMEOUT_MS = 20_000;
/** How long an incomplete partition index is served before a rebuild. */
const PARTIAL_INDEX_MS = 5 * 60_000;
const TILE_MAX_BYTES = 2 * 1024 * 1024;
const TILE_CACHE_LIMIT = 512;
const IN_FLIGHT_LIMIT = 64;
/**
* Availability of every entry on this instance, decided by the operator's
* environment alone: `GEV_LIBRARY_DISABLE` removes entries, opt-in entries
* need `GEV_LIBRARY_ENABLE` (or, for a pack, its host's own switch), and a
* keyed entry needs its key. The answer names a reason, never a secret.
* @param {object[]} entries
* @param {Record<string, string|undefined>} env
* @returns {Map<string, { available: boolean, reason: string|null }>}
*/
export function libraryAvailability(entries, env) {
const enabled = parseLibraryIdList(env.GEV_LIBRARY_ENABLE);
const disabled = parseLibraryIdList(env.GEV_LIBRARY_DISABLE);
const result = new Map();
for (const entry of entries) {
let reason = null;
if (disabled.has(entry.id)) reason = 'operator-disabled';
else if (entry.kind === 'pack') {
if (String(env[entry.pack.env] || '').trim() !== '1')
reason = 'operator-opt-in';
} else if (entry.licence.operatorOptIn && !enabled.has(entry.id))
reason = 'operator-opt-in';
else if (entry.source.key && !libraryKeyValue(entry.source.key, env))
reason = 'key-missing';
result.set(entry.id, { available: reason === null, reason });
}
return result;
}
/** The configured value for a manifest key, through the key registry. */
export function libraryKeyValue(key, env) {
const registered = KEY_SETUP_KEYS.find(({ id }) => id === key?.id);
for (const name of registered?.envVars || []) {
const value = String(env[name] || '').trim();
if (value) return value;
}
return null;
}
function withKey(url, key, env, headers) {
if (!key) return url;
const value = libraryKeyValue(key, env);
if (!value) throw Object.assign(new Error('key_missing'), { status: 503 });
if (key.in === 'header') {
headers[key.name] = value;
return url;
}
const keyed = new URL(url);
keyed.searchParams.set(key.name, value);
return keyed.href;
}
/**
* Parse `/<id>/<z>/<x>/<y>` against a tiles entry's zoom range.
* @returns {{ z: number, x: number, y: number } | null}
*/
export function parseLibraryTilePath(parts, source) {
if (parts.length !== 3 || !parts.every((part) => /^\d{1,8}$/.test(part)))
return null;
const [z, x, y] = parts.map(Number);
const minZoom = source.minZoom ?? 0;
const maxZoom = source.maxZoom ?? 22;
if (z < minZoom || z > maxZoom) return null;
const span = 2 ** z;
if (x >= span || y >= span) return null;
return { z, x, y };
}
/**
* The Data Library's one server route. Proxied datasets and tiles reach their
* manifest's fixed upstream only: the browser names an entry id, never a URL,
* so the allowlist is exactly the catalog. Redirects are refused, bodies are
* capped while streaming, keys come from this server's environment and
* upstream error text is never returned.
*/
export function libraryProxy({
entries = LIBRARY_ENTRIES,
env = process.env,
fetchImpl = (...args) => globalThis.fetch(...args),
now = () => Date.now(),
} = {}) {
const byId = new Map(entries.map((entry) => [entry.id, entry]));
const datasets = new Map();
const tiles = new Map();
const indexes = new Map();
const partialIndexes = new Map();
const inFlight = new Map();
const allowData = makeRateLimiter({
windowMs: 60_000,
max: 30,
globalMax: 600,
});
const allowTiles = makeRateLimiter({
windowMs: 60_000,
max: 600,
globalMax: 6000,
});
async function upstream(url, entry, headers, timeoutMs) {
// Defense in depth: every derived request stays on the manifest's host.
const target = libraryUpstreamUrl(url);
const declared = libraryUpstreamUrl(entry.source.url);
if (!target || !declared || target.host !== declared.host)
throw Object.assign(new Error('off_manifest_host'), { status: 502 });
const signal = AbortSignal.timeout(timeoutMs);
const response = await fetchImpl(url, {
signal,
redirect: 'error',
headers: { Accept: '*/*', ...headers },
});
if (!response.ok) {
await response.body?.cancel?.().catch?.(() => {});
throw Object.assign(new Error('upstream_unavailable'), { status: 502 });
}
return { response, signal };
}
async function fetchDataset(entry) {
const headers = {};
const collection = await loadLibraryDataset(entry, async (url, budget) => {
const keyed = withKey(url, entry.source.key, env, headers);
const { response, signal } = await upstream(
keyed,
entry,
headers,
DATASET_TIMEOUT_MS,
);
const text = await readResponseTextCapped(
response,
budget.remainingBytes,
signal,
);
budget.remainingBytes -= new TextEncoder().encode(text).byteLength;
return text;
});
return { ...collection, fetchedAt: now() };
}
async function cached(store, key, ttlMs, load, limit = Infinity) {
const previous = store.get(key);
if (previous && now() - previous.savedAt < ttlMs)
return { value: previous.value, stale: false };
if (!inFlight.has(key) && inFlight.size >= IN_FLIGHT_LIMIT)
throw Object.assign(new Error('busy'), { status: 429 });
try {
const { promise } = coalesceProxyRequest(inFlight, key, async () => {
const value = await load();
store.delete(key);
store.set(key, { value, savedAt: now() });
while (store.size > limit) store.delete(store.keys().next().value);
return value;
});
return { value: await promise, stale: false };
} catch (error) {
if (previous) return { value: previous.value, stale: true };
throw error;
}
}
function json(res, status, value, extra = {}) {
if (res.destroyed) return;
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(status === 405 ? { Allow: 'GET' } : {}),
...(status === 429 ? { 'Retry-After': '60' } : {}),
...extra,
});
res.end(JSON.stringify(value));
}
function gate(req, res, entry) {
// A keyed entry spends the operator's credential: refuse forwarded
// requests like the other cost-bearing routes. Keyless entries keep
// working behind a reverse proxy, like terrain heights.
return entry?.source?.key
? admitSameSite(req, res)
: admitSameSiteKeyless(req, res);
}
async function handler(req, res) {
if (req.method !== 'GET')
return json(res, 405, { error: 'method_not_allowed' });
const path = (req.url || '/').split('?')[0];
const parts = path.split('/').filter(Boolean);
const availability = libraryAvailability(entries, env);
if (parts.length === 1 && parts[0] === 'catalog') {
if (admitSameSiteKeyless(req, res)) return undefined;
return json(res, 200, {
entries: entries.map(({ id }) => ({ id, ...availability.get(id) })),
});
}
const [route, id, ...rest] = parts;
const entry = byId.get(id);
if (!entry || !['data', 'tile', 'index'].includes(route))
return json(res, 404, { error: 'unknown_entry' });
// A partitioned GeoParquet entry's index: which file covers where, read
// from each file's footer. Built once per cacheSeconds and shared, so
// visitors do not each list the bucket and read every footer.
if (route === 'index') {
if (rest.length || !entry.source?.partitions)
return json(res, 404, { error: 'not_partitioned' });
if (admitSameSiteKeyless(req, res)) return undefined;
const { available, reason } = availability.get(id);
if (!available) return json(res, 403, { error: reason });
if (!allowData(clientKey(req)))
return json(res, 429, { error: 'rate_limited' });
// Only a complete index is cached for cacheSeconds. An incomplete one
// (footers that kept failing) is served as `complete: false` and kept
// for PARTIAL_INDEX_MS only, so the next request soon rebuilds it; a
// complete index from before keeps being served while rebuilds fail.
const key = `index:${id}`;
const partial = partialIndexes.get(key);
if (partial && now() - partial.savedAt < PARTIAL_INDEX_MS) {
const previous = indexes.get(key);
return json(res, 200, previous?.value ?? partial.value);
}
try {
const { value } = await cached(
indexes,
key,
entry.source.cacheSeconds * 1000,
async () => {
const built = await buildPartitionIndex(entry, {
fetchImpl: (url, options = {}) =>
fetchImpl(url, {
...options,
signal:
options.signal ?? AbortSignal.timeout(INDEX_TIMEOUT_MS),
}),
});
const value = {
partitions: built.partitions,
complete: built.complete,
builtAt: now(),
};
if (!built.complete) {
// Recorded here too: with an earlier complete index, cached()
// serves that one and swallows this error, and the rebuild
// still waits PARTIAL_INDEX_MS.
partialIndexes.set(key, { value, savedAt: now() });
throw Object.assign(new Error('index_incomplete'), {
partialIndex: value,
});
}
partialIndexes.delete(key);
return value;
},
);
return json(res, 200, value);
} catch (error) {
if (error?.partialIndex) return json(res, 200, error.partialIndex);
const status = error?.status === 429 ? 429 : 502;
return json(res, status, { error: 'library_upstream_unavailable' });
}
}
const isDataset = route === 'data' && rest.length === 0;
const isTile = route === 'tile';
if (
entry.source.fetch !== 'proxied' ||
(isDataset && entry.kind !== 'dataset') ||
(isTile && entry.kind !== 'tiles') ||
(!isDataset && !isTile)
)
return json(res, 404, { error: 'not_proxied' });
if (gate(req, res, entry)) return undefined;
const { available, reason } = availability.get(id);
if (!available) return json(res, 403, { error: reason });
const ttlMs = entry.source.cacheSeconds * 1000;
if (isDataset) {
if (!allowData(clientKey(req)))
return json(res, 429, { error: 'rate_limited' });
try {
const { value, stale } = await cached(
datasets,
`data:${id}`,
ttlMs,
() => fetchDataset(entry),
);
return json(
res,
200,
stale ? { ...value, stale: true } : value,
stale ? { 'X-Data-Stale': 'true' } : {},
);
} catch (error) {
const status = [429, 503].includes(error?.status) ? error.status : 502;
return json(res, status, {
error:
status === 503 ? 'key_missing' : 'library_upstream_unavailable',
});
}
}
const tile = parseLibraryTilePath(rest, entry.source);
if (!tile) return json(res, 400, { error: 'invalid_tile' });
if (!allowTiles(clientKey(req)))
return json(res, 429, { error: 'rate_limited' });
try {
const key = `tile:${id}:${tile.z}/${tile.x}/${tile.y}`;
const { value } = await cached(
tiles,
key,
ttlMs,
async () => {
const headers = {};
const url = withKey(
libraryTileUrl(entry.source.url, tile.z, tile.x, tile.y),
entry.source.key,
env,
headers,
);
const { response, signal } = await upstream(
url,
entry,
headers,
TILE_TIMEOUT_MS,
);
const type = String(response.headers.get('content-type') || '')
.split(';')[0]
.trim()
.toLowerCase();
if (!/^image\/(png|jpeg|webp)$/.test(type)) {
await response.body?.cancel?.().catch?.(() => {});
throw Object.assign(new Error('not_an_image'), { status: 502 });
}
const bytes = await readResponseBytesCapped(
response,
TILE_MAX_BYTES,
signal,
);
return { type, bytes };
},
TILE_CACHE_LIMIT,
);
if (res.destroyed) return undefined;
res.writeHead(200, {
'Content-Type': value.type,
'Cache-Control': `private, max-age=${entry.source.cacheSeconds}`,
'X-Content-Type-Options': 'nosniff',
});
res.end(Buffer.from(value.bytes));
return undefined;
} catch (error) {
const status = [429, 503].includes(error?.status) ? error.status : 502;
return json(res, status, {
error: status === 503 ? 'key_missing' : 'library_upstream_unavailable',
});
}
}
return {
name: 'data-library',
handler,
configureServer({ middlewares }) {
middlewares.use('/api/library', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/library', handler);
},
};
}
+381
View File
@@ -0,0 +1,381 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
libraryAvailability,
libraryProxy,
parseLibraryTilePath,
} from './library.js';
import { validateLibraryCatalog } from '../../src/library/manifest.js';
import { libraryEntry } from '../../src/library/catalog.js';
const base = {
family: 'events',
provider: 'Fixture',
description: 'Fixture entry.',
maintainer: 'test',
coverage: { global: true },
licence: { name: 'Public domain', url: 'https://example.org/licence' },
attribution: { text: 'Fixture' },
status: 'community',
};
const { entries, errors } = validateLibraryCatalog([
{
...base,
id: 'proxied-points',
kind: 'dataset',
title: 'Proxied points',
source: {
format: 'geojson',
url: 'https://data.example.org/points.json',
fetch: 'proxied',
maxBytes: 4096,
},
display: { label: 'title' },
},
{
...base,
id: 'direct-points',
kind: 'dataset',
title: 'Direct points',
source: {
format: 'geojson',
url: 'https://data.example.org/direct.json',
fetch: 'browser-direct',
},
},
{
...base,
id: 'keyed-points',
kind: 'dataset',
title: 'Keyed points',
source: {
format: 'geojson',
url: 'https://keyed.example.org/points.json',
fetch: 'proxied',
key: { id: 'firms', in: 'query', name: 'MAP_KEY' },
},
},
{
...base,
id: 'opt-in-points',
kind: 'dataset',
title: 'Opt-in points',
source: {
format: 'geojson',
url: 'https://data.example.org/opt.json',
fetch: 'proxied',
},
licence: { ...base.licence, operatorOptIn: true },
},
{
...base,
id: 'proxied-tiles',
kind: 'tiles',
title: 'Proxied tiles',
source: {
format: 'xyz',
url: 'https://tiles.example.org/{z}/{x}/{y}.png',
fetch: 'proxied',
minZoom: 2,
maxZoom: 5,
},
},
{
...base,
id: 'fixture-pack',
kind: 'pack',
family: 'cameras',
title: 'Fixture cameras',
source: { format: 'host', url: 'https://cams.example.org/' },
pack: { host: 'cctv', env: 'CCTV_FIXTURE_ENABLED' },
licence: { ...base.licence, operatorOptIn: true },
},
]);
assert.deepEqual(errors, []);
const collection = (features) =>
JSON.stringify({ type: 'FeatureCollection', features });
const point = (title) => ({
type: 'Feature',
geometry: { type: 'Point', coordinates: [1, 2] },
properties: { title },
});
function install({ env = {}, fetchImpl, now, catalog = entries } = {}) {
const plugin = libraryProxy({ entries: catalog, env, fetchImpl, now });
let handler;
plugin.configurePreviewServer({
middlewares: {
use(path, callback) {
assert.equal(path, '/api/library');
handler = callback;
},
},
});
return async (url, headers = {}) => {
const res = {
headers: {},
writeHead(status, values) {
this.status = status;
this.headers = values;
},
setHeader(name, value) {
this.headers[name] = value;
},
end(body) {
this.raw = body;
try {
this.body = JSON.parse(body);
} catch {
this.body = null;
}
},
set statusCode(value) {
this.status = value;
},
};
await handler(
{ url, method: 'GET', headers, socket: { remoteAddress: '127.0.0.1' } },
res,
);
return res;
};
}
test('availability is decided by the operator environment and names no secret', () => {
const none = libraryAvailability(entries, {});
assert.deepEqual(none.get('proxied-points'), {
available: true,
reason: null,
});
assert.equal(none.get('keyed-points').reason, 'key-missing');
assert.equal(none.get('opt-in-points').reason, 'operator-opt-in');
assert.equal(none.get('fixture-pack').reason, 'operator-opt-in');
const configured = libraryAvailability(entries, {
FIRMS_MAP_KEY: 'secret',
GEV_LIBRARY_ENABLE: 'opt-in-points',
GEV_LIBRARY_DISABLE: 'proxied-points',
CCTV_FIXTURE_ENABLED: '1',
});
assert.equal(configured.get('keyed-points').available, true);
assert.equal(configured.get('opt-in-points').available, true);
assert.equal(configured.get('proxied-points').reason, 'operator-disabled');
assert.equal(configured.get('fixture-pack').available, true);
assert.doesNotMatch(JSON.stringify([...configured]), /secret/);
});
test('the catalog route lists every entry with its availability', async () => {
const request = install();
const res = await request('/catalog');
assert.equal(res.status, 200);
assert.equal(res.body.entries.length, entries.length);
assert.deepEqual(
res.body.entries.find(({ id }) => id === 'opt-in-points'),
{ id: 'opt-in-points', available: false, reason: 'operator-opt-in' },
);
});
test('a proxied dataset reaches only its manifest URL, refuses redirects and is normalized', async () => {
const calls = [];
const request = install({
now: () => 42,
fetchImpl: async (url, options) => {
calls.push({ url, options });
return new Response(collection([point('Storm'), { geometry: null }]));
},
});
const res = await request('/data/proxied-points?url=https://evil.example');
assert.equal(res.status, 200);
assert.equal(calls.length, 1);
assert.equal(calls[0].url, 'https://data.example.org/points.json');
assert.equal(calls[0].options.redirect, 'error');
assert.ok(calls[0].options.signal instanceof AbortSignal);
assert.equal(res.body.features.length, 1);
assert.equal(res.body.features[0].properties.name, 'Storm');
assert.equal(res.body.dropped, 1);
assert.equal(res.body.fetchedAt, 42);
// A fresh cache answers without another upstream request.
await request('/data/proxied-points');
assert.equal(calls.length, 1);
});
test('browser-direct, unknown, opt-in and wrong-kind entries are not proxied', async () => {
let calls = 0;
const request = install({
fetchImpl: async () => {
calls++;
return new Response(collection([]));
},
});
assert.equal((await request('/data/direct-points')).status, 404);
assert.equal((await request('/data/nope-nope')).status, 404);
assert.equal((await request('/data/proxied-tiles')).status, 404);
assert.equal((await request('/tile/proxied-points/2/0/0')).status, 404);
const optIn = await request('/data/opt-in-points');
assert.equal(optIn.status, 403);
assert.equal(optIn.body.error, 'operator-opt-in');
assert.equal(calls, 0);
});
test('a keyed entry gets the operator key server-side and refuses forwarded requests', async () => {
const calls = [];
const request = install({
env: { FIRMS_MAP_KEY: 'k3y' },
fetchImpl: async (url) => {
calls.push(new URL(url));
return new Response(collection([point('Fire')]));
},
});
const forwarded = await request('/data/keyed-points', {
'x-forwarded-for': '203.0.113.9',
});
assert.equal(forwarded.status, 403);
assert.equal(calls.length, 0);
const res = await request('/data/keyed-points');
assert.equal(res.status, 200);
assert.equal(calls[0].searchParams.get('MAP_KEY'), 'k3y');
assert.doesNotMatch(res.raw, /k3y/);
});
test('an oversized or failing upstream answers 502 without its text, then serves stale', async () => {
let clock = 0;
let mode = 'ok';
const request = install({
now: () => clock,
fetchImpl: async () => {
if (mode === 'big') return new Response('x'.repeat(8192));
if (mode === 'fail')
return new Response('secret upstream text', { status: 500 });
return new Response(collection([point('A')]));
},
});
mode = 'big';
const big = await request('/data/proxied-points');
assert.equal(big.status, 502);
assert.doesNotMatch(big.raw, /x{20}/);
mode = 'ok';
assert.equal((await request('/data/proxied-points')).status, 200);
clock = 10 * 3600 * 1000;
mode = 'fail';
const stale = await request('/data/proxied-points');
assert.equal(stale.status, 200);
assert.equal(stale.body.stale, true);
assert.equal(stale.headers['X-Data-Stale'], 'true');
assert.doesNotMatch(stale.raw, /secret upstream text/);
});
test('cross-site browser requests are refused before any upstream request', async () => {
let calls = 0;
const request = install({
fetchImpl: async () => {
calls++;
return new Response(collection([]));
},
});
const res = await request('/data/proxied-points', {
host: 'localhost:5173',
origin: 'https://evil.example',
'sec-fetch-site': 'cross-site',
});
assert.equal(res.status, 403);
assert.equal(calls, 0);
});
test('tiles: bounded z/x/y, image types only, cached', async () => {
const calls = [];
const request = install({
fetchImpl: async (url) => {
calls.push(url);
if (url.includes('/3/'))
return new Response('<html>', {
headers: { 'content-type': 'text/html' },
});
return new Response(new Uint8Array([137, 80, 78, 71]), {
headers: { 'content-type': 'image/png' },
});
},
});
assert.equal((await request('/tile/proxied-tiles/1/0/0')).status, 400);
assert.equal((await request('/tile/proxied-tiles/2/4/0')).status, 400);
assert.equal((await request('/tile/proxied-tiles/2/a/0')).status, 400);
const ok = await request('/tile/proxied-tiles/2/1/3');
assert.equal(ok.status, 200);
assert.equal(ok.headers['Content-Type'], 'image/png');
assert.equal(calls[0], 'https://tiles.example.org/2/1/3.png');
await request('/tile/proxied-tiles/2/1/3');
assert.equal(calls.length, 1);
assert.equal((await request('/tile/proxied-tiles/3/1/1')).status, 502);
});
test('tile paths are parsed against the zoom range', () => {
const source = { minZoom: 0, maxZoom: 3 };
assert.deepEqual(parseLibraryTilePath(['3', '7', '7'], source), {
z: 3,
x: 7,
y: 7,
});
assert.equal(parseLibraryTilePath(['3', '8', '0'], source), null);
assert.equal(parseLibraryTilePath(['4', '0', '0'], source), null);
assert.equal(parseLibraryTilePath(['1', '0'], source), null);
});
test('only GET is served', async () => {
const plugin = libraryProxy({ entries, env: {} });
const res = {
writeHead(status, headers) {
this.status = status;
this.headers = headers;
},
end() {},
};
await plugin.handler({ url: '/catalog', method: 'POST', headers: {} }, res);
assert.equal(res.status, 405);
});
test('a partition index is cached only when complete; a complete one outlives failed rebuilds', async () => {
const epc = libraryEntry('epc-england-wales');
let clock = 0;
let footers = 'ok';
let listings = 0;
const listing = `<ListBucketResult><Contents><Key>a.parquet</Key><Size>100</Size></Contents></ListBucketResult>`;
const fetchImpl = async (url) => {
if (url.includes('list-type')) {
listings += 1;
return new Response(listing);
}
return footers === 'ok'
? new Response(new Uint8Array(100))
: new Response('', { status: 503 });
};
const request = install({ catalog: [epc], now: () => clock, fetchImpl });
const first = await request('/index/epc-england-wales');
assert.equal(first.status, 200);
assert.equal(first.body.complete, true);
const builtAt = first.body.builtAt;
// Past cacheSeconds the rebuild fails: the complete index is still served,
// and no rebuild runs again until the partial pause has passed.
clock += (epc.source.cacheSeconds + 1) * 1000;
footers = 'fail';
const kept = await request('/index/epc-england-wales');
assert.equal(kept.body.complete, true);
assert.equal(kept.body.builtAt, builtAt);
const listed = listings;
await request('/index/epc-england-wales');
assert.equal(listings, listed);
// A server that never had a complete index serves the partial one as such,
// and rebuilds after the pause.
const fresh = install({ catalog: [epc], now: () => clock, fetchImpl });
const partial = await fresh('/index/epc-england-wales');
assert.equal(partial.status, 200);
assert.equal(partial.body.complete, false);
const before = listings;
await fresh('/index/epc-england-wales');
assert.equal(listings, before);
clock += 6 * 60_000;
footers = 'ok';
const rebuilt = await fresh('/index/epc-england-wales');
assert.equal(rebuilt.body.complete, true);
assert.equal(listings, before + 1);
});
+4
View File
@@ -16,6 +16,7 @@ import { radioBrowserProxy } from './radio.js';
import { gbfsProxy } from './gbfs.js';
import { localReceiversProxy } from './local-receivers.js';
import { transitProxy } from './transit.js';
import { railProxy } from './rail.js';
import { adsbLolProxy } from './aircraft/adsb-lol.js';
import { aisLiveProxy } from './vessels/ais-live.js';
import { trackBackfillProxies } from './aircraft/tracks.js';
@@ -35,6 +36,7 @@ import { inaturalistProxy } from './inaturalist/observations.js';
import { acartiaProxy } from './acartia.js';
import { cycloneProxy } from './cyclones.js';
import { windProxy } from './wind.js';
import { libraryProxy } from './library.js';
/**
* Construct the local provider plugins in their established order.
@@ -71,6 +73,7 @@ function localProviderPlugins({
gbfsProxy(),
localReceiversProxy(),
transitProxy(),
railProxy(),
adsbLolProxy(),
aisLiveProxy(),
trackBackfillProxies(),
@@ -83,6 +86,7 @@ function localProviderPlugins({
mapillaryProxy(),
firePerimetersProxy(),
gnssIntegrityProxy(),
libraryProxy(),
ghostMapsProxy(),
shipwrecksProxy(),
droneAirspaceProxy(),
+2 -1
View File
@@ -38,7 +38,7 @@ function realtimeDirectives(
'READING THE VIEW: with no selected object or overlay entities, answer from get_entity_context\'s basemap context (tiles source, centre target, reverse-geocoded place, altitude, style, enabled layers); say there is nothing only when that is unavailable too. Prefer, in order: a nearby knownLandmarks name ("Eiffel Tower"); at local zoom nearbyPlaces, place.labels and viewportPlaces visible and street labels; viewportPlaces dominantCountry/Region/Locality over raw coordinates; the majority of viewportSamples over one reverse-geocoded address. Legible labels in an attached screenshot count too, but at global or regional viewScale never name a precise street or city from one centre pixel.',
'Do not mention disabled layers or stale selections.',
'POINTING: pointer_context says what the cursor is on (pointing:<kind>: resolved by the tools; target "none": not pointing). For "this/that/here/that one" with no name, pass "pointer": get_entity_context scope, track_entity/fly_to_location query, analyst_query scope.kind ("around here" is pointer, not view), annotate_map target, find_imagery area. "The second/last one" is referent:n from the last numbered list. Without a pointer, ask or follow WHERE.',
'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, resolve_area, find_imagery, a searched fly_to_location, select_nearest_aircraft, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.',
'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, resolve_area, find_imagery, a searched fly_to_location, select_nearest_aircraft, show_street_level, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.',
'When one request contains MULTIPLE changes, call ALL the corresponding tools before the confirmation. Never confirm a partial subset; if one fails, say which.',
'AFTER TOOLS: speak the result\'s say line once, lightly rephrased at most, never repeating your preamble. For a requested analyst list or ranking, also name up to three returned items with the relevant returned values; keep their order. A count alone does not answer that request; if say is null, finish any remaining requested tools before confirming. Never add a count, list or action claim that the executed tool results do not support. Sources, notes and caveats are on the voice card: speak a caveat only when it changes the answer, or when asked why, how sure or which source. Lower bounds ("at least 500"), partial answers, approximate outlines and stale, degraded or unavailable feed states in say are part of the answer — always preserve them. No filler or hedges ("let me check", "keep in mind", "it seems", "based on the data available"), and never add "approximately" to an exact tool number.',
'For "show/open/turn on" layer requests, enable the matching layer. For "hide/close/turn off", disable it.',
@@ -65,6 +65,7 @@ function realtimeDirectives(
'"Track/follow <something specific>" (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.',
'"Show me which planes are overhead"/"frame the ships"/"show me the satellites above" use frame_overhead with the matching target.',
'Confirmations echo the RESULTING state, never the request: "HUD minimal layout", "Density twenty-five percent", "Bing aerial imagery", "Tracking UAL428", "Framed fourteen aircraft". On ok=false, state the failure plainly: "Nothing matched UAL999", "No ships within 120 kilometers". Never claim an action without ok=true in the tool result.',
'STREET LEVEL: "X on Street View/Mapillary/KartaView/Panoramax", "street view of X" → show_street_level once, not fly_to_location; no place = view centre. Street View is never a basemap.',
'For destination requests such as "take me to Italy", "go to NYC", or "show me the Eiffel Tower", call fly_to_location. Prefer known city IDs when available; otherwise pass the plain place query.',
'Navigation-only requests ("take me to X", "go to X", "fly to X") are NOT descriptions: call fly_to_location alone and do NOT also call annotate_map, unless the user explicitly asks to mark the place or you go on to explain specific places there. Never drop a point pin on a region-scale natural feature (a mountain range, desert, sea, or forest) — a single point in the middle of the Rockies is meaningless. If the user explicitly asks to mark such a region, prefer type=area.',
'For country and city destinations, omit rangeM so GEV frames the whole country or city in view. For landmarks and buildings, omit rangeM so GEV chooses a close landmark view.',
@@ -65,6 +65,28 @@ export const ACTION_DESCRIPTIONS = {
},
},
},
show_street_level: {
description:
'Fly to a place and open its nearest street-level photo; Street View switches to Google 3D (billed). No query: view centre.',
$position: 1,
parameters: {
properties: {
query: {
description: 'Place or address; "pointer" = under the cursor.',
$position: 1,
},
provider: {
description:
'google = Street View; others by name; omit: the lit source.',
$position: 1,
},
referent: {
description: 'Numbered item n from the last result list.',
$position: 1,
},
},
},
},
adjust_camera_zoom: {
description:
'Move the current Cesium camera closer to or farther from what it is presently looking at. Use for relative zoom requests without changing location.',
+45 -16
View File
@@ -72,6 +72,14 @@ class UpstreamRateLimitError extends Error {
}
}
/** Thrown when the routing service failed or sent no JSON; answered as 502. */
class UpstreamUnavailableError extends Error {
constructor() {
super('routing service unavailable');
this.name = 'UpstreamUnavailableError';
}
}
/**
* Run one outbound request no sooner than the shared gate allows.
* @template T
@@ -167,18 +175,26 @@ async function fetchRoute({
`${base.replace(/\/$/, '')}/route/v1/${osrmProfile}/${coords}` +
`?overview=full&geometries=geojson&alternatives=false&steps=${withSteps ? 'true' : 'false'}`;
const controller = new AbortController();
// The 12 s budget includes queue wait, so Directions (15 s) hears back.
const timer = setTimeout(() => controller.abort(), 12000);
let osrm;
try {
const upstreamRes = await throughUpstreamGate(() =>
fetchImpl(upstream, {
signal: controller.signal,
// The endpoint is configured above; a redirect is the one way out of
// it, so it is refused rather than followed.
redirect: 'error',
headers: { 'User-Agent': 'gods-eye-view/dev (local)' },
}),
);
const upstreamRes = await throughUpstreamGate(async () => {
// Expiry before dispatch reflects our backlog, not an upstream outage.
if (controller.signal.aborted)
throw new RouteBusyError(_upstreamIntervalMs);
try {
return await fetchImpl(upstream, {
signal: controller.signal,
// The endpoint is configured above; a redirect is the one way out of
// it, so it is refused rather than followed.
redirect: 'error',
headers: { 'User-Agent': 'gods-eye-view/dev (local)' },
});
} catch {
throw new UpstreamUnavailableError();
}
});
if (upstreamRes.status === 429) {
await cancelBody(upstreamRes);
const retryAfter = Number(upstreamRes.headers.get('retry-after'));
@@ -186,20 +202,28 @@ async function fetchRoute({
Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter : 5,
);
}
if (!upstreamRes.ok) {
// OSRM answers 400 for a route it cannot find; other failures are outages.
if (upstreamRes.status === 400) {
await cancelBody(upstreamRes);
return { payload: null, error: 'no route found' };
}
if (!upstreamRes.ok) {
await cancelBody(upstreamRes);
throw new UpstreamUnavailableError();
}
const ctype = upstreamRes.headers.get('content-type') || '';
if (!ctype.includes('json')) {
await cancelBody(upstreamRes);
return { payload: null, error: 'no route found' };
throw new UpstreamUnavailableError();
}
// Read and parse failures are outages; later ones stay "route proxy error".
try {
osrm = JSON.parse(
await readResponseTextCapped(upstreamRes, ROUTE_MAX_RESPONSE_BYTES),
);
} catch {
throw new UpstreamUnavailableError();
}
const text = await readResponseTextCapped(
upstreamRes,
ROUTE_MAX_RESPONSE_BYTES,
);
osrm = JSON.parse(text);
} finally {
clearTimeout(timer);
}
@@ -355,6 +379,11 @@ export function installRouteMiddleware(
upstreamError.retryAfterSec,
);
}
if (upstreamError instanceof UpstreamUnavailableError) {
res.writeHead(502, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ ok: false, error: upstreamError.message }));
return;
}
throw upstreamError;
}
if (error || !payload) return fail(error || 'no route found');
+285
View File
@@ -0,0 +1,285 @@
import { open, stat } from 'node:fs/promises';
import path from 'node:path';
import { Readable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import { fileURLToPath } from 'node:url';
import { createRailService } from '../../src/sources/railService.js';
import { RAIL_FEED_ID_PATTERN } from '../../src/data/railFeeds.js';
/**
* Where development and preview look for path and schedule packs when
* RAIL_PATHS_DIR is unset: the default output of scripts/build-rail-paths.mjs.
*/
export const DEFAULT_RAIL_PATHS_DIR = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
'../../.cache/rail-paths',
);
/**
* Pack bytes the development server keeps in memory, across path and
* schedule packs: a few of the largest national packs.
*/
export const RAIL_PACK_CACHE_MAX_BYTES = 64 * 1024 * 1024;
/**
* A bounded cache of pack files' bytes, by path, least recently read first
* out. An entry is used only while the file's modification time and size
* are the ones it was read with, so a rebuilt pack is read again; a file
* larger than the whole byte budget is never kept. Reads of the same file at
* the same time share one read.
* @param {number} [maxBytes]
* @returns {{load: (file: string, info: {mtimeMs: number, size: number},
* read: () => Promise<Buffer>) => Promise<Buffer>, bytes: () => number}}
*/
export function createRailPackCache(maxBytes = RAIL_PACK_CACHE_MAX_BYTES) {
/** @type {Map<string, {mtimeMs: number, size: number, bytes: Buffer}>} */
const entries = new Map();
/** @type {Map<string, Promise<Buffer>>} */
const pending = new Map();
let total = 0;
const drop = (file) => {
const entry = entries.get(file);
if (!entry) return;
entries.delete(file);
total -= entry.bytes.byteLength;
};
return {
async load(file, info, read) {
const entry = entries.get(file);
if (entry && entry.mtimeMs === info.mtimeMs && entry.size === info.size) {
// Most recently read last: the first entry is the next one out.
entries.delete(file);
entries.set(file, entry);
return entry.bytes;
}
const version = `${file}|${info.mtimeMs}|${info.size}`;
const reading = pending.get(version);
if (reading) return reading;
const promise = (async () => {
const bytes = await read();
drop(file);
if (bytes.byteLength <= maxBytes) {
entries.set(file, { mtimeMs: info.mtimeMs, size: info.size, bytes });
total += bytes.byteLength;
for (const [oldest] of entries) {
if (total <= maxBytes) break;
drop(oldest);
}
}
return bytes;
})().finally(() => pending.delete(version));
pending.set(version, promise);
return promise;
},
bytes: () => total,
};
}
/**
* A pack reader for the rail service that reads `<dir>/<feedId><suffix>`.
* Ids are re-checked against the registry's id pattern, so a name can never
* leave the directory; a missing file is "no pack", and a file over the cap is
* refused without being read (an error with `code: 'PACK_TOO_LARGE'`). The
* pack is returned as the file's bytes, never decoded to text; with a
* `cache` (createRailPackCache), an unchanged file is read once.
* @param {string} dir
* @param {string} suffix `.json` (path packs) or `.schedule.json`.
* @param {ReturnType<typeof createRailPackCache>|null} [cache]
* @returns {(feedId: string, options?: {maxBytes?: number}) =>
* Promise<Buffer|null>}
*/
function createFsPackReader(dir, suffix, cache = null) {
const root = path.resolve(dir);
return async (feedId, { maxBytes = Infinity } = {}) => {
if (typeof feedId !== 'string' || !RAIL_FEED_ID_PATTERN.test(feedId))
return null;
const file = path.join(root, `${feedId}${suffix}`);
let handle;
try {
handle = await open(file, 'r');
} catch (error) {
if (error?.code === 'ENOENT') return null;
throw error;
}
try {
const info = await handle.stat();
if (!info.isFile()) return null;
if (info.size > maxBytes) {
const error = new Error(`pack is ${info.size} bytes`);
error.code = 'PACK_TOO_LARGE';
throw error;
}
return cache
? await cache.load(file, info, () => handle.readFile())
: await handle.readFile();
} finally {
await handle.close();
}
};
}
/**
* A `readPathPack` that reads `<dir>/<feedId>.json`.
* @param {string} dir
* @param {{cache?: ReturnType<typeof createRailPackCache>|null}} [options]
*/
export function createFsPathPackReader(dir, { cache = null } = {}) {
return createFsPackReader(dir, '.json', cache);
}
/**
* A `readSchedulePack` that reads `<dir>/<feedId>.schedule.json`.
* @param {string} dir
* @param {{cache?: ReturnType<typeof createRailPackCache>|null}} [options]
*/
export function createFsSchedulePackReader(dir, { cache = null } = {}) {
return createFsPackReader(dir, '.schedule.json', cache);
}
/**
* A `hasSchedulePack` for the rail service: whether
* `<dir>/<feedId>.schedule.json` is a file. Only the directory entry is
* read, never the pack; a missing file is false, any other error is thrown
* (the service then lists the feed, so the read shows the failure).
* @param {string} dir
* @returns {(feedId: string) => Promise<boolean>}
*/
export function createFsSchedulePackProbe(dir) {
const root = path.resolve(dir);
return async (feedId) => {
if (typeof feedId !== 'string' || !RAIL_FEED_ID_PATTERN.test(feedId))
return false;
try {
return (await stat(path.join(root, `${feedId}.schedule.json`))).isFile();
} catch (error) {
if (error?.code === 'ENOENT') return false;
throw error;
}
};
}
/**
* A pack version for the rail service: the modification time and size of
* `<dir>/<feedId><suffix>`, or null when there is no such file. Only the
* directory entry is read; a rebuilt pack has a new version.
* @param {string} dir
* @param {string} suffix
* @returns {(feedId: string) => Promise<string|null>}
*/
function createFsPackVersion(dir, suffix) {
const root = path.resolve(dir);
return async (feedId) => {
if (typeof feedId !== 'string' || !RAIL_FEED_ID_PATTERN.test(feedId))
return null;
try {
const info = await stat(path.join(root, `${feedId}${suffix}`));
return info.isFile() ? `${info.mtimeMs}:${info.size}` : null;
} catch (error) {
if (error?.code === 'ENOENT') return null;
throw error;
}
};
}
/**
* A `schedulePackVersion` for the rail service: the modification time and
* size of `<dir>/<feedId>.schedule.json`, or null when there is no such
* file.
* @param {string} dir
* @returns {(feedId: string) => Promise<string|null>}
*/
export function createFsSchedulePackVersion(dir) {
return createFsPackVersion(dir, '.schedule.json');
}
/**
* A `pathPackVersion` for the rail service: the same for
* `<dir>/<feedId>.json`.
* @param {string} dir
* @returns {(feedId: string) => Promise<string|null>}
*/
export function createFsPathPackVersion(dir) {
return createFsPackVersion(dir, '.json');
}
/**
* Write a service Response to a Node response, streaming its body: a pack's
* bytes go to the socket as they are, with no text decode and no copy.
* @param {Response} response
* @param {import('node:http').ServerResponse} res
* @returns {Promise<void>}
*/
export async function sendRailResponse(response, res) {
res.writeHead(response.status, Object.fromEntries(response.headers));
if (!response.body) {
res.end();
return;
}
await pipeline(Readable.fromWeb(response.body), res);
}
/** Connect the reusable rail request service to development and preview. */
export function railProxy(options = {}) {
const pathsDir =
options.pathsDir || process.env.RAIL_PATHS_DIR || DEFAULT_RAIL_PATHS_DIR;
// One byte budget for both kinds of pack.
const cache = createRailPackCache();
const service = createRailService({
...options,
readPathPack:
options.readPathPack || createFsPathPackReader(pathsDir, { cache }),
readSchedulePack:
options.readSchedulePack ||
createFsSchedulePackReader(pathsDir, { cache }),
// Probe the same directory only when its fs reader is the one in use.
hasSchedulePack:
options.hasSchedulePack ||
(options.readSchedulePack ? null : createFsSchedulePackProbe(pathsDir)),
schedulePackVersion:
options.schedulePackVersion ||
(options.readSchedulePack ? null : createFsSchedulePackVersion(pathsDir)),
pathPackVersion:
options.pathPackVersion ||
(options.readPathPack ? null : createFsPathPackVersion(pathsDir)),
});
// A throw here would escape into the HTTP server's event loop and end the
// whole dev server (Node exits on an unhandled rejection or a throwing
// 'close' listener), so failures stay inside the rail routes.
function closeService() {
try {
service.close();
} catch (error) {
console.warn('[rail] provider close failed', error);
}
}
function install(server) {
server.middlewares.use('/api/rail', async (req, res) => {
try {
const response = await service.handle({
url: `http://localhost/api/rail${req.url || '/'}`,
method: req.method,
// Read for If-None-Match on a pack.
headers: req.headers,
});
await sendRailResponse(response, res);
} catch (error) {
// An answer already under way (a viewer that left mid-pack) has
// nothing to add: the stream is closed and that is all.
if (res.headersSent) {
res.destroy();
return;
}
console.warn('[rail] request failed', error);
res.writeHead(500, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'Rail provider error' }));
}
});
server.httpServer?.once('close', closeService);
}
return {
name: 'rail-proxy',
closeBundle: closeService,
configureServer: install,
configurePreviewServer: install,
};
}
+13 -116
View File
@@ -100,20 +100,6 @@ export function createScreenAnnotationRenderer(
const heightCache = new Map();
const HEIGHT_CACHE_SOFT = 600; // target size; over this we evict keys not used this frame
const HEIGHT_CACHE_HARD = 8000; // absolute ceiling (one big ring + others) — never exceeded
// Accepted surface-height range; samples outside it are treated as misses.
const MIN_SURFACE_HEIGHT = -430;
const MAX_SURFACE_HEIGHT = 9000;
const HEIGHT_SAMPLE_BUDGET = 32; // clampToHeight calls per projection pass
const HEIGHT_RETRY_MS = 1000; // minimum interval between misses per anchor
let cullingVolume = null;
const heightBounds = new Cesium.BoundingSphere(new Cesium.Cartesian3(), 9500);
// Fair sampling: keys refused by an exhausted budget queue in arrival order,
// and each pass reserves budget for the oldest queued keys first, so a fixed
// iteration order cannot starve anchors beyond the first budget's worth.
let heightQueue = new Set(); // keys refused in earlier passes, oldest first
let heightReserved = new Set(); // queued keys this pass serves first
let heightRefused = new Set(); // keys refused during this pass
let heightBudget = HEIGHT_SAMPLE_BUDGET; // unreserved slots left this pass
let projGen = 0; // bumped each projection frame; entries used this frame are "hot"
function trimHeightCache() {
if (heightCache.size <= HEIGHT_CACHE_SOFT) return;
@@ -141,20 +127,11 @@ export function createScreenAnnotationRenderer(
return PALETTE[anno.color] || PALETTE.primary;
}
function clearHeights() {
heightCache.clear();
heightQueue.clear();
}
function heightKey(lon, lat) {
return `${lon.toFixed(5)},${lat.toFixed(5)}`;
}
// Best-effort surface height under a coordinate (clamps onto the 3D tiles).
// Returns 0 until the tile loads, then caches the validated height.
// canSample=false returns the cached height (or 0) without surface sampling.
// `key` and `cached` are the caller's cache lookup for this coordinate.
function groundHeight(lon, lat, key, cached, canSample = true) {
function groundHeight(lon, lat) {
const key = `${lon.toFixed(5)},${lat.toFixed(5)}`;
const cached = heightCache.get(key);
if (cached && cached.settled) {
// Mark hot for this frame + bump to most-recently-used so heights for live
// marks survive eviction while places we've navigated away from age out.
@@ -163,20 +140,6 @@ export function createScreenAnnotationRenderer(
heightCache.set(key, cached);
return cached.h;
}
const now = Date.now();
if (cached && (!canSample || now < cached.nextAttempt)) {
cached.gen = projGen;
return cached.h;
}
if (!canSample) return 0;
if (heightReserved.has(key)) {
heightReserved.delete(key);
} else if (heightBudget > 0) {
heightBudget--;
} else {
heightRefused.add(key);
return cached?.h || 0;
}
try {
if (
scene.clampToHeightSupported &&
@@ -187,11 +150,7 @@ export function createScreenAnnotationRenderer(
);
if (c) {
const h = Cesium.Cartographic.fromCartesian(c).height;
if (
Number.isFinite(h) &&
h > MIN_SURFACE_HEIGHT &&
h < MAX_SURFACE_HEIGHT
) {
if (Number.isFinite(h) && h > -430 && h < 9000) {
heightCache.set(key, { h, settled: true, gen: projGen });
trimHeightCache();
return h;
@@ -202,12 +161,7 @@ export function createScreenAnnotationRenderer(
/* tiles not ready */
}
const fallback = cached ? cached.h : 0;
heightCache.set(key, {
h: fallback,
settled: false,
gen: projGen,
nextAttempt: now + HEIGHT_RETRY_MS,
});
heightCache.set(key, { h: fallback, settled: false, gen: projGen });
trimHeightCache();
return fallback;
}
@@ -353,7 +307,7 @@ export function createScreenAnnotationRenderer(
} catch {
/* the board is unreachable */
}
if (records.size === 0) clearHeights();
if (records.size === 0) heightCache.clear();
throw error;
}
}
@@ -409,32 +363,11 @@ export function createScreenAnnotationRenderer(
// Anchor at the real surface height (cached + validated), so marks sit on
// the ground/building instead of at sea level. The settle-once cache keeps
// it stable (no per-frame jitter that earlier broke nearby projections).
// A settled height is tested as the actual point below. An unknown height
// is rejected before GPU-backed surface sampling only when the whole
// accepted height range is proven hidden by one constraint.
const key = heightKey(lon, lat);
const cached = heightCache.get(key);
let height;
if (cached?.settled) {
height = groundHeight(lon, lat, key, cached);
} else {
if (heightRangeHidden(lon, lat)) return null;
// The frustum restricts sampling only; an endpoint outside it still
// projects (at its cached or zero height) so a partly visible arrow,
// route or ring keeps its on-screen connector and label.
Cesium.Cartesian3.fromDegrees(
lon,
lat,
0,
undefined,
heightBounds.center,
);
const inFrustum =
cullingVolume?.computeVisibility(heightBounds) !==
Cesium.Intersect.OUTSIDE;
height = groundHeight(lon, lat, key, cached, inFrustum);
}
const world = Cesium.Cartesian3.fromDegrees(lon, lat, height);
const world = Cesium.Cartesian3.fromDegrees(
lon,
lat,
groundHeight(lon, lat),
);
// Reject points behind the camera (they produce wrapped/extreme coords).
Cesium.Cartesian3.subtract(world, scene.camera.positionWC, scratchDir);
if (Cesium.Cartesian3.dot(scratchDir, scene.camera.directionWC) <= 0)
@@ -458,21 +391,6 @@ export function createScreenAnnotationRenderer(
return { x: win.x, y: win.y };
}
// Points along one surface normal are linear in height, and each rejection
// region (behind the camera plane; the occluder's cone beyond the horizon
// plane) is convex. So when both height-range endpoints fail the SAME test,
// every intermediate height fails it too. Mixed failures prove nothing.
function heightRangeHidden(lon, lat) {
const low = Cesium.Cartesian3.fromDegrees(lon, lat, MIN_SURFACE_HEIGHT);
const high = Cesium.Cartesian3.fromDegrees(lon, lat, MAX_SURFACE_HEIGHT);
const behind = (point) => {
Cesium.Cartesian3.subtract(point, scene.camera.positionWC, scratchDir);
return Cesium.Cartesian3.dot(scratchDir, scene.camera.directionWC) <= 0;
};
if (behind(low) && behind(high)) return true;
return !occluder.isPointVisible(low) && !occluder.isPointVisible(high);
}
// Tracked-entity z-order: the tracked aircraft is a Cesium billboard in the
// CANVAS, which sits BELOW this SVG overlay (z-90, HTML over canvas). So a screen mark
// could hide it. After layout, any mark whose ACTUAL projected bounding box (rings,
@@ -557,20 +475,6 @@ export function createScreenAnnotationRenderer(
publish();
return;
}
// Serve the oldest refused keys first; the rest of the budget is open.
heightReserved = new Set();
for (const key of heightQueue) {
if (heightReserved.size >= HEIGHT_SAMPLE_BUDGET) break;
heightReserved.add(key);
}
heightBudget = HEIGHT_SAMPLE_BUDGET - heightReserved.size;
heightRefused = new Set();
cullingVolume =
scene.camera.frustum?.computeCullingVolume?.(
scene.camera.positionWC,
scene.camera.directionWC,
scene.camera.upWC,
) || null;
projGen += 1; // new frame: entries touched below are "hot" and survive trimming
const trackedRect = trackedEntityRect();
occluder.cameraPosition = scene.camera.positionWC;
@@ -728,13 +632,6 @@ export function createScreenAnnotationRenderer(
}
}
}
// Still-refused queued keys keep their place; new refusals join the back.
// Keys no longer requesting a sample (settled, culled, removed) drop out.
const nextQueue = new Set();
for (const key of heightQueue)
if (heightRefused.has(key)) nextQueue.add(key);
for (const key of heightRefused) nextQueue.add(key);
heightQueue = nextQueue;
// Keep clustered callouts readable.
const placed = [];
@@ -914,7 +811,7 @@ export function createScreenAnnotationRenderer(
records.delete(anno.id);
publish();
// Board emptied → drop accumulated height samples (natural reset point).
if (records.size === 0) clearHeights();
if (records.size === 0) heightCache.clear();
}
function sync() {
@@ -937,7 +834,7 @@ export function createScreenAnnotationRenderer(
records.clear();
leaving.clear();
byKey.clear();
clearHeights();
heightCache.clear();
}
// Claim the board; from here on only this renderer's own board changes.
@@ -166,17 +166,10 @@ test('an arrow defines its arrowhead and drops the dash once drawn on', async (t
test('outline upgrade preserves the existing SVG group identity', async (t) => {
const { store, svg, groups } = board(t);
const camera = {
positionWC: Cesium.Cartesian3.fromDegrees(-97.5, 31.2, 1000000),
directionWC: Cesium.Cartesian3.normalize(
Cesium.Cartesian3.negate(
Cesium.Cartesian3.fromDegrees(-97.5, 31.2),
new Cesium.Cartesian3(),
),
new Cesium.Cartesian3(),
),
positionWC: Cesium.Cartesian3.ZERO,
directionWC: Cesium.Cartesian3.ZERO,
positionCartographic: { height: 1000 },
};
t.mock.method(Cesium.SceneTransforms, 'worldToWindowCoordinates', () => ({ x: 100, y: 100 }));
const sampledAnchors = [];
const scene = sceneFor(camera, {
canvas: { clientWidth: 1280, clientHeight: 720, width: 1280, height: 720 },
@@ -582,280 +575,3 @@ test('a mark hidden from Draw is not drawn or placed, and comes back when shown'
await settle();
assert.equal(drawn(), true, 'shown again, it is drawn');
});
test('hidden and back-facing annotations do not sample; visible unresolved heights have bounded retries', (t) => {
const store = samplingBoard(t);
t.mock.timers.enable({ apis: ['setTimeout', 'Date'], now: 1000 });
t.mock.method(Cesium.SceneTransforms, 'worldToWindowCoordinates', () => ({
x: 100,
y: 100,
}));
const camera = {
positionWC: Cesium.Cartesian3.fromDegrees(0, 0, 100000),
directionWC: new Cesium.Cartesian3(-1, 0, 0),
positionCartographic: { height: 100000 },
moveEnd: new Cesium.Event(),
};
let samples = 0;
const postRender = new Cesium.Event();
const scene = {
camera,
canvas: { width: 800, height: 600 },
postRender,
clampToHeightSupported: true,
clampToHeight() {
samples++;
},
requestRender() {},
};
const renderer = samplingRenderer(store, scene);
const mark = (id, lon) => ({
id,
type: 'label',
anchor: { lon, lat: 0 },
label: id,
});
renderer.add(mark('far-side', 180));
for (let i = 0; i < 100; i++) postRender.raiseEvent();
assert.equal(samples, 0);
renderer.add(mark('near-side', 0));
for (let i = 0; i < 100; i++) postRender.raiseEvent();
assert.equal(samples, 1);
for (let i = 0; i < 10; i++) {
t.mock.timers.tick(1000);
postRender.raiseEvent();
}
assert.equal(samples, 11);
camera.moveEnd.raiseEvent();
postRender.raiseEvent();
assert.equal(samples, 11, 'camera events cannot bypass the retry interval');
renderer.update(mark('near-side', 0), { empty: true });
t.mock.timers.tick(1000);
postRender.raiseEvent();
assert.equal(samples, 11, 'hidden records do not sample');
camera.directionWC = new Cesium.Cartesian3(1, 0, 0);
renderer.update(mark('near-side', 0));
postRender.raiseEvent();
assert.equal(samples, 11, 'behind-camera anchors do not sample');
camera.directionWC = new Cesium.Cartesian3(-1, 0, 0);
camera.frustum = {
computeCullingVolume: () => ({
computeVisibility: () => Cesium.Intersect.OUTSIDE,
}),
};
postRender.raiseEvent();
assert.equal(samples, 11, 'outside-frustum anchors do not sample');
camera.frustum = null;
const before = samples;
renderer.add({
id: 'large-area',
type: 'area',
label: 'Area',
ring: Array.from({ length: 100 }, (_, i) => [0.001 + i / 10000, 0]),
anchor: { lon: 0, lat: 0 },
});
assert.equal(
samples - before,
32,
'a projection pass caps synchronous surface work',
);
renderer.destroy();
assert.equal(camera.moveEnd.numberOfListeners, 0);
assert.equal(postRender.numberOfListeners, 0);
});
function sampledLon(world) {
const carto = Cesium.Cartographic.fromCartesian(world);
return Number(Cesium.Math.toDegrees(carto.longitude).toFixed(5));
}
/**
* A board with no view mounted, for tests that count surface samples: the
* renderer projects on `add` and on each post-render event, and the store
* holds what it would draw. Animation frames run at once.
*/
function samplingBoard(t) {
const previousRaf = globalThis.requestAnimationFrame;
globalThis.requestAnimationFrame = (fn) => {
fn();
return 1;
};
t.after(() => {
globalThis.requestAnimationFrame = previousRaf;
});
return useScreenAnnotationsStore(createPinia());
}
/** The renderer under test, on the given board and scene. */
function samplingRenderer(store, scene) {
return createScreenAnnotationRenderer(
{ scene, camera: scene.camera, trackedEntity: null },
{ store, activeTrackedReadoutId: () => null },
);
}
function squareFrustum() {
const frustum = new Cesium.PerspectiveFrustum();
frustum.fov = Cesium.Math.toRadians(60);
frustum.aspectRatio = 1;
frustum.near = 1;
frustum.far = 1e8;
return frustum;
}
test('an arrow with one endpoint outside a real frustum keeps its visible connector', (t) => {
const store = samplingBoard(t);
t.mock.method(Cesium.SceneTransforms, 'worldToWindowCoordinates', (_scene, world) =>
sampledLon(world) > 0.1 ? { x: 1100, y: 400 } : { x: 400, y: 400 },
);
// 5 km straight down over (0, 0): east is +Y, north is +Z in ECEF here.
const camera = {
positionWC: Cesium.Cartesian3.fromDegrees(0, 0, 5000),
directionWC: new Cesium.Cartesian3(-1, 0, 0),
upWC: new Cesium.Cartesian3(0, 0, 1),
positionCartographic: { height: 5000 },
frustum: squareFrustum(),
};
const sampled = [];
const postRender = new Cesium.Event();
const scene = {
camera,
canvas: { width: 800, height: 800 },
postRender,
clampToHeightSupported: true,
clampToHeight(world) {
sampled.push(sampledLon(world));
return world;
},
};
// The 0.2-degree endpoint (~22 km east) is well beyond the side plane plus
// the 9.5 km height-bound radius, yet in front of the camera and the horizon.
const far = Cesium.Cartesian3.fromDegrees(0.2, 0, 0);
const volume = camera.frustum.computeCullingVolume(
camera.positionWC,
camera.directionWC,
camera.upWC,
);
assert.equal(
volume.computeVisibility(new Cesium.BoundingSphere(far, 9500)),
Cesium.Intersect.OUTSIDE,
);
const renderer = samplingRenderer(store, scene);
renderer.add({
id: 'partial-arrow',
type: 'arrow',
label: 'Ridge',
anchor: { lon: 0, lat: 0 },
to: { lon: 0.2, lat: 0 },
});
postRender.raiseEvent();
const [mark] = store.board.marks;
assert.equal(mark.display, '', 'the arrow stays drawn');
const path = mark.children.find((child) => child.attrs.class?.includes('gev-anno-arrow'));
assert.match(path.attrs.d, /^M 400\.0 400\.0 Q .* 1100\.0 400\.0$/);
assert.deepEqual([...new Set(sampled)], [0], 'only the in-frustum endpoint samples');
renderer.destroy();
});
test('a low-angle cached roof stays visible when range endpoints fail different tests', (t) => {
const store = samplingBoard(t);
t.mock.method(Cesium.SceneTransforms, 'worldToWindowCoordinates', () => ({ x: 400, y: 380 }));
// Camera 1 m above the ellipsoid, heading east, pitched 25 degrees down.
const pitch = Cesium.Math.toRadians(25);
const camera = {
positionWC: Cesium.Cartesian3.fromDegrees(0, 0, 1),
directionWC: new Cesium.Cartesian3(-Math.sin(pitch), Math.cos(pitch), 0),
upWC: new Cesium.Cartesian3(Math.cos(pitch), Math.sin(pitch), 0),
positionCartographic: { height: 1 },
frustum: squareFrustum(),
};
const roofLon = 4000 / 111319.49;
let samples = 0;
const postRender = new Cesium.Event();
const scene = {
camera,
canvas: { width: 800, height: 800 },
postRender,
clampToHeightSupported: true,
clampToHeight() {
samples++;
return Cesium.Cartesian3.fromDegrees(roofLon, 0, 20);
},
};
// Sanity: height 0 is beyond the horizon and 9000 m is behind the camera,
// while the 20 m roof itself is in front and above the horizon.
const occluder = new Cesium.EllipsoidalOccluder(Cesium.Ellipsoid.WGS84, camera.positionWC);
const ahead = (h) => {
const p = Cesium.Cartesian3.fromDegrees(roofLon, 0, h);
const v = Cesium.Cartesian3.subtract(p, camera.positionWC, new Cesium.Cartesian3());
return Cesium.Cartesian3.dot(v, camera.directionWC) > 0;
};
assert.equal(occluder.isPointVisible(Cesium.Cartesian3.fromDegrees(roofLon, 0, 0)), false);
assert.equal(ahead(9000), false);
assert.equal(ahead(20), true);
assert.equal(occluder.isPointVisible(Cesium.Cartesian3.fromDegrees(roofLon, 0, 20)), true);
const renderer = samplingRenderer(store, scene);
renderer.add({ id: 'roof', type: 'label', label: 'Roof', anchor: { lon: roofLon, lat: 0 } });
assert.equal(samples, 1, 'the unknown height is sampled, not culled');
assert.equal(store.board.marks[0].display, '');
for (let i = 0; i < 5; i++) postRender.raiseEvent();
assert.equal(samples, 1, 'the settled height is reused');
assert.equal(store.board.marks[0].display, '', 'the cached roof point is tested and shown');
renderer.destroy();
});
test('height retries beyond the first budget are served under sparse passes', (t) => {
const store = samplingBoard(t);
t.mock.timers.enable({ apis: ['setTimeout', 'Date'], now: 1000 });
t.mock.method(Cesium.SceneTransforms, 'worldToWindowCoordinates', () => ({ x: 100, y: 100 }));
const camera = {
positionWC: Cesium.Cartesian3.fromDegrees(0, 0, 100000),
directionWC: new Cesium.Cartesian3(-1, 0, 0),
positionCartographic: { height: 100000 },
};
const lons = Array.from({ length: 33 }, (_, i) => Number(((i + 1) / 1000).toFixed(3)));
const last = lons.at(-1);
let tilesReady = false;
let pass = [];
const postRender = new Cesium.Event();
const scene = {
camera,
canvas: { width: 800, height: 600 },
postRender,
clampToHeightSupported: true,
clampToHeight(world) {
const lon = sampledLon(world);
pass.push(lon);
return tilesReady && lon === last ? Cesium.Cartesian3.fromDegrees(lon, 0, 12) : undefined;
},
};
const renderer = samplingRenderer(store, scene);
for (const lon of lons) {
renderer.add({ id: `p${lon}`, type: 'label', label: 'P', anchor: { lon, lat: 0 } });
}
assert.ok(pass.includes(last), 'every point gets its initial miss');
tilesReady = true; // tiles under the final point arrive after its miss
const lastSamples = [];
for (let i = 0; i < 4; i++) {
t.mock.timers.tick(1000);
pass = [];
postRender.raiseEvent();
assert.ok(pass.length <= 32, `pass ${i} keeps the 32-sample budget`);
lastSamples.push(pass.filter((lon) => lon === last).length);
}
assert.ok(lastSamples.slice(0, 2).includes(1), 'the 33rd point is served within two passes');
assert.equal(lastSamples.at(-1), 0, 'and then stays settled');
// Points refused in one pass are served first in the next: every point is
// retried within any two consecutive sparse passes.
pass = [];
t.mock.timers.tick(1000);
postRender.raiseEvent();
const firstPass = pass;
pass = [];
t.mock.timers.tick(1000);
postRender.raiseEvent();
const served = new Set([...firstPass, ...pass]);
for (const lon of lons.slice(0, -1)) assert.ok(served.has(lon), `${lon} retried`);
renderer.destroy();
});
+11 -1
View File
@@ -1,6 +1,7 @@
import { resolveCatalogSources } from './sourceComposition.js';
import { createWeatherClock } from '../layers/weather/clock.js';
import { worldClock } from '../time/worldClock.js';
import { NEWS_LAYER_ID } from '../layers/news/model.js';
import { createWeatherLayer } from '../layers/weather/index.js';
import { createApplicationCyclones } from './layers/cyclones.js';
import { createWindLayer } from '../layers/wind/index.js';
@@ -18,6 +19,7 @@ import { createApplicationDirections } from './layers/directions.js';
import { createApplicationRecentImagery } from './layers/recentImagery.js';
import { createApplicationOsmPlaces } from './layers/osmPlaces.js';
import { createApplicationTransit } from './layers/transit.js';
import { createApplicationRail } from './layers/rail.js';
import { createApplicationInstallations } from './layers/militaryInstallations.js';
import { createApplicationSatellites } from './layers/satellites.js';
import { createApplicationLaunches } from './layers/rocketLaunches.js';
@@ -68,6 +70,7 @@ export function createApplicationCatalog({
signal,
metadata = APPLICATION_LAYER_METADATA,
vesselOptions,
newsOptions,
resolveAsset,
nepalBoundaryResolver,
streetLevelProviders,
@@ -142,7 +145,13 @@ export function createApplicationCatalog({
createApplicationDroneAirspace({
source: sources['drone-airspace'],
}),
createApplicationNews({ source: sources.news }),
// Ages, the window and the counts read the world clock: a host that
// follows it for the layer shows the news at the time on the timeline.
createApplicationNews({
now: worldClock.timeFor(NEWS_LAYER_ID),
...newsOptions,
source: sources.news,
}),
createApplicationWildlifeSightings({
source: sources['wildlife-sightings'],
groundFloor: surface.groundFloor,
@@ -191,6 +200,7 @@ export function createApplicationCatalog({
...createOsmOverlayLayers(),
...createInfrastructureLayers(localGeoJsonServices),
createApplicationCables({ source: sources.cables }),
createApplicationRail({ surface, source: sources.rail }),
createApplicationFirms({
surface,
id: 'local-firms',
+2 -2
View File
@@ -40,7 +40,7 @@ test('catalogs construct distinct layers and classification from their supplied
signal: b.signal,
surface: fixtureSurface(b.signal),
});
assert.equal(first.layers.length, 40);
assert.equal(first.layers.length, 41);
assert.ok(first.get('local-adsb'), 'Local ADS-B is registered');
assert.deepEqual(
first.metadata.find(({ id }) => id === 'osm-places'),
@@ -162,7 +162,7 @@ test('Mapillary can be omitted or replaced without changing catalog membership',
surface: fixtureSurface(lifetime.signal),
};
const catalog = createApplicationCatalog(options);
assert.equal(catalog.layers.length, 40);
assert.equal(catalog.layers.length, 41);
assert.deepEqual(catalog.get('street-level').providerIds, []);
assert.equal(
catalogControlServices(catalog).streetLevelLayer,
+2 -1
View File
@@ -9,7 +9,8 @@ import { t } from '../i18n/index.js';
* Construct the existing controls and camera presentation. `initialShareCode`
* opens a share code (the `<code>` of `/share/<code>`) at start-up, for a host
* whose address does not carry one. `localePicker: false` removes the language
* picker, for a host that does not initialize Core's locale.
* picker, for a host that offers its own language choice (`switchLocale` from
* the `./i18n` entry).
*/
export function createApplicationControls({
scene: { viewer, mapStackController, operations },
+272 -24
View File
@@ -12,13 +12,25 @@
*/
import * as Cesium from 'cesium';
import { announceNavigationAuthority } from '../navigationPolicy.js';
import { annotationsFromParams, createView } from '../view/index.js';
import {
announceNavigationAuthority,
watchCameraOwner,
} from '../navigationPolicy.js';
import { t, translateKnown } from '../i18n/index.js';
import { GOOGLE_LABEL } from '../layers/streetLevel/providers/google/policy.js';
import {
annotationsFromParams,
createView,
followFromParams,
groundedLookAt,
streetLevelFromParams,
} from '../view/index.js';
const EMBED_VIEW_MESSAGE = 'gev:view';
const EMBED_APPLIED_MESSAGE = 'gev:view-applied';
const EMBED_READY_MESSAGE = 'gev:ready';
const STREET_LEVEL_LAYER = 'street-level';
const FOLLOW_LAYERS = {
aircraft: 'flights',
military_aircraft: 'military',
@@ -27,6 +39,8 @@ const FOLLOW_LAYERS = {
const FOLLOW_ATTEMPTS = 20;
const FOLLOW_RETRY_MS = 1000;
const FLIGHT_SECONDS = 2;
/** How long a posted view waits for the terrain under its camera. */
const TERRAIN_WAIT_MS = 2000;
/**
* Whether this page shows the app embedded: `?embed=1` when another page
@@ -45,6 +59,14 @@ export function isEmbeddedInline() {
return globalThis.GEV_EMBED_INLINE === true;
}
/**
* Whether the panel page started the app for a tool result issued just now,
* not one its host handed back on mounting the panel again.
*/
function isFreshPanelResult() {
return isEmbeddedInline() && globalThis.GEV_EMBED_FRESH === true;
}
/**
* `doc`'s visibility for work that pauses in a hidden page. A panel's host
* may report it hidden while it is on screen (see keepPanelRendering), so an
@@ -165,6 +187,40 @@ function flyTo(viewer, camera) {
});
}
/**
* The `show_street_level` arguments for a view's street-level photos. A plain
* link (`autoOpen: false`) only arms a source that bills every load; a view
* posted into the embedded app is the client's own request and opens.
*/
function streetLevelArgs(streetLevel, { autoOpen = true } = {}) {
return {
latitude: streetLevel.lat,
longitude: streetLevel.lon,
...(streetLevel.provider ? { provider: streetLevel.provider } : {}),
fly: true,
...(autoOpen ? {} : { autoOpen: false }),
};
}
const STREET_LEVEL_PANEL_ID = 'street-level-panel';
/** Dispatched on the window when a link opened after start-up has restored. */
const SHARE_NAVIGATION_RESTORED_EVENT = 'gev:share-navigation-restore-settled';
/** The armed link's toast, naming the OPEN button as the panel shows it. */
function armedToast() {
return t('Street View is ready: press {open} to load it', {
open: t('OPEN {labels}', { labels: translateKnown(GOOGLE_LABEL) }),
});
}
/** Why a link's photo stays shut: its source has no working key here. */
function keyToast(result) {
return t(
"{source} cannot open this link's street-level photo without a working key",
{ source: translateKnown(result.providerName || 'Street Level') },
);
}
/**
* Make the app show a view: style and map, exactly the view's layers,
* annotations, the camera, and the followed entity. Each step runs through the
@@ -173,13 +229,20 @@ function flyTo(viewer, camera) {
*/
export async function applyView(
view,
{ viewer, dataManager, run, signal, retryMs = FOLLOW_RETRY_MS },
{
viewer,
dataManager,
run,
signal,
autoOpen = true,
retryMs = FOLLOW_RETRY_MS,
},
) {
const steps = [];
const act = async (name, args) => {
const act = async (name, args, options) => {
let result = null;
try {
result = await run(name, args);
result = await run(name, args, options);
} catch (error) {
result = { ok: false, error: error?.message || String(error) };
}
@@ -198,6 +261,8 @@ export async function applyView(
await act('set_layer_visibility', { layerId: layer.id, enabled: false });
}
for (const layerId of view.layers) {
// The street-level action lights its source first, then the layer.
if (view.street_level && layerId === STREET_LEVEL_LAYER) continue;
if (!dataManager.isEnabled(layerId))
await act('set_layer_visibility', { layerId, enabled: true });
}
@@ -207,6 +272,29 @@ export async function applyView(
await act('clear_annotations', {});
if (view.annotations.length)
await act('annotate_map', { annotations: [...view.annotations] });
if (view.street_level) {
// The photos frame the camera; when the action fails before flying, the
// view's own camera still applies.
// A posted view replays someone else's view, as a link does: the source
// it lights is not saved as the visitor's own choice.
const result = await act(
'show_street_level',
streetLevelArgs(view.street_level, { autoOpen }),
{ origin: 'programmatic' },
);
if (result?.ok !== true) {
// Overtaken (a click on the map, a newer request): hands off.
if (result?.cancelled || result?.superseded) return steps;
// An earlier view's photo must not stay over a place this view left.
// From the imagery stage on the action has settled the panel itself
// (a photo it could not load keeps its error).
if (result?.stage !== 'imagery')
dataManager.layers?.get?.(STREET_LEVEL_LAYER)?.module?.closeViewer?.();
if (!result?.location)
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
}
return steps;
}
if (!view.follow) {
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
return steps;
@@ -250,18 +338,115 @@ export function installViews({
dataManager,
run,
signal,
terrain = null,
location = globalThis.location,
windowRef = globalThis.window,
}) {
const ready = Promise.resolve(shell.initialRestorePromise).catch(() => {});
const linked = annotationsFromParams(
new URLSearchParams(String(location?.hash || '').replace(/^#/, '')),
const hashParams = new URLSearchParams(
String(location?.hash || '').replace(/^#/, ''),
);
if (linked.length)
void ready.then(() => {
if (!signal?.aborted) void run('annotate_map', { annotations: linked });
// The link's own steps run first, one at a time, and posted views queue
// behind them; one failing step does not take the next with it. A passive
// link never flies over someone who took the camera: its steps are skipped
// when a newer link superseded it, when its flight never landed (the
// recipient dragged or searched), or after any navigation since the
// generation its restore flight owned.
const cameraLanded = (outcome) => outcome?.share?.camera === 'applied';
const restoreGeneration = (outcome) =>
Number.isInteger(outcome?.navigationToken)
? outcome.navigationToken
: shell._navigationGeneration;
let generationAtRestore = null;
let startSuperseded = false;
let queue = ready.then((outcome) => {
generationAtRestore = restoreGeneration(outcome);
startSuperseded =
outcome?.status === 'superseded' ||
(Boolean(outcome?.share) && !cameraLanded(outcome));
});
const navigatedSince = (generation) =>
Number.isFinite(generation) && shell._navigationGeneration !== generation;
const linkStep = (work) =>
queue.then(async () => {
if (signal?.aborted) return;
try {
await work();
} catch {
/* reported by the action itself */
}
});
if (!isEmbedded(location)) return () => {};
const linked = annotationsFromParams(hashParams);
if (linked.length)
queue = linkStep(() => run('annotate_map', { annotations: linked }));
// A link's street-level photos open once it has restored. A billed source
// only arms, except for the panel page's own link to a result issued just
// now. A link that follows an entity keeps the camera on it.
const linkedStreetLevel = (params) =>
followFromParams(params) ? null : streetLevelFromParams(params);
const openLinkedStreetLevel = async (
streetLevel,
{ generation, autoOpen = false },
) => {
if (navigatedSince(generation)) return;
// The link replays the sender's view: nothing it switches becomes the
// recipient's own preference.
const result = await run(
'show_street_level',
streetLevelArgs(streetLevel, { autoOpen }),
{ origin: 'programmatic' },
);
// A source without a working key changes nothing: say why, or the link
// looks broken. In a frame the host reads the result instead.
if (result?.stage === 'key' && !isEmbedded(location)) {
shell._showToast?.(keyToast(result));
return;
}
// OPEN must be in view, but embed mode hides the rail that holds it: in
// a frame a posted view is the way to open a photo.
if (result?.armed === true && !isEmbedded(location)) {
// Opened for this link only: the recipient's saved layout stays theirs.
shell.setPanelCollapsed?.(STREET_LEVEL_PANEL_ID, false, {
explicit: true,
persist: false,
});
// On a phone the panel sits in a tab sheet: open it so OPEN shows.
shell.revealPanel?.(STREET_LEVEL_PANEL_ID);
shell._showToast?.(armedToast());
}
};
const startStreetLevel = linkedStreetLevel(hashParams);
if (startStreetLevel)
queue = linkStep(() =>
startSuperseded
? undefined
: openLinkedStreetLevel(startStreetLevel, {
generation: generationAtRestore,
autoOpen: isFreshPanelResult(),
}),
);
// A link pasted into the open tab: its photos follow once its restore
// settles, under the same rules.
const onNavigatedRestore = (event) => {
if (event?.detail?.status !== 'settled' || !cameraLanded(event.detail))
return;
const streetLevel = linkedStreetLevel(
new URLSearchParams(String(location?.hash || '').replace(/^#/, '')),
);
if (!streetLevel) return;
const generation = restoreGeneration(event.detail);
queue = linkStep(() => openLinkedStreetLevel(streetLevel, { generation }));
};
windowRef?.addEventListener?.(
SHARE_NAVIGATION_RESTORED_EVENT,
onNavigatedRestore,
);
const stopNavigatedRestores = () =>
windowRef?.removeEventListener?.(
SHARE_NAVIGATION_RESTORED_EVENT,
onNavigatedRestore,
);
if (!isEmbedded(location)) return stopNavigatedRestores;
windowRef.document.body.classList.add('ui-embed');
shell.setCleanView?.(true);
@@ -278,7 +463,6 @@ export function installViews({
peer.postMessage(message, origin);
};
// Views apply one at a time, in the order they arrive.
let queue = ready;
const onMessage = (event) => {
if (event.source !== peer || event.data?.type !== EMBED_VIEW_MESSAGE)
return;
@@ -302,19 +486,82 @@ export function installViews({
);
return;
}
queue = queue.then(async () => {
if (signal?.aborted) return;
const steps = await applyView(view, { viewer, dataManager, run, signal });
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: steps.every((step) => step.ok),
steps,
},
replyOrigin,
// A panel's host can hand back old results on a remount: only one issued
// just now may open a billed photo. A framing page's view is its own.
const autoOpen = !isEmbeddedInline() || event.data.fresh === true;
// Photos asked for without a point open where the camera looks, measured
// from the ground under it, as show_in_gods_eye_view places them.
const asked = event.data.view?.street_level;
const unplaced =
asked &&
typeof asked === 'object' &&
asked.lat == null &&
asked.lon == null;
queue = queue
.then(async () => {
if (signal?.aborted) return;
if (unplaced && view.street_level) {
// A terrain service that does not answer soon leaves the view's
// own estimate.
const wait = AbortSignal.timeout(TERRAIN_WAIT_MS);
const owner = watchCameraOwner(viewer, shell);
const point = await groundedLookAt(
view.camera,
terrain,
signal ? AbortSignal.any([signal, wait]) : wait,
)
.catch(() => null)
.finally(() => owner.stop());
if (signal?.aborted) return;
// Whoever takes the camera while the terrain answers (a drag, a
// search, the assistant) keeps it: the view stands down.
if (owner.taken()) {
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: false,
steps: [{ step: 'street_level', ok: false, superseded: true }],
},
replyOrigin,
);
return;
}
if (point)
view = createView({
...event.data.view,
street_level: { ...asked, ...point },
});
}
const steps = await applyView(view, {
viewer,
dataManager,
run,
signal,
autoOpen,
});
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: steps.every((step) => step.ok),
steps,
},
replyOrigin,
);
})
// A view that fails is answered, and the views behind it still apply.
.catch((error) =>
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: false,
error: error?.message || String(error),
},
replyOrigin,
),
);
});
};
windowRef.addEventListener('message', onMessage);
void ready.then(() => {
@@ -322,6 +569,7 @@ export function installViews({
});
return () => {
stopRendering();
stopNavigatedRestores();
windowRef.removeEventListener('message', onMessage);
};
}
+901 -3
View File
@@ -8,7 +8,7 @@ import {
panelAwareVisibility,
} from './embed.js';
import { LayerLifecycle } from '../data/lifecycle.js';
import { createView } from '../view/index.js';
import { createView, viewToParams } from '../view/index.js';
const fakeViewer = () => {
const flights = [];
@@ -32,14 +32,16 @@ const fakeLayers = (enabled) => ({
});
const recorder = (answers = {}) => {
const calls = [];
const run = async (name, args) => {
const options = [];
const run = async (name, args, runOptions) => {
calls.push([name, args]);
options.push([name, runOptions]);
const answer = answers[name];
return typeof answer === 'function'
? answer(calls)
: (answer ?? { ok: true });
};
return { calls, run };
return { calls, run, options };
};
test('embed mode is opt-in by query parameter', () => {
@@ -83,6 +85,19 @@ test('a view is applied through the app actions in order', async () => {
assert.ok(steps.some((step) => step.step === 'camera'));
});
test("a view's street-level step replays the sender's view: nothing it switches is the visitor's choice", async () => {
const { options, run } = recorder();
await applyView(
createView({
camera: { lat: 48.86, lon: 2.29 },
street_level: { provider: 'panoramax', lat: 48.8584, lon: 2.2945 },
}),
{ viewer: fakeViewer(), dataManager: fakeLayers(new Set()), run },
);
const [, runOptions] = options.find(([name]) => name === 'show_street_level');
assert.equal(runOptions?.origin, 'programmatic');
});
test('a followed entity is retried until its layer has it', async () => {
let tries = 0;
const { run } = recorder({
@@ -102,6 +117,65 @@ test('a followed entity is retried until its layer has it', async () => {
assert.deepEqual(steps.at(-1), { step: 'follow', ok: true });
});
test('a posted view without a photo point opens its photos where the camera looks over the ground', async () => {
const parent = { postMessage() {} };
const windowRef = new EventTarget();
windowRef.parent = parent;
windowRef.document = { body: { classList: new Set() } };
windowRef.document.body.classList.add = Set.prototype.add;
const shell = { initialRestorePromise: Promise.resolve(), setCleanView() {} };
const { calls, run } = recorder();
const asked = [];
// Denver: the ground is 1,590 m above the ellipsoid under a camera at 1,909 m.
const terrain = {
getHeights: async (points) => {
asked.push(points);
return points.map(() => ({ ellipsoid: 1590 }));
},
};
const remove = installViews({
shell,
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
terrain,
location: { search: '?embed=1', hash: '' },
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
const event = new Event('message');
Object.assign(event, {
source: parent,
origin: 'https://host.example',
data: {
type: 'gev:view',
id: 1,
view: {
camera: {
lat: 39.74,
lon: -104.99,
altitude_m: 1909,
pitch_deg: -30,
heading_deg: 0,
},
street_level: { provider: 'mapillary' },
},
},
});
windowRef.dispatchEvent(event);
await new Promise((resolve) => setTimeout(resolve, 10));
remove();
const [, args] = calls.find(([name]) => name === 'show_street_level');
// 319 m above the ground at -30° looks ~550 m ahead, not the ~3.3 km the
// ellipsoid altitude would give.
assert.deepEqual(asked, [[{ lat: 39.74, lon: -104.99 }]]);
assert.ok(
args.latitude > 39.744 && args.latitude < 39.746,
String(args.latitude),
);
assert.equal(args.longitude, -104.99);
});
test('an embedded page takes views only from its parent and answers it', async () => {
const posted = [];
const targets = [];
@@ -186,6 +260,59 @@ test('an embedded page takes views only from its parent and answers it', async (
remove();
});
test('an inline panel opens a billed photo only for a view whose result is fresh', async () => {
const windowRef = new EventTarget();
windowRef.parent = windowRef;
windowRef.postMessage = () => {};
windowRef.document = { body: { classList: new Set() } };
windowRef.document.body.classList.add = Set.prototype.add;
Object.assign(windowRef, {
requestAnimationFrame: () => 1,
cancelAnimationFrame() {},
setInterval: () => 1,
clearInterval() {},
});
const { calls, run } = recorder();
globalThis.GEV_EMBED_INLINE = true;
try {
const remove = installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: { search: '', hash: '' },
windowRef,
});
const view = {
camera: { lat: 48.8584, lon: 2.2945 },
map: 'photoreal',
street_level: { provider: 'google', lat: 48.8584, lon: 2.2945 },
};
for (const [id, fresh] of [
[1, true],
[2, false],
[3, undefined],
]) {
const event = new Event('message');
Object.assign(event, {
source: windowRef,
data: { type: 'gev:view', id, view, fresh },
});
windowRef.dispatchEvent(event);
}
await new Promise((resolve) => setTimeout(resolve, 20));
assert.deepEqual(
calls
.filter(([name]) => name === 'show_street_level')
.map(([, args]) => args.autoOpen ?? true),
[true, false, false],
);
remove();
} finally {
delete globalThis.GEV_EMBED_INLINE;
}
});
test('an app loaded inline into a panel page talks through its own window', async () => {
const posted = [];
const windowRef = new EventTarget();
@@ -393,3 +520,774 @@ test('a followed aircraft owns the camera, and cockpit view is entered after it'
/no cockpit view/,
);
});
test('a view with street-level photos lets the action fly and open them', async () => {
const viewer = fakeViewer();
const { calls, run } = recorder();
const view = createView({
camera: { lat: 38.5799, lon: -121.4937, altitude_m: 500 },
street_level: { provider: 'google' },
});
const steps = await applyView(view, {
viewer,
dataManager: fakeLayers(new Set()),
run,
});
assert.deepEqual(
calls.map(([name]) => name),
[
'set_map_stack',
'stop_tracking',
'clear_annotations',
'show_street_level',
],
);
assert.deepEqual(calls.at(-1)[1], {
latitude: 38.5799,
longitude: -121.4937,
provider: 'google',
fly: true,
});
assert.equal(viewer.flights.length, 0, 'the action frames the camera');
assert.ok(steps.every((step) => step.ok));
assert.equal(steps.at(-1).step, 'show_street_level');
});
test('a link to street-level photos opens them after it restores', async () => {
const { calls, run } = recorder();
const toasts = [];
const panels = [];
installViews({
shell: {
initialRestorePromise: Promise.resolve(),
_showToast: (text) => toasts.push(text),
setPanelCollapsed: (...args) => panels.push(args),
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&sl=mapillary,38.58000,-121.49000',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [
[
'show_street_level',
{
latitude: 38.58,
longitude: -121.49,
provider: 'mapillary',
fly: true,
autoOpen: false,
},
],
]);
assert.deepEqual(
[toasts, panels],
[[], []],
'an opened photo needs no toast or panel nudge',
);
// A failure (no key) is not an armed state either: it says why instead.
const failed = recorder({
show_street_level: () => ({
ok: false,
stage: 'key',
error: 'needs its key',
}),
});
installViews({
shell: {
initialRestorePromise: Promise.resolve(),
_showToast: (text) => toasts.push(text),
setPanelCollapsed: (...args) => panels.push(args),
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: failed.run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&sl=google,1.00000,2.00000',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(
[toasts, panels],
[
[
"Street Level cannot open this link's street-level photo without a working key",
],
[],
],
'it says why, and nothing to press opens',
);
});
test('a link that follows an entity keeps the camera: its photos are not opened', async () => {
const { calls, run } = recorder();
const params = viewToParams(
createView({
camera: { lat: 1, lon: 2 },
follow: { kind: 'satellite', id: '25544' },
}),
);
params.set('sl', 'mapillary,1.00000,2.00000');
installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: { search: '', hash: `#${params}` },
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, []);
});
test('a failing link step does not take the next one with it', async () => {
const calls = [];
const run = async (name, args) => {
calls.push(name);
if (name === 'annotate_map') throw new Error('marks failed');
return { ok: true, armed: false, ...args };
};
const an = encodeURIComponent(
JSON.stringify([{ type: 'pin', target: 'Austin' }]),
);
installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`,
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, ['annotate_map', 'show_street_level']);
});
test('a link whose source has no working key says why instead of doing nothing; a frame stays quiet', async () => {
for (const embed of [false, true]) {
const toasts = [];
const { calls, run } = recorder({
show_street_level: () => ({
ok: false,
stage: 'key',
provider: 'google',
providerName: 'Google Street View',
error: 'Google Street View needs its key',
}),
});
const windowRef = new EventTarget();
if (embed) {
windowRef.document = { body: { classList: { add() {} } } };
windowRef.parent = { postMessage() {} };
}
installViews({
shell: {
initialRestorePromise: Promise.resolve(),
setCleanView() {},
_showToast: (text) => toasts.push(text),
setPanelCollapsed() {
throw new Error('nothing to arm');
},
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: embed ? '?embed=1' : '',
hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372',
},
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(calls[0][0], 'show_street_level');
assert.deepEqual(
toasts,
embed
? []
: [
"Google Street View cannot open this link's street-level photo without a working key",
],
);
}
});
test('a plain link never auto-loads a billed photo: it arms the panel and says so; an embedded view opens', async () => {
const toasts = [];
const panels = [];
const revealed = [];
const { calls, run } = recorder({
show_street_level: () => ({ ok: true, armed: true }),
});
installViews({
shell: {
initialRestorePromise: Promise.resolve(),
_showToast: (text) => toasts.push(text),
setPanelCollapsed: (id, collapsed, options) =>
panels.push([id, collapsed, options]),
revealPanel: (id) => revealed.push(id),
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(calls[0][1].autoOpen, false);
assert.deepEqual(toasts, [
'Street View is ready: press OPEN GOOGLE STREET VIEW to load it',
]);
assert.deepEqual(
panels,
[['street-level-panel', false, { explicit: true, persist: false }]],
"the panel opens so OPEN is in view, without saving the layout as the recipient's",
);
assert.deepEqual(revealed, ['street-level-panel'], 'its phone sheet too');
// Adding ?embed=1 to a published link does not lift the gate: the page is
// still a plain top-level load.
const framedToasts = [];
const framedPanels = [];
const framed = recorder({
show_street_level: () => ({ ok: true, armed: true }),
});
const windowRef = new EventTarget();
windowRef.document = { body: { classList: { add() {} } } };
windowRef.parent = { postMessage() {} };
installViews({
shell: {
initialRestorePromise: Promise.resolve(),
setCleanView() {},
_showToast: (text) => framedToasts.push(text),
setPanelCollapsed: (...args) => framedPanels.push(args),
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: framed.run,
location: {
search: '?embed=1',
hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372',
},
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(framed.calls[0][1].autoOpen, false);
// In a frame the rail that holds OPEN is hidden: no nudge, no toast.
assert.deepEqual([framedToasts, framedPanels], [[], []]);
// Only the panel page's own link for a result issued just now is the
// client's request: a host that mounts the panel again hands back an old
// result, and that one only arms, like any link.
globalThis.GEV_EMBED_INLINE = true;
try {
for (const fresh of [true, false]) {
globalThis.GEV_EMBED_FRESH = fresh;
const inline = recorder({ show_street_level: () => ({ ok: true }) });
const inlineWindow = new EventTarget();
inlineWindow.document = { body: { classList: { add() {} } } };
inlineWindow.postMessage = () => {};
// The inline panel keeps rendering from a timer; give it inert clocks.
Object.assign(inlineWindow, {
requestAnimationFrame: () => 1,
cancelAnimationFrame() {},
setTimeout: () => 1,
clearTimeout() {},
setInterval: () => 1,
clearInterval() {},
});
const stop = installViews({
shell: { initialRestorePromise: Promise.resolve(), setCleanView() {} },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: inline.run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&map=photoreal&sl=google,38.57986,-121.49372',
},
windowRef: inlineWindow,
});
await new Promise((resolve) => setTimeout(resolve, 0));
if (fresh) assert.equal('autoOpen' in inline.calls[0][1], false);
else assert.equal(inline.calls[0][1].autoOpen, false, 'a remount arms');
stop();
}
} finally {
delete globalThis.GEV_EMBED_INLINE;
delete globalThis.GEV_EMBED_FRESH;
}
// applyView (a posted view) opens too.
const posted = recorder();
await applyView(
createView({
camera: { lat: 1, lon: 2 },
street_level: { provider: 'google' },
}),
{
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: posted.run,
},
);
assert.equal('autoOpen' in posted.calls.at(-1)[1], false);
});
test('a view whose photos fail before any flight still gets its camera', async () => {
const viewer = fakeViewer();
const { calls, run } = recorder({
show_street_level: () => ({
ok: false,
stage: 'key',
error: 'needs its key',
}),
});
const view = createView({
camera: { lat: 1, lon: 2, altitude_m: 500 },
street_level: { provider: 'google' },
});
const steps = await applyView(view, {
viewer,
dataManager: fakeLayers(new Set()),
run,
});
assert.equal(viewer.flights.length, 1, 'the view flies itself');
assert.deepEqual(
steps.map((step) => [step.step, step.ok]),
[
['set_map_stack', true],
['stop_tracking', true],
['clear_annotations', true],
['show_street_level', false],
['camera', true],
],
);
assert.ok(calls.some(([name]) => name === 'show_street_level'));
// A flight that happened (the action reports where) is not repeated.
const flown = fakeViewer();
const { run: flownRun } = recorder({
show_street_level: () => ({
ok: false,
stage: 'imagery',
location: 'Somewhere',
error: 'no imagery',
}),
});
await applyView(view, {
viewer: flown,
dataManager: fakeLayers(new Set()),
run: flownRun,
});
assert.equal(flown.flights.length, 0);
});
test('a link runs its marks and photos in order, and posted views wait behind them', async () => {
const log = [];
let releaseStreetLevel = null;
let releaseMarks = null;
const run = async (name, args) => {
log.push(`start:${name}`);
if (name === 'annotate_map')
await new Promise((resolve) => {
releaseMarks = resolve;
});
if (name === 'show_street_level')
await new Promise((resolve) => {
releaseStreetLevel = resolve;
});
log.push(`end:${name}`);
return {
ok: true,
...(name === 'show_street_level' ? { location: 'x' } : {}),
};
};
const windowRef = new EventTarget();
windowRef.document = { body: { classList: { add() {} } } };
// A framing page: answers go to it, not to the app's own window.
windowRef.parent = {
postMessage: (message) => log.push(`post:${message.type}`),
};
const an = encodeURIComponent(
JSON.stringify([{ type: 'pin', target: 'Austin' }]),
);
installViews({
shell: { initialRestorePromise: Promise.resolve(), setCleanView() {} },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '?embed=1',
hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`,
},
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
// A view arrives while the link's photos are still opening.
windowRef.dispatchEvent(
Object.assign(new Event('message'), {
source: windowRef.parent,
origin: 'null',
data: {
type: 'gev:view',
id: 'v1',
view: { camera: { lat: 3, lon: 4 } },
},
}),
);
await new Promise((resolve) => setTimeout(resolve, 0));
const actions = () => log.filter((entry) => !entry.startsWith('post:'));
assert.deepEqual(
actions(),
['start:annotate_map'],
'the photos wait for the marks',
);
releaseMarks();
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(actions().slice(0, 3), [
'start:annotate_map',
'end:annotate_map',
'start:show_street_level',
]);
assert.ok(
!log.includes('start:stop_tracking'),
'the posted view waits for the link',
);
releaseStreetLevel();
for (
let tick = 0;
tick < 20 && !log.includes('post:gev:view-applied');
tick++
)
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(
log.indexOf('end:show_street_level') < log.indexOf('start:stop_tracking'),
true,
);
assert.ok(log.includes('post:gev:view-applied'), log.join(' '));
});
test('an aborted install never runs the link photos', async () => {
const { calls, run } = recorder();
const controller = new AbortController();
controller.abort();
installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
signal: controller.signal,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&sl=google,1.00000,2.00000',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, []);
});
test('a passive link stands down when someone navigated while it waited behind its marks', async () => {
const calls = [];
let releaseMarks = null;
const shell = {
initialRestorePromise: Promise.resolve(),
_navigationGeneration: 7,
};
const run = async (name) => {
calls.push(name);
if (name === 'annotate_map')
await new Promise((resolve) => {
releaseMarks = resolve;
});
return { ok: true };
};
const an = encodeURIComponent(
JSON.stringify([{ type: 'pin', target: 'Austin' }]),
);
installViews({
shell,
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`,
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, ['annotate_map']);
shell._navigationGeneration = 8; // the recipient searched, or the assistant flew
releaseMarks();
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(
calls,
['annotate_map'],
'the link did not fly over the newer navigation',
);
// Unchanged navigation: the photos open as before.
const quiet = [];
installViews({
shell: {
initialRestorePromise: Promise.resolve(),
_navigationGeneration: 3,
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: async (name) => {
quiet.push(name);
return { ok: true };
},
location: {
search: '',
hash: `#v=2&lat=1&lon=2&an=${an}&sl=mapillary,1.00000,2.00000`,
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(quiet, ['annotate_map', 'show_street_level']);
});
test('a link replays the sender’s view: its street-level step never writes the recipient’s preferences', async () => {
const { calls, run, options } = recorder();
installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&sl=mapillary,38.58000,-121.49000',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(calls.length, 1);
assert.deepEqual(options, [
['show_street_level', { origin: 'programmatic' }],
]);
});
test('a start link whose restore a newer link superseded opens nothing', async () => {
const { calls, run } = recorder();
installViews({
shell: {
initialRestorePromise: Promise.resolve({ status: 'superseded' }),
_navigationGeneration: 4,
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&sl=mapillary,38.58000,-121.49000',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [], 'start link A does not pull the camera from B');
});
test('a start link whose restore flight the recipient interrupted leaves the camera with them', async () => {
for (const [camera, expected] of [
['cancelled', 0],
['skipped', 0],
['applied', 1],
]) {
const { calls, run } = recorder();
installViews({
shell: {
initialRestorePromise: Promise.resolve({
status: 'settled',
share: { camera },
}),
_navigationGeneration: 4,
},
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: {
search: '',
hash: '#v=2&lat=1&lon=2&sl=mapillary,38.58000,-121.49000',
},
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(calls.length, expected, camera);
}
});
test('a link pasted into the open tab opens or arms its photos once its restore settles', async () => {
const { calls, run, options } = recorder();
const windowRef = new EventTarget();
const location = { search: '', hash: '#v=2&lat=1&lon=2' };
const shell = {
initialRestorePromise: Promise.resolve(),
_navigationGeneration: 1,
};
const remove = installViews({
shell,
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location,
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [], 'the start link has no photos');
const settled = (status, camera = 'applied') =>
windowRef.dispatchEvent(
Object.assign(new Event('gev:share-navigation-restore-settled'), {
detail: { status, share: { camera } },
}),
);
location.hash = '#v=2&lat=3&lon=4&sl=mapillary,48.88740,2.34950';
shell._navigationGeneration = 2; // the restore's own flight
settled('failed');
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [], 'a failed restore opens nothing');
// The recipient dragged or searched during the restore flight: the camera
// is theirs, and the link does not pull it back.
shell._navigationGeneration = 3;
for (const camera of ['cancelled', 'superseded']) {
settled('settled', camera);
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(
calls,
[],
`an interrupted restore (${camera}) opens nothing`,
);
}
settled('settled');
await new Promise((resolve) => setTimeout(resolve, 0));
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [
[
'show_street_level',
{
latitude: 48.8874,
longitude: 2.3495,
provider: 'mapillary',
fly: true,
autoOpen: false,
},
],
]);
assert.deepEqual(options.at(-1), [
'show_street_level',
{ origin: 'programmatic' },
]);
remove();
location.hash = '#v=2&lat=5&lon=6&sl=mapillary,1.00000,2.00000';
settled('settled');
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(calls.length, 1, 'removed with the install');
});
test('a view whose photo step the user overtook leaves their photo and camera alone', async () => {
// Photo B, opened by a click while the posted view looked up its photo.
const street = { open: false, imageId: null };
const dataManager = {
...fakeLayers(new Set()),
layers: new Map([
[
'street-level',
{
module: {
closeViewer: () =>
Object.assign(street, { open: false, imageId: null }),
},
},
],
]),
};
const { run } = recorder({
show_street_level: () => {
Object.assign(street, { open: true, imageId: 'b' });
return {
ok: false,
stage: 'imagery',
cancelled: true,
superseded: true,
error: 'The camera moved on before the photo opened',
};
},
});
const view = createView({
camera: { lat: 1, lon: 2, altitude_m: 500 },
street_level: { provider: 'mapillary' },
});
const viewer = fakeViewer();
const steps = await applyView(view, { viewer, dataManager, run });
assert.deepEqual(street, { open: true, imageId: 'b' }, 'photo B stays');
assert.equal(viewer.flights.length, 0, 'the camera stays with the user');
assert.equal(steps.at(-1).ok, false, 'reported as not applied');
});
test('a view whose photo was found but could not load keeps its error in view', async () => {
const closed = [];
const dataManager = {
...fakeLayers(new Set()),
layers: new Map([
['street-level', { module: { closeViewer: () => closed.push(true) } }],
]),
};
const { run } = recorder({
show_street_level: () => ({
ok: false,
stage: 'imagery',
imageryError: true,
location: '1.00000, 2.00000',
error: 'Image could not be opened',
}),
});
const view = createView({
camera: { lat: 1, lon: 2, altitude_m: 500 },
street_level: { provider: 'mapillary' },
});
await applyView(view, { viewer: fakeViewer(), dataManager, run });
assert.equal(closed.length, 0, 'the viewer keeps the new photo’s error');
});
test('a view whose photos fail closes the photo an earlier view opened', async () => {
const closed = [];
const dataManager = {
...fakeLayers(new Set()),
layers: new Map([
['street-level', { module: { closeViewer: () => closed.push(true) } }],
]),
};
const { run } = recorder({
show_street_level: () => ({
ok: false,
stage: 'key',
error: 'needs its key',
}),
});
const view = createView({
camera: { lat: 1, lon: 2, altitude_m: 500 },
street_level: { provider: 'google' },
});
const viewer = fakeViewer();
await applyView(view, { viewer, dataManager, run });
assert.equal(closed.length, 1, 'closed before the view flies away');
assert.equal(viewer.flights.length, 1);
const opened = recorder({ show_street_level: () => ({ ok: true }) });
await applyView(view, { viewer, dataManager, run: opened.run });
assert.equal(closed.length, 1, 'an opened photo stays');
});
+118
View File
@@ -0,0 +1,118 @@
import '../testSupport/registerDom.mjs';
import test from 'node:test';
import assert from 'node:assert/strict';
import * as Cesium from 'cesium';
import { installViews } from './embed.js';
import { createGevActionRunner } from '../voice/gevActions.js';
import { createAnnotationEngine } from '../annotations/annotationEngine.js';
import { fakeCesiumViewer } from '../testSupport/streetLevelFakes.mjs';
// A camera taken while a posted view waits on terrain stays with whoever took
// it: the stale view stands down whole, so it neither stops a newer follow nor
// frames its own marks. Offline: real installViews, action runner and
// annotation engine; the viewer, renderer, flights data and terrain are fixtures.
globalThis.fetch = async () => { throw new Error('Unexpected network request in offline reproduction'); };
globalThis.requestAnimationFrame = () => 1;
globalThis.cancelAnimationFrame = () => {};
const tick = () => new Promise(resolve => setTimeout(resolve, 0));
function deferred() {
let resolve;
const promise = new Promise(done => { resolve = done; });
return { promise, resolve };
}
async function scenario(t, effect) {
document.body.className = '';
const entered = deferred();
const terrain = deferred();
const viewer = fakeCesiumViewer();
viewer.clock = { onTick: new Cesium.Event() };
viewer.canvas = viewer.scene.canvas;
// A real perspective frustum looking down at 0,0; the old view's mark
// at 48N,2E is outside it. No override of the engine's visibility logic.
Object.assign(viewer.camera, {
position: Cesium.Cartesian3.fromDegrees(0, 0, 1000),
direction: new Cesium.Cartesian3(-1, 0, 0),
up: new Cesium.Cartesian3(0, 0, 1),
heading: 0,
frustum: new Cesium.PerspectiveFrustum({ fov: Math.PI / 3, aspectRatio: 1, near: 1, far: 20000000 }),
});
const entity = { id: 'fixture-aircraft' };
const aircraft = { icao24: 'fixture', callsign: 'Fixture', latitude: 0, longitude: 0 };
let stopCalls = 0;
const flights = {
findByQuery: () => aircraft,
trackById: () => { viewer.trackedEntity = entity; return true; },
getTrackedInfo: () => viewer.trackedEntity ? aircraft : null,
stopTracking: () => { stopCalls++; viewer.trackedEntity = undefined; return true; },
};
const dataManager = {
layers: new Map([['flights', { module: flights }]]),
getAll: () => [{ id: 'flights', enabled: true }],
isEnabled: id => id === 'flights',
setLayerParams: () => true,
};
const shell = {
_navigationGeneration: 1,
initialRestorePromise: Promise.resolve(),
setCleanView() {},
runImmediateNavigation(_noun, navigate) { this._navigationGeneration++; return navigate(); },
};
const annotations = createAnnotationEngine({ viewer,
renderer: { add() {}, update() {}, remove() {}, sync() {}, destroy() {} },
});
const runner = createGevActionRunner({ viewer, dataManager, styleManager: shell, annotations });
const calls = [];
const run = async (name, args, options) => {
calls.push(name);
return runner(name, args, options);
};
const replies = [];
const peer = { postMessage: message => replies.push(message) };
const windowRef = new EventTarget();
windowRef.parent = peer;
windowRef.document = document;
const remove = installViews({ shell, viewer, dataManager, run, windowRef,
location: { search: '?embed=1', hash: '' },
terrain: { getHeights: () => { entered.resolve(); return terrain.promise; } },
});
t.after(() => { remove(); runner.dispose(); annotations.destroy(); document.body.className = ''; });
await tick();
const event = new Event('message');
Object.assign(event, { source: peer, origin: 'https://host.example', data: {
type: 'gev:view', id: 1, view: {
camera: { lat: 48, lon: 2, altitude_m: 1000, pitch_deg: -45 },
layers: ['flights'],
street_level: { provider: 'mapillary' },
annotations: effect === 'annotations'
? [{ type: 'pin', latitude: 48, longitude: 2, label: 'Old view' }] : [],
},
} });
windowRef.dispatchEvent(event);
await entered.promise;
if (effect === 'follow') {
const follow = await runner('track_entity', { query: 'Fixture', layerId: 'flights' });
assert.equal(follow.ok, true, 'new follow starts through the real runner');
assert.equal(viewer.trackedEntity, entity);
} else {
viewer.canvas.dispatchEvent(new Event('pointerdown'));
}
terrain.resolve([{ ellipsoid: 200 }]);
for (let n = 0; n < 50 && !replies.some(reply => reply.id === 1); n++) await tick();
const reply = replies.find(reply => reply.id === 1);
t.diagnostic(JSON.stringify({ effect, calls, stopCalls, followed: viewer.trackedEntity === entity,
annotationCount: annotations.count(), annotationFlights: viewer.flights.started, reply }));
assert.ok(reply, 'host receives completion');
assert.equal(reply.ok, false, 'superseded view is acknowledged as not applied');
assert.ok(!calls.includes('show_street_level'), 'superseded request never reaches photo handler');
if (effect === 'follow') {
assert.equal(stopCalls, 0, 'stale view must not stop the newer follow');
assert.equal(viewer.trackedEntity, entity, 'the new tracked entity stays selected');
} else {
assert.equal(viewer.flights.started, 0, 'stale annotations must not fly the camera');
}
}
for (const effect of ['follow', 'annotations']) {
test(`a camera taken during the terrain wait keeps the user's ${effect}`, { timeout: 5000 }, t => scenario(t, effect));
}
+23 -7
View File
@@ -462,8 +462,11 @@ export function panelRuntime(config) {
* Load God's Eye View into this page from the app's panel build, in
* inline embed mode, starting at the first view as its link would.
*/
async function startApp(url, view) {
async function startApp(url, view, fresh = false) {
window.GEV_EMBED_INLINE = true;
// Whether this result was just issued, not handed back by a host that
// mounted the panel again: only then may its link open a billed photo.
window.GEV_EMBED_FRESH = fresh;
window.GEV_APP_BASE_URL = config.appBaseUrl;
try {
history.replaceState(
@@ -551,13 +554,21 @@ export function panelRuntime(config) {
);
}
// Tool results: the first view starts the app, later ones move it.
// Tool results: the first view starts the app, later ones move it. A
// result issued longer ago than this was handed back on a remount.
const FRESH_RESULT_MS = 5 * 60 * 1000;
let started = false;
let ready = false;
let queued = null;
let queuedFresh = false;
let currentUrl = null;
const postView = (view) =>
window.postMessage({ type: 'gev:view', id: nextId++, view }, '*');
// Every view carries whether its result was issued just now: only those may
// open a billed photo, whichever result on this mount they came in.
const postView = (view, fresh) =>
window.postMessage(
{ type: 'gev:view', id: nextId++, view, fresh: fresh === true },
'*',
);
function show(result) {
const data = result?.structuredContent;
const view = data?.view;
@@ -565,13 +576,18 @@ export function panelRuntime(config) {
if (!view || !url) return;
currentUrl = url;
open.hidden = false;
if (ready) return postView(view);
const issuedAt = Number(data.issued_at);
const fresh =
Number.isFinite(issuedAt) &&
Math.abs(Date.now() - issuedAt) <= FRESH_RESULT_MS;
if (ready) return postView(view, fresh);
queued = view;
queuedFresh = fresh;
if (started) return;
started = true;
// i18n-ignore: host-panel script, no app catalogs
say("Loading God's Eye View…");
startApp(url, view).catch((error) =>
startApp(url, view, fresh).catch((error) =>
// i18n-ignore: host-panel script, no app catalogs
say(`God's Eye View could not load here: ${error?.message || error}.`),
);
@@ -592,7 +608,7 @@ export function panelRuntime(config) {
if (message?.type === 'gev:ready') {
ready = true;
status.remove();
if (queued) postView(queued);
if (queued) postView(queued, queuedFresh);
queued = null;
}
return;
+71
View File
@@ -0,0 +1,71 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { panelRuntime } from './globePanelRuntime.js';
const element = () => ({
isConnected: true,
hidden: true,
textContent: '',
addEventListener() {},
remove() {
this.isConnected = false;
},
});
test('every view the panel posts says whether its result was issued just now', async () => {
const savedWindow = globalThis.window;
const savedDocument = globalThis.document;
const views = [];
const parent = { postMessage() {} };
const windowRef = new EventTarget();
Object.assign(windowRef, {
parent,
postMessage: (message) => {
if (message?.type === 'gev:view') views.push(message);
},
});
const elements = { status: element(), open: element(), expand: element() };
globalThis.window = windowRef;
globalThis.document = {
getElementById: (id) => elements[id] ?? null,
addEventListener() {},
baseURI: 'https://host.example/panel',
body: { clientWidth: 600 },
};
const deliver = (source, data) => {
const event = new Event('message');
Object.assign(event, { source, data });
windowRef.dispatchEvent(event);
};
const result = (issuedAt) => ({
jsonrpc: '2.0',
method: 'ui/notifications/tool-result',
params: {
structuredContent: {
url: 'http://localhost:4173/#sl=google,48.85840,2.29450',
issued_at: issuedAt,
view: { camera: { lat: 48.8584, lon: 2.2945 } },
},
},
});
try {
panelRuntime({
appBaseUrl: 'http://localhost:4173/',
panelBase: '/panel/',
protocolVersion: '2025-06-18',
panelHeight: 480,
});
deliver(windowRef, { type: 'gev:ready' });
deliver(parent, result(Date.now()));
// A host replaying an old result on the same mount.
deliver(parent, result(Date.now() - 60 * 60 * 1000));
deliver(parent, result(undefined));
} finally {
globalThis.window = savedWindow;
globalThis.document = savedDocument;
}
assert.deepEqual(
views.map((message) => message.fresh),
[true, false, false],
);
});
+1
View File
@@ -18,6 +18,7 @@ export class LayerPresentation {
this.pendingVisible = false;
this._unsubscribe = manager.subscribeActivity((change) => {
if (change.type === 'status') this.refresh();
else if (change.type === 'registrations') this._panel?.rerender();
else if (change.type === 'destroy-all') this.destroy();
else {
const reason =
+50
View File
@@ -0,0 +1,50 @@
import { createRailLayer } from '../../layers/rail/index.js';
import * as render from '../../renderGovernor.js';
import * as sprites from '../../data/spriteOrder.js';
import * as picking from '../../data/pickRegistry.js';
import * as overlays from '../../overlays/worldOverlay.js';
import { applyTrackedCameraFrame } from '../../data/trackedCamera.js';
import { WORLD_FOCUS_FRAMING, requestWorldFocus } from '../../worldFocus.js';
import {
railFeedCredit,
registerDynamicCredit,
} from '../../data/dataCredits.js';
/**
* Construct the Rail layer on the application's scene owners and its shared
* ground floor, the same floor Transit and the aircraft stand on.
* @param {{surface: object, source?: object}} options
* @returns {object}
*/
export function createApplicationRail({ surface, source }) {
if (!surface?.groundFloor)
throw new TypeError('Rail needs the application surface services');
const { groundFloor } = surface;
return createRailLayer({
source,
services: {
render,
sprites,
picking,
overlays,
credits: { registerDynamicCredit, railFeedCredit },
ground: {
GROUND_FLOOR_LIFT_M: groundFloor.GROUND_FLOOR_LIFT_M,
cachedGroundFloor: groundFloor.cachedGroundFloor,
warmGroundFloor: groundFloor.warmGroundFloor,
neighborFloorM: groundFloor.neighborFloorM,
coarseFloorCoord: groundFloor.coarseFloorCoord,
},
// A selected train is followed as a tracked aircraft is, or flown to
// once as a vessel is (src/layers/rail/follow.js).
camera: {
framing: WORLD_FOCUS_FRAMING.train,
requestWorldFocus,
applyTrackedCameraFrame,
isCockpitActive: () =>
globalThis.document?.body?.classList?.contains('cockpit-mode') ===
true,
},
},
});
}
+1
View File
@@ -55,6 +55,7 @@ export const CATALOG_SOURCE_CONTRACTS = Object.freeze({
'wildlife-sightings': contract(['wildlife-sightings'], ['getSnapshot']),
cables: contract(['telegeography-submarine-cables'], ['fetch']),
transit: contract(['transit'], ['requestSnapshot', 'getHistory']),
rail: contract(['rail'], ['requestSnapshot', 'getFeeds']),
});
/** An omitted provider is unavailable, never an authoritative empty dataset. */
+16 -1
View File
@@ -37,7 +37,7 @@ test('every acquisition source can be omitted without losing catalog membership'
test('all acquisition sources may be absent, and enable fails before initializing or fetching', async (t) => {
const actual = catalog({}, t);
assert.equal(actual.layers.length, 40);
assert.equal(actual.layers.length, 41);
const manager = new LayerLifecycle(
{},
{ getSourceAvailability: actual.getSourceAvailability },
@@ -247,3 +247,18 @@ test('absent military acquisition does not poll, suppress flights, or erase supp
assert.equal(registry.isMilitaryLayerActive(), false);
assert.equal(fetches, 0);
});
test('a rail source must carry the feed catalog that keeps unbuilt packs out', () => {
// Without getFeeds the layer could never learn which schedule packs are
// built, so the live feeds that read their routes from one would never run.
assert.throws(
() =>
resolveCatalogSources({ rail: { requestSnapshot: async () => null } }),
/Invalid catalog source: rail/,
);
const source = {
requestSnapshot: async () => null,
getFeeds: async () => [],
};
assert.equal(resolveCatalogSources({ rail: source }).sources.rail, source);
});
+53 -1
View File
@@ -11,6 +11,11 @@ import { useDisplayStore } from '../ui/stores/display.js';
import { applicationPinia } from '../ui/vueRuntime.js';
import { initImageryBoxTool } from '../ui/imageryBoxTool.js';
import { createRecentImageryPanel } from '../ui/recentImagery.js';
import { createDataLibrary } from '../library/controller.js';
import { flyToLibraryExtent } from '../library/camera.js';
import { createLibraryBrowser } from '../ui/libraryBrowser.js';
import { useDataLibraryStore } from '../ui/stores/dataLibrary.js';
import { localGeoJsonServices } from './localGeojsonServices.js';
import { initGevVoiceCommands } from '../voice/gevRealtime.js';
import { installViews, isEmbeddedInline } from './embed.js';
import {
@@ -109,6 +114,51 @@ export function createApplicationTools({
connect: connectDisplay,
});
defer(() => geoFileImport?.destroy());
// DATA ▸ Library: entries register their layers only when a viewer adds
// them, after the core catalog is sealed. The browser dialog and the
// controller share the application lifetime.
const dataLibrary = createDataLibrary({
dataManager,
services: localGeoJsonServices,
getTileset: () => tileset,
mapStackController,
shareLink: styleManager.shareLinkManager,
requestRender: governorRequestRender,
// Zoom to layer: an immediate navigation, so Cockpit can refuse it and
// the current follow owner is released first (as for imported files).
zoomTo: (extent) =>
styleManager.runImmediateNavigation('layer', () =>
flyToLibraryExtent(viewer, extent),
) !== false,
});
defer(() => dataLibrary.destroy());
// Browse: the owner presents the dialog through the store; LayerPanelsView
// draws the Browse button and the dialog from it.
const libraryStore = useDataLibraryStore(applicationPinia());
const libraryBrowser = createLibraryBrowser({
library: dataLibrary,
publish: (view) => libraryStore.publish(view),
});
libraryStore.connect(libraryBrowser.controls);
defer(() => {
libraryBrowser.destroy();
libraryStore.connect(null);
});
dataLibrary.start({ signal }).catch((error) => {
if (!signal?.aborted) console.warn('[Library] start failed:', error);
});
// The live gate's handle (scripts/qa-data-library.mjs).
const libraryHandle = {
library: dataLibrary,
browser: libraryBrowser,
// What Copy Link would copy.
buildShareLink: () => styleManager.shareLinkManager?.buildLink?.() ?? null,
};
window.__gevDataLibrary = libraryHandle;
defer(() => {
if (window.__gevDataLibrary === libraryHandle)
delete window.__gevDataLibrary;
});
// DATA ▸ Recent Imagery: the box tool claims the pointer like Draw and the
// panel lives on the right rail, so both belong to the application
// lifetime. The tileset lets the layer drape while the globe is hidden.
@@ -246,8 +296,10 @@ export function createApplicationTools({
shell: styleManager,
viewer,
dataManager,
run: (name, args) => voiceCommands.runner(name, args, { signal }),
run: (name, args, options = {}) =>
voiceCommands.runner(name, args, { ...options, signal }),
signal,
terrain: operations.requests.terrain,
}),
);
// An application's own tool takes the pointer as Draw does, and its own
+1
View File
@@ -29,6 +29,7 @@ export const writeCalibrationStoreV2 = layer.writeCalibrationStoreV2;
export const deriveCalBadge = layer.deriveCalBadge;
export const computeFrustumGeometry = layer.computeFrustumGeometry;
export const activationProbeClampRange = layer.activationProbeClampRange;
export const flatGroundAxisDistance = layer.flatGroundAxisDistance;
export const frameSignatureFromPixels = layer.frameSignatureFromPixels;
export const _createCctvProjectionPlaneForTest =
layer._createCctvProjectionPlaneForTest;
+12
View File
@@ -35,6 +35,7 @@ import cctvLayer, {
_pushAmbientCardEntriesForTest,
_setCctvOverlayHostForTest,
activationProbeClampRange,
flatGroundAxisDistance,
bindCctvWorldClickGesture,
clearProbeClampOnDeactivation,
computeFrustumGeometry,
@@ -655,6 +656,17 @@ test('activation obstruction clamp uses the field-derived 12 m floor', () => {
assert.equal(activationProbeClampRange(210, 250), null);
});
test('on a globe map the axis ends on flat ground at the mount ground', () => {
// A 24 m mount pitched 30° down meets the ground 48 m along its axis.
assert.ok(Math.abs(flatGroundAxisDistance(24, -30) - 48) < 1e-9);
// The default 700 m range then clamps just short of that, not 700 m out.
const clamp = activationProbeClampRange(700, flatGroundAxisDistance(24, -30));
assert.ok(Math.abs(clamp - 44) < 1e-9);
assert.equal(flatGroundAxisDistance(24, 0), null);
assert.equal(flatGroundAxisDistance(24, 10), null);
assert.equal(flatGroundAxisDistance(0, -30), null);
});
test('deactivation clears the probe clamp and restores nominal frustum geometry', () => {
const record = {
camera: { ...UNCLAMPED_CAMERA },
@@ -0,0 +1,129 @@
// An explicit pick of the camera that a settled move just re-opened must end
// the opening pick. That pick goes through setActiveCamera's UNCHANGED path
// (same camera, already activated); if that path keeps the provisional flag,
// the next settled move (a pan, or the landing of NEAREST's own flight)
// switches the camera away from the one the user chose.
import assert from 'node:assert/strict';
import test from 'node:test';
import * as Cesium from 'cesium';
import cctvLayer, {
_setCctvCoverageStateForTest,
_setCctvOverlayHostForTest,
reopenNearestCamera,
} from './cctv.js';
const BASE = {
headingDeg: 41,
pitchDeg: -5,
fovDeg: 56,
rangeM: 210,
mountHeightM: 100,
groundElevationM: 0,
city: 'London',
};
function record(id, lat, lon) {
return {
camera: { ...BASE, id, name: `Camera ${id}`, lat, lon },
coverageEntities: [],
projection: null,
activationDone: true,
groundResolved: {},
groundSamples: {},
groundPrior: 0,
frustumGeometry: null,
frustumPositions: null,
probeClampRangeM: null,
position: Cesium.Cartesian3.fromDegrees(lon, lat, 100),
};
}
function viewerAt(lat, lon) {
return {
entities: new Cesium.EntityCollection(),
isDestroyed: () => false,
trackedEntity: null,
camera: {
positionWC: Cesium.Cartesian3.fromDegrees(lon, lat, 300),
positionCartographic: Cesium.Cartographic.fromDegrees(lon, lat, 300),
heading: 0.4,
pitch: -0.7,
roll: 0,
transform: Cesium.Matrix4.clone(Cesium.Matrix4.IDENTITY),
flyToBoundingSphere() {},
},
scene: {
canvas: { clientWidth: 1200, clientHeight: 800 },
cartesianToCanvasCoordinates: () => undefined,
primitives: { add: (p) => p, remove: () => true },
requestRender() {},
},
};
}
function moveViewer(viewer, lat, lon) {
viewer.camera.positionCartographic = Cesium.Cartographic.fromDegrees(
lon,
lat,
300,
);
}
function withReopenedLayer(fn) {
_setCctvOverlayHostForTest({
setEntries() {},
setVisible() {},
clearSource() {},
});
// Activation builds the projection canvas and frame image.
const originalDocument = globalThis.document;
const originalImage = globalThis.Image;
globalThis.document = { createElement: () => ({ getContext: () => null }) };
globalThis.Image = class {};
try {
const trafalgar = record('a-trafalgar', 51.508, -0.128);
const cambridge = record('b-cambridge', 51.513, -0.129);
// enable() assigned the opening camera without activating it.
trafalgar.activationDone = false;
const viewer = viewerAt(51.5081, -0.1281);
_setCctvCoverageStateForTest({
viewer,
records: [trafalgar, cambridge],
activeCameraId: 'a-trafalgar',
enabled: true,
openingCameraProvisional: true,
});
// A settled move next to Cambridge Circus re-opens there, still as the
// opening pick.
moveViewer(viewer, 51.5131, -0.1291);
assert.equal(reopenNearestCamera(), true);
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
fn(viewer);
} finally {
globalThis.document = originalDocument;
globalThis.Image = originalImage;
_setCctvOverlayHostForTest();
_setCctvCoverageStateForTest({ enabled: false });
}
}
for (const [name, pick] of [
['NEAREST (focusNearest)', () => cctvLayer.focusNearest({ focus: false })],
['selectCamera', () => cctvLayer.selectCamera('b-cambridge')],
[
'setParams({ selectedCameraId })',
() => cctvLayer.setParams({ selectedCameraId: 'b-cambridge' }),
],
]) {
test(`an explicit ${name} pick of the re-opened camera ends the opening pick`, () => {
withReopenedLayer((viewer) => {
pick();
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
// A later settled move (a pan, or the landing of the pick's own flight)
// leaves the explicit choice alone.
moveViewer(viewer, 51.5081, -0.1281);
assert.equal(reopenNearestCamera(), false);
assert.equal(cctvLayer.getUIState().activeCameraId, 'b-cambridge');
});
});
}
+25
View File
@@ -141,6 +141,13 @@ export const DATA_CREDITS = [
'<a href="https://www.planespotters.net" target="_blank" rel="noopener">Planespotters.net</a> ' +
'· each photo is © its photographer, credited beside it',
},
{
key: 'openvesselphotos',
html:
'Vessel photos: ' +
'<a href="https://openvesselphotos.org" target="_blank" rel="noopener">Open Vessel Photos</a> ' +
"· each photo keeps its author's open licence, credited beside it",
},
{
key: 'aisstream',
html:
@@ -503,6 +510,10 @@ export const DATA_CREDITS = [
key: 'gtfs-rt',
html: 'Transit vehicles: operator GTFS-Realtime feeds (each operator is credited below when its vehicles are shown)',
},
{
key: 'rail-gtfs-rt',
html: 'Trains: operator realtime feeds (each operator is credited below when its trains are shown)',
},
{
key: 'radio-browser',
html:
@@ -759,6 +770,20 @@ export function transitFeedCredit(feed, snapshot = null) {
};
}
/**
* Per-feed rail credit, registered the first time that feed's trains render
* (see `src/data/railFeeds.js` for the licence of each).
* @param {{ id: string, attribution: string, license: string, licenseUrl: string }} feed
* @returns {{ key: string, html: string }}
*/
export function railFeedCredit(feed) {
const transit = transitFeedCredit(feed);
return {
key: `rail-${feed.id}`,
html: transit.html.replace(/^Transit \(/, 'Trains ('),
};
}
/** Registered when the Bhote Koshi event reconstruction activates. */
export const BHOTE_KOSHI_CREDIT = {
key: 'bhote-koshi-2026',
+16
View File
@@ -51,6 +51,13 @@ test('Planespotters is credited with a plain followable link', () => {
assert.doesNotMatch(credit.html, /nofollow/);
});
test('Open Vessel Photos is credited with a plain followable link', () => {
const credit = DATA_CREDITS.find((entry) => entry.key === 'openvesselphotos');
assert.ok(credit, 'the selected vessel photos come from Open Vessel Photos');
assert.match(credit.html, /href="https:\/\/openvesselphotos\.org"/);
assert.doesNotMatch(credit.html, /nofollow/);
});
test('the UCS Satellite Database is acknowledged with its version and webpage', () => {
// UCS database manual, section 3: references name the version (the Excel
// file's name) and link www.ucsusa.org/satellite_database.
@@ -244,3 +251,12 @@ test('Warsaw and Poznań credits carry the data date their licences ask for', ()
assert.equal(mbta.key, 'transit-mbta');
assert.doesNotMatch(mbta.html, /data of/);
});
test('the always-on rail credit names live feeds only, never timetables', () => {
// Every train shown is where its operator reported it; none is placed
// from a timetable.
const rail = DATA_CREDITS.find((entry) => entry.key === 'rail-gtfs-rt');
assert.ok(rail);
assert.match(rail.html, /realtime feeds/);
assert.doesNotMatch(rail.html, /timetable|placed from/i);
});
+257
View File
@@ -0,0 +1,257 @@
/**
* @module irishRail
* @description Adapter for the Irish Rail realtime API's running-trains list
* (`getCurrentTrainsXML`), turning it into the train records the GTFS-RT path
* produces.
*
* The document is a flat list of `<objTrainPositions>` elements, each with
* seven text children (TrainStatus, TrainLatitude, TrainLongitude, TrainCode,
* TrainDate, PublicMessage, Direction). That schema is all this reads, so it
* is read with a small scanner instead of an XML library: no DTDs, no entity
* definitions, no attributes, no nesting beyond those two levels. A document
* that declares a DTD or entities is refused, and every size is capped (whole
* text, train count, field length).
*
* Positions are NOT GPS: the operator places each train at the station or
* timing point its signalling system last reported. The record says so with
* `positionSource: 'event'` (added by the rail proxy). The feed carries no
* time for that report, so a record's time is the fetch time, labelled
* `timestampSource: 'fetch'`, which the layer ages conservatively.
*
* Pure: no Node built-ins, safe for node:test and any server runtime.
*/
import { isPlausibleVehiclePosition } from './gtfsRealtime.js';
/** Longest document read (characters). A full day's list is about 35 KB. */
export const IRISH_RAIL_MAX_TEXT_CHARS = 1024 * 1024;
/** Most trains read from one document; about 80 run at once. */
export const IRISH_RAIL_MAX_TRAINS = 2000;
/** Longest accepted field (characters); the public message is the longest. */
export const IRISH_RAIL_MAX_FIELD_CHARS = 512;
const FIELDS = Object.freeze([
'TrainStatus',
'TrainLatitude',
'TrainLongitude',
'TrainCode',
'TrainDate',
'PublicMessage',
'Direction',
]);
/** Error for a document this adapter will not read. */
export class IrishRailFormatError extends Error {
constructor(message) {
super(message);
this.name = 'IrishRailFormatError';
}
}
const NAMED_ENTITIES = Object.freeze({
amp: '&',
lt: '<',
gt: '>',
quot: '"',
apos: "'",
});
/**
* Decode the five predefined XML entities and numeric character references.
* An unknown entity is left as written (no DTD can define one here).
* @param {string} text
* @returns {string}
*/
export function decodeXmlText(text) {
return String(text).replace(
/&(#x[0-9a-fA-F]{1,6}|#[0-9]{1,7}|[a-zA-Z]{2,4});/g,
(whole, body) => {
if (body[0] === '#') {
const code =
body[1] === 'x' || body[1] === 'X'
? Number.parseInt(body.slice(2), 16)
: Number.parseInt(body.slice(1), 10);
return Number.isInteger(code) && code > 0 && code <= 0x10ffff
? String.fromCodePoint(code)
: whole;
}
return Object.hasOwn(NAMED_ENTITIES, body) ? NAMED_ENTITIES[body] : whole;
},
);
}
/**
* The text of `<name>...</name>` inside one train element, `''` for an empty
* or self-closing element, null when absent or over the field cap.
*/
function readField(block, name) {
const open = `<${name}>`;
const start = block.indexOf(open);
if (start < 0) {
return block.includes(`<${name}/>`) || block.includes(`<${name} />`)
? ''
: null;
}
const from = start + open.length;
const end = block.indexOf(`</${name}>`, from);
if (end < 0 || end - from > IRISH_RAIL_MAX_FIELD_CHARS * 6) return null;
const value = decodeXmlText(block.slice(from, end)).trim();
return value.length <= IRISH_RAIL_MAX_FIELD_CHARS ? value : null;
}
/**
* Split the running-trains document into raw train rows.
* @param {string} text
* @returns {{rows: Array<Record<string, string|null>>, truncated: boolean}}
*/
export function parseIrishRailTrainsXml(text) {
if (typeof text !== 'string') throw new IrishRailFormatError('not text');
if (text.length > IRISH_RAIL_MAX_TEXT_CHARS)
throw new IrishRailFormatError(`document is ${text.length} characters`);
if (/<!DOCTYPE|<!ENTITY/i.test(text))
throw new IrishRailFormatError('document declares a DTD');
if (!text.includes('<ArrayOfObjTrainPositions'))
throw new IrishRailFormatError('not a running-trains document');
const rows = [];
let truncated = false;
const OPEN = '<objTrainPositions>';
const CLOSE = '</objTrainPositions>';
let at = text.indexOf(OPEN);
while (at >= 0) {
const end = text.indexOf(CLOSE, at + OPEN.length);
if (end < 0) break;
if (rows.length >= IRISH_RAIL_MAX_TRAINS) {
truncated = true;
break;
}
const block = text.slice(at + OPEN.length, end);
const row = {};
for (const name of FIELDS) row[name] = readField(block, name);
rows.push(row);
at = text.indexOf(OPEN, end + CLOSE.length);
}
return { rows, truncated };
}
/**
* Read the three lines of a public message. The operator separates lines
* with a literal backslash-n; a real newline is accepted too.
*
* A118\n14:50 - Dublin Connolly to Belfast (0 mins late)\nDeparted Dundalk next stop Newry
* E244\nHowth to Bray\nExpected Departure 16:35
* A311\n15:05 - Tralee to Mallow(-2 mins late)\nTERMINATED Mallow at 16:34
*
* @param {string|null} message
* @returns {{origin: string|null, destination: string|null,
* scheduledDeparture: string|null, delaySec: number|null,
* event: 'departed'|'arrived'|'expected'|'terminated'|null,
* lastStop: string|null, nextStop: string|null}}
*/
export function parsePublicMessage(message) {
const result = {
origin: null,
destination: null,
scheduledDeparture: null,
delaySec: null,
event: null,
lastStop: null,
nextStop: null,
};
const lines = String(message ?? '')
.split(/\\n|\r?\n/)
.map((line) => line.trim())
.filter(Boolean);
const route =
/^(?:(\d{1,2}:\d{2})\s*-\s*)?(.+?)\s+to\s+(.+?)(?:\s*\((-?\d{1,4})\s*mins?\s+late\))?$/i.exec(
lines[1] || '',
);
if (route) {
result.scheduledDeparture = route[1] || null;
result.origin = route[2].trim();
result.destination = route[3].trim();
if (route[4] !== undefined) result.delaySec = Number(route[4]) * 60;
}
const event = lines[2] || '';
let match;
if (
(match = /^(Departed|Arrived)\s+(.+?)\s+next stop\s+(.+)$/i.exec(event))
) {
result.event =
match[1].toLowerCase() === 'departed' ? 'departed' : 'arrived';
result.lastStop = match[2].trim();
result.nextStop = match[3].trim();
} else if (/^Expected Departure\b/i.test(event)) {
result.event = 'expected';
} else if (
(match = /^TERMINATED\s+(.+?)\s+at\s+\d{1,2}:\d{2}/i.exec(event))
) {
result.event = 'terminated';
result.lastStop = match[1].trim();
}
return result;
}
/**
* Turn the running-trains document into unclassified train records in the
* shape the GTFS-RT path produces (src/data/gtfsRealtime.js), plus the
* delay the public message states (`delaySec`). The message's last and next
* stop and the row's direction only decide `status`; nothing reads them
* further, so they do not leave the server.
*
* Status `T` (terminated) trains are dropped: they are out of service, parked
* at the end of their run. Status `N` trains are due to start within ten
* minutes and stand at their origin (`STOPPED_AT`). A running train is
* `STOPPED_AT` after "Arrived" and `IN_TRANSIT_TO` its next stop after
* "Departed".
*
* @param {string} text The XML document.
* @param {number} fetchedAtS Fetch time (epoch seconds), the only time the
* feed lets a record carry.
* @returns {{entityCount: number, truncated: boolean, vehicles: object[]}}
*/
export function decodeIrishRailTrains(text, fetchedAtS) {
const { rows, truncated } = parseIrishRailTrainsXml(text);
const seen = new Set();
const vehicles = [];
for (const row of rows) {
const code = row.TrainCode;
if (!code || !/^[A-Z0-9]{1,8}$/.test(code) || seen.has(code)) continue;
const status = row.TrainStatus;
if (status !== 'R' && status !== 'N') continue;
const lat = Number.parseFloat(row.TrainLatitude ?? '');
const lon = Number.parseFloat(row.TrainLongitude ?? '');
if (!isPlausibleVehiclePosition(lat, lon)) continue;
seen.add(code);
const message = parsePublicMessage(row.PublicMessage);
const routeId =
message.origin && message.destination
? `${message.origin} - ${message.destination}`
: null;
vehicles.push({
id: code,
lat: Number(lat.toFixed(6)),
lon: Number(lon.toFixed(6)),
bearing: null,
speedMps: null,
timestamp: fetchedAtS,
timestampSource: 'fetch',
routeId,
tripId: code,
directionId: null,
label: routeId
? `${code} ${message.origin} to ${message.destination}`
: code,
stopId: null,
status:
// i18n-ignore: GTFS-Realtime status codes, not UI text
status === 'N' || message.event === 'arrived'
? 'STOPPED_AT'
: message.event === 'departed'
? 'IN_TRANSIT_TO'
: null,
occupancy: null,
delaySec: message.delaySec,
});
}
return { entityCount: rows.length, truncated, vehicles };
}
+193
View File
@@ -0,0 +1,193 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
IRISH_RAIL_MAX_TEXT_CHARS,
IRISH_RAIL_MAX_TRAINS,
IrishRailFormatError,
decodeIrishRailTrains,
decodeXmlText,
parseIrishRailTrainsXml,
parsePublicMessage,
} from './irishRail.js';
/** One `<objTrainPositions>` element, written by hand for these tests. */
function train({
status = 'R',
lat = '53.3531',
lon = '-6.24591',
code = 'A118',
date = '06 Nov 2026',
message = 'A118\\n14:50 - Dublin Connolly to Belfast (3 mins late)\\nDeparted Dundalk next stop Newry',
direction = 'Northbound',
} = {}) {
return ` <objTrainPositions>
<TrainStatus>${status}</TrainStatus>
<TrainLatitude>${lat}</TrainLatitude>
<TrainLongitude>${lon}</TrainLongitude>
<TrainCode>${code}</TrainCode>
<TrainDate>${date}</TrainDate>
<PublicMessage>${message}</PublicMessage>
<Direction>${direction}</Direction>
</objTrainPositions>`;
}
function document(...trains) {
return `<?xml version="1.0" encoding="utf-8"?>
<ArrayOfObjTrainPositions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://api.irishrail.ie/realtime/">
${trains.join('\n')}
</ArrayOfObjTrainPositions>`;
}
const FETCHED_S = 1_794_000_000;
test('a running-trains document becomes GTFS-shaped train records', () => {
const { vehicles, entityCount, truncated } = decodeIrishRailTrains(
document(
train(),
train({
code: 'E244',
status: 'N',
message: 'E244\\nHowth to Bray\\nExpected Departure 16:35',
direction: 'Southbound',
}),
train({
code: 'D825',
message:
'D825\\n16:40 - Dun Laoghaire to Dundalk (0 mins late)\\nArrived Dun Laoghaire next stop Blackrock',
}),
),
FETCHED_S,
);
assert.equal(entityCount, 3);
assert.equal(truncated, false);
assert.equal(vehicles.length, 3);
const [enterprise, dart, commuter] = vehicles;
assert.deepEqual(enterprise, {
id: 'A118',
lat: 53.3531,
lon: -6.24591,
bearing: null,
speedMps: null,
timestamp: FETCHED_S,
timestampSource: 'fetch',
routeId: 'Dublin Connolly - Belfast',
tripId: 'A118',
directionId: null,
label: 'A118 Dublin Connolly to Belfast',
stopId: null,
status: 'IN_TRANSIT_TO',
occupancy: null,
delaySec: 180,
});
// The message's stops and the row's direction decide the status only.
for (const field of ['serviceState', 'lastStop', 'nextStop', 'direction'])
assert.equal(field in dart, false, field);
assert.equal(dart.status, 'STOPPED_AT');
assert.equal(dart.routeId, 'Howth - Bray');
assert.equal(dart.delaySec, null);
assert.equal(commuter.status, 'STOPPED_AT');
assert.equal(commuter.delaySec, 0);
});
test('terminated, unplaced, malformed and repeated trains are dropped', () => {
const { vehicles, entityCount } = decodeIrishRailTrains(
document(
train({
code: 'A311',
status: 'T',
message:
'A311\\n15:05 - Tralee to Mallow(-2 mins late)\\nTERMINATED Mallow at 16:34',
}),
train({ code: 'A1', lat: '0', lon: '0' }),
train({ code: 'A2', lat: 'north', lon: '-6.2' }),
train({ code: 'A3', status: 'X' }),
train({ code: 'bad code!' }),
train({ code: 'A4' }),
train({ code: 'A4', lat: '53.0' }),
),
FETCHED_S,
);
assert.equal(entityCount, 7);
assert.deepEqual(
vehicles.map((vehicle) => [vehicle.id, vehicle.lat]),
[['A4', 53.3531]],
);
});
test('public messages: the three line shapes, with and without spacing', () => {
assert.deepEqual(
parsePublicMessage(
'A311\\n15:05 - Tralee to Mallow(-2 mins late)\\nTERMINATED Mallow at 16:34',
),
{
origin: 'Tralee',
destination: 'Mallow',
scheduledDeparture: '15:05',
delaySec: -120,
event: 'terminated',
lastStop: 'Mallow',
nextStop: null,
},
);
const real = parsePublicMessage(
'D319\n16:27 - Docklands to M3 Parkway (1 mins late)\nArrived Pelletstown next stop Ashtown',
);
assert.equal(real.destination, 'M3 Parkway');
assert.equal(real.event, 'arrived');
assert.equal(real.nextStop, 'Ashtown');
assert.equal(parsePublicMessage('').origin, null);
assert.equal(parsePublicMessage(null).event, null);
assert.equal(parsePublicMessage('X1\\nno route here\\n?').origin, null);
});
test('entities are decoded and DTDs refused', () => {
assert.equal(
decodeXmlText('Tom &amp; Jerry &lt;&gt; &quot;&apos; &#65;&#x42; &bogus;'),
'Tom & Jerry <> "\' AB &bogus;',
);
assert.equal(decodeXmlText('&#0; &#x110000;'), '&#0; &#x110000;');
const { rows } = parseIrishRailTrainsXml(
document(train({ direction: 'To Dun Laoghaire &amp; Bray' })),
);
assert.equal(rows[0].Direction, 'To Dun Laoghaire & Bray');
assert.throws(
() =>
parseIrishRailTrainsXml(
`<!DOCTYPE x [<!ENTITY a "aaaa">]>${document(train())}`,
),
IrishRailFormatError,
);
assert.throws(
() => parseIrishRailTrainsXml('<html>error</html>'),
IrishRailFormatError,
);
assert.throws(() => parseIrishRailTrainsXml(42), IrishRailFormatError);
});
test('sizes are capped: document, train count and field length', () => {
assert.throws(
() =>
parseIrishRailTrainsXml(
document(train()) + ' '.repeat(IRISH_RAIL_MAX_TEXT_CHARS),
),
/characters/,
);
const many = Array.from({ length: IRISH_RAIL_MAX_TRAINS + 3 }, (_, i) =>
train({ code: `E${i}` }),
);
const capped = parseIrishRailTrainsXml(document(...many));
assert.equal(capped.rows.length, IRISH_RAIL_MAX_TRAINS);
assert.equal(capped.truncated, true);
const { rows } = parseIrishRailTrainsXml(
document(train({ message: 'm'.repeat(600) }), '<objTrainPositions>'),
);
assert.equal(rows.length, 1, 'an unclosed element is not a train');
assert.equal(rows[0].PublicMessage, null, 'an over-long field is refused');
const empty = parseIrishRailTrainsXml(
document(
'<objTrainPositions><TrainCode>E1</TrainCode><Direction /></objTrainPositions>',
),
);
assert.equal(empty.rows[0].Direction, '');
assert.equal(empty.rows[0].TrainStatus, null);
});
+13
View File
@@ -313,6 +313,13 @@ const OPTION_GROUPS = Object.freeze({
high: 'h',
}),
]),
rail: Object.freeze([
booleanOption('transit', 't', true),
booleanOption('local', 'l', true),
booleanOption('regional', 'r', true),
booleanOption('national', 'n', true),
booleanOption('international', 'i', true),
]),
'wildlife-sightings': Object.freeze([
booleanOption('confirmed', 'c', false),
enumOption(
@@ -700,6 +707,12 @@ export const LAYER_STATE_REGISTRY = Object.freeze([
disposition: 'enabled+options',
optionOwner: 'radio',
}),
Object.freeze({
id: 'rail',
token: '02',
disposition: 'enabled+options',
optionOwner: 'rail',
}),
Object.freeze({
id: 'recent-imagery',
token: '1',
+7 -5
View File
@@ -203,8 +203,8 @@ function encode(state) {
test('production registry is exact, canonical, and rejects incomplete contracts', async () => {
assert.equal(validateLayerStateRegistry(), true);
assert.equal(REGISTERED_LAYER_IDS.length, 38);
assert.equal(new Set(REGISTERED_LAYER_IDS).size, 38);
assert.equal(REGISTERED_LAYER_IDS.length, 39);
assert.equal(new Set(REGISTERED_LAYER_IDS).size, 39);
assert.ok(REGISTERED_LAYER_IDS.includes('transit'));
assert.deepEqual(REGISTERED_LAYER_IDS, [...REGISTERED_LAYER_IDS].sort());
assert.deepEqual(LEGACY_LAYER_STATE_TOKENS, {
@@ -243,12 +243,14 @@ test('production registry is exact, canonical, and rejects incomplete contracts'
for (const [id, token] of Object.entries(LEGACY_LAYER_STATE_TOKENS)) {
assert.equal(LAYER_STATE_TOKEN_RESERVATIONS[id], token);
}
// News holds 00 and Shipwrecks 01, the first pairs: the next token is 02.
assert.equal(nextLayerStateToken(), '02');
// The allocation rules below read the ledger before either took a pair.
// News holds 00, Shipwrecks 01 and Rail 02, the first pairs: the next token
// is 03.
assert.equal(nextLayerStateToken(), '03');
// The allocation rules below read the ledger before any of them took a pair.
const beforeNews = { ...LAYER_STATE_TOKEN_RESERVATIONS };
delete beforeNews.news;
delete beforeNews.shipwrecks;
delete beforeNews.rail;
assert.equal(nextLayerStateToken(beforeNews), '00');
assert.equal(
nextLayerStateToken({
+19 -11
View File
@@ -97,10 +97,11 @@ function withPublishedBase(
}
}
// The ledger before Wildlife Sightings took the last digit and News and
// Shipwrecks the first pairs, for the allocation rules written against it.
// The ledger before Wildlife Sightings took the last digit and News,
// Shipwrecks and Rail the first pairs, for the allocation rules written
// against it.
const beforeLastDigitRows = reservationRows.filter(
([id]) => !['wildlife-sightings', 'news', 'shipwrecks'].includes(id),
([id]) => !['wildlife-sightings', 'news', 'shipwrecks', 'rail'].includes(id),
);
function writeCodecFixture(cwd, rows, entries) {
@@ -112,6 +113,7 @@ function writeCodecFixture(cwd, rows, entries) {
readFileSync(new URL(`./${name}`, import.meta.url)),
);
let source = readFileSync(CODEC_SOURCE_PATH, 'utf8');
// Layers published after the fixture's ledger are not in its registry.
if (!rows.some(([id]) => id === 'wildlife-sightings'))
source = source.replace(
/ Object\.freeze\(\{\s*id: 'wildlife-sightings',[\s\S]*?\}\),\n/,
@@ -127,6 +129,11 @@ function writeCodecFixture(cwd, rows, entries) {
/ Object\.freeze\(\{\s*id: 'shipwrecks',[\s\S]*?\}\),\n/,
'',
);
if (!rows.some(([id]) => id === 'rail'))
source = source.replace(
/ Object\.freeze\(\{\s*id: 'rail',[\s\S]*?\}\),\n/,
'',
);
const registryEnd = ']);\n\nexport const REGISTERED_LAYER_IDS';
assert.equal(source.split(registryEnd).length, 2);
const additions = entries
@@ -150,7 +157,7 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed
{ ...parseLayerStateTokenReservations(reservationRows) },
{ ...LAYER_STATE_TOKEN_RESERVATIONS },
);
assert.equal(Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length, 38);
assert.equal(Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length, 39);
assert.deepEqual(
{ ...LAYER_STATE_TOKEN_RESERVATIONS },
{
@@ -165,6 +172,7 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed
'wildlife-sightings': '9',
news: '00',
shipwrecks: '01',
rail: '02',
},
);
assert.throws(
@@ -190,9 +198,9 @@ test('reservation ledger is complete, pinned, and rejects duplicate or malformed
/Duplicate layer-state token reservation/,
);
assert.equal(
parseLayerStateTokenReservations([...reservationRows, ['future', '02']])
parseLayerStateTokenReservations([...reservationRows, ['future', '03']])
.future,
'02',
'03',
);
assert.throws(
() =>
@@ -423,19 +431,19 @@ test('an isolated valid two-character fixture round-trips an l field beyond the
test('checker reads a complete future base ledger and rejects retired-token reuse', () => {
withPublishedBase(
{
rows: [...reservationRows, ['retired-layer', '02']],
rows: [...reservationRows, ['retired-layer', '03']],
source:
'export const LAYER_STATE_TOKEN_RESERVATIONS = Object.freeze({ ["retired-layer"]: "02" });\n',
'export const LAYER_STATE_TOKEN_RESERVATIONS = Object.freeze({ ["retired-layer"]: "03" });\n',
},
(cwd) => {
const published = readPublishedLayerStateReservations('HEAD', cwd);
assert.equal(Object.keys(published).length, 39);
assert.equal(published['retired-layer'], '02');
assert.equal(Object.keys(published).length, 40);
assert.equal(published['retired-layer'], '03');
assert.throws(
() =>
validateLayerStateAllocations(published, {
...LAYER_STATE_TOKEN_RESERVATIONS,
newcomer: '02',
newcomer: '03',
}),
/changed or removed/,
);
@@ -20,6 +20,7 @@
["military-installations", "i"],
["news", "00"],
["radio", "r"],
["rail", "02"],
["recent-imagery", "1"],
["rocket-launches", "x"],
["s2-ghostmaps", "5"],
+22
View File
@@ -0,0 +1,22 @@
/**
* The Data Library's layer namespace, owned by the layer lifecycle: the
* manager accepts a late registration only under `lib-<entry id>`, and the
* library names its layers with the same rule.
*/
/** A library entry id: lowercase, digits and hyphens, 3–48 characters. */
export const LIBRARY_ID_PATTERN = /^[a-z0-9][a-z0-9-]{2,47}$/;
export const LIBRARY_LAYER_PREFIX = 'lib-';
/** The manager id a library entry's layer registers under. */
export function libraryLayerId(entryId) {
return `${LIBRARY_LAYER_PREFIX}${entryId}`;
}
/** The entry id behind a library layer id, or null. */
export function libraryEntryIdOf(layerId) {
if (typeof layerId !== 'string' || !layerId.startsWith(LIBRARY_LAYER_PREFIX))
return null;
const id = layerId.slice(LIBRARY_LAYER_PREFIX.length);
return LIBRARY_ID_PATTERN.test(id) ? id : null;
}
+54
View File
@@ -0,0 +1,54 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { LayerLifecycle } from './lifecycle.js';
const layer = (id) => ({
id,
name: id,
icon: '',
source: 'test',
async init() {},
async enable() {},
async update() {},
async disable() {},
async destroy() {},
getStats: () => ({ count: 0, lastUpdate: null }),
});
test('library layers register only after sealing and only in the lib- namespace', async () => {
const manager = new LayerLifecycle({});
const activity = [];
manager.subscribeActivity((change) => activity.push(change.type));
manager.register(layer('earthquakes'));
assert.throws(
() => manager.registerLibraryLayer(layer('lib-night-lights')),
/after the catalog is sealed/,
);
manager.finalizeRegistrations([
{ id: 'earthquakes', disposition: 'enabled-only' },
]);
assert.throws(
() => manager.registerLibraryLayer(layer('radio')),
/lib- namespace/,
);
assert.throws(() => manager.register(layer('lib-x-y-z')), /finalized/);
manager.registerLibraryLayer(layer('lib-night-lights'));
assert.equal(manager.isLibraryLayer('lib-night-lights'), true);
assert.equal(manager.isLibraryLayer('earthquakes'), false);
const rows = manager.getAll();
assert.equal(
rows.find(({ id }) => id === 'earthquakes').libraryEntryId,
null,
);
assert.equal(
rows.find(({ id }) => id === 'lib-night-lights').libraryEntryId,
'night-lights',
);
assert.equal(await manager.unregisterLibraryLayer('earthquakes'), false);
assert.equal(manager.layers.has('earthquakes'), true);
await manager.setEnabled('lib-night-lights', true, { origin: 'user' });
assert.equal(await manager.unregisterLibraryLayer('lib-night-lights'), true);
assert.equal(manager.layers.has('lib-night-lights'), false);
assert.equal(manager.isLibraryLayer('lib-night-lights'), false);
assert.equal(activity.filter((type) => type === 'registrations').length, 2);
});
+42
View File
@@ -1,4 +1,5 @@
import { cancelCameraArrival } from './cameraArrival.js';
import { libraryEntryIdOf } from './libraryLayerIds.js';
function cloneLayerParams(value) {
if (Array.isArray(value)) return value.map(cloneLayerParams);
@@ -97,6 +98,7 @@ export class LayerLifecycle {
this._registrationDispositions = null;
this._allowQaRegistration = allowQaRegistration === true;
this._qaLayerIds = new Set();
this._libraryLayerIds = new Set();
}
register(layerModule) {
@@ -125,6 +127,41 @@ export class LayerLifecycle {
return destroyed;
}
/**
* Register a Data Library layer after the catalog is sealed. Only ids in
* the library namespace (`lib-<entry id>`) are accepted, so this seam can
* never stand in for a core layer, and library layers never join the core
* share-token registry: their selection travels in `lib=` instead.
* @param {object} layerModule
* @returns {string} The registered id.
*/
registerLibraryLayer(layerModule) {
if (!this._registrationsFinalized)
throw new Error('Library layers register after the catalog is sealed');
if (!libraryEntryIdOf(layerModule?.id))
throw new Error('Library layer ids must use the lib- namespace');
this._registerLayer(layerModule);
this._libraryLayerIds.add(layerModule.id);
this._publishActivity({ type: 'registrations', layerId: layerModule.id });
return layerModule.id;
}
/** Destroy a layer registered through `registerLibraryLayer`. */
async unregisterLibraryLayer(layerId) {
if (!this._libraryLayerIds.has(layerId)) return false;
const destroyed = await this.destroyLayer(layerId);
if (destroyed) {
this._libraryLayerIds.delete(layerId);
this._publishActivity({ type: 'registrations', layerId });
}
return destroyed;
}
/** Whether a registered layer came from the Data Library. */
isLibraryLayer(layerId) {
return this._libraryLayerIds.has(layerId);
}
_registerLayer(layerModule) {
if (!layerModule || typeof layerModule.id !== 'string' || !layerModule.id) {
throw new Error('Data layer must provide a stable id');
@@ -2231,6 +2268,7 @@ export class LayerLifecycle {
}
this.layers.delete(layerId);
this._qaLayerIds.delete(layerId);
this._libraryLayerIds.delete(layerId);
return true;
}
@@ -2274,6 +2312,10 @@ export class LayerLifecycle {
// showInTogglePanel). The layer reports stats.keyRequired while that
// key is absent; the panel builds the guidance text from the pair.
requiresKeyId: entry.module.requiresKeyId || null,
// Data Library entry id; the panel groups these under Library.
libraryEntryId: this._libraryLayerIds.has(id)
? libraryEntryIdOf(id)
: null,
enabled: entry.enabled,
lifecycleState: entry.lifecycleState,
lifecycleUncertain: entry.lifecycleUncertain,
+172 -8
View File
@@ -86,6 +86,13 @@ export function localInfrastructureOverlayCopy(properties, layerId) {
if (river && river.toLocaleLowerCase() !== title.toLocaleLowerCase()) {
details.push(clampCardLine(river));
}
} else if (Array.isArray(props.__gevDetails)) {
// Data Library features carry their card lines from the manifest
// (src/library/formats.js presentLibraryProperties).
for (const line of props.__gevDetails.slice(0, 2)) {
const clean = cleanLabel(line);
if (clean) details.push(clampCardLine(clean));
}
}
return { title, details };
@@ -292,7 +299,10 @@ export function localDatasetError(error) {
* A minimal, rock-solid native implementation for loading local GeoJSON Data.
* Draws 3D stems (polylines) attached to Point entities and ensures
* standard scene.pick natively clicks them.
* @param {object} options Dataset URL, identity, appearance and optional Cesium adapters.
* @param {object} options Dataset URL (or a `loadFeatures({signal})` loader
* resolving to GeoJSON features), identity, appearance (an optional
* `featureColor(properties)` returns a CSS colour per feature) and optional
* Cesium adapters.
* @param {object} services Caller-owned operations; see docs/INFRASTRUCTURE-LAYERS.md.
* @returns {object} A fresh layer implementing init/enable/disable/update/destroy/getStats/getAnalystRecords.
* One live instance per layer id is allowed in a given viewer/context/overlay host.
@@ -310,6 +320,8 @@ export function createLocalGeoJsonLayer(
labels = true,
labelMax = DEFAULT_LABEL_MAX,
labelGridPx = DEFAULT_LABEL_GRID_PX,
loadFeatures = null,
featureColor = null,
screenSpaceEventHandlerFactory = (canvas) =>
new Cesium.ScreenSpaceEventHandler(canvas),
projectToWindow = (scene, position) =>
@@ -353,6 +365,12 @@ export function createLocalGeoJsonLayer(
* @type {Array<object>|null}
*/
let _cachedFeatures = null;
/**
* The source a `replaceFeatures()` rebuild is replacing: it stays drawn
* until the new one is in the scene, and comes back if the rebuild fails.
* @type {{source: object, stemRecords: object[], count: number, lastUpdate: number|null, error: string|null}|null}
*/
let _outgoing = null;
/**
* Globe-LOD active set: the record ids allowed to carry a live stem right
* now. This bounds geometry refreshes and ground-sample work to the
@@ -437,7 +455,19 @@ export function createLocalGeoJsonLayer(
* survive a disable is the per-frame visualizer walk and the entity memory.
* @param {object} viewer
*/
const releaseOutgoing = (viewer) => {
if (!_outgoing) return;
const { source } = _outgoing;
_outgoing = null;
try {
viewer?.dataSources?.remove(source, true);
} catch {
/* already gone */
}
};
const releaseDataSource = (viewer) => {
releaseOutgoing(viewer);
if (!_dataSource) return;
const source = _dataSource;
_dataSource = null;
@@ -477,7 +507,7 @@ export function createLocalGeoJsonLayer(
}
};
return {
const layer = {
id,
name,
icon,
@@ -585,7 +615,17 @@ export function createLocalGeoJsonLayer(
let addedToScene = false;
try {
let features = _cachedFeatures;
if (!features) {
if (!features && loadFeatures) {
// A caller-owned loader (the Data Library) fetches and bounds
// its own live dataset; this layer only renders it.
features = await loadFeatures({
signal: _loadController.signal,
});
if (_destroyed) return;
if (!Array.isArray(features))
throw new Error('dataset unavailable');
_cachedFeatures = features;
} else if (!features) {
const response = await fetch(url, {
signal: _loadController.signal,
});
@@ -639,9 +679,21 @@ export function createLocalGeoJsonLayer(
_stemRecords = [];
_stemGeometryDirty = true;
// An optional per-feature colour (Data Library colorBy); one
// Cesium colour per distinct CSS value.
const colorCache = new Map([[color, baseColor]]);
const colorFor = (entity) => {
if (typeof featureColor !== 'function') return baseColor;
const css = featureColor(propertyObject(entity)) || color;
if (!colorCache.has(css))
colorCache.set(css, Cesium.Color.fromCssColorString(css));
return colorCache.get(css);
};
for (let i = 0; i < entities.length; i++) {
const feature = entities[i];
feature.__localLayerId = id; // Tag it so our click handler knows it belongs to this layer
const featureBaseColor = colorFor(feature);
let pos = feature.position?.getValue(Cesium.JulianDate.now());
@@ -649,7 +701,10 @@ export function createLocalGeoJsonLayer(
// It's a polygon or line
if (feature.polygon) {
feature.polygon.outline = true;
feature.polygon.outlineColor = baseColor;
feature.polygon.outlineColor = featureBaseColor;
if (featureBaseColor !== baseColor)
feature.polygon.material =
featureBaseColor.withAlpha(0.3);
// Calculate center point for the stem
const hierarchy = feature.polygon.hierarchy?.getValue(
@@ -689,7 +744,7 @@ export function createLocalGeoJsonLayer(
// Store references for bounded stem scaling and native picking.
feature.__localBaseCarto = carto;
feature.__localBaseCartesian = base;
registerEntityContext(feature, {
const context = registerEntityContext(feature, {
id: `${id}:${recordId}`,
layerId: id,
layerName: name,
@@ -716,11 +771,11 @@ export function createLocalGeoJsonLayer(
feature.polyline = new Cesium.PolylineGraphics({
positions: stemPositionBuffers[0],
width: 3.5,
material: new Cesium.ColorMaterialProperty(baseColor),
material: new Cesium.ColorMaterialProperty(featureBaseColor),
});
feature.point = new Cesium.PointGraphics({
pixelSize: 10,
color: baseColor,
color: featureBaseColor,
outlineColor: Cesium.Color.BLACK,
outlineWidth: 2,
// Never depth-cull the anchor against the photoreal mesh —
@@ -732,6 +787,7 @@ export function createLocalGeoJsonLayer(
_stemRecords.push({
id: recordId,
entity: feature,
context,
carto,
base,
tip,
@@ -749,7 +805,7 @@ export function createLocalGeoJsonLayer(
position: tip,
properties,
priority,
accent: color,
accent: featureBaseColor.toCssHexString(),
})
: null,
});
@@ -1065,6 +1121,108 @@ export function createLocalGeoJsonLayer(
disable: disableLayer,
/**
* Forget the parsed features so the next enable() loads them again: a
* caller-owned live loader (the Data Library) uses it once its data is
* older than the source allows.
*/
dropCachedFeatures: () => {
_cachedFeatures = null;
},
/**
* Draw a new feature set in place of the current one while the layer is
* on (a live loader's refresh). The current features stay drawn until
* the new ones are in the scene; a selected feature stays selected when
* the new set still has it (same id, or same place and label); a failed
* rebuild keeps the current features and rejects. Off, the set is only
* kept for the next enable().
* @param {object} viewer
* @param {object[]} features GeoJSON features.
*/
replaceFeatures: async (viewer, features) => {
if (_destroyed) return;
const previousFeatures = _cachedFeatures;
_cachedFeatures = features;
if (!_enabled || !_dataSource || _loadPromise) return;
const selected =
viewer?.selectedEntity?.__localLayerId === id
? viewer.selectedEntity
: null;
const selectedLabel = selected
? _stemRecords.find((record) => record.entity === selected)?.context
?.label
: undefined;
_outgoing = {
source: _dataSource,
stemRecords: _stemRecords,
count: _count,
lastUpdate: _lastUpdate,
error: _error,
};
const outgoing = _outgoing;
_dataSource = null;
await coreEnable(viewer);
// Disabled or destroyed meanwhile: release took both sources.
if (_destroyed || !_enabled || _outgoing !== outgoing) return;
_outgoing = null;
if (!_dataSource) {
// The rebuild failed: the current features come back as they were.
const failure = _error;
_cachedFeatures = previousFeatures;
_dataSource = outgoing.source;
_stemRecords = outgoing.stemRecords;
_count = outgoing.count;
_lastUpdate = outgoing.lastUpdate;
_error = outgoing.error;
_stemGeometryDirty = true;
_lastVisibilityUpdate = Number.NEGATIVE_INFINITY;
for (const record of _stemRecords)
if (record.context)
registerEntityContext(record.entity, record.context);
if (selected && viewer.selectedEntity === selected)
selectEntityContext(selected);
throw new Error(failure || 'dataset unavailable');
}
try {
viewer?.dataSources?.remove(outgoing.source, true);
} catch {
/* already gone */
}
// Records the new set no longer has go with the old source.
removeEntityContextsForLayer(id, {
retainIds: new Set(
_stemRecords.map((record) => record.context?.id).filter(Boolean),
),
});
if (selected) {
const same = (record) => {
const old = selected.__localBaseCarto;
const next = record.entity.__localBaseCarto;
return (
old &&
next &&
Math.abs(old.longitude - next.longitude) < 1e-9 &&
Math.abs(old.latitude - next.latitude) < 1e-9 &&
record.context?.label === selectedLabel
);
};
const match = _stemRecords.find((record) =>
selected.id !== undefined && record.entity.id === selected.id
? true
: same(record),
);
if (match) {
viewer.selectedEntity = match.entity;
selectEntityContext(match.entity);
} else if (viewer.selectedEntity === selected) {
viewer.selectedEntity = undefined;
clearSelectedEntityContextForLayer(id);
}
}
viewer?.scene?.requestRender?.();
},
destroy: (viewer) => {
if (_destroyed) return;
_destroyed = true;
@@ -1089,6 +1247,8 @@ export function createLocalGeoJsonLayer(
_error = null;
},
};
const coreEnable = layer.enable;
return layer;
}
function compareLocalOverlayRecords(a, b) {
@@ -1323,6 +1483,10 @@ function labelPriorityFromProperties(props, layerId) {
if (props.output || tags['plant:output:electricity']) score += 120;
if (layerId === 'local-dams') score += 80;
if (layerId === 'local-datacenters') score += 60;
// Data Library features carry their manifest importance (0–600), below the
// named/unnamed gap so a rank never promotes an unnamed feature.
if (Number.isFinite(props.__gevRank))
score += Math.max(0, Math.min(600, props.__gevRank));
return score;
}
+3 -2
View File
@@ -3,9 +3,10 @@
Offline polygons with no network dependency. The voice-annotation resolver
(`src/data/naturalEarthRegions.js`) reads the named regions, so "outline the
Alps" resolves to the real range geometry; area annotations read the
countries, states and provinces (`src/data/adminBoundaries.js`); and the News
countries, states and provinces (`src/data/adminBoundaries.js`); the News
layer reads the countries and the disputed areas
(`server/providers/news/areas.js`).
(`server/providers/news/areas.js`); and Data Library imagery draws its
coverage outlines from the countries (`src/library/outlines.js`).
| File | Source dataset | Features |
|------|----------------|----------|
+135
View File
@@ -0,0 +1,135 @@
/**
* @module openVesselPhoto
* @description Reads an Open Vessel Photos answer into the one photo every
* surface shows. Open Vessel Photos (https://openvesselphotos.org) serves a
* free, keyless JSON and image API by MMSI. Every photo keeps the open licence
* its author chose (CC BY, CC BY-SA, CC0 or public domain), so it is shown
* with its credit and licence and links back to the vessel's page. A photo
* whose image is not on Open Vessel Photos' own host, or whose answer names
* no licence, is "no photo": nothing arbitrary is ever shown or linked.
*
* Accepts the upstream answer (`GET https://openvesselphotos.org/v1/mmsi/{mmsi}`,
* `{mmsi, photos: [...]}`) and an already-read photo
* (`{found, src, width, height, photographer, licence, link}`).
* Pure: no fetch, no DOM.
*/
/** The only host a photo may come from or link to. */
const HOST = 'openvesselphotos.org';
/** The API's per-vessel answer. */
const API = `https://${HOST}/v1/mmsi/`;
/** The image width asked for: sharp in the panel on a 2× screen, ~60 KB. */
const IMAGE_SIZE = '800';
const NO_PHOTO = Object.freeze({ found: false });
/** A vessel's MMSI: nine digits. */
export const MMSI = /^\d{9}$/;
/** The API answer for one MMSI. */
export function openVesselPhotosUrl(mmsi) {
return API + mmsi;
}
/** The vessel's page on Open Vessel Photos. */
function vesselPage(mmsi) {
return `https://${HOST}/vessels/mmsi/${mmsi}`;
}
/** The parsed `https:` URL on the Open Vessel Photos host, or null. */
function ownUrl(value) {
if (typeof value !== 'string' || !value) return null;
let url;
try {
url = new URL(value);
} catch {
return null;
}
return url.protocol === 'https:' &&
!url.username &&
!url.password &&
!url.port &&
url.hostname === HOST
? url
: null;
}
function dimension(value) {
const number = Number(value);
return value != null && Number.isFinite(number) && number > 0
? Math.round(number)
: null;
}
function text(value) {
return typeof value === 'string' ? value.trim() : '';
}
/** One upstream photo of the vessel `mmsi`, checked. */
function checkedUpstream(photo, mmsi) {
if (!photo || typeof photo !== 'object') return null;
const image = ownUrl(photo.urls?.mmsi);
const licence = text(photo.licenceName) || text(photo.licence);
if (!image || !licence) return null;
image.search = '';
image.hash = '';
image.searchParams.set('size', IMAGE_SIZE);
// The answer gives the full image's size; the 800 px one keeps its shape.
const width = dimension(photo.width);
const height = dimension(photo.height);
const scale =
width && width > Number(IMAGE_SIZE) ? Number(IMAGE_SIZE) / width : 1;
return {
found: true,
src: image.href,
width: width ? Math.round(width * scale) : null,
height: height ? Math.round(height * scale) : null,
photographer: text(photo.credit),
licence,
link: vesselPage(mmsi),
};
}
/** An already-read photo, checked again. */
function checkedRead(photo) {
const image = ownUrl(photo.src);
const page = ownUrl(photo.link);
const licence = text(photo.licence);
if (!image || !page || !licence) return null;
return {
found: true,
src: image.href,
width: dimension(photo.width),
height: dimension(photo.height),
photographer: text(photo.photographer),
licence,
link: page.href,
};
}
/**
* Read an Open Vessel Photos answer into the first photo that passes the
* checks.
* @param {*} payload Upstream answer or already-read photo.
* @param {?string} [requestedMmsi] The MMSI asked for; an upstream answer
* naming another is no photo.
* @returns {{found: true, src: string, width: ?number, height: ?number,
* photographer: string, licence: string, link: string} | {found: false}}
*/
export function parseOpenVesselPhoto(payload, requestedMmsi = null) {
if (!payload || typeof payload !== 'object') return NO_PHOTO;
if (Array.isArray(payload.photos)) {
const mmsi = String(payload.mmsi ?? '');
if (!MMSI.test(mmsi)) return NO_PHOTO;
// An answer about another vessel is no photo of this one.
if (requestedMmsi != null && mmsi !== String(requestedMmsi))
return NO_PHOTO;
for (const photo of payload.photos) {
const checked = checkedUpstream(photo, mmsi);
if (checked) return checked;
}
return NO_PHOTO;
}
if (payload.found !== true) return NO_PHOTO;
return checkedRead(payload) || NO_PHOTO;
}
+120
View File
@@ -0,0 +1,120 @@
// src/data/openVesselPhoto.test.mjs — an Open Vessel Photos answer is read
// into one credited, licensed photo, and nothing outside Open Vessel Photos'
// own host is ever shown or linked.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
openVesselPhotosUrl,
parseOpenVesselPhoto,
} from './openVesselPhoto.js';
const UPSTREAM = Object.freeze({
mmsi: '366999709',
name: '01 USCGC Sockeye WPB87337',
photos: [
{
n: 1,
urls: {
cfr: null,
mmsi: 'https://openvesselphotos.org/v1/mmsi/366999709/photos/1.jpg',
},
credit: 'Gordon Leggett',
licence: 'CC-BY-4.0',
licenceName: 'CC BY 4.0',
width: 1600,
height: 1200,
},
],
});
const PHOTO = Object.freeze({
found: true,
src: 'https://openvesselphotos.org/v1/mmsi/366999709/photos/1.jpg?size=800',
width: 800,
height: 600,
photographer: 'Gordon Leggett',
licence: 'CC BY 4.0',
link: 'https://openvesselphotos.org/vessels/mmsi/366999709',
});
const withPhoto = (overrides) => ({
...UPSTREAM,
photos: [{ ...UPSTREAM.photos[0], ...overrides }],
});
test('the upstream answer maps to the 800 px image, its credit, licence and page', () => {
assert.deepEqual(parseOpenVesselPhoto(UPSTREAM), PHOTO);
assert.equal(
openVesselPhotosUrl('366999709'),
'https://openvesselphotos.org/v1/mmsi/366999709',
);
});
test('an image narrower than 800 px keeps its own size', () => {
const photo = parseOpenVesselPhoto(withPhoto({ width: 640, height: 480 }));
assert.equal(photo.width, 640);
assert.equal(photo.height, 480);
});
test('an already-read photo passes through its checks again', () => {
assert.deepEqual(parseOpenVesselPhoto(PHOTO), PHOTO);
assert.deepEqual(parseOpenVesselPhoto({ ...PHOTO, licence: '' }), {
found: false,
});
});
test('a photo off the Open Vessel Photos host, or without a licence, is no photo', () => {
for (const overrides of [
{ urls: { mmsi: 'https://example.com/1.jpg' } },
{
urls: {
mmsi: 'http://openvesselphotos.org/v1/mmsi/366999709/photos/1.jpg',
},
},
{ urls: { mmsi: 'https://openvesselphotos.org.evil.com/1.jpg' } },
{ urls: { mmsi: 'https://[email protected]/1.jpg' } },
{ licence: '', licenceName: '' },
])
assert.deepEqual(parseOpenVesselPhoto(withPhoto(overrides)), {
found: false,
});
assert.deepEqual(
parseOpenVesselPhoto({ ...PHOTO, link: 'https://example.com/' }),
{ found: false },
);
});
test('a vessel without photos, or an answer with a bad MMSI, is no photo', () => {
assert.deepEqual(parseOpenVesselPhoto({ ...UPSTREAM, photos: [] }), {
found: false,
});
assert.deepEqual(parseOpenVesselPhoto({ ...UPSTREAM, mmsi: '12/../x' }), {
found: false,
});
assert.deepEqual(parseOpenVesselPhoto(null), { found: false });
});
test('the next photo stands in when the first fails its checks', () => {
const second = {
...UPSTREAM.photos[0],
n: 2,
urls: {
mmsi: 'https://openvesselphotos.org/v1/mmsi/366999709/photos/2.jpg',
},
};
const photo = parseOpenVesselPhoto({
...UPSTREAM,
photos: [{ ...UPSTREAM.photos[0], licenceName: '', licence: '' }, second],
});
assert.equal(
photo.src,
'https://openvesselphotos.org/v1/mmsi/366999709/photos/2.jpg?size=800',
);
});
test('an answer about another vessel than the one asked for is no photo', () => {
assert.deepEqual(parseOpenVesselPhoto(UPSTREAM, '366999709'), PHOTO);
assert.deepEqual(parseOpenVesselPhoto(UPSTREAM, '227000101'), {
found: false,
});
});
File diff suppressed because it is too large. Load diff
+491
View File
@@ -0,0 +1,491 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_CATEGORIES,
RAIL_ENABLED_FEEDS,
RAIL_FEED_ID_PATTERN,
RAIL_FEED_REGISTRY,
RAIL_MODES,
RAIL_POSITION_SOURCES,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
isRailFeedPollable,
publicRailCatalog,
railEnabledFeeds,
railFeedPositions,
railFeedsInRange,
} from './railFeeds.js';
const isHttps = (url) => new URL(url).protocol === 'https:';
test('every registered feed is complete, https and uniquely named', () => {
const ids = new Set();
for (const feed of RAIL_FEED_REGISTRY) {
assert.match(feed.id, RAIL_FEED_ID_PATTERN, feed.id);
assert.ok(!ids.has(feed.id), `duplicate id ${feed.id}`);
ids.add(feed.id);
const positions = railFeedPositions(feed);
assert.ok(RAIL_POSITION_SOURCES.includes(positions), feed.id);
assert.ok(isHttps(feed.url), feed.id);
for (const url of [feed.tripUpdatesUrl, feed.staticGtfsUrl])
if (url !== undefined) assert.ok(isHttps(url), `${feed.id} ${url}`);
assert.ok(
['gtfs-rt', 'irish-rail-xml', 'siri-vm'].includes(feed.format),
feed.id,
);
if (feed.format === 'irish-rail-xml')
assert.equal(positions, 'event', `${feed.id} positions are events`);
for (const key of ['name', 'operator', 'region', 'license', 'attribution'])
assert.ok(feed[key], `${feed.id} has ${key}`);
assert.ok(URL.canParse(feed.licenseUrl), `${feed.id} licence url`);
assert.equal(typeof feed.defaultEnabled, 'boolean', feed.id);
assert.equal(typeof feed.terms?.quote, 'string', feed.id);
assert.equal(typeof feed.classify, 'function', feed.id);
assert.ok(
Number.isFinite(feed.center.lat) && Number.isFinite(feed.center.lon),
);
assert.ok(feed.loadRadiusKm > 0 && feed.loadRadiusKm <= 1000, feed.id);
for (const ms of [feed.pollIntervalMs, feed.tripUpdatesIntervalMs])
if (ms !== undefined) assert.ok(ms >= 5_000 && ms <= 300_000, feed.id);
}
});
// The fields the registry documents (railFeeds.js, RAIL_FEED_REGISTRY):
// an entry carries nothing else, so no feed asks for a key, a sign-in or a
// timetable to place its trains.
const REGISTRY_FIELDS = new Set([
'id',
'name',
'operator',
'region',
'center',
'loadRadiusKm',
'format',
'url',
'positions',
'headers',
'headerClock',
'speedUnit',
'pollIntervalMs',
'tripUpdatesUrl',
'tripUpdatesIntervalMs',
'maxBodyBytes',
'limits',
'routesFrom',
'positionsByTrainNumber',
'staticGtfsUrl',
'staticGtfsEntry',
'staticHeaders',
'staticRouteTypes',
'staticTripEnds',
'staticSimplifyM',
'staticRouteRename',
'staticTripRename',
'license',
'licenseUrl',
'attribution',
'defaultEnabled',
'terms',
'classify',
]);
test('every feed reports live positions: GPS or signalling events', () => {
for (const feed of RAIL_FEED_REGISTRY) {
assert.notEqual(feed.positions, 'schedule', feed.id);
assert.ok(
['gps', 'event'].includes(railFeedPositions(feed)),
`${feed.id} positions`,
);
assert.equal(typeof feed.url, 'string', `${feed.id} has a live feed`);
}
for (const entry of publicRailCatalog())
assert.ok(['gps', 'event'].includes(entry.positions), entry.id);
});
/** Query parameter names that carry a key, a token or a signature. */
const ACCESS_PARAM =
/^(api[-_]?key|appid|app[-_]?key|key|token|access[-_]?token|auth|sig|signature|subscription[-_]?key)$/i;
/** `name=value` of each access parameter in a URL (values decoded). */
function accessParams(url) {
if (!url) return [];
return [...new URL(url).searchParams]
.filter(([name]) => ACCESS_PARAM.test(name))
.map(([name, value]) => `${name}=${value}`);
}
// The only access parameter allowed, a fixed public value the publisher
// prints in the published link for everyone, the same for every user and not
// a credential of the deployer's: Bordeaux Métropole's TBM `apiKey` (on the
// dataset page).
const PUBLISHED_ACCESS_VALUES = new Set([
'tbm-bordeaux apiKey=opendata-bordeaux-metropole-flux-gtfs-rt',
]);
test('an access parameter in a feed URL is found, whatever its spelling', () => {
for (const url of [
'https://x.test/a?apiKey=1',
'https://x.test/a?api_key=1',
'https://x.test/a?APIKEY=1',
'https://x.test/a?f=pb&key=1',
'https://x.test/a?token=1',
'https://x.test/a?access_token=1',
'https://x.test/a?sr=b&sig=abc%3D',
'https://x.test/a?signature=1',
'https://x.test/a?subscription-key=1',
])
assert.equal(accessParams(url).length, 1, url);
assert.deepEqual(accessParams('https://x.test/a?format=pb&route=key'), []);
assert.deepEqual(accessParams(null), []);
// Each published value is still in use, so the allow-list cannot go stale.
const used = new Set();
for (const feed of RAIL_FEED_REGISTRY)
for (const url of [feed.url, feed.tripUpdatesUrl, feed.staticGtfsUrl])
for (const param of accessParams(url)) used.add(`${feed.id} ${param}`);
assert.deepEqual([...used].sort(), [...PUBLISHED_ACCESS_VALUES].sort());
});
test('every feed is on, credited with its licence and terms, and fetched as published', () => {
const ids = new Set();
for (const feed of RAIL_FEED_REGISTRY) {
assert.ok(!ids.has(feed.id), `duplicate id ${feed.id}`);
ids.add(feed.id);
assert.equal(feed.defaultEnabled, true, `${feed.id} is on`);
for (const field of Object.keys(feed))
assert.ok(REGISTRY_FIELDS.has(field), `${feed.id}: ${field}`);
assert.ok(typeof feed.license === 'string' && feed.license, feed.id);
assert.ok(isHttps(feed.licenseUrl), `${feed.id} licence url`);
assert.ok(
typeof feed.attribution === 'string' && feed.attribution,
feed.id,
);
assert.equal(typeof feed.terms?.quote, 'string', `${feed.id} terms`);
assert.ok(feed.terms.quote || feed.terms.note, `${feed.id} terms basis`);
// No URL carries an access parameter, but for the fixed public values
// their publishers print for everyone (PUBLISHED_ACCESS_VALUES).
for (const url of [feed.url, feed.tripUpdatesUrl, feed.staticGtfsUrl])
for (const param of accessParams(url))
assert.ok(
PUBLISHED_ACCESS_VALUES.has(`${feed.id} ${param}`),
`${feed.id} ${url}`,
);
}
assert.equal(ids.size, RAIL_FEED_REGISTRY.length);
assert.deepEqual(
RAIL_ENABLED_FEEDS.map((feed) => feed.id),
RAIL_FEED_REGISTRY.map((feed) => feed.id),
);
});
test('a feed is switched on only when its terms were read and quoted', () => {
for (const feed of RAIL_ENABLED_FEEDS) {
assert.ok(
feed.terms.quote || feed.terms.note,
`${feed.id} is enabled without a recorded basis`,
);
}
// Tri-Rail states no licence; it is used without one by project owner
// decision, and the terms say so.
const triRail = getRailFeed('tri-rail');
assert.ok(triRail, 'tri-rail is pollable');
assert.ok(
triRail.terms.note.startsWith(
'Used without a published licence (project owner decision, 2026-11-10)',
),
);
assert.match(triRail.license, /^No licence stated/);
});
test('only enabled, well-formed ids resolve to a pollable feed', () => {
assert.equal(getRailFeed('mta-metro-north')?.id, 'mta-metro-north');
for (const id of ['', 'Nope', '../x', 'a', null, 42, 'unknown-feed'])
assert.equal(getRailFeed(id), null, String(id));
const mbta = getRegisteredRailFeed('mbta-rail');
assert.equal(isRailFeedPollable(mbta), true);
assert.equal(isRailFeedPollable({ ...mbta, defaultEnabled: false }), false);
assert.deepEqual(
railEnabledFeeds().map((entry) => entry.id),
RAIL_ENABLED_FEEDS.map((entry) => entry.id),
);
});
test('classifiers keep trains and drop buses in mixed feeds', () => {
const kind = (id, route) => classifyTrain(getRegisteredRailFeed(id), route);
assert.deepEqual(kind('mbta-rail', 'CR-Worcester'), {
category: 'local',
mode: 'rail',
});
assert.equal(kind('mbta-rail', 'Red').mode, 'metro');
assert.equal(kind('mbta-rail', 'Green-E').mode, 'tram');
assert.equal(kind('mbta-rail', 'Mattapan').category, 'transit');
assert.equal(kind('mbta-rail', '1'), null);
assert.equal(kind('mbta-rail', 'Boat-F1'), null);
assert.equal(kind('mbta-rail', 'Shuttle-Generic'), null);
assert.equal(kind('mbta-rail', null), null);
assert.equal(kind('rtd-denver-rail', 'A').category, 'local');
assert.equal(kind('rtd-denver-rail', '113G').category, 'local');
assert.equal(kind('rtd-denver-rail', '117N').category, 'local');
assert.equal(kind('rtd-denver-rail', '101E').category, 'transit');
assert.equal(kind('rtd-denver-rail', '107R').category, 'transit');
assert.equal(kind('rtd-denver-rail', '15'), null);
assert.equal(kind('rtd-denver-rail', '15L'), null);
assert.equal(kind('capmetro-rail', '550').category, 'local');
assert.equal(kind('capmetro-rail', '801'), null);
// Northstar's old route id is a bus now and must not come back as a train.
assert.equal(kind('metrotransit-rail', '888'), null);
assert.equal(kind('metrotransit-rail', '902').mode, 'tram');
assert.equal(kind('septa-trolley', 'M1').mode, 'metro');
assert.equal(kind('septa-trolley', '17'), null);
assert.equal(kind('translink-seq-rail', 'BRCA-5').category, 'local');
assert.equal(kind('translink-seq-rail', '61-3'), null);
assert.equal(kind('adelaide-metro', 'SEAFRD').category, 'local');
assert.equal(kind('adelaide-metro', 'GLNELG').mode, 'tram');
assert.equal(kind('adelaide-metro', 'J1A'), null);
assert.equal(kind('mta-metro-north', null).category, 'local');
});
test('Irish Rail: DART and Commuter local, InterCity by province, Enterprise international', () => {
const kind = (code, route) =>
classifyTrain(getRegisteredRailFeed('irish-rail'), route, {
tripId: code,
})?.category ?? null;
assert.equal(kind('E244', 'Howth - Bray'), 'local');
assert.equal(kind('D822', 'Grand Canal Dock - Drogheda'), 'local');
assert.equal(kind('P571', 'Midleton - Cork'), 'local');
assert.equal(kind('A216', 'Dublin Heuston - Cork'), 'national');
assert.equal(kind('A720', 'Dublin Heuston - Galway'), 'national');
assert.equal(kind('B701', 'Galway - Dublin Heuston'), 'national');
assert.equal(kind('A308', 'Mallow - Tralee'), 'regional', 'both Munster');
assert.equal(kind('A606', 'Dublin Connolly - Rosslare Europort'), 'regional');
assert.equal(kind('A118', 'Dublin Connolly - Belfast'), 'international');
assert.equal(kind('A131', 'Belfast - Dublin Connolly'), 'international');
assert.equal(kind('A999', 'Nowhere - Elsewhere'), 'national');
assert.equal(kind('A999', null), 'national');
assert.equal(kind('not a code', 'Howth - Bray'), null);
assert.equal(kind(null, 'Howth - Bray'), null);
});
test('a classifier answer outside the known lists is treated as not a train', () => {
const feed = (answer) => ({ classify: () => answer });
assert.equal(
classifyTrain(feed({ category: 'express', mode: 'rail' })),
null,
);
assert.equal(classifyTrain(feed({ category: 'local', mode: 'bus' })), null);
assert.equal(classifyTrain(feed(null)), null);
assert.equal(classifyTrain({}), null);
assert.equal(classifyTrain(null), null);
for (const feed of RAIL_FEED_REGISTRY) {
for (const route of [
'CR-Lowell',
'A',
'550',
'901',
'T1',
'BRCA-5',
'GAW',
null,
]) {
const kind = feed.classify(route);
if (!kind) continue;
assert.ok(RAIL_CATEGORIES.includes(kind.category), `${feed.id} ${route}`);
assert.ok(RAIL_MODES.includes(kind.mode), `${feed.id} ${route}`);
}
}
});
test('coverage selection returns enabled feeds near the point, nearest first', () => {
const nyc = railFeedsInRange(40.75, -73.99).map((feed) => feed.id);
assert.ok(nyc.includes('mta-metro-north') && nyc.includes('mta-lirr'));
assert.deepEqual(railFeedsInRange(0, -150), []);
assert.deepEqual(railFeedsInRange(Number.NaN, 0), []);
const brisbane = railFeedsInRange(-27.47, 153.02);
assert.equal(brisbane[0].id, 'translink-seq-rail');
// The browser passes the server's catalog, which lists a feed that reads
// its routes from a schedule pack only once that pack is built.
const madrid = (built) =>
railFeedsInRange(
40.42,
-3.7,
0,
publicRailCatalog({ schedulePackBuilt: () => built }),
).map((feed) => feed.id);
assert.ok(
!madrid(false).includes('renfe-larga-distancia'),
'no pack: not offered',
);
assert.ok(madrid(false).includes('renfe-cercanias'), 'needs no pack');
assert.ok(madrid(true).includes('renfe-larga-distancia'));
assert.deepEqual(railFeedsInRange(-27.47, 153.02, 0, null), []);
assert.deepEqual(railFeedsInRange(-27.47, 153.02, 0, [{ id: 'x' }]), []);
});
test('the public catalog exposes coverage and credit but never upstream URLs', () => {
const catalog = publicRailCatalog();
assert.equal(catalog.length, RAIL_ENABLED_FEEDS.length);
for (const entry of catalog) {
assert.equal(entry.url, undefined);
assert.equal(entry.tripUpdatesUrl, undefined);
assert.equal(entry.headers, undefined);
assert.equal(entry.classify, undefined);
assert.ok(entry.attribution && entry.licenseUrl);
assert.ok(RAIL_POSITION_SOURCES.includes(entry.positions));
assert.equal(typeof entry.tripUpdates, 'boolean');
}
const byId = new Map(catalog.map((entry) => [entry.id, entry]));
assert.equal(byId.get('irish-rail').positions, 'event');
assert.equal(byId.get('irish-rail').tripUpdates, false);
assert.equal(byId.get('mbta-rail').positions, 'gps');
assert.equal(byId.get('mbta-rail').tripUpdates, true);
// No upstream URL reaches the browser, by any field but the licence link
// (an access point's page can be both the licence and the download).
const text = JSON.stringify(
catalog.map((entry) => ({ ...entry, licenseUrl: null })),
);
for (const feed of RAIL_FEED_REGISTRY)
for (const url of [feed.url, feed.tripUpdatesUrl, feed.staticGtfsUrl])
if (url) assert.ok(!text.includes(url), `${feed.id} ${url}`);
});
test('Finland: commuter local, H/HDM regional, long distance national, Haaparanta international', () => {
const kind = (route) =>
classifyTrain(getRegisteredRailFeed('fintraffic-vr'), route)?.category ??
null;
assert.equal(kind('EPO_HKI_E_HL_10'), 'local');
assert.equal(kind('SK_ÄHT_480_HDM_10'), 'regional');
assert.equal(kind('TPE_PRI_1_H_10'), 'regional');
for (const route of [
'HKI_ROI_23_IC_10',
'HKI_OL_45_S_10',
'HKI_KOK_101_P_10',
'HKI_KLI_265_PYO_10',
])
assert.equal(kind(route), 'national', route);
assert.equal(kind('OL_HPA_407_IC_10'), 'international');
assert.equal(kind('HPA_OL_408_IC_10'), 'international');
for (const route of ['HKI_TKU_1_T_10', 'IC 23', null])
assert.equal(kind(route), null, String(route));
});
test('Renfe: products and the autonomous-community rule', () => {
const feed = getRegisteredRailFeed('renfe-larga-distancia');
const kind = (route) => classifyTrain(feed, route)?.category ?? null;
for (const product of ['AVE', 'AVLO', 'ALVIA', 'EUROMED', 'Intercity'])
for (const scope of ['S', 'X'])
assert.equal(kind(`${product}~${scope}~1`), 'national', product);
for (const product of ['MD', 'REGIONAL', 'REG.EXP.', 'PROXIMDAD', 'AVANT']) {
assert.equal(kind(`${product}~S~1`), 'regional', product);
assert.equal(kind(`${product}~X~1`), 'national', product);
}
assert.equal(kind('AVE INT~X~1'), 'international');
assert.equal(kind('TRENCELTA~X~1'), 'international');
assert.equal(kind('MD~B~1'), 'international', 'an end abroad');
assert.equal(kind('BUS~S~1'), null);
assert.equal(kind('1700037606GL023'), null, 'an unscoped operator id');
assert.equal(kind(null), null);
// The rename the path build applies to routes.txt rows.
const places = {
17000: { lat: 40.47, lon: -3.68 },
37606: { lat: 38.89, lon: -6.98 },
18000: { lat: 40.41, lon: -3.7 },
87089: { lat: 43.3, lon: 5.38 },
};
const helpers = {
stop: (id) => places[id] ?? null,
stateAt: (lat) =>
lat > 43
? { country: 'FR', state: 'FR-13' }
: lat < 39.5
? { country: 'ES', state: 'ES-EX' }
: { country: 'ES', state: 'ES-MD' },
};
const rename = (id, product) =>
feed.staticRouteRename(
{ route_id: id, route_short_name: product },
helpers,
);
assert.equal(rename('1700037606GL023', 'MD'), 'MD~X~1700037606GL023');
assert.equal(
rename('1700018000GL001', 'REGIONAL'),
'REGIONAL~S~1700018000GL001',
);
assert.equal(rename('1700087089AV016', 'AVE'), 'AVE~B~1700087089AV016');
assert.equal(
rename('1700099999GL001', 'MD'),
'MD~X~1700099999GL001',
'unplaced',
);
assert.equal(rename('short', 'MD'), null);
assert.equal(
classifyTrain(getRegisteredRailFeed('renfe-cercanias'), null)?.category,
'local',
);
});
test('Norway: by line and county, replacement buses dropped, Flåmsbana a tourist line', () => {
const kind = (id, route, tripId = 'VYG:ServiceJourney:536_906310-R') =>
classifyTrain(getRegisteredRailFeed(id), route, { tripId });
assert.equal(kind('entur-vy', 'VYG:Line:F4')?.category, 'national');
assert.equal(kind('entur-vy', 'VYG:Line:RE11')?.category, 'national');
assert.equal(kind('entur-vy', 'VYG:Line:L1')?.category, 'local');
assert.equal(kind('entur-vy', 'VYG:Line:RE20')?.category, 'international');
assert.equal(kind('entur-vy', 'VYG:Line:F1')?.category, 'international');
assert.equal(kind('entur-vy', 'VYG:Line:R55')?.category, 'regional');
const flam = kind('entur-vy', 'VYG:Line:R45');
assert.equal(flam.category, 'regional');
assert.equal(flam.tourist, true);
assert.equal(kind('entur-go-ahead', 'GOA:Line:50')?.category, 'national');
assert.equal(kind('entur-go-ahead', 'GOA:Line:59')?.category, 'local');
assert.equal(kind('entur-flytoget', 'FLT:Line:FLY1')?.category, 'local');
assert.equal(kind('entur-sj-nord', 'SJN:Line:21')?.category, 'national');
assert.equal(kind('entur-sj-nord', 'SJN:Line:72')?.category, 'international');
assert.equal(kind('entur-sj-nord', 'SJN:Line:26')?.category, 'regional');
assert.equal(
kind('entur-vy', 'VYG:Line:RE11', 'VYG:ServiceJourney:BUS-836_761763-R'),
null,
'a replacement bus on a train line',
);
assert.equal(kind('entur-vy', 'VYG:Line:X99'), null);
assert.match(getRegisteredRailFeed('entur-sj-nord').terms.note, /missing/);
// Its positions come from SIRI VM, the timetable still gives its stops.
assert.equal(
railFeedPositions(getRegisteredRailFeed('entur-sj-nord')),
'gps',
);
assert.equal(getRegisteredRailFeed('entur-sj-nord').format, 'siri-vm');
});
test('feeds that read routes from their schedule pack can build one', () => {
for (const feed of RAIL_FEED_REGISTRY.filter((entry) => entry.routesFrom)) {
assert.equal(feed.routesFrom, 'schedule', feed.id);
assert.ok(isHttps(feed.staticGtfsUrl), feed.id);
}
for (const feed of RAIL_FEED_REGISTRY.filter((entry) => entry.headerClock))
assert.equal(feed.headerClock, 'untrusted', feed.id);
// Identify-yourself headers never carry a person.
for (const feed of RAIL_FEED_REGISTRY)
for (const value of Object.values({
...(feed.headers || {}),
...(feed.staticHeaders || {}),
}))
assert.ok(!/@/.test(value), feed.id);
});
test('a feed Transit also reads declares the same speed unit', async () => {
const { TRANSIT_FEED_REGISTRY } = await import('./transitFeeds.js');
const transitByUrl = new Map(
TRANSIT_FEED_REGISTRY.map((feed) => [feed.url, feed]),
);
let shared = 0;
for (const feed of RAIL_FEED_REGISTRY) {
const transit = transitByUrl.get(feed.url);
if (!transit) continue;
shared += 1;
assert.equal(feed.speedUnit, transit.speedUnit, feed.id);
}
assert.ok(shared > 0);
assert.equal(getRegisteredRailFeed('tam-montpellier').speedUnit, 'km/h');
});
+63
View File
@@ -0,0 +1,63 @@
// Asian feeds: Japan's live tram feeds (Manyōsen and Toyama), registered,
// switched on with their licence basis recorded, and sorted into
// categories.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_ENABLED_FEEDS,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
railFeedPositions,
railFeedsInRange,
} from './railFeeds.js';
const LIVE = ['jp-manyosen', 'jp-toyama-tram'];
const feed = (id) => getRegisteredRailFeed(id);
const kind = (id, routeId) => classifyTrain(feed(id), routeId);
const TRAM = { category: 'transit', mode: 'tram' };
test('the Japanese feeds are on, with their licence basis recorded', () => {
for (const id of LIVE) {
assert.ok(getRailFeed(id), `${id} is pollable`);
assert.ok(RAIL_ENABLED_FEEDS.includes(feed(id)), id);
assert.ok(feed(id).license, id);
assert.match(feed(id).licenseUrl, /^https:\/\//, id);
assert.ok(feed(id).attribution, id);
assert.ok(feed(id).terms.quote, `${id} quotes its terms`);
assert.match(feed(id).terms.note, /read 2026-11-10/, id);
assert.doesNotMatch(feed(id).terms.note, /@/, id);
assert.equal(railFeedPositions(feed(id)), 'gps', id);
assert.match(feed(id).url, /^https:\/\/gtfs-rt-files\.buscatch\.jp\//);
}
assert.equal(
feed('jp-toyama-tram').staticGtfsUrl,
undefined,
'its static GTFS codes the tram as buses',
);
});
test('Japan: the Manyōsen and Toyama trams are transit', () => {
assert.deepEqual(kind('jp-manyosen', '1'), TRAM);
assert.deepEqual(
kind('jp-toyama-tram', '富山港線(南富山)(3004-1)'),
TRAM,
'a route id seen live',
);
});
test('each Japanese tram feed is polled around its own city only', () => {
const near = (lat, lon) => railFeedsInRange(lat, lon).map((f) => f.id);
// Takaoka for the Manyōsen, Toyama station for the city tram.
assert.ok(near(36.75, 137.02).includes('jp-manyosen'));
assert.ok(near(36.7, 137.21).includes('jp-toyama-tram'));
// Tokyo and Nagoya are hundreds of kilometres off.
for (const [lat, lon] of [
[35.68, 139.77],
[35.17, 136.88],
])
for (const id of LIVE) assert.ok(!near(lat, lon).includes(id), id);
for (const id of LIVE) assert.ok(feed(id).loadRadiusKm <= 25, id);
});
+97
View File
@@ -0,0 +1,97 @@
// The rail counts the documents state, against the registry: how many feeds
// there are, how many publish GPS or station reports, how many work out of
// the box, register trip updates or have a static GTFS, and one
// DATA_SOURCES.md row per feed, none for a feed the registry does not hold.
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import {
RAIL_ENABLED_FEEDS,
RAIL_FEED_REGISTRY,
railFeedNeedsSchedulePack,
railFeedPositions,
} from './railFeeds.js';
const read = (file) =>
readFileSync(new URL(`../../${file}`, import.meta.url), 'utf8');
const counts = {
registry: RAIL_FEED_REGISTRY.length,
gps: RAIL_FEED_REGISTRY.filter((feed) => railFeedPositions(feed) === 'gps')
.length,
stationReports: RAIL_FEED_REGISTRY.filter(
(feed) => railFeedPositions(feed) === 'event',
).length,
outOfTheBox: RAIL_ENABLED_FEEDS.filter(
(feed) => !railFeedNeedsSchedulePack(feed),
).length,
tripUpdates: RAIL_FEED_REGISTRY.filter(
(feed) => typeof feed.tripUpdatesUrl === 'string',
).length,
staticGtfs: RAIL_FEED_REGISTRY.filter(
(feed) => typeof feed.staticGtfsUrl === 'string',
).length,
};
test('the registry holds the feeds the documents count', () => {
assert.deepEqual(counts, {
registry: 30,
gps: 29,
stationReports: 1,
outOfTheBox: 28,
tripUpdates: 22,
staticGtfs: 29,
});
assert.equal(RAIL_ENABLED_FEEDS.length, counts.registry, 'all switched on');
assert.deepEqual(
RAIL_FEED_REGISTRY.filter(
(feed) => railFeedPositions(feed) === 'event',
).map((feed) => feed.id),
['irish-rail'],
'the station reports are Irish Rail’s',
);
});
test('DATA_SOURCES.md states the rail counts and has one row per feed', () => {
const sources = read('DATA_SOURCES.md');
const { registry, gps, outOfTheBox, tripUpdates, staticGtfs } = counts;
for (const phrase of [
`The registry lists ${registry} feeds, all switched on`,
`${gps} feeds publish live GPS positions (SJ Nord's through SIRI VM), and Irish Rail reports the station or timing point a train last passed`,
`${tripUpdates} feeds register trip updates`,
`${outOfTheBox} feeds work out of the box`,
`(${staticGtfs} of the ${registry} have one)`,
])
assert.ok(sources.includes(phrase), phrase);
const rows = sources
.split('\n')
.filter((line) => /^\| \*\*.+\| Rail layer trains, /.test(line));
assert.equal(rows.length, registry, 'one row per feed');
for (const feed of RAIL_FEED_REGISTRY)
assert.ok(
rows.some((row) => row.includes(`| Rail layer trains, ${feed.region}:`)),
`${feed.id} has a row`,
);
});
test('CURRENT-STATE.md states the rail counts', () => {
const state = read('docs/CURRENT-STATE.md');
const { registry, gps, outOfTheBox, tripUpdates, staticGtfs } = counts;
for (const phrase of [
`(${registry} keyless feeds, all on, ${outOfTheBox} out of the box;`,
`The registry lists ${registry} feeds, all`,
`a host that has built nothing shows ${outOfTheBox} feeds`,
`By position source, ${gps} feeds publish live GPS.`,
'Irish Rail publishes the\nstation or timing point the signalling last reported.',
`${tripUpdates} feeds register trip updates;`,
`that has a static GTFS (${staticGtfs} of the ${registry})`,
])
assert.ok(state.includes(phrase), phrase);
});
test('README.md states the rail counts', () => {
const readme = read('README.md');
const { registry, gps, outOfTheBox } = counts;
const phrase = `from ${registry} keyless operator feeds: live GPS positions from ${gps} of them and Irish Rail’s station reports, ${outOfTheBox} live out of the box`;
assert.ok(readme.includes(phrase), phrase);
});
+86
View File
@@ -0,0 +1,86 @@
// France (2026-11-10): the Bordeaux and Montpellier tram networks, from
// transport.data.gouv.fr. Registered, switched on with the dataset's licence
// field quoted, polled for live positions, and sorted into categories; no
// registry URL is on a download path its host's robots.txt disallows.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_FEED_REGISTRY,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
railFeedHasTripUpdates,
railFeedPositions,
} from './railFeeds.js';
const kind = (id, routeId) => {
const train = classifyTrain(getRegisteredRailFeed(id), routeId);
return train ? `${train.category}/${train.mode}` : null;
};
const FR_FEEDS = ['tbm-bordeaux', 'tam-montpellier'];
// ------------------------------------------------------------ registry
test('the French feeds are on and quote the dataset licence field', () => {
for (const id of FR_FEEDS) {
const feed = getRegisteredRailFeed(id);
assert.ok(feed, `${id} is registered`);
assert.ok(getRailFeed(id), `${id} is pollable`);
assert.equal(feed.defaultEnabled, true, id);
assert.match(feed.staticGtfsUrl, /^https:\/\//, id);
assert.ok(['lov2', 'odc-odbl', 'fr-lo'].includes(feed.terms.quote), id);
assert.match(feed.terms.note, /transport\.data\.gouv\.fr dataset "/, id);
assert.match(feed.terms.note, /2026-11-10/, id);
}
});
test('the ODbL dataset carries the share-alike note and the access point conditions', () => {
const feed = getRegisteredRailFeed('tam-montpellier');
assert.equal(feed.license, 'ODbL 1.0');
assert.match(feed.terms.note, /share-alike/);
assert.match(feed.terms.note, /dans le format d'origine/);
assert.equal(
getRegisteredRailFeed('tbm-bordeaux').license,
'Licence Ouverte / Open Licence 1.0',
);
});
test('positions: Bordeaux and Montpellier trams are GPS, with trip updates', () => {
for (const id of FR_FEEDS) {
assert.equal(railFeedPositions(getRegisteredRailFeed(id)), 'gps', id);
assert.ok(railFeedHasTripUpdates(getRegisteredRailFeed(id)), id);
}
});
// ------------------------------------------------------------ categories
test('tram lines are transit; buses are dropped', () => {
const cases = [
['tbm-bordeaux', '164', 'transit/tram'],
['tbm-bordeaux', '901', null],
['tam-montpellier', '5', 'transit/tram'],
['tam-montpellier', 'A', null],
];
for (const [id, route, expected] of cases)
assert.equal(kind(id, route), expected, `${id} ${route}`);
});
// ------------------------------------------------------------ download
// Hosts whose robots.txt disallows the download paths French publishers use.
const DISALLOWED = [
/^https:\/\/eu\.ftp\.opendatasoft\.com\//,
/^https:\/\/thello\.axelor\.com\//,
/^https:\/\/static\.data\.gouv\.fr\/resources\//,
/^https:\/\/www\.data\.gouv\.fr\/(fr\/)?(api\/1\/)?datasets\/r\//,
/\/explore\/dataset\/.*\/download/,
];
test('no registry URL is on a download path its robots.txt disallows', () => {
for (const feed of RAIL_FEED_REGISTRY)
for (const url of [feed.url, feed.tripUpdatesUrl, feed.staticGtfsUrl])
if (url)
for (const pattern of DISALLOWED)
assert.ok(!pattern.test(url), `${feed.id} ${url}`);
});
+103
View File
@@ -0,0 +1,103 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_ENABLED_FEEDS,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
} from './railFeeds.js';
import {
railAdmissionGlobalMax,
railNaturalRequestsPerMinute,
} from './railProxy.js';
/** One end of a run, as the path build's `readTripEnds` writes it. */
const at = (name, country, state) => ({ name, country, state });
const ends = (
from,
to,
countries = [...new Set([from.country, to.country])],
) => ({ from, to, countries });
/**
* The category a timetable trip gets: the feed's route rename (when it has
* one), its trip rename with the run's ends, then its classifier on the pack
* route id, as the build and the browser apply them.
*/
function tripKind(feedId, route, run = null, trip = { trip_id: 't1' }) {
const feed = getRegisteredRailFeed(feedId);
const base =
(typeof feed.staticRouteRename === 'function'
? feed.staticRouteRename(route, { stop: () => null, stateAt: () => null })
: null) ?? route.route_id;
const id =
typeof feed.staticTripRename === 'function'
? feed.staticTripRename(trip, base, { ends: run })
: base;
return classifyTrain(feed, id, { tripId: trip.trip_id });
}
const category = (...args) => tripKind(...args)?.category ?? null;
test('the national feeds are on and quote their terms', () => {
for (const id of ['ktmb', 'entur-sj-nord']) {
assert.ok(getRailFeed(id), `${id} is pollable`);
assert.ok(getRegisteredRailFeed(id).terms.quote, `${id} quotes its terms`);
}
});
test('the admission cap grows with the KTMB and SJ Nord feeds', () => {
const before = RAIL_ENABLED_FEEDS.filter(
(feed) => !['ktmb', 'entur-sj-nord'].includes(feed.id),
);
assert.ok(
railNaturalRequestsPerMinute() > railNaturalRequestsPerMinute(before),
);
assert.ok(railAdmissionGlobalMax() >= railAdmissionGlobalMax(before));
});
test('KTMB: Komuter local, Shuttle Tebrau international, ETS and Intercity by state', () => {
const route = (id) => ({ route_id: id, route_type: '2' });
const kl = at('KL SENTRAL', 'MY', 'MY-14');
const jb = at('JB SENTRAL', 'MY', 'MY-01');
const tumpat = at('TUMPAT', 'MY', 'MY-03');
const guaMusang = at('GUA MUSANG', 'MY', 'MY-03');
// The simplified polygons put Padang Besar station in Thailand.
const padangBesar = at('PADANG BESAR', 'TH', 'TH-90');
const butterworth = at('BUTTERWORTH', 'MY', 'MY-07');
assert.equal(category('ktmb', route('KA15_KD19'), ends(kl, kl)), 'local');
assert.equal(category('ktmb', route('KC05_KB18')), 'local');
assert.equal(
category('ktmb', route('100_47300'), ends(butterworth, padangBesar)),
'local',
'Komuter Utara',
);
assert.equal(category('ktmb', { route_id: 'SS', route_type: '0' }), 'local');
assert.equal(category('ktmb', route('ST')), 'international');
assert.equal(category('ktmb', route('ETS'), ends(kl, jb)), 'national');
assert.equal(
category('ktmb', route('ETS'), ends(kl, padangBesar)),
'national',
'Padang Besar is in Perlis',
);
assert.equal(
category('ktmb', route('ETS'), ends(kl, at('HAT YAI', 'TH', 'TH-90'))),
'international',
);
assert.equal(
category(
'ktmb',
route('ERT'),
ends(at('MERAPOH', 'MY', 'MY-06'), at('WOODLANDS CIQ', 'SG', 'SG-03')),
),
'international',
);
assert.equal(
category('ktmb', route('SH'), ends(tumpat, guaMusang)),
'regional',
);
assert.equal(category('ktmb', route('SH')), 'national', 'ends unknown');
assert.equal(category('ktmb', route('BUS1'), ends(kl, jb)), null);
// The realtime vehicles carry no route: it comes from the schedule pack.
assert.equal(getRegisteredRailFeed('ktmb').routesFrom, 'schedule');
assert.equal(getRegisteredRailFeed('ktmb').pollIntervalMs, 30_000);
});
+46
View File
@@ -0,0 +1,46 @@
// Tri-Rail, South Florida's commuter line: switched on with its basis,
// polled for live positions, and sorted into categories by route or by
// train number.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_ENABLED_FEEDS,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
railFeedHasTripUpdates,
railFeedPositions,
} from './railFeeds.js';
const feed = (id) => getRegisteredRailFeed(id);
const kind = (id, routeId, context) =>
classifyTrain(feed(id), routeId, context);
test('Tri-Rail is on, with its basis', () => {
const tri = feed('tri-rail');
assert.ok(getRailFeed('tri-rail'), 'pollable');
assert.ok(RAIL_ENABLED_FEEDS.includes(tri));
assert.ok(tri.terms.quote, 'quotes its terms');
assert.ok(tri.terms.note, 'explains its basis');
// Used by project owner decision, which it says, naming no person.
assert.match(tri.terms.note, /\(project owner decision, 2026-11-10\)/);
assert.doesNotMatch(tri.terms.note, /@/);
});
test('Tri-Rail: live positions with trip updates, trains by route or train number', () => {
const tri = feed('tri-rail');
assert.equal(railFeedPositions(tri), 'gps');
assert.ok(railFeedHasTripUpdates(tri));
assert.equal(tri.staticGtfsUrl, 'https://gtfs.tri-rail.com/gtfs.zip');
const local = { category: 'local', mode: 'rail' };
// Pack routes: Mainline and Downtown Miami Link.
assert.deepEqual(kind('tri-rail', '1'), local);
assert.deepEqual(kind('tri-rail', '2'), local);
// The airport shuttle bus, should a route id ever reach the classifier.
assert.equal(kind('tri-rail', '4'), null);
// Realtime vehicles carry no route, only the train number.
assert.deepEqual(kind('tri-rail', null, { tripId: 'P617' }), local);
assert.deepEqual(kind('tri-rail', '', { tripId: 'P601X' }), local);
assert.equal(kind('tri-rail', null, { tripId: 'FLA Loop_T01' }), null);
assert.equal(kind('tri-rail', null, {}), null);
});
+62
View File
@@ -0,0 +1,62 @@
// A United States feed whose publisher states no terms (Charlotte LYNX):
// switched on with the reason recorded, and sorted into categories.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_ENABLED_FEEDS,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
railFeedHasTripUpdates,
railFeedPositions,
} from './railFeeds.js';
import { keepsRouteType } from '../../scripts/build-rail-paths.mjs';
// The feeds that publish no terms, used without a published licence by project
// owner decision of 2026-11-10.
const ON = ['charlotte-lynx'];
const BASIS =
'Publishes no terms; used without a published licence (project owner decision, 2026-11-10).';
const TRAM = { category: 'transit', mode: 'tram' };
const feed = (id) => getRegisteredRailFeed(id);
const kind = (id, routeId, context) =>
classifyTrain(feed(id), routeId, context);
test('the no-terms feed is on and says why', () => {
for (const id of ON) {
const entry = feed(id);
assert.ok(getRailFeed(id), `${id} is pollable`);
assert.ok(RAIL_ENABLED_FEEDS.includes(entry), id);
assert.equal(entry.license, 'No licence published', id);
assert.ok(entry.terms.note.startsWith(BASIS), id);
// Nothing personal in the recorded basis.
assert.doesNotMatch(entry.terms.note, /@/, id);
}
});
test('Charlotte LYNX: live light rail and streetcar with trip updates, buses dropped', () => {
const lynx = feed('charlotte-lynx');
assert.equal(railFeedPositions(lynx), 'gps');
assert.ok(railFeedHasTripUpdates(lynx));
assert.match(
lynx.staticGtfsUrl,
/^https:\/\/gtfsrealtime\.ridetransit\.org\//,
);
// Routes seen live on 2026-11-10: 501 Blue Line, 510 Gold Line, buses.
assert.deepEqual(kind('charlotte-lynx', '501'), TRAM);
assert.deepEqual(kind('charlotte-lynx', '510'), TRAM);
for (const bus of ['9', '22', '5', null])
assert.equal(kind('charlotte-lynx', bus), null, String(bus));
});
test('a listed staticRouteTypes is the whole list, so trains coded as buses build', () => {
assert.equal(keepsRouteType({}, '3'), false, 'a bus by default');
assert.equal(keepsRouteType({}, ' 2 '), true);
assert.equal(keepsRouteType({ staticRouteTypes: [3] }, '3'), true);
assert.equal(keepsRouteType({ staticRouteTypes: [3] }, '2'), false);
assert.equal(keepsRouteType({ staticRouteTypes: [0, 2] }, '0'), true);
assert.equal(keepsRouteType({ staticRouteTypes: [0, 2] }, '1'), false);
assert.equal(keepsRouteType({ staticRouteTypes: [3] }, ''), false);
});
+57
View File
@@ -0,0 +1,57 @@
// United States streetcar feeds (Seattle, Cincinnati): switched on with
// their basis, polled for live positions, and sorted into categories with
// the buses dropped.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_ENABLED_FEEDS,
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
railFeedHasTripUpdates,
railFeedPositions,
} from './railFeeds.js';
// Licensed and switched on.
const ON = ['seattle-streetcar', 'cincinnati-connector'];
const TRAM = { category: 'transit', mode: 'tram' };
const feed = (id) => getRegisteredRailFeed(id);
const kind = (id, routeId, context) =>
classifyTrain(feed(id), routeId, context);
test('the US streetcar feeds: licensed feeds on, each with its basis', () => {
for (const id of ON) {
assert.ok(getRailFeed(id), `${id} is pollable`);
assert.ok(RAIL_ENABLED_FEEDS.includes(feed(id)), id);
assert.ok(feed(id).terms.note, `${id} explains its basis`);
assert.ok(feed(id).staticGtfsUrl, `${id} has a timetable`);
assert.ok(feed(id).terms.quote, `${id} quotes its terms`);
// Nothing personal in the recorded basis.
assert.doesNotMatch(feed(id).terms.note, /@/, id);
}
});
test('the US streetcar feeds poll positions and trip updates', () => {
for (const id of ON) {
assert.equal(railFeedPositions(feed(id)), 'gps', id);
assert.ok(railFeedHasTripUpdates(feed(id)), id);
}
});
test('Seattle Streetcar: the two City of Seattle routes of the King County feed', () => {
assert.deepEqual(kind('seattle-streetcar', '100340'), TRAM);
assert.deepEqual(kind('seattle-streetcar', '102638'), TRAM);
assert.equal(kind('seattle-streetcar', '100001'), null, 'a Metro bus');
assert.equal(kind('seattle-streetcar', null), null);
assert.equal(
feed('seattle-streetcar').attribution,
'Transit scheduling, geographic, and real-time data provided by permission of King County',
);
});
test('Cincinnati Connector: the streetcar route, buses dropped', () => {
assert.deepEqual(kind('cincinnati-connector', '100'), TRAM);
assert.equal(kind('cincinnati-connector', '78'), null);
});
+134
View File
@@ -0,0 +1,134 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
classifyTrain,
getRailFeed,
getRegisteredRailFeed,
publicRailCatalog,
railFeedHasTripUpdates,
railFeedPositions,
} from './railFeeds.js';
import {
buildRailSnapshot,
railMaxBodyBytes,
railUpstreamHeaders,
} from './railProxy.js';
const irish = getRegisteredRailFeed('irish-rail');
test('Irish Rail keeps its XML positions and gains a timetable', () => {
assert.equal(irish.format, 'irish-rail-xml');
assert.equal(irish.positions, 'event');
assert.ok(getRailFeed('irish-rail'));
assert.match(irish.staticGtfsUrl, /\/transitData\/Data\/GTFS_Realtime\.zip$/);
assert.deepEqual(irish.staticRouteTypes, [2]);
assert.equal(irish.positionsByTrainNumber, true);
assert.equal(irish.routesFrom, 'schedule');
// The XML's own headers and cap.
assert.match(railUpstreamHeaders(irish).Accept, /xml/);
assert.equal(railMaxBodyBytes(irish), 1024 * 1024);
// The positions publish no licence; the NTA timetable is CC BY 4.0.
assert.equal(
irish.license,
'No licence published (positions); CC BY 4.0 (NTA timetable)',
);
assert.equal(
irish.attribution,
'Iarnród Éireann / Irish Rail; timetable: National Transport Authority, provided as is',
);
});
test('Irish Rail is placed by its XML events and serves no trip updates', () => {
assert.equal(railFeedPositions(irish), 'event');
assert.equal(irish.tripUpdatesUrl, undefined);
assert.equal(railFeedHasTripUpdates(irish), false);
// Listed without a timetable pack: the XML's train codes classify alone.
const entry = publicRailCatalog({ schedulePackBuilt: () => false }).find(
(feed) => feed.id === 'irish-rail',
);
assert.equal(entry.positions, 'event');
assert.equal(entry.tripUpdates, false);
});
test('Irish Rail pack routes: the code letter and the provinces of the ends', () => {
const rename = (code, from, to, countries = ['IE']) =>
irish.staticTripRename({ trip_short_name: code }, 'R-1', {
ends: {
from: { name: from, country: countries[0] },
to: { name: to, country: countries.at(-1) },
countries,
},
});
assert.equal(
rename('A100', 'Dublin Connolly', 'Belfast', ['IE', 'GB']),
'A~B~R-1',
);
assert.equal(rename('A203', 'Cork (Kent)', 'Dublin Heuston'), 'A~X~R-1');
assert.equal(rename('A301', 'Mallow', 'Tralee (Casement)'), 'A~S~R-1');
assert.equal(rename('P501', 'Mallow', 'Cobh'), 'P~S~R-1');
assert.equal(rename('E206', 'Bray (Daly)', 'Howth'), 'E~X~R-1');
assert.equal(rename('195.MF-BH.93-GRN-y11', 'a', 'b'), null, 'no train code');
const kind = (route) => classifyTrain(irish, route, { tripId: '6024_1' });
assert.equal(
kind('A~B~DUB-BFT-O').category,
'international',
'the Enterprise',
);
assert.equal(kind('A~X~DUB-CORK-O').category, 'national');
assert.equal(kind('A~S~DUB-TRALEE-O').category, 'regional');
assert.equal(kind('E~X~BRAY-HOWTH-I').category, 'local', 'DART');
assert.equal(kind('P~S~MAL-COBH-O').category, 'local', 'commuter');
assert.equal(kind('D~S~MAL-COBH-O').category, 'local');
// The XML's own records still classify by their code and ends.
assert.equal(
classifyTrain(irish, 'Dublin Connolly - Belfast', { tripId: 'A100' })
.category,
'international',
);
assert.equal(
classifyTrain(irish, 'Bray - Howth', { tripId: 'E206' }).category,
'local',
);
});
test('an Irish Rail train code takes the timetable run it is on now', () => {
const xml = `<?xml version="1.0" encoding="utf-8"?>
<ArrayOfObjTrainPositions xmlns="http://api.irishrail.ie/realtime/">
<objTrainPositions>
<TrainStatus>R</TrainStatus><TrainLatitude>53.1442</TrainLatitude>
<TrainLongitude>-6.06085</TrainLongitude><TrainCode>A610</TrainCode>
<TrainDate>09 Nov 2026</TrainDate>
<PublicMessage>A610\\n18:37 - Dublin Connolly to Rosslare Europort (3 mins late)\\nDeparted Greystones next stop Kilcoole</PublicMessage>
<Direction>Southbound</Direction>
</objTrainPositions>
<objTrainPositions>
<TrainStatus>R</TrainStatus><TrainLatitude>53.35</TrainLatitude>
<TrainLongitude>-6.25</TrainLongitude><TrainCode>E999</TrainCode>
<TrainDate>09 Nov 2026</TrainDate>
<PublicMessage>E999\\n19:00 - Bray to Howth (0 mins late)\\nDeparted Connolly next stop Clontarf Road</PublicMessage>
<Direction>Northbound</Direction>
</objTrainPositions>
</ArrayOfObjTrainPositions>`;
const pack = { '6024_8435': 'A~X~DUB-EUROPORT-I' };
const snapshot = buildRailSnapshot(
irish,
new TextEncoder().encode(xml),
1_794_251_747_000,
{
routeFor: (tripId) => pack[tripId] ?? null,
tripFor: (code) => (code === 'A610' ? '6024_8435' : null),
},
);
const byCode = Object.fromEntries(
snapshot.trains.map((train) => [train.trainNumber ?? train.tripId, train]),
);
assert.equal(byCode.A610.tripId, '6024_8435');
assert.equal(byCode.A610.routeId, 'A~X~DUB-EUROPORT-I');
assert.equal(byCode.A610.category, 'national');
assert.equal(byCode.A610.positionSource, 'event');
assert.equal(byCode.A610.id, 'A610', 'the vehicle keeps its code');
// Not in the timetable: its own code and ends, as before.
assert.equal(byCode.E999.tripId, 'E999');
assert.equal(byCode.E999.trainNumber, undefined);
assert.equal(byCode.E999.category, 'local');
});
File diff suppressed because it is too large. Load diff
+297
View File
@@ -0,0 +1,297 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtemp, rm, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { crc32 } from 'node:zlib';
import { PbfWriter } from 'pbf';
import {
NO_TRAIN_POSITION_BOX,
NO_TRAIN_POSITION_COUNTRIES,
isNoTrainPositionPoint,
isNoTrainPositionStop,
stopIdCountry,
} from './railNoPositions.js';
import { buildRailSnapshot } from './railProxy.js';
import { createRunIndex } from './railScheduleIndex.js';
import {
buildPack,
buildSchedule,
openZip,
} from '../../scripts/build-rail-paths.mjs';
const KYIV = [50.45, 30.52];
const LVIV = [49.84, 24.03];
const SIMFEROPOL = [44.95, 34.1];
const AMSTERDAM = [52.37, 4.9];
const PRZEMYSL = [49.78, 22.77];
const CHISINAU = [47.0, 28.86];
/** A fake feed: every vehicle and route is a train, so only the rule drops. */
function fakeFeed(extra = {}) {
return Object.freeze({
id: 'test-no-positions',
name: 'Test',
format: 'gtfs-rt',
classify: () => ({ category: 'international', mode: 'rail' }),
...extra,
});
}
test('the area is Ukraine, Crimea included, behind a cheap box', () => {
assert.deepEqual(NO_TRAIN_POSITION_COUNTRIES, ['UA']);
for (const [lat, lon] of [KYIV, LVIV, SIMFEROPOL, [48.0, 37.8]])
assert.equal(isNoTrainPositionPoint(lat, lon), true, `${lat},${lon}`);
for (const [lat, lon] of [AMSTERDAM, PRZEMYSL, CHISINAU, [53.9, 27.56]])
assert.equal(isNoTrainPositionPoint(lat, lon), false, `${lat},${lon}`);
// Outside the box nothing more is computed.
assert.ok(NO_TRAIN_POSITION_BOX.west > 22 && NO_TRAIN_POSITION_BOX.east < 41);
for (const bad of [NaN, null, undefined])
assert.equal(isNoTrainPositionPoint(bad, 30), false);
});
test('stations on the border fall on their own side (OpenStreetMap positions)', () => {
// Outside Ukraine, within 1 km of the border: on the Tisa (Bocicoi), by
// the Polish crossing (Medyka) and on the Slovak line (Čierna nad Tisou).
for (const [name, lat, lon] of [
['Bocicoi', 47.95743, 24.02333],
['Valea Vișeului', 47.91463, 24.15557],
['Sighetu Marmației', 47.9358, 23.88536],
['Medyka', 49.80248, 22.93929],
['Čierna nad Tisou št. hr.', 48.43209, 22.13597],
['Záhony', 48.4017, 22.18376],
])
assert.equal(isNoTrainPositionPoint(lat, lon), false, name);
// Inside Ukraine, as close to the border.
for (const [name, lat, lon] of [
['Uhniv', 50.37859, 23.73241],
['Korchiv', 50.39422, 23.83512],
['Teresva', 47.99573, 23.70047],
['Solotvyno-Likarnia', 47.96591, 23.86276],
])
assert.equal(isNoTrainPositionPoint(lat, lon), true, name);
});
test('a stop is left out when its id names Ukraine or its position is inside', () => {
assert.equal(stopIdCountry('ua:22005:1:1'), 'UA');
assert.equal(stopIdCountry('Pua:22005'), 'UA');
assert.equal(stopIdCountry('de:09162:6'), 'DE');
assert.equal(stopIdCountry('8000105'), null);
// A Ukrainian stop the outline would put in Hungary (Chop, on the border).
assert.equal(
isNoTrainPositionStop({ id: 'ua:22005:1:1', lat: 48.43, lon: 22.1 }),
true,
);
// A stop whose id names Poland, at a position the outline puts in
// Ukraine (Shehyni, across the border from Medyka): its id does not keep
// it.
assert.equal(
isNoTrainPositionStop({ id: 'pl:1:1', lat: 49.7949, lon: 22.9768 }),
true,
);
// Border stations on their own side stay, by id or by position alone.
for (const [id, lat, lon] of [
['pl:1:2', 49.80248, 22.93929], // Medyka
['hu:1:1', 48.4017, 22.18376], // Záhony
['sk:1:1', 48.43209, 22.13597], // Čierna nad Tisou
['ro:1:1', 47.95743, 24.02333], // Bocicoi
['5001', 49.80248, 22.93929],
])
assert.equal(isNoTrainPositionStop({ id, lat, lon }), false, id);
assert.equal(
isNoTrainPositionStop({ id: '9001', lat: LVIV[0], lon: LVIV[1] }),
true,
);
assert.equal(isNoTrainPositionStop(null), false);
});
/** A minimal stored (uncompressed) zip. */
function makeZip(files) {
const locals = [];
const centrals = [];
let offset = 0;
for (const [name, text] of Object.entries(files)) {
const data = Buffer.from(text, 'utf8');
const nameBytes = Buffer.from(name, 'utf8');
const local = Buffer.alloc(30);
local.writeUInt32LE(0x04034b50, 0);
local.writeUInt16LE(20, 4);
local.writeUInt32LE(crc32(data), 14);
local.writeUInt32LE(data.length, 18);
local.writeUInt32LE(data.length, 22);
local.writeUInt16LE(nameBytes.length, 26);
const central = Buffer.alloc(46);
central.writeUInt32LE(0x02014b50, 0);
central.writeUInt16LE(20, 4);
central.writeUInt16LE(20, 6);
central.writeUInt32LE(crc32(data), 16);
central.writeUInt32LE(data.length, 20);
central.writeUInt32LE(data.length, 24);
central.writeUInt16LE(nameBytes.length, 28);
central.writeUInt32LE(offset, 42);
locals.push(local, nameBytes, data);
centrals.push(central, nameBytes);
offset += local.length + nameBytes.length + data.length;
}
const cd = Buffer.concat(centrals);
const end = Buffer.alloc(22);
end.writeUInt32LE(0x06054b50, 0);
end.writeUInt16LE(locals.length / 3, 8);
end.writeUInt16LE(locals.length / 3, 10);
end.writeUInt32LE(cd.length, 12);
end.writeUInt32LE(offset, 16);
return Buffer.concat([...locals, cd, end]);
}
/**
* Wien–Budapest, Budapest–Chop (a `ua:` stop the outline would miss),
* Budapest–Lviv (an id without a country, inside the outline) and
* Sighet–Bocicoi (a Romanian halt on the border river, which a coarser
* outline would place in Ukraine).
*/
const GTFS = {
'agency.txt':
'agency_id,agency_name,agency_url,agency_timezone\n1,Test,https://example.invalid,Europe/Vienna\n',
'routes.txt': 'route_id,route_short_name,route_type\nR,EC,2\n',
'trips.txt': [
'route_id,service_id,trip_id,trip_short_name',
'R,daily,vienna-budapest,EC 1',
'R,daily,budapest-chop,EC 2',
'R,daily,budapest-lviv,EC 3',
'R,daily,sighet-bocicoi,R 4',
].join('\n'),
'stops.txt': [
'stop_id,stop_name,stop_lat,stop_lon',
'at:49:1,Wien Hauptbahnhof,48.185,16.376',
'hu:55:1,Budapest-Keleti,47.500,19.083',
'ua:22005:1:1,Chop,48.43,22.1',
'9001,Lviv,49.84,24.03',
'43820,Sighetu Marmaţiei,47.9358,23.88536',
'43753,Bocicoi Hm.,47.95743,24.02333',
].join('\n'),
'stop_times.txt': [
'trip_id,arrival_time,departure_time,stop_id,stop_sequence',
'vienna-budapest,08:00:00,08:00:00,at:49:1,1',
'vienna-budapest,10:40:00,10:40:00,hu:55:1,2',
'budapest-chop,11:00:00,11:00:00,hu:55:1,1',
'budapest-chop,16:00:00,16:00:00,ua:22005:1:1,2',
'budapest-lviv,12:00:00,12:00:00,hu:55:1,1',
'budapest-lviv,20:00:00,20:00:00,9001,2',
'sighet-bocicoi,09:00:00,09:00:00,43820,1',
'sighet-bocicoi,09:20:00,09:20:00,43753,2',
].join('\n'),
'calendar.txt': [
'service_id,monday,tuesday,wednesday,thursday,friday,saturday,sunday,start_date,end_date',
'daily,1,1,1,1,1,1,1,20260101,20271231',
].join('\n'),
};
async function buildFake(t) {
const dir = await mkdtemp(path.join(os.tmpdir(), 'gev-rail-no-pos-test-'));
t.after(() => rm(dir, { recursive: true, force: true }));
const file = path.join(dir, 'feed.zip');
await writeFile(file, makeZip(GTFS));
const zip = await openZip(file);
t.after(() => zip.handle.close());
const feed = fakeFeed();
const { pack } = await buildPack(feed, zip, { url: 'x' });
return buildSchedule(feed, zip, { url: 'x' }, pack, {
from: '2026-11-09',
days: 2,
});
}
test('the build leaves out every timetable trip that stops in Ukraine', async (t) => {
const { schedule, stats } = await buildFake(t);
assert.deepEqual(Object.keys(schedule.trips).sort(), [
'sighet-bocicoi',
'vienna-budapest',
]);
assert.equal(stats.tripsNoPositions, 2);
assert.deepEqual(schedule.stops.map((stop) => stop.name).sort(), [
'Bocicoi Hm.',
'Budapest-Keleti',
'Sighetu Marmaţiei',
'Wien Hauptbahnhof',
]);
});
test('a kept stop records no country', async (t) => {
const { schedule } = await buildFake(t);
for (const stop of schedule.stops) {
assert.equal(Object.hasOwn(stop, 'cc'), false, stop.name);
assert.equal(Object.hasOwn(stop, 'country'), false, stop.name);
}
});
test('a left-out trip has no runs for a reported train to take', async (t) => {
const { schedule } = await buildFake(t);
const runs = createRunIndex(schedule);
for (const tripId of ['budapest-chop', 'budapest-lviv'])
assert.deepEqual(runs.runsOf(tripId), [], tripId);
assert.ok(runs.runsOf('vienna-budapest').length > 0);
});
/** A GTFS-RT VehiclePositions message with one vehicle per position. */
function encodeVehicles(positions) {
const writer = new PbfWriter();
const header = new PbfWriter();
header.writeStringField(1, '2.0');
header.writeVarintField(3, 1_794_000_000);
writer.writeBytesField(1, header.finish());
positions.forEach(([lat, lon], i) => {
const entity = new PbfWriter();
entity.writeStringField(1, `v${i}`);
const vehicle = new PbfWriter();
const trip = new PbfWriter();
trip.writeStringField(1, `trip-${i}`);
trip.writeStringField(5, 'R');
vehicle.writeBytesField(1, trip.finish());
const position = new PbfWriter();
position.writeFloatField(1, lat);
position.writeFloatField(2, lon);
vehicle.writeBytesField(2, position.finish());
vehicle.writeVarintField(5, 1_793_999_990);
entity.writeBytesField(4, vehicle.finish());
writer.writeBytesField(2, entity.finish());
});
return writer.finish();
}
test('a live GPS position in Ukraine is dropped and counted, never sent', () => {
const snapshot = buildRailSnapshot(
fakeFeed({ url: 'https://example.invalid/vp' }),
encodeVehicles([AMSTERDAM, KYIV, SIMFEROPOL, PRZEMYSL]),
1_794_000_000_000,
);
assert.equal(snapshot.count, 2);
assert.equal(snapshot.dropped, 2);
for (const train of snapshot.trains)
assert.equal(isNoTrainPositionPoint(train.lat, train.lon), false);
});
test('an adapter position (Irish Rail XML) in Ukraine is dropped too', () => {
const position = (code, [lat, lon]) => `
<objTrainPositions>
<TrainStatus>R</TrainStatus>
<TrainLatitude>${lat}</TrainLatitude>
<TrainLongitude>${lon}</TrainLongitude>
<TrainCode>${code}</TrainCode>
<TrainDate>06 Nov 2026</TrainDate>
<PublicMessage>${code}</PublicMessage>
<Direction>Northbound</Direction>
</objTrainPositions>`;
const body = `<?xml version="1.0" encoding="utf-8"?>
<ArrayOfObjTrainPositions xmlns="http://api.irishrail.ie/realtime/">${position('A101', AMSTERDAM)}${position('A102', LVIV)}
</ArrayOfObjTrainPositions>`;
const snapshot = buildRailSnapshot(
fakeFeed({ format: 'irish-rail-xml', url: 'https://example.invalid/xml' }),
new TextEncoder().encode(body),
1_794_000_000_000,
);
assert.deepEqual(
snapshot.trains.map((train) => train.id),
['A101'],
);
assert.equal(snapshot.dropped, 1);
});
+288
View File
@@ -0,0 +1,288 @@
/**
* @module railPaths
* @description Track geometry for the Rail layer: a railway line as an indexed
* polyline measured by arc length, with projection of a position onto it and
* the point at a given arc length.
*
* Paths come from per-feed path packs (scripts/build-rail-paths.mjs), served by
* `/api/rail/paths/<feedId>`. A pack holds each unique shape as a flat
* `[lat, lon, lat, lon, ...]` array with its cumulative distances in metres.
*
* Distances are measured in one local equirectangular frame per path, centred
* on the path's middle latitude. Over the extent of a commuter network (a few
* hundred kilometres) the east-west scale error stays within a few per cent,
* which moves a projection by metres, well under what a marker shows.
*
* Pure: no Cesium, no DOM, no Node built-ins.
*/
/** Metres per degree of latitude (and of longitude at the equator). */
const M_PER_DEG = 111_320;
/** Segments per bounding box in the whole-path search index. */
const CHUNK = 32;
/** Longitude difference wrapped into (-180, 180]. */
function wrapLon(delta) {
return ((((delta + 180) % 360) + 360) % 360) - 180;
}
/**
* Build an indexed path from a flat lat/lon array.
* @param {ArrayLike<number>} coords `[lat, lon, lat, lon, ...]`, at least two points.
* @param {ArrayLike<number>} [dist] Cumulative metres per point. When absent,
* wrong-sized or decreasing, the distances are measured in the local frame.
* @returns {object|null} The path, or null when the input is not a polyline.
*/
export function createRailPath(coords, dist = null) {
const n = Math.floor((coords?.length ?? 0) / 2);
if (n < 2) return null;
const lat = new Float64Array(n);
const lon = new Float64Array(n);
let minLat = Infinity;
let maxLat = -Infinity;
for (let i = 0; i < n; i++) {
const a = Number(coords[2 * i]);
const o = Number(coords[2 * i + 1]);
if (!Number.isFinite(a) || !Number.isFinite(o) || Math.abs(a) > 90)
return null;
lat[i] = a;
lon[i] = o;
if (a < minLat) minLat = a;
if (a > maxLat) maxLat = a;
}
const lat0 = (minLat + maxLat) / 2;
const lon0 = lon[0];
const kx = M_PER_DEG * Math.cos((lat0 * Math.PI) / 180);
const ky = M_PER_DEG;
const x = new Float64Array(n);
const y = new Float64Array(n);
for (let i = 0; i < n; i++) {
x[i] = wrapLon(lon[i] - lon0) * kx;
y[i] = (lat[i] - lat0) * ky;
}
const s = new Float64Array(n);
let given = dist?.length === n && Number(dist[0]) >= 0;
for (let i = 0; given && i < n; i++) {
const d = Number(dist[i]);
if (!Number.isFinite(d) || (i > 0 && d < s[i - 1])) given = false;
else s[i] = d;
}
if (!given) {
s[0] = 0;
for (let i = 1; i < n; i++)
s[i] = s[i - 1] + Math.hypot(x[i] - x[i - 1], y[i] - y[i - 1]);
}
const chunks = Math.ceil((n - 1) / CHUNK);
const box = new Float64Array(chunks * 4);
for (let c = 0; c < chunks; c++) {
let x0 = Infinity;
let x1 = -Infinity;
let y0 = Infinity;
let y1 = -Infinity;
const end = Math.min(n - 1, (c + 1) * CHUNK);
for (let i = c * CHUNK; i <= end; i++) {
if (x[i] < x0) x0 = x[i];
if (x[i] > x1) x1 = x[i];
if (y[i] < y0) y0 = y[i];
if (y[i] > y1) y1 = y[i];
}
box[4 * c] = x0;
box[4 * c + 1] = x1;
box[4 * c + 2] = y0;
box[4 * c + 3] = y1;
}
return Object.freeze({
n,
lat,
lon,
x,
y,
s,
box,
lat0,
lon0,
kx,
ky,
length: s[n - 1],
});
}
/** Index of the segment [i, i+1] whose arc range holds `s`, clamped. */
function segmentAt(path, s) {
const arc = path.s;
let lo = 0;
let hi = path.n - 2;
if (!(s > arc[0])) return 0;
if (s >= arc[hi + 1]) return hi;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (arc[mid] <= s) lo = mid;
else hi = mid - 1;
}
return lo;
}
/**
* Squared distance from (px, py) to segment i, writing the parameter to `hit`.
* @returns {number}
*/
function segmentDistance2(path, i, px, py, hit) {
const { x, y } = path;
const dx = x[i + 1] - x[i];
const dy = y[i + 1] - y[i];
const len2 = dx * dx + dy * dy;
let t = len2 > 0 ? ((px - x[i]) * dx + (py - y[i]) * dy) / len2 : 0;
if (t < 0) t = 0;
else if (t > 1) t = 1;
const ex = x[i] + dx * t - px;
const ey = y[i] + dy * t - py;
hit.t = t;
return ex * ex + ey * ey;
}
/**
* Project a position onto a path.
*
* With a hint, only the segments whose arc range overlaps
* `[hintS - windowM, hintS + windowM]` are searched: a train known to be near
* arc `hintS` cannot jump to the far side of a loop or to a parallel branch
* that the same shape passes later. Without one, the whole path is searched.
*
* @param {object} path From createRailPath.
* @param {number} lat
* @param {number} lon
* @param {number} [hintS] Expected arc length (m).
* @param {number} [windowM] Half-width of the search window around the hint (m).
* @param {{s?: number, offsetM?: number, index?: number}} [out]
* @returns {{s: number, offsetM: number, index: number}|null}
*/
export function projectOnPath(path, lat, lon, hintS, windowM, out = {}) {
if (!path || !Number.isFinite(lat) || !Number.isFinite(lon)) return null;
const px = wrapLon(lon - path.lon0) * path.kx;
const py = (lat - path.lat0) * path.ky;
const hit = { t: 0 };
let best = Infinity;
let bestIndex = -1;
let bestT = 0;
if (Number.isFinite(hintS) && Number.isFinite(windowM) && windowM >= 0) {
const first = segmentAt(path, hintS - windowM);
const last = segmentAt(path, hintS + windowM);
for (let i = first; i <= last; i++) {
const d2 = segmentDistance2(path, i, px, py, hit);
if (d2 < best) {
best = d2;
bestIndex = i;
bestT = hit.t;
}
}
} else {
const { box } = path;
const chunks = box.length / 4;
for (let c = 0; c < chunks; c++) {
const ox = Math.max(box[4 * c] - px, 0, px - box[4 * c + 1]);
const oy = Math.max(box[4 * c + 2] - py, 0, py - box[4 * c + 3]);
if (ox * ox + oy * oy >= best) continue;
const end = Math.min(path.n - 2, (c + 1) * CHUNK - 1);
for (let i = c * CHUNK; i <= end; i++) {
const d2 = segmentDistance2(path, i, px, py, hit);
if (d2 < best) {
best = d2;
bestIndex = i;
bestT = hit.t;
}
}
}
}
if (bestIndex < 0) return null;
const s = path.s;
out.s = s[bestIndex] + (s[bestIndex + 1] - s[bestIndex]) * bestT;
out.offsetM = Math.sqrt(best);
out.index = bestIndex;
return out;
}
/**
* The position and direction of travel at an arc length, clamped to the path.
* @param {object} path From createRailPath.
* @param {number} s Arc length (m).
* @param {{lat?: number, lon?: number, bearingDeg?: number}} [out]
* @returns {{lat: number, lon: number, bearingDeg: number}|null}
*/
export function pointAtS(path, s, out = {}) {
if (!path || !Number.isFinite(s)) return null;
const clamped = Math.max(0, Math.min(path.length, s));
let i = segmentAt(path, clamped);
// A zero-length segment has no direction; borrow its neighbour's.
while (i < path.n - 2 && path.s[i + 1] === path.s[i]) i++;
const span = path.s[i + 1] - path.s[i];
const t =
span > 0 ? Math.max(0, Math.min(1, (clamped - path.s[i]) / span)) : 0;
out.lat = path.lat[i] + (path.lat[i + 1] - path.lat[i]) * t;
let lon = path.lon[i] + wrapLon(path.lon[i + 1] - path.lon[i]) * t;
if (lon > 180) lon -= 360;
else if (lon < -180) lon += 360;
out.lon = lon;
const dx = path.x[i + 1] - path.x[i];
const dy = path.y[i + 1] - path.y[i];
out.bearingDeg =
dx === 0 && dy === 0
? NaN
: ((Math.atan2(dx, dy) * 180) / Math.PI + 360) % 360;
return out;
}
/** Path-pack format this module reads. */
export const RAIL_PATH_PACK_VERSION = 1;
/**
* Wrap a decoded path pack for lookups. Shapes are indexed lazily, the first
* time a train asks for one.
* @param {object} raw The JSON body of `/api/rail/paths/<feedId>`.
* @returns {{feedId: string, shapeCount: number,
* pathAt: (index: number) => object|null,
* shapeForTrip: (tripId: string) => number|null,
* shapesForRoute: (routeId: string) => number[]}}
*/
export function createRailPathPack(raw) {
if (
!raw ||
raw.version !== RAIL_PATH_PACK_VERSION ||
typeof raw.feedId !== 'string' ||
!Array.isArray(raw.shapes) ||
!Array.isArray(raw.dist) ||
raw.dist.length !== raw.shapes.length
)
throw new TypeError('Malformed rail path pack');
const shapes = raw.shapes;
const dist = raw.dist;
const trips = raw.trips && typeof raw.trips === 'object' ? raw.trips : {};
const routes = raw.routes && typeof raw.routes === 'object' ? raw.routes : {};
const built = new Map();
const validIndex = (value) =>
Number.isInteger(value) && value >= 0 && value < shapes.length
? value
: null;
return Object.freeze({
feedId: raw.feedId,
/** When the pack was built (ISO), matched by a schedule's pathsBuiltAt. */
builtAt: typeof raw.builtAt === 'string' ? raw.builtAt : null,
shapeCount: shapes.length,
pathAt(index) {
if (validIndex(index) === null) return null;
if (!built.has(index))
built.set(index, createRailPath(shapes[index], dist[index]));
return built.get(index);
},
shapeForTrip(tripId) {
const id = String(tripId ?? '');
return id && Object.hasOwn(trips, id) ? validIndex(trips[id]) : null;
},
shapesForRoute(routeId) {
const id = String(routeId ?? '');
const list = id && Object.hasOwn(routes, id) ? routes[id] : null;
return Array.isArray(list)
? list.filter((value) => validIndex(value) !== null)
: [];
},
});
}
+163
View File
@@ -0,0 +1,163 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
RAIL_PATH_PACK_VERSION,
createRailPath,
createRailPathPack,
pointAtS,
projectOnPath,
} from './railPaths.js';
const M_PER_DEG = 111_320;
/** An L: 1 km east along the equator, then 1 km north. */
function lPath() {
const dLon = 1000 / M_PER_DEG;
return createRailPath([0, 0, 0, dLon, dLon, dLon]);
}
/**
* A hairpin: 2 km east along lat 0, a 100 m step north, then 2 km back west.
* The two legs are parallel and 100 m apart, like the two sides of a loop.
*/
function hairpin() {
const d = 1 / M_PER_DEG;
return createRailPath([0, 0, 0, 2000 * d, 100 * d, 2000 * d, 100 * d, 0]);
}
test('a path measures its arc length in metres', () => {
const path = lPath();
assert.equal(path.n, 3);
assert.ok(Math.abs(path.length - 2000) < 0.5, String(path.length));
assert.equal(createRailPath([0, 0]), null, 'one point is not a path');
assert.equal(createRailPath([0, 0, Number.NaN, 1]), null);
assert.equal(createRailPath(null), null);
});
test('given distances are used, and bad ones are measured instead', () => {
const coords = [0, 0, 0, 0.01, 0, 0.02];
assert.deepEqual([...createRailPath(coords, [0, 5, 10]).s], [0, 5, 10]);
const measured = createRailPath(coords, [0, 10, 5]);
assert.ok(Math.abs(measured.s[2] - 2 * 0.01 * M_PER_DEG) < 0.5);
const wrongSize = createRailPath(coords, [0, 1]);
assert.ok(wrongSize.s[1] > 1000);
});
test('projection finds the arc and the offset from the line', () => {
const path = lPath();
const d = 1 / M_PER_DEG;
const hit = projectOnPath(path, 20 * d, 500 * d);
assert.ok(Math.abs(hit.s - 500) < 0.5, String(hit.s));
assert.ok(Math.abs(hit.offsetM - 20) < 0.5, String(hit.offsetM));
assert.equal(hit.index, 0);
const second = projectOnPath(path, 300 * d, 1030 * d);
assert.ok(Math.abs(second.s - 1300) < 0.5, String(second.s));
assert.ok(Math.abs(second.offsetM - 30) < 0.5);
assert.equal(second.index, 1);
const beyond = projectOnPath(path, 2000 * d, 1000 * d);
assert.ok(Math.abs(beyond.s - 2000) < 0.5, 'clamped to the end');
assert.ok(Math.abs(beyond.offsetM - 1000) < 1);
assert.equal(projectOnPath(path, Number.NaN, 0), null);
assert.equal(projectOnPath(null, 0, 0), null);
});
test('a hint keeps the projection on its own side of a loop', () => {
const path = hairpin();
const d = 1 / M_PER_DEG;
// 45 m north of the outbound leg, 55 m south of the return leg: the
// whole-path answer is the outbound leg...
const point = [45 * d, 1000 * d];
const free = projectOnPath(path, ...point);
assert.ok(Math.abs(free.s - 1000) < 1, String(free.s));
// ...but a train known to be on the return leg stays on it.
const hinted = projectOnPath(path, ...point, 3100, 500);
assert.ok(Math.abs(hinted.s - 3100) < 1, String(hinted.s));
assert.ok(Math.abs(hinted.offsetM - 55) < 0.5);
});
test('the whole-path search agrees with a brute-force scan on a long path', () => {
const coords = [];
for (let i = 0; i < 500; i++)
coords.push(40 + Math.sin(i / 20) * 0.01, -74 + i * 0.001);
const path = createRailPath(coords);
for (const [lat, lon] of [
[40.004, -73.9],
[40.02, -73.75],
[39.99, -73.6],
]) {
const fast = projectOnPath(path, lat, lon);
let best = Infinity;
for (let i = 0; i < path.n - 1; i++) {
const hit = projectOnPath(
path,
lat,
lon,
(path.s[i] + path.s[i + 1]) / 2,
0,
);
best = Math.min(best, hit.offsetM);
}
assert.ok(
Math.abs(fast.offsetM - best) < 1e-6,
`${fast.offsetM} vs ${best}`,
);
}
});
test('the point at an arc length carries the direction of travel', () => {
const path = lPath();
const d = 1 / M_PER_DEG;
const east = pointAtS(path, 250);
assert.ok(Math.abs(east.lon - 250 * d) < 1e-9);
assert.equal(east.lat, 0);
assert.ok(Math.abs(east.bearingDeg - 90) < 1e-6, String(east.bearingDeg));
const north = pointAtS(path, 1500);
assert.ok(Math.abs(north.lat - 500 * d) < 1e-6);
assert.ok(
Math.abs(north.bearingDeg - 0) < 1e-6 ||
Math.abs(north.bearingDeg - 360) < 1e-6,
);
const out = {};
assert.equal(pointAtS(path, -50, out), out, 'writes into the caller scratch');
assert.equal(out.lat, 0);
assert.equal(out.lon, 0);
assert.ok(Math.abs(pointAtS(path, 9e9).lat - 1000 * d) < 1e-9);
assert.equal(pointAtS(path, Number.NaN), null);
});
test('a pack resolves trips and routes to shapes and builds paths lazily', () => {
const raw = {
version: RAIL_PATH_PACK_VERSION,
feedId: 'grt-ion',
shapes: [
[43.4, -80.5, 43.41, -80.5],
[43.4, -80.5, 43.4, -80.49],
],
dist: [
[0, 1113.2],
[0, 808.9],
],
trips: { t1: 0, t2: 1, bad: 7 },
routes: { 301: [0, 1, 9] },
};
const pack = createRailPathPack(raw);
assert.equal(pack.feedId, 'grt-ion');
assert.equal(pack.shapeCount, 2);
assert.equal(pack.shapeForTrip('t1'), 0);
assert.equal(pack.shapeForTrip('bad'), null, 'out-of-range index');
assert.equal(pack.shapeForTrip('constructor'), null, 'no prototype keys');
assert.equal(pack.shapeForTrip(''), null);
assert.deepEqual(pack.shapesForRoute('301'), [0, 1]);
assert.deepEqual(pack.shapesForRoute('302'), []);
const path = pack.pathAt(1);
assert.equal(path.length, 808.9);
assert.equal(pack.pathAt(1), path, 'built once');
assert.equal(pack.pathAt(5), null);
for (const broken of [
null,
{ ...raw, version: 2 },
{ ...raw, feedId: 3 },
{ ...raw, dist: [] },
])
assert.throws(() => createRailPathPack(broken), TypeError);
});
+518
View File
@@ -0,0 +1,518 @@
/**
* @module railProxy
* @description Pure server-side mechanics for the `/api/rail` proxy.
*
* The upstream mechanics are the Transit proxy's (src/data/transitProxy.js):
* the same decoder, timestamp repair, redirect rule, conditional headers, cache
* windows and backoff ladder. What differs is which feeds are reachable and
* what leaves the server: only vehicles a feed's classifier recognises as
* trains, each tagged with its category, mode and position source. On top of
* that come the Irish Rail XML and SIRI-VM adapters and trip updates. Every
* train is placed where its operator reports it. Every feed is open: no
* request carries a credential.
*/
import {
TRANSIT_ADMISSION_MAX_GLOBAL,
TRANSIT_PROXY_MAX_BODY_BYTES,
TRANSIT_PROXY_STALE_MAX_MS,
TRANSIT_PROXY_TTL_MS,
TransitFeedShapeError,
buildTransitSnapshot,
} from './transitProxy.js';
import { GTFS_INCREMENTALITY_FULL_DATASET } from './gtfsRealtime.js';
import {
RAIL_FEED_REGISTRY,
classifyTrain,
getRailFeed,
railFeedHasTripUpdates,
railFeedPositions,
} from './railFeeds.js';
import { decodeIrishRailTrains } from './irishRail.js';
import { decodeSiriVehicleMonitoring } from './railSiriVm.js';
import { decodeTripUpdates } from './railTripUpdates.js';
import { isNoTrainPositionPoint } from './railNoPositions.js';
/**
* Text formats with their own adapter, by registry `format`. Each turns the
* document and the fetch time (epoch s) into `{entityCount, truncated,
* vehicles, feedTimestamp?}`.
*/
const TEXT_ADAPTERS = Object.freeze({
'irish-rail-xml': decodeIrishRailTrains,
'siri-vm': decodeSiriVehicleMonitoring,
});
export {
TRANSIT_ADMISSION_MAX_PER_FEED as RAIL_ADMISSION_MAX_PER_FEED,
TRANSIT_ADMISSION_WINDOW_MS as RAIL_ADMISSION_WINDOW_MS,
TRANSIT_PROXY_TIMEOUT_MS as RAIL_PROXY_TIMEOUT_MS,
TransitFeedShapeError as RailFeedShapeError,
isAcceptableTransitUpstreamUrl as isAcceptableRailUpstreamUrl,
nextTransitBackoffMs as nextRailBackoffMs,
} from './transitProxy.js';
/**
* Largest path or schedule pack the proxy serves or reads (bytes). The
* biggest national packs built so far were 14 to 16 MB; anything far beyond
* that is a build fault, not a pack to ship. A pack over the cap is refused
* with its own answer, not taken for a missing one.
*/
export const RAIL_PATH_PACK_MAX_BYTES = 24 * 1024 * 1024;
/**
* Default freshness window for trip updates. Predictions move more slowly
* than positions, and several operators refresh them every 30 to 60 s.
*/
export const RAIL_TRIP_UPDATES_TTL_MS = 30_000;
/** Shortest window a registry entry may ask for. */
export const RAIL_MIN_TTL_MS = 5_000;
/** Trips one `/updates` answer carries, after the rail filter. */
export const RAIL_UPDATES_MAX_TRIPS = 3_000;
/** Stop updates one `/updates` answer carries, across all its trips. */
export const RAIL_UPDATES_MAX_STOPS = 40_000;
/**
* Freshness window for one route of a feed: its own interval when the
* registry sets one (never below RAIL_MIN_TTL_MS), else the default.
* @param {object} feed
* @param {'trains'|'updates'} route
* @returns {number} ms
*/
export function railFeedTtlMs(feed, route = 'trains') {
const own =
route === 'updates' ? feed?.tripUpdatesIntervalMs : feed?.pollIntervalMs;
if (Number.isFinite(own) && own > 0) return Math.max(RAIL_MIN_TTL_MS, own);
return route === 'updates' ? RAIL_TRIP_UPDATES_TTL_MS : TRANSIT_PROXY_TTL_MS;
}
/**
* Largest byte cap a feed may set for itself (`maxBodyBytes`): nothing
* larger is read into memory.
*/
export const RAIL_PROXY_MAX_BODY_CEILING_BYTES = 64 * 1024 * 1024;
/**
* Upstream byte cap for a feed: its own cap when it sets one (never above
* RAIL_PROXY_MAX_BODY_CEILING_BYTES), else the Transit proxy's 8 MiB.
* @param {object} feed
* @returns {number}
*/
export function railMaxBodyBytes(feed) {
const own = feed?.maxBodyBytes;
return Number.isFinite(own) && own > 0
? Math.min(own, RAIL_PROXY_MAX_BODY_CEILING_BYTES)
: TRANSIT_PROXY_MAX_BODY_BYTES;
}
/**
* Classify a cache entry against a freshness window: the Transit policy with
* a per-feed TTL. The serve-stale window is the same ten minutes.
* @param {{at: number}|null|undefined} entry
* @param {number} now
* @param {number} [ttlMs]
* @returns {'none'|'fresh'|'stale'|'expired'}
*/
export function railCacheStateFor(entry, now, ttlMs = TRANSIT_PROXY_TTL_MS) {
if (!entry || !Number.isFinite(entry.at)) return 'none';
const age = now - entry.at;
if (age < ttlMs) return 'fresh';
if (age < Math.max(TRANSIT_PROXY_STALE_MAX_MS, ttlMs)) return 'stale';
return 'expired';
}
/**
* Resolve `/trains/<feedId>`, `/paths/<feedId>`, `/schedule/<feedId>` or
* `/updates/<feedId>` (the path after the `/api/rail` mount) to a pollable
* feed. Anything else (another route, an unknown or disabled id, an
* `/updates` request for a feed with no trip updates, path tricks) resolves
* to null and the caller 404s.
* @param {string} url Request URL relative to the mount point.
* @returns {{ route: 'feeds' } |
* { route: 'trains'|'paths'|'schedule'|'updates', feed: object } | null}
*/
export function resolveRailRoute(url) {
const pathname = String(url || '').split('?')[0];
if (pathname === '/feeds' || pathname === '/feeds/')
return { route: 'feeds' };
const match = /^\/(trains|paths|schedule|updates)\/([^/]+)\/?$/.exec(
pathname,
);
if (!match) return null;
let id;
try {
id = decodeURIComponent(match[2]);
} catch {
return null;
}
const feed = getRailFeed(id);
if (!feed) return null;
if (match[1] === 'updates' && !railFeedHasTripUpdates(feed)) return null;
return { route: match[1], feed };
}
/**
* Classify decoded vehicles and keep only trains. A position where no train
* is placed (src/data/railNoPositions.js: Ukraine) is dropped first, so it
* is only counted and never leaves the server.
*/
function keepTrains(feed, vehicles, routeFor = null) {
const positionSource = railFeedPositions(feed) === 'event' ? 'event' : 'gps';
const trains = [];
for (const vehicle of vehicles) {
if (isNoTrainPositionPoint(vehicle.lat, vehicle.lon)) continue;
const routeId = resolvedRouteId(feed, vehicle, routeFor);
const kind = classifyTrain(feed, routeId, { tripId: vehicle.tripId });
if (!kind) continue;
trains.push({
...vehicle,
routeId,
category: kind.category,
mode: kind.mode,
...(kind.tourist ? { tourist: true } : {}),
positionSource,
});
}
return trains;
}
/**
* The route a record is classified by. A feed with `routesFrom: 'schedule'`
* publishes trips whose route is missing or not the one its classifier reads;
* `routeFor(tripId)` answers from the feed's schedule pack, whose route ids
* the build wrote to carry the product. A trip the pack does not know keeps
* what the feed said, which its classifier then drops.
*/
function resolvedRouteId(feed, record, routeFor) {
if (feed?.routesFrom !== 'schedule' || typeof routeFor !== 'function')
return record.routeId;
return (record.tripId && routeFor(record.tripId)) || record.routeId;
}
/**
* For a feed whose positions name each train by its number
* (`positionsByTrainNumber`, Irish Rail's train codes), the record takes
* the schedule pack's trip that number is running now (`tripFor`, given the
* number and the record, whose stated delay moves its timetable), so the
* train is drawn on its line and its card lists its stops; the number stays
* as `trainNumber`. A number the timetable does not place keeps its own id.
*/
function byTrainNumber(feed, vehicles, tripFor) {
if (feed?.positionsByTrainNumber !== true || typeof tripFor !== 'function')
return vehicles;
return vehicles.map((vehicle) => {
const tripId = vehicle.tripId ? tripFor(vehicle.tripId, vehicle) : null;
return tripId
? { ...vehicle, tripId, trainNumber: vehicle.tripId }
: vehicle;
});
}
/**
* For a feed whose header clock cannot be trusted (`headerClock:
* 'untrusted'`, Renfe: local time written as UTC), a time the Transit
* ladder took from the header is replaced by the fetch time and labelled so.
*/
function distrustHeaderClock(feed, vehicles, now) {
if (feed?.headerClock !== 'untrusted') return vehicles;
const fetchedS = Math.floor(now / 1000);
return vehicles.map((vehicle) =>
vehicle.timestampSource === 'header'
? { ...vehicle, timestamp: fetchedS, timestampSource: 'fetch' }
: vehicle,
);
}
/**
* An upstream answer that is no document of the feed's: a failed fetch, not
* a reason to distrust the last good snapshot, which the service keeps
* serving while it is young enough (`code: 'UPSTREAM_UNREADABLE'`, a 502).
* @param {string} detail
* @returns {Error & {code: string}}
*/
function unreadableUpstream(detail) {
const failure = new Error(`upstream document unreadable: ${detail}`);
failure.code = 'UPSTREAM_UNREADABLE';
return failure;
}
/**
* Run a GTFS-RT decode and refuse what is not a FeedMessage: bytes protobuf
* cannot read (an HTML or JSON error page served as 200), and a message
* without a FeedHeader or its gtfs_realtime_version, which every
* FeedMessage carries. An empty body (a 200 or 204 with nothing in it)
* decodes to exactly that, and would otherwise read as a full snapshot with
* no trains. A DIFFERENTIAL feed's refusal passes through as it is.
* @template {{version: string|null}} T
* @param {() => T} decode
* @returns {T}
*/
function decodeGtfsRealtime(decode) {
let decoded;
try {
decoded = decode();
} catch (error) {
if (error instanceof TransitFeedShapeError) throw error;
throw unreadableUpstream(error?.message || String(error));
}
if (decoded?.version === null || decoded?.version === undefined)
throw unreadableUpstream('no FeedHeader with a gtfs_realtime_version');
return decoded;
}
/**
* Decode upstream bytes into the JSON snapshot the Rail layer consumes.
*
* For GTFS-RT feeds, decoding, the refusal of DIFFERENTIAL feeds and the
* report-time ladder are the Transit proxy's; a body that is no FeedMessage
* (no header, no version, not protobuf) is refused as unreadable
* (decodeGtfsRealtime). Irish Rail's XML and SIRI-VM go through their
* adapters. The classifier then runs on every vehicle: a vehicle it does
* not recognise (a bus in a mixed feed, a shuttle, an out-of-service vehicle
* with no route in a mixed feed) is dropped and only counted, so the browser
* receives trains and nothing else. Every train carries `positionSource`:
* `gps` for GTFS-RT positions, `event` for signalling events.
*
* @param {object} feed Rail registry entry.
* @param {Uint8Array|ArrayBuffer} bytes Raw upstream body.
* @param {number} [now=Date.now()] Fetch time (ms epoch).
* @param {{routeFor?: Function|null, tripFor?: Function|null}} [lookups]
* `routeFor` resolves a trip's route from the schedule pack, and
* `tripFor(number, record)` a train number's running trip
* (`positionsByTrainNumber`).
* @returns {{ feedId: string, name: string, fetchedAt: number, feedTimestamp: number|null,
* version: string|null, entityCount: number, truncated: boolean, count: number,
* dropped: number, positions: string, trains: object[] }}
*/
export function buildRailSnapshot(
feed,
bytes,
now = Date.now(),
{ routeFor = null, tripFor = null } = {},
) {
const adapter = Object.hasOwn(TEXT_ADAPTERS, feed?.format)
? TEXT_ADAPTERS[feed.format]
: null;
if (adapter) {
const view = bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes);
const text = new TextDecoder('utf-8').decode(view);
let decoded;
try {
decoded = adapter(text, Math.floor(now / 1000));
} catch (error) {
// An unreadable document (an error page served as 200, a truncated
// body) is a failed fetch, not a reason to distrust the last good one.
throw unreadableUpstream(error?.message || String(error));
}
const trains = keepTrains(
feed,
byTrainNumber(feed, decoded.vehicles, tripFor),
routeFor,
);
return {
feedId: feed.id,
name: feed.name,
fetchedAt: now,
feedTimestamp: Number.isFinite(decoded.feedTimestamp)
? decoded.feedTimestamp
: null,
version: null,
entityCount: decoded.entityCount,
truncated: decoded.truncated,
count: trains.length,
dropped: decoded.vehicles.length - trains.length,
positions: railFeedPositions(feed),
trains,
};
}
const decoded = decodeGtfsRealtime(() =>
buildTransitSnapshot(feed, bytes, now),
);
const trains = keepTrains(
feed,
distrustHeaderClock(feed, decoded.vehicles, now),
routeFor,
);
return {
feedId: decoded.feedId,
name: decoded.name,
fetchedAt: decoded.fetchedAt,
feedTimestamp:
feed.headerClock === 'untrusted' ? null : decoded.feedTimestamp,
version: decoded.version,
entityCount: decoded.entityCount,
truncated: decoded.truncated,
count: trains.length,
dropped: decoded.vehicles.length - trains.length,
positions: railFeedPositions(feed),
trains,
};
}
/**
* Decode a TripUpdates feed into the `/api/rail/updates/<feedId>` body: one
* record per rail trip, filtered with the same classifier as the positions
* (a feed's buses never leave the server), capped in trips and stops.
*
* A DIFFERENTIAL feed is refused, for the same reason positions are: an
* unmentioned trip would read as one with no prediction. A body that is no
* FeedMessage is refused as unreadable (decodeGtfsRealtime).
*
* @param {object} feed Rail registry entry.
* @param {Uint8Array|ArrayBuffer} bytes Raw GTFS-RT FeedMessage.
* @param {number} [now=Date.now()] Fetch time (ms epoch).
* @param {{routeFor?: ((tripId: string) => string|null)|null}} [lookups]
* `routeFor` is the schedule pack's trip-to-route lookup, for a feed
* whose trips do not carry the route its classifier reads (`routesFrom:
* 'schedule'`).
* @returns {{ feedId: string, name: string, fetchedAt: number,
* feedTimestamp: number|null, entityCount: number, truncated: boolean,
* count: number, dropped: number, trips: object[] }}
*/
export function buildRailTripUpdatesSnapshot(
feed,
bytes,
now = Date.now(),
{ routeFor = null } = {},
) {
const decoded = decodeGtfsRealtime(() => decodeTripUpdates(bytes));
if (decoded.incrementality !== GTFS_INCREMENTALITY_FULL_DATASET) {
throw new TransitFeedShapeError(
`feed is differential (incrementality ${decoded.incrementality})`,
'differential',
);
}
const trips = [];
let truncated = decoded.truncated;
let stops = 0;
let rail = 0;
for (const trip of decoded.trips) {
const routeId = resolvedRouteId(feed, trip, routeFor);
const kind = classifyTrain(feed, routeId, { tripId: trip.tripId });
if (!kind) continue;
rail += 1;
if (
trips.length >= RAIL_UPDATES_MAX_TRIPS ||
stops + trip.stops.length > RAIL_UPDATES_MAX_STOPS
) {
truncated = true;
continue;
}
stops += trip.stops.length;
trips.push({
...trip,
routeId,
category: kind.category,
mode: kind.mode,
...(kind.tourist ? { tourist: true } : {}),
});
}
return {
feedId: feed.id,
name: feed.name,
fetchedAt: now,
feedTimestamp: decoded.timestamp,
entityCount: decoded.entityCount,
truncated,
count: trips.length,
dropped: decoded.trips.length - rail,
trips,
};
}
/**
* Upstream request headers. Same obligations as Transit's (conditional
* validators, gzip, a User-Agent that says who is asking, any identify-yourself
* header the registry names), with this proxy's own name in the User-Agent.
* A feed's own `headers` (an Accept for an XML feed) come last.
* @param {object} feed Registry entry.
* @param {{etag?: string|null, lastModified?: string|null}} [validators]
* @returns {Record<string, string>}
*/
export function railUpstreamHeaders(feed, validators = null) {
return {
'User-Agent':
'gods-eye-view-rail-proxy/1.0 (+https://github.com/bilawalsidhu/gods-eye-view)',
Accept: 'application/x-protobuf, application/octet-stream;q=0.9, */*;q=0.1',
'Accept-Encoding': 'gzip',
...(validators?.etag ? { 'If-None-Match': validators.etag } : {}),
...(validators?.lastModified
? { 'If-Modified-Since': validators.lastModified }
: {}),
...(feed?.headers || {}),
};
}
/**
* Upstream requests one minute may need when every feed in the registry is
* polled at once: one per freshness window for each feed's positions and
* trip updates.
* @param {ReadonlyArray<object>} [feeds]
* @returns {number}
*/
export function railNaturalRequestsPerMinute(feeds = RAIL_FEED_REGISTRY) {
let total = 0;
for (const feed of feeds) {
if (feed?.defaultEnabled !== true) continue;
if (feed.url) total += Math.ceil(60_000 / railFeedTtlMs(feed, 'trains'));
if (railFeedHasTripUpdates(feed))
total += Math.ceil(60_000 / railFeedTtlMs(feed, 'updates'));
}
return total;
}
/**
* The process-wide admission cap (requests per admission window, all
* operators): the natural load of every enabled feed, half again on top for
* retries on the backoff ladder, never below the Transit proxy's own cap
* (TRANSIT_ADMISSION_MAX_GLOBAL, 40). It is a backstop against a bug or a
* runaway client, not the protection an operator relies on: that is each
* feed's freshness window and the per-feed admission count. Transit's fixed
* cap alone sat below the natural load once the registry passed about ten
* feeds polled every 15 s, so a viewer of several regions got 503s from a
* healthy proxy.
* @param {ReadonlyArray<object>} [feeds]
* @returns {number}
*/
export function railAdmissionGlobalMax(feeds = RAIL_FEED_REGISTRY) {
return Math.max(
TRANSIT_ADMISSION_MAX_GLOBAL,
Math.ceil(railNaturalRequestsPerMinute(feeds) * 1.5),
);
}
/**
* Response headers for a snapshot. `X-GEV-Cache` mirrors the other proxies
* (HIT / MISS / INFLIGHT / STALE-ERROR); `X-Rail-Contact` is when the operator
* last ANSWERED, which a revalidated (304) feed advances without changing the
* body's own fetch time. `max-age` is what is left of the feed's own
* freshness window once the snapshot is `ageMs` old, so a HIT near the end
* of the window is not kept a whole window more downstream.
* @param {'HIT'|'MISS'|'INFLIGHT'|'STALE-ERROR'} cacheState
* @param {string} [upstreamHost]
* @param {number|null} [contactedAt]
* @param {number} [ttlMs]
* @param {number} [ageMs] How old the snapshot is (ms); 0 for a fresh one.
* @returns {Record<string, string>}
*/
export function railResponseHeaders(
cacheState,
upstreamHost = '',
contactedAt = null,
ttlMs = TRANSIT_PROXY_TTL_MS,
ageMs = 0,
) {
const age = Number.isFinite(ageMs) && ageMs > 0 ? ageMs : 0;
return {
'Content-Type': 'application/json; charset=utf-8',
'Cache-Control':
cacheState === 'STALE-ERROR'
? 'no-store'
: `public, max-age=${Math.max(0, Math.ceil((ttlMs - age) / 1000))}`,
'X-GEV-Cache': cacheState,
...(upstreamHost ? { 'X-Rail-Upstream': upstreamHost } : {}),
...(Number.isFinite(contactedAt)
? { 'X-Rail-Contact': String(Math.floor(contactedAt)) }
: {}),
};
}
+586
View File
@@ -0,0 +1,586 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { PbfWriter } from 'pbf';
import {
RAIL_UPDATES_MAX_TRIPS,
RailFeedShapeError,
buildRailSnapshot,
buildRailTripUpdatesSnapshot,
railAdmissionGlobalMax,
railNaturalRequestsPerMinute,
railCacheStateFor,
railFeedTtlMs,
railMaxBodyBytes,
railResponseHeaders,
railUpstreamHeaders,
resolveRailRoute,
} from './railProxy.js';
import {
RAIL_FEED_REGISTRY,
getRailFeed,
getRegisteredRailFeed,
railFeedPositions,
} from './railFeeds.js';
/** A minimal TripUpdates FeedMessage: `[tripId, routeId, delay]` rows. */
function encodeTripUpdates(rows, { incrementality = null } = {}) {
const writer = new PbfWriter();
const header = new PbfWriter();
header.writeStringField(1, '2.0');
if (incrementality !== null) header.writeVarintField(2, incrementality);
header.writeVarintField(3, 1_794_000_000);
writer.writeBytesField(1, header.finish());
for (const [tripId, routeId, delay, stopCount = 1] of rows) {
const entity = new PbfWriter();
entity.writeStringField(1, tripId);
const update = new PbfWriter();
const trip = new PbfWriter();
trip.writeStringField(1, tripId);
if (routeId) trip.writeStringField(5, routeId);
update.writeBytesField(1, trip.finish());
for (let i = 0; i < stopCount; i += 1) {
const stop = new PbfWriter();
stop.writeVarintField(1, i + 1);
const arrival = new PbfWriter();
arrival.writeVarintField(1, delay);
stop.writeBytesField(2, arrival.finish());
stop.writeStringField(4, `s${i}`);
update.writeBytesField(2, stop.finish());
}
entity.writeBytesField(3, update.finish());
writer.writeBytesField(2, entity.finish());
}
return writer.finish();
}
const IRISH_XML = `<?xml version="1.0" encoding="utf-8"?>
<ArrayOfObjTrainPositions xmlns="http://api.irishrail.ie/realtime/">
<objTrainPositions>
<TrainStatus>R</TrainStatus>
<TrainLatitude>54.0</TrainLatitude>
<TrainLongitude>-6.4</TrainLongitude>
<TrainCode>A118</TrainCode>
<TrainDate>06 Nov 2026</TrainDate>
<PublicMessage>A118\\n14:50 - Dublin Connolly to Belfast (0 mins late)\\nDeparted Dundalk next stop Newry</PublicMessage>
<Direction>Northbound</Direction>
</objTrainPositions>
<objTrainPositions>
<TrainStatus>R</TrainStatus>
<TrainLatitude>53.29</TrainLatitude>
<TrainLongitude>-6.13</TrainLongitude>
<TrainCode>E931</TrainCode>
<TrainDate>06 Nov 2026</TrainDate>
<PublicMessage>E931\\n15:03 - Greystones to Howth (16 mins late)\\nDeparted Sutton next stop Howth</PublicMessage>
<Direction>Northbound</Direction>
</objTrainPositions>
</ArrayOfObjTrainPositions>`;
/** A minimal FeedMessage whose vehicles carry a trip route id. */
function encodeFeed({ incrementality = null, entities = [] }) {
const writer = new PbfWriter();
const header = new PbfWriter();
header.writeStringField(1, '2.0');
if (incrementality !== null) header.writeVarintField(2, incrementality);
header.writeVarintField(3, 1_700_000_000);
writer.writeBytesField(1, header.finish());
for (const entity of entities) {
const feedEntity = new PbfWriter();
feedEntity.writeStringField(1, entity.id);
const vehicle = new PbfWriter();
if (entity.route) {
const trip = new PbfWriter();
trip.writeStringField(1, `trip-${entity.id}`);
trip.writeStringField(5, entity.route);
vehicle.writeBytesField(1, trip.finish());
}
const position = new PbfWriter();
position.writeFloatField(1, entity.lat ?? 42.35);
position.writeFloatField(2, entity.lon ?? -71.06);
vehicle.writeBytesField(2, position.finish());
vehicle.writeVarintField(5, 1_699_999_990);
const descriptor = new PbfWriter();
descriptor.writeStringField(1, entity.id);
// A vendor extension (field 1999), as Adelaide Metro sends: it must be
// skipped, not fail the snapshot.
const extension = new PbfWriter();
extension.writeVarintField(1, 1);
descriptor.writeBytesField(1999, extension.finish());
vehicle.writeBytesField(8, descriptor.finish());
feedEntity.writeBytesField(4, vehicle.finish());
writer.writeBytesField(2, feedEntity.finish());
}
return writer.finish();
}
test('route resolution admits only the catalog and enabled rail feeds', () => {
assert.deepEqual(resolveRailRoute('/feeds'), { route: 'feeds' });
assert.deepEqual(resolveRailRoute('/feeds/?x=1'), { route: 'feeds' });
assert.equal(resolveRailRoute('/trains/mbta-rail')?.feed?.id, 'mbta-rail');
assert.equal(resolveRailRoute('/trains/mbta-rail/')?.feed?.id, 'mbta-rail');
assert.equal(resolveRailRoute('/trains/vre'), null, 'not registered');
assert.equal(
resolveRailRoute('/trains/mbta'),
null,
'a transit id is not a rail id',
);
assert.equal(resolveRailRoute('/vehicles/mbta-rail'), null);
assert.equal(resolveRailRoute('/trains/mbta-rail/extra'), null);
assert.equal(resolveRailRoute('/trains/..%2F..%2Fetc'), null);
assert.equal(resolveRailRoute('/trains/%E0%A4%A'), null);
assert.equal(resolveRailRoute(''), null);
assert.equal(resolveRailRoute(undefined), null);
});
test('path packs resolve for enabled rail feeds only', () => {
const route = resolveRailRoute('/paths/mta-lirr');
assert.equal(route?.route, 'paths');
assert.equal(route?.feed?.id, 'mta-lirr');
assert.equal(resolveRailRoute('/paths/mta-lirr/?v=1')?.route, 'paths');
assert.equal(resolveRailRoute('/trains/mta-lirr')?.route, 'trains');
assert.equal(resolveRailRoute('/paths/vre'), null, 'not registered');
assert.equal(resolveRailRoute('/paths/unknown'), null);
assert.equal(resolveRailRoute('/paths/'), null);
assert.equal(resolveRailRoute('/paths/mta-lirr/extra'), null);
assert.equal(resolveRailRoute('/paths/..%2F..%2Fetc%2Fpasswd'), null);
assert.equal(resolveRailRoute('/paths/mta-lirr.json'), null);
assert.equal(resolveRailRoute('/paths/%E0%A4%A'), null);
});
test('snapshots keep classified trains only and count what was dropped', () => {
const feed = getRailFeed('mbta-rail');
const snapshot = buildRailSnapshot(
feed,
encodeFeed({
entities: [
{ id: 'cr1', route: 'CR-Lowell' },
{ id: 'red1', route: 'Red' },
{ id: 'grn1', route: 'Green-D' },
{ id: 'bus1', route: '39' },
{ id: 'none' },
],
}),
1_700_000_000_000,
);
assert.equal(snapshot.feedId, 'mbta-rail');
assert.equal(snapshot.entityCount, 5);
assert.equal(snapshot.count, 3);
assert.equal(snapshot.dropped, 2);
assert.deepEqual(
snapshot.trains.map(({ id, category, mode }) => [id, category, mode]),
[
['cr1', 'local', 'rail'],
['red1', 'transit', 'metro'],
['grn1', 'transit', 'tram'],
],
);
assert.equal(snapshot.trains[0].timestampSource, 'vehicle');
assert.equal(snapshot.vehicles, undefined, 'buses never leave the server');
});
test('a rail-only feed keeps vehicles that carry no route', () => {
const snapshot = buildRailSnapshot(
getRailFeed('mta-metro-north'),
encodeFeed({ entities: [{ id: 'mnr1' }] }),
);
assert.equal(snapshot.count, 1);
assert.equal(snapshot.trains[0].category, 'local');
});
test('a differential feed is refused, as in the Transit proxy', () => {
assert.throws(
() =>
buildRailSnapshot(
getRailFeed('mbta-rail'),
encodeFeed({ incrementality: 1, entities: [] }),
),
RailFeedShapeError,
);
});
test('a body that is no FeedMessage is unreadable, not an empty snapshot', () => {
const feed = getRailFeed('mbta-rail');
const headerless = new PbfWriter();
const entity = new PbfWriter();
entity.writeStringField(1, 'x');
headerless.writeBytesField(2, entity.finish());
const versionless = new PbfWriter();
const header = new PbfWriter();
header.writeVarintField(3, 1_700_000_000);
versionless.writeBytesField(1, header.finish());
for (const [label, bytes] of [
['empty', new Uint8Array(0)],
['no header', headerless.finish()],
['no version', versionless.finish()],
['html', new TextEncoder().encode('<!DOCTYPE html><html></html>')],
['json', new TextEncoder().encode('{"error":"quota"}')],
]) {
for (const build of [buildRailSnapshot, buildRailTripUpdatesSnapshot])
assert.throws(
() => build(feed, bytes, 1_700_000_000_000),
(error) =>
error.code === 'UPSTREAM_UNREADABLE' &&
!(error instanceof RailFeedShapeError),
`${build.name}: ${label}`,
);
}
// A FeedMessage with its header and nothing else is a real empty feed.
const quiet = encodeFeed({ entities: [] });
assert.equal(buildRailSnapshot(feed, quiet).count, 0);
assert.equal(buildRailTripUpdatesSnapshot(feed, quiet).count, 0);
});
test('upstream headers identify the rail proxy and let a feed override Accept', () => {
const plain = railUpstreamHeaders(getRailFeed('mbta-rail'), {
etag: '"abc"',
lastModified: 'Thu, 01 Jan 2026 00:00:00 GMT',
});
assert.match(plain['User-Agent'], /gods-eye-view-rail-proxy/);
assert.equal(plain['If-None-Match'], '"abc"');
assert.equal(plain['If-Modified-Since'], 'Thu, 01 Jan 2026 00:00:00 GMT');
const adelaide = railUpstreamHeaders(getRegisteredRailFeed('adelaide-metro'));
assert.equal(adelaide.Accept, 'application/x-google-protobuf');
assert.equal(
railUpstreamHeaders(getRailFeed('irish-rail')).Accept,
'text/xml, application/xml;q=0.9',
);
});
test('response headers report cache state and operator contact', () => {
const fresh = railResponseHeaders('MISS', 'cdn.mbta.com', 1_700_000_000_123);
assert.equal(fresh['X-GEV-Cache'], 'MISS');
assert.equal(fresh['X-Rail-Upstream'], 'cdn.mbta.com');
assert.equal(fresh['X-Rail-Contact'], '1700000000123');
assert.match(fresh['Cache-Control'], /max-age=15/);
assert.equal(railResponseHeaders('STALE-ERROR')['Cache-Control'], 'no-store');
assert.equal(railResponseHeaders('HIT')['X-Rail-Contact'], undefined);
});
test('GTFS trains say their positions are GPS', () => {
const snapshot = buildRailSnapshot(
getRailFeed('mbta-rail'),
encodeFeed({ entities: [{ id: 'cr1', route: 'CR-Lowell' }] }),
);
assert.equal(snapshot.positions, 'gps');
assert.equal(snapshot.trains[0].positionSource, 'gps');
});
test('Irish Rail XML becomes classified, event-positioned trains', () => {
const now = 1_794_000_000_000;
const snapshot = buildRailSnapshot(
getRailFeed('irish-rail'),
new TextEncoder().encode(IRISH_XML),
now,
);
assert.equal(snapshot.feedId, 'irish-rail');
assert.equal(snapshot.positions, 'event');
assert.equal(snapshot.entityCount, 2);
assert.equal(snapshot.count, 2);
assert.equal(snapshot.feedTimestamp, null);
const byId = new Map(snapshot.trains.map((t) => [t.id, t]));
assert.equal(byId.get('A118').category, 'international');
assert.equal(byId.get('A118').positionSource, 'event');
assert.equal(byId.get('A118').timestampSource, 'fetch');
assert.equal(byId.get('A118').timestamp, now / 1000);
assert.equal(byId.get('E931').category, 'local');
assert.equal(byId.get('E931').delaySec, 960);
assert.throws(
() =>
buildRailSnapshot(
getRailFeed('irish-rail'),
new TextEncoder().encode('<html>Service Unavailable</html>'),
),
(error) =>
error.code === 'UPSTREAM_UNREADABLE' &&
!(error instanceof RailFeedShapeError),
);
});
test('no enabled feed is placed from its timetable', () => {
// Every train is where its operator reports it: no feed answers `/trains`
// with a timetable-only list.
for (const feed of RAIL_FEED_REGISTRY.filter((f) => f.defaultEnabled))
assert.ok(
['gps', 'event'].includes(railFeedPositions(feed)) &&
typeof feed.url === 'string',
feed.id,
);
});
test('trip updates keep rail trips only, with category, and refuse differential feeds', () => {
const feed = getRailFeed('mbta-rail');
const snapshot = buildRailTripUpdatesSnapshot(
feed,
encodeTripUpdates([
['cr', 'CR-Worcester', 120],
['red', 'Red', -30],
['bus', '39', 60],
['none', null, 0],
]),
5_000,
);
assert.equal(snapshot.feedId, 'mbta-rail');
assert.equal(snapshot.fetchedAt, 5_000);
assert.equal(snapshot.feedTimestamp, 1_794_000_000);
assert.equal(snapshot.count, 2);
assert.equal(snapshot.dropped, 2);
assert.deepEqual(
snapshot.trips.map((t) => [t.tripId, t.routeId, t.delaySec, t.category]),
[
['cr', 'CR-Worcester', 120, 'local'],
['red', 'Red', -30, 'transit'],
],
);
assert.deepEqual(snapshot.trips[0].stops[0], {
stopId: 's0',
stopSequence: 1,
arrival: { time: null, delay: 120 },
departure: null,
});
assert.throws(
() =>
buildRailTripUpdatesSnapshot(
feed,
encodeTripUpdates([], { incrementality: 1 }),
),
RailFeedShapeError,
);
});
test('a train named by its number takes its run, never a timetable position', () => {
const now = 1_794_000_000_000;
const bytes = new TextEncoder().encode(IRISH_XML);
const plain = buildRailSnapshot(getRailFeed('irish-rail'), bytes, now);
const named = buildRailSnapshot(getRailFeed('irish-rail'), bytes, now, {
tripFor: (code) => (code === 'A118' ? 'pack-trip' : null),
routeFor: (tripId) =>
tripId === 'pack-trip' ? 'A~X~DUB-EUROPORT-I' : null,
});
const at = (snapshot) =>
Object.fromEntries(
snapshot.trains.map((t) => [t.id, [t.lat, t.lon, t.positionSource]]),
);
assert.deepEqual(at(named), at(plain), 'positions as reported');
const train = named.trains.find((t) => t.id === 'A118');
assert.equal(train.tripId, 'pack-trip');
assert.equal(train.trainNumber, 'A118');
});
test('trip updates are kept by their own route', () => {
const feed = getRailFeed('mbta-rail');
const snapshot = buildRailTripUpdatesSnapshot(
feed,
encodeTripUpdates([
['cr', 'CR-Worcester', 120],
['bus', '39', 60],
]),
5_000,
{ routeFor: () => null },
);
assert.deepEqual(
snapshot.trips.map((t) => [t.tripId, t.routeId]),
[['cr', 'CR-Worcester']],
);
assert.equal(snapshot.dropped, 1);
});
test('trip update answers are capped in trips', () => {
const rows = Array.from({ length: RAIL_UPDATES_MAX_TRIPS + 2 }, (_, i) => [
`t${i}`,
'CR-Lowell',
0,
]);
const snapshot = buildRailTripUpdatesSnapshot(
getRailFeed('mbta-rail'),
encodeTripUpdates(rows),
);
assert.equal(snapshot.count, RAIL_UPDATES_MAX_TRIPS);
assert.equal(snapshot.truncated, true);
assert.equal(snapshot.dropped, 0, 'capped trips are rail, not dropped');
});
test('/updates resolves only for feeds that publish trip updates', () => {
assert.equal(resolveRailRoute('/updates/mbta-rail')?.route, 'updates');
assert.equal(resolveRailRoute('/updates/marc')?.feed?.id, 'marc');
assert.equal(resolveRailRoute('/updates/irish-rail'), null, 'no updates');
assert.equal(resolveRailRoute('/updates/capmetro-rail'), null);
assert.equal(resolveRailRoute('/updates/vre'), null, 'not registered');
assert.equal(resolveRailRoute('/updates/mbta-rail/x'), null);
});
test('feed windows: own interval when set, a floor, and per-route defaults', () => {
assert.equal(railFeedTtlMs(getRailFeed('mbta-rail')), 15_000);
assert.equal(railFeedTtlMs(getRailFeed('mbta-rail'), 'updates'), 30_000);
assert.equal(railFeedTtlMs(getRailFeed('irish-rail')), 30_000);
assert.equal(railFeedTtlMs(getRailFeed('adelaide-metro'), 'updates'), 60_000);
assert.equal(railFeedTtlMs({ pollIntervalMs: 10 }), 5_000);
assert.equal(railMaxBodyBytes(getRailFeed('irish-rail')), 1024 * 1024);
assert.equal(railMaxBodyBytes(getRailFeed('mbta-rail')), 8 * 1024 * 1024);
assert.equal(railCacheStateFor(null, 0), 'none');
assert.equal(railCacheStateFor({ at: 0 }, 29_000, 30_000), 'fresh');
assert.equal(railCacheStateFor({ at: 0 }, 31_000, 30_000), 'stale');
assert.equal(railCacheStateFor({ at: 0 }, 11 * 60_000, 30_000), 'expired');
});
test('response headers follow the feed window', () => {
assert.match(
railResponseHeaders('MISS', 'h', null, 60_000)['Cache-Control'],
/max-age=60/,
);
// An older snapshot is kept downstream only for the rest of its window.
const maxAge = (ageMs) =>
railResponseHeaders('HIT', 'h', null, 60_000, ageMs)['Cache-Control'];
assert.equal(maxAge(45_000), 'public, max-age=15');
assert.equal(maxAge(59_200), 'public, max-age=1');
assert.equal(maxAge(60_000), 'public, max-age=0');
assert.equal(maxAge(-5), 'public, max-age=60', 'a clock step is no age');
});
test('/schedule resolves for every pollable feed', () => {
assert.equal(resolveRailRoute('/schedule/ktmb')?.route, 'schedule');
assert.equal(
resolveRailRoute('/schedule/irish-rail')?.feed?.id,
'irish-rail',
);
assert.equal(resolveRailRoute('/schedule/mbta-rail')?.route, 'schedule');
assert.equal(resolveRailRoute('/schedule/vre'), null, 'not registered');
assert.equal(resolveRailRoute('/schedule/ktmb/x'), null);
assert.equal(resolveRailRoute('/schedule/..%2Fktmb'), null);
});
test('the global admission cap scales with the registry', () => {
const natural = railNaturalRequestsPerMinute();
const cap = railAdmissionGlobalMax();
assert.ok(natural > 40, 'the registry outgrew the old fixed cap');
assert.ok(
cap >= natural * 1.5,
'room for retries on top of the natural load',
);
assert.equal(railAdmissionGlobalMax([]), 40, 'never below the Transit floor');
const one = [{ defaultEnabled: true, url: 'https://x.test/vp' }];
assert.equal(railNaturalRequestsPerMinute(one), 4, 'one feed at 15 s');
const updatesOnly = [
{ defaultEnabled: true, tripUpdatesUrl: 'https://x.test/tu' },
];
assert.equal(railNaturalRequestsPerMinute(updatesOnly), 2, 'no positions');
assert.equal(
railNaturalRequestsPerMinute([{ ...one[0], defaultEnabled: false }]),
0,
);
});
/** Vehicles with a trip id and no route or time, as Renfe sends them. */
function encodeRenfeLike(headerTime, trips) {
const writer = new PbfWriter();
const header = new PbfWriter();
header.writeStringField(1, '2.0');
header.writeVarintField(3, headerTime);
writer.writeBytesField(1, header.finish());
for (const [id, tripId, route] of trips) {
const entity = new PbfWriter();
entity.writeStringField(1, id);
const vehicle = new PbfWriter();
const trip = new PbfWriter();
trip.writeStringField(1, tripId);
if (route) trip.writeStringField(5, route);
vehicle.writeBytesField(1, trip.finish());
const position = new PbfWriter();
position.writeFloatField(1, 40.4);
position.writeFloatField(2, -3.7);
vehicle.writeBytesField(2, position.finish());
const descriptor = new PbfWriter();
descriptor.writeStringField(1, id);
vehicle.writeBytesField(8, descriptor.finish());
entity.writeBytesField(4, vehicle.finish());
writer.writeBytesField(2, entity.finish());
}
return writer.finish();
}
test('a feed with an untrusted header clock stamps its trains with the fetch time', () => {
const now = 1_794_000_000_000;
// Renfe writes local time as UTC: the header reads an hour behind.
const bytes = encodeRenfeLike(now / 1000 - 3600, [
['04192', '0419212026-11-06'],
]);
const routes = new Map([['0419212026-11-06', 'ALVIA~X~1700037606GL023']]);
const snapshot = buildRailSnapshot(
getRailFeed('renfe-larga-distancia'),
bytes,
now,
{
routeFor: (tripId) => routes.get(tripId) ?? null,
},
);
assert.equal(snapshot.count, 1);
const [train] = snapshot.trains;
assert.equal(train.timestampSource, 'fetch');
assert.equal(train.timestamp, now / 1000, 'not the header time');
assert.equal(
snapshot.feedTimestamp,
null,
'the header clock is not reported',
);
assert.equal(
train.routeId,
'ALVIA~X~1700037606GL023',
'from the schedule pack',
);
assert.equal(train.category, 'national');
// A trusted feed keeps the header ladder.
const trusted = buildRailSnapshot(
getRailFeed('mta-metro-north'),
encodeRenfeLike(now / 1000 - 5, [['mnr1', 't1']]),
now,
);
assert.equal(trusted.trains[0].timestampSource, 'header');
});
test('a trip the schedule pack does not know is dropped, not guessed', () => {
const snapshot = buildRailSnapshot(
getRailFeed('renfe-larga-distancia'),
encodeRenfeLike(1_794_000_000, [
['a', 'known'],
['b', 'unknown'],
]),
1_794_000_000_000,
{ routeFor: (tripId) => (tripId === 'known' ? 'MD~S~1' : null) },
);
assert.deepEqual(
snapshot.trains.map((t) => [t.id, t.category]),
[['a', 'regional']],
);
assert.equal(snapshot.dropped, 1);
// Without a lookup nothing in this feed can be classified.
const blind = buildRailSnapshot(
getRailFeed('renfe-larga-distancia'),
encodeRenfeLike(1_794_000_000, [['a', 'known']]),
);
assert.equal(blind.count, 0);
});
test('trip updates take their route from the schedule pack too, and carry the tourist flag', () => {
const snapshot = buildRailTripUpdatesSnapshot(
getRailFeed('renfe-larga-distancia'),
encodeTripUpdates([
['0419212026-11-06', null, 60],
['999', null, 0],
]),
5_000,
{
routeFor: (tripId) =>
tripId === '0419212026-11-06' ? 'ALVIA~X~1700037606GL023' : null,
},
);
assert.deepEqual(
snapshot.trips.map((t) => [t.tripId, t.routeId, t.category]),
[['0419212026-11-06', 'ALVIA~X~1700037606GL023', 'national']],
);
assert.equal(snapshot.dropped, 1, 'the trip the pack does not know');
const flam = buildRailSnapshot(
getRailFeed('entur-vy'),
encodeFeed({ entities: [{ id: 'flam', route: 'VYG:Line:R45' }] }),
);
assert.equal(flam.trains[0].tourist, true);
assert.equal(flam.trains[0].category, 'regional');
});
+741
View File
@@ -0,0 +1,741 @@
import {
dayNumberOf,
formatClock,
isTimeZone,
parseServiceDate,
serviceDayOriginMs,
} from './railTimeZones.js';
/**
* @module railSchedule
* @description Timetable tier of the Rail layer: which trains run now
* according to the operator's static GTFS, where along their line the
* timetable puts them (only to match a live train to its run, never drawn),
* and the stop list a train card shows.
*
* Schedule packs come from scripts/build-rail-paths.mjs (`<feedId>.schedule.json`).
* Service calendars are resolved to concrete dates at build time, so this
* module never evaluates a calendar. Stop times are seconds after the service
* day's GTFS origin (noon minus 12 h, agency time zone) and may exceed 24 h.
*
* Realtime delays follow GTFS-RT TripUpdate semantics:
*
* delays = { [tripId]: { delaySec?, startDate?, relationship?, stops?: [{
* stopId? | stopSequence?, relationship?,
* arrival?: { time?, delay? }, departure?: { time?, delay? } }] } }
*
* (the trip records of src/data/railTripUpdates.js, keyed by trip id). A
* trip `relationship` of 'CANCELED' or 'DELETED' drops the run from
* activeTrips; a stop `relationship` of 'SKIPPED' (or `skipped: true`) is
* passed without a stop, and one of 'NO_DATA' has no prediction: that stop
* and the stops after it keep their timetable times until the next update.
*
* `time` is POSIX seconds (milliseconds are accepted too), `delay` seconds.
* A stop update's delay carries on to later stops until the next update;
* stops before the first update use the trip's `delaySec` (else the
* timetable). `startDate` ('YYYYMMDD' or 'YYYY-MM-DD') pins the entry to one
* run of a trip that runs on several days.
*
* Clock strings are 24-hour 'HH:MM' in each STATION's own time zone.
*
* Pure: no Cesium, no DOM, no I/O.
*/
/** Schedule-pack format this module reads. */
export const RAIL_SCHEDULE_PACK_VERSION = 1;
/** Earliest a delayed or early train is looked for around its timetable (ms). */
const EARLY_SLACK_MS = 60 * 60_000;
const LATE_SLACK_MS = 12 * 60 * 60_000;
const MINUS = '−';
/** TripUpdate relationships under which a scheduled run does not run. */
const CANCELED = new Set(['CANCELED', 'DELETED']);
/**
* A relative time chip as the Launches layer writes its countdown: `T−4`
* for an event four minutes ahead, `T+3` for one three minutes past. Minutes
* round up toward the event (Math.ceil, as the Launches countdown rounds its
* seconds), so `T−1` shows until the moment itself and `T+0` for the first
* minute after. An hour or more reads `T−1:05`.
* @param {number} deltaMs Event time minus now.
* @returns {string}
*/
export function formatRelative(deltaMs) {
if (!Number.isFinite(deltaMs)) return '';
const minutes = Math.ceil(deltaMs / 60_000);
const sign = deltaMs > 0 ? MINUS : '+';
const total = Math.abs(minutes);
const text =
total >= 60
? `${Math.floor(total / 60)}:${String(total % 60).padStart(2, '0')}`
: String(total);
return `T${sign}${text}`;
}
const isIndex = (value, length) =>
Number.isInteger(value) && value >= 0 && value < length;
/** Largest delay a live report may state, either way (s): a day. */
const LIVE_DELAY_MAX_S = 86_400;
/**
* The delay a live train's own report states (`delaySec`: Irish Rail's
* public message, SIRI VM's `Delay`), for a feed that publishes no trip
* updates; null when it states none, or more than a day either way (a unit
* error, not a late train).
* @param {{delaySec?: unknown}|null|undefined} record
* @returns {number|null} Seconds, positive when late.
*/
export function liveDelaySec(record) {
const value = record?.delaySec;
return typeof value === 'number' &&
Number.isFinite(value) &&
Math.abs(value) <= LIVE_DELAY_MAX_S
? value
: null;
}
/**
* Validate and index a decoded schedule pack.
* @param {object} raw The JSON body of `<feedId>.schedule.json`.
* @returns {object} A schedule for the functions below.
*/
export function createSchedule(raw) {
if (
!raw ||
raw.version !== RAIL_SCHEDULE_PACK_VERSION ||
typeof raw.feedId !== 'string' ||
!Array.isArray(raw.stops) ||
!Array.isArray(raw.patterns) ||
!Array.isArray(raw.profiles) ||
!raw.trips ||
typeof raw.trips !== 'object' ||
!raw.services ||
typeof raw.services !== 'object'
)
throw new TypeError('Malformed rail schedule pack');
const timezone = isTimeZone(raw.timezone) ? raw.timezone : 'UTC';
const stops = raw.stops.map((stop, index) => ({
index,
id: String(stop?.id ?? ''),
name: String(stop?.name ?? ''),
lat: Number(stop?.lat),
lon: Number(stop?.lon),
parent: isIndex(stop?.parent, raw.stops.length) ? stop.parent : null,
platform: stop?.platform ? String(stop.platform) : null,
ownZone: isTimeZone(stop?.tz) ? stop.tz : null,
}));
for (const stop of stops) {
const parent = stop.parent === null ? null : stops[stop.parent];
stop.timezone = stop.ownZone ?? parent?.ownZone ?? timezone;
delete stop.ownZone;
Object.freeze(stop);
}
const stopIndexById = new Map(stops.map((stop) => [stop.id, stop.index]));
const patterns = raw.patterns.map((pattern, index) => {
const list = Array.isArray(pattern?.stops) ? pattern.stops : [];
const n = list.length;
if (n < 2 || !list.every((value) => isIndex(value, stops.length)))
throw new TypeError(`Malformed rail schedule pattern ${index}`);
const dist =
Array.isArray(pattern.dist) && pattern.dist.length === n
? pattern.dist.map(Number)
: new Array(n).fill(0);
const seq =
Array.isArray(pattern.seq) && pattern.seq.length === n
? pattern.seq.map(Number)
: list.map((_, k) => k + 1);
return Object.freeze({
index,
shape: Number.isInteger(pattern.shape) ? pattern.shape : null,
stops: Object.freeze(list.slice()),
seq: Object.freeze(seq),
dist: Object.freeze(dist),
pass: Object.freeze(
Array.isArray(pattern.pass) && pattern.pass.length === n
? pattern.pass.map((value) => value === 1)
: new Array(n).fill(false),
),
});
});
const origins = new Map();
const originOf = (date) => {
let origin = origins.get(date);
if (origin === undefined) {
origin = serviceDayOriginMs(date, timezone);
origins.set(date, origin);
}
return origin;
};
const trips = new Map();
const instances = [];
let maxDurationMs = 0;
for (const [tripId, value] of Object.entries(raw.trips)) {
const pattern = patterns[value?.pattern];
const profile = raw.profiles[value?.profile];
if (!pattern || !Array.isArray(profile)) continue;
if (profile.length !== pattern.stops.length * 2) continue;
const startSec = Number(value.startSec);
if (!Number.isFinite(startSec)) continue;
const trip = Object.freeze({
id: tripId,
pattern,
offsets: profile,
startSec,
routeId: value.route == null ? null : String(value.route),
service: String(value.service ?? ''),
headsign: value.headsign ? String(value.headsign) : null,
name: value.name ? String(value.name) : null,
direction: Number.isInteger(value.direction) ? value.direction : null,
});
trips.set(tripId, trip);
const dates = raw.services[trip.service];
if (!Array.isArray(dates)) continue;
const durationMs = profile[profile.length - 1] * 1000;
if (durationMs > maxDurationMs) maxDurationMs = durationMs;
for (const date of dates) {
if (!parseServiceDate(date)) continue;
const startMs = originOf(date) + startSec * 1000;
if (Number.isFinite(startMs))
instances.push({
trip,
serviceDate: date,
startMs,
endMs: startMs + durationMs,
});
}
}
instances.sort((a, b) => a.startMs - b.startMs);
const runsByTrip = new Map();
for (const instance of instances) {
let list = runsByTrip.get(instance.trip.id);
if (!list) runsByTrip.set(instance.trip.id, (list = []));
list.push(instance);
}
return Object.freeze({
feedId: raw.feedId,
/**
* The `builtAt` of the path pack this timetable's shape indices point
* into (written by the same build), or null in packs that predate it.
*/
pathsBuiltAt:
typeof raw.pathsBuiltAt === 'string' ? raw.pathsBuiltAt : null,
timezone,
window: raw.window ?? null,
routes: raw.routes && typeof raw.routes === 'object' ? raw.routes : {},
stops: Object.freeze(stops),
patterns: Object.freeze(patterns),
trips,
stopIndexById,
instances: Object.freeze(instances),
runsByTrip,
instanceStarts: Float64Array.from(
instances,
(instance) => instance.startMs,
),
maxDurationMs,
});
}
/**
* When the last run a decoded schedule pack holds arrives (epoch ms): past
* it the pack's window has ended, and it names no train that can still be
* running. Each trip counts on the latest date of its service. Read from the
* raw pack, without indexing it (the server checks a pack it reads only for
* its routes). Null when the pack names no window (`window.to`), so its end
* is not known; -Infinity when it holds no run.
* @param {object} raw The JSON body of `<feedId>.schedule.json`.
* @returns {number|null}
*/
export function schedulePackEndMs(raw) {
if (!parseServiceDate(raw?.window?.to)) return null;
const trips = raw?.trips && typeof raw.trips === 'object' ? raw.trips : {};
const services =
raw?.services && typeof raw.services === 'object' ? raw.services : {};
const profiles = Array.isArray(raw?.profiles) ? raw.profiles : [];
const zone = isTimeZone(raw?.timezone) ? raw.timezone : 'UTC';
/** Service id → the origin of its latest date (-Infinity: no date). */
const lastOrigin = new Map();
const latestOrigin = (service) => {
let origin = lastOrigin.get(service);
if (origin !== undefined) return origin;
const dates = Object.hasOwn(services, service) ? services[service] : null;
let latest = -Infinity;
let latestDate = null;
for (const date of Array.isArray(dates) ? dates : []) {
const day = dayNumberOf(date);
if (day > latest) {
latest = day;
latestDate = date;
}
}
origin = latestDate === null ? NaN : serviceDayOriginMs(latestDate, zone);
if (!Number.isFinite(origin)) origin = -Infinity;
lastOrigin.set(service, origin);
return origin;
};
let end = -Infinity;
for (const trip of Object.values(trips)) {
const profile = profiles[trip?.profile];
const startSec = Number(trip?.startSec);
if (!Array.isArray(profile) || !Number.isFinite(startSec)) continue;
const origin = latestOrigin(String(trip.service ?? ''));
if (origin === -Infinity) continue;
const durationSec = Number(profile[profile.length - 1]) || 0;
end = Math.max(end, origin + (startSec + durationSec) * 1000);
}
return end;
}
/** First index whose start is greater than `ms`. */
function upperBound(starts, ms) {
let lo = 0;
let hi = starts.length;
while (lo < hi) {
const mid = (lo + hi) >> 1;
if (starts[mid] <= ms) lo = mid + 1;
else hi = mid;
}
return lo;
}
/** The public view of one run of a trip. */
function runOf(instance) {
const { trip } = instance;
return {
tripId: trip.id,
serviceDate: instance.serviceDate,
routeId: trip.routeId,
headsign: trip.headsign,
name: trip.name,
direction: trip.direction,
pattern: trip.pattern.index,
shape: trip.pattern.shape,
scheduledStartMs: instance.startMs,
scheduledEndMs: instance.endMs,
};
}
/** The delay entry for a run, honouring a `startDate` pin. */
function delayFor(delays, run) {
const entry =
delays && Object.hasOwn(delays, run.tripId) ? delays[run.tripId] : null;
if (!entry || typeof entry !== 'object') return null;
if (entry.startDate != null) {
const pinned = parseServiceDate(entry.startDate);
const own = parseServiceDate(run.serviceDate);
if (
!pinned ||
!own ||
pinned.year !== own.year ||
pinned.month !== own.month ||
pinned.day !== own.day
)
return null;
}
return entry;
}
const toMs = (time) => {
const value = Number(time);
if (!Number.isFinite(value) || value <= 0) return NaN;
return value > 1e11 ? value : value * 1000;
};
/** The pattern index each stop update applies to, in order. */
function matchUpdates(schedule, pattern, updates) {
const byIndex = new Map();
let from = 0;
for (const update of updates) {
if (!update || typeof update !== 'object') continue;
let found = -1;
for (let pass = 0; pass < 2 && found < 0; pass++) {
const start = pass === 0 ? from : 0;
for (let k = start; k < pattern.stops.length; k++) {
if (stopMatches(schedule, pattern, k, update)) {
found = k;
break;
}
}
}
if (found < 0) continue;
byIndex.set(found, update);
from = found + 1;
}
return byIndex;
}
function stopMatches(schedule, pattern, k, update) {
if (update.stopSequence != null && update.stopSequence !== '')
return Number(update.stopSequence) === pattern.seq[k];
if (update.stopId == null) return false;
const id = String(update.stopId);
const stop = schedule.stops[pattern.stops[k]];
if (stop.id === id) return true;
if (stop.parent !== null && schedule.stops[stop.parent].id === id)
return true;
const other = schedule.stopIndexById.get(id);
if (other === undefined) return false;
const otherParent = schedule.stops[other].parent;
return otherParent !== null && otherParent === (stop.parent ?? stop.index);
}
/** Whether a stop update carries this StopTimeUpdate relationship. */
function stopRelationshipIs(update, relationship) {
return (
update.relationship === relationship ||
update.scheduleRelationship === relationship
);
}
/**
* Scheduled and expected times of every stop of a run.
* @returns {{schedArr: number[], schedDep: number[], arr: number[],
* dep: number[], skipped: boolean[], hasRealtime: boolean}}
*/
function runTimes(schedule, instance, delays) {
const { trip } = instance;
const n = trip.pattern.stops.length;
const schedArr = new Array(n);
const schedDep = new Array(n);
for (let k = 0; k < n; k++) {
schedArr[k] = instance.startMs + trip.offsets[2 * k] * 1000;
schedDep[k] = instance.startMs + trip.offsets[2 * k + 1] * 1000;
}
const arr = schedArr.slice();
const dep = schedDep.slice();
const skipped = new Array(n).fill(false);
const entry = delayFor(delays, {
tripId: trip.id,
serviceDate: instance.serviceDate,
});
if (!entry)
return { schedArr, schedDep, arr, dep, skipped, hasRealtime: false };
const updates = matchUpdates(
schedule,
trip.pattern,
Array.isArray(entry.stops) ? entry.stops : [],
);
let carried = Number.isFinite(Number(entry.delaySec))
? Number(entry.delaySec)
: 0;
for (let k = 0; k < n; k++) {
const update = updates.get(k);
let arrDelay = carried;
let depDelay = carried;
if (update && stopRelationshipIs(update, 'NO_DATA')) {
// No prediction (GTFS-RT NO_DATA): the delay carried so far no longer
// holds, here or at the stops after it.
carried = 0;
arrDelay = 0;
depDelay = 0;
} else if (update) {
if (update.skipped === true || stopRelationshipIs(update, 'SKIPPED'))
skipped[k] = true;
const a = update.arrival ?? null;
const d = update.departure ?? null;
const aTime = toMs(a?.time);
const dTime = toMs(d?.time);
const aDelay = Number(a?.delay);
const dDelay = Number(d?.delay);
let arrivalGiven = true;
if (Number.isFinite(aTime)) arrDelay = (aTime - schedArr[k]) / 1000;
else if (a && a.delay != null && Number.isFinite(aDelay))
arrDelay = aDelay;
else arrivalGiven = false;
if (Number.isFinite(dTime)) depDelay = (dTime - schedDep[k]) / 1000;
else if (d && d.delay != null && Number.isFinite(dDelay))
depDelay = dDelay;
else if (arrivalGiven) depDelay = arrDelay;
else depDelay = carried;
if (!arrivalGiven && (Number.isFinite(dTime) || (d && d.delay != null)))
arrDelay = Math.min(carried, depDelay);
carried = depDelay;
}
arr[k] = schedArr[k] + arrDelay * 1000;
dep[k] = schedDep[k] + depDelay * 1000;
}
// Predictions are not always consistent; time never runs backwards.
for (let k = 0; k < n; k++) {
if (k > 0 && arr[k] < dep[k - 1]) arr[k] = dep[k - 1];
if (dep[k] < arr[k]) dep[k] = arr[k];
}
return { schedArr, schedDep, arr, dep, skipped, hasRealtime: true };
}
/** The instance of `trip` (a run object or a trip id) meant at `nowMs`. */
function resolveInstance(schedule, trip, nowMs, delays) {
const pinned = trip && typeof trip === 'object';
const runs = schedule.runsByTrip.get(
String((pinned ? trip.tripId : trip) ?? ''),
);
if (!runs) return null;
if (pinned && trip.serviceDate != null) {
const match = runs.find(
(instance) => instance.serviceDate === trip.serviceDate,
);
if (match) return match;
}
let best = null;
let bestScore = Infinity;
for (const instance of runs) {
const times = runTimes(schedule, instance, delays);
const start = times.dep[0];
const end = times.arr[times.arr.length - 1];
// Running now wins; else the nearest run, upcoming before past.
const score =
nowMs >= start && nowMs <= end
? -1
: nowMs < start
? start - nowMs
: (nowMs - end) * 2;
if (score < bestScore) {
bestScore = score;
best = instance;
}
}
return best;
}
/**
* Trains the timetable has running at `nowMs`: from the departure at the
* first stop to the arrival at the last, shifted by realtime delays.
* @param {object} schedule From createSchedule.
* @param {number} nowMs
* @param {{delays?: object, beforeMs?: number, afterMs?: number}} [options]
* `beforeMs` / `afterMs` widen the window (a train waiting at its origin,
* one just arrived).
* @returns {object[]} Runs: {tripId, serviceDate, routeId, headsign, name,
* direction, pattern, shape, scheduledStartMs, scheduledEndMs,
* expectedStartMs, expectedEndMs}.
*/
export function activeTrips(
schedule,
nowMs,
{ delays = null, beforeMs = 0, afterMs = 0 } = {},
) {
if (!schedule || !Number.isFinite(nowMs)) return [];
const lead = Math.max(0, beforeMs);
const tail = Math.max(0, afterMs);
const last = upperBound(
schedule.instanceStarts,
nowMs + lead + EARLY_SLACK_MS,
);
const floor = nowMs - tail - LATE_SLACK_MS - schedule.maxDurationMs;
const out = [];
for (let i = last - 1; i >= 0; i--) {
const instance = schedule.instances[i];
if (instance.startMs < floor) break;
const run = runOf(instance);
let start = instance.startMs + instance.trip.offsets[1] * 1000;
let end = instance.endMs;
const entry = delayFor(delays, run);
if (entry && CANCELED.has(entry.relationship)) continue;
if (entry) {
const times = runTimes(schedule, instance, delays);
start = times.dep[0];
end = times.arr[times.arr.length - 1];
} else if (instance.endMs < nowMs - tail) continue;
if (nowMs < start - lead || nowMs > end + tail) continue;
run.expectedStartMs = start;
run.expectedEndMs = end;
out.push(run);
}
return out.reverse();
}
/**
* Where the timetable puts a train at `nowMs`, along its pattern's shape.
* Between two stops the arc runs linearly in time from the departure to the
* next arrival; during a dwell it holds at the stop. Before the first
* departure it waits at the origin, after the last arrival at the terminus.
*
* @param {object} schedule From createSchedule.
* @param {object|string} trip A run from activeTrips, or a trip id.
* @param {number} nowMs
* @param {object} [delays]
* @returns {{tripId: string, serviceDate: string, shape: number|null,
* s: number, prevStop: number, nextStop: number,
* prevStopId: string|null, nextStopId: string|null, dwell: boolean,
* state: 'before'|'running'|'arrived', fraction: number, delaySec: number,
* lat?: number, lon?: number}|null} Stop fields are pattern indices (-1 for
* none). `lat`/`lon` interpolate between the stops and are set only when
* the pattern has no shape.
*/
export function scheduledPosition(schedule, trip, nowMs, delays = null) {
if (!schedule || !Number.isFinite(nowMs)) return null;
const instance = resolveInstance(schedule, trip, nowMs, delays);
if (!instance) return null;
const { pattern } = instance.trip;
const times = runTimes(schedule, instance, delays);
// Skipped stops are passed without a stop: their times do not pace the run.
const nodes = [];
for (let k = 0; k < pattern.stops.length; k++)
if (!times.skipped[k]) nodes.push(k);
if (nodes.length < 2)
nodes.splice(0, nodes.length, 0, pattern.stops.length - 1);
const result = {
tripId: instance.trip.id,
serviceDate: instance.serviceDate,
shape: pattern.shape,
s: 0,
prevStop: -1,
nextStop: -1,
prevStopId: null,
nextStopId: null,
dwell: false,
state: 'running',
fraction: 0,
delaySec: 0,
};
const first = nodes[0];
const lastNode = nodes[nodes.length - 1];
let at = -1;
let from = -1;
let to = -1;
if (nowMs < times.dep[first]) {
result.state = 'before';
at = first;
result.prevStop = -1;
result.nextStop = first;
} else if (nowMs >= times.arr[lastNode]) {
result.state = 'arrived';
at = lastNode;
result.prevStop = lastNode;
result.nextStop = -1;
} else {
for (let j = 0; j < nodes.length; j++) {
const k = nodes[j];
if (nowMs >= times.arr[k] && nowMs < times.dep[k]) {
at = k;
result.prevStop = k;
result.nextStop = j + 1 < nodes.length ? nodes[j + 1] : -1;
break;
}
if (
j + 1 < nodes.length &&
nowMs >= times.dep[k] &&
nowMs < times.arr[nodes[j + 1]]
) {
from = k;
to = nodes[j + 1];
break;
}
}
}
if (at >= 0) {
result.dwell = true;
result.s = pattern.dist[at];
result.fraction = 0;
result.delaySec = Math.round(
(result.state === 'arrived'
? times.arr[at] - times.schedArr[at]
: times.dep[at] - times.schedDep[at]) / 1000,
);
} else {
const span = times.arr[to] - times.dep[from];
const fraction =
span > 0 ? Math.min(1, Math.max(0, (nowMs - times.dep[from]) / span)) : 1;
result.prevStop = from;
result.nextStop = to;
result.fraction = fraction;
result.s =
pattern.dist[from] + (pattern.dist[to] - pattern.dist[from]) * fraction;
result.delaySec = Math.round((times.arr[to] - times.schedArr[to]) / 1000);
}
if (result.prevStop >= 0)
result.prevStopId = schedule.stops[pattern.stops[result.prevStop]].id;
if (result.nextStop >= 0)
result.nextStopId = schedule.stops[pattern.stops[result.nextStop]].id;
if (pattern.shape === null) {
const a =
schedule.stops[
pattern.stops[result.prevStop >= 0 ? result.prevStop : result.nextStop]
];
const b = result.dwell ? a : schedule.stops[pattern.stops[result.nextStop]];
result.lat = a.lat + (b.lat - a.lat) * result.fraction;
result.lon = a.lon + (b.lon - a.lon) * result.fraction;
}
return result;
}
/**
* A run's stop list for a train card.
*
* `state` is 'passed' once the train has left (or, at the terminus, reached)
* the stop, 'next' for the first stop not yet passed (with `atStop` while the
* train dwells there) and 'upcoming' after it. `relative` is the chip from
* formatRelative: for a passed stop the departure (terminus: arrival), for
* the stop the train stands at its departure, else the arrival.
*
* @param {object} schedule
* @param {object|string} trip A run from activeTrips, or a trip id.
* @param {number} nowMs
* @param {object} [delays]
* @returns {object[]} Per stop: {index, stopId, name, platform, parentId,
* parentName, lat, lon, timezone, scheduledArrivalMs,
* scheduledDepartureMs, expectedArrivalMs, expectedDepartureMs,
* arrivalDelaySec, departureDelaySec, scheduledArrival,
* scheduledDeparture, expectedArrival, expectedDeparture (local 'HH:MM'),
* passThrough, skipped, realtime, state, atStop, relative, relativeMs}.
*/
export function tripStops(schedule, trip, nowMs, delays = null) {
if (!schedule || !Number.isFinite(nowMs)) return [];
const instance = resolveInstance(schedule, trip, nowMs, delays);
if (!instance) return [];
const { pattern } = instance.trip;
const times = runTimes(schedule, instance, delays);
const n = pattern.stops.length;
const out = [];
let nextFound = false;
for (let k = 0; k < n; k++) {
const stop = schedule.stops[pattern.stops[k]];
const parent = stop.parent === null ? null : schedule.stops[stop.parent];
const zone = stop.timezone;
const terminus = k === n - 1;
const leftAt = terminus ? times.arr[k] : times.dep[k];
let state;
let atStop = false;
if (nowMs >= leftAt) state = 'passed';
else if (!nextFound) {
state = 'next';
nextFound = true;
atStop = nowMs >= times.arr[k] && !times.skipped[k];
} else state = 'upcoming';
const eventMs =
state === 'passed'
? leftAt
: atStop || k === 0
? times.dep[k]
: times.arr[k];
out.push({
index: k,
stopId: stop.id,
name: stop.name || parent?.name || stop.id,
platform: stop.platform,
parentId: parent?.id ?? null,
parentName: parent?.name ?? null,
lat: stop.lat,
lon: stop.lon,
timezone: zone,
scheduledArrivalMs: times.schedArr[k],
scheduledDepartureMs: times.schedDep[k],
expectedArrivalMs: times.arr[k],
expectedDepartureMs: times.dep[k],
arrivalDelaySec: Math.round((times.arr[k] - times.schedArr[k]) / 1000),
departureDelaySec: Math.round((times.dep[k] - times.schedDep[k]) / 1000),
scheduledArrival: formatClock(times.schedArr[k], zone),
scheduledDeparture: formatClock(times.schedDep[k], zone),
expectedArrival: formatClock(times.arr[k], zone),
expectedDeparture: formatClock(times.dep[k], zone),
passThrough: pattern.pass[k],
skipped: times.skipped[k],
realtime: times.hasRealtime,
state,
atStop,
relativeMs: eventMs - nowMs,
relative: formatRelative(eventMs - nowMs),
});
}
return out;
}
+444
View File
@@ -0,0 +1,444 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
activeTrips,
createSchedule,
formatRelative,
liveDelaySec,
schedulePackEndMs,
scheduledPosition,
tripStops,
} from './railSchedule.js';
import {
formatClock,
localDate,
serviceDayOriginMs,
zonedTimeToEpochMs,
} from './railTimeZones.js';
const MIN = 60_000;
const H = 3_600_000;
/**
* Four stops on one shape, New York time except the last (Chicago). Run
* times: A 0, B arrive +5 min / leave +6 min, C +15 min, D +25 min.
*/
function pack(overrides = {}) {
return {
version: 1,
feedId: 'test-feed',
timezone: 'America/New_York',
window: { from: '2026-10-31', to: '2026-11-03' },
routes: { R: { type: 2, short: 'R' } },
stops: [
{ id: 'A', name: 'Alpha', lat: 40, lon: -74 },
{
id: 'B1',
name: 'Track 1',
lat: 40,
lon: -73.99,
parent: 4,
platform: '1',
},
{ id: 'C', name: 'Charlie', lat: 40, lon: -73.97 },
{ id: 'D', name: 'Delta', lat: 40, lon: -73.94, tz: 'America/Chicago' },
{ id: 'B', name: 'Bravo', lat: 40, lon: -73.99 },
],
patterns: [
{
shape: 0,
stops: [0, 1, 2, 3],
seq: [1, 2, 3, 4],
dist: [0, 1000, 3000, 6000],
},
{ shape: null, stops: [0, 2], seq: [1, 2], dist: [0, 2500] },
],
profiles: [
[0, 0, 300, 360, 900, 900, 1500, 1500],
[0, 0, 600, 600],
],
services: {
SAT: ['2026-10-31'],
SUN: ['2026-11-01'],
DAILY: ['2026-10-31', '2026-11-01', '2026-11-02'],
},
trips: {
morning: {
pattern: 0,
profile: 0,
startSec: 8 * 3600,
route: 'R',
service: 'SUN',
headsign: 'Delta',
name: '101',
direction: 0,
},
late: {
pattern: 0,
profile: 0,
startSec: 25 * 3600,
route: 'R',
service: 'SAT',
name: '199',
},
fallback: {
pattern: 0,
profile: 0,
startSec: 1.5 * 3600,
route: 'R',
service: 'SUN',
},
shuttle: {
pattern: 1,
profile: 1,
startSec: 12 * 3600,
route: 'R',
service: 'DAILY',
},
},
...overrides,
};
}
const at = (iso) => Date.parse(iso);
/** 08:00 EST on 2026-11-01, the morning trip's departure. */
const MORNING = at('2026-11-01T13:00:00Z');
test('relative chips read T−n ahead and T+n past, rounded like the Launches countdown', () => {
assert.equal(formatRelative(4 * MIN), 'T−4');
assert.equal(formatRelative(3.5 * MIN), 'T−4');
assert.equal(formatRelative(30_000), 'T−1');
assert.equal(formatRelative(0), 'T+0');
assert.equal(formatRelative(-30_000), 'T+0');
assert.equal(formatRelative(-3 * MIN), 'T+3');
assert.equal(formatRelative(-3.5 * MIN), 'T+3');
assert.equal(formatRelative(65 * MIN), 'T−1:05');
assert.equal(formatRelative(-125 * MIN), 'T+2:05');
assert.equal(formatRelative(NaN), '');
});
test('service days count from noon minus 12 h, across both DST changes', () => {
// Ordinary day: local midnight.
assert.equal(
serviceDayOriginMs('2026-10-31', 'America/New_York'),
at('2026-10-31T04:00:00Z'),
);
// Fall back: 01:00 EDT, so 08:00:00 is 08:00 EST.
const fall = serviceDayOriginMs('20261101', 'America/New_York');
assert.equal(fall, at('2026-11-01T05:00:00Z'));
assert.equal(formatClock(fall + 8 * H, 'America/New_York'), '08:00');
// Spring forward: 23:00 EST the evening before, so 06:00:00 is 06:00 EDT.
const spring = serviceDayOriginMs('2026-03-08', 'America/New_York');
assert.equal(spring, at('2026-03-08T04:00:00Z'));
assert.equal(formatClock(spring + 6 * H, 'America/New_York'), '06:00');
assert.ok(Number.isNaN(serviceDayOriginMs('2026-02-30', 'UTC')));
});
test('wall times resolve to instants: overlaps take the earlier, gaps move on', () => {
assert.equal(
zonedTimeToEpochMs('America/New_York', 2026, 11, 1, 1, 30),
at('2026-11-01T05:30:00Z'),
);
assert.equal(
zonedTimeToEpochMs('America/New_York', 2026, 3, 8, 2, 30),
at('2026-03-08T07:30:00Z'),
);
assert.equal(
zonedTimeToEpochMs('Europe/London', 2026, 7, 1, 12),
at('2026-07-01T11:00:00Z'),
);
assert.equal(
localDate(at('2026-11-01T03:59:00Z'), 'America/New_York'),
'2026-10-31',
);
assert.equal(formatClock(at('2026-11-01T23:59:00Z'), 'Asia/Tokyo'), '08:59');
});
test('a malformed pack is refused', () => {
assert.throws(() => createSchedule(null), TypeError);
assert.throws(() => createSchedule({ ...pack(), version: 2 }), TypeError);
assert.throws(
() => createSchedule(pack({ patterns: [{ shape: 0, stops: [0, 99] }] })),
TypeError,
);
});
test('active trips come from service dates in the agency zone', () => {
const schedule = createSchedule(pack());
const ids = (now) =>
activeTrips(schedule, now).map((run) => `${run.tripId}@${run.serviceDate}`);
assert.deepEqual(ids(MORNING + 10 * MIN), ['morning@2026-11-01']);
assert.deepEqual(ids(MORNING - MIN), []);
assert.deepEqual(ids(MORNING + 26 * MIN), []);
// Saturday's 25:00:00 trip runs at 01:00 EDT on Sunday, the first 01:00.
assert.deepEqual(ids(at('2026-11-01T05:10:00Z')), ['late@2026-10-31']);
// Sunday's 01:30:00 trip runs at 01:30 EST, the second 01:30.
assert.deepEqual(ids(at('2026-11-01T06:40:00Z')), ['fallback@2026-11-01']);
// A daily trip appears once per run; widening the window catches a train
// waiting at its origin.
assert.deepEqual(ids(at('2026-11-02T17:05:00Z')), ['shuttle@2026-11-02']);
assert.deepEqual(
activeTrips(schedule, MORNING - 2 * MIN, { beforeMs: 5 * MIN }).map(
(run) => run.tripId,
),
['morning'],
);
const [run] = activeTrips(schedule, MORNING + MIN);
assert.equal(run.name, '101');
assert.equal(run.headsign, 'Delta');
assert.equal(run.shape, 0);
assert.equal(run.scheduledStartMs, MORNING);
assert.equal(run.scheduledEndMs, MORNING + 25 * MIN);
});
test('the timetable position runs linearly between stops and holds through a dwell', () => {
const schedule = createSchedule(pack());
const [run] = activeTrips(schedule, MORNING + MIN);
const moving = scheduledPosition(schedule, run, MORNING + 150_000);
assert.equal(moving.state, 'running');
assert.equal(moving.dwell, false);
assert.equal(moving.s, 500);
assert.equal(moving.prevStop, 0);
assert.equal(moving.nextStop, 1);
assert.equal(moving.nextStopId, 'B1');
assert.equal(
moving.lat,
undefined,
'a shaped pattern leaves lat/lon to the path',
);
const dwell = scheduledPosition(schedule, run, MORNING + 330_000);
assert.equal(dwell.dwell, true);
assert.equal(dwell.s, 1000);
assert.equal(dwell.prevStop, 1);
assert.equal(dwell.nextStop, 2);
const leg = scheduledPosition(schedule, 'morning', MORNING + 630_000);
assert.equal(leg.s, 2000, 'halfway from B (leaves +6) to C (arrives +15)');
const before = scheduledPosition(schedule, run, MORNING - MIN);
assert.equal(before.state, 'before');
assert.equal(before.s, 0);
assert.equal(before.nextStop, 0);
const after = scheduledPosition(schedule, run, MORNING + 30 * MIN);
assert.equal(after.state, 'arrived');
assert.equal(after.s, 6000);
assert.equal(after.nextStop, -1);
});
test('a pattern without a shape interpolates between its stops', () => {
const schedule = createSchedule(pack());
const noon = at('2026-11-01T17:00:00Z');
const position = scheduledPosition(
schedule,
{ tripId: 'shuttle', serviceDate: '2026-11-01' },
noon + 5 * MIN,
);
assert.equal(position.shape, null);
assert.equal(position.s, 1250);
assert.ok(Math.abs(position.lon - -73.985) < 1e-9);
assert.equal(position.lat, 40);
});
test('the stop list shows station-local 24 h clocks, states and chips', () => {
const schedule = createSchedule(pack());
const stops = tripStops(schedule, 'morning', MORNING + 320_000);
assert.deepEqual(
stops.map((stop) => [stop.stopId, stop.state, stop.atStop, stop.relative]),
[
['A', 'passed', false, 'T+5'],
['B1', 'next', true, 'T−1'],
['C', 'upcoming', false, 'T−10'],
['D', 'upcoming', false, 'T−20'],
],
);
assert.equal(stops[0].scheduledDeparture, '08:00');
assert.equal(stops[1].scheduledArrival, '08:05');
assert.equal(stops[1].scheduledDeparture, '08:06');
assert.equal(stops[1].name, 'Track 1');
assert.equal(stops[1].parentName, 'Bravo');
assert.equal(stops[1].platform, '1');
assert.equal(stops[1].timezone, 'America/New_York');
// Delta keeps Chicago time: 08:25 Eastern is 07:25 Central.
assert.equal(stops[3].timezone, 'America/Chicago');
assert.equal(stops[3].scheduledArrival, '07:25');
assert.equal(stops[3].scheduledArrivalMs, MORNING + 25 * MIN);
assert.equal(stops[3].realtime, false);
});
test('times past 24:00:00 land on the next calendar day', () => {
const schedule = createSchedule(pack());
const stops = tripStops(schedule, 'late', at('2026-11-01T05:00:00Z'));
assert.equal(stops[0].scheduledDepartureMs, at('2026-11-01T05:00:00Z'));
assert.equal(stops[0].scheduledDeparture, '01:00');
assert.equal(stops[1].scheduledArrival, '01:05');
// The fall-back trip at 01:30:00 runs an hour of real time later.
const fallback = tripStops(schedule, 'fallback', at('2026-11-01T06:30:00Z'));
assert.equal(fallback[0].scheduledDepartureMs, at('2026-11-01T06:30:00Z'));
assert.equal(fallback[0].scheduledDeparture, '01:30');
});
test('a stop delay carries on to later stops until the next update', () => {
const schedule = createSchedule(pack());
const delays = {
morning: {
stops: [
{ stopId: 'B1', arrival: { delay: 120 } },
{ stopSequence: 4, arrival: { time: (MORNING + 26 * MIN) / 1000 } },
],
},
};
const stops = tripStops(schedule, 'morning', MORNING, delays);
assert.deepEqual(
stops.map((stop) => [stop.arrivalDelaySec, stop.departureDelaySec]),
[
[0, 0],
[120, 120],
[120, 120],
[60, 60],
],
);
assert.equal(stops[1].expectedArrival, '08:07');
assert.equal(stops[1].scheduledArrival, '08:05');
assert.equal(stops[1].realtime, true);
// Between A (left on time) and B (arrives +7) the run is slower.
const position = scheduledPosition(
schedule,
'morning',
MORNING + 210_000,
delays,
);
assert.equal(position.s, 500);
assert.equal(position.delaySec, 120);
// A late train is still active after its timetable ends.
const late = { morning: { delaySec: 600 } };
assert.deepEqual(
activeTrips(schedule, MORNING + 30 * MIN).map((run) => run.tripId),
[],
);
const [run] = activeTrips(schedule, MORNING + 30 * MIN, { delays: late });
assert.equal(run.tripId, 'morning');
assert.equal(run.expectedEndMs, MORNING + 35 * MIN);
});
test('a delay pinned to a start date leaves other runs alone; parent ids match platforms', () => {
const schedule = createSchedule(pack());
const pinned = { morning: { startDate: '20261031', delaySec: 300 } };
assert.equal(
tripStops(schedule, 'morning', MORNING, pinned)[2].arrivalDelaySec,
0,
);
const own = {
morning: {
startDate: '2026-11-01',
stops: [{ stopId: 'B', departure: { delay: 60 } }],
},
};
const stops = tripStops(schedule, 'morning', MORNING, own);
assert.equal(stops[1].departureDelaySec, 60);
assert.equal(stops[2].arrivalDelaySec, 60);
});
test('a skipped stop is passed without a dwell', () => {
const schedule = createSchedule(pack());
// The record shape src/data/railTripUpdates.js serves.
const delays = {
morning: {
tripId: 'morning',
startDate: '20261101',
delaySec: null,
stops: [
{
stopId: null,
stopSequence: 2,
arrival: null,
departure: null,
relationship: 'SKIPPED',
},
],
},
};
const position = scheduledPosition(
schedule,
'morning',
MORNING + 450_000,
delays,
);
assert.equal(position.dwell, false);
assert.equal(position.prevStop, 0);
assert.equal(position.nextStop, 2);
assert.equal(position.s, 1500, 'halfway from A (+0) to C (+15)');
assert.equal(
tripStops(schedule, 'morning', MORNING, delays)[1].skipped,
true,
);
});
test('a canceled run leaves the active list', () => {
const schedule = createSchedule(pack());
const now = MORNING + 10 * MIN;
const canceled = { morning: { relationship: 'CANCELED' } };
assert.equal(activeTrips(schedule, now).length, 1);
assert.equal(activeTrips(schedule, now, { delays: canceled }).length, 0);
});
test('NO_DATA at a stop ends the delay carried to it: the timetable applies again', () => {
const schedule = createSchedule(pack());
const delays = {
morning: {
delaySec: 300,
stops: [
{ stopId: 'B1', arrival: { delay: 240 } },
{ stopId: 'C', relationship: 'NO_DATA' },
],
},
};
const stops = tripStops(schedule, 'morning', MORNING, delays);
assert.deepEqual(
stops.map((stop) => [stop.arrivalDelaySec, stop.departureDelaySec]),
[
[300, 300],
[240, 240],
// No prediction from C on: its time and D's are the timetable's.
[0, 0],
[0, 0],
],
);
assert.equal(stops[2].expectedArrival, stops[2].scheduledArrival);
// A later update after NO_DATA counts again.
const later = {
morning: {
stops: [
{ stopId: 'B1', arrival: { delay: 240 } },
{ stopId: 'C', scheduleRelationship: 'NO_DATA' },
{ stopSequence: 4, arrival: { delay: 60 } },
],
},
};
assert.deepEqual(
tripStops(schedule, 'morning', MORNING, later).map(
(stop) => stop.arrivalDelaySec,
),
[0, 240, 0, 60],
);
});
test('a pack ends when the last run it holds arrives', () => {
// The latest run: the shuttle on its last date, 2 November, at noon New
// York time for 10 minutes (31 October's 25:00 train arrives earlier).
const end = schedulePackEndMs(pack());
assert.equal(
end,
serviceDayOriginMs('2026-11-02', 'America/New_York') + 12 * H + 10 * MIN,
);
// Without a window the end is not known; with no run there is none.
assert.equal(schedulePackEndMs(pack({ window: undefined })), null);
assert.equal(schedulePackEndMs(pack({ trips: {} })), -Infinity);
assert.equal(schedulePackEndMs(null), null);
});
test('a live report states its own delay; a day or more either way is no delay', () => {
assert.equal(liveDelaySec({ delaySec: 540 }), 540);
assert.equal(liveDelaySec({ delaySec: -120 }), -120);
assert.equal(liveDelaySec({ delaySec: 0 }), 0);
for (const value of [null, undefined, '540', NaN, Infinity, 86_401, -86_401])
assert.equal(liveDelaySec({ delaySec: value }), null, String(value));
assert.equal(liveDelaySec(null), null);
});
+124
View File
@@ -0,0 +1,124 @@
/**
* @module railScheduleIndex
* @description Server-side lookup over a schedule pack (the
* `<feedId>.schedule.json` scripts/build-rail-paths.mjs writes) for a
* realtime source that names a train by its number (Irish Rail's train
* codes): `createRunIndex`, the dated runs of a pack's trips, so a reported
* train takes the trip it is on (its line and stops). The train's position
* is never taken from the timetable.
*
* Pure: no I/O. Times are epoch milliseconds; a run starts at the service
* day's GTFS origin (noon minus 12 h in the pack's time zone) plus the
* trip's start, and lasts its profile's last offset, as
* src/data/railSchedule.js reads them.
*/
import { serviceDayOriginMs } from './railTimeZones.js';
function packTrips(raw) {
return raw?.trips && typeof raw.trips === 'object' ? raw.trips : {};
}
function packServices(raw) {
return raw?.services && typeof raw.services === 'object' ? raw.services : {};
}
function serviceDates(services, service) {
const dates = services[service];
return Array.isArray(dates) ? dates : [];
}
/**
* The dated runs of a pack's trips. Built lazily per trip, so a source that
* looks up a few hundred train numbers never expands the whole pack. It
* keeps the parts of the pack it reads (trips, profiles, services), not the
* pack itself.
* @param {object} raw Decoded schedule pack.
* @returns {{runsOf: (tripId: string) => Array<{tripId: string, date: string,
* startMs: number, endMs: number}>,
* tripsNamed: (name: string) => string[],
* activeTrip: (name: string, nowMs: number,
* options?: {slackMs?: number, delayMs?: number}) =>
* ({tripId: string, date: string, startMs: number, endMs: number}|null)}}
*/
export function createRunIndex(raw) {
const trips = packTrips(raw);
const services = packServices(raw);
const zone = typeof raw?.timezone === 'string' ? raw.timezone : 'UTC';
const profiles = Array.isArray(raw?.profiles) ? raw.profiles : [];
const origins = new Map();
const originOf = (date) => {
let origin = origins.get(date);
if (origin === undefined) {
origin = serviceDayOriginMs(date, zone);
origins.set(date, origin);
}
return origin;
};
let names = null;
const runs = new Map();
function runsOf(tripId) {
let list = runs.get(tripId);
if (list) return list;
list = [];
const trip = Object.hasOwn(trips, tripId) ? trips[tripId] : null;
const profile = trip ? profiles[trip.profile] : null;
if (trip && Array.isArray(profile) && Number.isFinite(trip.startSec)) {
const durationMs = (Number(profile[profile.length - 1]) || 0) * 1000;
for (const date of serviceDates(services, trip.service)) {
const origin = originOf(date);
if (!Number.isFinite(origin)) continue;
const startMs = origin + trip.startSec * 1000;
list.push({ tripId, date, startMs, endMs: startMs + durationMs });
}
}
runs.set(tripId, list);
return list;
}
return {
runsOf,
tripsNamed(name) {
if (!names) {
names = new Map();
for (const [tripId, trip] of Object.entries(trips)) {
if (typeof trip?.name !== 'string' || !trip.name) continue;
const list = names.get(trip.name);
if (list) list.push(tripId);
else names.set(trip.name, [tripId]);
}
}
return names.get(String(name ?? '')) ?? [];
},
/**
* The run of a train number that is under way at `nowMs` (or within
* `slackMs` of its start or end); the one whose span is nearest when
* several qualify, null when none or when two tie. A train running
* `delayMs` late (negative: early) is where its timetable was that long
* ago, so its runs are compared at `nowMs - delayMs`.
*/
activeTrip(name, nowMs, { slackMs = 10 * 60_000, delayMs = 0 } = {}) {
const at = nowMs - (Number.isFinite(delayMs) ? delayMs : 0);
let best = null;
let bestScore = Infinity;
let tie = false;
for (const tripId of this.tripsNamed(name)) {
for (const run of runsOf(tripId)) {
if (at < run.startMs - slackMs || at > run.endMs + slackMs) continue;
const score =
at < run.startMs
? run.startMs - at
: at > run.endMs
? at - run.endMs
: 0;
if (score < bestScore) {
best = run;
bestScore = score;
tie = false;
} else if (score === bestScore && best && run.tripId !== best.tripId)
tie = true;
}
}
return tie ? null : best;
},
};
}
+188
View File
@@ -0,0 +1,188 @@
/**
* @module railSiriVm
* @description Adapter for SIRI Vehicle Monitoring (SIRI Lite `vm`, as
* Entur publishes it at `api.entur.io/realtime/v1/rest/vm`), turning each
* `<VehicleActivity>` into the train records the GTFS-RT path produces.
*
* Only the handful of elements a position needs are read (RecordedAtTime,
* ValidUntilTime, LineRef, VehicleMode, VehicleLocation, Bearing, Delay,
* VehicleJourneyRef, VehicleRef), with a small scanner instead of an XML
* library: no DTDs, no entity definitions, attributes ignored, an optional
* namespace prefix on any element. A document that declares a DTD or
* entities is refused, and every size is capped (whole text, activity count,
* field length).
*
* The positions are the operator's own GPS reports, each with the time it
* was recorded, so records carry `timestampSource: 'vehicle'`. An activity
* whose `ValidUntilTime` has passed is dropped, as is one whose
* `VehicleMode` says it is not a train (Entur lists replacement buses under
* the train lines).
*
* Pure: no Node built-ins, safe for node:test and any server runtime.
*/
import { isPlausibleVehiclePosition } from './gtfsRealtime.js';
import { decodeXmlText } from './irishRail.js';
/** Longest document read (characters). SJ Nord's is about 7 KB. */
export const SIRI_VM_MAX_TEXT_CHARS = 2 * 1024 * 1024;
/** Most activities read from one document. */
export const SIRI_VM_MAX_ACTIVITIES = 5000;
/** Longest accepted field (characters). */
export const SIRI_VM_MAX_FIELD_CHARS = 256;
/** Error for a document this adapter will not read. */
export class SiriVmFormatError extends Error {
constructor(message) {
super(message);
this.name = 'SiriVmFormatError';
}
}
/** `<Name>` or `<prefix:Name>` (no attributes kept), up to its close. */
function readField(block, name) {
const match = new RegExp(
`<(?:[A-Za-z_][\\w.-]*:)?${name}(?:\\s[^>]*)?>([^<]*)</(?:[A-Za-z_][\\w.-]*:)?${name}>`,
).exec(block);
if (!match || match[1].length > SIRI_VM_MAX_FIELD_CHARS * 6) return null;
const value = decodeXmlText(match[1]).trim();
return value.length <= SIRI_VM_MAX_FIELD_CHARS ? value : null;
}
/**
* Seconds in an ISO 8601 duration as SIRI writes a delay (`PT0S`,
* `PT2M30S`, `-PT45S`, `P1DT1H`), or null.
* @param {string|null} text
* @returns {number|null}
*/
export function parseIsoDurationSec(text) {
const match =
/^(-)?P(?:(\d+(?:\.\d+)?)D)?(?:T(?:(\d+(?:\.\d+)?)H)?(?:(\d+(?:\.\d+)?)M)?(?:(\d+(?:\.\d+)?)S)?)?$/.exec(
String(text ?? '').trim(),
);
if (!match || !(match[2] || match[3] || match[4] || match[5])) return null;
const [, sign, d = 0, h = 0, m = 0, s = 0] = match;
const total =
Number(d) * 86_400 + Number(h) * 3600 + Number(m) * 60 + Number(s);
return Math.round(sign ? -total : total);
}
/** Epoch seconds of an ISO date-time with its offset, or null. */
function epochSeconds(text) {
const ms = Date.parse(String(text ?? ''));
return Number.isFinite(ms) ? Math.floor(ms / 1000) : null;
}
/**
* Split a SIRI VM document into raw activity rows.
* @param {string} text
* @returns {{rows: Array<Record<string, string|null>>, truncated: boolean,
* responseTimestamp: string|null}}
*/
export function parseSiriVmXml(text) {
if (typeof text !== 'string') throw new SiriVmFormatError('not text');
if (text.length > SIRI_VM_MAX_TEXT_CHARS)
throw new SiriVmFormatError(`document is ${text.length} characters`);
if (/<!DOCTYPE|<!ENTITY/i.test(text))
throw new SiriVmFormatError('document declares a DTD');
if (!/<(?:[A-Za-z_][\w.-]*:)?VehicleMonitoringDelivery[\s>]/.test(text))
throw new SiriVmFormatError('not a vehicle monitoring document');
const open = /<(?:[A-Za-z_][\w.-]*:)?VehicleActivity(?:\s[^>]*)?>/g;
const close = /<\/(?:[A-Za-z_][\w.-]*:)?VehicleActivity>/g;
const rows = [];
let truncated = false;
let found;
while ((found = open.exec(text))) {
close.lastIndex = open.lastIndex;
const end = close.exec(text);
if (!end) break;
if (rows.length >= SIRI_VM_MAX_ACTIVITIES) {
truncated = true;
break;
}
const block = text.slice(open.lastIndex, end.index);
rows.push({
recordedAt: readField(block, 'RecordedAtTime'),
validUntil: readField(block, 'ValidUntilTime'),
lineRef: readField(block, 'LineRef'),
mode: readField(block, 'VehicleMode'),
latitude: readField(block, 'Latitude'),
longitude: readField(block, 'Longitude'),
bearing: readField(block, 'Bearing'),
delay: readField(block, 'Delay'),
journeyRef:
readField(block, 'VehicleJourneyRef') ??
readField(block, 'DatedVehicleJourneyRef'),
vehicleRef: readField(block, 'VehicleRef'),
lineName: readField(block, 'PublishedLineName'),
});
open.lastIndex = end.index + end[0].length;
}
const head = text.slice(0, 4096);
return {
rows,
truncated,
responseTimestamp: readField(head, 'ResponseTimestamp'),
};
}
/**
* Turn a SIRI VM document into unclassified train records in the shape the
* GTFS-RT path produces (src/data/gtfsRealtime.js), plus `delaySec`.
*
* `routeId` is the `LineRef` (Entur: the same `<codespace>:Line:<n>` ids as
* the static GTFS), `tripId` the `VehicleJourneyRef` (Entur: a
* `DatedServiceJourney`, not the GTFS trip id; the timetable matches the
* train to a run by line and position instead). One record per vehicle:
* a vehicle listed twice keeps its latest report.
*
* @param {string} text The XML document.
* @param {number} fetchedAtS Fetch time (epoch seconds), used when an
* activity carries no RecordedAtTime and to drop expired activities.
* @returns {{entityCount: number, truncated: boolean,
* feedTimestamp: number|null, vehicles: object[]}}
*/
export function decodeSiriVehicleMonitoring(text, fetchedAtS) {
const { rows, truncated, responseTimestamp } = parseSiriVmXml(text);
const byId = new Map();
for (const row of rows) {
if (row.mode && !/^rail$/i.test(row.mode)) continue;
const lat = Number.parseFloat(row.latitude ?? '');
const lon = Number.parseFloat(row.longitude ?? '');
if (!isPlausibleVehiclePosition(lat, lon)) continue;
const validUntil = epochSeconds(row.validUntil);
if (validUntil !== null && validUntil < fetchedAtS) continue;
const id = row.vehicleRef || row.journeyRef;
if (!id || !/^[\w:.-]{1,128}$/.test(id)) continue;
const recorded = epochSeconds(row.recordedAt);
const bearing = Number.parseFloat(row.bearing ?? '');
const record = {
id,
lat: Number(lat.toFixed(6)),
lon: Number(lon.toFixed(6)),
bearing:
Number.isFinite(bearing) && bearing >= 0 && bearing <= 360
? bearing
: null,
speedMps: null,
timestamp: recorded ?? fetchedAtS,
timestampSource: recorded === null ? 'fetch' : 'vehicle',
routeId: row.lineRef || null,
tripId: row.journeyRef || null,
directionId: null,
label: row.lineName || null,
stopId: null,
status: null,
occupancy: null,
delaySec: parseIsoDurationSec(row.delay),
};
const known = byId.get(id);
if (!known || record.timestamp >= known.timestamp) byId.set(id, record);
}
return {
entityCount: rows.length,
truncated,
feedTimestamp: epochSeconds(responseTimestamp),
vehicles: [...byId.values()],
};
}
Loaded 100 of 395 files, more files were not shown because too many files have changed in this diff. Show more