From 76c20be2773018164f274f3e48a25f857ceb1dd6 Mon Sep 17 00:00:00 2001 From: Bilawal Sidhu <106619546+bilawalsidhu@users.noreply.github.com> Date: Tue, 15 Sep 2026 05:18:30 -0500 Subject: [PATCH] feat(transit): live public-transit vehicles from open GTFS-Realtime feeds (#587) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Supersedes #218 by @spcpza, ported onto the current component layout with fixes from review. Verified before merge: format, package boundaries, unit suite, build, and the browser tracking suite; a browser harness for the change where one applies; adversarial review applied. Contributor: @spcpza. His original commit is the base of this branch, and his own re-port onto main (2026-09-15, `5b69011`) was consulted while finishing the height work. That port reads every vehicle's surface from the app's shared ground-floor system (`cachedGroundFloor` / `warmGroundFloor` / `GROUND_FLOOR_LIFT_M`) rather than sampling the scene per layer, keeps a vehicle hidden until its cell answers, warms a bounded batch per poll on a timer instead of per frame, and renders at the ellipsoid above the altitude gate. That is the sounder primitive — one floor system for the app, with the rooftop-versus-street question already settled against a datum-correct DEM prior — and this branch's height work was rebuilt on it, replacing a bespoke cell cache, ground prior and rooftop rule written here. Credit for that direction is his. Future work (not in this PR), both written up in docs/CURRENT-STATE.md: - Readable route names. The realtime feeds carry only `route_id`; `route_short_name` / `route_long_name` live in each operator's static GTFS `routes.txt`, and headsigns in `trips.txt`. Doing it properly wants a server-side metadata loader beside the transit proxy with fixed registered static sources, bounded indexes built once and persisted, a slow conditional refresh, and a versioned route dictionary the browser caches. Shipped here: ids are never rewritten, and a cut one carries an ellipsis instead of reading as a typo. - Below-ground rendering. Subway vehicles are drawn on the surface directly above themselves, because the feeds carry no depth and a train nobody can see is not worth drawing; the card says so plainly. Real treatment wants OSM tunnel geometry per city and a ghosted sprite tethered to the surface. The owner recorded it and left it. Round four and after (2026-09-15). The owner tested round three under FLIR and NVG, and then asked for the design to be taken further: elegant in performance, beautiful on camera, trails with a cache for public transport, Cesium-native, and a core other moving layers can adopt. Codex Astra reviewed the plan (it confirmed the speed fault: a glide started from the drawn position at receipt time but spanned the gap between report times, so with MBTA reporting every ~10 s against a 15 s poll every second glide ran at 1.5x; it rejected a queued-segment patch), then wrote the design, then implemented it in commit-sized passes with adversarial review between them and real-browser harness runs by the integrator. What this PR now contains beyond the contributor's original port: - Delayed playback in a contact-agnostic core (`src/data/contactPlayback.js`): each vehicle keeps a typed, bounded fix history and a display clock that runs at one second per second through it, a feed-derived lag behind real time, holding at its newest fix rather than extrapolating and never sprinting to catch up. Marker, course, motion word, card and trail all read the same displayed sample. The clock has an explicit time-offset and seek API so a rewind is a later feature, not a rewrite; flights and vessels are untouched. - A selected-vehicle trail (`src/data/contactTrailRenderer.js`, `src/layers/transit/trails.js`): the retained history drawn with batched Cesium primitives, clipped at display time so the head is exactly the sprite, faded by age, in the mode colour (white under sensor presets), on the same surface path as the marker. - Bounded proxy-side history (`server/providers/transitHistory.js`): fifteen minutes and at most 128 fixes per vehicle with hard byte ceilings, MBTA only for now (opt-in per feed), served by a small endpoint that never fetches upstream, so a selected trail survives a browser reload while the server keeps running. - Scene work bounded to what is visible and moving, an occluder, cached DETECT candidates, holds on the render loop only while something visible moves. - Sensor presets: white-hot haloed sprites under NVG/FLIR/noir (black-hot inverts), saturated mode colours under CRT, per-mode bracket tiers in every theme, the selected card keeping its mode colour. - A discriminating browser harness (`scripts/qa-transit.mjs`) that samples postRender frames, reads the real card anchor, and measures fleet budgets at 800 and 3,000 vehicles. Not built, recorded as follow-ups: curated route display names (lowest priority per the owner), fleet-wide tails, route snapping, a rewind UI, adoption of the playback core by flights and vessels. Verified at merge (2026-09-15 morning): unit suite 3,896 passing, build clean, format and boundary gates clean, tracking regression 109/109, browser harness keyless 200 pass / 0 fail (13 sensor checks unexercised because the live view offered fewer than six isolated sprites; every judged sprite passed), keyed on Google 3D tiles 198 pass with the per-frame allocation gate as the only failure. Follow-ups recorded: the keyed build's playback sampler still allocates per frame (CPU and frame time stay within budget; the 3,000-vehicle metro view sits at a 27.6 ms frame interval against a 25 ms budget), the shared detection overlay's per-paint allocation, friendlier route names, fleet-wide tails, a rewind UI, and adopting the playback core for vessels and flights. Co-authored-by: spcpza --- CHANGELOG.md | 22 + DATA_SOURCES.md | 8 + README.md | 168 +- SECURITY.md | 1 + docs/CURRENT-STATE.md | 158 + package.json | 11 +- scripts/format-scope.json | 49 +- scripts/package-boundaries.json | 170 +- scripts/qa-transit-browser.mjs | 595 ++++ scripts/qa-transit-controls.mjs | 204 ++ scripts/qa-transit-scenes.mjs | 583 ++++ scripts/qa-transit.mjs | 1951 ++++++++++++ server/providers/common/http.js | 44 + server/providers/local.js | 2 + server/providers/transit.js | 427 +++ server/providers/transitHistory.js | 493 +++ server/providers/transitHistory.test.mjs | 179 ++ src/app/catalog.js | 1 + src/app/catalog.test.mjs | 4 +- src/app/constructCatalog.js | 2 + src/app/constructCatalog.test.mjs | 2 +- src/app/layers/transit.js | 48 + src/app/layers/transit.test.mjs | 93 + src/data/cameraSensitivity.js | 93 + src/data/contactPlayback.js | 691 ++++ src/data/contactPlayback.test.mjs | 302 ++ src/data/contactTrailRenderer.js | 229 ++ src/data/contactTrailRenderer.test.mjs | 210 ++ src/data/dataCredits.js | 25 + src/data/detection.js | 27 +- src/data/detectionDraw.js | 21 + src/data/detectionDraw.test.mjs | 31 + src/data/detectionHost.test.mjs | 88 +- src/data/gtfsRealtime.js | 274 ++ src/data/gtfsRealtime.test.mjs | 348 +++ src/data/iconOrientation.js | 164 +- src/data/iconOrientation.test.mjs | 395 ++- src/data/layerState.js | 1 + src/data/layerState.test.mjs | 4 +- src/data/spriteOrder.js | 2 + src/data/transit.js | 56 + src/data/transit.test.mjs | 225 ++ src/data/transitFeeds.js | 404 +++ src/data/transitFeeds.test.mjs | 241 ++ src/data/transitIcons.js | 214 ++ src/data/transitIcons.test.mjs | 136 + src/data/transitPresetStyle.js | 147 + src/data/transitPresetStyle.test.mjs | 96 + src/data/transitProxy.js | 401 +++ src/data/transitProxy.test.mjs | 367 +++ src/layers/bikeshare/lifecycle.js | 14 +- src/layers/traffic/lifecycle.js | 19 +- src/layers/transit/controls.js | 18 + src/layers/transit/height.js | 257 ++ src/layers/transit/index.js | 70 + src/layers/transit/ingestion.js | 513 +++ src/layers/transit/lifecycle.js | 303 ++ src/layers/transit/lifecycle.test.mjs | 3641 ++++++++++++++++++++++ src/layers/transit/movement.js | 313 ++ src/layers/transit/movement.test.mjs | 146 + src/layers/transit/policy.js | 565 ++++ src/layers/transit/qaMetrics.js | 331 ++ src/layers/transit/qaMetrics.test.mjs | 330 ++ src/layers/transit/queries.js | 237 ++ src/layers/transit/rendering.js | 598 ++++ src/layers/transit/selection.js | 209 ++ src/layers/transit/state.js | 126 + src/layers/transit/testing.js | 340 ++ src/layers/transit/trails.js | 328 ++ src/layers/transit/viewport.js | 137 + src/overlays/worldOverlay.js | 2 + src/overlays/worldOverlay.test.mjs | 2 + src/overlays/worldOverlayTokens.js | 68 +- src/overlays/worldOverlayTokens.test.mjs | 24 + src/tooling/transitProvider.test.mjs | 386 +++ src/tooling/transitQa.test.mjs | 631 ++++ src/ui/applicationShell.js | 2 + src/ui/visualSettings.js | 11 + 78 files changed, 19842 insertions(+), 186 deletions(-) create mode 100644 scripts/qa-transit-browser.mjs create mode 100644 scripts/qa-transit-controls.mjs create mode 100644 scripts/qa-transit-scenes.mjs create mode 100644 scripts/qa-transit.mjs create mode 100644 server/providers/transit.js create mode 100644 server/providers/transitHistory.js create mode 100644 server/providers/transitHistory.test.mjs create mode 100644 src/app/layers/transit.js create mode 100644 src/app/layers/transit.test.mjs create mode 100644 src/data/cameraSensitivity.js create mode 100644 src/data/contactPlayback.js create mode 100644 src/data/contactPlayback.test.mjs create mode 100644 src/data/contactTrailRenderer.js create mode 100644 src/data/contactTrailRenderer.test.mjs create mode 100644 src/data/gtfsRealtime.js create mode 100644 src/data/gtfsRealtime.test.mjs create mode 100644 src/data/transit.js create mode 100644 src/data/transit.test.mjs create mode 100644 src/data/transitFeeds.js create mode 100644 src/data/transitFeeds.test.mjs create mode 100644 src/data/transitIcons.js create mode 100644 src/data/transitIcons.test.mjs create mode 100644 src/data/transitPresetStyle.js create mode 100644 src/data/transitPresetStyle.test.mjs create mode 100644 src/data/transitProxy.js create mode 100644 src/data/transitProxy.test.mjs create mode 100644 src/layers/transit/controls.js create mode 100644 src/layers/transit/height.js create mode 100644 src/layers/transit/index.js create mode 100644 src/layers/transit/ingestion.js create mode 100644 src/layers/transit/lifecycle.js create mode 100644 src/layers/transit/lifecycle.test.mjs create mode 100644 src/layers/transit/movement.js create mode 100644 src/layers/transit/movement.test.mjs create mode 100644 src/layers/transit/policy.js create mode 100644 src/layers/transit/qaMetrics.js create mode 100644 src/layers/transit/qaMetrics.test.mjs create mode 100644 src/layers/transit/queries.js create mode 100644 src/layers/transit/rendering.js create mode 100644 src/layers/transit/selection.js create mode 100644 src/layers/transit/state.js create mode 100644 src/layers/transit/testing.js create mode 100644 src/layers/transit/trails.js create mode 100644 src/layers/transit/viewport.js create mode 100644 src/overlays/worldOverlayTokens.test.mjs create mode 100644 src/tooling/transitProvider.test.mjs create mode 100644 src/tooling/transitQa.test.mjs diff --git a/CHANGELOG.md b/CHANGELOG.md index 515c20845..4efe85ba9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,17 @@ # Changelog +- Widen opaque sensor halos to 3 display pixels with a 30 px fleet core while retaining at least 60% opaque core coverage, adding contrast margin through thermal blur and bloom on bright roofs. + +- Exclude Noir's intentionally vignetted outer field from transit core measurements and add separate Boston live oblique selection/readability coverage. + +- Judge NVG transit readability by phosphor peak and halo contrast, use opaque black sensor halos, reject overlapping pixel samples, and report each selected-trail acceptance condition. Simplify transit source credits and ground-placement wording. + +- Reserve numeric playback scratch before sampling, including resolved transit surface heights, and reuse the same output and segment objects on every frame. + +- Keep moving and stationary transit above CCTV/Bikeshare consistently, and share camera sensitivity across the actual Traffic, Bikeshare and Transit lifecycles. + +- Reject isolated transit history outliers without changing the active playback bracket; rejected live reports retain the last accepted route, mode and detection text. + - Add an optional Nominatim geocoding adapter with configurable search/reverse endpoints, cancellation, bounded responses and retryable upstream errors. Extract portable response-reading and Overpass lexical helpers while retaining existing server exports. - Expose reference feed factories independently of standalone catalog construction; preserve source choices and asset attribution. @@ -7,6 +19,7 @@ - Add source-only layer exports and enforce source, browser, standalone and voice import directions. Move plain record/feed helpers and settings filesystem hardening to their owners while preserving compatibility and behavior. - Separate voice session lifetime and common controls from the default Realtime protocol adapter. +- Add live public transit: vehicles from seven open GTFS-Realtime feeds played back a lag behind real time at their reported speed, a selected-vehicle trail with bounded MBTA history caching, mode-coloured detection brackets in every preset, and sprites that stay readable in night-vision and thermal views. - Separate canonical voice action arguments from descriptive wording, preserving the existing Realtime tool inventory. - Expose portable radio, camera-type and regional source helpers; keep HTTP transport separate from record normalization. @@ -226,6 +239,15 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md address grid — so headings use the shared id-hash fallback at low confidence and are corrected with the calibration gizmo. `CCTV_CALGARY_MAX_SOURCES` sets the cap and `CCTV_CALGARY_ENABLED=0` turns the pack off. +- **Transit layer** — keyless buses, trams, subways, trains and ferries in + Boston, Austin, Minneapolis–St Paul, Helsinki, the Netherlands, Norway and + South East Queensland. Vehicles use delayed timestamp playback and explicit + waiting states. Selection shows available recent history, mode-coloured cards + and clear report ages; MBTA history can survive a browser reload while the + proxy remains running. Heights are aligned to work with Google 3D tiles. + Subways are projected to street level and their cards explain that choice. + Visible animation, detection membership, history storage and proxy requests + are bounded. Share links carry Transit as token `j`. - Add Ontario 511 as a keyless CCTV source pack, including Kitchener-area highway cameras, with server-registered still URLs and attribution. diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 9c3262770..1de1b9dc1 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -46,6 +46,13 @@ How to read this: | **DriveBC** (Government of British Columbia) | CCTV camera catalog + frames, British Columbia highways | [Open Government Licence – British Columbia](https://www2.gov.bc.ca/gov/content/data/open-data/open-government-licence-bc) — attribution REQUIRED | "DriveBC. Contains information licensed under the Open Government Licence – British Columbia" | | **Open Calgary** (`data.calgary.ca`) | CCTV camera catalog + frames, Calgary | [Open Government Licence – City of Calgary](https://data.calgary.ca/stories/s/Open-Calgary-Terms-of-Use/u45n-7awa) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – City of Calgary" | | **GBFS (Lyft / BCycle)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` | +| **MBTA / MassDOT** (cdn.mbta.com) | Transit layer live vehicles and up to 15 minutes of recently observed positions, Greater Boston | MBTA / MassDOT open realtime vehicle data, used for the live view and up to fifteen minutes of recently observed positions; published for developer use and not against their use. | Attribution to MBTA / MassDOT (courtesy text) | +| **CapMetro** (data.texas.gov) | Transit layer live vehicles, Austin TX | CapMetro open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to CapMetro (courtesy text) | +| **Metro Transit** (Metropolitan Council) | Transit layer live vehicles, Minneapolis–St Paul | Metro Transit / Metropolitan Council open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to Metro Transit / Metropolitan Council (courtesy text) | +| **OVapi / Stichting OpenGeo** | Transit layer live vehicles, Netherlands | OVapi / Stichting OpenGeo open realtime vehicle data, used for the live view; published for developer use and not against their use. The client sends a User-Agent and conditional requests. | Attribution to OVapi / Stichting OpenGeo (courtesy text) | +| **Entur** (api.entur.io) | Transit layer live vehicles, Norway | Entur open realtime vehicle data, used for the live view; published for developer use and not against their use. The client sends the `ET-Client-Name` identifying header. | Attribution to Entur (courtesy text) | +| **TransLink** (Queensland Government) | Transit layer live vehicles, South East Queensland | TransLink / Queensland Government open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to TransLink / Queensland Government (courtesy text) | +| **HSL** (realtime.hsl.fi) | Transit layer live vehicles, Helsinki | HSL (Helsinki Region Transport) open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to HSL (Helsinki Region Transport) (courtesy text) | | **Radio Browser** | Geolocated internet-radio station directory and station-level tags | Public-domain directory data under PDDL 1.0; individual broadcaster stream terms apply | "Radio Browser" plus a link to the selected broadcaster | | **Re:Earth Terrain** (Mapterhorn) | Terrain (keyless globe stacks — OSM etc. — + `/api/terrain/heights` ellipsoidal-height lookups) | Terrain mesh: CC BY 4.0; geoid: EGM2008 (NGA, public domain) | "Terrain (keyless globe stacks): Re:Earth Terrain / Mapterhorn (CC BY 4.0) / EGM2008 (NGA)" | | **OSRM on the FOSSGIS routing servers** (`routing.openstreetmap.de`) | Street-following routes for the Directions layer and voice route annotations, via `/api/route` | [FOSSGIS routing usage policy](https://routing.openstreetmap.de/about.html): "Display the required attribution and display a link to 'fix the map'", "Use a valid user agent and, if applicable, a correct referrer", "One request per second max", "No scraping, no heavy usage". The full policy is the German [FOSSGIS Nutzungsbedingungen](https://www.fossgis.de/arbeitsgruppen/osm-server/nutzungsbedingungen/); FOSSGIS also states that the server may be embedded in your own pages but "eine gewerbliche Nutzung ist nur mit Einschränkungen erlaubt" (commercial use only with restrictions). Route data derives from OpenStreetMap (ODbL 1.0) | "Routing: OSRM on the FOSSGIS servers" + "© OpenStreetMap contributors" + a "fix the map" link — shown in the Data attribution popover | @@ -67,6 +74,7 @@ How to read this: - **Live Traffic NSW cameras.** The camera list is the keyless public `data.livetraffic.com/cameras/traffic-cam.json` feed (Transport for NSW open data, CC BY 4.0); frames are stills on `webcams.transport.nsw.gov.au`, and only that host is registered. That image host answers non-browser clients with an HTML placeholder, so the proxy identifies as a browser for that one host and as itself everywhere else. Frames are fetched at request time and never stored or redistributed. Disable with `CCTV_NSW_ENABLED=0`. - **Open Calgary traffic cameras.** Keyless. The camera list is the public Socrata dataset `k7p9-kppz` on `data.calgary.ca`; frames are stills on the city's own `trafficcam.calgary.ca` host, and only that host is registered. Most rows publish an `http://` frame URL and the host redirects those to HTTPS, so URLs are upgraded and pinned to that origin before registration. Frames are fetched at request time and never stored or redistributed. The dataset publishes no camera facing: the `quadrant` field and the quadrant suffix on each camera name are Calgary's address grid, not a bearing, so headings use the shared id-hash fallback at low confidence and are corrected with the in-app calibration gizmo. The licence requires the attribution statement above. Disable with `CCTV_CALGARY_ENABLED=0`. - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals. +- **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All seven registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. - **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply. - **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 40,000, a configurable application safety ceiling, not a guarantee of staying within TomTom's monthly free allowance; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app). - **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line. diff --git a/README.md b/README.md index 5b0622ba5..6558a10b5 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Photorealistic 3D globe. Live aircraft, ships, satellites, earthquakes, traffic, and public cameras. Hands-free voice control powered by a realtime AI agent. -*No place left behind.* +_No place left behind._ ![Orbital HUD, a tracked live globe, FLIR terrain — then OPEN SOURCED](docs/media/hero-open-source-reveal.gif) @@ -16,7 +16,7 @@ Photorealistic 3D globe. Live aircraft, ships, satellites, earthquakes, traffic, The God's Eye View video series on YouTube -▶️ **From the project behind the viral God's Eye View series** *(formerly WorldView)* — [5M+ on YouTube](https://youtube.com/playlist?list=PL6qSg2I-7_koPbDnSMo0QeeHX_RknA2uv&si=nBGYMoHWQw41v93Q) · [25M+ across socials](https://www.google.com/search?q=god%27s+eye+view) +▶️ **From the project behind the viral God's Eye View series** _(formerly WorldView)_ — [5M+ on YouTube](https://youtube.com/playlist?list=PL6qSg2I-7_koPbDnSMo0QeeHX_RknA2uv&si=nBGYMoHWQw41v93Q) · [25M+ across socials](https://www.google.com/search?q=god%27s+eye+view) [![Reached #1 on GitHub Trending](https://img.shields.io/badge/%231_GitHub_Trending-thank_you!-F0A63C?style=flat-square&logo=github)](https://x.com/bilawalsidhu/status/2093798887815348521) @@ -24,7 +24,7 @@ Photorealistic 3D globe. Live aircraft, ships, satellites, earthquakes, traffic, **[#8 Product of the Day](https://www.producthunt.com/products/god-s-eye-view?launch=god-s-eye-view)** · Hunted by Chris Messina, creator of the hashtag -*“pretty cool”* — [Brendan Eich](https://x.com/BrendanEich/status/2094592096401490266), creator of JavaScript and co-founder of Mozilla and Brave · Featured on **[Pinokio](https://pinokio.co/posts/01m1m4p9xxm3qw7dnnpj2wr93g)** +_“pretty cool”_ — [Brendan Eich](https://x.com/BrendanEich/status/2094592096401490266), creator of JavaScript and co-founder of Mozilla and Brave · Featured on **[Pinokio](https://pinokio.co/posts/01m1m4p9xxm3qw7dnnpj2wr93g)** ⚡ **Start without API keys.** Install with [Pinokio](https://pinokio.co/apps/github-com-bilawalsidhu-gods-eye-view) or run locally from the terminal. Add optional keys inside the app. **[→ Quick Start](#-quick-start)** @@ -146,10 +146,10 @@ reopens the same panel. - **Where keys land:** Pinokio → the app's ignored `pinokio/ENVIRONMENT`; a terminal clone → the repo-root `.env`. Either file is made owner-only - *before* a secret is written into it. These are local plaintext files, + _before_ a secret is written into it. These are local plaintext files, excluded from Git; the app uses your keys to contact the providers. - **Keys you already have stay yours:** values from your shell or the macOS - Keychain show as *configured externally* and are read-only to the panel. + Keychain show as _configured externally_ and are read-only to the panel. - **What to get first:** the free [Cesium ion](https://cesium.com/ion) token (eligible personal, non-commercial use; current terms and quotas apply) for photorealistic 3D and world terrain; a Google Maps key only for the @@ -188,7 +188,7 @@ Choose a first-run mission, or try these in order. The GIFs show Google Photorea ![Moving from a full airport overhead down to close taxiway inspection with 3D flight models](docs/media/start-here/airport-ground-traffic-google-3d.gif) -4. **Look through a public camera.** Turn on **CCTV** over Austin, London, California, or Finland. The feeds aren't webcam embeds — they project *into* the 3D city. Cycle coverage to **VIEWSHED** and every camera draws its estimated coverage volume — where it reaches, and where it goes blind. +4. **Look through a public camera.** Turn on **CCTV** over Austin, London, California, or Finland. The feeds aren't webcam embeds — they project _into_ the 3D city. Cycle coverage to **VIEWSHED** and every camera draws its estimated coverage volume — where it reaches, and where it goes blind. ![Diving into an Austin intersection with a live public camera projected into the 3D scene](docs/media/03-austin-cctv.gif) @@ -200,8 +200,8 @@ Choose a first-run mission, or try these in order. The GIFs show Google Photorea ![Cycling a dense live globe through CRT, FLIR, and NVG in one continuous view](docs/media/01-style-sweep.gif) -7. **Talk to it** *(needs an OpenAI key)*: *"Take me to LAX and select the nearest airborne aircraft."* -8. **Come home.** Hit **Reset Globe** — or just say *"zoom out to a globe view."* +7. **Talk to it** _(needs an OpenAI key)_: _"Take me to LAX and select the nearest airborne aircraft."_ +8. **Come home.** Hit **Reset Globe** — or just say _"zoom out to a globe view."_ **Keyboard:** `1`–`7` visual styles · `H` HUD · `D` detection · `C` cockpit · `Esc` out. @@ -219,7 +219,7 @@ The cockpit even carries its own briefing strip: nearby live signals, regional h ![A live military contact ridden through Normal, NVG, and Ironbow FLIR with dense detection](docs/media/start-here/military-cockpit-dense-google-3d.gif) -*Why cockpit mode exists: you're riding a real aircraft over real terrain — and you get to pick which sensor you see the world through.* +_Why cockpit mode exists: you're riding a real aircraft over real terrain — and you get to pick which sensor you see the world through._ --- @@ -229,19 +229,21 @@ The cockpit even carries its own briefing strip: nearby live signals, regional h Click **GEV MIC**, grant the microphone, and just talk. This is more than a voice-controlled remote: -- **🧠 It knows what it's looking at.** The agent pulls live scene context before answering — including coordinates, street names, active layers, and view scale. Ask *"what city is this?"* mid-flight and it knows. -- **🎯 Entity Q&A.** Click any plane, ship, or datacenter and ask *"what's this?"* It answers using the object's live telemetry. +- **🧠 It knows what it's looking at.** The agent pulls live scene context before answering — including coordinates, street names, active layers, and view scale. Ask _"what city is this?"_ mid-flight and it knows. +- **🎯 Entity Q&A.** Click any plane, ship, or datacenter and ask _"what's this?"_ It answers using the object's live telemetry. - **👁️ Visual grounding.** At street level, it reads a viewport screenshot to identify legible signage and building names, and is instructed never to hallucinate labels. -- **🎬 Cinematic framing.** *"Show me the planes overhead"* pulls the camera back, angles it, and frames the live traffic like a director. +- **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director. - **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token. Twenty-eight tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook: **🎥 Direct it** — drone-operator camera verbs: -> 🗣️ *"Take me to Tokyo."* · *"Orbit around this area slowly."* · *"Draw the walking route from the Capitol to Zilker Park."* → *"Fly the route we just drew."* · *"Zoom out to a globe view."* + +> 🗣️ _"Take me to Tokyo."_ · _"Orbit around this area slowly."_ · _"Draw the walking route from the Capitol to Zilker Park."_ → _"Fly the route we just drew."_ · _"Zoom out to a globe view."_ **🖊️ Annotate it** — a whiteboard over the real world: -> 🗣️ *"Outline the state of Texas."* · *"Annotate the Texas State Capitol and its grounds"* — it draws the **actual enclosing boundary**, not a circle. · *"How far is the Eiffel Tower from the Louvre?"* — a connector arrow appears and it speaks the distance. Everything persists until you say *"clear the map."* + +> 🗣️ _"Outline the state of Texas."_ · _"Annotate the Texas State Capitol and its grounds"_ — it draws the **actual enclosing boundary**, not a circle. · _"How far is the Eiffel Tower from the Louvre?"_ — a connector arrow appears and it speaks the distance. Everything persists until you say _"clear the map."_ **✍️ Or draw it yourself** — DISPLAY ▸ **Draw**: pick Area, Line or Pin, click the vertices on the real world, double-click to finish, label it. Same whiteboard, same persistence, no microphone needed. @@ -250,52 +252,56 @@ Twenty-eight tools, four jobs — the commands below come straight from the prod ![A spoken distance measurement spanning an airport, inspected from orbit](docs/media/04-airport-distance.gif) **🔎 Interrogate it** — analyst queries against the live layers: -> 🗣️ *"How many flights are over Texas right now?"* · *"Which ships are headed to Oakland?"* · *"What is the biggest fire near Los Angeles?"* · *"Is anything flying above forty thousand feet?"* · *"When does the ISS pass over next?"* + +> 🗣️ _"How many flights are over Texas right now?"_ · _"Which ships are headed to Oakland?"_ · _"What is the biggest fire near Los Angeles?"_ · _"Is anything flying above forty thousand feet?"_ · _"When does the ISS pass over next?"_ **🎛️ Operate it** — the whole console, hands-free: -> 🗣️ *"Switch to night vision and turn on the flights layer."* · *"Turn on the camera viewsheds."* · *"Play a news radio station near Austin."* · *"Track that plane."* → *"Enter Cockpit."* + +> 🗣️ _"Switch to night vision and turn on the flights layer."_ · _"Turn on the camera viewsheds."_ · _"Play a news radio station near Austin."_ · _"Track that plane."_ → _"Enter Cockpit."_ **And the rapid-fire tier** — one sentence each: -> 🗣️ *"Show me global infrastructure."* (stages the layers and pulls back to the globe) · *"Play Orbital Watch."* (a full cinematic scene) · *"Set detection density to fifty percent."* · *"Next contact — helicopters only."* (mid-cockpit) · *"Show me space missions."* · *"Switch to OSM."* · *"Sharpen the image a touch."* · *"Switch to the tactical layout."* · *"What's turned on right now?"* + +> 🗣️ _"Show me global infrastructure."_ (stages the layers and pulls back to the globe) · _"Play Orbital Watch."_ (a full cinematic scene) · _"Set detection density to fifty percent."_ · _"Next contact — helicopters only."_ (mid-cockpit) · _"Show me space missions."_ · _"Switch to OSM."_ · _"Sharpen the image a touch."_ · _"Switch to the tactical layout."_ · _"What's turned on right now?"_ ![The globe populating with the world's radio stations as another live layer](docs/media/15-global-radio-layer.gif) -*Ask for radio near anywhere and the globe starts broadcasting — every station is a real place you can fly to.* +_Ask for radio near anywhere and the globe starts broadcasting — every station is a real place you can fly to._ --- ## 🛰️ What's on the Globe -Fourteen layers and map sources. **Twelve have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.) +Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.) -| Layer | What you get | Source | Auth | -|-------|--------------|--------|------| -| 🗺️ **Map Stack** | Esri satellite imagery, Google Photorealistic 3D, OSM, plus additional ion-hosted stacks | Esri / Google / Ion / OSM | 🟢 Esri satellite + OSM · 🟡 ion-hosted Google 3D + world terrain · 🔴 direct Google + place search | -| ✈️ **Live Flights** | 11,000+ live aircraft + route history | OpenSky + adsb.lol | 🟢 (🟡 optional for more polling credits) | -| 🎖️ **Military Flights** | ADS-B military traffic in amber | adsb.lol | 🟢 | -| 🚢 **Live Vessels** | Thousands of ships worldwide | AISStream | 🟡 | -| 🛰️ **Satellites** | 838-object catalog, color-coded by class with a live legend — the **DENSE** chip drops in the whole Starlink shell | CelesTrak | 🟢 | -| 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 | -| 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds | -| 📹 **CCTV Mesh** | ~3,600 public cameras projected *into* the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · New South Wales (Live Traffic NSW) · Calgary. Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | -| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 | -| 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 | -| 🧭 **Directions** | Click A and B on the globe for a street-following drive, walk or cycle route draped on the terrain with turn-by-turn steps — then FLY the camera along it. No key, no geocoder, no mic | OSRM on FOSSGIS servers (OpenStreetMap) | 🟢 | -| 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 | -| 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🟡 optional token raises the allowance) | -| 🎖️ **Mapped Installations** | Viewport-bounded military-site context from community mapping — incomplete by nature, and labeled that way | OpenStreetMap | 🟢 | +| Layer | What you get | Source | Auth | +| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------- | +| 🗺️ **Map Stack** | Esri satellite imagery, Google Photorealistic 3D, OSM, plus additional ion-hosted stacks | Esri / Google / Ion / OSM | 🟢 Esri satellite + OSM · 🟡 ion-hosted Google 3D + world terrain · 🔴 direct Google + place search | +| ✈️ **Live Flights** | 11,000+ live aircraft + route history | OpenSky + adsb.lol | 🟢 (🟡 optional for more polling credits) | +| 🎖️ **Military Flights** | ADS-B military traffic in amber | adsb.lol | 🟢 | +| 🚢 **Live Vessels** | Thousands of ships worldwide | AISStream | 🟡 | +| 🛰️ **Satellites** | 838-object catalog, color-coded by class with a live legend — the **DENSE** chip drops in the whole Starlink shell | CelesTrak | 🟢 | +| 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 | +| 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds | +| 📹 **CCTV Mesh** | ~3,600 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · New South Wales (Live Traffic NSW) · Calgary. Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 | +| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 | +| 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 | +| 🧭 **Directions** | Click A and B on the globe for a street-following drive, walk or cycle route draped on the terrain with turn-by-turn steps — then FLY the camera along it. No key, no geocoder, no mic | OSRM on FOSSGIS servers (OpenStreetMap) | 🟢 | +| 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland | Operator GTFS-Realtime feeds | 🟢 | +| 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 | +| 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🟡 optional token raises the allowance) | +| 🎖️ **Mapped Installations** | Viewport-bounded military-site context from community mapping — incomplete by nature, and labeled that way | OpenStreetMap | 🟢 | **The basemap ladder — what each tier buys you:** -| You have | The globe you get | -|---|---| -| 🟢 Nothing | Esri World Imagery satellite basemap + keyless terrain, in 2D. OSM takes over automatically if Esri is unreachable; if terrain is unavailable the globe continues without it | -| 🟡 A free Cesium ion token | **Google Photorealistic 3D cities** and world terrain — eligible personal, non-commercial use; current ion terms and quotas apply | -| 🔴 A Google Maps key | The same 3D direct from Google, plus in-app place search — the billing-enabled, metered route | +| You have | The globe you get | +| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 🟢 Nothing | Esri World Imagery satellite basemap + keyless terrain, in 2D. OSM takes over automatically if Esri is unreachable; if terrain is unavailable the globe continues without it | +| 🟡 A free Cesium ion token | **Google Photorealistic 3D cities** and world terrain — eligible personal, non-commercial use; current ion terms and quotas apply | +| 🔴 A Google Maps key | The same 3D direct from Google, plus in-app place search — the billing-enabled, metered route | ![A reconstructed Falcon 9 ascent climbing and curving into its projected orbit](docs/media/08-falcon9-replay.gif) -*The Space Missions layer replaying a Falcon 9 ascent — labeled `RECONSTRUCTED ESTIMATE`, scrubbable 0.25×–4×.* +_The Space Missions layer replaying a Falcon 9 ascent — labeled `RECONSTRUCTED ESTIMATE`, scrubbable 0.25×–4×._ **Also on the globe:** neighborhood overlays · an optional cockpit WX cloud effect. **Bundled static infrastructure:** Datacenters (4,351), Dams (704), and Submarine Cables (712). @@ -309,33 +315,33 @@ Fourteen layers and map sources. **Twelve have a keyless path.** Some offer addi Once the basics click, run these: -| Mission | How | -|---|---| -| **🚁 Ask the planet** | *"Why are all these military helicopters flying in circles?"* Select a military track — it silently backfills ~24 h of real trace history — and see what it's been doing, resolved as stacked 3D loops. | -| **✈️ Final approach** | Click-track an airliner lining up for a runway, hop into the **cockpit**, and ride it down. | -| **🌃 Night watch** | Fly to your own city, switch to **NVG**, and let the detection mesh and HUD read the scene. | -| **🚢 Port call** | Vessels on over the Port of Long Beach. Click a tanker for its tactical card and wake trail — then hit **NEAREST** in the CCTV panel and look at the same water through a public camera. | -| **📻 Tokyo FM** | Orbit Shibuya with the **Radio** layer on — then drag the analog tuner needle: every position snaps to a real station and the globe flies to whoever's broadcasting. | -| **🔥 Fire line** | FIRMS over California. Click a detection — the camera dives to it — read the intensity, then hit **NEAREST** in the CCTV panel for a ground view. | -| **🚶 Ask for a walking route** *🎙️* | Tell the world where you want to go and watch a real street-following route trace itself through the 3D city — then *"fly it"*: banked turns, eased ends, a camera that leads the path like a drone shot. | -| **📏 Measure LAX to DFW** *🎙️* | *"How far is LAX from DFW?"* — an arrow spans the country, the distance lands in the caption, and the endpoints stay pinned to the real world as you orbit. | -| **🚀 Launch replay** | Open **Space Missions**, pick a launch from the last 30 days, and ride the T-minus countdown through ascent to orbit — scrub it at 0.25×–4×. Labeled `RECONSTRUCTED ESTIMATE`, because it is one. | -| **🪦 Walk the boneyard** | Fly from regional context down into dense, fully resolved rows of retired aircraft. | -| **🏗️ Orbit Three Gorges** | Sweep the dam and its terrain at a glance — then flip on the **Dams** layer and find 703 more. | +| Mission | How | +| ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **🚁 Ask the planet** | _"Why are all these military helicopters flying in circles?"_ Select a military track — it silently backfills ~24 h of real trace history — and see what it's been doing, resolved as stacked 3D loops. | +| **✈️ Final approach** | Click-track an airliner lining up for a runway, hop into the **cockpit**, and ride it down. | +| **🌃 Night watch** | Fly to your own city, switch to **NVG**, and let the detection mesh and HUD read the scene. | +| **🚢 Port call** | Vessels on over the Port of Long Beach. Click a tanker for its tactical card and wake trail — then hit **NEAREST** in the CCTV panel and look at the same water through a public camera. | +| **📻 Tokyo FM** | Orbit Shibuya with the **Radio** layer on — then drag the analog tuner needle: every position snaps to a real station and the globe flies to whoever's broadcasting. | +| **🔥 Fire line** | FIRMS over California. Click a detection — the camera dives to it — read the intensity, then hit **NEAREST** in the CCTV panel for a ground view. | +| **🚶 Ask for a walking route** _🎙️_ | Tell the world where you want to go and watch a real street-following route trace itself through the 3D city — then _"fly it"_: banked turns, eased ends, a camera that leads the path like a drone shot. | +| **📏 Measure LAX to DFW** _🎙️_ | _"How far is LAX from DFW?"_ — an arrow spans the country, the distance lands in the caption, and the endpoints stay pinned to the real world as you orbit. | +| **🚀 Launch replay** | Open **Space Missions**, pick a launch from the last 30 days, and ride the T-minus countdown through ascent to orbit — scrub it at 0.25×–4×. Labeled `RECONSTRUCTED ESTIMATE`, because it is one. | +| **🪦 Walk the boneyard** | Fly from regional context down into dense, fully resolved rows of retired aircraft. | +| **🏗️ Orbit Three Gorges** | Sweep the dam and its terrain at a glance — then flip on the **Dams** layer and find 703 more. | -*🎙️ = voice missions — they need an OpenAI key.* +_🎙️ = voice missions — they need an OpenAI key._ ![Resolving a selected aircraft's recent flight path into stacked 3D loops above the terrain](docs/media/07-helicopter-loops.gif) -*Ask the planet: a military contact's last ~24 hours of real trace history, resolved as stacked 3D loops.* +_Ask the planet: a military contact's last ~24 hours of real trace history, resolved as stacked 3D loops._ ![Asking for a walking route and flying the generated path through the 3D city](docs/media/10-walking-route-flythrough.gif) -*"Draw the walking route… now fly it" — banked turns, eased ends, the camera leading the path like a drone shot.* +_"Draw the walking route… now fly it" — banked turns, eased ends, the camera leading the path like a drone shot._ ![Descending from regional context into dense rows of retired aircraft at the boneyard](docs/media/08-boneyard.gif) -*Walk the boneyard: rows of retired airframes, fully resolved in 3D.* +_Walk the boneyard: rows of retired airframes, fully resolved in 3D._ --- @@ -343,10 +349,10 @@ Once the basics click, run these: How the globe handles live data: -- **World-stable icons.** Aircraft and ships point along their *true real-world heading* at every camera angle — tracked or not, looking straight down or across the horizon — via per-frame screen-space course projection. No spinning, no viewport-locking. +- **World-stable icons.** Aircraft and ships point along their _true real-world heading_ at every camera angle — tracked or not, looking straight down or across the horizon — via per-frame screen-space course projection. No spinning, no viewport-locking. - **Smooth motion from choppy data.** Live feeds arrive every 15–30s; the globe renders one interval behind real time and interpolates between known fixes. Dead reckoning fills the gaps. - **Honest satellites.** SGP4 propagation with orbit rings that stay locked to their satellites via GMST realignment — no drift, no per-second flicker. -- **Sits on the real ground.** Entity heights run through a real vertical datum — geoid-aware, sampled against the *rendered* terrain mesh — so aircraft park on aprons and cameras stand on street corners instead of floating. +- **Sits on the real ground.** Entity heights are aligned to work with Google 3D tiles, so aircraft park on aprons and cameras stand on street corners instead of floating. - **Caching and request budgets.** An OpenSky credit governor, a TomTom daily tile budget, and disk-cached TLEs reduce repeated requests. These controls do not replace provider quotas or billing controls. - **Server-side credentials.** Every API that touches a private key (OpenAI, AISStream, OpenSky OAuth, camera frames) is brokered through a hardened server-side proxy with SSRF protection, response caps, and sanitized errors. The only keys the browser sees are Google Maps and Cesium ion (restrict both at the provider). - **No framework.** Vanilla JavaScript, **CesiumJS**, and **Vite** — plus **Google Photorealistic 3D Tiles** for the planet and the **OpenAI Realtime API** for voice. Fast to read, fast to hack on. @@ -382,25 +388,25 @@ and configuration details. Six keys. Four have a free tier, and the two 🔴 ones are metered: -| | Key | Why | Get it | -|---|-----|-----|--------| -| 🟡 | **Cesium ion** | 🗺️ Google Photorealistic 3D, world terrain, and additional ion-hosted imagery stacks. The free Community plan is for eligible individual, personal/non-commercial use and has quotas | [cesium.com/ion](https://cesium.com/ion) — use a public `assets:read` token and check current [pricing/eligibility](https://cesium.com/platform/cesium-ion/pricing/) | -| 🔴 | **Google Maps** | Direct Google Photorealistic 3D + Google place search ([Map Tiles API](https://developers.google.com/maps/documentation/tile)) | [Google Cloud Console](https://console.cloud.google.com/) — URL-restrict it | -| 🔴 | **OpenAI** | 🎙️ The voice experience + AI HUD summary. The mini model works; the standard model is noticeably smarter. Want Gemini or another provider behind the mic? PRs welcome | [platform.openai.com](https://platform.openai.com) — metered, see costs below | -| 🟡 | **AISStream** | 🚢 Live global ships | [aisstream.io](https://aisstream.io) — free signup | -| 🟡 | **NASA FIRMS** | 🔥 Live active fires | [firms.modaps.eosdis.nasa.gov](https://firms.modaps.eosdis.nasa.gov/api/map_key/) — free | -| 🟡 | **TomTom** | 🚦 Live flow speeds and congestion colors for the simulated traffic layer | [developer.tomtom.com](https://developer.tomtom.com) — free tier available | +| | Key | Why | Get it | +| --- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 🟡 | **Cesium ion** | 🗺️ Google Photorealistic 3D, world terrain, and additional ion-hosted imagery stacks. The free Community plan is for eligible individual, personal/non-commercial use and has quotas | [cesium.com/ion](https://cesium.com/ion) — use a public `assets:read` token and check current [pricing/eligibility](https://cesium.com/platform/cesium-ion/pricing/) | +| 🔴 | **Google Maps** | Direct Google Photorealistic 3D + Google place search ([Map Tiles API](https://developers.google.com/maps/documentation/tile)) | [Google Cloud Console](https://console.cloud.google.com/) — URL-restrict it | +| 🔴 | **OpenAI** | 🎙️ The voice experience + AI HUD summary. The mini model works; the standard model is noticeably smarter. Want Gemini or another provider behind the mic? PRs welcome | [platform.openai.com](https://platform.openai.com) — metered, see costs below | +| 🟡 | **AISStream** | 🚢 Live global ships | [aisstream.io](https://aisstream.io) — free signup | +| 🟡 | **NASA FIRMS** | 🔥 Live active fires | [firms.modaps.eosdis.nasa.gov](https://firms.modaps.eosdis.nasa.gov/api/map_key/) — free | +| 🟡 | **TomTom** | 🚦 Live flow speeds and congestion colors for the simulated traffic layer | [developer.tomtom.com](https://developer.tomtom.com) — free tier available | ![Diving from city-scale live congestion straight into an intersection's public camera](docs/media/05-traffic-to-cctv.gif) -*What the TomTom key buys you: rush-hour density painted on the city — then dive from the jam straight into the camera watching it.* +_What the TomTom key buys you: rush-hour density painted on the city — then dive from the jam straight into the camera watching it._ ### Cherry on top -| | Key | Why | Get it | -|---|-----|-----|--------| -| 🟡 | **OpenSky** | ✈️ More flight-polling credits (🟢 anonymous works without) | [opensky-network.org](https://opensky-network.org) | -| 🟡 | **Launch Library 2** | 🚀 Higher space-missions request allowance (🟢 works without) | [thespacedevs.com](https://thespacedevs.com) | +| | Key | Why | Get it | +| --- | -------------------- | ------------------------------------------------------------- | -------------------------------------------------- | +| 🟡 | **OpenSky** | ✈️ More flight-polling credits (🟢 anonymous works without) | [opensky-network.org](https://opensky-network.org) | +| 🟡 | **Launch Library 2** | 🚀 Higher space-missions request allowance (🟢 works without) | [thespacedevs.com](https://thespacedevs.com) | Add these if you need higher polling allowances. @@ -437,12 +443,12 @@ OpenSky can run fully anonymous (`OPENSKY_AUTH_MODE=anon`), or import OAuth cred Honest numbers, roughly, as of mid-2026 — always check the provider pricing pages: -| | Cost reality | -|---|---| -| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, bundled datasets. | -| **🟡 The free-key tier** | **$0 with a signup.** AISStream, FIRMS, TomTom, OpenSky, plus Cesium ion for eligible personal/non-commercial use. Provider quotas and eligibility still apply. | -| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for GEV place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. | -| **🔴 OpenAI voice** | **The one that costs real money — so the app meters it for you.** Realtime audio runs a few cents per active minute; an evening of heavy use is single-digit dollars. A live session-spend readout sits next to the mic, with an STD/MINI model toggle, a $2 warning, and a **$5 hard cap that ends the session**. The voice context window is kept deliberately short too. | +| | Cost reality | +| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, bundled datasets. | +| **🟡 The free-key tier** | **$0 with a signup.** AISStream, FIRMS, TomTom, OpenSky, plus Cesium ion for eligible personal/non-commercial use. Provider quotas and eligibility still apply. | +| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for GEV place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. | +| **🔴 OpenAI voice** | **The one that costs real money — so the app meters it for you.** Realtime audio runs a few cents per active minute; an evening of heavy use is single-digit dollars. A live session-spend readout sits next to the mic, with an STD/MINI model toggle, a $2 warning, and a **$5 hard cap that ends the session**. The voice context window is kept deliberately short too. | Google's direct 3D route is surprisingly generous: the first 1,000 Photorealistic 3D Tiles sessions each month are currently free, and one root request supports @@ -500,9 +506,9 @@ First — thank you. To everyone who watched the God-view demos and went off to So here it is. Step inside the spy-thriller cockpit — except the data is real — and let's turn this into our shared sandbox for making sense of the world, and have fun doing it. This repo is the baseline, it stays open, and the whole point is for you to break things and bolt on layers we haven't thought of yet. -One heads-up from the inside: build in this space for a week and you learn that **the present is the cheap part**. The moment you try to go back in time — tiling, serving, and scrubbing *what happened* and *what changed* at any real resolution — the data gets expensive and the compute gets brutal. That's the long game. +One heads-up from the inside: build in this space for a week and you learn that **the present is the cheap part**. The moment you try to go back in time — tiling, serving, and scrubbing _what happened_ and _what changed_ at any real resolution — the data gets expensive and the compute gets brutal. That's the long game. -**Update — a hosted version is coming.** We originally planned to keep this repository as the open-source client and build a separate professional product. Then the launch happened, and the loudest request wasn't another feature — it was *"just give me a link."* So we're building an official hosted God's Eye View at [Halfpixel](https://halfpixel.ai): no installation, just open it in your browser. The hosted version is the easiest way into this open-source project. More soon. +**Update — a hosted version is coming.** We originally planned to keep this repository as the open-source client and build a separate professional product. Then the launch happened, and the loudest request wasn't another feature — it was _"just give me a link."_ So we're building an official hosted God's Eye View at [Halfpixel](https://halfpixel.ai): no installation, just open it in your browser. The hosted version is the easiest way into this open-source project. More soon. --- diff --git a/SECURITY.md b/SECURITY.md index 9bf718b74..a21ea99a4 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -64,6 +64,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur - **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). Other proxies target fixed upstream hosts. - **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio. - **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body. +- **Transit fetches registered feeds only.** `/api/transit/vehicles/` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator. - **Response-size caps and timeouts** on proxied responses. - **Sanitized errors** — internal error details are not echoed back to clients. - **Coalesced OAuth refresh** and cached successful responses only (OpenSky). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 5e533f4e5..36a838492 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2407,6 +2407,7 @@ its criteria cannot be silently ignored. | Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh | | Bikeshare 🚲 | GBFS (Lyft + BCycle) | `src/data/bikeshare.js` | `/api/gbfs` | 60s | | Directions 🧭 | OSRM on FOSSGIS servers (OpenStreetMap) | `src/data/directions.js` | `/api/route` (`steps=1`) | on placement / mode change | +| Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) | | Datacenters ▣ | OSM extract (bundled) | `src/data/localLayers.js` | — | static | | Dams ▰ | OpenInfraMap/OSM extract (bundled) | `src/data/localLayers.js` | — | static | | Submarine Cables ◠ | TeleGeography public map (bundled) | `src/data/telegeographySubmarineCables.js` | — | static | @@ -2494,6 +2495,163 @@ What a card needs is a resolved ground anchor: a step whose ground cell has not answered yet opens nothing rather than a card hanging at sea level. The cells are re-read for up to 90 seconds, which outlasts a cold terrain round trip; reroute, CLEAR and disable cancel that. +Transit is off by default, and share links use token `j`. The layer polls +registered feeds near the camera every 15 seconds, below a 3,000 km altitude +gate. Seven keyless regions are available: Boston, Austin, Minneapolis–St Paul, +Helsinki, the Netherlands, Norway and South East Queensland. Mode silhouettes +identify buses, trams, subways, trains, ferries and unclassified vehicles. + +Retained transit history uses the same outlier quarantine as live observations and +preserves the active live bracket. Rejected reports do not publish route or mode +changes to sprites, cards or detection text. + +**Timestamp playback.** Vehicles play between reports at 1× or hold while +waiting for the next report. The display delay adapts within 25–120 seconds; +late updates do not make a vehicle accelerate. A short reported crawl still +plays between fixes. Stationary segments schedule a wake at the next motion +boundary. Motion wording and orientation come from the displayed segment; +underflow says “Waiting for update,” and stale reports override confident +motion wording. Reported speed and stop status remain explicitly labelled as +reports. Duplicate timestamps and implausible observations cannot create a +zero-duration journey or immediately relocate a visible vehicle. + +**Selected history.** Selecting one vehicle draws its available recent path, +clipped to the displayed time. The browser retains up to 15 minutes and 128 +fixes per vehicle, under a global 32 MiB history budget. Capacity pressure may +shorten that history. Only the selected vehicle has a trail: completed segments +are batched primitives, and a small moving head ends at the marker. Trail +geometry is bounded to 2,048 prepared vertices; history changes rebuild the +body, while ordinary playback updates the head and segment visibility. +Both normal and depth-fail instance colours follow the age ramp once per display +second or style change. Subdivision crossings update only changed visibility +attributes. Deselecting releases selection geometry; marker paths retain only +active and future corridors. + +MBTA history is also retained in the running proxy: up to 15 minutes and 128 +observations per vehicle, with a 16 MiB feed limit and 64 MiB process limit. +`GET /api/transit/trail/mbta/` reads only observations already +received by this process; it never contacts the operator. Selection reads it +once and cancels the request on deselection or disable. Reload can recover +available history while the same server stays running. Restart clears it, and +there is no background collection or promise of complete coverage. Other feeds +remain live-enabled with session-local trails; proxy history is opt-in per feed. +The catalog and source tests pin that distinction and the MBTA / MassDOT credit. +Epoch metadata grows as contexts arrive. Snapshot members are protected during +ingestion, eviction uses receipt-ordered queues, and capacity loss is disclosed +when history is recreated. + +**Surface and route limitations.** Heights are aligned to work with Google 3D +tiles. Unknown historical surfaces leave gaps in a trail. Sparse reports can +cut across corners; these paths are not surveyed route geometry. Subway markers +are projected to street level because the feed does not provide depth, and the +selected card explains that choice. Actual street-level views still require +visual inspection. + +**Appearance.** Normal fleet frames are 20 px, growing to 30 px on selection; +NVG, thermal and noir frames are 30/36 px. Source rasters are 48/96 px before padding. +External halos are 1/1.5 px in normal and 3/3 px in sensors, calculated from the +final unpadded display size. Six modes, two sizes and three profiles cap the +raster cache at 36 variants. Normal mode colours are bus `#5EF08A`, tram +`#FFC24A`, subway `#FF4538`, rail `#D9A6FF`, ferry `#5FD6FF`, and unknown +`#D8DDE5`. Selection keeps the mode colour. Sensor bodies use solid rounded envelopes with at least 60% opaque white coverage +of the padded display footprint, without interior panel lines. Sensor input is opaque white with a +fully opaque black halo; black-hot reverses the contrast through the +thermal effect. Creation, mode changes, selection, deselection, map presets and +Cockpit vision use the same sprite styling function. + +Sensor signatures remain in-scene. The thresholds in `qaMetrics.js` require +white-hot, Ironbow and noir centres ≥0.85 luma and halo minima ≤0.25; +black-hot uses centre ≤0.15 and halo maximum ≥0.75. Surveillance uses ≥90% +of its phosphor peak white (0.65992 from RGB 0.16/1/0.22), halo minimum ≤0.25 +with the centre minus sampled background reported without gating. Each preset needs at least +six verified, unobscured sprites; insufficient coverage is UNEXERCISED. +The sampler excludes overlapping footprints, including otherwise ineligible +sprites, and verifies pick ownership across all nine sampled core pixels. +Noir samples also exclude its deliberately vignetted outer field, using shader +settings and screen position rather than observed brightness. Selected-trail +diagnostics report every condition even when selection fails. + +Transit brackets use the normal-composite overlay surface: a 1.25 px mode-colour +stroke over 3.25 px dark backing. Other contact themes retain their existing +behavior. The selected trail uses a 2 px core over 4 px backing, mode-coloured +in normal and white input in sensors. Alpha falls linearly from 0.80 at the head +to 0.45 at two minutes, 0.20 at ten minutes and 0.08 at fifteen minutes. + +**Cards and DETECT.** The card presents route, mode/operator, displayed motion, +display delay and newest-report age, reported speed/stop, vehicle ID, and +available trail duration. Its anchor follows the rendered marker; text is +checked at 4 Hz and published when it changes. DETECT labels retain route, +mode and motion, with the selected label suppressed beside its fuller card. +Candidate membership changes on revisions, coalesced at 4 Hz; paints read +cached contacts and strings. Marker migration explicitly updates cached +position references. Freshness and delay use the shared sample’s anchored clock; +newest bearings are explicitly labelled as reports. Rejected fixes cannot alter +lag reservoirs or the target delay. Returning from a hidden tab explicitly +synchronizes visible playback before its next frame. + +Playback reserves numeric scratch before the first sample, including unknown +heights, and reuses caller-owned sample and segment storage. + +**Scene work.** Stationary and animated billboards use separate, adjacent +collections above CCTV and Bikeshare. Traffic, Bikeshare and Transit share camera +sensitivity claims; the original sensitivity returns after the last release. +Visibility checks combine the camera frustum and ellipsoid occlusion at no more +than 4 Hz; missing map bounds do not admit the globe's far side. Frame work +visits moving visible vehicles, up to 64 pending height changes, and the selected +head. Rotation visits visible membership at 5 Hz, with selected rotation per +frame. The transit render hold exists only for visible positional or orientation +animation. Polls, surface completions and trail fades request individual renders. +Disable clears vehicles, detection references, requests, timers and holds; +destruction also releases collections and registrations. + +**Acceptance harness.** Run against an already running development server: + +```sh +QA_BASE_URL=http://localhost:4305 QA_TAG=keyless node scripts/qa-transit.mjs +``` + +The full run samples completed frames at 1920×1080 CSS pixels, records DPR, and +measures moving fleets of 800 and 3,000 vehicles with 128 fixes each and DETECT +on. Street/metro targets are respectively: transit frame CPU p95 ≤1/4 ms, frame +interval p95 ≤20/25 ms, billboard uploads ≤256 KiB/1 MiB per frame, and retained +heap growth ≤24/48 MiB. The 3,000-vehicle CPU budget is 4 ms (recorded browser +measurement 3.4 ms); the 800-vehicle budget stays at 1 ms (measured 0.9 ms). +Transit-authored frame allocation is ≤8 KiB/frame at both fleets, attributed +by source stack to `src/layers/transit/**`, `src/data/contactPlayback.js`, +`src/data/contactTrailRenderer.js` and `src/data/transit*.js`. Whole-frame +allocation and its three largest allocators are reported without gating: +Cesium billboard buffer uploads and the shared DETECT painter are outside +this layer's allocation budget. **Follow-up: pool shared DETECT placement and +paint scratch** in `_buildLabelPlacements`, `_materializeCandidate`, +`_drawOverlay` and their cohort/arbiter helpers. Heap growth is measured +conservatively as GC-bracketed page growth; allocation sampling includes objects +collected during the interval. These are acceptance budgets, not measured claims +for every machine. + +Artifacts in `qa-shots/transit/` include per-budget metrics and allocation +profiles, the Boston Red/741/742/Green-E matrix on measured bright and dark +backgrounds across normal, NVG, white-hot, black-hot and Ironbow, selected and +restored screenshots, reload recovery, and disable checks. Matrix routes are +scripted fixtures; the separate live-feed/reload checks use actual observations. +The separate Boston live pass uses actual MBTA observations at 120–600 m and +20–45 degree oblique views, with live selection and sensor readback. +Live visual cities are otherwise chosen for local daytime operation. Motion checks use +injected playback independently of live fleet activity. Thermal checks verify +style activation before reading generated controls; DETECT checks set density +and exercise the real OFF/restore button. Cesium render errors fail selection +acceptance. +Required empty or unexercised checks fail acceptance. The passing final line is +`HARNESS: PASS (0 unexercised)`. A `QA_SECTIONS` subset is diagnostic only and +cannot claim full acceptance. + +**Lifecycle and proxy.** Contacts temporarily disables Transit and restores its +prior state on exit; Space Missions refuses enablement while isolating replay +data. Cockpit enters through Contacts and does not independently enable Transit. +The proxy fetches registered URLs only, validates redirect origins, bounds +responses and decoding, caches successful snapshots, and backs off after failed +refreshes. A 304 confirms contact without appending history. Missing vehicles +are removed after two missed polls; reports older than ten minutes and feeds +silent for five minutes are removed. Credits appear when the feed is used. `src/data/militaryAwareness.js` remains registered internally as the Contacts coordinator, but it is not a user-visible Data Layers entry. Its visible entry diff --git a/package.json b/package.json index 6f8a5b411..8e3e5556f 100644 --- a/package.json +++ b/package.json @@ -43,7 +43,8 @@ "test": "node scripts/run-unit-tests.mjs", "test:track": "node scripts/track-regression.mjs", "qa:map-source-tray": "node scripts/qa-map-source-tray.mjs", - "check:boundaries": "node scripts/check-import-directions.mjs && node scripts/check-package-boundaries.mjs" + "check:boundaries": "node scripts/check-import-directions.mjs && node scripts/check-package-boundaries.mjs", + "qa:transit": "node scripts/qa-transit.mjs" }, "dependencies": { "@mapbox/vector-tile": "^3.0.0", @@ -230,6 +231,12 @@ "./sources/http-body": "./src/sources/httpBody.js", "./sources/overpass": "./src/sources/overpass.js", "./search/nominatim": "./src/search/nominatim.js", - "./sources/nominatim": "./src/sources/nominatim.js" + "./sources/nominatim": "./src/sources/nominatim.js", + "./server/providers/transit": { + "node": "./server/providers/transit.js" + }, + "./sources/transit": "./src/data/transitProxy.js", + "./layers/transit": "./src/layers/transit/index.js", + "./layers/transit/standalone": "./src/data/transit.js" } } diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 30b035a0e..ad13e1b96 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -675,5 +675,52 @@ "src/search/nominatim.test.mjs", "src/sources/httpBody.js", "src/sources/overpass.js", - "src/sources/nominatim.js" + "src/sources/nominatim.js", + "server/providers/transit.js", + "server/providers/transitHistory.js", + "server/providers/transitHistory.test.mjs", + "src/data/transitFeeds.js", + "src/data/transitFeeds.test.mjs", + "src/data/transitProxy.js", + "src/data/transitProxy.test.mjs", + "src/data/gtfsRealtime.js", + "src/data/gtfsRealtime.test.mjs", + "src/data/transit.js", + "src/data/transit.test.mjs", + "src/layers/transit/controls.js", + "src/layers/transit/height.js", + "src/layers/transit/index.js", + "src/layers/transit/ingestion.js", + "src/layers/transit/lifecycle.js", + "src/layers/transit/policy.js", + "src/layers/transit/queries.js", + "src/layers/transit/rendering.js", + "src/layers/transit/selection.js", + "src/layers/transit/state.js", + "src/layers/transit/testing.js", + "src/layers/transit/viewport.js", + "src/layers/transit/lifecycle.test.mjs", + "scripts/qa-transit.mjs", + "src/tooling/transitProvider.test.mjs", + "src/app/layers/transit.js", + "src/data/transitIcons.js", + "src/data/cameraSensitivity.js", + "src/data/contactPlayback.js", + "src/data/contactTrailRenderer.js", + "src/data/contactTrailRenderer.test.mjs", + "src/layers/transit/trails.js", + "src/data/contactPlayback.test.mjs", + "src/layers/transit/movement.js", + "src/layers/transit/movement.test.mjs", + "src/data/transitPresetStyle.js", + "src/data/transitPresetStyle.test.mjs", + "src/data/transitIcons.test.mjs", + "src/overlays/worldOverlayTokens.test.mjs", + "src/app/layers/transit.test.mjs", + "src/layers/transit/qaMetrics.js", + "src/layers/transit/qaMetrics.test.mjs", + "scripts/qa-transit-scenes.mjs", + "scripts/qa-transit-controls.mjs", + "src/tooling/transitQa.test.mjs", + "scripts/qa-transit-browser.mjs" ] diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index cdd9fe6bb..9eb44433c 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -878,6 +878,7 @@ "traffic-layer": { "exports": ["./layers/traffic"], "modules": [ + "src/data/cameraSensitivity.js", "src/data/flowMatch.js", "src/data/tomtomTiles.js", "src/data/trafficBounds.js", @@ -920,7 +921,8 @@ "src/layers/bikeshare/state.js", "src/layers/bikeshare/testing.js", "src/layers/bikeshare/viewport.js", - "src/data/inputOwnership.js" + "src/data/inputOwnership.js", + "src/data/cameraSensitivity.js" ], "external": ["cesium"] }, @@ -1638,7 +1640,32 @@ "src/search/nominatim.js", "src/nominatimGeocode.js", "src/sources/httpBody.js", - "src/sources/nominatim.js" + "src/sources/nominatim.js", + "src/app/layers/transit.js", + "src/data/transit.js", + "src/data/transitFeeds.js", + "src/data/transitIcons.js", + "src/data/transitPresetStyle.js", + "src/layers/transit/controls.js", + "src/layers/transit/height.js", + "src/layers/transit/index.js", + "src/layers/transit/ingestion.js", + "src/layers/transit/lifecycle.js", + "src/layers/transit/policy.js", + "src/layers/transit/queries.js", + "src/layers/transit/rendering.js", + "src/layers/transit/selection.js", + "src/layers/transit/state.js", + "src/layers/transit/testing.js", + "src/layers/transit/viewport.js", + "src/data/cameraSensitivity.js", + "src/data/contactPlayback.js", + "src/data/contactTrailRenderer.js", + "src/data/transitProxy.js", + "src/data/gtfsRealtime.js", + "src/layers/transit/trails.js", + "src/layers/transit/movement.js", + "src/layers/transit/qaMetrics.js" ], "external": [ "@mapbox/vector-tile", @@ -1995,7 +2022,31 @@ "src/nominatimGeocode.js", "src/data/regionalModel.js", "src/sources/httpBody.js", - "src/sources/nominatim.js" + "src/sources/nominatim.js", + "src/app/layers/transit.js", + "src/data/transitFeeds.js", + "src/data/transitIcons.js", + "src/data/transitPresetStyle.js", + "src/layers/transit/controls.js", + "src/layers/transit/height.js", + "src/layers/transit/index.js", + "src/layers/transit/ingestion.js", + "src/layers/transit/lifecycle.js", + "src/layers/transit/policy.js", + "src/layers/transit/queries.js", + "src/layers/transit/rendering.js", + "src/layers/transit/selection.js", + "src/layers/transit/state.js", + "src/layers/transit/testing.js", + "src/layers/transit/viewport.js", + "src/data/cameraSensitivity.js", + "src/data/contactPlayback.js", + "src/data/contactTrailRenderer.js", + "src/data/transitProxy.js", + "src/data/gtfsRealtime.js", + "src/layers/transit/trails.js", + "src/layers/transit/movement.js", + "src/layers/transit/qaMetrics.js" ], "external": [ "cesium", @@ -2316,5 +2367,118 @@ "src/sources/httpBody.js" ], "external": [] + }, + "transit-provider": { + "runtime": "node", + "exports": ["./server/providers/transit"], + "modules": [ + "server/providers/common/http.js", + "server/providers/common/rate-limit.js", + "server/providers/transit.js", + "server/providers/transitHistory.js", + "src/data/gtfsRealtime.js", + "src/data/transitFeeds.js", + "src/data/transitProxy.js", + "src/sources/httpBody.js" + ], + "external": ["pbf"] + }, + "transit-source": { + "exports": ["./sources/transit"], + "modules": [ + "src/data/gtfsRealtime.js", + "src/data/transitFeeds.js", + "src/data/transitProxy.js" + ], + "external": ["pbf"] + }, + "transit-standalone": { + "exports": ["./layers/transit/standalone"], + "modules": [ + "src/app/sourceSlot.js", + "src/celestialRing.js", + "src/data/cameraSensitivity.js", + "src/data/contactPlayback.js", + "src/data/contactTrailRenderer.js", + "src/data/dataCredits.js", + "src/data/detectionCohort.js", + "src/data/detectionPolicy.js", + "src/data/geoid.js", + "src/data/groundFloor.js", + "src/data/gtfsRealtime.js", + "src/data/iconOrientation.js", + "src/data/inputOwnership.js", + "src/data/labelArbiter.js", + "src/data/meshFloorSampler.js", + "src/data/pickRegistry.js", + "src/data/spriteOrder.js", + "src/data/terrainHeights.js", + "src/data/transit.js", + "src/data/transitFeeds.js", + "src/data/transitIcons.js", + "src/data/transitPresetStyle.js", + "src/data/transitProxy.js", + "src/layers/transit/controls.js", + "src/layers/transit/height.js", + "src/layers/transit/index.js", + "src/layers/transit/ingestion.js", + "src/layers/transit/lifecycle.js", + "src/layers/transit/movement.js", + "src/layers/transit/policy.js", + "src/layers/transit/qaMetrics.js", + "src/layers/transit/queries.js", + "src/layers/transit/rendering.js", + "src/layers/transit/selection.js", + "src/layers/transit/state.js", + "src/layers/transit/testing.js", + "src/layers/transit/trails.js", + "src/layers/transit/viewport.js", + "src/overlays/worldOverlay.js", + "src/overlays/worldOverlayDraw.js", + "src/overlays/worldOverlayTokens.js", + "src/renderGovernor.js", + "src/services/application.js", + "src/services/groundFloor.js", + "src/services/meshFloorSampler.js", + "src/services/requests.js", + "src/services/terrainHeights.js", + "src/sources/sourceSlot.js" + ], + "external": ["cesium", "egm96-universal", "pbf"] + }, + "transit-layer": { + "exports": ["./layers/transit"], + "modules": [ + "src/data/iconOrientation.js", + "src/data/inputOwnership.js", + "src/data/transitFeeds.js", + "src/data/transitIcons.js", + "src/data/transitPresetStyle.js", + "src/layers/transit/controls.js", + "src/layers/transit/height.js", + "src/layers/transit/index.js", + "src/layers/transit/ingestion.js", + "src/layers/transit/lifecycle.js", + "src/layers/transit/policy.js", + "src/layers/transit/queries.js", + "src/layers/transit/rendering.js", + "src/layers/transit/selection.js", + "src/layers/transit/state.js", + "src/layers/transit/testing.js", + "src/layers/transit/viewport.js", + "src/data/cameraSensitivity.js", + "src/data/contactPlayback.js", + "src/data/contactTrailRenderer.js", + "src/data/transitProxy.js", + "src/data/gtfsRealtime.js", + "src/layers/transit/trails.js", + "src/layers/transit/movement.js", + "src/layers/transit/qaMetrics.js", + "src/layers/transit/qaMetrics.test.mjs", + "scripts/qa-transit-scenes.mjs", + "scripts/qa-transit-controls.mjs", + "scripts/qa-transit-browser.mjs" + ], + "external": ["cesium", "pbf"] } } diff --git a/scripts/qa-transit-browser.mjs b/scripts/qa-transit-browser.mjs new file mode 100644 index 000000000..1e71e484e --- /dev/null +++ b/scripts/qa-transit-browser.mjs @@ -0,0 +1,595 @@ +/** Bound page promises as well as CDP itself; labels identify a stalled section. */ +export function boundPageEvaluations(page, defaultTimeoutMs = 30000) { + const evaluate = page.evaluate.bind(page); + page.evaluate = (fn, ...args) => { + // The two long trace collectors pass their duration explicitly. + const duration = + args.find((v) => typeof v === 'number' && v >= 6000 && v <= 120000) || 0; + const timeoutMs = Math.max( + defaultTimeoutMs, + duration ? duration + 5000 : 0, + ); + const label = String(fn).slice(0, 120).replace(/\s+/g, ' '); + return evaluate( + async (source, values, timeoutMs, label) => { + let timer; + try { + return await Promise.race([ + // Puppeteer callbacks are standalone functions, just as in evaluate. + Promise.resolve().then(() => (0, eval)(`(${source})`)(...values)), + new Promise((_, reject) => { + timer = setTimeout( + () => + reject( + new Error( + `Transit QA page wait timed out after ${timeoutMs} ms: ${label}`, + ), + ), + timeoutMs, + ); + }), + ]); + } finally { + clearTimeout(timer); + } + }, + String(fn), + args, + timeoutMs, + label, + ); + }; +} + +export async function reportTransitVisibility(page, label) { + const result = await page.evaluate(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + return { + total: state._vehicles.size, + visible: state._visible.size, + moving: state._moving.size, + held: state._renderHeld, + requestRenderMode: app.viewer.scene.requestRenderMode, + renderLoop: app.viewer.useDefaultRenderLoop, + renderErrors: window.__transitRenderErrors || [], + bounds: state._viewBounds, + hidden: layer._transitVisibilityForTest().filter((e) => !e.shown), + }; + }); + console.log(` ${label}: ${JSON.stringify(result)}`); + if (result.renderLoop === false || result.renderErrors.length) + throw new Error( + `${label}: Cesium render loop stopped or raised an error: ${JSON.stringify(result.renderErrors)}`, + ); + return result; +} + +/** Intersect primed histories with the new snapshot; disappearance is availability. */ +export async function restoreRetainedTransit(retained) { + if (!retained.length) + return { + error: 'No multi-fix history observed during the probe', + unexercised: true, + }; + const app = window.__godsEyeView; + await app.dataManager.setEnabled('transit', true, { source: 'qa' }); + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + // enable() schedules proximity discovery; an empty pre-discovery map is not + // evidence that every primed vehicle departed. Wait for this feed's poll. + const deadline = Date.now() + 12000; + while (state._activeFeeds && !state._vehicles.size && Date.now() < deadline) + await new Promise((r) => setTimeout(r, 100)); + await layer.update(); + if ( + retained[0]?.primedAt && + Math.floor(Date.now() / 60000) !== Math.floor(retained[0].primedAt / 60000) + ) + return { + error: 'Reload crossed the priming minute; same-minute proof unavailable', + unexercised: true, + }; + const history = retained.find((h) => + state._vehicles.has(`${h.feedId || 'mbta'}:${h.vehicleId}`), + ); + if (!history) + return { + error: `All ${retained.length} primed vehicles left the current snapshot`, + unexercised: true, + }; + const entry = state._vehicles.get( + `${history.feedId || 'mbta'}:${history.vehicleId}`, + ); + const camera = app.viewer.camera, + C = camera.positionCartographic.constructor; + camera.setView({ + destination: C.toCartesian( + C.fromDegrees(entry.record.lon, entry.record.lat, 600), + ), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + layer._transitPartsForTest().selection.selectVehicle(entry.key); + return { key: entry.key, oldest: history.oldestT }; +} + +export async function sampleTransitPixels(page, palette) { + return page.evaluate(async (palette) => { + const app = window.__godsEyeView; + const state = app.dataManager.layers + .get('transit') + .module._transitStateForTest(); + // Decode the actual billboard rasters before the paint being judged. + const overlay = window.location + ? await import('/src/overlays/worldOverlay.js') + : null; + const rasters = new Map(); + await Promise.all( + [ + ...new Set( + [...state._vehicles.values()] + .map((e) => e.marker?.image) + .filter(Boolean), + ), + ].map(async (url) => { + const img = new Image(); + img.src = url; + await img.decode(); + const c = document.createElement('canvas'); + c.width = img.width; + c.height = img.height; + const ctx = c.getContext('2d'); + ctx.drawImage(img, 0, 0); + rasters.set(url, { + width: c.width, + height: c.height, + data: ctx.getImageData(0, 0, c.width, c.height).data, + }); + }), + ); + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + remove(); + reject( + new Error('Transit pixel readback: no postRender within 5000 ms'), + ); + }, 5000); + const remove = + window.__godsEyeView.viewer.scene.postRender.addEventListener(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const scene = app.viewer.scene; + const canvas = scene.canvas; + const view = scene.camera.viewMatrix; + const proj = scene.camera.frustum.projectionMatrix; + const mul = (m, v) => [ + m[0] * v[0] + m[4] * v[1] + m[8] * v[2] + m[12] * v[3], + m[1] * v[0] + m[5] * v[1] + m[9] * v[2] + m[13] * v[3], + m[2] * v[0] + m[6] * v[1] + m[10] * v[2] + m[14] * v[3], + m[3] * v[0] + m[7] * v[1] + m[11] * v[2] + m[15] * v[3], + ]; + const width = canvas.clientWidth; + const height = canvas.clientHeight; + const sprites = []; + const footprints = []; + const intersects = (a, b) => + a.x < b.x + b.w && + a.x + a.w > b.x && + a.y < b.y + b.h && + a.y + a.h > b.y; + const plates = + app.styleManager + ?.getDetectionDiagnostics?.() + ?.calloutRects?.filter((r) => r.alpha > 0.01) || []; + if (state._selectedKey) { + const card = overlay?.getOverlayPaintRect( + 'transit-selected', + state._selectedKey, + ); + if (card) plates.push(card); + } + const panels = (overlay?.WORLD_OVERLAY_OCCLUDER_SELECTORS || []) + .flatMap((selector) => [...document.querySelectorAll(selector)]) + .filter( + (el) => + !el.hidden && + el.checkVisibility({ + checkOpacity: true, + checkVisibilityCSS: true, + }), + ) + .map((el) => { + const r = el.getBoundingClientRect(); + return { x: r.left, y: r.top, w: r.width, h: r.height }; + }); + const skipped = []; + const style = state._stylePreset; + const uniforms = app.styleManager?.stages?.[style]?.uniforms || {}; + const intensity = uniforms.intensity ?? 1; + const pixelation = + uniforms.pixelation ?? (style === 'surveillance' ? 2.5 : 1.5); + const sensor = style === 'surveillance' || style === 'thermal'; + // Invert NVG's barrel transform; thermal only quantizes the source. + const screenPoint = (sx, sy) => { + let x = sx, + y = sy; + if (style === 'surveillance') { + const tx = (sx / width) * 2 - 1, + ty = (sy / height) * 2 - 1; + let cx = tx, + cy = ty; + for (let i = 0; i < 16; i++) { + const r2 = cx * cx + cy * cy; + const d = + 1 + r2 * intensity * 0.25 + r2 * r2 * intensity * 0.075; + cx = tx / d; + cy = ty / d; + } + x = ((cx + 1) * width) / 2; + y = ((cy + 1) * height) / 2; + } + const offset = sensor + ? ((1 + (pixelation - 1) * intensity) * intensity) / + 2 / + (canvas.width / width) + : 0; + return { x: x + offset, y: y - offset }; + }; + const uiBlocked = (x, y) => { + const el = document.elementFromPoint?.(x, y); + return ( + el && + el !== canvas && + !el.closest?.( + '#world-overlay-root, #world-overlay-detection-surface', + ) && + el.tagName !== 'BODY' && + el.tagName !== 'HTML' + ); + }; + for (const entry of state._vehicles.values()) { + if (!entry.marker || entry.marker.show === false) continue; + const p = entry.marker.position; + const clip = mul(proj, mul(view, [p.x, p.y, p.z, 1])); + if (!(clip[3] > 0)) continue; + const screen = entry.marker.computeScreenSpacePosition?.(scene) || { + x: ((clip[0] / clip[3]) * 0.5 + 0.5) * width, + y: (1 - ((clip[1] / clip[3]) * 0.5 + 0.5)) * height, + }; + const rect = entry.marker.constructor.getScreenSpaceBoundingBox?.( + entry.marker, + screen, + ) || { x: screen.x - 16, y: screen.y - 16, width: 32, height: 32 }; + const { x, y } = screenPoint(screen.x, screen.y); + const c = Math.abs(Math.cos(entry.marker.rotation || 0)), + sn = Math.abs(Math.sin(entry.marker.rotation || 0)); + const w = rect.width * c + rect.height * sn, + h = rect.height * c + rect.width * sn; + const footprint = { x: x - w / 2, y: y - h / 2, w, h }; + footprints.push({ key: entry.key, ...footprint }); + let reason = null; + if ( + rect.x < 0 || + rect.y < 0 || + rect.x + rect.width > width || + rect.y + rect.height > height + ) + reason = 'billboard rectangle outside frame'; + if ( + entry.marker.color?.alpha < 0.95 || + entry.markerCollection?.show === false + ) + reason = 'billboard not opaque/collection hidden'; + const raster = rasters.get(entry.marker.image); + if (!raster) reason = 'billboard alpha unavailable'; + else if ( + raster.data[ + (Math.floor(raster.height / 2) * raster.width + + Math.floor(raster.width / 2)) * + 4 + + 3 + ] < 242 + ) + reason = 'billboard centre is transparent'; + if ( + sensor && + Math.hypot( + (((x / width) * 2 - 1) * width) / height, + (y / height) * 2 - 1, + ) > 0.6 + ) + reason = 'outside unmasked sensor lens'; + if (style === 'noir') { + // Noir deliberately darkens the frame edges. Judge the core in + // its unmasked field, independently of the measured brightness. + const u = x / width, + v = y / height; + const transmission = Math.pow( + 16 * u * (1 - u) * v * (1 - v), + 0.3 + 0.4 * (uniforms.vignetteAmt ?? 0.5) * intensity, + ); + if (1 - intensity + transmission * intensity < 0.9) + reason = 'outside unmasked noir field'; + } + + if ( + [ + [x, y], + [rect.x, rect.y], + [rect.x + rect.width, rect.y + rect.height], + [rect.x + rect.width, rect.y], + [rect.x, rect.y + rect.height], + ].some(([px, py]) => uiBlocked(px, py)) + ) + reason = 'UI panel overlaps billboard'; + if (panels.some((r) => intersects(footprint, r))) + reason = 'UI panel rectangle overlaps billboard'; + if (plates.some((r) => intersects(footprint, r))) + reason = 'DETECT label/card rectangle overlaps billboard'; + if (reason) { + skipped.push({ key: entry.key, reason }); + continue; + } + if (x < 40 || y < 40 || x > width - 40 || y > height - 40) continue; + sprites.push({ + key: entry.key, + mode: entry.mode, + x, + y, + selected: state._selectedKey === entry.key, + screenX: screen.x, + screenY: screen.y, + rect, + footprint, + bracketX: entry.detectContact?._candidateScreenX ?? screen.x, + bracketY: entry.detectContact?._candidateScreenY ?? screen.y, + raster, + rotation: entry.marker.rotation || 0, + halfW: Number.isFinite(entry.marker.width) + ? Math.ceil(entry.marker.width / 2) + 2 + : 11, + halfH: Number.isFinite(entry.marker.height) + ? Math.ceil(entry.marker.height / 2) + 2 + : 7, + }); + } + const isolated = sprites.filter( + (a) => + !a.selected && + footprints.every( + (b) => b.key === a.key || !intersects(a.footprint, b), + ), + ); + const gl = canvas.getContext('webgl2') || canvas.getContext('webgl'); + const scale = canvas.width / canvas.clientWidth; + const luma = (r, g, b) => (0.299 * r + 0.587 * g + 0.114 * b) / 255; + const readLuma = (cx, cy) => { + const px = Math.floor(cx * scale); + const py = Math.floor(canvas.height - 1 - cy * scale); + const buf = new Uint8Array(4); + gl.readPixels(px, py, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, buf); + return luma(buf[0], buf[1], buf[2]); + }; + const surface = document.getElementById('world-overlay-canvas'); + const ctx = surface?.getContext('2d'); + // A hidden surface has no client size; nothing to read there. + const sscale = + surface && surface.clientWidth > 0 + ? surface.width / surface.clientWidth + : 0; + const out = []; + for (const s of isolated) { + if (out.length >= 12) break; + // The normal canvas includes DETECT plates and selected cards. + // A plate over the body invalidates this contact, even if WebGL below is hot. + if (ctx && sscale > 0) { + const body = ctx.getImageData( + Math.floor((s.x - 3) * sscale), + Math.floor((s.y - 3) * sscale), + Math.max(1, Math.ceil(6 * sscale)), + Math.max(1, Math.ceil(6 * sscale)), + ).data; + if (body.some((v, i) => i % 4 === 3 && v > 0)) { + skipped.push({ + key: s.key, + reason: 'DETECT label/card overlaps core', + }); + continue; + } + } + // Picking is independent of rendered brightness: an overlapping + // label or sprite must not masquerade as a dim sensor signature. + // Check the whole sampled core, not just its centre pick. + let coreOwned = true; + if (scene.pick) { + const framebuffer = gl.getParameter?.(gl.FRAMEBUFFER_BINDING); + for (let dx = -1; dx <= 1; dx++) + for (let dy = -1; dy <= 1; dy++) { + let u = (s.x + dx) / width, + v = 1 - (s.y + dy) / height; + if (style === 'surveillance') { + const x = u * 2 - 1, + y = v * 2 - 1, + r2 = x * x + y * y; + const d = + 1 + r2 * intensity * 0.25 + r2 * r2 * intensity * 0.075; + u = (x * d + 1) / 2; + v = (y * d + 1) / 2; + } + if (sensor) { + const grid = 1 + (pixelation - 1) * intensity; + u += + ((Math.floor((u * canvas.width) / grid) * grid) / + canvas.width - + u) * + intensity; + v += + ((Math.floor((v * canvas.height) / grid) * grid) / + canvas.height - + v) * + intensity; + } + const picked = scene.pick( + { x: u * width, y: (1 - v) * height }, + 1, + 1, + ); + if (picked?.id !== s.key && picked?.primitive?.id !== s.key) + coreOwned = false; + } + if (gl.bindFramebuffer) + gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer); + } + if (!coreOwned) { + skipped.push({ + key: s.key, + reason: + 'sampled core is partially obscured or outside the sprite', + }); + continue; + } + let centre = 0; + for (let dx = -1; dx <= 1; dx += 1) + for (let dy = -1; dy <= 1; dy += 1) + centre += readLuma(s.x + dx, s.y + dy) / 9; + let ringMin = 1; + let ringMax = 0; + // Sample the actual rotated raster halo, not a fixed-radius circle + // that can hit the hot body or terrain after a display-size change. + const raster = s.raster; + const cos = Math.cos(s.rotation), + sin = Math.sin(s.rotation); + for (let ry = 1; ry < raster.height - 1; ry++) + for (let rx = 1; rx < raster.width - 1; rx++) { + const i = (ry * raster.width + rx) * 4; + if ( + raster.data[i + 3] < 200 || + raster.data[i] > 30 || + raster.data[i + 1] > 30 || + raster.data[i + 2] > 30 + ) + continue; + const dx = ((rx + 0.5) / raster.width - 0.5) * s.rect.width; + const dy = ((ry + 0.5) / raster.height - 0.5) * s.rect.height; + const q = screenPoint( + s.screenX + dx * cos + dy * sin, + s.screenY - dx * sin + dy * cos, + ); + const v = readLuma(q.x, q.y); + ringMin = Math.min(ringMin, v); + ringMax = Math.max(ringMax, v); + } + let bracket = null; + if (ctx && sscale > 0) { + // Transit brackets follow the padded sprite extent and snapped + // stroke centres. Sample the middle of each straight segment, + // never a colour histogram containing antialiased edges/callouts. + const cx = Math.floor(s.bracketX) + 0.5, + cy = Math.floor(s.bracketY) + 0.5; + const hw = s.halfW, + hh = s.halfH, + mid = Math.max(4, Math.floor(Math.min(hw, hh) * 0.55)) / 2; + const points = [ + [cx - hw + mid, cy - hh], + [cx + hw - mid, cy - hh], + [cx - hw + mid, cy + hh], + [cx + hw - mid, cy + hh], + [cx - hw, cy - hh + mid], + [cx + hw, cy - hh + mid], + [cx - hw, cy + hh - mid], + [cx + hw, cy + hh - mid], + ]; + let bestAlpha = 0; + for (const [index, [x, y]] of points.entries()) { + if ( + uiBlocked(x, y) || + plates.some((r) => + intersects({ x: x - 1, y: y - 1, w: 2, h: 2 }, r), + ) + ) + continue; + // A label plate fills both sides of a stroke. Require empty + // pixels beyond its dark backing on the perpendicular axis. + const normal = index < 4 ? [0, 3] : [3, 0]; + if ( + [-1, 1].some( + (sign) => + ctx.getImageData( + Math.floor((x + sign * normal[0]) * sscale), + Math.floor((y + sign * normal[1]) * sscale), + 1, + 1, + ).data[3] > 0, + ) + ) + continue; + const data = ctx.getImageData( + Math.floor(x * sscale), + Math.floor(y * sscale), + 1, + 1, + ).data; + if (data[3] > bestAlpha) { + bestAlpha = data[3]; + bracket = [data[0], data[1], data[2]]; + } + } + } + out.push({ + key: s.key, + mode: s.mode, + verified: true, + rectangle: s.rect, + centre, + background: + [ + readLuma(s.x + 25, s.y), + readLuma(s.x - 25, s.y), + readLuma(s.x, s.y + 25), + readLuma(s.x, s.y - 25), + ].reduce((a, b) => a + b, 0) / 4, + ringMin, + ringMax, + bracket, + bracketTier: `transit_${s.mode}`, + bracketExpected: palette[s.mode] + ? [1, 3, 5].map((i) => + parseInt(palette[s.mode].slice(i, i + 2), 16), + ) + : null, + x: s.x, + y: s.y, + }); + } + clearTimeout(timer); + remove(); + console.log( + `TRANSIT_SAMPLER ${style}: ${out.length} verified; skipped=${JSON.stringify(skipped)}`, + ); + resolve(out); + }); + window.__godsEyeView.viewer.scene.requestRender(); + }); + }, palette); +} + +/** A fleet-to-fleet comparison must not inherit the city's selected marker. */ +export function readTransitFleetStyle({ key, resetSelection = false } = {}) { + const layer = window.__godsEyeView.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + if (resetSelection) layer._transitPartsForTest().selection.clearSelection(); + const entry = key + ? state._vehicles.get(key) + : [...state._vehicles.values()].find((e) => e.marker?.show); + return entry + ? { + key: entry.key, + selected: state._selectedKey === entry.key, + color: entry.marker.color.toCssColorString(), + width: entry.marker.width, + image: entry.marker.image, + } + : null; +} diff --git a/scripts/qa-transit-controls.mjs b/scripts/qa-transit-controls.mjs new file mode 100644 index 000000000..b76e14706 --- /dev/null +++ b/scripts/qa-transit-controls.mjs @@ -0,0 +1,204 @@ +const pause = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); +export const BOSTON = { + id: 'boston', + label: 'Boston (MBTA)', + lat: 42.3554, + lon: -71.0605, + heading: 25, + zone: 'America/New_York', +}; +const LIVE_CITIES = [ + BOSTON, + { + id: 'austin', + label: 'Austin (CapMetro)', + lat: 30.2672, + lon: -97.7431, + heading: 340, + zone: 'America/Chicago', + }, + { + id: 'minneapolis', + label: 'Minneapolis (Metro Transit)', + lat: 44.9778, + lon: -93.265, + heading: 0, + zone: 'America/Chicago', + }, + { + id: 'helsinki', + label: 'Helsinki (HSL)', + lat: 60.1699, + lon: 24.9384, + heading: 0, + zone: 'Europe/Helsinki', + }, + { + id: 'oslo', + label: 'Oslo (Entur)', + lat: 59.9139, + lon: 10.7522, + heading: 0, + zone: 'Europe/Oslo', + }, +]; +export function chooseLiveCities(now = new Date()) { + return LIVE_CITIES.filter((city) => { + const hour = Number( + new Intl.DateTimeFormat('en-US', { + timeZone: city.zone, + hour: 'numeric', + hourCycle: 'h23', + }).format(now), + ); + return hour >= 6 && hour < 22; + }).slice(0, 2); +} + +/** + * Switch the post-FX style the way a user does — the preset button — and + * fall back to the style manager only when the tray is not clickable, saying + * so. Then set any style parameters through their sliders. + * @param {import('puppeteer').Page} page + * @param {string} name + * @param {Record} [params] aria-label → slider value + * @returns {Promise} how the style was set + */ +export async function selectStyle(page, name, params = {}, wait = pause) { + let how = 'button'; + // The presets live in the bottom control panel, which auto-dismisses when + // the pointer leaves it: pin it open the way a user does, once. + await page.evaluate(() => { + const panel = document.getElementById('control-panel'); + if (panel && !panel.classList.contains('dock-pinned')) { + document + .querySelector('.dock-pin-btn[data-pin-target="control-panel"]') + ?.click(); + } + window.__godsEyeView.styleManager.setPanelCollapsed?.( + 'control-panel', + false, + { explicit: true }, + ); + }); + await wait(400); + try { + await page.$eval(`.style-btn[data-style="${name}"]`, (el) => + el.scrollIntoView({ block: 'center' }), + ); + await page.click(`.style-btn[data-style="${name}"]`); + } catch { + how = 'styleManager.setStyle (button not clickable)'; + await page.evaluate( + (n) => window.__godsEyeView.styleManager.setStyle(n), + name, + ); + } + // A click can be swallowed by a loading cover or a collapsed tray without + // Puppeteer throwing. Verify the active style before querying generated rows. + if ( + !(await page.evaluate( + (n) => document.documentElement.dataset.gevStyle === n, + name, + )) + ) { + how = 'styleManager.setStyle (button did not activate style)'; + await page.evaluate( + (n) => window.__godsEyeView.styleManager.setStyle(n), + name, + ); + } + await page.waitForFunction( + (n) => document.documentElement.dataset.gevStyle === n, + { timeout: 10000 }, + name, + ); + await wait(1_800); + for (const [label, value] of Object.entries(params)) { + const selector = `#param-sliders input.param-slider[aria-label="${label}"]`; + await page.waitForSelector(selector, { timeout: 10000 }); + await page.$eval( + selector, + (el, v) => { + el.value = String(v); + el.dispatchEvent(new Event('input', { bubbles: true })); + }, + value, + ); + await wait(700); + } + return how; +} + +/** + * Turn DETECT on through its own button, cycling until the dense profile is + * reported on the detection surface. + * @param {import('puppeteer').Page} page + * @returns {Promise} the profile reached + */ +export async function detectOn(page, wait = pause) { + // The real button toggles OFF/restore. Density determines its on-profile. + const mode = () => + page.evaluate(() => { + const button = document.getElementById('detection-toggle'); + const label = button?.getAttribute('aria-label') || ''; + return { + on: button?.getAttribute('aria-pressed') === 'true', + mode: (label.split(':')[1] || 'off').trim().toUpperCase(), + }; + }); + // The loading screen swallows the first click of a fresh page. + await page + .waitForFunction(() => !document.getElementById('loading-screen'), { + timeout: 30_000, + }) + .catch(() => {}); + // The DISPLAY rail starts collapsed on a first run: open it the way a user + // does, then press DETECT until it reports on. + await page.evaluate(() => { + const rail = document.getElementById('pp-toggles'); + if (rail?.classList.contains('collapsed')) + document + .querySelector('.panel-collapse-btn[data-collapse-target="pp-toggles"]') + ?.click(); + }); + await page.waitForFunction( + () => + document.getElementById('detection-toggle')?.getBoundingClientRect() + .width > 0, + { timeout: 10_000 }, + ); + if (!(await mode()).on) { + await page.click('#detection-toggle'); + await wait(300); + } + await page.$eval('#detection-density-slider', (el) => { + el.value = '100'; + el.dispatchEvent(new Event('input', { bubbles: true })); + }); + await page.waitForFunction( + () => + document + .getElementById('detection-toggle') + ?.getAttribute('aria-label') === 'Detection overlay: dense', + { timeout: 10000 }, + ); + // Prove the button's actual OFF/restore cycle preserves DENSE. + await page.click('#detection-toggle'); + await page.waitForFunction( + () => + document + .getElementById('detection-toggle') + ?.getAttribute('aria-pressed') === 'false', + { timeout: 10000 }, + ); + await page.click('#detection-toggle'); + await page.waitForFunction( + () => + document + .getElementById('detection-toggle') + ?.getAttribute('aria-label') === 'Detection overlay: dense', + { timeout: 10000 }, + ); + return (await mode()).mode; +} diff --git a/scripts/qa-transit-scenes.mjs b/scripts/qa-transit-scenes.mjs new file mode 100644 index 000000000..94ec134bb --- /dev/null +++ b/scripts/qa-transit-scenes.mjs @@ -0,0 +1,583 @@ +import { reportTransitVisibility } from './qa-transit-browser.mjs'; +import { writeFile } from 'node:fs/promises'; +import { + reduceFleet, + reduceMotion, + reduceBackground, + reduceSensorContrast, +} from '../src/layers/transit/qaMetrics.js'; + +/** CDP includes objects collected during the sample, not just surviving objects. */ +export function allocationTotals(profile) { + let frameBytes = 0, + layerFrameBytes = 0, + sharedOverlayBytes = 0, + qaBytes = 0; + const allocators = new Map(); + function visit(node, inFrame = false, inQa = false, inOverlay = false) { + const frame = node.callFrame || {}, + name = frame.functionName || '', + url = frame.url || ''; + const nested = + inFrame || + name === 'onPreRender' || + /^render\d*$/.test(name) || + /^Scene\d*\.render$/.test(name); + // QA wraps the real frame callback: re-enter production attribution at its + // onPreRender boundary, but exclude QA's own Math/Cesium helper descendants. + const qa = + name === 'onPreRender' + ? false + : inQa || /\/transit\/testing\.js(?:\?|$)/.test(url); + const overlay = inOverlay || name === '_drawOverlay'; + const bytes = node.selfSize || 0; + if (nested) { + if (qa) qaBytes += bytes; + else { + frameBytes += bytes; + const allocator = `${name || '(anonymous)'} ${url.split('?')[0]}:${(frame.lineNumber ?? -1) + 1}`; + allocators.set(allocator, (allocators.get(allocator) || 0) + bytes); + if (overlay) sharedOverlayBytes += bytes; + if ( + /\/layers\/transit\/|\/data\/(?:contactPlayback|contactTrailRenderer|transit[^/]*)\.js/.test( + url, + ) + ) + layerFrameBytes += bytes; + } + } + for (const child of node.children || []) visit(child, nested, qa, overlay); + } + visit(profile.head); + return { + frameBytes, + layerFrameBytes, + sharedOverlayBytes, + qaBytes, + topAllocators: [...allocators] + .sort((a, b) => b[1] - a[1]) + .slice(0, 3) + .map(([allocator, bytes]) => ({ allocator, bytes })), + }; +} + +export async function runFleetBudgets({ + page, + check, + wait, + shots, + tag, + detectOn, + selectStyle, +}) { + console.log( + '\n== moving fleet budgets: 1920×1080 CSS, warmed view, DETECT dense ==', + ); + await selectStyle(page, 'normal'); + await detectOn(page); + const cdp = await page.createCDPSession(); + await cdp.send('HeapProfiler.enable'); + for (const count of [800, 3000]) { + await page.evaluate(async (count) => { + const app = window.__godsEyeView; + await app.dataManager.setEnabled('transit', true, { source: 'qa' }); + const camera = app.viewer.camera, + C = camera.positionCartographic.constructor; + camera.setView({ + destination: C.toCartesian( + C.fromDegrees(-71.0605, 42.3554, count === 800 ? 600 : 30000), + ), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + const layer = app.dataManager.layers.get('transit').module; + layer._transitPartsForTest().ingestion.abortAllInFlight(); + layer._transitStateForTest()._activeFeeds.clear(); + for (const key of layer._transitStateForTest()._vehicles.keys()) + layer._transitPartsForTest().ingestion.removeVehicle(key); + }, count); + await wait(8000); + // Camera proximity can finish a live poll during tile warm-up. Remove it + // before the heap baseline, or its memory would discount the fixture fleet. + await page.evaluate(() => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(), + parts = layer._transitPartsForTest(); + parts.ingestion.abortAllInFlight(); + state._activeFeeds.clear(); + clearTimeout(state._cameraDebounceTimer); + state._cameraDebounceTimer = null; + for (const key of state._vehicles.keys()) + parts.ingestion.removeVehicle(key); + parts.rendering.syncRenderHold(); + window.__godsEyeView.viewer.scene.requestRender(); + }); + await wait(500); + await cdp.send('HeapProfiler.collectGarbage'); + const before = await cdp.send('Runtime.getHeapUsage'); + const loaded = await page.evaluate( + (count) => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._loadTransitFleetForTest( + count, + { lat: 42.3554, lon: -71.0605 }, + count === 800 ? 5 : 120, + ), + count, + ); + await reportTransitVisibility( + page, + `${count} budget visibility after load`, + ); + await wait(2000); + await cdp.send('HeapProfiler.collectGarbage'); + // The retained delta is conservative: it includes concurrent page growth. + const retained = await cdp.send('Runtime.getHeapUsage'); + // Timing/upload acceptance runs without the allocation profiler attached. + const trace = await page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._measureTransitFramesForTest(8000), + ); + await page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._resetTransitFixtureClockForTest(), + ); + await wait(1500); + await cdp.send('HeapProfiler.startSampling', { + samplingInterval: 128, + includeObjectsCollectedByMajorGC: true, + includeObjectsCollectedByMinorGC: true, + }); + await reportTransitVisibility( + page, + `${count} budget visibility before allocation trace`, + ); + const allocationTrace = await page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._measureTransitFramesForTest(6000), + ); + const { profile } = await cdp.send('HeapProfiler.stopSampling'); + const allocations = allocationTotals(profile); + const retainedBytes = retained.usedSize - before.usedSize; + const evidence = reduceFleet( + trace.rows, + count, + allocations.layerFrameBytes / allocationTrace.rows.length, + retainedBytes, + loaded.fixBytes, + ); + const details = { + ...evidence, + ...allocations, + wholeFrameBytesPerFrame: + allocations.frameBytes / allocationTrace.rows.length, + topAllocatorsPerFrame: allocations.topAllocators.map((a) => ({ + ...a, + bytesPerFrame: a.bytes / allocationTrace.rows.length, + })), + allocationFrames: allocationTrace.rows.length, + loaded, + dpr: trace.dpr, + viewport: [trace.width, trace.height], + heapMethod: + 'GC-bracketed whole-page growth after fleet load (conservative)', + allocationMethod: + 'CDP 128-byte sampling including collected objects; transit source self allocation gates, whole render stacks reported, QA scaffolding excluded', + }; + await writeFile( + `${shots}/${tag}-fleet-${count}.json`, + JSON.stringify(details, null, 2), + ); + await writeFile( + `${shots}/${tag}-fleet-${count}-allocations.json`, + JSON.stringify(profile), + ); + check( + `${count} moving vehicles meet CPU, frame interval, billboard upload, heap and allocation budgets`, + evidence.pass && trace.width === 1920 && trace.height === 1080, + JSON.stringify(details), + ); + check( + `${count}: transit-authored frame allocation <= 8 KiB`, + allocationTrace.rows.length >= 100 && + reduceMotion(allocationTrace.rows).pass && + allocationTrace.rows.every((r) => r.moving === count) && + allocations.layerFrameBytes / allocationTrace.rows.length <= 8192, + `${allocations.layerFrameBytes / allocationTrace.rows.length} transit-authored bytes/frame; whole frame ${allocations.frameBytes / allocationTrace.rows.length}; top three ${JSON.stringify(details.topAllocatorsPerFrame)}; collected objects included. Full profile saved.`, + ); + await page.screenshot({ + path: `${shots}/${tag}-fleet-${count}.jpg`, + type: 'jpeg', + quality: 85, + }); + const settled = await page.evaluate(async () => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const { seek } = await import('/src/data/contactPlayback.js'); + const { updatePlayback } = + await import('/src/layers/transit/movement.js'); + for (const entry of state._vehicles.values()) { + entry.clocks.wallNowMs = Date.now(); + entry.clocks.monoNowMs = performance.now(); + seek(entry.track, entry.fixes.at(-1).t, entry.clocks); + updatePlayback(entry, Date.now(), performance.now()); + layer._transitPartsForTest().rendering.schedulePlayback(entry); + } + layer._transitPartsForTest().rendering.syncRenderHold(); + return { moving: state._moving.size, held: state._renderHeld }; + }); + check( + `${count}: settling releases the transit hold`, + settled.moving === 0 && !settled.held, + JSON.stringify(settled), + ); + } + await cdp.detach(); +} + +export async function runBostonMatrix({ + page, + check, + wait, + shots, + tag, + selectStyle, + sampleRendered, + detectOn, +}) { + console.log( + '\n== Boston matrix: scripted Red, 741, 742, Green-E; actual map backgrounds ==', + ); + await detectOn(page); + const routes = ['Red', '741', '742', 'Green-E']; + const looks = [ + { name: 'normal', style: 'normal', params: {} }, + { name: 'nvg', style: 'surveillance', params: {} }, + { name: 'noir', style: 'noir', params: {} }, + { + name: 'white-hot', + style: 'thermal', + params: { 'WHOT/BHOT': 0, Ironbow: 0 }, + }, + { + name: 'black-hot', + style: 'thermal', + params: { 'WHOT/BHOT': 1, Ironbow: 0 }, + }, + { + name: 'ironbow', + style: 'thermal', + params: { 'WHOT/BHOT': 0, Ironbow: 1 }, + }, + ]; + // Actual Boston views. Acceptance checks the measured background; a tile/key + // failure or an unsuitable view is explicitly unexercised, never assumed bright/dark. + const backgrounds = [ + { + name: 'bright', + candidates: [ + { lat: 42.3458, lon: -71.0464 }, + { lat: 42.3659, lon: -71.018 }, + { lat: 42.3526, lon: -71.0461 }, + { lat: 42.3486, lon: -71.0825 }, + ], + }, + { + name: 'dark', + candidates: [ + { lat: 42.3574, lon: -71.0375 }, + { lat: 42.3554, lon: -71.0655 }, + { lat: 42.3588, lon: -71.0738 }, + { lat: 42.348, lon: -71.032 }, + ], + }, + ]; + const evidence = []; + for (const group of backgrounds) { + let background, backgroundResult; + const attempts = []; + await selectStyle(page, 'normal'); + for (const candidate of group.candidates) { + background = { ...candidate, name: group.name }; + await page.evaluate((b) => { + const camera = window.__godsEyeView.viewer.camera, + C = camera.positionCartographic.constructor; + camera.setView({ + destination: C.toCartesian(C.fromDegrees(b.lon, b.lat, 600)), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + }, background); + await wait(8000); + await page.evaluate( + ({ background, routes }) => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + layer._loadTransitFleetForTest(8, background, 45, routes); + for (const entry of layer._transitStateForTest()._vehicles.values()) { + entry.track.rate = 0; + layer._transitPartsForTest().rendering.schedulePlayback(entry); + } + }, + { background, routes }, + ); + await wait(1000); + backgroundResult = reduceBackground( + group.name, + await sampleRendered(page), + ); + attempts.push({ background, ...backgroundResult }); + if (backgroundResult.pass) break; + } + check( + `Boston ${background.name}: all four routes over measured ${background.name} background`, + backgroundResult.pass, + JSON.stringify({ attempts, chosen: background }), + { unexercised: !backgroundResult.pass }, + ); + for (const look of looks) { + await selectStyle(page, look.style, look.params); + await wait(1200); + await reportTransitVisibility( + page, + `Boston ${background.name}/${look.name} visibility`, + ); + const pixels = await sampleRendered(page); + const modes = await page.evaluate(() => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + return [...layer._transitStateForTest()._vehicles.values()].map( + (e) => ({ + key: e.key, + route: e.record.routeId, + image: e.marker.image, + width: e.marker.width, + color: e.marker.color.toCssHexString(), + }), + ); + }); + const contrast = + look.name === 'normal' + ? null + : reduceSensorContrast( + pixels, + look.name === 'black-hot' ? 'black' : 'white', + look.style, + ); + check( + `Boston ${background.name}/${look.name}: all four silhouettes and mode brackets readable`, + pixels.length >= 6 && + pixels.every( + (p) => + p.bracket && + p.bracketExpected && + Math.hypot(...p.bracket.map((v, i) => v - p.bracketExpected[i])) < + 48, + ) && + (!contrast || contrast.pass), + JSON.stringify({ pixels, contrast }), + { unexercised: pixels.length < 6 }, + ); + await page.screenshot({ + path: `${shots}/${tag}-matrix-${background.name}-${look.name}-fleet.jpg`, + type: 'jpeg', + quality: 90, + }); + for (const target of modes) { + await page.evaluate( + (key) => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitPartsForTest() + .selection.selectVehicle(key), + target.key, + ); + await wait(500); + await page.screenshot({ + path: `${shots}/${tag}-matrix-${background.name}-${look.name}-${target.route}-selected.jpg`, + type: 'jpeg', + quality: 90, + }); + const restored = await page.evaluate((key) => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + layer._transitPartsForTest().selection.clearSelection(); + const e = layer._transitStateForTest()._vehicles.get(key); + return { + image: e.marker.image, + width: e.marker.width, + color: e.marker.color.toCssHexString(), + }; + }, target.key); + check( + `Boston ${background.name}/${look.name}/${target.route}: deselect restores fleet styling`, + restored.image === target.image && + restored.width === target.width && + restored.color === target.color, + ); + } + await wait(300); + await page.screenshot({ + path: `${shots}/${tag}-matrix-${background.name}-${look.name}-restored.jpg`, + type: 'jpeg', + quality: 90, + }); + evidence.push({ + background, + look, + pixels, + routes: modes.map((m) => m.route), + }); + } + } + await selectStyle(page, 'normal'); + await writeFile( + `${shots}/${tag}-boston-matrix.json`, + JSON.stringify(evidence, null, 2), + ); +} + +/** Real MBTA observations at oblique street views, separate from matrix fixtures. */ +export async function runBostonLive({ + page, + check, + wait, + shots, + tag, + detectOn, + selectStyle, + sampleRendered, +}) { + console.log('\n== Boston live oblique streets =='); + await page.evaluate(async () => { + const app = window.__godsEyeView; + await app.dataManager.setEnabled('transit', false, { source: 'qa' }); + const camera = app.viewer.camera, + C = camera.positionCartographic.constructor; + camera.setView({ + destination: C.toCartesian(C.fromDegrees(-71.0605, 42.3554, 600)), + orientation: { heading: 0, pitch: -Math.PI / 4, roll: 0 }, + }); + await app.dataManager.setEnabled('transit', true, { source: 'qa' }); + const layer = app.dataManager.layers.get('transit').module; + layer._setTransitFixtureFloorsForTest([], 0); + await layer.update(); + }); + await wait(20000); + await detectOn(page); + const evidence = []; + for (const view of [ + { + name: 'white-hot', + style: 'thermal', + altitude: 120, + heading: 0, + pitch: -20, + params: { 'WHOT/BHOT': 0, Ironbow: 0 }, + }, + { + name: 'nvg', + style: 'surveillance', + altitude: 600, + heading: 30, + pitch: -45, + params: {}, + }, + { + name: 'noir', + style: 'noir', + altitude: 400, + heading: 60, + pitch: -35, + params: {}, + }, + ]) { + await page.evaluate((view) => { + const app = window.__godsEyeView, + camera = app.viewer.camera, + C = camera.positionCartographic.constructor; + app.dataManager.layers + .get('transit') + .module._transitPartsForTest() + .selection.clearSelection(); + camera.setView({ + destination: C.toCartesian( + C.fromDegrees(-71.0605, 42.3554, view.altitude), + ), + orientation: { + heading: (view.heading * Math.PI) / 180, + pitch: (view.pitch * Math.PI) / 180, + roll: 0, + }, + }); + }, view); + await selectStyle(page, view.style, view.params); + await wait(8000); + const fleet = await page.evaluate(() => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + const entries = [...state._vehicles.values()]; + return { + total: entries.length, + visible: entries.filter((e) => e.marker?.show).length, + fixtures: entries.filter((e) => e.qaFixture).length, + routes: [ + ...new Set( + entries.filter((e) => e.marker?.show).map((e) => e.record.routeId), + ), + ], + }; + }); + check( + `Boston live/${view.name}: observed vehicles in the oblique street view`, + fleet.total > 0 && fleet.visible > 0 && fleet.fixtures === 0, + JSON.stringify({ view, fleet }), + { unexercised: fleet.visible === 0 }, + ); + const pixels = await sampleRendered(page); + const contrast = reduceSensorContrast(pixels, 'white', view.style); + check( + `Boston live/${view.name}: unobscured sensor contrast`, + contrast.pass, + JSON.stringify(contrast), + { unexercised: contrast.unexercised }, + ); + const target = pixels[0]; + let selection = null; + if (target) { + await page.mouse.click(target.x, target.y); + await wait(250); + selection = await page.evaluate((key) => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + return { + expected: key, + actual: state._selectedKey, + pick: state._lastPickForTest ?? null, + }; + }, target.key); + } + check( + `Boston live/${view.name}: real mouse selects an observed vehicle`, + selection?.actual === target?.key && !!target, + JSON.stringify(selection), + { unexercised: !target }, + ); + await page.screenshot({ + path: `${shots}/${tag}-boston-live-${view.name}.jpg`, + type: 'jpeg', + quality: 90, + }); + evidence.push({ view, fleet, pixels, contrast, selection }); + } + await writeFile( + `${shots}/${tag}-boston-live.json`, + JSON.stringify(evidence, null, 2), + ); +} diff --git a/scripts/qa-transit.mjs b/scripts/qa-transit.mjs new file mode 100644 index 000000000..de09a6222 --- /dev/null +++ b/scripts/qa-transit.mjs @@ -0,0 +1,1951 @@ +#!/usr/bin/env node +import { + boundPageEvaluations, + reportTransitVisibility, + restoreRetainedTransit, + sampleTransitPixels, + readTransitFleetStyle, +} from './qa-transit-browser.mjs'; +import { + BOSTON, + chooseLiveCities, + selectStyle, + detectOn, +} from './qa-transit-controls.mjs'; +import { + runFleetBudgets, + runBostonMatrix, + runBostonLive, +} from './qa-transit-scenes.mjs'; +/** + * Browser proof for the Transit layer against a running app. + * + * npm run qa:transit (defaults to http://localhost:4173) + * QA_BASE_URL=http://localhost:4305 node scripts/qa-transit.mjs + * + * Drives the real layer toggle and the real CONTEXT chooser, watches the real + * /api/transit traffic, proves vehicles move between polls in the SCENE (not + * just in the JSON), captures street-level shots in two cities, and proves that + * turning the layer off stops every request and leaves nothing rendered. + * + * Screenshots land in the gitignored qa-shots/ directory unless QA_SHOTS says + * otherwise. Needs a running server; the feeds are keyless, so a keyless boot + * exercises the same paths. + */ +import { mkdir, readFile, writeFile } from 'node:fs/promises'; +import puppeteer from 'puppeteer'; +import { createTransitHistory } from '../server/providers/transitHistory.js'; +import { DETECTION_THEME_MAP } from '../src/overlays/worldOverlayTokens.js'; +import { transitModeTier } from '../src/data/transitPresetStyle.js'; + +import { + reduceAnchor, + reduceTrailHead, + reduceScenario, + reduceSensorContrast, + reduceScriptedMotion, + reduceMotion, + reduceFleet, +} from '../src/layers/transit/qaMetrics.js'; + +const BASE = process.env.QA_BASE_URL || 'http://localhost:4173'; +const SHOTS = process.env.QA_SHOTS || 'qa-shots/transit'; +const TAG = process.env.QA_TAG || 'keyed'; +/** Which sections to run: all, or a comma list of cities,playback,disable,context. */ +const SECTIONS = new Set((process.env.QA_SECTIONS || 'all').split(',')); +const runs = (section) => SECTIONS.has('all') || SECTIONS.has(section); +const CITIES = chooseLiveCities(); + +let failures = 0; +let unexercised = 0; +/** + * A check has three outcomes. PASS and FAIL are what they say. UNEXERCISED + * means the run could not put the claim to the test — a parked fleet has no + * motion to judge — and it is reported as its own word rather than as a + * pass, because a pass that proved nothing is how a broken interpolation + * sailed through this harness once. + */ +const check = ( + name, + passed, + detail = '', + { unexercised: idle = false } = {}, +) => { + const word = idle ? 'UNEXERCISED' : passed ? 'PASS' : 'FAIL'; + console.log(`[${word}] ${name}${detail ? ` — ${detail}` : ''}`); + if (idle) { + unexercised += 1; + failures += 1; + } else if (!passed) failures += 1; +}; +const renderingSource = await readFile( + new URL('../src/layers/transit/rendering.js', import.meta.url), + 'utf8', +); +const frameSource = renderingSource + .split('function onPreRender()')[1] + .split('const occluder')[0]; +check( + 'frame pass does not scan the fleet or copy the moving set', + !/_vehicles\.values|\[\.\.\.state\._moving\]/.test(frameSource), +); +const palette = { + bus: '#5EF08A', + tram: '#FFC24A', + subway: '#FF4538', + rail: '#D9A6FF', + ferry: '#5FD6FF', + unknown: '#D8DDE5', +}; +check( + 'all preset bracket themes preserve the six mode colours', + Object.values(DETECTION_THEME_MAP).every((theme) => + Object.entries(palette).every( + ([mode, color]) => + theme.tiers[transitModeTier(mode)].toUpperCase() === color, + ), + ), +); +check( + 'all required sections enabled', + SECTIONS.has('all'), + 'Partial runs are diagnostic and cannot pass acceptance.', +); +const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +/** Hex → [r,g,b]. */ +const rgbOf = (hex) => [1, 3, 5].map((i) => parseInt(hex.slice(i, i + 2), 16)); +const rgbDistance = (a, b) => Math.hypot(a[0] - b[0], a[1] - b[1], a[2] - b[2]); + +/** + * Rendered pixels of the scene and of the detection surface around each + * visible transit sprite: centre and darkest/brightest ring luma from the + * WebGL canvas (the app preserves its drawing buffer), and the dominant + * opaque colour painted on the detection surface in a bracket-sized window. + * Sprites closer than 44 px to another sprite are skipped so a ring is not + * read off a neighbour. + * @param {import('puppeteer').Page} page + * @returns {Promise>} + */ +const sampleRendered = (page) => sampleTransitPixels(page, palette); + +/** + * The rendered position of every VISIBLE transit billboard, read back through + * Cesium. Visible only: off-screen vehicles take their newest fix directly, so + * including them would let hidden vehicles snapping stand in for motion a + * person can see. + * @param {import('puppeteer').Page} page + * @returns {Promise>} + */ +/** + * Window-space position and billboard rotation of every visible transit + * sprite. Orientation is a screen-space claim, so it has to be checked in + * screen space. + * @param {import('puppeteer').Page} page + * @returns {Promise>} + */ +async function sampleScreen(page) { + return page.evaluate(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const scene = app.viewer.scene; + const canvas = scene.canvas; + // The app exposes no `window.Cesium`, so the projection is done here in + // plain numbers: Cesium matrices are column-major arrays of 16 doubles, + // and window coordinates are the standard clip -> NDC -> viewport walk. + const view = scene.camera.viewMatrix; + const proj = scene.camera.frustum.projectionMatrix; + const mul = (m, v) => [ + m[0] * v[0] + m[4] * v[1] + m[8] * v[2] + m[12] * v[3], + m[1] * v[0] + m[5] * v[1] + m[9] * v[2] + m[13] * v[3], + m[2] * v[0] + m[6] * v[1] + m[10] * v[2] + m[14] * v[3], + m[3] * v[0] + m[7] * v[1] + m[11] * v[2] + m[15] * v[3], + ]; + const width = canvas.clientWidth; + const height = canvas.clientHeight; + const out = []; + for (const entry of state._vehicles.values()) { + if (!entry.marker || entry.marker.show === false) continue; + const p = entry.marker.position; + const clip = mul(proj, mul(view, [p.x, p.y, p.z, 1])); + if (!(clip[3] > 0)) continue; + const ndcX = clip[0] / clip[3]; + const ndcY = clip[1] / clip[3]; + out.push({ + entityCount: app.viewer.entities.values.length, + key: entry.key, + x: (ndcX * 0.5 + 0.5) * width, + y: (1 - (ndcY * 0.5 + 0.5)) * height, + rotation: entry.marker.rotation, + pollSeq: entry.pollSeq, + segment: + entry.from && entry.to ? `${entry.from.t}-${entry.to.t}` : null, + }); + if (out.length >= 200) break; + } + return out; + }); +} + +async function sampleVisible(page) { + return page.evaluate(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const Cartographic = app.viewer.camera.positionCartographic.constructor; + return [...state._vehicles.values()] + .filter((entry) => entry.marker && entry.marker.show !== false) + .slice(0, 200) + .map((entry) => { + const carto = Cartographic.fromCartesian(entry.marker.position); + return { + key: entry.key, + lat: (carto.latitude * 180) / Math.PI, + lon: (carto.longitude * 180) / Math.PI, + gliding: state._moving.has(entry), + // Which poll and which displayed segment this reading belongs to: + // two readings that straddle a poll or a segment change are not + // evidence about interpolation, whatever the elapsed time says. + pollSeq: entry.pollSeq, + segment: + entry.from && entry.to ? `${entry.from.t}-${entry.to.t}` : null, + }; + }); + }); +} + +// Accelerate the storage clock without collecting a live feed in the background. +const historyProbe = createTransitHistory({ + sweep: false, + storage: { bytes: 0, keys: 0, stores: new Set() }, + limits: { feedBytes: 128000, processBytes: 128000 }, +}); +const retainedSizes = []; +for (let now = 0; now <= 7200000; now += 15000) { + historyProbe.ingest( + { id: 'probe', historyRetention: true, defaultMode: 'bus' }, + Array.from({ length: 12 }, (_, i) => ({ + id: String(i), + timestamp: now / 1000, + lat: 42, + lon: -71, + })), + now, + ); + if (now >= 900000) + retainedSizes.push(historyProbe.diagnostics().allocatedBytes); +} +check( + 'two-hour proxy history allocation plateaus', + Math.max(...retainedSizes) === Math.min(...retainedSizes) && + Math.max(...retainedSizes) <= 128000, + `${retainedSizes.length} warmed samples, ${Math.max(...retainedSizes)} allocated bytes`, +); +historyProbe.clear(); +await mkdir(SHOTS, { recursive: true }); + +const browser = await puppeteer.launch({ + headless: false, + protocolTimeout: 150000, + executablePath: + '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', + args: [ + '--no-sandbox', + '--disable-background-timer-throttling', + '--disable-renderer-backgrounding', + '--window-size=1920,1080', + '--enable-precise-memory-info', + ], +}); +const page = await browser.newPage(); +boundPageEvaluations(page); +await page.setViewport({ width: 1920, height: 1080 }); +const pageErrors = []; +page.on('pageerror', (error) => pageErrors.push(error.message)); +page.on('console', (message) => { + if (message.text().startsWith('TRANSIT_SAMPLER')) console.log(message.text()); +}); +const transitRequests = []; +page.on('request', (request) => { + if (request.url().includes('/api/transit')) { + transitRequests.push({ at: Date.now(), url: request.url() }); + } +}); + +const shot = (name, caption) => + page + .screenshot({ path: `${SHOTS}/${name}`, type: 'jpeg', quality: 82 }) + .then(() => { + console.log(` shot ${name} — ${caption}`); + }); + +const view = (city) => + `${BASE}/?welcome=0#lat=${city.lat}&lon=${city.lon}&alt=600&heading=${city.heading}&pitch=-45`; + +try { + await page.goto(view(BOSTON), { waitUntil: 'domcontentloaded' }); + // Ready when the catalog is registered and its rows exist. (The manager's + // private panel handle moved into a presentation facade; waiting on it here + // was waiting for something that no longer exists.) + await page.waitForFunction( + () => + Boolean(window.__godsEyeView?.dataManager?.layers?.size) && + Boolean(document.querySelector('[data-layer-id="transit"]')), + { timeout: 90_000 }, + ); + await wait(6_000); + + // The Data Layers panel starts collapsed; a user opens it before toggling. + const expanded = await page.$eval( + '#data-panel .panel-collapse-btn', + (button) => { + if (button.getAttribute('aria-expanded') === 'true') return true; + button.click(); + return false; + }, + ); + if (!expanded) await wait(800); + check( + 'the Data Layers panel opens from its own header button', + await page.$eval( + '#data-panel .panel-collapse-btn', + (b) => b.getAttribute('aria-expanded') === 'true', + ), + ); + + // Record what the layer hands the shared overlay host, so the card check can + // quote the operator's own words rather than assert an internal key was set. + await page.evaluate(() => { + window.__gevTransitCards = new Map(); + const layer = window.__godsEyeView.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const host = state.DEFAULT_OVERLAY_HOST; + layer._setTransitOverlayHostForTest({ + setEntries(sourceId, entries, options) { + for (const entry of entries || []) { + window.__gevTransitCards.set(entry.id, { + title: entry.title, + details: entry.details, + }); + } + return host.setEntries(sourceId, entries, options); + }, + setVisible: host.setVisible, + clearSource(sourceId) { + window.__gevTransitCards.clear(); + return host.clearSource(sourceId); + }, + }); + }); + + const row = '[data-layer-id="transit"]'; + const hasRow = await page.$(row); + check('the Transit layer has a row in the layer panel', Boolean(hasRow)); + await page.$eval(row, (element) => + element.scrollIntoView({ block: 'center' }), + ); + await wait(400); + const before = transitRequests.length; + await page.click(`${row} .data-toggle-btn`); + await page.waitForFunction( + () => window.__godsEyeView.dataManager.isEnabled('transit'), + { timeout: 20_000 }, + ); + check( + 'clicking the row button enables the layer and starts real traffic', + transitRequests.length > before, + `${transitRequests.length - before} request(s)`, + ); + + await page.evaluate(() => { + window.__transitRenderErrors = []; + window.__godsEyeView.viewer.scene.renderError.addEventListener( + (scene, error) => window.__transitRenderErrors.push(error.message), + ); + }); + for (const city of runs('cities') ? CITIES : []) { + console.log(`\n== ${city.label} ==`); + // City-wide first: enough altitude to see the fleet as a fleet. + await page.evaluate((c) => { + const camera = window.__godsEyeView.viewer.camera; + const Cartographic = camera.positionCartographic.constructor; + camera.setView({ + destination: Cartographic.toCartesian( + Cartographic.fromDegrees(c.lon, c.lat - 0.03, 3_500), + ), + orientation: { heading: 0, pitch: (-40 * Math.PI) / 180, roll: 0 }, + }); + }, city); + await page.waitForFunction( + () => + (window.__godsEyeView.dataManager.layers + .get('transit') + .module.getStats().count || 0) > 0, + { timeout: 90_000 }, + ); + await wait(9_000); + const fleet = await page.evaluate(() => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + return { + count: layer._transitStateForTest()._vehicles.size, + stats: layer.getStats(), + }; + }); + check( + `${city.label}: vehicles are rendered`, + fleet.count >= 20, + `${fleet.count} vehicles, coverage "${fleet.stats.coverage}"`, + ); + await shot( + `${TAG}-${city.id}-00-fleet.jpg`, + `${city.label} from 3.5 km — ${fleet.count} vehicles across the metro`, + ); + + // Street level: 600 m up, 45 degrees down, framed on a real vehicle so the + // shot shows what a user sees when they drop in on one. + const framed = await page.evaluate((c) => { + const app = window.__godsEyeView; + const camera = app.viewer.camera; + const Cartographic = camera.positionCartographic.constructor; + const state = app.dataManager.layers + .get('transit') + .module._transitStateForTest(); + let best = null; + for (const entry of state._vehicles.values()) { + const d = Math.hypot(entry.to.lat - c.lat, entry.to.lon - c.lon); + if (!best || d < best.d) + best = { d, lat: entry.to.lat, lon: entry.to.lon, key: entry.key }; + } + if (!best) return null; + // Camera 600 m back along the heading, 600 m up, looking 45 degrees down: + // the vehicle lands in the middle of the frame. + const heading = (c.heading * Math.PI) / 180; + const lat = best.lat - (600 * Math.cos(heading)) / 111_320; + const lon = + best.lon - + (600 * Math.sin(heading)) / + (111_320 * Math.cos((best.lat * Math.PI) / 180)); + camera.setView({ + destination: Cartographic.toCartesian( + Cartographic.fromDegrees(lon, lat, 600), + ), + orientation: { heading, pitch: (-45 * Math.PI) / 180, roll: 0 }, + }); + return best; + }, city); + check( + `${city.label}: a vehicle to fly to`, + Boolean(framed), + framed?.key || 'none', + ); + await wait(9_000); + + const first = await page.evaluate(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const Cartographic = app.viewer.camera.positionCartographic.constructor; + return { + count: state._vehicles.size, + stats: layer.getStats(), + // The RENDERED primitive position, read back through Cesium — not the + // fix the layer is aiming at. A layer that never moved a point would + // still show two different `entry.to` values, so comparing those would + // prove the data changed, not that anything on screen did. + // VISIBLE markers only. Off-screen vehicles take their newest fix + // directly — that is the whole point of the view gate — so sampling + // them would let a layer whose visible interpolation is broken pass + // this check on the strength of hidden vehicles snapping. + sample: [...state._vehicles.values()] + .filter((entry) => entry.marker && entry.marker.show !== false) + .slice(0, 200) + .map((entry) => { + const carto = Cartographic.fromCartesian(entry.marker.position); + return { + key: entry.key, + lat: (carto.latitude * 180) / Math.PI, + lon: (carto.longitude * 180) / Math.PI, + }; + }), + }; + }); + + // Click a vehicle that is actually on screen, recomputing its position + // immediately before each attempt — these dots are gliding. + let selected = null; + for (let attempt = 0; attempt < 6 && !selected; attempt += 1) { + const target = await page.evaluate((skip) => { + const app = window.__godsEyeView; + const state = app.dataManager.layers + .get('transit') + .module._transitStateForTest(); + const scene = app.viewer.scene; + const seen = []; + for (const entry of state._vehicles.values()) { + const screen = scene.cartesianToCanvasCoordinates( + entry.marker.position, + ); + if ( + screen && + screen.x > 360 && + screen.x < scene.canvas.clientWidth - 120 && + screen.y > 120 && + screen.y < scene.canvas.clientHeight - 160 + ) { + seen.push({ + key: entry.key, + x: Math.round(screen.x), + y: Math.round(screen.y), + }); + } + } + return seen[skip] || null; + }, attempt); + if (!target) break; + await page.mouse.click(target.x, target.y); + await wait(900); + selected = await page.evaluate(() => { + const app = window.__godsEyeView; + const key = app.dataManager.layers + .get('transit') + .module._transitStateForTest()._selectedKey; + if (!key) return null; + // A selected key is bookkeeping. The claim is that the user SEES a + // card, and these cards are painted onto the shared overlay CANVAS + // rather than built from DOM nodes — so the proof is the operator's own + // words the layer published, plus pixels that actually landed there. + const published = window.__gevTransitCards?.get(key) || null; + const canvas = document.getElementById('world-overlay-canvas'); + let painted = 0; + if (canvas?.width) { + const context = canvas.getContext('2d', { willReadFrequently: true }); + const pixels = context.getImageData( + 0, + 0, + canvas.width, + canvas.height, + ).data; + for (let i = 3; i < pixels.length; i += 64) { + if (pixels[i] > 8) painted += 1; + } + } + return { key, published, painted }; + }); + if (selected && (!selected.published || selected.painted === 0)) { + selected = null; + } + } + const cardLines = selected + ? [selected.published.title, ...(selected.published.details || [])] + : []; + check( + `${city.label}: clicking a vehicle raises a card with its details on screen`, + cardLines.length >= 3 && + /Route |Vehicle /.test(cardLines[0] || '') && + cardLines.some((line) => /Reported /.test(line)) && + selected.painted > 0, + selected + ? `${selected.key} — "${cardLines.slice(0, 4).join(' / ')}" over ${selected.painted} painted overlay samples` + : 'no vehicle card became visible', + ); + // DETECT is on by default at DENSE. Prove the overlay actually boxes and + // labels transit contacts rather than ignoring them, which is what the + // owner found: the layer drew dots and the sensor overlay looked past them. + const detect = await page.evaluate(() => { + const gev = window.__godsEyeView; + const layer = gev.dataManager.layers.get('transit').module; + const objects = layer.getDetectableObjects({ maxCount: 200 }); + const button = document.getElementById('detection-toggle'); + return { + mode: (button?.textContent || '').replace(/\s+/g, ' ').trim(), + count: objects.length, + sample: objects.slice(0, 3).map((o) => `${o.id}/${o.metric || ''}`), + types: [...new Set(objects.map((o) => o.type))], + labelled: objects.filter((o) => o.id && o.metric).length, + skipped: objects.filter((o) => o.skipLabel).length, + }; + }); + // What the host DOES with these is pinned by the detection-host unit test, + // which drives the real painter and reads the route and mode back off the + // canvas. Here the claim is narrower and fully attributable: the layer + // offers DETECT properly formed contacts, which is precisely what it used + // to offer none of. + check( + `${city.label}: the layer offers DETECT real contacts, not bare dots`, + detect.count > 0 && + detect.types.length === 1 && + detect.types[0] === 'VEH' && + detect.labelled === detect.count, + `${detect.count} contacts, all labelled, e.g. ${detect.sample.join(', ')} (DETECT ${detect.mode})`, + ); + + // The glyphs themselves: modes must not all be one shape. + const sprites = await page.evaluate(() => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + const images = new Map(); + for (const entry of state._vehicles.values()) { + if (!entry.marker?.show) continue; + // Compare the WHOLE data URI: every glyph shares a long base64 prefix, + // so a truncated comparison says they are all the same picture. + images.set(entry.mode, String(entry.marker.image || '')); + } + return { + modes: [...images.keys()], + distinct: new Set(images.values()).size, + glyphBytes: [...images.values()].map((uri) => uri.length), + billboards: [...state._vehicles.values()].filter((e) => e.marker) + .length, + shown: state._shownCount, + }; + }); + check( + `${city.label}: vehicles draw as mode sprites, not plain dots`, + sprites.distinct === sprites.modes.length && sprites.modes.length > 0, + `${sprites.modes.join(', ')} → ${sprites.distinct} distinct glyph(s) of sizes ${sprites.glyphBytes.join('/')}; ${sprites.shown} shown of ${sprites.billboards}`, + ); + + await shot( + `${TAG}-${city.id}-01-vehicles.jpg`, + `${city.label} at 600 m, 45° pitch — ${first.count} vehicles in the layer, selected card on the one in frame`, + ); + + // ---- Rendered appearance under the sensor presets (first active live city) -------- + // In-scene sprites pass through the post-FX chain; what matters is what + // reaches the screen. DETECT on, then each preset the owner tested: + // pixels of the sprite core and its ring from the WebGL canvas, the + // colour painted for its bracket on the detection surface, and one + // screenshot per preset for the diary. + if (city.id === CITIES[0].id) { + console.log('\n== rendered appearance =='); + const profile = await detectOn(page); + check( + 'DETECT reaches the dense profile through its own button', + profile === 'DENSE', + profile, + ); + const themeFor = (style) => + DETECTION_THEME_MAP[style] || DETECTION_THEME_MAP._default; + const presets = [ + { name: 'normal', style: 'normal', params: {}, hot: null }, + { + name: 'thermal-whot', + style: 'thermal', + params: { 'WHOT/BHOT': 0, Ironbow: 0 }, + hot: 'white', + }, + { + name: 'thermal-bhot', + style: 'thermal', + params: { 'WHOT/BHOT': 1, Ironbow: 0 }, + hot: 'black', + }, + { + name: 'thermal-ironbow', + style: 'thermal', + params: { 'WHOT/BHOT': 0, Ironbow: 1 }, + hot: 'white', + }, + { + name: 'surveillance', + style: 'surveillance', + params: {}, + hot: 'white', + }, + { name: 'noir', style: 'noir', params: {}, hot: 'white' }, + { name: 'retro', style: 'retro', params: {}, hot: null }, + ]; + await selectStyle(page, 'normal'); + const baseline = await page.evaluate(readTransitFleetStyle, { + resetSelection: true, + }); + for (const preset of presets) { + const how = await selectStyle(page, preset.style, preset.params); + await wait(1_200); + const styling = await page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStylingForTest(), + ); + const rendered = await sampleRendered(page); + await shot( + `${TAG}-boston-style-${preset.name}.jpg`, + `${city.label} at street level under ${preset.name}, DETECT on — ${rendered.length} isolated sprites sampled (style set via ${how})`, + ); + const modes = new Set(rendered.map((r) => r.mode)); + if (preset.hot) { + const contrast = reduceSensorContrast( + rendered, + preset.hot, + preset.style, + ); + check( + `${preset.name}: sprite cores have the expected sensor contrast`, + contrast.pass, + JSON.stringify(contrast), + { unexercised: rendered.length < 6 }, + ); + check( + `${preset.name}: the layer styled its sprites for ${preset.style}`, + styling.stylePreset === preset.style, + JSON.stringify(styling), + ); + } + // Brackets: the detection surface paints each contact in its tier + // colour, which must be the mode's colour for this theme. + const theme = themeFor(preset.style); + const matched = rendered.filter( + (r) => + r.bracket && + rgbDistance( + r.bracket, + rgbOf(theme.tiers[transitModeTier(r.mode)]), + ) < 48, + ); + const byMode = new Map(); + for (const r of rendered) if (r.bracket) byMode.set(r.mode, r.bracket); + const modeColours = [...byMode.values()]; + let distinct = true; + for (let i = 0; i < modeColours.length; i += 1) + for (let j = i + 1; j < modeColours.length; j += 1) + if (rgbDistance(modeColours[i], modeColours[j]) < 40) + distinct = false; + check( + `${preset.name}: brackets carry the mode colour (${[...modes].join(', ') || 'no modes'})`, + rendered.length >= 6 && + matched.length >= Math.ceil(rendered.length * 0.75) && + (byMode.size < 2 || distinct), + rendered.length === 0 + ? 'no isolated sprite in frame to sample' + : `${matched.length} of ${rendered.length} within tolerance of their tier; ${byMode.size} mode(s) painted${byMode.size >= 2 ? (distinct ? ', distinguishable' : ', NOT distinguishable') : ''}; misses=${JSON.stringify( + rendered + .filter((r) => !matched.includes(r)) + .map((r) => ({ + key: r.key, + sampled: r.bracket, + tier: transitModeTier(r.mode), + expected: rgbOf(theme.tiers[transitModeTier(r.mode)]), + })), + )}`, + { unexercised: rendered.length < 6 }, + ); + const clickTarget = rendered[0]; + if (clickTarget) { + await page.mouse.click(clickTarget.x, clickTarget.y); + await wait(100); + const picked = await page.evaluate((key) => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + return { + selected: state._selectedKey === key, + pickOwner: state._lastPickForTest, + }; + }, clickTarget.key); + check( + `${preset.name}: mouse click selects the displayed sprite`, + picked.selected, + JSON.stringify(picked), + ); + await page.keyboard.press('Escape'); + } else + check( + `${preset.name}: mouse click selects the displayed sprite`, + false, + 'No verified unobscured sprite', + { unexercised: true }, + ); + check( + `${preset.name}: the icon raster cache stays bounded`, + styling.iconCacheSize <= 6 * 2 * 3, + `${styling.iconCacheSize} rasters`, + ); + } + // Selection under FLIR keeps the sprite white; leaving FLIR restores it. + await selectStyle(page, 'thermal', { 'WHOT/BHOT': 0, Ironbow: 0 }); + const target = (await sampleRendered(page))[0]; + if (target) { + await page.mouse.click(target.x, target.y); + await wait(800); + const underFlir = await page.evaluate((key) => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const entry = state._vehicles.get(key); + return { + selected: state._selectedKey === key, + color: entry?.marker.color.toCssColorString(), + width: entry?.marker.width, + pickOwner: (() => { + const p = window.__godsEyeView.viewer.scene.pick( + entry.marker.computeScreenSpacePosition( + window.__godsEyeView.viewer.scene, + ), + ); + return { + id: typeof p?.id === 'string' ? p.id : null, + primitiveId: + typeof p?.primitive?.id === 'string' ? p.primitive.id : null, + collection: + entry.markerCollection === state._animatedMarkers + ? 'animated' + : 'stationary', + }; + })(), + }; + }, target.key); + check( + 'clicking a sprite under FLIR selects it and keeps it white-hot', + underFlir.selected && underFlir.color === 'rgb(255,255,255)', + JSON.stringify(underFlir), + ); + await page.keyboard.press('Escape'); + await wait(500); + } else { + check( + 'clicking a sprite under FLIR selects it and keeps it white-hot', + false, + 'no isolated sprite to click', + { unexercised: true }, + ); + } + await selectStyle(page, 'normal'); + const restored = await page.evaluate(readTransitFleetStyle, { + key: baseline?.key, + }); + check( + 'returning to normal restores the shipped sprite exactly', + Boolean(restored && baseline) && + !restored.selected && + !baseline.selected && + restored.color === baseline.color && + restored.width === baseline.width && + restored.image === baseline.image, + restored && baseline + ? `${baseline.key}: selected ${baseline.selected}→${restored.selected}, ${baseline.color}/${baseline.width}px → ${restored.color}/${restored.width}px; raster equal ${baseline.image === restored.image}` + : 'the baseline vehicle left the fleet', + { unexercised: !(restored && baseline) }, + ); + } + + // Motion acceptance uses known report-time playback even when live fleets park. + await selectStyle(page, 'normal'); + await page.evaluate((city) => { + const app = window.__godsEyeView, + camera = app.viewer.camera; + const C = camera.positionCartographic.constructor; + camera.setView({ + destination: C.toCartesian(C.fromDegrees(city.lon, city.lat, 600)), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + }, city); + await wait(2000); + await page.evaluate( + (city) => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._loadTransitFleetForTest(16, city, 25), + city, + ); + await wait(1000); + const motionFirst = await sampleVisible(page); + await wait(30_000); + const second = await page.evaluate(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const Cartographic = app.viewer.camera.positionCartographic.constructor; + return { + count: state._vehicles.size, + // The RENDERED primitive position, read back through Cesium — not the + // fix the layer is aiming at. A layer that never moved a point would + // still show two different `entry.to` values, so comparing those would + // prove the data changed, not that anything on screen did. + // VISIBLE markers only. Off-screen vehicles take their newest fix + // directly — that is the whole point of the view gate — so sampling + // them would let a layer whose visible interpolation is broken pass + // this check on the strength of hidden vehicles snapping. + sample: [...state._vehicles.values()] + .filter((entry) => entry.marker && entry.marker.show !== false) + .slice(0, 200) + .map((entry) => { + const carto = Cartographic.fromCartesian(entry.marker.position); + return { + key: entry.key, + lat: (carto.latitude * 180) / Math.PI, + lon: (carto.longitude * 180) / Math.PI, + }; + }), + }; + }); + const beforeById = new Map(motionFirst.map((v) => [v.key, v])); + let moved = 0; + let matched = 0; + for (const v of second.sample) { + const was = beforeById.get(v.key); + if (!was) continue; + matched += 1; + const dLat = (v.lat - was.lat) * 111_320; + const dLon = + (v.lon - was.lon) * 111_320 * Math.cos((v.lat * Math.PI) / 180); + if (Math.hypot(dLat, dLon) > 5) moved += 1; + } + check( + `${city.label} deterministic fixture: the same vehicles are still drawn 30 s later, in new places`, + matched >= 5 && moved >= 3, + `${matched} same visible ids, ${moved} whose RENDERED point moved more than 5 m`, + ); + + // Between polls, not across them. Positions changing over thirty seconds + // only proves new data arrived; a layer that snapped every fifteen seconds + // and drew nothing in between would pass that. Two reads two seconds apart + // fall inside one poll interval, so anything that moves is being + // INTERPOLATED on screen. + const glideA = await sampleVisible(page); + await wait(2_000); + const glideB = await sampleVisible(page); + const glideBefore = new Map(glideA.map((v) => [v.key, v])); + let glided = 0; + let glideMatched = 0; + let straddled = 0; + // Only vehicles the LAYER says are mid-segment can testify here, and only + // readings from the SAME poll and the same displayed segment: a pair that + // straddles a poll could have snapped, and would pass as motion. + let claimed = 0; + for (const v of glideB) { + const was = glideBefore.get(v.key); + if (!was) continue; + glideMatched += 1; + if (!(was.gliding || v.gliding)) continue; + if (was.pollSeq !== v.pollSeq || was.segment !== v.segment) { + straddled += 1; + continue; + } + claimed += 1; + const dLat = (v.lat - was.lat) * 111_320; + const dLon = + (v.lon - was.lon) * 111_320 * Math.cos((v.lat * Math.PI) / 180); + if (Math.hypot(dLat, dLon) > 1) glided += 1; + } + check( + `${city.label} deterministic fixture: a vehicle the layer says is mid-segment moves on screen between polls`, + claimed > 0 && glided >= 1, + claimed === 0 + ? `nothing was mid-segment inside one poll across ${glideMatched} visible vehicles (${straddled} pairs straddled a poll)` + : `${glided} of ${claimed} mid-segment vehicles moved within one poll and one segment (${straddled} straddling pairs discarded)`, + { unexercised: claimed === 0 }, + ); + + // Orientation, measured the only way that settles it: where the sprite + // points against where the sprite actually travelled on screen. The + // projection is perspective-exact, so an off-centre vehicle at an oblique + // pitch must agree too — that is precisely the case the orthographic + // approximation got wrong by twenty-odd degrees. + // + // Only sprites that visibly moved can testify, and a keyless Austin view + // holds a handful of them; the floor is what that view can honestly + // supply, and the accuracy bar does the work. + const headingA = await sampleScreen(page); + await wait(6_000); + const headingB = await sampleScreen(page); + const wasById = new Map(headingA.map((v) => [v.key, v])); + let aimed = 0; + let checked = 0; + let worstDeg = 0; + for (const now of headingB) { + const was = wasById.get(now.key); + if (!was) continue; + // A poll in between could have placed it; that is not travel. + if (was.pollSeq !== now.pollSeq) continue; + const dx = now.x - was.x; + const dy = now.y - was.y; + // Only vehicles that visibly travelled can testify about direction. + if (Math.hypot(dx, dy) < 6) continue; + checked += 1; + // The sprite is authored nose-up and rotated CCW, so at rotation r its + // nose points along (-sin r, -cos r) in window coordinates. + const noseX = -Math.sin(now.rotation); + const noseY = -Math.cos(now.rotation); + const travel = Math.hypot(dx, dy); + const cosErr = (noseX * dx + noseY * dy) / travel; + const errDeg = + (Math.acos(Math.max(-1, Math.min(1, cosErr))) * 180) / Math.PI; + if (errDeg <= 35) aimed += 1; + else worstDeg = Math.max(worstDeg, errDeg); + } + check( + `${city.label} deterministic fixture: sprites point the way they are travelling on screen`, + checked > 0 && aimed >= Math.ceil(checked * 0.8), + checked === 0 + ? 'no sprite travelled far enough on screen inside one poll to testify' + : `${aimed} of ${checked} moving sprites within 35° of their screen travel` + + (worstDeg ? `, worst miss ${Math.round(worstDeg)}°` : ''), + { unexercised: checked === 0 }, + ); + + await shot( + `${TAG}-${city.id}-02-thirty-seconds-later.jpg`, + `${city.label} deterministic fixture 30 s later — ${second.count} vehicles, ${moved}/${matched} moved`, + ); + } + + check( + 'vehicle selection never raises a Cesium render error', + (await page.evaluate(() => window.__transitRenderErrors || [])).length === + 0, + JSON.stringify( + await page.evaluate(() => window.__transitRenderErrors || []), + ), + ); + + if (runs('history')) { + console.log('\n== retained history reload =='); + const recovery = await browser.newPage(); + boundPageEvaluations(recovery); + try { + await recovery.goto(view(BOSTON), { waitUntil: 'domcontentloaded' }); + await recovery.waitForFunction( + () => window.__godsEyeView?.dataManager?.layers?.has('transit'), + { timeout: 90000 }, + ); + // Leave enough time for two snapshots, priming and reload in one minute. + if (Date.now() % 60000 > 20000) + await wait(60000 - (Date.now() % 60000) + 50); + const first = await recovery.evaluate(async () => + ( + await fetch('/api/transit/vehicles/mbta', { + signal: AbortSignal.timeout(10000), + }) + ).json(), + ); + await wait(16000); + const retained = await recovery.evaluate(async (prior) => { + const current = await ( + await fetch('/api/transit/vehicles/mbta', { + signal: AbortSignal.timeout(10000), + }) + ).json(); + const candidates = current.vehicles.filter((v) => + prior.vehicles.some((p) => p.id === v.id), + ); + // Prime the newest fixes from this exact feed, not evenly spaced stale IDs. + const primedAt = Date.now(); + const picked = candidates + .sort((a, b) => (b.timestamp || 0) - (a.timestamp || 0)) + .slice(0, 12); + const histories = await Promise.all( + picked.map(async (vehicle) => { + const response = await fetch( + `/api/transit/trail/mbta/${encodeURIComponent(vehicle.id)}`, + { signal: AbortSignal.timeout(10000) }, + ); + if (!response.ok) + throw new Error(`Trail prime HTTP ${response.status}`); + return response.json(); + }), + ); + return histories + .filter((history) => history.fixes?.length >= 2) + .map((history) => ({ ...history, feedId: 'mbta', primedAt })); + }, first); + await recovery.reload({ waitUntil: 'domcontentloaded' }); + await recovery.waitForFunction( + () => window.__godsEyeView?.dataManager?.layers?.has('transit'), + { timeout: 90000 }, + ); + await recovery.evaluate(() => { + window.__transitRenderErrors = []; + window.__godsEyeView.viewer.scene.renderError.addEventListener( + (scene, error) => window.__transitRenderErrors.push(error.message), + ); + }); + const restored = await recovery.evaluate( + restoreRetainedTransit, + retained, + ); + if (restored.key) { + await recovery.waitForFunction( + () => + !window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitPartsForTest() + .trails.requestDiagnostics().pending, + { timeout: 20000 }, + ); + await wait(4000); + const proof = await recovery.evaluate( + ({ key, oldest }) => { + const layer = + window.__godsEyeView.dataManager.layers.get('transit').module; + const entry = layer._transitStateForTest()._vehicles.get(key), + parts = layer._transitPartsForTest(); + if (!entry) return { gone: true }; + const segments = entry.trailSegments?.length || 0; + const renderErrors = window.__transitRenderErrors || []; + const count = entry.track.count, + restoredOldest = entry.fixes[0].t; + const before = parts.trails.requestDiagnostics().abortCount; + parts.selection.selectVehicle(key); + parts.selection.clearSelection(); + return { + count, + segments, + renderErrors, + restoredOldest, + oldest, + aborted: parts.trails.requestDiagnostics().abortCount > before, + }; + }, + { key: restored.key, oldest: restored.oldest }, + ); + check( + 'reload restores observed history and selection churn aborts its request', + proof.count >= 2 && + proof.restoredOldest <= proof.oldest && + proof.aborted, + proof.gone + ? 'Selected primed vehicle left during recovery' + : JSON.stringify(proof), + { unexercised: proof.gone === true }, + ); + if (!proof.gone) + check( + 'reloaded MBTA trail prepares geometry without a Cesium render error', + proof.segments > 0 && proof.renderErrors.length === 0, + JSON.stringify(proof), + { + unexercised: + proof.segments === 0 && proof.renderErrors.length === 0, + }, + ); + } else + check('reload restores observed history', false, restored.error, { + unexercised: restored.unexercised === true, + }); + } finally { + await recovery.close(); + await page.bringToFront(); + } + } + + // ---- Deterministic playback and card anchor (synthetic feed) ----------- + // The live feeds prove the layer against reality; they cannot prove it + // against a script. For the next two minutes the MBTA response is replaced + // at the network layer by six scripted vehicles near the Boston view — + // straight travel, a corner, a genuine stop, a ninety-second silence, a + // late packet and a feed that only refreshes every thirty seconds — and + // every rendered frame is sampled. + if (runs('playback')) { + console.log('\n== deterministic playback =='); + await page.evaluate((c) => { + const camera = window.__godsEyeView.viewer.camera; + const Cartographic = camera.positionCartographic.constructor; + camera.setView({ + destination: Cartographic.toCartesian( + Cartographic.fromDegrees(c.lon, c.lat + 0.005, 2400), + ), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + }, BOSTON); + await page.evaluate(() => { + window.__godsEyeView.dataManager.layers + .get('transit') + .module._setTransitFixtureFloorsForTest( + ['straight', 'corner', 'stop', 'gap', 'late', 'dup'].map( + (id) => `mbta:sim-${id}`, + ), + 20, + ); + }); + const ORIGIN = { lat: BOSTON.lat, lon: BOSTON.lon }; + const M_LAT = 1 / 111_320; + const M_LON = 1 / (111_320 * Math.cos((ORIGIN.lat * Math.PI) / 180)); + const at = (eastM, northM) => ({ + lat: ORIGIN.lat + northM * M_LAT, + lon: ORIGIN.lon + eastM * M_LON, + }); + const simStart = Date.now(); + const reportTime = (elapsedS, cadenceS) => + Math.floor(elapsedS / cadenceS) * cadenceS; + const SIM = { + straight: { speed: 10 }, + corner: { speed: 8, turnAt: 60 }, + stop: { speed: 8, stopFrom: 30, stopTo: 75 }, + gap: { speed: 6, silentFrom: 40, silentTo: 130 }, + late: { speed: 8, lateFrom: 50, lateTo: 60 }, + dup: { speed: 8, cadence: 30 }, + }; + const synthetic = (nowMs) => { + const elapsed = (nowMs - simStart) / 1000; + const vehicles = []; + const push = (id, pos, tS) => + vehicles.push({ + id, + lat: pos.lat, + lon: pos.lon, + bearing: null, + speedMps: null, + timestamp: Math.round(simStart / 1000 + tS), + timestampSource: 'vehicle', + routeId: id.toUpperCase(), + tripId: null, + directionId: null, + label: null, + stopId: null, + status: null, + occupancy: null, + }); + { + const t = reportTime(elapsed, 10); + push('sim-straight', at(-220, SIM.straight.speed * t), t); + } + { + const t = reportTime(elapsed, 10); + const n = Math.min(t, SIM.corner.turnAt) * SIM.corner.speed; + const e = Math.max(0, t - SIM.corner.turnAt) * SIM.corner.speed; + push('sim-corner', at(110 + e, n), t); + } + { + const t = reportTime(elapsed, 10); + const moved = + t <= SIM.stop.stopFrom + ? t + : t <= SIM.stop.stopTo + ? SIM.stop.stopFrom + : SIM.stop.stopFrom + (t - SIM.stop.stopTo); + push('sim-stop', at(-110, moved * SIM.stop.speed), t); + } + { + const t0 = reportTime(elapsed, 10); + const t = + t0 >= SIM.gap.silentFrom && t0 < SIM.gap.silentTo + ? SIM.gap.silentFrom + : t0; + push('sim-gap', at(220, t * SIM.gap.speed), t); + } + { + const t0 = reportTime(elapsed, 10); + const t = + elapsed >= SIM.late.lateFrom && elapsed < SIM.late.lateTo + ? t0 - 30 + : t0; + push('sim-late', at(330, t * SIM.late.speed), t); + } + { + const t = reportTime(elapsed, SIM.dup.cadence); + push('sim-dup', at(-330, t * SIM.dup.speed), t); + } + return { + feedId: 'mbta', + name: 'MBTA', + fetchedAt: nowMs, + feedTimestamp: null, + version: '2.0', + entityCount: vehicles.length, + truncated: false, + count: vehicles.length, + vehicles, + }; + }; + let intercepting = true; + await page.setRequestInterception(true); + const onRequest = (request) => { + if ( + intercepting && + request.url().includes('/api/transit/vehicles/mbta') + ) { + const now = Date.now(); + request.respond({ + status: 200, + contentType: 'application/json', + headers: { 'x-transit-contact': String(now) }, + body: JSON.stringify(synthetic(now)), + }); + return; + } + request.continue(); + }; + page.on('request', onRequest); + // Wait for the scripted fleet to replace the live one. + await page.waitForFunction( + () => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + return state._vehicles.has('mbta:sim-straight'); + }, + { timeout: 60_000 }, + ); + const simState = () => + page.evaluate(() => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + const now = Date.now(); + return ['straight', 'corner', 'stop', 'gap', 'late', 'dup'] + .map((k) => { + const e = state._vehicles.get(`mbta:sim-${k}`); + if (!e) return `${k}:absent`; + return `${k}:${e.marker ? (e.marker.show === false ? 'hidden' : 'shown') : 'nomarker'}/lag${((now - e.playT) / 1000).toFixed(0)}/${state._moving.has(e) ? 'mv' : 'idle'}/fx${e.fixes.length}/r${e.resets || 0}`; + }) + .join(' '); + }); + console.log(` scripted fleet before trace: ${await simState()}`); + await reportTransitVisibility(page, 'scripted visibility before trace'); + // Sample every rendered frame for two minutes. + const trace = await page.evaluate( + (durationMs) => + new Promise((resolve) => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const Cartographic = + app.viewer.camera.positionCartographic.constructor; + const keys = [ + 'sim-straight', + 'sim-corner', + 'sim-stop', + 'sim-gap', + 'sim-late', + 'sim-dup', + ].map((k) => `mbta:${k}`); + const out = Object.fromEntries(keys.map((k) => [k, []])); + const words = Object.fromEntries(keys.map((k) => [k, []])); + const t0 = performance.now(); + let lastWordAt = -1e9; + let remove; + const timer = setTimeout(() => { + remove?.(); + resolve({ out, words }); + }, durationMs + 1000); + const tick = () => { + const now = performance.now(); + const wall = Date.now(); + for (const key of keys) { + const entry = state._vehicles.get(key); + if (!entry?.marker?.show) continue; + const carto = Cartographic.fromCartesian(entry.marker.position); + out[key].push([ + now - t0, + (carto.latitude * 180) / Math.PI, + (carto.longitude * 180) / Math.PI, + entry.playT, + entry.resets || 0, + state._moving.has(entry) ? 1 : 0, + entry.courseDeg, + wall, + entry.track.count, + entry.sample.segmentSpeedMps, + entry.sample.fromSeq, + entry.marker.position.x, + entry.marker.position.y, + entry.marker.position.z, + entry.sample.toSeq, + entry.sample.phase, + entry.clocks.monoNowMs, + ]); + } + if (now - lastWordAt > 1_000) { + lastWordAt = now; + for (const contact of layer.getDetectableObjects()) { + if (words[contact.sourceId]) + words[contact.sourceId].push([now - t0, contact.metric]); + } + } + if (now - t0 >= durationMs) { + clearTimeout(timer); + remove(); + resolve({ out, words }); + } + }; + remove = app.viewer.scene.postRender.addEventListener(tick); + }), + 120_000, + ); + console.log(` scripted fleet after trace: ${await simState()}`); + await reportTransitVisibility(page, 'scripted visibility after trace'); + const metres = (a, b) => + Math.hypot( + (b[1] - a[1]) * 111_320, + (b[2] - a[2]) * 111_320 * Math.cos((a[1] * Math.PI) / 180), + ); + const judge = (key) => reduceScenario(trace, key); + // Preserve every frame and both bracket ends; never join moving rows across a hold. + const speedEvidence = Object.entries(trace.out).map(([key, rows]) => ({ + key, + ...reduceScriptedMotion(key, rows), + })); + await writeFile( + `${SHOTS}/${TAG}-scripted-frames.json`, + JSON.stringify(trace), + ); + for (const evidence of speedEvidence) + check( + `${evidence.key}: ordinary postRender frames follow the displayed segment within 2%`, + evidence.pass, + JSON.stringify(evidence), + ); + const scenarioCheck = (name, conditions, detail) => + check( + name, + Object.values(conditions).every(Boolean), + `${detail}; conditions=${JSON.stringify(conditions)}; failed=${JSON.stringify(Object.keys(conditions).filter((k) => !conditions[k]))}`, + ); + const straight = judge('straight'); + scenarioCheck( + 'straight travel is played back continuously at its own speed', + { + 'straight.rows > 3_000': straight.rows > 3_000, + 'straight.total > 300': straight.total > 300, + 'straight.moved > straight.rows * 0.4': + straight.moved > straight.rows * 0.4, + 'straight.worstJump <= SIM.straight.speed * 1.02 + 0.1': + straight.worstJump <= SIM.straight.speed * 1.02 + 0.1, + 'straight.speedWorst >= SIM.straight.speed * 0.98': + straight.speedWorst >= SIM.straight.speed * 0.98, + 'straight.speedWorst <= SIM.straight.speed * 1.02': + straight.speedWorst <= SIM.straight.speed * 1.02, + 'straight.wallFaster === 0': straight.wallFaster === 0, + 'straight.resets === 0': straight.resets === 0, + }, + `${straight.rows} frames (${straight.keys}), ${straight.total.toFixed(0)} m drawn, ${straight.moved} moving frames, worst per-frame ${straight.worstJump.toFixed(1)} m/s, peak segment speed ${straight.speedWorst.toFixed(2)} m/s vs ${SIM.straight.speed}, ${straight.wallFaster} frames faster than real time, ${straight.resets} resets`, + ); + scenarioCheck( + 'the display lag is bounded and never below the feed floor', + { + 'straight.lagValid': straight.lagValid, + 'straight.lagMin >= 24': straight.lagMin >= 24, + 'straight.lagMax <= 215': straight.lagMax <= 215, + }, + `lag ${straight.lagMin.toFixed(1)}–${straight.lagMax.toFixed(1)} s`, + ); + const corner = judge('corner'); + const cornerRows = trace.out['mbta:sim-corner'] || []; + const courseLate = cornerRows + .slice(-300) + .map((r) => r[6]) + .filter(Number.isFinite); + const courseEarly = cornerRows + .filter((r) => r[5] === 1) + .slice(0, 300) + .map((r) => r[6]) + .filter(Number.isFinite); + const mean = (xs) => + xs.length ? xs.reduce((s, x) => s + x, 0) / xs.length : NaN; + scenarioCheck( + 'a corner is turned on the displayed segment: north first, east once the turn is played', + { + 'courseEarly.length > 0': courseEarly.length > 0, + 'courseLate.length > 0': courseLate.length > 0, + 'Math.abs(mean(courseEarly)) < 12': Math.abs(mean(courseEarly)) < 12, + 'Math.abs(mean(courseLate) - 90) < 15': + Math.abs(mean(courseLate) - 90) < 15, + 'corner.worstJump <= SIM.corner.speed * 1.5 + 2': + corner.worstJump <= SIM.corner.speed * 1.5 + 2, + 'corner.resets === 0': corner.resets === 0, + }, + `early course ${mean(courseEarly).toFixed(1)}°, late course ${mean(courseLate).toFixed(1)}°, worst per-frame ${corner.worstJump.toFixed(1)} m/s`, + ); + const stopWords = (trace.words['mbta:sim-stop'] || []).map((w) => w[1]); + const stopRows = trace.out['mbta:sim-stop'] || []; + const stopStill = stopRows.filter( + (r, i) => i > 0 && metres(stopRows[i - 1], r) < 0.01, + ).length; + scenarioCheck( + 'a genuine stop reads STOPPED while the marker stands still, then moves on', + { + 'stopWords.some((w) => /STOPPED/.test(w))': stopWords.some((w) => + /STOPPED/.test(w), + ), + 'stopStill > 600': stopStill > 600, + "judge('stop').total > 200": judge('stop').total > 200, + "judge('stop').resets === 0": judge('stop').resets === 0, + }, + `words seen: ${[...new Set(stopWords)].join(' | ')}; ${stopStill} still frames; ${judge('stop').total.toFixed(0)} m drawn`, + ); + const gap = judge('gap'); + scenarioCheck( + 'a ninety-second silence holds at the newest fix — no extrapolation, no reset — and resumes', + { + 'gap.resets === 0': gap.resets === 0, + 'gap.worstJump <= SIM.gap.speed * 1.5 + 2': + gap.worstJump <= SIM.gap.speed * 1.5 + 2, + 'gap.total > 100': gap.total > 100, + }, + `${gap.total.toFixed(0)} m drawn, worst per-frame ${gap.worstJump.toFixed(1)} m/s, ${gap.resets} resets`, + ); + const late = judge('late'); + scenarioCheck( + 'a late packet is refused: no backward motion, no reset', + { + 'late.backwards === 0': late.backwards === 0, + 'late.resets === 0': late.resets === 0, + 'late.total > 200': late.total > 200, + }, + `${late.backwards} backward frames, ${late.resets} resets, ${late.total.toFixed(0)} m drawn`, + ); + const dup = judge('dup'); + scenarioCheck( + 'a feed that refreshes every thirty seconds plays at the fix speed, not in bursts', + { + 'dup.speedWorst <= SIM.dup.speed * 1.06': + dup.speedWorst <= SIM.dup.speed * 1.06, + 'dup.worstJump <= SIM.dup.speed * 1.5 + 2': + dup.worstJump <= SIM.dup.speed * 1.5 + 2, + 'dup.total > 200': dup.total > 200, + 'dup.resets === 0': dup.resets === 0, + }, + `peak segment speed ${dup.speedWorst.toFixed(2)} m/s vs ${SIM.dup.speed}, worst per-frame ${dup.worstJump.toFixed(1)} m/s, ${dup.total.toFixed(0)} m drawn`, + ); + + // ---- Card anchor synchronisation --------------------------------------- + console.log('\n== card anchor =='); + await page.evaluate(() => { + const app = window.__godsEyeView, + camera = app.viewer.camera; + const entry = app.dataManager.layers + .get('transit') + .module._transitStateForTest() + ._vehicles.get('mbta:sim-straight'); + if (!entry) return; + const C = camera.positionCartographic.constructor; + const at = C.fromCartesian(entry.marker.position); + at.height = 600; + camera.setView({ + destination: C.toCartesian(at), + orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, + }); + }); + await wait(1500); + await reportTransitVisibility(page, 'scripted visibility before click'); + const target = await page.evaluate(() => { + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + const entry = state._vehicles.get('mbta:sim-straight'); + if (!entry?.marker?.show) return null; + const scene = app.viewer.scene; + const canvas = scene.canvas; + const view = scene.camera.viewMatrix; + const proj = scene.camera.frustum.projectionMatrix; + const mul = (m, v) => [ + m[0] * v[0] + m[4] * v[1] + m[8] * v[2] + m[12] * v[3], + m[1] * v[0] + m[5] * v[1] + m[9] * v[2] + m[13] * v[3], + m[2] * v[0] + m[6] * v[1] + m[10] * v[2] + m[14] * v[3], + m[3] * v[0] + m[7] * v[1] + m[11] * v[2] + m[15] * v[3], + ]; + const p = entry.marker.position; + const clip = mul(proj, mul(view, [p.x, p.y, p.z, 1])); + const x = ((clip[0] / clip[3]) * 0.5 + 0.5) * canvas.clientWidth; + const y = (1 - ((clip[1] / clip[3]) * 0.5 + 0.5)) * canvas.clientHeight; + if ( + !(clip[3] > 0) || + x < 0 || + y < 0 || + x >= canvas.clientWidth || + y >= canvas.clientHeight + ) + return null; + const picked = scene.pick({ x, y }); + return { + entityCount: app.viewer.entities.values.length, + key: entry.key, + x, + y, + pickKey: picked?.id ?? picked?.primitive?.id ?? null, + }; + }); + let anchor = null; + if (target) { + check( + 'the scripted sprite owns its screen-space pick', + target.pickKey === target.key, + JSON.stringify(target), + ); + await page.mouse.click(target.x, target.y); + await wait(600); + anchor = await page.evaluate( + async (key, durationMs) => { + const overlay = await import('/src/overlays/worldOverlay.js'); + const app = window.__godsEyeView; + const layer = app.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + if (state._selectedKey !== key) + return { + selected: false, + selection: { + expected: key, + actual: state._selectedKey, + handler: !!state._clickHandler, + lastPick: state._lastPickForTest ?? null, + pointerOwner: ( + await import('/src/data/inputOwnership.js') + ).pointerOwner(), + }, + }; + const scene = app.viewer.scene; + const canvas = scene.canvas; + const mul = (m, v) => [ + m[0] * v[0] + m[4] * v[1] + m[8] * v[2] + m[12] * v[3], + m[1] * v[0] + m[5] * v[1] + m[9] * v[2] + m[13] * v[3], + m[2] * v[0] + m[6] * v[1] + m[10] * v[2] + m[14] * v[3], + m[3] * v[0] + m[7] * v[1] + m[11] * v[2] + m[15] * v[3], + ]; + const frames = []; + const publicationsBefore = state._cardPublications || 0; + let headSamples = 0, + headErrorPx = 0, + bodyMutation = 0, + previousTrail = null; + const entityCount = app.viewer.entities.values.length; + return new Promise((resolve) => { + const t0 = performance.now(); + const tick = () => { + const entry = state._vehicles.get(key); + const rect = overlay.getOverlayPaintRect('transit-selected', key); + if (entry?.marker) { + const view = scene.camera.viewMatrix; + const proj = scene.camera.frustum.projectionMatrix; + const p = entry.marker.position; + const clip = mul(proj, mul(view, [p.x, p.y, p.z, 1])); + frames.push({ + t: performance.now() - t0, + mx: ((clip[0] / clip[3]) * 0.5 + 0.5) * canvas.clientWidth, + my: + (1 - ((clip[1] / clip[3]) * 0.5 + 0.5)) * + canvas.clientHeight, + markerVisible: entry.marker.show !== false, + worldFinite: [p.x, p.y, p.z].every(Number.isFinite), + cardPresent: !!rect, + surfaceReady: entry.surfaceReady, + ax: rect?.anchorX ?? null, + ay: rect?.anchorY ?? null, + poll: entry.pollSeq, + moving: state._moving.has(entry) ? 1 : 0, + }); + } + const trail = layer._transitPartsForTest().trails.diagnostics(); + if (trail && entry?.marker) { + if ( + previousTrail && + previousTrail.revision === trail.revision && + previousTrail.body !== trail.body + ) + bodyMutation++; + previousTrail = trail; + if (trail.head.show && trail.head.positions.length) { + const project = (p) => { + const clip = mul( + scene.camera.frustum.projectionMatrix, + mul(scene.camera.viewMatrix, [p.x, p.y, p.z, 1]), + ); + return [ + ((clip[0] / clip[3]) * canvas.clientWidth) / 2, + ((clip[1] / clip[3]) * canvas.clientHeight) / 2, + ]; + }; + const h = project(trail.head.positions.at(-1)), + m = project(entry.marker.position); + headErrorPx = Math.max( + headErrorPx, + Math.hypot(h[0] - m[0], h[1] - m[1]), + ); + headSamples++; + } + } + if (performance.now() - t0 >= durationMs) finish(); + }; + let remove; + const finish = () => { + clearTimeout(timer); + remove?.(); + resolve({ + selected: true, + frames, + textPublishes: + (state._cardPublications || 0) - publicationsBefore, + headSamples, + headErrorPx, + bodyMutation, + entityCount, + }); + }; + const timer = setTimeout(finish, durationMs + 1000); + remove = scene.postRender.addEventListener(tick); + }); + }, + target.key, + 20_000, + ); + } + check( + 'selected trail head stays within 1 CSS px of the marker, without entity additions or body rebuilds between revisions', + reduceTrailHead(anchor, target).pass, + JSON.stringify(reduceTrailHead(anchor, target)), + ); + if (anchor?.selected) { + await writeFile( + `${SHOTS}/${TAG}-anchor-frames.json`, + JSON.stringify(anchor.frames), + ); + const evidence = reduceAnchor(anchor.frames); + check( + 'actual painted host anchor stays within 1 CSS px of the marker', + evidence.pass, + JSON.stringify(evidence), + ); + check( + 'card text publication stays throttled while the anchor runs at frame rate', + anchor.textPublishes > 0 && anchor.textPublishes <= 20_000 / 250 + 3, + `${anchor.textPublishes} text publications in 20 s`, + ); + } else { + check( + 'the card anchor moves in lockstep with the sprite on every frame, across polls and a settle', + false, + target + ? 'the click did not select the scripted vehicle' + : 'no scripted vehicle on screen', + { unexercised: true }, + ); + } + // Removal: the scripted vehicle leaves the feed; after two missed polls its + // card must go with it. + intercepting = false; + await page.evaluate(() => + window.__godsEyeView.dataManager.layers + .get('transit') + .module._setTransitFixtureFloorsForTest([], null), + ); + page.off('request', onRequest); + await page.setRequestInterception(false); + await page + .waitForFunction( + () => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + return !state._vehicles.has('mbta:sim-straight'); + }, + { timeout: 60_000 }, + ) + .catch(() => {}); + const afterRemoval = await page.evaluate(() => { + const state = window.__godsEyeView.dataManager.layers + .get('transit') + .module._transitStateForTest(); + return { + gone: !state._vehicles.has('mbta:sim-straight'), + selected: state._selectedKey, + }; + }); + check( + 'a removed vehicle takes its card with it', + afterRemoval.gone && afterRemoval.selected !== 'mbta:sim-straight', + JSON.stringify(afterRemoval), + ); + await page.evaluate((c) => { + const camera = window.__godsEyeView.viewer.camera; + const Cartographic = camera.positionCartographic.constructor; + camera.setView({ + destination: Cartographic.toCartesian( + Cartographic.fromDegrees(c.lon, c.lat, 600), + ), + orientation: { + heading: (c.heading * Math.PI) / 180, + pitch: (-45 * Math.PI) / 180, + roll: 0, + }, + }); + }, CITIES.at(-1)); + await wait(4_000); + } + + const sceneQA = { + page, + check, + wait, + shots: SHOTS, + tag: TAG, + detectOn, + selectStyle, + sampleRendered, + }; + if (runs('budgets')) await runFleetBudgets(sceneQA); + if (runs('matrix')) await runBostonMatrix(sceneQA); + if (runs('boston-live')) await runBostonLive(sceneQA); + + // Turning it off must stop the traffic and leave nothing behind. + console.log('\n== disable =='); + await page.$eval(row, (element) => + element.scrollIntoView({ block: 'center' }), + ); + await page.click(`${row} .data-toggle-btn`); + await page.waitForFunction( + () => !window.__godsEyeView.dataManager.isEnabled('transit'), + { timeout: 20_000 }, + ); + const afterDisable = transitRequests.length; + const teardown = await page.evaluate(() => { + const layer = window.__godsEyeView.dataManager.layers.get('transit').module; + const state = layer._transitStateForTest(); + return { + vehicles: state._vehicles.size, + primitives: + (state._markers?.length || 0) + (state._animatedMarkers?.length || 0), + visible: state._visible.size, + moving: state._moving.size, + detect: state._detectCache, + timers: [ + state._visibilityTimer, + state._maintenanceTimer, + state._floorTimer, + state._cameraDebounceTimer, + ], + trailPending: layer._transitPartsForTest().trails.requestDiagnostics() + .pending, + shown: state._markers ? state._markers.show : null, + inFlight: state._inFlight.size, + activeFeeds: state._activeFeeds.size, + holds: window.__godsEyeView.getRenderGovernorDiagnostics().holds, + }; + }); + check( + 'disable leaves no vehicles and no point primitives', + teardown.vehicles === 0 && + teardown.primitives === 0 && + teardown.visible === 0 && + teardown.moving === 0 && + teardown.detect === null && + teardown.timers.every((t) => t === null) && + !teardown.trailPending && + !teardown.holds.includes('transit'), + JSON.stringify(teardown), + ); + await shot( + `${TAG}-disabled.jpg`, + 'Transit off — same view, nothing rendered', + ); + await wait(25_000); + check( + 'no further transit requests after disable', + transitRequests.length === afterDisable, + `${transitRequests.length - afterDisable} request(s) in 25 s`, + ); + + // ---- Context modes ------------------------------------------------------- + console.log('\n== context modes =='); + // The CONTEXT panel starts collapsed; a user expands it before choosing a mode. + await page.$eval( + '[data-collapse-target="global-context-panel"]', + (button) => { + if (button.getAttribute('aria-expanded') !== 'true') button.click(); + }, + ); + await wait(1_200); + check( + 'the CONTEXT panel opens from its own header button', + await page.$eval( + '[data-collapse-target="global-context-panel"]', + (b) => b.getAttribute('aria-expanded') === 'true', + ), + ); + + const isOn = () => + page.evaluate(() => window.__godsEyeView.dataManager.isEnabled('transit')); + await page.evaluate(() => + window.__godsEyeView.dataManager.setEnabled('transit', true, { + origin: 'user', + }), + ); + await wait(6_000); + check('transit is on before entering a Context mode', await isOn()); + + // CONTACTS + await page.click('#global-context-flights-btn'); + await wait(18_000); + const inContacts = { + transit: await isOn(), + vehicles: await page.evaluate( + () => + window.__godsEyeView.dataManager.layers.get('transit').module.getStats() + .count, + ), + }; + // Contacts is not a passive overlay: it puts every non-dependency layer away + // for the duration, Transit included, and gives it back on exit. + check( + 'Contacts puts Transit away for the duration', + inContacts.transit === false && inContacts.vehicles === 0, + JSON.stringify(inContacts), + ); + await page.click('#global-context-flights-btn'); + await wait(12_000); + check('leaving Contacts restores Transit', await isOn()); + + // SPACE MISSIONS + await page.evaluate(() => + window.__godsEyeView.dataManager.setEnabled('transit', false, { + origin: 'user', + }), + ); + await wait(2_000); + await page.click('#global-context-missions-btn'); + await wait(20_000); + const blocked = await page.evaluate(async () => { + const settled = await window.__godsEyeView.dataManager.setEnabled( + 'transit', + true, + { origin: 'user' }, + ); + return { + settled, + enabled: window.__godsEyeView.dataManager.isEnabled('transit'), + }; + }); + check( + 'Space Missions refuses to enable Transit while it isolates replay data', + blocked.enabled === false, + JSON.stringify(blocked), + ); + await page.click('#global-context-missions-btn'); + await wait(12_000); + check( + 'leaving Space Missions leaves Transit off, as it was on entry', + (await isOn()) === false, + ); + + check( + 'no uncaught page errors', + pageErrors.length === 0, + pageErrors.join(' | '), + ); + console.log( + `\ntotal /api/transit requests this run: ${transitRequests.length}`, + ); + check( + 'scripted selections never raise a Cesium render error', + (await page.evaluate(() => window.__transitRenderErrors || [])).length === + 0, + JSON.stringify( + await page.evaluate(() => window.__transitRenderErrors || []), + ), + ); +} catch (error) { + check( + 'browser harness completed within its bounded waits', + false, + error.stack || error.message, + ); +} finally { + await browser.close(); +} +console.log( + failures + ? `HARNESS: FAIL (${failures} failed, ${unexercised} unexercised)` + : `HARNESS: PASS (${unexercised} unexercised)`, +); +process.exit(failures || unexercised ? 1 : 0); diff --git a/server/providers/common/http.js b/server/providers/common/http.js index 0e530bb29..654d8e3d7 100644 --- a/server/providers/common/http.js +++ b/server/providers/common/http.js @@ -3,6 +3,50 @@ export { readResponseJsonCapped, } from '../../../src/sources/httpBody.js'; +/** + * Read a fetch() Response body as bytes with the same hard cap as + * readResponseTextCapped — for protobuf upstreams (GTFS-Realtime). + * Throws { code:'RESPONSE_TOO_LARGE' }. + */ +export async function readResponseBytesCapped(response, maxBytes) { + const tooLarge = () => { + const err = new Error('Upstream response too large'); + err.code = 'RESPONSE_TOO_LARGE'; + return err; + }; + const declared = Number(response.headers.get('content-length')); + if (Number.isFinite(declared) && declared > maxBytes) throw tooLarge(); + const reader = response.body?.getReader?.(); + if (!reader) { + const bytes = new Uint8Array(await response.arrayBuffer()); + if (bytes.byteLength > maxBytes) throw tooLarge(); + return bytes; + } + const chunks = []; + let total = 0; + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + total += value.byteLength; + if (total > maxBytes) { + try { + await reader.cancel(); + } catch { + /* no-op */ + } + throw tooLarge(); + } + chunks.push(value); + } + const out = new Uint8Array(total); + let offset = 0; + for (const chunk of chunks) { + out.set(chunk, offset); + offset += chunk.byteLength; + } + return out; +} + /** * Return the existing promise for a cache key, or create one and remove it * only when that exact promise settles. diff --git a/server/providers/local.js b/server/providers/local.js index eb9059bd5..68f61a3cb 100644 --- a/server/providers/local.js +++ b/server/providers/local.js @@ -13,6 +13,7 @@ import { cctvProxy } from './cctv.js'; import { defaultSourceRoot } from './common/source-root.js'; import { radioBrowserProxy } from './radio.js'; import { gbfsProxy } from './gbfs.js'; +import { transitProxy } from './transit.js'; import { adsbLolProxy } from './aircraft/adsb-lol.js'; import { aisLiveProxy } from './vessels/ais-live.js'; import { trackBackfillProxies } from './aircraft/tracks.js'; @@ -38,6 +39,7 @@ function localProviderPlugins() { cctvProxy({ sourceRoot: defaultSourceRoot }), radioBrowserProxy(), gbfsProxy(), + transitProxy(), adsbLolProxy(), aisLiveProxy(), trackBackfillProxies(), diff --git a/server/providers/transit.js b/server/providers/transit.js new file mode 100644 index 000000000..14c2de52d --- /dev/null +++ b/server/providers/transit.js @@ -0,0 +1,427 @@ +import { createTransitHistory } from './transitHistory.js'; +/** + * GTFS-Realtime transit proxy for the Transit data layer. + * + * Only URLs registered in `src/data/transitFeeds.js` are ever fetched — the + * browser names a registered id, never a URL. Pure request/cache mechanics + * live in `src/data/transitProxy.js` so the offline suite covers them. + */ + +import { + coalesceProxyRequest, + readResponseBytesCapped, +} from './common/http.js'; +import { makeRateLimiter } from './common/rate-limit.js'; +import { publicTransitCatalog } from '../../src/data/transitFeeds.js'; +import { + TRANSIT_ADMISSION_MAX_GLOBAL, + TRANSIT_ADMISSION_MAX_PER_FEED, + TRANSIT_ADMISSION_WINDOW_MS, + TRANSIT_MAX_REDIRECTS, + TRANSIT_PROXY_MAX_BODY_BYTES, + TRANSIT_PROXY_TIMEOUT_MS, + TRANSIT_PROXY_TTL_MS, + TransitFeedShapeError, + buildTransitSnapshot, + isAcceptableTransitUpstreamUrl, + isTransitRedirectStatus, + nextTransitBackoffMs, + resolveTransitRoute, + transitCacheState, + transitRedirectDecision, + transitResponseHeaders, + transitUpstreamHeaders, +} from '../../src/data/transitProxy.js'; + +/** + * Fetch a registered feed, resolving each redirect hop BEFORE following it. + * + * `redirect: 'follow'` hands the hop to the runtime and only lets us inspect + * where it landed — by which point a disallowed host has already been + * contacted and has already answered. Each hop is therefore requested with + * `redirect: 'manual'` and its `Location` checked against the feed's own https + * origin first, the rule the CCTV frame proxy applies to camera images. + * + * @param {object} feed Registry entry. + * @param {AbortSignal} signal Abort signal for the whole hop chain. + * @param {typeof fetch} [fetchImpl] Injected for tests. + * @param {{etag?: string|null, lastModified?: string|null}} [validators] Cached validators. + * @returns {Promise<{response: Response, finalUrl: string}>} + */ +export async function fetchTransitFeed( + feed, + signal, + fetchImpl = fetch, + validators = null, +) { + let current = feed.url; + for (let hop = 0; hop <= TRANSIT_MAX_REDIRECTS; hop += 1) { + const response = await fetchImpl(current, { + signal, + headers: transitUpstreamHeaders(feed, validators), + redirect: 'manual', + }); + // Only a real redirect is followed. A 304 is the successful answer to our + // own conditional request and has no Location; sending it round this loop + // fails a feed that is simply unchanged. + if (!isTransitRedirectStatus(response?.status)) { + return { response, finalUrl: current }; + } + const decision = transitRedirectDecision( + feed.url, + current, + response.headers.get('location'), + ); + try { + await response.body?.cancel(); + } catch { + /* no-op */ + } + if (!decision.ok) throw new Error(`upstream ${decision.reason}`); + if (hop === TRANSIT_MAX_REDIRECTS) { + throw new Error('upstream redirected too many times'); + } + current = decision.url; + } + throw new Error('upstream redirected too many times'); +} + +/** + * Vite plugin: GTFS-Realtime VehiclePositions proxy for the Transit layer. + * + * GET /api/transit/feeds → public catalog (coverage + credit) + * GET /api/transit/vehicles/ → decoded snapshot as JSON + * + * Only URLs in `src/data/transitFeeds.js` are ever fetched — the browser + * names a registered id, never a URL (SECURITY.md). Redirects are resolved hop + * by hop and must stay on the feed's own https origin. Bytes are capped at + * TRANSIT_PROXY_MAX_BODY_BYTES and decoded server-side, so the browser never + * parses protobuf. Per feed: 15 s memory cache, single-flight refresh, and + * serve-stale-on-failure for up to 10 minutes (the launch-library pattern). + * No disk cache — transit positions are worthless after a few minutes. + * + * Two backstops sit in front of every upstream request. A fixed-window + * admission limiter bounds how often this process may ask any one operator, + * and all of them together, no matter how many browser tabs are open. A + * per-feed cooldown ladder then keeps a feed that is genuinely down from being + * re-asked once per poll for a whole session: consecutive failures push the + * next permitted attempt out to five minutes, and one success resets it. + * + * @param {{fetchImpl?: typeof fetch}} [options] + * @returns {object} Vite plugin. + */ +export function transitProxy({ fetchImpl = fetch } = {}) { + /** @type {Map} feedId → snapshot */ + const cache = new Map(); + const history = createTransitHistory(); + const admitHistory = makeRateLimiter({ + windowMs: 60000, + max: 30, + globalMax: 120, + }); + let closed = false; + const inFlight = new Map(); + const controllers = new Set(); + /** @type {Map} */ + const cooldown = new Map(); + const admit = makeRateLimiter({ + windowMs: TRANSIT_ADMISSION_WINDOW_MS, + max: TRANSIT_ADMISSION_MAX_PER_FEED, + globalMax: TRANSIT_ADMISSION_MAX_GLOBAL, + }); + const ttlSeconds = Math.ceil(TRANSIT_PROXY_TTL_MS / 1000); + + function send(res, status, body, headers) { + res.writeHead(status, headers); + res.end(body); + } + + function noteFailure(feedId, reason, now) { + const previous = cooldown.get(feedId); + const failures = (previous?.failures || 0) + 1; + const wait = nextTransitBackoffMs(failures); + cooldown.set(feedId, { failures, nextAttemptAt: now + wait, reason }); + return wait; + } + + async function refresh(feed, now) { + const controller = new AbortController(); + controllers.add(controller); + const timeout = setTimeout( + () => controller.abort(), + TRANSIT_PROXY_TIMEOUT_MS, + ); + const previous = cache.get(feed.id); + try { + const { response, finalUrl } = await fetchTransitFeed( + feed, + controller.signal, + fetchImpl, + previous + ? { etag: previous.etag, lastModified: previous.lastModified } + : null, + ); + if (closed) throw new Error('Transit provider closed'); + if (!isAcceptableTransitUpstreamUrl(finalUrl)) { + throw new Error('upstream is not https'); + } + // Nothing changed upstream: keep the snapshot we have and restart its + // freshness window rather than asking for a body we already hold. + if (response.status === 304 && previous) { + try { + await response.body?.cancel(); + } catch { + /* no-op */ + } + // `at` restarts the freshness window; `contactedAt` records that the + // operator answered just now. They are different facts: the body still + // carries the fetch time of the payload it holds, and reading that as + // "when did we last hear from them" would age a feed that is answering + // every single request into silence. + const revalidated = { ...previous, at: now, contactedAt: now }; + cache.set(feed.id, revalidated); + cooldown.delete(feed.id); + return revalidated; + } + if (!response.ok) { + const error = new Error(`upstream HTTP ${response.status}`); + error.upstreamStatus = response.status; + throw error; + } + const bytes = await readResponseBytesCapped( + response, + TRANSIT_PROXY_MAX_BODY_BYTES, + ); + const snapshot = buildTransitSnapshot(feed, bytes, now); + if (closed) throw new Error('Transit provider closed'); + history.ingest(feed, snapshot.vehicles, Date.now()); + const entry = { + at: snapshot.fetchedAt, + contactedAt: snapshot.fetchedAt, + body: JSON.stringify(snapshot), + host: new URL(finalUrl).hostname, + etag: response.headers.get('etag'), + lastModified: response.headers.get('last-modified'), + }; + cache.set(feed.id, entry); + cooldown.delete(feed.id); + return entry; + } finally { + clearTimeout(timeout); + controller.abort(); + controllers.delete(controller); + } + } + + function install(middlewares) { + middlewares.use('/api/transit', async (req, res) => { + if (req.method !== 'GET') { + send( + res, + 405, + JSON.stringify({ error: 'Method Not Allowed' }), + transitResponseHeaders('NONE'), + ); + return; + } + const route = resolveTransitRoute(req.url); + if (!route) { + send(res, 404, JSON.stringify({ error: 'Unknown transit feed' }), { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + }); + return; + } + if (route.route === 'feeds') { + send(res, 200, JSON.stringify({ feeds: publicTransitCatalog() }), { + 'Content-Type': 'application/json; charset=utf-8', + 'Cache-Control': 'public, max-age=3600', + }); + return; + } + const { feed } = route; + const now = Date.now(); + if (route.route === 'trail') { + if (!admitHistory(feed.id)) { + send( + res, + 429, + JSON.stringify({ error: 'Transit history read limit reached' }), + { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + 'Retry-After': '60', + }, + ); + return; + } + send( + res, + 200, + JSON.stringify(history.get(feed.id, route.vehicleId, now)), + { 'Content-Type': 'application/json', 'Cache-Control': 'no-store' }, + ); + return; + } + const cached = cache.get(feed.id); + const state = transitCacheState(cached, now); + if (state === 'fresh') { + send( + res, + 200, + cached.body, + transitResponseHeaders('HIT', cached.host, cached.contactedAt), + ); + return; + } + + // Answer without contacting the operator: the cached snapshot when there + // is one, otherwise an honest "unavailable, come back in N seconds". + const answerWithoutUpstream = (reason, retryAfterS) => { + if (state === 'stale' && cached) { + // The contact time stays as old as it is: a feed being served from + // cache during an outage has NOT answered, and saying otherwise + // would keep a dead feed on the globe. + send( + res, + 200, + cached.body, + transitResponseHeaders( + 'STALE-ERROR', + cached.host, + cached.contactedAt, + ), + ); + return; + } + send( + res, + 503, + JSON.stringify({ + error: 'Transit feed unavailable', + feedId: feed.id, + retryInSec: retryAfterS, + }), + { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + 'X-GEV-Cache': 'NONE', + 'Retry-After': String(retryAfterS), + 'X-Transit-Backoff': reason, + }, + ); + }; + + const waiting = cooldown.get(feed.id); + if (waiting && waiting.nextAttemptAt > now && !inFlight.has(feed.id)) { + answerWithoutUpstream( + 'cooldown', + Math.max(1, Math.ceil((waiting.nextAttemptAt - now) / 1000)), + ); + return; + } + if (!inFlight.has(feed.id) && !admit(feed.id)) { + answerWithoutUpstream('rate-limited', ttlSeconds); + return; + } + + const request = coalesceProxyRequest(inFlight, feed.id, () => + refresh(feed, now), + ); + try { + const fresh = await request.promise; + send( + res, + 200, + fresh.body, + transitResponseHeaders( + request.shared ? 'INFLIGHT' : 'MISS', + fresh.host, + fresh.contactedAt, + ), + ); + } catch (error) { + const differential = error instanceof TransitFeedShapeError; + // A shape fault condemns what we already hold. The cached copy was + // decoded under the assumption this feed is a full snapshot, and that + // assumption has just been shown false — leaving it in place would let + // the very next request serve it back as a 200. + if (differential) cache.delete(feed.id); + if (!request.shared) { + const wait = noteFailure( + feed.id, + error?.message || String(error), + Date.now(), + ); + console.warn( + `[transit-proxy] ${feed.id} unavailable: ${error?.message || error} — next attempt in ${Math.round(wait / 1000)}s`, + ); + } + // A differential feed is a shape fault, not an outage: serving the old + // snapshot would hide a feed this decoder must not render at all. + if (state === 'stale' && cached && !differential) { + // The contact time is the last SUCCESSFUL one. This request failed, + // so it does not advance — otherwise a dead feed would look like one + // answering on time. + send( + res, + 200, + cached.body, + transitResponseHeaders( + 'STALE-ERROR', + cached.host, + cached.contactedAt, + ), + ); + return; + } + const pending = cooldown.get(feed.id); + const retryAfterS = pending + ? Math.max(1, Math.ceil((pending.nextAttemptAt - Date.now()) / 1000)) + : ttlSeconds; + send( + res, + differential || + error?.code === 'RESPONSE_TOO_LARGE' || + Number.isInteger(error?.upstreamStatus) + ? 502 + : 504, + JSON.stringify({ + error: differential + ? 'Transit feed is differential and is not supported' + : 'Transit feed unavailable', + feedId: feed.id, + retryInSec: retryAfterS, + }), + { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + 'X-GEV-Cache': 'NONE', + 'Retry-After': String(retryAfterS), + }, + ); + } + }); + } + + function close() { + closed = true; + for (const controller of controllers) controller.abort(); + controllers.clear(); + history.clear(); + cache.clear(); + cooldown.clear(); + } + return { + closeBundle: close, + name: 'transit-proxy', + configureServer(server) { + install(server.middlewares); + server.httpServer?.once('close', close); + }, + configurePreviewServer(server) { + install(server.middlewares); + server.httpServer?.once('close', close); + }, + }; +} diff --git a/server/providers/transitHistory.js b/server/providers/transitHistory.js new file mode 100644 index 000000000..1cdb387e1 --- /dev/null +++ b/server/providers/transitHistory.js @@ -0,0 +1,493 @@ +import { validTransitIdentifier } from '../../src/data/transitProxy.js'; +export { validTransitIdentifier } from '../../src/data/transitProxy.js'; +/** Bounded, process-local operator position history. No upstream or scene access. */ +export const TRANSIT_HISTORY_LIMITS = Object.freeze({ + retentionMs: 900000, + fixes: 128, + feedBytes: 16 * 1024 * 1024, + processBytes: 64 * 1024 * 1024, + keys: 15000, + identifierBytes: 256, + responseBytes: 32768, +}); +const FIX_BYTES = 32, + CHUNK = 8, + EPOCH_SLOTS = 8, + EPOCH_BYTES = 544; +const META_BYTES = 64; +const encoder = new TextEncoder(), + decoder = new TextDecoder(); +// All stores in this process share the hard process allocation ceiling. +const processStorage = { bytes: 0, keys: 0, stores: new Set() }; + +function contextOf(record, mode) { + const trip = typeof record.tripId === 'string' ? record.tripId : ''; + const route = typeof record.routeId === 'string' ? record.routeId : ''; + if (encoder.encode(trip).length > 256 || encoder.encode(route).length > 256) + return null; + return { trip, route, mode: mode || 'unknown' }; +} +function contextAt(entry, slot) { + const offset = 0, + view = entry.contexts[slot]; + const bytes = new Uint8Array(view.buffer); + const length = view.getUint16(offset + 4), + routeLength = view.getUint16(offset + 6); + return { + id: view.getUint32(offset), + trip: decoder.decode(bytes.subarray(offset + 16, offset + 16 + length)), + route: decoder.decode( + bytes.subarray(offset + 272, offset + 272 + routeLength), + ), + mode: decoder.decode( + bytes.subarray(offset + 528, offset + 528 + view.getUint8(offset + 8)), + ), + }; +} +function storeContext(entry, context, id) { + const slot = (id - 1) % EPOCH_SLOTS, + offset = 0; + const trip = encoder.encode(context.trip), + route = encoder.encode(context.route), + mode = encoder.encode(context.mode).subarray(0, 16); + const view = entry.contexts[slot], + bytes = new Uint8Array(view.buffer); + view.setUint32(offset, id); + view.setUint16(offset + 4, trip.length); + view.setUint16(offset + 6, route.length); + view.setUint8(offset + 8, mode.length); + bytes.set(trip, offset + 16); + bytes.set(route, offset + 272); + bytes.set(mode, offset + 528); +} +function offset(entry, index) { + return ((entry.head + index) % entry.capacity) * FIX_BYTES; +} +function read(entry, index) { + const at = offset(entry, index), + v = entry.fixes; + return [ + v.getFloat64(at), + v.getFloat64(at + 8), + v.getFloat64(at + 16), + v.getUint32(at + 24), + v.getUint32(at + 28), + ]; +} +function firstTime(entry) { + return entry.count ? entry.fixes.getFloat64(offset(entry, 0)) : Infinity; +} +function drop(entry) { + entry.head = (entry.head + 1) % entry.capacity; + entry.count--; + entry.truncated = true; +} + +// Indexed eviction queues keep removal and receipt-order updates logarithmic. +function evictionQueue() { + const items = [], + positions = new Map(); + const less = (a, b) => + a.observedAt < b.observedAt || + (a.observedAt === b.observedAt && a.key < b.key); + function swap(a, b) { + [items[a], items[b]] = [items[b], items[a]]; + positions.set(items[a], a); + positions.set(items[b], b); + } + function repair(index) { + while (index > 0) { + const parent = (index - 1) >>> 1; + if (!less(items[index], items[parent])) break; + swap(index, parent); + index = parent; + } + for (;;) { + let child = index * 2 + 1; + if (child >= items.length) break; + if (child + 1 < items.length && less(items[child + 1], items[child])) + child++; + if (!less(items[child], items[index])) break; + swap(index, child); + index = child; + } + } + function remove(entry) { + const index = positions.get(entry); + if (index === undefined) return; + positions.delete(entry); + const last = items.pop(); + if (index < items.length) { + items[index] = last; + positions.set(last, index); + repair(index); + } + } + return { + remove, + add(entry) { + remove(entry); + positions.set(entry, items.length); + items.push(entry); + repair(items.length - 1); + }, + oldest(except) { + if (items[0] !== except) return items[0] || null; + if (!items[1]) return null; + return items[2] && less(items[2], items[1]) ? items[2] : items[1]; + }, + get size() { + return items.length; + }, + }; +} + +export function createTransitHistory({ + limits = {}, + storage = processStorage, + sweep = true, +} = {}) { + const policy = { ...TRANSIT_HISTORY_LIMITS }; + for (const [key, limit] of Object.entries(limits)) + if (key in policy && Number.isFinite(limit) && limit > 0) + policy[key] = Math.min(policy[key], Math.floor(limit)); + const entries = new Map(), + feedBytes = new Map(), + eviction = evictionQueue(), + feedEviction = new Map(); + storage.feedBytes ||= new Map(); + storage.lossUntil ||= new Map(); + let protectedEntries = null; + function queue(entry) { + if (!feedEviction.has(entry.feed)) + feedEviction.set(entry.feed, evictionQueue()); + eviction.add(entry); + feedEviction.get(entry.feed).add(entry); + } + function unqueue(entry) { + eviction.remove(entry); + const feedQueue = feedEviction.get(entry.feed); + feedQueue?.remove(entry); + if (feedQueue?.size === 0) feedEviction.delete(entry.feed); + } + function recordLoss(feed, until) { + storage.lossUntil.set( + feed, + Math.max(storage.lossUntil.get(feed) || 0, until), + ); + } + let prunedAt = -Infinity; + let allocatedBytes = 0, + timer = null, + closed = false; + function account(feed, delta) { + allocatedBytes += delta; + storage.bytes += delta; + storage.feedBytes.set(feed, (storage.feedBytes.get(feed) || 0) + delta); + if (storage.feedBytes.get(feed) === 0) storage.feedBytes.delete(feed); + feedBytes.set(feed, (feedBytes.get(feed) || 0) + delta); + if (feedBytes.get(feed) === 0) feedBytes.delete(feed); + } + function remove(entry) { + unqueue(entry); + entries.delete(entry.key); + storage.keys--; + account(entry.feed, -entry.bytes); + } + function canAllocate(feed, bytes) { + return ( + storage.bytes + bytes <= policy.processBytes && + (storage.feedBytes.get(feed) || 0) + bytes <= policy.feedBytes + ); + } + function resize(entry, capacity) { + const next = new Uint8Array(capacity * FIX_BYTES), + before = new Uint8Array(entry.fixes.buffer); + for (let i = 0; i < entry.count; i++) + next.set( + before.subarray(offset(entry, i), offset(entry, i) + FIX_BYTES), + i * FIX_BYTES, + ); + const delta = next.byteLength - entry.fixes.byteLength; + entry.fixes = new DataView(next.buffer); + entry.head = 0; + entry.capacity = capacity; + entry.bytes += delta; + account(entry.feed, delta); + } + function prune(now) { + if (now === prunedAt) return; + prunedAt = now; + for (const [feed, until] of storage.lossUntil) + if (until <= now) storage.lossUntil.delete(feed); + for (const entry of entries.values()) { + while (entry.count && firstTime(entry) < now - policy.retentionMs) + drop(entry); + if (!entry.count || now - entry.observedAt >= policy.retentionMs) + remove(entry); + else if ( + entry.capacity > CHUNK && + entry.count <= entry.capacity / 2 && + canAllocate( + entry.feed, + Math.max(CHUNK, Math.ceil(entry.count / CHUNK) * CHUNK) * FIX_BYTES, + ) + ) + resize(entry, Math.max(CHUNK, Math.ceil(entry.count / CHUNK) * CHUNK)); + } + } + function oldest(feed = null, except = null) { + return (feed ? feedEviction.get(feed) : eviction)?.oldest(except) || null; + } + function evictOne(feed = null, except = null) { + const entry = oldest(feed, except); + if (!entry) return false; + recordLoss(entry.feed, entry.observedAt + policy.retentionMs); + if (entry.count > CHUNK) { + const target = Math.max(CHUNK, entry.capacity - CHUNK); + if (!canAllocate(entry.feed, target * FIX_BYTES)) { + remove(entry); + return true; + } + while (entry.count > target) drop(entry); + resize(entry, target); + } else remove(entry); + return true; + } + const api = { + prune, + evictOne, + oldest, + get allocatedBytes() { + return allocatedBytes; + }, + }; + storage.stores.add(api); + function reserve(feed, bytes, except = null, newKey = false) { + while ((storage.feedBytes.get(feed) || 0) + bytes > policy.feedBytes) { + let owner = null, + candidate = null; + for (const store of storage.stores) { + const entry = store.oldest(feed, except); + if (entry && (!candidate || entry.observedAt < candidate.observedAt)) { + owner = store; + candidate = entry; + } + } + if (!owner || !owner.evictOne(feed, except)) return false; + } + while ( + storage.bytes + bytes > policy.processBytes || + (newKey && storage.keys >= policy.keys) + ) { + let owner = null, + candidate = null; + for (const store of storage.stores) { + const entry = store.oldest(null, except); + if (entry && (!candidate || entry.observedAt < candidate.observedAt)) { + owner = store; + candidate = entry; + } + } + if (!owner || !owner.evictOne(null, except)) return false; + } + return true; + } + function ingest(feed, records, now) { + if ( + closed || + feed.historyRetention !== true || + !validTransitIdentifier(feed.id) + ) + return; + prune(now); + // Pin all existing snapshot members before admitting any new key. A later + // row must not lose its retained path to an earlier row in the same feed. + protectedEntries = new Set(); + for (const record of records) { + const entry = entries.get(`${feed.id}\u0000${record.id}`); + if (entry) { + protectedEntries.add(entry); + unqueue(entry); + } + } + try { + for (const record of records) { + if ( + !validTransitIdentifier(record.id) || + !Number.isFinite(record.lat) || + Math.abs(record.lat) > 90 || + !Number.isFinite(record.lon) || + Math.abs(record.lon) > 180 + ) + continue; + const t = record.timestamp * 1000; + if ( + !Number.isFinite(t) || + t < now - policy.retentionMs || + t > now + 10000 + ) + continue; + const mode = + feed.routeMode?.(record.routeId) || feed.defaultMode || 'unknown'; + const context = contextOf(record, mode); + if (!context) continue; + const key = `${feed.id}\u0000${record.id}`; + let entry = entries.get(key); + if (entry) { + const last = read(entry, entry.count - 1); + if (t <= last[0]) continue; + } else { + const encodedKey = encoder.encode(key), + bytes = + META_BYTES + + EPOCH_BYTES + + encodedKey.byteLength + + CHUNK * FIX_BYTES; + if (!reserve(feed.id, bytes, null, true)) { + recordLoss(feed.id, now + policy.retentionMs); + continue; + } + entry = { + key, + encodedKey, + feed: feed.id, + vehicle: record.id, + metadata: new DataView(new ArrayBuffer(META_BYTES)), + contexts: [new DataView(new ArrayBuffer(EPOCH_BYTES))], + fixes: new DataView(new ArrayBuffer(CHUNK * FIX_BYTES)), + capacity: CHUNK, + head: 0, + count: 0, + epoch: 0, + observedAt: now, + truncated: (storage.lossUntil.get(feed.id) || 0) > now, + bytes, + }; + entries.set(key, entry); + protectedEntries.add(entry); + storage.keys++; + account(feed.id, bytes); + } + let flags = + record.timestampSource === 'vehicle' + ? 1 + : record.timestampSource === 'header' + ? 2 + : 4; + const previous = entry.epoch + ? contextAt(entry, (entry.epoch - 1) % EPOCH_SLOTS) + : null; + if ( + !previous || + previous.trip !== context.trip || + previous.route !== context.route || + previous.mode !== context.mode + ) { + const slot = entry.epoch % EPOCH_SLOTS; + if (!entry.contexts[slot]) { + if (!reserve(feed.id, EPOCH_BYTES, entry)) { + entry.truncated = true; + recordLoss(feed.id, now + policy.retentionMs); + continue; + } + entry.contexts[slot] = new DataView(new ArrayBuffer(EPOCH_BYTES)); + entry.bytes += EPOCH_BYTES; + account(feed.id, EPOCH_BYTES); + } + entry.epoch++; + // Expiring epoch metadata also expires its positions; never label a + // retained point with a reused context slot. + const expiredEpoch = entry.epoch - EPOCH_SLOTS; + while (entry.count && read(entry, 0)[4] <= expiredEpoch) drop(entry); + storeContext(entry, context, entry.epoch); + flags |= 16; + } + if (entry.count === entry.capacity) { + const next = Math.min(policy.fixes, entry.capacity + CHUNK); + if ( + next > entry.capacity && + reserve(feed.id, next * FIX_BYTES, entry) + ) + resize(entry, next); + else drop(entry); + } + const at = offset(entry, entry.count++), + v = entry.fixes; + v.setFloat64(at, t); + v.setFloat64(at + 8, record.lat); + v.setFloat64(at + 16, record.lon); + v.setUint32(at + 24, flags); + v.setUint32(at + 28, entry.epoch); + entry.observedAt = now; + } + } finally { + for (const entry of protectedEntries) queue(entry); + protectedEntries = null; + } + } + function get(feedId, vehicleId, now) { + prune(now); + const entry = entries.get(`${feedId}\u0000${vehicleId}`); + const fixes = [], + epochs = []; + if (entry) { + const used = new Set(); + for (let i = 0; i < entry.count; i++) { + const fix = read(entry, i); + fixes.push(fix); + used.add(fix[4]); + } + for (const id of used) + epochs.push(contextAt(entry, (id - 1) % EPOCH_SLOTS)); + } + const payload = { + version: 1, + feedId, + vehicleId, + oldestT: fixes[0]?.[0] ?? null, + newestT: fixes.at(-1)?.[0] ?? null, + truncated: + entry?.truncated || + (!entry && (storage.lossUntil.get(feedId) || 0) > now), + epochs, + fixes, + }; + // The normal bound is smaller; keep a byte guard for worst-case Unicode. + while ( + encoder.encode(JSON.stringify(payload)).length > policy.responseBytes && + payload.fixes.length + ) { + payload.fixes.shift(); + payload.oldestT = payload.fixes[0]?.[0] ?? null; + payload.truncated = true; + const used = new Set(payload.fixes.map((f) => f[4])); + payload.epochs = payload.epochs.filter((e) => used.has(e.id)); + } + return payload; + } + function clear() { + if (timer) clearInterval(timer); + timer = null; + closed = true; + for (const entry of entries.values()) remove(entry); + storage.stores.delete(api); + } + if (sweep) { + timer = setInterval(() => prune(Date.now()), 60000); + timer.unref?.(); + } + return { + ingest, + get, + prune, + clear, + diagnostics: () => ({ + allocatedBytes, + processBytes: storage.bytes, + keys: entries.size, + feedBytes: Object.fromEntries(feedBytes), + sweepActive: timer !== null, + }), + }; +} diff --git a/server/providers/transitHistory.test.mjs b/server/providers/transitHistory.test.mjs new file mode 100644 index 000000000..6ea9209f1 --- /dev/null +++ b/server/providers/transitHistory.test.mjs @@ -0,0 +1,179 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + createTransitHistory, + validTransitIdentifier, +} from './transitHistory.js'; +const feed = { id: 'mbta', historyRetention: true, defaultMode: 'bus' }; +const row = (t, id = 'bus', extra = {}) => ({ + id, + timestamp: t / 1000, + lat: 42, + lon: -71, + tripId: 'trip', + routeId: '1', + timestampSource: 'vehicle', + ...extra, +}); +const storage = () => ({ bytes: 0, keys: 0, stores: new Set() }); +function history(t, options = {}) { + const h = createTransitHistory({ + sweep: false, + storage: storage(), + ...options, + }); + t.after(() => h.clear()); + return h; +} +test('TTL ages reports and reads do not extend it; duplicate/conflicting times never append', (t) => { + const h = history(t); + h.ingest(feed, [row(100000)], 100000); + h.ingest(feed, [row(100000), row(100000, 'bus', { lat: 43 })], 500000); + assert.equal(h.get('mbta', 'bus', 500000).fixes.length, 1); + assert.equal(h.get('mbta', 'bus', 999999).fixes.length, 1); + assert.equal(h.get('mbta', 'bus', 1000000).fixes.length, 0); + assert.equal(h.diagnostics().allocatedBytes, 0); +}); +test('128-fix ring and bounded epoch contexts preserve breaks, never scene heights', (t) => { + const h = history(t); + for (let i = 0; i < 200; i++) + h.ingest( + feed, + [ + row(i * 1000, 'bus', { + tripId: i < 100 ? 'first' : 'second', + heightM: 999, + }), + ], + i * 1000, + ); + const result = h.get('mbta', 'bus', 199000); + assert.equal(result.fixes.length, 128); + assert.equal(result.truncated, true); + assert.equal(result.epochs.length, 2); + assert.ok(result.fixes.find((f) => f[0] === 100000)[3] & 16); + assert.ok(result.fixes.every((f) => f.length === 5)); + assert.ok(Buffer.byteLength(JSON.stringify(result)) < 32768); + for (let i = 200; i < 220; i++) + h.ingest(feed, [row(i * 1000, 'bus', { tripId: `trip-${i}` })], i * 1000); + const churn = h.get('mbta', 'bus', 219000); + assert.ok(churn.epochs.length <= 8); + assert.ok(churn.fixes.every((f) => churn.epochs.some((e) => e.id === f[4]))); +}); +test('allocated key, metadata and chunk bytes obey feed/process ceilings across stores', (t) => { + const shared = storage(), + limits = { feedBytes: 16000, processBytes: 24000, keys: 4 }; + const a = history(t, { storage: shared, limits }), + b = history(t, { storage: shared, limits }); + for (let i = 0; i < 60; i++) { + a.ingest(feed, [row(i * 1000, `a${i}`)], i * 1000); + b.ingest( + { ...feed, id: i % 2 ? 'mbta' : 'other' }, + [row(i * 1000, `b${i}`)], + i * 1000, + ); + assert.ok(shared.bytes <= limits.processBytes); + assert.ok(shared.keys <= limits.keys); + assert.ok( + [...shared.feedBytes.values()].every( + (bytes) => bytes <= limits.feedBytes, + ), + ); + assert.equal( + a.diagnostics().allocatedBytes + b.diagnostics().allocatedBytes, + shared.bytes, + ); + } + a.clear(); + b.clear(); + assert.equal(shared.bytes, 0); + assert.equal(shared.keys, 0); +}); +test('accelerated two-hour ingestion plateaus and expires idle keys', (t) => { + const h = history(t, { limits: { feedBytes: 128000, processBytes: 128000 } }); + const samples = []; + for (let now = 0; now <= 7200000; now += 15000) { + h.ingest( + feed, + Array.from({ length: 12 }, (_, i) => row(now, `v${i}`)), + now, + ); + if (now >= 900000) samples.push(h.diagnostics().allocatedBytes); + } + assert.ok(Math.max(...samples) <= 128000); + assert.equal(Math.max(...samples), Math.min(...samples)); + h.prune(8100001); + assert.equal(h.diagnostics().keys, 0); +}); +test('retention is opt-in and entire decoded identifiers are bounded', (t) => { + const h = history(t); + h.ingest({ ...feed, historyRetention: false }, [row(1000)], 1000); + assert.equal(h.diagnostics().keys, 0); + for (const id of [ + '', + '.', + '..', + 'abc/def', + 'abc\\def', + 'abc\0def', + 'é'.repeat(129), + '\ud800', + ]) { + assert.equal(validTransitIdentifier(id), false); + h.ingest(feed, [row(1000, id)], 1000); + } + assert.equal(h.diagnostics().keys, 0); + assert.equal(validTransitIdentifier('🚋 vehicle 1'), true); +}); +test('unref sweep is owned and clear closes the cache', (t) => { + const h = history(t, { sweep: true }); + assert.equal(h.diagnostics().sweepActive, true); + h.clear(); + h.ingest(feed, [row(1000)], 1000); + assert.equal(h.diagnostics().sweepActive, false); + assert.equal(h.diagnostics().keys, 0); +}); + +test('three identical-order 15000-vehicle snapshots retain paths within a bounded ingestion time', (t) => { + const h = history(t); + const times = []; + for (let poll = 0; poll < 3; poll++) { + const now = 100000 + poll * 15000; + const rows = Array.from({ length: 15000 }, (_, i) => + row(now, `vehicle-${i}`), + ); + const start = performance.now(); + h.ingest(feed, rows, now); + times.push(performance.now() - start); + } + assert.equal(h.diagnostics().keys, 15000); + for (let i = 0; i < 15000; i++) { + const result = h.get('mbta', `vehicle-${i}`, 130000); + assert.equal(result.fixes.length, 3); + assert.equal(result.truncated, false); + } + assert.ok(h.diagnostics().allocatedBytes < 16 * 1024 * 1024); + t.diagnostic( + `15000-vehicle ingestion ms: ${times.map((n) => n.toFixed(1)).join('/')}`, + ); + // A ceiling that says "not quadratic", not a benchmark: 15,000 rows in + // well under two seconds on a loaded laptop or a slow CI runner. The + // per-run figure above is the diagnostic to read. + assert.ok(Math.max(...times) < 2000, `ingestion ms: ${times}`); +}); + +test('snapshot capacity pressure preserves members not yet visited and discloses recreated history', (t) => { + const h = history(t, { limits: { keys: 2 } }); + h.ingest(feed, [row(1000, 'a'), row(1000, 'b')], 1000); + // Newcomers precede the retained members in every snapshot. + for (const now of [2000, 3000]) + h.ingest(feed, [row(now, 'c'), row(now, 'a'), row(now, 'b')], now); + assert.equal(h.get('mbta', 'a', 3000).fixes.length, 3); + assert.equal(h.get('mbta', 'b', 3000).fixes.length, 3); + assert.equal(h.get('mbta', 'c', 3000).truncated, true); + h.ingest(feed, [row(4000, 'c')], 4000); + h.ingest(feed, [row(5000, 'a')], 5000); + const recreated = h.get('mbta', 'a', 5000); + assert.equal(recreated.fixes.length, 1); + assert.equal(recreated.truncated, true); +}); diff --git a/src/app/catalog.js b/src/app/catalog.js index aa75899d3..0d22ec9a1 100644 --- a/src/app/catalog.js +++ b/src/app/catalog.js @@ -6,6 +6,7 @@ const CONTROL_LAYER_IDS = Object.freeze({ cctvLayer: 'cctv', radioLayer: 'radio', bikeshareLayer: 'bikeshare', + transitLayer: 'transit', aisLiveVesselsLayer: 'ais-live-vessels', militaryAwarenessLayer: 'military-awareness', militaryInstallationsLayer: 'military-installations', diff --git a/src/app/catalog.test.mjs b/src/app/catalog.test.mjs index 5cee846ba..820943049 100644 --- a/src/app/catalog.test.mjs +++ b/src/app/catalog.test.mjs @@ -88,6 +88,7 @@ test('controls bind catalog instances rather than similarly named defaults', () 'cctv', 'radio', 'bikeshare', + 'transit', 'ais-live-vessels', 'military-awareness', 'military-installations', @@ -97,7 +98,8 @@ test('controls bind catalog instances rather than similarly named defaults', () const catalog = createLayerCatalog(layers, metadata(layers)); const services = catalogControlServices(catalog); assert.equal(services.flightsLayer, layers[1]); - assert.equal(services.aisLiveVesselsLayer, layers[7]); + assert.equal(services.transitLayer, layers[7]); + assert.equal(services.aisLiveVesselsLayer, layers[8]); assert.equal(new Set(Object.values(services)).size, layers.length); assert.throws( () => catalogControlServices(createLayerCatalog([], [])), diff --git a/src/app/constructCatalog.js b/src/app/constructCatalog.js index 18d872209..64a4c9843 100644 --- a/src/app/constructCatalog.js +++ b/src/app/constructCatalog.js @@ -9,6 +9,7 @@ import { createApplicationRadio } from './layers/radio.js'; import { createApplicationTraffic } from './layers/traffic.js'; import { createApplicationBikeshare } from './layers/bikeshare.js'; import { createApplicationDirections } from './layers/directions.js'; +import { createApplicationTransit } from './layers/transit.js'; import { createApplicationInstallations } from './layers/militaryInstallations.js'; import { createApplicationSatellites } from './layers/satellites.js'; import { createApplicationLaunches } from './layers/rocketLaunches.js'; @@ -111,6 +112,7 @@ export function createApplicationCatalog({ createApplicationRadio({ surface, source: sources.radio }), createApplicationBikeshare({ source: sources.bikeshare }), createApplicationDirections(), + createApplicationTransit({ surface }), vessels, installations, createApplicationAwareness({ diff --git a/src/app/constructCatalog.test.mjs b/src/app/constructCatalog.test.mjs index d25ad45ee..ba434ef61 100644 --- a/src/app/constructCatalog.test.mjs +++ b/src/app/constructCatalog.test.mjs @@ -38,7 +38,7 @@ test('catalogs construct distinct layers and classification from their supplied signal: b.signal, surface: fixtureSurface(b.signal), }); - assert.equal(first.layers.length, 18); + assert.equal(first.layers.length, 19); assert.deepEqual( first.layers.map(({ id }) => id), second.layers.map(({ id }) => id), diff --git a/src/app/layers/transit.js b/src/app/layers/transit.js new file mode 100644 index 000000000..34724d6f0 --- /dev/null +++ b/src/app/layers/transit.js @@ -0,0 +1,48 @@ +import { createTransitLayer } from '../../layers/transit/index.js'; +import * as render from '../../renderGovernor.js'; +import * as sprites from '../../data/spriteOrder.js'; +import * as picking from '../../data/pickRegistry.js'; +import * as overlays from '../../overlays/worldOverlay.js'; +import { + registerDynamicCredit, + transitFeedCredit, +} from '../../data/dataCredits.js'; + +/** + * Construct one layer using the application scene owners and the + * application's surface services. + * + * The floor comes from `surface.groundFloor` — the ONE instance aircraft, + * trails, fires and camera geometry stand on — and the rendered-mesh sampler + * from `surface.meshFloor`, so a mesh floor the application resolves for a + * cell is the floor transit draws on, and a transit-only session warms its + * own mesh floors. An earlier cut imported the legacy module-level singleton + * instead and could not see either. + * + * @param {{surface: object}} options + * @returns {object} + */ +export function createApplicationTransit({ surface }) { + if (!surface?.groundFloor || !surface?.meshFloor) + throw new TypeError('Transit needs the application surface services'); + const { groundFloor, meshFloor } = surface; + return createTransitLayer({ + services: { + render, + sprites, + picking, + overlays, + credits: { registerDynamicCredit, transitFeedCredit }, + ground: { + GROUND_FLOOR_LIFT_M: groundFloor.GROUND_FLOOR_LIFT_M, + cachedGroundFloor: groundFloor.cachedGroundFloor, + coarseFloorCoord: groundFloor.coarseFloorCoord, + neighborFloorM: groundFloor.neighborFloorM, + warmGroundFloor: groundFloor.warmGroundFloor, + corridorFloorCells: groundFloor.corridorFloorCells, + allocateCorridorCells: groundFloor.allocateCorridorCells, + }, + mesh: { sampleMeshFloorCells: meshFloor.sampleMeshFloorCells }, + }, + }); +} diff --git a/src/app/layers/transit.test.mjs b/src/app/layers/transit.test.mjs new file mode 100644 index 000000000..3f585ded5 --- /dev/null +++ b/src/app/layers/transit.test.mjs @@ -0,0 +1,93 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createSurfaceServices } from '../surfaceServices.js'; +import { createApplicationTransit } from './transit.js'; +import { createApplicationCatalog } from '../constructCatalog.js'; +import { createStandaloneLayerSources } from '../../standalone/layerSources.js'; + +/** A terrain source that answers every point with one ellipsoidal height. */ +function fixtureSurface(signal, demM) { + return createSurfaceServices({ + terrainSource: { + getHeights: async (chunk) => chunk.map(() => ({ ellipsoid: demM })), + }, + signal, + eventTarget: null, + }); +} + +test('transit reads the mesh floor the application resolved, through the application surface', async (t) => { + // The floor transit draws on is the ONE the application constructs — the + // instance aircraft and camera geometry stand on — not a module-level copy. + // An earlier cut imported the legacy singleton and could not see a mesh + // floor the app had already resolved for a cell; it drew on bare earth. + const lifetime = new AbortController(); + t.after(() => lifetime.abort()); + const surface = fixtureSurface(lifetime.signal, 20); + const layer = createApplicationTransit({ surface }); + const height = layer._transitPartsForTest().height; + const lat = 42.3601; + const lon = -71.0589; + + // Nothing warm yet: the layer has no floor and no prior to borrow. + assert.equal(height.requestHeight(lat, lon).height, null); + + // The application resolves the DEM for that cell... + await surface.groundFloor.resolveGroundFloorCells([{ lat, lon }]); + const lift = surface.groundFloor.GROUND_FLOOR_LIFT_M; + assert.equal( + height.requestHeight(lat, lon).height, + 20 + lift, + 'the DEM floor, lifted', + ); + + // ...and then a rendered-mesh sample lands on the same cell, seventeen + // metres above bare earth, the way a photogrammetric street sits. + assert.equal( + surface.groundFloor.reportValidatedMeshFloorCell(lat, lon, 37), + true, + ); + assert.equal( + surface.groundFloor.cachedGroundFloor(lat, lon), + 37, + 'the app now floors on the mesh', + ); + assert.equal( + height.requestHeight(lat, lon).height, + 37 + lift, + 'and so does transit — the same number', + ); + + // Leaving the photoreal regime drops back to the DEM for everyone at once. + surface.groundFloor.setMeshFloorPreferred(false); + assert.equal(height.requestHeight(lat, lon).height, 20 + lift); +}); + +test('the catalog hands transit the same surface it hands everything else', async (t) => { + const lifetime = new AbortController(); + t.after(() => lifetime.abort()); + const surface = fixtureSurface(lifetime.signal, 5); + const catalog = createApplicationCatalog({ + sources: createStandaloneLayerSources(), + signal: lifetime.signal, + surface, + }); + const transit = catalog.get('transit'); + assert.ok(transit, 'transit is in the catalog'); + const lat = 30.2672; + const lon = -97.7431; + await surface.groundFloor.resolveGroundFloorCells([{ lat, lon }]); + surface.groundFloor.reportValidatedMeshFloorCell(lat, lon, 22); + assert.equal( + transit._transitPartsForTest().height.requestHeight(lat, lon).height, + 22 + surface.groundFloor.GROUND_FLOOR_LIFT_M, + ); +}); + +test('transit refuses to build without the application surface', () => { + assert.throws(() => createApplicationTransit({}), /surface/); + assert.throws( + () => createApplicationTransit({ surface: { groundFloor: {} } }), + /surface/, + ); +}); diff --git a/src/data/cameraSensitivity.js b/src/data/cameraSensitivity.js new file mode 100644 index 000000000..87933dc9e --- /dev/null +++ b/src/data/cameraSensitivity.js @@ -0,0 +1,93 @@ +// src/data/cameraSensitivity.js +/** + * Shared claims on `camera.percentageChanged`. + * + * `camera.changed` fires once the view has moved by this fraction, and it is a + * single number on the camera that every listener in the app shares. Several + * layers want it finer than the default — traffic, bikeshare and transit all + * ask for 0.05 so they notice they have entered a coverage area — and each of + * them used to write the number directly and hand back whatever it remembered. + * + * That cannot work, because a number is not an owner. Two layers asking for the + * same 0.05 write the same value, so when the first one leaves it sees its own + * number still in place, concludes nothing else has claimed it, and restores + * the coarse default out from under a layer that is still running. The symptom + * is a camera-driven layer that quietly stops noticing the camera: Transit is + * turned off and Bikeshare stops loading new cities until something else nudges + * it. + * + * So ownership is tracked explicitly. Claims are named, the camera gets the + * finest value anyone has asked for, and the original value returns only when + * the last claim is gone. Claiming twice under one name replaces that claim. + * + * This is bookkeeping, not policy: it never decides what a layer should ask + * for, only that nobody's answer is taken away while they still need it. + */ + +/** @type {WeakMap}>} */ +const _cameras = new WeakMap(); + +/** The value Cesium starts a camera at, used when the camera reports nothing. */ +const DEFAULT_PERCENTAGE_CHANGED = 0.5; + +function ledgerFor(camera) { + let ledger = _cameras.get(camera); + if (!ledger) { + const base = Number.isFinite(camera.percentageChanged) + ? camera.percentageChanged + : DEFAULT_PERCENTAGE_CHANGED; + ledger = { base, claims: new Map() }; + _cameras.set(camera, ledger); + } + return ledger; +} + +function applyLedger(camera, ledger) { + let value = ledger.base; + for (const claimed of ledger.claims.values()) { + if (claimed < value) value = claimed; + } + camera.percentageChanged = value; + return value; +} + +/** + * Ask for a camera-change sensitivity at least as fine as `value`. + * @param {object|null|undefined} camera A Cesium camera (or anything with `percentageChanged`). + * @param {string} owner Stable claim name, usually the layer id. + * @param {number} value Requested `percentageChanged`. + * @returns {number|null} The value now in force, or null when there is no camera. + */ +export function claimCameraSensitivity(camera, owner, value) { + if (!camera || !owner || !Number.isFinite(value)) return null; + const ledger = ledgerFor(camera); + ledger.claims.set(owner, value); + return applyLedger(camera, ledger); +} + +/** + * Give up one claim. The camera keeps the finest value still claimed by + * somebody else, and returns to its original only when none are left. + * @param {object|null|undefined} camera + * @param {string} owner + * @returns {number|null} The value now in force, or null when there is no camera. + */ +export function releaseCameraSensitivity(camera, owner) { + if (!camera || !owner) return null; + const ledger = _cameras.get(camera); + if (!ledger) return null; + if (!ledger.claims.delete(owner)) return camera.percentageChanged; + const applied = applyLedger(camera, ledger); + if (ledger.claims.size === 0) _cameras.delete(camera); + return applied; +} + +/** + * Who currently holds a claim. For tests and diagnostics only. + * @param {object|null|undefined} camera + * @returns {Array} + */ +export function cameraSensitivityClaims(camera) { + const ledger = camera ? _cameras.get(camera) : null; + return ledger ? [...ledger.claims.keys()] : []; +} diff --git a/src/data/contactPlayback.js b/src/data/contactPlayback.js new file mode 100644 index 000000000..2aa1c9372 --- /dev/null +++ b/src/data/contactPlayback.js @@ -0,0 +1,691 @@ +/** Portable retained report-time playback. No scene or scheduling ownership. */ +export const FIX_BYTES = 48; +export const FIX_FLAGS = Object.freeze({ + VEHICLE_TIME: 1, + FEED_TIME: 2, + RECEIPT_TIME: 4, + FLOOR: 8, + BREAK: 16, + LIVE: 32, +}); +const clampDelay = (n) => Math.max(25_000, Math.min(120_000, n)); + +export function createLagReservoir(capacity = 256) { + return { values: new Float64Array(capacity), count: 0, cursor: 0 }; +} +function addLag(reservoir, value) { + if (!reservoir || !Number.isFinite(value) || value < 0) return; + reservoir.values[reservoir.cursor] = value; + reservoir.cursor = (reservoir.cursor + 1) % reservoir.values.length; + reservoir.count = Math.min(reservoir.count + 1, reservoir.values.length); +} +function percentile(reservoir, rank) { + if (!reservoir?.count) return 0; + const sorted = reservoir.values.slice(0, reservoir.count).sort(); + return sorted[Math.ceil(sorted.length * rank) - 1]; +} + +/** Optional shared payload budget. Call trimTrack on lower-priority tracks first. */ +export function createHistoryBudget(maxBytes = 32 * 1024 * 1024) { + return { maxBytes, allocatedBytes: 0 }; +} +/** Numeric fields are reserved up front, including unknown mesh heights. */ +export function createFixSample() { + return { + t: NaN, + lat: NaN, + lon: NaN, + heightM: NaN, + h: NaN, + receivedAt: NaN, + bearingDeg: NaN, + flags: 0, + epoch: 0, + seq: -1, + }; +} +export function createTrack({ + capacity = 128, + retentionMs = 900_000, + policy = {}, + budget = null, +} = {}) { + capacity = Math.max(2, Math.min(128, Math.floor(capacity))); + return { + capacity, + retentionMs, + policy, + budget, + metricsScratchA: createFixSample(), + metricsScratchB: createFixSample(), + storage: new DataView(new ArrayBuffer(0)), + slots: 0, + head: 0, + count: 0, + baseSeq: 0, + revision: 0, + truncated: false, + epochs: new Map(), + lag: createLagReservoir(32), + targetDelayMs: 25_000, + displayT: NaN, + wallAnchor: NaN, + monoAnchor: NaN, + lastMono: NaN, + offset: 0, + rate: 1, + phase: 'buffering', + resets: 0, + resetReason: null, + acceptedAt: NaN, + acceptedMono: NaN, + resumePending: false, + bracket: 0, + metricsRevision: -1, + metricsIndex: -1, + from: createFixSample(), + to: createFixSample(), + metrics: { speed: NaN, course: NaN, nextMotionT: Infinity }, + latest: createFixSample(), + pending: null, + pendingCount: 0, + }; +} +function address(track, index) { + return ((track.head + index) % track.slots) * FIX_BYTES; +} +export function readFix(track, index, out = {}) { + if (index < 0 || index >= track.count) return null; + const v = track.storage, + o = address(track, index); + out.t = v.getFloat64(o); + out.lat = v.getFloat64(o + 8); + out.lon = v.getFloat64(o + 16); + out.heightM = v.getFloat32(o + 24); + out.h = out.heightM; + out.receivedAt = v.getFloat64(o + 28); + out.bearingDeg = v.getFloat32(o + 36); + out.flags = v.getUint32(o + 40); + out.epoch = v.getUint32(o + 44); + out.seq = track.baseSeq + index; + return out; +} +function writeFix(track, index, fix) { + const v = track.storage, + o = address(track, index); + v.setFloat64(o, fix.t); + v.setFloat64(o + 8, fix.lat); + v.setFloat64(o + 16, fix.lon); + v.setFloat32(o + 24, fix.heightM ?? fix.h ?? NaN); + v.setFloat64(o + 28, fix.receivedAt ?? NaN); + v.setFloat32(o + 36, fix.bearingDeg ?? NaN); + v.setUint32(o + 40, fix.flags || 0); + v.setUint32(o + 44, fix.epoch || 0); +} +function resize(track, slots) { + const bytes = slots * FIX_BYTES, + delta = bytes - track.storage.byteLength; + if ( + delta > 0 && + track.budget && + track.budget.allocatedBytes + delta > track.budget.maxBytes + ) + return false; + const old = track.storage, + head = track.head, + oldSlots = track.slots; + const next = new Uint8Array(bytes), + previous = new Uint8Array(old.buffer); + for (let i = 0; i < track.count; i++) { + const o = ((head + i) % oldSlots) * FIX_BYTES; + next.set(previous.subarray(o, o + FIX_BYTES), i * FIX_BYTES); + } + track.storage = new DataView(next.buffer); + track.slots = slots; + track.head = 0; + if (track.budget) track.budget.allocatedBytes += delta; + return true; +} +export function destroyTrack(track) { + if (track.budget) track.budget.allocatedBytes -= track.storage.byteLength; + track.storage = new DataView(new ArrayBuffer(0)); + track.count = 0; + track.slots = 0; + track.epochs.clear(); +} +function timeAt(track, i) { + return track.storage.getFloat64(address(track, i)); +} +function dropOldest(track) { + track.head = (track.head + 1) % track.slots; + track.count--; + track.baseSeq++; + track.revision++; + track.truncated = true; +} +/** Retention never evicts the bracket currently on screen. */ +export function trimTrack(track, nowMs, keep = track.capacity) { + while ( + track.count > 2 && + (track.count > keep || timeAt(track, 1) < nowMs - track.retentionMs) + ) { + if (Number.isFinite(track.displayT) && timeAt(track, 1) > track.displayT) + break; + dropOldest(track); + } + if (keep < track.slots && track.count <= keep) + resize(track, Math.max(2, track.count)); + const used = new Set(); + for (let i = 0; i < track.count; i++) + used.add(track.storage.getUint32(address(track, i) + 44)); + for (const id of track.epochs.keys()) + if (!used.has(id)) track.epochs.delete(id); +} +export function validFix(fix) { + return ( + Number.isFinite(fix?.t) && + fix.t >= 0 && + Number.isFinite(fix.lat) && + Math.abs(fix.lat) <= 90 && + Number.isFinite(fix.lon) && + Math.abs(fix.lon) <= 180 + ); +} +function distance(a, b) { + const lon = ((b.lon - a.lon + 540) % 360) - 180; + return Math.hypot( + (b.lat - a.lat) * 111320, + lon * 111320 * Math.cos((a.lat * Math.PI) / 180), + ); +} +function append(track, fix) { + if (track.count === track.slots && track.slots < track.capacity) + resize(track, Math.min(track.capacity, Math.max(8, track.slots * 2))); + if (track.count === track.slots) { + if (track.count < 2) return false; + if (Number.isFinite(track.displayT) && timeAt(track, 1) > track.displayT) { + // Pin both active ends; drop the next future observation and mark the gap. + if (track.count < 4) return false; + const scratch = {}; + for (let i = 2; i < track.count - 1; i++) { + readFix(track, i + 1, scratch); + if (i === 2) scratch.flags |= FIX_FLAGS.BREAK; + writeFix(track, i, scratch); + } + track.count--; + track.truncated = true; + } else dropOldest(track); + } + writeFix(track, track.count++, fix); + track.revision++; + return true; +} + +export function insertFix(track, fix, observation = {}) { + if ( + !validFix(fix) || + (Number.isFinite(observation.wallNowMs) && + fix.t > observation.wallNowMs + 10_000) + ) + return { accepted: false, reason: 'malformed' }; + const last = readFix(track, track.count - 1, track.latest); + if (last && fix.t === last.t) + return { + accepted: false, + reason: + fix.lat === last.lat && fix.lon === last.lon ? 'repeat' : 'conflict', + }; + let reason = last ? 'moved' : 'first'; + if ( + last && + (fix.t < last.t || (track.policy.accept && !track.policy.accept(last, fix))) + ) { + reason = fix.t < last.t ? 'out-of-order' : 'implausible'; + const prior = track.pending; + if ( + !prior || + prior.reason !== reason || + fix.t <= prior.t || + (track.policy.accept && !track.policy.accept(prior, fix)) + ) { + if (!prior || fix.t !== prior.t) { + track.pending = { ...fix, reason }; + track.pendingCount = 1; + } + return { accepted: false, reason }; + } + track.pendingCount++; + const required = reason === 'out-of-order' ? 3 : 2; + if (track.pendingCount < required) { + track.pending = { ...fix, reason }; + return { accepted: false, reason }; + } + // Clock correction starts a new time domain. A spatial replacement retains a visible gap. + if (reason === 'out-of-order') { + track.count = 0; + track.head = 0; + track.baseSeq += 128; + } + const epoch = (last.epoch + 1) >>> 0; + append(track, { + ...prior, + epoch, + flags: (prior.flags || 0) | FIX_FLAGS.BREAK, + }); + fix = { ...fix, epoch }; + if (reason === 'out-of-order') track.displayT = NaN; + track.resetReason = reason; + track.resets++; + } + const receivedAt = + observation.receivedAt ?? fix.receivedAt ?? observation.wallNowMs ?? fix.t; + trimTrack(track, receivedAt); + const accepted = append(track, { + ...fix, + receivedAt, + flags: (fix.flags || 0) | FIX_FLAGS.LIVE, + }); + if (accepted) { + if (!observation.replayed) { + if (last && fix.t > last.t && reason === 'moved') { + addLag(track.lag, receivedAt - last.t); + addLag(track.policy.feedLag, receivedAt - last.t); + track.targetDelayMs = clampDelay( + Math.max( + percentile(track.lag, 0.95), + percentile(track.policy.feedLag, 0.9), + ) + 5000, + ); + } else if (!last) + track.targetDelayMs = clampDelay( + Math.max(0, receivedAt - fix.t) + 20_000, + ); + } + track.acceptedAt = receivedAt; + track.acceptedMono = observation.monoNowMs ?? NaN; + track.resumePending = true; + track.pending = null; + track.pendingCount = 0; + } + return { accepted, reason: accepted ? reason : 'capacity' }; +} + +/** Bounded backfill; live positions win conflicts and the clock/health are untouched. */ +export function mergeHistory(track, fixes) { + const merged = new Map(); + for (const fix of fixes.slice(-128)) + if ( + validFix(fix) && + (!Number.isFinite(track.acceptedAt) || + fix.t >= track.acceptedAt - track.retentionMs) + ) + merged.set(fix.t, { ...fix, flags: (fix.flags || 0) & ~FIX_FLAGS.LIVE }); + for (let i = 0; i < track.count; i++) { + const fix = readFix(track, i); + merged.set(fix.t, fix); + } + const retained = Array.from({ length: track.count }, (_, i) => + readFix(track, i), + ); + const active = Number.isFinite(track.displayT) + ? retained.findLastIndex((f) => f.t <= track.displayT) + : -1; + const pinnedFrom = retained[active], + pinnedTo = retained[active + 1]; + const trusted = new Set(retained.map((f) => f.t)); + const candidates = [...merged.values()] + .sort((a, b) => a.t - b.t) + .filter( + (f) => + trusted.has(f.t) || + !pinnedFrom || + !pinnedTo || + f.t < pinnedFrom.t || + f.t > pinnedTo.t, + ); + const ordered = []; + // Start from an admitted live position, including when walking backwards + // into retained history. A lone outlier is quarantined in either direction. + // Never borrow the live pending observation as a second vote for itself. + function admit(sequence, anchor, backwards = false) { + let last = anchor, + pending = null; + const plausible = (a, b) => + !track.policy.accept || + (backwards ? track.policy.accept(b, a) : track.policy.accept(a, b)); + for (const fix of sequence) { + if (trusted.has(fix.t) || !last || plausible(last, fix)) { + ordered.push(fix); + last = fix; + pending = null; + } else if (pending && plausible(pending, fix)) { + // Two distinct coherent observations establish a replacement segment. + // Keep a visible gap between it and the authoritative live path. + if (backwards) last.flags |= FIX_FLAGS.BREAK; + else pending.flags |= FIX_FLAGS.BREAK; + ordered.push(pending, fix); + last = fix; + pending = null; + } else pending = fix; + } + } + const first = retained[0]; + if (first) { + admit( + candidates.filter((f) => f.t < first.t).reverse(), + merged.get(first.t), + true, + ); + admit( + candidates.filter((f) => f.t >= first.t), + null, + ); + } else admit(candidates, null); + ordered.sort((a, b) => a.t - b.t); + while (ordered.length > track.capacity) { + const i = ordered.findIndex( + (f) => f.t !== pinnedFrom?.t && f.t !== pinnedTo?.t, + ); + if (i > 0 && i + 1 < ordered.length) + ordered[i + 1].flags |= FIX_FLAGS.BREAK; + ordered.splice(Math.max(0, i), 1); + track.truncated = true; + } + for (let i = 1; i < ordered.length; i++) { + if (track.policy.accept && !track.policy.accept(ordered[i - 1], ordered[i])) + ordered[i].flags |= FIX_FLAGS.BREAK; + } + track.count = 0; + track.head = 0; + track.baseSeq += 128; + for (const fix of ordered) append(track, fix); + track.revision++; + track.bracket = 0; +} +export function correctHeight(track, seq, heightM) { + const i = seq - track.baseSeq; + if (i < 0 || i >= track.count || !Number.isFinite(heightM)) return false; + const o = address(track, i), + value = Math.fround(heightM); + if (track.storage.getFloat32(o + 24) === value) return false; + track.storage.setFloat32(o + 24, value); + track.storage.setUint32( + o + 40, + track.storage.getUint32(o + 40) | FIX_FLAGS.FLOOR, + ); + track.revision++; + return true; +} +function anchoredWall(track, clocks) { + if (!Number.isFinite(track.wallAnchor)) { + track.wallAnchor = clocks.wallNowMs; + track.monoAnchor = clocks.monoNowMs; + } + return track.wallAnchor + Math.max(0, clocks.monoNowMs - track.monoAnchor); +} +/** Stable numeric fields avoid dictionary properties and boxed doubles in hot samples. */ +export function createSample() { + return { + displayT: NaN, + revision: 0, + nextWakeMonoMs: Infinity, + phase: 'buffering', + motion: 'unknown', + fromSeq: -1, + toSeq: -1, + lat: NaN, + lon: NaN, + heightM: NaN, + segmentCourseDeg: NaN, + segmentSpeedMps: NaN, + fraction: NaN, + fromT: NaN, + toT: NaN, + nextMotionT: Infinity, + latestReportAgeMs: NaN, + actualDelayMs: NaN, + }; +} +/** Sample an explicit report time into caller-owned storage. */ +export function sampleAt(track, displayTimeMs, out) { + out.displayT = displayTimeMs; + return sampleCurrent(track, out); +} +/** The frame path keeps its changing double in storage across this call. */ +function sampleCurrent(track, out) { + const sampleTime = out.displayT; + out.revision = track.revision; + out.nextWakeMonoMs = Infinity; + out.phase = track.phase; + out.motion = 'unknown'; + out.fromSeq = -1; + out.toSeq = -1; + out.lat = NaN; + out.lon = NaN; + out.heightM = NaN; + out.segmentCourseDeg = NaN; + out.segmentSpeedMps = 0; + if (!track.count) return out; + let i = Math.min(track.bracket, track.count - 1); + if (timeAt(track, i) > sampleTime) i = 0; + while (i + 1 < track.count && timeAt(track, i + 1) <= sampleTime) i++; + track.bracket = i; + const a = readFix(track, i, track.from); + const j = sampleTime < a.t ? i : Math.min(i + 1, track.count - 1); + const b = readFix(track, j, track.to); + const continuous = + i !== j && a.epoch === b.epoch && !(b.flags & FIX_FLAGS.BREAK); + out.fromSeq = a.seq; + out.toSeq = continuous ? b.seq : a.seq; + out.fraction = continuous + ? Math.max(0, Math.min(1, (sampleTime - a.t) / (b.t - a.t))) + : 1; + out.fromT = a.t; + out.toT = continuous ? b.t : a.t; + if ( + track.metricsRevision !== track.revision || + track.metricsIndex !== i || + track.metricsTo !== j + ) { + const metres = continuous ? distance(a, b) : 0; + const metrics = track.metrics; + metrics.speed = continuous ? (metres * 1000) / (b.t - a.t) : 0; + metrics.course = NaN; + if (metres >= (track.policy.courseMinTravelM ?? 12)) { + metrics.course = + ((Math.atan2( + (((b.lon - a.lon + 540) % 360) - 180) * + Math.cos((a.lat * Math.PI) / 180), + b.lat - a.lat, + ) * + 180) / + Math.PI + + 360) % + 360; + } else { + // Tiny displacement holds the last reliable course in this epoch. + // With no reliable displacement yet, use the earliest known bearing. + const p = track.metricsScratchA, + q = track.metricsScratchB; + for (let k = i; k >= 0; k--) { + readFix(track, k, q); + if (q.epoch !== a.epoch) break; + if (Number.isFinite(q.bearingDeg)) metrics.course = q.bearingDeg; + if (k === 0 || q.flags & FIX_FLAGS.BREAK) continue; + readFix(track, k - 1, p); + if ( + p.epoch === q.epoch && + distance(p, q) >= (track.policy.courseMinTravelM ?? 12) + ) { + metrics.course = + ((Math.atan2( + (((q.lon - p.lon + 540) % 360) - 180) * + Math.cos((p.lat * Math.PI) / 180), + q.lat - p.lat, + ) * + 180) / + Math.PI + + 360) % + 360; + break; + } + } + } + metrics.nextMotionT = Infinity; + const p = track.metricsScratchA, + q = track.metricsScratchB; + for (let k = i; k + 1 < track.count; k++) { + readFix(track, k, p); + readFix(track, k + 1, q); + if ( + p.lat !== q.lat || + p.lon !== q.lon || + p.epoch !== q.epoch || + q.flags & FIX_FLAGS.BREAK + ) { + metrics.nextMotionT = + p.epoch !== q.epoch || q.flags & FIX_FLAGS.BREAK ? q.t : p.t; + break; + } + } + track.metricsRevision = track.revision; + track.metricsIndex = i; + track.metricsTo = j; + } + const f = out.fraction; + out.lat = continuous ? a.lat + (b.lat - a.lat) * f : a.lat; + out.lon = continuous + ? a.lon + (((b.lon - a.lon + 540) % 360) - 180) * f + : a.lon; + if (out.lon > 180) out.lon -= 360; + if (out.lon < -180) out.lon += 360; + out.heightM = continuous + ? a.heightM + (b.heightM - a.heightM) * f + : a.heightM; + out.segmentCourseDeg = track.metrics.course; + out.segmentSpeedMps = track.metrics.speed; + out.motion = continuous + ? track.metrics.speed > 0 + ? 'moving' + : 'stopped' + : 'unknown'; + out.nextMotionT = track.metrics.nextMotionT; + out.latestReportAgeMs = Number.isFinite(track.wallAnchor) + ? track.wallAnchor + + track.lastMono - + track.monoAnchor - + timeAt(track, track.count - 1) + : NaN; + out.actualDelayMs = Number.isFinite(track.wallAnchor) + ? track.wallAnchor + track.lastMono - track.monoAnchor - sampleTime + : NaN; + return out; +} +export function advance(track, clocks, out) { + const wall = anchoredWall(track, clocks), + previousMono = track.lastMono; + const dt = Number.isFinite(previousMono) + ? Math.max(0, clocks.monoNowMs - previousMono) + : 0; + track.lastMono = clocks.monoNowMs; + if (!track.count) return sampleAt(track, NaN, out); + const newest = timeAt(track, track.count - 1), + oldest = timeAt(track, 0); + const ceiling = Math.min(newest, wall - track.targetDelayMs + track.offset); + const delayBlocked = + track.offset === 0 && + track.phase !== 'playing' && + ceiling < track.displayT && + track.displayT < newest; + if (!Number.isFinite(track.displayT)) + track.displayT = Math.min( + newest, + wall - track.targetDelayMs + track.offset, + ); + else if ( + track.offset === 0 && + track.rate !== 0 && + (ceiling - track.displayT > 90_000 || + (track.truncated && track.displayT < oldest)) + ) { + track.resetReason = + ceiling - track.displayT > 90_000 ? 'backlog' : 'unavailable-bracket'; + track.displayT = ceiling; + track.resets++; + } else if (track.rate !== 0) { + // While playing, finish the active segment at 1x even if lag grows. + const activeCeiling = + track.offset < 0 || track.phase === 'playing' ? newest : ceiling; + let next = track.displayT + dt * track.rate; + if (track.phase === 'held' && track.resumePending) { + const sinceReceipt = Number.isFinite(track.acceptedMono) + ? Math.max(0, clocks.monoNowMs - track.acceptedMono) + : 0; + next = track.displayT + Math.min(dt, sinceReceipt) * track.rate; + } + track.displayT = Math.max(track.displayT, Math.min(next, activeCeiling)); + } + track.phase = + track.rate === 0 + ? 'paused' + : track.displayT < oldest || delayBlocked + ? 'buffering' + : track.displayT >= newest + ? 'held' + : 'playing'; + track.resumePending = false; + out.displayT = track.displayT; + sampleCurrent(track, out); + if (out.motion !== 'moving' || track.phase !== 'playing') { + const boundary = Math.max(oldest, out.nextMotionT); + if ( + Number.isFinite(boundary) && + boundary > track.displayT && + track.rate > 0 + ) + out.nextWakeMonoMs = + clocks.monoNowMs + (boundary - track.displayT) / track.rate; + } + if (delayBlocked && track.rate > 0) + out.nextWakeMonoMs = + clocks.monoNowMs + Math.max(1, track.displayT - ceiling); + if (out.latestReportAgeMs > (track.policy.staleMs ?? 90_000)) + out.motion = 'stale'; + return out; +} +export function seek(track, displayTimeMs, clocks) { + if (!Number.isFinite(displayTimeMs)) + throw new TypeError('A seek needs a finite time'); + anchoredWall(track, clocks); + track.displayT = displayTimeMs; + track.offset = -1; + track.rate = 0; + track.phase = 'paused'; + track.lastMono = clocks.monoNowMs; + track.bracket = 0; + track.resets++; + track.resetReason = 'seek'; +} +export function setTimeOffsetMs(track, offset, clocks) { + if (!Number.isFinite(offset) || offset > 0) + throw new RangeError('Offset must be zero or negative'); + if (Number.isFinite(track.displayT)) { + track.resets++; + track.resetReason = 'time-offset'; + } + track.offset = offset; + track.rate = 1; + track.displayT = anchoredWall(track, clocks) - track.targetDelayMs + offset; + track.lastMono = clocks.monoNowMs; + track.phase = 'buffering'; + track.bracket = 0; +} +export function setRate(track, rate, clocks) { + if (!Number.isFinite(rate) || rate < 0 || rate > 16) + throw new RangeError('Rate must be in [0, 16]'); + if (track.offset === 0 && rate !== 0 && rate !== 1) + throw new RangeError('Live playback uses 1x or hold'); + track.rate = rate; + track.lastMono = clocks.monoNowMs; + track.phase = rate === 0 ? 'paused' : 'buffering'; +} diff --git a/src/data/contactPlayback.test.mjs b/src/data/contactPlayback.test.mjs new file mode 100644 index 000000000..67028b2f1 --- /dev/null +++ b/src/data/contactPlayback.test.mjs @@ -0,0 +1,302 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + createTrack, + createLagReservoir, + createHistoryBudget, + insertFix, + readFix, + advance, + sampleAt, + seek, + setRate, + setTimeOffsetMs, + mergeHistory, + correctHeight, + trimTrack, + destroyTrack, + FIX_BYTES, + FIX_FLAGS, +} from './contactPlayback.js'; +const fix = (t, metres = t / 100) => ({ + t, + lat: 42 + metres / 111320, + lon: -71, + bearingDeg: 0, +}); +const clocks = (t) => ({ wallNowMs: t, monoNowMs: t }); +const put = (track, t, metres, receipt = t) => + insertFix(track, fix(t, metres), { receivedAt: receipt, wallNowMs: receipt }); + +test('10 s reports / 15 s polls and a 6 m crawl move at report speed on ordinary frames', () => { + for (const speed of [10, 0.6]) { + const track = createTrack(), + out = {}; + let prior = null, + moving = 0, + pollJumps = 0; + for (let now = 5000; now <= 180000; now += 16) { + if ( + !prior || + Math.floor((now - 5000) / 15000) !== Math.floor((now - 5016) / 15000) + ) { + const t = Math.floor((now - 5000) / 10000) * 10000; + put(track, t, (t / 1000) * speed, now); + } + advance(track, clocks(now), out); + if (prior && out.displayT >= 0 && prior.displayT >= 0) { + const d = (out.lat - prior.lat) * 111320; + const dt = out.displayT - prior.displayT; + assert.ok(dt >= 0 && dt <= 16.0001); + if (dt > 0 && d > 0) { + assert.ok(Math.abs(d / (dt / 1000) - speed) < speed * 0.02); + moving++; + } + if (d > speed * 0.0161) pollJumps++; + } + prior = { ...out }; + } + assert.ok( + moving > 5000, + `non-vacuous ${speed} m/s trace: ${moving} moving frames`, + ); + assert.equal(pollJumps, 0); + } +}); +test('stationary observations survive, schedule departure, and distinguish underflow', () => { + const track = createTrack(), + out = {}; + put(track, 0, 0); + put(track, 15000, 0); + put(track, 30000, 6); + setTimeOffsetMs(track, 0, clocks(35000)); + advance(track, clocks(35000), out); + assert.equal(out.motion, 'stopped'); + assert.equal(out.nextWakeMonoMs, 40000); + advance(track, clocks(40000), out); + assert.equal(out.motion, 'moving'); + advance(track, clocks(55000), out); + assert.equal(out.phase, 'held'); + assert.equal(out.segmentSpeedMps, 0); +}); +test('same-time conflicts and invalid coordinates never make zero-duration segments', () => { + const track = createTrack(); + put(track, 0, 0); + assert.equal(put(track, 0, 0).reason, 'repeat'); + assert.equal(put(track, 0, 1).reason, 'conflict'); + for (const value of [ + { t: 1, lat: 91, lon: 0 }, + { t: 1, lat: 0, lon: NaN }, + { t: -1, lat: 0, lon: 0 }, + ]) + assert.equal(insertFix(track, value).accepted, false); + assert.equal(track.count, 1); + assert.equal(put(track, 15000, 0).accepted, true); +}); +test('ring wrap retains 15 minutes and pins an active bracket under capacity pressure', () => { + const track = createTrack({ capacity: 8 }); + for (let i = 0; i < 8; i++) put(track, i * 10000, i); + seek(track, 5000, clocks(80000)); + for (let i = 8; i < 30; i++) put(track, i * 10000, i); + const sample = sampleAt(track, 5000, {}); + assert.equal(sample.fromT, 0); + assert.equal(sample.toT, 10000); + assert.equal(track.count, 8); + assert.equal(track.storage.byteLength, 8 * FIX_BYTES); + assert.equal(track.truncated, true); + assert.ok(readFix(track, 2).flags & FIX_FLAGS.BREAK); + seek(track, 250000, clocks(300000)); + trimTrack(track, 300000, 2); + assert.ok(track.count <= 5); +}); +test('backward seek and history merge leave clock and live health alone', () => { + const track = createTrack(), + out = {}; + put(track, 30000, 30); + put(track, 45000, 45); + seek(track, 37500, clocks(80000)); + const health = track.acceptedAt; + mergeHistory(track, [fix(0, 0), fix(15000, 15), fix(30000, 999)]); + assert.equal(track.displayT, 37500); + assert.equal(track.acceptedAt, health); + assert.equal(readFix(track, 2).lat, fix(30000, 30).lat); + seek(track, 7500, clocks(80000)); + advance(track, clocks(90000), out); + assert.equal(out.displayT, 7500); + assert.equal(out.phase, 'paused'); + setRate(track, 1, clocks(90000)); + advance(track, clocks(91000), out); + assert.equal(out.displayT, 8500); +}); +test('system-clock changes do not alter animation elapsed time', () => { + const track = createTrack(), + out = {}; + put(track, 0, 0); + put(track, 60000, 600); + advance(track, clocks(60000), out); + const before = out.displayT; + advance(track, { wallNowMs: 999999999, monoNowMs: 60016 }, out); + assert.equal(out.displayT - before, 16); + setTimeOffsetMs(track, -10000, clocks(60016)); + assert.equal(track.resetReason, 'time-offset'); + assert.equal(track.resets, 1); +}); +test('a 90 s outage holds, then resumes without a sprint; backlog reset adds no second delay', () => { + const track = createTrack(), + out = {}; + put(track, 0, 0); + put(track, 15000, 15); + for (let t = 25000; t <= 120000; t += 100) advance(track, clocks(t), out); + assert.equal(out.displayT, 15000); + put(track, 105000, 105, 120000); + advance(track, clocks(120100), out); + assert.ok(out.displayT - 15000 <= 100); + track.targetDelayMs = 25000; + put(track, 300000, 300, 300000); + track.targetDelayMs = 25000; + advance(track, clocks(300000), out); + assert.equal(track.resets, 1); + assert.equal(track.resetReason, 'backlog'); + assert.equal(out.displayT, 275000); +}); +test('lag measures receipt of current minus report of previous and ignores cached deliveries', () => { + const feedLag = createLagReservoir(), + track = createTrack({ policy: { feedLag } }); + put(track, 0, 0, 5000); + put(track, 10000, 100, 20000); + assert.equal(track.lag.values[0], 20000); + put(track, 10000, 100, 80000); + assert.equal(track.lag.count, 1); + put(track, 20000, 200, 50000); + assert.equal(track.targetDelayMs, 45000); +}); +test('displayed bearing never reads a future fix; height corrections revise typed storage', () => { + const track = createTrack(); + insertFix(track, { ...fix(0, 0), bearingDeg: 90 }); + insertFix(track, { ...fix(15000, 6), bearingDeg: 270 }); + const out = sampleAt(track, 7500, {}); + assert.equal(out.segmentCourseDeg, 90); + assert.ok(Number.isNaN(out.heightM)); + correctHeight(track, out.fromSeq, 11.5); + correctHeight(track, out.toSeq, 21.5); + assert.equal(sampleAt(track, 7500, out).heightM, 16.5); +}); +test('allocation budget is hard and destruction returns payload bytes', () => { + const budget = createHistoryBudget(48 * 16), + tracks = Array.from({ length: 3 }, () => createTrack({ budget })); + for (const track of tracks) + for (let t = 0; t < 100; t++) put(track, t * 1000, t); + assert.ok(budget.allocatedBytes <= budget.maxBytes); + for (const track of tracks) destroyTrack(track); + assert.equal(budget.allocatedBytes, 0); +}); + +test('tiny displacement holds a preceding reliable course instead of a changed bearing', () => { + const track = createTrack(); + insertFix(track, { ...fix(0, 0), bearingDeg: 90 }); + insertFix(track, { ...fix(15000, 100), bearingDeg: 270 }); + insertFix(track, { ...fix(30000, 106), bearingDeg: 180 }); + assert.equal(sampleAt(track, 20000, {}).segmentCourseDeg, 0); +}); +test('an epoch gap wakes at its far boundary without drawing a connecting segment', () => { + const track = createTrack(), + out = {}; + insertFix(track, { ...fix(0, 0), epoch: 1 }); + insertFix(track, { ...fix(30000, 100), epoch: 2 }); + advance(track, clocks(40000), out); + assert.equal(out.segmentSpeedMps, 0); + assert.equal(out.nextWakeMonoMs, 65000); +}); + +test('increased lag is adopted while rebuffering, with one monotonic wake and no slow playback', () => { + const track = createTrack(), + out = {}; + put(track, 0, 0); + put(track, 15000, 15); + for (let now = 25000; now <= 120000; now += 100) + advance(track, clocks(now), out); + insertFix(track, fix(105000, 105), { + receivedAt: 120000, + wallNowMs: 120000, + monoNowMs: 120000, + }); + advance(track, clocks(120100), out); + assert.equal(out.displayT, 15000); + assert.equal(out.phase, 'buffering'); + assert.equal(out.nextWakeMonoMs, 125000); + advance(track, { wallNowMs: -9999999, monoNowMs: 125000 }, out); + assert.equal(out.displayT, 15000); + advance(track, { wallNowMs: 9999999, monoNowMs: 125016 }, out); + assert.equal(out.displayT, 15016); +}); + +test('capacity rejection leaves both lag reservoirs and clock policy unchanged', () => { + const feedLag = createLagReservoir(); + const track = createTrack({ capacity: 2, policy: { feedLag } }); + put(track, 0, 0); + put(track, 10000, 10); + seek(track, 5000, clocks(10000)); + const policyState = () => + structuredClone({ + lag: track.lag, + feedLag, + targetDelayMs: track.targetDelayMs, + acceptedAt: track.acceptedAt, + acceptedMono: track.acceptedMono, + resumePending: track.resumePending, + displayT: track.displayT, + wallAnchor: track.wallAnchor, + monoAnchor: track.monoAnchor, + resetReason: track.resetReason, + resets: track.resets, + }); + const before = policyState(); + assert.deepEqual(put(track, 20000, 20, 300000), { + accepted: false, + reason: 'capacity', + }); + assert.deepEqual(policyState(), before); +}); + +test('sampleAt writes the supplied output without replacing nested scratch', () => { + const track = createTrack(); + put(track, 0, 0); + put(track, 15000, 150); + const out = {}, + from = track.from, + to = track.to, + metrics = track.metrics; + for (let i = 0; i < 10000; i++) { + assert.equal(sampleAt(track, i, out), out); + assert.equal(track.from, from); + assert.equal(track.to, to); + assert.equal(track.metrics, metrics); + } +}); + +test('backfill pins the live bracket and quarantines outliers on either side', () => { + const track = createTrack({ + policy: { + accept: (a, b) => + Math.abs(b.lat - a.lat) * 111320 <= ((b.t - a.t) / 1000) * 36, + }, + }); + put(track, 10000, 0); + put(track, 25000, 100); + seek(track, 17500, clocks(40000)); + const before = { ...sampleAt(track, 17500, {}) }; + mergeHistory(track, [fix(0, -111320), fix(17500, 25), fix(40000, 111320)]); + const after = sampleAt(track, 17500, {}); + assert.equal(track.count, 2); + assert.equal(after.fromT, before.fromT); + assert.equal(after.toT, before.toT); + assert.equal(after.lat, before.lat); + mergeHistory(track, [fix(40000, 111320), fix(55000, 111420)]); + assert.equal( + track.count, + 4, + 'two coherent replacement fixes can be retained', + ); + assert.ok(readFix(track, 2).flags & FIX_FLAGS.BREAK); + assert.equal(sampleAt(track, 30000, {}).lat, fix(25000, 100).lat); +}); diff --git a/src/data/contactTrailRenderer.js b/src/data/contactTrailRenderer.js new file mode 100644 index 000000000..ae8712f9d --- /dev/null +++ b/src/data/contactTrailRenderer.js @@ -0,0 +1,229 @@ +import * as Cesium from 'cesium'; + +const ALPHA_STOPS = [ + [0, 0.8], + [120000, 0.45], + [600000, 0.2], + [900000, 0.08], +]; +export function trailAlpha(ageMs) { + const stops = ALPHA_STOPS; + for (let i = 1; i < stops.length; i++) { + if (ageMs === stops[i][0]) return stops[i][1]; + if (ageMs < stops[i][0]) { + const [a, x] = stops[i - 1], + [b, y] = stops[i]; + return x + ((y - x) * Math.max(0, ageMs - a)) / (b - a); + } + } + return 0.08; +} + +/** One static history batch and one frequently updated head. Never entities. */ +export function createContactTrailRenderer(scene) { + const heads = scene.primitives.add(new Cesium.PolylineCollection()); + const backing = heads.add({ + positions: [], + width: 4, + show: false, + material: Cesium.Material.fromType('Color', { + color: new Cesium.Color(0.02, 0.03, 0.05, 0.8), + }), + }); + const head = heads.add({ + positions: [], + width: 2, + show: false, + material: Cesium.Material.fromType('Color', { + color: Cesium.Color.WHITE.clone(), + }), + }); + let body = null, + segments = [], + revision = -1, + visible = true, + style = '#5EF08A'; + let lastSecond = -1, + completed = -1, + active = null; + const styleColor = Cesium.Color.fromCssColorString(style); + const attributeIds = []; + const color = new Cesium.Color(), + endpoint = new Cesium.Cartesian3(); + const headPositions = [new Cesium.Cartesian3(), endpoint]; + let rebuilds = 0; + function tint(alpha, dark = false) { + if (dark) { + color.red = 0.02; + color.green = 0.03; + color.blue = 0.05; + } else Cesium.Color.clone(styleColor, color); + color.alpha = alpha; + return color; + } + function replaceHistory(next) { + if (next.revision === revision) return; + revision = next.revision; + segments = next.segments; + completed = -1; + lastSecond = -1; + active = null; + if (body) scene.primitives.remove(body); + body = null; + const instances = []; + attributeIds.length = 0; + for (let i = 0; i < segments.length; i++) { + const segment = segments[i]; + attributeIds.push([`${i}:0`, `${i}:1`]); + if (segment.positions.length < 2) continue; + for (let lane = 0; lane < 2; lane++) { + instances.push( + new Cesium.GeometryInstance({ + id: `${i}:${lane}`, + geometry: new Cesium.PolylineGeometry({ + positions: segment.positions, + width: lane ? 2 : 4, + arcType: Cesium.ArcType.NONE, + vertexFormat: Cesium.PolylineColorAppearance.VERTEX_FORMAT, + }), + attributes: { + color: Cesium.ColorGeometryInstanceAttribute.fromColor( + tint(0.8, !lane), + ), + depthFailColor: Cesium.ColorGeometryInstanceAttribute.fromColor( + tint(0.8, !lane), + ), + show: new Cesium.ShowGeometryInstanceAttribute(false), + }, + }), + ); + } + } + if (instances.length) { + const depthFailAppearance = new Cesium.PolylineColorAppearance({ + translucent: true, + fragmentShaderSource: + 'in vec4 v_color; void main() { out_FragColor = czm_gammaCorrect(vec4(v_color.rgb, v_color.a * 0.2)); }', + }); + body = scene.primitives.add( + new Cesium.Primitive({ + geometryInstances: instances, + appearance: new Cesium.PolylineColorAppearance({ translucent: true }), + depthFailAppearance, + allowPicking: false, + asynchronous: false, + show: visible, + }), + ); + rebuilds++; + } + } + function setDisplaySample(sample, renderedPosition) { + if (!sample || !renderedPosition) { + head.show = false; + backing.show = false; + return; + } + const second = Math.floor(sample.displayT / 1000); + let lo = 0, + hi = segments.length; + while (lo < hi) { + const mid = (lo + hi) >>> 1; + if (segments[mid].toT <= sample.displayT) lo = mid + 1; + else hi = mid; + } + const done = lo - 1; + const candidate = segments[lo]; + const segment = + candidate && + candidate.fromT <= sample.displayT && + candidate.fromSeq === sample.fromSeq && + candidate.toSeq === sample.toSeq + ? candidate + : null; + if (body?.ready && done !== completed) { + for ( + let i = Math.min(done, completed) + 1; + i <= Math.max(done, completed); + i++ + ) { + for (let lane = 0; lane < 2; lane++) { + const attributes = body.getGeometryInstanceAttributes( + attributeIds[i][lane], + ); + if (!attributes) continue; + attributes.show[0] = i <= done ? 1 : 0; + attributes.show = attributes.show; + } + } + completed = done; + } + if (body?.ready && second !== lastSecond) { + for (let i = 0; i < segments.length; i++) { + const alpha = trailAlpha(sample.displayT - segments[i].toT); + for (let lane = 0; lane < 2; lane++) { + const attributes = body.getGeometryInstanceAttributes( + attributeIds[i][lane], + ); + if (!attributes) continue; + Cesium.ColorGeometryInstanceAttribute.toValue( + tint(alpha, !lane), + attributes.color, + ); + attributes.color = attributes.color; + Cesium.ColorGeometryInstanceAttribute.toValue( + tint(alpha, !lane), + attributes.depthFailColor, + ); + attributes.depthFailColor = attributes.depthFailColor; + } + } + lastSecond = second; + } + const show = visible && !!segment && sample.fraction < 1; + head.show = show; + backing.show = show; + if (!show) return; + if (active !== segment) { + active = segment; + headPositions[0] = segment.positions[0]; + headPositions[1] = endpoint; + } + Cesium.Cartesian3.clone(renderedPosition, endpoint); + head.positions = headPositions; + backing.positions = headPositions; + Cesium.Color.clone(tint(0.8), head.material.uniforms.color); + } + return { + replaceHistory, + setDisplaySample, + setStyle(next) { + if (next === style) return; + style = next; + Cesium.Color.fromCssColorString(style, styleColor); + lastSecond = -1; + }, + setVisible(next) { + visible = next; + heads.show = next; + if (body) body.show = next; + }, + destroy() { + if (body) scene.primitives.remove(body); + scene.primitives.remove(heads); + body = null; + segments = []; + }, + diagnostics() { + return { + revision, + rebuilds, + segments: segments.length, + body, + head, + backing, + entitiesAdded: 0, + }; + }, + }; +} diff --git a/src/data/contactTrailRenderer.test.mjs b/src/data/contactTrailRenderer.test.mjs new file mode 100644 index 000000000..04c7cc617 --- /dev/null +++ b/src/data/contactTrailRenderer.test.mjs @@ -0,0 +1,210 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as Cesium from 'cesium'; +import { + createContactTrailRenderer, + trailAlpha, +} from './contactTrailRenderer.js'; + +function fixture(t) { + for (const key of [ + 'HTMLCanvasElement', + 'HTMLImageElement', + 'ImageBitmap', + 'OffscreenCanvas', + ]) { + const prior = globalThis[key]; + globalThis[key] = class {}; + t.after(() => { + if (prior === undefined) delete globalThis[key]; + else globalThis[key] = prior; + }); + } + const primitives = new Cesium.PrimitiveCollection(); + t.after(() => primitives.destroy()); + return { primitives }; +} +test('body clips future subdivisions, head ends at exact marker, and frames retain geometry', (t) => { + const scene = fixture(t), + renderer = createContactTrailRenderer(scene); + const positions = [0, 1, 2].map((i) => + Cesium.Cartesian3.fromDegrees(-71, 42 + i * 0.001, 11.5), + ); + renderer.replaceHistory({ + revision: 1, + segments: [ + { + fromSeq: 0, + toSeq: 1, + fromT: 0, + toT: 10000, + positions: positions.slice(0, 2), + }, + { + fromSeq: 0, + toSeq: 1, + fromT: 10000, + toT: 20000, + positions: positions.slice(1, 3), + }, + ], + }); + const { body } = renderer.diagnostics(); + assert.equal(body.allowPicking, false); + // The constructor defers this check until shader preparation. Run the real + // Cesium validator without WebGL, including a missing-attribute control. + const source = body.depthFailAppearance.vertexShaderSource; + body._batchTableAttributeIndices = { color: 0 }; + assert.throws( + () => Cesium.Primitive._updateColorAttribute(body, source, true), + /depthFailColor/, + ); + for (const instance of body.geometryInstances) { + assert.deepEqual( + instance.attributes.depthFailColor.value, + instance.attributes.color.value, + ); + } + body._batchTableAttributeIndices = { color: 0, depthFailColor: 1 }; + assert.doesNotThrow(() => + Cesium.Primitive._updateColorAttribute(body, source, true), + ); + const attributes = new Map( + body.geometryInstances.map((instance) => [ + instance.id, + { + show: new Uint8Array([0]), + color: new Uint8Array(4), + depthFailColor: new Uint8Array(4), + }, + ]), + ); + // Simulate Cesium's completed upload; exercise public attribute writes. + body._ready = true; + body.getGeometryInstanceAttributes = (id) => attributes.get(id); + const marker = Cesium.Cartesian3.lerp( + positions[1], + positions[2], + 0.5, + new Cesium.Cartesian3(), + ); + const sample = { displayT: 15000, fromSeq: 0, toSeq: 1, fraction: 0.75 }; + for (let i = 0; i < 120; i++) renderer.setDisplaySample(sample, marker); + assert.deepEqual( + [...attributes.values()].map((a) => a.show[0]), + [1, 1, 0, 0], + ); + for (const a of attributes.values()) + assert.deepEqual(a.depthFailColor, a.color); + const result = renderer.diagnostics(); + assert.equal(result.body, body); + assert.equal(result.rebuilds, 1); + assert.ok(Cesium.Cartesian3.equals(result.head.positions.at(-1), marker)); + assert.equal(result.head.positions.length, 2); + renderer.setDisplaySample( + { ...sample, displayT: 5000, fraction: 0.25 }, + positions[0], + ); + assert.ok( + [...attributes.values()].every((a) => a.show[0] === 0), + 'rewind hides future geometry again', + ); + renderer.destroy(); + assert.equal(scene.primitives.length, 0); +}); +test('trail age curve uses the specified four alpha anchors', () => { + assert.equal(trailAlpha(0), 0.8); + assert.equal(trailAlpha(120000), 0.45); + assert.equal(trailAlpha(600000), 0.2); + assert.equal(trailAlpha(900000), 0.08); + assert.ok(Math.abs(trailAlpha(60000) - 0.625) < 1e-10); +}); + +test('the maximum prepared body stays below the 2 MiB geometry budget', () => { + const geometry = Cesium.PolylineGeometry.createGeometry( + new Cesium.PolylineGeometry({ + positions: [ + Cesium.Cartesian3.fromDegrees(-71, 42, 11.5), + Cesium.Cartesian3.fromDegrees(-71, 42.0002, 11.5), + ], + width: 2, + arcType: Cesium.ArcType.NONE, + vertexFormat: Cesium.PolylineColorAppearance.VERTEX_FORMAT, + }), + ); + const bytes = + Object.values(geometry.attributes).reduce( + (total, attribute) => total + (attribute?.values?.byteLength || 0), + 0, + ) + geometry.indices.byteLength; + assert.ok(bytes * 2 * 2047 < 2 * 1024 * 1024); +}); + +test('subdivision crossings only write the changed show range and age colors once per second', (t) => { + const renderer = createContactTrailRenderer(fixture(t)); + const positions = [ + Cesium.Cartesian3.fromDegrees(-71, 42), + Cesium.Cartesian3.fromDegrees(-71, 42.001), + ]; + renderer.replaceHistory({ + revision: 1, + segments: Array.from({ length: 100 }, (_, i) => ({ + fromSeq: 0, + toSeq: 1, + fromT: i * 500, + toT: (i + 1) * 500, + positions, + })), + }); + const { body } = renderer.diagnostics(); + const counts = { show: 0, color: 0, depthFailColor: 0 }; + const attributes = new Map( + body.geometryInstances.map((instance) => { + const result = {}; + for (const name of Object.keys(counts)) { + let value = instance.attributes[name].value; + Object.defineProperty(result, name, { + get: () => value, + set(next) { + value = next; + counts[name]++; + }, + }); + } + return [instance.id, result]; + }), + ); + body._ready = true; + body.getGeometryInstanceAttributes = (id) => attributes.get(id); + const display = (displayT) => + renderer.setDisplaySample( + { displayT, fromSeq: 0, toSeq: 1, fraction: 0.1 }, + positions[0], + ); + const reset = () => { + for (const name of Object.keys(counts)) counts[name] = 0; + }; + display(100); + assert.deepEqual(counts, { show: 0, color: 200, depthFailColor: 200 }); + reset(); + display(600); + assert.deepEqual(counts, { show: 2, color: 0, depthFailColor: 0 }); + reset(); + display(100); + assert.deepEqual(counts, { show: 2, color: 0, depthFailColor: 0 }); + assert.ok( + [...attributes.values()].every((a) => a.show[0] === 0), + 'backward seek hides the completed range', + ); + reset(); + display(1100); + assert.deepEqual(counts, { show: 4, color: 200, depthFailColor: 200 }); + reset(); + display(1200); + assert.deepEqual(counts, { show: 0, color: 0, depthFailColor: 0 }); + renderer.setStyle('#FF4538'); + display(1200); + assert.deepEqual(counts, { show: 0, color: 200, depthFailColor: 200 }); + for (const a of attributes.values()) + assert.deepEqual(a.color, a.depthFailColor); +}); diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index 29a76a15e..9196e22ba 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -183,6 +183,11 @@ export const DATA_CREDITS = [ '© OpenStreetMap contributors (ODbL) · ' + 'fix the map', }, + { + key: 'gtfs-rt', + html: + 'Transit vehicles: operator GTFS-Realtime feeds (each operator is credited below when its vehicles are shown)', + }, { key: 'radio-browser', html: @@ -287,6 +292,26 @@ export const NATURAL_EARTH_CREDIT = { 'Natural Earth (public domain)', }; +/** + * Per-feed transit credit, registered the first time that feed's vehicles + * render (see `src/data/transitFeeds.js` for the license of each). + * @param {{ id: string, attribution: string, license: string, licenseUrl: string }} feed + * @returns {{ key: string, html: string }} + */ +export function transitFeedCredit(feed) { + const escape = (text) => String(text) + .replaceAll('&', '&') + .replaceAll('<', '<') + .replaceAll('>', '>') + .replaceAll('"', '"'); + return { + key: `transit-${feed.id}`, + html: + `Transit (${escape(feed.attribution)}): ` + + `${escape(feed.license)}`, + }; +} + /** @type {Set} Keys of dynamic credits already registered this session. */ const _dynamicCreditKeys = new Set(); diff --git a/src/data/detection.js b/src/data/detection.js index ca31e71f2..a42dcec2d 100644 --- a/src/data/detection.js +++ b/src/data/detection.js @@ -1,3 +1,4 @@ +import { paintTransitBracket } from './detectionDraw.js'; import * as Cesium from 'cesium'; import { governorRequestRender } from '../renderGovernor.js'; import { @@ -10,6 +11,7 @@ import { import { acquireAlpha, appendCornerBracket, + appendTransitBracket, resolveTier, measureTrackLabel, nearFarScale, @@ -217,6 +219,8 @@ let _hostLane = null; * @type {{surface:HTMLCanvasElement|null,setActive:Function,requestPaint:Function,unregister:Function}|null} */ let _calloutLane = null; +let _transitBracketPaths = new Map(); +let _transitBracketAlpha = 1; /** * Callouts solved by the sensor lane this frame, replayed by the callout lane. * The array and its rows are POOLED — `_calloutCount` is the live length — so a @@ -329,6 +333,7 @@ export function destroyDetection() { _hostLane = null; _calloutLane?.unregister?.(); _calloutLane = null; + _transitBracketPaths.clear(); _calloutCount = 0; _viewer = null; _layers = []; @@ -517,7 +522,12 @@ export function markDetectionSourcesChanged(reason = 'sources-changed') { /** Read-only diagnostics for unit/browser QA. */ export function getDetectionDiagnostics() { - return _lastDiagnostics ? JSON.parse(JSON.stringify(_lastDiagnostics)) : null; + if (!_lastDiagnostics) return null; + const result = JSON.parse(JSON.stringify(_lastDiagnostics)); + // Readback only: expose painted plates so pixel QA can exclude occluded + // sprites/strokes. Nothing is copied on the production paint path. + result.calloutRects = _calloutPool.slice(0, _calloutCount).map(({ x, y, w, h, alpha }) => ({ x, y, w, h, alpha })); + return result; } function _publishDiagnostics() { @@ -981,9 +991,14 @@ function _stashCallout(entry, acquireFade, keyhole) { * @param {Object} frame Host paint frame. */ function _paintCalloutLane(frame) { - if (_mode === MODE_OFF || _suspended || _calloutCount === 0) return; + if (_mode === MODE_OFF || _suspended) return; const ctx = frame.ctx; if (!ctx) return; + for (const bands of _transitBracketPaths.values()) { + for (const entry of bands) { + if (entry) paintTransitBracket(ctx, entry.path, entry.color, entry.alpha * _transitBracketAlpha); + } + } ctx.textAlign = 'left'; ctx.textBaseline = 'alphabetic'; for (let i = 0; i < _calloutCount; i++) { @@ -1061,6 +1076,7 @@ function _materializeCandidate(obj, width, height, keyhole, occlusionRects, came * for a bounded cohort on solve ticks. */ function _drawOverlay(frame) { + _transitBracketPaths.clear(); const { width, height } = frame; if (!_ctx || width <= 0 || height <= 0) { return { didSolve: false, solveMs: 0, fadingCount: 0, animatingCount: 0, solvePending: false }; @@ -1217,6 +1233,9 @@ function _drawOverlay(frame) { ); halfW = _clamp((isTracked ? 14 : 9) * bscale, 7, 48); halfH = _clamp((isTracked ? 11 : 7) * bscale, 5, 38); + } else if (obj.tier?.startsWith('transit_') && obj.bracketHalfWidth > 0) { + halfW = obj.bracketHalfWidth; + halfH = obj.bracketHalfHeight; } else { halfW = _mode === MODE_DENSE ? (isTracked ? 28 : 11) : 16; halfH = _mode === MODE_DENSE ? (isTracked ? 22 : 7) : 10; @@ -1230,7 +1249,8 @@ function _drawOverlay(frame) { const keyholeAlpha = keyholeLabelAlphaFromGeometry(sx, sy, keyhole); const bracketAlpha = detectionBracketAlpha(obj.type, keyholeAlpha, keyholeOutsideOpacity); if (bracketAlpha > 0) { - appendCornerBracket(pathFor(bracketPaths, color, bracketAlpha), sx, sy, halfW, halfH); + const transit = obj.tier?.startsWith('transit_'); + (transit ? appendTransitBracket : appendCornerBracket)(pathFor(transit ? _transitBracketPaths : bracketPaths, color, bracketAlpha), sx, sy, halfW, halfH); visibleCount++; if (obj.type === 'AIR') aircraftBracketSectors[detectionHorizontalSector(sx, width)]++; if (bracketAlpha >= 1) bracketOpacityCounts.full++; @@ -1346,6 +1366,7 @@ function _drawOverlay(frame) { const fade = acquireAlpha(_enableTime, now, FADE_MS); const bracketWidth = _mode === MODE_DENSE ? 1 : 1.25; + _transitBracketAlpha = fade * bracketPresentationOpacity; // Brackets — batched by tier color and linear radial-opacity band. _ctx.lineWidth = bracketWidth; diff --git a/src/data/detectionDraw.js b/src/data/detectionDraw.js index 78ccea568..4b4c7aa33 100644 --- a/src/data/detectionDraw.js +++ b/src/data/detectionDraw.js @@ -237,3 +237,24 @@ export function nearFarScale(distance, near, nearValue, far, farValue) { const t = (distance - near) / (far - near); return nearValue + t * (farValue - nearValue); } + + +/** Transit keeps literal mode colour on the normal-composite surface. */ +export function paintTransitBracket(ctx, path, color, alpha = 1) { + ctx.save?.(); + ctx.globalCompositeOperation = 'source-over'; + ctx.globalAlpha = alpha; + ctx.strokeStyle = '#05080C'; + ctx.lineWidth = 3.25; + ctx.stroke(path); + ctx.strokeStyle = color; + ctx.lineWidth = 1.25; + ctx.stroke(path); + ctx.restore?.(); +} + +/** Centre 1.25 px strokes on pixels, so their cores are fully covered. */ +export function appendTransitBracket(sink, sx, sy, halfW, halfH) { + appendCornerBracket(sink, Math.floor(sx) + 0.5, Math.floor(sy) + 0.5, + Math.round(halfW), Math.round(halfH)); +} diff --git a/src/data/detectionDraw.test.mjs b/src/data/detectionDraw.test.mjs index dd9dfb67d..a7f896971 100644 --- a/src/data/detectionDraw.test.mjs +++ b/src/data/detectionDraw.test.mjs @@ -161,3 +161,34 @@ test('nearFarScale interpolates by distance and clamps to the near/far values', const mid = nearFarScale((1000 + 8000000) / 2, 1000, 3.0, 8000000, 0.5); assert.ok(Math.abs(mid - 1.75) < 1e-6, `midpoint ~1.75, got ${mid}`); }); + + +test('transit brackets paint a 1.25 px mode stroke over 3.25 px dark backing', async () => { + const { paintTransitBracket } = await import('./detectionDraw.js'); + const strokes = []; + const ctx = { stroke(path) { strokes.push([path, this.lineWidth, this.strokeStyle, this.globalAlpha]); } }; + const path = {}; + paintTransitBracket(ctx, path, '#FF4538', 0.75); + assert.deepEqual(strokes, [[path, 3.25, '#05080C', 0.75], [path, 1.25, '#FF4538', 0.75]]); +}); + +test('transit bracket backing and colour use source-over under every inherited sensor theme', async () => { + const {paintTransitBracket}=await import('./detectionDraw.js'); + for(const theme of ['normal','thermal','surveillance','noir','retro']) { + const strokes=[]; + const ctx={globalCompositeOperation:'screen',globalAlpha:0.3,stroke(p){strokes.push([this.globalCompositeOperation,this.lineWidth,this.strokeStyle]);},save(){this.saved=[this.globalCompositeOperation,this.globalAlpha];},restore(){[this.globalCompositeOperation,this.globalAlpha]=this.saved;}}; + paintTransitBracket(ctx,{},'#FF4538',1); + assert.deepEqual(strokes,[['source-over',3.25,'#05080C'],['source-over',1.25,'#FF4538']],theme); + assert.equal(ctx.globalCompositeOperation,'screen'); + } +}); + +test('transit bracket stroke centres have full pixel coverage at the failing fractional positions', async () => { + const {appendTransitBracket}=await import('./detectionDraw.js'); + for(const sy of [305.076,490.54,279.56]) { + const points=[],sink={moveTo:(x,y)=>points.push([x,y]),lineTo:(x,y)=>points.push([x,y])}; + appendTransitBracket(sink,759.016,sy,11,7); + assert.equal(points.length,12); + for(const [x,y] of points){assert.equal(x%1,0.5);assert.equal(y%1,0.5);} + } +}); diff --git a/src/data/detectionHost.test.mjs b/src/data/detectionHost.test.mjs index 7ad94c43b..06bfe01af 100644 --- a/src/data/detectionHost.test.mjs +++ b/src/data/detectionHost.test.mjs @@ -24,6 +24,7 @@ import { setOverlayEntries, } from '../overlays/worldOverlay.js'; import { DETECTION_THEME_MAP } from '../overlays/worldOverlayTokens.js'; +import { composeLabel, resolveTier } from './detectionDraw.js'; test('detection diagnostics count rendered fading rows instead of absent selected identities', () => { assert.equal(countFadingRenderEntries([ @@ -97,7 +98,7 @@ function mockContext(target, trace) { // set at some point during the frame. globalAlpha rides along for the same // reason: the backdrop feather is an alpha, not a colour. fill(path) { record('fill', path, this.fillStyle, this.globalAlpha); }, - stroke(path) { record('stroke', path); }, + stroke(path) { record('stroke', path, this.strokeStyle, this.lineWidth); }, fillRect(...args) { record('fillRect', ...args); }, fillText(...args) { record('fillText', ...args); }, drawImage(...args) { record('drawImage', ...args); }, @@ -378,6 +379,12 @@ test('detection lifecycle re-hosts unchanged painters behind the sole host liste env.postRender.raise(); const first = getDetectionDiagnostics(); assert.equal(first.profile, 'DENSE'); + assert.ok(Array.isArray(first.calloutRects)); + if (first.calloutRects.length) { + const x=first.calloutRects[0].x; + first.calloutRects[0].x=NaN; + assert.equal(getDetectionDiagnostics().calloutRects[0].x,x,'QA owns a copy, never a live plate'); + } assert.equal(first.observationCount, 2); assert.equal(first.didSolve, true); assert.ok(first.solveRevision > 0); @@ -874,3 +881,82 @@ test('the backdrop feather reaches the canvas as a lighter plate against sky', ( env.cleanup(); } }); + +test('a transit vehicle is a contact: box, route, mode and operator on the shared canvas', () => { + // The owner's finding was that buses were "green dots moving around + // clumsily" — the layer rendered them and the detection overlay ignored + // them, because transit published no detectable objects at all. This drives + // the real host with the shape the layer now returns. + const env = installEnvironment(); + const objects = [ + { + position: new Cesium.Cartesian3(0, 0, 6_356_752), + sourceId: 'mbta:y1276', + tier: 'transit_bus', + id: '111', + type: 'VEH', + klass: 'MBTA', + metric: 'BUS 34 KM/H', + }, + { + position: new Cesium.Cartesian3(0, 900, 6_356_752), + sourceId: 'mbta:R-548BBB14', + id: 'RED', + type: 'VEH', + klass: 'MBTA', + metric: 'METRO STOPPED', + skipLabel: true, + }, + ]; + try { + initWorldOverlay(env.viewer); + initDetection( + env.viewer, + [{ id: 'transit', getDetectableObjects: () => objects }], + () => {}, + ); + setMode('DENSE'); + env.advance(250); + env.postRender.raise(); + env.advance(250); + env.postRender.raise(); + const painted = env.ctx.calls + .filter(([name]) => name === 'fillText') + .map(([, text]) => text); + assert.ok( + painted.includes('111'), + `the route is the bright line; painted ${JSON.stringify(painted)}`, + ); + // The host paints a single-line track label — bright id, one dim micro + // field — so the mode and what the vehicle is doing share that field. + assert.ok( + painted.some((text) => String(text).includes('BUS')), + `the mode rides along on the micro line; painted ${JSON.stringify(painted)}`, + ); + assert.ok( + painted.some((text) => String(text).includes('34 KM/H')), + 'and so does what the vehicle is doing', + ); + assert.ok(env.ctx.calls.some(([name, , color, width]) => name === 'stroke' && color === '#5EF08A' && width === 1.25)); + assert.ok(env.ctx.calls.some(([name, , color, width]) => name === 'stroke' && color === '#05080C' && width === 3.25)); + assert.ok(!env.detectionCtx.calls.some(([name, , color]) => name === 'stroke' && color === '#5EF08A')); + // The operator travels on the object for the two-tier card form, which is + // where `klass` is read; the host's track label does not paint it. + assert.equal(composeLabel(objects[0]).secondary, 'MBTA · BUS 34 KM/H'); + // The selected vehicle carries its own fuller card, so detection must not + // label it a second time. + assert.equal( + painted.includes('RED'), + false, + 'skipLabel keeps the selected vehicle from being labelled twice', + ); + } finally { + env.cleanup(); + } +}); + +test('transit contacts take the vehicle tier, not the civil-air one', () => { + assert.equal(resolveTier({ type: 'VEH' }), 'vehicle'); + assert.equal(resolveTier({ type: 'AIR' }), 'civil'); + assert.equal(resolveTier({ type: 'SEA' }), 'sea'); +}); diff --git a/src/data/gtfsRealtime.js b/src/data/gtfsRealtime.js new file mode 100644 index 000000000..96d6b28a5 --- /dev/null +++ b/src/data/gtfsRealtime.js @@ -0,0 +1,274 @@ +/** + * @module gtfsRealtime + * @description Minimal GTFS-Realtime decoder for VehiclePosition feeds. + * + * Decodes only what the Transit layer needs from a `FeedMessage` — the header + * and every entity's `VehiclePosition` — straight from the protobuf wire + * format with `pbf` (already a dependency via the TomTom flow tiles). No + * `gtfs-realtime-bindings`, no protobufjs, no generated code: the GTFS-RT + * schema is stable, and field numbers are the contract. + * + * Field numbers (gtfs-realtime.proto, v2.0): + * FeedMessage 1 header, 2 entity[] + * FeedHeader 1 gtfs_realtime_version, 2 incrementality, 3 timestamp + * FeedEntity 1 id, 2 is_deleted, 3 trip_update, 4 vehicle, 5 alert + * VehiclePosition 1 trip, 2 position, 3 current_stop_sequence, + * 4 current_status, 5 timestamp, 6 congestion_level, + * 7 stop_id, 8 vehicle, 9 occupancy_status + * TripDescriptor 1 trip_id, 2 start_time, 3 start_date, + * 4 schedule_relationship, 5 route_id, 6 direction_id + * Position 1 latitude, 2 longitude, 3 bearing, 4 odometer, 5 speed + * VehicleDescriptor 1 id, 2 label, 3 license_plate + * + * Unknown fields (extensions, newer additions) are skipped by pbf, so a feed + * that carries OVapi/NYCT extensions decodes the same as a plain one. + * + * Decoding is bounded, because a byte cap alone does not bound what a decode + * COSTS: entities past `GTFS_MAX_ENTITIES` are left unread so pbf skips them + * whole (the result says it truncated), and every string field — the feed + * header's version included — is refused past `GTFS_MAX_STRING_CHARS` rather + * than carried into the snapshot. + * + * Pure: safe to import from the browser layer, the Vite proxy, and node:test. + */ + +import { PbfReader } from 'pbf'; + +/** VehiclePosition.VehicleStopStatus enum → label. */ +export const VEHICLE_STOP_STATUS = Object.freeze({ + 0: 'INCOMING_AT', + 1: 'STOPPED_AT', + 2: 'IN_TRANSIT_TO', +}); + +/** VehiclePosition.OccupancyStatus enum → label. */ +export const OCCUPANCY_STATUS = Object.freeze({ + 0: 'EMPTY', + 1: 'MANY_SEATS_AVAILABLE', + 2: 'FEW_SEATS_AVAILABLE', + 3: 'STANDING_ROOM_ONLY', + 4: 'CRUSHED_STANDING_ROOM_ONLY', + 5: 'FULL', + 6: 'NOT_ACCEPTING_PASSENGERS', + 7: 'NO_DATA_AVAILABLE', + 8: 'NOT_BOARDABLE', +}); + +/** + * Hard ceiling on decoded entities. Every registered feed is a metropolitan or + * national fleet — the largest observed is a few thousand vehicles — so this is + * two orders of magnitude above any honest payload. + */ +export const GTFS_MAX_ENTITIES = 50_000; +/** Longest accepted string field. GTFS ids, labels and stop ids are short. */ +export const GTFS_MAX_STRING_CHARS = 256; +/** FeedHeader.Incrementality: only FULL_DATASET carries a complete snapshot. */ +export const GTFS_INCREMENTALITY_FULL_DATASET = 0; +export const GTFS_INCREMENTALITY_DIFFERENTIAL = 1; + +/** + * Read a length-delimited string, refusing one that is implausibly long. + * An over-long field marks its containing message `oversize`, and the whole + * record is then dropped — never truncated. Half an id is a different vehicle, + * not a shorter name for the same one. + * @param {object} pbf + * @param {object} target Message being read, flagged when the field is refused. + * @returns {string|null} + */ +function readBoundedString(pbf, target) { + const value = pbf.readString(); + if (value.length <= GTFS_MAX_STRING_CHARS) return value; + target.oversize = true; + return null; +} + +function readFeedHeader(tag, header, pbf) { + if (tag === 1) header.version = readBoundedString(pbf, header); + else if (tag === 2) header.incrementality = pbf.readVarint(); + else if (tag === 3) header.timestamp = pbf.readVarint(); +} + +function readTripDescriptor(tag, trip, pbf) { + if (tag === 1) trip.tripId = readBoundedString(pbf, trip); + else if (tag === 2) trip.startTime = readBoundedString(pbf, trip); + else if (tag === 3) trip.startDate = readBoundedString(pbf, trip); + else if (tag === 4) trip.scheduleRelationship = pbf.readVarint(); + else if (tag === 5) trip.routeId = readBoundedString(pbf, trip); + else if (tag === 6) trip.directionId = pbf.readVarint(); +} + +function readPosition(tag, position, pbf) { + if (tag === 1) position.latitude = pbf.readFloat(); + else if (tag === 2) position.longitude = pbf.readFloat(); + else if (tag === 3) position.bearing = pbf.readFloat(); + else if (tag === 4) position.odometer = pbf.readDouble(); + else if (tag === 5) position.speed = pbf.readFloat(); +} + +function readVehicleDescriptor(tag, descriptor, pbf) { + if (tag === 1) descriptor.id = readBoundedString(pbf, descriptor); + else if (tag === 2) descriptor.label = readBoundedString(pbf, descriptor); + else if (tag === 3) + descriptor.licensePlate = readBoundedString(pbf, descriptor); +} + +function readVehiclePosition(tag, vehicle, pbf) { + if (tag === 1) vehicle.trip = pbf.readMessage(readTripDescriptor, {}); + else if (tag === 2) vehicle.position = pbf.readMessage(readPosition, {}); + else if (tag === 3) vehicle.currentStopSequence = pbf.readVarint(); + else if (tag === 4) vehicle.currentStatus = pbf.readVarint(); + else if (tag === 5) vehicle.timestamp = pbf.readVarint(); + else if (tag === 6) vehicle.congestionLevel = pbf.readVarint(); + else if (tag === 7) vehicle.stopId = readBoundedString(pbf, vehicle); + else if (tag === 8) + vehicle.vehicle = pbf.readMessage(readVehicleDescriptor, {}); + else if (tag === 9) vehicle.occupancyStatus = pbf.readVarint(); +} + +function readFeedEntity(tag, entity, pbf) { + if (tag === 1) entity.id = readBoundedString(pbf, entity); + else if (tag === 2) entity.isDeleted = pbf.readBoolean(); + else if (tag === 4) entity.vehicle = pbf.readMessage(readVehiclePosition, {}); + // 3 (trip_update) and 5 (alert) are skipped: pbf advances past any tag the + // reader leaves untouched. +} + +function readFeedMessage(tag, message, pbf) { + if (tag === 1) message.header = pbf.readMessage(readFeedHeader, {}); + else if (tag === 2) { + // Past the cap the field is left UNREAD. pbf skips any field its handler + // did not consume, so the excess costs one length varint each instead of a + // full nested decode that is then thrown away. + if (message.entities.length >= GTFS_MAX_ENTITIES) { + message.truncated = true; + return; + } + message.entities.push(pbf.readMessage(readFeedEntity, {})); + } +} + +/** + * Decode a raw GTFS-Realtime FeedMessage. + * @param {Uint8Array|ArrayBuffer} bytes Protobuf wire bytes. + * @returns {{ header: {version?: string, incrementality?: number, timestamp?: number}, + * entities: object[], truncated: boolean }} + */ +export function decodeFeedMessage(bytes) { + const view = bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes); + const pbf = new PbfReader(view); + return pbf.readFields(readFeedMessage, { + header: {}, + entities: [], + truncated: false, + }); +} + +function finiteOrNull(value) { + return Number.isFinite(value) ? value : null; +} + +function nonEmptyString(value) { + const text = typeof value === 'string' ? value.trim() : ''; + return text ? text : null; +} + +/** + * True when a lat/lon pair is a usable surface position. Rejects non-finite + * values, out-of-range degrees, and the (0,0) null island a cold GPS reports. + * @param {number} lat + * @param {number} lon + * @returns {boolean} + */ +export function isPlausibleVehiclePosition(lat, lon) { + if (!Number.isFinite(lat) || !Number.isFinite(lon)) return false; + if (Math.abs(lat) > 90 || Math.abs(lon) > 180) return false; + if (Math.abs(lat) < 1e-6 && Math.abs(lon) < 1e-6) return false; + return true; +} + +/** + * Flatten one decoded FeedEntity into the record the Transit layer renders, + * or null when the entity carries no usable vehicle position. + * @param {object} entity Decoded FeedEntity. + * @returns {object|null} + */ +export function normalizeVehicleEntity(entity) { + if (!entity || entity.isDeleted === true) return null; + const vehicle = entity.vehicle; + const position = vehicle?.position; + if (!vehicle || !position) return null; + // Any refused (over-long) string anywhere in this entity makes the whole + // record untrustworthy — drop it rather than render a half-identified dot. + if ( + entity.oversize || + vehicle.oversize || + vehicle.trip?.oversize || + vehicle.vehicle?.oversize + ) { + return null; + } + const lat = finiteOrNull(position.latitude); + const lon = finiteOrNull(position.longitude); + if (!isPlausibleVehiclePosition(lat, lon)) return null; + const id = nonEmptyString(vehicle.vehicle?.id) || nonEmptyString(entity.id); + if (!id) return null; + const bearing = finiteOrNull(position.bearing); + const speed = finiteOrNull(position.speed); + const timestamp = + Number.isFinite(vehicle.timestamp) && vehicle.timestamp > 0 + ? vehicle.timestamp + : null; + return { + id, + lat: Number(lat.toFixed(6)), + lon: Number(lon.toFixed(6)), + bearing: bearing === null ? null : ((bearing % 360) + 360) % 360, + speedMps: speed === null || speed < 0 ? null : speed, + timestamp, + routeId: nonEmptyString(vehicle.trip?.routeId), + tripId: nonEmptyString(vehicle.trip?.tripId), + directionId: Number.isInteger(vehicle.trip?.directionId) + ? vehicle.trip.directionId + : null, + label: nonEmptyString(vehicle.vehicle?.label), + stopId: nonEmptyString(vehicle.stopId), + status: VEHICLE_STOP_STATUS[vehicle.currentStatus] || null, + occupancy: OCCUPANCY_STATUS[vehicle.occupancyStatus] || null, + }; +} + +/** + * Decode a VehiclePositions feed into the compact snapshot the proxy serves. + * Duplicate vehicle ids keep the newest timestamp (feeds occasionally repeat + * a vehicle across two trip entities during a handover). + * @param {Uint8Array|ArrayBuffer} bytes Protobuf wire bytes. + * @returns {{ version: string|null, timestamp: number|null, incrementality: number, + * entityCount: number, truncated: boolean, vehicles: object[] }} + */ +export function decodeVehiclePositions(bytes) { + const message = decodeFeedMessage(bytes); + const byId = new Map(); + for (const entity of message.entities) { + const record = normalizeVehicleEntity(entity); + if (!record) continue; + const existing = byId.get(record.id); + if (!existing || (record.timestamp ?? 0) >= (existing.timestamp ?? 0)) { + byId.set(record.id, record); + } + } + const headerTimestamp = message.header?.timestamp; + const incrementality = Number.isFinite(message.header?.incrementality) + ? message.header.incrementality + : GTFS_INCREMENTALITY_FULL_DATASET; + return { + version: nonEmptyString(message.header?.version), + timestamp: + Number.isFinite(headerTimestamp) && headerTimestamp > 0 + ? headerTimestamp + : null, + incrementality, + entityCount: message.entities.length, + truncated: message.truncated === true, + vehicles: [...byId.values()], + }; +} diff --git a/src/data/gtfsRealtime.test.mjs b/src/data/gtfsRealtime.test.mjs new file mode 100644 index 000000000..04aecfc4d --- /dev/null +++ b/src/data/gtfsRealtime.test.mjs @@ -0,0 +1,348 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { PbfWriter } from 'pbf'; +import { + GTFS_MAX_ENTITIES, + GTFS_MAX_STRING_CHARS, + decodeFeedMessage, + decodeVehiclePositions, + isPlausibleVehiclePosition, + normalizeVehicleEntity, +} from './gtfsRealtime.js'; + +/** Encode a nested message with a fresh writer and embed it as a bytes field. */ +function embed(writer, tag, build) { + const inner = new PbfWriter(); + build(inner); + writer.writeBytesField(tag, inner.finish()); +} + +/** + * Hand-encode one FeedEntity carrying a VehiclePosition, from the proto field + * numbers — the same contract the decoder reads, written independently. + */ +function writeVehicleEntity( + writer, + { + entityId, + vehicleId, + label, + lat, + lon, + bearing, + speed, + timestamp, + routeId, + tripId, + directionId, + stopId, + status, + occupancy, + isDeleted, + extraUnknownFields = false, + }, +) { + embed(writer, 2, (entity) => { + if (entityId != null) entity.writeStringField(1, entityId); + if (isDeleted) entity.writeBooleanField(2, true); + embed(entity, 4, (vehicle) => { + if (routeId != null || tripId != null || directionId != null) { + embed(vehicle, 1, (trip) => { + if (tripId != null) trip.writeStringField(1, tripId); + if (routeId != null) trip.writeStringField(5, routeId); + if (directionId != null) trip.writeVarintField(6, directionId); + }); + } + if (lat != null) { + embed(vehicle, 2, (position) => { + position.writeFloatField(1, lat); + position.writeFloatField(2, lon); + if (bearing != null) position.writeFloatField(3, bearing); + if (speed != null) position.writeFloatField(5, speed); + if (extraUnknownFields) position.writeDoubleField(4, 123456.5); // odometer + }); + } + if (status != null) vehicle.writeVarintField(4, status); + if (timestamp != null) vehicle.writeVarintField(5, timestamp); + if (stopId != null) vehicle.writeStringField(7, stopId); + if (vehicleId != null || label != null) { + embed(vehicle, 8, (descriptor) => { + if (vehicleId != null) descriptor.writeStringField(1, vehicleId); + if (label != null) descriptor.writeStringField(2, label); + }); + } + if (occupancy != null) vehicle.writeVarintField(9, occupancy); + if (extraUnknownFields) + vehicle.writeStringField(1001, 'operator-extension'); + }); + }); +} + +function encodeFeed( + entities, + { version = '2.0', timestamp = 1_788_936_000 } = {}, +) { + const writer = new PbfWriter(); + embed(writer, 1, (header) => { + header.writeStringField(1, version); + header.writeVarintField(2, 0); + header.writeVarintField(3, timestamp); + }); + for (const entity of entities) writeVehicleEntity(writer, entity); + return writer.finish(); +} + +test('decodes header and every VehiclePosition field the layer renders', () => { + const bytes = encodeFeed([ + { + entityId: 'e1', + vehicleId: 'bus-17', + label: '1557', + lat: 44.927551, + lon: -93.386711, + bearing: 248, + speed: 11.2, + timestamp: 1_788_936_945, + routeId: '17', + tripId: '1134773', + directionId: 0, + stopId: '57458', + status: 1, + occupancy: 2, + }, + ]); + const snapshot = decodeVehiclePositions(bytes); + assert.equal(snapshot.version, '2.0'); + assert.equal(snapshot.timestamp, 1_788_936_000); + assert.equal(snapshot.entityCount, 1); + assert.equal(snapshot.vehicles.length, 1); + const [v] = snapshot.vehicles; + assert.equal(v.id, 'bus-17'); + assert.equal(v.label, '1557'); + assert.ok(Math.abs(v.lat - 44.927551) < 1e-5); + assert.ok(Math.abs(v.lon + 93.386711) < 1e-5); + assert.equal(v.bearing, 248); + assert.ok(Math.abs(v.speedMps - 11.2) < 1e-5); + assert.equal(v.timestamp, 1_788_936_945); + assert.equal(v.routeId, '17'); + assert.equal(v.tripId, '1134773'); + assert.equal(v.directionId, 0); + assert.equal(v.stopId, '57458'); + assert.equal(v.status, 'STOPPED_AT'); + assert.equal(v.occupancy, 'FEW_SEATS_AVAILABLE'); +}); + +test('unknown and extension fields are skipped, not misread', () => { + const bytes = encodeFeed([ + { entityId: 'x', lat: 51.3, lon: 5.33, extraUnknownFields: true }, + ]); + const snapshot = decodeVehiclePositions(bytes); + assert.equal(snapshot.vehicles.length, 1); + assert.equal(snapshot.vehicles[0].id, 'x'); + assert.ok(Math.abs(snapshot.vehicles[0].lat - 51.3) < 1e-5); +}); + +test('entities without a usable position are dropped and never throw', () => { + const bytes = encodeFeed([ + { entityId: 'no-position' }, + { entityId: 'deleted', lat: 1, lon: 1, isDeleted: true }, + { entityId: 'null-island', lat: 0, lon: 0 }, + { entityId: 'out-of-range', lat: 91, lon: 10 }, + { entityId: 'nan', lat: Number.NaN, lon: 10 }, + { entityId: 'keeper', lat: 60.17, lon: 24.94 }, + ]); + const snapshot = decodeVehiclePositions(bytes); + assert.equal(snapshot.entityCount, 6); + assert.deepEqual( + snapshot.vehicles.map((v) => v.id), + ['keeper'], + ); +}); + +test('vehicle id falls back to the entity id and the vehicle is dropped when neither exists', () => { + const bytes = encodeFeed([ + { entityId: 'entity-only', lat: 42.3, lon: -71.1 }, + { lat: 42.3, lon: -71.1 }, + ]); + const snapshot = decodeVehiclePositions(bytes); + assert.deepEqual( + snapshot.vehicles.map((v) => v.id), + ['entity-only'], + ); +}); + +test('duplicate vehicle ids keep the newest timestamp', () => { + const bytes = encodeFeed([ + { entityId: 'a', vehicleId: 'v1', lat: 42.3, lon: -71.1, timestamp: 100 }, + { entityId: 'b', vehicleId: 'v1', lat: 42.31, lon: -71.11, timestamp: 200 }, + { entityId: 'c', vehicleId: 'v1', lat: 42.32, lon: -71.12, timestamp: 150 }, + ]); + const snapshot = decodeVehiclePositions(bytes); + assert.equal(snapshot.vehicles.length, 1); + assert.equal(snapshot.vehicles[0].timestamp, 200); + assert.ok(Math.abs(snapshot.vehicles[0].lat - 42.31) < 1e-5); +}); + +test('bearing is normalized to [0, 360) and negative speed is treated as unknown', () => { + const bytes = encodeFeed([ + { entityId: 'neg', lat: 30.2, lon: -97.7, bearing: -90, speed: -1 }, + { entityId: 'wrap', lat: 30.2, lon: -97.7, bearing: 450 }, + ]); + const [neg, wrap] = decodeVehiclePositions(bytes).vehicles; + assert.equal(neg.bearing, 270); + assert.equal(neg.speedMps, null); + assert.equal(wrap.bearing, 90); +}); + +test('missing header yields null version/timestamp rather than garbage', () => { + const writer = new PbfWriter(); + writeVehicleEntity(writer, { entityId: 'solo', lat: 10, lon: 10 }); + const snapshot = decodeVehiclePositions(writer.finish()); + assert.equal(snapshot.version, null); + assert.equal(snapshot.timestamp, null); + assert.equal(snapshot.vehicles.length, 1); +}); + +test('decodeFeedMessage accepts an ArrayBuffer and an empty feed', () => { + const empty = decodeFeedMessage(new ArrayBuffer(0)); + assert.deepEqual(empty, { header: {}, entities: [], truncated: false }); + const bytes = encodeFeed([]); + const asBuffer = bytes.buffer.slice( + bytes.byteOffset, + bytes.byteOffset + bytes.byteLength, + ); + assert.equal(decodeFeedMessage(asBuffer).header.version, '2.0'); +}); + +test('isPlausibleVehiclePosition and normalizeVehicleEntity guard their inputs', () => { + assert.equal(isPlausibleVehiclePosition(45, 90), true); + assert.equal(isPlausibleVehiclePosition(0, 0), false); + assert.equal(isPlausibleVehiclePosition(-90.0001, 0), false); + assert.equal(isPlausibleVehiclePosition(1, Infinity), false); + assert.equal(normalizeVehicleEntity(null), null); + assert.equal(normalizeVehicleEntity({ id: 'no-vehicle' }), null); + assert.equal( + normalizeVehicleEntity({ + id: 'v', + vehicle: { position: { latitude: 1, longitude: 2 }, currentStatus: 99 }, + })?.status, + null, + ); +}); + +test('the decode is bounded by entity count as well as by bytes', () => { + const entities = []; + for (let index = 0; index < GTFS_MAX_ENTITIES + 25; index += 1) { + entities.push({ + entityId: `e${index}`, + vehicleId: `v${index}`, + lat: 42 + index * 1e-6, + lon: -71, + timestamp: 1_788_936_000, + }); + } + const decoded = decodeVehiclePositions(encodeFeed(entities)); + assert.equal(decoded.entityCount, GTFS_MAX_ENTITIES); + assert.equal(decoded.truncated, true); + assert.equal(decoded.vehicles.length, GTFS_MAX_ENTITIES); + // A feed inside the cap is not flagged. + assert.equal( + decodeVehiclePositions(encodeFeed(entities.slice(0, 3))).truncated, + false, + ); +}); + +test('an implausibly long string drops the whole record rather than truncating an id', () => { + const long = 'x'.repeat(GTFS_MAX_STRING_CHARS + 1); + const decoded = decodeVehiclePositions( + encodeFeed([ + { + entityId: 'ok', + vehicleId: 'good', + lat: 42.1, + lon: -71.1, + timestamp: 1_788_936_000, + }, + { + entityId: 'bad', + vehicleId: long, + lat: 42.2, + lon: -71.2, + timestamp: 1_788_936_000, + }, + { + entityId: 'bad-route', + vehicleId: 'v3', + routeId: long, + lat: 42.3, + lon: -71.3, + timestamp: 1_788_936_000, + }, + ]), + ); + // The entity is still counted — the decoder had to walk past it — but it is + // never rendered, because half an id is a different vehicle. + assert.equal(decoded.entityCount, 3); + assert.deepEqual( + decoded.vehicles.map((vehicle) => vehicle.id), + ['good'], + ); + // A string exactly at the cap is fine. + const atCap = 'y'.repeat(GTFS_MAX_STRING_CHARS); + const kept = decodeVehiclePositions( + encodeFeed([ + { entityId: 'e', vehicleId: atCap, lat: 42, lon: -71, timestamp: 1 }, + ]), + ); + assert.deepEqual( + kept.vehicles.map((vehicle) => vehicle.id), + [atCap], + ); +}); + +test('incrementality is reported, not silently discarded', () => { + assert.equal(decodeVehiclePositions(encodeFeed([])).incrementality, 0); + const writer = new PbfWriter(); + embed(writer, 1, (header) => { + header.writeStringField(1, '2.0'); + header.writeVarintField(2, 1); + header.writeVarintField(3, 1_788_936_000); + }); + assert.equal(decodeVehiclePositions(writer.finish()).incrementality, 1); +}); + +test('the feed header version is bounded like every other string', () => { + const writer = new PbfWriter(); + embed(writer, 1, (header) => { + header.writeStringField(1, 'v'.repeat(GTFS_MAX_STRING_CHARS + 1)); + header.writeVarintField(2, 0); + header.writeVarintField(3, 1_788_936_000); + }); + const decoded = decodeVehiclePositions(writer.finish()); + // Refused, not carried: an unbounded header field is the one string a feed + // can grow without the entity cap ever seeing it. + assert.equal(decoded.version, null); + assert.equal(decoded.timestamp, 1_788_936_000); +}); + +test('entities past the cap are skipped whole rather than decoded and discarded', () => { + const decoded = decodeVehiclePositions( + encodeFeed( + Array.from({ length: GTFS_MAX_ENTITIES + 5 }, (_, index) => ({ + entityId: `e${index}`, + vehicleId: `v${index}`, + lat: 42 + index * 1e-6, + lon: -71, + timestamp: 1_788_936_000, + })), + ), + ); + assert.equal(decoded.entityCount, GTFS_MAX_ENTITIES); + assert.equal(decoded.truncated, true); + // The skipped tail never becomes an object, so the last retained vehicle is + // the one at the cap and nothing beyond it appears. + assert.equal( + decoded.vehicles.some((item) => item.id === `v${GTFS_MAX_ENTITIES}`), + false, + ); +}); diff --git a/src/data/iconOrientation.js b/src/data/iconOrientation.js index 96e3a1470..857f1aeec 100644 --- a/src/data/iconOrientation.js +++ b/src/data/iconOrientation.js @@ -49,7 +49,12 @@ const _scratchWorldForward = new Cesium.Cartesian3(); * unavailable (off-screen/behind camera/degenerate). * @returns {number|null} Rotation in radians, or `previous` when unknown. */ -export function screenProjectedRotation(scene, position, courseDeg, previous = null) { +export function screenProjectedRotation( + scene, + position, + courseDeg, + previous = null, +) { const camera = scene?.camera; if (!camera?.rightWC || !camera?.upWC || !position) return previous; @@ -58,20 +63,26 @@ export function screenProjectedRotation(scene, position, courseDeg, previous = n Math.sin(courseRad) * FORWARD_PROBE_M, Math.cos(courseRad) * FORWARD_PROBE_M, 0, - _scratchForward + _scratchForward, + ); + const enu = Cesium.Transforms.eastNorthUpToFixedFrame( + position, + Cesium.Ellipsoid.WGS84, + _scratchEnu, + ); + Cesium.Matrix4.multiplyByPointAsVector( + enu, + _scratchForward, + _scratchWorldForward, ); - const enu = Cesium.Transforms.eastNorthUpToFixedFrame(position, Cesium.Ellipsoid.WGS84, _scratchEnu); - Cesium.Matrix4.multiplyByPointAsVector(enu, _scratchForward, _scratchWorldForward); // Screen x follows camera-right. Window y grows downward, the opposite of // camera-up. Projecting the vector itself avoids clipping/behind-camera // failure modes from the old forward-point worldToWindowCoordinates probe. const dx = Cesium.Cartesian3.dot(_scratchWorldForward, camera.rightWC); const dy = -Cesium.Cartesian3.dot(_scratchWorldForward, camera.upWC); - if ( - (dx * dx + dy * dy) - < MIN_SCREEN_COMPONENT_M * MIN_SCREEN_COMPONENT_M - ) return previous; + if (dx * dx + dy * dy < MIN_SCREEN_COMPONENT_M * MIN_SCREEN_COMPONENT_M) + return previous; // Window y grows downward; rotation 0 = icon pointing screen-up. // Icon direction in window coords after CCW rotation r is (-sin r, -cos r), @@ -94,13 +105,20 @@ export function stabilizeScreenRotation( next, deadbandRad = ROTATION_DEADBAND_RAD, ) { - if (!Number.isFinite(next)) return Number.isFinite(previous) ? previous : null; + if (!Number.isFinite(next)) + return Number.isFinite(previous) ? previous : null; if (!Number.isFinite(previous)) return next; - const delta = Math.atan2(Math.sin(next - previous), Math.cos(next - previous)); + const delta = Math.atan2( + Math.sin(next - previous), + Math.cos(next - previous), + ); return Math.abs(delta) < Math.max(0, deadbandRad) ? previous : next; } -const _occluder = new Cesium.EllipsoidalOccluder(Cesium.Ellipsoid.WGS84, new Cesium.Cartesian3()); +const _occluder = new Cesium.EllipsoidalOccluder( + Cesium.Ellipsoid.WGS84, + new Cesium.Cartesian3(), +); /** * Half-width of the crossfade band centred on the ellipsoid silhouette, in @@ -169,7 +187,11 @@ const _wgs84OneOverRadii = Cesium.Ellipsoid.WGS84.oneOverRadii; * @param {number} [featherRad=HORIZON_FEATHER_RAD] Half-width of the blend band. * @returns {number} 0 (ground behind) … 1 (sky behind). */ -export function skyBackdropFactor(cameraPosition, position, featherRad = HORIZON_FEATHER_RAD) { +export function skyBackdropFactor( + cameraPosition, + position, + featherRad = HORIZON_FEATHER_RAD, +) { if (!cameraPosition || !position) return 0; const s = _wgs84OneOverRadii; @@ -236,6 +258,120 @@ export function horizonOccluder(camera) { */ export function cameraPoseSignature(camera) { const p = camera.positionWC; - return `${Math.round(p.x / 10)}:${Math.round(p.y / 10)}:${Math.round(p.z / 10)}:` + - `${camera.heading.toFixed(3)}:${camera.pitch.toFixed(3)}:${camera.roll.toFixed(3)}`; + return ( + `${Math.round(p.x / 10)}:${Math.round(p.y / 10)}:${Math.round(p.z / 10)}:` + + `${camera.heading.toFixed(3)}:${camera.pitch.toFixed(3)}:${camera.roll.toFixed(3)}` + ); } + +/** + * Meters ahead used by the exact projection below. Short on purpose: the point + * of a perspective projection is that it is LOCAL to the contact, and a two + * kilometre probe on a bus at six hundred metres would sample a part of the + * frustum the vehicle is nowhere near. + */ +const NEAR_PROBE_M = 30; +const _scratchProbe = new Cesium.Cartesian3(); +const _scratchWindowAt = new Cesium.Cartesian2(); +const _scratchWindowAhead = new Cesium.Cartesian2(); + +/** + * Billboard rotation for a SURFACE contact, projected exactly — perspective + * division included — instead of through the camera-basis approximation. + * + * `screenProjectedRotation` above deliberately projects the course vector onto + * the camera's right/up basis, which is orthographic: exact at the viewport + * centre and an approximation away from it. That is the right trade for + * aircraft, where the alternative loses continuity through a tracked orbit and + * the contacts are far enough away for the error to be small. + * + * It is the wrong trade for street-level vehicles. A probe with a real Cesium + * camera at 600 m and −45° pitch, on a northbound contact 350 m right of + * centre, put the orthographic answer at 0° where the true screen direction of + * travel was 22.5°. At that pitch and that range the vehicles fill the frame + * away from the centre, and every one of them points visibly wrong. + * + * So this projects two nearby world points through the actual view-projection + * and reads the angle between them on screen. When either point cannot be + * projected — behind the camera, off the frustum, degenerate — it falls back to + * the camera-basis answer rather than inventing one. + * + * @param {Cesium.Scene} scene The scene (projection source). + * @param {Cesium.Cartesian3} position Entity world position. + * @param {number} courseDeg Course in degrees clockwise from north. + * @param {number|null} previous Rotation to keep when projection is unavailable. + * @returns {number|null} Rotation in radians, or the fallback. + */ +export function perspectiveProjectedRotation( + scene, + position, + courseDeg, + previous = null, +) { + if (!scene?.camera || !position || !Number.isFinite(courseDeg)) { + return previous; + } + const courseRad = Cesium.Math.toRadians(courseDeg); + Cesium.Cartesian3.fromElements( + Math.sin(courseRad) * NEAR_PROBE_M, + Math.cos(courseRad) * NEAR_PROBE_M, + 0, + _scratchForward, + ); + const enu = Cesium.Transforms.eastNorthUpToFixedFrame( + position, + Cesium.Ellipsoid.WGS84, + _scratchEnu, + ); + Cesium.Matrix4.multiplyByPointAsVector( + enu, + _scratchForward, + _scratchWorldForward, + ); + Cesium.Cartesian3.add(position, _scratchWorldForward, _scratchProbe); + + const at = Cesium.SceneTransforms.worldToWindowCoordinates( + scene, + position, + _scratchWindowAt, + ); + const ahead = Cesium.SceneTransforms.worldToWindowCoordinates( + scene, + _scratchProbe, + _scratchWindowAhead, + ); + if (!at || !ahead) { + return screenProjectedRotation(scene, position, courseDeg, previous); + } + const dx = ahead.x - at.x; + const dy = ahead.y - at.y; + const separation = Math.hypot(dx, dy); + // Window y grows downward; rotation 0 = icon pointing screen-up. + const exact = Math.atan2(-dx, -dy); + if (separation >= PERSPECTIVE_BLEND_MAX_PX) return exact; + // Below a pixel or so of separation the exact answer is dominated by + // projection noise, and above it the camera-basis answer is the wrong one + // by up to twenty-odd degrees. A hard cutoff between the two flipped a + // sprite by 23° for a 20 m change of camera altitude; the two answers are + // blended across the band instead, the short way round, so the rotation is + // continuous through it. + const basis = screenProjectedRotation(scene, position, courseDeg, previous); + if (!Number.isFinite(basis)) return exact; + if (separation <= PERSPECTIVE_BLEND_MIN_PX) return basis; + const weight = + (separation - PERSPECTIVE_BLEND_MIN_PX) / + (PERSPECTIVE_BLEND_MAX_PX - PERSPECTIVE_BLEND_MIN_PX); + const delta = Math.atan2(Math.sin(exact - basis), Math.cos(exact - basis)); + return basis + delta * weight; +} + +/** + * Probe separations (px) between which the exact and camera-basis answers + * are blended. At or below the floor the camera-basis answer is used alone; + * at or above the ceiling the exact one. + */ +export const PERSPECTIVE_BLEND_MIN_PX = 0.5; +export const PERSPECTIVE_BLEND_MAX_PX = 3; + +/** Exported for tests: how far ahead the exact projection probes. */ +export const PERSPECTIVE_PROBE_M = NEAR_PROBE_M; diff --git a/src/data/iconOrientation.test.mjs b/src/data/iconOrientation.test.mjs index f6111981a..62808ce79 100644 --- a/src/data/iconOrientation.test.mjs +++ b/src/data/iconOrientation.test.mjs @@ -47,8 +47,12 @@ test('off-center oblique contacts pin the documented perspective divergence', () const right = new Cesium.Cartesian3(0, 1, 0); const up = new Cesium.Cartesian3(Math.sin(pitch), 0, Math.cos(pitch)); const forward = new Cesium.Cartesian3(-Math.cos(pitch), 0, Math.sin(pitch)); - const basisRotation = screenProjectedRotation({ camera: { rightWC: right, upWC: up } }, - equatorPosition, 0, null); + const basisRotation = screenProjectedRotation( + { camera: { rightWC: right, upWC: up } }, + equatorPosition, + 0, + null, + ); // Exact pinhole derivative for a contact 2 km right of center and 5 km in // front. North has a depth component under this oblique camera, so exact @@ -65,26 +69,29 @@ test('off-center oblique contacts pin the documented perspective divergence', () const exactRotation = Math.atan2(-exactScreenX, exactScreenUp); assert.ok(Math.abs(basisRotation) < 1e-12); - assert.ok(Math.abs(wrappedDelta(exactRotation, basisRotation)) > Cesium.Math.toRadians(5)); + assert.ok( + Math.abs(wrappedDelta(exactRotation, basisRotation)) > + Cesium.Math.toRadians(5), + ); }); test('screen rotation holds sub-degree projection noise', () => { const previous = 1; assert.equal( - stabilizeScreenRotation(previous, previous + (0.25 * Math.PI / 180)), + stabilizeScreenRotation(previous, previous + (0.25 * Math.PI) / 180), previous, ); }); test('screen rotation accepts deliberate camera-orbit movement', () => { const previous = 1; - const next = previous + (2 * Math.PI / 180); + const next = previous + (2 * Math.PI) / 180; assert.equal(stabilizeScreenRotation(previous, next), next); }); test('screen rotation compares across the wrapped angle boundary', () => { - const previous = Math.PI - (0.1 * Math.PI / 180); - const next = -Math.PI + (0.1 * Math.PI / 180); + const previous = Math.PI - (0.1 * Math.PI) / 180; + const next = -Math.PI + (0.1 * Math.PI) / 180; assert.equal(stabilizeScreenRotation(previous, next), previous); }); @@ -102,10 +109,12 @@ const cruiseCamera = Cesium.Cartesian3.fromDegrees(0, 0, CRUISE_M); * tangent condition is exact here and the expected answer is derived, not * guessed. */ -const tangentLonDeg = Cesium.Math.toDegrees(Math.acos( - Cesium.Ellipsoid.WGS84.maximumRadius - / (Cesium.Ellipsoid.WGS84.maximumRadius + CRUISE_M), -)); +const tangentLonDeg = Cesium.Math.toDegrees( + Math.acos( + Cesium.Ellipsoid.WGS84.maximumRadius / + (Cesium.Ellipsoid.WGS84.maximumRadius + CRUISE_M), + ), +); test('a contact above the silhouette is read against sky', () => { // Another aircraft at the same flight level 200 km ahead. The chord between @@ -124,7 +133,10 @@ test('a contact below the silhouette keeps ground behind it', () => { // the horizon, so the plate has real imagery to separate the text from. const below = Cesium.Cartesian3.fromDegrees(30_000 / 111_320, 0, 2_000); assert.equal(skyBackdropFactor(cruiseCamera, below), 0); - assert.equal(skyBackdropFactor(cruiseCamera, Cesium.Cartesian3.fromDegrees(0.05, 0, 0)), 0); + assert.equal( + skyBackdropFactor(cruiseCamera, Cesium.Cartesian3.fromDegrees(0.05, 0, 0)), + 0, + ); }); test('the silhouette itself lands exactly mid-band', () => { @@ -144,10 +156,12 @@ test('the silhouette itself lands exactly mid-band', () => { function climbSweep(step = 500) { const samples = []; for (let height = 0; height <= 12_000; height += step) { - samples.push(skyBackdropFactor( - cruiseCamera, - Cesium.Cartesian3.fromDegrees(100_000 / 111_320, 0, height), - )); + samples.push( + skyBackdropFactor( + cruiseCamera, + Cesium.Cartesian3.fromDegrees(100_000 / 111_320, 0, height), + ), + ); } return samples; } @@ -163,9 +177,16 @@ test('the band lerps monotonically and saturates on both sides', () => { ); } assert.equal(samples[0], 0, 'well below the horizon is fully grounded'); - assert.equal(samples[samples.length - 1], 1, 'well above the horizon is fully sky'); + assert.equal( + samples[samples.length - 1], + 1, + 'well above the horizon is fully sky', + ); const partial = samples.filter((value) => value > 0 && value < 1); - assert.ok(partial.length >= 3, `expected a real blend band, saw ${partial.length} partial samples`); + assert.ok( + partial.length >= 3, + `expected a real blend band, saw ${partial.length} partial samples`, + ); }); test('the band, not the geometry, is what smooths the crossing', () => { @@ -174,13 +195,19 @@ test('the band, not the geometry, is what smooths the crossing', () => { // if the two ever get conflated, only one of these two tests goes red. const hard = []; for (let height = 0; height <= 12_000; height += 500) { - hard.push(skyBackdropFactor( - cruiseCamera, - Cesium.Cartesian3.fromDegrees(100_000 / 111_320, 0, height), - 0, - )); + hard.push( + skyBackdropFactor( + cruiseCamera, + Cesium.Cartesian3.fromDegrees(100_000 / 111_320, 0, height), + 0, + ), + ); } - assert.deepEqual([...new Set(hard)], [0, 1], 'a zero band must produce only the two end states'); + assert.deepEqual( + [...new Set(hard)], + [0, 1], + 'a zero band must produce only the two end states', + ); }); /** @@ -200,12 +227,19 @@ test('below the ellipsoid, a contact against open sky still reads as sky', () => // ~6.5° above eye level — the whole band is 1.09° per side, so this is not a // near call. Before the fix a sub-ellipsoid camera fell into the degenerate // guard and every one of these came back 0: a row of dark boxes on empty sky. - const approach = Cesium.Cartesian3.fromDegrees(JFK_LON, JFK_LAT + 8_000 * DEG_PER_M_LAT, 900); + const approach = Cesium.Cartesian3.fromDegrees( + JFK_LON, + JFK_LAT + 8_000 * DEG_PER_M_LAT, + 900, + ); assert.equal(skyBackdropFactor(jfkCockpit, approach), 1); // Straight overhead is the unambiguous case, from under the surface as much // as from cruise altitude. assert.equal( - skyBackdropFactor(jfkCockpit, Cesium.Cartesian3.fromDegrees(JFK_LON, JFK_LAT, 10_000)), + skyBackdropFactor( + jfkCockpit, + Cesium.Cartesian3.fromDegrees(JFK_LON, JFK_LAT, 10_000), + ), 1, ); }); @@ -225,7 +259,11 @@ test('below the ellipsoid, a contact on the ramp still keeps its plate', () => { assert.equal( skyBackdropFactor( jfkCockpit, - Cesium.Cartesian3.fromDegrees(JFK_LON, JFK_LAT + 400 * DEG_PER_M_LAT, JFK_RAMP_H - 5), + Cesium.Cartesian3.fromDegrees( + JFK_LON, + JFK_LAT + 400 * DEG_PER_M_LAT, + JFK_RAMP_H - 5, + ), ), 0, ); @@ -246,14 +284,21 @@ test('at the surface the horizon is exactly eye level, geodetic not geocentric', new Cesium.Cartesian3(20_000, 0, 0), new Cesium.Cartesian3(), ); - const alongHorizontal = Cesium.Cartesian3.add(onSurface, east, new Cesium.Cartesian3()); + const alongHorizontal = Cesium.Cartesian3.add( + onSurface, + east, + new Cesium.Cartesian3(), + ); assert.ok( Math.abs(skyBackdropFactor(onSurface, alongHorizontal) - 0.5) < 1e-5, `eye level must be the band centre, got ${skyBackdropFactor(onSurface, alongHorizontal)}`, ); // The geocentric radial is NOT the answer: a contact perpendicular to it // lands measurably off centre, so this test cannot pass by accident. - const radialUp = Cesium.Cartesian3.normalize(onSurface, new Cesium.Cartesian3()); + const radialUp = Cesium.Cartesian3.normalize( + onSurface, + new Cesium.Cartesian3(), + ); const perpToRadial = Cesium.Cartesian3.cross( radialUp, Cesium.Cartesian3.UNIT_Z, @@ -265,7 +310,11 @@ test('at the surface the horizon is exactly eye level, geodetic not geocentric', 20_000, perpToRadial, ); - const alongRadialPlane = Cesium.Cartesian3.add(onSurface, perpToRadial, new Cesium.Cartesian3()); + const alongRadialPlane = Cesium.Cartesian3.add( + onSurface, + perpToRadial, + new Cesium.Cartesian3(), + ); assert.ok( Math.abs(skyBackdropFactor(onSurface, alongRadialPlane) - 0.5) > 0.05, 'a geocentric horizontal would be a different plane; the test must be able to tell', @@ -284,7 +333,9 @@ function surfaceCrossingSweep(from = -50, to = 50, step = 1) { const contact = Cesium.Cartesian3.fromDegrees(30_000 / 111_319.49, 0, 0); const samples = []; for (let height = from; height <= to; height += step) { - samples.push(skyBackdropFactor(Cesium.Cartesian3.fromDegrees(0, 0, height), contact)); + samples.push( + skyBackdropFactor(Cesium.Cartesian3.fromDegrees(0, 0, height), contact), + ); } return samples; } @@ -300,7 +351,10 @@ test('the answer stays continuous as the camera rises through the surface', () = for (let i = 1; i < samples.length; i++) { worst = Math.max(worst, Math.abs(samples[i] - samples[i - 1])); } - assert.ok(worst < 0.05, `no step across the surface may pop; worst adjacent jump ${worst}`); + assert.ok( + worst < 0.05, + `no step across the surface may pop; worst adjacent jump ${worst}`, + ); }); test('the crossover has no seam at the surface itself', () => { @@ -308,9 +362,18 @@ test('the crossover has no seam at the surface itself', () => { // 1 cm is 5.6e-5 rad, a thousandth of the band, so any visible difference // here is a discontinuity in the code rather than in the geometry. const contact = Cesium.Cartesian3.fromDegrees(30_000 / 111_319.49, 0, 0); - const under = skyBackdropFactor(Cesium.Cartesian3.fromDegrees(0, 0, -0.01), contact); - const over = skyBackdropFactor(Cesium.Cartesian3.fromDegrees(0, 0, 0.01), contact); - assert.ok(Math.abs(under - over) < 5e-3, `seam at the surface: ${under} vs ${over}`); + const under = skyBackdropFactor( + Cesium.Cartesian3.fromDegrees(0, 0, -0.01), + contact, + ); + const over = skyBackdropFactor( + Cesium.Cartesian3.fromDegrees(0, 0, 0.01), + contact, + ); + assert.ok( + Math.abs(under - over) < 5e-3, + `seam at the surface: ${under} vs ${over}`, + ); }); test('a sub-surface camera carries real information, not one frozen answer', () => { @@ -319,7 +382,10 @@ test('a sub-surface camera carries real information, not one frozen answer', () // sub-surface camera heights must produce different answers. const belowOnly = surfaceCrossingSweep(-200, -1, 1); const spread = Math.max(...belowOnly) - Math.min(...belowOnly); - assert.ok(spread > 0.05, `a sub-surface camera must still discriminate; spread ${spread}`); + assert.ok( + spread > 0.05, + `a sub-surface camera must still discriminate; spread ${spread}`, + ); assert.ok( belowOnly.every((value) => value > 0 && value < 1), 'this sweep is deliberately mid-band the whole way', @@ -334,7 +400,11 @@ test('a sub-surface camera carries real information, not one frozen answer', () * a future edit to the shared lines could drift the cruise answer while every * sub-surface test stayed green. */ -function legacyAboveSurfaceFactor(cameraPosition, position, featherRad = HORIZON_FEATHER_RAD) { +function legacyAboveSurfaceFactor( + cameraPosition, + position, + featherRad = HORIZON_FEATHER_RAD, +) { const s = Cesium.Ellipsoid.WGS84.oneOverRadii; const cx = cameraPosition.x * s.x; const cy = cameraPosition.y * s.y; @@ -415,22 +485,39 @@ test('below the surface the backdrop test agrees with the occluder beside it', ( const camera = Cesium.Cartesian3.fromDegrees(JFK_LON, JFK_LAT, -18); const occluder = horizonOccluder({ positionWC: camera }); const enu = Cesium.Transforms.eastNorthUpToFixedFrame(camera); - const offsetBy = (up) => Cesium.Cartesian3.add( - camera, - Cesium.Matrix4.multiplyByPointAsVector( - enu, - new Cesium.Cartesian3(0, 20_000, up), + const offsetBy = (up) => + Cesium.Cartesian3.add( + camera, + Cesium.Matrix4.multiplyByPointAsVector( + enu, + new Cesium.Cartesian3(0, 20_000, up), + new Cesium.Cartesian3(), + ), new Cesium.Cartesian3(), - ), - new Cesium.Cartesian3(), - ); + ); // 20 km out and 600 m clear of eye level is well outside the 1.09° band. const aboveEye = offsetBy(600); const belowEye = offsetBy(-600); - assert.equal(occluder.isPointVisible(aboveEye), true, 'the occluder keeps what is above eye level'); - assert.equal(skyBackdropFactor(camera, aboveEye), 1, 'and the backdrop test must call it sky'); - assert.equal(occluder.isPointVisible(belowEye), false, 'the occluder culls what is below it'); - assert.equal(skyBackdropFactor(camera, belowEye), 0, 'and the backdrop test must call it ground'); + assert.equal( + occluder.isPointVisible(aboveEye), + true, + 'the occluder keeps what is above eye level', + ); + assert.equal( + skyBackdropFactor(camera, aboveEye), + 1, + 'and the backdrop test must call it sky', + ); + assert.equal( + occluder.isPointVisible(belowEye), + false, + 'the occluder culls what is below it', + ); + assert.equal( + skyBackdropFactor(camera, belowEye), + 0, + 'and the backdrop test must call it ground', + ); }); test('a degenerate camera or contact reports ground rather than throwing', () => { @@ -442,12 +529,24 @@ test('a degenerate camera or contact reports ground rather than throwing', () => assert.equal(skyBackdropFactor(cruiseCamera, cruiseCamera), 0); // The planet's centre has no local vertical to measure against. assert.equal(skyBackdropFactor(Cesium.Cartesian3.ZERO, cruiseCamera), 0); - assert.equal(skyBackdropFactor(new Cesium.Cartesian3(NaN, NaN, NaN), cruiseCamera), 0); - assert.equal(skyBackdropFactor(cruiseCamera, new Cesium.Cartesian3(NaN, NaN, NaN)), 0); + assert.equal( + skyBackdropFactor(new Cesium.Cartesian3(NaN, NaN, NaN), cruiseCamera), + 0, + ); + assert.equal( + skyBackdropFactor(cruiseCamera, new Cesium.Cartesian3(NaN, NaN, NaN)), + 0, + ); // An infinite coordinate normalizes to NaN a few lines later, and a NaN plate // alpha reaches the canvas as an invisible label rather than a caught error. - assert.equal(skyBackdropFactor(new Cesium.Cartesian3(Infinity, 0, 0), cruiseCamera), 0); - assert.equal(skyBackdropFactor(cruiseCamera, new Cesium.Cartesian3(Infinity, 0, 0)), 0); + assert.equal( + skyBackdropFactor(new Cesium.Cartesian3(Infinity, 0, 0), cruiseCamera), + 0, + ); + assert.equal( + skyBackdropFactor(cruiseCamera, new Cesium.Cartesian3(Infinity, 0, 0)), + 0, + ); }); test('the feather band is a named constant in a sane screen-space range', () => { @@ -456,3 +555,193 @@ test('the feather band is a named constant in a sane screen-space range', () => // band of arc-seconds would pop. assert.ok(HORIZON_FEATHER_RAD > 0.002 && HORIZON_FEATHER_RAD < 0.09); }); + +// --- Perspective projection through a real view-projection -------------------- + +import { + PERSPECTIVE_BLEND_MAX_PX, + PERSPECTIVE_BLEND_MIN_PX, + PERSPECTIVE_PROBE_M, + perspectiveProjectedRotation, +} from './iconOrientation.js'; + +/** + * A scene the exact projector accepts: 3D mode, a canvas, and a camera with a + * real view matrix and perspective frustum. The camera sits `altitudeM` above + * the origin and looks north, pitched down by `pitchDeg`, with world east as + * screen right — the oblique street-level pose where the camera-basis answer + * is wrong off-centre and the exact one is not. + */ +function obliqueScene( + altitudeM, + pitchDeg, + { width = 1280, height = 800 } = {}, +) { + const position = Cesium.Cartesian3.fromDegrees(0, 0, altitudeM); + const enu = Cesium.Transforms.eastNorthUpToFixedFrame(position); + const east = Cesium.Matrix4.getColumn(enu, 0, new Cesium.Cartesian3()); + const north = Cesium.Matrix4.getColumn(enu, 1, new Cesium.Cartesian3()); + const up = Cesium.Matrix4.getColumn(enu, 2, new Cesium.Cartesian3()); + const pitch = Cesium.Math.toRadians(pitchDeg); + const direction = Cesium.Cartesian3.normalize( + Cesium.Cartesian3.add( + Cesium.Cartesian3.multiplyByScalar( + north, + Math.cos(pitch), + new Cesium.Cartesian3(), + ), + Cesium.Cartesian3.multiplyByScalar( + up, + -Math.sin(pitch), + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ); + const cameraUp = Cesium.Cartesian3.normalize( + Cesium.Cartesian3.add( + Cesium.Cartesian3.multiplyByScalar( + north, + Math.sin(pitch), + new Cesium.Cartesian3(), + ), + Cesium.Cartesian3.multiplyByScalar( + up, + Math.cos(pitch), + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ); + const viewMatrix = Cesium.Matrix4.computeView( + position, + direction, + cameraUp, + east, + new Cesium.Matrix4(), + ); + const frustum = new Cesium.PerspectiveFrustum({ + fov: Cesium.Math.toRadians(60), + aspectRatio: width / height, + near: 1, + far: 1e7, + }); + return { + frameState: { mode: Cesium.SceneMode.SCENE3D }, + canvas: { clientWidth: width, clientHeight: height }, + camera: { + viewMatrix, + frustum, + rightWC: east, + upWC: cameraUp, + positionWC: position, + }, + }; +} + +/** Probe separation (px) the helper sees for a contact and course in a scene. */ +function probeSeparation(scene, contact, courseDeg) { + const courseRad = Cesium.Math.toRadians(courseDeg); + const enu = Cesium.Transforms.eastNorthUpToFixedFrame(contact); + const forward = Cesium.Matrix4.multiplyByPointAsVector( + enu, + new Cesium.Cartesian3( + Math.sin(courseRad) * PERSPECTIVE_PROBE_M, + Math.cos(courseRad) * PERSPECTIVE_PROBE_M, + 0, + ), + new Cesium.Cartesian3(), + ); + const probe = Cesium.Cartesian3.add( + contact, + forward, + new Cesium.Cartesian3(), + ); + const at = Cesium.SceneTransforms.worldToWindowCoordinates(scene, contact); + const ahead = Cesium.SceneTransforms.worldToWindowCoordinates(scene, probe); + const dx = ahead.x - at.x; + const dy = ahead.y - at.y; + return { separation: Math.hypot(dx, dy), exact: Math.atan2(-dx, -dy) }; +} + +test('the perspective helper is continuous through the sub-pixel band, not a 23° flip', () => { + // Codex's reproduction: a contact near the right edge of an oblique view, + // the camera altitude walked so the probe separation crosses half a pixel. + // The rotation jumped 23° → 0° between two adjacent altitudes. The exact + // and camera-basis answers now blend across a band, the short way round. + // + // The contact keeps the SAME screen position across the walk — its ground + // offsets scale with the altitude — so the two answers disagree by the + // same twenty-odd degrees at every altitude while the separation alone + // walks from well above the band to well below it. + const course = 0; // northbound + const contactFor = (altitude) => + Cesium.Cartesian3.fromDegrees( + (0.7 * altitude) / 111_320, + (1.2 * altitude) / 111_320, + 0, + ); + const samples = []; + // Geometric steps: separation scales with altitude, so equal ratios give + // equal-sized steps in the quantity the band is defined on. + for (let altitude = 500; altitude <= 600_000; altitude *= 1.02) { + const scene = obliqueScene(altitude, 45); + const contact = contactFor(altitude); + if (!Cesium.SceneTransforms.worldToWindowCoordinates(scene, contact)) + continue; + const { separation, exact } = probeSeparation(scene, contact, course); + samples.push({ + altitude, + separation, + exact, + rotation: perspectiveProjectedRotation(scene, contact, course, null), + basis: screenProjectedRotation(scene, contact, course, null), + }); + } + const inBand = samples.filter( + (s) => + s.separation > PERSPECTIVE_BLEND_MIN_PX && + s.separation < PERSPECTIVE_BLEND_MAX_PX, + ); + const above = samples.filter((s) => s.separation >= PERSPECTIVE_BLEND_MAX_PX); + const below = samples.filter((s) => s.separation <= PERSPECTIVE_BLEND_MIN_PX); + assert.ok( + inBand.length >= 5 && above.length >= 5 && below.length >= 5, + `the walk crosses the whole band (${below.length}/${inBand.length}/${above.length})`, + ); + + // Off-centre and oblique, the two answers genuinely disagree — that is the + // divergence the exact projection exists to fix. + const disagreement = Math.abs( + wrappedDelta(above[0].rotation, above[0].basis), + ); + assert.ok( + disagreement > Cesium.Math.toRadians(5), + `exact and camera-basis differ by ${Cesium.Math.toDegrees(disagreement).toFixed(1)}° above the band`, + ); + + let worstStep = 0; + for (let i = 1; i < samples.length; i += 1) { + worstStep = Math.max( + worstStep, + Math.abs(wrappedDelta(samples[i].rotation, samples[i - 1].rotation)), + ); + } + assert.ok( + worstStep < Cesium.Math.toRadians(3), + `no step between neighbouring altitudes exceeds 3° (worst ${Cesium.Math.toDegrees(worstStep).toFixed(2)}°)`, + ); + // Below the band it is the camera-basis answer; above it the exact one, unblended. + for (const s of below) + assert.ok( + Math.abs(wrappedDelta(s.rotation, s.basis)) < 1e-6, + `basis alone at ${s.separation.toFixed(2)} px`, + ); + for (const s of above) + assert.ok( + Math.abs(wrappedDelta(s.rotation, s.exact)) < 1e-6, + `exact alone at ${s.separation.toFixed(2)} px`, + ); +}); diff --git a/src/data/layerState.js b/src/data/layerState.js index 4d380db27..abad5d64c 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -293,6 +293,7 @@ export const LAYER_STATE_REGISTRY = Object.freeze([ Object.freeze({ id: 'satellites', token: 's', disposition: 'enabled+options', optionOwner: 'satellites' }), Object.freeze({ id: 'telegeography-submarine-cables', token: 'u', disposition: 'enabled-only' }), Object.freeze({ id: 'traffic', token: 't', disposition: 'enabled-only' }), + Object.freeze({ id: 'transit', token: 'j', disposition: 'enabled-only' }), ]); export const REGISTERED_LAYER_IDS = Object.freeze(LAYER_STATE_REGISTRY.map((entry) => entry.id)); diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index f985eb71a..cd6f3924e 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -158,8 +158,8 @@ function encode(state) { test('production registry is exact, canonical, and rejects incomplete contracts', async () => { assert.equal(validateLayerStateRegistry(), true); - assert.equal(REGISTERED_LAYER_IDS.length, 18); - assert.equal(new Set(REGISTERED_LAYER_IDS).size, 18); + assert.equal(REGISTERED_LAYER_IDS.length, 19); + assert.equal(new Set(REGISTERED_LAYER_IDS).size, 19); assert.deepEqual(REGISTERED_LAYER_IDS, [...REGISTERED_LAYER_IDS].sort()); assert.throws( () => validateLayerStateRegistry([...LAYER_STATE_REGISTRY, LAYER_STATE_REGISTRY[0]]), diff --git a/src/data/spriteOrder.js b/src/data/spriteOrder.js index aaabe14a5..3f1ff899e 100644 --- a/src/data/spriteOrder.js +++ b/src/data/spriteOrder.js @@ -3,6 +3,8 @@ export const SPRITE_LAYER_ORDER = Object.freeze([ 'cctv', 'firms', 'bikeshare', + 'transit', + 'transit-motion', 'directions', 'ais', 'military', diff --git a/src/data/transit.js b/src/data/transit.js new file mode 100644 index 000000000..1d33d4ae7 --- /dev/null +++ b/src/data/transit.js @@ -0,0 +1,56 @@ +import { createTransitLayer } from '../layers/transit/index.js'; +import * as render from '../renderGovernor.js'; +import * as sprites from './spriteOrder.js'; +import * as picking from './pickRegistry.js'; +import * as overlays from '../overlays/worldOverlay.js'; +import { registerDynamicCredit, transitFeedCredit } from './dataCredits.js'; +import { + GROUND_FLOOR_LIFT_M, + cachedGroundFloor, + coarseFloorCoord, + neighborFloorM, + warmGroundFloor, +} from './groundFloor.js'; +import { sampleMeshFloorCells } from './meshFloorSampler.js'; + +const layer = createTransitLayer({ + services: { + render, + sprites, + picking, + overlays, + credits: { registerDynamicCredit, transitFeedCredit }, + ground: { + GROUND_FLOOR_LIFT_M, + cachedGroundFloor, + coarseFloorCoord, + neighborFloorM, + warmGroundFloor, + }, + // The legacy sampler over the same legacy floor, so this standalone + // module warms its own mesh floors the way the application layer does. + mesh: { sampleMeshFloorCells }, + }, +}); + +export const _setTransitOverlayHostForTest = + layer._setTransitOverlayHostForTest; +export const _transitStateForTest = layer._transitStateForTest; +export { + TRANSIT_MODE_COLORS, + TRANSIT_POLL_MS, + TRANSIT_SELECTED_OVERLAY_SOURCE_ID, + TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS, + aggregateTransitFeedHealth, + buildTransitSelectionCopy, + createTransitSelectedOverlayEntry, + interpolatedVehiclePosition, + isStaleVehicleFix, + transitDetectionClass, + transitDetectionId, + transitDetectionMetric, + transitVehicleKey, + vehicleFixAgeMs, + createTransitLayer, +} from '../layers/transit/index.js'; +export default layer; diff --git a/src/data/transit.test.mjs b/src/data/transit.test.mjs new file mode 100644 index 000000000..902f5f3be --- /dev/null +++ b/src/data/transit.test.mjs @@ -0,0 +1,225 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as Cesium from 'cesium'; +import transitLayer, { + TRANSIT_MODE_COLORS, + TRANSIT_POLL_MS, + TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS, + buildTransitSelectionCopy, + createTransitSelectedOverlayEntry, + interpolatedVehiclePosition, + isStaleVehicleFix, + transitVehicleKey, +} from './transit.js'; +import { TRANSIT_MODES, getTransitFeed } from './transitFeeds.js'; +import { LAYER_STATE_REGISTRY } from './layerState.js'; +import { SPRITE_LAYER_ORDER } from './spriteOrder.js'; + +test('layer module declares the manager contract', () => { + assert.equal(transitLayer.id, 'transit'); + assert.equal(typeof transitLayer.name, 'string'); + assert.equal(typeof transitLayer.icon, 'string'); + assert.equal(transitLayer.updateInterval, TRANSIT_POLL_MS); + for (const method of [ + 'init', + 'enable', + 'disable', + 'update', + 'getStats', + 'destroy', + ]) { + assert.equal( + typeof transitLayer[method], + 'function', + `${method} is implemented`, + ); + } +}); + +test('the layer is registered for share links and sprite stacking', () => { + const entry = LAYER_STATE_REGISTRY.find((row) => row.id === 'transit'); + assert.ok(entry, 'transit has a share-link token'); + assert.equal(entry.disposition, 'enabled-only'); + assert.equal( + LAYER_STATE_REGISTRY.filter((row) => row.token === entry.token).length, + 1, + 'token is unique', + ); + const index = SPRITE_LAYER_ORDER.indexOf('transit'); + assert.ok( + index > SPRITE_LAYER_ORDER.indexOf('bikeshare'), + 'vehicles draw above bikeshare stations', + ); + assert.ok( + index < SPRITE_LAYER_ORDER.indexOf('flights'), + 'aircraft stay on top', + ); +}); + +test('every transit mode has a colour and the selected card uses it as accent', () => { + for (const mode of TRANSIT_MODES) { + assert.match( + TRANSIT_MODE_COLORS[mode], + /^#[0-9a-f]{6}$/i, + `${mode} has a colour`, + ); + } + const position = Cesium.Cartesian3.fromDegrees(-71.06, 42.36, 3); + const card = createTransitSelectedOverlayEntry( + 'mbta:1', + position, + { title: 'T', details: ['d'] }, + 'subway', + ); + assert.equal(card.accent, TRANSIT_MODE_COLORS.subway); + assert.equal(card.selected, true); + assert.equal(card.protected, true); + assert.equal(card.position, position); + assert.equal( + createTransitSelectedOverlayEntry( + '', + position, + { title: 'T', details: [] }, + 'bus', + ), + null, + ); + assert.equal( + createTransitSelectedOverlayEntry( + 'k', + null, + { title: 'T', details: [] }, + 'bus', + ), + null, + ); + assert.equal( + TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS.moving, + true, + 'the card follows a moving vehicle', + ); +}); + +test('position queries read the published display sample', () => { + const entry = { track: {}, sample: { lat: 1, lon: 2, phase: 'playing' } }; + assert.deepEqual(interpolatedVehiclePosition(entry), { + lat: 1, + lon: 2, + settled: false, + }); + entry.sample.phase = 'held'; + assert.equal(interpolatedVehiclePosition(entry).settled, true); +}); +test('fixes older than ten minutes are stale; feeds without timestamps are trusted', () => { + const now = 1_700_000_000_000; + assert.equal(isStaleVehicleFix({ timestamp: now / 1000 - 30 }, now), false); + assert.equal(isStaleVehicleFix({ timestamp: now / 1000 - 601 }, now), true); + assert.equal(isStaleVehicleFix({ timestamp: null }, now), false); + assert.equal(isStaleVehicleFix({}, now), false); +}); + +test('selection copy reads like a transit card and never leaks nulls', () => { + const feed = getTransitFeed('metrotransit-msp'); + const now = 1_788_936_960_000; + const full = buildTransitSelectionCopy( + feed, + { + id: '1557', + label: '1557', + routeId: '17', + lat: 44.9, + lon: -93.4, + bearing: 248, + speedMps: 11.2, + timestamp: 1_788_936_945, + stopId: '57458', + status: 'STOPPED_AT', + occupancy: 'FEW_SEATS_AVAILABLE', + timestampSource: 'vehicle', + }, + 'bus', + now, + ); + assert.equal(full.title, '🚌 Route 17'); + assert.deepEqual(full.details, [ + // The mode is said in words: a glyph and a route number do not tell a + // reader whether the dot on the street is a bus or a subway train. + 'Bus · Metro Transit · Minneapolis–St Paul, MN', + 'Reported 15 s ago', + // The operator's speed is labelled as a report: it describes the newest + // fix, not the older segment the screen is drawing. + 'reported 40 km/h · reported hdg 248°', + // The operator's claim is kept as a REPORT, not as the live state. It + // describes the moment of the report, which the display may not have + // reached yet — presenting it as "now" is what put STOPPED on a metro + // gliding across the owner's screen. + 'Last report: stopped at stop 57458 · few seats available', + 'Vehicle 1557', + ]); + // A time the feed gave for the SNAPSHOT rather than for this vehicle is + // labelled, so the card never presents a borrowed time as the bus's own. + const borrowed = buildTransitSelectionCopy( + feed, + { + id: '9', + lat: 1, + lon: 1, + timestamp: 1_788_936_900, + timestampSource: 'header', + }, + 'bus', + now, + ); + assert.equal(borrowed.details[1], 'Reported 60 s ago (feed time)'); + + // A subway says where it is drawn, because a train on a road is otherwise + // read as a bug rather than as a deliberate surface projection. + const underground = buildTransitSelectionCopy( + feed, + { + id: 'T1', + routeId: 'Blue', + lat: 44.9, + lon: -93.4, + timestamp: 1_788_936_945, + }, + 'subway', + now, + ); + assert.deepEqual(underground.details, [ + 'Subway · Metro Transit · Minneapolis–St Paul, MN', + 'Reported 15 s ago', + 'Shown at street level · depth not in the feed', + 'Vehicle T1', + ]); + // No timestamp at all is aged from the fetch, never from "now". + const sparse = buildTransitSelectionCopy( + feed, + { id: 'abc', lat: 1, lon: 1 }, + 'rail', + now, + now - 120_000, + ); + assert.equal(sparse.title, '🚆 Vehicle abc'); + assert.deepEqual(sparse.details, [ + 'Train · Metro Transit · Minneapolis–St Paul, MN', + 'Reported 2 min ago (feed time)', + 'Vehicle abc', + ]); + for (const line of [...full.details, ...sparse.details]) + assert.doesNotMatch(line, /null|undefined|NaN/); + assert.equal(transitVehicleKey('mbta', '17'), 'mbta:17'); +}); + +test('the layer accepts a manager handle for out-of-tick panel repaints', () => { + assert.equal(typeof transitLayer.attachDataManager, 'function'); + transitLayer.attachDataManager({ refreshLayerStats() {} }); + transitLayer.attachDataManager(null); +}); + +test('stats before enable are an honest zero, not a fake feed state', () => { + const stats = transitLayer.getStats(); + assert.equal(stats.count, 0); + assert.equal(stats.source, 'GTFS-RT'); + assert.equal(stats.error, null); +}); diff --git a/src/data/transitFeeds.js b/src/data/transitFeeds.js new file mode 100644 index 000000000..2858efa62 --- /dev/null +++ b/src/data/transitFeeds.js @@ -0,0 +1,404 @@ +/** + * @module transitFeeds + * @description Registry of keyless, openly licensed GTFS-Realtime + * VehiclePositions feeds the Transit layer can show. + * + * Every entry here is a URL the SERVER fetches — the browser only ever asks + * `/api/transit/vehicles/` for a registered id (see SECURITY.md: proxies + * never fetch client-supplied URLs). Adding a feed means adding a row here, + * a DATA_SOURCES.md row with its license, and a credit in dataCredits.js. + * + * Admission rules for a feed: + * - No key, token, or registration required (identify-yourself headers are + * fine — Entur asks for `ET-Client-Name`, OVapi for a User-Agent). + * - An open license that permits display with attribution. + * - Real coordinates in VehiclePosition.position — NYCT subway, for example, + * publishes stop-relative positions only and is deliberately absent. + * + * `defaultEnabled` is separate from being registered. A feed ships switched ON + * only when its operator's own published terms were READ and plainly cover this + * use. A registered feed with `defaultEnabled: false` is never polled and never + * offered to the browser; it is here so the decision is visible and reversible + * in one line. `terms` records the passage that decision rests on, so the next + * reader does not have to take the licence string on faith. + * + * Pure data + pure helpers: imported by the browser layer, the Vite proxy, and + * node:test. No Cesium, no Node built-ins. + */ + +/** Transit modes the layer colors. `routeMode` hints refine a feed's default. */ +export const TRANSIT_MODES = Object.freeze([ + 'bus', + 'tram', + 'subway', + 'rail', + 'ferry', + 'unknown', +]); + +/** Per-mode icon used in labels and the selection card. */ +export const TRANSIT_MODE_ICON = Object.freeze({ + bus: '🚌', + tram: '🚊', + subway: '🚇', + rail: '🚆', + ferry: '⛴️', + unknown: '🚏', +}); + +/** + * MBTA route ids are human-readable and mode-typed: rapid-transit lines carry + * colour names, commuter rail is prefixed `CR-`, ferries `Boat-`, buses are + * numeric (or `SL`/`CT` express families). + * @param {string|null} routeId + * @returns {string} + */ +function mbtaRouteMode(routeId) { + if (!routeId) return 'unknown'; + if (/^(Red|Orange|Blue)\b/.test(routeId)) return 'subway'; + if (/^(Green|Mattapan)/.test(routeId)) return 'tram'; + if (/^CR-/.test(routeId)) return 'rail'; + if (/^Boat-/.test(routeId)) return 'ferry'; + if (/^Shuttle/i.test(routeId)) return 'bus'; + return 'bus'; +} + +/** + * Entur (Norway) route ids are `:Line:`; the codespace + * tells the operator, not the mode, but a few are single-mode operators. + * @param {string|null} routeId + * @returns {string} + */ +function enturRouteMode(routeId) { + if (!routeId) return 'unknown'; + const codespace = routeId.split(':')[0]; + if ( + codespace === 'VYG' || + codespace === 'GJB' || + codespace === 'SJN' || + codespace === 'FLT' || + codespace === 'GOA' || + codespace === 'NSB' || + codespace === 'VYT' + ) + return 'rail'; + if (codespace === 'FLB') return 'rail'; + return 'bus'; +} + +/** + * HSL route ids start with a four-digit code whose first digit is the mode + * family in HSL's numbering: 1xxx/2xxx… are trams (1001–1010) for 4-digit ids + * beginning with `10`, metro routes are `31M…`, ferries `1019`. + * @param {string|null} routeId + * @returns {string} + */ +function hslRouteMode(routeId) { + if (!routeId) return 'unknown'; + if (/^31M/.test(routeId)) return 'subway'; + if (/^10(0[1-9]|10|15)/.test(routeId)) return 'tram'; + if (/^1019/.test(routeId)) return 'ferry'; + if (/^300[0-9A-Z]/.test(routeId)) return 'rail'; + return 'bus'; +} + +/** + * Metro Transit (Minneapolis–St Paul): light rail is the Blue/Green line + * (route ids 901/902), Northstar commuter rail is 888. + * @param {string|null} routeId + * @returns {string} + */ +function metroTransitRouteMode(routeId) { + if (!routeId) return 'unknown'; + if (routeId === '901' || routeId === '902') return 'tram'; + if (routeId === '888') return 'rail'; + return 'bus'; +} + +/** + * Registry of feeds. Order is presentation order in the stats/credit text. + * `loadRadiusKm` is the distance from `center` inside which the feed is polled. + * @type {ReadonlyArray, + * license: string, licenseUrl: string, attribution: string, + * defaultMode: string, routeMode?: (routeId: string|null) => string, + * }>>} + */ +export const TRANSIT_FEED_REGISTRY = Object.freeze([ + Object.freeze({ + id: 'mbta', + historyRetention: true, + name: 'MBTA', + operator: 'Massachusetts Bay Transportation Authority', + region: 'Boston, MA', + center: Object.freeze({ lat: 42.3601, lon: -71.0589 }), + loadRadiusKm: 70, + url: 'https://cdn.mbta.com/realtime/VehiclePositions.pb', + license: 'MassDOT Developers License Agreement', + licenseUrl: + 'https://cdn.mbta.com/sites/default/files/2023-08/mbta-massdot-develop-license-agreement.pdf', + attribution: 'MBTA / MassDOT', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'MassDOT ... hereby grants You (Licensee) non-exclusive, limited, and revocable rights to use, reproduce, and redistribute the Data. ... Clearly acknowledge MassDOT as the provider of the Data.', + note: 'Credit is text only: the agreement forbids using MBTA/MassDOT logos or trademarks with the data.', + }), + defaultMode: 'bus', + routeMode: mbtaRouteMode, + }), + Object.freeze({ + id: 'capmetro-austin', + name: 'CapMetro', + operator: 'Capital Metropolitan Transportation Authority', + region: 'Austin, TX', + center: Object.freeze({ lat: 30.2672, lon: -97.7431 }), + loadRadiusKm: 60, + url: 'https://data.texas.gov/download/eiei-9rpf/application%2Foctet-stream', + license: 'CapMetro Developer Tools license', + licenseUrl: 'https://www.capmetro.org/developertools', + attribution: + 'Capital Metropolitan Transportation Authority — data.texas.gov', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'Capital Metropolitan Transportation Authority (CMTA) hereby grants you (Licensee) non-exclusive, limited and revocable rights to use, reproduce, and redistribute CMTA Data.', + note: 'CMTA trademarks may not be used in association with the data, so the credit is the operator name as text.', + }), + defaultMode: 'bus', + }), + Object.freeze({ + id: 'metrotransit-msp', + name: 'Metro Transit', + operator: 'Metro Transit (Metropolitan Council)', + region: 'Minneapolis–St Paul, MN', + center: Object.freeze({ lat: 44.9778, lon: -93.265 }), + loadRadiusKm: 70, + url: 'https://svc.metrotransit.org/mtgtfs/vehiclepositions.pb', + license: 'Public Metro Transit vehicle position data', + licenseUrl: 'https://svc.metrotransit.org/', + attribution: 'Metro Transit — Metropolitan Council', + defaultEnabled: true, + terms: Object.freeze({ + quote: + "Metro Transit's GTFS-realtime feeds are refreshed every 5 seconds.", + note: 'The realtime feed is published from a developer page carrying operational guidance and no licence text of its own. The public-domain statement below belongs to the Metropolitan Council metadata for the COMPANION SCHEDULE dataset, not to this feed: "None. This dataset is public domain under the Minnesota Government Data Practices Act". Credited as a courtesy, the way the other public agency feeds are.', + }), + defaultMode: 'bus', + routeMode: metroTransitRouteMode, + }), + Object.freeze({ + id: 'hsl-helsinki', + name: 'HSL', + operator: 'Helsinki Region Transport (HSL)', + region: 'Helsinki, Finland', + center: Object.freeze({ lat: 60.1699, lon: 24.9384 }), + loadRadiusKm: 70, + url: 'https://realtime.hsl.fi/realtime/vehicle-positions/v2/hsl', + license: 'CC BY 4.0', + licenseUrl: 'https://www.hsl.fi/en/hsl/open-data', + attribution: 'HSL (Helsinki Region Transport)', + defaultEnabled: true, + terms: Object.freeze({ + quote: null, + note: 'The open-data page refuses automated readers (HTTP 403, bot protection), so this entry rests on the owner opening it and accepting it rather than on a passage quoted here. HSL publishes its open data under CC BY 4.0 and asks to be credited as the source; the feed needs no key.', + }), + defaultMode: 'bus', + routeMode: hslRouteMode, + }), + Object.freeze({ + id: 'ovapi-nl', + name: 'OVapi', + operator: 'Stichting OpenGeo (NDOV data)', + region: 'Netherlands', + center: Object.freeze({ lat: 52.2, lon: 5.3 }), + loadRadiusKm: 220, + url: 'https://gtfs.ovapi.nl/nl/vehiclePositions.pb', + license: 'Free to use per the OVapi README (best effort, no SLA)', + licenseUrl: 'https://gtfs.ovapi.nl/README', + attribution: + 'OVapi / Stichting OpenGeo — Dutch integrated real-time transit data', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'You are free to use this data, but there is no service level agreement (best-effort) nor are you allowed to say you represent or impersonate any of the transit agencies listed here.', + note: 'The README asks consumers to identify themselves in the User-Agent, to send If-Modified-Since / If-None-Match when polling faster than once a minute, and to accept gzip. The proxy does all three.', + }), + defaultMode: 'bus', + }), + Object.freeze({ + id: 'entur-norway', + name: 'Entur', + operator: 'Entur AS (Norwegian national transit data)', + region: 'Norway', + // Circle chosen to hold Oslo, Bergen, Bodø and Tromsø while leaving + // Helsinki (≈820 km) out — a national feed must not poll from next door. + center: Object.freeze({ lat: 64.0, lon: 11.5 }), + loadRadiusKm: 720, + url: 'https://api.entur.io/realtime/v1/gtfs-rt/vehicle-positions', + headers: Object.freeze({ 'ET-Client-Name': 'gods-eye-view-transit' }), + license: 'Norwegian Licence for Open Government Data (NLOD)', + licenseUrl: 'https://developer.entur.org/pages-intro-authentication', + attribution: 'Entur — data under NLOD', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'This API is open under NLOD licence, however, it is required that all consumers identify themselves by using the header ET-Client-Name.', + note: 'The header is mandatory — unidentified consumers may be rate-limited or blocked — and the proxy sends it on every request.', + }), + defaultMode: 'bus', + routeMode: enturRouteMode, + }), + Object.freeze({ + id: 'translink-seq', + name: 'TransLink', + operator: 'TransLink (Queensland Government)', + region: 'South East Queensland, Australia', + center: Object.freeze({ lat: -27.4698, lon: 153.0251 }), + loadRadiusKm: 150, + url: 'https://gtfsrt.api.translink.com.au/api/realtime/seq/VehiclePositions', + license: 'CC BY 4.0', + licenseUrl: 'https://translink.com.au/about-translink/open-data', + attribution: 'TransLink — Queensland Government (CC BY 4.0)', + defaultEnabled: true, + terms: Object.freeze({ + quote: + 'Our data is licensed under a Creative Commons Attribution 4.0 International License. ... You must not represent that the State in any way endorses your Application.', + note: 'No key, no stated rate limit, no caching rule. Logos and network imagery need separate approval, so the credit is text only.', + }), + defaultMode: 'bus', + }), +]); + +/** + * The feeds this build actually polls. A registered feed that is not here is + * inert everywhere: it resolves to no route, is never offered to the browser, + * and never appears in a coverage check. + */ +export const TRANSIT_ENABLED_FEEDS = Object.freeze( + TRANSIT_FEED_REGISTRY.filter((feed) => feed.defaultEnabled === true), +); + +const FEED_BY_ID = new Map( + TRANSIT_ENABLED_FEEDS.map((feed) => [feed.id, feed]), +); +const ANY_FEED_BY_ID = new Map( + TRANSIT_FEED_REGISTRY.map((feed) => [feed.id, feed]), +); + +/** Feed ids are path segments: lowercase letters, digits, hyphens only. */ +export const TRANSIT_FEED_ID_PATTERN = /^[a-z0-9][a-z0-9-]{1,63}$/; + +/** + * Look up a POLLABLE feed by id. Unknown ids, malformed ids, and feeds that are + * registered but switched off all return null — this is the only door from a + * request path to an upstream URL, so a feed the owner has not cleared must not + * be reachable through it. + * @param {string} id + * @returns {object|null} + */ +export function getTransitFeed(id) { + if (typeof id !== 'string' || !TRANSIT_FEED_ID_PATTERN.test(id)) return null; + return FEED_BY_ID.get(id) || null; +} + +/** + * Look up any registered feed, enabled or not. For credits and documentation + * only — never for resolving a request into an upstream fetch. + * @param {string} id + * @returns {object|null} + */ +export function getRegisteredTransitFeed(id) { + if (typeof id !== 'string' || !TRANSIT_FEED_ID_PATTERN.test(id)) return null; + return ANY_FEED_BY_ID.get(id) || null; +} + +/** + * Great-circle distance in km. + * @param {number} aLat + * @param {number} aLon + * @param {number} bLat + * @param {number} bLon + * @returns {number} + */ +export function haversineKm(aLat, aLon, bLat, bLon) { + const toRad = (deg) => (deg * Math.PI) / 180; + const dLat = toRad(bLat - aLat); + const dLon = toRad(bLon - aLon); + const h = + Math.sin(dLat / 2) ** 2 + + Math.cos(toRad(aLat)) * Math.cos(toRad(bLat)) * Math.sin(dLon / 2) ** 2; + return 2 * 6371 * Math.asin(Math.min(1, Math.sqrt(h))); +} + +/** + * Feeds whose coverage circle contains the point, nearest first. + * @param {number} lat + * @param {number} lon + * @param {number} [slackKm=0] Extra radius (hysteresis) so a feed at the edge + * of coverage does not flap on small camera moves. + * @returns {object[]} + */ +export function transitFeedsInRange(lat, lon, slackKm = 0) { + if (!Number.isFinite(lat) || !Number.isFinite(lon)) return []; + return TRANSIT_ENABLED_FEEDS.map((feed) => ({ + feed, + km: haversineKm(lat, lon, feed.center.lat, feed.center.lon), + })) + .filter(({ feed, km }) => km <= feed.loadRadiusKm + Math.max(0, slackKm)) + .sort((a, b) => a.km - b.km) + .map(({ feed }) => feed); +} + +/** + * Mode for a vehicle: the feed's route hint when it has one, else its default. + * @param {object} feed Registry entry. + * @param {string|null} routeId GTFS route_id from the vehicle's trip. + * @returns {string} One of TRANSIT_MODES. + */ +export function transitModeFor(feed, routeId) { + const hinted = + typeof feed?.routeMode === 'function' ? feed.routeMode(routeId) : null; + const mode = + hinted && hinted !== 'unknown' ? hinted : feed?.defaultMode || 'unknown'; + return TRANSIT_MODES.includes(mode) ? mode : 'unknown'; +} + +/** + * Whether the mode came from the route id rather than the feed's default. A + * default is a guess about a fleet — TransLink's is "bus" while it publishes + * rail in the same feed — so a consumer judging physical plausibility must + * not hold a defaulted vehicle to a bus's limits. + * @param {object} feed Registry entry. + * @param {string|null} routeId + * @returns {boolean} + */ +export function transitModeResolved(feed, routeId) { + const hinted = + typeof feed?.routeMode === 'function' ? feed.routeMode(routeId) : null; + return ( + Boolean(hinted) && hinted !== 'unknown' && TRANSIT_MODES.includes(hinted) + ); +} + +/** + * Public catalog shape served by `/api/transit/feeds` — everything the browser + * needs to gate polling and credit the source, and nothing it could misuse. + * @returns {object[]} + */ +export function publicTransitCatalog() { + return TRANSIT_ENABLED_FEEDS.map((feed) => ({ + id: feed.id, + name: feed.name, + operator: feed.operator, + region: feed.region, + center: { ...feed.center }, + loadRadiusKm: feed.loadRadiusKm, + license: feed.license, + licenseUrl: feed.licenseUrl, + attribution: feed.attribution, + historyRetention: feed.historyRetention === true, + })); +} diff --git a/src/data/transitFeeds.test.mjs b/src/data/transitFeeds.test.mjs new file mode 100644 index 000000000..d0e38524f --- /dev/null +++ b/src/data/transitFeeds.test.mjs @@ -0,0 +1,241 @@ +import { readFileSync } from 'node:fs'; +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + TRANSIT_ENABLED_FEEDS, + TRANSIT_FEED_REGISTRY, + TRANSIT_FEED_ID_PATTERN, + TRANSIT_MODES, + getRegisteredTransitFeed, + getTransitFeed, + haversineKm, + publicTransitCatalog, + transitFeedsInRange, + transitModeFor, + transitModeResolved, +} from './transitFeeds.js'; + +test('every registered feed is keyless, https, licensed, and uniquely identified', () => { + const ids = new Set(); + for (const feed of TRANSIT_FEED_REGISTRY) { + assert.match( + feed.id, + TRANSIT_FEED_ID_PATTERN, + `${feed.id} is a valid path segment`, + ); + assert.equal(ids.has(feed.id), false, `${feed.id} is unique`); + ids.add(feed.id); + const url = new URL(feed.url); + assert.equal(url.protocol, 'https:', `${feed.id} fetches over https`); + assert.equal( + url.search.includes('key='), + false, + `${feed.id} carries no key in its URL`, + ); + assert.ok( + feed.license && feed.licenseUrl && feed.attribution, + `${feed.id} names its license`, + ); + assert.ok( + feed.loadRadiusKm > 0 && feed.loadRadiusKm <= 1000, + `${feed.id} radius is sane`, + ); + assert.ok( + Math.abs(feed.center.lat) <= 90 && Math.abs(feed.center.lon) <= 180, + ); + assert.ok( + TRANSIT_MODES.includes(feed.defaultMode), + `${feed.id} default mode is known`, + ); + assert.ok(Object.isFrozen(feed), `${feed.id} is immutable`); + } +}); + +test('getTransitFeed is the only door to an upstream URL and refuses anything unregistered', () => { + assert.equal( + getTransitFeed('mbta')?.url, + 'https://cdn.mbta.com/realtime/VehiclePositions.pb', + ); + assert.equal(getTransitFeed('MBTA'), null); + assert.equal(getTransitFeed('../etc/passwd'), null); + assert.equal(getTransitFeed('https://evil.example'), null); + assert.equal(getTransitFeed(''), null); + assert.equal(getTransitFeed(null), null); + assert.equal(getTransitFeed(42), null); +}); + +test('haversine matches known city distances', () => { + const bostonToNyc = haversineKm(42.3601, -71.0589, 40.7128, -74.006); + assert.ok( + Math.abs(bostonToNyc - 306) < 5, + `Boston–NYC ≈ 306 km, got ${bostonToNyc}`, + ); + assert.equal(haversineKm(0, 0, 0, 0), 0); +}); + +test('feeds in range are nearest-first and honor slack as hysteresis', () => { + // Camera over Cambridge, MA → MBTA only. + const boston = transitFeedsInRange(42.37, -71.11); + assert.deepEqual( + boston.map((f) => f.id), + ['mbta'], + ); + // Mid-Atlantic: nothing. + assert.deepEqual(transitFeedsInRange(40, -40), []); + // Just outside MBTA's 70 km circle (≈ 80 km south) — out without slack, in with 20 km slack. + const farLat = 42.3601 - 80 / 111; + assert.deepEqual(transitFeedsInRange(farLat, -71.0589), []); + assert.deepEqual( + transitFeedsInRange(farLat, -71.0589, 20).map((f) => f.id), + ['mbta'], + ); + // Bad input never throws. + assert.deepEqual(transitFeedsInRange(NaN, 1), []); + // A national feed covers its own cities and not the neighbour's capital. + for (const [city, lat, lon] of [ + ['Oslo', 59.91, 10.75], + ['Bergen', 60.39, 5.32], + ['Tromsø', 69.65, 18.96], + ]) { + assert.ok( + transitFeedsInRange(lat, lon).some((f) => f.id === 'entur-norway'), + `${city} is covered by Entur`, + ); + } + assert.deepEqual( + transitFeedsInRange(60.17, 24.94).map((f) => f.id), + ['hsl-helsinki'], + 'Helsinki polls HSL only', + ); +}); + +test('route hints refine a feed default and never escape the known modes', () => { + const mbta = getTransitFeed('mbta'); + assert.equal(transitModeFor(mbta, 'Red'), 'subway'); + assert.equal(transitModeFor(mbta, 'Green-B'), 'tram'); + assert.equal(transitModeFor(mbta, 'CR-Fitchburg'), 'rail'); + assert.equal(transitModeFor(mbta, 'Boat-F1'), 'ferry'); + assert.equal(transitModeFor(mbta, '66'), 'bus'); + assert.equal(transitModeFor(mbta, null), 'bus'); + const hsl = getTransitFeed('hsl-helsinki'); + assert.equal(transitModeFor(hsl, '31M1'), 'subway'); + assert.equal(transitModeFor(hsl, '1006'), 'tram'); + assert.equal(transitModeFor(hsl, '9982'), 'bus'); + const msp = getTransitFeed('metrotransit-msp'); + assert.equal(transitModeFor(msp, '901'), 'tram'); + assert.equal(transitModeFor(msp, '17'), 'bus'); + const entur = getTransitFeed('entur-norway'); + assert.equal(transitModeFor(entur, 'VYG:Line:R10'), 'rail'); + assert.equal(transitModeFor(entur, 'TRO:Line:1_310'), 'bus'); + assert.equal(transitModeFor({ defaultMode: 'spaceship' }, 'x'), 'unknown'); + assert.equal(transitModeFor(null, 'x'), 'unknown'); +}); + +test('the public catalog exposes coverage and credit, never the upstream URL or headers', () => { + const catalog = publicTransitCatalog(); + assert.equal(catalog.length, TRANSIT_ENABLED_FEEDS.length); + for (const entry of catalog) { + assert.equal('url' in entry, false); + assert.equal('headers' in entry, false); + assert.ok(entry.id && entry.name && entry.region && entry.attribution); + assert.ok( + Number.isFinite(entry.center.lat) && Number.isFinite(entry.loadRadiusKm), + ); + } +}); + +test('the enabled set is a gate on the registry, not a copy of it', () => { + // Every registered feed is switched on today. The gate still has to be the + // thing that decides that, because the moment an operator's terms change the + // owner flips one flag and expects the feed to become unreachable — not to + // stay routable because the filter had quietly become a no-op. + assert.deepEqual( + TRANSIT_ENABLED_FEEDS.map((feed) => feed.id), + TRANSIT_FEED_REGISTRY.filter((feed) => feed.defaultEnabled === true).map( + (feed) => feed.id, + ), + ); + for (const feed of TRANSIT_FEED_REGISTRY) { + assert.equal( + typeof feed.defaultEnabled, + 'boolean', + `${feed.id} states whether it ships on`, + ); + assert.equal( + Boolean(getTransitFeed(feed.id)), + feed.defaultEnabled, + `${feed.id} is routable exactly when it is enabled`, + ); + assert.ok(getRegisteredTransitFeed(feed.id), `${feed.id} stays documented`); + } +}); + +test('a mode the route id established is resolved; a feed default is only a guess', () => { + // TransLink publishes rail in a feed whose default is bus. A consumer that + // judges physical plausibility must know the difference, or a 140 km/h + // train is refused as an impossible bus. + const translink = getTransitFeed('translink-seq'); + assert.equal(transitModeFor(translink, '600'), 'bus'); + assert.equal(transitModeResolved(translink, '600'), false, 'defaulted'); + const mbta = getTransitFeed('mbta'); + assert.equal(transitModeResolved(mbta, 'Red'), true); + assert.equal( + transitModeResolved(mbta, '66'), + true, + 'a numeric MBTA route is a bus by rule, not by default', + ); + assert.equal(transitModeResolved(mbta, null), false); + assert.equal(transitModeResolved({ defaultMode: 'bus' }, 'x'), false); + assert.equal( + transitModeResolved({ routeMode: () => 'spaceship' }, 'x'), + false, + 'an unknown hint resolves nothing', + ); +}); + +test('only MBTA retains proxy history while every other registered feed stays live-enabled', () => { + assert.deepEqual( + publicTransitCatalog() + .filter((feed) => feed.historyRetention) + .map((feed) => feed.id), + ['mbta'], + ); + assert.equal(getTransitFeed('capmetro-austin').defaultEnabled, true); + assert.equal(getTransitFeed('mbta').attribution, 'MBTA / MassDOT'); +}); + +test('published transit history scope agrees with the catalog capability', () => { + const retained = publicTransitCatalog().filter( + (feed) => feed.historyRetention, + ); + assert.deepEqual( + retained.map((feed) => feed.id), + ['mbta'], + ); + const sources = readFileSync( + new URL('../../DATA_SOURCES.md', import.meta.url), + 'utf8', + ); + const row = sources + .split('\n') + .find((line) => line.startsWith('| **MBTA / MassDOT**')); + assert.match( + row, + /live vehicles and up to 15 minutes of recently observed positions/, + ); + assert.match(row, /MBTA \/ MassDOT.*courtesy/); + assert.match(sources, /other feeds have no proxy retention/); + const state = readFileSync( + new URL('../../docs/CURRENT-STATE.md', import.meta.url), + 'utf8', + ); + const transit = state.slice( + state.indexOf('Transit is off by default'), + state.indexOf('`src/data/militaryAwareness.js` remains'), + ); + assert.match(transit, /Restart clears it/); + assert.doesNotMatch( + transit, + /four fixes|0\.95x|guaranteed lower|one poll interval behind/, + ); +}); diff --git a/src/data/transitIcons.js b/src/data/transitIcons.js new file mode 100644 index 000000000..6d5d5066f --- /dev/null +++ b/src/data/transitIcons.js @@ -0,0 +1,214 @@ +import { + presetSpriteScale, + presetSpriteOutlinePx, + transitStyleProfile, +} from './transitPresetStyle.js'; +// src/data/transitIcons.js +/** + * Front-up transit-vehicle silhouettes, one per mode, as SVG data URIs for + * Cesium billboards. + * + * Same house language as `aircraftIcons.js`: a 96-unit viewBox with the glyph + * centred at (48,48), drawn nose toward -Y so the screen-projected rotation in + * `iconOrientation.js` can spin the whole glyph, filled WHITE with a dark + * hairline edge so the tint pipeline (billboard.color = the mode colour, plus + * `.withAlpha` fades) keeps working. No glyph carries a colour of its own; a + * hardcoded fill would fight the tint that tells a person what they are looking + * at. + * + * These are plan views, the way a vehicle reads from a camera above a street — + * not side elevations. Each mode has a DISTINCT outline so it is recognisable + * at the ~18 px the fleet renders at, by shape and not only by colour: a bus is + * short and blunt, a tram is long and slender with articulation joints, a metro + * car is squared off, a mainline train has a tapered nose, and a ferry is a + * hull with a bow. + * + * Interior detail is drawn with the same dark hairline as the edge — windscreen + * lines, articulation joints, a deck line — because a solid dark panel would + * darken the tint and a white-on-white panel would vanish. + */ + +const VIEW = 96; +const C = VIEW / 2; // 48 — glyph centre + +/** Hairline dark edge, matching the aircraft glyph weight at this box size. */ +const STROKE = + 'stroke="rgba(0,0,0,0.34)" stroke-width="1.5" stroke-linejoin="round"'; +/** Interior panel lines: the same ink, thinner, capped so joints read cleanly. */ +const DETAIL = + 'stroke="rgba(0,0,0,0.30)" stroke-width="1.5" stroke-linecap="round" fill="none"'; + +/** + * Each body is drawn in a centred frame (origin 0,0 = glyph centre), front + * toward -Y, in the 96-unit space (half-extent up to ~45). + */ +const BODIES = { + // ── Bus: short, blunt, square-shouldered. The shape everything else reads + // against, and the one most people will see most often. + bus: ` + + + `, + + // ── Tram / light rail: longer and slimmer than a bus, with the two + // articulation joints that make a modern low-floor tram unmistakable. + tram: ` + + + + `, + + // ── Metro / subway: squared off at both ends, wider than a tram, a single + // cab line at the front. Reads as rolling stock, not a road vehicle. + subway: ` + + + + `, + + // ── Mainline / commuter rail: the tapered nose of an intercity set, longer + // than anything else in the set. + rail: ` + + + `, + + // ── Ferry: a hull. Pointed bow, square stern, superstructure amidships — + // the only glyph in the set that is not a box on rails or rubber. + ferry: ` + + `, + + // ── Unknown: a plain rounded marker. Deliberately featureless, so an + // unclassified vehicle never pretends to be a mode it is not. + unknown: ` + `, +}; + +/** The modes this module can draw. */ +export const TRANSIT_ICON_KINDS = Object.freeze(Object.keys(BODIES)); + +const _iconCache = new Map(); + +const _b64 = (s) => + typeof btoa === 'function' + ? btoa(s) + : Buffer.from(s, 'utf8').toString('base64'); + +/** + * Fleet raster size. Same reasoning as the aircraft fleet glyphs: Cesium's + * billboard atlas has no mipmaps, so a texture far larger than its on-screen + * footprint is GPU-minified into mush. Transit billboards render around 14–20 + * CSS px, so 48 px of source covers the band on a Retina display with very + * little minification. + */ +const FLEET_RASTER_PX = 48; +/** The selected vehicle draws bigger and deserves a crisper source. */ +export const SELECTED_ICON_PX = 96; + +/** + * On-screen size the fleet glyph is authored for. A halo width asked for in + * screen pixels is converted to viewBox units against this, so "two pixels of + * ring" means two pixels on an eighteen-pixel billboard. + */ +export const FLEET_SCREEN_PX = 20; + +/** + * How a halo changes the glyph's frame. A thick outer stroke would clip at + * the 96-unit box (the tram already reaches 42.5 of 48), so the viewBox is + * padded by the stroke and the billboard has to grow by the same ratio to + * keep the body its shipped size on screen. + * + * @param {number} haloScreenPx Ring width in final screen pixels (0 = none). + * @param {number} [screenPx] Billboard size the ring is specified against. + * @returns {{units: number, pad: number, ratio: number}} Stroke width in + * viewBox units (the visible ring is half of it, outside the body), the + * padding added on every side, and the frame growth factor to apply to the + * billboard's width and height. + */ +export function haloFrame(haloScreenPx, screenPx = FLEET_SCREEN_PX) { + const ring = Number.isFinite(haloScreenPx) ? Math.max(0, haloScreenPx) : 0; + if (ring === 0) return { units: 0, pad: 0, ratio: 1 }; + // Half the stroke lies outside the body, so a ring of R px needs 2R of + // stroke. One screen pixel is VIEW/screenPx units. + const units = (2 * ring * VIEW) / Math.max(1, screenPx); + const pad = Math.ceil(units / 2) + 1; + return { units, pad, ratio: (VIEW + 2 * pad) / VIEW }; +} + +/** Number of raster variants ever built — a bound a test can hold the cache to. */ +export function transitIconCacheSize() { + return _iconCache.size; +} + +/** + * Data URI for one mode's silhouette, lazily built and cached per kind, size + * and halo. + * + * With a halo, the body is drawn twice: first as a wide dark stroke, then as + * the shipped white fill with its hairline edge on top, so the ring sits + * OUTSIDE the silhouette and the interior detail is untouched. The dark ring + * is what survives NVG's auto-gain and FLIR's luma mapping when the bright + * core alone would merge with a bright road. + * + * @param {string} kind One of TRANSIT_ICON_KINDS; anything else draws `unknown`. + * @param {number} [px] Raster size. + * @param {{haloScreenPx?: number, screenPx?: number}} [options] + * @returns {string} `data:image/svg+xml;base64,...` + */ +export function transitIcon(kind, px = FLEET_RASTER_PX, options = {}) { + const k = BODIES[kind] ? kind : 'unknown'; + const selected = px > FLEET_RASTER_PX; + px = selected ? SELECTED_ICON_PX : FLEET_RASTER_PX; + const profile = transitStyleProfile( + options.style || + (options.haloScreenPx >= 2 + ? 'thermal' + : options.haloScreenPx === 1.25 + ? 'retro' + : 'normal'), + ); + const style = + profile === 'mono' ? 'thermal' : profile === 'crt' ? 'retro' : 'normal'; + const displayPx = (selected ? 30 : 20) * presetSpriteScale(style, selected); + const frame = haloFrame(presetSpriteOutlinePx(style, selected), displayPx); + const key = `${k}:${selected ? 1 : 0}:${profile}`; + let uri = _iconCache.get(key); + if (!uri) { + const size = VIEW + 2 * frame.pad; + // Sensor signatures use a broad rounded envelope, without dark interior + // joints or a second outline eating into the hot core at display resolution. + const halfW = k === 'bus' || k === 'ferry' || k === 'unknown' ? 48 : 47; + const halfH = 48; + const radius = k === 'subway' ? 8 : k === 'rail' ? 12 : 14; + const capsule = ``; + const body = profile === 'mono' ? capsule : BODIES[k]; + const halo = + frame.units > 0 + ? body.replace( + //, + (_m, d) => + ``, + ) + : ''; + const svg = + `` + + `${halo}${body}`; + uri = 'data:image/svg+xml;base64,' + _b64(svg); + _iconCache.set(key, uri); + } + return uri; +} diff --git a/src/data/transitIcons.test.mjs b/src/data/transitIcons.test.mjs new file mode 100644 index 000000000..17cfb20c4 --- /dev/null +++ b/src/data/transitIcons.test.mjs @@ -0,0 +1,136 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + FLEET_SCREEN_PX, + TRANSIT_ICON_KINDS, + haloFrame, + transitIcon, + transitIconCacheSize, +} from './transitIcons.js'; + +const decode = (uri) => + Buffer.from(uri.replace('data:image/svg+xml;base64,', ''), 'base64').toString( + 'utf8', + ); + +test('a halo is sized in screen pixels and pads the frame so it cannot clip', () => { + const none = haloFrame(0); + assert.deepEqual(none, { units: 0, pad: 0, ratio: 1 }); + const two = haloFrame(2, FLEET_SCREEN_PX); + // Two pixels of ring on an eighteen-pixel glyph is a 4-px stroke: 21.3 of 96 units. + assert.ok(Math.abs(two.units - (4 * 96) / 20) < 1e-9); + assert.ok( + two.pad >= two.units / 2, + 'padding covers the half-stroke outside the body', + ); + assert.ok(two.ratio > 1, 'and the billboard grows by the same ratio'); +}); + +test('a haloed glyph draws the dark ring under the shipped white body', () => { + const plain = decode(transitIcon('bus')); + const haloed = decode(transitIcon('bus', 48, { haloScreenPx: 2 })); + assert.notEqual(plain, haloed); + assert.match( + haloed, + /stroke="#000000" stroke-opacity="1" stroke-width="19\.20"/, + ); + assert.ok( + haloed.indexOf('#000000') < haloed.indexOf('fill="white"'), + 'ring first, body on top', + ); + assert.match(haloed, /viewBox="-\d+ -\d+ \d+ \d+"/, 'padded frame'); + assert.match(plain, /stroke-width="9\.60"/, 'normal has a one-pixel halo'); +}); + +test('the raster cache is bounded by kind, size and halo — never by call count', () => { + const before = transitIconCacheSize(); + for (let i = 0; i < 50; i += 1) { + transitIcon('tram', 48, { haloScreenPx: 2 }); + transitIcon('tram', 48); + transitIcon('tram', 128, { haloScreenPx: 1.25 }); + } + assert.ok( + transitIconCacheSize() - before <= 3, + `three variants, not ${transitIconCacheSize() - before}`, + ); + assert.equal( + transitIcon('tram', 48, { haloScreenPx: 2 }), + transitIcon('tram', 48, { haloScreenPx: 2 }), + ); + for (const kind of TRANSIT_ICON_KINDS) + transitIcon(kind, 48, { haloScreenPx: 2 }); + assert.ok(transitIconCacheSize() <= before + 3 + TRANSIT_ICON_KINDS.length); +}); + +test('all 36 variants retain the specified final halo at both display sizes', async () => { + const { presetSpriteScale, presetSpriteOutlinePx } = + await import('./transitPresetStyle.js'); + for (const kind of TRANSIT_ICON_KINDS) { + for (const selected of [false, true]) { + for (const style of ['normal', 'thermal', 'retro']) { + const display = + (selected ? 30 : 20) * presetSpriteScale(style, selected); + const svg = decode(transitIcon(kind, selected ? 96 : 48, { style })); + const units = Number( + svg.match(/stroke-opacity="(?:0.95|1)" stroke-width="([\d.]+)"/)[1], + ); + assert.ok( + Math.abs( + (units * display) / 192 - presetSpriteOutlinePx(style, selected), + ) < 0.001, + ); + const frame = haloFrame( + presetSpriteOutlinePx(style, selected), + display, + ); + assert.match( + svg, + new RegExp( + `width="${Math.round((selected ? 96 : 48) * frame.ratio)}"`, + ), + ); + } + } + } + for (let i = 0; i < 100; i++) + transitIcon('invalid', i, { haloScreenPx: i, screenPx: i }); + assert.equal(transitIconCacheSize(), 36); +}); + +test('sensor bodies cover sixty percent of the padded display with opaque white', async () => { + const sharp = (await import('sharp')).default; + const { presetSpriteScale, presetSpriteOutlinePx } = + await import('./transitPresetStyle.js'); + for (const selected of [false, true]) + for (const kind of TRANSIT_ICON_KINDS) { + const display = + (selected ? 30 : 20) * presetSpriteScale('thermal', selected); + assert.equal(display, selected ? 36 : 30); + assert.equal(presetSpriteOutlinePx('thermal', selected), 3); + const svg = decode( + transitIcon(kind, selected ? 96 : 48, { style: 'thermal' }), + ); + const frame = haloFrame(3, display); + const width = Math.round(display * frame.ratio); + const { data, info } = await sharp(Buffer.from(svg)) + .resize(width, width) + .ensureAlpha() + .raw() + .toBuffer({ resolveWithObject: true }); + let white = 0; + for (let i = 0; i < data.length; i += 4) + if ( + data[i] >= 240 && + data[i + 1] >= 240 && + data[i + 2] >= 240 && + data[i + 3] >= 240 + ) + white++; + assert.ok( + white / (info.width * info.height) >= 0.6, + `${kind}/${display}: white ${white / (info.width * info.height)}`, + ); + const c = (Math.floor(width / 2) * width + Math.floor(width / 2)) * 4; + assert.deepEqual([...data.subarray(c, c + 4)], [255, 255, 255, 255]); + } +}); diff --git a/src/data/transitPresetStyle.js b/src/data/transitPresetStyle.js new file mode 100644 index 000000000..4848924f4 --- /dev/null +++ b/src/data/transitPresetStyle.js @@ -0,0 +1,147 @@ +/** + * @file Preset-aware transit sprite styling — pure lookup tables mapping the + * active post-FX style to a sprite treatment the shaders cannot destroy, and + * the detection tier that carries each mode's colour above them. + * + * Why: transit sprites are in-scene billboards, so they pass THROUGH the + * post-FX chain. NVG and FLIR reduce the scene to luminance, and the mode + * palette's luma is middling — a green bus, a red metro and an amber tram all + * collapse to the same dim grey, on a road NVG's auto-gain has pushed to + * white. The owner's Boston test: "you can barely see these guys". + * + * Encoding per profile, the traffic-dot precedent applied to a glyph: + * - `mono` (surveillance/NVG, thermal/FLIR, noir, cockpit nvg/thermal): the + * sprite is WHITE — the hottest end of grayscale FLIR and of the Ironbow + * ramp, and the brightest phosphor under NVG — drawn larger, with a dark + * halo sized in SCREEN pixels so it keeps local contrast on a bright road + * through any luma mapping. Black-hot FLIR inverts it, deliberately: a + * black sprite with a light ring is what a hot object looks like there. + * - `crt` (retro): saturated per-mode colours that survive posterisation, + * a size boost to out-shout the pixel grid, and a thinner halo. + * - `normal` (normal/anime/snow/unknown): every lookup returns the shipped + * value — mode colour, shipped size, hairline edge only. + * + * Brackets are a different matter. The detection canvas composites ABOVE the + * post-FX chain, so a tier colour is literal screen RGB in every preset: the + * brackets carry the mode colour the sprites can no longer encode, the way + * traffic's brackets carry congestion. Tier colours per theme live in + * `src/overlays/worldOverlayTokens.js`. + * + * Cesium-free so the tables are unit-testable; the layer maps them to Cesium + * colours and billboard sizes at spawn and restyle time. + * + * @module data/transitPresetStyle + */ + +/** @const {Object} Style name → non-normal profile. */ +const PROFILE_BY_STYLE = Object.freeze({ + surveillance: 'mono', // NVG — P43 phosphor × luma + thermal: 'mono', // FLIR — grayscale/Ironbow × luma + noir: 'mono', // full desaturation + nvg: 'mono', // cockpit vision names its modes directly + retro: 'crt', // CRT — hue survives, small glyphs don't +}); + +/** + * Per-profile sprite treatment. `rgba` is [r,g,b] 0–255 + alpha 0–1; + * `scale` multiplies the shipped billboard size; `outlinePx` is the dark + * halo's width in FINAL SCREEN PIXELS, which is the only unit that means + * anything for a glyph rendered at eighteen pixels — the shipped 1.5-unit + * hairline in a 96-unit box is 0.28 px on screen, and vanishes. + */ +const SPRITE_STYLE = Object.freeze({ + mono: Object.freeze({ + rgba: Object.freeze([255, 255, 255, 1]), + scale: 1.5, + outlinePx: 3, + }), + crt: Object.freeze({ + scale: 1.3, + outlinePx: 1.25, + rgbaByMode: Object.freeze({ + bus: Object.freeze([0, 255, 102, 1]), + tram: Object.freeze([255, 179, 0, 1]), + subway: Object.freeze([255, 59, 48, 1]), + rail: Object.freeze([215, 139, 255, 1]), + ferry: Object.freeze([58, 208, 255, 1]), + unknown: Object.freeze([255, 255, 255, 1]), + }), + }), +}); + +/** Modes with a bracket tier of their own. */ +const TIER_MODES = Object.freeze([ + 'bus', + 'tram', + 'subway', + 'rail', + 'ferry', + 'unknown', +]); + +/** + * Classify a style name into a transit styling profile. + * @param {string|null|undefined} styleName - Active style (e.g. 'thermal'). + * @returns {'normal'|'mono'|'crt'} Styling profile; unknown → 'normal'. + */ +export function transitStyleProfile(styleName) { + return PROFILE_BY_STYLE[styleName] || 'normal'; +} + +/** + * Preset sprite colour, or null to keep the shipped mode colour. + * @param {string|null|undefined} styleName + * @param {string} mode Transit mode. + * @returns {number[]|null} [r,g,b,a] with rgb 0–255, alpha 0–1, or null. + */ +export function presetSpriteRgba(styleName, mode) { + const profile = transitStyleProfile(styleName); + if (profile === 'mono') return SPRITE_STYLE.mono.rgba; + if (profile === 'crt') { + return ( + SPRITE_STYLE.crt.rgbaByMode[mode] || SPRITE_STYLE.crt.rgbaByMode.unknown + ); + } + return null; +} + +/** + * Multiplier a preset applies to the shipped billboard size. + * @param {string|null|undefined} styleName + * @returns {number} 1 under the normal profile. + */ +export function presetSpriteScale(styleName, selected = false) { + if (transitStyleProfile(styleName) === 'mono') + return selected ? 36 / 30 : 30 / 20; + return SPRITE_STYLE[transitStyleProfile(styleName)]?.scale ?? 1; +} + +/** + * Dark-halo width in final screen pixels, or 0 for the shipped hairline only. + * @param {string|null|undefined} styleName + * @returns {number} + */ +export function presetSpriteOutlinePx(styleName, selected = false) { + return ( + SPRITE_STYLE[transitStyleProfile(styleName)]?.outlinePx ?? + (selected ? 1.5 : 1) + ); +} + +/** + * Detection-overlay tier key for a transit mode. Every theme's `tiers` + * table carries these six, so the bracket colour is per mode in every + * preset. Unlike traffic, transit sets its tier keyless as well: the feeds + * are keyless, and there is no separate keyless experience to keep + * byte-identical. + * @param {string} mode + * @returns {string} + */ +export function transitModeTier(mode) { + return `transit_${TIER_MODES.includes(mode) ? mode : 'unknown'}`; +} + +/** The tier keys this module can produce, for the tables that must carry them. */ +export const TRANSIT_TIER_KEYS = Object.freeze( + TIER_MODES.map((mode) => `transit_${mode}`), +); diff --git a/src/data/transitPresetStyle.test.mjs b/src/data/transitPresetStyle.test.mjs new file mode 100644 index 000000000..db9f413a8 --- /dev/null +++ b/src/data/transitPresetStyle.test.mjs @@ -0,0 +1,96 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + TRANSIT_TIER_KEYS, + presetSpriteOutlinePx, + presetSpriteRgba, + presetSpriteScale, + transitModeTier, + transitStyleProfile, +} from './transitPresetStyle.js'; +import { TRANSIT_MODES } from './transitFeeds.js'; + +test('profile mapping: NVG/FLIR/noir and cockpit nvg are mono, retro is crt, everything else normal', () => { + for (const mono of ['surveillance', 'thermal', 'noir', 'nvg']) + assert.equal(transitStyleProfile(mono), 'mono', mono); + assert.equal(transitStyleProfile('retro'), 'crt'); + for (const plain of [ + 'normal', + 'anime', + 'snow', + 'minecraft', + '', + null, + undefined, + 'optical', + ]) + assert.equal(transitStyleProfile(plain), 'normal', String(plain)); +}); + +test('normal uses mode colour, unit scale and a one-pixel halo', () => { + for (const mode of TRANSIT_MODES) { + assert.equal(presetSpriteRgba('normal', mode), null); + assert.equal(presetSpriteRgba(undefined, mode), null); + } + assert.equal(presetSpriteScale('normal'), 1); + assert.equal(presetSpriteOutlinePx('normal'), 1); +}); + +test('mono: every mode is a white core, larger, with a dark halo in screen pixels', () => { + for (const style of ['surveillance', 'thermal', 'noir', 'nvg']) { + for (const mode of TRANSIT_MODES) { + assert.deepEqual( + presetSpriteRgba(style, mode), + [255, 255, 255, 1], + `${style} ${mode}`, + ); + } + assert.ok(presetSpriteScale(style) === 1.5, `${style} is drawn larger`); + // At least a whole screen pixel and a half of ring around a five-pixel + // body: thinner than that vanishes through blur and pixelation. + assert.ok( + presetSpriteOutlinePx(style) >= 1.5, + `${style} halo is ${presetSpriteOutlinePx(style)} px`, + ); + } +}); + +test('crt: saturated per-mode colours, distinct, upsized, thinner halo', () => { + const seen = new Set(); + for (const mode of TRANSIT_MODES) { + const rgba = presetSpriteRgba('retro', mode); + assert.ok(Array.isArray(rgba) && rgba.length === 4, mode); + seen.add(rgba.slice(0, 3).join(',')); + const [r, g, b] = rgba; + assert.ok( + Math.max(r, g, b) === 255, + `${mode} is fully saturated on one channel`, + ); + } + assert.equal(seen.size, TRANSIT_MODES.length, 'no two modes share a colour'); + assert.ok(presetSpriteScale('retro') > 1.2); + assert.ok( + presetSpriteOutlinePx('retro') > 0 && + presetSpriteOutlinePx('retro') < presetSpriteOutlinePx('thermal'), + ); +}); + +test('every mode has its own bracket tier, unknown modes fall to transit_unknown', () => { + const tiers = new Set(); + for (const mode of TRANSIT_MODES) tiers.add(transitModeTier(mode)); + assert.equal(tiers.size, TRANSIT_MODES.length); + assert.equal(transitModeTier('spaceship'), 'transit_unknown'); + assert.equal(transitModeTier(undefined), 'transit_unknown'); + assert.deepEqual([...tiers].sort(), [...TRANSIT_TIER_KEYS].sort()); +}); + +test('all styled tuples are valid rgba: channels 0–255 ints, alpha in (0, 1]', () => { + for (const style of ['thermal', 'retro']) { + for (const mode of TRANSIT_MODES) { + const [r, g, b, a] = presetSpriteRgba(style, mode); + for (const c of [r, g, b]) + assert.ok(Number.isInteger(c) && c >= 0 && c <= 255); + assert.ok(a > 0 && a <= 1); + } + } +}); diff --git a/src/data/transitProxy.js b/src/data/transitProxy.js new file mode 100644 index 000000000..b27ae2358 --- /dev/null +++ b/src/data/transitProxy.js @@ -0,0 +1,401 @@ +/** + * @module transitProxy + * @description Pure server-side mechanics for the `/api/transit` proxy. + * + * Kept free of Vite/Node middleware state (the terrainHeightsProxy pattern) + * so path resolution, snapshot shaping, and the cache/stale policy can be + * exercised by the offline node:test suite. The middleware in vite.config.js + * only does I/O: fetch the registered upstream, hand the bytes here, send. + */ + +import { + GTFS_INCREMENTALITY_FULL_DATASET, + decodeVehiclePositions, +} from './gtfsRealtime.js'; +import { getTransitFeed } from './transitFeeds.js'; + +/** Fresh window: a snapshot younger than this is served without refetching. */ +export const TRANSIT_PROXY_TTL_MS = 15_000; +/** Serve-stale window: after an upstream failure, a snapshot this old still ships (marked stale). */ +export const TRANSIT_PROXY_STALE_MAX_MS = 10 * 60_000; +/** Upstream fetch timeout. National feeds (Entur ≈ 1.4 MB) need headroom. */ +export const TRANSIT_PROXY_TIMEOUT_MS = 15_000; +/** Hard cap on upstream bytes: the largest registered feed is well under 1 MB. */ +export const TRANSIT_PROXY_MAX_BODY_BYTES = 8 * 1024 * 1024; +/** Redirect hops followed before a feed is declared unreachable. */ +export const TRANSIT_MAX_REDIRECTS = 3; +/** + * The 3xx codes that actually mean "go somewhere else". 304 is NOT one of them: + * it is the successful answer to a conditional request and carries no Location, + * so treating the whole 3xx range as a redirect turns every healthy unchanged + * feed into a failure and walks it up the backoff ladder. + */ +export const TRANSIT_REDIRECT_STATUSES = Object.freeze([ + 301, 302, 303, 307, 308, +]); + +/** + * Whether an upstream status is a redirect this proxy should follow. + * @param {number} status + * @returns {boolean} + */ +export function isTransitRedirectStatus(status) { + return TRANSIT_REDIRECT_STATUSES.includes(Number(status)); +} +/** + * Admission window for upstream requests. One browser polls a feed every 15 s + * (4/min); the allowance leaves room for a second tab and a manual reload, and + * the global figure bounds what this process can ask of ALL operators at once. + */ +export const TRANSIT_ADMISSION_WINDOW_MS = 60_000; +export const TRANSIT_ADMISSION_MAX_PER_FEED = 8; +export const TRANSIT_ADMISSION_MAX_GLOBAL = 40; +/** + * Cooldown ladder after consecutive upstream failures, in ms. A cold feed that + * is simply down must not be re-asked once per poll for the whole session: the + * first retry is quick, the last rung is five minutes, and the ladder resets on + * the first success. + */ +export const TRANSIT_BACKOFF_LADDER_MS = Object.freeze([ + 5_000, 15_000, 60_000, 300_000, +]); + +/** + * Resolve `/vehicles/` (the path after the `/api/transit` mount) to a + * registered feed. Anything else — a different route, an unknown id, path + * tricks, a query string — resolves to null and the caller 404s. + * @param {string} url Request URL relative to the mount point. + * @returns {{ route: 'feeds' } | { route: 'vehicles', feed: object } | null} + */ +/** Validate the entire decoded identifier, including UTF-8 length. */ +export function validTransitIdentifier(value) { + return ( + typeof value === 'string' && + value.length > 0 && + !/[\u0000-\u001f\u007f/\\]/u.test(value) && + value !== '.' && + value !== '..' && + new TextEncoder().encode(value).length <= 256 && + !/[\uD800-\uDFFF]/u.test(value) + ); +} +export function resolveTransitRoute(url) { + const pathname = String(url || '').split('?')[0]; + if (pathname === '/feeds' || pathname === '/feeds/') + return { route: 'feeds' }; + const match = /^\/(vehicles|trail)\/([^/]+)(?:\/([^/]+))?\/?$/.exec(pathname); + if (!match) return null; + let id, vehicleId; + try { + id = decodeURIComponent(match[2]); + vehicleId = match[3] === undefined ? null : decodeURIComponent(match[3]); + } catch { + return null; + } + const feed = getTransitFeed(id); + if (!feed) return null; + if (match[1] === 'vehicles') + return vehicleId === null ? { route: 'vehicles', feed } : null; + return feed.historyRetention === true && validTransitIdentifier(vehicleId) + ? { route: 'trail', feed, vehicleId } + : null; +} + +/** Selection reads are bounded and cancellable through body parsing. */ +export async function fetchTransitHistory( + feedId, + vehicleId, + signal, + fetchImpl = fetch, +) { + if (!validTransitIdentifier(vehicleId)) + throw new TypeError('Invalid transit vehicle identifier'); + const response = await fetchImpl( + `/api/transit/trail/${encodeURIComponent(feedId)}/${encodeURIComponent(vehicleId)}`, + { signal, headers: { Accept: 'application/json' } }, + ); + if (!response.ok) throw new Error(`Transit history HTTP ${response.status}`); + const reader = response.body?.getReader?.(); + let text = ''; + if (reader) { + const decoder = new TextDecoder(); + let bytes = 0; + try { + while (true) { + signal?.throwIfAborted(); + const chunk = await reader.read(); + if (chunk.done) break; + bytes += chunk.value.byteLength; + if (bytes > 32768) + throw new RangeError('Transit history exceeds response limit'); + text += decoder.decode(chunk.value, { stream: true }); + } + text += decoder.decode(); + } finally { + await reader.cancel().catch(() => {}); + } + } else { + text = await response.text(); + if (new TextEncoder().encode(text).length > 32768) + throw new RangeError('Transit history exceeds response limit'); + } + signal?.throwIfAborted(); + const payload = JSON.parse(text); + if ( + payload.version !== 1 || + payload.feedId !== feedId || + payload.vehicleId !== vehicleId || + !Array.isArray(payload.fixes) || + payload.fixes.length > 128 || + !Array.isArray(payload.epochs) || + payload.epochs.length > 8 + ) + throw new TypeError('Invalid transit history response'); + let previous = -Infinity; + const epochs = new Set(); + for (const epoch of payload.epochs) { + if ( + !Number.isInteger(epoch.id) || + epoch.id < 1 || + epoch.id > 0xffffffff || + epochs.has(epoch.id) || + !['trip', 'route', 'mode'].every( + (key) => + typeof epoch[key] === 'string' && + new TextEncoder().encode(epoch[key]).length <= 256, + ) + ) + throw new TypeError('Invalid transit history epoch'); + epochs.add(epoch.id); + } + for (const fix of payload.fixes) { + if ( + !Array.isArray(fix) || + fix.length !== 5 || + !fix.every(Number.isFinite) || + fix[0] < 0 || + fix[0] <= previous || + Math.abs(fix[1]) > 90 || + Math.abs(fix[2]) > 180 || + !Number.isInteger(fix[3]) || + !epochs.has(fix[4]) + ) + throw new TypeError('Invalid transit history fix'); + previous = fix[0]; + } + return payload; +} + +/** + * Request headers for one upstream fetch. + * + * Three of these are obligations, not politeness. Feeds that ask consumers to + * identify themselves get their header from the registry (Entur requires + * `ET-Client-Name`; OVapi asks that the User-Agent say who you are). OVapi also + * asks anyone polling faster than once a minute to send conditional-request + * validators and to accept gzip, so both are sent to every feed: harmless + * where it is not asked for, and it spares each operator a full body whenever + * nothing has changed. + * + * @param {object} feed Registry entry. + * @param {{etag?: string|null, lastModified?: string|null}} [validators] From the cached snapshot. + * @returns {Record} + */ +export function transitUpstreamHeaders(feed, validators = null) { + return { + 'User-Agent': + 'gods-eye-view-transit-proxy/1.0 (+https://github.com/bilawalsidhu/gods-eye-view)', + Accept: 'application/x-protobuf, application/octet-stream;q=0.9, */*;q=0.1', + 'Accept-Encoding': 'gzip', + ...(validators?.etag ? { 'If-None-Match': validators.etag } : {}), + ...(validators?.lastModified + ? { 'If-Modified-Since': validators.lastModified } + : {}), + ...(feed?.headers || {}), + }; +} + +/** + * Only https upstreams are fetched. + * @param {string} url Request or response URL. + * @returns {boolean} + */ +export function isAcceptableTransitUpstreamUrl(url) { + try { + return new URL(url).protocol === 'https:'; + } catch { + return false; + } +} + +/** + * Decide whether one redirect hop may be followed, BEFORE it is requested. + * + * Checking the FINAL url after `redirect: 'follow'` is too late: by then the + * disallowed host has already been contacted, has already seen this server's + * request, and has already answered. So every hop is resolved here first and a + * hop that leaves the feed's own origin, or drops to plain http, is refused — + * the same rule the CCTV frame proxy applies to camera images. + * + * @param {string} originUrl The registered feed URL (defines the allowed origin). + * @param {string} currentUrl The URL that produced this redirect (for relative Location). + * @param {string|null} location Raw `Location` header value. + * @returns {{ ok: true, url: string } | { ok: false, reason: string }} + */ +export function transitRedirectDecision(originUrl, currentUrl, location) { + if (!location) + return { ok: false, reason: 'redirect without a Location header' }; + let origin; + let next; + try { + origin = new URL(originUrl).origin; + next = new URL(location, currentUrl); + } catch { + return { ok: false, reason: 'redirect target is not a valid URL' }; + } + if (next.protocol !== 'https:') + return { ok: false, reason: 'redirect left https' }; + if (next.origin !== origin) + return { ok: false, reason: `redirect left ${origin}` }; + return { ok: true, url: next.toString() }; +} + +/** + * Cooldown before the next upstream attempt after `failures` consecutive + * failures. Zero while the feed is healthy. + * @param {number} failures Consecutive failures (0 = none). + * @returns {number} ms to wait before the next upstream attempt. + */ +export function nextTransitBackoffMs(failures) { + const count = Number.isFinite(failures) ? Math.floor(failures) : 0; + if (count <= 0) return 0; + const index = Math.min(count, TRANSIT_BACKOFF_LADDER_MS.length) - 1; + return TRANSIT_BACKOFF_LADDER_MS[index]; +} + +/** + * Error thrown when a feed's shape is unusable rather than merely unavailable. + * Carries `transitReason` so the middleware can answer honestly. + */ +export class TransitFeedShapeError extends Error { + constructor(message, reason) { + super(message); + this.name = 'TransitFeedShapeError'; + this.transitReason = reason; + } +} + +/** + * Give every vehicle a best-available report time, and say WHERE it came from. + * + * A feed that omits `VehiclePosition.timestamp` is not reporting "now" — it is + * reporting nothing, and treating the moment WE fetched as the moment the bus + * was there is how a five-minute-old fix gets drawn as live. The ladder is + * explicit instead: the vehicle's own timestamp, else the feed header's + * timestamp (the operator's own statement about the snapshot), else fetch time, + * which is labelled as a guess so the layer can age it conservatively. + * + * @param {object[]} vehicles Normalized vehicle records. + * @param {number|null} headerTimestamp Feed header timestamp (epoch seconds). + * @param {number} fetchedAtS Fetch time (epoch seconds). + * @returns {object[]} Records with `timestamp` and `timestampSource` set. + */ +export function repairVehicleTimestamps(vehicles, headerTimestamp, fetchedAtS) { + const header = + Number.isFinite(headerTimestamp) && headerTimestamp > 0 + ? headerTimestamp + : null; + return (vehicles || []).map((vehicle) => { + if (Number.isFinite(vehicle.timestamp) && vehicle.timestamp > 0) { + return { ...vehicle, timestampSource: 'vehicle' }; + } + if (header !== null) + return { ...vehicle, timestamp: header, timestampSource: 'header' }; + return { ...vehicle, timestamp: fetchedAtS, timestampSource: 'fetch' }; + }); +} + +/** + * Decode upstream bytes into the JSON snapshot the browser consumes. + * + * A DIFFERENTIAL feed is refused outright. Differential GTFS-Realtime carries + * only what changed, so reading one as if it were a full snapshot would make + * every unmentioned vehicle look like it had vanished. Deletion semantics are + * not implemented, so the honest answer is to decline the feed, not to render + * a wrong one. + * + * @param {object} feed Registry entry. + * @param {Uint8Array|ArrayBuffer} bytes Raw GTFS-RT FeedMessage. + * @param {number} [now=Date.now()] Fetch time (ms epoch). + * @returns {{ feedId: string, name: string, fetchedAt: number, feedTimestamp: number|null, + * version: string|null, entityCount: number, truncated: boolean, count: number, vehicles: object[] }} + */ +export function buildTransitSnapshot(feed, bytes, now = Date.now()) { + const decoded = decodeVehiclePositions(bytes); + if (decoded.incrementality !== GTFS_INCREMENTALITY_FULL_DATASET) { + throw new TransitFeedShapeError( + `feed is differential (incrementality ${decoded.incrementality})`, + 'differential', + ); + } + return { + feedId: feed.id, + name: feed.name, + fetchedAt: now, + feedTimestamp: decoded.timestamp, + version: decoded.version, + entityCount: decoded.entityCount, + truncated: decoded.truncated, + count: decoded.vehicles.length, + vehicles: repairVehicleTimestamps( + decoded.vehicles, + decoded.timestamp, + Math.floor(now / 1000), + ), + }; +} + +/** + * Classify a cache entry for the request policy. + * @param {{ at: number }|null|undefined} entry Cached snapshot (`at` = fetch ms). + * @param {number} now + * @returns {'none'|'fresh'|'stale'|'expired'} + */ +export function transitCacheState(entry, now) { + if (!entry || !Number.isFinite(entry.at)) return 'none'; + const age = now - entry.at; + if (age < 0) return 'fresh'; + if (age < TRANSIT_PROXY_TTL_MS) return 'fresh'; + if (age < TRANSIT_PROXY_STALE_MAX_MS) return 'stale'; + return 'expired'; +} + +/** + * Response headers for a snapshot. `X-GEV-Cache` mirrors the other proxies + * (HIT / MISS / INFLIGHT / STALE-ERROR) so the layer can surface staleness. + * @param {'HIT'|'MISS'|'INFLIGHT'|'STALE-ERROR'} cacheState + * @param {string} [upstreamHost] + * @returns {Record} + */ +export function transitResponseHeaders( + cacheState, + upstreamHost = '', + contactedAt = null, +) { + return { + 'Content-Type': 'application/json; charset=utf-8', + 'Cache-Control': + cacheState === 'STALE-ERROR' + ? 'no-store' + : `public, max-age=${Math.floor(TRANSIT_PROXY_TTL_MS / 1000)}`, + 'X-GEV-Cache': cacheState, + ...(upstreamHost ? { 'X-Transit-Upstream': upstreamHost } : {}), + // When the operator last ANSWERED, which is not when the body was fetched. + // A feed whose file has not changed answers 304 forever, and the body we + // keep serving carries its original fetch time; without this the browser + // would read a healthy revalidated feed as one that had gone silent. + ...(Number.isFinite(contactedAt) + ? { 'X-Transit-Contact': String(Math.floor(contactedAt)) } + : {}), + }; +} diff --git a/src/data/transitProxy.test.mjs b/src/data/transitProxy.test.mjs new file mode 100644 index 000000000..dea10ab25 --- /dev/null +++ b/src/data/transitProxy.test.mjs @@ -0,0 +1,367 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { PbfWriter } from 'pbf'; +import { + TRANSIT_BACKOFF_LADDER_MS, + TRANSIT_PROXY_STALE_MAX_MS, + TRANSIT_PROXY_TTL_MS, + TransitFeedShapeError, + buildTransitSnapshot, + isAcceptableTransitUpstreamUrl, + nextTransitBackoffMs, + repairVehicleTimestamps, + resolveTransitRoute, + transitCacheState, + transitRedirectDecision, + transitResponseHeaders, + transitUpstreamHeaders, +} from './transitProxy.js'; +import { getTransitFeed } from './transitFeeds.js'; + +test('route resolution admits only the catalog and registered feed ids', () => { + assert.deepEqual(resolveTransitRoute('/feeds'), { route: 'feeds' }); + assert.deepEqual(resolveTransitRoute('/feeds/?x=1'), { route: 'feeds' }); + assert.equal(resolveTransitRoute('/vehicles/mbta')?.feed?.id, 'mbta'); + assert.equal(resolveTransitRoute('/vehicles/mbta/')?.feed?.id, 'mbta'); + assert.equal(resolveTransitRoute('/vehicles/mbta?trip=1')?.feed?.id, 'mbta'); + assert.equal(resolveTransitRoute('/vehicles/nope'), null); + assert.equal(resolveTransitRoute('/vehicles/'), null); + assert.equal(resolveTransitRoute('/vehicles/mbta/extra'), null); + assert.equal(resolveTransitRoute('/vehicles/..%2F..%2Fetc'), null); + assert.equal(resolveTransitRoute('/vehicles/%E0%A4%A'), null); // malformed escape never throws + assert.equal(resolveTransitRoute('/'), null); + assert.equal(resolveTransitRoute(''), null); + assert.equal(resolveTransitRoute(undefined), null); +}); + +test('upstream headers identify the proxy and carry feed-specific identification', () => { + const plain = transitUpstreamHeaders(getTransitFeed('mbta')); + assert.match(plain['User-Agent'], /gods-eye-view-transit-proxy/); + assert.match(plain.Accept, /x-protobuf/); + const entur = transitUpstreamHeaders(getTransitFeed('entur-norway')); + assert.equal(entur['ET-Client-Name'], 'gods-eye-view-transit'); + assert.ok(transitUpstreamHeaders(null)['User-Agent']); +}); + +test('only https upstreams are acceptable, including after a redirect', () => { + assert.equal( + isAcceptableTransitUpstreamUrl('https://cdn.mbta.com/x.pb'), + true, + ); + assert.equal( + isAcceptableTransitUpstreamUrl( + 'http://gtfs.ovapi.nl/nl/vehiclePositions.pb', + ), + false, + ); + assert.equal(isAcceptableTransitUpstreamUrl('ftp://x'), false); + assert.equal(isAcceptableTransitUpstreamUrl('not a url'), false); +}); + +test('snapshot shape carries provenance the layer displays', () => { + const writer = new PbfWriter(); + const header = new PbfWriter(); + header.writeStringField(1, '2.0'); + header.writeVarintField(3, 1_700_000_000); + writer.writeBytesField(1, header.finish()); + const snapshot = buildTransitSnapshot( + getTransitFeed('mbta'), + writer.finish(), + 12345, + ); + assert.equal(snapshot.feedId, 'mbta'); + assert.equal(snapshot.name, 'MBTA'); + assert.equal(snapshot.fetchedAt, 12345); + assert.equal(snapshot.feedTimestamp, 1_700_000_000); + assert.equal(snapshot.version, '2.0'); + assert.equal(snapshot.count, 0); + assert.deepEqual(snapshot.vehicles, []); +}); + +test('cache policy: fresh within TTL, stale until the serve-stale window, expired after', () => { + const now = 1_000_000; + assert.equal(transitCacheState(null, now), 'none'); + assert.equal(transitCacheState({ at: Number.NaN }, now), 'none'); + assert.equal(transitCacheState({ at: now }, now), 'fresh'); + assert.equal( + transitCacheState({ at: now - TRANSIT_PROXY_TTL_MS + 1 }, now), + 'fresh', + ); + assert.equal( + transitCacheState({ at: now - TRANSIT_PROXY_TTL_MS }, now), + 'stale', + ); + assert.equal( + transitCacheState({ at: now - TRANSIT_PROXY_STALE_MAX_MS + 1 }, now), + 'stale', + ); + assert.equal( + transitCacheState({ at: now - TRANSIT_PROXY_STALE_MAX_MS }, now), + 'expired', + ); + assert.equal(transitCacheState({ at: now + 5000 }, now), 'fresh'); // clock skew never expires a fresh fetch +}); + +test('response headers mark cache state and never let a stale-error response be cached downstream', () => { + assert.equal( + transitResponseHeaders('HIT', 'cdn.mbta.com')['X-Transit-Upstream'], + 'cdn.mbta.com', + ); + assert.equal( + transitResponseHeaders('HIT')['Cache-Control'], + 'public, max-age=15', + ); + assert.equal( + transitResponseHeaders('STALE-ERROR')['Cache-Control'], + 'no-store', + ); + assert.equal(transitResponseHeaders('MISS')['X-GEV-Cache'], 'MISS'); +}); + +/** A minimal FeedMessage with the given header fields and vehicle entities. */ +function encodeFeed({ + incrementality = null, + timestamp = 1_700_000_000, + entities = [], +}) { + const writer = new PbfWriter(); + const header = new PbfWriter(); + header.writeStringField(1, '2.0'); + if (incrementality !== null) header.writeVarintField(2, incrementality); + if (timestamp !== null) header.writeVarintField(3, timestamp); + writer.writeBytesField(1, header.finish()); + for (const entity of entities) { + const feedEntity = new PbfWriter(); + feedEntity.writeStringField(1, entity.id); + const vehicle = new PbfWriter(); + const position = new PbfWriter(); + position.writeFloatField(1, entity.lat); + position.writeFloatField(2, entity.lon); + vehicle.writeBytesField(2, position.finish()); + if (entity.timestamp) vehicle.writeVarintField(5, entity.timestamp); + const descriptor = new PbfWriter(); + descriptor.writeStringField(1, entity.vehicleId || entity.id); + vehicle.writeBytesField(8, descriptor.finish()); + feedEntity.writeBytesField(4, vehicle.finish()); + writer.writeBytesField(2, feedEntity.finish()); + } + return writer.finish(); +} + +test('a redirect is judged before it is followed, not after it has been answered', () => { + const feed = { url: 'https://data.example.gov/download/feed.pb' }; + // Same origin, absolute and relative, is the only case that may proceed. + assert.deepEqual( + transitRedirectDecision( + feed.url, + feed.url, + 'https://data.example.gov/files/a.pb', + ), + { ok: true, url: 'https://data.example.gov/files/a.pb' }, + ); + assert.deepEqual(transitRedirectDecision(feed.url, feed.url, '/files/b.pb'), { + ok: true, + url: 'https://data.example.gov/files/b.pb', + }); + // Another host is refused even though it is https — this is the hop that used + // to be contacted first and inspected afterwards. + const offOrigin = transitRedirectDecision( + feed.url, + feed.url, + 'https://cdn.elsewhere.test/a.pb', + ); + assert.equal(offOrigin.ok, false); + assert.match(offOrigin.reason, /left https:\/\/data\.example\.gov/); + // A downgrade to http, a missing Location, and a malformed target all refuse. + assert.equal( + transitRedirectDecision(feed.url, feed.url, 'http://data.example.gov/a.pb') + .ok, + false, + ); + assert.equal(transitRedirectDecision(feed.url, feed.url, null).ok, false); + assert.equal( + transitRedirectDecision(feed.url, feed.url, 'javascript:alert(1)').ok, + false, + ); + assert.equal( + transitRedirectDecision(feed.url, feed.url, 'data:text/plain,x').ok, + false, + ); + assert.equal( + transitRedirectDecision('not-a-url', 'not-a-url', '/a.pb').ok, + false, + ); + // A sibling port is a different origin. + assert.equal( + transitRedirectDecision( + feed.url, + feed.url, + 'https://data.example.gov:8443/a.pb', + ).ok, + false, + ); +}); + +test('failure cooldown climbs and stops climbing, and a healthy feed waits for nothing', () => { + assert.equal(nextTransitBackoffMs(0), 0); + assert.equal(nextTransitBackoffMs(-3), 0); + assert.deepEqual([1, 2, 3, 4].map(nextTransitBackoffMs), [ + ...TRANSIT_BACKOFF_LADDER_MS, + ]); + // Past the ladder the wait stays at its last rung rather than growing forever. + assert.equal(nextTransitBackoffMs(9), TRANSIT_BACKOFF_LADDER_MS.at(-1)); + assert.equal(nextTransitBackoffMs(500), TRANSIT_BACKOFF_LADDER_MS.at(-1)); +}); + +test('a differential feed is refused rather than read as a full snapshot', () => { + const feed = getTransitFeed('mbta'); + const differential = encodeFeed({ + incrementality: 1, + entities: [{ id: 'v1', lat: 42.36, lon: -71.05, timestamp: 1_700_000_000 }], + }); + assert.throws( + () => buildTransitSnapshot(feed, differential, 1_700_000_000_000), + (error) => + error instanceof TransitFeedShapeError && + error.transitReason === 'differential', + ); + // The same bytes with FULL_DATASET decode normally, so the refusal is about + // the header and not about the payload. + const full = encodeFeed({ + incrementality: 0, + entities: [{ id: 'v1', lat: 42.36, lon: -71.05, timestamp: 1_700_000_000 }], + }); + assert.equal(buildTransitSnapshot(feed, full, 1_700_000_000_000).count, 1); + // An absent incrementality field means FULL_DATASET by the spec's default. + assert.equal( + buildTransitSnapshot(feed, encodeFeed({ entities: [] }), 1).count, + 0, + ); +}); + +test('a vehicle with no time of its own borrows the feed header time and says so', () => { + const fetchedAtS = 1_700_000_500; + const repaired = repairVehicleTimestamps( + [ + { id: 'own', timestamp: 1_700_000_400 }, + { id: 'none' }, + { id: 'zero', timestamp: 0 }, + ], + 1_700_000_450, + fetchedAtS, + ); + assert.deepEqual( + repaired.map((vehicle) => [ + vehicle.id, + vehicle.timestamp, + vehicle.timestampSource, + ]), + [ + ['own', 1_700_000_400, 'vehicle'], + ['none', 1_700_000_450, 'header'], + ['zero', 1_700_000_450, 'header'], + ], + ); + // With no header either, fetch time is used and is labelled as such — never + // presented as the vehicle's own report. + const noHeader = repairVehicleTimestamps([{ id: 'none' }], null, fetchedAtS); + assert.deepEqual( + [noHeader[0].timestamp, noHeader[0].timestampSource], + [fetchedAtS, 'fetch'], + ); + assert.deepEqual(repairVehicleTimestamps(null, null, fetchedAtS), []); +}); + +test('the snapshot builder fills timestamps from the header it decoded', () => { + const feed = getTransitFeed('mbta'); + const bytes = encodeFeed({ + timestamp: 1_700_000_000, + entities: [{ id: 'unstamped', lat: 42.36, lon: -71.05 }], + }); + const snapshot = buildTransitSnapshot(feed, bytes, 1_700_000_900_000); + assert.equal(snapshot.vehicles[0].timestamp, 1_700_000_000); + assert.equal(snapshot.vehicles[0].timestampSource, 'header'); + assert.equal(snapshot.feedTimestamp, 1_700_000_000); +}); + +test('conditional-request validators and gzip ride along with feed identification', () => { + const entur = getTransitFeed('entur-norway'); + const plain = transitUpstreamHeaders(entur); + assert.equal(plain['Accept-Encoding'], 'gzip'); + assert.equal(plain['ET-Client-Name'], 'gods-eye-view-transit'); + assert.equal('If-None-Match' in plain, false); + const conditional = transitUpstreamHeaders(entur, { + etag: 'W/"abc"', + lastModified: 'Wed, 10 Sep 2026 12:00:00 GMT', + }); + assert.equal(conditional['If-None-Match'], 'W/"abc"'); + assert.equal( + conditional['If-Modified-Since'], + 'Wed, 10 Sep 2026 12:00:00 GMT', + ); + assert.match(conditional['User-Agent'], /gods-eye-view/); +}); + +test('retained routes validate the entire identifier and only MBTA is opted in', () => { + assert.equal(resolveTransitRoute('/trail/mbta/bus%201').vehicleId, 'bus 1'); + for (const path of [ + '/trail/mbta/a%2Fb', + '/trail/mbta/%00', + '/trail/mbta/%5C', + '/trail/mbta/%2E%2E', + '/trail/mbta/%E0%A4%A', + `/trail/mbta/${encodeURIComponent('é'.repeat(129))}`, + '/trail/capmetro-austin/b', + ]) + assert.equal(resolveTransitRoute(path), null); +}); +test('client history validates response bounds, epochs and cancellation through parsing', async () => { + const { fetchTransitHistory } = await import('./transitProxy.js'); + const payload = { + version: 1, + feedId: 'mbta', + vehicleId: 'b', + epochs: [{ id: 1, trip: '', route: '1', mode: 'bus' }], + fixes: [[1000, 42, -71, 1, 1]], + }; + assert.deepEqual( + await fetchTransitHistory( + 'mbta', + 'b', + new AbortController().signal, + async () => new Response(JSON.stringify(payload)), + ), + payload, + ); + for (const value of [ + { ...payload, vehicleId: 'wrong' }, + { ...payload, fixes: [[1000, 91, -71, 1, 1]] }, + { ...payload, fixes: Array(129).fill(payload.fixes[0]) }, + { ...payload, epochs: [] }, + ]) + await assert.rejects( + fetchTransitHistory( + 'mbta', + 'b', + undefined, + async () => new Response(JSON.stringify(value)), + ), + ); + await assert.rejects( + fetchTransitHistory( + 'mbta', + 'b', + undefined, + async () => new Response('x'.repeat(32769)), + ), + /limit/, + ); + const controller = new AbortController(); + await assert.rejects( + fetchTransitHistory('mbta', 'b', controller.signal, async () => { + controller.abort(); + return new Response(JSON.stringify(payload)); + }), + { name: 'AbortError' }, + ); +}); diff --git a/src/layers/bikeshare/lifecycle.js b/src/layers/bikeshare/lifecycle.js index aeabac64b..17af85f11 100644 --- a/src/layers/bikeshare/lifecycle.js +++ b/src/layers/bikeshare/lifecycle.js @@ -1,6 +1,10 @@ import * as Cesium from 'cesium'; import { GBFS_CITY_REGISTRY } from './registry.js'; import { BIKESHARE_SELECTED_OVERLAY_SOURCE_ID } from './policy.js'; +import { + claimCameraSensitivity, + releaseCameraSensitivity, +} from '../../data/cameraSensitivity.js'; export function createLifecycle({ state: layerState, @@ -85,10 +89,10 @@ export function createLifecycle({ if (!layerState._cameraChangedAttached) { viewer.camera.changed.addEventListener(parts.viewport.onCameraChanged); - viewer.camera.percentageChanged = Math.min( - viewer.camera.percentageChanged || 1, - 0.05, - ); + // Through the shared ledger: this is a single number every layer's + // camera.changed listener shares, and a layer leaving must not hand + // back a coarse default while this one is still driving off it. + claimCameraSensitivity(viewer.camera, 'bikeshare', 0.05); layerState._cameraChangedAttached = true; } @@ -124,6 +128,7 @@ export function createLifecycle({ viewer.camera.changed.removeEventListener( parts.viewport.onCameraChanged, ); + releaseCameraSensitivity(viewer.camera, 'bikeshare'); layerState._cameraChangedAttached = false; } @@ -156,6 +161,7 @@ export function createLifecycle({ viewer.camera.changed.removeEventListener( parts.viewport.onCameraChanged, ); + releaseCameraSensitivity(viewer.camera, 'bikeshare'); layerState._cameraChangedAttached = false; } parts.ingestion.abortAllInFlight(); diff --git a/src/layers/traffic/lifecycle.js b/src/layers/traffic/lifecycle.js index bb452a984..c04ccbd62 100644 --- a/src/layers/traffic/lifecycle.js +++ b/src/layers/traffic/lifecycle.js @@ -1,5 +1,9 @@ import * as Cesium from 'cesium'; import { TRAFFIC_TIMING_ENABLED } from './policy.js'; +import { + claimCameraSensitivity, + releaseCameraSensitivity, +} from '../../data/cameraSensitivity.js'; export function createLifecycle({ state: layerState, @@ -93,18 +97,14 @@ export function createLifecycle({ ); } - // Subscribe to camera changes with a 5% movement threshold. Save the prior - // value so disable() can restore it — percentageChanged is a shared global - // on the camera and leaving it mutated affects every other camera.changed - // listener in the app. + // Share the 5% movement threshold with other camera-driven layers. viewer.camera.changed.addEventListener(parts.viewport.onCameraChanged); // Always inspect the final view, even when the last flight step is below // camera.changed's movement threshold. layerState._arrivalRemover = viewer.camera.moveEnd.addEventListener( parts.viewport.onCameraChanged, ); - layerState._prevPercentageChanged = viewer.camera.percentageChanged; - viewer.camera.percentageChanged = 0.05; + claimCameraSensitivity(viewer.camera, 'traffic', 0.05); // Kick off initial viewport check parts.viewport.onCameraChanged(); @@ -169,12 +169,7 @@ export function createLifecycle({ viewer.camera.changed.removeEventListener(parts.viewport.onCameraChanged); layerState._arrivalRemover?.(); layerState._arrivalRemover = null; - // Restore the camera's global percentageChanged so other layers/listeners - // keep their expected sensitivity. - if (layerState._prevPercentageChanged != null) { - viewer.camera.percentageChanged = layerState._prevPercentageChanged; - layerState._prevPercentageChanged = null; - } + releaseCameraSensitivity(viewer.camera, 'traffic'); if (layerState._pointCollection) layerState._pointCollection.show = false; }, diff --git a/src/layers/transit/controls.js b/src/layers/transit/controls.js new file mode 100644 index 000000000..ffae4fb4c --- /dev/null +++ b/src/layers/transit/controls.js @@ -0,0 +1,18 @@ +/** + * Manager wiring. Transit's data arrives from camera-driven proximity polls, + * not only from the manager's own tick, so the layer keeps a handle it can use + * to repaint its panel row the moment a snapshot lands. + * @param {object} context + * @returns {object} + */ +export function createControls({ state }) { + const methods = { + /** + * @param {object} dataManager DataLayerManager instance. + */ + attachDataManager(dataManager) { + state._dataManager = dataManager; + }, + }; + return { methods }; +} diff --git a/src/layers/transit/height.js b/src/layers/transit/height.js new file mode 100644 index 000000000..2e2945a88 --- /dev/null +++ b/src/layers/transit/height.js @@ -0,0 +1,257 @@ +import * as Cesium from 'cesium'; +import { + FLOOR_REREAD_ATTEMPTS, + FLOOR_REREAD_MS, + FLOOR_WARM_PER_POLL, + TRANSIT_POLL_MS, + HEIGHT_SAMPLE_MAX_ALTITUDE_M, + withinViewBounds, +} from './policy.js'; + +/** Surface preparation off the render path, shared by markers and history. */ +export function createHeight({ state, services, parts }) { + const { governorRequestRender } = services.render; + const { + cachedGroundFloor, + neighborFloorM, + warmGroundFloor, + coarseFloorCoord, + GROUND_FLOOR_LIFT_M, + } = services.ground; + // The application's rendered-mesh sampler. Optional so a layer built on a + // bare floor stub still runs; the application always supplies it. + const sampleMeshFloorCells = services.mesh?.sampleMeshFloorCells || null; + + /** The camera's subpoint in degrees, for the sampler's proximity gate. */ + function viewerSubpoint() { + const carto = state._viewer?.camera?.positionCartographic; + if (!carto) return { viewerLat: undefined, viewerLon: undefined }; + return { + viewerLat: (carto.latitude * 180) / Math.PI, + viewerLon: (carto.longitude * 180) / Math.PI, + }; + } + + function cameraAltitude() { + const carto = state._viewer?.camera?.positionCartographic; + return carto && Number.isFinite(carto.height) ? carto.height : Infinity; + } + + /** + * Whether ground matters at this camera. Above the gate a few metres of + * terrain are invisible, so the fleet renders on the ellipsoid and nothing + * is warmed — a national view must not queue a country's worth of cells. + * @returns {boolean} + */ + function nearGround() { + return cameraAltitude() <= HEIGHT_SAMPLE_MAX_ALTITUDE_M; + } + + /** + * The floor under a vehicle, and the best fallback if its own cell is cold. + * @param {number} lat + * @param {number} lon + * @returns {{cell: string, height: number|null, prior: number|null}} + */ + const terrainPoint = new Cesium.Cartographic(); + function requestHeight(lat, lon, entry = null) { + const coarse = coarseFloorCoord(lat, lon); + const cell = `${coarse.lat},${coarse.lon}`; + const fixtureFloor = + entry?.qaFloorM ?? state._qaFixtureFloors?.get(entry?.key); + if (Number.isFinite(fixtureFloor)) + return { cell, height: fixtureFloor, prior: null }; + // On a terrain globe, use the loaded surface under the marker. This is + // synchronous cache work and does not launch a terrain request. + const globe = state._viewer?.scene?.globe; + if (globe?.show === true) { + Cesium.Cartographic.fromDegrees(lon, lat, 0, terrainPoint); + const rendered = globe.getHeight?.(terrainPoint); + const flat = + state._viewer?.terrainProvider instanceof + Cesium.EllipsoidTerrainProvider; + if (Number.isFinite(rendered) || flat) + return { + cell, + height: (flat ? 0 : rendered) + GROUND_FLOOR_LIFT_M, + prior: null, + }; + } + const floor = cachedGroundFloor(lat, lon); + if (Number.isFinite(floor)) { + return { cell, height: floor + GROUND_FLOOR_LIFT_M, prior: null }; + } + if (!nearGround()) { + // Far enough away that the ellipsoid is indistinguishable from the + // street: draw, do not wait, do not warm. + return { cell, height: null, prior: 0 }; + } + const neighbour = neighborFloorM(coarse); + return { + cell, + height: null, + // A neighbouring floor is a coarse prior, never a clearance guarantee. + prior: Number.isFinite(neighbour) + ? neighbour + GROUND_FLOOR_LIFT_M + : null, + }; + } + + /** A vehicle no longer rendered needs nothing released: cells are shared. */ + function releaseVehicle() {} + + let cellCursors = new WeakMap(); + let budgetAt = -Infinity; + let demUsed = new Set(), + meshUsed = 0; + function rereadFloors() { + state._floorTimer = null; + if (!state._enabled) return; + const onGround = nearGround(), + candidates = []; + const entries = [...state._vehicles.values()].filter( + (entry) => + entry.key === state._selectedKey || + state._visible.has(entry) || + withinViewBounds(state._viewBounds, entry.segment?.from || entry.to), + ); + entries.sort( + (a, b) => + Number(b.key === state._selectedKey) - + Number(a.key === state._selectedKey), + ); + for (const entry of entries) { + const cells = new Map(); + const collect = (point, answer) => { + if (!onGround) return; + if (!cells.has(answer.cell)) + cells.set(answer.cell, { + lat: point.lat, + lon: point.lon, + cell: answer.cell, + warm: answer.height !== null, + }); + }; + parts.trails.prepareEntry(entry, collect); + const newest = entry.fixes.at(-1); + const answer = requestHeight(newest.lat, newest.lon, entry); + entry.heightCell = answer.cell; + if (answer.height !== null) { + entry.heightM = answer.height; + entry.heightResolved = true; + if (entry.heightPending) { + entry.heightPending = false; + state._visibilityDirty = true; + } + } + candidates.push({ + entry, + cells: [...cells.values()], + cursor: 0, + start: cellCursors.get(entry) || 0, + }); + } + const selectedCells = + candidates[0]?.entry.key === state._selectedKey + ? candidates[0].cells.slice(0, 12) + : []; + const ordered = [...selectedCells], + seen = new Set(selectedCells.map((cell) => cell.cell)); + const start = candidates.length + ? state._floorCursor % candidates.length + : 0; + let remaining = true; + while (remaining) { + remaining = false; + for (let i = 0; i < candidates.length; i++) { + const candidate = candidates[(start + i) % candidates.length]; + if (candidate.cursor >= candidate.cells.length) continue; + const index = + (candidate.start + candidate.cursor++) % candidate.cells.length; + const cell = candidate.cells[index]; + remaining = true; + if (!seen.has(cell.cell)) { + seen.add(cell.cell); + ordered.push({ + ...cell, + entry: candidate.entry, + next: (index + 1) % candidate.cells.length, + }); + } + } + } + state._floorCursor += FLOOR_WARM_PER_POLL; + const dem = [], + mesh = []; + for (const cell of ordered) { + let admitted = false; + if ( + !cell.warm && + (demUsed.has(cell.cell) || demUsed.size < FLOOR_WARM_PER_POLL) + ) { + demUsed.add(cell.cell); + dem.push(cell); + admitted = true; + } + if (cell.warm && meshUsed + mesh.length < 40) { + mesh.push(cell); + admitted = true; + } + if (admitted && cell.entry) cellCursors.set(cell.entry, cell.next); + } + meshUsed += mesh.length; + const markers = entries.map((entry) => entry.marker).filter(Boolean); + for (let i = 0; i < dem.length; i += 8) + warmGroundFloor(dem.slice(i, i + 8)); + for (let i = 0; i < mesh.length; i += 8) + sampleMeshFloorCells?.(state._viewer?.scene, mesh.slice(i, i + 8), { + excludeObjects: markers, + ...viewerSubpoint(), + }); + // Cached completions can be adopted now; asynchronous completions are + // adopted by the next bounded reread. + for (const entry of entries) parts.trails.prepareEntry(entry); + parts.rendering.requestVisibility(); + governorRequestRender('transit-floor'); + if (state._floorAttempts++ < FLOOR_REREAD_ATTEMPTS) + state._floorTimer = setTimeout(rereadFloors, FLOOR_REREAD_MS); + } + + /** Start a fresh re-read cycle. Called once per poll, never per frame. */ + function anchorFloors() { + if (state._floorTimer) { + clearTimeout(state._floorTimer); + state._floorTimer = null; + } + state._floorAttempts = 0; + const now = performance.now(); + if (now - budgetAt >= TRANSIT_POLL_MS) { + demUsed = new Set(); + meshUsed = 0; + budgetAt = now; + } + rereadFloors(); + } + + function clear() { + budgetAt = -Infinity; + cellCursors = new WeakMap(); + demUsed.clear(); + meshUsed = 0; + if (state._floorTimer) { + clearTimeout(state._floorTimer); + state._floorTimer = null; + } + state._floorAttempts = 0; + state._heightDirty.clear(); + } + + return { + nearGround, + requestHeight, + releaseVehicle, + anchorFloors, + rereadFloors, + clear, + }; +} diff --git a/src/layers/transit/index.js b/src/layers/transit/index.js new file mode 100644 index 000000000..0116b3829 --- /dev/null +++ b/src/layers/transit/index.js @@ -0,0 +1,70 @@ +import { createTrails } from './trails.js'; +import { createState } from './state.js'; +import { createHeight } from './height.js'; +import { createRendering } from './rendering.js'; +import { createSelection } from './selection.js'; +import { createIngestion } from './ingestion.js'; +import { createViewport } from './viewport.js'; +import { createQueries } from './queries.js'; +import { createControls } from './controls.js'; +import { createLifecycle } from './lifecycle.js'; +import { createTesting } from './testing.js'; +import { TRANSIT_POLL_MS } from './policy.js'; + +/** + * Construct one Transit layer with its own scene state and supplied services. + * + * Nothing here is module-global: the viewer, the point collection, the vehicle + * and height maps, the in-flight requests, the manager handle and the overlay + * host all live on the instance this returns, and `destroy()` releases every + * shared registration it took (sprite collection, pick owner, render hold, + * camera sensitivity). + * + * @param {{services: object}} options + * @returns {object} The data-layer module the manager registers. + */ +export function createTransitLayer({ services }) { + if (!services?.overlays || !services?.render || !services?.sprites) { + throw new TypeError( + 'A transit layer needs overlay, render and sprite services', + ); + } + const state = createState({ services }); + const parts = {}; + const context = { state, services, parts }; + parts.trails = createTrails(context); + parts.height = createHeight(context); + parts.selection = createSelection(context); + parts.rendering = createRendering(context); + parts.ingestion = createIngestion(context); + parts.viewport = createViewport(context); + parts.queries = createQueries(context); + parts.controls = createControls(context); + parts.lifecycle = createLifecycle(context); + parts.testing = createTesting(context); + + return Object.assign( + { updateInterval: TRANSIT_POLL_MS }, + parts.lifecycle.methods, + parts.queries.methods, + parts.controls.methods, + parts.testing, + ); +} + +export { + TRANSIT_MODE_COLORS, + TRANSIT_POLL_MS, + TRANSIT_SELECTED_OVERLAY_SOURCE_ID, + TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS, + aggregateTransitFeedHealth, + buildTransitSelectionCopy, + createTransitSelectedOverlayEntry, + interpolatedVehiclePosition, + isStaleVehicleFix, + transitDetectionClass, + transitDetectionId, + transitDetectionMetric, + transitVehicleKey, + vehicleFixAgeMs, +} from './policy.js'; diff --git a/src/layers/transit/ingestion.js b/src/layers/transit/ingestion.js new file mode 100644 index 000000000..9146ae86c --- /dev/null +++ b/src/layers/transit/ingestion.js @@ -0,0 +1,513 @@ +import { + createLagReservoir, + destroyTrack, + FIX_FLAGS, + trimTrack, +} from '../../data/contactPlayback.js'; +import { + FEED_EVICT_AFTER_MS, + FEED_STALE_AFTER_MS, + MAX_VEHICLES_TOTAL, + MISSED_POLLS_TO_DROP, + TRANSIT_POLL_MS, + isStaleVehicleFix, + transitVehicleKey, + vehicleReportTimeMs, +} from './policy.js'; +import { + transitModeFor, + transitModeResolved, +} from '../../data/transitFeeds.js'; +import { + attachFloorToPlace, + initializePlayback, + recordFix, + updatePlayback, + syncPlayback, +} from './movement.js'; + +/** + * How far ahead of the snapshot's own fetch time a vehicle stamp may sit + * before it is a clock fault rather than a report. A stamp an hour ahead once + * made every honest fix that followed read as out of order, so the marker + * froze for as long as the clock stayed wrong. Beyond this the fetch time is + * the honest bound. + */ +export const FUTURE_STAMP_TOLERANCE_MS = 10_000; +/** + * Consecutive out-of-order refusals after which the history is reset to the + * incoming fix. A feed whose clock was corrected backwards would otherwise be + * refused until real time caught up with its old mistake. + */ +export const ORDER_REJECTS_BEFORE_RESET = 3; + +/** + * Fetching feeds through the proxy and turning snapshots into rendered points. + * @param {object} context + * @returns {object} + */ +export function createIngestion({ state, services, parts }) { + const { governorRequestRender } = services.render; + const { registerDynamicCredit, transitFeedCredit } = services.credits; + + function feedStatus(feedId) { + let status = state._feedStatus.get(feedId); + if (!status) { + status = { + count: 0, + lastUpdate: null, + upstreamAt: null, + receivedAt: null, + error: null, + retryInSec: null, + stale: false, + pollSeq: 0, + loading: false, + }; + state._feedStatus.set(feedId, status); + } + return status; + } + + function removeVehicle(key) { + const entry = state._vehicles.get(key); + if (!entry) return; + if (state._selectedKey === key) parts.selection.clearSelection(); + if (entry.marker) entry.markerCollection?.remove(entry.marker); + entry.marker = null; + entry.detectContact = null; + state._visible.delete(entry); + parts.height.releaseVehicle(key, entry.heightCell); + // A vehicle removed mid-glide takes its share of the hold with it, so the + // set stays true without waiting for a frame that may never be requested. + state._moving.delete(entry); + state._heightDirty.delete(entry); + parts.rendering.cancelWake(entry); + destroyTrack(entry.track); + state._vehicles.delete(key); + state._detectRevision += 1; + } + + function removeFeedVehicles(feedId) { + let removed = 0; + for (const [key, entry] of state._vehicles) { + if (entry.feedId === feedId) { + removeVehicle(key); + removed += 1; + } + } + if (removed > 0) parts.rendering.syncRenderHold(); + return removed; + } + + /** + * Drop what has aged out, whatever the reason. + * + * Two independent clocks matter. A single vehicle can go quiet inside a feed + * that is otherwise healthy, and a whole feed can stop answering while its + * last snapshot still sits on the globe. The proxy serves a stale snapshot + * for up to ten minutes and then stops answering at all, and before this + * sweep existed that silence left the last fleet frozen in place, indefinitely + * and unlabelled. Both cases now end in removal. + * + * @param {number} nowMs + * @returns {number} Vehicles removed. + */ + function sweepAgedVehicles(nowMs) { + let removed = 0; + const deadFeeds = new Set(); + for (const [feedId, status] of state._feedStatus) { + // `upstreamAt` — when the OPERATOR last answered — not when this browser + // last received bytes. The proxy hands back its cached copy every 15 s + // during an outage, and counting those as updates meant a dead feed never + // aged at all. + if ( + Number.isFinite(status.upstreamAt) && + nowMs - status.upstreamAt > FEED_EVICT_AFTER_MS + ) { + deadFeeds.add(feedId); + } + } + for (const [key, entry] of state._vehicles) { + if ( + deadFeeds.has(entry.feedId) || + isStaleVehicleFix(entry.record, nowMs, entry.fetchedAt) + ) { + removeVehicle(key); + removed += 1; + } + } + if (removed > 0) { + for (const feedId of deadFeeds) { + const status = state._feedStatus.get(feedId); + if (status) { + status.count = 0; + status.stale = true; + } + } + // Removing the last vehicle is the moment the layer stops having work. + // Nothing else would notice: the per-frame pass returns early on an empty + // fleet, so the hold would sit there held by a fleet that no longer exists. + parts.rendering.syncRenderHold(); + governorRequestRender('transit-evict'); + state._dataManager?.refreshLayerStats?.(); + } + return removed; + } + + function applySnapshot(feed, snapshot, { stale, contactedAt = null }) { + const now = Date.now(); + // When the OPERATOR produced these positions, not when we received the + // bytes. A snapshot replayed from the proxy's cache carries its original + // fetch time, and that is the only honest clock for "how old are these + // buses". + const fetchedAt = Math.min( + Number.isFinite(snapshot?.fetchedAt) ? snapshot.fetchedAt : now, + now, + ); + // When the operator last ANSWERED, which is a different question. A feed + // whose file has not changed answers 304, and the proxy rightly keeps + // serving the body it already holds — but that body's fetch time stops + // advancing, so reading it as "when did we last hear from them" aged a + // perfectly healthy feed into DEGRADED at ninety seconds and emptied it at + // five minutes while every single request was succeeding. The proxy + // reports its last successful contact separately; absent that header the + // fetch time is the best we know. + const contacted = Math.min( + Number.isFinite(contactedAt) ? contactedAt : fetchedAt, + now, + ); + const status = feedStatus(feed.id); + const upstreamAgeMs = now - Math.max(contacted, fetchedAt); + status.upstreamAt = Math.max(contacted, fetchedAt); + status.receivedAt = now; + status.loading = false; + status.error = null; + status.retryInSec = null; + status.stale = stale === true || upstreamAgeMs > FEED_STALE_AFTER_MS; + + // Past the eviction window the answer is bytes, not news. Drawing them + // would put a five-minute-old fleet on the globe with a fresh-looking row. + if (upstreamAgeMs > FEED_EVICT_AFTER_MS) { + removeFeedVehicles(feed.id); + status.count = 0; + parts.rendering.syncRenderHold(); + governorRequestRender('transit-stale-refused'); + state._dataManager?.refreshLayerStats?.(); + return; + } + + status.pollSeq += 1; + const pollSeq = status.pollSeq; + let seen = 0; + let rejectedFixes = 0; + + const records = snapshot.vehicles || []; + if (!status.lag) status.lag = createLagReservoir(); + const replayed = status.snapshotAt === fetchedAt || stale === true; + status.snapshotAt = fetchedAt; + const monoNow = performance.now(); + // Under pressure, release hidden retention before visible histories. + if ( + state._historyBudget.allocatedBytes > + state._historyBudget.maxBytes * 0.9 + ) { + for (const vehicle of state._vehicles.values()) { + if (!vehicle.marker?.show && vehicle.key !== state._selectedKey) + trimTrack(vehicle.track, now, 2); + } + } + + for (const record of records) { + if (isStaleVehicleFix(record, now, fetchedAt)) continue; + const key = transitVehicleKey(feed.id, record.id); + const mode = transitModeFor(feed, record.routeId); + const modeInferred = !transitModeResolved(feed, record.routeId); + // A stamp ahead of the fetch is a clock fault; the fetch time bounds it. + const stamped = vehicleReportTimeMs(record, fetchedAt); + const reportAt = + stamped > fetchedAt + FUTURE_STAMP_TOLERANCE_MS ? fetchedAt : stamped; + const fix = { + t: reportAt, + lat: record.lat, + lon: record.lon, + bearingDeg: Number.isFinite(record.bearing) ? record.bearing : NaN, + flags: + stamped !== reportAt + ? FIX_FLAGS.RECEIPT_TIME + : record.timestampSource === 'vehicle' + ? FIX_FLAGS.VEHICLE_TIME + : record.timestampSource === 'header' + ? FIX_FLAGS.FEED_TIME + : FIX_FLAGS.RECEIPT_TIME, + }; + let entry = state._vehicles.get(key); + if (entry) { + const heard = recordFix(entry, fix, { + receivedAt: now, + monoNowMs: monoNow, + wallNowMs: now, + replayed, + modeInferred, + context: { + trip: record.tripId || '', + route: record.routeId || '', + mode, + }, + }); + if (!heard.accepted) { + entry.pollSeq = pollSeq; + seen += 1; + if (heard.reason !== 'repeat') rejectedFixes += 1; + continue; + } + if (entry.mode !== mode) parts.rendering.paintMode(entry, mode); + entry.modeInferred = modeInferred; + updatePlayback(entry, now, monoNow); + parts.rendering.schedulePlayback(entry); + // Ground follows the vehicle, every poll: a mesh floor landing over a + // DEM one, or a change of map regime, moves a parked vehicle too. A + // cell still cold takes the lowest neighbour as a prior, as before. + const priorHeightM = entry.heightM; + const height = parts.height.requestHeight(record.lat, record.lon); + if (height.height !== null) { + entry.heightCell = height.cell; + entry.heightM = height.height; + entry.heightResolved = true; + entry.heightPending = false; + } else if (height.cell !== entry.heightCell || !entry.heightResolved) { + entry.heightCell = height.cell; + entry.heightResolved = false; + if (height.prior !== null) { + entry.heightM = height.prior; + entry.heightPending = false; + } + } + // The floor belongs to the FIX it was asked for: the display may still + // be drawing an older fix on lower ground, and lifting that one onto + // the newest fix's surface is the flew-up-then-travelled jump. The + // frame pass owns the move, for a moving vehicle and a parked one + // alike. + const newest = entry.fixes[entry.fixes.length - 1]; + const stood = newest.h; + if (height.height !== null) { + attachFloorToPlace(entry, newest, height.height); + } + if (entry.heightM !== priorHeightM || newest.h !== stood) { + state._heightDirty.add(entry); + } + } else { + if (state._vehicles.size >= MAX_VEHICLES_TOTAL) { + if (!state._limitWarned) { + state._limitWarned = true; + console.warn( + `[Data:Transit] vehicle cap ${MAX_VEHICLES_TOTAL} reached — extra vehicles are not rendered`, + ); + } + continue; + } + const height = parts.height.requestHeight(record.lat, record.lon); + // Keep the local prior separate from resolved historical floors. + // A neighbouring cell is not a guaranteed lower bound. + const prior = height.height === null ? height.prior : height.height; + entry = { + key, + feedId: feed.id, + marker: null, + mode, + modeInferred, + heightM: prior === null ? 0 : prior, + heightCell: height.cell, + heightResolved: height.height !== null, + // Nothing trustworthy to stand on and no prior to borrow: the + // vehicle waits one sample rather than floating. + heightPending: height.height === null && prior === null, + from: null, + to: null, + fromCart: null, + toCart: null, + playT: NaN, + resets: 0, + courseDeg: Number.isFinite(record.bearing) ? record.bearing : null, + courseAt: now, + courseEvalAt: monoNow, + record, + fetchedAt, + pollSeq, + }; + initializePlayback(entry, state._historyBudget, status.lag); + const first = recordFix(entry, fix, { + receivedAt: now, + monoNowMs: monoNow, + wallNowMs: now, + replayed, + modeInferred, + context: { + trip: record.tripId || '', + route: record.routeId || '', + mode, + }, + }); + if (!first.accepted) { + destroyTrack(entry.track); + continue; + } + if (height.height !== null) + attachFloorToPlace(entry, fix, height.height); + // A new vehicle holds at its only fix; its clock starts a full lag + // behind, so by the time playback reaches the fix the next one has + // arrived and the segment can be drawn at 1x. + syncPlayback(entry, now, monoNow); + entry.from = entry.segment.from; + entry.to = entry.segment.to; + parts.rendering.refreshEndpoints(entry); + entry.marker = parts.rendering.addMarker(key, entry); + state._vehicles.set(key, entry); + parts.rendering.schedulePlayback(entry); + if (entry.courseDeg !== null) state._rotationDirty = true; + } + entry.record = record; + entry.fetchedAt = fetchedAt; + entry.pollSeq = pollSeq; + seen += 1; + } + + // Drop vehicles this feed stopped reporting. + for (const [key, entry] of state._vehicles) { + if ( + entry.feedId === feed.id && + pollSeq - entry.pollSeq >= MISSED_POLLS_TO_DROP + ) { + removeVehicle(key); + } + } + sweepAgedVehicles(now); + + status.count = seen; + status.lastUpdate = fetchedAt; + status.receivedAt = now; + // The layer-level timestamp is the freshest UPSTREAM time across feeds, so + // the panel's "just now" means the positions are from just now. + state._lastUpdate = + Math.max(state._lastUpdate || 0, fetchedAt) || fetchedAt; + state._error = null; + if (rejectedFixes > 0 && !state._jumpWarned) { + state._jumpWarned = true; + console.warn( + `[Data:Transit] ${feed.id}: ${rejectedFixes} fix(es) implied impossible travel and were not animated`, + ); + } + if (state._viewer) + registerDynamicCredit(state._viewer, transitFeedCredit(feed)); + if ( + state._selectedKey && + state._vehicles.get(state._selectedKey)?.feedId === feed.id + ) { + parts.selection.refreshSelectedCard(true); + } + // Records changed hands this poll, so DETECT's list is rebuilt once — + // the visibility sweep below bumps the revision. + // One floor cycle per poll, on its own timer: warming is a network round + // trip and must never sit on the render path. + parts.height.anchorFloors(); + state._detectRevision++; + parts.rendering.requestVisibility(); + parts.rendering.syncRenderHold(); + governorRequestRender('transit-poll'); + state._dataManager?.refreshLayerStats?.(); + } + + /** + * Poll one feed. A request already in flight is younger than one poll + * interval, so a second caller (enable() and the manager's first update() + * both ask within the same tick) awaits that request instead of aborting it — + * the manager's first update then settles with data on the globe. + * @param {object} feed Registry entry. + * @param {number} generation Enable generation the poll belongs to. + * @returns {Promise} + */ + function pollFeed(feed, generation) { + if (!state._enabled || generation !== state._generation) + return Promise.resolve(); + const existing = state._inFlight.get(feed.id); + if (existing) return existing.promise; + const controller = new AbortController(); + const status = feedStatus(feed.id); + status.loading = status.count === 0; + const promise = (async () => { + try { + const response = await fetch( + `/api/transit/vehicles/${encodeURIComponent(feed.id)}`, + { + signal: controller.signal, + headers: { Accept: 'application/json' }, + }, + ); + if (!response.ok) { + // The proxy says when it will next try the operator; carrying that + // through means the row can offer a time instead of just a shrug. + const detail = await response.json().catch(() => null); + const error = new Error(`transit proxy HTTP ${response.status}`); + if (Number.isFinite(detail?.retryInSec)) { + error.retryInSec = detail.retryInSec; + } + throw error; + } + const snapshot = await response.json(); + if ( + !state._enabled || + generation !== state._generation || + !state._activeFeeds.has(feed.id) + ) { + return; + } + const contactHeader = Number.parseInt( + response.headers.get('x-transit-contact') || '', + 10, + ); + applySnapshot(feed, snapshot, { + stale: response.headers.get('x-gev-cache') === 'STALE-ERROR', + contactedAt: Number.isFinite(contactHeader) ? contactHeader : null, + }); + } catch (error) { + if (error?.name === 'AbortError') return; + if (generation !== state._generation) return; + console.warn( + `[Data:Transit] ${feed.id} poll failed:`, + error?.message || error, + ); + status.error = `${feed.name} feed unavailable`; + status.retryInSec = Number.isFinite(error?.retryInSec) + ? error.retryInSec + : null; + status.loading = false; + state._error = status.error; + // A failed poll is exactly when old vehicles must be re-examined: + // nothing else will tell the globe this feed has gone quiet. + sweepAgedVehicles(Date.now()); + state._dataManager?.refreshLayerStats?.(); + } finally { + if (state._inFlight.get(feed.id)?.controller === controller) { + state._inFlight.delete(feed.id); + } + } + })(); + state._inFlight.set(feed.id, { controller, promise }); + return promise; + } + + function abortAllInFlight() { + for (const { controller } of state._inFlight.values()) controller.abort(); + state._inFlight.clear(); + } + + return { + feedStatus, + removeVehicle, + removeFeedVehicles, + sweepAgedVehicles, + applySnapshot, + pollFeed, + abortAllInFlight, + }; +} diff --git a/src/layers/transit/lifecycle.js b/src/layers/transit/lifecycle.js new file mode 100644 index 000000000..11b670519 --- /dev/null +++ b/src/layers/transit/lifecycle.js @@ -0,0 +1,303 @@ +import { destroyTrack } from '../../data/contactPlayback.js'; +import * as Cesium from 'cesium'; +import { + claimCameraSensitivity, + releaseCameraSensitivity, +} from '../../data/cameraSensitivity.js'; +import { + CAMERA_PERCENTAGE_CHANGED, + TRANSIT_SELECTED_OVERLAY_SOURCE_ID, +} from './policy.js'; +import { TRANSIT_ENABLED_FEEDS } from '../../data/transitFeeds.js'; + +/** + * Init / enable / disable / update / destroy for one Transit layer instance. + * Everything acquired here is released here — primitives, listeners, the sprite + * registration, the pick owner, the render hold, and the camera sensitivity + * this layer borrowed. + * @param {object} context + * @returns {object} + */ +export function createLifecycle({ state, services, parts }) { + const { + registerSpriteCollection, + unregisterSpriteCollection, + restoreSpriteOrder, + } = services.sprites; + const { registerPickOwner, unregisterPickOwner } = services.picking; + const { releaseContinuousRender } = services.render; + + /** + * Borrow the camera's change sensitivity through the shared claim ledger. + * + * `camera.changed` fires on a percentage of the view moving, and this layer + * needs a finer trigger than the default to notice it has entered a feed's + * coverage. It is a single shared number, so the claim is registered by name + * rather than written directly: comparing the VALUE on the way out could not + * tell this layer's 0.05 from anybody else's, and handing back the coarse + * default while Bikeshare was still relying on a fine one was exactly the + * bug that produced. + */ + function borrowCameraSensitivity(viewer) { + claimCameraSensitivity( + viewer?.camera, + 'transit', + CAMERA_PERCENTAGE_CHANGED, + ); + } + + /** Give the claim up. The camera keeps whatever anyone else still asks for. */ + function returnCameraSensitivity(viewer) { + releaseCameraSensitivity(viewer?.camera, 'transit'); + } + + /** + * Follow the active post-FX style so the sprites can be restyled for it. + * The map style arrives on `gev:style-change`; a cockpit vision override + * (NVG/thermal inside the cockpit view, which sets no map style) arrives on + * `gev:vision-change` with the effective style and wins while it lasts. + * Bound once per instance and removed on destroy. Read the document's + * current style first: a persisted style restores before layers register. + */ + function bindStyleEvents() { + if (typeof window === 'undefined') return; + if (state._styleListeners.length > 0) return; + parts.rendering.setStylePreset( + document?.documentElement?.dataset?.gevStyle || 'normal', + ); + const onStyle = (event) => { + if (state._cockpitVision) return; + parts.rendering.setStylePreset(event?.detail?.style); + }; + const onVision = (event) => { + state._cockpitVision = event?.detail?.cockpit === true; + parts.rendering.setStylePreset(event?.detail?.style); + }; + window.addEventListener('gev:style-change', onStyle); + window.addEventListener('gev:vision-change', onVision); + state._styleListeners.push( + ['gev:style-change', onStyle], + ['gev:vision-change', onVision], + ); + } + + function unbindStyleEvents() { + if (typeof window === 'undefined') return; + for (const [name, listener] of state._styleListeners) { + window.removeEventListener(name, listener); + } + state._styleListeners.length = 0; + state._cockpitVision = false; + } + + let visibilityDocument = null, + hidden = false; + function onVisibilityChange() { + if (visibilityDocument.visibilityState === 'hidden') hidden = true; + else if (hidden) { + hidden = false; + parts.rendering.resumePlayback(); + } + } + function bindVisibility() { + if (visibilityDocument || typeof document === 'undefined') return; + visibilityDocument = document; + hidden = document.visibilityState === 'hidden'; + visibilityDocument.addEventListener('visibilitychange', onVisibilityChange); + } + function unbindVisibility() { + visibilityDocument?.removeEventListener( + 'visibilitychange', + onVisibilityChange, + ); + visibilityDocument = null; + hidden = false; + } + + const methods = { + id: 'transit', + name: 'Transit', + icon: '🚌', + source: 'GTFS-RT', + + /** + * Create the point collection. Called once at bootstrap. + * @param {Cesium.Viewer} viewer + */ + init(viewer) { + state._viewer = viewer; + state._markers = new Cesium.BillboardCollection({ + blendOption: Cesium.BlendOption.TRANSLUCENT, + }); + viewer.scene.primitives.add(state._markers); + registerSpriteCollection('transit', state._markers); + state._animatedMarkers = new Cesium.BillboardCollection({ + blendOption: Cesium.BlendOption.TRANSLUCENT, + }); + viewer.scene.primitives.add(state._animatedMarkers); + registerSpriteCollection('transit-motion', state._animatedMarkers); + state._animatedMarkers.show = false; + state._markers.show = false; + state._enabled = false; + state._activeFeeds.clear(); + state._feedStatus.clear(); + state._inFlight.clear(); + for (const entry of state._vehicles.values()) { + entry.detectContact = null; + parts.rendering.cancelWake(entry); + destroyTrack(entry.track); + } + state._vehicles.clear(); + parts.height.clear(); + state._selectedKey = null; + state._lastUpdate = null; + state._error = null; + state._limitWarned = false; + state._altitudeGateOpen = false; + state._moving.clear(); + state._heightDirty.clear(); + state._rotationPose = null; + state._rotationDirty = false; + state._overlayHost.setVisible(TRANSIT_SELECTED_OVERLAY_SOURCE_ID, false); + restoreSpriteOrder(viewer); + bindStyleEvents(); + console.log( + `[Data:Transit] Initialized with ${TRANSIT_ENABLED_FEEDS.length} GTFS-RT feeds`, + ); + }, + + /** + * Show vehicles, watch the camera, and poll every feed in range. + * @param {Cesium.Viewer} viewer + */ + enable(viewer) { + state._enabled = true; + bindVisibility(); + state._generation += 1; + state._error = null; + state._markers.show = true; + state._animatedMarkers.show = true; + state._maintenanceTimer = setInterval( + parts.rendering.maintainPresentation, + 250, + ); + state._overlayHost.setVisible(TRANSIT_SELECTED_OVERLAY_SOURCE_ID, true); + parts.selection.installClickHandler(viewer); + registerPickOwner('transit', (pickedId) => state._vehicles.has(pickedId)); + if (!state._cameraChangedAttached) { + viewer.camera.changed.addEventListener(parts.viewport.onCameraChanged); + borrowCameraSensitivity(viewer); + state._cameraChangedAttached = true; + } + if (!state._preRenderRemove) { + state._preRenderRemove = viewer.scene.preRender.addEventListener( + parts.rendering.onPreRender, + ); + } + parts.viewport.runProximityCheck(); + restoreSpriteOrder(viewer); + }, + + /** + * Hide everything, stop polling, drop all vehicles. + * @param {Cesium.Viewer} viewer + */ + disable(viewer) { + state._enabled = false; + unbindVisibility(); + parts.testing._stopTransitProbeForTest(); + state._qaFixtureFloors?.clear(); + state._generation += 1; + clearInterval(state._maintenanceTimer); + state._maintenanceTimer = null; + clearTimeout(state._visibilityTimer); + state._visibilityTimer = null; + state._detectCache = null; + state._detectBuiltAt = -Infinity; + state._visible.clear(); + clearTimeout(state._cameraDebounceTimer); + state._cameraDebounceTimer = null; + parts.selection.clearSelection(); + state._overlayHost.setVisible(TRANSIT_SELECTED_OVERLAY_SOURCE_ID, false); + parts.selection.removeClickHandler(); + unregisterPickOwner('transit'); + if (state._cameraChangedAttached) { + viewer.camera.changed.removeEventListener( + parts.viewport.onCameraChanged, + ); + returnCameraSensitivity(viewer); + state._cameraChangedAttached = false; + } + if (state._preRenderRemove) { + state._preRenderRemove(); + state._preRenderRemove = null; + } + parts.ingestion.abortAllInFlight(); + state._activeFeeds.clear(); + state._feedStatus.clear(); + for (const entry of state._vehicles.values()) { + entry.detectContact = null; + parts.rendering.cancelWake(entry); + destroyTrack(entry.track); + } + state._vehicles.clear(); + parts.height.clear(); + state._markers?.removeAll(); + state._animatedMarkers?.removeAll(); + if (state._animatedMarkers) state._animatedMarkers.show = false; + if (state._markers) state._markers.show = false; + state._altitudeGateOpen = false; + state._moving.clear(); + state._heightDirty.clear(); + state._rotationPose = null; + state._rotationDirty = false; + if (state._renderHeld) { + releaseContinuousRender('transit'); + state._renderHeld = false; + } + }, + + /** + * Manager tick (every TRANSIT_POLL_MS): re-poll every active feed, and + * sweep aged vehicles even when nothing is active, so a layer left over a + * quiet region does not keep an old fleet on screen. + * @returns {Promise} + */ + async update() { + if (!state._enabled) return; + parts.ingestion.sweepAgedVehicles(Date.now()); + if (state._activeFeeds.size === 0) return; + const generation = state._generation; + await Promise.all( + [...state._activeFeeds.values()].map((feed) => + parts.ingestion.pollFeed(feed, generation), + ), + ); + }, + + /** + * Tear down the collection entirely and release every shared registration. + * @param {Cesium.Viewer} viewer + */ + destroy(viewer) { + this.disable(viewer); + if (state._markers) { + unregisterSpriteCollection('transit', state._markers); + viewer?.scene?.primitives?.remove(state._markers); + state._markers = null; + } + if (state._animatedMarkers) { + unregisterSpriteCollection('transit-motion', state._animatedMarkers); + viewer?.scene?.primitives?.remove(state._animatedMarkers); + state._animatedMarkers = null; + } + state._overlayHost.clearSource(TRANSIT_SELECTED_OVERLAY_SOURCE_ID); + parts.height.clear(); + unbindStyleEvents(); + state._dataManager = null; + state._viewer = null; + }, + }; + + return { methods }; +} diff --git a/src/layers/transit/lifecycle.test.mjs b/src/layers/transit/lifecycle.test.mjs new file mode 100644 index 000000000..dc3635dd1 --- /dev/null +++ b/src/layers/transit/lifecycle.test.mjs @@ -0,0 +1,3641 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as Cesium from 'cesium'; +import { createTransitLayer } from './index.js'; +import { + FEED_EVICT_AFTER_MS, + FEED_STALE_AFTER_MS, + MISSED_POLLS_TO_DROP, + SELECTED_CARD_REFRESH_MS, + TRANSIT_POLL_MS, + VEHICLE_MAX_FIX_AGE_MS, +} from './policy.js'; +import { FLOOR_WARM_PER_POLL } from './policy.js'; +import { + FUTURE_STAMP_TOLERANCE_MS, + ORDER_REJECTS_BEFORE_RESET, +} from './ingestion.js'; + +/** + * Fixes in these tests are AGE_MS old when they arrive, which makes the lag + * the feed earns equal to the lag a fifteen-second reporter earns on its own + * — one number, `LAG_MS`, and no slew — so the timelines below are exact. + */ +const AGE_MS = 15_000; +const LAG_MS = AGE_MS + TRANSIT_POLL_MS + 5_000; +/** A fix stamped AGE_MS before now, the way a live feed delivers them. */ +const reported = () => (Date.now() - AGE_MS) / 1000; +const lat = (entry) => + Cesium.Math.toDegrees( + Cesium.Cartographic.fromCartesian(entry.marker.position).latitude, + ); +import { + cameraSensitivityClaims, + claimCameraSensitivity, + releaseCameraSensitivity, +} from '../../data/cameraSensitivity.js'; +import { + _resetRenderGovernorForTest, + getRenderGovernorDiagnostics, + governorRequestRender, + holdContinuousRender, + installRenderGovernor, + releaseContinuousRender, + uninstallRenderGovernor, +} from '../../renderGovernor.js'; + +const BOSTON = { lat: 42.3601, lon: -71.0589 }; + +/** One decoded snapshot in the shape `/api/transit/vehicles/` returns. */ +function snapshot( + feedId, + name, + vehicles, + { fetchedAt, feedTimestamp = null } = {}, +) { + return { + feedId, + name, + fetchedAt, + feedTimestamp, + version: '2.0', + entityCount: vehicles.length, + truncated: false, + count: vehicles.length, + vehicles, + }; +} + +/** A vehicle record as the proxy emits it, with its timestamp source resolved. */ +function vehicle(id, lat, lon, atSeconds, extra = {}) { + return { + id, + lat, + lon, + bearing: null, + speedMps: null, + timestamp: atSeconds, + timestampSource: 'vehicle', + routeId: '1', + tripId: null, + directionId: null, + label: null, + stopId: null, + status: null, + occupancy: null, + ...extra, + }; +} + +/** + * A Transit layer wired to a stub scene and a scripted proxy. + * + * The render governor is the real one, so "does this layer keep the render loop + * awake?" is answered by the same code the app runs. + */ +/** + * A stand-in for the app's shared ground-floor system: warm cells, the coarse + * grid, and a lowest-neighbour read. `floorAt` decides what a cell resolves to + * when it is warmed; returning undefined models terrain that will not answer. + * @param {(lat: number, lon: number) => number|undefined} floorAt + */ +function groundStub(floorAt) { + const warm = new Map(); + const warmed = []; + const coarseFloorCoord = (lat, lon) => ({ + lat: Number(lat.toFixed(3)), + lon: Number(lon.toFixed(3)), + }); + const key = (c) => `${c.lat},${c.lon}`; + const cachedGroundFloor = (lat, lon) => { + const value = warm.get(key(coarseFloorCoord(lat, lon))); + return value === undefined ? null : value; + }; + return { + GROUND_FLOOR_LIFT_M: 1.5, + coarseFloorCoord, + cachedGroundFloor, + neighborFloorM(cell) { + let lowest = null; + let resolved = 0; + for (let dLat = -1; dLat <= 1; dLat += 1) { + for (let dLon = -1; dLon <= 1; dLon += 1) { + if (dLat === 0 && dLon === 0) continue; + const h = cachedGroundFloor( + cell.lat + dLat * 0.001, + cell.lon + dLon * 0.001, + ); + if (h === null) continue; + resolved += 1; + if (lowest === null || h < lowest) lowest = h; + } + } + return resolved >= 3 ? lowest : null; + }, + warmGroundFloor(points) { + warmed.push(...points); + for (const point of points) { + const value = floorAt(point.lat, point.lon); + if (Number.isFinite(value)) { + warm.set(key(coarseFloorCoord(point.lat, point.lon)), value); + } + } + }, + _warm: warm, + _warmed: warmed, + _key: (lat, lon) => key(coarseFloorCoord(lat, lon)), + }; +} + +function harness( + t, + { + altitude = 4_000, + floorAt = () => 12, + at = BOSTON, + view = { + west: at.lon - 0.1, + south: at.lat - 0.1, + east: at.lon + 0.1, + north: at.lat + 0.1, + }, + } = {}, +) { + const viewRect = { ...view }; + t.mock.timers.enable({ + apis: ['Date', 'setTimeout'], + now: 1_789_000_000_000, + }); + + for (const name of [ + 'HTMLCanvasElement', + 'HTMLImageElement', + 'ImageBitmap', + 'OffscreenCanvas', + ]) { + const prior = globalThis[name]; + globalThis[name] = class {}; + t.after(() => { + if (prior === undefined) delete globalThis[name]; + else globalThis[name] = prior; + }); + } + t.mock.method(performance, 'now', () => Date.now() - 1_789_000_000_000); + const originalFetch = globalThis.fetch; + const originalDocument = globalThis.document; + const primitives = []; + const sprites = new Map(); + const pickOwners = new Map(); + const overlaySources = new Map(); + const credits = []; + /** @type {Map {status:number, body?:object, headers?:object}>} */ + const responders = new Map(); + const requested = []; + + globalThis.document = Object.assign(new EventTarget(), { + visibilityState: 'visible', + documentElement: { dataset: {} }, + }); + // The style and vision events arrive on `window`, which Node does not have. + const originalWindow = globalThis.window; + globalThis.window = new EventTarget(); + + const preRender = new Cesium.Event(); + const cameraChanged = new Cesium.Event(); + const viewer = { + camera: { + changed: cameraChanged, + percentageChanged: 0.5, + positionCartographic: { + height: altitude, + latitude: Cesium.Math.toRadians(at.lat), + longitude: Cesium.Math.toRadians(at.lon), + }, + computeViewRectangle() { + return Cesium.Rectangle.fromDegrees( + viewRect.west, + viewRect.south, + viewRect.east, + viewRect.north, + ); + }, + }, + scene: { + canvas: { addEventListener() {}, removeEventListener() {} }, + globe: { ellipsoid: Cesium.Ellipsoid.WGS84 }, + pick: () => undefined, + requestRenderMode: false, + maximumRenderTimeChange: 0, + requestRender() {}, + preRender, + primitives: { + add(primitive) { + primitives.push(primitive); + return primitive; + }, + remove(primitive) { + const index = primitives.indexOf(primitive); + if (index >= 0) primitives.splice(index, 1); + return index >= 0; + }, + contains: (primitive) => primitives.includes(primitive), + raiseToTop() {}, + }, + }, + }; + + globalThis.fetch = async (url) => { + requested.push(String(url)); + const feedId = String(url).split('/').pop(); + const responder = responders.get(feedId); + const answer = responder + ? responder(String(url)) + : { status: 504, body: { error: 'Transit feed unavailable' } }; + return { + ok: answer.status >= 200 && answer.status < 300, + status: answer.status, + headers: { get: (name) => answer.headers?.[name.toLowerCase()] ?? null }, + json: async () => answer.body, + }; + }; + + const ground = groundStub(floorAt); + /** Every call the floor cycle makes to the rendered-mesh sampler. */ + const meshCalls = []; + const layer = createTransitLayer({ + services: { + ground, + mesh: { + sampleMeshFloorCells(scene, points, options) { + meshCalls.push({ scene, points: [...points], options }); + }, + }, + render: { + governorRequestRender, + holdContinuousRender, + releaseContinuousRender, + }, + sprites: { + registerSpriteCollection: (id, collection) => + sprites.set(id, collection), + unregisterSpriteCollection: (id) => sprites.delete(id), + restoreSpriteOrder() {}, + }, + picking: { + registerPickOwner: (id, predicate) => pickOwners.set(id, predicate), + unregisterPickOwner: (id) => pickOwners.delete(id), + }, + overlays: { + setOverlayEntries: (id, entries) => overlaySources.set(id, entries), + setOverlaySourceVisible() {}, + clearOverlaySource: (id) => overlaySources.delete(id), + }, + credits: { + registerDynamicCredit: (_viewer, credit) => credits.push(credit?.key), + transitFeedCredit: (feed) => ({ + key: `transit-${feed.id}`, + html: feed.name, + }), + }, + }, + }); + + installRenderGovernor(viewer); + layer.init(viewer); + + t.after(() => { + try { + layer.destroy(viewer); + } catch { + /* a test may already have destroyed it */ + } + uninstallRenderGovernor(viewer); + _resetRenderGovernorForTest(); + globalThis.fetch = originalFetch; + if (originalDocument === undefined) delete globalThis.document; + else globalThis.document = originalDocument; + if (originalWindow === undefined) delete globalThis.window; + else globalThis.window = originalWindow; + }); + + return { + layer, + viewer, + primitives, + sprites, + pickOwners, + overlaySources, + credits, + requested, + serve(feedId, responder) { + responders.set(feedId, responder); + }, + silence(feedId) { + responders.delete(feedId); + }, + state: () => layer._transitStateForTest(), + vehicles: () => [...layer._transitStateForTest()._vehicles.values()], + frame() { + preRender.raiseEvent(); + }, + advance(ms) { + t.mock.timers.tick(ms); + }, + /** + * Let wall time pass the way it does in the app: with frames along the + * way. The display clock only advances on frames, so a test that moves + * the clock without them models a tab in the background, not a viewer. + * @param {number} ms + * @param {number} [step] + */ + run(ms, step = 1_000) { + const until = Date.now() + ms; + while (Date.now() + step <= until) { + t.mock.timers.tick(step); + preRender.raiseEvent(); + } + if (Date.now() < until) { + t.mock.timers.tick(until - Date.now()); + preRender.raiseEvent(); + } + }, + holds: () => getRenderGovernorDiagnostics().holds, + ground, + meshCalls, + /** + * Run the floor cycle the way its timer would, then a frame. Ground + * resolution is a network round trip in the real app and deliberately + * lives off the render path, so a vehicle with no floor yet is not drawn + * until a re-read picks one up. + * @param {number} [rounds] + */ + settle(rounds = 4) { + for (let i = 0; i < rounds; i += 1) { + const parts = layer._transitPartsForTest(); + parts.height.rereadFloors(); + // The shown/hidden sweep is throttled in the app; running it here + // keeps the clock out of it, so a test about glide timing is not also + // a test about how long a sweep waits. + parts.rendering.refreshVisibility(); + t.mock.timers.tick(16); + preRender.raiseEvent(); + } + }, + /** What the style manager dispatches when the user picks a preset. */ + style(name) { + globalThis.window.dispatchEvent( + new CustomEvent('gev:style-change', { detail: { style: name } }), + ); + }, + /** What the style manager dispatches after every sync, cockpit included. */ + vision(style, cockpit) { + globalThis.window.dispatchEvent( + new CustomEvent('gev:vision-change', { detail: { style, cockpit } }), + ); + }, + /** Move the camera's view rectangle, the way a real camera move would. */ + lookAt(next) { + Object.assign(viewRect, next); + cameraChanged.raiseEvent(); + }, + }; +} + +test('a vehicle is played back a fixed lag behind its fixes, at 1x', async (t) => { + const app = harness(t); + const start = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + const [entry] = app.vehicles(); + assert.equal(entry.to.lat, 42.36); + assert.equal( + entry.sample.fromSeq, + entry.sample.toSeq, + 'a brand-new vehicle holds at its only fix', + ); + assert.equal(app.state()._moving.has(entry), false); + assert.ok( + Math.abs(Date.now() - entry.playT - LAG_MS) < 1, + 'its display clock starts a full lag behind', + ); + + // A later poll gives it a new fix. Nothing is drawn yet: the display clock + // has not reached the first fix. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.362, -71.06, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + await app.layer.update(); + assert.ok( + entry.wakeTimer != null, + 'one wake is scheduled at the first motion boundary', + ); + app.frame(); + assert.ok(Math.abs(lat(entry) - 42.36) < 1e-6, 'still at the first fix'); + + // The clock reaches the first fix LAG_MS after the wall time it was + // reported at; halfway along the fifteen seconds to the second fix the + // vehicle is drawn halfway between them. + const firstFixAt = start - AGE_MS; + app.run(firstFixAt + LAG_MS + TRANSIT_POLL_MS / 2 - Date.now()); + assert.ok(Math.abs(lat(entry) - 42.361) < 1e-4, `halfway, got ${lat(entry)}`); + + // Past the second fix it holds exactly there and stops being animated. + app.run(TRANSIT_POLL_MS); + assert.ok(Math.abs(lat(entry) - 42.362) < 1e-6); + assert.equal(app.state()._moving.has(entry), false, 'held, not animated'); +}); +test('a vehicle the feed stops reporting is removed, and its primitive with it', async (t) => { + const app = harness(t); + const now = Date.now(); + const both = [ + vehicle('bus-1', 42.36, -71.06, now / 1000), + vehicle('bus-2', 42.37, -71.07, now / 1000), + ]; + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', both, { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 2); + const collection = app.sprites.get('transit'); + assert.equal(collection.length, 2); + + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.361, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + // One missed poll is tolerated (feeds drop a vehicle for a beat), two is not. + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + assert.equal(app.vehicles().length, 2); + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + assert.deepEqual( + app.vehicles().map((entry) => entry.record.id), + ['bus-1'], + ); + assert.equal( + collection.length + app.sprites.get('transit-motion').length, + 1, + 'the removed vehicle released its point primitive', + ); +}); + +test('one feed stale beside one fresh reads DEGRADED, not nominal', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + + const state = app.state(); + // Pretend a second feed is also in range and is being served from the proxy's + // stale cache — exactly the mixed case the old `every(stale)` call missed. + state._activeFeeds.set('ovapi-nl', { + id: 'ovapi-nl', + name: 'OVapi', + region: 'Netherlands', + }); + state._feedStatus.set('ovapi-nl', { + count: 4, + lastUpdate: Date.now(), + error: null, + stale: true, + pollSeq: 1, + loading: false, + }); + const mixed = app.layer.getStats(); + assert.equal(mixed.stale, false, 'not everything is stale'); + assert.equal(mixed.degraded, true, 'but the view is part live and part not'); + + // When every feed is stale the row says STALE outright. + state._feedStatus.get('mbta').stale = true; + const allStale = app.layer.getStats(); + assert.equal(allStale.stale, true); +}); + +test('a feed that goes silent has its vehicles evicted instead of frozen on the globe', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 1); + + // The proxy's stale window expires and the feed stops answering at all. + app.silence('mbta'); + app.advance(FEED_EVICT_AFTER_MS + 1_000); + await app.layer.update(); + assert.equal(app.vehicles().length, 0, 'the last fleet is not left standing'); + assert.equal(app.sprites.get('transit').length, 0); + const stats = app.layer.getStats(); + assert.equal(stats.count, 0); + assert.equal(stats.stale, true); +}); + +test('inside a fresh snapshot, only the vehicle fixes that are too old are dropped', async (t) => { + const app = harness(t); + const now = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('fresh', 42.36, -71.06, Date.now() / 1000), + // A live feed really does carry these: a vehicle parked since this + // morning, still listed, still reporting its last known position. + vehicle( + 'ancient', + 42.37, + -71.07, + (Date.now() - 4 * 60 * 60 * 1000) / 1000, + ), + // A clock ahead of ours is treated as current, never as negative age. + vehicle('future', 42.38, -71.08, (Date.now() + 60_000) / 1000), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.deepEqual( + app + .vehicles() + .map((entry) => entry.record.id) + .sort(), + ['fresh', 'future'], + ); + + // The same vehicles, unrefreshed, age out of the layer on a later sweep. + app.silence('mbta'); + app.advance(VEHICLE_MAX_FIX_AGE_MS + 1_000); + await app.layer.update(); + assert.equal(app.vehicles().length, 0); + assert.equal(now <= Date.now(), true); +}); + +test('a snapshot the proxy replayed from its own cache for six minutes is refused, not drawn', async (t) => { + const app = harness(t); + const fetchedAt = Date.now(); + const vehicles = [ + vehicle('bus-1', 42.36, -71.06, fetchedAt / 1000), + vehicle('bus-2', 42.37, -71.07, fetchedAt / 1000), + ]; + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', vehicles, { fetchedAt }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 2); + + // The operator stops answering. The proxy keeps handing back the SAME + // snapshot every 15 s with STALE-ERROR; receiving bytes on a fresh cadence is + // not the same as the buses being where they say. + app.serve('mbta', () => ({ + status: 200, + headers: { 'x-gev-cache': 'STALE-ERROR' }, + body: snapshot('mbta', 'MBTA', vehicles, { fetchedAt }), + })); + for (let tick = 0; tick < 24; tick += 1) { + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + } + assert.ok( + Date.now() - fetchedAt > FEED_EVICT_AFTER_MS, + 'six minutes have passed', + ); + assert.equal( + app.vehicles().length, + 0, + 'the replayed fleet is gone, not frozen', + ); + assert.equal(app.sprites.get('transit').length, 0); + const stats = app.layer.getStats(); + assert.equal(stats.count, 0); + assert.equal(stats.stale, true); + assert.match(stats.coverage, /\(\d+s old\)/); + assert.equal( + app.holds().includes('transit'), + false, + 'and it holds no frames', + ); +}); + +test('disable aborts in-flight work and a response that lands late is ignored', async (t) => { + const app = harness(t); + let release; + const pending = new Promise((resolve) => { + release = resolve; + }); + for (const name of [ + 'HTMLCanvasElement', + 'HTMLImageElement', + 'ImageBitmap', + 'OffscreenCanvas', + ]) { + const prior = globalThis[name]; + globalThis[name] = class {}; + t.after(() => { + if (prior === undefined) delete globalThis[name]; + else globalThis[name] = prior; + }); + } + t.mock.method(performance, 'now', () => Date.now() - 1_789_000_000_000); + const originalFetch = globalThis.fetch; + globalThis.fetch = async (url) => { + await pending; + return originalFetch(url); + }; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + const inFlight = app.layer.update(); + + app.layer.disable(app.viewer); + release(); + await inFlight; + assert.equal( + app.vehicles().length, + 0, + 'a poll from the disabled generation never lands', + ); + assert.equal(app.state()._inFlight.size, 0); + assert.equal(app.holds().includes('transit'), false); + + // Re-enabling starts a clean generation and does take data. + globalThis.fetch = originalFetch; + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 1); +}); + +test('a vehicle waits for its floor rather than standing on the ellipsoid', async (t) => { + // The shared ground-floor system answers from the rendered mesh cell or the + // real DEM cell; until one of them is warm there is no honest height to draw + // at. Ellipsoid zero is not street level, and drawing there and dropping + // afterwards is exactly the floating the owner reported. + let terrainReady = false; + const app = harness(t, { floorAt: () => (terrainReady ? 47 : undefined) }); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + const entry = app.vehicles()[0]; + assert.equal(entry.heightResolved, false); + app.settle(); + assert.equal(entry.heightPending, true, 'it has nothing to stand on'); + assert.equal(entry.marker.show, false, 'so it is not drawn'); + + // Terrain arrives; the next re-read picks the cell up and the vehicle + // appears, standing on it. + terrainReady = true; + app.settle(); + assert.equal(entry.heightResolved, true); + assert.equal(entry.heightM, 47 + 1.5, 'the floor plus the shared lift'); + assert.equal(entry.marker.show, true); + + // Driving into a new cell asks that cell rather than keeping the one it was + // born in. + app.advance(TRANSIT_POLL_MS); + const cellBefore = entry.heightCell; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.362, -71.06, Date.now() / 1000)], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + assert.notEqual(app.vehicles()[0].heightCell, cellBefore); +}); + +test('the render hold follows real motion, and is released when everything settles', async (t) => { + const app = harness(t); + const start = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + // The first poll draws each vehicle at its fix; there is nothing to play, + // and ground resolution runs off the render path, so the loop is never held. + assert.equal( + app.holds().includes('transit'), + false, + 'a fleet that is not moving does not hold the loop', + ); + app.frame(); + assert.equal(app.holds().includes('transit'), false); + assert.equal(app.viewer.scene.requestRenderMode, true); + + // A second fix IS motion to come: the vehicle is armed and the loop held. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.362, -71.06, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + await app.layer.update(); + const firstFixAt = start - AGE_MS; + app.run(firstFixAt + LAG_MS + TRANSIT_POLL_MS / 2 - Date.now()); + assert.ok( + app.holds().includes('transit'), + 'a moving vehicle keeps the loop awake', + ); + assert.equal(app.viewer.scene.requestRenderMode, false); + + // Past the second fix it holds and the loop is released again. + app.run(TRANSIT_POLL_MS); + assert.equal( + app.holds().includes('transit'), + false, + 'a settled fleet does not hold the GPU', + ); + assert.equal(app.viewer.scene.requestRenderMode, true); +}); +test('disable hands back the camera sensitivity it borrowed', async (t) => { + const app = harness(t); + assert.equal(app.viewer.camera.percentageChanged, 0.5); + app.layer.enable(app.viewer); + assert.equal( + app.viewer.camera.percentageChanged, + 0.05, + 'the layer needs a finer camera trigger', + ); + app.layer.disable(app.viewer); + assert.equal(app.viewer.camera.percentageChanged, 0.5, 'and gives it back'); +}); + +test('disable leaves the fine camera trigger another layer asked for', async (t) => { + // The sequence that broke it: two layers ask for the SAME sensitivity, so + // the value on the camera cannot say who owns it. Transit used to compare + // the number, find its own 0.05 still there, and hand the coarse default + // back while Bikeshare was still driving off the fine one. + const app = harness(t); + assert.equal(app.viewer.camera.percentageChanged, 0.5); + + app.layer.enable(app.viewer); + assert.equal(app.viewer.camera.percentageChanged, 0.05); + + claimCameraSensitivity(app.viewer.camera, 'bikeshare', 0.05); + app.layer.disable(app.viewer); + + assert.equal( + app.viewer.camera.percentageChanged, + 0.05, + 'bikeshare is still enabled, so the fine trigger stays', + ); + assert.deepEqual(cameraSensitivityClaims(app.viewer.camera), ['bikeshare']); + + releaseCameraSensitivity(app.viewer.camera, 'bikeshare'); + assert.equal( + app.viewer.camera.percentageChanged, + 0.5, + 'and the original returns once the last claim is gone', + ); + assert.deepEqual(cameraSensitivityClaims(app.viewer.camera), []); +}); + +test('a finer claim wins while it stands, and the coarser one survives it', async (t) => { + const app = harness(t); + app.layer.enable(app.viewer); + claimCameraSensitivity(app.viewer.camera, 'traffic', 0.01); + assert.equal(app.viewer.camera.percentageChanged, 0.01); + + releaseCameraSensitivity(app.viewer.camera, 'traffic'); + assert.equal( + app.viewer.camera.percentageChanged, + 0.05, + 'transit still needs its own trigger', + ); + app.layer.disable(app.viewer); + assert.equal(app.viewer.camera.percentageChanged, 0.5); +}); + +test('destroy releases every shared registration this layer took', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.ok(app.sprites.has('transit')); + assert.ok(app.pickOwners.has('transit')); + assert.equal(app.primitives.length, 2); + + app.layer.destroy(app.viewer); + assert.equal( + app.sprites.has('transit'), + false, + 'the sprite collection is unregistered', + ); + assert.equal( + app.pickOwners.has('transit'), + false, + 'the pick owner is released', + ); + assert.equal( + app.primitives.length, + 0, + 'the primitive collection leaves the scene', + ); + assert.equal(app.holds().includes('transit'), false); + assert.equal(app.state()._viewer, null); + assert.equal(app.state()._dataManager, null); +}); + +test('two layers built from the factory share no state', async (t) => { + const app = harness(t); + const second = createTransitLayer({ + services: { + ground: groundStub(() => 12), + render: { + governorRequestRender, + holdContinuousRender, + releaseContinuousRender, + }, + sprites: { + registerSpriteCollection() {}, + unregisterSpriteCollection() {}, + restoreSpriteOrder() {}, + }, + picking: { registerPickOwner() {}, unregisterPickOwner() {} }, + overlays: { + setOverlayEntries() {}, + setOverlaySourceVisible() {}, + clearOverlaySource() {}, + }, + credits: { + registerDynamicCredit() {}, + transitFeedCredit: (feed) => ({ key: feed.id, html: feed.name }), + }, + }, + }); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 1); + assert.equal(second._transitStateForTest()._vehicles.size, 0); + assert.notEqual(second._transitStateForTest(), app.state()); + assert.equal(second.getStats().count, 0); +}); + +test('a proxy cooldown reaches the row as a time, not just a shrug', async (t) => { + const app = harness(t); + // The proxy answers 503 with how long it intends to leave the operator alone. + app.serve('mbta', () => ({ + status: 503, + body: { error: 'Transit feed unavailable', feedId: 'mbta', retryInSec: 60 }, + })); + app.layer.enable(app.viewer); + await app.layer.update(); + const stats = app.layer.getStats(); + assert.equal(stats.count, 0); + assert.match(String(stats.error), /MBTA/); + assert.equal(stats.retryInSec, 60); + + // A successful poll clears both the error and the countdown. + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + const recovered = app.layer.getStats(); + assert.equal(recovered.error, null); + assert.equal(recovered.retryInSec, undefined); +}); + +test('the coverage text ages a quiet feed by the same rule the row state uses', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, Date.now() / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.layer.getStats().coverage, 'MBTA 1'); + + // Past the stale threshold the text names the age, and the row agrees. + app.silence('mbta'); + app.advance(FEED_STALE_AFTER_MS + 10_000); + const stale = app.layer.getStats(); + assert.match(stale.coverage, /^MBTA 1 \(\d+s old\)$/); + assert.equal(stale.stale, true); +}); + +test('a parked fleet that keeps reporting the same position stops asking for frames', async (t) => { + const app = harness(t); + // A depot at night: the feed answers on time, every 15 s, with coordinates + // that do not change. Each of those is a real observation — the bus is + // still there — but there is nothing to draw, and drawing it anyway kept + // the render loop awake all night. + const parked = () => [ + vehicle('bus-1', 42.36, -71.06, reported()), + vehicle('bus-2', 42.37, -71.07, reported()), + ]; + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', parked(), { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.frame(); + assert.equal( + app.holds().includes('transit'), + false, + 'nothing moved on the first poll', + ); + + for (let poll = 0; poll < 4; poll += 1) { + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + app.frame(); + assert.equal( + app.holds().includes('transit'), + false, + `identical poll ${poll + 1} must not count as motion`, + ); + assert.equal(app.state()._moving.size, 0); + } + assert.equal(app.viewer.scene.requestRenderMode, true); + assert.equal(app.vehicles().length, 2, 'and the fleet is still on screen'); + + // One vehicle finally moves: that IS work, and the hold comes back. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('bus-1', 42.3605, -71.06, reported()), + vehicle('bus-2', 42.37, -71.07, reported()), + ], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + assert.equal( + app.holds().includes('transit'), + false, + 'departure remains buffered until its boundary', + ); + app.advance(6000); + app.frame(); + assert.ok( + app.holds().includes('transit'), + 'departure wakes the loop at its boundary', + ); + app.advance(TRANSIT_POLL_MS / 2); + app.frame(); + assert.ok(app.holds().includes('transit')); + assert.equal( + app.state()._moving.size, + 1, + 'only the vehicle that actually moved is animating', + ); +}); +test('the last vehicle leaving mid-segment takes the render hold with it', async (t) => { + const app = harness(t); + const fetchedAt = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.36, -71.06, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + + // Second poll: the vehicle has somewhere to go, so the hold is taken. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus-1', 42.362, -71.06, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + await app.layer.update(); + app.advance(TRANSIT_POLL_MS / 3); + app.frame(); + assert.ok(app.holds().includes('transit'), 'in motion, so the loop is held'); + assert.equal(app.state()._moving.has(app.vehicles()[0]), true); + + // The feed dies. The sweep removes the last vehicle while it is still + // in motion, and the poll that discovers this fails — so no frame follows to + // notice the fleet is gone. The hold has to be released by the eviction. + app.silence('mbta'); + app.advance(FEED_EVICT_AFTER_MS + 1_000); + await app.layer.update(); + assert.equal(app.vehicles().length, 0); + assert.equal( + app.holds().includes('transit'), + false, + 'no hold may outlive the fleet that justified it', + ); + assert.equal(app.viewer.scene.requestRenderMode, true); + assert.ok(Date.now() - fetchedAt > FEED_EVICT_AFTER_MS); +}); +test('only the vehicles near the view are animated frame by frame', async (t) => { + const app = harness(t); + const near = vehicle('near', 42.36, -71.06, reported()); + // Far enough away to be outside any plausible inflation of the harness view + // rectangle, but well inside the feed's coverage circle. + const far = vehicle('far', 42.62, -71.44, reported()); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [near, far], { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.frame(); + + // Both move on the next poll; only the one in view is armed at all. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('near', 42.3625, -71.06, reported()), + vehicle('far', 42.6225, -71.44, reported()), + ], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + const byId = new Map(app.vehicles().map((entry) => [entry.record.id, entry])); + app.advance(6000); + app.frame(); + assert.equal( + app.state()._moving.size, + 1, + 'a poll arms only the vehicle in view', + ); + assert.ok(app.state()._moving.has(byId.get('near'))); + assert.equal( + byId.get('far').marker.show, + false, + 'the off-screen vehicle is hidden', + ); + + app.advance(TRANSIT_POLL_MS / 4); + app.frame(); + assert.equal( + app.state()._moving.has(byId.get('far')), + false, + 'and is never animated', + ); + assert.ok( + app.state()._moving.has(byId.get('near')), + 'the one in view keeps playing', + ); + + // When the view reaches it, its clock is re-synced and the frame pass places + // it: nothing is owed for motion nobody watched. + app.lookAt({ west: -71.54, south: 42.52, east: -71.34, north: 42.72 }); + // Two floor passes: it was never warmed while hidden, and a warmed cell + // answers on the pass after the one that asked. + app.settle(2); + app.advance(250); + app.frame(); + assert.notEqual(byId.get('far').marker.show, false, 'now shown'); + assert.ok( + Math.abs(lat(byId.get('far')) - 42.62) < 1e-3, + 'placed on its history', + ); +}); +test('a wide oblique view does not hide the fleet that is plainly in it', async (t) => { + // Codex's reproduction: a high oblique camera over Europe sees roughly + // -78°..88° of longitude. Padding each endpoint on its own wrapped that span + // into a five-degree slice of the Pacific, so every Dutch vehicle failed the + // view gate — hidden, unanimated, and invisible to DETECT — while filling + // the screen. + const app = harness(t, { + at: { lat: 52.09, lon: 5.12 }, + altitude: 1_500_000, + view: { west: -78.084, south: 20, east: 88.084, north: 84 }, + }); + const now = Date.now(); + const utrecht = vehicle('nl-1', 52.09, 5.12, now / 1000); + const rotterdam = vehicle('nl-2', 51.92, 4.48, now / 1000); + app.serve('ovapi-nl', () => ({ + status: 200, + body: snapshot('ovapi-nl', 'OVapi', [utrecht, rotterdam], { + fetchedAt: Date.now(), + }), + })); + + app.layer.enable(app.viewer); + await app.layer.update(); + app.advance(250); + app.frame(); + + const shown = app.vehicles().filter((entry) => entry.marker?.show !== false); + assert.equal(shown.length, 2, 'both vehicles are in the view the camera has'); + assert.equal( + app.layer.getDetectableObjects({ maxCount: 50, seed: 0 }).length, + 2, + 'and DETECT can see them', + ); +}); + +test('a lone visible vehicle is offered to DETECT out of a large hidden fleet', async (t) => { + // The stride used to run over the WHOLE fleet and discard the hidden ones + // afterwards, so one visible bus among thousands could fall outside the + // sample and go unlabelled while the overlay had capacity to spare. + const app = harness(t); + const now = Date.now(); + const vehicles = []; + for (let i = 0; i < 400; i += 1) { + // Far outside the harness view rectangle, inside MBTA coverage. + vehicles.push(vehicle(`away-${i}`, 42.6 + i * 0.001, -71.45, now / 1000)); + // Deliberately NOT first in the map: an ordinal stride happens to pick + // index zero, which would hide the fault rather than show it. + if (i === 0) vehicles.push(vehicle('visible', 42.361, -71.058, now / 1000)); + } + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', vehicles, { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + const contacts = app.layer.getDetectableObjects({ maxCount: 8, seed: 0 }); + assert.equal(contacts.length, 1, 'only the visible one is a candidate'); + assert.equal(contacts[0].sourceId, 'mbta:visible'); +}); + +test('removing one vehicle does not hand every label to its neighbour', async (t) => { + // Striding by position meant that one bus leaving at the front renumbered + // everyone behind it: the sampled set flipped from even indices to odd, and + // not one of the previous identities survived. + const app = harness(t, { + view: { west: -71.3, south: 42.2, east: -70.8, north: 42.5 }, + }); + const now = Date.now(); + const fleet = []; + for (let i = 0; i < 40; i += 1) { + fleet.push(vehicle(`bus-${i}`, 42.35 + i * 0.001, -71.06, now / 1000)); + } + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', fleet, { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + const before = app.layer + .getDetectableObjects({ maxCount: 10, seed: 0 }) + .map((contact) => contact.sourceId); + assert.equal(before.length, 10); + + // The first vehicle stops being reported; everyone else is unchanged. It + // takes MISSED_POLLS_TO_DROP polls of silence for it to actually leave. + for (let poll = 0; poll < MISSED_POLLS_TO_DROP + 1; poll += 1) { + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + fleet.slice(1).map((v) => ({ ...v, timestamp: Date.now() / 1000 })), + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + app.frame(); + } + assert.equal( + app.vehicles().some((entry) => entry.record.id === 'bus-0'), + false, + 'the vehicle really did leave', + ); + + const after = new Set( + app.layer + .getDetectableObjects({ maxCount: 10, seed: 0 }) + .map((contact) => contact.sourceId), + ); + const survived = before.filter( + (id) => id !== 'mbta:bus-0' && after.has(id), + ).length; + const expected = before.filter((id) => id !== 'mbta:bus-0').length; + assert.equal( + survived, + expected, + 'every still-present labelled vehicle keeps its label', + ); +}); + +test('a vehicle inside the view drives out of it instead of vanishing', async (t) => { + // A fix placing the DESTINATION just past the padded edge used to hide the + // marker on the spot — while the vehicle was still drawn well inside the + // view — and throw its glide away in the same frame. + const app = harness(t, { + view: { west: -71.062, south: 42.3585, east: -71.058, north: 42.3605 }, + }); + const now = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, now / 1000)], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + assert.notEqual(entry.marker.show, false, 'it starts on screen'); + + // Next fix is beyond the inflated rectangle; the vehicle is still drawn + // inside it. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', 42.3625, -71.06, Date.now() / 1000)], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + app.frame(); + + assert.notEqual( + entry.marker.show, + false, + 'it is still on screen, because it still is', + ); + assert.ok( + entry.wakeTimer != null || app.state()._moving.has(entry), + 'and it has a departure wake rather than teleporting', + ); +}); + +test('the selected card lands where the vehicle stopped, not short of it', async (t) => { + // The card is refreshed on a 250 ms throttle. The last tick before a + // segment ends left it a few metres behind, and then the render hold was + // released with no frame scheduled to close the gap. + const app = harness(t); + const start = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + // Its floor resolves, so the marker is shown: a hidden vehicle has no card. + app.settle(); + + const entry = app.vehicles()[0]; + app.layer._transitPartsForTest().selection.selectVehicle(entry.key); + + app.advance(TRANSIT_POLL_MS); + // Stamped exactly one poll after the first fix, so the vehicle's own lag + // equals the feed's and the clock runs at exactly 1x through the segment. + const secondFixS = (entry.fixes[0].t + TRANSIT_POLL_MS) / 1000; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', 42.3625, -71.0625, secondFixS)], + { + fetchedAt: Date.now(), + }, + ), + })); + await app.layer.update(); + + // Codex's sequence: a card refresh 200 ms before the segment completes, then + // the settling frame — which falls INSIDE the 250 ms card throttle, so + // nothing but a forced refresh can close the gap. + // The segment ends when the clock reaches the second fix, a lag after it. + const segmentEndsAt = entry.fixes[1].t + LAG_MS; + app.run(segmentEndsAt - 200 - Date.now()); + // The settling frame — whichever frame that turns out to be — falls INSIDE + // the card throttle, so only the forced refresh can close the gap. + let frames = 0; + while (app.state()._moving.has(entry) && frames < 60) { + app.advance(16); + app.frame(); + frames += 1; + } + assert.equal(app.state()._moving.has(entry), false, 'settled'); + assert.equal( + app.state()._selectedCardAt, + Date.now(), + 'the settling frame forced a card refresh, throttle or no throttle', + ); + + const card = app.overlaySources.get('transit-selected')?.[0]; + assert.ok(card, 'the card is still on screen'); + const anchor = + typeof card.position === 'function' ? card.position() : card.position; + const drift = Cesium.Cartesian3.distance(anchor, entry.marker.position); + assert.ok( + drift < 0.5, + `the card sits on the vehicle it follows (was ${drift.toFixed(2)} m behind)`, + ); +}); +test('a parked fleet over unsampled ground off screen stops asking for frames', async (t) => { + // 200 stationary vehicles in distinct off-screen cells whose terrain never + // resolves used to keep the height queue non-empty for ever, and a non-empty + // queue is a held render loop over a view with nothing moving in it. + const app = harness(t, { floorAt: () => undefined }); + const now = Date.now(); + const parked = []; + for (let i = 0; i < 200; i += 1) { + parked.push(vehicle(`p-${i}`, 42.5 + i * 0.003, -71.4, now / 1000)); + } + const serve = () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', parked, { fetchedAt: Date.now() }), + }); + app.serve('mbta', serve); + app.layer.enable(app.viewer); + await app.layer.update(); + app.frame(); + + // Eight polls — two minutes — with nothing moving and nothing in view. + for (let poll = 0; poll < 8; poll += 1) { + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', serve); + await app.layer.update(); + for (let f = 0; f < 4; f += 1) { + app.advance(100); + app.frame(); + } + } + + assert.equal( + app.ground._warmed.length, + 0, + 'no ground was asked for on behalf of vehicles nobody can see', + ); + assert.equal( + app.holds().includes('transit'), + false, + 'and the render loop is not held over an empty view', + ); +}); + +test('a height that arrives from a neighbour lifts a stationary vehicle too', async (t) => { + // A cache hit updated heightM but the billboard endpoints only refreshed on + // a lat/lon change, so a bus that had not moved kept drawing at ellipsoid + // height beside a neighbour standing on the street. + const app = harness(t, { floorAt: () => 125 }); + const now = Date.now(); + const parked = vehicle('still', 42.36, -71.06, now / 1000); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [parked], { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + + const entry = app.vehicles()[0]; + // Pretend the floor landed for this cell while the vehicle was absent: the + // shared cache knows the height, the entry does not. + entry.heightM = 0; + entry.heightResolved = false; + entry.heightPending = false; + app.ground._warm.set(app.ground._key(42.36, -71.06), 125); + + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('still', 42.36, -71.06, Date.now() / 1000)], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + app.frame(); + + const carto = Cesium.Cartographic.fromCartesian(entry.marker.position); + assert.ok( + Math.abs(carto.height - 126.5) < 1, + `the vehicle stands on the ground its cell reported (drew at ${carto.height.toFixed(1)} m)`, + ); +}); + +test('a feed answering 304 every poll stays healthy instead of ageing out', async (t) => { + // The proxy keeps serving the body it holds, whose fetch time stops + // advancing. Reading that as "when did we last hear from them" marked a feed + // whose every request succeeded as DEGRADED at 90 s and emptied it at five + // minutes. + const app = harness(t); + const firstFetch = Date.now(); + const body = snapshot( + 'mbta', + 'MBTA', + [vehicle('b', 42.36, -71.06, firstFetch / 1000)], + { fetchedAt: firstFetch }, + ); + app.serve('mbta', () => ({ + status: 200, + body, + headers: { 'x-transit-contact': String(Date.now()) }, + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 1); + + // Twenty-four polls — six minutes — of successful revalidation. + for (let poll = 0; poll < 24; poll += 1) { + app.advance(16_000); + app.serve('mbta', () => ({ + status: 200, + body, + headers: { 'x-transit-contact': String(Date.now()) }, + })); + await app.layer.update(); + } + + const stats = app.layer.getStats(); + assert.equal(app.vehicles().length, 1, 'the vehicle is still on the globe'); + assert.equal(stats.stale, false, 'and the feed does not read as gone quiet'); +}); + +test('a feed that has genuinely stopped answering still ages out', async (t) => { + // The other half of the same rule: a contact time that does NOT advance is + // still silence, and silence must empty the map. + const app = harness(t); + const firstFetch = Date.now(); + const body = snapshot( + 'mbta', + 'MBTA', + [vehicle('b', 42.36, -71.06, firstFetch / 1000)], + { fetchedAt: firstFetch }, + ); + app.serve('mbta', () => ({ + status: 200, + body, + headers: { 'x-transit-contact': String(firstFetch) }, + })); + app.layer.enable(app.viewer); + await app.layer.update(); + assert.equal(app.vehicles().length, 1); + + app.advance(FEED_EVICT_AFTER_MS + 30_000); + app.serve('mbta', () => ({ + status: 200, + body, + headers: { 'x-transit-contact': String(firstFetch) }, + })); + await app.layer.update(); + assert.equal(app.vehicles().length, 0, 'a silent feed empties the globe'); +}); + +test('a vehicle with no surface yet waits instead of floating', async (t) => { + // Ellipsoid zero is not street level — in Boston it is about thirty metres + // underground — so a marker drawn at +3 m while its sample is pending sits + // above its final height and then drops. The owner saw that as floating. + let allowSamples = false; + const app = harness(t, { + floorAt: () => (allowSamples ? 30 : undefined), + }); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', 42.36, -71.06, Date.now() / 1000)], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.frame(); + + const entry = app.vehicles()[0]; + assert.equal( + entry.heightPending, + true, + 'it knows it has nothing to stand on', + ); + assert.equal(entry.marker.show, false, 'so it is not drawn'); + + allowSamples = true; + app.settle(); + assert.equal(entry.heightPending, false); + assert.equal(entry.marker.show, true, 'and appears once the ground answers'); + const carto = Cesium.Cartographic.fromCartesian(entry.marker.position); + assert.ok( + Math.abs(carto.height - 31.5) < 1, + `standing on its floor plus the shared lift, got ${carto.height.toFixed(1)} m`, + ); +}); + +test('a new vehicle borrows the lowest surface resolved nearby', async (t) => { + // The first visible frame uses the best prior available rather than a flat + // +3 m, and the prior is biased LOW so a marker can only rise onto its own + // surface, never appear above it and fall. + const app = harness(t, { floorAt: () => 40 }); + const early = [ + vehicle('a', 42.36, -71.06, Date.now() / 1000), + vehicle('b', 42.361, -71.06, Date.now() / 1000), + vehicle('c', 42.36, -71.061, Date.now() / 1000), + vehicle('d', 42.361, -71.061, Date.now() / 1000), + ]; + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', early, { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + // A second vehicle appears a block away, in a cell of its own whose + // neighbours are warm. The shared floor wants at least three resolved + // neighbours before it will speak for a cold cell. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + ...early.map((v) => ({ ...v, timestamp: Date.now() / 1000 })), + // Its own cell is cold; the three around it are warm. + vehicle('second', 42.3605, -71.0605, Date.now() / 1000), + ], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + + const second = app.vehicles().find((e) => e.record.id === 'second'); + assert.equal( + second.heightPending, + false, + 'it has ground to stand on at once', + ); + assert.equal(second.heightM, 41.5, 'borrowed from the neighbourhood'); + app.advance(250); + app.frame(); + assert.equal(second.marker.show, true); +}); + +test('only the vehicles in view have their ground warmed', async (t) => { + // Warming is a network round trip through the shared resolver. A parked + // metropolitan fleet off screen used to keep a queue — and with it the + // render loop — alive indefinitely over an empty view. + const app = harness(t, { floorAt: () => undefined }); + const warmed = app.ground._warmed; + const fleet = [vehicle('near', 42.36, -71.06, Date.now() / 1000)]; + for (let i = 0; i < 50; i += 1) { + fleet.push( + vehicle(`far-${i}`, 42.6 + i * 0.002, -71.45, Date.now() / 1000), + ); + } + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', fleet, { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + assert.ok(warmed.length > 0, 'the one in view is warmed'); + const offScreen = warmed.filter((point) => point.lat > 42.5); + assert.equal(offScreen.length, 0, 'and nothing off screen is'); + assert.equal( + app.holds().includes('transit'), + false, + 'a fleet waiting for ground is not a fleet worth burning frames on', + ); +}); + +test('a segment climbs from the ground it left to the ground it reaches', async (t) => { + // Rebuilding both ends of a segment at the destination's altitude lifted the + // vehicle onto the new height on the FIRST frame: it flew up, then + // travelled. With the display running a lag behind, the trap is subtler — + // the floor is asked for the NEWEST fix while an older one is still being + // drawn — so each fix carries its own floor. + const app = harness(t, { floorAt: (lat) => (lat > 42.3615 ? 100 : 10) }); + const start = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + assert.ok(Math.abs(entry.heightM - 11.5) < 1, 'it starts on the low ground'); + + // The destination's surface is already known — a hundred metres uphill — so + // the poll resolves it immediately, on the fix that stands there. + app.ground._warm.set(app.ground._key(42.362, -71.06), 100); + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.362, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + await app.layer.update(); + assert.ok(Math.abs(entry.heightM - 101.5) < 1, 'the destination is uphill'); + const height = () => + Cesium.Cartographic.fromCartesian(entry.marker.position).height; + app.run(400); + assert.ok( + height() < 30, + `still on the ground it is drawn on, got ${height().toFixed(1)} m`, + ); + + // Halfway along the segment it is halfway up; at the end, on the high ground. + const firstFixAt = start - AGE_MS; + app.run(firstFixAt + LAG_MS + TRANSIT_POLL_MS / 2 - Date.now()); + assert.ok( + Math.abs(height() - 11.5) < 3, + `at the corridor floor, got ${height().toFixed(1)} m`, + ); + app.run(TRANSIT_POLL_MS); + assert.ok( + Math.abs(height() - 101.5) < 1.5, + `and arrives on the high ground, got ${height().toFixed(1)} m`, + ); +}); +test('a lone impossible fix leaves the displayed vehicle and health untouched', async (t) => { + const app = harness(t, { floorAt: () => 5 }); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + + // Eleven kilometres in one poll: 2,600 km/h. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.46, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + await app.layer.update(); + + assert.equal( + entry.sample.fromSeq, + entry.sample.toSeq, + 'no segment was started', + ); + assert.equal(app.state()._moving.has(entry), false); + assert.equal(entry.fixes.length, 1, 'the accepted history is preserved'); + assert.equal(entry.resets, 0, 'one bad fix cannot reset playback'); + assert.deepEqual( + [entry.to.lat, entry.to.lon], + [42.36, -71.06], + 'it remains at its accepted position', + ); + assert.ok(Math.abs(lat(entry) - 42.36) < 1e-6); +}); +test('the detection metric says what the display is doing, not what the feed claims', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('m', 42.36, -71.06, Date.now() / 1000, { + routeId: 'Green-C', + status: 'STOPPED_AT', + stopId: '49001', + }), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + // It moves; the feed still says STOPPED_AT. + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('m', 42.362, -71.06, Date.now() / 1000, { + routeId: 'Green-C', + status: 'STOPPED_AT', + stopId: '49001', + }), + ], + { fetchedAt: Date.now() }, + ), + })); + await app.layer.update(); + app.advance(2_000); + app.frame(); + + const [contact] = app.layer.getDetectableObjects({ maxCount: 5, seed: 0 }); + assert.ok(contact, 'the vehicle is a contact'); + assert.equal( + contact.metric.includes('STOPPED'), + false, + `a moving vehicle is not stopped, got "${contact.metric}"`, + ); + assert.equal(contact.id, 'GREEN-C'); + assert.ok( + contact.klass.includes('METRO') || contact.klass.includes('TRAM'), + `the class carries the mode, got "${contact.klass}"`, + ); +}); + +test('the card keeps the operator claim as a report and says the mode plainly', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('m', 42.36, -71.06, Date.now() / 1000, { + routeId: 'Red', + status: 'STOPPED_AT', + stopId: '49001', + }), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + const entry = app.vehicles()[0]; + app.layer._transitPartsForTest().selection.selectVehicle(entry.key); + const card = app.overlaySources.get('transit-selected')?.[0]; + assert.ok(card, 'a card is shown'); + const text = card.details.join(' | '); + assert.match(text, /Subway/, 'the mode is said in words'); + assert.match(text, /Last report: stopped at stop 49001/); + assert.match( + text, + /Shown at street level/, + 'and the surface caveat is stated', + ); +}); + +test('a vehicle reporting every 30 s moves continuously at its own speed', async (t) => { + // Measured on the live MBTA feed: half the fleet updates every thirty + // seconds while the browser polls every fifteen. Gliding over a fixed poll + // interval drew that half-minute of travel in fifteen seconds and then left + // the fleet perfectly still for the other fifteen — 46 vehicles moving, then + // zero, then 43, then zero. Played back a lag behind, the segment takes the + // thirty seconds the fixes describe and runs straight into the next one. + const app = harness(t, { floorAt: () => 5 }); + const start = Date.now(); + const firstFixAt = start - AGE_MS; + let fixAt = firstFixAt; + let north = 42.36; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', north, -71.06, fixAt / 1000)], + { + fetchedAt: Date.now(), + }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + + // Polls at 15 s (a repeat), 30 s (moved 0.0025°), 45 s (repeat), 60 s (moved)… + const samples = []; + // Each poll lands at the end of the previous fifteen frames, the way the + // manager tick does in the app: frames run continuously between polls. + for (let poll = 1; poll <= 6; poll += 1) { + if (poll % 2 === 0) { + fixAt += 2 * TRANSIT_POLL_MS; + north += 0.0025; + } + await app.layer.update(); + for (let f = 0; f < 15; f += 1) { + app.advance(1_000); + app.frame(); + samples.push({ at: Date.now(), lat: lat(entry), playT: entry.playT }); + } + } + // A thirty-second reporter under a thirty-five-second prior holds once, + // before its first segment: nobody knows its cadence until the second fix + // arrives, and the hold is what earns it the longer lag. From the first + // frame it moves, the clock runs at exactly one second per second with no + // dead interval, and the drawn speed is the fix speed. + const firstMove = samples.findIndex( + (s, i) => i > 0 && s.lat !== samples[i - 1].lat, + ); + assert.ok(firstMove > 0, 'it starts moving'); + const playing = samples.filter( + (s, i) => i >= firstMove && s.playT < entry.fixes.at(-1).t, + ); + assert.ok( + playing.length >= 40, + `long enough to judge (${playing.length} samples)`, + ); + for (let i = 1; i < playing.length; i += 1) { + const dPlay = playing[i].playT - playing[i - 1].playT; + const dWall = playing[i].at - playing[i - 1].at; + // 1x, or the 0.95x it runs at while banking the longer lag it has just + // learned it needs — never a freeze, never a rush. + assert.ok( + dPlay <= dWall + 1 && dPlay >= 0.95 * dWall - 1, + `no freeze and no rush (${dPlay} ms of play in ${dWall} ms)`, + ); + const metresPerS = + ((playing[i].lat - playing[i - 1].lat) * 111_320) / (dWall / 1000); + const fixSpeed = (0.0025 * 111_320) / ((2 * TRANSIT_POLL_MS) / 1000); + assert.ok( + metresPerS <= fixSpeed * 1.01 + 0.05 && + metresPerS >= fixSpeed * 0.94 - 0.05, + `at the fix speed: ${metresPerS.toFixed(2)} vs ${fixSpeed.toFixed(2)} m/s`, + ); + } +}); + +test('the selected card anchors to the live marker every frame and republishes text only on change', async (t) => { + // The anchor was a clone refreshed on a 250 ms throttle under a 60 Hz + // sprite, so the card stepped behind a smooth vehicle. It is a getter now, + // read by the overlay host on every paint; the text keeps its throttle and + // is re-sent only when it has changed. + const app = harness(t); + const start = Date.now(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + // Its floor resolves, so the marker is shown: a hidden vehicle has no card. + app.settle(); + const entry = app.vehicles()[0]; + const published = []; + app.layer._setTransitOverlayHostForTest({ + setEntries: (id, entries) => published.push(entries[0]), + setVisible() {}, + clearSource() {}, + }); + app.layer._transitPartsForTest().selection.selectVehicle(entry.key); + assert.equal(published.length, 1); + const card = published[0]; + assert.equal(typeof card.position, 'function', 'the anchor is a getter'); + + app.advance(TRANSIT_POLL_MS); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', 42.3625, -71.0625, reported())], + { + fetchedAt: Date.now(), + }, + ), + })); + await app.layer.update(); + // Frames at 16 ms through the segment: the anchor never lags the marker, + // whether or not a card refresh happened to run. + const firstFixAt = start - AGE_MS; + app.advance(firstFixAt + LAG_MS + 2_000 - Date.now()); + let worst = 0; + for (let f = 0; f < 60; f += 1) { + app.advance(16); + app.frame(); + const anchor = card.position(); + worst = Math.max( + worst, + Cesium.Cartesian3.distance(anchor, entry.marker.position), + ); + } + assert.ok( + worst < 1e-6, + `the anchor is the marker's own position (worst gap ${worst} m)`, + ); + assert.ok(app.state()._moving.has(entry), 'and it was moving throughout'); + + // The text: throttled, and republished only when it changes. Sixty frames + // of one second is sixty throttle windows, but the card copy only changes + // when the reported age or the motion state does. + const before = published.length; + for (let f = 0; f < 5; f += 1) { + app.advance(16); + app.frame(); + } + assert.equal( + published.length, + before, + 'identical text within a second is not re-sent', + ); + + // A hidden marker anchors nothing. + entry.marker.show = false; + assert.equal(card.position(), null); +}); + +test('DETECT gets the same candidate list back until something could have changed it', async (t) => { + // Detection pulls the list on every paint. It used to be a fleet scan with + // an allocation and, above the cap, a sort — on the demo path, sixty times + // a second. The list is rebuilt on polls, visibility sweeps, removals and + // selection, and handed back as-is between them. + const app = harness(t); + const fleet = () => + Array.from({ length: 40 }, (_, i) => + vehicle(`v${i}`, 42.36 + i * 0.0002, -71.06, reported(), { + routeId: `${i}`, + }), + ); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', fleet(), { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + + const first = app.layer.getDetectableObjects({ maxCount: 10, seed: 7 }); + assert.equal(first.length, 10); + const again = app.layer.getDetectableObjects({ maxCount: 10, seed: 7 }); + assert.equal(again, first, 'the same array, not a rebuilt one'); + app.layer.getDetectableObjects({ maxCount: 5, seed: 7 }); + app.advance(250); + app.layer._transitPartsForTest().queries.refreshDetectCache(); + const other = app.layer.getDetectableObjects({ maxCount: 5, seed: 7 }); + assert.notEqual(other, first, 'a different cap is a different list'); + assert.equal(other.length, 5); + + // Positions are the markers' own Cartesians: live, no copy. + const entry = app.vehicles().find((v) => v.key === first[0].sourceId); + assert.equal(first[0].position, entry.marker.position); + assert.equal(first[0].id, entry.record.routeId); + + // Selecting a vehicle changes what the list says, so it is rebuilt. + app.layer._transitPartsForTest().selection.selectVehicle(first[0].sourceId); + app.layer.getDetectableObjects({ maxCount: 10, seed: 7 }); + app.advance(250); + app.layer._transitPartsForTest().queries.refreshDetectCache(); + const afterSelect = app.layer.getDetectableObjects({ maxCount: 10, seed: 7 }); + assert.notEqual(afterSelect, first); + assert.equal(afterSelect[0].skipLabel, true); + + // A poll rebuilds it; removing an UNSELECTED vehicle leaves the chosen set. + const chosenBefore = afterSelect.map((c) => c.sourceId); + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + app.settle(); + app.layer._transitPartsForTest().queries.refreshDetectCache(); + const afterPoll = app.layer.getDetectableObjects({ maxCount: 10, seed: 7 }); + assert.notEqual(afterPoll, afterSelect, 'rebuilt after the poll'); + assert.deepEqual( + afterPoll.map((c) => c.sourceId), + chosenBefore, + 'same stable selection', + ); +}); + +test('under FLIR and NVG every sprite is white-hot with a dark halo, and selection cannot undo it', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('bus', 42.36, -71.06, reported(), { routeId: '742' }), + vehicle('metro', 42.361, -71.06, reported(), { routeId: 'Red' }), + vehicle('tram', 42.362, -71.06, reported(), { routeId: 'Green-E' }), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const byRoute = new Map(app.vehicles().map((v) => [v.record.routeId, v])); + const bus = byRoute.get('742'); + const plainImage = bus.marker.image; + const plainWidth = bus.marker.width; + const modeColour = bus.marker.color.clone(); + + app.style('thermal'); + for (const entry of app.vehicles()) { + assert.ok( + Cesium.Color.equals(entry.marker.color, Cesium.Color.WHITE), + `${entry.record.routeId} is white-hot`, + ); + assert.notEqual(entry.marker.image, plainImage, 'a haloed raster'); + assert.match( + Buffer.from(entry.marker.image.split(',')[1], 'base64').toString(), + /#000000/, + 'with the dark ring', + ); + assert.ok(entry.marker.width > plainWidth * 1.2, 'and larger'); + } + // Selecting under FLIR grows the sprite and keeps it white — cyan would dim it. + app.layer._transitPartsForTest().selection.selectVehicle(bus.key); + assert.ok( + Cesium.Color.equals(bus.marker.color, Cesium.Color.WHITE), + 'still white when selected', + ); + assert.ok( + bus.marker.width > byRoute.get('Red').marker.width, + 'and bigger than its neighbours', + ); + app.layer._transitPartsForTest().selection.clearSelection(); + assert.ok( + Cesium.Color.equals(bus.marker.color, Cesium.Color.WHITE), + 'and white when deselected', + ); + assert.equal(bus.marker.width, byRoute.get('Red').marker.width); + + // Back to normal: the shipped look, exactly. + app.style('normal'); + assert.ok( + Cesium.Color.equals(bus.marker.color, modeColour), + 'mode colour restored', + ); + assert.equal(bus.marker.image, plainImage, 'plain raster restored'); + assert.equal(bus.marker.width, plainWidth, 'shipped size restored'); + // A style that is not a sensor changes nothing. + app.style('anime'); + assert.equal(bus.marker.image, plainImage); +}); + +test('a cockpit vision override wins over the map style while it lasts, and destroy stops listening', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + const plain = entry.marker.image; + app.vision('nvg', true); + assert.ok( + Cesium.Color.equals(entry.marker.color, Cesium.Color.WHITE), + 'cockpit NVG restyles', + ); + app.style('normal'); + assert.ok( + Cesium.Color.equals(entry.marker.color, Cesium.Color.WHITE), + 'the map style cannot undo a live cockpit override', + ); + app.vision('normal', false); + assert.equal( + entry.marker.image, + plain, + 'leaving the cockpit restores the map style', + ); + assert.equal(app.state()._styleListeners.length, 2); + app.layer.destroy(app.viewer); + assert.equal(app.state()._styleListeners.length, 0, 'listeners removed'); + assert.doesNotThrow( + () => app.style('thermal'), + "and a later event is nobody's business", + ); +}); + +test('DETECT contacts carry a bracket tier per mode, keyed or keyless alike', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('bus', 42.36, -71.06, reported(), { routeId: '742' }), + vehicle('metro', 42.361, -71.06, reported(), { routeId: 'Red' }), + vehicle('tram', 42.362, -71.06, reported(), { routeId: 'Green-E' }), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const tiers = new Map( + app.layer.getDetectableObjects().map((c) => [c.id, c.tier]), + ); + assert.equal(tiers.get('742'), 'transit_bus'); + assert.equal(tiers.get('RED'), 'transit_subway'); + assert.equal(tiers.get('GREEN-E'), 'transit_tram'); +}); + +test('the floor cycle samples the rendered mesh for the cells it warms, with its own sprites excluded', async (t) => { + // A transit-only session never warmed a mesh floor: the layer warmed the + // DEM and stopped, so under a photogrammetric city it drew on bare earth + // seventeen metres down. The same cells now go to the application's mesh + // sampler after every warm — bounded by the sampler's own per-call cap and + // proximity gate — with the layer's billboards excluded so a vertical probe + // cannot land on a bus. + const app = harness(t, { floorAt: () => 5 }); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('a', 42.36, -71.06, reported()), + vehicle('b', 42.37, -71.07, reported()), + vehicle('far', 42.62, -71.44, reported()), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + assert.ok( + app.meshCalls.length >= 1, + 'the sampler was asked after the floor resolved', + ); + const call = app.meshCalls[0]; + assert.equal(call.scene, app.viewer.scene); + assert.deepEqual( + call.points.map((p) => p.lat).sort(), + [42.36, 42.37], + 'the in-view cells, and not the one off screen', + ); + assert.equal( + call.options.excludeObjects.length, + 2, + 'both in-view billboards excluded', + ); + assert.ok( + Math.abs(call.options.viewerLat - BOSTON.lat) < 1e-6, + 'viewer subpoint in degrees', + ); + assert.ok(Math.abs(call.options.viewerLon - BOSTON.lon) < 1e-6); +}); + +test('a failing prefix of cells cannot starve the vehicle behind it of a floor', async (t) => { + // Codex's reproduction: 301 visible cold entries, the first 300 in cells + // that never resolve. The budget used to take the first 300 unresolved + // vehicles in map order every cycle, so the 301st was never even asked for. + const count = FLOOR_WARM_PER_POLL + 1; + const app = harness(t, { + // Only the LAST vehicle's cell ever answers. + floorAt: (lat) => + Math.abs(lat - (42.36 + (count - 1) * 0.002)) < 1e-9 ? 12 : undefined, + view: { west: -71.2, south: 42.3, east: -70.9, north: 43.2 }, + }); + const fleet = Array.from({ length: count }, (_, i) => + vehicle(`v${i}`, 42.36 + i * 0.002, -71.06, reported()), + ); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', fleet, { fetchedAt: Date.now() }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + // The unique-cell budget resets per poll; the next poll rotates priority. + app.settle(3); + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + app.settle(3); + const last = app.vehicles().find((v) => v.record.id === `v${count - 1}`); + assert.equal(last.heightResolved, true, 'the 301st vehicle got its floor'); + assert.ok(Math.abs(last.heightM - 13.5) < 1e-9); + assert.ok( + app.ground._warmed.some((p) => Math.abs(p.lat - last.record.lat) < 1e-9), + 'because its cell was actually asked for', + ); +}); + +test('a resolved floor still adopts a later mesh sample or a regime change on a parked vehicle', async (t) => { + // Codex's reproduction: resolve the DEM at 10 m, then the shared floor + // changes to a 50 m mesh sample; the parked entry kept 11.5 m for ever + // because ingestion only re-read a resolved height when the cell changed. + const app = harness(t, { floorAt: () => 10 }); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('p', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + assert.ok(Math.abs(entry.heightM - 11.5) < 1e-9, 'DEM floor, lifted'); + const height = () => + Cesium.Cartographic.fromCartesian(entry.marker.position).height; + assert.ok(Math.abs(height() - 11.5) < 0.5); + + // The mesh lands on the same cell. + app.ground._warm.set(app.ground._key(42.36, -71.06), 50); + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + app.frame(); + assert.ok( + Math.abs(entry.heightM - 51.5) < 1e-9, + 'the parked vehicle adopts the mesh floor', + ); + assert.ok(Math.abs(height() - 51.5) < 0.5, 'and is drawn on it'); + + // Leaving the photoreal regime: the shared floor answers the DEM again. + app.ground._warm.set(app.ground._key(42.36, -71.06), 10); + app.advance(TRANSIT_POLL_MS); + await app.layer.update(); + app.frame(); + assert.ok(Math.abs(height() - 11.5) < 0.5, 'and follows it back down'); +}); + +test('a stamp an hour ahead is bounded, so the honest fixes that follow are not refused', async (t) => { + // Codex's reproduction: the first fix stamped one hour ahead, then a + // corrected clock and normal travel. The marker stayed at its first + // position for as long as the wrong clock stayed newer than real time, and + // survived eviction because every refused snapshot refreshed its record. + const app = harness(t, { floorAt: () => 5 }); + const ahead = (Date.now() + 3_600_000) / 1000; + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('c', 42.36, -71.06, ahead)], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + assert.ok( + entry.fixes[0].t <= Date.now() + FUTURE_STAMP_TOLERANCE_MS, + 'the stamp is bounded by the fetch time', + ); + // The clock is corrected; the bus travels north 200 m per poll. + let north = 42.36; + for (let poll = 1; poll <= 4; poll += 1) { + app.advance(TRANSIT_POLL_MS); + north += 0.0018; + const fixAtS = (Date.now() - 2_000) / 1000; + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('c', north, -71.06, fixAtS)], { + fetchedAt: Date.now(), + }), + })); + await app.layer.update(); + } + assert.ok( + entry.fixes.length >= 3, + `honest fixes were accepted (${entry.fixes.length} in history)`, + ); + assert.equal(entry.track.pendingCount, 0); + app.run(120_000); + assert.ok( + Math.abs(lat(entry) - 42.36) > 0.003, + 'and the marker has moved on', + ); +}); + +test('a run of out-of-order fixes restarts the history instead of freezing the vehicle', async (t) => { + const app = harness(t, { floorAt: () => 5 }); + const later = (Date.now() + 5_000) / 1000; // within tolerance, but newer than what follows + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('o', 42.36, -71.06, later)], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + const firstRecord = entry.record; + // The feed's clock steps BACK: every following stamp is older than the first. + for (let poll = 1; poll <= ORDER_REJECTS_BEFORE_RESET; poll += 1) { + app.advance(TRANSIT_POLL_MS); + const stale = later - 60 + poll; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('o', 42.365, -71.06, stale, { label: `poll${poll}` })], + { + fetchedAt: Date.now(), + }, + ), + })); + await app.layer.update(); + if (poll < ORDER_REJECTS_BEFORE_RESET) { + assert.equal( + entry.record, + firstRecord, + `refusal ${poll} does not refresh the record`, + ); + assert.equal(entry.track.pendingCount, poll); + } + } + assert.equal( + entry.track.pendingCount, + 0, + 'reset after three advancing observations', + ); + assert.equal(entry.fixes.length, 2); + app.settle(); + assert.equal(entry.resets, 1); + assert.ok( + Math.abs(lat(entry) - 42.365) < 1e-6, + 'placed at the fix it restarted from', + ); +}); + +test('selected trail and marker share revised start floors, with unknown history left as gaps', async (t) => { + const app = harness(t, { floorAt: () => 10 }); + let position = 42.36; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('trail', position, -71.06, reported())], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + app.advance(15000); + position += 0.0001; + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0], + parts = app.layer._transitPartsForTest(); + parts.selection.selectVehicle(entry.key); + app.run(8000, 16); + let trail = parts.trails.diagnostics(); + assert.ok(trail.segments > 0); + assert.equal(trail.entitiesAdded, 0); + assert.ok( + Cesium.Cartesian3.equals( + trail.head.positions.at(-1), + entry.marker.position, + ), + ); + const body = trail.body, + revision = trail.revision; + app.run(1000, 16); + assert.equal(parts.trails.diagnostics().body, body); + app.ground._warm.set(app.ground._key(42.36, -71.06), 30); + parts.height.rereadFloors(); + app.frame(); + trail = parts.trails.diagnostics(); + assert.ok(trail.revision > revision); + assert.ok( + Cesium.Cartesian3.equals( + trail.head.positions.at(-1), + entry.marker.position, + ), + ); + assert.ok( + Cesium.Cartographic.fromCartesian(entry.marker.position).height > 30, + ); + assert.ok(entry.trailVertices <= 2048); + parts.selection.clearSelection(); + assert.equal(parts.trails.diagnostics(), null); +}); + +test('selection restores retained history without changing clock or health; churn aborts late reads', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot('mbta', 'MBTA', [vehicle('b', 42.36, -71.06, reported())], { + fetchedAt: Date.now(), + }), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0], + parts = app.layer._transitPartsForTest(); + const liveT = entry.fixes[0].t; + const payload = { + version: 1, + feedId: 'mbta', + vehicleId: 'b', + oldestT: liveT - 30000, + newestT: liveT, + truncated: false, + epochs: [{ id: 1, trip: '', route: '1', mode: 'bus' }], + fixes: [ + [liveT - 30000, 42.359, -71.06, 1, 1], + [liveT - 15000, 42.3595, -71.06, 1, 1], + [liveT, 42.37, -71.06, 1, 1], + ], + }; + const priorFetch = globalThis.fetch; + let complete, + signal, + requests = 0; + globalThis.fetch = (url, options) => { + if (!String(url).includes('/trail/')) return priorFetch(url, options); + requests++; + signal = options.signal; + return new Promise((resolve) => { + complete = () => resolve(new Response(JSON.stringify(payload))); + }); + }; + const health = entry.track.acceptedAt, + clock = entry.playT; + parts.selection.selectVehicle(entry.key); + complete(); + for (let i = 0; i < 20; i++) await Promise.resolve(); + assert.equal(entry.track.count, 3); + assert.equal( + entry.fixes.at(-1).lat, + 42.36, + 'live position wins the conflict', + ); + assert.equal(entry.track.acceptedAt, health); + assert.equal(entry.playT, clock); + assert.equal(requests, 1); + parts.selection.selectVehicle(entry.key); + assert.equal(requests, 2); + parts.selection.clearSelection(); + assert.equal(signal.aborted, true); + const count = entry.track.count; + complete(); + for (let i = 0; i < 20; i++) await Promise.resolve(); + assert.equal(entry.track.count, count); + assert.equal(parts.trails.requestDiagnostics().pending, false); + parts.selection.selectVehicle(entry.key); + app.layer.disable(app.viewer); + assert.equal(signal.aborted, true); + complete(); + for (let i = 0; i < 20; i++) await Promise.resolve(); + assert.equal(app.vehicles().length, 0); +}); + +test('a long selected history shortens its oldest geometry before sacrificing the active corridor', async (t) => { + const app = harness(t); + const parts = app.layer._transitPartsForTest(); + const { mergeHistory, seek } = await import('../../data/contactPlayback.js'); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('long', 42.36, -71.06, reported(), { routeId: 'CR-1' })], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0], + newest = entry.fixes[0].t; + mergeHistory( + entry.track, + Array.from({ length: 100 }, (_, i) => ({ + t: newest - (99 - i) * 15000, + lat: 42.36 - (99 - i) * 0.009, + lon: -71.06, + epoch: entry.currentEpoch, + })), + ); + for (let i = 0; i < 1000; i++) + app.ground._warm.set(app.ground._key(41.45 + i * 0.001, -71.06), 12); + seek(entry.track, newest - 7500, { + wallNowMs: Date.now(), + monoNowMs: performance.now(), + }); + parts.rendering.sampleIdle(entry); + parts.selection.selectVehicle(entry.key); + assert.equal(entry.trailTruncated, true); + assert.ok(entry.trailVertices <= 2048); + assert.ok( + entry.displayPaths.has(entry.sample.fromSeq), + 'active path is prepared first', + ); + assert.ok( + entry.trailSegments.every( + (segment, i, list) => i === 0 || list[i - 1].fromT <= segment.fromT, + ), + ); +}); + +test('missing rectangle still rejects the far side, and frustum rejection wins', async (t) => { + const app = harness(t, { altitude: 100_000 }); + app.viewer.camera.computeViewRectangle = () => undefined; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [ + vehicle('near', 42.36, -71.06, reported()), + vehicle('far', -42.36, 108.94, reported()), + ], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.advance(250); + app.frame(); + assert.deepEqual( + [...app.state()._visible].map((e) => e.record.id), + ['near'], + ); + app.viewer.camera.frustum = { + computeCullingVolume: () => ({ + computeVisibility: () => Cesium.Intersect.OUTSIDE, + }), + }; + app.advance(250); + app.layer._transitPartsForTest().rendering.refreshVisibility(); + assert.equal(app.state()._visible.size, 0); + assert.equal(app.state()._moving.size, 0); + assert.equal(app.holds().includes('transit'), false); +}); + +test('motion transitions migrate billboards and update cached positions; disable clears owners', async (t) => { + const app = harness(t); + let latitude = 42.36; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus', latitude, -71.06, reported())], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + const old = entry.marker; + app.layer.getDetectableObjects(); + app.advance(15000); + latitude += 0.001; + await app.layer.update(); + app.run(8000, 16); + assert.equal(entry.markerCollection, app.state()._animatedMarkers); + assert.notEqual(entry.marker, old); + assert.equal(entry.detectContact.position, entry.marker.position); + app.run(30000, 16); + assert.equal(entry.markerCollection, app.state()._markers); + const parts = app.layer._transitPartsForTest(); + const contacts = app.layer.getDetectableObjects(); + const metric = contacts[0].metric; + for (let i = 0; i < 20; i++) { + assert.equal(app.layer.getDetectableObjects(), contacts); + assert.equal(contacts[0].metric, metric); + } + parts.rendering.requestVisibility(); + app.layer.disable(app.viewer); + assert.equal(app.state()._detectCache, null); + assert.equal(entry.detectContact, null); + assert.equal(app.state()._visible.size, 0); + assert.equal(app.state()._visibilityTimer, null); + assert.equal(app.state()._maintenanceTimer, null); + assert.equal(entry.wakeTimer, null); + assert.equal(app.state()._inFlight.size, 0); + assert.equal(parts.trails.requestDiagnostics().pending, false); +}); + +test('every styling path retains the mode palette and exact unpadded size', async (t) => { + const app = harness(t); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus', 42.36, -71.06, reported(), { routeId: '741' })], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0], + parts = app.layer._transitPartsForTest(); + const check = (hex, display) => { + assert.equal(entry.marker.color.toCssHexString().toUpperCase(), hex); + const svg = Buffer.from( + entry.marker.image.split(',')[1], + 'base64', + ).toString(); + const width = Number(svg.match(/viewBox="-\d+ -\d+ (\d+)/)[1]); + assert.ok(Math.abs((entry.marker.width * 96) / width - display) < 1e-9); + }; + check('#5EF08A', 20); + parts.selection.selectVehicle(entry.key); + check('#5EF08A', 30); + parts.rendering.paintMode(entry, 'subway'); + check('#FF4538', 30); + app.style('thermal'); + check('#FFFFFF', 36); + parts.selection.clearSelection(); + check('#FFFFFF', 30); + app.vision('nvg', true); + check('#FFFFFF', 30); + parts.selection.selectVehicle(entry.key); + check('#FFFFFF', 36); + app.vision('normal', false); + check('#FF4538', 30); + parts.selection.clearSelection(); + check('#FF4538', 20); +}); + +test('QA fleets at both budgets exercise real marker motion', async (t) => { + const app = harness(t, { altitude: 30000 }); + app.layer.enable(app.viewer); + await app.layer.update(); + for (const count of [800, 3000]) { + const loaded = app.layer._loadTransitFleetForTest(count, BOSTON, 1); + assert.equal(loaded.count, count); + assert.equal(loaded.moving, count); + assert.equal(loaded.fixBytes, count * 128 * 48); + const entry = app.vehicles()[0]; + const before = Cesium.Cartesian3.clone(entry.marker.position); + app.advance(100); + app.frame(); + assert.ok(Cesium.Cartesian3.distance(before, entry.marker.position) > 0.09); + } +}); + +test('visible motion is recullable without a camera event and releases its hold offscreen', async (t) => { + const app = harness(t, { altitude: 30000 }); + app.layer.enable(app.viewer); + await app.layer.update(); + app.layer._loadTransitFleetForTest(1, BOSTON, 1); + assert.equal(app.state()._moving.size, 1); + app.viewer.camera.frustum = { + computeCullingVolume: () => ({ + computeVisibility: () => Cesium.Intersect.OUTSIDE, + }), + }; + app.layer._transitPartsForTest().rendering.maintainPresentation(); + app.advance(250); + assert.equal(app.state()._moving.size, 0); + assert.equal(app.state()._renderHeld, false); +}); + +test('mesh refinement reaches all 100 retained DEM cells within five poll budgets', async (t) => { + const { createHeight } = await import('./height.js'); + t.mock.timers.enable({ apis: ['setTimeout'] }); + let mono = 0; + t.mock.method(performance, 'now', () => mono); + const cells = Array.from({ length: 100 }, (_, i) => ({ + lat: i / 1000, + lon: 0, + })); + const entry = { key: 'selected', to: cells[0], fixes: [cells.at(-1)] }; + const probed = new Set(); + const state = { + _enabled: true, + _selectedKey: entry.key, + _vehicles: new Map([[entry.key, entry]]), + _floorCursor: 0, + _heightDirty: new Set(), + _viewer: { camera: { positionCartographic: { height: 100 } } }, + }; + const height = createHeight({ + state, + services: { + render: { governorRequestRender() {} }, + ground: { + coarseFloorCoord: (lat, lon) => ({ lat, lon }), + cachedGroundFloor: () => 10, + GROUND_FLOOR_LIFT_M: 1.5, + }, + mesh: { + sampleMeshFloorCells(scene, points) { + for (const point of points) probed.add(point.cell); + }, + }, + }, + parts: { + trails: { + prepareEntry(e, collect) { + if (collect) + for (const point of cells) + collect(point, height.requestHeight(point.lat, point.lon)); + }, + }, + rendering: { requestVisibility() {} }, + }, + }); + t.after(() => height.clear()); + for (let poll = 0; poll < 5; poll++) { + mono = poll * TRANSIT_POLL_MS; + height.anchorFloors(); + } + assert.equal(probed.size, 100); +}); + +test('selection churn releases past geometry on every retained vehicle', async (t) => { + const app = harness(t, { floorAt: () => 10 }); + const { mergeHistory, seek } = await import('../../data/contactPlayback.js'); + const parts = app.layer._transitPartsForTest(); + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + Array.from({ length: 5 }, (_, i) => + vehicle(`churn-${i}`, 42.36, -71.06, reported()), + ), + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + for (const entry of app.vehicles()) { + const newest = entry.fixes[0].t; + mergeHistory( + entry.track, + Array.from({ length: 100 }, (_, i) => ({ + t: newest - (99 - i) * 5000, + lat: 42.36 - (99 - i) * 0.0001, + lon: -71.06, + epoch: entry.currentEpoch, + })), + ); + for (let i = 0; i < 20; i++) + app.ground._warm.set(app.ground._key(42.345 + i * 0.001, -71.06), 10); + seek(entry.track, newest - 7500, { + wallNowMs: Date.now(), + monoNowMs: performance.now(), + }); + parts.rendering.sampleIdle(entry); + parts.selection.selectVehicle(entry.key); + assert.ok(entry.trailSegments.length > 50); + parts.selection.clearSelection(); + for (const retained of app.vehicles()) { + assert.equal(retained.trailSegments?.length || 0, 0); + assert.equal(retained.trailVertices || 0, 0); + assert.ok( + [...retained.displayPaths.keys()].every( + (seq) => seq >= retained.sample.fromSeq, + ), + ); + } + assert.equal(parts.trails.diagnostics(), null); + assert.equal( + entry.displayPaths.size, + 2, + 'only active and future corridors remain', + ); + seek(entry.track, newest - 1000, { + wallNowMs: Date.now(), + monoNowMs: performance.now(), + }); + parts.rendering.sampleIdle(entry); + assert.equal( + entry.displayPaths.size, + 1, + 'marker playback releases completed corridors', + ); + } +}); + +test('tab visibility synchronizes playback before the returning frame and removes its listener', async (t) => { + const app = harness(t, { floorAt: () => 10 }); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(1, BOSTON, 5); + app.settle(); + const entry = app.vehicles()[0]; + app.frame(); + const before = entry.playT; + const resets = entry.track.resets; + document.visibilityState = 'hidden'; + document.dispatchEvent(new Event('visibilitychange')); + app.advance(30000); + document.visibilityState = 'visible'; + document.dispatchEvent(new Event('visibilitychange')); + assert.equal(entry.track.resetReason, 're-entry'); + assert.equal(entry.track.resets, resets + 1); + assert.ok(entry.playT > before); + assert.equal(entry.playT, Date.now() - entry.track.targetDelayMs); + const synchronized = entry.playT; + app.frame(); + assert.equal( + entry.playT, + synchronized, + 'first frame does not replay hidden elapsed time', + ); + document.dispatchEvent(new Event('visibilitychange')); + assert.equal(entry.track.resets, resets + 1, 'visible duplicate is a no-op'); + const remove = t.mock.method(document, 'removeEventListener'); + app.layer.disable(app.viewer); + assert.ok( + remove.mock.calls.some((call) => call.arguments[0] === 'visibilitychange'), + ); +}); + +for (const bounds of ['null', 'stale']) { + test(`browser visibility starts an injected fleet with ${bounds} bounds and an unsampled hidden marker`, async (t) => { + const app = harness(t); + app.viewer.camera.frustum = { + computeCullingVolume: () => ({ + computeVisibility: () => Cesium.Intersect.INSIDE, + }), + }; + app.layer.enable(app.viewer); + const loaded = app.layer._loadTransitFleetForTest(4, BOSTON, 70, [ + 'Red', + '741', + '742', + 'Green-E', + ]); + const state = app.state(), + parts = app.layer._transitPartsForTest(); + state._viewBounds = + bounds === 'null' ? null : { south: 0, north: 1, west: 0, east: 1 }; + for (const entry of app.vehicles()) { + state._visible.delete(entry); + state._moving.delete(entry); + entry.marker.show = false; + entry.sample = null; + } + state._visibilityAt = 0; + parts.rendering.refreshVisibility(); + assert.equal(state._visible.size, loaded.count); + assert.equal(state._moving.size, loaded.count); + assert.equal(app.viewer.scene.requestRenderMode, false); + const entry = app.vehicles()[0], + before = Cesium.Cartesian3.clone(entry.marker.position); + app.run(1000, 16); + assert.ok(Cesium.Cartesian3.distance(before, entry.marker.position) > 0.9); + }); +} + +test('keyless loaded terrain resolves a pending surface without needing a frame first', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + let floor; + app.viewer.scene.globe.show = true; + app.viewer.scene.globe.getHeight = () => floor; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('bus', 42.36, -71.06, reported())], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + const entry = app.vehicles()[0], + parts = app.layer._transitPartsForTest(); + assert.equal(entry.heightPending, true); + assert.equal(entry.marker.show, false); + floor = 12; + parts.height.rereadFloors(); + app.state()._visibilityAt = 0; + parts.rendering.refreshVisibility(); + assert.equal(entry.heightPending, false); + assert.equal(entry.surfaceReady, true); + assert.equal(entry.marker.show, true); +}); + +test('scripted keyless fleet keeps its known floor through selection and floor rereads', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.viewer.scene.globe.show = true; + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(4, BOSTON, 70); + const parts = app.layer._transitPartsForTest(), + entry = app.vehicles()[0]; + parts.selection.selectVehicle(entry.key); + parts.height.rereadFloors(); + app.run(1000, 16); + assert.equal(entry.surfaceReady, true); + assert.equal(entry.marker.show, true); + assert.ok(entry.trailSegments.length > 0); + assert.ok(app.state()._moving.has(entry)); +}); + +for (const count of [800, 3000]) { + test(`real perspective frustum admits all ${count} moving budget fixtures`, async (t) => { + const altitude = count === 800 ? 600 : 30000; + const app = harness(t, { altitude, floorAt: () => undefined }); + const camera = app.viewer.camera; + camera.positionWC = Cesium.Cartesian3.fromDegrees( + BOSTON.lon, + BOSTON.lat, + altitude, + ); + camera.directionWC = Cesium.Cartesian3.negate( + Cesium.Ellipsoid.WGS84.geodeticSurfaceNormal( + camera.positionWC, + new Cesium.Cartesian3(), + ), + new Cesium.Cartesian3(), + ); + const enu = Cesium.Transforms.eastNorthUpToFixedFrame(camera.positionWC); + camera.upWC = Cesium.Matrix4.getColumn(enu, 1, new Cesium.Cartesian3()); + camera.frustum = new Cesium.PerspectiveFrustum({ + fov: Math.PI / 3, + aspectRatio: 1920 / 1080, + near: 1, + far: 100000, + }); + app.layer.enable(app.viewer); + const loaded = app.layer._loadTransitFleetForTest( + count, + BOSTON, + count === 800 ? 5 : 120, + ); + assert.equal(loaded.moving, count); + assert.equal(app.state()._visible.size, count); + const entry = app.vehicles()[0], + before = Cesium.Cartesian3.clone(entry.marker.position); + app.run(1000, 16); + assert.ok(Cesium.Cartesian3.distance(before, entry.marker.position) > 0.9); + assert.equal(app.viewer.scene.requestRenderMode, false); + }); +} + +test('click handler selects a migrated scripted billboard and prepares a valid depth-fail trail', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(4, BOSTON, 70); + const entry = app.vehicles()[0]; + assert.equal(entry.markerCollection, app.state()._animatedMarkers); + app.viewer.scene.pick = () => ({ primitive: entry.marker, id: entry.key }); + app + .state() + ._clickHandler.getInputAction(Cesium.ScreenSpaceEventType.LEFT_CLICK)({ + position: new Cesium.Cartesian2(500, 400), + }); + assert.equal(app.state()._selectedKey, entry.key); + const body = app.layer._transitPartsForTest().trails.diagnostics().body; + assert.ok(body.geometryInstances.length > 0); + body._batchTableAttributeIndices = { color: 0, depthFailColor: 1 }; + assert.doesNotThrow(() => + Cesium.Primitive._updateColorAttribute( + body, + body.depthFailAppearance.vertexShaderSource, + true, + ), + ); + assert.ok(entry.trailSegments.length > 0); + app.run(1000, 16); + assert.equal(entry.marker.show, true); +}); + +test('intercepted scripted reports resolve their explicit floor and wake without a preRender bootstrap', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.layer._setTransitFixtureFloorsForTest(['mbta:sim-straight'], 20); + let latitude = BOSTON.lat; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('sim-straight', latitude, BOSTON.lon, reported())], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.advance(15000); + latitude += 0.001; + await app.layer.update(); + app.advance(8000); + const entry = app.vehicles()[0]; + assert.equal(entry.heightPending, false); + assert.equal(entry.surfaceReady, true); + assert.equal(entry.marker.show, true); + assert.ok(app.state()._moving.has(entry)); + assert.equal(app.viewer.scene.requestRenderMode, false); + const before = Cesium.Cartesian3.clone(entry.marker.position); + app.run(1000, 16); + assert.ok(Cesium.Cartesian3.distance(before, entry.marker.position) > 1); + app.layer.disable(app.viewer); + assert.equal(app.state()._qaFixtureFloors.size, 0); +}); + +test('hot path reuses sample outputs and reads camera altitude once per frame', async (t) => { + const h = harness(t); + await h.layer.enable(h.viewer); + h.layer._loadTransitFleetForTest(800, BOSTON, 5); + const state = h.state(); + const parts = h.layer._transitPartsForTest(); + const entry = state._vehicles.values().next().value; + const out = new Cesium.Cartesian3(); + assert.equal(parts.trails.samplePosition(entry, entry.sample, out), out); + const sample = entry.sample; + const carto = h.viewer.camera.positionCartographic; + let reads = 0; + Object.defineProperty(h.viewer.camera, 'positionCartographic', { + configurable: true, + get() { + reads++; + return carto; + }, + }); + h.advance(17); + h.frame(); + assert.equal(entry.sample, sample); + assert.ok(reads <= 2, `camera altitude reads/frame ${reads}, limit 2`); + const contacts = h.layer.getDetectableObjects(); + const contact = contacts[0]; + for (let i = 0; i < 100; i++) { + state._detectRevision++; + assert.equal(h.layer.getDetectableObjects(), contacts); + assert.equal(contacts[0], contact); + } +}); + +test('scripted straight and changing delay preserve ordinary-frame speed through polls', async (t) => { + const h = harness(t, { altitude: 600 }); + const start = Date.now(); + const modes = ['straight', 'gap', 'dup']; + h.serve('mbta', () => { + const elapsed = (Date.now() - start) / 1000; + return { + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + modes.map((id) => { + let sec = + Math.floor(elapsed / (id === 'dup' ? 30 : 10)) * + (id === 'dup' ? 30 : 10); + if (id === 'gap' && sec >= 40 && sec < 130) sec = 40; + return vehicle( + id, + BOSTON.lat + (sec * 8) / 111320, + BOSTON.lon, + (start + sec * 1000) / 1000, + ); + }), + { fetchedAt: Date.now() }, + ), + }; + }); + await h.layer.enable(h.viewer); + let prior = new Map(), + worst = 0, + pair = null, + samples = 0; + for (let ms = 17; ms <= 200000; ms += 17) { + h.advance(17); + if (ms % 15011 === 0) await h.layer.update(); + h.frame(); + for (const e of h.vehicles()) { + const a = prior.get(e.key), + s = e.sample, + p = e.marker.position; + const b = { + x: p.x, + y: p.y, + z: p.z, + t: s.displayT, + from: s.fromSeq, + to: s.toSeq, + speed: s.segmentSpeedMps, + phase: s.phase, + }; + if ( + a && + a.phase === 'playing' && + b.phase === 'playing' && + a.from === b.from && + a.to === b.to && + a.speed > 0 && + b.t > a.t + ) { + const error = Math.abs( + Math.hypot(b.x - a.x, b.y - a.y, b.z - a.z) / + ((b.t - a.t) / 1000) / + a.speed - + 1, + ); + samples++; + if (error > worst) { + worst = error; + pair = { key: e.key, ms, a, b }; + } + } + prior.set(e.key, b); + } + } + assert.ok(samples > 1000); + assert.ok(worst < 0.02, JSON.stringify({ worst, pair, samples })); +}); + +test('a lost prepared surface releases moving playback on that frame', async (t) => { + const h = harness(t); + await h.layer.enable(h.viewer); + h.layer._loadTransitFleetForTest(1, BOSTON, 5); + const parts = h.layer._transitPartsForTest(); + assert.equal(h.state()._moving.size, 1); + h.state()._heightDirty.clear(); + clearTimeout(h.state()._visibilityTimer); + h.state()._visibilityTimer = null; + t.mock.method(parts.trails, 'samplePosition', () => null); + h.advance(17); + h.frame(); + assert.equal(h.state()._moving.size, 0); + assert.equal(h.state()._renderHeld, false); +}); + +test('sensor brackets clear the padded hot body and retain their contact during style changes', async (t) => { + const h = harness(t); + await h.layer.enable(h.viewer); + h.layer._loadTransitFleetForTest(1, BOSTON, 5); + const parts = h.layer._transitPartsForTest(), + entry = h.vehicles()[0]; + const contact = h.layer.getDetectableObjects()[0]; + for (const style of ['thermal', 'surveillance', 'noir', 'normal']) { + parts.rendering.setStylePreset(style); + assert.equal(entry.detectContact, contact); + assert.ok(contact.bracketHalfWidth >= entry.marker.width / 2 + 2, style); + assert.ok(contact.bracketHalfHeight >= entry.marker.height / 2 + 2, style); + } +}); + +test('selected anchor rejects nonfinite marker coordinates before sampling and during surface changes', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(4, BOSTON, 70); + const entry = app.vehicles()[0]; + app.layer._transitPartsForTest().selection.selectVehicle(entry.key); + const card = app.overlaySources.get('transit-selected')[0]; + assert.ok(card.position()); + const position = Cesium.Cartesian3.clone(entry.marker.position); + entry.marker.position = new Cesium.Cartesian3(NaN, 0, 0); + assert.equal(card.position(), null); + entry.marker.position = position; + assert.equal(card.position(), entry.marker.position); + entry.marker.show = false; + assert.equal(card.position(), null); +}); + +test('preset clicks preserve both pick id paths through moving and stationary collection migration', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(4, BOSTON, 70); + const entry = app.vehicles()[0], + parts = app.layer._transitPartsForTest(); + for (const style of ['normal', 'thermal', 'surveillance', 'noir', 'retro']) + for (const animated of [true, false]) { + parts.rendering.setStylePreset(style); + entry.sample.phase = animated ? 'playing' : 'held'; + entry.sample.segmentSpeedMps = animated ? 1 : 0; + entry.sample.nextWakeMonoMs = Infinity; + entry.sample.segmentCourseDeg = entry.courseDeg; + parts.rendering.schedulePlayback(entry); + assert.equal( + entry.markerCollection, + animated ? app.state()._animatedMarkers : app.state()._markers, + ); + assert.equal(entry.marker.id, entry.key); + for (const pick of [{ id: entry.key }, { primitive: entry.marker }]) { + app.viewer.scene.pick = () => pick; + app + .state() + ._clickHandler.getInputAction(Cesium.ScreenSpaceEventType.LEFT_CLICK)( + { position: new Cesium.Cartesian2(500, 400) }, + ); + assert.equal(app.state()._selectedKey, entry.key); + parts.selection.clearSelection(); + } + } +}); + +test('QA backwards seek reconstructs the pruned fixture paths before allocation measurement', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(4, BOSTON, 70); + app.run(10000, 16); + const entry = app.vehicles()[0]; + entry.displayPaths.clear(); + app.layer._resetTransitFixtureClockForTest(); + assert.ok( + entry.displayPaths.size <= 6, + 'rewind retains only the active and future fixture corridor', + ); + assert.equal(entry.marker.show, true); + assert.equal(app.state()._moving.size, 4); + const before = Cesium.Cartesian3.clone(entry.marker.position); + app.run(1000, 16); + assert.ok(Cesium.Cartesian3.distance(before, entry.marker.position) > 0.9); +}); + +test('sensor clicks map displayed pixels into the pre-shader pick buffer', async (t) => { + const app = harness(t, { floorAt: () => undefined }); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(4, BOSTON, 70); + const entry = app.vehicles()[0], + scene = app.viewer.scene; + let actual; + scene.pick = (p) => { + actual = { x: p.x, y: p.y }; + return { id: entry.key }; + }; + scene.canvas = { + width: 1920, + height: 1080, + clientWidth: 1920, + clientHeight: 1080, + }; + const uniforms = { intensity: 1, pixelation: 1 }; + scene.postProcessStages = { + getStageByName: () => ({ enabled: true, uniforms }), + }; + const click = app + .state() + ._clickHandler.getInputAction(Cesium.ScreenSpaceEventType.LEFT_CLICK); + app.layer._transitPartsForTest().rendering.setStylePreset('surveillance'); + click({ position: new Cesium.Cartesian2(1440, 540) }); + assert.ok(Math.abs(actual.x - 1472) < 1e-6); + assert.equal(actual.y, 540); + app.layer._transitPartsForTest().rendering.setStylePreset('thermal'); + uniforms.pixelation = 1.5; + click({ position: new Cesium.Cartesian2(100.9, 200.1) }); + assert.ok(Math.abs(actual.x - 100.5) < 1e-6); + assert.ok(Math.abs(actual.y - 201) < 1e-6); + assert.equal(app.state()._selectedKey, entry.key); +}); + +test('rejected route metadata stays on the accepted bus', async (t) => { + const app = harness(t); + let routeId = '741', + latitude = 42.36; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', latitude, -71.06, reported(), { routeId })], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + const image = entry.marker.image; + app.advance(TRANSIT_POLL_MS); + routeId = 'Red'; + latitude = 43.36; + await app.layer.update(); + assert.equal(entry.mode, 'bus'); + assert.equal(entry.record.routeId, '741'); + assert.equal(entry.marker.image, image); + assert.equal(entry.currentEpoch, 1); + assert.equal(entry.track.count, 1); + assert.doesNotMatch( + app.layer + .getDetectableObjects() + .map((c) => c.metric) + .join(' '), + /METRO/, + ); +}); + +test('reject then selected history backfill cannot resurrect a 111 km displacement', async (t) => { + const app = harness(t); + let latitude = 42.36; + app.serve('mbta', () => ({ + status: 200, + body: snapshot( + 'mbta', + 'MBTA', + [vehicle('b', latitude, -71.06, reported())], + { fetchedAt: Date.now() }, + ), + })); + app.layer.enable(app.viewer); + await app.layer.update(); + app.settle(); + const entry = app.vehicles()[0]; + app.advance(15000); + latitude += 0.001; + await app.layer.update(); + app.settle(); + const accepted = entry.fixes.map((f) => [f.t, f.lat, f.lon, 1, 1]); + app.advance(15000); + latitude = 43.36; + await app.layer.update(); + assert.equal(entry.track.count, 2); + const pending = entry.track.pending; + const clock = entry.playT; + const priorFetch = globalThis.fetch; + globalThis.fetch = (url, options) => + String(url).includes('/trail/') + ? Promise.resolve( + new Response( + JSON.stringify({ + version: 1, + feedId: 'mbta', + vehicleId: 'b', + oldestT: accepted[0][0], + newestT: pending.t, + truncated: false, + epochs: [{ id: 1, trip: '', route: '1', mode: 'bus' }], + fixes: [...accepted, [pending.t, latitude, -71.06, 1, 1]], + }), + ), + ) + : priorFetch(url, options); + const parts = app.layer._transitPartsForTest(); + parts.selection.selectVehicle(entry.key); + for (let i = 0; i < 30; i++) await Promise.resolve(); + assert.equal(entry.track.count, 2); + assert.equal(entry.playT, clock); + assert.equal(entry.track.pending, pending); + app.run(60000); + assert.ok(Math.abs(lat(entry) - 42.361) * 111320 < 1); + assert.equal(entry.track.resets, 0); +}); + +test('actual Traffic, Bikeshare and Transit lifecycles retain sensitivity in every enable/disable order', async (t) => { + const { createLifecycle: trafficLifecycle } = + await import('../traffic/lifecycle.js'); + const { createLifecycle: bikeLifecycle } = + await import('../bikeshare/lifecycle.js'); + const app = harness(t); + const noop = () => {}; + const traffic = trafficLifecycle({ + state: { _pointCollection: {}, _loadGeneration: 0 }, + services: { + render: { holdContinuousRender: noop, releaseContinuousRender: noop }, + }, + source: {}, + parts: { + flow: { ensureFlowStatus: noop }, + animation: { animate: noop, clearDots: noop }, + viewport: { onCameraChanged: noop }, + ingestion: { cancelActiveFetch: noop }, + }, + }).methods; + const bike = bikeLifecycle({ + state: { + _pointCollection: {}, + _overlayHost: { setVisible: noop }, + _cityRuntime: new Map(), + }, + services: { + sprites: { restoreSpriteOrder: noop }, + picking: { registerPickOwner: noop, unregisterPickOwner: noop }, + }, + source: {}, + parts: { + selection: { + _installClickHandler: noop, + _clearSelection: noop, + _onKeyDown: noop, + }, + viewport: { + onCameraChanged: noop, + runProximityCheck: noop, + deactivateAllCities: noop, + }, + ingestion: { abortAllInFlight: noop }, + }, + }).methods; + app.viewer.camera.moveEnd = new Cesium.Event(); + const layers = { traffic, bikeshare: bike, transit: app.layer }; + const orders = [ + ['traffic', 'bikeshare', 'transit'], + ['traffic', 'transit', 'bikeshare'], + ['bikeshare', 'traffic', 'transit'], + ['bikeshare', 'transit', 'traffic'], + ['transit', 'traffic', 'bikeshare'], + ['transit', 'bikeshare', 'traffic'], + ]; + try { + for (const enable of orders) + for (const disable of orders) { + const active = new Set(); + for (const id of enable) { + layers[id].enable(app.viewer); + active.add(id); + assert.equal(app.viewer.camera.percentageChanged, 0.05); + assert.deepEqual( + new Set(cameraSensitivityClaims(app.viewer.camera)), + active, + ); + } + for (const id of disable) { + layers[id].disable(app.viewer); + active.delete(id); + assert.equal( + app.viewer.camera.percentageChanged, + active.size ? 0.05 : 0.5, + `${enable} / ${disable} after ${id}`, + ); + assert.deepEqual( + new Set(cameraSensitivityClaims(app.viewer.camera)), + active, + ); + } + } + } finally { + for (const layer of Object.values(layers)) layer.disable(app.viewer); + } +}); + +test('moving and stationary transit stay adjacent above CCTV and Bikeshare during migration', async (t) => { + const { + registerSpriteCollection, + unregisterSpriteCollection, + restoreSpriteOrder, + } = await import('../../data/spriteOrder.js'); + const app = harness(t); + app.layer.enable(app.viewer); + app.layer._loadTransitFleetForTest(1, BOSTON, 70); + const cctv = {}, + bike = {}, + flights = {}; + app.primitives.push(cctv, bike, flights); + app.viewer.scene.primitives.raiseToTop = (c) => { + app.primitives.splice(app.primitives.indexOf(c), 1); + app.primitives.push(c); + }; + const collections = new Map([ + ...app.sprites, + ['cctv', cctv], + ['bikeshare', bike], + ['flights', flights], + ]); + for (const [id, collection] of collections) + registerSpriteCollection(id, collection); + t.after(() => { + for (const [id, c] of collections) unregisterSpriteCollection(id, c); + }); + const entry = app.vehicles()[0], + parts = app.layer._transitPartsForTest(); + for (const moving of [true, false, true]) { + entry.sample.phase = moving ? 'playing' : 'held'; + entry.sample.segmentSpeedMps = moving ? 1 : 0; + entry.sample.segmentCourseDeg = NaN; + parts.rendering.schedulePlayback(entry); + restoreSpriteOrder(app.viewer); + const active = app.primitives.indexOf(entry.markerCollection); + assert.ok(active > app.primitives.indexOf(cctv)); + assert.ok(active > app.primitives.indexOf(bike)); + assert.ok(active < app.primitives.indexOf(flights)); + assert.equal( + Math.abs( + app.primitives.indexOf(app.state()._markers) - + app.primitives.indexOf(app.state()._animatedMarkers), + ), + 1, + ); + assert.equal(entry.marker.id, entry.key); + } +}); diff --git a/src/layers/transit/movement.js b/src/layers/transit/movement.js new file mode 100644 index 000000000..716bc9082 --- /dev/null +++ b/src/layers/transit/movement.js @@ -0,0 +1,313 @@ +/** Transit admission and orientation policy; storage and clocks are portable. */ +import { + createTrack, + createSample, + createFixSample, + insertFix, + advance, + readFix, + correctHeight, + setTimeOffsetMs, +} from '../../data/contactPlayback.js'; +export const FIX_HISTORY_MAX = 128; +export const DISPLAY_LAG_MIN_MS = 25_000; +export const DISPLAY_LAG_RESET_MS = 90_000; +export const MODE_MAX_SPEED_MPS = Object.freeze({ + bus: 36, // ~130 km/h — a coach on a motorway + tram: 28, // ~100 km/h — interurban light rail + subway: 42, // ~150 km/h + rail: 111, // ~400 km/h — high-speed services exist on some feeds + ferry: 25, // ~90 km/h — fast catamarans + unknown: 56, // ~200 km/h +}); + +/** + * The ceiling a fix is judged against. A mode a feed DEFAULTED to is a guess + * about a fleet, not a fact about a vehicle: TransLink publishes rail in a + * feed whose default is bus, and a bus ceiling refused its 140 km/h services + * as impossible and placed them instead of animating them. An inferred mode + * gets the permissive ceiling; only a mode the route id actually established + * gets its own. + * @param {string} mode + * @param {boolean} [inferred] Whether the mode came from a feed default. + * @returns {number} m/s + */ +export function speedCeilingMps(mode, inferred = false) { + if (inferred) return MODE_MAX_SPEED_MPS.unknown; + return MODE_MAX_SPEED_MPS[mode] ?? MODE_MAX_SPEED_MPS.unknown; +} + +/** Position deadband is disabled: even a short reported crawl is played. */ +export const FIX_NOISE_M = 0; + +/** + * Displacement (m) a glide must cover before its direction is worth believing. + * Below this the course derived from it is noise; the last good one is held. + */ +export const COURSE_MIN_TRAVEL_M = 12; + +/** Maximum display-course change per second (deg). A bus is not a gyroscope. */ +export const COURSE_MAX_RATE_DEG_PER_S = 120; +/** + * Longest stretch of elapsed time (s) one evaluation may spend. The limiter + * used to bank the time since the course last CHANGED, so a vehicle that held + * north for thirty seconds had thirty seconds of allowance and snapped through + * a corner in one step. Elapsed time is now measured since the course was last + * EVALUATED, and capped here so that even a first look after a long idle + * sweeps visibly instead of snapping. + */ +export const COURSE_STEP_MAX_S = 0.25; + +/** + * Consecutive fixes at the same place before a vehicle is called STOPPED. + * One is not enough: a fifteen-second gap between identical fixes is also what + * a vehicle waiting at a light looks like, and so is a feed that repeated + * itself. + */ +export const STOPPED_FIXES = 2; + +/** + * How long after its newest fix a vehicle stops being described as travelling. + * + * "En route" means the last two fixes are in different places, which is a fact + * about the past. A bus that reported movement and then went quiet for three + * minutes is not en route — we simply do not know what it is doing, and saying + * otherwise is the same overreach as reading STOPPED off a stale status. Ninety + * seconds matches the threshold at which a whole feed reads as gone quiet. + */ +export const MOTION_UNKNOWN_AFTER_MS = 90_000; + +const EARTH_M_PER_DEG = 111_320; + +/** + * Metres between two lat/lon points, flat-earth. Over the few hundred metres a + * surface vehicle covers between fixes this is exact to well under a metre. + * @param {{lat:number, lon:number}} a + * @param {{lat:number, lon:number}} b + * @returns {number} + */ +export function fixDistanceM(a, b) { + if (!a || !b) return 0; + const dLat = (b.lat - a.lat) * EARTH_M_PER_DEG; + const dLon = + (b.lon - a.lon) * EARTH_M_PER_DEG * Math.cos((a.lat * Math.PI) / 180); + return Math.hypot(dLat, dLon); +} + +/** + * Course in degrees clockwise from north, from one point to another. + * @param {{lat:number, lon:number}} from + * @param {{lat:number, lon:number}} to + * @returns {number} 0-360 + */ +export function courseBetween(from, to) { + const dLat = (to.lat - from.lat) * EARTH_M_PER_DEG; + const dLon = + (to.lon - from.lon) * + EARTH_M_PER_DEG * + Math.cos((from.lat * Math.PI) / 180); + const deg = (Math.atan2(dLon, dLat) * 180) / Math.PI; + return (deg + 360) % 360; +} + +/** + * Shortest signed difference between two courses, in degrees (-180, 180]. + * @param {number} from + * @param {number} to + * @returns {number} + */ +export function courseDelta(from, to) { + let delta = ((to - from + 540) % 360) - 180; + if (delta === -180) delta = 180; + return delta; +} + +/** + * Whether a displacement between two fixes could have been travelled. + * @param {string} mode + * @param {number} meters + * @param {number} seconds + * @param {boolean} [inferred] The mode is a feed default, not a route fact. + * @returns {boolean} False when the implied speed is impossible for the mode. + */ +export function displacementPlausible(mode, meters, seconds, inferred = false) { + if (!(meters > 0)) return true; + const ceiling = speedCeilingMps(mode, inferred); + // Below a second of separation the implied speed is dominated by clock + // resolution, so judge those against a flat displacement instead. + if (!(seconds >= 1)) return meters <= ceiling; + return meters / seconds <= ceiling; +} + +export function initializePlayback(entry, budget, feedLag) { + entry.track = createTrack({ + budget, + policy: { + feedLag, + courseMinTravelM: COURSE_MIN_TRAVEL_M, + staleMs: MOTION_UNKNOWN_AFTER_MS, + accept: (a, b) => + displacementPlausible( + b.admissionMode ?? entry.mode, + fixDistanceM(a, b), + (b.t - a.t) / 1000, + b.admissionModeInferred ?? entry.modeInferred, + ), + }, + }); + entry.sample = createSample(); + entry.segment = { + from: createFixSample(), + to: createFixSample(), + fraction: NaN, + }; + entry.clocks = { wallNowMs: NaN, monoNowMs: NaN }; + // Inspection only. Scene animation reads the typed store into owned scratch. + Object.defineProperty(entry, 'playT', { + configurable: true, + get: () => entry.sample.displayT, + }); + Object.defineProperty(entry, 'lagTarget', { + configurable: true, + get: () => entry.track.targetDelayMs, + }); + Object.defineProperty(entry, 'fixes', { + configurable: true, + get() { + return Array.from({ length: entry.track.count }, (_, i) => + readFix(entry.track, i), + ); + }, + }); +} +export function recordFix(entry, fix, observation = {}) { + const context = observation.context; + let epoch = entry.currentEpoch || 1; + if (context) { + const previous = entry.track.epochs.get(epoch); + if ( + previous && + (previous.trip !== context.trip || + previous.route !== context.route || + previous.mode !== context.mode) + ) + epoch = Math.max(0, ...entry.track.epochs.keys()) + 1; + } + const result = insertFix( + entry.track, + { + ...fix, + epoch, + admissionMode: context?.mode, + admissionModeInferred: observation.modeInferred, + }, + observation, + ); + if (result.accepted) { + const latest = readFix( + entry.track, + entry.track.count - 1, + entry.track.latest, + ); + entry.currentEpoch = latest.epoch; + if (context) entry.track.epochs.set(latest.epoch, context); + } + return result; +} +export function updatePlayback(entry, wallNowMs, monoNowMs) { + entry.clocks.wallNowMs = wallNowMs; + entry.clocks.monoNowMs = monoNowMs; + advance(entry.track, entry.clocks, entry.sample); + entry.resets = entry.track.resets; + const segment = entry.segment; + readFix( + entry.track, + entry.sample.fromSeq - entry.track.baseSeq, + segment.from, + ); + readFix(entry.track, entry.sample.toSeq - entry.track.baseSeq, segment.to); + segment.fraction = entry.sample.fraction; + return entry.sample; +} +export function syncPlayback(entry, wallNowMs, monoNowMs = performance.now()) { + entry.clocks.wallNowMs = wallNowMs; + entry.clocks.monoNowMs = monoNowMs; + const resets = entry.track.resets; + const reason = entry.track.resetReason; + setTimeOffsetMs(entry.track, 0, entry.clocks); + if (entry.hasRendered) entry.track.resetReason = 're-entry'; + else { + entry.track.resets = resets; + entry.track.resetReason = reason; + } + updatePlayback(entry, wallNowMs, monoNowMs); +} +export function playbackPosition(entry) { + if (!entry?.track) return null; + return { + lat: entry.sample.lat, + lon: entry.sample.lon, + settled: entry.sample.phase !== 'playing', + }; +} +export function attachFloorToPlace(entry, fix, heightM) { + if (!entry.track || !Number.isFinite(heightM)) return; + const scratch = {}; + for (let i = 0; i < entry.track.count; i++) { + readFix(entry.track, i, scratch); + if (scratch.lat === fix.lat && scratch.lon === fix.lon) + correctHeight(entry.track, scratch.seq, heightM); + } +} +export function displayCourse(entry, nowMs) { + if (!entry) return null; + const target = Number.isFinite(entry.sample?.segmentCourseDeg) + ? entry.sample.segmentCourseDeg + : null; + if (target === null) return entry.courseDeg ?? null; + if (entry.courseDeg === null || !Number.isFinite(entry.courseEvalAt)) { + return target; + } + + // Rate-limit, the way the aircraft display course does: a fix arriving from + // around a corner should sweep, not snap. Time since the last EVALUATION, + // capped, so idle time is never banked into one big turn. + const elapsedS = Math.min( + COURSE_STEP_MAX_S, + Math.max(0, (nowMs - entry.courseEvalAt) / 1000), + ); + const allowed = COURSE_MAX_RATE_DEG_PER_S * elapsedS; + const delta = courseDelta(entry.courseDeg, target); + if (Math.abs(delta) <= allowed) return target; + return (entry.courseDeg + Math.sign(delta) * allowed + 360) % 360; +} + +/** + * Evaluate the display course and write it back — the ONE writer both the + * poll and the rotation pass go through, so the limiter sees every look. + * @param {object} entry + * @param {number} nowMs + * @returns {boolean} Whether the course changed. + */ +export function applyDisplayCourse(entry, nowMs) { + const course = displayCourse(entry, nowMs); + entry.courseEvalAt = nowMs; + if (course === entry.courseDeg) return false; + entry.courseDeg = course; + entry.courseAt = nowMs; + return true; +} + +export function displayMotion(entry, nowMs) { + if (!entry) return { moving: false, word: '' }; + const sample = entry.sample; + if (!sample) return { moving: false, word: 'WAITING' }; + if (sample.latestReportAgeMs > MOTION_UNKNOWN_AFTER_MS) + return { moving: false, word: 'NO FIX' }; + if (sample.phase !== 'playing') return { moving: false, word: 'WAITING' }; + if (sample.motion === 'stopped') return { moving: false, word: 'STOPPED' }; + return { + moving: sample.motion === 'moving', + word: sample.motion === 'moving' ? 'EN ROUTE' : 'WAITING', + }; +} diff --git a/src/layers/transit/movement.test.mjs b/src/layers/transit/movement.test.mjs new file mode 100644 index 000000000..fd71360e4 --- /dev/null +++ b/src/layers/transit/movement.test.mjs @@ -0,0 +1,146 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + initializePlayback, + recordFix, + updatePlayback, + displayMotion, + applyDisplayCourse, + displacementPlausible, +} from './movement.js'; +import { buildTransitSelectionCopy } from './policy.js'; +import { seek, sampleAt } from '../../data/contactPlayback.js'; +const entry = () => { + const e = { mode: 'bus', modeInferred: false, courseDeg: null }; + initializePlayback(e); + return e; +}; +const fix = (t, lat = 42) => ({ t, lat, lon: -71 }); +test('transit mode admission retains the permissive ceiling for inferred modes', () => { + assert.equal(displacementPlausible('bus', 583, 15), false); + assert.equal(displacementPlausible('bus', 583, 15, true), true); + assert.equal(displacementPlausible('rail', 583, 15), true); +}); +test('a lone implausible fix cannot teleport; two coherent replacement fixes break the path', () => { + const e = entry(); + recordFix(e, fix(0)); + recordFix(e, fix(15000, 42.001)); + assert.equal(recordFix(e, fix(30000, 43)).accepted, false); + assert.equal(e.track.count, 2); + assert.equal(recordFix(e, fix(45000, 43.001)).accepted, true); + assert.equal(e.track.count, 4); + assert.equal(sampleAt(e.track, 22500, {}).segmentSpeedMps, 0); +}); +test('clock correction requires three distinct advancing observations, never three cached packets', () => { + const e = entry(); + recordFix(e, fix(100000)); + for (let i = 0; i < 10; i++) recordFix(e, fix(10000)); + assert.equal(e.track.resets, 0); + recordFix(e, fix(20000)); + recordFix(e, fix(30000)); + assert.equal(e.track.resets, 1); + assert.equal(e.track.count, 2); +}); +test('orientation alone smooths at 120 degrees/s and waiting/stale override moving copy', () => { + const e = entry(); + recordFix(e, { ...fix(0), bearingDeg: 90 }); + recordFix(e, { ...fix(15000, 42.00005), bearingDeg: 270 }); + updatePlayback(e, 30000, 0); + e.courseDeg = 0; + e.courseEvalAt = 0; + applyDisplayCourse(e, 100); + assert.equal(e.courseDeg, 12); + applyDisplayCourse(e, 100); + assert.equal(e.courseDeg, 12); + updatePlayback(e, 45000, 15000); + assert.equal(displayMotion(e, 45000).word, 'WAITING'); + updatePlayback(e, 200000, 170000); + assert.equal(displayMotion(e, 200000).word, 'NO FIX'); +}); + +test('display copy never borrows a future bearing and follows anchored freshness after a wall jump', () => { + const e = entry(); + recordFix(e, fix(10000)); + recordFix(e, { ...fix(25000, 42.000005), bearingDeg: 270 }); + updatePlayback(e, 35000, 0); + assert.equal(e.sample.displayT, 10000); + assert.ok(!Number.isFinite(e.sample.segmentCourseDeg)); + const record = { id: 'test', timestamp: 25, bearing: 270 }; + const copy = (wall) => + buildTransitSelectionCopy( + { name: 'Test', region: 'Test' }, + record, + 'bus', + wall, + null, + e, + ); + const before = copy(35000); + assert.equal(before.details[1], 'Moving'); + assert.ok(before.details.includes('reported hdg 270°')); + assert.ok(!before.details[1].includes('270')); + updatePlayback(e, 3600000, 16); + assert.equal(e.sample.motion, 'moving'); + assert.equal(e.sample.latestReportAgeMs, 10016); + assert.equal(displayMotion(e, 3600000).word, 'EN ROUTE'); + assert.equal(copy(3600000).details[2], before.details[2]); + assert.match( + copy(3600000).details[2], + /Reported 10 s ago · shown 25 s behind/, + ); +}); + +test('mesh-floor sampling owns fully initialized numeric scratch before the first frame', async () => { + const { correctHeight } = await import('../../data/contactPlayback.js'); + const e = entry(); + const scratch = [ + e.track.from, + e.track.to, + e.track.latest, + e.track.metricsScratchA, + e.track.metricsScratchB, + e.segment.from, + e.segment.to, + ]; + const fields = [ + 't', + 'lat', + 'lon', + 'heightM', + 'h', + 'receivedAt', + 'bearingDeg', + 'flags', + 'epoch', + 'seq', + ]; + for (const value of scratch) + for (const field of fields) + assert.equal( + typeof value[field], + 'number', + `preallocated numeric ${field}`, + ); + const shapes = scratch.map((value) => Object.keys(value)); + recordFix(e, fix(10000)); + recordFix(e, fix(25000, 42.001)); + correctHeight(e.track, e.track.baseSeq, 12.125); + correctHeight(e.track, e.track.baseSeq + 1, 37.375); + for (let i = 0; i < 10000; i++) { + assert.equal(updatePlayback(e, 35000 + i, i), e.sample); + assert.equal(sampleAt(e.track, 10000 + i, e.sample), e.sample); + assert.ok(Number.isFinite(e.sample.heightM)); + } + [ + e.track.from, + e.track.to, + e.track.latest, + e.track.metricsScratchA, + e.track.metricsScratchB, + e.segment.from, + e.segment.to, + ].forEach((value, i) => assert.equal(value, scratch[i])); + scratch.forEach((value, i) => + assert.deepEqual(Object.keys(value), shapes[i]), + ); +}); diff --git a/src/layers/transit/policy.js b/src/layers/transit/policy.js new file mode 100644 index 000000000..b672581bb --- /dev/null +++ b/src/layers/transit/policy.js @@ -0,0 +1,565 @@ +/** + * @module layers/transit/policy + * @description Constants and pure decisions for the Transit layer. No Cesium + * scene state, no network: everything here is exercised by node:test. + */ + +import { TRANSIT_MODE_ICON } from '../../data/transitFeeds.js'; +import { displayMotion, playbackPosition } from './movement.js'; + +export const TRANSIT_SELECTED_OVERLAY_SOURCE_ID = 'transit-selected'; +export const TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS = Object.freeze({ + cohortLimit: 1, + collisionCapacity: 0, + moving: true, +}); + +// --- Polling / activation --- +/** Poll interval (ms). */ +export const TRANSIT_POLL_MS = 15_000; +/** + * How a vehicle moves between fixes is delayed playback, not a glide: the + * display clock runs a feed-derived lag behind real time and interpolates + * between the two fixes bracketing it (`src/layers/transit/movement.js`). + * A glide that started from the drawn position at receipt time and spanned the + * report gap ran every second segment at 1.5x when MBTA vehicles reported + * every ten seconds against this fifteen-second poll — buses plainly too fast. + */ +/** Camera altitude (m) above which the layer idles: a national fleet still reads at 3,000 km. */ +export const ACTIVATION_ALTITUDE_M = 3_000_000; +export const ACTIVATION_ENTER_ALTITUDE_M = ACTIVATION_ALTITUDE_M - 150_000; +export const ACTIVATION_EXIT_ALTITUDE_M = ACTIVATION_ALTITUDE_M + 150_000; +/** Debounce (ms) for camera-change proximity checks. */ +export const CAMERA_DEBOUNCE_MS = 340; +/** Camera sensitivity this layer asks for while it is enabled. */ +export const CAMERA_PERCENTAGE_CHANGED = 0.05; +/** Extra coverage radius (km) granted to a feed that is already active, so it does not flap at the edge. */ +export const RANGE_SLACK_KM = 40; +/** Hard cap on rendered vehicles across all active feeds. */ +export const MAX_VEHICLES_TOTAL = 15_000; +/** A vehicle absent from this many consecutive polls is removed. */ +export const MISSED_POLLS_TO_DROP = 2; + +// --- Age policy --- +/** + * Oldest fix still drawn, in ms. Past this a vehicle is REMOVED, not merely + * skipped on arrival: a feed that stops answering must not leave the last + * snapshot frozen on the globe pretending to be live traffic. + */ +export const VEHICLE_MAX_FIX_AGE_MS = 10 * 60_000; +/** A feed with no successful poll for this long reads as stale in the panel. */ +export const FEED_STALE_AFTER_MS = 90_000; +/** + * A feed with no successful poll for this long has its vehicles dropped. It is + * deliberately shorter than VEHICLE_MAX_FIX_AGE_MS so a dead feed empties even + * when its last snapshot carried optimistic timestamps. + */ +export const FEED_EVICT_AFTER_MS = 5 * 60_000; + +// --- Ground anchoring --- +/** + * Above this camera altitude (m) a few metres of terrain are invisible, so the + * fleet renders on the ellipsoid and no floor cells are warmed. A national view + * must not queue a country's worth of ground lookups. + */ +export const HEIGHT_SAMPLE_MAX_ALTITUDE_M = 60_000; +/** + * Floor cells warmed per poll. The shared resolver batches and chunks, but the + * bound belongs here: a metropolitan fleet is hundreds of vehicles and only the + * ones in view are worth a lookup at all. + */ +export const FLOOR_WARM_PER_POLL = 300; +/** How soon after a warm the layer looks again to see which cells landed. */ +export const FLOOR_REREAD_MS = 900; +/** Re-reads before a poll's cycle gives up and waits for the next poll. */ +export const FLOOR_REREAD_ATTEMPTS = 6; + +// --- Rendering --- +/** On-screen glyph size in CSS px at the near end of the distance ramp. */ +export const MARKER_PIXEL_SIZE = 20; +export const SELECTED_PIXEL_SIZE = 30; +/** + * How often the fleet's screen-space rotations may be recomputed. Projecting a + * course into screen space depends on the camera, so a moving camera would + * otherwise re-solve every glyph every frame; a fifth of a second of rotation + * lag on a bus icon is invisible, and the saving is not. + */ +export const ROTATION_REFRESH_MS = 200; +/** + * How often the shown/hidden sweep may run. `billboard.show` is a dirty flag, + * so doing this on every camera event during a drag would re-upload the whole + * fleet once a frame — which is the cost it exists to avoid. + */ +export const VISIBILITY_REFRESH_MS = 250; +/** + * How much wider than the camera's own view rectangle a vehicle still counts as + * on screen. Generous on purpose: a vehicle that crosses the edge mid-glide + * should already have been moving, not pop into motion at the border. + */ +export const VIEW_MARGIN_RATIO = 0.6; + +/** + * Whether a vehicle is close enough to the camera's look-at area to be worth + * animating frame by frame. + * + * At street level the layer holds an entire metropolitan fleet — five hundred + * and more vehicles — while the camera can see about fifteen of them. Gliding + * the other five hundred means writing five hundred billboard positions every + * frame, which makes Cesium re-upload the whole buffer, for motion nobody can + * observe. Off-screen vehicles take their newest fix directly instead; they are + * in exactly the right place, and the frame that would have drawn them moving + * is spent on the ones a person is actually looking at. + * + * @param {{south:number,north:number,west:number,east:number}|null} bounds Inflated view bounds, degrees. + * @param {{lat:number, lon:number}} at Vehicle position. + * @returns {boolean} True when there are no bounds to apply, or it is inside them. + */ +export function withinViewBounds(bounds, at) { + if (!bounds || !at) return true; + if (at.lat < bounds.south || at.lat > bounds.north) return false; + if (bounds.west <= bounds.east) { + return at.lon >= bounds.west && at.lon <= bounds.east; + } + // The rectangle straddles the antimeridian, so "inside" is the union of the + // two halves rather than the span between them. + return at.lon >= bounds.west || at.lon <= bounds.east; +} + +/** + * Whether a gliding vehicle is near enough the view to be worth animating and + * showing — judged on where it is NOW as well as where it is going. + * + * Testing only the destination hid a bus that was plainly on screen: a fix + * arrives placing it just past the padded edge, and it disappeared from inside + * the view instead of driving out of it, its glide thrown away in the same + * frame. A vehicle counts as near the view while either end of its current + * glide is inside the bounds, so it leaves the way a person expects — by + * travelling out — and arrives the same way. + * + * @param {{south:number,north:number,west:number,east:number}|null} bounds + * @param {{to:{lat:number,lon:number}, from:{lat:number,lon:number}|null}} entry + * @returns {boolean} + */ +export function vehicleNearView(bounds, entry) { + if (!bounds || !entry) return true; + if (withinViewBounds(bounds, entry.to)) return true; + return entry.from ? withinViewBounds(bounds, entry.from) : false; +} + +/** + * Inflate a camera view rectangle (degrees) into the bounds above. + * + * Longitude is a circle, so padding it has a ceiling: once the padded span + * reaches 360° the rectangle is every longitude there is, and wrapping each + * endpoint on its own would turn that into a narrow band pointing the wrong + * way. A high oblique view over Europe spans about 166°, pads to well past a + * full turn, and used to come back as -177.8°..-172.2° — five degrees of the + * Pacific — which hid an entire national fleet that was plainly on screen. So + * a padded span that closes the circle is reported as the whole circle, and + * only a span that stays under it is wrapped. + * + * @param {{south:number,north:number,west:number,east:number}|null} rect + * @param {number} [ratio=VIEW_MARGIN_RATIO] + * @returns {{south:number,north:number,west:number,east:number}|null} + */ +export function inflateViewBounds(rect, ratio = VIEW_MARGIN_RATIO) { + if (!rect) return null; + const { south, north, west, east } = rect; + if (![south, north, west, east].every(Number.isFinite)) return null; + const padLat = Math.max(0, (north - south) * ratio); + const spanLon = west <= east ? east - west : 360 - west + east; + const padLon = Math.max(0, spanLon * ratio); + const bounds = { + south: Math.max(-90, south - padLat), + north: Math.min(90, north + padLat), + west: -180, + east: 180, + }; + if (spanLon + padLon * 2 >= 360) return bounds; + bounds.west = west - padLon < -180 ? west - padLon + 360 : west - padLon; + bounds.east = east + padLon > 180 ? east + padLon - 360 : east + padLon; + return bounds; +} +/** Throttle (ms) for re-anchoring the selected vehicle's card while it glides. */ +export const SELECTED_CARD_REFRESH_MS = 250; +/** Mode palette: distinct at a glance, none reused by flights (white/cyan), military (amber), or vessels. */ +export const TRANSIT_MODE_COLORS = Object.freeze({ + bus: '#5EF08A', + tram: '#FFC24A', + subway: '#FF4538', + rail: '#D9A6FF', + ferry: '#5FD6FF', + unknown: '#D8DDE5', +}); + +/** + * Stable key for a vehicle across polls. + * @param {string} feedId + * @param {string} vehicleId + * @returns {string} + */ +export function transitVehicleKey(feedId, vehicleId) { + return `${feedId}:${vehicleId}`; +} + +/** + * Where the vehicle is drawn: the display clock's position along its fix + * history. `now` is accepted for the older callers and ignored — the clock is + * advanced by the frame pass, and reading it here must not move it. + * @param {{fixes?:Array, to?:{lat:number,lon:number}}} entry + * @param {number} [now] Unused. + * @returns {{lat:number, lon:number, settled:boolean}} + */ +export function interpolatedVehiclePosition(entry, now) { + void now; + const played = entry?.track ? playbackPosition(entry) : null; + if (played) return played; + const to = entry?.to || { lat: NaN, lon: NaN }; + return { lat: to.lat, lon: to.lon, settled: true }; +} + +/** + * When a fix was REPORTED, in ms, from the best timestamp available. + * + * The ordering questions — is this fix newer than the one we drew, how much + * time passed between them — need an absolute time, not an age. Same ladder as + * the age helper: the vehicle's own stamp, else the snapshot's fetch time. + * + * @param {object} record Normalized vehicle record. + * @param {number|null} [fetchedAtMs] Snapshot fetch time. + * @returns {number} Report time in ms. + */ +export function vehicleReportTimeMs(record, fetchedAtMs = null) { + const seconds = Number(record?.timestamp); + if (Number.isFinite(seconds) && seconds > 0) return seconds * 1000; + if (Number.isFinite(fetchedAtMs)) return fetchedAtMs; + return Date.now(); +} + +/** + * Age of a vehicle's reported fix in ms, from the best timestamp available. + * + * The proxy already resolves the ladder (the vehicle's own timestamp, else the + * feed header's, else fetch time) and says which it used. A record that somehow + * arrives with no usable timestamp at all is aged from the snapshot's fetch + * time, never from "now" — treating an unstamped fix as fresh is exactly how a + * ten-minute-old bus gets drawn as live. + * + * @param {object} record Normalized vehicle record. + * @param {number} nowMs + * @param {number|null} [fetchedAtMs] Snapshot fetch time. + * @returns {number} Age in ms (never negative). + */ +export function vehicleFixAgeMs(record, nowMs, fetchedAtMs = null) { + const seconds = Number(record?.timestamp); + if (Number.isFinite(seconds) && seconds > 0) { + return Math.max(0, nowMs - seconds * 1000); + } + if (Number.isFinite(fetchedAtMs)) return Math.max(0, nowMs - fetchedAtMs); + return 0; +} + +/** + * Whether a feed fix is too old to draw. + * @param {object} record Normalized vehicle record. + * @param {number} nowMs + * @param {number|null} [fetchedAtMs] Snapshot fetch time. + * @returns {boolean} + */ +export function isStaleVehicleFix(record, nowMs, fetchedAtMs = null) { + return vehicleFixAgeMs(record, nowMs, fetchedAtMs) > VEHICLE_MAX_FIX_AGE_MS; +} + +/** + * Roll per-feed poll health into the one honest row state the layer panel reads. + * + * `every(stale)` was the bug this replaces: with Boston fresh and Helsinki an + * hour old, "not every feed is stale" read as nominal and the row claimed + * everything was fine. One stale feed beside a fresh one is DEGRADED — the + * view is part live and part not, and the row has to say so. + * + * @param {Array<{stale:boolean, error:string|null, retryInSec?:number|null, + * upstreamAt:number|null, loading:boolean}>} statuses + * @param {number} nowMs + * @returns {{stale:boolean, degraded:boolean, error:string|null, retryInSec:number|null, + * loading:boolean, staleFeeds:number}} + */ +export function aggregateTransitFeedHealth(statuses, nowMs) { + const list = Array.isArray(statuses) ? statuses : []; + if (list.length === 0) { + return { + stale: false, + degraded: false, + error: null, + retryInSec: null, + loading: false, + staleFeeds: 0, + }; + } + // Age is measured from when the OPERATOR last answered, never from when this + // browser last received bytes — the proxy replays its cached copy on a fixed + // cadence during an outage, and that cadence is not freshness. + const isStale = (status) => + status.stale === true || + (Number.isFinite(status.upstreamAt) && + nowMs - status.upstreamAt > FEED_STALE_AFTER_MS); + const staleFeeds = list.filter(isStale).length; + const failing = list.find((status) => status.error) || null; + const error = failing?.error || null; + return { + stale: staleFeeds === list.length, + degraded: (staleFeeds > 0 && staleFeeds < list.length) || Boolean(error), + error, + retryInSec: Number.isFinite(failing?.retryInSec) + ? failing.retryInSec + : null, + loading: list.some((status) => status.loading), + staleFeeds, + }; +} + +/** + * Build the human lines of the selection card from a vehicle record. + * @param {object} feed Registry entry. + * @param {object} record Normalized vehicle record. + * @param {string} mode Resolved transit mode. + * @param {number} nowMs + * @param {number|null} [fetchedAtMs] Snapshot fetch time. + * @returns {{title:string, details:string[]}} + */ +export function buildTransitSelectionCopy( + feed, + record, + mode, + nowMs, + fetchedAtMs = null, + entry = null, +) { + const icon = TRANSIT_MODE_ICON[mode] || TRANSIT_MODE_ICON.unknown; + const routeLabel = record.routeId + ? `Route ${record.routeId}` + : record.label + ? `Vehicle ${record.label}` + : `Vehicle ${record.id}`; + const title = `${icon} ${routeLabel}`; + // The mode is said in words on its own line. A glyph and a route id do not + // tell a reader that the dot on Tremont Street is a subway train, and a + // subway drawn on a street reads as a bug until the card says what it is. + const kindWord = TRANSIT_MODE_WORD[mode] || TRANSIT_MODE_WORD.unknown; + const details = [`${kindWord} · ${feed.name} · ${feed.region}`]; + const motion = []; + // What the SCREEN is doing leads, because that is what the reader can check. + const shown = entry ? displayMotion(entry, nowMs) : null; + if (shown?.word === 'STOPPED') motion.push('Stopped'); + else if (shown?.word === 'NO FIX') motion.push('No recent fix'); + else if (shown?.word === 'WAITING') motion.push('Waiting for update'); + else if (shown) motion.push('Moving'); + if (motion.length) details.push(motion.join(' · ')); + const reportedMotion = []; + if (Number.isFinite(record.speedMps)) + reportedMotion.push(`reported ${Math.round(record.speedMps * 3.6)} km/h`); + if (Number.isFinite(record.bearing)) + reportedMotion.push(`reported hdg ${Math.round(record.bearing)}°`); + // The operator's own claim, kept as a REPORT rather than as the live state. + // It describes the moment of the report, which the display may not have + // reached yet; presenting it as "now" is what put STOPPED on a moving metro. + const state = []; + if (record.status === 'STOPPED_AT' && record.stopId) + state.push(`Last report: stopped at stop ${record.stopId}`); + else if (record.status === 'INCOMING_AT' && record.stopId) + state.push(`Last report: arriving at stop ${record.stopId}`); + else if (record.status === 'IN_TRANSIT_TO' && record.stopId) + state.push(`Last report: next stop ${record.stopId}`); + if (record.occupancy && record.occupancy !== 'NO_DATA_AVAILABLE') { + state.push(record.occupancy.toLowerCase().replaceAll('_', ' ')); + } + if (reportedMotion.length) details.push(reportedMotion.join(' · ')); + if (state.length) details.push(state.join(' · ')); + // Underground services are drawn on the surface directly above themselves. + // That is a deliberate choice — a train nobody can see is not worth drawing — + // but it has to be stated, or a metro sitting on a road looks like an error. + if (UNDERGROUND_MODES.has(mode)) { + details.push('Shown at street level · depth not in the feed'); + } + details.push( + `Vehicle ${record.id}${record.label && record.label !== record.id ? ` · ${record.label}` : ''}`, + ); + const ageS = Math.round( + (Number.isFinite(entry?.sample?.latestReportAgeMs) + ? entry.sample.latestReportAgeMs + : vehicleFixAgeMs(record, nowMs, fetchedAtMs)) / 1000, + ); + const reported = + ageS < 90 + ? `Reported ${ageS} s ago` + : `Reported ${Math.round(ageS / 60)} min ago`; + // A time the feed did not give for THIS vehicle is labelled, so the card + // never presents a borrowed or inferred time as the vehicle's own report. + const ownTime = + Number.isFinite(record.timestamp) && + record.timestamp > 0 && + (!record.timestampSource || record.timestampSource === 'vehicle'); + // How far behind real time the display is running, said separately from + // how old the newest report is: the two are different facts. + const lagS = Number.isFinite(entry?.sample?.actualDelayMs) + ? Math.max(0, Math.round(entry.sample.actualDelayMs / 1000)) + : null; + const behind = lagS === null ? '' : ` · shown ${lagS} s behind`; + details.splice( + motion.length ? 2 : 1, + 0, + (ownTime ? reported : `${reported} (feed time)`) + behind, + ); + if (entry?.track?.count) { + const duration = Math.max( + 0, + Math.min(entry.playT, entry.fixes.at(-1).t) - entry.fixes[0].t, + ); + details.push( + `Available trail: ${Math.round(duration / 60000)} min${entry.track.truncated || entry.trailTruncated ? ' (limited)' : ''}`, + ); + } + return { title, details }; +} + +/** + * Shared-host card for the selected vehicle. + * @param {string} key + * @param {object|(() => object|null)} position Cartesian position, or a + * getter the host calls on every paint so the anchor follows the marker. + * @param {{title:string, details:string[]}} copy + * @param {string} mode + * @returns {object|null} + */ +export function createTransitSelectedOverlayEntry(key, position, copy, mode) { + if (!key || !position) return null; + return { + id: String(key), + position, + variant: 'selected', + selected: true, + protected: true, + paintLane: 'selected', + collisionGroup: 'ambient-card', + priority: Number.MAX_SAFE_INTEGER, + title: copy.title, + details: copy.details, + accent: TRANSIT_MODE_COLORS[mode] || TRANSIT_MODE_COLORS.unknown, + interactive: false, + anchorRadiusPx: 9, + minAnchorGapPx: 11, + verticalOnly: true, + placement: 'above', + edgeFade: 'keyhole', + horizonCull: true, + terrainOcclusion: false, + }; +} + +/** Modes that ordinarily run below the street, drawn on the surface above themselves. */ +export const UNDERGROUND_MODES = new Set(['subway']); + +/** The mode in plain words, for a card a person reads rather than scans. */ +export const TRANSIT_MODE_WORD = Object.freeze({ + bus: 'Bus', + tram: 'Tram', + subway: 'Subway', + rail: 'Train', + ferry: 'Ferry', + unknown: 'Transit vehicle', +}); + +/** Short, fixed-width mode word for the detection card's class field. */ +export const TRANSIT_MODE_ABBR = Object.freeze({ + bus: 'BUS', + tram: 'TRAM', + subway: 'METRO', + rail: 'RAIL', + ferry: 'FERRY', + unknown: 'TRANSIT', +}); + +/** + * The bright first line of a vehicle's detection label: its route when the feed + * names one, else the fleet number, else the raw id. What a person standing at + * the stop would call it. + * @param {object} record Normalized vehicle record. + * @returns {string} + */ +export function transitDetectionId(record) { + const route = + typeof record?.routeId === 'string' ? record.routeId.trim() : ''; + if (route) return elideRouteText(route); + const label = typeof record?.label === 'string' ? record.label.trim() : ''; + if (label) return elideRouteText(label); + return elideRouteText(String(record?.id || 'TRANSIT')); +} + +/** Longest route text the single-line detection label can carry. */ +export const DETECTION_ID_MAX_CHARS = 14; + +/** + * Cut long route text with an ellipsis, so a reader can see it was cut. + * + * `Shuttle-Generic` used to reach the screen as `SHUTTLE-GENERI`, which does + * not read as an abbreviation — it reads as a typo, or as a route that exists. + * A trailing ellipsis costs one character and says plainly that there is more. + * + * @param {string} text + * @returns {string} Uppercase, at most DETECTION_ID_MAX_CHARS characters. + */ +export function elideRouteText(text) { + const value = String(text || '').toUpperCase(); + if (value.length <= DETECTION_ID_MAX_CHARS) return value; + return `${value.slice(0, DETECTION_ID_MAX_CHARS - 1)}\u2026`; +} + +/** + * The dim second line: what kind of vehicle it is and who runs it. + * @param {string} mode Resolved transit mode. + * @param {object} feed Registry entry. + * @returns {string} + */ +export function transitDetectionClass(mode, feed) { + const kind = TRANSIT_MODE_ABBR[mode] || TRANSIT_MODE_ABBR.unknown; + const operator = + typeof feed?.name === 'string' ? feed.name.toUpperCase() : ''; + return operator ? `${kind} ${operator}`.slice(0, 20) : kind; +} + +/** + * The micro field the detection host actually paints beside the route. + * + * The host draws a single-line track label — the bright id and one dim + * micro-field — so mode and status have to share that field or go unseen. Mode + * leads because it tells you what you are looking at (and matches the glyph + * shape), then what the vehicle is doing: a stopped bus reporting 0 km/h is + * worth saying out loud, it is the most common state in a city at rush hour. + * + * @param {object} record Normalized vehicle record. + * @param {string} mode Resolved transit mode. + * @returns {string} + */ +export function transitDetectionMetric(entry, mode, nowMs = Date.now()) { + const kind = TRANSIT_MODE_ABBR[mode] || TRANSIT_MODE_ABBR.unknown; + const state = transitVehicleState(entry, nowMs); + return state ? `${kind} ${state}` : kind; +} + +/** + * What the vehicle is doing right now, in as few characters as it takes — + * judged on what the screen is showing, not on what the feed claims. + * + * This used to read `current_status` straight out of the newest record, which + * is how a metro came to glide across the owner's screen with STOPPED printed + * beside it: the status describes the moment of the report, and the display is + * still animating the segment before it. The feed is not lying; it is early. + * Its claim is kept, verbatim and dated, on the selected card. + * + * @param {object} entry Vehicle runtime entry. + * @param {number} [nowMs] + * @returns {string} + */ +export function transitVehicleState(entry, nowMs = Date.now()) { + return displayMotion(entry, nowMs).word; +} diff --git a/src/layers/transit/qaMetrics.js b/src/layers/transit/qaMetrics.js new file mode 100644 index 000000000..908aaa168 --- /dev/null +++ b/src/layers/transit/qaMetrics.js @@ -0,0 +1,331 @@ +/** Acceptance reducers. Missing, non-finite or unexercised evidence never passes. */ +export function percentile(values, fraction = 0.95) { + if (!values.length || values.some((v) => !Number.isFinite(v))) + return Infinity; + const sorted = [...values].sort((a, b) => a - b); + return sorted[Math.max(0, Math.ceil(sorted.length * fraction) - 1)]; +} + +export function reduceMotion(rows, tolerance = 0.02) { + let tested = 0, + moving = 0, + worstRelativeError = 0, + distance = 0, + worstFrame = null; + for (let i = 1; i < rows.length; i++) { + const a = rows[i - 1], + b = rows[i]; + if ( + !(a.expectedMps > 0) || + a.segment !== b.segment || + a.toSeq !== b.toSeq || + (a.phase && a.phase !== 'playing') || + (b.phase && b.phase !== 'playing') + ) + continue; + const seconds = + (Number.isFinite(a.monoMs) && Number.isFinite(b.monoMs) + ? b.monoMs - a.monoMs + : b.displayT - a.displayT) / 1000; + if (!(seconds > 0)) { + tested++; + worstRelativeError = Infinity; + continue; + } + const metres = Math.hypot(b.x - a.x, b.y - a.y, b.z - a.z); + const error = Math.abs(metres / seconds / a.expectedMps - 1); + tested++; + if (error > worstRelativeError || !Number.isFinite(error)) + worstFrame = { + key: b.key, + fromIndex: i - 1, + toIndex: i, + from: a, + to: b, + seconds, + metres, + error, + }; + if (metres > 0.00001) moving++; + distance += metres; + worstRelativeError = Math.max( + worstRelativeError, + Number.isFinite(error) ? error : Infinity, + ); + } + return { + pass: + tested > 30 && + moving === tested && + distance > 1 && + worstRelativeError <= tolerance, + tested, + moving, + distance, + worstRelativeError, + worstFrame, + tolerance, + }; +} + +export function reduceAnchor(rows) { + const hiddenFrames = rows.filter((r) => r.markerVisible === false).length; + const invalidWorld = rows.filter((r) => r.worldFinite === false); + rows = rows.filter((r) => r.markerVisible !== false); + // The host acquires its first card asynchronously. Only its leading acquisition + // frames are outside the anchor test; any later missing anchor is a failure. + const acquired = rows.findIndex((r) => + [r.mx, r.my, r.ax, r.ay].every(Number.isFinite), + ); + const acquisitionFrames = acquired < 0 ? rows.length : acquired; + rows = rows.slice(acquisitionFrames); + let samples = 0, + moving = 0, + maxErrorPx = 0; + for (let i = 0; i < rows.length; i++) { + const r = rows[i]; + if (![r.mx, r.my, r.ax, r.ay].every(Number.isFinite)) continue; + samples++; + maxErrorPx = Math.max(maxErrorPx, Math.hypot(r.mx - r.ax, r.my - r.ay)); + const a = rows[i - 1]; + if (a && Math.hypot(r.mx - a.mx, r.my - a.my) > 0.0001) moving++; + } + const polls = new Set(rows.map((r) => r.poll).filter(Number.isFinite)).size; + const conditions = { + enoughSamples: samples > 30, + allFinite: samples === rows.length && invalidWorld.length === 0, + moving: moving > 30, + withinOnePixel: maxErrorPx <= 1, + pollTransition: polls >= 2, + }; + return { + pass: Object.values(conditions).every(Boolean), + conditions, + hiddenFrames, + invalidSamples: rows + .filter((r) => ![r.mx, r.my, r.ax, r.ay].every(Number.isFinite)) + .slice(0, 8), + invalidWorld: invalidWorld.slice(0, 8), + acquisitionFrames, + inputFrames: rows.length + acquisitionFrames + hiddenFrames, + polls, + samples, + moving, + maxErrorPx, + }; +} + +export function reduceFleet( + rows, + count, + allocationBytesPerFrame, + retainedBytes, + fixBytes, +) { + const street = count === 800; + const limits = { + cpuMs: street ? 1 : 4, + intervalMs: street ? 20 : 25, + uploadBytes: street ? 262144 : 1048576, + allocationBytes: 8192, + retainedBytes: (street ? 24 : 48) * 1048576, + fixBytes: count * 128 * 48, + }; + const metrics = { + frames: rows.length, + cpuP95: percentile(rows.map((r) => r.cpuMs)), + intervalP95: percentile(rows.map((r) => r.intervalMs)), + uploadMax: rows.length + ? Math.max(...rows.map((r) => r.uploadBytes)) + : Infinity, + movingMin: rows.length ? Math.min(...rows.map((r) => r.moving)) : 0, + allocationBytesPerFrame, + retainedBytes, + fixBytes, + }; + const motion = reduceMotion(rows); + return { + pass: + rows.length >= 300 && + motion.pass && + metrics.movingMin === count && + metrics.cpuP95 <= limits.cpuMs && + metrics.intervalP95 <= limits.intervalMs && + metrics.uploadMax > 0 && + metrics.uploadMax <= limits.uploadBytes && + Number.isFinite(allocationBytesPerFrame) && + allocationBytesPerFrame <= limits.allocationBytes && + Number.isFinite(retainedBytes) && + retainedBytes > 0 && + retainedBytes <= limits.retainedBytes && + fixBytes > 0 && + fixBytes <= limits.fixBytes, + limits, + metrics, + motion, + }; +} + +export function reduceScenario(trace, key) { + const keys = key.startsWith('mbta:') ? key : `mbta:sim-${key}`; + const rows = trace.out[keys] || []; + let moved = 0; + let total = 0; + let worstJump = 0; + let speedWorst = 0; + let wallFaster = 0; + let lagMax = 0; + let lagMin = Infinity; + const initialResets = (rows.find((r) => r[5] === 1) || rows[0])?.[4] || 0; + let resets = 0; + let backwards = 0; + for (let i = 1; i < rows.length; i += 1) { + const a = rows[i - 1]; + const b = rows[i]; + const d = Math.hypot( + (b[1] - a[1]) * 111320, + (b[2] - a[2]) * 111320 * Math.cos((a[1] * Math.PI) / 180), + ); + // Position and playback time were sampled in preRender; GPU completion + // jitter in column zero must not masquerade as acceleration or catch-up. + const dWall = + (Number.isFinite(a[16]) && Number.isFinite(b[16]) + ? b[16] - a[16] + : b[0] - a[0]) / 1000; + const dPlay = (b[3] - a[3]) / 1000; + total += d; + if (d > 0.05) moved += 1; + worstJump = Math.max(worstJump, d / Math.max(dWall, 1 / 120)); + if (dPlay > dWall * 1.02 + 0.002) wallFaster += 1; + if (dPlay > 0 && d > 0.001) speedWorst = Math.max(speedWorst, d / dPlay); + if (b[1] < a[1] - 0.5 / 111320) backwards += 1; + resets = Math.max(resets, b[4] - initialResets); + const lag = (b[7] - b[3]) / 1000; + lagMax = Math.max(lagMax, lag); + lagMin = Math.min(lagMin, lag); + } + return { + keys, + lagValid: + rows.length > 1 && Number.isFinite(lagMin) && Number.isFinite(lagMax), + rows: rows.length, + moved, + total, + worstJump, + speedWorst, + wallFaster, + lagMax, + lagMin, + resets, + initialResets, + backwards, + last: rows.at(-1), + first: rows[0], + }; +} + +/** Choose actual sampled backgrounds, never infer brightness from a place name. */ +export function reduceBackground(name, pixels) { + const values = pixels.map((p) => p.background); + const luma = values.length + ? values.reduce((a, b) => a + b, 0) / values.length + : NaN; + const threshold = name === 'bright' ? 0.55 : 0.35; + return { + pass: + values.length >= 4 && + values.every(Number.isFinite) && + (name === 'bright' ? luma >= threshold : luma <= threshold), + luma, + samples: values.length, + threshold, + comparison: name === 'bright' ? '>=' : '<=', + }; +} + +/** Convert captured tuples without confusing paint completion with the playback clock. */ +export function reduceScriptedMotion(key, rows) { + return reduceMotion( + rows.map((r) => ({ + key, + monoMs: r[16], + x: r[11], + y: r[12], + z: r[13], + displayT: r[3], + expectedMps: r[9], + segment: r[10], + toSeq: r[14], + phase: r[15], + })), + ); +} + +/** + * In-scene sensor signatures retain the active shader. Surveillance peak white + * is P43 phosphor (0.16, 1, 0.22): Rec.601 luma 0.65992. D1 requires 90% of + * that peak and a ring <= 0.25. Local background delta is diagnostic. + * White-hot, Ironbow and noir require 0.85; black-hot keeps the inverse. + * Six verified, unobscured sprites are required; absence is UNEXERCISED. + */ +export function reduceSensorContrast(pixels, polarity = 'white', preset = '') { + const limits = + polarity === 'black' + ? { centreMax: 0.15, ringMin: 0.75 } + : { + centreMin: + preset === 'surveillance' + ? 0.9 * (0.299 * 0.16 + 0.587 + 0.114 * 0.22) + : 0.85, + ringMax: 0.25, + backgroundDeltaMin: null, + }; + const judged = pixels + .filter((p) => p.verified === true) + .map((p) => ({ + key: p.key, + centre: p.centre, + background: p.background, + backgroundDelta: p.centre - p.background, + ring: polarity === 'black' ? p.ringMax : p.ringMin, + pass: + polarity === 'black' + ? p.centre <= limits.centreMax && p.ringMax >= limits.ringMin + : p.centre >= limits.centreMin && + p.ringMin <= limits.ringMax && + (limits.backgroundDeltaMin === null || + p.centre - p.background >= limits.backgroundDeltaMin), + })); + return { + pass: judged.length >= 6 && judged.every((p) => p.pass), + unexercised: judged.length < 6, + reason: + judged.length < 6 + ? `Only ${judged.length} verified unobscured sprites; need 6` + : null, + limits, + judged, + }; +} + +/** Selected trail evidence stays inspectable even when the click failed. */ +export function reduceTrailHead(anchor, target) { + const conditions = { + selected: anchor?.selected === true, + enoughHeadSamples: + Number.isFinite(anchor?.headSamples) && anchor.headSamples > 30, + aligned: Number.isFinite(anchor?.headErrorPx) && anchor.headErrorPx <= 1, + stableBody: anchor?.bodyMutation === 0, + stableEntityCount: + Number.isFinite(anchor?.entityCount) && + anchor.entityCount === target?.entityCount, + }; + return { + pass: Object.values(conditions).every(Boolean), + conditions, + headSamples: anchor?.headSamples ?? null, + errorPx: anchor?.headErrorPx ?? null, + bodyMutation: anchor?.bodyMutation ?? null, + selection: anchor?.selection ?? null, + }; +} diff --git a/src/layers/transit/qaMetrics.test.mjs b/src/layers/transit/qaMetrics.test.mjs new file mode 100644 index 000000000..47a8a1be3 --- /dev/null +++ b/src/layers/transit/qaMetrics.test.mjs @@ -0,0 +1,330 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + reduceMotion, + reduceAnchor, + reduceFleet, + percentile, +} from './qaMetrics.js'; +const rows = () => + Array.from({ length: 400 }, (_, i) => ({ + x: i / 6, + y: 0, + z: 0, + displayT: (i * 1000) / 60, + expectedMps: 10, + segment: 1, + mx: i, + my: 20, + ax: i, + ay: 20, + cpuMs: 0.5, + intervalMs: 1000 / 60, + uploadBytes: 128000, + moving: 800, + poll: Math.floor(i / 200), + })); +test('ordinary 60 Hz motion passes; frozen motion, time and overspeed fail', () => { + assert.equal(reduceMotion(rows()).pass, true); + assert.equal(reduceMotion(rows().map((r) => ({ ...r, x: 0 }))).pass, false); + assert.equal( + reduceMotion(rows().map((r) => ({ ...r, displayT: 0 }))).pass, + false, + ); + assert.equal( + reduceMotion(rows().map((r) => ({ ...r, x: r.x * 1.03 }))).pass, + false, + ); +}); +test('actual anchor agreement passes; equal travel with a fixed offset fails', () => { + assert.equal(reduceAnchor(rows()).pass, true); + assert.equal( + reduceAnchor(rows().map((r) => ({ ...r, ax: r.ax + 2 }))).pass, + false, + ); + assert.equal( + reduceAnchor(rows().map((r) => ({ ...r, ax: null }))).pass, + false, + ); +}); +test('empty samples and unmeasured allocation never pass acceptance', () => { + assert.equal(percentile([]), Infinity); + assert.equal(reduceMotion([]).pass, false); + assert.equal(reduceAnchor([]).pass, false); + assert.equal(reduceFleet([], 800, 0, 1, 1).pass, false); + assert.equal(reduceFleet(rows(), 800, undefined, 1, 1).pass, false); + assert.equal(reduceFleet(rows(), 800, 100, 10000, 800 * 128 * 48).pass, true); + assert.equal( + reduceFleet( + rows().map((r) => ({ ...r, moving: 1 })), + 800, + 100, + 10000, + 10000, + ).pass, + false, + ); +}); + +test('ordinary frames judge drawn speed against elapsed time and name the offending pair', () => { + const slow = rows().map((r, i) => ({ + ...r, + key: 'mbta:sim-dup', + monoMs: (i * 1000) / 60, + displayT: r.displayT * 0.8858664517, + x: r.x * 0.8858664517, + })); + const result = reduceMotion(slow); + assert.equal(result.pass, false); + assert.equal(result.worstFrame.key, 'mbta:sim-dup'); + assert.ok(result.worstRelativeError > 0.114); +}); + +test('per-scenario reducer resolves full vehicle keys and cannot pass empty lag', async () => { + const { reduceScenario } = await import('./qaMetrics.js'); + const trace = { + out: { + 'mbta:sim-straight': Array.from({ length: 5645 }, (_, i) => [ + i * 17, + 42 + i / 111320, + -71, + i * 17, + 0, + 1, + 0, + i * 17 + 25000, + ]), + }, + }; + const result = reduceScenario(trace, 'straight'); + assert.equal(result.rows, 5645); + assert.equal(result.keys, 'mbta:sim-straight'); + assert.equal(result.lagMin, 25); + assert.equal(reduceScenario({ out: {} }, 'gap').lagValid, false); +}); + +test('anchor names its judged conditions and accepts 1128 aligned frames across two polls', () => { + const fixture = Array.from({ length: 1128 }, (_, i) => ({ + mx: i, + my: 0, + ax: i + 4e-9, + ay: 0, + poll: i < 564 ? 1 : 2, + })); + const result = reduceAnchor([ + { mx: 0, my: 0, ax: null, ay: null }, + ...fixture, + ]); + assert.equal(result.pass, true); + assert.equal(result.conditions.pollTransition, true); + assert.equal(result.polls, 2); + assert.equal( + reduceAnchor(fixture.map((r) => ({ ...r, poll: 1 }))).pass, + false, + ); +}); + +test('Boston mid-luma fixtures cannot qualify as bright or dark', async () => { + const { reduceBackground } = await import('./qaMetrics.js'); + assert.equal( + reduceBackground('bright', Array(4).fill({ background: 0.423 })).pass, + false, + ); + assert.equal( + reduceBackground('dark', Array(4).fill({ background: 0.403 })).pass, + false, + ); + assert.equal( + reduceBackground('bright', Array(4).fill({ background: 0.65 })).pass, + true, + ); + assert.equal( + reduceBackground('dark', Array(4).fill({ background: 0.2 })).pass, + true, + ); +}); + +test('variable GPU work after sampling cannot change the judged playback speed', async () => { + const { reduceScriptedMotion } = await import('./qaMetrics.js'); + const frames = Array.from({ length: 400 }, (_, i) => { + const r = Array(17).fill(0), + stamp = (i * 1000) / 60; + r[0] = stamp + (i % 2 ? 4 : 0); + r[3] = stamp; + r[9] = 10; + r[10] = 1; + r[11] = i / 6; + r[14] = 2; + r[15] = 'playing'; + r[16] = stamp; + return r; + }); + assert.equal(reduceScriptedMotion('mbta:sim-straight', frames).pass, true); +}); + +test('sensor thresholds retain in-scene NVG peak, halo and six-sprite minimum', async () => { + const { reduceSensorContrast } = await import('./qaMetrics.js'); + const pixels = (p, n = 6) => + Array.from({ length: n }, (_, i) => ({ + key: `bus-${i}`, + verified: true, + ...p, + })); + assert.equal( + reduceSensorContrast( + pixels({ centre: 0.64, ringMin: 0.03, background: 0.3 }), + 'white', + 'surveillance', + ).pass, + true, + ); + assert.equal( + reduceSensorContrast( + pixels({ centre: 0.64, ringMin: 0.03, background: 0.9 }), + 'white', + 'surveillance', + ).pass, + true, + ); + assert.equal( + reduceSensorContrast( + pixels({ centre: 0.59, ringMin: 0.03, background: 0.1 }), + 'white', + 'surveillance', + ).pass, + false, + ); + assert.equal( + reduceSensorContrast(pixels({ centre: 0.64, ringMin: 0.03 })).pass, + false, + ); + assert.equal( + reduceSensorContrast(pixels({ centre: 0.86, ringMin: 0.2 })).pass, + true, + ); + assert.equal( + reduceSensorContrast(pixels({ centre: 0.14, ringMax: 0.8 }), 'black').pass, + true, + ); + assert.equal( + reduceSensorContrast(pixels({ centre: 1, ringMin: 0.1 }, 5)).unexercised, + true, + ); +}); + +test('initial scripted synchronization is disclosed but subsequent resets fail', async () => { + const { reduceScenario } = await import('./qaMetrics.js'); + const trace = { + out: { + 'mbta:sim-straight': Array.from({ length: 100 }, (_, i) => [ + i * 17, + 42 + i / 111320, + -71, + i * 17, + i < 2 ? 0 : 1, + i < 2 ? 0 : 1, + 0, + i * 17 + 25000, + ]), + }, + }; + assert.equal(reduceScenario(trace, 'straight').resets, 0); + assert.equal(reduceScenario(trace, 'straight').initialResets, 1); + trace.out['mbta:sim-straight'][50][4] = 2; + assert.equal(reduceScenario(trace, 'straight').resets, 1); +}); + +test('D2 and D3 gate authored allocation inclusively and metro CPU at four milliseconds', () => { + const fleet = rows().map((r) => ({ ...r, moving: 3000, cpuMs: 3.4 })); + assert.equal( + reduceFleet(fleet, 3000, 8192, 10000, 3000 * 128 * 48).pass, + true, + ); + assert.equal( + reduceFleet(fleet, 3000, 8193, 10000, 3000 * 128 * 48).pass, + false, + ); + assert.equal( + reduceFleet( + fleet.map((r) => ({ ...r, cpuMs: 4.1 })), + 3000, + 0, + 10000, + 10000, + ).pass, + false, + ); +}); + +test('anchor ignores explicitly hidden markers but exposes missing painted cards and nonfinite world positions', () => { + const samples = rows(); + samples.splice(100, 0, { + markerVisible: false, + mx: 0, + my: 0, + ax: null, + ay: null, + worldFinite: true, + }); + assert.equal(reduceAnchor(samples).pass, true); + assert.equal(reduceAnchor(samples).hiddenFrames, 1); + samples[100].markerVisible = true; + assert.equal(reduceAnchor(samples).pass, false); + assert.equal(reduceAnchor(samples).invalidSamples.length, 1); + samples[100].markerVisible = false; + samples[100].worldFinite = false; + assert.equal(reduceAnchor(samples).pass, false); +}); + +test('scenario speed and catch-up use sampling time, not variable GPU completion time', async () => { + const { reduceScenario } = await import('./qaMetrics.js'); + const rows = Array.from({ length: 100 }, (_, i) => { + const r = Array(17).fill(0); + r[0] = i * 17 + (i % 2 ? 4 : 0); + r[1] = 42 + (i * 0.17) / 111320; + r[2] = -71; + r[3] = i * 17; + r[5] = 1; + r[7] = r[3] + 25000; + r[16] = i * 17; + return r; + }); + const result = reduceScenario( + { out: { 'mbta:sim-straight': rows } }, + 'straight', + ); + assert.ok(Math.abs(result.worstJump - 10) < 1e-6); + assert.equal(result.wallFaster, 0); + for (const r of rows) r[16] *= 0.8; + const fast = reduceScenario( + { out: { 'mbta:sim-straight': rows } }, + 'straight', + ); + assert.ok(fast.worstJump > 12); + assert.ok(fast.wallFaster > 0); +}); + +test('trail-head acceptance names every missing and failed condition', async () => { + const { reduceTrailHead } = await import('./qaMetrics.js'); + const missing = reduceTrailHead({ selected: false }, { entityCount: 0 }); + assert.equal(missing.pass, false); + assert.deepEqual(missing.conditions, { + selected: false, + enoughHeadSamples: false, + aligned: false, + stableBody: false, + stableEntityCount: false, + }); + const good = { + selected: true, + headSamples: 100, + headErrorPx: 0, + bodyMutation: 0, + entityCount: 0, + }; + assert.equal(reduceTrailHead(good, { entityCount: 0 }).pass, true); + assert.equal( + reduceTrailHead({ ...good, headErrorPx: 2 }, { entityCount: 0 }).pass, + false, + ); +}); diff --git a/src/layers/transit/queries.js b/src/layers/transit/queries.js new file mode 100644 index 000000000..762004f1e --- /dev/null +++ b/src/layers/transit/queries.js @@ -0,0 +1,237 @@ +import { + ACTIVATION_ALTITUDE_M, + FEED_STALE_AFTER_MS, + aggregateTransitFeedHealth, + transitDetectionClass, + transitDetectionId, + transitDetectionMetric, +} from './policy.js'; +import { + TRANSIT_ENABLED_FEEDS, + getRegisteredTransitFeed, +} from '../../data/transitFeeds.js'; +import { transitModeTier } from '../../data/transitPresetStyle.js'; + +/** + * The layer row's answer to "what is on screen and is it honest?". + * @param {object} context + * @returns {object} + */ +export function createQueries({ state, parts }) { + const empty = []; + const defaults = {}; + let requestedMax = Infinity, + requestedSeed = 0; + /** + * A stable 32-bit hash of a vehicle key, mixed with the caller's seed. + * + * Selection has to survive ordinary churn. Striding by position in the map + * meant that one bus leaving at the front renumbered every bus behind it, so + * a single removal swapped the whole labelled set for its neighbours — + * visible as every callout jumping one vehicle sideways. A hash of the key + * belongs to the vehicle, not to its place in a list, so a removal anywhere + * leaves every other vehicle's standing untouched. + * + * @param {string} key + * @param {number} seed + * @returns {number} Unsigned 32-bit. + */ + function selectionRank(key, seed) { + // FNV-1a, seeded. Cheap, no allocation, and well spread for short ids. + let hash = (2166136261 ^ seed) >>> 0; + for (let i = 0; i < key.length; i += 1) { + hash ^= key.charCodeAt(i); + hash = Math.imul(hash, 16777619) >>> 0; + } + return hash >>> 0; + } + + /** + * The vehicles the detection overlay may box and label this frame. + * + * Detection PULLS this on every paint, not only on label solves, so it must + * not be a fleet scan: the answer is CACHED and rebuilt only when something + * that could change it has happened — a poll, a visibility sweep, a + * removal, a selection — which the rest of the layer signals by bumping + * `state._detectRevision`. Between those, every paint gets the same array + * back. Positions inside it are the markers' own Cartesians, which the frame + * pass writes in place, so they are live without a copy; only the motion + * word is re-read per paint, and only for the entries actually handed over. + * + * Two older rules stand. Visibility is applied FIRST: sampling the whole + * fleet and then discarding the hidden ones meant a single visible bus in a + * fleet of five thousand could fall outside the sample and go unlabelled + * while the overlay had thousands of slots free. And selection among the + * visible is by hash, not by position, so the labelled set is stable as + * vehicles come and go. The selected vehicle is handed over with + * `skipLabel` — it has its own fuller card. + * + * @param {{maxCount?: number, seed?: number}} [options] + * @returns {Array} + */ + function collectDetectableVehicles(options = defaults) { + if (!state._enabled || !state._markers?.show) return empty; + if (state._vehicles.size === 0) return empty; + const maxCount = Number.isFinite(options.maxCount) + ? Math.max(1, Math.floor(options.maxCount)) + : Infinity; + const seed = Number.isFinite(options.seed) ? Math.floor(options.seed) : 0; + + requestedMax = maxCount; + requestedSeed = seed; + if (!state._detectCache) refreshDetectCache(); + return state._detectCache?.contacts || empty; + } + function refreshDetectCache() { + if (!state._enabled) return; + const cache = state._detectCache; + const now = Date.now(); + const maxCount = requestedMax, + seed = requestedSeed; + if (now - state._detectBuiltAt < 250) return; + state._detectBuiltAt = now; + if ( + cache && + cache.revision === state._detectRevision && + cache.maxCount === maxCount && + cache.seed === seed + ) { + for (const contact of cache.contacts) + contact.metric = transitDetectionMetric( + contact._entry, + contact._entry.mode, + now, + ); + return; + } + if (cache) + for (const contact of cache.contacts) contact._entry.detectContact = null; + const visible = []; + for (const entry of state._visible) { + const marker = entry.marker; + if (!marker?.position || marker.show === false) continue; + visible.push(entry); + } + let chosen = visible; + if (visible.length > maxCount) { + // Take the lowest-ranked keys. Removing an unselected vehicle cannot + // change this set at all, and removing a selected one promotes exactly + // one replacement instead of reshuffling everybody. + chosen = visible + .map((entry) => ({ entry, rank: selectionRank(entry.key, seed) })) + .sort((a, b) => a.rank - b.rank || (a.entry.key < b.entry.key ? -1 : 1)) + .slice(0, maxCount) + .map((scored) => scored.entry); + } + const contacts = []; + for (const entry of chosen) { + // Route and operator text change with the record and the mode, both of + // which arrive on polls — the same events that rebuild this list. + if (!entry.labelId || entry.labelRecord !== entry.record) { + entry.labelId = transitDetectionId(entry.record); + entry.labelRecord = entry.record; + } + if (!entry.labelClass || entry.labelMode !== entry.mode) { + const feed = + state._activeFeeds.get(entry.feedId) || + getRegisteredTransitFeed(entry.feedId); + entry.labelClass = transitDetectionClass(entry.mode, feed); + entry.labelMode = entry.mode; + } + const contact = { + position: entry.marker.position, + bracketHalfWidth: Math.ceil(entry.marker.width / 2) + 2, + bracketHalfHeight: Math.ceil(entry.marker.height / 2) + 2, + sourceId: entry.key, + id: entry.labelId, + type: 'VEH', + klass: entry.labelClass, + metric: transitDetectionMetric(entry, entry.mode, now), + // The bracket carries the mode colour in every preset: the detection + // canvas sits above the post-FX chain, where the sprites' own colour + // is lost under NVG and FLIR. Keyless too — the feeds are keyless. + tier: transitModeTier(entry.mode), + skipLabel: entry.key === state._selectedKey, + _entry: entry, + }; + entry.detectContact = contact; + contacts.push(contact); + } + state._detectCache = { + revision: state._detectRevision, + maxCount, + seed, + contacts, + }; + return contacts; + } + + const methods = { + /** + * Detection contract: buses, trams and trains are contacts like any other. + * @param {{maxCount?: number, seed?: number}} [options] + * @returns {Array} + */ + getDetectableObjects(options = defaults) { + return collectDetectableVehicles(options); + }, + + getStats() { + const active = [...state._activeFeeds.values()]; + const statuses = active.map((feed) => + parts.ingestion.feedStatus(feed.id), + ); + const now = Date.now(); + const health = aggregateTransitFeedHealth(statuses, now); + const count = state._vehicles.size; + if (state._enabled && active.length === 0) { + return { + count: 0, + lastUpdate: state._lastUpdate, + error: null, + source: 'GTFS-RT', + status: 'zoom-in', + coverage: state._altitudeGateOpen + ? `No feed here yet · ${TRANSIT_ENABLED_FEEDS.length} regions available` + : `Fly below ${Math.round(ACTIVATION_ALTITUDE_M / 1000).toLocaleString()} km to a covered region`, + }; + } + // Age is shown by the same rule the row state uses, so the text and the + // chip can never disagree about which feed has gone quiet. + const coverage = active + .map((feed) => { + const status = parts.ingestion.feedStatus(feed.id); + const quiet = + status.stale === true || + (Number.isFinite(status.upstreamAt) && + now - status.upstreamAt > FEED_STALE_AFTER_MS); + const aged = + quiet && Number.isFinite(status.upstreamAt) + ? ` (${Math.round((now - status.upstreamAt) / 1000)}s old)` + : ''; + return `${feed.name} ${status.count}${aged}`; + }) + .join(' · '); + return { + count, + lastUpdate: state._lastUpdate, + error: count === 0 ? health.error : null, + ...(count === 0 && Number.isFinite(health.retryInSec) + ? { retryInSec: health.retryInSec } + : {}), + degraded: count > 0 && health.degraded, + loading: health.loading && count === 0, + loadingLabel: + health.loading && count === 0 + ? `Loading ${active.map((feed) => feed.name).join(', ')}` + : undefined, + stale: health.stale, + source: 'GTFS-RT', + coverage, + feeds: active.map((feed) => feed.id), + }; + }, + }; + + return { methods, refreshDetectCache }; +} diff --git a/src/layers/transit/rendering.js b/src/layers/transit/rendering.js new file mode 100644 index 000000000..8fb7a954d --- /dev/null +++ b/src/layers/transit/rendering.js @@ -0,0 +1,598 @@ +import * as Cesium from 'cesium'; +import { perspectiveProjectedRotation } from '../../data/iconOrientation.js'; +import { + updatePlayback, + applyDisplayCourse, + syncPlayback, +} from './movement.js'; +import { + SELECTED_ICON_PX, + haloFrame, + transitIcon, +} from '../../data/transitIcons.js'; +import { + presetSpriteOutlinePx, + presetSpriteRgba, + presetSpriteScale, + transitStyleProfile, +} from '../../data/transitPresetStyle.js'; +import { + MARKER_PIXEL_SIZE, + ROTATION_REFRESH_MS, + SELECTED_CARD_REFRESH_MS, + VISIBILITY_REFRESH_MS, + SELECTED_PIXEL_SIZE, + TRANSIT_MODE_COLORS, + vehicleNearView, +} from './policy.js'; + +export const MODE_CESIUM_COLORS = Object.fromEntries( + Object.entries(TRANSIT_MODE_COLORS).map(([mode, css]) => [ + mode, + Cesium.Color.fromCssColorString(css), + ]), +); + +/** + * Billboards and the per-frame glide. + * @param {object} context + * @returns {object} + */ +export function createRendering({ state, services, parts }) { + const { + governorRequestRender, + holdContinuousRender, + releaseContinuousRender, + } = services.render; + + /** + * Cartesian position for a lat/lon at a height. Called on POLL, not per + * frame: `fromDegrees` runs trigonometry, and doing that for every vehicle on + * every frame was most of what made this layer heavy. + */ + function cartesianFor(lat, lon, heightM, out) { + // `heightM` already carries the shared floor's own lift, so nothing is + // added here: two lifts stacked is a vehicle hovering. + return Cesium.Cartesian3.fromDegrees( + lon, + lat, + heightM || 0, + undefined, + out, + ); + } + + /** Heights are aligned to work with Google 3D tiles. Rebuild both ends. */ + function refreshEndpoints(entry) { + if (!entry.sample || entry.sample.fromSeq < 0) return; + entry.from = entry.segment.from; + entry.to = entry.segment.to; + entry.fromCart = cartesianFor( + entry.from.lat, + entry.from.lon, + entry.from.h, + entry.fromCart || new Cesium.Cartesian3(), + ); + entry.toCart = cartesianFor( + entry.to.lat, + entry.to.lon, + entry.to.h, + entry.toCart || new Cesium.Cartesian3(), + ); + entry.endpointRevision = entry.sample.revision; + entry.endpointFromSeq = entry.sample.fromSeq; + entry.endpointToSeq = entry.sample.toSeq; + } + function placeSample(entry, nearGround = parts.height.nearGround()) { + if ( + entry.endpointRevision !== entry.sample.revision || + entry.endpointFromSeq !== entry.sample.fromSeq || + entry.endpointToSeq !== entry.sample.toSeq + ) + refreshEndpoints(entry); + const known = parts.trails.samplePosition( + entry, + entry.sample, + state._scratchCartesian, + nearGround, + ); + entry.surfaceReady = !!known; + entry.marker.show = !!known && vehicleInView(entry); + if (known) { + entry.marker.position = state._scratchCartesian; + entry.hasRendered = true; + } + } + function sampleIdle(entry) { + if (!state._moving.has(entry)) { + updatePlayback(entry, Date.now(), performance.now()); + placeSample(entry); + } + } + + function cancelWake(entry) { + if (entry.wakeTimer != null) clearTimeout(entry.wakeTimer); + entry.wakeTimer = null; + } + function migrateMarker(entry, animated) { + const collection = animated ? state._animatedMarkers : state._markers; + if (!entry.marker || entry.markerCollection === collection) return; + const old = entry.marker; + const marker = collection.add({ + id: entry.key, + position: old.position, + image: old.image, + width: old.width, + height: old.height, + color: old.color, + scaleByDistance: old.scaleByDistance, + rotation: old.rotation, + alignedAxis: old.alignedAxis, + show: old.show, + disableDepthTestDistance: old.disableDepthTestDistance, + }); + entry.markerCollection.remove(old); + entry.marker = marker; + entry.markerCollection = collection; + entry.markerRevision = (entry.markerRevision || 0) + 1; + // Cached contacts and card getters must follow the replacement billboard. + if (entry.detectContact) entry.detectContact.position = marker.position; + } + function schedulePlayback(entry) { + cancelWake(entry); + if (!entry.marker || !vehicleInView(entry)) { + state._moving.delete(entry); + migrateMarker(entry, false); + return; + } + const sample = entry.sample; + if (!sample || entry.heightPending || entry.surfaceReady === false) { + state._moving.delete(entry); + migrateMarker(entry, false); + return; + } + const positional = sample.phase === 'playing' && sample.segmentSpeedMps > 0; + const turning = + Number.isFinite(sample.segmentCourseDeg) && + Number.isFinite(entry.courseDeg) && + Math.abs( + ((sample.segmentCourseDeg - entry.courseDeg + 540) % 360) - 180, + ) > 0.01; + if (positional || turning) { + state._moving.add(entry); + migrateMarker(entry, true); + } else { + state._moving.delete(entry); + migrateMarker(entry, false); + if (Number.isFinite(sample.nextWakeMonoMs)) { + entry.wakeTimer = setTimeout( + () => { + entry.wakeTimer = null; + if (!state._enabled || !state._vehicles.has(entry.key)) return; + updatePlayback(entry, Date.now(), performance.now()); + placeSample(entry); + schedulePlayback(entry); + syncRenderHold(); + governorRequestRender('transit-motion-boundary'); + }, + Math.max(1, sample.nextWakeMonoMs - performance.now()), + ); + } + } + } + + /** + * Whether either end of the segment being drawn is near the view — the + * test visibility, animation and rotation all share. + * @param {object} entry + * @returns {boolean} + */ + function vehicleInView(entry) { + return state._visible.has(entry); + } + + /** Cesium colour for a preset tuple, cached per profile+mode. */ + const _presetColors = new Map(); + function presetColor(style, mode) { + const rgba = presetSpriteRgba(style, mode); + if (!rgba) return null; + const key = `${transitStyleProfile(style)}:${mode}`; + let color = _presetColors.get(key); + if (!color) { + color = new Cesium.Color( + rgba[0] / 255, + rgba[1] / 255, + rgba[2] / 255, + rgba[3], + ); + _presetColors.set(key, color); + } + return color; + } + + /** + * The ONE place a sprite's look is decided: creation, a mode change, select, + * deselect and a preset change all come here, so no path can undo another. + * Under a mono preset (NVG, FLIR, noir) the sprite is white-hot with a dark + * halo sized in screen pixels; under CRT a saturated mode colour with a + * thinner halo; otherwise the shipped mode colour and hairline. Selection + * grows the glyph and retains the mode colour; sensors use opaque white. + * @param {object} entry + */ + function applySpriteStyle(entry) { + const marker = entry.marker; + if (!marker) return; + const style = state._stylePreset; + const selected = state._selectedKey === entry.key; + const basePx = selected ? SELECTED_PIXEL_SIZE : MARKER_PIXEL_SIZE; + const halo = presetSpriteOutlinePx(style, selected); + const displayPx = basePx * presetSpriteScale(style, selected); + const frame = haloFrame(halo, displayPx); + const px = displayPx * frame.ratio; + marker.image = transitIcon( + entry.mode, + selected ? SELECTED_ICON_PX : undefined, + { + style, + }, + ); + marker.width = px; + marker.height = px; + if (entry.detectContact) { + entry.detectContact.bracketHalfWidth = Math.ceil(px / 2) + 2; + entry.detectContact.bracketHalfHeight = Math.ceil(px / 2) + 2; + } + const preset = presetColor(style, entry.mode); + if (preset) marker.color = preset; + else + marker.color = + MODE_CESIUM_COLORS[entry.mode] || MODE_CESIUM_COLORS.unknown; + } + + function addMarker(key, entry) { + const marker = state._markers.add({ + id: key, + position: entry.toCart, + image: transitIcon(entry.mode), + width: MARKER_PIXEL_SIZE, + height: MARKER_PIXEL_SIZE, + color: MODE_CESIUM_COLORS[entry.mode], + rotation: 0, + alignedAxis: Cesium.Cartesian3.ZERO, + disableDepthTestDistance: Number.POSITIVE_INFINITY, + }); + entry.marker = marker; + entry.markerCollection = state._markers; + marker.show = false; + applySpriteStyle(entry); + return marker; + } + + function paintMode(entry, mode) { + entry.mode = mode; + if (entry.key === state._selectedKey) { + parts.trails.style(); + parts.selection.refreshSelectedCard(true); + } + state._detectRevision += 1; + applySpriteStyle(entry); + } + + /** + * Adopt a post-FX style. One pass over the fleet, on the event, never per + * frame; a style already applied is a no-op. + * @param {string|null|undefined} styleName + */ + function setStylePreset(styleName) { + const next = styleName || 'normal'; + if (next === state._stylePreset) return; + state._stylePreset = next; + for (const entry of state._vehicles.values()) applySpriteStyle(entry); + parts.trails.style(); + governorRequestRender('transit-style'); + } + + /** + * Point every glyph along its reported course. + * + * Billboards are camera-facing quads, so a course has to be projected into + * screen space — which depends on where the camera is, not only on the + * vehicle. Doing that for the whole fleet on every frame is exactly the kind + * of cost that made this layer heavy, and it buys nothing: a bus icon whose + * rotation is a fifth of a second stale is indistinguishable. So the pass is + * throttled, and runs only when the camera has actually moved or a poll + * changed somebody's heading. + */ + function refreshRotations(now) { + const scene = state._viewer?.scene; + if (!scene?.camera) return; + if (now - state._rotationAt < ROTATION_REFRESH_MS) return; + if ( + state._rotationRevision === state._cameraRevision && + !state._rotationDirty && + state._moving.size === 0 + ) + return; + state._rotationAt = now; + const cameraChanged = state._rotationRevision !== state._cameraRevision; + state._rotationRevision = state._cameraRevision; + state._rotationDirty = false; + for (const entry of state._visible) { + if (entry.key === state._selectedKey) continue; + const course = entry.courseDeg; + if (course === null) continue; + const position = entry.marker.position; + if ( + !cameraChanged && + entry.rotationCourse === course && + Math.abs(position.x - entry.rotationX) + + Math.abs(position.y - entry.rotationY) + + Math.abs(position.z - entry.rotationZ) < + 10 + ) + continue; + entry.rotationCourse = course; + entry.rotationX = position.x; + entry.rotationY = position.y; + entry.rotationZ = position.z; + // Exact projection, perspective division included. The shared + // camera-basis helper aircraft use is orthographic — fine at altitude, + // and wrong by twenty-odd degrees for a street-level contact away from + // the centre of an obliquely pitched view, which is most of them. + entry.marker.rotation = perspectiveProjectedRotation( + scene, + entry.marker.position, + course, + entry.marker.rotation, + ); + } + } + + /** Publish final motion wording when the selected contact settles. */ + function settleSelectedCard(entry) { + if (state._selectedKey && state._selectedKey === entry.key) { + parts.selection.refreshSelectedCard(true); + } + } + + /** Hold only for visible position or orientation animation. */ + function syncRenderHold() { + // Held for motion only. Ground resolution runs on its own timer off the + // render path, so a fleet waiting for floors is not a fleet worth burning + // frames on — it is not moving. + const working = state._enabled && state._moving.size > 0; + if (working && !state._renderHeld) { + holdContinuousRender('transit'); + state._renderHeld = true; + } else if (!working && state._renderHeld) { + releaseContinuousRender('transit'); + state._renderHeld = false; + governorRequestRender('transit-settled'); + } + } + + function onPreRender() { + if ( + typeof document !== 'undefined' && + document.visibilityState === 'hidden' + ) + return; + if (!state._enabled) return; + const now = Date.now(); + if (state._vehicles.size === 0) { + state._moving.clear(); + syncRenderHold(); + return; + } + + const monoNow = performance.now(); + const nearGround = parts.height.nearGround(); + let dirtyCount = 0; + for (const entry of state._heightDirty) { + if (dirtyCount++ >= 64) break; + state._heightDirty.delete(entry); + if (!entry.marker) continue; + updatePlayback(entry, now, monoNow); + placeSample(entry, nearGround); + schedulePlayback(entry); + } + for (const entry of state._moving) { + if (!entry.marker || !vehicleInView(entry)) { + state._moving.delete(entry); + continue; + } + updatePlayback(entry, now, monoNow); + placeSample(entry, nearGround); + if (applyDisplayCourse(entry, monoNow)) state._rotationDirty = true; + if ( + !entry.surfaceReady || + entry.sample.phase !== 'playing' || + entry.sample.segmentSpeedMps === 0 + ) + schedulePlayback(entry); + if (!state._moving.has(entry)) settleSelectedCard(entry); + } + + parts.trails.update(); + refreshRotations(now); + const selected = state._vehicles.get(state._selectedKey); + if ( + selected && + state._visible.has(selected) && + selected.courseDeg !== null + ) { + selected.marker.rotation = perspectiveProjectedRotation( + state._viewer.scene, + selected.marker.position, + selected.courseDeg, + selected.marker.rotation, + ); + } + if (state._heightDirty.size) governorRequestRender('transit-height-batch'); + syncRenderHold(); + } + + const occluder = new Cesium.EllipsoidalOccluder(Cesium.Ellipsoid.WGS84); + const sphere = new Cesium.BoundingSphere(undefined, 5); + const cameraPosition = new Cesium.Cartesian3(); + const candidatePosition = new Cesium.Cartesian3(); + /** Visibility work runs off the frame path, coalesced at four sweeps/second. */ + function requestVisibility() { + state._visibilityDirty = true; + if (!state._enabled || state._visibilityTimer !== null) return; + state._visibilityTimer = setTimeout( + () => { + state._visibilityTimer = null; + refreshVisibility(); + }, + Math.max(0, VISIBILITY_REFRESH_MS - (Date.now() - state._visibilityAt)), + ); + } + function refreshVisibility() { + if (!state._enabled) return 0; + const now = Date.now(); + if (now - state._visibilityAt < VISIBILITY_REFRESH_MS) { + requestVisibility(); + return state._shownCount; + } + state._visibilityAt = now; + state._visibilityDirty = false; + const viewer = state._viewer, + camera = viewer?.camera; + if (!camera) return 0; + const position = + camera.positionWC || + Cesium.Ellipsoid.WGS84.cartographicToCartesian( + camera.positionCartographic, + cameraPosition, + ); + occluder.cameraPosition = position; + const frustum = camera.frustum?.computeCullingVolume( + position, + camera.directionWC, + camera.upWC, + ); + const planes = frustum?.planes; + let changed = false; + for (const entry of state._vehicles.values()) { + if (!entry.marker) continue; + // Refresh hidden clocks before testing admission, even if no frame has + // ever sampled them. Do not write the billboard until it is admitted. + const noSample = !entry.sample || !Number.isFinite(entry.sample.lat); + if (entry.track?.count && (!state._visible.has(entry) || noSample)) { + entry.sample ||= {}; + updatePlayback(entry, now, performance.now()); + } + sphere.center = + state._visible.has(entry) || !entry.sample + ? entry.marker.position + : cartesianFor( + entry.sample.lat, + entry.sample.lon, + entry.heightM, + candidatePosition, + ); + // A settled geographic rectangle can lag setView or miss an oblique + // view. The current frustum is authoritative; the horizon still rejects + // the far side when the rectangle is missing or stale. + const bounds = vehicleNearView(state._viewBounds, entry); + const horizon = occluder.isPointVisible(sphere.center); + let inFrustum = true; + if (planes) { + for (let i = 0; i < planes.length; i++) { + const p = planes[i], + c = sphere.center; + if (p.x * c.x + p.y * c.y + p.z * c.z + p.w < -sphere.radius) { + inFrustum = false; + break; + } + } + } else if (frustum) + inFrustum = + frustum.computeVisibility(sphere) !== Cesium.Intersect.OUTSIDE; + const visible = (frustum || bounds) && horizon && inFrustum; + const visibility = (entry.visibility ||= {}); + visibility.bounds = bounds; + visibility.boundsApplied = !frustum; + visibility.occluder = horizon; + visibility.frustum = inFrustum; + visibility.noSample = noSample; + const wasShown = entry.marker.show; + const wasVisible = state._visible.has(entry); + if (visible) state._visible.add(entry); + else state._visible.delete(entry); + if (visible && entry.track?.count) { + if (!wasVisible && !entry.qaFixture) syncPlayback(entry, now); + // Surface completion must recover while requestRenderMode is idle. + // Otherwise surfaceReady=false prevents the very frame that clears it. + if (!state._moving.has(entry)) { + updatePlayback(entry, now, performance.now()); + placeSample(entry); + } + } + entry.marker.show = + visible && !entry.heightPending && entry.surfaceReady !== false; + schedulePlayback(entry); + changed ||= visible !== wasVisible || wasShown !== entry.marker.show; + } + state._shownCount = state._visible.size; + if (changed) state._detectRevision++; + syncRenderHold(); + governorRequestRender('transit-visibility'); + return state._shownCount; + } + function resumePlayback() { + if (!state._enabled) return; + const now = Date.now(), + mono = performance.now(); + const entries = new Set(state._visible); + const selected = state._vehicles.get(state._selectedKey); + if (selected) entries.add(selected); + for (const entry of entries) { + if (!entry.track?.count) continue; + syncPlayback(entry, now, mono); + parts.trails.prepareEntry(entry); + placeSample(entry); + schedulePlayback(entry); + } + parts.trails.update(); + parts.selection.refreshSelectedCard(true); + state._detectRevision++; + syncRenderHold(); + governorRequestRender('transit-re-entry'); + } + // Text and membership refreshes are bounded timer work, never paint work. + function maintainPresentation() { + if (!state._enabled) return; + if (state._moving.size) requestVisibility(); + parts.queries.refreshDetectCache(); + parts.selection.refreshSelectedCard(false); + } + + /** Grow the selected glyph (or shrink it back) through the one styling path. */ + function paintSelected(entry, selected) { + void selected; // decided from state._selectedKey, which the caller has set + applySpriteStyle(entry); + } + + return { + resumePlayback, + maintainPresentation, + requestVisibility, + sampleIdle, + cancelWake, + schedulePlayback, + placeSample, + cartesianFor, + refreshEndpoints, + refreshVisibility, + vehicleInView, + addMarker, + paintMode, + paintSelected, + applySpriteStyle, + setStylePreset, + onPreRender, + syncRenderHold, + }; +} diff --git a/src/layers/transit/selection.js b/src/layers/transit/selection.js new file mode 100644 index 000000000..cd7d33968 --- /dev/null +++ b/src/layers/transit/selection.js @@ -0,0 +1,209 @@ +import * as Cesium from 'cesium'; +import { + SELECTED_CARD_REFRESH_MS, + TRANSIT_SELECTED_OVERLAY_SOURCE_ID, + TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS, + buildTransitSelectionCopy, + createTransitSelectedOverlayEntry, +} from './policy.js'; +import { getRegisteredTransitFeed } from '../../data/transitFeeds.js'; +import { isPointerFree } from '../../data/inputOwnership.js'; + +/** + * Click-to-select one vehicle and a card that follows it. + * @param {object} context + * @returns {object} + */ +export function createSelection({ state, services, parts }) { + const { governorRequestRender } = services.render; + + /** + * The card's anchor: the selected marker's CURRENT position, read by the + * overlay host on every paint. A getter, not a clone — the host accepts one + * (the tracked-aircraft readout uses the same), so the anchor moves with the + * sprite at frame rate, through poll arrivals, settling, a late floor and a + * reset alike, with no per-frame work here. Null when there is nothing to + * anchor to: a hidden or removed marker draws no card. + * @param {object} entry + * @returns {() => Cesium.Cartesian3|null} + */ + function anchorFor(entry) { + return () => { + const marker = entry.marker; + const p = marker?.show !== false ? marker?.position : null; + return p && + Number.isFinite(p.x) && + Number.isFinite(p.y) && + Number.isFinite(p.z) + ? p + : null; + }; + } + + function refreshSelectedCard(force) { + const entry = state._selectedKey + ? state._vehicles.get(state._selectedKey) + : null; + if (!entry) return; + const now = Date.now(); + if (!force && now - state._selectedCardAt < SELECTED_CARD_REFRESH_MS) + return; + state._selectedCardAt = now; + const feed = + state._activeFeeds.get(entry.feedId) || + getRegisteredTransitFeed(entry.feedId); + if (!feed) return; + const copy = buildTransitSelectionCopy( + feed, + entry.record, + entry.mode, + now, + entry.fetchedAt, + entry, + ); + // The TEXT is what the throttle is for, and it is republished only when + // it has changed: the anchor is live through its getter, so re-sending + // an identical card would only make the host re-solve its layout. + const text = `${entry.key}\u0000${copy.title}\u0000${copy.details.join('\u0000')}`; + if (!force && text === state._selectedCardText) return; + const card = createTransitSelectedOverlayEntry( + entry.key, + anchorFor(entry), + copy, + entry.mode, + ); + if (card) { + state._selectedCardText = text; + state._overlayHost.setEntries( + TRANSIT_SELECTED_OVERLAY_SOURCE_ID, + [card], + TRANSIT_SELECTED_OVERLAY_SOURCE_OPTIONS, + ); + state._cardPublications = (state._cardPublications || 0) + 1; + } + } + + function clearSelection() { + const entry = state._selectedKey + ? state._vehicles.get(state._selectedKey) + : null; + parts.trails.clear(); + state._selectedKey = null; + state._selectedCardText = null; + state._detectRevision += 1; + // Cleared first: the styling path reads the selection from state. + if (entry) parts.rendering.paintSelected(entry, false); + state._overlayHost.clearSource(TRANSIT_SELECTED_OVERLAY_SOURCE_ID); + } + + function selectVehicle(key) { + clearSelection(); + const entry = state._vehicles.get(key); + if (!entry?.marker) return; + state._selectedKey = key; + state._detectRevision += 1; + parts.rendering.paintSelected(entry, true); + parts.trails.select(entry); + refreshSelectedCard(true); + governorRequestRender('transit-select'); + } + + function onKeyDown(event) { + if (event.key === 'Escape' && state._selectedKey) clearSelection(); + } + + function installClickHandler(viewer) { + if (state._clickHandler) return; + state._clickHandler = new Cesium.ScreenSpaceEventHandler( + viewer.scene.canvas, + ); + state._clickHandler.setInputAction((click) => { + // Ambient selection never competes for a pointer a tool has claimed: + // placing a draw vertex on top of a bus must not also select the bus. + if (!isPointerFree()) return; + const scene = viewer.scene; + // Pick buffers precede post-processing. Map a sensor pixel back through + // the active shader's barrel distortion and quantization before picking. + const position = Cesium.Cartesian2.clone(click.position); + const style = state._stylePreset; + const stage = scene.postProcessStages?.getStageByName?.( + `godsEyeView_${style}`, + ); + if (stage?.enabled && (style === 'thermal' || style === 'surveillance')) { + const intensity = stage.uniforms.intensity; + const canvas = scene.canvas, + w = canvas.clientWidth, + h = canvas.clientHeight; + let u = position.x / w, + v = 1 - position.y / h; + if (style === 'surveillance') { + const x = u * 2 - 1, + y = v * 2 - 1, + r2 = x * x + y * y; + const d = 1 + r2 * intensity * 0.25 + r2 * r2 * intensity * 0.075; + u = (x * d + 1) / 2; + v = (y * d + 1) / 2; + } + const grid = 1 + (stage.uniforms.pixelation - 1) * intensity; + u += + ((Math.floor((u * canvas.width) / grid) * grid) / canvas.width - u) * + intensity; + v += + ((Math.floor((v * canvas.height) / grid) * grid) / canvas.height - + v) * + intensity; + position.x = u * w; + position.y = (1 - v) * h; + } + const picked = scene.pick(position); + state._lastPickForTest = { + id: typeof picked?.id === 'string' ? picked.id : null, + primitiveId: + typeof picked?.primitive?.id === 'string' + ? picked.primitive.id + : null, + x: position.x, + y: position.y, + collection: + picked?.primitive?._billboardCollection === state._animatedMarkers + ? 'animated' + : picked?.primitive?._billboardCollection === state._markers + ? 'stationary' + : null, + }; + if (picked) { + const primitiveId = picked.primitive?.id; + if ( + typeof primitiveId === 'string' && + state._vehicles.has(primitiveId) + ) { + selectVehicle(primitiveId); + return; + } + if (typeof picked.id === 'string' && state._vehicles.has(picked.id)) { + selectVehicle(picked.id); + return; + } + } + if (state._selectedKey) clearSelection(); + }, Cesium.ScreenSpaceEventType.LEFT_CLICK); + document.addEventListener('keydown', onKeyDown); + } + + function removeClickHandler() { + if (state._clickHandler) { + state._clickHandler.destroy(); + state._clickHandler = null; + } + document.removeEventListener('keydown', onKeyDown); + } + + return { + refreshSelectedCard, + clearSelection, + selectVehicle, + onKeyDown, + installClickHandler, + removeClickHandler, + }; +} diff --git a/src/layers/transit/state.js b/src/layers/transit/state.js new file mode 100644 index 000000000..75b13d84b --- /dev/null +++ b/src/layers/transit/state.js @@ -0,0 +1,126 @@ +import * as Cesium from 'cesium'; +import { createHistoryBudget } from '../../data/contactPlayback.js'; + +/** + * One Transit layer's mutable scene state. Every field lives on the instance, + * so two layers built from this factory never share a viewer, a primitive + * collection, a request map or a height cache. + * @param {{services: object}} context + * @returns {object} + */ +export function createState({ services }) { + const { setOverlayEntries, setOverlaySourceVisible, clearOverlaySource } = + services.overlays; + + const state = {}; + state._historyBudget = createHistoryBudget(); + + state.DEFAULT_OVERLAY_HOST = Object.freeze({ + setEntries: setOverlayEntries, + setVisible: setOverlaySourceVisible, + clearSource: clearOverlaySource, + }); + state._overlayHost = state.DEFAULT_OVERLAY_HOST; + + /** @type {Cesium.Viewer|null} */ + state._viewer = null; + /** @type {Cesium.BillboardCollection|null} */ + state._markers = null; + state._animatedMarkers = null; + state._visible = new Set(); + state._visibilityTimer = null; + state._maintenanceTimer = null; + state._detectBuiltAt = -Infinity; + state._cameraRevision = 0; + state._rotationRevision = -1; + state._enabled = false; + state._generation = 0; + + state._cameraChangedAttached = false; + state._cameraDebounceTimer = null; + state._altitudeGateOpen = false; + /** Camera sensitivity as we found it, so disable() can hand it back. */ + state._priorPercentageChanged = null; + /** The value this layer actually wrote, for an identity-guarded restore. */ + state._appliedPercentageChanged = null; + + /** @type {(() => void)|null} */ + state._preRenderRemove = null; + state._renderHeld = false; + /** @type {Cesium.ScreenSpaceEventHandler|null} */ + state._clickHandler = null; + /** + * The vehicles currently mid-glide. The per-frame pass walks THIS, not the + * whole fleet, and the render hold is derived from it together with the + * height queue — so both track work that exists rather than a window of time + * the layer hoped would contain some. + * @type {Set} + */ + state._moving = new Set(); + /** Vehicles whose sampled height landed after they were drawn. @type {Set} */ + state._heightDirty = new Set(); + /** + * Inflated camera view bounds in degrees, or null before the first camera + * read. Decides which vehicles are worth animating frame by frame. + * @type {{south:number,north:number,west:number,east:number}|null} + */ + state._viewBounds = null; + /** How many vehicles are currently drawn, as opposed to held. */ + state._shownCount = 0; + /** Deferred shown/hidden sweep bookkeeping. */ + state._visibilityDirty = true; + state._visibilityAt = 0; + /** Throttle bookkeeping for the screen-space rotation pass. */ + state._rotationAt = 0; + state._rotationPose = null; + state._rotationDirty = false; + + /** @type {Map} feedId → registry entry currently polled */ + state._activeFeeds = new Map(); + /** @type {Map} */ + state._feedStatus = new Map(); + /** @type {Map}>} */ + state._inFlight = new Map(); + /** @type {Map} vehicle key → runtime entry */ + state._vehicles = new Map(); + + /** Timer for the floor re-read cycle, and how many it has spent. */ + state._floorTimer = null; + state._floorAttempts = 0; + /** Where the next warm window starts among the cold cells: no cell waits for ever. */ + state._floorCursor = 0; + + /** @type {string|null} */ + state._selectedKey = null; + state._selectedCardAt = 0; + /** The card text last published, so an unchanged card is not re-sent. */ + state._selectedCardText = null; + /** + * Bumped whenever the set of vehicles DETECT may see could have changed — + * a poll, a visibility sweep, a removal, a selection — so the candidate + * list is rebuilt on change rather than on every paint. + */ + state._detectRevision = 0; + /** The candidate list DETECT was last handed, and the revision it was built at. */ + state._detectCache = null; + /** + * Active post-FX style the sprites are drawn for. Read from the document + * at init and followed through the style and cockpit-vision events; a + * cockpit override wins over the map style while it lasts. + */ + state._stylePreset = 'normal'; + state._cockpitVision = false; + /** @type {Array<[string, EventListener]>} window listeners to remove on destroy. */ + state._styleListeners = []; + /** @type {{ refreshLayerStats?: () => void }|null} */ + state._dataManager = null; + /** @type {number|null} */ + state._lastUpdate = null; + /** @type {string|null} */ + state._error = null; + state._limitWarned = false; + + state._scratchCartesian = new Cesium.Cartesian3(); + + return state; +} diff --git a/src/layers/transit/testing.js b/src/layers/transit/testing.js new file mode 100644 index 000000000..06bd315a1 --- /dev/null +++ b/src/layers/transit/testing.js @@ -0,0 +1,340 @@ +import * as Cesium from 'cesium'; +import { recordFix, updatePlayback, attachFloorToPlace } from './movement.js'; +import { seek, setRate, mergeHistory } from '../../data/contactPlayback.js'; +import { getRegisteredTransitFeed } from '../../data/transitFeeds.js'; +/** + * Test seams. Each one reaches into THIS instance's state, so a test that + * swaps the overlay host or inspects vehicles cannot leak into another layer. + * @param {object} context + * @returns {object} + */ +import { transitIconCacheSize } from '../../data/transitIcons.js'; + +export function createTesting({ state, parts }) { + let stopProbe = null; + return { + _setTransitFixtureFloorsForTest(keys, heightM) { + state._qaFixtureFloors = new Map(keys.map((key) => [key, heightM])); + }, + _stopTransitProbeForTest() { + stopProbe?.(); + }, + /** Deterministic observed-position fixtures, independent of upstream availability. */ + _loadTransitFleetForTest(count, origin, spacingM = 6, routes = ['741']) { + parts.selection.clearSelection(); + parts.ingestion.abortAllInFlight(); + state._activeFeeds.clear(); + for (const key of state._vehicles.keys()) + parts.ingestion.removeVehicle(key); + clearTimeout(state._cameraDebounceTimer); + state._cameraDebounceTimer = null; + const feed = getRegisteredTransitFeed('mbta'); + const now = Date.now(), + mono = performance.now(); + const cols = Math.ceil(Math.sqrt(count)); + const vehicles = Array.from({ length: count }, (_, i) => ({ + id: `qa-${i}`, + routeId: routes[i % routes.length], + tripId: 'qa', + lat: + origin.lat + ((Math.floor(i / cols) - cols / 2) * spacingM) / 111320, + lon: + origin.lon + + (((i % cols) - cols / 2) * spacingM) / + (111320 * Math.cos((origin.lat * Math.PI) / 180)), + timestamp: (now - 120000) / 1000, + timestampSource: 'vehicle', + bearing: 0, + })); + parts.ingestion.applySnapshot( + feed, + { fetchedAt: now, vehicles }, + { stale: false }, + ); + let fixtureIndex = 0; + for (const entry of state._vehicles.values()) { + entry.qaIndex = fixtureIndex++; + entry.qaFixture = true; + entry.qaFloorM = 20; + for (let j = 1; j <= 8; j++) { + const t = now - 120000 + j * 15000; + recordFix( + entry, + { + t, + lat: entry.record.lat + (j * 15) / 111320, + lon: entry.record.lon, + bearingDeg: 0, + }, + { + receivedAt: t + 1000, + wallNowMs: t + 1000, + monoNowMs: mono - now + t + 1000, + }, + ); + } + mergeHistory( + entry.track, + Array.from({ length: 119 }, (_, i) => { + const t = now - 899000 + i * (764000 / 118); + return { + t, + lat: entry.record.lat + (t - now + 120000) / 1000 / 111320, + lon: entry.record.lon, + bearingDeg: 0, + epoch: entry.currentEpoch, + }; + }), + ); + // This fixture deliberately provides a flat, known surface for load measurements. + for (const fix of entry.fixes) attachFloorToPlace(entry, fix, 20); + entry.heightPending = false; + entry.surfaceReady = true; + entry.displayPaths = new Map(); + const fixes = entry.fixes; + for (let i = 0; i + 1 < fixes.length; i++) { + if (fixes[i + 1].t < now - 72000) continue; + entry.displayPaths.set(fixes[i].seq, { + toSeq: fixes[i + 1].seq, + positions: [ + Cesium.Cartesian3.fromDegrees(fixes[i].lon, fixes[i].lat, 20), + Cesium.Cartesian3.fromDegrees( + fixes[i + 1].lon, + fixes[i + 1].lat, + 20, + ), + ], + }); + } + entry.clocks.wallNowMs = now; + entry.clocks.monoNowMs = mono; + seek(entry.track, now - 72000, entry.clocks); + setRate(entry.track, 1, entry.clocks); + updatePlayback(entry, now, mono); + parts.rendering.placeSample(entry); + } + clearTimeout(state._floorTimer); + state._floorTimer = null; + // Establish the fixture clock without the live re-entry reset. + for (const entry of state._vehicles.values()) state._visible.add(entry); + state._visibilityAt = 0; + parts.rendering.refreshVisibility(); + parts.rendering.maintainPresentation(); + return { + count: state._vehicles.size, + moving: state._moving.size, + fixBytes: state._historyBudget.allocatedBytes, + }; + }, + _resetTransitFixtureClockForTest() { + const now = Date.now(), + mono = performance.now(); + for (const entry of state._vehicles.values()) { + // A backwards QA seek revisits paths pruned during the timing run. + // Rebuild the same explicitly flat fixture surface before allocation QA. + const fixes = entry.fixes; + entry.displayPaths = new Map(); + for (let i = 0; i + 1 < fixes.length; i++) { + if (fixes[i + 1].t < fixes.at(-1).t - 72000) continue; + entry.displayPaths.set(fixes[i].seq, { + toSeq: fixes[i + 1].seq, + positions: [ + Cesium.Cartesian3.fromDegrees(fixes[i].lon, fixes[i].lat, 20), + Cesium.Cartesian3.fromDegrees( + fixes[i + 1].lon, + fixes[i + 1].lat, + 20, + ), + ], + }); + } + entry.clocks.wallNowMs = now; + entry.clocks.monoNowMs = mono; + seek(entry.track, entry.fixes.at(-1).t - 72000, entry.clocks); + setRate(entry.track, 1, entry.clocks); + updatePlayback(entry, now, mono); + parts.rendering.placeSample(entry); + parts.rendering.schedulePlayback(entry); + } + parts.rendering.syncRenderHold(); + }, + /** Measure inside Cesium postRender with fixed typed storage; no sample objects per frame. */ + _measureTransitFramesForTest(durationMs = 12000) { + stopProbe?.(); + const scene = state._viewer.scene; + const capacity = 20000, + stride = 12, + data = new Float64Array(capacity * stride); + const previousPositions = new Float64Array(state._vehicles.size * 3); + const gl = scene.context._gl; + let active = false, + upload = 0, + cpu = 0, + count = 0, + previous = 0; + const saved = []; + for (const name of ['bufferData', 'bufferSubData']) { + const original = gl[name]; + gl[name] = function (...args) { + if (active) { + const value = args[name === 'bufferData' ? 1 : 2]; + const offset = args[name === 'bufferData' ? 3 : 3] || 0; + const length = args[4]; + upload += + typeof value === 'number' + ? value + : value + ? length !== undefined + ? length * (value.BYTES_PER_ELEMENT || 1) + : value.byteLength - offset * (value.BYTES_PER_ELEMENT || 1) + : 0; + } + return original.apply(this, args); + }; + saved.push(() => { + gl[name] = original; + }); + } + for (const collection of [state._markers, state._animatedMarkers]) { + const original = collection.update; + collection.update = function (frame) { + active = true; + try { + return original.call(this, frame); + } finally { + active = false; + } + }; + saved.push(() => { + collection.update = original; + }); + } + state._preRenderRemove?.(); + const removePre = scene.preRender.addEventListener(() => { + upload = 0; + const start = performance.now(); + parts.rendering.onPreRender(); + cpu = performance.now() - start; + }); + let removePost, timer; + return new Promise((resolve) => { + const finish = () => { + clearTimeout(timer); + removePre(); + removePost?.(); + for (const restore of saved) restore(); + if (state._enabled) + state._preRenderRemove = scene.preRender.addEventListener( + parts.rendering.onPreRender, + ); + stopProbe = null; + const rows = []; + for (let i = 1; i < count; i++) { + const k = i * stride; + rows.push({ + intervalMs: data[k], + cpuMs: data[k + 1], + uploadBytes: data[k + 2], + moving: data[k + 3], + x: data[k + 4], + y: data[k + 5], + z: data[k + 6], + displayT: data[k + 7], + expectedMps: data[k + 8], + segment: data[k + 9], + }); + } + resolve({ + rows, + dpr: devicePixelRatio, + width: scene.canvas.clientWidth, + height: scene.canvas.clientHeight, + }); + }; + stopProbe = finish; + removePost = scene.postRender.addEventListener(() => { + const now = performance.now(); + if (count >= capacity) return; + const k = count++ * stride; + const entry = state._vehicles.get('mbta:qa-0'); + data[k] = previous ? now - previous : 0; + previous = now; + data[k + 1] = cpu; + data[k + 2] = upload; + let actuallyMoved = 0; + for (const vehicle of state._visible) { + const p = vehicle.marker.position, + index = vehicle.qaIndex * 3; + if ( + count > 1 && + Math.hypot( + p.x - previousPositions[index], + p.y - previousPositions[index + 1], + p.z - previousPositions[index + 2], + ) > 0.00001 + ) + actuallyMoved++; + previousPositions[index] = p.x; + previousPositions[index + 1] = p.y; + previousPositions[index + 2] = p.z; + } + data[k + 3] = actuallyMoved; + if (entry?.marker) { + const p = entry.marker.position; + data[k + 4] = p.x; + data[k + 5] = p.y; + data[k + 6] = p.z; + data[k + 7] = entry.sample.displayT; + data[k + 8] = entry.sample.segmentSpeedMps; + data[k + 9] = entry.sample.fromSeq; + } + }); + timer = setTimeout(finish, durationMs); + }); + }, + _transitVisibilityForTest() { + return [...state._vehicles.values()].map((entry) => { + const gate = entry.visibility || {}; + const why = []; + if (!entry.marker) why.push('no marker'); + if (gate.bounds === false && gate.boundsApplied) why.push('bounds'); + if (gate.occluder === false) why.push('occluder'); + if (gate.frustum === false) why.push('frustum'); + if (entry.heightPending) why.push('heightPending'); + if (entry.surfaceReady === false) why.push('surfaceReady'); + if (!entry.sample || !Number.isFinite(entry.sample.lat)) + why.push('no sample'); + if (!state._enabled) why.push('disabled'); + return { + key: entry.key, + shown: entry.marker?.show === true, + why, + ...gate, + heightPending: !!entry.heightPending, + surfaceReady: entry.surfaceReady ?? null, + moving: state._moving.has(entry), + held: state._renderHeld, + heightM: entry.heightM, + phase: entry.sample?.phase, + }; + }); + }, + /** What the sprites are currently styled for, and how many rasters exist. */ + _transitStylingForTest() { + return { + stylePreset: state._stylePreset, + cockpitVision: state._cockpitVision, + iconCacheSize: transitIconCacheSize(), + }; + }, + _setTransitOverlayHostForTest(host) { + state._overlayHost = host || state.DEFAULT_OVERLAY_HOST; + }, + _transitStateForTest() { + return state; + }, + _transitPartsForTest() { + return parts; + }, + }; +} diff --git a/src/layers/transit/trails.js b/src/layers/transit/trails.js new file mode 100644 index 000000000..0dd1b6c3a --- /dev/null +++ b/src/layers/transit/trails.js @@ -0,0 +1,328 @@ +import { fetchTransitHistory } from '../../data/transitProxy.js'; +import { getRegisteredTransitFeed } from '../../data/transitFeeds.js'; +import * as Cesium from 'cesium'; +import { + readFix, + correctHeight, + FIX_FLAGS, + mergeHistory, +} from '../../data/contactPlayback.js'; +import { createContactTrailRenderer } from '../../data/contactTrailRenderer.js'; +import { fixDistanceM } from './movement.js'; +import { transitStyleProfile } from '../../data/transitPresetStyle.js'; + +const MODE_COLOR = { + bus: '#5EF08A', + tram: '#FFC24A', + subway: '#FF4538', + rail: '#D9A6FF', + ferry: '#5FD6FF', + unknown: '#D8DDE5', +}; +export const TRAIL_VERTEX_LIMIT = 2048; +const STEP_M = 25; + +/** Shared prepared paths. Floors are aligned to work with Google 3D tiles. */ +export function createTrails({ state, services, parts }) { + let historyRequest = null; + let requestCount = 0, + abortCount = 0; + let renderer = null, + selected = null, + fadeTimer = null, + revision = 0; + function ensureRenderer() { + if (!renderer && state._viewer) + renderer = createContactTrailRenderer(state._viewer.scene); + return renderer; + } + function style() { + renderer?.setStyle( + transitStyleProfile(state._stylePreset) === 'mono' + ? '#FFFFFF' + : MODE_COLOR[selected?.mode] || MODE_COLOR.unknown, + ); + } + function releaseSelectionGeometry(entry) { + entry.trailSegments = []; + entry.trailVertices = 0; + entry.trailTruncated = false; + entry.pathRevision = -1; + entry.markerPathSeq = NaN; + pruneMarkerPaths(entry, entry.sample); + } + function pruneMarkerPaths(entry, sample) { + if (entry === selected || entry.markerPathSeq === sample.fromSeq) return; + entry.markerPathSeq = sample.fromSeq; + for (const seq of entry.displayPaths?.keys() || []) + if (seq < sample.fromSeq) entry.displayPaths.delete(seq); + } + /** Runs only after reports, selection, or surface rereads. */ + function prepareEntry(entry, collect = null) { + if (!entry.track?.count) return; + if (entry !== selected) { + entry.trailSegments = []; + entry.trailVertices = 0; + } + const paths = new Map(), + all = [], + a = {}, + b = {}; + let vertices = 0, + signature = ''; + const floor = (point) => { + const answer = parts.height.requestHeight(point.lat, point.lon, entry); + if (collect) collect(point, answer); + return answer.height; + }; + // Selected active corridor gets first access to both surface budgets. + if (collect && entry.key === state._selectedKey) { + readFix( + entry.track, + Math.max(0, entry.sample.fromSeq - entry.track.baseSeq), + a, + ); + readFix( + entry.track, + Math.max(0, entry.sample.toSeq - entry.track.baseSeq), + b, + ); + const n = Math.min( + TRAIL_VERTEX_LIMIT - 1, + Math.max(1, Math.ceil(fixDistanceM(a, b) / STEP_M)), + ); + for (let i = 0; i <= n; i++) + floor({ + lat: a.lat + ((b.lat - a.lat) * i) / n, + lon: a.lon + ((b.lon - a.lon) * i) / n, + }); + } + for (let i = 0; i < entry.track.count; i++) { + readFix(entry.track, i, a); + const h = floor(a); + if (h !== null) correctHeight(entry.track, a.seq, h); + } + const order = []; + const activeIndex = Math.max( + 0, + Math.min( + entry.track.count - 2, + entry.sample.fromSeq - entry.track.baseSeq, + ), + ); + for (let i = activeIndex; i + 1 < entry.track.count; i++) order.push(i); + if (entry === selected) + for (let i = activeIndex - 1; i >= 0; i--) order.push(i); + entry.trailTruncated = false; + for (const i of order) { + readFix(entry.track, i, a); + readFix(entry.track, i + 1, b); + if (a.epoch !== b.epoch || b.flags & FIX_FLAGS.BREAK) continue; + if (entry !== selected && b.t < entry.sample.displayT) continue; + const n = Math.max(1, Math.ceil(fixDistanceM(a, b) / STEP_M)); + if (vertices + n + 1 > TRAIL_VERTEX_LIMIT) { + entry.trailTruncated = true; + continue; + } + const positions = [], + heights = []; + let valid = true; + for (let k = 0; k <= n; k++) { + const fraction = k / n; + const point = { + lat: a.lat + (b.lat - a.lat) * fraction, + lon: a.lon + (((b.lon - a.lon + 540) % 360) - 180) * fraction, + }; + const h = floor(point); + heights.push(h); + if (h === null) { + valid = false; + positions.push(null); + } else + positions.push( + Cesium.Cartesian3.fromDegrees(point.lon, point.lat, h), + ); + } + signature += `${a.seq}/${b.seq}:${heights.join(',')};`; + vertices += n + 1; + if (!valid) continue; + paths.set(a.seq, { positions, toSeq: b.seq }); + if (entry === selected && fixDistanceM(a, b) > 0) { + for (let k = 0; k < n; k++) + all.push({ + fromSeq: a.seq, + toSeq: b.seq, + fromT: a.t + ((b.t - a.t) * k) / n, + toT: a.t + ((b.t - a.t) * (k + 1)) / n, + positions: [positions[k], positions[k + 1]], + }); + } + } + const changed = + entry.pathSignature !== signature || + entry.pathRevision !== entry.track.revision; + if (changed) { + entry.displayPaths = paths; + entry.pathSignature = signature; + entry.pathRevision = entry.track.revision; + state._heightDirty.add(entry); + if (entry === selected) { + all.sort((a, b) => a.fromT - b.fromT); + ensureRenderer()?.replaceHistory({ + segments: all, + revision: ++revision, + }); + entry.trailVertices = vertices; + entry.trailSegments = all; + services.render.governorRequestRender('transit-trail-history'); + } + } + return changed; + } + /** Allocation-free sampling of the exact path also used by the trail. */ + function samplePosition( + entry, + sample, + out, + nearGround = parts.height.nearGround(), + ) { + pruneMarkerPaths(entry, sample); + if (!nearGround) { + const a = entry.fromCart, + b = entry.toCart, + f = sample.fraction; + out.x = a.x + (b.x - a.x) * f; + out.y = a.y + (b.y - a.y) * f; + out.z = a.z + (b.z - a.z) * f; + return out; + } + if (sample.fromSeq === sample.toSeq) { + if (!Number.isFinite(entry.segment.from.h)) return null; + Cesium.Cartesian3.clone(entry.fromCart, out); + return out; + } + const path = entry.displayPaths?.get(sample.fromSeq); + if (!path || path.toSeq !== sample.toSeq) return null; + const scaled = sample.fraction * (path.positions.length - 1); + const index = Math.min(path.positions.length - 2, Math.floor(scaled)); + // Keep scalar interpolation in this frame function. Passing the changing + // fraction to the general Cesium helper boxes it in sampled V8 stacks. + const a = path.positions[index], + b = path.positions[index + 1], + f = scaled - index; + out.x = a.x + (b.x - a.x) * f; + out.y = a.y + (b.y - a.y) * f; + out.z = a.z + (b.z - a.z) * f; + return out; + } + function update() { + if (!selected || !renderer) return; + renderer.setVisible(state._enabled && selected.marker?.show !== false); + renderer.setDisplaySample(selected.sample, selected.marker?.position); + } + function select(entry) { + clear(); + selected = entry; + ensureRenderer(); + style(); + entry.pathRevision = -1; + prepareEntry(entry); + update(); + const feed = getRegisteredTransitFeed(entry.feedId); + if (feed?.historyRetention) { + const controller = new AbortController(); + historyRequest = controller; + requestCount++; + fetchTransitHistory(entry.feedId, entry.record.id, controller.signal) + .then((payload) => { + if ( + controller.signal.aborted || + !state._enabled || + selected !== entry || + historyRequest !== controller + ) + return; + const mapping = new Map(); + for (const epoch of payload.epochs) { + let id = null; + for (const [candidate, context] of entry.track.epochs) { + if ( + context.trip === epoch.trip && + context.route === epoch.route && + context.mode === epoch.mode + ) { + id = candidate; + break; + } + } + if (id === null) { + id = Math.max(0, ...entry.track.epochs.keys()) + 1; + entry.track.epochs.set(id, { + trip: epoch.trip, + route: epoch.route, + mode: epoch.mode, + }); + } + mapping.set(epoch.id, id); + } + mergeHistory( + entry.track, + payload.fixes.map(([t, lat, lon, flags, epoch]) => ({ + t, + lat, + lon, + flags, + epoch: mapping.get(epoch), + })), + ); + entry.track.truncated ||= payload.truncated; + entry.pathRevision = -1; + parts.height.anchorFloors(); + prepareEntry(entry); + update(); + services.render.governorRequestRender('transit-trail-backfill'); + }) + .catch(() => { + // Session history remains usable if the optional retained read fails. + }) + .finally(() => { + if (historyRequest === controller) historyRequest = null; + }); + } + fadeTimer = setInterval(() => { + if (!state._enabled || selected !== entry) return; + parts.rendering.sampleIdle(entry); + update(); + services.render.governorRequestRender('transit-trail-age'); + }, 1000); + } + function clear() { + if (historyRequest) { + historyRequest.abort(); + abortCount++; + historyRequest = null; + } + if (fadeTimer != null) clearInterval(fadeTimer); + fadeTimer = null; + const previous = selected; + selected = null; + if (previous) releaseSelectionGeometry(previous); + renderer?.destroy(); + renderer = null; + } + return { + prepareEntry, + samplePosition, + select, + clear, + update, + style, + destroy: clear, + requestDiagnostics: () => ({ + requestCount, + abortCount, + pending: historyRequest !== null, + }), + diagnostics: () => renderer?.diagnostics() || null, + }; +} diff --git a/src/layers/transit/viewport.js b/src/layers/transit/viewport.js new file mode 100644 index 000000000..641fab5a2 --- /dev/null +++ b/src/layers/transit/viewport.js @@ -0,0 +1,137 @@ +import * as Cesium from 'cesium'; +import { + ACTIVATION_ENTER_ALTITUDE_M, + ACTIVATION_EXIT_ALTITUDE_M, + CAMERA_DEBOUNCE_MS, + RANGE_SLACK_KM, + inflateViewBounds, +} from './policy.js'; +import { transitFeedsInRange } from '../../data/transitFeeds.js'; + +/** + * Camera-driven feed activation: only the feeds whose coverage circle contains + * the look-at point are polled, and only below the altitude gate. + * @param {object} context + * @returns {object} + */ +export function createViewport({ state, services, parts }) { + const { governorRequestRender } = services.render; + + function getCameraAltitude(viewer = state._viewer) { + const carto = viewer?.camera?.positionCartographic; + return carto && Number.isFinite(carto.height) ? carto.height : Infinity; + } + + function getCameraCenterLatLon(viewer = state._viewer) { + const rect = viewer?.camera?.computeViewRectangle?.( + viewer.scene.globe?.ellipsoid, + ); + if (rect) { + const center = Cesium.Rectangle.center(rect); + return { + lat: Cesium.Math.toDegrees(center.latitude), + lon: Cesium.Math.toDegrees(center.longitude), + }; + } + const carto = viewer?.camera?.positionCartographic; + if (carto) { + return { + lat: Cesium.Math.toDegrees(carto.latitude), + lon: Cesium.Math.toDegrees(carto.longitude), + }; + } + return null; + } + + function altitudeGateOpen(altitude) { + if (state._altitudeGateOpen) return altitude <= ACTIVATION_EXIT_ALTITUDE_M; + return altitude <= ACTIVATION_ENTER_ALTITUDE_M; + } + + /** + * Refresh the bounds the per-frame pass uses to decide what is worth + * animating. Cheap, and only on a settled camera — never per frame. + */ + function refreshViewBounds() { + const viewer = state._viewer; + const rect = viewer?.camera?.computeViewRectangle?.( + viewer.scene.globe?.ellipsoid, + ); + state._viewBounds = rect + ? inflateViewBounds({ + south: Cesium.Math.toDegrees(rect.south), + north: Cesium.Math.toDegrees(rect.north), + west: Cesium.Math.toDegrees(rect.west), + east: Cesium.Math.toDegrees(rect.east), + }) + : null; + // The sweep itself is deferred: `marker.show` is a dirty flag, and a drag + // fires this on every frame, so re-deciding visibility inline would put the + // whole fleet back into the vertex buffer once a frame — the exact cost + // this is meant to remove. + state._visibilityDirty = true; + state._cameraRevision++; + parts.rendering.requestVisibility(); + } + + function runProximityCheck() { + if (!state._enabled || !state._viewer) return; + refreshViewBounds(); + const altitude = getCameraAltitude(); + state._altitudeGateOpen = altitudeGateOpen(altitude); + const center = state._altitudeGateOpen ? getCameraCenterLatLon() : null; + const desired = new Map(); + if (center) { + for (const feed of transitFeedsInRange(center.lat, center.lon)) { + desired.set(feed.id, feed); + } + // Hysteresis: a feed already active stays active a little past its edge. + for (const feed of transitFeedsInRange( + center.lat, + center.lon, + RANGE_SLACK_KM, + )) { + if (state._activeFeeds.has(feed.id)) desired.set(feed.id, feed); + } + } + for (const feedId of [...state._activeFeeds.keys()]) { + if (!desired.has(feedId)) { + state._inFlight.get(feedId)?.controller.abort(); + state._inFlight.delete(feedId); + state._activeFeeds.delete(feedId); + state._feedStatus.delete(feedId); + parts.ingestion.removeFeedVehicles(feedId); + } + } + for (const [feedId, feed] of desired) { + if (!state._activeFeeds.has(feedId)) { + state._activeFeeds.set(feedId, feed); + void parts.ingestion.pollFeed(feed, state._generation); + } + } + parts.ingestion.sweepAgedVehicles(Date.now()); + parts.rendering.syncRenderHold(); + governorRequestRender('transit-proximity'); + } + + function onCameraChanged() { + // The bounds update immediately; the feed proximity check stays debounced. + // A person dragging the globe should not wait a third of a second for the + // vehicles under the cursor to start moving again. + refreshViewBounds(); + clearTimeout(state._cameraDebounceTimer); + state._cameraDebounceTimer = setTimeout(() => { + state._cameraDebounceTimer = null; + runProximityCheck(); + }, CAMERA_DEBOUNCE_MS); + } + + return { + refreshViewBounds, + getCameraAltitude, + getCameraCenterLatLon, + altitudeGateOpen, + runProximityCheck, + onCameraChanged, + }; +} diff --git a/src/overlays/worldOverlay.js b/src/overlays/worldOverlay.js index 9f76a45e6..4db3cef7a 100644 --- a/src/overlays/worldOverlay.js +++ b/src/overlays/worldOverlay.js @@ -1962,6 +1962,8 @@ function publishPaintRect(item) { const { record, placement } = item; const rect = _paintRectPool[_paintRectCount] || (_paintRectPool[_paintRectCount] = {}); _paintRectCount++; + rect.anchorX = placement.anchorX; + rect.anchorY = placement.anchorY; rect.x = placement.rect.x; rect.y = placement.rect.y; rect.w = placement.rect.w; diff --git a/src/overlays/worldOverlay.test.mjs b/src/overlays/worldOverlay.test.mjs index edb6fa1b4..90ff52b0f 100644 --- a/src/overlays/worldOverlay.test.mjs +++ b/src/overlays/worldOverlay.test.mjs @@ -1299,6 +1299,8 @@ test('UI exclusions stay per-rect: overlapping chrome never merges into a boundi assert.equal(getWorldOverlayDiagnostics().paintedCount, 1); const painted = getOverlayPaintRect('ambient', 'CLEAR-OF-BOTH'); + assert.ok(Number.isFinite(painted.anchorX)); + assert.ok(Number.isFinite(painted.anchorY)); assert.ok(painted, 'the entry published a paint rectangle'); for (const [name, rect] of Object.entries(chrome)) { assert.equal(rectsIntersect(painted, inflatedRect(rect)), false, diff --git a/src/overlays/worldOverlayTokens.js b/src/overlays/worldOverlayTokens.js index 1e4c0f4c8..a08c3985a 100644 --- a/src/overlays/worldOverlayTokens.js +++ b/src/overlays/worldOverlayTokens.js @@ -114,8 +114,21 @@ export const DETECTION_THEME_MAP = Object.freeze({ filter: 'contrast(1.08) saturate(1.04)', scanline: 0.085, tiers: { - civil: '#ffd27a', military: '#ff8a3c', sea: '#ffc06a', space: '#ffe0a0', vehicle: '#d0a060', - veh_jam: '#ff3b30', veh_slow: '#ffb300', veh_free: '#00ff66', veh_nodata: '#c9c9c9', + civil: '#ffd27a', + military: '#ff8a3c', + sea: '#ffc06a', + space: '#ffe0a0', + vehicle: '#d0a060', + veh_jam: '#ff3b30', + veh_slow: '#ffb300', + veh_free: '#00ff66', + veh_nodata: '#c9c9c9', + transit_bus: '#5EF08A', + transit_tram: '#FFC24A', + transit_subway: '#FF4538', + transit_rail: '#D9A6FF', + transit_ferry: '#5FD6FF', + transit_unknown: '#D8DDE5', }, }, surveillance: { @@ -131,8 +144,21 @@ export const DETECTION_THEME_MAP = Object.freeze({ filter: 'contrast(1.12) saturate(1.12)', scanline: 0.09, tiers: { - civil: '#8fe89a', military: '#ff5a47', sea: '#a6f0c0', space: '#9fe8ff', vehicle: '#ffc24a', - veh_jam: '#ff4538', veh_slow: '#ffc24a', veh_free: '#45d8ff', veh_nodata: '#dcdcdc', + civil: '#8fe89a', + military: '#ff5a47', + sea: '#a6f0c0', + space: '#9fe8ff', + vehicle: '#ffc24a', + veh_jam: '#ff4538', + veh_slow: '#ffc24a', + veh_free: '#45d8ff', + veh_nodata: '#dcdcdc', + transit_bus: '#5EF08A', + transit_tram: '#FFC24A', + transit_subway: '#FF4538', + transit_rail: '#D9A6FF', + transit_ferry: '#5FD6FF', + transit_unknown: '#D8DDE5', }, }, thermal: { @@ -148,8 +174,21 @@ export const DETECTION_THEME_MAP = Object.freeze({ filter: 'contrast(1.1) saturate(1.08)', scanline: 0.04, tiers: { - civil: '#ffffff', military: '#ff7a5c', sea: '#ffd0b0', space: '#d0e0ff', vehicle: '#ffcf9f', - veh_jam: '#ff4538', veh_slow: '#ffc24a', veh_free: '#2ecc71', veh_nodata: '#d8d8d8', + civil: '#ffffff', + military: '#ff7a5c', + sea: '#ffd0b0', + space: '#d0e0ff', + vehicle: '#ffcf9f', + veh_jam: '#ff4538', + veh_slow: '#ffc24a', + veh_free: '#2ecc71', + veh_nodata: '#d8d8d8', + transit_bus: '#5EF08A', + transit_tram: '#FFC24A', + transit_subway: '#FF4538', + transit_rail: '#D9A6FF', + transit_ferry: '#5FD6FF', + transit_unknown: '#D8DDE5', }, }, _default: { @@ -165,8 +204,21 @@ export const DETECTION_THEME_MAP = Object.freeze({ filter: 'contrast(1.05) saturate(1.05)', scanline: 0.05, tiers: { - civil: '#22e0ff', military: '#ffb347', sea: '#3fe0c8', space: '#bda4ff', vehicle: '#8fa6b4', - veh_jam: '#e05252', veh_slow: '#f0b23e', veh_free: '#2ecc71', veh_nodata: '#c9c9c9', + civil: '#22e0ff', + military: '#ffb347', + sea: '#3fe0c8', + space: '#bda4ff', + vehicle: '#8fa6b4', + veh_jam: '#e05252', + veh_slow: '#f0b23e', + veh_free: '#2ecc71', + veh_nodata: '#c9c9c9', + transit_bus: '#5EF08A', + transit_tram: '#FFC24A', + transit_subway: '#FF4538', + transit_rail: '#D9A6FF', + transit_ferry: '#5FD6FF', + transit_unknown: '#D8DDE5', }, }, }); diff --git a/src/overlays/worldOverlayTokens.test.mjs b/src/overlays/worldOverlayTokens.test.mjs new file mode 100644 index 000000000..a1a02fe89 --- /dev/null +++ b/src/overlays/worldOverlayTokens.test.mjs @@ -0,0 +1,24 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { DETECTION_THEME_MAP } from './worldOverlayTokens.js'; +import { TRANSIT_TIER_KEYS } from '../data/transitPresetStyle.js'; + +test('every detection theme carries a distinct bracket colour for every transit mode', () => { + // The detection canvas composites above the post-FX chain, so these are + // literal screen colours in every preset — the mode signal the sprites + // lose under NVG and FLIR. A missing key would silently fall back to the + // theme's line colour and every mode would bracket alike. + for (const [name, theme] of Object.entries(DETECTION_THEME_MAP)) { + const seen = new Set(); + for (const key of TRANSIT_TIER_KEYS) { + const colour = theme.tiers?.[key]; + assert.match(colour || '', /^#[0-9a-f]{6}$/i, `${name} ${key}`); + if (key !== 'transit_unknown') seen.add(colour.toLowerCase()); + } + assert.equal( + seen.size, + TRANSIT_TIER_KEYS.length - 1, + `${name}: the five real modes differ`, + ); + } +}); diff --git a/src/tooling/transitProvider.test.mjs b/src/tooling/transitProvider.test.mjs new file mode 100644 index 000000000..b10768912 --- /dev/null +++ b/src/tooling/transitProvider.test.mjs @@ -0,0 +1,386 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { PbfWriter } from 'pbf'; +import { transitProxy } from 'gods-eye-view/server/providers/transit'; +import { + TRANSIT_BACKOFF_LADDER_MS, + TRANSIT_PROXY_TTL_MS, +} from 'gods-eye-view/sources/transit'; + +/** Mount the plugin and return a caller for its single route. */ +function install(plugin) { + const routes = new Map(); + plugin.configureServer({ + middlewares: { + use(route, handler) { + routes.set(route, handler); + }, + }, + }); + assert.equal(routes.size, 1); + return async (url = '/', method = 'GET') => { + const res = { + headersSent: false, + writeHead(status, headers) { + Object.assign(this, { status, headers, headersSent: true }); + }, + end(body) { + this.body = body; + }, + }; + await [...routes.values()][0]({ url, method }, res); + return res; + }; +} + +/** One GTFS-RT FeedMessage with the given header and vehicles. */ +function feedBytes({ + incrementality = 0, + timestamp = 1_700_000_000, + vehicles = [], +}) { + const writer = new PbfWriter(); + const header = new PbfWriter(); + header.writeStringField(1, '2.0'); + header.writeVarintField(2, incrementality); + header.writeVarintField(3, timestamp); + writer.writeBytesField(1, header.finish()); + for (const item of vehicles) { + const entity = new PbfWriter(); + entity.writeStringField(1, item.id); + const vehicle = new PbfWriter(); + const position = new PbfWriter(); + position.writeFloatField(1, item.lat); + position.writeFloatField(2, item.lon); + vehicle.writeBytesField(2, position.finish()); + vehicle.writeVarintField(5, item.timestamp ?? timestamp); + const descriptor = new PbfWriter(); + descriptor.writeStringField(1, item.id); + vehicle.writeBytesField(8, descriptor.finish()); + entity.writeBytesField(4, vehicle.finish()); + writer.writeBytesField(2, entity.finish()); + } + return writer.finish(); +} + +function upstreamResponse({ status = 200, headers = {}, bytes = null }) { + const map = new Map( + Object.entries(headers).map(([name, value]) => [name.toLowerCase(), value]), + ); + return { + status, + ok: status >= 200 && status < 300, + headers: { get: (name) => map.get(String(name).toLowerCase()) ?? null }, + body: null, + async arrayBuffer() { + const view = bytes || new Uint8Array(0); + return view.buffer.slice( + view.byteOffset, + view.byteOffset + view.byteLength, + ); + }, + }; +} + +/** A scripted upstream that records every request it is actually given. */ +function scriptedUpstream(steps) { + const calls = []; + let index = 0; + const fetchImpl = async (url, init) => { + calls.push({ url, headers: init?.headers || {}, redirect: init?.redirect }); + const step = steps[Math.min(index, steps.length - 1)]; + index += 1; + if (typeof step === 'function') return step(url, init); + return step; + }; + return { fetchImpl, calls }; +} + +const FULL = feedBytes({ + vehicles: [{ id: 'bus-1', lat: 42.36, lon: -71.06 }], +}); + +test('a 304 revalidation is a success, not a redirect with a missing Location', async (t) => { + t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 }); + const { fetchImpl, calls } = scriptedUpstream([ + upstreamResponse({ headers: { etag: 'W/"v1"' }, bytes: FULL }), + upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }), + upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }), + ]); + const call = install(transitProxy({ fetchImpl })); + + const first = await call('/vehicles/mbta'); + assert.equal(first.status, 200); + assert.equal(first.headers['X-GEV-Cache'], 'MISS'); + assert.equal(JSON.parse(first.body).count, 1); + + // Past the freshness window the proxy asks again, conditionally. + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000); + const second = await call('/vehicles/mbta'); + assert.equal(calls[1].headers['If-None-Match'], 'W/"v1"'); + assert.equal(second.status, 200, 'an unchanged feed is not a failure'); + // MISS, not STALE-ERROR: the refresh SUCCEEDED. Treating 304 as a redirect + // made this a failure that happened to be masked by the serve-stale path. + assert.equal(second.headers['X-GEV-Cache'], 'MISS'); + assert.equal(JSON.parse(second.body).count, 1); + + // And the third one too: a 304 must not have started a backoff ladder. + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000); + const third = await call('/vehicles/mbta'); + assert.equal(third.status, 200); + assert.equal(third.headers['X-GEV-Cache'], 'MISS'); + assert.equal(calls.length, 3, 'every poll past the TTL reached the operator'); + assert.equal(third.headers['Retry-After'], undefined); + assert.equal( + third.headers['X-Transit-Backoff'], + undefined, + 'no ladder was started', + ); +}); + +test('a differential feed condemns the snapshot that preceded it', async (t) => { + t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 }); + const differential = feedBytes({ + incrementality: 1, + vehicles: [{ id: 'bus-1', lat: 42.36, lon: -71.06 }], + }); + const { fetchImpl } = scriptedUpstream([ + upstreamResponse({ bytes: FULL }), + upstreamResponse({ bytes: differential }), + upstreamResponse({ bytes: differential }), + ]); + const call = install(transitProxy({ fetchImpl })); + + assert.equal((await call('/vehicles/mbta')).status, 200); + + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000); + const refused = await call('/vehicles/mbta'); + assert.equal(refused.status, 502); + assert.match(JSON.parse(refused.body).error, /differential/); + + // The next caller arrives INSIDE the cooldown the failure just started, which + // is precisely the path that used to hand back the condemned snapshot as a + // 200 because it was still inside the serve-stale window. + t.mock.timers.setTime(Date.now() + 2_000); + const after = await call('/vehicles/mbta'); + assert.notEqual( + after.status, + 200, + `served ${after.status} with ${after.body}`, + ); + assert.equal(after.headers['X-GEV-Cache'], 'NONE'); + assert.equal(after.headers['X-Transit-Backoff'], 'cooldown'); +}); + +test('a failing operator is put on a cooldown instead of being re-asked every poll', async (t) => { + t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 }); + const { fetchImpl, calls } = scriptedUpstream([ + upstreamResponse({ status: 500 }), + upstreamResponse({ status: 500 }), + ]); + const call = install(transitProxy({ fetchImpl })); + + const failed = await call('/vehicles/mbta'); + assert.equal(failed.status, 502); + assert.equal( + failed.headers['Retry-After'], + String(TRANSIT_BACKOFF_LADDER_MS[0] / 1000), + ); + assert.equal(calls.length, 1); + + // Inside the cooldown the operator is not contacted at all. + t.mock.timers.setTime(Date.now() + 1_000); + const held = await call('/vehicles/mbta'); + assert.equal(held.status, 503); + assert.equal(held.headers['X-Transit-Backoff'], 'cooldown'); + assert.equal(calls.length, 1, 'the operator was left alone'); + + // Past the first rung it is tried again, and a second failure waits longer. + t.mock.timers.setTime(Date.now() + TRANSIT_BACKOFF_LADDER_MS[0]); + const retried = await call('/vehicles/mbta'); + assert.equal(calls.length, 2); + assert.equal( + retried.headers['Retry-After'], + String(TRANSIT_BACKOFF_LADDER_MS[1] / 1000), + ); +}); + +test('a redirect is followed only within the feed origin, and never contacted first', async (t) => { + t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 }); + const sameOrigin = scriptedUpstream([ + upstreamResponse({ + status: 302, + headers: { location: '/files/current.pb' }, + }), + upstreamResponse({ bytes: FULL }), + ]); + const call = install(transitProxy({ fetchImpl: sameOrigin.fetchImpl })); + const ok = await call('/vehicles/capmetro-austin'); + assert.equal(ok.status, 200); + assert.equal(sameOrigin.calls.length, 2); + assert.equal( + sameOrigin.calls[1].url, + 'https://data.texas.gov/files/current.pb', + ); + assert.equal(sameOrigin.calls[0].redirect, 'manual'); + + const offOrigin = scriptedUpstream([ + upstreamResponse({ + status: 302, + headers: { location: 'https://cdn.elsewhere.test/current.pb' }, + }), + upstreamResponse({ bytes: FULL }), + ]); + const refuse = install(transitProxy({ fetchImpl: offOrigin.fetchImpl })); + const blocked = await refuse('/vehicles/capmetro-austin'); + assert.equal(blocked.status, 504); + assert.equal( + offOrigin.calls.length, + 1, + 'the disallowed host was never contacted, only its Location read', + ); +}); + +test('only registered feed ids reach an operator', async (t) => { + const { fetchImpl, calls } = scriptedUpstream([ + upstreamResponse({ bytes: FULL }), + ]); + const call = install(transitProxy({ fetchImpl })); + for (const path of [ + '/vehicles/not-a-feed', + '/vehicles/..%2F..%2Fetc', + '/vehicles/', + '/elsewhere', + ]) { + const missing = await call(path); + assert.equal(missing.status, 404, `${path} is not a door to an upstream`); + } + assert.equal(calls.length, 0, 'no operator was contacted for any of them'); + const catalog = JSON.parse((await call('/feeds')).body); + assert.ok(catalog.feeds.length > 0); + for (const feed of catalog.feeds) { + assert.equal('url' in feed, false); + assert.equal('headers' in feed, false); + } +}); + +test('a 304 reports that the operator answered, not that the body is new', async (t) => { + // The body a revalidated feed keeps serving carries the fetch time of the + // payload it holds, which stops advancing the moment the operator's file + // stops changing. That is the right answer for "how old are these positions" + // and the wrong one for "when did we last hear from them" — read as the + // latter it aged a feed whose every request succeeded into silence. + t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 }); + const firstAt = Date.now(); + const { fetchImpl } = scriptedUpstream([ + upstreamResponse({ headers: { etag: 'W/"v1"' }, bytes: FULL }), + upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }), + upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }), + ]); + const call = install(transitProxy({ fetchImpl })); + + const first = await call('/vehicles/mbta'); + assert.equal(Number(first.headers['X-Transit-Contact']), firstAt); + assert.equal(JSON.parse(first.body).fetchedAt, firstAt); + + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000); + const revalidatedAt = Date.now(); + const second = await call('/vehicles/mbta'); + assert.equal( + JSON.parse(second.body).fetchedAt, + firstAt, + 'the positions are as old as they are', + ); + assert.equal( + Number(second.headers['X-Transit-Contact']), + revalidatedAt, + 'but the operator answered just now', + ); + + // Served from the fresh cache, the contact time is the last real contact — + // not the moment this request happened to arrive. + const third = await call('/vehicles/mbta'); + assert.equal(third.headers['X-GEV-Cache'], 'HIT'); + assert.equal(Number(third.headers['X-Transit-Contact']), revalidatedAt); +}); + +test('a feed served from cache during an outage does not claim fresh contact', async (t) => { + // The other half: STALE-ERROR must carry the LAST SUCCESSFUL contact, so the + // browser can tell a revalidated feed from a dead one. + t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 }); + const contactedAt = Date.now(); + const { fetchImpl } = scriptedUpstream([ + upstreamResponse({ bytes: FULL }), + upstreamResponse({ status: 500 }), + upstreamResponse({ status: 500 }), + ]); + const call = install(transitProxy({ fetchImpl })); + await call('/vehicles/mbta'); + + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000); + const during = await call('/vehicles/mbta'); + assert.equal(during.headers['X-GEV-Cache'], 'STALE-ERROR'); + assert.equal( + Number(during.headers['X-Transit-Contact']), + contactedAt, + 'silence does not count as an answer', + ); +}); + +// Include the provider-local storage contract in the repository's src test discovery. +import '../../server/providers/transitHistory.test.mjs'; + +test('history reads never fetch upstream, and 304/cache/failure replays never append', async (t) => { + t.mock.timers.enable({ apis: ['Date'], now: 1700000000000 }); + const { calls, fetchImpl } = scriptedUpstream([ + upstreamResponse({ bytes: FULL }), + upstreamResponse({ status: 304 }), + upstreamResponse({ status: 503 }), + ]); + const plugin = transitProxy({ fetchImpl }); + t.after(() => plugin.closeBundle()); + const call = install(plugin); + const read = () => call('/trail/mbta/bus-1'); + assert.equal(JSON.parse((await read()).body).fixes.length, 0); + assert.equal(calls.length, 0); + await call('/vehicles/mbta'); + const initial = JSON.parse((await read()).body); + assert.equal(initial.fixes.length, 1); + await call('/vehicles/mbta'); + assert.equal(calls.length, 1); + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1); + await call('/vehicles/mbta'); + assert.deepEqual(JSON.parse((await read()).body), initial); + t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1); + await call('/vehicles/mbta'); + assert.deepEqual(JSON.parse((await read()).body), initial); + assert.equal(calls.length, 3); + for (const path of [ + '/trail/mbta/%2Fescape', + '/trail/mbta/abc%00bad', + '/trail/mbta/%E0%A4%A', + `/trail/mbta/${'a'.repeat(257)}`, + '/trail/capmetro-austin/bus-1', + ]) + assert.equal((await call(path)).status, 404); + assert.equal( + calls.length, + 3, + 'malformed and non-retained reads cannot reach an operator', + ); +}); +test('history read admission is separate from upstream admission and closes with the server', async (t) => { + t.mock.timers.enable({ apis: ['Date'], now: 1700000000000 }); + const { calls, fetchImpl } = scriptedUpstream([ + upstreamResponse({ bytes: FULL }), + ]); + const plugin = transitProxy({ fetchImpl }); + t.after(() => plugin.closeBundle()); + const call = install(plugin); + for (let i = 0; i < 30; i++) + assert.equal((await call('/trail/mbta/bus-1')).status, 200); + assert.equal((await call('/trail/mbta/bus-1')).status, 429); + assert.equal(calls.length, 0); + assert.equal((await call('/vehicles/mbta')).status, 200); + assert.equal(calls.length, 1); +}); diff --git a/src/tooling/transitQa.test.mjs b/src/tooling/transitQa.test.mjs new file mode 100644 index 000000000..0c5fd106d --- /dev/null +++ b/src/tooling/transitQa.test.mjs @@ -0,0 +1,631 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import { + chooseLiveCities, + selectStyle, + detectOn, +} from '../../scripts/qa-transit-controls.mjs'; +import { createStyleParameters } from '../ui/styleParameters.js'; +import { bindDisplayControls } from '../ui/displayControls.js'; +import { thermalShader } from '../styles/thermal.js'; +import { + cycleMode, + setMode, + getMode, + setDetectionTuning, +} from '../data/detection.js'; + +const pause = async () => {}; +class Element extends EventTarget { + constructor() { + super(); + this.children = []; + this.attributes = new Map(); + this.classList = { contains: () => false }; + } + setAttribute(name, value) { + this.attributes.set(name, value); + } + getAttribute(name) { + return this.attributes.get(name); + } + append(...children) { + this.children.push(...children); + } + appendChild(child) { + this.children.push(child); + } + replaceChildren() { + this.children = []; + } + scrollIntoView() {} + getBoundingClientRect() { + return { width: 100 }; + } +} +function pageFixture(t) { + const prior = { document: globalThis.document, window: globalThis.window }; + const elements = new Map(); + const document = { + documentElement: { dataset: { gevStyle: 'normal' } }, + createElement: () => new Element(), + getElementById: (id) => elements.get(id), + querySelector: () => null, + }; + globalThis.document = document; + globalThis.window = {}; + t.after(() => { + for (const [name, value] of Object.entries(prior)) { + if (value === undefined) delete globalThis[name]; + else globalThis[name] = value; + } + }); + const find = (selector) => { + if (selector.startsWith('#param-sliders ')) { + const label = selector.match(/aria-label="([^"]+)"/)[1]; + return elements + .get('param-sliders') + .children.flatMap((row) => row.children) + .find((el) => el.getAttribute('aria-label') === label); + } + return elements.get(selector.slice(1)); + }; + const page = { + evaluate: async (fn, ...args) => fn(...args), + waitForFunction: async (fn, options, ...args) => + assert.ok(fn(...args), 'UI condition must hold'), + waitForSelector: async (selector) => + assert.ok(find(selector), `missing ${selector}`), + $eval: async (selector, fn, ...args) => { + const el = find(selector); + assert.ok(el, `missing ${selector}`); + return fn(el, ...args); + }, + click: async (selector) => + find(selector)?.dispatchEvent(new Event('click')), + }; + return { page, elements, document }; +} + +test('live visual cities follow local operating hours in summer and winter', () => { + for (const date of ['2026-09-15T06:30:00Z', '2026-01-15T06:30:00Z']) { + assert.equal(chooseLiveCities(new Date(date))[0].id, 'helsinki'); + } + assert.deepEqual( + chooseLiveCities(new Date('2026-09-15T17:00:00Z')).map((c) => c.id), + ['boston', 'austin'], + ); + for (let hour = 0; hour < 24; hour++) { + const now = new Date(`2026-09-15T${String(hour).padStart(2, '0')}:00:00Z`); + const cities = chooseLiveCities(now); + assert.ok(cities.length >= 1); + for (const city of cities) { + const local = Number( + new Intl.DateTimeFormat('en', { + timeZone: city.zone, + hour: 'numeric', + hourCycle: 'h23', + }).format(now), + ); + assert.ok(local >= 6 && local < 22); + } + } +}); + +test('thermal controls wait for activation and use the real generated parameter rows', async (t) => { + const { page, document, elements } = pageFixture(t); + const container = new Element(); + container.ownerDocument = document; + elements.set('param-sliders', container); + const parameters = createStyleParameters({ container }); + t.after(() => parameters.destroy()); + const values = Object.fromEntries( + Object.entries(thermalShader.uniforms).map(([key, meta]) => [ + key, + meta.default, + ]), + ); + window.__godsEyeView = { + styleManager: { + setPanelCollapsed() {}, + setStyle(name) { + document.documentElement.dataset.gevStyle = name; + parameters.render({ + uniforms: thermalShader.uniforms, + readValue: (key) => values[key], + writeValue: (key, value) => { + values[key] = value; + }, + onChange() {}, + }); + }, + }, + }; + // A swallowed click resolves normally; the helper must notice no activation. + page.$eval = async (selector, fn, ...args) => { + if (selector.startsWith('.style-btn')) return fn(new Element(), ...args); + const label = selector.match(/aria-label="([^"]+)"/)[1]; + const slider = container.children + .flatMap((row) => row.children) + .find((el) => el.getAttribute('aria-label') === label); + assert.ok(slider); + return fn(slider, ...args); + }; + const how = await selectStyle( + page, + 'thermal', + { 'WHOT/BHOT': 1, Ironbow: 1 }, + pause, + ); + assert.match(how, /button did not activate/); + assert.equal(values.mode, 1); + assert.equal(values.palette, 1); +}); + +test('DETECT uses the real OFF/restore button and density input to reach DENSE', async (t) => { + const { page, elements } = pageFixture(t); + const button = new Element(), + slider = new Element(); + elements.set('detection-toggle', button); + elements.set('detection-density-slider', slider); + button.getAttribute = (name) => + name === 'aria-pressed' + ? String(getMode() !== 'OFF') + : `Detection overlay: ${getMode().toLowerCase()}`; + setMode('BALANCED'); + setMode('OFF'); + const modes = []; + const controls = bindDisplayControls({ + elements: { detectionButton: button, densitySlider: slider }, + actions: { + cycleDetection() { + cycleMode(); + modes.push(getMode()); + }, + setDensity(value) { + setDetectionTuning({ densityPct: Number(value) }); + }, + }, + }); + t.after(() => { + controls.destroy(); + setMode('OFF'); + }); + assert.equal(await detectOn(page, pause), 'DENSE'); + assert.deepEqual(modes, ['BALANCED', 'OFF', 'DENSE']); + const template = readFileSync( + new URL('../ui/templates/display-controls.html', import.meta.url), + 'utf8', + ); + assert.match(template, /id="detection-density-slider"/); + assert.match(template, /id="detection-toggle"/); +}); + +test('city motion checks load deterministic playback before reading rendered samples', () => { + const source = readFileSync( + new URL('../../scripts/qa-transit.mjs', import.meta.url), + 'utf8', + ); + const fixture = source.indexOf('_loadTransitFleetForTest(16'); + assert.ok(fixture >= 0); + assert.ok( + source.indexOf('const motionFirst = await sampleVisible(page)', fixture) > + fixture, + ); + assert.ok( + source.indexOf('const glideA = await sampleVisible(page)', fixture) > + fixture, + ); + assert.ok( + source.indexOf('const headingA = await sampleScreen(page)', fixture) > + fixture, + ); + assert.match(source, /scene.renderError.addEventListener/); +}); + +const { boundPageEvaluations, restoreRetainedTransit, sampleTransitPixels } = + await import('../../scripts/qa-transit-browser.mjs'); + +test('page promise deadline rejects a stuck enable or teardown and preserves ordinary results', async () => { + const page = { evaluate: async (fn, ...args) => fn(...args) }; + boundPageEvaluations(page, 15); + assert.equal(await page.evaluate((a, b) => a + b, 3, 4), 7); + await assert.rejects( + page.evaluate(() => new Promise(() => {})), + /Transit QA page wait timed out after 15 ms/, + ); +}); + +test('reload picks a surviving primed vehicle and reports unavailable snapshots as unexercised', async (t) => { + const previous = globalThis.window; + t.after(() => { + globalThis.window = previous; + }); + let selected; + const entry = { key: 'mbta:survivor', record: { lon: -71.06, lat: 42.36 } }; + const vehicles = new Map([[entry.key, entry]]); + globalThis.window = { + __godsEyeView: { + viewer: { + camera: { + positionCartographic: { + constructor: { fromDegrees: (...v) => v, toCartesian: (v) => v }, + }, + setView() {}, + }, + }, + dataManager: { + setEnabled: async () => {}, + layers: new Map([ + [ + 'transit', + { + module: { + update: async () => {}, + _transitStateForTest: () => ({ _vehicles: vehicles }), + _transitPartsForTest: () => ({ + selection: { + selectVehicle: (key) => { + selected = key; + }, + }, + }), + }, + }, + ], + ]), + }, + }, + }; + const primed = [ + { vehicleId: 'departed', oldestT: 10 }, + { vehicleId: 'survivor', oldestT: 20 }, + ]; + assert.deepEqual(await restoreRetainedTransit(primed), { + key: entry.key, + oldest: 20, + }); + assert.equal(selected, entry.key); + vehicles.clear(); + const manager = window.__godsEyeView.dataManager; + const layer = manager.layers.get('transit').module; + layer._transitStateForTest = () => ({ + _vehicles: vehicles, + _activeFeeds: new Map([['mbta', {}]]), + }); + manager.setEnabled = async () => { + setTimeout(() => vehicles.set(entry.key, entry), 5); + }; + assert.equal( + (await restoreRetainedTransit(primed)).key, + entry.key, + 'wait for discovery instead of declaring every vehicle departed', + ); + manager.setEnabled = async () => {}; + assert.equal( + ( + await restoreRetainedTransit([ + { ...primed[1], feedId: 'mbta', primedAt: Date.now() - 60000 }, + ]) + ).unexercised, + true, + 'expired priming cannot pass', + ); + layer._transitStateForTest = () => ({ _vehicles: vehicles }); + vehicles.clear(); + assert.equal((await restoreRetainedTransit(primed)).unexercised, true); + assert.equal((await restoreRetainedTransit([])).unexercised, true); +}); + +test('sensor sampler waits for postRender and reads framebuffer pixels; stopped rendering rejects and removes listener', async (t) => { + const previousWindow = globalThis.window, + previousDocument = globalThis.document, + previousImage = globalThis.Image; + t.after(() => { + globalThis.window = previousWindow; + globalThis.document = previousDocument; + globalThis.Image = previousImage; + }); + const identity = [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]; + let listener, + reads = 0, + render = true; + const gl = { + RGBA: 1, + UNSIGNED_BYTE: 2, + readPixels(x, y, w, h, format, type, buffer) { + reads++; + buffer.set([255, 255, 255, 255]); + }, + }; + const scene = { + camera: { viewMatrix: identity, frustum: { projectionMatrix: identity } }, + canvas: { + width: 400, + height: 400, + clientWidth: 400, + clientHeight: 400, + getContext: () => gl, + }, + postRender: { + addEventListener(fn) { + listener = fn; + return () => { + listener = null; + }; + }, + }, + requestRender() { + if (render) queueMicrotask(() => listener?.()); + }, + }; + const raster = new Uint8ClampedArray(16 * 16 * 4).fill(255); + globalThis.Image = class { + width = 16; + height = 16; + async decode() {} + }; + globalThis.document = { + getElementById: () => null, + createElement: () => ({ + getContext: () => ({ + drawImage() {}, + getImageData: () => ({ data: raster }), + }), + }), + }; + globalThis.window = { + __godsEyeView: { + viewer: { scene }, + dataManager: { + layers: new Map([ + [ + 'transit', + { + module: { + _transitStateForTest: () => ({ + _vehicles: new Map([ + [ + 'bus', + { + key: 'bus', + mode: 'bus', + marker: { + show: true, + image: 'raster', + position: { x: 0, y: 0, z: 0 }, + }, + }, + ], + ]), + }), + }, + }, + ], + ]), + }, + }, + }; + const page = { evaluate: async (fn, ...args) => fn(...args) }; + const pixels = await sampleTransitPixels(page, {}); + assert.ok( + reads >= 13, + 'centre, rings and background come from WebGL readPixels', + ); + assert.ok(Math.abs(pixels[0].centre - 1) < 1e-6); + assert.equal(listener, null); + scene.pick = () => ({ id: 'overlapping-label' }); + assert.equal( + (await sampleTransitPixels(page, {})).length, + 0, + 'a foreign pick across the sampled core disqualifies an obscured sprite', + ); + scene.pick = ({ x }) => ({ id: x < 200 ? 'overlapping-sprite' : 'bus' }); + assert.equal( + (await sampleTransitPixels(page, {})).length, + 0, + 'a clear centre cannot qualify a partially obscured core', + ); + delete scene.pick; + const module = window.__godsEyeView.dataManager.layers.get('transit').module; + const getState = module._transitStateForTest; + module._transitStateForTest = () => { + const state = getState(); + state._stylePreset = 'noir'; + state._vehicles.get('bus').marker.position.x = 0.65; + return state; + }; + assert.equal( + (await sampleTransitPixels(page, {})).length, + 0, + 'the intentional noir edge vignette is outside unobscured-core acceptance', + ); + module._transitStateForTest = getState; + // The fixture returns fresh wrapper objects, so exercise raster and DOM + // eligibility independently of framebuffer brightness. + raster.fill(0); + assert.equal((await sampleTransitPixels(page, {})).length, 0); + raster.fill(255); + document.elementFromPoint = () => ({ tagName: 'DIV', closest: () => null }); + assert.equal((await sampleTransitPixels(page, {})).length, 0); + delete document.elementFromPoint; + document.getElementById = () => ({ + width: 400, + clientWidth: 400, + getContext: () => ({ + getImageData: () => ({ data: new Uint8ClampedArray(144).fill(255) }), + }), + }); + assert.equal((await sampleTransitPixels(page, {})).length, 0); + document.getElementById = () => null; + window.__godsEyeView.styleManager = { + getDetectionDiagnostics: () => ({ + calloutRects: [{ x: 205, y: 195, w: 10, h: 10, alpha: 1 }], + }), + }; + assert.equal( + (await sampleTransitPixels(page, {})).length, + 0, + 'plate over the edge excludes the whole sprite even with a clear centre', + ); + delete window.__godsEyeView.styleManager; + document.getElementById = () => ({ + width: 400, + clientWidth: 400, + getContext: () => ({ + getImageData: (x, y, w = 1, h = 1) => ({ + data: + w > 1 + ? new Uint8ClampedArray(w * h * 4) + : new Uint8ClampedArray( + x === 191 && [190, 193, 196].includes(y) + ? [20, 30, 40, 255] + : x === 191 && y === 207 + ? [94, 240, 138, 255] + : [0, 0, 0, 0], + ), + }), + }), + }); + assert.deepEqual( + (await sampleTransitPixels(page, {}))[0].bracket, + [94, 240, 138], + 'stroke centre is sampled away from the top label plate', + ); + document.getElementById = () => null; + render = false; + t.mock.timers.enable({ apis: ['setTimeout'] }); + const pending = sampleTransitPixels(page, {}); + const rejected = assert.rejects(pending, /no postRender within 5000 ms/); + await new Promise((resolve) => setImmediate(resolve)); + t.mock.timers.tick(5000); + await rejected; + assert.equal(listener, null); +}); + +test('preset baseline clears a previous city selection before comparing fleet sizes', async (t) => { + const { readTransitFleetStyle } = + await import('../../scripts/qa-transit-browser.mjs'); + const prior = globalThis.window; + t.after(() => { + globalThis.window = prior; + }); + const entry = { + key: 'mbta:tram', + marker: { + show: true, + width: 33.75, + image: 'raster-selected', + color: { toCssColorString: () => 'rgb(255,194,74)' }, + }, + }; + const state = { + _selectedKey: entry.key, + _vehicles: new Map([[entry.key, entry]]), + }; + globalThis.window = { + __godsEyeView: { + dataManager: { + layers: new Map([ + [ + 'transit', + { + module: { + _transitStateForTest: () => state, + _transitPartsForTest: () => ({ + selection: { + clearSelection() { + state._selectedKey = null; + entry.marker.width = 22.5; + entry.marker.image = 'raster-fleet'; + }, + }, + }), + }, + }, + ], + ]), + }, + }, + }; + const baseline = readTransitFleetStyle({ resetSelection: true }); + assert.equal(baseline.width, 22.5); + assert.equal(baseline.selected, false); + assert.deepEqual(readTransitFleetStyle({ key: entry.key }), baseline); +}); + +test('allocation accounting includes Cesium render aliases and excludes QA helper subtrees', async () => { + const { allocationTotals } = + await import('../../scripts/qa-transit-scenes.mjs'); + const node = (functionName, url, selfSize, children = []) => ({ + callFrame: { functionName, url }, + selfSize, + children, + }); + const profile = { + head: node('(root)', '', 0, [ + node('render2', 'cesium.js', 0, [ + node('Scene4.render', 'cesium.js', 10, [ + node('_drawOverlay', '/src/data/detection.js', 100), + node('', '/src/layers/transit/testing.js', 200, [ + node('hypot', '', 300), + node('onPreRender', '/src/layers/transit/rendering.js', 20, [ + node('sampleAt', '/src/data/contactPlayback.js', 30), + node('style', '/src/data/transitPresetStyle.js', 10), + ]), + ]), + ]), + ]), + ]), + }; + const { topAllocators, ...totals } = allocationTotals(profile); + assert.equal(topAllocators[0].bytes, 100); + assert.deepEqual(totals, { + frameBytes: 170, + layerFrameBytes: 60, + sharedOverlayBytes: 100, + qaBytes: 500, + }); +}); + +test('public transit documentation uses courtesy attribution and the approved ground sentence', () => { + const data = readFileSync( + new URL('../../DATA_SOURCES.md', import.meta.url), + 'utf8', + ); + const rows = data + .split('\n') + .filter((line) => + /^\| \*\*(MBTA|CapMetro|Metro Transit|OVapi|Entur|TransLink|HSL)/.test( + line, + ), + ); + assert.equal(rows.length, 7); + for (const row of rows) { + assert.doesNotMatch( + row, + /licen[cs]e|unlicensed|unstated|grants|permits|public.domain|CC BY|NLOD|terms|trademarks/i, + ); + assert.match(row, /open realtime vehicle data/); + assert.match(row, /published for developer use and not against their use/); + } + const readme = readFileSync( + new URL('../../README.md', import.meta.url), + 'utf8', + ); + assert.ok( + readme.includes( + 'Fifteen layers and map sources. **Thirteen have a keyless path.**', + ), + ); + assert.ok( + readme.includes( + '**Sits on the real ground.** Entity heights are aligned to work with Google 3D tiles, so aircraft park on aprons and cameras stand on street corners instead of floating.', + ), + ); + assert.doesNotMatch( + readme, + /geoid-aware|sampled against the _rendered_ terrain mesh/, + ); +}); diff --git a/src/ui/applicationShell.js b/src/ui/applicationShell.js index e6ba0f8c0..aa2dab4ee 100644 --- a/src/ui/applicationShell.js +++ b/src/ui/applicationShell.js @@ -98,6 +98,7 @@ export class StyleManager { satellitesLayer, cctvLayer, bikeshareLayer, + transitLayer, aisLiveVesselsLayer, militaryAwarenessLayer, cachedGroundFloor, @@ -570,6 +571,7 @@ export class StyleManager { satellitesLayer, cctvLayer, bikeshareLayer, + transitLayer, aisLiveVesselsLayer, ], (modeLabel) => { diff --git a/src/ui/visualSettings.js b/src/ui/visualSettings.js index 24711caed..a1eac1bc6 100644 --- a/src/ui/visualSettings.js +++ b/src/ui/visualSettings.js @@ -253,6 +253,17 @@ export class VisualSettings { effective === 'nvg'; this._dataManager?.setLayerParams('flights', { irBoost }); this._dataManager?.setLayerParams('military', { irBoost }); + // Layers whose in-scene sprites restyle for a sensor preset need the + // EFFECTIVE style — a cockpit vision override sets no map style, so the + // map's own style event never fires for it. + window.dispatchEvent( + new CustomEvent('gev:vision-change', { + detail: { + style: effective, + cockpit: Boolean(cockpitMode && cockpitMode !== 'optical'), + }, + }), + ); // Fog blends distant geometry toward an effectively-BLACK color in this // app (the Cesium globe is hidden), so beyond ~100 km every 3D aircraft // fogs to a black silhouette — lighting and shaders can't reach past it