Compare commits

...
Author SHA1 Message Date
manjunath22466 01979bba1e chore(stack): refresh voice core and outlines parents
merge the latest #837 and #842 draft heads

keep the stack parent-only so #846 shows geometry changes once
2026-10-02 23:28:25 +05:30
manjunath22466 e57b60e511 fix(voice): detach nested contacts snapshots
deep-clone retained aircraft items, provenance and subject data

add a regression for nested mutation across repeated snapshot reads
2026-10-02 23:21:14 +05:30
manjunath22466 177b5951a0 fix(outlines): harden shared cache and provider lifecycle 2026-10-02 22:32:46 +05:30
manjunath22466 095981eee7 fix(voice): preserve analyst result and tracking contracts
keep contacts counts, rows, scope and provenance on one retained snapshot

report capped cohorts as lower bounds

separate sibling tool cancellation from newer user turns

align track_entity validation across runtime and benchmark paths
2026-10-02 11:15:31 +05:30
Bilawal Sidhu 9559e7a282 feat(outlines): voice outlines without Overpass — bundled US places and neighborhoods, guarded Nominatim, OpenFreeMap streets and buildings 2026-09-28 22:49:23 -05:00
Bilawal Sidhu a19897ad15 feat(voice): voice core — honest layer reach, spoken summary with on-screen card, narration, point-and-ask
- Voice layer manifest (src/voice/layerManifest.js) generates the layer enums,
  aliases and analyst field hints; a test fails when a catalog layer has no entry.
- analyst_query validates fields, operators, units, values and scopes, counts up
  to 250,000 loaded records per layer, reports complete/partial/unanswered, tells
  LAYER_OFF / NOT_READY / FEED_UNAVAILABLE apart from zero, refuses mismatched
  follow-ups and never lets a superseded query replace follow-up memory. Seven
  more layers expose analyst records.
- Tool results carry a code-built say line and a display card; the voice card in
  the mic control shows captions, plan steps and the result
  (src/voice/resultDisplay.js is the one adapter). Slow tools narrate their
  current step out of band; holding Space while the assistant talks cuts it off.
  Turn spans go to the debug log; caption transcription is metered; the log
  omits what was said unless GEV_VOICE_LOG_CONTENT=1.
- Point and ask: the Space-down pointer snapshot resolves "this", "that one" and
  "around here" through 'pointer' targets, and "the second one" through
  referent:n from the last numbered list.
- Prompt states each policy once (instructions + tools 48.5k -> 46.7k chars).
- Voice benchmark harness (scripts/qa-voice-bench.mjs, scripts/voice-bench/)
  shares its phrase table and grading with scripts/qa-voice-routing.mjs.
2026-09-28 21:19:23 -05:00
Bilawal SidhuandBilawal Sidhu e7707d9a0f docs: tell existing installs to update for OSM-backed layers (#821)
Co-authored-by: Bilawal Sidhu <[email protected]>
2026-09-28 19:33:34 -05:00
f4fcf267c6 fix(osm): stop using public Overpass by default (#648) (#742)
Traffic roads come from OpenFreeMap vector tiles, or TomTom flow tiles with a
key (Hybrid by default). Military areas use OpenFreeMap polygons with a bundled
worldwide name index; wide views show named points. ALPR reads an hourly
OpenStreetMap extract. Cockpit place context and country, state and US county
outlines use bundled Natural Earth and US Census data. Overpass is used only
when an operator configures their own endpoint.

Also: camera arrivals no longer land below ground, keyless terrain retries on
429, bounded relation geometry for configured Overpass, one shared
OpenStreetMap credit, and separate tile and API transports for embedders.

Co-authored-by: theGaryLarson <[email protected]>
Co-authored-by: Milan Khanal <[email protected]>
2026-09-28 19:07:57 -05:00
Sameh Khamis 81eb44340d Merge pull request #744 from manjunath22466/fix/share-link-layer-token-allocation
fix: allocate share-link layer tokens without collisions (addresses #658)
2026-09-27 13:14:59 -07:00
manjunath22466 d0105f1e9d fix: reconcile share-link token allocation with main
Preserve all published token assignments and add regression coverage for
all-enabled URL restoration, longer token lists, and manual provisional
token recovery after competing merges.

Builds on daikaginza's digits-first and merge-time allocation proposal.
Thanks to KnottyDyes for the review and regression-test suggestions.
2026-09-25 22:33:03 +05:30
Sameh Khamis b210ab0fe4 Merge pull request #643 from ch-bas/fix/heading-confidence-consumers
fix(cctv): surface heading confidence — estimated bearings are now visible (fixes #639)
2026-09-24 22:50:38 -07:00
Sameh KhamisandClaude Opus 5.5 00235365c3 build: register headingConfidence.js with the CCTV package boundaries (#643)
main's boundary manifest now requires every module to be owned; add the new
helper beside presentation.js in cctv-layer and the two application groups.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:46:58 -07:00
Sameh KhamisandClaude Opus 5.5 303429a421 docs: record estimated CCTV bearings in CHANGELOG and CURRENT-STATE (#643)
Credit the contributor and describe the HUD tag, dashed wireframe and
calibration precedence for #639.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:44:30 -07:00
Sameh KhamisandClaude Opus 5.5 9a971fef50 Merge main into pr-643
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:44:16 -07:00
Sameh Khamis 818b654812 Merge pull request #665 from daikaginza/fix/traffic-road-error-diagnosis
fix(traffic): say which upstream declined a road load (refs #661)
2026-09-24 22:41:49 -07:00
Sameh KhamisandClaude Opus 5.5 31224d8bd2 fix(traffic): name the proxy's own 502/503 accurately (#665)
502 (every mirror unreachable) and 503 (the local limiter, before any mirror
is asked) come from our proxy, so they no longer read as a mirror refusing
the query. Credit the fix in CHANGELOG and note the mapping in CURRENT-STATE.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:36:53 -07:00
Sameh KhamisandClaude Opus 5.5 5472e54f73 Merge main into pr-665
Resolve the CHANGELOG conflict by keeping both sides' entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:36:35 -07:00
Sameh Khamis 80126a7aaa Merge pull request #594 from Lob26/fix/terrain-height-cache-bound
fix(terrain): bound the client terrain-height cache with LRU eviction (closes #327)
2026-09-24 22:28:13 -07:00
Sameh KhamisandClaude Opus 5.5 95f0664e4e docs: credit the terrain-height cache bound in CHANGELOG (#594)
Move the entry to the top of the changelog, where main's newer entries had
left it behind, and credit the contributor.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:23:48 -07:00
Sameh KhamisandClaude Opus 5.5 714d3dd1bb Merge main into pr-594
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:23:42 -07:00
Sameh Khamis a96cb09bc9 Merge pull request #688 from ethanstoner/fix/cctv-media-idle-deadline
fix(cctv): bound media bodies with an idle deadline
2026-09-24 22:21:53 -07:00
Sameh KhamisandClaude Opus 5.5 12bf66ef6f docs: credit the CCTV idle-deadline fix in CHANGELOG (#688)
Move the entry to the top of the changelog, where main's newer entries had
left it behind, and credit the contributor.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:17:28 -07:00
Sameh KhamisandClaude Opus 5.5 9b1c75e539 Merge main into pr-688
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:17:12 -07:00
Sameh Khamis e16a1a7f7e Merge pull request #722 from Raushankumar0720/fix/track-proxy-oversized-response
fix(live): return 502 and stop caching oversized track responses as 200 (fixes #720)
2026-09-24 22:14:01 -07:00
Sameh KhamisandClaude Opus 5.5 3a23dc393d fix(live): cache the oversized-track 502 like other upstream errors (#722)
Keep the new 502, but store it in the 60 s cache as the proxy already does
for 404/429: /tracks/all costs OpenSky credits, so an oversized track must
not be re-downloaded on every reselect. Update the regression test and
credit the fix in CHANGELOG.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:09:17 -07:00
Sameh KhamisandClaude Opus 5.5 0d58d6e1c4 Merge main into pr-722
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:08:54 -07:00
Sameh Khamis d508a406dc Merge pull request #694 from abogaart/fix/macos-chmod-absolute-path
fix: spawn Apple /bin/chmod by absolute path when hardening credentials
2026-09-24 22:06:27 -07:00
Sameh KhamisandClaude Opus 5.5 f631d5428b docs: changelog entry for the macOS chmod fix (#694)
Credit the fix in CHANGELOG and tidy the wording of its inline comment.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:01:56 -07:00
Sameh KhamisandClaude Opus 5.5 fb8a93752f Merge main into pr-694
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:01:50 -07:00
Sameh Khamis a28327b890 Merge pull request #705 from KnottyDyes/fix/ipad-metal-model-atmosphere
fix(render): keep Cesium's model atmosphere off Apple Metal
2026-09-24 21:59:30 -07:00
Sameh KhamisandClaude Opus 5.5 20a03aad87 fix(render): release the atmosphere probe's WebGL context
Lose the throwaway WebGL2 probe context right after the link probe instead
of leaving it for GC; browsers (Safari especially) cap live contexts. Note
that affected devices also lose globe-basemap fog, credit the changelog
entry and move it to the top.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 21:53:06 -07:00
Sameh KhamisandClaude Opus 5.5 351b5fcaa4 Merge main into pr-705
Resolve the CHANGELOG conflict with Fire Perimeters (#737) by keeping both
entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 21:52:39 -07:00
manjunath22466 54735a9e6f fix: allocate share-link layer tokens without collisions 2026-09-24 16:25:55 +05:30
Bilawal SidhuandBilawal Sidhu ce671ce500 fix(data): load bundled region packs in the browser; bound region lookups (#743)
The Natural Earth and neighborhood packs were loaded with a JSON-attributed
import(). The Vite dev server answers that request with a JavaScript module,
which the browser rejects, so every analyst region scope failed and fell
through to a geocode lookup that could take 12-28 s.

- src/data/bundledJson.js fetches the pack from a new URL(..., import.meta.url)
  asset in the browser; under Node the caller's attributed import runs.
- resolveRegionRingForQuery caps the geocode + admin-boundary rung at 3 s and
  returns { error: 'region-timeout' }; the lookup keeps running and fills the
  cache. The analyst query reports it as code 'region-timeout'. The Nepal
  locator opts out of the budget.
- Unit tests for the loader, the timeout and the analyst error; the behavior
  harness checks the Gulf of Mexico scope and retries Texas on region-timeout.

Co-authored-by: Bilawal Sidhu <[email protected]>
2026-09-24 01:52:41 -05:00
Raushankumar0720 b22dd34f45 Merge remote-tracking branch 'upstream/main' into fix/track-proxy-oversized-response 2026-09-24 09:19:33 +05:30
513dc87a0e feat(hud): Cyber HUD theme with GPU contact sonar (#706)
Add the opt-in Cyber HUD layout with coordinated map and cockpit panel
styling, GPU-only contact sonar (rings, range, power, opacity, sector) with
native-rendering fallback, and set_cyber_sonar voice controls. Sonar visual
style inspired by kk376's tactical naval sonar (#214).

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-23 20:31:56 -07:00
f913066cf2 feat(fires): MODIS in Active Fires and a Fire Perimeters layer (#737)
Adds MODIS NRT to Active Fires and a keyless Fire Perimeters layer (NIFC WFIGS) with
InciWeb incident links, served through a capped, cached /api/fire-perimeters proxy.

Co-authored-by: Gustavo Beneduzi <[email protected]>
Co-authored-by: James Cooke <[email protected]>
2026-09-23 21:21:52 -05:00
Bilawal SidhuandBilawal Sidhu 4b56d0e917 fix(radio): restore the Radio QA harness and the row refresh it depended on (#736)
The harness called the compatibility facade's private _refreshTogglePanel,
which the layer-lifecycle refactor removed; Transit and Directions call
refreshLayerStats() on the bare lifecycle, which lacked it, so their rows
only repainted on the next tick. Add refreshLayerStats() to the lifecycle
and use it from the harness. With the crash gone, two further checks ran:
the selected-entity paint wait now also waits for Radio's own reveal, and
on phones the title bar sits 16 px from the top so both broadcast waves
stay on-screen.

Co-authored-by: Bilawal Sidhu <[email protected]>
2026-09-23 19:51:56 -05:00
1fc795577a feat(sdr): local receivers — browser RTL-SDR, Local ADS-B layer and decoder feeds (#732)
Adds a Local ADS-B layer fed by your own receivers: a browser WebUSB
RTL-SDR (with FM listening) and local 1090/978 MHz decoder feeds
(LOCAL_RECEIVER_FEEDS, local and private addresses only). Local aircraft
draw in magenta with class, silhouettes, 3D models, trails and a position
sanity filter; the click card names the band and input that heard them.

Builds on Sameh Khamis's SDR prototype and the initial GEV port by
Mazeyar Moeini Feizabadi (#134); the decoder-feed design follows the
notes on #57. Closes #57.

Co-authored-by: Sameh Khamis <[email protected]>
Co-authored-by: Mazeyar Moeini Feizabadi <[email protected]>
2026-09-23 16:00:53 -05:00
Bilawal SidhuandBilawal Sidhu 143cffd7ca fix(ui): keep panel scroll position through rail layout passes (#731)
The right-rail layout pass lifts the Context, CCTV, Weather and Recent
Imagery panel scrollers to measure their natural heights, which clamps
their scroll offset to zero, but it only restored scrollers tagged
data-rail-scroller. A layer that refreshes its Layers-panel row every
second snapped the Context panel back to the top while scrolling. Tag
every lifted scroller and pin the pairing with a test that reads the
measuring CSS.

Co-authored-by: Bilawal Sidhu <[email protected]>
2026-09-23 15:55:41 -05:00
Bilawal Sidhu 005254fcd1 fix(weather): upright N. America cloud tiles; lift shells over coarse 3D tiles (#730) 2026-09-23 14:49:39 -05:00
Sean ArmstrongandClaude Opus 5 b456eb8ff1 fix(render): keep Cesium's model atmosphere off Apple Metal
iPad and iPhone show "An error occurred while rendering. Rendering has
stopped." on load. Cesium's AtmosphereStageVS calls
czm_computeGroundAtmosphereScattering with its `out` parameters bound
directly to varyings:

    czm_computeGroundAtmosphereScattering(
        v_positionWC, lightDirection,
        v_atmosphereRayleighColor, v_atmosphereMieColor,
        v_atmosphereOpacity);

ANGLE's Metal backend wraps each `out` argument in an ANGLE_out(...)
helper taking `thread T &`, but vertex-output members live in
`__metal_generic` and cannot bind to `thread`, so the program fails to
LINK and Cesium tears down the render loop.

This is the only shader in Cesium that binds varyings to `out` params --
SkyAtmosphere (czm_computeScattering) and the ground-atmosphere fragment
path both route through locals, so they are unaffected and stay on.

ModelSceneGraph.configurePipeline attaches the stage under a single
condition, `fog.enabled && fog.renderable`, so clearing `fog.renderable`
keeps the broken program from ever being built.

`renderable` rather than `enabled` is deliberate: Fog.update returns
early when `enabled` is false and forfeits the density that drives 3D
Tiles screen-space-error scaling, and visualSettings saves/restores
`fog.enabled` around IR boost, which would resurrect the broken shader
mid-session.

Detection is a WebGL2 link probe of the same out-param/varying pattern,
so a future driver fix silently restores the effect with no code change,
OR'd with iOS/iPadOS platform detection as a backstop for when no probe
context can be created. iPadOS desktop mode is identified by the
Macintosh UA token plus maxTouchPoints > 1, since real Macs report zero
and navigator.platform is deprecated.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HoRD3yHVgfTHeikTmsUoHf
2026-09-23 09:25:16 -06:00
Raushankumar0720 34d258b394 fix(live): return 502 and prevent caching on oversized track responses (fixes #720) 2026-09-23 15:52:21 +05:30
Bilawal Sidhuandmanjunath22466 082074a006 feat(layers): Recent Imagery — open satellite imagery for a selected box (#716)
Select a box on the world and browse the last 30 days of open satellite imagery over it, keyless: NASA GIBS Harmonized Landsat Sentinel-2 (30 m) and the VIIRS daily overview (250 m), with a day strip, IMAGE / VS BASEMAP / A / B modes with a swipe divider, PNG export and share links. The before/after swipe from the Nepal scene moves into shared modules so both use it.

Co-authored-by: manjunath22466 <[email protected]>
2026-09-22 20:34:11 -05:00
25491f7e33 feat(taps): shared address contract for receiver taps (#382)
* feat(taps): shared address contract for receiver taps

Receiver taps (#57 RTL-SDR, #56 Rayhunter, #7 TAK, #6 Meshtastic) differ from
every other provider in this tree in one way that matters: the upstream address
comes from the user at runtime rather than from a fixed constant, so each tap
route is an SSRF surface unless the address is constrained. Today each tap
would answer that question for itself, and they would disagree.

parseTapAddress() is the single place the rule lives:

  A tap may address loopback or an RFC1918 private network, plus the
  `localhost` and `*.local` (mDNS) names. Nothing else.

That is what a tap is for — reading your own device on your own network. A tap
pointed at a public address is either a mistake or the proxy being used as a
confused deputy, and both get the same answer. Link-local (169.254/16) is
excluded wholesale, which covers the cloud metadata address by construction
rather than by blocklist.

Deliberately NOT reusing isNonGlobalIpv4() from radio/stations.js: it reports
malformed input like 999.1.1.1 as non-global, which is the safe direction for
the radio proxy's "is this public enough to fetch?" test but inverts into
*allowing* garbage under a tap's "is this local enough to fetch?" test. A
positive allowlist cannot invert that way, and the tests pin that case.

Two documented non-goals so they don't read as oversights: no general hostname
resolution (a name other than localhost/*.local is rejected rather than
resolved-and-pinned, avoiding DNS-rebinding defence for no gain), and no IPv6
(zone-index handling is platform-specific and no tap has needed it).

Scoped to the address rule alone. The recorded-fixture harness lands with the
first tap, designed against a real recorded response rather than guessed at
with no consumer.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KGKBNNjPh4iQe8sviGZNPb

* style: adopt the tap address contract into the format scope

Mechanical. Adds server/providers/common/tap-address.js and its test to
scripts/format-scope.json and applies prettier, matching the six sibling
modules in server/providers/common/ that are already adopted. No behaviour
change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KGKBNNjPh4iQe8sviGZNPb

* docs(taps): state accurately which taps this contract governs

The header claimed #57, #56, #7 and #6 as consumers. Only #56 (Rayhunter)
actually is today: it takes ?base=host:port from the browser, which is the
shape that makes a route an SSRF surface. #57 would qualify if built
server-side.

The other three are different shapes, and forcing them through a private-only
check would be wrong rather than merely unnecessary:

- Meshtastic (#6) reads a fixed public broker, mqtt.meshtastic.org.
- WiGLE (#11) reads a fixed public API behind a token.
- TAK (#7) takes its host from operator-set env config rather than from the
  browser, and a TAK server is frequently a legitimate remote host.

No behaviour change; comment only.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KGKBNNjPh4iQe8sviGZNPb

* refactor(taps): move the tap address contract to src/data so both sides share it

The browser needs the same answer the server does — the server as a trust
boundary, the client to give immediate feedback on a typed address without a
round trip. Keeping the rule under server/providers/ would have meant the
client re-deriving it, which is how the Rayhunter branch already carried a
hand-synced copy of the regex with a comment admitting it.

#274 is open about eight independent copies of haversine, one of which had
silently swapped its arguments. Shipping a new duplicated validator across a
trust boundary in the same tree would be hard to defend.

server/providers/gbfs.js already imports src/data/gbfsSource.js, so shared
validation living in src/data with the server importing it is the established
direction here rather than a new one.

Pure move plus the path/docblock updates. No behaviour change; the 12 contract
tests are untouched and still pass.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KGKBNNjPh4iQe8sviGZNPb

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
Co-authored-by: Bilawal Sidhu <[email protected]>
2026-09-22 19:22:55 -05:00
Bilawal SidhuandMatt Van Horn 09a5eaa302 docs: bring the README, data sources and known issues up to date (#712)
Reflect the weather stack, analyst records and feed provenance, satellite pass predictions and live camera video that landed since the last full pass.

Co-authored-by: Matt Van Horn <[email protected]>
2026-09-22 18:31:02 -05:00
0251c74109 feat(weather): add globe wind, observed weather and cyclone context (#617)
Wind (NOAA GFS and ECMWF IFS: animated particles, colour field, a reading at map centre), observed weather (NOAA MRMS radar, GOES and NESDIS infrared clouds, lightning density) on one shared history timeline, NHC and CPHC cyclone advisories with tracks and cones, and a WEATHER panel in the right rail. On 3D Tiles the observed products render as raised shells with a sharper window around the view.

Co-authored-by: Gustavo Beneduzi <[email protected]>
Co-authored-by: Daniel Slay <[email protected]>
Co-authored-by: Rehaan Delmotra <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
2026-09-22 17:45:39 -05:00
Bilawal Sidhu f01b6a5d84 fix(ui): stop panel glows from adding scrollbars to the narrow-screen rails (#699)
At 720px and below both panel stacks scroll vertically, so each panel's absolutely positioned glow (negative inset) became scrollable overflow on both axes and drew a horizontal scrollbar band plus a spurious vertical scrollbar whenever CCTV, Context, Data Layers or Scenes was expanded. Pin the glow to its panel box in the narrow-screen block and add a stylesheet test that checks every 720px media block.
2026-09-21 20:19:55 -05:00
Arthur Bogaart 2a47bdb8e7 fix: spawn Apple /bin/chmod by absolute path when hardening credentials
The macOS credential hardener strips ACLs with `chmod -N`, spawning
`chmod` by bare name. On machines where Nix home-manager or Homebrew
coreutils sit ahead of /bin on PATH, that resolves to GNU chmod, which
has no `-N` flag and exits non-zero. The hardener treated that as a
failure and refused every save from the Provider Settings panel with
"could not restrict the credential file to your account; nothing was
saved".

Spawn /bin/chmod by absolute path instead, matching how the Windows
branch already resolves its native tools without consulting PATH, and
add a regression test that pins the absolute path.
2026-09-21 17:28:23 +02:00
daikaginza c24399e99d fix(traffic): say which upstream declined a road load
Street Traffic draws on OpenStreetMap for geometry and TomTom for flow,
and the row said "Road data temporarily unavailable" whichever one
failed. The reporter of #661 went and verified a TomTom key while every
public Overpass mirror was refusing their network with HTTP 406 — a
status fetchRoads already had and dropped on the floor.

Classify the refusal where the layer talks to Overpass: 429 is
"Overpass rate-limited", 504 is "Overpass timed out", anything else is
"Overpass refused the road query (HTTP <status>)". The first two are the
words alpr's Overpass source already uses; the status is what says which
upstream declined, which a single-source layer never has to.

The error carries its status as a RoadRequestError, following
LiveSourceError in src/sources/live/contract.js, so a caller can branch
on the code without parsing English. The catch promotes a classified
refusal and keeps the general line for everything else — the branch is
on the error type, not on its text, because a dropped socket throws
whatever the platform felt like saying and "Failed to fetch" on a panel
row helps nobody.

Retries, backoff, caching and the mirror rotation are untouched.
2026-09-21 10:49:37 +07:00
Ethan Stoner 899ca4f56e fix: bound CCTV media bodies with an idle deadline
The media relay only deadlined the wait for upstream response headers. Once
headers arrived the timer stopped, so an upstream that answered and then sent
nothing held both the client response and the upstream socket open for as long
as the camera host allowed. The declared Content-Length ceiling was the only
body bound, and a chunked or length-less body had none.

A 30-second idle deadline now measures the gap between upstream chunks and
tears the stream down through the existing release path. Because a paused pipe
also stops upstream bytes arriving, the deadline reschedules itself whenever
the response is still waiting to drain, so a viewer on a slow link is not read
as a dead camera. Live feeds that keep delivering frames are unaffected.
2026-09-20 20:38:59 -07:00
Sameh Khamis 0dbde1e36c Merge pull request #284 from jrmagnus/fix/trackpad-pinch-zoom
Fix trackpad pinch-to-zoom on macOS
2026-09-20 20:01:25 -07:00
Sameh Khamis b641ad2019 fix: integrate bounded trackpad pinch zoom 2026-09-20 19:49:07 -07:00
Sameh Khamis 97927302bd Merge origin/main into fix/trackpad-pinch-zoom
# Conflicts:
#	src/main.js
2026-09-20 19:44:57 -07:00
Sameh Khamis a8e1c0965b Merge pull request #310 from SurefireStudios/docs/qa-gates
docs: point contributors at the per-feature regression gates (#271)
2026-09-20 19:40:02 -07:00
Sameh Khamis d01d10393c docs: distinguish QA gates from helper modules 2026-09-20 19:35:56 -07:00
Sameh Khamis 4f941266c2 Merge origin/main into docs/qa-gates 2026-09-20 19:35:29 -07:00
Sameh Khamis 3d640303f1 Merge pull request #354 from Lob26/fix/realtime-debug-log-hardening
fix(realtime): bound the debug-log sink and stop blocking on its writes
2026-09-20 19:31:09 -07:00
Sameh Khamis dd627891f8 fix: complete debug-log request bounds 2026-09-20 19:21:22 -07:00
Sameh Khamis a948274326 Merge origin/main into fix/realtime-debug-log-hardening
# Conflicts:
#	CHANGELOG.md
#	src/tooling/localServices.test.mjs
2026-09-20 19:19:49 -07:00
Sameh Khamis de2a41927b Merge pull request #355 from Lob26/fix/sanitize-openai-proxy-errors
fix(server): stop relaying upstream and JS error text to the client
2026-09-20 18:05:45 -07:00
Sameh Khamis 2608f187d3 fix: sanitize rejected Realtime token responses 2026-09-20 17:45:43 -07:00
Sameh Khamis a88904a02f Merge origin/main into fix/sanitize-openai-proxy-errors 2026-09-20 17:42:54 -07:00
Sameh Khamis 3586a904d6 Merge pull request #392 from Ashfaqbs/fix/antimeridian-traffic-bounds
fix: handle antimeridian crossing in clampBoundsAroundCenter
2026-09-20 17:16:02 -07:00
Sameh Khamis 62abcfdd16 fix: keep antimeridian traffic requests in range 2026-09-20 17:11:27 -07:00
Sameh Khamis 2f16e34cad Merge origin/main into fix/antimeridian-traffic-bounds 2026-09-20 17:09:53 -07:00
Sameh Khamis a5f820bb0d Merge pull request #444 from vortsghost2025/fix/issue-362-opensky-doctor
fix: report OpenSky anonymous mode in setup doctor
2026-09-20 16:47:59 -07:00
Sameh Khamis 9ad64112dc fix: report OpenSky Basic and auto modes accurately 2026-09-20 16:43:16 -07:00
Sameh Khamis a716b059b8 Merge origin/main into fix/issue-362-opensky-doctor 2026-09-20 16:43:01 -07:00
Sameh Khamis ebca84e87a Merge pull request #649 from theerayutbo/fix/ais-unavailable-sog-cog
fix: treat AIS "not available" speed and course as unknown
2026-09-20 16:38:40 -07:00
Sameh Khamis 5d684c22b3 Merge origin/main into fix/ais-unavailable-sog-cog
# Conflicts:
#	CHANGELOG.md
#	docs/CURRENT-STATE.md
2026-09-20 16:33:53 -07:00
Sameh Khamis 678763d3c0 Merge pull request #664 from Udaydsmls/fix/ui-dark-select-palette
fix: render native select option lists in the dark palette
2026-09-20 16:26:20 -07:00
Sameh Khamis 6eeb2fce11 Merge origin/main into fix/ui-dark-select-palette
# Conflicts:
#	CHANGELOG.md
2026-09-20 16:21:29 -07:00
Sameh Khamis d15cc383c3 Merge pull request #622 from cyberspace-cs/docs/add-contributing-nav-link
docs: add Contributing link to top navigation bar
2026-09-20 16:01:24 -07:00
Sameh Khamis 79a828bb90 Merge branch 'main' into docs/add-contributing-nav-link 2026-09-20 15:56:45 -07:00
Sameh Khamis 7ef9b885ee Merge pull request #669 from daikaginza/docs/adsbdb-attribution
docs(attribution): credit adsbdb and carry its route-data terms
2026-09-20 15:53:51 -07:00
Sameh Khamis 9f8f6284b5 docs(attribution): clarify adsbdb data-use wording 2026-09-20 15:49:22 -07:00
Sameh Khamis 581afbecea Merge origin/main into docs/adsbdb-attribution
# Conflicts:
#	CHANGELOG.md
2026-09-20 15:48:49 -07:00
Sameh Khamis fb7e1ef568 Merge pull request #677 from GGPOShadows/pr/tomtom-budget
fix(tomtom): size the daily tile budget to the real free allowance
2026-09-20 15:43:10 -07:00
Sameh Khamis 783f68a36a Merge branch 'main' into pr/tomtom-budget 2026-09-20 15:39:17 -07:00
Sameh Khamis 7fc8cd9fd0 Merge pull request #678 from GGPOShadows/pr/gitignore-env-variants
fix(security): ignore the whole dotenv ladder, not just .env
2026-09-20 15:33:53 -07:00
GGPOShadowsandClaude Opus 5 82f3516166 fix(security): ignore the whole dotenv ladder, not just .env
.gitignore listed only the bare filename `.env`, but the app reads four
credential files, not one. scripts/setup-doctor.mjs:92 walks
`['.env', '.env.local', `.env.${mode}`, `.env.${mode}.local`]`, and
server/standalone/vite.config.js:11 calls Vite's `loadEnv(mode, root, '')`,
which reads the same ladder.

gitignore matches literally, so three of those four files were
committable. A key placed in .env.local is fully live for the app --
the doctor reports it as configured and the provider proxies use it --
while `git add -A` stages it. Verified on a clean checkout of main:

    .env                    ignored
    .env.local              COMMITTABLE
    .env.production         COMMITTABLE
    .env.production.local   COMMITTABLE

This matters more than usual here because the in-app Provider Settings
panel trains people to keep real keys in the checkout rather than in a
shell profile, and because the repository has ~7.7k forks that inherit
the same gap.

Replaces the single `.env` entry with `.env` plus `.env.*`, and re-includes
`.env.example` with a negation so the tracked template is unaffected.

Deliberately scoped to the dotenv ladder alone. Other credential-bearing
artifacts (a repo-local OPENSKY_CREDENTIALS_FILE, *.pem/*.key) are a
separate question and are left for a follow-up rather than widened here.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-19 22:55:56 -07:00
GGPOShadowsandClaude Opus 5 5e99adc978 fix(tomtom): size the daily tile budget to the real free allowance
TOMTOM_DAILY_TILE_BUDGET defaulted to 40,000 per day against a free
tier that is granted per MONTH, not per day. TomTom currently allows
200,000 Traffic Flow & Incidents tile requests per month
(https://docs.tomtom.com/pricing/), so the default exhausted a user's
entire monthly allowance in five days and left the traffic layer dead
for the rest of the billing period.

The docs contradicted themselves on this. traffic.js described the
free tier as "~50k/day", while .env.example cited the correct 200K
per month two lines below quoting the same 40,000 default. 50k/day
would be 1.5M/month, 7.5x the actual allowance.

Sets the default to 6,000/day. A 31-day month lands at 186,000,
inside the allowance with headroom. Corrects the stale figures in
traffic.js, .env.example and DATA_SOURCES.md.

This remains an application-side safety ceiling, not a billing cap,
and the surrounding text still says so. TomTom's free tier requires
no card, so overrunning it cuts the layer off rather than generating
a charge — but a layer that dies three weeks into every month is
still worth avoiding by default.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-19 22:44:19 -07:00
daikaginza ae35a2c939 docs(attribution): credit adsbdb and carry its route-data terms
adsbdb supplies the aircraft type, model name and registration behind every
enriched flight, and the airline and origin/destination pair behind the
tracked contact's FROM -> TO strip. It appeared in neither attribution
surface: DATA_SOURCES.md and src/data/dataCredits.js each held zero
occurrences of it, against the rule both files state for themselves and
CONTRIBUTING's fourth pull-request item. adsb.lol is a different operator and
its ODbL credit does not reach this data.

The reporter of #256 left the patch off deliberately, saying the terms were
what should not be guessed. They are published, so this takes them from the
source rather than inventing them: adsbdb credits PlaneBase for the aircraft
data and Guillaume Michel for the ICAO-to-N-number conversion, and states that
the flight route data "is the work of David Taylor, Edinburgh and Jim Mason,
Glasgow, and may not be copied, published, or incorporated into other
databases without the explicit permission of David J Taylor, Edinburgh". No
licence or rate limit is published for the service itself.

That restriction is the reason the note records what the proxy does with what
it fetches: responses are cached per key for 24 hours in .gev-cache/adsbdb.json,
which is gitignored, so the data stays on the operator's machine and is never
bundled with this repo or redistributed.

A test pins the credit. The attribution is a requirement rather than a
preference, so it should fail loudly if a later edit drops the names or the
restriction, instead of surviving on luck.
2026-09-19 20:12:36 +07:00
Uday S 59b2637fb7 fix: render native select option lists in the dark palette 2026-09-19 00:06:19 -04:00
Theerayut BoonkirdandClaude Opus 5 75869d0c10 chore: adopt the new AIS test into the explicit formatting scope
scripts/format.mjs skips *.test.mjs during runtime discovery, so a test
added under a discovered root is never seen by format:check until it is
listed in scripts/format-scope.json — which is what CODE-BOUNDARIES.md
means by adding tests to the explicit list as they are adopted. The file
went in unformatted and the check stayed green, hiding it.

Listed and formatted; format:check now covers 917 files instead of 916.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-18 00:43:02 +01:00
Theerayut BoonkirdandClaude Opus 5 b0f5072347 docs: record the AIS unavailable-code handling
CONTRIBUTING asks for docs/CURRENT-STATE.md and CHANGELOG.md in the same PR
as a runtime behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-18 00:41:06 +01:00
Theerayut BoonkirdandClaude Opus 5 88f6e8bcf8 fix: treat AIS "not available" speed and course as unknown
ITU-R M.1371 encodes speed over ground in 0.1-knot units and course over
ground in 0.1-degree units, reserving the top code of each field for "not
available": SOG 1023 reaches us as 102.3 knots and COG 3600 as 360 degrees.
Both are finite, so numberValue passed them through as ordinary readings and
every consumer — vessel cards, the HUD, voice analyst queries — treated a
missing measurement as a real one.

Observed on a live AISStream feed of 3,062 vessels: exactly 3 records at
102.3 knots with no reading anywhere between 40 and 102.3, and 404 records
at exactly 360 degrees. One Dutch rescue boat moored at its station carried
both sentinels at once, so an analyst summary reported a maximum speed of
102.3 knots — faster than any vessel afloat.

Filter both at the store, where AISStream messages are decoded, so the
sentinels never enter the cache. Heading already had this treatment through
normalizedHeading with its own separate sentinel of 511.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-09-18 00:41:06 +01:00
bassem chagra 5f27f6edf4 fix(cctv): surface heading confidence — estimated bearings are now visible
Every pack sets headingConfidence on each camera, but nothing consumed it:
a bearing synthesized by fallbackHeadingFromId (a hash of the id string,
no geographic input) rendered exactly like a surveyed one — 2,827 of 3,997
cameras in a default catalog. Fixes #639.

Two consumers, plus a passthrough:

- HUD: the heading token is tagged when the bearing is synthetic —
  `HDG 194° (ESTIMATED)` — so a hash never reads as a measured facing.
- Wireframe: a low-confidence camera draws its coverage frustum with dashed
  polylines instead of solid ones. Colors, widths, and the active/idle
  emphasis are unchanged; the dash is the only signal.
- Public camera state gains `headingConfidence` (the raw pack flag) and
  `headingEstimated` (the derived bit) so UI consumers never re-derive it.

A human-vouched pose always wins over the pack flag: a manually saved
calibration (calSource:'manual') or a curated catalog entry
(poseSource:'curated') is never presented as estimated, mirroring the CAL
badge states in calibration.js.

- src/layers/cctv/headingConfidence.js: pure helpers (isHeadingEstimated,
  headingHudToken)
- src/layers/cctv/headingConfidence.test.mjs: confidence normalization,
  calibration/curation overrides, HUD token formatting
- src/layers/cctv/presentation.js: HUD token + public-state passthrough
- src/layers/cctv/rendering.js: dashed materials for estimated bearings
2026-09-17 10:20:26 +02:00
SurefireStudios 4b161ea7af docs: point contributors at the per-feature regression gates (#271)
CONTRIBUTING.md named only npm run build, npm test and npm run test:track,
so a contributor following it literally never learns that most features
have a dedicated gate in scripts/qa-*.mjs. Today 25 of the 40 gates are
not named in CURRENT-STATE.md either, so for those there was nothing in
the written guide to find them by.

Point at the directory rather than transcribing a list that would go stale
the same way: every gate already opens with a comment saying what it
asserts and how to run it, so `ls scripts/qa-*.mjs` plus the script header
is the canonical, self-maintaining reference.

Also state plainly that CI runs neither test:track nor any qa gate, since
that is what makes running them locally matter.
2026-09-16 14:35:31 -07:00
Sameh Khamis 0d41b6be54 Merge pull request #626 from bilawalsidhu/fix/vessel-partial-feed-status
fix: distinguish partial vessel snapshots from stale data
2026-09-16 11:57:56 -07:00
Sameh Khamis a25ee64209 fix: distinguish partial vessel snapshots from stale data 2026-09-16 11:52:26 -07:00
cyberspace-cs 0dcb02c6da docs: add Contributing link to top navigation bar
Make the contribution guide discoverable from the README's main
navigation alongside Quick Start, Under the Hood, and Keys & Costs.
2026-09-17 00:06:33 +08:00
Sameh Khamis 531ef52ca4 Merge pull request #600 from manjunath22466/fix/pinokio-nepal-media-fallback
fix: prevent Nepal media from opening external browsers in Pinokio
2026-09-16 08:41:06 -07:00
Ashfaq dadaaf1f27 Merge remote-tracking branch 'origin/fix/antimeridian-traffic-bounds' 2026-09-16 20:02:45 +05:30
Ashfaq 830c372c12 fix: keep clamped traffic bounds monotonic across the antimeridian
The previous fix wrapped the span correctly but normalized the
returned west/east back into [-180, 180), which re-inverts the box
whenever it crosses the antimeridian (west > east again). Downstream
consumers (getBoundsCenter, boundsOverlap) assume west <= east and
silently misbehave on an inverted box: the fetch center lands on the
opposite side of the planet, and the overlap gate never skips a
re-fetch.

Drop the normalization and leave west/east unnormalized instead, so
the box stays monotonic. The only remaining edge case (coordinates
outside the conventional range reaching the Overpass query builder)
is a visible, separate problem rather than a silently wrong center.

Addresses review feedback on #392.
2026-09-16 20:02:26 +05:30
manjunath22466 621d186de1 fix: scope Nepal media playback to explicit scene actions 2026-09-16 12:14:15 +05:30
manjunath22466 cf0b9927de Merge public main into Nepal media follow-up 2026-09-16 10:40:29 +05:30
Sameh Khamis a65d9d85f1 Merge pull request #616 from bilawalsidhu/feat/director-sharing
Add Director authoring, import previews and scene bundles
2026-09-15 20:19:57 -07:00
Sameh Khamis 078fc290ec Add Director authoring, import previews and scene bundles 2026-09-15 20:15:32 -07:00
Sameh Khamis 62f66c4b86 Merge pull request #614 from bilawalsidhu/feat/director-interactions
feat: add declarative Director feature actions
2026-09-15 19:20:40 -07:00
Sameh Khamis 80027756ef feat: add declarative Director feature actions 2026-09-15 19:16:49 -07:00
manjunath22466 94f83c59de fix: prevent Nepal media from opening external browsers in Pinokio 2026-09-15 23:54:07 +05:30
Pedro Lobato 3308569edd docs(cctv): the media-fetch log line drops the error text on purpose
The comment claimed "Keep the detail in the log" while the line it introduces
carries none, so the next reader would have read the omission as an oversight
and added the errno back.

Detail-free is the deliberate choice: a media-fetch failure names the camera's
upstream host, and these lines get pasted into issues. Say that, and say why
the catch at the bottom of this file may still log its error text.
2026-09-15 12:06:11 -05:00
Pedro Lobato 92350b8412 Merge remote-tracking branch 'origin/main' into fix/sanitize-openai-proxy-errors
# Conflicts:
#	CHANGELOG.md
2026-09-15 11:58:25 -05:00
Pedro Lobato 4fd168ff3c Merge remote-tracking branch 'origin/main' into fix/realtime-debug-log-hardening
# Conflicts:
#	CHANGELOG.md
2026-09-15 11:49:03 -05:00
Pedro Lobato fa9af39e7b fix(terrain): bound the client terrain-height cache with LRU eviction
The in-memory cache in the terrain-height resolver had no eviction, so a
session retained every coordinate it ever resolved for the lifetime of the
page. The 5-decimal rounding documented in the file header makes repeated
visits to one place share a key, but it does not bound how many distinct
places a session visits.

Cap it at 20 000 entries, overridable per instance so the bound is testable
without minting twenty thousand coordinates. Eviction is least-recently-used
rather than insertion-ordered because revisits here are spatial: the cell the
camera is parked on must not be dropped for having been resolved early.

The bound sits far above a city-scale session at the ~111 m floor grid, so
normal use never evicts and issues no additional proxy requests. Batch results
are assembled from what the call resolved, so eviction during a large batch
cannot report a just-resolved point as unresolved.
2026-09-15 11:33:56 -05:00
Pedro Lobato 44817e1286 fix(realtime): bound the debug-log sink and stop blocking on its writes
Ported onto the split provider layout: the route now lives in
server/providers/openai/debug-log.js and its state belongs to the handler
the factory returns rather than to module scope.

/api/realtime/debug-log is the one route in its family left unhardened.
Both siblings enforce a limiter through openai/rate-limit.js, and this was
the only handler still writing with fs.appendFileSync on the request path.

- Always-on per-IP limiter (120/min, 400 global). Deliberately not the
  opt-in openAiRateLimiter(): that one defaults to unlimited so throttling
  cannot silently change what a user spends, and this sink spends disk and
  event-loop time rather than provider quota.
- Serialized async appends with a 32 MB rotation ceiling and one retained
  generation. REALTIME_DEBUG_LOG_MAX_BYTES caps a single request body; it
  never bounded the file those bodies accumulate into, so any local page
  could grow .gev-logs/realtime-conversations.jsonl for as long as the dev
  server ran. Serializing also closes a read-then-write race on the size
  check, where concurrent callers all stat under the ceiling and none
  rotates.
- Generic 400/500 answers separating a malformed record or oversized body
  from a failed write, instead of echoing error.message — an errno and an
  absolute path for an fs failure.
- readRequestBody marks an oversized body with code 'BODY_TOO_LARGE', the
  marker its sibling readRequestBodyCapped already sets, so the two cases
  are distinguishable without matching on text.

Coverage extends src/tooling/localServices.test.mjs, which already drives
the real handler over a temporary sourceRoot. Both new tests fail on the
previous implementation and pass with this one.

Refs #23. Does not add authentication: these proxies have no auth
mechanism today, so introducing one is a maintainer design decision.
2026-09-14 21:24:12 -05:00
Pedro Lobato 9e71081a5a fix(server): stop relaying upstream and JS error text to the client
Ported onto the split provider layout: the routes now live in
server/providers/openai/{hud-summary,realtime}.js and the camera health
entry in server/providers/cctv.js.

The routes #263 names — celestrak, terrain, adsbdb — were sanitized by
#289. The same defect had moved:

- /api/openai/hud-summary relayed `data.error.message` verbatim on a
  non-ok upstream. That is OpenAI's own error text, carrying request ids,
  organization hints and quota wording. Its catch echoed the JS error too.
- /api/realtime/token echoed the JS error when minting failed, which for a
  network fault is a resolver message naming the upstream host.
- cctvProxy stored the raw error as a camera's health `message`. This is
  the least obvious of the three and the reason the PR is worth its size:
  GET /api/cctv/health serializes `message`, and src/layers/cctv/frames.js
  renders it as the camera's status label, so the detail reached the
  screen through a different door than the sanitized 502 beside it.

Logs name the failure and the upstream status without the text, matching
the Launch Library and terrain paths; these lines get pasted into issues.

Coverage extends the existing harnesses rather than adding files —
localServices.test.mjs drives both OpenAI routes over a mocked upstream,
mediaProviders.test.mjs drives a failing media fetch and then reads
/health. All three assertions fail on the previous implementation.

Refs #263. Does not touch /api/realtime/debug-log's identical echo; that
route is the subject of the #23 PR, so the two do not overlap a line.
2026-09-14 21:24:05 -05:00
DeliberateEnsemble 7da812eddd fix: report keyless OpenSky access in setup doctor 2026-09-13 23:06:51 -04:00
Ashfaq 5c2382dd37 Merge branch 'main' into fix/antimeridian-traffic-bounds 2026-09-14 07:07:43 +05:30
Ashfaq c22ad6af32 fix: handle antimeridian crossing in clampBoundsAroundCenter
east - west assumed a linear longitude axis, so a viewport crossing
the antimeridian (west=179.98, east=-179.98) produced a negative span
and unnormalized output coordinates outside [-180, 180). Wrap the span
through 360 and normalize the clamped west/east back into range.

Fixes #352.
2026-09-13 10:15:55 +05:30
jrmagnus a8fae8d87a fix: enable trackpad pinch-to-zoom on the globe
macOS reports trackpad pinch as wheel events with ctrlKey=true, but
Cesium's default zoomEventTypes only binds plain WHEEL (no modifier),
so pinch was silently ignored — only two-finger scroll worked.

Adds the CTRL-modified WHEEL binding so Cesium recognizes pinch as a
zoom gesture, plus a capture-phase relay that amplifies the pinch
event's deltaY before Cesium processes it, since macOS reports much
smaller deltaY per unit of finger movement for pinch than for scroll
(same fixed Cesium formula otherwise made pinch feel unresponsive).
2026-09-10 23:23:16 -03:00
995 changed files with 133183 additions and 4058 deletions

No files matched your search

+46 -6
View File
@@ -64,6 +64,13 @@ OPENAI_REALTIME_REASONING_EFFORT=low
# Keep only a short conversational window. Current map state is fetched live.
OPENAI_REALTIME_CONTEXT_TOKENS=3000
OPENAI_REALTIME_CONTEXT_RETENTION=0.5
# Voice-card captions of what you said. Billed separately by OpenAI and
# counted in the session cost meter and cap; off disables them.
OPENAI_REALTIME_TRANSCRIBE_MODEL=gpt-4o-mini-transcribe
# The voice debug log (.gev-logs/realtime-conversations.jsonl, rotated at
# 32 MB, never expired) omits transcripts and typed or spoken text by default.
# Set to 1 to keep them while debugging; delete the log when you are done.
GEV_VOICE_LOG_CONTENT=
OPENAI_HUD_SUMMARY_MODEL=gpt-5-nano
# Optional: opt-in per-IP rate limit for the OpenAI cost endpoints
# (/api/realtime/token + /api/openai/hud-summary), in requests per minute.
@@ -129,18 +136,27 @@ AISSTREAM_API_KEY=
# 0 disables the silence watch entirely.
# AISSTREAM_SILENCE_TIMEOUT_MS=120000
# Optional: TomTom live traffic flow (BYOK freemium — free tier is ~50,000 tile
# requests/day, get a key at https://developer.tomtom.com). SERVER-SIDE ONLY:
# Optional: local ADS-B decoder feeds for the Local ADS-B layer. SERVER-SIDE
# ONLY. Comma-separated band=url entries; band is 1090 or 978 (UAT). Each host
# must be loopback, a private (RFC1918) address, localhost or a *.local name;
# the path must end in aircraft.json. Invalid entries are logged and skipped.
# See docs/LOCAL-RECEIVERS.md.
# LOCAL_RECEIVER_FEEDS=1090=http://localhost:8080/data/aircraft.json,978=http://localhost:8978/data/aircraft.json
# Optional: TomTom live traffic flow (BYOK freemium — the free tier is 200,000
# Traffic Flow & Incidents tile requests per MONTH, not per day; no credit card
# required. Get a key at https://developer.tomtom.com). SERVER-SIDE ONLY:
# the browser fetches same-origin /api/tomtom/* and never sees the key.
# Keyless fallback: the traffic layer runs its built-in simulation (white dots,
# hardcoded per-road-class speeds) — no key required for the layer to work.
# TOMTOM_API_KEY=
# Optional soft cap on upstream tile fetches per UTC day (default 40000) — a
# Optional soft cap on upstream tile fetches per UTC day (default 6000) — a
# configurable application safety ceiling, not a guarantee of staying within
# TomTom's free allowance (currently 200K tile requests/month:
# https://docs.tomtom.com/pricing/). Over the cap the proxy serves cached/stale
# tiles instead of hitting upstream.
# TOMTOM_DAILY_TILE_BUDGET=40000
# https://docs.tomtom.com/pricing/). 6000/day keeps a 31-day month at 186K,
# inside that allowance. Over the cap the proxy serves cached/stale tiles
# instead of hitting upstream.
# TOMTOM_DAILY_TILE_BUDGET=6000
# Optional: CCTV layer tuning (advanced). Defaults are sensible — leave unset
# unless you're customizing the camera source pack.
@@ -174,3 +190,27 @@ AISSTREAM_API_KEY=
# CCTV_MAX_SOURCES=4000 # catalog-wide ceiling, shared round-robin across packs
# CCTV_PREFER_AUSTIN=1 # prioritize Austin cameras in the registry
# CCTV_FORCE_AUSTIN= # force live default sources when a source-pack JSON is configured
# Optional: comma-separated Overpass URLs; use only instances you run or pay for.
OVERPASS_UPSTREAMS=
# Optional: place-lookup endpoints. Lookups try bundled data first, then the
# endpoint set here, or the public service under its usage limits when unset.
# NOMINATIM_URL: search endpoint for last-resort place search and voice
# outlines (e.g. https://nominatim.example.org/search). A configured URL
# replaces the public service: nothing falls back to nominatim.openstreetmap.org,
# and an invalid value disables Nominatim. Unset uses the public
# nominatim.openstreetmap.org: one request at a time, 1.1 s apart, a short
# queue, cached answers (outlines 90 days, places 30 days, no-results 24 h in
# .gev-cache/nominatim) and a daily cap per install. Set it empty to disable.
# The public service's limit applies to all installs of this app together; the
# per-install cap bounds each install, not the total, so heavy or shared
# deployments should point this at their own server.
# NOMINATIM_URL=
# Public-instance requests per UTC day, outlines and search together. Not
# applied to your own NOMINATIM_URL.
# NOMINATIM_DAILY_CAP=50
# PHOTON_URL: keyless place search endpoint used by the browser
# (e.g. https://photon.example.org/api/). Unset uses photon.komoot.io; set it
# empty to disable.
# PHOTON_URL=
+5
View File
@@ -29,6 +29,7 @@ jobs:
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
persist-credentials: false
fetch-depth: 0
- name: Use Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
@@ -48,6 +49,10 @@ jobs:
- name: Check package boundaries
run: npm run check:boundaries
- name: Check published layer-token allocations
if: github.event_name == 'pull_request'
run: npm run layer-token:check -- --base-ref origin/main
- name: Run unit tests
run: npm test
+7
View File
@@ -1,6 +1,13 @@
node_modules
dist
# Credential files. `.env` alone is not enough: Vite's loadEnv (see
# server/standalone/vite.config.js) and scripts/setup-doctor.mjs both read the
# full dotenv ladder — .env, .env.local, .env.<mode> and .env.<mode>.local —
# so a key placed in any of them is live for the app. Only .env was ignored,
# leaving the other three committable. .env.example is the tracked template.
.env
.env.*
!.env.example
*.log
screenshots/
qa-shots/
+517 -3
View File
@@ -1,5 +1,482 @@
# Changelog
- The voice card shows analyst counts ("At least 250,000 aircraft …") with
their scope, feed window and caveats, and marks partial answers. Analyst
speech uses the same deterministic headline, preserves lower bounds and says
when an answer is partial. Follow-ups retain unanswered-layer coverage, and
keep the original scope label when they narrow a remembered result.
Contacts-window answers read one immutable panel snapshot, mark a 20,000-row
layer cap as a lower bound, and commit that displayed cohort only after
cancellation checks pass. Concurrent analyst calls from one model response
remain independent; a stopped or replaced user turn still cancels them.
- Voice answers are shorter and fill slow work instead of going silent. Slow
or multi-step requests may open with one short spoken plan; instant ones
just happen. Tool results carry a code-built `say` line and a `display`
card: sources and notes stay on screen, a caveat is spoken only when it
changes the answer, and "at least" and partial answers are always said. A card above the mic shows captions, the plan's steps and the
result. When a tool that reports steps (layers, place lookups, outlines,
nearest-aircraft) is still running after 1.5 s its current step is spoken
once; others say "still working" once after 8 s. A soft
tick plays in push-to-talk. Holding Space while the assistant talks cuts it
off. Each turn writes latency and speech figures to the debug log
(`node scripts/voice-turn-spans.mjs`). Captions of what you say are
counted in the voice cost meter; the debug log leaves out what was said
unless `GEV_VOICE_LOG_CONTENT=1`.
- Point and ask: hold Space with the cursor on an aircraft, ship or spot on
the map and say "what's that", "track that one" or "how many flights
around here". A reticle marks the point and the voice card shows what
"this" meant. In open mic it works when you have just moved the cursor
there. "The second one" picks from the last numbered list. Asking about
"the last one" resolves to the last of the five rows actually shown, and
referent-only tracking calls are accepted while empty targets are rejected.
Asking about
bare ground up close gives the assistant a small crop of the map around the
point.
- Public Overpass instances are no longer used by default. Street Traffic
roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen
on the layer row (TomTom / OSM / Hybrid) or with `?trafficRoads=`. With a
TomTom key the default is Hybrid: TomTom roads with live flow, plus
OpenFreeMap roads TomTom does not cover, simulated. Without a key every
choice draws OpenFreeMap roads. TomTom mode shows only roads with flow; OSM
mode matches TomTom flow onto OpenFreeMap roads for congestion colors,
speeds and closures, and unmatched roads stay simulated. Road failures name
OpenFreeMap and retain HTTP status or timeout reasons separately from TomTom.
Flow coverage counts
rendered dots on roads with flow; mapped military areas come from OpenFreeMap and
follow settled camera changes with cached tiles and retained site entities.
Superseded requests preserve the next view’s pending load.
Tile and API transports can be injected independently; the optional Overpass
ALPR adapter remains available.
OSM-derived layers share one attribution entry and a persistent inline map credit,
including routing, Warendorf cameras, datacenters, dams and the Nepal locator.
Military areas use bundled Overture/OSM names; wide views show bounded,
decluttered named points that follow camera motion while Contacts holds a subject,
using the shared world-overlay host for wide-point and polygon titles. Polygons appear while names load; cancelled views
cannot publish late names, and stalled name downloads can retry. Merged groups
keep their identity and parent name when a pan hides the largest member. Late
names reconcile ids across zooms while retaining selection and marker ownership. Wide
named points remain visible until matching close polygons take over. Installation
titles share datacenter/dam card arbitration and keep incumbents while panning.
Selecting an installation replaces its label with one card; deselection restores it.
Selected footprints receive a translucent fill draped on the active map surface.
ALPR cameras come from an hourly OpenStreetMap extract (US and Canada). Features
without a replacement say they are unavailable; area and footprint
annotations check once whether an Overpass instance is configured and skip
the query when none is. Area annotations outline countries, states and provinces
(Natural Earth, including UK constituent countries) and US counties (US Census Bureau) from bundled data, with no
lookup. County names are disambiguated across countries by aliases, qualifiers
and geography. `OVERPASS_UPSTREAMS` sets an
Overpass instance you run or pay for. The cockpit regional brief resolves
regions from bundled Natural Earth data instead of Nominatim. Traffic starts on enable and paints
locally grounded roads incrementally, shares concurrent tile requests and
reuses validated heights across pans. Camera moves and arriving tiles retain
existing traffic dots on their roads; budget changes fade in/out incrementally,
height refinements ease vertically, and live flow changes color and speed in
place. The traffic footprint follows the reticle square at oblique angles, with
cached detailed near tiles and coarse distant roads. Failed detail tiles retain
coarse roads and retry at most three times. Road-pass deadlines also cancel
queued tile paints and terrain callbacks;
off-screen height corrections cannot hold loading open, and visible corrections
stay pixel-limited at canvas and camera-plane boundaries. Completed road snapshots replace
streamed previews without duplicating roads. Unresolved previews no longer delay or suppress detailed street roads.
Road admission excludes private, parking, walking and unknown traffic categories;
OpenStreetMap service ways and tunnels no longer receive surface vehicles.
Detail tiles load alongside previews, arrival bypasses the movement debounce,
and geographically validated rendered depth avoids repeated mesh redraws.
Traffic releases surface-frame listeners on disable/destroy and reuses unchanged
surface identities without frame-by-frame allocations.
Grounded dots respect building occlusion instead of drawing through roofs,
and the traffic status chip reports completion without a delayed text animation.
Short road chunks and local surface elevation keep the budget on visible
streets in elevated cities. Congestion and closures respect travel
direction. ALPR distinguishes unsupported coverage from empty results and
asks for zoom-in before exceeding its tile budget. Installation footprints
retain visible fragments without joining separate parcels across zooms or
across quantization gaps between different mapped identities. Keyless
terrain tiles retry HTTP 429 and transient gateway failures with bounded,
shared backoff. Operator-configured Overpass area and footprint queries
retain relation member geometry so their outlines remain available.
Contacts filters mapped installations to a true 100 km surface radius around
the current tracked subject,
including in Cockpit, instead of "?" or nothing. Search arrivals frame against the resolved
ground and lift the eye above the rendered surface, so Camp Mabry and Denver
no longer land underground (Milan Khanal, #118). Arrival corrections yield to
new camera owners and are cancelled on layer/app teardown. ALPR floor preparation
and marker placement share terrain requests, cancelled when the layer is disabled.
Map-source changes reposition every ALPR marker without replacing entities. ALPR loads
whole-city views from z9-z12 tiles, drops native ground wedges and clamping,
keeps frame time with ALPR on within a few percent of ALPR off, seats nearby
badges on the rendered surface and says when no loaded camera is on screen.
- CCTV cameras whose bearing is a guess now say so. Packs mark bearings derived
from a hash of the camera id as `headingConfidence: 'low'`, but nothing read the
flag, so roughly 70% of a default catalog rendered like surveyed facings. The HUD
now reads `HDG n° (ESTIMATED)` and the coverage wireframe draws dashed; manual
calibrations and curated poses are never marked estimated (bassem chagra, #643).
- Report which upstream declined a Street Traffic road load. The layer row now
reads `Overpass rate-limited`, `Overpass timed out`, or
`Overpass refused the road query (HTTP 406)` instead of a general
"Road data temporarily unavailable", so a reader is not sent to check a
TomTom key when the public OpenStreetMap mirrors are the side that failed.
The proxy's own 502 (every mirror unreachable) and 503 (local limiter busy)
read `Overpass mirrors unreachable` and `Overpass temporarily unavailable`.
Failures the layer cannot classify keep the general line (daikaginza, #665).
- Bound the client terrain-height cache at 20 000 entries with least-recently-used eviction, so a long session no longer retains every coordinate it ever resolved. Consumer reads promote their entry and a batch still reports every point it resolved (Pedro Lobato, #594).
- Release CCTV media streams whose upstream falls silent after answering. The
15-second media deadline covered only the wait for response headers, so a
camera that replied and then stopped sending held both the proxy connection
and its upstream socket open for as long as the camera host allowed; the
declared `Content-Length` ceiling was the only body bound, and a chunked or
length-less body had none. A 30-second idle deadline now bounds the gap
between upstream chunks and releases a body that has gone silent. It is
rescheduled while the response is still waiting to drain, so a viewer on a
slow link is not mistaken for a dead camera; live feeds are unaffected
(Ethan Stoner, #688).
- Aircraft track backfill (`/api/opensky-track`, `/api/adsblol/trace`) now
answers 502 when the upstream body exceeds the 5 MB cap, instead of a 200
whose error body the client read as an empty track. The failure is cached
like other upstream errors, so retries inside the 60 s window do not spend
OpenSky credits (Raushankumar0720, #720, #722).
- Saving a key from Provider Settings works again on Macs where Nix or
Homebrew coreutils sit ahead of `/bin` on `PATH`. The credential hardener
now spawns Apple's `/bin/chmod -N` by absolute path; GNU `chmod` has no `-N`,
so the ACL strip failed closed and every save was refused (Arthur Bogaart, #694).
- Render on iPad and iPhone instead of stopping with "An error occurred while
rendering." Cesium's per-vertex model atmosphere binds shader `out`
parameters directly to varyings, which Apple's Metal/ANGLE backend cannot
link, so the program failed and the render loop was torn down. The stage is
now kept out of the pipeline on affected devices by clearing
`scene.fog.renderable`, which leaves `fog.enabled` — and the fog density that
drives 3D Tiles refinement — untouched. Detection is a WebGL2 link probe of
the same pattern, so a future driver fix restores the effect with no code
change, with iOS/iPadOS detection as a backstop. Sky atmosphere and the
ground-atmosphere fragment path route through locals and are unaffected;
affected devices lose distance fog on 3D tiles and on the globe basemaps.
`src/app/atmosphereCompat.test.mjs` pins the probe, the platform matrix and
the `renderable`-not-`enabled` choice (KnottyDyes, #705).
- Make share-link layer tokens durable allocations instead of ad hoc picks.
Existing one-character mappings are pinned permanently, while new layers use
the next free single-character digit, then two-character base-36 tokens.
Authors record each token in the permanent JSON ledger after running
`npm run layer-token:next -- <layer-id>` on a rebased branch.
`npm run layer-token:check -- --base-ref origin/main` guards published
assignments and allocation order in
pull-request CI. Existing v2 links keep their exact meaning.
Malformed enabled-layer lists now reject the whole layer payload instead of
restoring a partial list. A QA-only browser fixture exercises the future
two-character path after a fixture-only exhausted-digit scenario, without
reserving `00` in production.
- Region scopes in voice analyst queries ("in the Gulf of Mexico", "over
the Alps") work again in the dev server: the bundled Natural Earth and
neighborhood packs are fetched as JSON in the browser
(`src/data/bundledJson.js`). When a region is not in the bundled packs, the
geocode and admin-boundary fallback answers `region-timeout` after 3 s
instead of holding the reply.
- Transit and Directions rows repaint as soon as their data lands again:
`refreshLayerStats()` now lives on the layer lifecycle, not only on the
compatibility facade. `scripts/qa-radio.mjs` uses it instead of a private
panel method.
- On phones the title bar sits 16 px from the top so both Radio broadcast
waves stay on-screen.
- Keep Cyber right-rail panels mutually exclusive and Display, CCTV and Context
headers and frames fixed during content scrolling. Restore Radio's nested Context placement and compact
player. Add Cyber Sonar voice controls with settings and effect-state readback.
Keep keyboard-focus outlines visible inside Cyber's clipped map and cockpit
expand/collapse buttons, with a matching red hover border.
- Add the opt-in Cyber HUD layout with coordinated map and cockpit panel
styling. Display exposes Sonar on/off, rings, range, power, opacity and sector.
Native point, billboard and label highlighting uses GPU draw commands; there
is no scene-dimming effect selector. Unsupported shaders retain native contact
rendering, and leaving Cyber restores the standard shell and contact treatment.
- Add a **Recent Imagery** data layer (NASA GIBS · HLS + VIIRS, keyless).
Select a box (drag, the current view, or around a pin; up to 1,000 km a
side) and the right-rail panel lists the last 30 days of Sentinel-2 /
Landsat (30 m) and, when switched on, VIIRS daily overview imagery over it,
with thumbnails and scene cloud. Three modes: IMAGE shows one day, VS
BASEMAP swipes it against the map, A / B swipes two days; a SHOW or A / B
chip on each day card pins it, arrow keys preview the focused day while a
slot is empty, and no control moves when anything changes. While imagery is
shown on Google 3D the map switches to Esri and comes back when it is
cleared. Either image exports as a PNG; box, pins, mode and split travel in
share links; the NASA acknowledgement is in the credits and
`DATA_SOURCES.md`. The swipe is now shared with the Nepal scene
(`src/ui/imagerySplit.js`, `src/maps/imageryComparison.js`), and
`MapSourceController.subscribe()` reports every settled map switch.
## Unreleased — voice reaches every layer; honest analyst answers
- Voice can switch on every shipped data layer (the two scene-driven
Bhote Koshi layers stay scene-only), including transit, Recent
Imagery, wind, the weather layers, cyclones and mapped installations, and
understands their spoken names ("buses", "hurricanes", "military bases").
One voice manifest (`src/voice/layerManifest.js`) generates the tool enums
and aliases, and a unit test fails when a new layer has no entry.
- `analyst_query` counts up to 250,000 loaded records per layer instead of the
first 2,000, and says when a layer went past that (`complete: false`). It
also answers over bikeshare stations, transit vehicles, rocket launches,
cyclones, ALPR cameras, mapped installations, local ADS-B aircraft and fire perimeters; fire
records carry their sensor (VIIRS or MODIS).
- Unknown fields, operators, units, value types, scope kinds and invalid
centres are refused with the allowed values instead of answering zero or widening to anywhere. A
layer that is off, still loading or unavailable is reported as such, not
as a count of zero; a partly answerable query is marked `partial`.
Distance sorts honour `sortDir`, a written unit such as `altitudeM(ft)`
converts, and large rankings no longer sort the whole set.
- Results carry `lat`/`lon` for each item and a `display` object with the
scope, the feed window (earthquakes and fires cover the last 24 h) and any
caveat for the screen. The model is asked to answer in one short phrase.
- The weather and Recent Imagery panels can be opened by voice.
## Unreleased — voice outlines without Overpass
- Voice outlines work again without Overpass. Bundled US Census places and
Who's On First neighborhoods (lazy-loaded; `src/data/local_data/`) answer
first, then streets, grounds and pointed-at buildings from OpenFreeMap
tiles, then Nominatim for one explicit ask at a time. Traffic and outlines
share pending fetch/decode ownership and a projection-aware retained-byte
ceiling for the same immutable tile; consumer caches and request lifetimes
remain independent, and eviction or teardown retires queued idle projections.
Public-provider answers are cached and capped per
install. Public Nominatim requests now reserve their daily slot and paced
start atomically across processes sharing that installation's state file,
and fail closed when durable state is unavailable or invalid. Operator-selected
endpoints remain outside the public cap, spacing and state policy, and an
upstream cooldown is isolated to the endpoint that returned it. Anything else
stays a point marked "outline unavailable". Administrative Nominatim
outlines carry the selected record's verified country/admin-1/admin-2
identity; ambiguous, missing, contradictory or retired evidence is not
promoted into a verified level.
- Who's On First rebuilds accept only exact raw registry licence values in the
reviewed open-licence allowlist. New, unknown, restricted or noncommercial
source licences stay excluded until they are reviewed explicitly.
- New settings: `NOMINATIM_URL` (your own Nominatim instead of the public
one; empty disables it), `NOMINATIM_DAILY_CAP` (public requests per day,
default 50) and `PHOTON_URL` (browser place search; empty disables it).
## Unreleased — local receiver feeds
- The Local ADS-B layer also reads local 1090 MHz and 978 MHz UAT decoder
feeds: the `aircraft.json` that dump1090-fa, readsb, tar1090 or skyaware978
serves. Configure them server-side with `LOCAL_RECEIVER_FEEDS`
(`band=url`, comma-separated); there is no feed editing in the browser.
Hosts must be loopback, RFC1918, `localhost` or `*.local`, the scheme http or
https and the path must end in `aircraft.json`; any other entry is logged at
startup, reported `invalid` and never fetched.
- Add `GET /api/local-receivers/aircraft`. It reads every configured feed in
parallel (2 s timeout, redirects refused, 2 MB body cap, about 1 s of shared
cache) and reports each feed `live`, `stale` (its own `now` is over 10 s old),
`unreachable` or `invalid`. Unconfigured, it answers
`{ configured: false }` and fetches nothing. Upstream error text is never
returned.
- The layer merges browser-SDR and feed aircraft by ICAO, keeping the newest
position, polls the route every second only while it is enabled, and remembers
which bands and sources heard each aircraft in the last 60 s. The click card
names them (for example "Heard by your receiver · 978 MHz UAT · decoder
feed"); aircraft heard only on 978 MHz carry a thin ring. The row status
covers both inputs ("2 feeds live · 14 heard", "feed 978 unreachable"), and
the Local RTL-SDR card shows a read-only decoder-feed line.
- While any input is producing aircraft (the browser receiver streaming, or a
feed live), the Local ADS-B row stays ON and lists feeds that are not live as
a trailing note ("3 heard · USB 5.8 msg/s · feed 1090 stale") instead of
showing DEGRADED.
- Records carry `band` (`1090`/`978`) and `source` (`webusb`/`feed`).
- See `docs/LOCAL-RECEIVERS.md`.
## Unreleased — local RTL-SDR and Local ADS-B
- Add a Local RTL-SDR card to the Radio panel. It connects a USB RTL-SDR in
desktop Chrome or Edge through WebUSB and receives broadcast FM (tune, seek,
volume) or 1090 MHz ADS-B. Local FM and internet-radio playback never play
together: starting one stops the other.
- Add a gain control: AUTO or a manual R820T step, remembered per mode. ADS-B
defaults to 28.0 dB (the earlier 20.7 dB gave about 1 msg/s against about 9
at 28.0 dB on the same antenna; a stored choice still wins), FM to AUTO;
changes apply without reconnecting. In
ADS-B mode the card shows CRC-valid messages per second, aircraft heard,
aircraft positioned and the IQ level.
- Prefer the ADS-B/1090 MHz channel of a dual-channel receiver, remember an
explicitly chosen device per mode, and add CHANGE DEVICE to reopen the
WebUSB picker.
- Add the Local ADS-B layer (`local-adsb`, off by default, not part of share
links). Aircraft heard by the receiver draw in magenta beside public Flights;
a marker drops when its position is 60 s old and the aircraft is forgotten
after 60 s without a message. Clicking one opens a card; markers are not
camera-followed. Voice `set_layer_visibility` accepts `local-adsb`.
- Normalize local ADS-B into one record shape, with a pure adapter for
dump1090/readsb `aircraft.json` documents.
- Add `@jtarrio/webrtlsdr` and `@jtarrio/signals` (Apache-2.0); see
`THIRD_PARTY_NOTICES.md`.
## Unreleased — weather review
- On 3D Tiles, draw a 4096×2048 detail window around the view on each
observed-weather shell except global infrared, sampled by the shell's own
surface. It follows the view on camera move end, keeps its place while the
view stays near its centre, and hides until its image is ready after a move;
an older frame's detail stays over at most one newer frame. The image proxy
accepts a 2:1 `bbox` inside the product bounds, rounded to 0.25°.
Lightning's whole-extent image is now 4096×2048; each shell caches up to
128 MiB of decoded images.
- On 3D Tiles, show observed weather and the wind color field as raised,
translucent shells (5.0–6.6 km, lightning highest) with one full-extent image
per frame instead of draping onto tiles; they show at any camera height.
Globe hosts are unchanged. The weather image proxy serves every product
(radar and regional infrared up to 4096×2048, lightning and global infrared
up to 2048×1024) with a size parameter and a 16 MiB cap. Accept bounded tile
sizes in the proxy with separate immutable cache entries.
- Cache exact-time weather images and tiles for 24 hours. Retain up to 6 decoded
global mosaics per renderer and warm the next observation during playback;
tile prefetch is bounded to eight requests and cancels when suspended.
- Move weather times, coverage, legends and cyclone advisory details into keyed
right-rail cards; left rows keep status and configuration. Use one native
observed-history timeline with local preview and coalesced drag commits.
- Add reusable rail card and timeline components. Keep panel collapse, count,
hidden-empty behavior and first-appearance expansion.
- Show wind unit controls beside speed legends and inside inspection readings;
changing units preserves the sampled location and open reading.
- Name satellite imagery Satellite clouds, with Clouds only / Full image modes
and explicit regional coverage. Existing share parameters are unchanged.
- Share one observed history clock across radar, infrared and lightning. Step
through their union timeline with bounded nearest-at-or-before selection;
hide products without an eligible frame. Latest uses each product's newest
observation; playback waits for all frame loads to settle before advancing.
- Label wind as a forecast that does not follow observed history. Keep history
transient and product readouts synchronized.
- Keep Google 3D Tiles drawing while draped weather imagery loads and retain the
old observation until the replacement has rendered.
- Decode one bounded global infrared mosaic per frame on both map hosts, then
crop local tiles to avoid request-dependent brightness seams.
- Add Clouds only / Full image controls and share-link state. Filtered mode uses
a soft linear-brightness alpha ramp from 0.40 to 0.70 around the old 0.55 cut.
- Dock the weather summary in the right rail with standard panel collapse and drag chrome.
- Filter infrared brightness so cold cloud tops stand out; this is a display
filter, not a cloud mask.
- Retry throttled weather tiles up to three times per tile.
- Cull regional wind batches per frame, fade curves below 60 km, and stop idle
rendering when no curve is visible.
- Fade the wind color field below ~1,200 km camera height and hide it at 200 km.
On 3D Tiles, update alpha in 0.1 steps only at camera move end or installation;
globe imagery keeps its smooth per-frame fade.
- Hide cyclone markers, labels, tracks and cones beyond the horizon on every map source.
- Reserve stable weather status space and coalesce panel refreshes per frame.
- Add keyless NOAA/NHC cyclone advisory positions, coherent forecast tracks and
uncertainty cones, plus NOAA's observed 15-minute lightning density imagery.
Preserve source clocks, basin coverage and explicit pending/stale states.
- Make wind default to trails, preserve earlier share-link appearance, retain
geometry across scalar changes, and show a compact weather summary with a
location marker and selected-field emphasis for forecast inspection.
- Add keyless NOAA observed rain radar and infrared satellite layers to Weather,
with explicit coverage/freshness, recent observation playback and native Cesium tiles.
- Increase desktop wind density to 7,200 paths and improve temperature contrast
while retaining the 1,200-path narrow-screen budget and unchanged forecast values.
- Expand Wind into a surface-weather prototype: globe-draped speed shading,
optional same-run 2 m temperature and mean sea-level pressure, GFS/ECMWF model
selection, a numeric legend, wind units, Pause, and a dismissible map-center
reading. Keep wind visible when an optional field is unavailable; respect
reduced motion and stop animation while hidden or disabled. Bake bounded
forecast-following curves once per field and animate their phase on the GPU,
with a canvas fallback and globe view lighting owned only while Wind is enabled.
Display lift does not
change the 10 m forecast level; this adds no cloud volume, radar or forecast-time
playback. Native hardware GPU behavior remains unverified.
Add feed provenance to analyst/view answers and HUD context while retaining existing response fields and runner ownership (Matt Van Horn, #347).
Analyst records for loaded satellites, datacenters and dams, with explicit bounded count/rank coverage (Matt Van Horn, #351).
- New Fire Perimeters layer (Events group): live NIFC WFIGS interagency
wildfire incident perimeters as ground-clamped polygons with a
containment-colored fire line, refreshed every 5 minutes from the public
keyless feature service with truncation paging. Clicking a perimeter shows
an incident card (acreage, containment, cause, behavior, personnel, county,
cost, complex membership) and, when the incident has a state- and
recency-verified InciWeb page, a click-through link to it. Recency uses
incident page origin and update times because the publication API was retired.
The layer is reachable from the panel, voice control, share links (token `2`), and the
analyst query engine. WFIGS and InciWeb requests use a capped, cached
same-origin proxy with timeouts and a per-client limit. Unchanged refreshes
retain geometry; incident-link checks abort on disable or selection change,
and the row includes a containment legend.
- Remove the spurious scrollbars that appeared on both panel stacks at narrow
widths (720px and below) as soon as a panel was expanded. The stacks scroll
vertically there, and each panel's decorative glow, absolutely positioned
with a negative inset, became 18–20px of scrollable overflow on both axes: a
horizontal scrollbar band under the expanded CCTV, Context, Data Layers or
Scenes panel plus a vertical scrollbar that scrolled nothing but glow. The
narrow-screen rules now pin the glow to its panel box; the stacks still
scroll for genuinely tall content such as an expanded DISPLAY panel, and the
Context radio popover is not clipped. `src/ui/panelRails.test.mjs` pins the
rule against every 720px media block.
- Enable responsive trackpad pinch zoom on the globe. Browser pixel-mode
`Ctrl+wheel` pinch gestures now reach Cesium with bounded amplification,
while ordinary wheel, line-mode and touch-pinch inputs retain their existing
behavior; the listener is removed with the application scene.
- Report AIS speed and course that carry the standard "not available" code as
unknown instead of 102.3 knots and 360 degrees. Genuine readings, including a
stopped vessel's zero and the highest encodable values, are unchanged.
- Render native `<select>` option lists in the dark UI palette. The closed
controls were already skinned, but the browser-painted popups fell back to the
platform light palette, leaving near-white option text on a white surface in
the HUD layout, Scenes, CCTV camera, Radio filter and Draw colour menus.
- Credit adsbdb, which supplies the aircraft type, model name and registration
on enriched flights and the airline and origin/destination pair behind the
tracked contact's route strip. `DATA_SOURCES.md` now records adsbdb's
published credits and route-data restriction, along with the request bounds
and gitignored 24-hour local cache. A matching `DATA_CREDITS` entry surfaces
the credit in the in-app Data attribution popover, and a test protects it
against accidental removal.
- Size the TomTom daily tile budget to the provider's real free allowance.
`TOMTOM_DAILY_TILE_BUDGET` defaulted to 40,000/day against an allowance
granted monthly (200,000 tile requests/month), exhausting a month in five
days and leaving the traffic layer dead for the rest of the period. The
default is now 6,000/day (186,000 over a 31-day month). Corrects the stale
"~50k/day" free-tier figure in the proxy, `.env.example` and
`DATA_SOURCES.md`. Still an application-side ceiling, not a billing cap.
- Distinguish PARTIAL vessel snapshots from STALE data in the layer panel, with
accepted-record counts and unchanged retention, freshness and outage safeguards.
- Keep Nepal provider media inside the Pinokio compatibility boundary: use
source-linked fallback cards instead of automatic embeds or hidden preloads
that launch an external browser. Ordinary browsers retain embedded playback.
Source-only timed shots use their authored card dwell and continue normally.
Media autoplay now requires a live Play Scene or Play Shot action; passive
loading, saved state and seeking do not grant playback authority.
- Add Director import previews, validated scene/shot detail drafts and selected-scene
JSON or asset-bundle sharing. Preserve attribution; verify bounded bundle bytes
before admission and release staged work on cancellation or teardown.
- Director scene documents now support bounded data-pack manifests, per-shot
selection and registered GeoJSON/PNG/media loaders with explicit placement,
visible attribution and cancellation/disposal on Stop or replacement.
@@ -8,17 +485,14 @@
with shared playback/seek interpolation, easing and holds. Navigation and
manual input cancel authored motion; older scene files retain existing flights.
- Director validates bounded version-3 scene files before replacing a project,
preserves unreadable browser saves, migrates legacy bloom once and preserves
zero-pitch/low-altitude camera and scope/detection edits. Project normalization has a separate owner.
- Separate Director timing, seek calculations, playback clocks and registered
scene-pack presentation rules. Preserve authored content and controls; Stop
releases pending hold timers and stale ticks cannot affect replacement playback.
- Keep parked transit vehicles aligned to their world course during camera orbits, fall back to reported bearing, and keep vehicles with no course consistently screen-up.
- Separate Realtime connection, response/tool, Radio, input/audio, cost, viewport
@@ -94,6 +568,10 @@
- Expose portable radio, camera-type and regional source helpers; keep HTTP transport separate from record normalization.
- The Realtime debug-log endpoint is bounded on every axis it was not: an always-on per-client rate limit, asynchronous appends through a serialized queue instead of a synchronous write on the request path, and rotation of the log file at 32 MB keeping one prior generation. The 8 MB cap applied to a single request body and never to the file those requests accumulated into, so any local page could grow it for as long as the dev server ran. A malformed record and a failed write are now told apart, 400 from 500, and neither answer carries the error text.
- Three proxy paths no longer relay upstream or JS error text to the client. The HUD summary passed OpenAI's own `error.message` through whenever upstream was not ok, carrying request ids and quota wording; the Realtime token route passed through non-success response bodies and echoed JS errors, which can expose upstream details; and a failed CCTV media fetch stored the raw errno as the camera's health message, which reaches the screen through `GET /api/cctv/health` rather than through the sanitized response beside it. Logs now name the failure and the upstream status without the text.
- Separate vessel records and feed acquisition from rendering while preserving selection, partial-feed retention, sea-surface placement and request cancellation.
- Separate military-flight records and acquisition from rendering while preserving ground-model ownership, source units and follow behavior.
@@ -108,6 +586,13 @@
- Split application scene, controls, catalog, tools and HTML into reusable components; configure application request services and sources without changing global fetch. Preserve standalone markup and voice behavior. Explicit annotation navigation may resolve a distant named target.
## Satellite pass prediction
- Bisect pass rise/set to ~0.2 s and fit peak elevation with a parabola.
- Mark passes visible from Earth-shadow and civil-twilight checks.
- Add `getNextSatellitePass(noradId, options)` for any loaded catalog satellite.
- `next_iss_pass` retains the next geometric pass and adds visibility metadata. `next_satellite_pass` adds bounded loaded-catalog name/NORAD lookup and optional visible-only filtering (Rehaan Delmotra, #451; maintainer adaptation).
## Voice component boundaries
- Separate voice controls, Realtime connection requests and the action runner.
@@ -199,6 +684,11 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
## [Unreleased]
- Add ECMWF IFS model selection to Wind (#464, thanks @beneduzi), with model-scoped forecast-step caches, cancellation of replaced requests, and separate issue/valid timestamps.
- Add bounded Director feature actions with accessible controls, explicit camera/layer admission and cancellation; restore pack geometry on same-shot seek. Preserve existing scenes and content attribution.
- Give application request services, terrain/floor caches and annotation lookup state explicit owners and cancellation; share them across controls, layers and voice.
- Construct application layers from explicit sources, with standalone provider selection and catalog-owned aircraft classification; controls and voice queries use those instances.
@@ -225,6 +715,15 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
### Fixed
- Keep traffic-road bounds crossing the antimeridian monotonic and inside the
longitude range accepted by the Overpass request path, preserving the small
wrapped span instead of producing an inverted or rejected box (#392 — thanks
@Ashfaqbs).
- Make `npm run doctor` report keyless anonymous OpenSky access for explicit
`OPENSKY_AUTH_MODE=anon` and OAuth mode without a client pair, retain the
existing OAuth-pair capability wording, and identify selected Basic or auto
modes without guessing their eventual credential choice. OpenSky proxy
authentication is unchanged.
- Bikeshare stations load again. The extracted station source addressed the
proxy as `/api/gbfs?url=`, but the proxy reads its upstream target from the
path, so every request answered 400 and the layer reported a fetch error for
@@ -283,6 +782,8 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
### Added
- Add MODIS NRT (Terra+Aqua, ~1 km) active fires to the FIRMS layer, sharing the
existing `FIRMS_MAP_KEY` and 30-minute cache.
- Two map-orientation controls sit beside Share in the top-center globe
actions. Tilt Map swings between a straight-down map and a 35-degree oblique
around the point under the centre of the view, keeping that point and the
@@ -317,6 +818,12 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
Visible animation, detection membership, history storage and proxy requests
are bounded. Share links carry Transit as token `j`.
- Add a keyless **Wind** layer from NOAA GFS 10 m wind (#459, thanks @beneduzi). The `/api/wind` proxy
byte-range fetches only the UGRD/VGRD GRIB2 messages from the public AWS bucket,
decodes them with ecCodes (WASM), and serves a compact Float32 U/V grid; the
client renders nullschool-style animated particles in a canvas overlay that
follows the Cesium camera and skips globe-occluded points. Forecast, not
observations. Requires Node ≥24 for the WASM decoder.
- Add Ontario 511 as a keyless CCTV source pack, including Kitchener-area
highway cameras, with server-registered still URLs and attribution.
- CCTV Mesh adds Finland: Fintraffic road weather cameras, keyless, nationwide, 300 by default. Each camera view of a station is placed separately; ambient stills refresh on the source's 10-minute cadence (the active camera keeps the usual 10-second refresh).
@@ -979,3 +1486,10 @@ represent previously published GitHub Releases.
## [0.1.0] — 2026-02-09
- Initial project version.
### Live CCTV integration candidate
- Live HLS video shares one decoder between the camera panel and projection,
with a DelDOT HTTPS source pack. Credit: Daniel Slay (@Danielslay86), PR #489.
- Maintainer adjustments bound sessions and downloads, remove disk/subprocess
remuxing, reject redirects, and clean up playback on switching or disabling.
+67 -1
View File
@@ -37,6 +37,72 @@ configuration through the development app or environment file. Unknown API
paths return JSON 404 responses. Vite preview is for checking a local build;
it is not a production server.
## Feature regression gates
Those three are the baseline, not the whole story. Most features also have a
dedicated headless gate under `scripts/qa-*.mjs` that drives the real app and
asserts that feature's contract. **Run the gate covering whatever you touched**,
and say which one you ran in the PR.
Feature gates and a few supporting modules live under the same filename pattern:
```bash
ls scripts/qa-*.mjs # gates and their supporting modules
head -40 scripts/qa-radio.mjs # what this runnable gate proves and how to run it
```
Runnable gate entrypoints document what they assert and how to invoke them.
Most need a dev server; some also need a specific provider key or port. Follow
the entrypoint's header rather than assuming every matching file accepts the
same arguments. Only a couple (`qa:map-source-tray`, `qa:transit`) have an
`npm run` alias.
If you aren't sure which gate covers your change, search `docs/CURRENT-STATE.md`
for the feature: it names the gate for many of them, and it's the authoritative
runtime reference either way. `scripts/qa-l9-matrix.mjs` aggregates the broader
release-candidate checks and selected harnesses; it does not replace the focused
gate for the feature you changed.
> **CI does not cover this for you.** The workflow runs the setup policy
> checks, the formatting and package-boundary checks, the unit suite and the
> production build, plus a Windows onboarding job. It runs neither
> `npm run test:track` nor any `qa-*.mjs` gate — both need a live dev server
> and a browser. Include the applicable local run in your PR's validation
> evidence.
## Share-link layer tokens
Every shareable layer owns an immutable token in `LAYER_STATE_REGISTRY` and
`src/data/layerStateTokenReservations.json`. Never change or reuse an existing
token: published v2 links are authored state, and the reservation remains even
if its layer is later removed.
New layers first use the remaining unreserved single-character digits (`0`
through `9`), then the two-character base-36 namespace (`00` through `zz`). On the
current published ledger, `1` and `2` are already owned, so the next choices
are `0`, `3` through `9`, then `00`. After rebasing onto the latest `main`, run
`npm run layer-token:next -- <layer-id>`, add that result to both the permanent
JSON reservation ledger and registry entry, and keep the exact registered-layer
count assertion up to date. The token is assigned against the merge-time base, not
claimed when a PR opens, so another merged layer may require rerunning the
command after a rebase. Never repurpose a published or retired token.
Run `npm run layer-token:check -- --base-ref origin/main` after the rebase;
this checks that every published assignment remains reserved and that new
assignments consume free digits before the two-character tokens, in order.
Pull-request CI runs the same check against `main`.
The helper reads the local ledger; it does not run during a rebase or edit files.
If another PR publishes your provisional token first, keep every published
registry entry and ledger reservation from the refreshed `main`. Temporarily
remove only your still-unpublished layer's registry entry and ledger row so the
local registry is valid again, then rerun `npm run layer-token:next -- <layer-id>`.
For example, if PR A publishes `0` while PR B still provisionally uses `0`,
PR B removes only its own provisional entry, reruns the helper, and receives
`3`. Put `3` on PR B's registry entry and ledger row, update its pinned count
and token tests, then rerun `npm run layer-token:check -- --base-ref origin/main`.
An invalid duplicate ledger can fail during module import before the helper
prints anything; never fix that by renaming or deleting PR A's published token.
## Good first contributions
The highest-leverage places to jump in:
@@ -82,7 +148,7 @@ ownership and adoption process.
## Pull requests
1. Branch off `main`.
2. Keep `npm run build`, `npm test`, and `npm run test:track` green and avoid new console errors.
2. Keep `npm run build`, `npm test`, and `npm run test:track` green and avoid new console errors, plus the [feature gate](#feature-regression-gates) for the area you touched.
3. If you change runtime behavior, update `docs/CURRENT-STATE.md` and `CHANGELOG.md` in the same PR.
4. If you add or change a data source, update [DATA_SOURCES.md](DATA_SOURCES.md) with its license and attribution. **Don't add data you don't have the right to redistribute** — fetch it at runtime instead.
5. Describe what you changed and how you verified it (screenshots welcome for anything visual).
+186 -40
View File
@@ -15,21 +15,29 @@ How to read this:
| Source | Used for | License / terms | Attribution |
| --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| **OpenStreetMap ALPR camera locations** (including DeFlock community mapping) | Optional mapped automatic license-plate-reader camera layer | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/); commercial use permitted with applicable attribution and database share-alike obligations | [© OpenStreetMap contributors](https://www.openstreetmap.org/copyright); [DeFlock](https://deflock.org) community mapping |
| **NOAA GFS (wind)** (`noaa-gfs-bdp-pds.s3.amazonaws.com`) | Global 10 m wind, optional 2 m temperature and mean sea-level pressure for Wind. Keyless; latest 6-hourly 0.25° cycle, byte-range GRIB2 reads, cached for an hour | U.S. public domain (NOAA); keyless via NOAA Open Data on AWS | "NOAA Global Forecast System (GFS)" (courtesy; not an endorsement) |
| **ECMWF IFS (wind)** (`data.ecmwf.int/forecasts`, ECMWF Open Data) | The alternative Wind model: 10 m wind, optional 2 m temperature and mean sea-level pressure. Keyless; latest 6-hourly 0.25° run, byte-range GRIB2 reads, cached for an hour | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) plus the [ECMWF Terms of Use](https://apps.ecmwf.int/datasets/licences/general/) | "This service is based on data and products of the European Centre for Medium-Range Weather Forecasts (ECMWF)", with the CC BY 4.0 link, the modification notice and ECMWF's liability disclaimer — shown in-app |
| **NOAA nowCOAST observed weather** (`nowcoast.noaa.gov/geoserver/observations/{weather_radar,satellite}/ows`, WMS 1.1.1) | Rain radar (MRMS base reflectivity, contiguous US, about 4-minute updates) and Satellite clouds (GOES-19/18 Band 14 regional infrared, about 5-minute updates; global infrared mosaic, hourly). Keyless; layer metadata refreshed every 2 minutes, exact-time images cached for up to 24 hours | [NOAA disclaimer](https://oceanservice.noaa.gov/disclaimer.html) | "NOAA nowCOAST · NWS/OAR MRMS radar; NESDIS GOES and global satellite partners" |
| **NOAA nowCOAST lightning density** (`nowcoast.noaa.gov/geoserver/observations/lightning_detection/ows`) | Lightning density: 15-minute accumulated strike density on an approximately 8 km grid, Americas and Pacific. Keyless; layer metadata refreshed every 10 minutes, exact-time images cached for up to 24 hours | NOAA Level-5 derived product from Vaisala NLDN/GLD360; the [product description](https://ocean.weather.gov/lightning/lightning_pdd.php) permits public distribution of this density product, not of raw Vaisala detections | "NOAA/NWS nowCOAST; derived from Vaisala NLDN/GLD360" |
| **NOAA NHC / CPHC advisories** (`www.nhc.noaa.gov/CurrentStorms.json` + the `mapservices.weather.noaa.gov` tropical weather summary MapServer) | Cyclone advisories: storm positions, forecast tracks, points and cones for the Atlantic and eastern/central North Pacific. Keyless through `/api/cyclones`; cached for 5 minutes | [NWS public-data terms](https://www.weather.gov/disclaimer); no endorsement implied | "NOAA/NWS NHC / CPHC" |
| **OpenStreetMap** | Map and place data (roads, military areas and names, ALPR, place search, outlines); via vector tiles, hourly ALPR extract, bundled data, Nominatim and optional self-configured Overpass | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | © OpenStreetMap contributors |
| **Google Map Tiles API** (Photorealistic 3D Tiles) + Places/Geocoding | The 3D globe, voice scene context, and on-demand nearby installation search | Google Maps Platform ToS (proprietary, your own key + billing) | "Google" / "Google Maps" logo — **shown in-app**, required |
| **OpenSky Network** | Primary worldwide live-flight snapshot | Non-commercial research/education license | Schäfer et al., _"Bringing Up OpenSky"_, IPSN 2014 + opensky-network.org |
| **adsb.lol point API** | Bounded live-flight fallback when OpenSky has no usable snapshot | ODbL 1.0 | adsb.lol contributors; `api.adsb.lol/v2/lat/{lat}/lon/{lon}/dist/{radius}` |
| **adsb.lol** | Military flights + aircraft traces | ODbL 1.0 | "adsb.lol" (ODbL) |
| **adsbdb** (`api.adsbdb.com`) | Aircraft type, model name and registration by ICAO hex for enriched flights; airline and origin/destination airports for the tracked flight's callsign | No licence or rate limit is published for the service. Its own credits: aircraft data from PlaneBase, ICAO-to-N-number conversion by Guillaume Michel, and flight route data that "is the work of David Taylor, Edinburgh and Jim Mason, Glasgow, and may not be copied, published, or incorporated into other databases without the explicit permission of David J Taylor, Edinburgh" | "adsbdb — api.adsbdb.com" (courtesy); route data credited to David Taylor, Edinburgh, and Jim Mason, Glasgow |
| **AISStream.io** | Live vessels (AIS) | Free, beta, no formal ToS; AIS is a public broadcast | "AISStream.io" (courtesy) |
| **CelesTrak** | Satellite TLEs (SGP4) | US-government-origin data, no license; citation requested | "CelesTrak (celestrak.org), Dr. T.S. Kelso" |
| **The Space Devs — Launch Library 2 v2.3** | Recent launch, payload, stage, and recovery metadata for Space Missions (30d) | [The Space Devs terms of use](https://github.com/TheSpaceDevs/Tutorials/blob/main/faqs/faq_TSD.md#terms-of-use): data may be used and shared in any form; avoid forwarding it without added value; attribution is encouraged (not mandatory). [Official API limits](https://ll.thespacedevs.com/docs/): 15 unauthenticated calls/hour; optional token | "Launch Library 2 — The Space Devs" (courtesy attribution) |
| **Esri World Imagery** (ArcGIS Online tile service) | The keyless satellite basemap — the default landing when no Google/ion credential is configured, and the "Esri Satellite" map stack | [Esri Master Agreement](https://www.esri.com/en-us/legal/terms/full-master-agreement): the public World Imagery service is usable in public-facing apps with attribution; no key is required for this classic endpoint, but Esri governs and can change access — an app at scale should review current ArcGIS Location Platform terms | "Powered by Esri — Source: Esri, Maxar, Earthstar Geographics, and the GIS User Community" (provider carries the service's own credit line) |
| **USGS** | Earthquakes | U.S. public domain | "Data courtesy of the U.S. Geological Survey" |
| **OpenStreetMap (Overpass API)** | Road geometry for traffic | ODbL 1.0 | "© OpenStreetMap contributors" |
| **TomTom Traffic API** (flow vector tiles) | Live congestion coloring for the traffic layer (optional, BYOK) | [TomTom for Developers terms](https://developer.tomtom.com) (proprietary, your own key; free tier currently 200K tile requests/month — see [current pricing](https://docs.tomtom.com/pricing/)) | "Traffic flow data © TomTom" — registered when live mode activates |
| **OpenStreetMap (Overpass API)** | Viewport-bounded mapped installation context for Global Context | ODbL 1.0 | "© OpenStreetMap contributors" (incomplete mapped context) |
| **Photon** (komoot) | Keyless place search — the fallback when no Google Maps key is configured, or when Google declines the Geocoding request | Service: free public instance, [fair use](https://photon.komoot.io) (no bulk/heavy use); underlying data: **ODbL 1.0** | "Photon (komoot)" + "© OpenStreetMap contributors" |
| **OpenStreetMap (Nominatim)** | Reverse-geocoded place label in the cockpit Local Info page, and last-resort forward place search via `/api/geocode` when Google and Photon do not answer | ODbL 1.0 + [Nominatim usage policy](https://operations.osmfoundation.org/policies/nominatim/) (maximum 1 request/second, results cached, identifying User-Agent) | "© OpenStreetMap contributors" |
| **NASA GIBS** (WMTS tiles + Worldview Snapshots) and **NASA CMR** (granule catalog) | Recent Imagery layer: HLS S30/L30 (Sentinel-2, Landsat 8/9) and VIIRS true-colour tiles draped on the map, per-day thumbnails and PNG exports; the HLS granule catalog (dates, cloud, footprints) for a selected box | U.S. public domain (NASA data); keyless, browser-direct (CORS `*`); no bulk use — one 30-day catalog query per box, tiles bounded to the box, six tile requests in flight | "We acknowledge the use of imagery provided by services from NASA's Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS)." — [gibs.earthdata.nasa.gov](https://gibs.earthdata.nasa.gov), [HLS](https://lpdaac.usgs.gov/products/hlss30v002/) |
| **NIFC WFIGS** (Wildland Fire Interagency Geospatial Services, ArcGIS feature service) | Fire Perimeters layer: current interagency wildfire incident perimeters, containment, and incident facts | U.S. public domain (interagency wildland-fire data published through the [NIFC Open Data portal](https://data-nifc.opendata.arcgis.com/)); fetched keyless through `/api/fire-perimeters` (5-minute server cache) with `exceededTransferLimit` paging | "Wildfire perimeters: National Interagency Fire Center (WFIGS)" |
| **InciWeb** (inciweb.wildfire.gov) | Per-incident "InciWeb ↗" card links to official incident pages (narratives, evacuation/closure notices) | U.S. government public incident information. Links are matched best-effort by incident name and state, with incident page origin and update times checked on selection. Same-origin `/api/fire-perimeters/inciweb` routes cache the catalog for 1 hour and publication metadata for 30 minutes; linked pages remain InciWeb content | "Incident information: InciWeb (inciweb.wildfire.gov)" |
| **OpenFreeMap / OpenMapTiles** | Vector tiles | OpenStreetMap data under ODbL 1.0 | OpenFreeMap © OpenMapTiles |
| **Overture Maps Foundation** | Military area names pack | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | Overture Maps Foundation |
| **TomTom Traffic API** (flow vector tiles) | Traffic roads and live congestion (optional, BYOK) | [TomTom for Developers terms](https://developer.tomtom.com) (proprietary, your own key; free tier currently 200K tile requests/month — see [current pricing](https://docs.tomtom.com/pricing/)) | "Traffic flow data © TomTom" — registered when live mode activates |
| **Photon** (komoot) | Keyless place search — the fallback when no Google Maps key is configured, or when Google declines the Geocoding request | Service: free public instance, [fair use](https://photon.komoot.io) (no bulk/heavy use); underlying data: **ODbL 1.0** | "Photon (komoot)" |
| **Open-Meteo** | Current weather in the cockpit Local Info page and cockpit-local dynamic atmospheric effects | [CC BY 4.0 data licence and adjacent-link attribution requirement](https://open-meteo.com/en/licence) | Linked "Weather data by Open-Meteo.com" beside the displayed local data |
| **Google News RSS** | Primary locality-matched headlines in the cockpit Regional News page | [Google News Terms of Service](https://www.google.com/intl/en_us/terms_google_news.html) restrict use to personal, noncommercial use; linked articles remain third-party publisher content and retain publisher terms | "Google News RSS" plus each article's linked publisher/domain |
| **GDELT Project DOC 2.0** | Fail-soft fallback for location-matched cockpit headlines | [GDELT Terms of Use](https://www.gdeltproject.org/about.html#termsofuse): unrestricted academic/commercial/governmental dataset use, with citation and link required; linked articles retain publisher terms | "GDELT Project" plus each article's linked publisher/domain |
@@ -37,9 +45,10 @@ How to read this:
| **TxDOT ITS (its.txdot.gov)** | CCTV camera catalogs + frames, Texas districts | Public TxDOT traffic camera data | "Texas Department of Transportation" (courtesy) |
| **City of Tallinn (ristmikud.tallinn.ee)** | CCTV camera catalog + frames, Tallinn intersections | Public City of Tallinn traffic camera data | "City of Tallinn — ristmikud.tallinn.ee" (courtesy) |
| **Transpordiamet / Tarktee** | CCTV camera catalog + frames, Estonia road-weather cameras | Public Transpordiamet / Tarktee road camera data | "Transpordiamet / Tarktee — tarktee.transpordiamet.ee" (courtesy) |
| **Stadt Warendorf webcam** | Webcam frames, Marktplatz / Historisches Rathaus (`config/cctv_sources.warendorf.json`) | Public municipal webcam data | "Stadt Warendorf — webcam.warendorf.de" (courtesy); mount and heading derived from OSM geometry, "© OpenStreetMap contributors" (ODbL) |
| **Stadt Warendorf webcam** | Webcam frames, Marktplatz / Historisches Rathaus (`config/cctv_sources.warendorf.json`) | Public municipal webcam data | "Stadt Warendorf — webcam.warendorf.de" (courtesy) |
| **Live Traffic NSW (Transport for NSW)** | CCTV camera catalog + frames, New South Wales | [CC BY 4.0](https://opendata.transport.nsw.gov.au/) — attribution REQUIRED | "Live Traffic NSW — Transport for NSW" |
| **Caltrans (cwwp2.dot.ca.gov)** | CCTV camera catalogs + frames, California districts | Public Caltrans traffic camera data | "Caltrans — cwwp2.dot.ca.gov" (courtesy) |
| **DelDOT (video.deldot.gov)** | CCTV camera catalog (`tmc.deldot.gov/json/videocamera.json`, cached for 15 minutes) + live HTTPS HLS video, Delaware. Keyless | Public DelDOT traffic camera streams via the [DelDOT traffic map](https://deldot.gov/map/) | "DelDOT — Delaware Department of Transportation" (courtesy) |
| **TfL Open Data (JamCams)** | CCTV camera catalog + frames, London | [TfL Open Data terms](https://tfl.gov.uk/info-for/open-data-users/) — attribution REQUIRED | "Powered by TfL Open Data. Contains OS data © Crown copyright and database rights" |
| **Ontario 511** | CCTV camera catalog + frames, Ontario highways including Kitchener-area routes | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) | "Ontario 511" plus Open Government Licence - Ontario |
| **Fintraffic / Digitraffic (weathercams)** | CCTV camera catalog + frames, Finland | [CC BY 4.0](https://www.digitraffic.fi/en/terms-of-service/) — attribution REQUIRED | "Fintraffic / digitraffic.fi, license CC BY 4.0" |
@@ -55,16 +64,22 @@ How to read this:
| **HSL** (realtime.hsl.fi) | Transit layer live vehicles, Helsinki | HSL (Helsinki Region Transport) open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to HSL (Helsinki Region Transport) (courtesy text) |
| **Radio Browser** | Geolocated internet-radio station directory and station-level tags | Public-domain directory data under PDDL 1.0; individual broadcaster stream terms apply | "Radio Browser" plus a link to the selected broadcaster |
| **Re:Earth Terrain** (Mapterhorn) | Terrain (keyless globe stacks — OSM etc. — + `/api/terrain/heights` ellipsoidal-height lookups) | Terrain mesh: CC BY 4.0; geoid: EGM2008 (NGA, public domain) | "Terrain (keyless globe stacks): Re:Earth Terrain / Mapterhorn (CC BY 4.0) / EGM2008 (NGA)" |
| **OSRM on the FOSSGIS routing servers** (`routing.openstreetmap.de`) | Street-following routes for the Directions layer and voice route annotations, via `/api/route` | [FOSSGIS routing usage policy](https://routing.openstreetmap.de/about.html): "Display the required attribution and display a link to 'fix the map'", "Use a valid user agent and, if applicable, a correct referrer", "One request per second max", "No scraping, no heavy usage". The full policy is the German [FOSSGIS Nutzungsbedingungen](https://www.fossgis.de/arbeitsgruppen/osm-server/nutzungsbedingungen/); FOSSGIS also states that the server may be embedded in your own pages but "eine gewerbliche Nutzung ist nur mit Einschränkungen erlaubt" (commercial use only with restrictions). Route data derives from OpenStreetMap (ODbL 1.0) | "Routing: OSRM on the FOSSGIS servers" + "© OpenStreetMap contributors" + a "fix the map" link — shown in the Data attribution popover |
| **OSRM on the FOSSGIS routing servers** (`routing.openstreetmap.de`) | Street-following routes for the Directions layer and voice route annotations, via `/api/route` | [FOSSGIS routing usage policy](https://routing.openstreetmap.de/about.html): "Display the required attribution and display a link to 'fix the map'", "Use a valid user agent and, if applicable, a correct referrer", "One request per second max", "No scraping, no heavy usage". The full policy is the German [FOSSGIS Nutzungsbedingungen](https://www.fossgis.de/arbeitsgruppen/osm-server/nutzungsbedingungen/); FOSSGIS also states that the server may be embedded in your own pages but "eine gewerbliche Nutzung ist nur mit Einschränkungen erlaubt" (commercial use only with restrictions). Route data derives from OpenStreetMap (ODbL 1.0) | "Routing: OSRM on the FOSSGIS servers" + a "fix the map" link — shown in the Data attribution popover |
### Local receiver input (not fetched from a service)
The optional Local ADS-B layer and the broadcast-FM receiver use radio you receive yourself: a USB RTL-SDR opened in the browser through WebUSB, or the `aircraft.json` of a decoder you run on your own machine or network (`LOCAL_RECEIVER_FEEDS`). Radio samples, audio and the optional receiver location stay in the browser. Decoder feeds are read by your local GEV server and relayed only to your browser, with about a second of in-memory caching. When you select a local aircraft, its ICAO address (and an airline-style callsign) is looked up, and aircraft that qualify for a 3D model are looked up by ICAO address, through the same adsbdb proxy the public flight layers use; your server caches those answers on disk and treats them as fresh for 24 hours. Received radio data is not stored or redistributed. ADS-B is an unauthenticated public broadcast and may be incomplete, delayed, spoofed or decoded in error; it is not suitable for navigation or operational use.
### Notes on the live sources
- **Google Maps Platform.** You supply your own API key and are bound by [Google's ToS](https://cloud.google.com/maps-platform/terms). Google Maps Content (tiles, geocodes, places) **may not be cached, stored, rehosted, or committed** — this app only ever uses it live, which is the compliant pattern. The "Google" attribution is displayed on the globe and must stay visible. Restrict your key (see [SECURITY.md](SECURITY.md)).
- **OpenSky Network.** Its license is **non-commercial**, and operational use of the REST API in a live product can require a prior written agreement with OpenSky — even for non-profit/government use. If you deploy this commercially, contact OpenSky for your own terms. The flights layer is a toggle and runs anonymously by default.
- **adsb.lol flight fallback.** When OpenSky is unavailable and no last-good OpenSky response exists, the server requests a cached, capped 250 nm adsb.lol point snapshot around the current camera subpoint. This is regional observed context, not worldwide completeness; provenance is exposed in the Flights stats/context row. Military ICAOs remain reconciled through the existing dedicated military registry rather than duplicated.
- **adsbdb enrichment.** `/api/adsbdb` resolves an ICAO hex to aircraft type, model name and registration, and the tracked flight's callsign to an airline and an origin/destination pair. Fan-out is deliberately small because adsbdb is a free community API that publishes no quota: at most 4 requests in flight, dispatches dripped to roughly 5/s, and each key asked for at most once per session, with route lookups reserved for the tracked aircraft. The dev-server proxy caches each key for 24 hours in `.gev-cache/adsbdb.json`, negative answers included, so fresh entries survive server restarts. That gitignored cache remains local to the deployment and is not bundled with this repo. adsbdb publishes no general service license; its README says the route data may not be copied, published, or incorporated into other databases without the explicit permission of David J Taylor, Edinburgh. Deployers are responsible for obtaining any permission their use requires. Enrichment is best-effort and fail-silent — a flight that gets no answer keeps what the flight feed already gave it.
- **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state.
- **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover.
- **Ontario 511 CCTV.** The camera list comes from the keyless `511on.ca/api/v2/get/cameras` endpoint; frames are server-registered still images under `511on.ca/map/Cctv/`. The source is live-fetched at runtime and attributed under the Open Government Licence - Ontario.
- **DelDOT live video cameras.** The keyless `tmc.deldot.gov/json/videocamera.json` catalog supplies active cameras and official `https://video.deldot.gov/live/…/playlist.m3u8` links. The local server transiently buffers bounded MPEG-TS segments in memory and deletes them on session release or idle expiry. No recordings are retained. `CCTV_DELDOT_ENABLED=0` disables the pack. Public availability is not a blanket redistribution license; deployment beyond personal local viewing needs confirmation of applicable DelDOT terms.
- **Fintraffic weathercams.** The camera list is one keyless GeoJSON request to `tie.digitraffic.fi/api/weathercam/v1/stations` (809 stations / 2,275 presets), sent with the `Digitraffic-User: gods-eye-view` header the service asks clients to identify themselves with and with gzip; frames are stills on `weathercam.digitraffic.fi`, refreshed on each station's 600 s collection interval, which is also the client's ambient-still refresh cadence for this pack (the active camera keeps the usual 10-second refresh). One preset (a fixed camera view) is one camera; stations not `GATHERING` and presets not `inCollection` are dropped. Attribution is required by the CC BY 4.0 licence and is registered in the Data attribution popover. `CCTV_FINTRAFFIC_ENABLED=0` is the kill switch.
- **DriveBC highway cameras.** The camera list comes from the keyless `https://www.drivebc.ca/api/webcams/` endpoint served by the DriveBC.ca site ([bcgov/DriveBC.ca](https://github.com/bcgov/DriveBC.ca)); frames come from `https://www.drivebc.ca/images/<id>.jpg`, built from the numeric camera id. DataBC's [DriveBC HighwayCams](https://catalogue.data.gov.bc.ca/dataset/bc-highwaycams) catalogue entry publishes the same cameras under the Open Government Licence – British Columbia, but its CSV still lists the retired `images.drivebc.ca` frame URLs, which now return a placeholder. The licence's attribution statement is required and is registered in the Data attribution popover. By default the 250 cameras nearest Vancouver and Victoria load; `CCTV_DRIVEBC_MAX_SOURCES` changes the cap and `CCTV_DRIVEBC_ENABLED=0` turns the pack off. Some cameras in the feed are supplied by partners (TransLink, the City of Vancouver, the City of Surrey, Parks Canada / Alberta Motor Association and others); the feed's per-camera credit is carried onto the camera record and shown beside the provider in the CCTV panel.
- **TxDOT ITS cameras.** The catalog comes from the keyless `its.txdot.gov/its/DistrictIts/GetCctvStatusListByDistrict` endpoint, one call per district. Frames come from the per-camera `GetCctvSnapshotByIcdId` endpoint, which answers JSON carrying a base64 JPEG rather than an image body; the proxy decodes it only for responses from the official TxDOT origin and validates the JPEG header before serving. Only cameras reporting `Device Online` are registered. Districts default to Austin and San Antonio (`CCTV_TXDOT_DISTRICTS`; 25 codes exist statewide); `CCTV_TXDOT_MAX_SOURCES` caps the pack and `CCTV_TXDOT_ENABLED=0` turns it off. Frames are fetched live at request time and are never stored or redistributed.
@@ -76,13 +91,14 @@ How to read this:
- **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals.
- **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All seven registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth.
- **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply.
- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 40,000, a configurable application safety ceiling, not a guarantee of staying within TomTom's monthly free allowance; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app).
- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 6,000 — sized so a 31-day month stays inside TomTom's 200K monthly free allowance; still a configurable application safety ceiling, not a guarantee; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app).
- **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line.
- **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable.
- **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit.
- **Global Context installation context.** `/api/military-installations` queries only an allow-listed subset of OSM `military=*` and `landuse=military` features inside a maximum 10° non-dateline viewport. It caches and may serve stale mapped context, but it is neither a global installation database nor evidence of capability, activity, or absence. User-requested Google Places results remain separately sourced candidates unless their returned types explicitly establish military classification; generic offices, museums, and similarly ambiguous matches are excluded from military proximity counts.
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon's public instance, then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` shares the cockpit's Nominatim queue at no more than one request per second, sends an identifying `User-Agent` and `Referer`, caches answers for five minutes, shares one upstream call between identical searches already in flight, bounds the queue so a burst is refused rather than held, and drops a queued search whose caller has given up. It asks for one result per search and is not used for systematic or bulk queries.
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and serializes Nominatim calls at no more than one request per second. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
- **Mapped installations.** OpenFreeMap military polygons use a bundled Overture/OSM name index; wide views show named points. Optional operator-configured Overpass and explicit Google Places searches keep their own source paths. Mapped data does not establish current activity or complete coverage.
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon (`PHOTON_URL`; unset uses the public instance, empty disables it), then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` and the voice outline route `/api/geocode/outline` share one Nominatim queue (`NOMINATIM_URL`; unset uses the public instance, empty disables it): one request at a time at least 1.1 s apart, an identifying `User-Agent` and `Referer`, a short queue that refuses a burst, a pause after 429/403/5xx answers (honouring `Retry-After` in full), at most one retry after a network failure, and a per-install daily cap on public-instance requests (`NOMINATIM_DAILY_CAP`, default 50). Processes using the same installation state file atomically reserve the next public start and daily slot; restarts share that count, spacing and any pause. Public access fails closed if the state file cannot be read, locked, written or semantically validated. This coordination does not span independent installations or state directories, and the public service's total limit still applies across all installs, so heavy or shared deployments should set their own `NOMINATIM_URL`. An operator-selected endpoint is explicitly outside the public cap, spacing and durable-state requirement; an upstream cooldown is kept with the endpoint that returned it rather than leaking across operator URLs. Answers are cached on disk (outlines 90 days, places 30 days, no-results 24 hours); failures are not cached. Outlines are looked up only for an explicit ask, after bundled data and any configured Overpass, and only results whose type suits the ask are used. Administrative outline responses expose country/admin-1/admin-2 identity only when the selected Nominatim row has a recognized, unambiguous address type consistent with its own name; numeric rank, ambiguous types, missing fields and contradictory names do not manufacture identity. It is not used for systematic or bulk queries.
- **Street and building outlines.** Voice street highlights, pointed-at buildings and grounds use the OpenFreeMap vector tiles the traffic layer already loads. Traffic and outlines share one bounded pending fetch/decode owner per immutable XYZ tile; materialized outline geometry is charged to the retained-byte ceiling and capped by coordinate count. Consumer result caches and request lifetimes remain independent, while eviction or explicit teardown cancels queued idle projections and releases their decoded owner. No Overpass request is made unless `OVERPASS_UPSTREAMS` is configured.
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and resolves physical regions from bundled public-domain Natural Earth polygons without reverse-geocoding requests. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
- **Dynamic weather presentation.** While cockpit mode is active, `/api/weather-effects` requests current Open-Meteo observations for the aircraft/camera location, rounds coordinates into 0.1° cache cells, caches results for five minutes, and may retain a stale observation for up to 30 minutes during a transient outage. WMO condition code selects the visual family; observed cloud cover, precipitation, visibility, wind speed, and wind direction bound its strength and motion. Missing or expired weather renders no synthetic atmospheric effect, and normal globe view never renders the weather overlay.
---
@@ -95,11 +111,16 @@ The [Bhote Koshi event pack](public/events/bhote-koshi-2026/README.md), under `p
| Dataset | Folder | License | Commercial use? | Attribution |
| --------------------------------------------------------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------- |
| **Datacenters** (~4.3K) | `datacenters/` | **ODbL 1.0** (OpenStreetMap extract) | ✅ (attribution + share-alike on data) | "© OpenStreetMap contributors" |
| **Dams** (704) | `dams/` | **ODbL 1.0** (OpenInfraMap / OSM extract) | ✅ (attribution + share-alike on data) | "© OpenStreetMap contributors" (+ Open Infrastructure Map) |
| **Datacenters** (4,351) | `datacenters/` | **ODbL 1.0** (OpenStreetMap extract) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) |
| **Dams** (704) | `dams/` | **ODbL 1.0** (OpenInfraMap / OSM extract) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + Open Infrastructure Map |
| **Military area names** (36,466) | `osm_military_names/` | **ODbL 1.0** (OpenStreetMap, via Overture Maps) | ✅ (attribution + share-alike on data) | © OpenStreetMap contributors (shared credit) + Overture Maps Foundation |
| **TeleGeography Submarine Cable Map** (712 cables + 1,917 landing points) | `telegeography_submarine_cables/` | **CC BY-NC-SA 3.0** | ❌ **NonCommercial — remove for commercial use** | "© TeleGeography — submarinecablemap.com" |
| **Natural Earth physical regions** (1,046 land + 292 marine named polygons) | `natural_earth/` | **Public domain** | ✅ (no restrictions) | "Made with Natural Earth" (courtesy credit — not legally required) |
| **DataSF Analysis Neighborhoods** (41 SF neighborhood polygons) | `neighborhoods/` | **PDDL 1.0** (public domain) | ✅ (no restrictions) | "City & County of San Francisco — DataSF" (courtesy — not legally required) |
| **Natural Earth countries, states and provinces** (260 admin-0/map units + 4,587 admin-1 polygons) | `natural_earth/` | **Public domain** | ✅ (no restrictions) | "Made with Natural Earth" (courtesy credit — not legally required) |
| **US Census Bureau counties** (3,235 county polygons) | `us_census_counties/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
| **US Census Bureau places** (32,629 city, town and CDP polygons) | `us_census_places/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
| **Who's On First neighborhoods** (61,755 polygons) | `wof_neighborhoods/` | Per source: CC BY, ODC-By, Open Government licences, CC0, PDDL, public domain (folder `ATTRIBUTION.md`) | ✅ (attribution) | "Neighborhood boundaries: Who's On First and contributors"; per-source credits in the folder |
| **CCTV ground heights** (3,445 cameras) | `cctv_ground_heights/` | Precomputed camera placement heights, aligned to work with Google Photorealistic 3D Tiles (folder README) | — | — |
### ⚠️ TeleGeography is bundled but NonCommercial
@@ -112,33 +133,23 @@ The richer structured dataset is licensed separately/commercially by TeleGeograp
### ALPR camera mapping
The optional ALPR layer queries OpenStreetMap nodes tagged `man_made=surveillance`
and `surveillance:type=ALPR` (including case-insensitive semicolon multi-values)
through the shared Overpass proxy. Source/tag reference:
[OSM ALPR tagging](https://wiki.openstreetmap.org/wiki/Tag:surveillance:type%3DALPR).
No DeFlock software, branded assets or separate database is bundled. The layer
shows contributor-supplied locations and tags, not plate records, camera video,
current operating status, or exhaustive coverage. All manufacturers share the
ALPR category and marker color. Manufacturer, operator and camera type are shown
only when tagged, not inferred or independently verified. Every selected card
names OpenStreetMap in a smaller, muted source line beneath those details.
The optional ALPR layer reads OpenStreetMap `surveillance:type=ALPR` nodes
from a community-hosted hourly US/Canada extract, using detail vector tiles.
It shows mapped locations and available tags, not plate records, camera video,
current operating status or exhaustive coverage. OSM edit timestamps are not
presented as field-verification dates. Other regions show no ALPR data.
Queries are limited to a city-scale viewport (at most 3° per axis), at most
1,500 returned nodes, with outward-snapped query reuse and explicit stale/limited
coverage notices. A linked © OpenStreetMap credit appears for five
seconds when camera locations first display after enabling, then collapses;
Data attribution retains the full source/license entry, following the
[OSMF interactive-map guidance](https://osmfoundation.org/wiki/Licence/Attribution_Guidelines).
OSM data remains separately attributed under ODbL; querying
at runtime does not replace the license obligations. ODbL permits commercial
use and does not license the application code. If publicly using a derivative
database, follow its share-alike and access requirements; keep independent
datasets separate rather than treating the whole collection as MIT data.
Overpass hosting capacity and usage policies are separate from the data license.
Reads are viewport-bounded, with at most 16 tiles per country, four concurrent
reads per country, 1,500 displayed records and an hour-long decoded-tile cache.
Displayed OSM-derived layers retain a short inline OpenStreetMap credit, plus
OpenMapTiles while OpenFreeMap tiles are used; Data attribution retains the full credits. ODbL permits
commercial use with attribution and applicable database share-alike/access
obligations; it does not license the application code. Trimmed test-only tile
fixtures retain their provenance in `src/data/fixtures/README.md`.
### ODbL share-alike (datacenters, dams)
### Bundled OpenStreetMap data
The OSM-derived datasets are under the **Open Database License**. ODbL's share-alike applies to the **data / derived database, not this MIT-licensed code** — the two coexist (exactly how Open Infrastructure Map ships: MIT software + ODbL data). If you publicly distribute a _modified_ version of these databases, you must offer it under ODbL. Keep the "© OpenStreetMap contributors" notice (link: https://www.openstreetmap.org/copyright).
Bundled OSM-derived data is distributed under [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/), separately from the MIT code: [datacenters](src/data/local_data/datacenters/), [dams](src/data/local_data/dams/) (via Open Infrastructure Map), [military names](src/data/local_data/osm_military_names/), and [OSM test fixtures](src/data/fixtures/). Keep © OpenStreetMap contributors and offer modified databases under ODbL.
### NASA FIRMS acknowledgement
@@ -146,7 +157,7 @@ The OSM-derived datasets are under the **Open Database License**. ODbL's share-a
FIRMS active fires are **fetched live at runtime** (CC0 / U.S. public domain data): the
`/api/firms` server-side proxy merges the three VIIRS NRT sources (NOAA-20, NOAA-21,
Suomi-NPP) clamped to the trailing 24 h, cached 30 min to respect the shared MAP_KEY
Suomi-NPP) and MODIS NRT (Terra + Aqua), clamped to the trailing 24 h, cached 30 min to respect the shared MAP_KEY
transaction quota. Requires a free `FIRMS_MAP_KEY`
(https://firms.modaps.eosdis.nasa.gov/api/map_key/); the layer is empty without it.
The former bundled 2026-05-25 snapshot was removed 2026-07-16.
@@ -189,6 +200,141 @@ Douglas-Peucker simplification, 6-decimal rounding).
---
## Weather sources
### ECMWF IFS wind
Wind also offers ECMWF IFS 10 m forecasts from the keyless [ECMWF Open Data](https://www.ecmwf.int/en/forecasts/datasets/open-data) service. The proxy reads its JSON Lines inventory and byte-range fetches 10u/10v GRIB messages, optionally adding 2t (2 m temperature) or msl (mean sea-level pressure) from the same run and forecast time. It resamples these fields to the approximately 1° display grid. Data is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The [ECMWF Terms of Use](https://apps.ecmwf.int/datasets/licences/general/) also apply. In-app attribution identifies this service as based on ECMWF data and products, links the CC BY 4.0 licence, identifies resampling and animation as modifications, and retains the required ECMWF liability disclaimer. Both model issue time and the selected forecast valid time appear in the WEATHER panel. These are model forecasts, not observations.
### NOAA GFS wind
The optional **Wind** layer animates the global 10 m wind field from NOAA's
Global Forecast System (GFS). The `/api/wind` server-side proxy selects the
latest available 0.25° cycle, reads its `.idx` inventory, byte-range fetches only
the `UGRD`/`VGRD` 10 m GRIB2 messages, optionally adding `TMP` at 2 m or
`PRMSL` at mean sea level from the same run and forecast time. It decodes the
selected messages with ecCodes (WASM), resamples to a compact grid (1° by
default), and serves a manifest plus Float32 U/V and optional scalar values. It is keyless, cached
per cycle for an hour, and is a **forecast, not an observation**: the particles
show model flow, not measured wind. The renderer bakes bounded curves through
the sampled field and animates their phase on the GPU, with a canvas fallback.
Any display lift is a visual aid and does not change the forecast's 10 m level.
NOAA GFS data is U.S. public domain; the credit above is a courtesy and does not
imply endorsement.
For both models, optional temperature is normalized from K to °C and pressure
from Pa to hPa before delivery. A missing, malformed or timed-out companion field
does not discard valid wind: the manifest marks that field unavailable, and the
UI retains wind without inventing scalar values. The proxy caches by model and
requested field, with bounded current/previous grids and shared in-flight loads.
Wind speed shading is computed locally from U/V and needs no companion download.
The legend and map-center reading explain units and forecast validity; the
animation is a visual flow through one forecast, not advancing forecast time.
Optional globe relief uses existing terrain vertex normals for view lighting;
without those normals it shades global globe curvature only, not local terrain
relief. Neither mode represents measured sunlight or a new elevation source.
This Wind prototype adds no cloud-volume or radar data. Mapped.earth's public
bundles were studied for rendering ideas, but
no code or assets were reused and no application licence granting reuse was
found. Its presentation is not a weather-data source for this implementation.
### Observed weather: NOAA nowCOAST
The Weather section provides keyless, observed **MRMS radar reflectivity** for the
contiguous United States and **infrared satellite imagery**. Fixed upstream WMS
services: `https://nowcoast.noaa.gov/geoserver/observations/weather_radar/ows` and
`https://nowcoast.noaa.gov/geoserver/observations/satellite/ows`.
- `conus_base_reflectivity_mosaic`: approximately 1 km, usually 4-minute updates.
dBZ measures radar reflectivity, not rainfall rate, probability or future rain.
Coverage gaps do not mean no precipitation. The numeric legend follows NOAA's
`weather_radar_base_reflectivity` style.
- `goes_longwave_imagery`: GOES-19/18 Band 14, approximately 2 km, 5-minute updates,
regional North America. **Clouds only** softly dims pixels with a linear
brightness ramp from 0.40 to 0.70; **Full image** retains source alpha. This
is a display filter, not a cloud mask or measured cloud volume.
- `global_longwave_imagery_mosaic`: approximately 3 km, hourly, nominal 60°S–60°N
coverage and typically 2–3-hour latency. It is slower global context.
The UI displays the exact advertised observation time separately from acquisition;
latest means the newest available observation, not zero-delay real time. Up to 13
recent advertised frames can be replayed as history. No nowcast is synthesized.
Tiles use WMS 1.1.1 EPSG:4326 longitude/latitude bounds; Cesium's geographic 2×1
root grid is capped at level 6 for radar and regional GOES. Global infrared uses
one fixed 2048×1024 geographic image on both hosts: the source's request-dependent
contrast otherwise creates brightness seams between tiles. The browser decodes
and filters it once, then crops 256 px tiles bounded to the manifest extent at
maximum level 3. This broad context view has a coarser display resolution than
its 3 km source. No browser reprojection is used. Browser imagery owns a displayed
frame and a staging frame, retaining the outgoing layer for one render after
admission.
On Google 3D Tiles each observed product is instead a raised shell showing one
whole-extent image per frame (up to 4096×2048; global infrared 2048×1024). Every
product except global infrared also draws a 4096×2048 detail window around the
view, requested with a 2:1 `bbox` inside the product bounds, rounded to 0.25°.
The proxy has an 8-request concurrency budget and a 12-second deadline; tiles
(256, 512 or 1024 px) are capped at 1 MiB or 4 bytes per pixel plus 64 KiB,
whichever is larger, and whole-extent and detail images at 16 MiB. Its shared
cache holds up to 128 images and 16 MiB, each exact-time image for at most 24
hours.
Metadata refreshes every two minutes with explicitly stale last-good fallback.
No key, new dependency, image reprojection job, or full-disk image download is needed.
Credit: NOAA nowCOAST, NWS/OAR MRMS, NESDIS GOES and global satellite partners.
[NOAA disclaimer](https://oceanservice.noaa.gov/disclaimer.html).
Community context: [#85](https://github.com/bilawalsidhu/gods-eye-view/issues/85),
[#588 radar](https://github.com/bilawalsidhu/gods-eye-view/pull/588), and
[#457 clouds](https://github.com/bilawalsidhu/gods-eye-view/pull/457).
This implementation is original; those contributions have not been merged here.
### Weather: NHC/CPHC cyclone advisories
- **Sources:** [NHC current tropical cyclone status](https://www.nhc.noaa.gov/CurrentStorms.json) and
[NOAA tropical weather summary GIS](https://mapservices.weather.noaa.gov/tropical/rest/services/tropical/NHC_tropical_weather_summary/MapServer).
- **Rights/credit:** NOAA/NWS National Hurricane Center / Central Pacific Hurricane Center;
[NWS public-data terms](https://www.weather.gov/disclaimer). Fetch official products at runtime;
no bundled advisory archive and no NOAA endorsement implied.
- **Coverage/meaning:** Atlantic and eastern/central North Pacific, not global cyclone coverage.
Status position time and advisory issue time remain separate. Forecast track, points and cone
require matching advisory numbers across all GIS parts. Cone means uncertainty in the
forecast center track, not storm size or a complete hazard boundary.
- **Delivery:** keyless same-origin `/api/cyclones`, five-minute singleflight cache,
fixed upstream endpoints, bounded bodies/geometry/deadline, explicit unavailable/stale states.
Direct status fetching is unsuitable in browsers because NHC does not advertise CORS.
### Weather: observed lightning density
- **Source:** [NOAA nowCOAST lightning detection WMS](https://nowcoast.noaa.gov/geoserver/observations/lightning_detection/ows),
fixed `ldn_lightning_strike_density` layer and `lightning_density` style.
- **Rights/credit:** NOAA/NWS nowCOAST; derived from Vaisala NLDN/GLD360.
[Official product description and public distribution terms](https://ocean.weather.gov/lightning/lightning_pdd.php)
permit distribution of this NOAA Level-5 derived product, not raw Vaisala detections.
- **Meaning:** 15-minute accumulated density on an approximately 8 km grid;
source color scale is strikes per km² per minute ×10³. Not individual GLM optical flashes,
a live ground-strike counter, an all-clear indication, or global coverage.
- **Coverage:** 110°E across the Pacific/Americas to 0°, 25°S–80°N. Display exact advertised
observations; ten-minute metadata refresh, source-matched colors, bounded shared imagery cache.
### Community weather prior art
The weather design builds on Gustavo Beneduzi's retained GFS/ECMWF contribution
commits ([#459](https://github.com/bilawalsidhu/gods-eye-view/pull/459),
[#464](https://github.com/bilawalsidhu/gods-eye-view/pull/464)). Related community
proposals informed the observed-weather experience: Sandiv D's on-demand radar
and imagery controls ([#588](https://github.com/bilawalsidhu/gods-eye-view/pull/588));
Gustavo Beneduzi's GOES and GLM source work
([#457](https://github.com/bilawalsidhu/gods-eye-view/pull/457),
[#458](https://github.com/bilawalsidhu/gods-eye-view/pull/458)); and HadiMuhammed's
NOAA hazard provenance and freshness work
([#414](https://github.com/bilawalsidhu/gods-eye-view/pull/414)).
The fixed nowCOAST imagery and NHC advisory implementations are original maintainer
work; this acknowledgement does not represent merging those four PRs or equate
NOAA density imagery with the raw GLM product.
---
## In-app attribution
The required Google Maps / Cesium credit renders on the on-globe credit line (`#cesium-credits`, bottom-left) and must stay visible — including in clean-view and recording modes (the whole line, logo + "Google Maps" + the "Data attribution" link, stays on screen; only the GEV panels/HUD fade). The layer-specific credits (adsb.lol, TeleGeography, OSM datacenters/dams/roads, NASA FIRMS, CelesTrak, USGS, City of Austin, Fintraffic, GBFS, Radio Browser, OpenSky, AISStream) are registered into the expandable **"Data attribution"** popover on that credit line via `viewer.creditDisplay.addStaticCredit(new Cesium.Credit(html, /* showOnScreen */ false))` — see `src/data/dataCredits.js`. When you add a new data source, add its license and attribution to this file **and** append an entry to `DATA_CREDITS` in `src/data/dataCredits.js` so it surfaces in the app.
The required Google Maps / Cesium credit renders on the on-globe credit line (`#cesium-credits`, bottom-left) and must stay visible — including in clean-view and recording modes (the whole line, logo + "Google Maps" + the "Data attribution" link, stays on screen; only the GEV panels/HUD fade). The layer-specific credits (adsb.lol, adsbdb, TeleGeography, OSM datacenters/dams/roads, NASA FIRMS, CelesTrak, USGS, City of Austin, Fintraffic, GBFS, Radio Browser, OpenSky, AISStream) are registered into the expandable **"Data attribution"** popover on that credit line via `viewer.creditDisplay.addStaticCredit(new Cesium.Credit(html, /* showOnScreen */ false))` — see `src/data/dataCredits.js`. When you add a new data source, add its license and attribution to this file **and** append an entry to `DATA_CREDITS` in `src/data/dataCredits.js` so it surfaces in the app.
+65 -19
View File
@@ -2,7 +2,7 @@
# 🌐 God's Eye View
[![CI](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml)
[![CI](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bilawalsidhu/gods-eye-view/actions/workflows/ci.yml) [![Reached #1 on GitHub Trending](https://img.shields.io/badge/%231_GitHub_Trending-thank_you!-F0A63C?style=flat-square&logo=github)](https://x.com/bilawalsidhu/status/2093798887815348521)
### A spy-satellite simulator in your browser — then you realize the sources are public and the data is real.
@@ -18,8 +18,6 @@ _No place left behind._
▶️ **From the project behind the viral God's Eye View series** _(formerly WorldView)_ — [5M+ on YouTube](https://youtube.com/playlist?list=PL6qSg2I-7_koPbDnSMo0QeeHX_RknA2uv&si=nBGYMoHWQw41v93Q) · [25M+ across socials](https://www.google.com/search?q=god%27s+eye+view)
[![Reached #1 on GitHub Trending](https://img.shields.io/badge/%231_GitHub_Trending-thank_you!-F0A63C?style=flat-square&logo=github)](https://x.com/bilawalsidhu/status/2093798887815348521)
🏆 **Reached #1 on GitHub Trending, daily and weekly · August 2026**
**[#8 Product of the Day](https://www.producthunt.com/products/god-s-eye-view?launch=god-s-eye-view)** · Hunted by Chris Messina, creator of the hashtag
@@ -34,7 +32,7 @@ _“pretty cool”_ — [Brendan Eich](https://x.com/BrendanEich/status/20945920
<div align="center">
**[Quick Start](#-quick-start) · [First Five Minutes](#-the-first-five-minutes) · [Talk to It](#-talk-to-it) · [What's Live](#-whats-on-the-globe) · [Under the Hood](#-under-the-hood) · [Keys & Costs](#-api-keys)**
**[Quick Start](#-quick-start) · [First Five Minutes](#-the-first-five-minutes) · [Talk to It](#-talk-to-it) · [What's Live](#-whats-on-the-globe) · [Under the Hood](#-under-the-hood) · [Keys & Costs](#-api-keys) · [Contributing](CONTRIBUTING.md)**
</div>
@@ -70,6 +68,14 @@ Start with the included data sources, then add your own. Each layer is a separat
- **🎥 Scene director:** Capture cinematic camera tours for clips and demos.
- **🔗 Share Links:** Camera, style, layers, and even one tracked target serialize into a URL — a live target is a handoff, not a bookmark.
- **🏠 Reset Globe:** One control — or one sentence — back to the full Earth.
- **🌦️ Weather:** Animate GFS or ECMWF forecast wind, replay observed radar, satellite clouds and lightning on one timeline, and follow NHC/CPHC cyclone tracks. No key needed.
- **📷 Mapped ALPR cameras:** License-plate-reader camera locations tagged in OpenStreetMap, one city at a time. Locations and tags only; no key needed.
- **🔭 Satellite passes:** Ask by voice when any loaded satellite next rises over you: rise, peak and set times, and whether you can see it.
- **🔎 Analyst answers:** Count, filter and rank satellites, datacenters and dams by voice, alongside flights, ships, fires and quakes; answers say when a feed is stale or on a fallback.
- **🧭 Tilt and North Up:** Switch between straight-down and a 35° oblique, or put north at the top, while a tracked target stays centered.
- **🔍 Keyless search:** Type coordinates or a bundled city or landmark to fly there with no network request; other names use Google when configured, then Photon and Nominatim.
- **📦 Shareable scenes:** Import, preview and share Director scenes as files or bundles, with camera anchors, authored moves and data packs.
- **🌊 Nepal flood scene:** Replay the Bhote Koshi flood: flood path, witness sources and before-and-after imagery (bundled data is non-commercial; see [DATA_SOURCES.md](DATA_SOURCES.md)).
---
@@ -90,9 +96,13 @@ cameras, radio, and launches are available without keys.
For photorealistic 3D, add a **Cesium ion token** for eligible personal,
non-commercial use, or a **Google Maps key** for the direct, metered route and
in-app place search. Provider terms and quotas apply. Add keys through the
Google place search. Provider terms and quotas apply. Add keys through the
app's **POWER UP** panel; [Keys & Costs](#-api-keys) explains the options.
> **Already installed?** Update to the latest version. Older versions query
> public OpenStreetMap Overpass servers, which now refuse them, so Traffic,
> Mapped Installations and ALPR stay empty until you update.
### Path 1 — One click, no terminal
1. Install or update [Pinokio](https://desktop.pinokio.co/) to **8.2 or later**.
@@ -153,8 +163,8 @@ reopens the same panel.
- **What to get first:** the free [Cesium ion](https://cesium.com/ion) token
(eligible personal, non-commercial use; current terms and quotas apply) for
photorealistic 3D and world terrain; a Google Maps key only for the
billing-enabled, metered route + place search; OpenAI when you want to talk
to the world. Full map, costs included, in [Keys & Costs](#-api-keys).
billing-enabled, metered route + Google place search; OpenAI when you want to
talk to the world. Full map, costs included, in [Keys & Costs](#-api-keys).
<details>
<summary>Older Pinokio versions and credential storage</summary>
@@ -188,7 +198,7 @@ Choose a first-run mission, or try these in order. The GIFs show Google Photorea
![Moving from a full airport overhead down to close taxiway inspection with 3D flight models](docs/media/start-here/airport-ground-traffic-google-3d.gif)
4. **Look through a public camera.** Turn on **CCTV** over Austin, London, California, or Finland. The feeds aren't webcam embeds — they project _into_ the 3D city. Cycle coverage to **VIEWSHED** and every camera draws its estimated coverage volume — where it reaches, and where it goes blind.
4. **Look through a public camera.** Turn on **Cameras** over Austin, London, California, Finland, or Delaware (live video). The feeds aren't webcam embeds — they project _into_ the 3D city. Cycle coverage to **VIEWSHED** and every camera draws its estimated coverage volume — where it reaches, and where it goes blind.
![Diving into an Austin intersection with a live public camera projected into the 3D scene](docs/media/03-austin-cctv.gif)
@@ -203,7 +213,7 @@ Choose a first-run mission, or try these in order. The GIFs show Google Photorea
7. **Talk to it** _(needs an OpenAI key)_: _"Take me to LAX and select the nearest airborne aircraft."_
8. **Come home.** Hit **Reset Globe** — or just say _"zoom out to a globe view."_
**Keyboard:** `1`–`7` visual styles · `H` HUD · `D` detection · `C` cockpit · `Esc` out.
**Keyboard:** `1`–`7` visual styles · `H` HUD · `D` detection · `C` cockpit · `` ` `` frame rate · `Esc` out. Trackpad pinch zooms the globe.
---
@@ -235,7 +245,7 @@ Click **GEV MIC**, grant the microphone, and just talk. This is more than a voic
- **🎬 Cinematic framing.** _"Show me the planes overhead"_ pulls the camera back, angles it, and frames the live traffic like a director.
- **🔒 Honest and secure.** The agent only confirms actions that succeeded. Your `OPENAI_API_KEY` never touches the browser; the client only gets a short-lived session token.
Twenty-eight tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook:
Twenty-nine tools, four jobs — the commands below come straight from the product's voice test suite and tool playbook:
**🎥 Direct it** — drone-operator camera verbs:
@@ -253,7 +263,7 @@ Twenty-eight tools, four jobs — the commands below come straight from the prod
**🔎 Interrogate it** — analyst queries against the live layers:
> 🗣️ _"How many flights are over Texas right now?"_ · _"Which ships are headed to Oakland?"_ · _"What is the biggest fire near Los Angeles?"_ · _"Is anything flying above forty thousand feet?"_ · _"When does the ISS pass over next?"_
> 🗣️ _"How many flights are over Texas right now?"_ · _"Which ships are headed to Oakland?"_ · _"What is the biggest fire near Los Angeles?"_ · _"Is anything flying above forty thousand feet?"_ · _"When does the ISS pass over next?"_ · _"How many datacenters are in view?"_
**🎛️ Operate it** — the whole console, hands-free:
@@ -271,7 +281,7 @@ _Ask for radio near anywhere and the globe starts broadcasting — every station
## 🛰️ What's on the Globe
Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
Nineteen layers and map sources. **Seventeen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
| Layer | What you get | Source | Auth |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------- |
@@ -282,7 +292,8 @@ Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer add
| 🛰️ **Satellites** | 838-object catalog, color-coded by class with a live legend — the **DENSE** chip drops in the whole Starlink shell | CelesTrak | 🟢 |
| 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 |
| 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds |
| 📹 **CCTV Mesh** | ~3,600 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · New South Wales (Live Traffic NSW) · Calgary. Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
| 📹 **CCTV Mesh** | ~3,600 public cameras projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary. Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
| 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 |
| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 |
| 🚌 **Transit** | Live buses, trams, metros, trains and ferries with delayed playback between reports, selected-vehicle trails, and mode-coloured DETECT labels — Boston, Austin, Minneapolis, Helsinki, the Netherlands, Norway, South East Queensland | Operator GTFS-Realtime feeds | 🟢 |
| 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 |
@@ -290,6 +301,13 @@ Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer add
| 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 |
| 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🟡 optional token raises the allowance) |
| 🎖️ **Mapped Installations** | Viewport-bounded military-site context from community mapping — incomplete by nature, and labeled that way | OpenStreetMap | 🟢 |
| 🌬️ **Wind** | Animated 10 m forecast wind with an optional speed, temperature or pressure color field and a reading at map center | NOAA GFS / ECMWF IFS | 🟢 |
| 🌦️ **Observed Weather** | Rain radar (contiguous US), Satellite clouds (GOES regional or global infrared, Clouds only or Full image) and Lightning density (Americas and Pacific) on one history timeline | NOAA nowCOAST | 🟢 |
| 🌀 **Cyclones** | Advisory positions, forecast tracks and uncertainty cones for the Atlantic and eastern/central North Pacific | NOAA NHC / CPHC | 🟢 |
**Weather, keyless.** **Wind** animates 10 m forecast flow from NOAA GFS or ECMWF IFS, with an optional color field and a reading at map center. **Rain radar**, **Satellite clouds** and **Lightning density** are observations on one history timeline in the **WEATHER** panel (step back, **Play**, **Latest**); **Cyclone advisories** draw NHC and CPHC positions, forecast tracks and cones. On Google 3D Tiles the observed layers float above the city as translucent shells, sharpened around your view.
In the app, **Data Layers** groups them as Movement, Cameras, Infrastructure, Events, Weather and Utilities.
**The basemap ladder — what each tier buys you:**
@@ -297,7 +315,7 @@ Fifteen layers and map sources. **Thirteen have a keyless path.** Some offer add
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 🟢 Nothing | Esri World Imagery satellite basemap + keyless terrain, in 2D. OSM takes over automatically if Esri is unreachable; if terrain is unavailable the globe continues without it |
| 🟡 A free Cesium ion token | **Google Photorealistic 3D cities** and world terrain — eligible personal, non-commercial use; current ion terms and quotas apply |
| 🔴 A Google Maps key | The same 3D direct from Google, plus in-app place search — the billing-enabled, metered route |
| 🔴 A Google Maps key | The same 3D direct from Google, plus Google place search — the billing-enabled, metered route |
![A reconstructed Falcon 9 ascent climbing and curving into its projected orbit](docs/media/08-falcon9-replay.gif)
@@ -364,7 +382,8 @@ src/
├── hud.js # Intelligence HUD + AI scene summary
├── keySetup.js # POWER UP panel — in-app provider keys (dev server only)
├── mapStackController.js # Basemap switching — Google 3D / Esri / OSM / ion stacks
├── voice/ # OpenAI Realtime session + 28 voice tools
├── voice/ # OpenAI Realtime session + 29 voice tools
├── layers/ # Layer components — weather, wind, cyclones, transit, ALPR, …
├── data/ # One module per layer + orchestration + context store
│ ├── iconOrientation.js # Screen-projected headings + horizon cull
│ └── local_data/ # Bundled datasets (per-folder provenance)
@@ -384,6 +403,11 @@ each provider enables; none is required to start. See the
[setup instructions](#then-power-it-up--in-the-app-not-in-a-file) for storage
and configuration details.
No key needed for Wind, Rain radar, Satellite clouds, Lightning density,
Cyclone advisories, Mapped ALPR Cameras, Transit, Directions, Delaware live
video, and coordinate or bundled-place search. A layer row waiting on a key
names it.
### Choose the capabilities you want
Six keys. Four have a free tier, and the two 🔴 ones are metered:
@@ -414,8 +438,10 @@ Add these if you need higher polling allowances.
where each configured provider was found without printing credential values.
On macOS its Keychain-aware result previews `./scripts/dev-fresh.sh`; plain
`npm run dev` reads only explicit environment and Vite dotenv values. The
OpenSky summary reports only OAuth client-pair presence, not the resolved
runtime mode or credential validity; Basic and credentials-file modes remain
OpenSky summary reports keyless anonymous access for explicit `anon` or an
OAuth mode without a client pair, retains presence-only wording for a complete
OAuth pair, and identifies selected Basic or auto mode without guessing which
credentials runtime will accept. Basic and credentials-file modes remain
advanced `dev-fresh.sh` configuration.
<details>
@@ -439,15 +465,35 @@ OpenSky can run fully anonymous (`OPENSKY_AUTH_MODE=anon`), or import OAuth cred
</details>
<details>
<summary>Live video cameras (HLS)</summary>
CCTV sources with `"feedType": "hls"` and a registered HTTP(S) `.m3u8`
URL play through a lazily loaded hls.js decoder shared by the monitor plane
and panel. DelDOT uses its official HTTPS HLS catalog links. Disable that
pack with `CCTV_DELDOT_ENABLED=0`.
The server allows two concurrent sessions. Each retains at most 12 segments
and 24 MiB in memory; individual downloads are capped at 4 MiB with a ten
second deadline. There are no segment files or ffmpeg processes. Redirects,
off-origin references, encrypted playlists and non-MPEG-TS segments are refused.
Each decoder has its own client lease (at most eight per session), including
native HLS. Closing it releases only that lease; abandoned leases expire after
15 seconds without access. The last release stops upstream work. Failed live video
uses the existing still/Street View/synthetic fallback, which is not live video.
RTMP-only sources are not supported by this integration.
</details>
### 💸 What it actually costs
Honest numbers, roughly, as of mid-2026 — always check the provider pricing pages:
| | Cost reality |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, bundled datasets. |
| **🟢 Most layers** | **$0, no signup.** OpenSky anon, USGS, CelesTrak, adsb.lol, city CCTV, Radio Browser, GBFS, Launch Library 2, transit feeds, OSRM routing, NOAA and ECMWF weather, OpenStreetMap ALPR mapping, bundled datasets. |
| **🟡 The free-key tier** | **$0 with a signup.** AISStream, FIRMS, TomTom, OpenSky, plus Cesium ion for eligible personal/non-commercial use. Provider quotas and eligibility still apply. |
| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for GEV place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. |
| **🗺️ Google 3D tiles** | **Free through an eligible Cesium ion Community account within its quota; metered through a direct Google key.** Use the direct route for Google place search or commercial deployment, verify current provider terms, and set budget alerts where billing is enabled. |
| **🔴 OpenAI voice** | **The one that costs real money — so the app meters it for you.** Realtime audio runs a few cents per active minute; an evening of heavy use is single-digit dollars. A live session-spend readout sits next to the mic, with an STD/MINI model toggle, a $2 warning, and a **$5 hard cap that ends the session**. The voice context window is kept deliberately short too. |
Google's direct 3D route is surprisingly generous: the first 1,000 Photorealistic
+2
View File
@@ -62,9 +62,11 @@ browser-key fallback for existing single-key setups.
The data proxies under `server/providers/` are written so the browser cannot turn the server into an open relay:
- **No arbitrary-URL fetching.** The CCTV frame proxy fetches only server-registered camera/frame URLs — clients cannot pass an upstream URL to fetch (SSRF mitigation). Other proxies target fixed upstream hosts.
- **Live HLS is bounded and origin-pinned.** Registered HTTP(S) HLS sources use a Node puller that rejects redirects and off-origin playlist references, bounds playlists and segment bodies while streaming, and aborts timed-out or released sessions. At most two sessions retain 12 segments/24 MiB each, plus bounded download buffers. Stores are memory-only and idle sessions expire after 15 seconds. RTMP/ffmpeg execution is not enabled. Locally configured source URLs remain an operator trust boundary.
- **Radio is not an audio relay.** `/api/radio/stations` contacts only allowlisted Radio Browser HTTPS hosts and paths, rejects redirects, rejects any hostname with a loopback/private/link-local/metadata/non-public A or AAAA result, and pins each TLS connection to a validated address. It returns normalized public HTTPS stream URLs; `/api/radio/click/:uuid` applies the same destination policy and accepts only station IDs from the current bounded catalog. The browser then connects directly to the broadcaster after an explicit playback action, so the broadcaster sees the listener's IP address. GEV never proxies, caches, records, or redistributes audio.
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
- **Transit fetches registered feeds only.** `/api/transit/vehicles/<id>` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator.
- **Local receiver feeds are operator-configured and local-only.** `/api/local-receivers/aircraft` reads only the `aircraft.json` URLs in the server's `LOCAL_RECEIVER_FEEDS`; the browser never supplies an address. Each host must pass the shared tap address rule (`src/data/tapAddress.js`: loopback, RFC1918, `localhost`, `*.local`; no link-local, IPv6 or other names), the scheme must be http(s), the path must end in `aircraft.json`, and credentials, queries and fragments are refused. Invalid entries are logged at startup and never fetched. Reads use a 2 s timeout, refuse redirects, cap bodies at 2 MB and share one read per second; responses name each feed by band only and never carry upstream error text.
- **Response-size caps and timeouts** on proxied responses.
- **Sanitized errors** — internal error details are not echoed back to clients.
- **Coalesced OAuth refresh** and cached successful responses only (OpenSky).
+17
View File
@@ -210,6 +210,18 @@ While recording, call out anything in these areas — this is the feedback I mos
happened? Any command it misunderstood?
- **Anything that looks wrong, janky, or surprising** — screenshot it; that's the gold.
---
## 6. Live video CCTV (HTTPS HLS) — soak, no mic
> Requires the DelDOT pack (loads by default; `CCTV_DELDOT_ENABLED=0` disables it).
1. Turn on **CCTV**, fly to Dover, DE, and activate **DE 1 @ Main Toll Plaza**. ✅ Within ~30 s the monitor plane and the panel preview both show moving video,.
2. Leave it for **five minutes**. ✅ Continuous playback; at most a short hitch about once a minute (the agency restarts its stream on a timer). ❌ A freeze that does not recover, or the panel picture stopping while the plane keeps moving.
3. Toggle **PROJECTION** off. ✅ The panel keeps playing; the plane and the active camera's cone hide.
4. Switch to **DE 8 @ Saulsbury Rd**, then back. ✅ Each switch resumes within ~10 s with no stale frame from the previous camera.
5. Optional: DevTools → Network, filter `media/`. ✅ Playlist responses carry `X-CCTV-Source: hls-pull`; segments return `video/mp2t`.
## If something looks off
- **Grey globe / slow tiles:** wait a few seconds after a camera flight; photoreal streams in.
@@ -237,3 +249,8 @@ FIRMS refactoring can be checked without a configured server key using
fixture mode exercises populated display, aggregation, cards, keyless and stale
responses, and selection/camera handoff. It does not establish live-source
acceptance; omit `--fixtures` with a configured FIRMS key for that check.
Director authoring and sharing acceptance: `node scripts/qa-director-sharing.mjs`
checks installed import previews, draft validation, file-bundle round trips,
cancellation, resource ownership and narrow-screen controls. Run alongside the
scene-controls, camera, pack, interaction and timing harnesses.
+21
View File
@@ -0,0 +1,21 @@
# Third-party software notices
The project depends on third-party npm packages whose own licenses apply. This
file records the packages added for the browser-local SDR feature; the complete
resolved dependency inventory remains in `package-lock.json`.
## Web RTL-SDR
- Package: `@jtarrio/webrtlsdr` 3.0.6
- Author: Jacobo Tarrio Barreiro; portions copyright Google Inc.
- Source: <https://github.com/jtarrio/webrtlsdr>
- License: Apache License 2.0
- License text: <https://www.apache.org/licenses/LICENSE-2.0>
## Signals
- Package: `@jtarrio/signals` 0.10.1
- Author: Jacobo Tarrio Barreiro
- Source: <https://github.com/jtarrio/signals>
- License: Apache License 2.0
- License text: <https://www.apache.org/licenses/LICENSE-2.0>
+20
View File
@@ -7,12 +7,28 @@ export function createBrowserViteConfig({
publicDir,
googleApiKey,
cesiumToken,
photonUrl,
host = 'localhost',
port = 4173,
command,
} = {}) {
return {
plugins: [cesium(), applicationHtmlPlugin(), ...plugins],
...(publicDir === undefined ? {} : { publicDir }),
// A production build must not clean the dependency cache a running dev
// server is still serving optimized module URLs from.
...(command === 'build' ? { cacheDir: 'node_modules/.vite-build' } : {}),
optimizeDeps: {
// First reached through the SDR worker or a dynamic import. Pre-bundle
// them at startup so first use cannot invalidate already-transformed
// URLs with Vite's "Outdated Optimize Dep" 504 response.
include: [
'@jtarrio/signals/demod/demodulator.js',
'@jtarrio/signals/demod/modes.js',
'@jtarrio/webrtlsdr/rtlsdr.js',
'egm96-universal',
],
},
server: {
host: host || 'localhost',
port: parseInt(port, 10) || 4173,
@@ -32,6 +48,10 @@ export function createBrowserViteConfig({
define: {
'import.meta.env.GOOGLE_MAPS_API_KEY': JSON.stringify(googleApiKey),
'import.meta.env.CESIUM_ION_TOKEN': JSON.stringify(cesiumToken),
// Only when set: '' disables Photon in the browser, a URL replaces it.
...(photonUrl === undefined
? {}
: { 'import.meta.env.PHOTON_URL': JSON.stringify(String(photonUrl)) }),
},
build: { chunkSizeWarningLimit: 1500 },
};
+16 -9
View File
@@ -105,14 +105,17 @@ checks when changing this wiring.
`src/search` owns forward/reverse geocoding, place search and route provider
interfaces. `createStandaloneApplication({ geospatial })` accepts `endpoints`
and `providers`; the defaults preserve Google/Photon search, Google place context
and the local OSRM route proxy. Location, annotations, HUD labels and voice use
the composed service. Provider-specific credentials belong in the selected
transport; server secrets never belong in browser configuration.
and `providers`. Default forward search tries decimal coordinates and bundled
place names (`presets`) first, without a network request, then Google when a key
is configured, Photon, and the local `/api/geocode` Nominatim route. The
defaults also keep Google place context and the local OSRM route proxy.
Location, annotations, HUD labels and voice use the composed service.
Provider-specific credentials belong in the selected transport; server secrets
never belong in browser configuration.
For a compatible protocol, configure `geocode`, `photon`, `reverse`, `textSearch`,
`nearby` or `route` endpoints. These are developer-selected configuration, not
URLs accepted from page queries or model arguments. A different protocol supplies
For a compatible protocol, configure `geocode`, `photon`, `nominatim`, `reverse`,
`textSearch`, `nearby` or `route` endpoints. These are developer-selected
configuration, not URLs accepted from page queries or model arguments. A different protocol supplies
an adapter function instead. The exported `createDefaultPlaceSearch` constructor
supports the same options for direct composition:
@@ -181,10 +184,14 @@ separate constructors. Pass these to `createApplication`; use its `signal` and
`defer` for ownership. The compatibility catalog is still page-scoped: construct
one application per page. Controls, actions and renderers use the same layer
instances. Configure sources with `application/sources` before registration or
state restoration, and release them after the consumers stop.
state restoration, and release them after the consumers stop. The catalog
constructor (`application/layers`) also requires `wind`, `weather` and
`cyclones` sources with `getSnapshot` for the Weather layers, and owns the
observed-history clock those layers share for the catalog's lifetime.
`application/services` accepts boundary, terrain, regional-context, weather and
summary services. `application/requests` supplies the existing HTTP protocols
summary services; its weather service supplies cockpit local conditions, not the
Weather layers. `application/requests` supplies the existing HTTP protocols
with configurable endpoints and a scoped transport. Changing an endpoint works
only for a compatible protocol; another protocol supplies a service adapter.
Cancellation discards late response bodies, and replacing a source invalidates
+91 -3
View File
@@ -17,6 +17,8 @@ until deliberately adopted.
Generated output, local configuration, browser evidence and bundled datasets are
excluded. The formatter validates every entry before writing any file.
New world text labels must use the world-overlay host; Cesium text labels are forbidden.
## Current component ownership
Package imports use `gods-eye-view`; `package.json` is the authoritative
@@ -277,6 +279,90 @@ share restoration and movement of controls between containers remain caller-owne
Existing helper imports from `cockpitMath.js` and `rightRailPolicy.js` remain
compatible through re-exports.
### Adding a right-rail readout panel
Add the shell in `src/ui/templates/context.html` inside `#right-context-rail`:
a `.panel-collapsible` with a stable `id` and matching `data-panel-id`, the
existing `.panel-header`, collapse button and a dedicated body. Register it
with the existing panel chrome/layout owners; do not allocate rail space in
the body renderer. Set `hidden` while empty so the allocator ignores it, and
expand through the existing collapse button only on first appearance per page
session, and only when the chrome marked the restored state as a default
(`data-collapsed-preference`), never over a stored or shared choice. Preserve
the user's later collapse choice, including body remounts. Mark the body
`data-rail-scroller` when it scrolls: the rail's measuring pass lifts its
`max-height` and puts the scroll offset back afterwards. Panel bodies only
scroll on an open-card change, with one write, never on a refresh.
Compose `createRailCards({ container, document, onParams, onOpen })` and
`createRailTimeline({ container, document, onCommit, onPreview, onStep,
onLatest, onPlay })`. Cards reconcile `{ id, title, badge, open, compact,
compactStatus, blocks }` in place. Their full-width native button headers expose
`aria-expanded` and `aria-controls`; articles expose `data-open`. The feature
adapter owns accordion state. Generic cards do not select data or enable layers.
`blocks` is an ordered array of loose, optional `{ id, type, ... }` descriptors,
rendered by `railCardBlocks.js`. There are no per-section disclosures:
- `summary`: `text`, a single summary line.
- `list`: `list: { ariaLabel, items }`, reconciled by `syncRowList` in `rowList.js`.
- `lines`: keyed `lines: [{ id, text, muted? }]` for details and status.
- `settings`: `settings: [{ id, label, chips }]`; each labelled row has a 56 px
label column and a wrapping `syncChipGroup` from `chipGroup.js`.
- `legend`: `legend: { colors, labels, units?, zeroIndex?, categorical? }` for a
ramp/scale (including a physical freezing anchor) or separate keyed swatches.
- `actions`: keyed `actions: [{ id, label, params?, href?, onClick?, disabled?,
hint? }]` in a footer, using buttons or safe new-tab links.
- `result`: `label`, keyed `lines`, and optional `clear: { params | onClick }`;
its accessible × button sits beside the title.
Every block and control has a stable id. Lists, settings, actions and results
can appear in any requested order, including a scene panel's rectangle action,
results list, then selected-scene details. Chip/list/action `params` dispatch
through `onParams(cardId, params)`; the feature adapter supplies
`setLayerParams(id, params, { origin: 'user' })`. Current descriptors and disabled
state are read at dispatch. Identical updates do not move nodes or write DOM;
focused controls survive feed refreshes.
Weather descriptors retain top-level `chips`, `list` and `legend` for non-DOM
consumers. They expose `summary.settings`, `summary.actions` and optional
`summary.result`; wind also retains its raw captured `summary.reading`.
`readout: true` layer rows render only the toggle and source/meta line; their
subscriptions still refresh the cards. Cards own configuration and readings.
Weather orders cyclones, wind, then a bordered Observed history group containing
the timeline and active radar, satellite and lightning cards. The group heading
and scope name its active products. The timeline stays visible whenever any
observed product is enabled; fewer than two ticks disables transport. Pass
`heading: false` to the timeline when the containing group owns the heading.
Exactly one active card is open. Header clicks and newly enabled ids choose it
(the last new entry wins a batch); refreshes and clock ticks retain it. First
appearance uses cyclones with storms, otherwise the first card. Disabling the
open layer falls back to the first remaining card. A WeakMap per document keeps
explicit choices through body remounts without adding share state. Card opening,
storm selection and layer enabling remain independent.
Optional card/badge/slider class names provide feature styling. Card titles use
`.data-name` typography; meta/time/badges use `.data-toggle-meta` typography;
setting/result labels use `.panel-title`; chips retain `.data-toggle-chip`.
Reserve one status line, even without a legend. Use app tokens. Wind's footer
reads the map center into a result after the action. The captured location stays
fixed until another read; model changes resample that location, units reformat
it, and × clears both reading and map marker.
The timeline shows endpoint times and receives ticks, index, mode, playing,
readout and disabled state. Its preview callback receives the tick and index and
may return readout text; commits receive the same values after a 150 ms coalesced
drag or final change. Feature adapters own data mapping and service subscriptions;
generic modules own DOM/listeners/timers and release them in `destroy()`.
Keep persistence with panel chrome: collapse uses
`godsEyeView.v6.panelCollapsed.<panelId>`, position uses
`godsEyeView.v8.panelPos.<panelId>`, and share state uses the registered panel id.
Do not rename existing
ids or reset persistence versions when adding a body. The WEATHER adapter is
`src/ui/weatherPanel.js`; its observed-history selection remains transient.
## Visual input
`gods-eye-view/ui/input` exports `bindApplicationShortcuts` and
@@ -584,8 +670,8 @@ An empty search array is a definitive miss. HTTP failures, malformed results
and oversized responses remain retryable. Attribution identifies OpenStreetMap /
Nominatim.
Overpass currently supplies traffic road geometry, ALPR camera records, military
installation footprints, and annotation geometry (administrative boundaries,
Vector tile sources supply traffic roads, ALPR records and keyless military
areas. Operator-configured Overpass supplies named installations and annotation geometry (administrative boundaries,
neighborhoods, streets, building/grounds outlines and monument candidates).
Nominatim forward/reverse lookup does not replace those queries. Layer source
interfaces select traffic, camera and installation ingestion separately;
@@ -627,7 +713,9 @@ query-language strings or decoding backend tags and relation members.
Points use `{lat, lon}`. Options carry `signal`. A successful array, including an
empty array, is definitive; `null` means a transient failure;
`{rateLimited: true, retryAfterMs}` retains a provider's retry delay. Candidates
`{rateLimited: true, retryAfterMs}` retains a provider's retry delay.
`{unavailable:true, code:'OVERPASS_NOT_CONFIGURED', retryable:false}` is a
capability miss that stops retries while preserving pins and offline geometry. Candidates
carry `id`, `names` (`primary`, `english`, `official`, `alternate`, `short`),
`coordinates` (objects with `lat` and `lon`), `building` and optional `heightM`,
`center` and `point`. Administrative candidates use `category: 'administrative'`
+1121 -117
View File
File diff suppressed because it is too large. Load diff
+2 -1
View File
@@ -3,7 +3,8 @@
Version 4 scene documents can name geographic anchors and describe an explicit
camera move followed by a hold. Existing camera-pose shots still use the ordinary
camera flight. Import, playback, Load, Replay and seek accept the new directions;
a visual anchor/path editor is a later authoring step.
[EDIT DETAILS](DIRECTOR-SHARING.md) captures anchors and move starts; a graphical
path editor remains a later extension.
```json
{
+4 -3
View File
@@ -74,7 +74,8 @@ is a lowercase hexadecimal SHA-256 digest checked before rendering. There are at
most 8 packs per scene, 8 MiB per asset, and 32 MiB of selected assets per shot.
GeoJSON additionally allows at most 2,000 features and 50,000 positions.
Transforms, terrain-relative placement, terrain collision avoidance, arbitrary
GeoJSON styles, archives and local filesystem bundles are not implemented.
GeoJSON styles and archive extraction are not implemented. Explicit local files
can be shared through [JSON asset bundles](DIRECTOR-SHARING.md).
## Loading and resource ownership
@@ -110,5 +111,5 @@ status and resource counts for lifecycle diagnostics.
Run `node scripts/qa-director-packs.mjs` against a credentialed local server to
exercise the real import/load/seek/Stop path with self-authored synthetic assets.
This stage provides the manifest/loading boundary; interactive actions and
visual authoring follow separately.
See [actions](DIRECTOR-INTERACTIONS.md) and [authoring/sharing](DIRECTOR-SHARING.md)
for the controls built on this manifest/loading boundary.
+66
View File
@@ -0,0 +1,66 @@
# Director scene actions
Version-6 scene documents can attach actions to stable GeoJSON feature IDs in a
shot's selected data packs. Existing versions 1–5 migrate without adding actions.
Import validates data and references; it does not load assets or execute actions.
Add `interactions` to a shot, for example:
```json
{
"id": "read-outline",
"label": "About this outline",
"target": { "packId": "outline", "featureId": "boundary" },
"action": {
"type": "card",
"text": "An authored explanation, displayed as plain text.",
"url": "https://example.org/source"
}
}
```
The target must be a feature in a GeoJSON pack selected by that shot. Points,
lines and polygons retain stable pack/feature identities. Assets load before
feature existence is checked; a missing feature refuses the entire action set.
Images and native media are not interaction targets in this version.
| Action | Fields | Behavior |
| --- | --- | --- |
| `card` | `text`, optional `url` | Show plain text and an optional HTTPS source link; never HTML |
| `focus` | `anchorId` | Set a top-down camera pose at a scene anchor through camera ownership arbitration |
| `shot` | `shotId` | Load another shot in the same scene at its authored start |
| `layer` | `layerId`, boolean `enabled` | Request an explicit state through the ordinary layer admission path |
There are at most 64 interactions per shot. Card text is limited to 4,096
characters; source links cannot contain credentials, query strings or fragments.
No script, module, arbitrary fetch endpoint or request header is accepted.
Actions appear after LOAD or seek finishes. Click a pack feature to select its
actions and focus the first button; choose a button explicitly to execute it.
Tab traverses buttons and Enter/Space activates them. The selected buttons have
visible outlines, and status feedback is announced to assistive technology.
There are no global keyboard shortcuts to collide with navigation or typing.
Drawing ownership blocks both pointer selection and action execution. Camera
focus uses existing navigation admission, including Cockpit restrictions.
Actions are inactive during automatic playback and camera travel. They do not
introduce automatic branching or loops. A chain of user-triggered shot changes
is limited to 64 transitions; an explicit LOAD starts a new chain. Import/Stop,
replacement, playback and teardown remove selection, cards and picking handlers,
and abort pending layer requests. No per-frame interaction loop runs.
A layer action must name a layer with an explicit baseline in the current shot.
A target shot must declare a baseline for every layer the departing shot can
change. LOAD and seek reapply those baselines and clear selection. Stop cancels
pending transitions; already completed layer changes remain visible, consistent
with ordinary scene controls. Camera focus is an immediate anchor pose, not a
new timeline segment; seek restores the authored camera pose. Scene JSON is not
modified by executing an action.
The portable interaction session owns action admission/cancellation. The scene
interaction owner supplies accessible DOM and picking; pack renderers own the
feature identities and release them with their geometry. Layer/source admission
and camera arbitration remain with their existing owners.
Author actions in scene JSON or the validated EDIT DETAILS draft.
[Import preview and file sharing](DIRECTOR-SHARING.md) preserve those declarations. Existing content, assets, notices and contributor credit are preserved.
+96
View File
@@ -0,0 +1,96 @@
# Director authoring and file sharing
Open Scenes and select a scene and shot. Existing capture, update, playback and
whole-project export controls remain available. Authored scene content, source
links, assets and contributor credit are preserved.
## Review imports before applying
IMPORT accepts scene JSON (including older supported versions) or a
`.gevbundle.json` asset bundle. A preview lists scenes, shots, data-pack attribution,
configured or unavailable sources, missing layer IDs and bundled bytes. Preview
makes no asset requests and does not change the current project. Configured
sources do not guarantee that a file exists: ordinary assets are checked on LOAD.
Bundle preview verifies the manifest and byte integrity; format decoding and
geometry limits are still enforced when rendering.
Apply replaces the current project after validation and playback cleanup. Export
your current project first if you want to keep both. Cancel or Escape discards
the staged import. A changed project invalidates an open draft rather than being
overwritten. Invalid files leave the existing project and browser save intact.
The programmatic `importProjectFile(file)` API still applies trusted callers'
imports directly; the installed file chooser supplies the review step.
## Edit scene details
EDIT DETAILS provides validated JSON fields for:
- Scene anchors and data-pack manifests.
- The selected shot's camera, optional move, duration, hold, pack IDs and actions.
Capture camera as anchor adds a named geographic position to the draft. Set move
start to current camera creates a pose-to-pose move ending at the shot's existing
camera; Use ordinary flight removes that explicit move. Heights are meters above
the WGS84 ellipsoid. Apply details validates the complete project, preserves IDs,
layer declarations and content notices, and keeps the selected shot. Cancel
leaves the project unchanged. See [camera directions](DIRECTOR-CAMERA.md),
[data packs](DIRECTOR-DATA-PACKS.md) and [actions](DIRECTOR-INTERACTIONS.md) for
field definitions. This is a basic draft editor, not a node graph or path editor.
## Share a selected scene
SHARE SCENE exports only the selected scene. Download scene JSON preserves its
camera, timing, layers, anchors, manifests, interactions and attribution, but
contains no asset bytes. The recipient needs the referenced source registrations
and files. Existing EXPORT PRESETS continues to export the entire project.
For a bundle, select the declared pack files or a folder, then Download asset
bundle. Folder paths match a manifest path either directly or after removing the
chosen root directory. Individual filenames must be unambiguous across all pack
sources; ambiguous or missing matches fail without downloading a partial bundle.
Previously imported bundle files can be reused. Export never automatically fetches
remote files or copies registered recipe media. Existing integrity declarations
must match the files you supply.
Bundles include only declared data-pack files. Registered scene content, linked
media, map tiles and live data are not included. Source notices and licenses remain
in the document; packaging a file does not grant permission to redistribute it.
## Bundle format and limits
A bundle is uncompressed JSON with these fields:
```json
{
"format": "gev-scene-bundle",
"version": 1,
"project": { "version": 6, "scenes": [] },
"assets": []
}
```
Each asset contains `path`, `mimeType`, `base64` and lowercase hexadecimal `sha256`.
Export copies the document and rewrites pack references to the reserved
`scene-bundle` source, adding exact `byteLength` and `sha256`. Paths are confined
relative names; URLs, traversal, duplicate or unreferenced assets, unknown fields,
missing packs and mismatched hashes are rejected. Scene documents remain bounded
at 5 MiB. A bundle allows 50 MiB of JSON, 64 files, 8 MiB per file and 32 MiB of
asset bytes in total. Supported MIME types are JSON/GeoJSON, PNG, MP4/WebM video
and MPEG/Ogg/WAV/WebM audio. Normal pack format and geometry checks still apply.
There are no executable modules, request headers or credentials in the format.
Imported bytes stay in memory for the current project and session. Stop releases
rendered resources while retaining these bytes for replay. Project replacement
or application teardown releases them. Only scene JSON is saved in browser
storage: **reimport the bundle after reloading the app**. Missing bundle bytes
fail explicitly, with no network fallback. This is file sharing, not a storage
service or an offline basemap.
Portable helpers are exported through `gods-eye-view/director`: `parseSceneShare`,
`readSceneShare`, `createSceneBundle`, `createBundleAssets`, `describeSceneShare`,
`editSceneDetails` and `selectSceneDocument`. Loading and UI remain separate owners.
`getSharingState()` reports copied dialog/asset counts for lifecycle diagnostics.
Run `node scripts/qa-director-sharing.mjs` against the local server for installed
controls, cancellation, stale drafts, export/import round trips, resource cleanup
and narrow-screen acceptance. Fixtures are synthetic and carry their own notices.
+19 -18
View File
@@ -2,8 +2,8 @@
Director is the scene playback system. Ordered shot execution, authored timing and playback clocks are separated
from the scene editor and application rendering. This document
also records the proposed next steps; the sharing format and interactions below
are a design plan, not features already available.
records implemented camera directions, data packs, declarative interactions and
file sharing, along with the remaining extensions.
## Implemented boundary
@@ -76,8 +76,8 @@ and river coordinates are bundled. See the
[Version 4](DIRECTOR-CAMERA.md) adds scene-local geographic anchors, explicit
start/destination poses, easing, duration and holds. Playback and seek share
the same sampler. Ellipsoidal heights are explicit; terrain-relative input
is rejected. Existing shots retain ordinary flights. Curved paths, look-at
targets and visual authoring are later extensions.
is rejected. Existing shots retain ordinary flights. Basic anchor/move authoring is available through EDIT DETAILS; curved paths and
look-at targets remain later extensions.
4. **Data packs and placement — implemented for bounded assets.**
[Version 5](DIRECTOR-DATA-PACKS.md) defines a pack manifest with format/version,
source, attribution/license and optional integrity/size metadata. It distinguishes
@@ -85,23 +85,24 @@ and river coordinates are bundled. See the
media anchors. GeoJSON, PNG and native media adapters load with byte/time
limits and explicit ellipsoid heights. Stop and replacement release resources.
Paths resolve against a trusted directory base; missing/offline assets fail
explicitly. Bundles, transforms and terrain-relative placement remain future
extensions. Scene files carry no credentials or executable modules.
5. **Declarative interactions.** Give pickable features stable IDs and connect
clicks to a small action registry: show a text/source card, focus an anchor,
seek a shot, or change an allowed layer state. Define selection feedback,
keyboard equivalents and precedence with drawing/tracking controls. Validate
target references and cancel pending actions on Stop, replacement or teardown.
No JavaScript evaluation or arbitrary HTML in scene files. Branching must
have explicit loop/transition limits and reproducible reset/seek semantics.
6. **Sharing and authoring.** Add validated preview/import/export, missing-pack
diagnostics and an optional self-contained bundle. Preserve attribution with
each pack; a share link does not change an asset's license. A future node graph
could author this same document, but playback and the format will not require
a graph editor.
explicitly. File bundles are available; transforms and terrain-relative placement
remain future extensions. Scene files carry no credentials or executable modules.
5. **Declarative interactions — implemented.** [Version 6](DIRECTOR-INTERACTIONS.md)
connects selected feature IDs to accessible text/source cards, anchor focus,
shot seek and admitted layer changes. References are validated and pending
actions cancel on Stop, replacement or teardown. Scene files contain no
JavaScript or arbitrary HTML; actions require explicit user input.
6. **Sharing and basic authoring — implemented.** [Authoring and sharing](DIRECTOR-SHARING.md)
adds import preview, missing-source/layer diagnostics, validated scene/shot
drafts and selected-scene JSON or explicit-file bundles. Pack attribution is
preserved. Bundle bytes last for the session and must be reimported after a
reload. A future node graph can author the same document without becoming a
playback or format requirement.
Each step must preserve existing scene editing, keyboard playback, camera
arbitration, layer cleanup, media timing and provider fallback. Acceptance includes
portable runner tests, existing Director regressions and browser playback with
Stop/seek/replay, plus tracking and voice checks. Use reproducible local fixtures
for failures and cancellation, alongside live visual checks for actual content.
Declarative actions are implemented in version 6; see [scene actions](DIRECTOR-INTERACTIONS.md) for supported targets, execution and reset semantics.
+2 -1
View File
@@ -36,7 +36,8 @@ const layers = createInfrastructureLayers({
| `governorRequestRender` | `(reason)`; request a frame through the existing render scheduler |
Each returned layer supports `init(viewer)`, `enable(viewer)`, `disable(viewer)`,
`update(viewer)`, `destroy(viewer)`, `getStats()` and `getLodDiagnostics()`.
`update(viewer)`, `destroy(viewer)`, `getStats()`, `getLodDiagnostics()` and
`getAnalystRecords()`.
Use the same viewer for the instance's entire lifetime. Each viewer/context/overlay
host permits one live instance per layer ID; destroy an old instance before
replacing it. Separate applications may use identical IDs with separate hosts.
+62 -3
View File
@@ -1,6 +1,6 @@
# KNOWN ISSUES
Updated: July 8, 2026
Updated: September 22, 2026
This file tracks active runtime issues only.
@@ -78,6 +78,56 @@ Status: Open (owner-accepted 2026-07-08, documented)
---
### Weather layers: coverage and meaning
Status: Open (source limits, by design)
- **Rain radar covers the contiguous United States only.** It shows MRMS
reflectivity in dBZ, not rainfall rate or a forecast; a gap in coverage does not
mean no precipitation.
- **Lightning density is a ground-network grid, not GLM flashes.** It is NOAA's
15-minute density product derived from Vaisala NLDN/GLD360 on an approximately
8 km grid, for the Americas and Pacific only (110°E across the dateline to 0°,
25°S–80°N). It is not a live strike counter or an all-clear.
- **Cyclone advisories cover NOAA's basins only:** the Atlantic and the
eastern/central North Pacific (NHC and CPHC). Storms elsewhere do not appear.
The cone is forecast-center uncertainty, not storm size.
- **Satellite clouds:** global infrared is hourly with typically 2–3 hours of
source latency. Clouds only is a brightness filter, not a cloud mask.
- **Wind is a forecast.** The animation flows through one forecast on an
approximately 1° grid and does not advance forecast time or follow the
observed-history timeline.
- **History is not shared.** Share links carry the weather layers and their
settings but open at Latest.
- **3D Tiles detail window during history playback:** when a frame's detail image
is slow to arrive, the window may fall back to the coarser full-extent image
until it does.
- Native hardware GPU behavior of the wind and weather renderers is not yet
verified.
---
### Satellite passes and analyst answers
Status: Open (by design)
- Pass predictions search the next 24 hours for satellites in the loaded catalog.
Visibility is a geometric estimate (satellite sunlit, observer's Sun at or
below −6°); it ignores weather, satellite brightness and orbital-element age.
- Analyst counts and ranks cover a bounded set of loaded records (by default
2,000 per layer for satellites, datacenters and dams), so omitted records can
change a nearest or count answer. Satellite distance is ground distance.
---
### Live camera video (HLS)
Status: Open (limits)
- At most two live-video sessions run at once. RTMP-only sources and encrypted,
fMP4 or byte-range playlists are not supported.
- When live video fails, the camera falls back to its labeled still, Street View
or placeholder frame, which is not live video.
---
## Closed / Intentional (for clarity)
### Proxy SSRF and error-surface hardening gaps
@@ -118,5 +168,14 @@ Context:
converted to **live NASA FIRMS data** on 2026-07-16: the `/api/firms` proxy merges
three VIIRS NRT sources (trailing 24 h, 30 min cache, serve-stale-on-failure) and the
bundled snapshot was deleted. Requires a free server-side `FIRMS_MAP_KEY`; without it
the layer shows a KEY REQUIRED state.
- Weather radar is still held out of OSS v1 after QA found the previous overlay did not provide reliable visible value.
the layer row reads "Needs FIRMS_MAP_KEY — add it in Provider Settings".
---
### Weather radar held out of the open-source release
Status: Closed — observed rain radar shipped (September 2026)
Context:
- Rain radar, Satellite clouds, Lightning density, Wind and Cyclone advisories
are in the Weather group of Data Layers. Their current limits are listed under
"Weather layers: coverage and meaning" above.
+82
View File
@@ -0,0 +1,82 @@
# Local receivers
The Local ADS-B layer draws aircraft heard by your own receivers. It has two
inputs, and it can use both at once.
| Input | What it is | Bands |
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| Browser SDR | A USB RTL-SDR opened by the browser through WebUSB (Radio panel, Local RTL-SDR card). Decoding runs in the page. | 1090 MHz |
| Decoder feed | The `aircraft.json` served by a decoder you run: dump1090-fa, readsb or tar1090 (1090 MHz), or dump978-fa + skyaware978 (978 MHz). | 1090 MHz, 978 MHz UAT |
Aircraft from both inputs are merged by ICAO address. The newest position
wins, and the click card names every band and source that heard the aircraft
in the last 60 s. Aircraft heard only on 978 MHz UAT draw with a thin ring.
Local aircraft usually lead their public twins. The public Flights layer
renders positions about one poll interval (~30 s) behind so it can interpolate
smoothly between fixes, while local receiver aircraft are shown as heard. An
aircraft in both layers therefore draws twice, the magenta local marker ahead
of the public one; both markers are drawn on purpose.
## Which to use
- **One plain RTL-SDR dongle, desktop Chrome or Edge:** use the browser SDR.
Nothing else to install.
- **A dual-band board, a Raspberry Pi feeder, or a dongle already used by a
decoder:** use a decoder feed. A USB dongle can be opened by only one program
at a time, so a dongle that dump1090 or dump978 owns cannot also be opened by
the browser.
## Configuring feeds
Feeds are set in the server environment only (`.env`), never in the browser:
```sh
LOCAL_RECEIVER_FEEDS=1090=http://localhost:8080/data/aircraft.json,978=http://localhost:8978/data/aircraft.json
```
Each entry is `band=url`, where `band` is `1090` or `978`. The URL must:
- use `http` or `https`;
- point at loopback, a private (RFC1918) address, `localhost` or a `*.local`
name. Public addresses, link-local addresses (including `169.254.169.254`),
IPv6 and other host names are refused;
- have a path ending in `aircraft.json`, with no credentials, query or
fragment.
An entry that breaks a rule is logged when the server starts, shown as
`invalid` and never fetched. Restart the server after changing the value.
The server reads every feed about once a second while the Local ADS-B layer is
on. A feed is `live`, `stale` (its own `now` is more than 10 s old, usually a
stopped decoder), `unreachable` or `invalid`. The Layers panel row and the
Local RTL-SDR card show these states.
## Example: dump1090-fa and dump978-fa on macOS or Linux
Write each decoder's JSON to a directory, then serve the directory on
localhost. Each command keeps running, so start each in its own terminal:
```sh
# 1090 MHz
dump1090-fa --device-index 0 --write-json /tmp/adsb1090
python3 -m http.server 8080 --bind 127.0.0.1 --directory /tmp/adsb1090
# 978 MHz UAT: dump978-fa decodes, skyaware978 writes aircraft.json
dump978-fa --sdr driver=rtlsdr,rtl=1 --raw-port 30978
skyaware978 --connect localhost:30978 --json-dir /tmp/adsb978
python3 -m http.server 8978 --bind 127.0.0.1 --directory /tmp/adsb978
```
```sh
LOCAL_RECEIVER_FEEDS=1090=http://localhost:8080/aircraft.json,978=http://localhost:8978/aircraft.json
```
A board with two identical-serial channels, such as the Nooelec FlyCatcher,
cannot be told apart by serial number. Select each channel by index:
`--device-index <n>` for dump1090-fa and `--sdr driver=rtlsdr,rtl=<n>` for
dump978-fa. `rtl_test` lists the indexes.
A tar1090 or readsb install on a Raspberry Pi already serves
`http://<pi-address>/tar1090/data/aircraft.json`. Use the Pi's private address
or its `.local` name.
+7 -1
View File
@@ -14,6 +14,12 @@ The session adapter, action runner and backend interfaces remain unchanged.
| `realtimeCost.js` | Next-session preferences and the current session's model-bound meter |
| `realtimeViewport.js` | One retained image, bounded deletion identities and capture generation |
| `realtimeDiagnostics.js` | Bounded error history and sanitized optional diagnostics |
| `turnMetrics.js` | Per-turn latency and speech spans written to the debug log |
| `narration.js` | Progress lines and the earcon while tools run; spoken through the turns owner |
| `pointerContext.js` | Pointer position, dwell pick and the current turn's pointer snapshot (session-scoped) |
| `referents.js` | Numbered items from the latest result for `referent:n` (session-scoped) |
| `resultDisplay.js` | What the voice card shows for a result and which items it numbers (read by the card and `referents.js`) |
| `gevActions.js` runner | Analyst follow-up memory (session: `resetConversation()` on stop, `dispose()` on removal) |
Pure input policy, preferences and protocol-response policy have separate modules.
Owners receive named readers, operations or focused collaborators. Cross-owner
@@ -29,7 +35,7 @@ Action and post-capture continuations retain their conversation identity before
publishing output, queueing a response or changing status.
Behavioral invariants remain in force: a 500 ms Space hold claims push-to-talk;
short control taps and text entry remain native; a click-started session stays
short control taps and text entry remain native; a claimed hold while the assistant speaks interrupts it (never during a Radio handoff); a click-started session stays
open-mic. Every accepted or refused tool call receives its terminal output once.
New typed input supersedes old intent. Radio waits for the spoken confirmation and
verified muted playback. A live cost meter stays bound to its negotiated model,
+50
View File
@@ -9,9 +9,13 @@
"version": "0.1.1",
"license": "MIT",
"dependencies": {
"@jtarrio/signals": "^0.10.0",
"@jtarrio/webrtlsdr": "^3.0.6",
"@mapbox/vector-tile": "^3.0.0",
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"hls.js": "^1.7.3",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
"satellite.js": "^6.0.2"
@@ -1059,6 +1063,22 @@
"url": "https://opencollective.com/libvips"
}
},
"node_modules/@jtarrio/signals": {
"version": "0.10.1",
"resolved": "https://registry.npmjs.org/@jtarrio/signals/-/signals-0.10.1.tgz",
"integrity": "sha512-X6tJOugEATYH9KJy6j2feEBVCUmRv1U9SSbg9ITQZCx/Qruw2fE2zsS1HKH3yOsxrZBE+eQTGM+d0roX4xuDdw==",
"license": "Apache-2.0"
},
"node_modules/@jtarrio/webrtlsdr": {
"version": "3.0.6",
"resolved": "https://registry.npmjs.org/@jtarrio/webrtlsdr/-/webrtlsdr-3.0.6.tgz",
"integrity": "sha512-+P8dsbXQ/6Mwru/vV613yHH68nHFyUJ9zPK68hE/Hh2F3jwyirUAjjBtIAfN/MlJbFioMJ3NS6dMGrRj9deFvw==",
"license": "Apache-2.0",
"dependencies": {
"@jtarrio/signals": "^0.10.0",
"@types/w3c-web-usb": "^1.0.10"
}
},
"node_modules/@mapbox/point-geometry": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@mapbox/point-geometry/-/point-geometry-1.1.0.tgz",
@@ -1076,6 +1096,24 @@
"pbf": "^5.0.0"
}
},
"node_modules/@meri-imperiumi/eccodes-wasm": {
"version": "2.48.2",
"resolved": "https://registry.npmjs.org/@meri-imperiumi/eccodes-wasm/-/eccodes-wasm-2.48.2.tgz",
"integrity": "sha512-Z7rD3xInomPG7FS97ORD1UGKAdgeeRP9ofa9k4m8d+k2sDfDQeAjTzvs9UOndyaZovaSbEQeIoNX8gTmQNhiSA==",
"cpu": [
"x64",
"arm64"
],
"license": "Apache-2.0",
"os": [
"darwin",
"linux",
"win32"
],
"engines": {
"node": ">=24.0.0"
}
},
"node_modules/@puppeteer/browsers": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/@puppeteer/browsers/-/browsers-3.2.2.tgz",
@@ -1518,6 +1556,12 @@
"license": "MIT",
"optional": true
},
"node_modules/@types/w3c-web-usb": {
"version": "1.0.14",
"resolved": "https://registry.npmjs.org/@types/w3c-web-usb/-/w3c-web-usb-1.0.14.tgz",
"integrity": "sha512-Qu3Nn6JFuF4+sHKYl+IcX9vYiI40ogleXzFFSxoE1W94rG98o/kXs8uJ0QSfFzuwBCZWlGfUGpPkgwuuX4PchA==",
"license": "MIT"
},
"node_modules/@zip.js/zip.js": {
"version": "2.8.18",
"resolved": "https://registry.npmjs.org/@zip.js/zip.js/-/zip.js-2.8.18.tgz",
@@ -1944,6 +1988,12 @@
"integrity": "sha512-bzh50DW9kTPM00T8y4o8vQg89Di9oLJVLW/KaOGIXJWP/iqCN6WKYkbNOF04vFLJhwcpYUh9ydh/+5vpOqV4YQ==",
"license": "MIT"
},
"node_modules/hls.js": {
"version": "1.7.3",
"resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.7.3.tgz",
"integrity": "sha512-MsPlx6yVW4Qv4C7mEVou4/gk/5cN2dVxOTnrNjPSmyH2f0Ln+/UkG10Ax1PE6OvrmTVPKgQYozTtBaPOuCzq7A==",
"license": "Apache-2.0"
},
"node_modules/http-errors": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz",
+10 -1
View File
@@ -2,7 +2,7 @@
"name": "gods-eye-view",
"private": true,
"version": "0.1.1",
"description": "A real-time intelligence console for planet Earth \u2014 photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
"description": "A real-time intelligence console for planet Earth — photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
"type": "module",
"license": "MIT",
"author": "Bilawal Sidhu",
@@ -35,6 +35,8 @@
"doctor": "node scripts/setup-doctor.mjs",
"format": "node scripts/format.mjs --write",
"format:check": "node scripts/format.mjs --check",
"layer-token:next": "node scripts/next-layer-state-token.mjs",
"layer-token:check": "node scripts/check-layer-state-tokens.mjs",
"dev": "vite",
"dev:secure": "./scripts/dev-secure.sh",
"opensky:import": "./scripts/opensky-import-client.sh",
@@ -47,9 +49,13 @@
"qa:transit": "node scripts/qa-transit.mjs"
},
"dependencies": {
"@jtarrio/signals": "^0.10.0",
"@jtarrio/webrtlsdr": "^3.0.6",
"@mapbox/vector-tile": "^3.0.0",
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"hls.js": "^1.7.3",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
"satellite.js": "^6.0.2"
@@ -150,6 +156,8 @@
"./aircraft": "./src/layers/aircraft/index.js",
"./layers/vessels": "./src/layers/vessels/index.js",
"./layers/earthquakes": "./src/layers/earthquakes/index.js",
"./layers/recent-imagery": "./src/layers/recentImagery/index.js",
"./layers/perimeters": "./src/layers/perimeters/index.js",
"./sources/firms-csv": "./src/data/firmsCsv.js",
"./layers/firms": "./src/layers/firms/index.js",
"./layers/satellites": "./src/layers/satellites/index.js",
@@ -221,6 +229,7 @@
"./layers/bikeshare/source": "./src/layers/bikeshare/source.js",
"./layers/cctv/source": "./src/layers/cctv/source.js",
"./layers/earthquakes/source": "./src/layers/earthquakes/source.js",
"./layers/perimeters/source": "./src/layers/perimeters/source.js",
"./layers/firms/source": "./src/layers/firms/source.js",
"./layers/installations/source": "./src/layers/installations/source.js",
"./layers/launches/source": "./src/layers/launches/source.js",
+771
View File
@@ -0,0 +1,771 @@
#!/usr/bin/env node
/**
* Build the bundled administrative-boundary packs read by
* `src/data/adminBoundaries.js`:
*
* src/data/local_data/natural_earth/states_provinces.json
* Natural Earth 10m admin-1 states and provinces (public domain).
* src/data/local_data/us_census_counties/counties.json
* US Census Bureau cartographic boundary counties (public domain).
*
* Both sources are pinned by URL and SHA-256, so a rerun reproduces the packs
* byte for byte. Downloads are cached outside the repository (default: the
* system temp directory) and are never committed.
*
* Usage:
* node scripts/build-admin-packs.mjs [--cache <dir>] [--only countries|admin1|counties]
*
* The provenance READMEs beside each pack record these parameters.
*/
import { createHash } from 'node:crypto';
import { mkdir, readFile, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { inflateRawSync } from 'node:zlib';
import { normalizeAdminName } from '../src/data/adminBoundaries.js';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const NE_COMMIT = 'ca96624a56bd078437bca8184e78163e5039ad19';
const NE_BASE = `https://raw.githubusercontent.com/nvkelso/natural-earth-vector/${NE_COMMIT}/geojson`;
export const SOURCES = Object.freeze({
countries: {
url: `${NE_BASE}/ne_10m_admin_0_countries.geojson`,
sha256: '239eec57ac17f100a11e2536cffc56752c318b50ae765b0918ff7aab4ce8f255',
},
mapUnits: {
url: `${NE_BASE}/ne_10m_admin_0_map_units.geojson`,
sha256: '57da82be755f4afccd8f3b14251bb2752f5df1395f47d2d86f817470c4a48862',
},
admin1: {
url: `${NE_BASE}/ne_10m_admin_1_states_provinces.geojson`,
sha256: '22d0e3ad85eb3e27f17cabf8ba2d50e554fbc27a87796ff891d958185da62fb5',
},
places: {
url: `${NE_BASE}/ne_10m_populated_places_simple.geojson`,
sha256: 'fd3fa867a320cbd5c5b6bb5bc550afeec2939fb2cef688e508007282a55ac42f',
},
counties: {
url: 'https://www2.census.gov/geo/tiger/GENZ2025/shp/cb_2025_us_county_5m.zip',
sha256: 'faec522080681e79be5be435c981009a77891206ff8a7f1d142f3bf5da9ebd74',
},
});
/**
* Pack parameters. The Douglas-Peucker tolerance scales with the unit's size
* (`toleranceFactor` × √area, clamped to `[minToleranceDeg, maxToleranceDeg]`)
* because a unit is viewed at a zoom that fits it: Texas keeps ~2 km detail,
* Bavaria ~1 km, Travis County ~150 m. Parts smaller than `minPartKm2` or than
* `minPartShare` of the largest part are dropped (the largest part always
* stays); holes under `minHoleKm2` are dropped.
*/
export const PARAMS = Object.freeze({
countries: {
toleranceFactor: 0.004,
minToleranceDeg: 0.005,
maxToleranceDeg: 0.02,
decimals: 3,
minPartKm2: 20,
minPartShare: 0.00003,
minHoleKm2: 20,
},
admin1: {
toleranceFactor: 0.004,
minToleranceDeg: 0.005,
maxToleranceDeg: 0.02,
decimals: 3,
minPartKm2: 20,
minPartShare: 0.0003,
minHoleKm2: 20,
},
counties: {
toleranceFactor: 0.003,
minToleranceDeg: 0.0005,
maxToleranceDeg: 0.01,
decimals: 4,
minPartKm2: 1,
minPartShare: 0.001,
minHoleKm2: 1,
},
// A bare admin-1 name is marked ambiguous — the resolver then lets a
// geocoder decide — when it is also a country, the name of a city of at
// least `insideMinPopulation` inside the unit's box ("New York", "Madrid"),
// or the name of a city of at least `elsewhereMinPopulation` outside it
// ("Santa Fe", "Victoria"). A unit no larger than `cityStateMaxKm2` whose
// own city lies inside is the city itself (Berlin, Vienna, Paris): it is
// marked `cityState` and may be named alone however small.
ambiguity: {
insideMinPopulation: 200_000,
elsewhereMinPopulation: 50_000,
cityStateMaxKm2: 1_000,
},
});
const args = process.argv.slice(2);
const cacheDir = path.resolve(
args.includes('--cache')
? args[args.indexOf('--cache') + 1]
: path.join(os.tmpdir(), 'gev-admin-packs'),
);
const only = args.includes('--only') ? args[args.indexOf('--only') + 1] : null;
export async function fetchPinned(name, source = SOURCES[name]) {
const { url, sha256 } = source;
const file = path.join(cacheDir, path.basename(new URL(url).pathname));
let bytes = await readFile(file).catch(() => null);
if (!bytes) {
console.log(`Downloading ${url}`);
const response = await fetch(url);
if (!response.ok) throw new Error(`HTTP ${response.status} for ${url}`);
bytes = Buffer.from(await response.arrayBuffer());
await mkdir(cacheDir, { recursive: true });
await writeFile(file, bytes);
}
const digest = createHash('sha256').update(bytes).digest('hex');
if (digest !== sha256)
throw new Error(
`${name}: SHA-256 ${digest} does not match the pinned ${sha256}`,
);
return { bytes, digest };
}
// ── geometry ────────────────────────────────────────────────────────────
const EARTH_RADIUS_KM = 6371;
const toRad = (d) => (d * Math.PI) / 180;
/** Spherical ring area (km²); matches src/data/naturalEarthRegions.js. */
export function ringAreaKm2(ring) {
const n = ring.length;
if (n < 3) return 0;
let sum = 0;
for (let i = 0; i < n; i++) {
const [lon1, lat1] = ring[i];
const [lon2, lat2] = ring[(i + 1) % n];
sum +=
toRad(lon2 - lon1) * (2 + Math.sin(toRad(lat1)) + Math.sin(toRad(lat2)));
}
return Math.abs((sum * EARTH_RADIUS_KM * EARTH_RADIUS_KM) / 2);
}
/** Signed planar (shoelace) area: positive counter-clockwise in lon/lat. */
export function signedArea(ring) {
let sum = 0;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
sum += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
return sum / 2;
}
function segDist(p, a, b) {
let [x, y] = p;
const [x1, y1] = a;
const [x2, y2] = b;
const dx = x2 - x1;
const dy = y2 - y1;
if (dx !== 0 || dy !== 0) {
const t = ((x - x1) * dx + (y - y1) * dy) / (dx * dx + dy * dy);
if (t > 1) {
x -= x2;
y -= y2;
return Math.hypot(x, y);
}
if (t > 0) {
x -= x1 + dx * t;
y -= y1 + dy * t;
return Math.hypot(x, y);
}
}
return Math.hypot(x - x1, y - y1);
}
/** Iterative Douglas-Peucker over an open point list. */
function douglasPeucker(points, tolerance) {
if (points.length <= 2) return points.slice();
const keep = new Uint8Array(points.length);
keep[0] = keep[points.length - 1] = 1;
const stack = [[0, points.length - 1]];
while (stack.length) {
const [first, last] = stack.pop();
let maxDist = 0;
let index = -1;
for (let i = first + 1; i < last; i++) {
const d = segDist(points[i], points[first], points[last]);
if (d > maxDist) {
maxDist = d;
index = i;
}
}
if (maxDist > tolerance && index !== -1) {
keep[index] = 1;
stack.push([first, index], [index, last]);
}
}
return points.filter((_, i) => keep[i]);
}
/**
* Simplify a closed ring to an OPEN rounded ring. The ring is split at its
* farthest vertex from the start so Douglas-Peucker has two anchored halves
* (a closed ring has no chord to measure against). Returns null when fewer
* than `minVertices` distinct vertices survive (four by default; a pack that
* keeps triangles passes three).
*/
function simplifyRing(ring, tolerance, decimals, minVertices = 4) {
let open = ring.slice();
const first = open[0];
const last = open[open.length - 1];
if (open.length > 1 && first[0] === last[0] && first[1] === last[1])
open = open.slice(0, -1);
if (open.length < minVertices) return null;
let far = 1;
let farDist = -1;
for (let i = 1; i < open.length; i++) {
const d = Math.hypot(open[i][0] - first[0], open[i][1] - first[1]);
if (d > farDist) {
farDist = d;
far = i;
}
}
const a = douglasPeucker(open.slice(0, far + 1), tolerance);
const b = douglasPeucker([...open.slice(far), first], tolerance);
const simplified = [...a, ...b.slice(1, -1)];
const factor = 10 ** decimals;
const out = [];
for (const [lon, lat] of simplified) {
const p = [
Math.round(lon * factor) / factor,
Math.round(lat * factor) / factor,
];
const prev = out[out.length - 1];
if (prev && prev[0] === p[0] && prev[1] === p[1]) continue;
out.push(p);
}
while (
out.length > 1 &&
out[0][0] === out[out.length - 1][0] &&
out[0][1] === out[out.length - 1][1]
)
out.pop();
return out.length >= minVertices ? out : null;
}
/**
* Simplify polygons ([outer, ...holes][]): drop parts and holes under the
* area floors (the largest part always stays), orient outers
* counter-clockwise and holes clockwise, sort parts by area descending.
*/
export function simplifyPolygons(polygons, params, tolerance, decimals) {
const minVertices = params.minRingVertices ?? 4;
const parts = [];
for (const poly of polygons) {
const outer = simplifyRing(poly[0], tolerance, decimals, minVertices);
if (!outer) continue;
const area = ringAreaKm2(outer);
if (signedArea(outer) < 0) outer.reverse();
const holes = [];
for (const hole of poly.slice(1)) {
const ring = simplifyRing(hole, tolerance, decimals, minVertices);
if (!ring || ringAreaKm2(ring) < params.minHoleKm2) continue;
if (signedArea(ring) > 0) ring.reverse();
holes.push(ring);
}
parts.push({ rings: [outer, ...holes], area });
}
parts.sort((x, y) => y.area - x.area);
const floor = Math.max(
params.minPartKm2,
(parts[0]?.area || 0) * params.minPartShare,
);
return parts
.filter((part, index) => index === 0 || part.area >= floor)
.map((part) => part.rings);
}
/**
* Simplify one unit at its size-scaled tolerance. A unit too small to
* survive the pack precision (a Maldives atoll) is kept at one more decimal
* and a fifth of the minimum tolerance, or failing that unsimplified at two
* more decimals; the caller records `d`.
*/
export function unitTolerance(polygons, params) {
const sourceArea = polygons.reduce(
(sum, poly) => sum + ringAreaKm2(poly[0]),
0,
);
return Math.min(
params.maxToleranceDeg,
Math.max(
params.minToleranceDeg,
(params.toleranceFactor * Math.sqrt(sourceArea)) / 111.32,
),
);
}
export function simplifyUnit(polygons, params) {
const tolerance = unitTolerance(polygons, params);
const coarse = simplifyPolygons(polygons, params, tolerance, params.decimals);
if (coarse.length) return { polygons: coarse, decimals: params.decimals };
const fine = simplifyPolygons(
polygons,
params,
params.minToleranceDeg / 5,
params.decimals + 1,
);
if (fine.length) return { polygons: fine, decimals: params.decimals + 1 };
// Smaller still (Vatican City): the source ring as drawn.
const exact = simplifyPolygons(polygons, params, 0, params.decimals + 2);
return { polygons: exact, decimals: params.decimals + 2 };
}
/**
* Encode a rounded open ring as integers in units of 10^-decimals degrees:
* the first vertex absolute, every later vertex as a delta from the previous
* one — [lon0, lat0, dLon1, dLat1, …]. Decoded by `decodeRing` in
* src/data/adminBoundaries.js.
*/
function encodeRing(ring, decimals) {
const factor = 10 ** decimals;
const out = [];
let px = 0;
let py = 0;
for (const [lon, lat] of ring) {
const x = Math.round(lon * factor);
const y = Math.round(lat * factor);
out.push(x - px, y - py);
px = x;
py = y;
}
return out;
}
export function encodeUnit({ polygons, decimals }) {
return polygons.map((poly) => poly.map((ring) => encodeRing(ring, decimals)));
}
function geojsonPolygons(geometry) {
if (geometry?.type === 'Polygon') return [geometry.coordinates];
if (geometry?.type === 'MultiPolygon') return geometry.coordinates;
return [];
}
function bboxOf(polygons) {
let w = Infinity;
let s = Infinity;
let e = -Infinity;
let n = -Infinity;
for (const poly of polygons)
for (const [lon, lat] of poly[0]) {
if (lon < w) w = lon;
if (lon > e) e = lon;
if (lat < s) s = lat;
if (lat > n) n = lat;
}
return [w, s, e, n];
}
// ── zip / shapefile / dbf ───────────────────────────────────────────────
/** Read named entries from a zip archive (stored or deflated). */
export function unzip(buffer) {
let eocd = buffer.length - 22;
while (eocd >= 0 && buffer.readUInt32LE(eocd) !== 0x06054b50) eocd--;
if (eocd < 0) throw new Error('zip: end of central directory not found');
const count = buffer.readUInt16LE(eocd + 10);
let offset = buffer.readUInt32LE(eocd + 16);
const files = new Map();
for (let i = 0; i < count; i++) {
if (buffer.readUInt32LE(offset) !== 0x02014b50)
throw new Error('zip: bad central directory entry');
const method = buffer.readUInt16LE(offset + 10);
const compressed = buffer.readUInt32LE(offset + 20);
const nameLength = buffer.readUInt16LE(offset + 28);
const extraLength = buffer.readUInt16LE(offset + 30);
const commentLength = buffer.readUInt16LE(offset + 32);
const local = buffer.readUInt32LE(offset + 42);
const name = buffer.toString('utf8', offset + 46, offset + 46 + nameLength);
const dataStart =
local +
30 +
buffer.readUInt16LE(local + 26) +
buffer.readUInt16LE(local + 28);
const data = buffer.subarray(dataStart, dataStart + compressed);
if (method === 0) files.set(name, data);
else if (method === 8) files.set(name, inflateRawSync(data));
else throw new Error(`zip: unsupported method ${method} for ${name}`);
offset += 46 + nameLength + extraLength + commentLength;
}
return files;
}
/** Polygon records (shape type 5) as arrays of closed [lon, lat] rings. */
export function readShp(buffer) {
const shapes = [];
let offset = 100;
while (offset < buffer.length) {
const length = buffer.readInt32BE(offset + 4) * 2;
const content = offset + 8;
const type = buffer.readInt32LE(content);
const rings = [];
if (type === 5) {
const numParts = buffer.readInt32LE(content + 36);
const numPoints = buffer.readInt32LE(content + 40);
const partsAt = content + 44;
const pointsAt = partsAt + numParts * 4;
for (let p = 0; p < numParts; p++) {
const start = buffer.readInt32LE(partsAt + p * 4);
const end =
p + 1 < numParts
? buffer.readInt32LE(partsAt + (p + 1) * 4)
: numPoints;
const ring = [];
for (let i = start; i < end; i++)
ring.push([
buffer.readDoubleLE(pointsAt + i * 16),
buffer.readDoubleLE(pointsAt + i * 16 + 8),
]);
rings.push(ring);
}
} else if (type !== 0) {
throw new Error(`shp: unsupported shape type ${type}`);
}
shapes.push(rings);
offset = content + length;
}
return shapes;
}
/** dBASE records as plain objects (character and numeric fields). */
export function readDbf(buffer) {
const count = buffer.readUInt32LE(4);
const headerLength = buffer.readUInt16LE(8);
const recordLength = buffer.readUInt16LE(10);
const fields = [];
for (let at = 32; buffer[at] !== 0x0d; at += 32) {
fields.push({
name: buffer.toString('latin1', at, at + 11).replace(/\0.*$/, ''),
length: buffer[at + 16],
});
}
const records = [];
for (let r = 0; r < count; r++) {
let at = headerLength + r * recordLength + 1;
const record = {};
for (const field of fields) {
record[field.name] = buffer
.toString('utf8', at, at + field.length)
.trim();
at += field.length;
}
records.push(record);
}
return records;
}
/** Group shapefile rings into polygons: clockwise outers, holes assigned by containment. */
export function shpPolygons(rings) {
const outers = [];
const holes = [];
for (const ring of rings) {
if (ring.length < 4) continue;
(signedArea(ring) < 0 ? outers : holes).push(ring);
}
const polygons = outers.map((outer) => [outer]);
for (const hole of holes) {
const [x, y] = hole[0];
const owner = polygons.find(([outer]) => pointInRing(outer, x, y));
if (owner) owner.push(hole);
else polygons.push([hole.slice().reverse()]);
}
return polygons;
}
export function pointInRing(ring, x, y) {
let inside = false;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++) {
const [xi, yi] = ring[i];
const [xj, yj] = ring[j];
if (yi > y !== yj > y && x < ((xj - xi) * (y - yi)) / (yj - yi) + xi)
inside = !inside;
}
return inside;
}
// ── packs ────────────────────────────────────────────────────────────────
const LATIN = /^[\p{Script=Latin}\p{N}\s.,'’()\-/&]+$/u;
function uniqueNames(values, exclude) {
const seen = new Set(exclude.map(normalizeAdminName));
const out = [];
for (const value of values) {
const text = String(value || '').trim();
const key = normalizeAdminName(text);
if (!text || !key || seen.has(key) || !LATIN.test(text)) continue;
seen.add(key);
out.push(text);
}
return out;
}
async function buildCountries() {
const source = await fetchPinned('countries');
const mapUnits = await fetchPinned('mapUnits');
const params = PARAMS.countries;
const features = [];
for (const { properties: p, geometry } of JSON.parse(source.bytes).features) {
const unit = simplifyUnit(geojsonPolygons(geometry), params);
if (!unit.polygons.length) continue;
const name = p.ADMIN || p.NAME_EN || p.NAME;
features.push({
name,
alt: uniqueNames(
[p.NAME, p.NAME_LONG, p.NAME_EN, p.FORMAL_EN, p.BRK_NAME].filter(
Boolean,
),
[name],
),
country: name,
iso: p.ADM0_A3,
iso2: p.ISO_A2_EH || p.ISO_A2,
type: 'Country',
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
}
for (const { properties: p, geometry } of JSON.parse(mapUnits.bytes)
.features) {
if (p.ADM0_A3 !== 'GBR') continue;
const unit = simplifyUnit(geojsonPolygons(geometry), params);
features.push({
name: p.GEOUNIT,
country: p.ADMIN,
iso: p.GU_A3,
iso2: 'GB',
type: 'Constituent country',
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
}
features.sort((a, b) => a.name.localeCompare(b.name));
return {
meta: {
title: 'Natural Earth admin-0 countries',
mapUnits: SOURCES.mapUnits,
source: 'ne_10m_admin_0_countries',
url: SOURCES.countries.url,
sha256: source.digest,
commit: NE_COMMIT,
license:
'Public domain (https://www.naturalearthdata.com/about/terms-of-use/)',
decimals: params.decimals,
curation: params,
script: 'scripts/build-admin-packs.mjs',
},
features,
};
}
async function buildAdmin1() {
const params = PARAMS.admin1;
const source = await fetchPinned('admin1');
const places = await fetchPinned('places');
const collection = JSON.parse(source.bytes.toString('utf8'));
const { ambiguity } = PARAMS;
const cities = JSON.parse(places.bytes.toString('utf8'))
.features.map((f) => f.properties)
.filter((p) => p.pop_max >= ambiguity.elsewhereMinPopulation)
.map((p) => ({
keys: new Set([p.name, p.nameascii].map(normalizeAdminName)),
lon: p.longitude,
lat: p.latitude,
population: p.pop_max,
country: p.adm0name,
}));
const countryNames = new Set();
for (const { properties: p } of collection.features)
for (const name of [p.admin, p.geonunit])
if (name) countryNames.add(normalizeAdminName(name));
const features = [];
for (const { properties: p, geometry } of collection.features) {
const name = String(p.name || '').trim();
if (!name) continue;
const unit = simplifyUnit(geojsonPolygons(geometry), params);
if (!unit.polygons.length) continue;
const nameEn =
p.name_en && normalizeAdminName(p.name_en) !== normalizeAdminName(name)
? String(p.name_en).trim()
: null;
const alt = uniqueNames(
String(p.name_alt || '').split('|'),
[name, nameEn].filter(Boolean),
);
const bbox = bboxOf(unit.polygons);
const areaKm2 = unit.polygons.reduce(
(sum, poly) => sum + ringAreaKm2(poly[0]),
0,
);
const keys = new Set(
[name, nameEn, ...alt].filter(Boolean).map(normalizeAdminName),
);
const amb = [];
const ambAbroad = [];
let cityState = false;
const inBox = (c) =>
c.lon >= bbox[0] &&
c.lon <= bbox[2] &&
c.lat >= bbox[1] &&
c.lat <= bbox[3];
for (const key of keys) {
const namesakes = cities.filter((c) => c.keys.has(key));
const ownCity = namesakes.some(
(c) => inBox(c) && c.population >= ambiguity.insideMinPopulation,
);
const cityInside = ownCity && areaKm2 > ambiguity.cityStateMaxKm2;
if (ownCity && !cityInside) cityState = true;
const elsewhere = namesakes.filter((c) => !inBox(c));
if (
countryNames.has(key) ||
cityInside ||
elsewhere.some((c) => c.country === p.admin)
)
amb.push(key);
// Only abroad: "Victoria, Australia" settles it; "Victoria" alone not.
else if (elsewhere.length) ambAbroad.push(key);
}
const postal = String(p.postal || '').trim();
features.push({
name,
...(nameEn ? { nameEn } : {}),
...(alt.length ? { alt } : {}),
...(/^[A-Z]{2}$/.test(postal) ? { postal } : {}),
type: p.type_en || p.type || null,
...(Number.isFinite(p.min_label) ? { rank: p.min_label } : {}),
country: p.admin,
iso2: p.iso_a2 && p.iso_a2 !== '-99' ? p.iso_a2 : null,
iso: p.iso_3166_2 && !p.iso_3166_2.includes('~') ? p.iso_3166_2 : null,
...(amb.length ? { amb: amb.sort() } : {}),
...(ambAbroad.length ? { ambAbroad: ambAbroad.sort() } : {}),
...(cityState ? { cityState } : {}),
...(Number.isFinite(p.longitude) && Number.isFinite(p.latitude)
? {
label: [
Math.round(p.longitude * 1000) / 1000,
Math.round(p.latitude * 1000) / 1000,
],
}
: {}),
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
}
features.sort(
(a, b) =>
a.country.localeCompare(b.country) ||
a.name.localeCompare(b.name) ||
(a.iso || '').localeCompare(b.iso || ''),
);
return {
meta: {
title: 'Natural Earth admin-1 states and provinces',
source: 'ne_10m_admin_1_states_provinces',
url: SOURCES.admin1.url,
sha256: source.digest,
ambiguitySource: SOURCES.places.url,
ambiguitySha256: places.digest,
commit: NE_COMMIT,
license:
'Public domain (https://www.naturalearthdata.com/about/terms-of-use/)',
schema:
'features[]: name, nameEn?, alt?[], postal?, type, rank? (min_label: the zoom at which Natural Earth labels the unit; lower is more prominent), country, iso2, iso (ISO 3166-2), amb?[] (normalized names that are also a country or a city — alone or with the country named), ambAbroad?[] (names shared only with a city in another country — ambiguous alone, settled by naming the country), cityState? (the unit is its own city), label? [lon, lat] label point, d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
decimals: params.decimals,
curation: { ...params, ambiguity },
script: 'scripts/build-admin-packs.mjs',
},
features,
};
}
async function buildCounties() {
const params = PARAMS.counties;
const source = await fetchPinned('counties');
const files = unzip(source.bytes);
const base = path.basename(new URL(SOURCES.counties.url).pathname, '.zip');
const shapes = readShp(files.get(`${base}.shp`));
const records = readDbf(files.get(`${base}.dbf`));
if (shapes.length !== records.length)
throw new Error('counties: shp/dbf record counts differ');
const features = [];
records.forEach((record, index) => {
const unit = simplifyUnit(shpPolygons(shapes[index]), params);
if (!unit.polygons.length) return;
const lsad = record.NAMELSAD.startsWith(record.NAME)
? record.NAMELSAD.slice(record.NAME.length).trim()
: '';
features.push({
name: record.NAME,
full: record.NAMELSAD,
...(lsad ? { lsad } : {}),
state: record.STATE_NAME,
st: record.STUSPS,
geoid: record.GEOID,
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
});
features.sort((a, b) => a.geoid.localeCompare(b.geoid));
return {
meta: {
title: 'US Census Bureau cartographic boundary counties',
source: base,
url: SOURCES.counties.url,
sha256: source.digest,
license: 'Public domain (U.S. Government work, 17 U.S.C. § 105)',
schema:
'features[]: name, full (NAMELSAD), lsad? (County, Parish, Borough, …), state, st (USPS), geoid, d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
decimals: params.decimals,
curation: params,
script: 'scripts/build-admin-packs.mjs',
},
features,
};
}
async function writePack(relative, pack) {
const file = path.join(ROOT, relative);
await mkdir(path.dirname(file), { recursive: true });
const text = JSON.stringify(pack) + '\n';
await writeFile(file, text);
const parts = pack.features.reduce((n, f) => n + f.polygons.length, 0);
const vertices = pack.features.reduce(
(n, f) => n + f.polygons.flat().reduce((m, ring) => m + ring.length / 2, 0),
0,
);
console.log(
`${relative}: ${pack.features.length} features, ${parts} parts, ${vertices} vertices, ${Buffer.byteLength(text)} bytes`,
);
}
const isMain =
process.argv[1] &&
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
if (isMain) {
if (!only || only === 'countries')
await writePack(
'src/data/local_data/natural_earth/countries.json',
await buildCountries(),
);
if (!only || only === 'admin1')
await writePack(
'src/data/local_data/natural_earth/states_provinces.json',
await buildAdmin1(),
);
if (!only || only === 'counties')
await writePack(
'src/data/local_data/us_census_counties/counties.json',
await buildCounties(),
);
}
+418
View File
@@ -0,0 +1,418 @@
#!/usr/bin/env node
/**
* Build the bundled US places pack read by `src/data/placeBoundaries.js`:
*
* src/data/local_data/us_census_places/index.json state list, boxes, meta
* src/data/local_data/us_census_places/<ST>.json one file per state
* src/data/local_data/us_census_places/files.js their URLs, for bundlers
*
* Source: US Census Bureau 2025 cartographic boundary places
* (cb_2025_us_place_500k, public domain), pinned by URL and SHA-256, so a
* rerun reproduces the pack byte for byte. The download is cached outside the
* repository (default: the system temp directory).
*
* Geometry uses the county pack's simplification and encoding
* (scripts/build-admin-packs.mjs) with two differences: triangles are kept,
* and a unit whose simplified rings cross themselves or each other is
* simplified again at half the tolerance (finer precision last). Then
* `scripts/pack_geometry.py repair census` checks every feature as a whole
* MultiPolygon under GEOS and repairs the ones that are still invalid (rings
* touching themselves, overlapping parts), so it needs Python with
* shapely==2.1.2 on GEOS 3.13.1.
*
* Usage:
* node scripts/build-census-places.mjs [--cache <dir>] [--python <exe>]
*/
import { execFileSync } from 'node:child_process';
import { mkdir, readdir, rm, writeFile } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import {
PARAMS as ADMIN_PARAMS,
encodeUnit,
fetchPinned,
readDbf,
readShp,
shpPolygons,
simplifyPolygons,
unitTolerance,
unzip,
} from './build-admin-packs.mjs';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const OUT = path.join(ROOT, 'src/data/local_data/us_census_places');
export const SOURCE = Object.freeze({
url: 'https://www2.census.gov/geo/tiger/GENZ2025/shp/cb_2025_us_place_500k.zip',
sha256: 'ce0e4019ecd4123d03d53aaa936eed0459b82e3e14b89a3dcd4d5e8b3308627d',
});
/** County-pack parameters; triangles are valid rings here. */
export const PARAMS = Object.freeze({
...ADMIN_PARAMS.counties,
minRingVertices: 3,
maxRepairSteps: 8,
});
/** Census LSAD codes for census designated places (unincorporated). */
const CDP_LSAD = new Set(['55', '57', '62']);
// ── validity ─────────────────────────────────────────────────────────────
function orient(ax, ay, bx, by, cx, cy) {
const v = (bx - ax) * (cy - ay) - (by - ay) * (cx - ax);
return v > 0 ? 1 : v < 0 ? -1 : 0;
}
function onSegment(ax, ay, bx, by, px, py) {
return (
Math.min(ax, bx) <= px &&
px <= Math.max(ax, bx) &&
Math.min(ay, by) <= py &&
py <= Math.max(ay, by)
);
}
/** Whether segments ab and cd share any point (touching counts). */
export function segmentsIntersect(a, b, c, d) {
const o1 = orient(a[0], a[1], b[0], b[1], c[0], c[1]);
const o2 = orient(a[0], a[1], b[0], b[1], d[0], d[1]);
const o3 = orient(c[0], c[1], d[0], d[1], a[0], a[1]);
const o4 = orient(c[0], c[1], d[0], d[1], b[0], b[1]);
if (o1 !== o2 && o3 !== o4) return true;
if (o1 === 0 && onSegment(a[0], a[1], b[0], b[1], c[0], c[1])) return true;
if (o2 === 0 && onSegment(a[0], a[1], b[0], b[1], d[0], d[1])) return true;
if (o3 === 0 && onSegment(c[0], c[1], d[0], d[1], a[0], a[1])) return true;
if (o4 === 0 && onSegment(c[0], c[1], d[0], d[1], b[0], b[1])) return true;
return false;
}
/**
* Whether segments of two different rings cross or overlap. Rings may touch
* at a point (a hole meeting its outer ring at a vertex is valid).
*/
function segmentsCross(s, t) {
const [a, b, c, d] = [s.a, s.b, t.a, t.b];
const o1 = orient(a[0], a[1], b[0], b[1], c[0], c[1]);
const o2 = orient(a[0], a[1], b[0], b[1], d[0], d[1]);
const o3 = orient(c[0], c[1], d[0], d[1], a[0], a[1]);
const o4 = orient(c[0], c[1], d[0], d[1], b[0], b[1]);
if (o1 && o2 && o3 && o4) return o1 !== o2 && o3 !== o4;
if (o1 === 0 && o2 === 0) {
// Collinear: overlapping by more than a point.
const key = a[0] !== b[0] ? 0 : 1;
const lo = Math.max(Math.min(a[key], b[key]), Math.min(c[key], d[key]));
const hi = Math.min(Math.max(a[key], b[key]), Math.max(c[key], d[key]));
return hi > lo;
}
// One endpoint on the other segment: a crossing only when the rings pass
// through each other there; a touch at a point is allowed.
return false;
}
/**
* Whether the rings of one part ([outer, ...holes], open, integer
* coordinates) are simple: no ring crosses or touches itself away from its
* shared vertices, no two rings cross or overlap (they may touch at a point),
* and no ring is flat.
*/
export function partIsValid(rings) {
const segments = [];
for (let r = 0; r < rings.length; r++) {
const ring = rings[r];
if (ring.length < 3) return false;
let twice = 0;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
twice += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
if (twice === 0) return false;
for (let i = 0; i < ring.length; i++) {
const a = ring[i];
const b = ring[(i + 1) % ring.length];
segments.push({
r,
i,
n: ring.length,
a,
b,
minX: Math.min(a[0], b[0]),
maxX: Math.max(a[0], b[0]),
});
}
}
segments.sort((s, t) => s.minX - t.minX);
for (let x = 0; x < segments.length; x++) {
const s = segments[x];
for (let y = x + 1; y < segments.length; y++) {
const t = segments[y];
if (t.minX > s.maxX) break;
if (s.r === t.r) {
const gap = Math.abs(s.i - t.i);
// Neighbours share exactly one vertex; that is not a crossing — unless
// they fold back over each other.
if (gap === 1 || gap === s.n - 1) {
if (s.n > 3 && collinearOverlap(s, t)) return false;
continue;
}
}
if (segmentsCross(s, t)) return false;
}
}
return true;
}
function collinearOverlap(s, t) {
const shared =
s.a === t.a || s.a === t.b ? s.a : s.b === t.a || s.b === t.b ? s.b : null;
if (!shared) return false;
const p = s.a === shared ? s.b : s.a;
const q = t.a === shared ? t.b : t.a;
if (orient(shared[0], shared[1], p[0], p[1], q[0], q[1]) !== 0) return false;
// Collinear and pointing the same way from the shared vertex: a spike.
return (
(p[0] - shared[0]) * (q[0] - shared[0]) +
(p[1] - shared[1]) * (q[1] - shared[1]) >
0
);
}
function toIntegers(polygons, decimals) {
const f = 10 ** decimals;
return polygons.map((poly) =>
poly.map((ring) =>
ring.map(([lon, lat]) => [Math.round(lon * f), Math.round(lat * f)]),
),
);
}
export function polygonsValid(polygons, decimals) {
return toIntegers(polygons, decimals).every(partIsValid);
}
/**
* Simplify a place like `simplifyUnit`, then halve the tolerance until every
* part is valid; then one more decimal; then the source rings at two more.
* @returns {{polygons: Array, decimals: number, repaired: boolean, valid: boolean}}
*/
export function simplifyPlace(source, params = PARAMS) {
let tolerance = unitTolerance(source, params);
const attempts = [];
for (let step = 0; step <= params.maxRepairSteps; step++) {
attempts.push([tolerance, params.decimals]);
tolerance /= 2;
}
attempts.push([params.minToleranceDeg / 5, params.decimals + 1]);
attempts.push([0, params.decimals + 1]);
attempts.push([0, params.decimals + 2]);
let first = null;
for (let i = 0; i < attempts.length; i++) {
const [tol, decimals] = attempts[i];
const polygons = simplifyPolygons(source, params, tol, decimals);
if (!polygons.length) continue;
first ||= { polygons, decimals };
if (polygonsValid(polygons, decimals))
return { polygons, decimals, repaired: i > 0, valid: true };
}
return first
? { ...first, repaired: false, valid: false }
: {
polygons: [],
decimals: params.decimals,
repaired: false,
valid: false,
};
}
// ── names ────────────────────────────────────────────────────────────────
const GOVERNMENT =
/\s+(?:metropolitan government|unified government|consolidated government|metro government|city and county|urban county)\b.*$/i;
/**
* Everyday names for records whose Census name is formal: "Nashville-Davidson
* metropolitan government (balance)" is also "Nashville"; "Urban Honolulu"
* is "Honolulu".
*/
export function placeAliases(name) {
const out = new Set();
let base = String(name).replace(/\s*\(balance\)\s*$/i, '');
if (base !== name) out.add(base);
const stripped = base.replace(GOVERNMENT, '');
if (stripped !== base) {
out.add(stripped);
base = stripped;
}
if (base !== name && /[-/]/.test(base)) out.add(base.split(/[-/]/)[0].trim());
const urban = /^Urban (.+)$/.exec(name);
if (urban) out.add(urban[1]);
out.delete(name);
return [...out].filter(Boolean).sort();
}
// ── pack ─────────────────────────────────────────────────────────────────
function bboxOf(polygons) {
let w = Infinity;
let s = Infinity;
let e = -Infinity;
let n = -Infinity;
for (const poly of polygons)
for (const [lon, lat] of poly[0]) {
if (lon < w) w = lon;
if (lon > e) e = lon;
if (lat < s) s = lat;
if (lat > n) n = lat;
}
return [w, s, e, n];
}
function union(boxes) {
return [
Math.min(...boxes.map((b) => b[0])),
Math.min(...boxes.map((b) => b[1])),
Math.max(...boxes.map((b) => b[2])),
Math.max(...boxes.map((b) => b[3])),
];
}
/**
* The box of a state's places. Alaska crosses the antimeridian: its box is
* taken with western longitudes shifted past 180 when that is narrower.
*/
function unionBox(boxes) {
const plain = union(boxes);
const shifted = union(
boxes.map(([w, s, e, n]) =>
e < 0 ? [w + 360, s, e + 360, n] : [w, s, e, n],
),
);
return shifted[2] - shifted[0] < plain[2] - plain[0] ? shifted : plain;
}
async function build() {
const params = PARAMS;
const source = await fetchPinned('places', SOURCE);
const files = unzip(source.bytes);
const base = path.basename(new URL(SOURCE.url).pathname, '.zip');
const shapes = readShp(files.get(`${base}.shp`));
const records = readDbf(files.get(`${base}.dbf`));
if (shapes.length !== records.length)
throw new Error('places: shp/dbf record counts differ');
const states = new Map();
const stats = { records: records.length, repaired: [], invalid: [] };
records.forEach((record, index) => {
const unit = simplifyPlace(shpPolygons(shapes[index]), params);
if (!unit.polygons.length) throw new Error(`no geometry: ${record.GEOID}`);
if (unit.repaired) stats.repaired.push(record.GEOID);
if (!unit.valid) stats.invalid.push(record.GEOID);
const alt = placeAliases(record.NAME);
const lsad = record.NAMELSAD.startsWith(record.NAME)
? record.NAMELSAD.slice(record.NAME.length).trim()
: '';
let state = states.get(record.STUSPS);
if (!state) {
state = {
st: record.STUSPS,
fips: record.STATEFP,
name: record.STATE_NAME,
features: [],
boxes: [],
};
states.set(record.STUSPS, state);
}
state.boxes.push(bboxOf(unit.polygons));
state.features.push({
geoid: record.GEOID,
name: record.NAME,
...(alt.length ? { alt } : {}),
full: record.NAMELSAD,
...(lsad ? { lsad } : {}),
...(CDP_LSAD.has(record.LSAD) ? { cdp: true } : {}),
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
});
if (stats.invalid.length)
throw new Error(`places still invalid: ${stats.invalid.join(', ')}`);
await mkdir(OUT, { recursive: true });
for (const file of await readdir(OUT))
if (/^[A-Z]{2}\.json$/.test(file)) await rm(path.join(OUT, file));
const list = [...states.values()].sort((a, b) => a.st.localeCompare(b.st));
let bytes = 0;
for (const state of list) {
state.features.sort((a, b) => a.geoid.localeCompare(b.geoid));
const text =
JSON.stringify({
st: state.st,
state: state.name,
fips: state.fips,
features: state.features,
}) + '\n';
bytes += Buffer.byteLength(text);
await writeFile(path.join(OUT, `${state.st}.json`), text);
}
const round = (v) => Math.round(v * 1e4) / 1e4;
const index = {
meta: {
title: 'US Census Bureau cartographic boundary places',
source: base,
url: SOURCE.url,
sha256: source.digest,
license: 'Public domain (U.S. Government work, 17 U.S.C. § 105)',
schema:
'states[]: st (USPS), fips, name, count, bbox [w, s, e, n] (e > 180 across the antimeridian). <st>.json: st, state, fips, features[]: geoid, name (NAME), alt?[] (everyday names for formal records), full (NAMELSAD), lsad? (city, town, CDP, …), cdp? (census designated place), d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
decimals: params.decimals,
curation: params,
script: 'scripts/build-census-places.mjs',
},
states: list.map((s) => ({
st: s.st,
fips: s.fips,
name: s.name,
count: s.features.length,
bbox: unionBox(s.boxes).map(round),
})),
};
await writeFile(
path.join(OUT, 'index.json'),
JSON.stringify(index, null, 1) + '\n',
);
await writeFile(path.join(OUT, 'files.js'), filesModule(list));
const count = list.reduce((n, s) => n + s.features.length, 0);
console.log(
`us_census_places: ${count} places in ${list.length} files, ${bytes} bytes; ` +
`${stats.repaired.length} re-simplified to stay valid`,
);
const args = process.argv.slice(2);
const python = args.includes('--python')
? args[args.indexOf('--python') + 1]
: 'python3';
execFileSync(
python,
[path.join(ROOT, 'scripts/pack_geometry.py'), 'repair', 'census'],
{ stdio: 'inherit', env: { ...process.env, PYTHONDONTWRITEBYTECODE: '1' } },
);
}
/**
* The file list as literal URLs, so a bundler emits every state file as an
* asset and the loader can fetch one by its code.
*/
function filesModule(list) {
const lines = [
'// Generated by scripts/build-census-places.mjs; do not edit.',
"export const INDEX_URL = new URL('./index.json', import.meta.url);",
'export const STATE_URLS = {',
...list.map(
(s) => ` ${s.st}: new URL('./${s.st}.json', import.meta.url),`,
),
'};',
];
return lines.join('\n') + '\n';
}
const isMain =
process.argv[1] &&
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
if (isMain) await build();
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Maintainer-only Overture military-name pack build (DuckDB 1.5.5)."""
import argparse
import gzip
import hashlib
import json
import math
from pathlib import Path
import re
ROOT = Path(__file__).resolve().parent.parent
RELEASE = '2026-09-23.1'
CLASSES = ['military_land', 'airfield', 'naval_base', 'range', 'barracks', 'base', 'training_area']
PACK = ROOT / 'src/data/local_data/osm_military_names'
def assemble(rows):
records = []
snapshots = set()
for row in rows:
source = next(s for s in row['sources'] if s['dataset'] == 'OpenStreetMap')
match = re.fullmatch(r'([nwr])(\d+)@\d+', source['record_id'])
if not match:
raise ValueError('Invalid typed OSM identity')
ident = match[1] + match[2]
bbox = [row['bbox'][k] for k in ['xmin', 'ymin', 'xmax', 'ymax']]
lon, lat, area = row['lon'], row['lat'], row['area']
if not all(math.isfinite(v) for v in [lon, lat, area, *bbox]):
raise ValueError('Invalid geometry measurement')
if not bbox[0] - 1e-7 <= lon <= bbox[2] + 1e-7 or not bbox[1] - 1e-7 <= lat <= bbox[3] + 1e-7:
raise ValueError('Label outside source bounds')
kind = row['military'] if row['military'] in CLASSES else 'military_land'
records.append([ident, row['name'], round(lon, 5), round(lat, 5),
*[round(v, 4) for v in bbox], CLASSES.index(kind), max(1, round(abs(area)))])
snapshots.add(source['version'][:10])
records.sort(key=lambda r: (r[0][0], int(r[0][1:])))
if len(set(r[0] for r in records)) != len(records):
raise ValueError('Duplicate OSM identity')
return {'release': RELEASE, 'snapshots': sorted(snapshots), 'classes': CLASSES, 'records': records}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--cache', type=Path, required=True)
parser.add_argument('--extensions', type=Path)
parser.add_argument('--verify', action='store_true')
args = parser.parse_args()
args.cache.mkdir(parents=True, exist_ok=True)
cached = args.cache / ('military-names-' + RELEASE + '.json')
if not cached.exists():
import duckdb
if duckdb.__version__ != '1.5.5':
raise RuntimeError('Use duckdb==1.5.5')
config = {'threads': '8', 'memory_limit': '3GB', 'temp_directory': str(args.cache / 'temp')}
if args.extensions:
config['extension_directory'] = str(args.extensions)
connection = duckdb.connect(config=config)
if not args.extensions:
connection.execute('INSTALL httpfs; INSTALL spatial;')
connection.execute('LOAD httpfs; LOAD spatial; SET geometry_always_xy=true; SET enable_progress_bar=false;')
files = json.loads((ROOT / 'scripts/military-names-files.json').read_text())
if not all('/' + RELEASE + '/' in url for url in files):
raise ValueError('Unexpected source release')
cursor = connection.execute((ROOT / 'scripts/military-names.sql').read_text(), [files])
fields = [d[0] for d in cursor.description]
rows = [dict(zip(fields, row)) for row in cursor.fetchall()]
cached.write_text(json.dumps(rows, ensure_ascii=False, separators=(',', ':')))
pack = assemble(json.loads(cached.read_text()))
raw = json.dumps(pack, ensure_ascii=False, separators=(',', ':')).encode()
digest = hashlib.sha256(raw).hexdigest()
if args.verify:
expected = (PACK / 'names.sha256').read_text().split()[0]
if digest != expected:
raise ValueError(f'Output SHA-256 mismatch: {digest}')
else:
PACK.mkdir(parents=True, exist_ok=True)
(PACK / 'names.json').write_bytes(raw)
(PACK / 'names.sha256').write_text(digest + ' names.json\n')
print(json.dumps({'records': len(pack['records']), 'rawBytes': len(raw),
'gzipBytes': len(gzip.compress(raw, compresslevel=9, mtime=0)), 'sha256': digest}))
if __name__ == '__main__':
main()
+382
View File
@@ -0,0 +1,382 @@
#!/usr/bin/env python3
"""Maintainer-only Who's On First neighborhood pack build (Shapely 2.1).
Reads the pinned Who's On First neighbourhood, macrohood and microhood
archives and the WOF sources registry, keeps current polygon records whose
geometry source matches the reviewed open-licence allowlist, simplifies them
to about 15 m, and
writes src/data/local_data/wof_neighborhoods/:
index.json format, licences per source, tile list with boxes
<tile>.json records whose box centre falls in that quadtree tile
files.js their URLs, for bundlers
ATTRIBUTION.md per-source credits
Usage:
python scripts/build-wof-neighborhoods.py --cache /tmp/wof-neighborhoods [--verify]
Requires shapely==2.1.2 on GEOS 3.13.1 (checked). --verify rebuilds in memory
and fails unless every shipped file matches byte for byte.
Downloads go to --cache and are checked against the pinned SHA-256. The
neighbourhood archive is about 400 MB.
"""
import argparse
import collections
import hashlib
import json
import math
import subprocess
import tarfile
from pathlib import Path
from shapely import make_valid
from shapely.affinity import scale
from shapely.geometry import MultiPolygon, shape
import pack_geometry
ROOT = Path(__file__).resolve().parent.parent
PACK = ROOT / 'src/data/local_data/wof_neighborhoods'
# The legacy distribution is frozen (last published 2025-10-14) and only
# offers `-latest` names; the SHA-256 pins make a changed file fail loudly.
DIST = 'https://data.geocode.earth/wof/dist/legacy'
ARCHIVES = {
'neighbourhood': {
'url': DIST + '/whosonfirst-data-neighbourhood-latest.tar.bz2',
'sha256': '8e5e413aacf2244e4e4167fbd9be860bc407ae6de8b733e47f13abe6d3d10455',
},
'macrohood': {
'url': DIST + '/whosonfirst-data-macrohood-latest.tar.bz2',
'sha256': 'f5fbfabcb8229b83242dbd38530c27e61867e8b3b097e3393d9c841962a99679',
},
'microhood': {
'url': DIST + '/whosonfirst-data-microhood-latest.tar.bz2',
'sha256': 'e8197fa576e8831be8f091ac7e219d29224d89404ab0c56d377339e42a8f29ac',
},
}
# The registry at a fixed commit.
SOURCES = {
'url': 'https://raw.githubusercontent.com/whosonfirst/whosonfirst-sources/'
'e17af153c144fe6e08aa8db2607b4bf8aecd448d/data/sources-spec-latest.json',
'sha256': 'd8a5dd873d26eee11583de4438185c046dec33848d5bde27047ca4bff44d9c39',
}
TOLERANCE_M = 15
PRECISION = 5
TILE_MAX_BYTES = 200_000
TILE_MAX_DEPTH = 16
LICENSE_POLICY = json.loads((ROOT / 'scripts/wof-open-licenses.json').read_text())
ALLOWED_LICENSES = frozenset(LICENSE_POLICY['licenses'])
REVIEWED_SOURCES = LICENSE_POLICY.get('reviewedSources', {})
FIELDS = ['id', 'name', 'country', 'type', 'bbox', 'aliases', 'source', 'rings', 'label']
def fetch(cache, name, url, sha256):
path = cache / name
if not path.exists():
print('Downloading', url, flush=True)
# Plain urllib requests are refused by the WOF host; curl is not.
subprocess.run(['curl', '--fail', '-L', '-o', str(path), url], check=True)
digest = hashlib.sha256()
with open(path, 'rb') as handle:
for chunk in iter(lambda: handle.read(1 << 20), b''):
digest.update(chunk)
if digest.hexdigest() != sha256:
raise ValueError(f'{name}: SHA-256 {digest.hexdigest()} is not the pinned {sha256}')
return path
# ── geometry ────────────────────────────────────────────────────────────
def simplify(geometry):
"""Simplify at TOLERANCE_M in local metres, snap to the pack grid, keep it
valid, and order parts largest first. Returns (rings, label) where the
label is a point inside the largest part, or ([], None)."""
geometry = MultiPolygon(pack_geometry.polygon_parts(make_valid(geometry) if not geometry.is_valid else geometry))
if geometry.is_empty:
return [], None
lat = geometry.centroid.y
sx = 111320 * max(0.001, math.cos(math.radians(lat)))
sy = 110574
metric = scale(geometry, xfact=sx, yfact=sy, origin=(0, 0))
metric = metric.simplify(TOLERANCE_M, preserve_topology=True)
geometry = scale(metric, xfact=1 / sx, yfact=1 / sy, origin=(0, 0))
# To whole pack units (10^-PRECISION degrees), where GEOS is exact.
units = scale(geometry, xfact=10 ** PRECISION, yfact=10 ** PRECISION, origin=(0, 0))
parts = pack_geometry.ordered_parts(pack_geometry.settle(units))
if not parts:
return [], None
rings = [
[pack_geometry.encode_polyline(pack_geometry.open_ring(p.exterior.coords))]
+ [pack_geometry.encode_polyline(pack_geometry.open_ring(r.coords)) for r in p.interiors]
for p in parts
]
# Check what ships: the geometry decoded from the encoded strings.
decoded = [[pack_geometry.decode_polyline(r) for r in poly] for poly in rings]
why = pack_geometry.problems(decoded)
if why:
raise ValueError(f'encoded geometry invalid: {why}')
point = parts[0].representative_point()
point = shape({'type': 'Point', 'coordinates': [round(point.x) / 10 ** PRECISION, round(point.y) / 10 ** PRECISION]})
return rings, [point.x, point.y]
# ── records ─────────────────────────────────────────────────────────────
def read_archive(path, records):
with tarfile.open(path, 'r|bz2') as archive:
for member in archive:
if not member.name.endswith('.geojson') or '-alt-' in member.name:
continue
feature = json.load(archive.extractfile(member))
if feature['geometry']['type'] not in ('Polygon', 'MultiPolygon'):
records.pop(feature['properties']['wof:id'], None)
continue
records[feature['properties']['wof:id']] = feature
def keep(properties):
deprecated = properties.get('edtf:deprecated', 'uuuu') not in ('uuuu', '', None)
superseded = bool(properties.get('wof:superseded_by', []))
return properties.get('mz:is_current', -1) != 0 and not deprecated and not superseded
def source_table(spec):
lookup = {}
for source in spec.values():
for key in (source.get('name'), source.get('prefix')):
if key:
lookup[key] = source
return lookup
def verify_source_reviews(lookup):
"""Pin exceptional source decisions to the exact reviewed registry evidence."""
registry_commit = SOURCES['url'].split('/')[-3]
for key, review in REVIEWED_SOURCES.items():
source = lookup.get(key)
if not source:
raise ValueError(f'{key}: reviewed source is absent from the pinned registry')
if review.get('registryCommit') != registry_commit:
raise ValueError(f'{key}: review does not match the pinned registry commit')
if review.get('registrySha256') != SOURCES['sha256']:
raise ValueError(f'{key}: review does not match the pinned registry SHA-256')
if review.get('rawLicenseType') != source.get('license_type'):
raise ValueError(f'{key}: reviewed raw licence no longer matches the registry')
text = source.get('license_text') or ''
if review.get('licenseTextSha256') != hashlib.sha256(text.encode()).hexdigest():
raise ValueError(f'{key}: reviewed licence text no longer matches the registry')
if review['rawLicenseType'] not in ALLOWED_LICENSES:
raise ValueError(f'{key}: reviewed raw licence is not admitted by policy')
def licence_allowed(value):
"""Only exact raw registry strings reviewed in the policy are accepted."""
return isinstance(value, str) and value in ALLOWED_LICENSES
def source_allowed(key, source):
return (key != 'unknown' and bool(source) and
licence_allowed(source.get('license_type')))
def build_records(archives, lookup):
raw = {}
for kind in ('neighbourhood', 'macrohood', 'microhood'):
read_archive(archives[kind], raw)
print(kind, len(raw), flush=True)
features = []
excluded = collections.Counter()
for wof_id in sorted(raw):
properties = raw[wof_id]['properties']
if not keep(properties):
continue
key = properties.get('src:geom', 'unknown')
if not source_allowed(key, lookup.get(key)):
excluded[key] += 1
continue
name = properties['wof:name']
aliases = sorted({
alias
for k, values in properties.items()
if k.startswith('name:') and isinstance(values, list)
for alias in values
if isinstance(alias, str) and alias != name
})
if not name.strip() and not aliases:
continue # nothing to ask for it by
geometry = shape(raw[wof_id]['geometry'])
if geometry.is_empty:
continue
rings, label = simplify(geometry)
if not rings:
continue
country = properties.get('iso:country', properties.get('wof:country'))
features.append([
wof_id, name, country, properties['wof:placetype'],
[round(v, PRECISION) for v in geometry.bounds], aliases, key, rings, label,
])
return features, excluded
# ── tiles ───────────────────────────────────────────────────────────────
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
def tiles(features, key='', box=(-180.0, -90.0, 180.0, 90.0)):
size = sum(len(compact(f).encode()) for f in features)
if size <= TILE_MAX_BYTES or len(key) >= TILE_MAX_DEPTH:
yield key or 'world', features
return
west, south, east, north = box
mx, my = (west + east) / 2, (south + north) / 2
quads = [[], [], [], []]
for f in features:
x = (f[4][0] + f[4][2]) / 2
y = (f[4][1] + f[4][3]) / 2
quads[(1 if x >= mx else 0) + (2 if y >= my else 0)].append(f)
boxes = [(west, south, mx, my), (mx, south, east, my), (west, my, mx, north), (mx, my, east, north)]
for i, quad in enumerate(quads):
if quad:
yield from tiles(quad, key + str(i), boxes[i])
def union_box(features):
return [
min(f[4][0] for f in features), min(f[4][1] for f in features),
max(f[4][2] for f in features), max(f[4][3] for f in features),
]
# ── credits ─────────────────────────────────────────────────────────────
def licence_label(key, source):
licence = source['license_type']
if key == 'karmashapes':
return 'CC BY 4.0'
return {
'CC-BY': 'CC BY', 'Attribution 3.0 Unported': 'CC BY 3.0', 'Public domain': 'Public Domain',
'Open Government License - British Columbia, v2.0': 'Open Government Licence - British Columbia, v2.0',
}.get(licence, licence)
def attribution(counts, lookup):
lines = [
'# Neighborhood boundary credits',
'',
"Neighborhood boundaries from [Who's On First](https://whosonfirst.org/docs/licenses/) "
'and contributors, used under the licences below. Modified: boundaries simplified to about 15 m, repaired and re-encoded.',
'',
'| Source | Records | Licence | Terms |',
'|---|---:|---|---|',
]
for key, count in sorted(counts.items(), key=lambda kv: (-kv[1], kv[0])):
source = lookup[key]
name = source.get('fullname') or key
terms = source.get('license') or ''
link = f'[terms]({terms})' if terms.startswith('http') else terms
lines.append(f'| {name} (`{key}`) | {count:,} | {licence_label(key, source)} | {link} |')
via = [(key, lookup[key]) for key in sorted(counts) if lookup[key].get('src:via')]
if via:
lines += ['', '## Upstream sources', '']
for key, source in via:
lines += [f'### {source.get("fullname") or key}', '']
for v in source['src:via']:
note = f' ({v["source_note"]})' if v.get('source_note') else ''
context = f'{v["context"]}: ' if v.get('context') else ''
lines.append(f'- {context}{v.get("source_name", "")}{note} {v.get("source_link", "")}'.rstrip())
lines.append('')
return '\n'.join(lines).rstrip() + '\n'
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('--cache', type=Path, required=True)
parser.add_argument('--verify', action='store_true', help='rebuild in memory and compare with the shipped pack')
args = parser.parse_args()
pack_geometry.require_pinned_tooling()
args.cache.mkdir(parents=True, exist_ok=True)
archives = {
kind: fetch(args.cache, Path(a['url']).name, a['url'], a['sha256'])
for kind, a in ARCHIVES.items()
}
spec = json.loads(fetch(args.cache, 'sources-spec-latest.json', SOURCES['url'], SOURCES['sha256']).read_text())
lookup = source_table(spec)
verify_source_reviews(lookup)
features, excluded = build_records(archives, lookup)
if len({f[0] for f in features}) != len(features):
raise ValueError('duplicate WOF id')
out = {}
tile_list = []
for key, members in tiles(features):
members.sort(key=lambda f: f[0])
out[f'{key}.json'] = compact({'tile': key, 'features': members}) + '\n'
tile_list.append({'key': key, 'count': len(members), 'bbox': union_box(members)})
counts = collections.Counter(f[6] for f in features)
index = {
'meta': {
'title': "Who's On First neighbourhood, macrohood and microhood polygons",
'archives': ARCHIVES,
'sourcesRegistry': SOURCES,
'license': 'Per source; see sources and ATTRIBUTION.md',
'format': 'neighborhood-polylines-v1',
'fields': FIELDS,
'precision': PRECISION,
'coordinateOrder': 'lon,lat',
'toleranceM': TOLERANCE_M,
'schema': (
'tiles[]: key, count, bbox [w, s, e, n] of its records. <key>.json: tile, features[] as '
'fields: WOF id, name, ISO country, placetype, bbox, aliases[] (other-language names), '
'geometry source key, rings (polygons, largest first, each [outer, ...holes]; each ring '
'an encoded polyline string: zigzag varint deltas of lon then lat at 10^-precision '
'degrees), label [lon, lat] (a point inside the largest polygon)'
),
'sources': {
key: {
'name': lookup[key].get('fullname') or key,
'license': licence_label(key, lookup[key]),
'terms': lookup[key].get('license') or '',
'records': counts[key],
}
for key in sorted(counts)
},
'script': 'scripts/build-wof-neighborhoods.py',
},
'tiles': tile_list,
}
out['index.json'] = json.dumps(index, ensure_ascii=False, indent=1) + '\n'
out['ATTRIBUTION.md'] = attribution(counts, lookup)
# Literal URLs, so a bundler emits every tile as an asset.
out['files.js'] = '\n'.join([
'// Generated by scripts/build-wof-neighborhoods.py; do not edit.',
"export const INDEX_URL = new URL('./index.json', import.meta.url);",
'export const TILE_URLS = {',
*[f" t{t['key']}: new URL('./{t['key']}.json', import.meta.url)," for t in tile_list],
'};',
]) + '\n'
if args.verify:
# Reproducibility: a rebuild must match what ships, byte for byte.
shipped = {p.name for p in PACK.glob('*.json')} | {'ATTRIBUTION.md', 'files.js'}
differ = sorted(
name for name in shipped | set(out)
if not (PACK / name).exists() or name not in out
or (PACK / name).read_bytes() != out[name].encode('utf-8')
)
if differ:
raise SystemExit(f'rebuild differs from the shipped pack: {differ[:10]}')
print(f'wof_neighborhoods: rebuild matches all {len(out)} shipped files')
return
PACK.mkdir(parents=True, exist_ok=True)
for old in PACK.glob('*.json'):
old.unlink()
for name, text in out.items():
(PACK / name).write_text(text, encoding='utf-8')
print(f'wof_neighborhoods: {len(features)} records in {len(tile_list)} tiles; '
f'excluded {sum(excluded.values())} with an unapproved licence: {dict(excluded)}', flush=True)
if __name__ == '__main__':
main()
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env node
import { execFileSync } from 'node:child_process';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import {
LAYER_STATE_TOKEN_RESERVATIONS,
parseLayerStateTokenReservations,
validateLayerStateAllocations,
validateLayerStateRegistry,
} from '../src/data/layerState.js';
// The pre-ledger main revision had exactly the 28 pinned v2 registry entries.
// Accept only that source blob as the one-time bootstrap; any other published
// source without a ledger must fail closed rather than risk losing a token.
const PRE_LEDGER_LAYER_STATE_BLOB = 'e559a4c40191512dcefb83134958a3bfd4a98a69';
// This snapshot belongs to the verified published blob above, not to the PR's
// editable legacy map. Otherwise a candidate could redefine both its ledger
// and its comparison baseline before the first ledger reaches main.
export const PRE_LEDGER_LAYER_STATE_TOKENS = Object.freeze({
'ais-live-vessels': 'a',
'alpr-cameras': 'p',
'bhote-koshi-2026': 'h',
'bhote-koshi-locator': 'z',
bikeshare: 'b',
cctv: 'c',
directions: 'n',
earthquakes: 'e',
'fire-perimeters': '2',
flights: 'f',
'local-dams': 'q',
'local-datacenters': 'd',
'local-firms': 'w',
military: 'm',
'military-awareness': 'g',
'military-installations': 'i',
radio: 'r',
'recent-imagery': '1',
'rocket-launches': 'x',
satellites: 's',
'telegeography-submarine-cables': 'u',
traffic: 't',
transit: 'j',
'weather-cyclones': 'y',
'weather-lightning': 'l',
'weather-radar': 'v',
'weather-satellite': 'o',
wind: 'k',
});
const LEDGER_PATH = 'src/data/layerStateTokenReservations.json';
const SOURCE_PATH = 'src/data/layerState.js';
function git(args, cwd) {
return execFileSync('git', args, {
cwd,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
}).trim();
}
/** Read every permanent assignment from a published Git revision. */
export function readPublishedLayerStateReservations(
baseRef,
cwd = process.cwd(),
) {
const commit = git(
['rev-parse', '--verify', '--end-of-options', `${baseRef}^{commit}`],
cwd,
);
const ledgerSpec = `${commit}:${LEDGER_PATH}`;
let hasLedger = true;
try {
git(['cat-file', '-e', ledgerSpec], cwd);
} catch {
hasLedger = false;
}
if (!hasLedger) {
const sourceBlob = git(['rev-parse', `${commit}:${SOURCE_PATH}`], cwd);
if (sourceBlob !== PRE_LEDGER_LAYER_STATE_BLOB) {
throw new Error(
'Published base has no recognizable layer-state token ledger',
);
}
return PRE_LEDGER_LAYER_STATE_TOKENS;
}
let rows;
try {
rows = JSON.parse(git(['show', ledgerSpec], cwd));
} catch (error) {
throw new Error(
`Cannot read published layer-state token ledger: ${error.message}`,
);
}
return parseLayerStateTokenReservations(rows);
}
export function checkLayerStateTokens(baseRef, cwd = process.cwd()) {
const base = readPublishedLayerStateReservations(baseRef, cwd);
validateLayerStateRegistry();
validateLayerStateAllocations(base, LAYER_STATE_TOKEN_RESERVATIONS);
return {
published: Object.keys(base).length,
added:
Object.keys(LAYER_STATE_TOKEN_RESERVATIONS).length -
Object.keys(base).length,
};
}
const invokedPath = process.argv[1]
? pathToFileURL(path.resolve(process.argv[1])).href
: '';
if (import.meta.url === invokedPath) {
const args = process.argv.slice(2);
const baseRef =
args[0] === '--base-ref' && args.length === 2 ? args[1] : null;
if (!baseRef || baseRef.startsWith('-')) {
console.error('Usage: npm run layer-token:check -- --base-ref origin/main');
process.exitCode = 1;
} else {
try {
const result = checkLayerStateTokens(baseRef);
console.log(
`Layer tokens valid against ${baseRef}: ${result.published} published, ` +
`${result.added} new.`,
);
} catch (error) {
console.error(`Layer token check failed: ${error.message}`);
process.exitCode = 1;
}
}
}
+10 -1
View File
@@ -89,6 +89,10 @@ export async function checkPackageBoundaries(root) {
const seen = new Set();
await build({
root,
// Keep this gate out of a running dev server's dependency cache
// (node_modules/.vite): a build there re-optimizes dependencies and the
// dev server's workers then fail with 504 "Outdated Optimize Dep".
cacheDir: path.join(root, 'node_modules/.vite-boundaries'),
configFile: false,
envFile: false,
publicDir: false,
@@ -115,7 +119,12 @@ export async function checkPackageBoundaries(root) {
assetsInlineLimit: 0,
rollupOptions: {
input,
external: group.external,
// A declared dependency also owns its package subpath exports
// (for example `@jtarrio/webrtlsdr/rtlsdr.js`).
external: (id) =>
group.external.some(
(name) => id === name || id.startsWith(`${name}/`),
),
// Unused imports must still obey ownership; tree shaking is not a boundary.
treeshake: false,
preserveEntrySignatures: 'strict',
+131 -1
View File
@@ -4,6 +4,11 @@
"scripts/format-scope.json",
"scripts/package-boundaries.json",
"scripts/format.mjs",
"scripts/next-layer-state-token.mjs",
"scripts/check-layer-state-tokens.mjs",
"scripts/qa-layer-token-twochar.mjs",
"src/data/layerStateTokenReservations.json",
"src/data/layerStateTokenLedger.test.mjs",
"scripts/check-package-boundaries.mjs",
"scripts/precompute-cctv-heights.mjs",
"src/tooling/precomputeCctvHeights.test.mjs",
@@ -21,16 +26,28 @@
"src/annotations/drawMode.test.mjs",
"src/annotations/drawTool.js",
"src/annotations/drawTool.test.mjs",
"src/annotations/adminOutline.test.mjs",
"src/annotations/regionRing.test.mjs",
"src/data/adminBoundaries.test.mjs",
"scripts/build-admin-packs.mjs",
"scripts/build-census-places.mjs",
"src/data/placeBoundaries.test.mjs",
"src/data/boundaryLabels.test.mjs",
"src/data/packGeometry.test.mjs",
"src/annotations/placeOutline.test.mjs",
"scripts/qa-admin-outlines.mjs",
"src/data/inputOwnership.test.mjs",
"docs/CODE-BOUNDARIES.md",
"src/main.js",
"docs/APPLICATION.md",
"scripts/qa-application.mjs",
"scripts/qa-terrain-429.mjs",
"src/app/application.js",
"src/app/application.test.mjs",
"src/app/mapLifecycle.test.mjs",
"src/app/startupCamera.test.mjs",
"src/app/viewer.js",
"src/app/atmosphereCompat.test.mjs",
"src/standalone/application.js",
"src/standalone/controls.js",
"src/standalone/data.js",
@@ -99,6 +116,8 @@
"src/tooling/previewServing.test.mjs",
"server/providers/common/request.js",
"server/providers/common/source-root.js",
"src/data/tapAddress.test.mjs",
"src/data/tapAddress.js",
"server/providers/local.js",
"server/providers/military-installations.js",
"server/providers/military-installations/cache.js",
@@ -142,6 +161,8 @@
"src/keylessGeocoder.js",
"src/standalone/placeSearch.js",
"src/ui/panelDisclosure.js",
"src/ui/imagerySplit.js",
"src/ui/imagerySplit.test.mjs",
"src/ui/surfaceKeyboard.js",
"src/ui/surfaceKeyboard.test.mjs",
"src/ui/panelRails.js",
@@ -334,6 +355,16 @@
"src/layers/earthquakes/model.js",
"src/layers/earthquakes/source.js",
"src/layers/earthquakes/ownership.test.mjs",
"src/layers/perimeters/index.js",
"src/layers/perimeters/records.js",
"src/layers/perimeters/source.js",
"src/layers/perimeters/cards.js",
"src/layers/perimeters/inciweb.js",
"src/layers/perimeters/records.test.mjs",
"src/layers/perimeters/source.test.mjs",
"src/layers/perimeters/cards.test.mjs",
"src/layers/perimeters/inciweb.test.mjs",
"src/layers/perimeters/ownership.test.mjs",
"src/data/firmsHeatmap.js",
"src/data/fireAnchors.js",
"src/layers/firms/anchors.js",
@@ -383,6 +414,10 @@
"src/layers/launches/model.js",
"src/layers/launches/selection.js",
"src/layers/launches/camera.js",
"src/data/localGeojson.analyst.test.mjs",
"src/data/satellites.analyst.test.mjs",
"src/data/layerSnapshot.test.mjs",
"src/data/satellitePass.test.mjs",
"src/data/satellites.js",
"src/data/rocketLaunches.js",
"src/layers/satellites/source.test.mjs",
@@ -495,6 +530,7 @@
"src/layers/cctv/source.test.mjs",
"src/data/alprCameras.js",
"src/data/alprCameras.test.mjs",
"src/data/dataCredits.test.mjs",
"src/layers/alpr/index.js",
"src/layers/alpr/model.js",
"src/layers/alpr/policy.js",
@@ -543,17 +579,21 @@
"src/data/telegeographySubmarineCables.test.mjs",
"src/data/naturalEarthRegions.js",
"src/data/neighborhoodPolygons.js",
"src/data/bundledJson.js",
"src/layers/submarineCables/bundledSource.test.mjs",
"src/maps/availability.js",
"src/maps/catalog.js",
"src/maps/controller.js",
"src/maps/controller.test.mjs",
"src/maps/imageryComparison.js",
"src/maps/credits.js",
"src/maps/defaultSources.js",
"src/maps/google3d.js",
"src/maps/imagery.js",
"src/maps/registry.js",
"src/maps/terrain.js",
"src/maps/terrainRetry.js",
"src/maps/terrainRetry.test.mjs",
"src/mapStackController.js",
"src/mapStartup.js",
"src/mapStackChips.test.mjs",
@@ -595,6 +635,7 @@
"src/app/layers/flights.js",
"src/app/layers/radio.js",
"src/app/layers/earthquakes.js",
"src/app/layers/perimeters.js",
"src/app/layers/cctv.js",
"src/app/layers/firms.js",
"src/app/layers/satellites.js",
@@ -610,6 +651,7 @@
"src/data/surfaceServices.js",
"src/annotations/resolver.js",
"src/services/terrainHeights.js",
"src/services/terrainHeights.test.mjs",
"src/services/groundFloor.js",
"src/services/meshFloorSampler.js",
"src/services/groundSnap.js",
@@ -729,6 +771,27 @@
"scripts/format-runtime.json",
"src/tooling/transitService.test.mjs",
"src/sources/overpassFeatures.test.mjs",
"src/data/cctvHlsStream.test.mjs",
"src/layers/cctv/videoPlayback.test.mjs",
"src/ui/cctvVideo.test.mjs",
"src/ui/styles/weather.css",
"src/layers/weather/index.js",
"src/layers/weather/clock.js",
"src/layers/weather/clock.test.mjs",
"src/layers/weather/rendering.js",
"src/layers/weather/shellRendering.js",
"src/layers/weather/shellRendering.test.mjs",
"src/layers/weather/infraredAlpha.js",
"src/layers/weather/infraredAlpha.test.mjs",
"src/layers/weather/infraredImage.js",
"src/layers/weather/infraredImage.test.mjs",
"src/layers/weather/source.js",
"src/layers/weather/weather.test.mjs",
"src/layers/weather/rasterTiles.test.mjs",
"src/layers/wind/rendering.test.mjs",
"src/layers/weather/source.test.mjs",
"src/data/weatherProxy.test.mjs",
"server/providers/weather.js",
"src/director/clock.test.mjs",
"src/director/timeline.test.mjs",
"src/scenes/packs/registry.test.mjs",
@@ -739,5 +802,72 @@
"scripts/qa-director-camera.mjs",
"src/director/packs/packs.test.mjs",
"scripts/qa-director-packs.mjs",
"src/scenes/dataPacks/presentation.test.mjs"
"src/scenes/dataPacks/presentation.test.mjs",
"src/director/interactions/interactions.test.mjs",
"scripts/qa-director-interactions.mjs",
"src/director/sharing/sharing.test.mjs",
"scripts/qa-director-sharing.mjs",
"src/ui/weatherPanel.js",
"src/ui/weatherPanel.test.mjs",
"src/ui/railCards.js",
"src/ui/railCards.test.mjs",
"src/ui/railTimeline.js",
"src/ui/railTimeline.test.mjs",
"src/layers/weather/imageryOrder.js",
"src/layers/weather/imageryOrder.test.mjs",
"server/providers/cyclones.js",
"server/providers/firePerimeters.js",
"src/data/cyclonesProxy.test.mjs",
"src/data/firePerimetersProxy.test.mjs",
"src/layers/cyclones/index.js",
"src/layers/cyclones/rendering.js",
"src/layers/cyclones/labels.js",
"src/layers/cyclones/source.js",
"src/layers/cyclones/rendering.test.mjs",
"src/layers/cyclones/labels.test.mjs",
"src/layers/cyclones/index.test.mjs",
"src/layers/cyclones/source.test.mjs",
"src/data/aisStreamSentinels.test.mjs",
"scripts/qa-weather-perf.mjs",
"scripts/qa-weather-journey.mjs",
"scripts/qa-weather-swap.mjs",
"scripts/qa-weather-teardown.mjs",
"src/layers/weather/imageryHost.test.mjs",
"src/sdr/adsbDecoder.test.mjs",
"src/sdr/controller.test.mjs",
"src/sdr/gain.test.mjs",
"src/sdr/spectrum.test.mjs",
"src/sdr/usbDevices.test.mjs",
"src/sources/adsbRecords.test.mjs",
"src/layers/localAdsb/index.test.mjs",
"src/ui/localSdrControls.test.mjs",
"src/ui/localSdrMarkup.test.mjs",
"src/ui/localSdrPresentation.test.mjs",
"src/data/localReceiversProxy.test.mjs",
"src/layers/localAdsb/feeds.test.mjs",
"src/ui/railScrollers.test.mjs",
"src/layers/localAdsb/motion.test.mjs",
"src/data/modelEligibility.test.mjs",
"src/layers/recentImagery/model.test.mjs",
"src/layers/recentImagery/catalog.test.mjs",
"src/maps/imageryHost.test.mjs",
"src/layers/recentImagery/rendering.test.mjs",
"src/layers/recentImagery/thumbnails.test.mjs",
"src/layers/recentImagery/index.test.mjs",
"src/ui/imageryBoxTool.test.mjs",
"src/ui/recentImagery.test.mjs",
"src/layers/recentImagery/testDoubles.mjs",
"scripts/qa-recent-imagery.mjs",
"scripts/qa-overpass-offload.mjs",
"src/overpassOffload.test.mjs",
"src/sources/openFreeMap.test.mjs",
"src/sources/vectorTiles.test.mjs",
"src/sources/militaryTileGeometry.test.mjs",
"src/layers/traffic/surface.test.mjs",
"src/layers/traffic/roadModes.test.mjs",
"scripts/qa-traffic-motion.mjs",
"src/layers/traffic/motion.test.mjs",
"scripts/qa-journey-recorder.mjs",
"scripts/qa-alpr-journey.mjs",
"scripts/qa-context-arrivals.mjs"
]
+66
View File
@@ -0,0 +1,66 @@
[
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00000-18dcb239-f29e-5e57-b36f-4bd46c12fb86-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00001-4a0ec645-f2e2-5fd1-aeaf-a2210290fe76-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00002-0bfb9ec4-5947-583a-9b20-5b39a6635e7d-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00003-0aa711cc-b475-5914-9670-7295640f184f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00004-9a1af7ec-89dd-5aa1-a419-d6325719c330-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00005-e85508c1-3f80-5883-8539-35302da710f3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00006-184886ec-67b4-5a65-852f-f26f757aaeea-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00007-adbd870a-4116-50a2-9554-57064af19a18-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00008-6362555a-8b0c-5589-a6fe-b71ddca82083-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00009-19bb41f3-6c3b-579a-adae-8f69030c37c7-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00010-0e372a9c-a60d-5b06-89e2-866989d612dd-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00011-2ae04990-7b00-5def-bf72-12b2fdc13b48-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00012-dcfe8859-26a4-544b-ad61-dfbc7e7f272a-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00013-a369ed98-5e3b-5bda-94c9-460f06e8746b-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00014-d03d3cde-6ae3-5938-abe9-b2018f7ab0e3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00015-57a736a1-7d58-5851-877a-2fca7de484ba-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00016-141b8ad5-f6d4-5f5e-8353-450a53188e27-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00017-14133e77-297b-57f3-b028-c5133bb56578-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00018-65485e6e-8939-51ab-8566-47dd3887b6c3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00019-64fb52b8-5ba4-5f00-860f-4fe71e1c9f3a-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00020-0feea924-6adb-5118-9a2b-e0a0d3ef6963-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00021-a9115b0c-a202-5468-bfae-0a1e6f953be3-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00022-440ee3b6-f6ae-5bf9-b5d1-cc81fcb0a430-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00023-53a4d462-62b9-589c-9650-f7c9652354e7-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00024-54089af7-32f4-5ad8-8970-cea7350cbb20-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00025-59cc63a7-e7da-58e2-8d06-1df15f3a0636-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00026-ba0e8b28-756c-5b5e-aa01-0939a15e45c8-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00027-8237ebb3-2fcb-50d7-884b-81c8e81bda1e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00028-6fb209d3-9fbe-5155-9983-896b1bb9e84e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00029-e86227ec-f5f7-51c6-b537-a10334796cc4-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00030-a2779b0d-003a-5f5d-812c-02eb4a5db71f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00031-554320d6-a62d-5be7-80b0-d32fb67d87ca-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00032-a49d2756-469a-571f-86ef-be60690cfaea-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00033-0b36146b-6963-5133-8a67-ced0bbb37d6a-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00034-b676aef2-1e90-5966-8531-40f42c5d01d6-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00035-c9b04728-fc59-50f2-bfa8-5f8692ea731e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00036-22a0fbd7-fd24-5832-ac6c-85e82b56ced6-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00037-d157ddb7-8962-515b-9ec3-d54333fcf836-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00038-39d77c44-55f2-5792-83e1-cac6416d0630-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00039-f6871cf2-753b-53f1-aacf-645ea035826d-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00040-3d142e7a-4189-5e50-9f68-0f77e9a51047-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00041-9e6d1cdd-f130-5089-9d0f-703626552100-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00042-4cefe8fe-4966-5e1e-9cf5-19fd90384db8-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00043-65360671-af63-58dd-9784-bceb07abe219-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00044-b423bff1-dd2b-52c4-90f6-cd1605e4d83f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00045-1f4d97be-99fc-50de-b9cb-f3624c9a1817-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00046-32a9756a-e3a0-5980-a8d9-88dd475f25b9-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00047-cf80d668-e9f1-5f69-8e21-0a73f1732e5f-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00048-8d932283-4f3f-533d-895b-684faaa9cc34-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00049-996168f3-0c3a-5b58-8444-168c7d859a04-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00050-f9c3c526-3ea5-5dab-a46d-05c623c3e905-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00051-72b4d98d-4b3b-5b3b-a565-62c206d4fa29-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00052-7385fa90-2fa9-53c8-9ed4-96d5703f3a37-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00053-96804709-94fa-56d2-bafd-69b2cbce3881-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00054-520c3c50-82f8-5f3d-97b5-fa8f5f4e65e7-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00055-8742f073-69cd-579e-bf00-b16b6e6a7669-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00056-58083e97-7020-5522-ac19-d8a5b4dcbc98-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00057-bf201c73-1fb3-5dbc-8cec-d344d999820e-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00058-9e1561c3-ba15-528b-8db3-62ce04b1570b-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00059-dd8cd7ca-410f-54f8-9b50-66beb004d640-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00060-203c7177-ec4a-55c2-8003-d828b144a6d6-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00061-0da76969-7fb1-56b9-bb83-4e40f3acea8d-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00062-13344b87-db6c-5415-a48e-afd95a928a85-c000.zstd.parquet",
"https://overturemaps-us-west-2.s3.us-west-2.amazonaws.com/release/2026-09-23.1/theme=base/type=land_use/part-00063-a61ab643-210f-518d-b6b8-894f72e5c094-c000.zstd.parquet"
]
+11
View File
@@ -0,0 +1,11 @@
SELECT names.primary AS name, bbox, sources,
source_tags['military'] AS military,
ST_X(ST_PointOnSurface(geometry)) AS lon,
ST_Y(ST_PointOnSurface(geometry)) AS lat,
ST_Area_Spheroid(geometry) AS area
FROM read_parquet(?)
WHERE subtype = 'military'
AND nullif(trim(names.primary), '') IS NOT NULL
AND (source_tags['landuse'] = 'military'
OR source_tags['military'] IN
('airfield', 'naval_base', 'range', 'barracks', 'base', 'training_area'))
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env node
import {
LAYER_STATE_TOKEN_RESERVATIONS,
nextLayerStateToken,
} from '../src/data/layerState.js';
const layerId = String(process.argv[2] || '').trim();
if (!/^[a-z0-9-]+$/.test(layerId)) {
console.error('Usage: npm run layer-token:next -- <layer-id>');
process.exitCode = 1;
} else if (Object.hasOwn(LAYER_STATE_TOKEN_RESERVATIONS, layerId)) {
console.error(
`${layerId} already owns token ${LAYER_STATE_TOKEN_RESERVATIONS[layerId]}`,
);
process.exitCode = 1;
} else {
console.log(`${layerId}: ${nextLayerStateToken()}`);
console.log('Order: free digits 0-9, then two-character base-36 00-zz.');
console.log('Re-run after rebasing onto the latest main before merge.');
}
+247
View File
@@ -0,0 +1,247 @@
#!/usr/bin/env python3
"""Maintainer-only topology check and repair for the bundled outline packs.
python scripts/pack_geometry.py check census|wof every shipped feature,
decoded as a whole MultiPolygon, is valid under GEOS, keeps at least
three distinct vertices per ring, and lists its largest part first
python scripts/pack_geometry.py repair census rewrite the Census place
files so every feature passes `check` (run by build-census-places.mjs)
Geometry is checked and repaired in the packs' own integer units (10^-d
degrees), where GEOS arithmetic is exact: a vertex that lies on another edge
after rounding is seen as the touch it is.
Requires Shapely 2.1.2 on GEOS 3.13.1, pinned so repairs are reproducible.
"""
import argparse
import json
import sys
from pathlib import Path
import shapely
from shapely import make_valid, set_precision
from shapely.geometry import MultiPolygon, Polygon
from shapely.geometry.polygon import orient
ROOT = Path(__file__).resolve().parent.parent
DATA = ROOT / 'src/data/local_data'
SHAPELY_VERSION = '2.1.2'
GEOS_VERSION = (3, 13, 1)
def require_pinned_tooling():
if shapely.__version__ != SHAPELY_VERSION or shapely.geos_version != GEOS_VERSION:
raise RuntimeError(
f'Use shapely=={SHAPELY_VERSION} on GEOS {".".join(map(str, GEOS_VERSION))}; '
f'found {shapely.__version__} on GEOS {shapely.geos_version}'
)
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
# ── encodings ────────────────────────────────────────────────────────────
def decode_deltas(encoded):
"""[x0, y0, dx1, dy1, …] → [(x, y), …] in pack units (Census/admin packs)."""
x = y = 0
ring = []
for i in range(0, len(encoded) - 1, 2):
x += encoded[i]
y += encoded[i + 1]
ring.append((x, y))
return ring
def encode_deltas(coords):
out = []
px = py = 0
for fx, fy in coords:
x, y = round(fx), round(fy)
out += [x - px, y - py]
px, py = x, y
return out
def decode_polyline(encoded):
"""Zigzag varint deltas, 5-bit groups offset by 63, lon then lat; pack units."""
values = []
last = [0, 0]
i = 0
while i < len(encoded):
n = shift = 0
while True:
b = ord(encoded[i]) - 63
i += 1
n |= (b & 31) << shift
shift += 5
if b < 32:
break
axis = len(values) % 2
last[axis] += ~(n >> 1) if n & 1 else n >> 1
values.append(last[axis])
return list(zip(values[::2], values[1::2]))
def encode_polyline(coords):
out = []
last = [0, 0]
for x, y in coords:
for i, value in enumerate([round(x), round(y)]):
n = value - last[i]
last[i] = value
n = (n << 1) ^ (n >> 63)
while n >= 32:
out.append(chr((32 | (n & 31)) + 63))
n >>= 5
out.append(chr(n + 63))
return ''.join(out)
# ── geometry ─────────────────────────────────────────────────────────────
def polygon_parts(geometry):
if geometry.geom_type == 'Polygon':
return [] if geometry.is_empty else [geometry]
if hasattr(geometry, 'geoms'):
return [p for g in geometry.geoms for p in polygon_parts(g)]
return []
def open_ring(coords):
coords = [tuple(c[:2]) for c in coords]
return coords[:-1] if len(coords) > 1 and coords[0] == coords[-1] else coords
def ordered_parts(geometry):
"""Polygon parts oriented (outer counter-clockwise), largest area first."""
parts = [orient(p, 1.0) for p in polygon_parts(geometry)]
return sorted(parts, key=lambda p: (-p.area, p.bounds))
def on_grid(geometry):
"""Snap to whole pack units keeping the result valid; polygon parts only."""
snapped = set_precision(geometry, 1, mode='valid_output')
return MultiPolygon(polygon_parts(snapped))
def settle(geometry):
"""A valid MultiPolygon of `geometry` (in pack units) on whole units."""
if not geometry.is_valid:
geometry = make_valid(geometry)
geometry = on_grid(MultiPolygon(polygon_parts(geometry)))
for _ in range(3):
if geometry.is_valid:
break
geometry = on_grid(make_valid(geometry))
return geometry
def rings_ok(polygons):
return all(len(set(ring)) >= 3 for poly in polygons for ring in poly)
def problems(polygons):
"""Why decoded `polygons` ([[outer, *holes], …]) fail the pack contract."""
if not polygons or not rings_ok(polygons):
return 'ring with fewer than three distinct vertices'
try:
geometry = MultiPolygon([Polygon(p[0], p[1:]) for p in polygons])
except Exception as error: # noqa: BLE001 - malformed input is a finding
return f'unbuildable: {error}'
if not geometry.is_valid:
return shapely.is_valid_reason(geometry)
areas = [part.area for part in geometry.geoms]
if max(areas) > areas[0]:
return 'largest part is not first'
return None
# ── packs ────────────────────────────────────────────────────────────────
def census_features():
folder = DATA / 'us_census_places'
decimals = json.loads((folder / 'index.json').read_text())['meta']['decimals']
for path in sorted(folder.glob('[A-Z][A-Z].json')):
pack = json.loads(path.read_text(encoding='utf-8'))
yield path, pack, decimals
def census_polygons(feature):
return [[decode_deltas(r) for r in poly] for poly in feature['polygons']]
def wof_features():
folder = DATA / 'wof_neighborhoods'
index = json.loads((folder / 'index.json').read_text(encoding='utf-8'))
for tile in index['tiles']:
pack = json.loads((folder / f'{tile["key"]}.json').read_text(encoding='utf-8'))
for feature in pack['features']:
yield feature, index['meta']['precision']
def check(pack):
bad = []
total = 0
if pack == 'census':
for _, data, decimals in census_features():
for feature in data['features']:
total += 1
why = problems(census_polygons(feature))
if why:
bad.append((feature['geoid'], feature['name'], why))
else:
for feature, precision in wof_features():
total += 1
polygons = [[decode_polyline(r) for r in poly] for poly in feature[7]]
why = problems(polygons)
if why:
bad.append((feature[0], feature[1], why))
for row in bad[:20]:
print('invalid', *row)
print(f'{pack}: {total} features, {len(bad)} failing')
return not bad
def repair_census():
repaired = 0
for path, data, decimals in census_features():
changed = False
for feature in data['features']:
polygons = census_polygons(feature)
if not problems(polygons):
continue
geometry = MultiPolygon([Polygon(p[0], p[1:]) for p in polygons if len(set(p[0])) >= 3])
parts = ordered_parts(settle(geometry))
feature['polygons'] = [
[encode_deltas(open_ring(p.exterior.coords))]
+ [encode_deltas(open_ring(r.coords)) for r in p.interiors]
for p in parts
]
why = problems(census_polygons(feature))
if why:
raise ValueError(f'{feature["geoid"]} {feature["name"]}: still {why}')
repaired += 1
changed = True
if changed:
path.write_text(compact(data) + '\n', encoding='utf-8')
print(f'us_census_places: {repaired} features repaired to valid GEOS topology')
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('command', choices=['check', 'repair'])
parser.add_argument('pack', choices=['census', 'wof'])
args = parser.parse_args()
require_pinned_tooling()
if args.command == 'repair':
if args.pack != 'census':
parser.error('the WOF build repairs its own geometry')
repair_census()
sys.exit(0 if check('census') else 1)
sys.exit(0 if check(args.pack) else 1)
if __name__ == '__main__':
main()
File diff suppressed because it is too large. Load diff
+695
View File
@@ -0,0 +1,695 @@
#!/usr/bin/env node
/**
* qa-admin-outlines.mjs — bundled state/province/county outlines, driven like a
* user would drive them.
*
* Run: node scripts/qa-admin-outlines.mjs http://localhost:4173 [--headful] [--no-video] [--offline-imagery]
*
* Journey: frame Texas at state scale and outline "Texas"; pan and zoom toward
* Austin in smooth 2–3 s moves; outline "Travis County, Texas"; pan north and
* outline the neighbouring "Williamson County, Texas"; zoom out to see both;
* then outline "Bavaria" (framed, as a voice request for a distant place is),
* and finally Brandenburg (Berlin is a hole) and Hawaii (eight islands).
*
* Asserts for every outline: it resolves from the bundled packs (no geocode,
* boundary or Overpass request), is drawn as a draped outline within the
* budget, and holds still — no rebuilt primitives and no new entities while
* the camera is still. The whole journey must log zero console/page errors and
* send zero requests to any Overpass host or nominatim.openstreetmap.org.
*
* Evidence in qa-shots/ (gitignored): admin-outlines-<step>.png screenshots,
* admin-outlines-journey.mp4 (CDP screencast at ~24 fps, frames re-timed from
* their capture timestamps), and admin-outlines-result.json with per-step
* timings, frame-time p50/p95 and still-camera frame-difference statistics.
* Exits nonzero on any failed assertion.
*/
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import fs from 'node:fs/promises';
import path from 'node:path';
import puppeteer from 'puppeteer';
import sharp from 'sharp';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-admin-outlines.mjs <dev-server-url> [--headful] [--no-video] [--offline-imagery]',
);
process.exit(0);
}
const url = args.find((arg) => !arg.startsWith('--'));
if (!url || !['http:', 'https:'].includes(new URL(url).protocol))
throw new Error('Supply a running dev server URL; see --help');
const recordVideo = !args.includes('--no-video');
const offlineImagery = args.includes('--offline-imagery');
const offlineImageryTile = offlineImagery
? await fs.readFile(
new URL(
'../node_modules/cesium/Build/Cesium/Assets/Textures/NaturalEarthII/0/0/0.jpg',
import.meta.url,
),
)
: null;
const shots = path.resolve('qa-shots');
const framesDir = path.join(shots, 'admin-outlines-frames');
await fs.mkdir(shots, { recursive: true });
await fs.rm(framesDir, { recursive: true, force: true });
await fs.mkdir(framesDir, { recursive: true });
const OUTLINE_BUDGET_MS = 1500;
const STILL_MS = 3000;
const FRAME_INTERVAL_S = 1 / 24;
const result = {
url,
steps: [],
forbiddenRequests: [],
lookups: [],
packRequests: [],
consoleErrors: [],
pageErrors: [],
screenshots: [],
video: null,
offlineImagery,
offlineImageryRequests: 0,
};
const browser = await puppeteer.launch({
headless: !args.includes('--headful'),
protocolTimeout: 300_000,
args: [
'--no-sandbox',
'--use-gl=angle',
'--disable-dev-shm-usage',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
let page;
const frames = [];
let lastKept = -Infinity;
let recording = false;
let marker = 'startup';
try {
page = await browser.newPage();
await page.evaluateOnNewDocument(() => {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
});
await page.setViewport({ width: 1280, height: 800 });
await page.setRequestInterception(true);
page.on('request', (request) => {
let target;
try {
target = new URL(request.url());
} catch {
request.continue();
return;
}
const knownImageryTile =
request.method() === 'GET' &&
((target.origin === 'https://services.arcgisonline.com' &&
/^\/ArcGIS\/rest\/services\/World_Imagery\/MapServer\/tile\/\d+\/\d+\/\d+$/.test(
target.pathname,
)) ||
(target.origin === 'https://tile.openstreetmap.org' &&
/^\/\d+\/\d+\/\d+\.png$/.test(target.pathname)));
if (offlineImagery && knownImageryTile) {
result.offlineImageryRequests += 1;
request.respond({
status: 200,
contentType: 'image/jpeg',
headers: { 'Access-Control-Allow-Origin': '*' },
body: offlineImageryTile,
});
return;
}
request.continue();
});
const client = await page.createCDPSession();
await client.send('Network.enable');
client.on('Network.requestWillBeSent', ({ request }) => {
const target = new URL(request.url);
const where = `${request.method} ${target.origin}${target.pathname}`;
if (
target.hostname.includes('overpass') ||
target.hostname === 'nominatim.openstreetmap.org'
)
result.forbiddenRequests.push({ step: marker, request: where });
// Lookups an outline could make: a forward geocode, a place search, an
// Overpass boundary query. The HUD's reverse geocode of the camera centre
// (latlng=) is not one of them.
const local = target.origin === new URL(url).origin;
const reverse = target.searchParams.has('latlng');
const lookup =
!reverse &&
((local &&
/^\/api\/(overpass|geocode|google\/text-search|nominatim|photon)/.test(
target.pathname,
)) ||
(target.hostname === 'maps.googleapis.com' &&
target.pathname.includes('/geocode/')) ||
target.hostname === 'photon.komoot.io');
if (lookup) result.lookups.push({ step: marker, request: where });
if (/\/(states_provinces|countries|counties)\.json$/.test(target.pathname))
result.packRequests.push({ step: marker, pack: target.pathname });
});
page.on('pageerror', (error) =>
result.pageErrors.push({ step: marker, message: error.message }),
);
page.on('console', (message) => {
if (message.type() === 'error')
result.consoleErrors.push({ step: marker, message: message.text() });
});
client.on('Page.screencastFrame', async ({ data, metadata, sessionId }) => {
client.send('Page.screencastFrameAck', { sessionId }).catch(() => {});
if (!recording) return;
const t = metadata.timestamp;
if (t - lastKept < FRAME_INTERVAL_S * 0.9) return;
lastKept = t;
const file = path.join(
framesDir,
`${String(frames.length).padStart(6, '0')}.jpg`,
);
frames.push({ file, t, step: marker });
await fs.writeFile(file, Buffer.from(data, 'base64'));
});
console.log('Loading viewer...');
const navigationUrl = new URL(url);
navigationUrl.searchParams.set('welcome', '0');
await page.goto(navigationUrl.href, {
waitUntil: 'domcontentloaded',
timeout: 60_000,
});
await page.waitForFunction(
() => window.__godsEyeView?.viewer && window.__godsEyeView?.annotations,
{ timeout: 90_000, polling: 500 },
);
await page.waitForFunction(
() =>
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 90_000, polling: 250 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.waitForFunction(
() => !document.querySelector('#first-run-launcher:not([hidden])'),
{ timeout: 10_000 },
);
result.renderer = await page.evaluate(() => {
const gl = window.__godsEyeView.viewer.scene.context._gl;
const debug = gl.getExtension('WEBGL_debug_renderer_info');
return debug
? gl.getParameter(debug.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER);
});
console.log('Renderer:', result.renderer);
// In-page helpers: camera moves, frame timing and annotation state.
await page.evaluate(() => {
const { viewer } = window.__godsEyeView;
const Cesium = window.__CESIUM__;
window.__qaOutlines = {
fly(view, duration) {
return new Promise((resolve) => {
viewer.camera.cancelFlight();
viewer.camera.flyTo({
destination: Cesium.Cartesian3.fromDegrees(
view.lon,
view.lat,
view.height,
),
orientation: {
heading: Cesium.Math.toRadians(view.heading ?? 0),
pitch: Cesium.Math.toRadians(view.pitch ?? -70),
roll: 0,
},
duration,
easingFunction: Cesium.EasingFunction.QUADRATIC_IN_OUT,
complete: resolve,
cancel: resolve,
});
});
},
frameTimes(ms) {
return new Promise((resolve) => {
const deltas = [];
let last = performance.now();
const end = last + ms;
const step = (now) => {
deltas.push(now - last);
last = now;
if (now < end) requestAnimationFrame(step);
else resolve(deltas.slice(1));
};
requestAnimationFrame(step);
});
},
annotationSource() {
for (let i = 0; i < viewer.dataSources.length; i++) {
const source = viewer.dataSources.get(i);
if (source.name === 'gev-annotations') return source;
}
return null;
},
groundPrimitives() {
const list = [];
const walk = (collection) => {
for (let i = 0; i < collection.length; i++) {
const item = collection.get(i);
if (item?.length !== undefined && typeof item.get === 'function')
walk(item);
else list.push(item);
}
};
walk(viewer.scene.groundPrimitives);
return list;
},
snapshot() {
const source = this.annotationSource();
const entities = source ? source.entities.values : [];
const primitives = this.groundPrimitives();
window.__qaPrimitiveIds ??= new WeakMap();
let next = window.__qaPrimitiveNext ?? 1;
const ids = primitives.map((primitive) => {
if (!window.__qaPrimitiveIds.has(primitive))
window.__qaPrimitiveIds.set(primitive, next++);
return window.__qaPrimitiveIds.get(primitive);
});
window.__qaPrimitiveNext = next;
return {
entities: entities.length,
polygons: entities.filter((e) => e.polygon).length,
polylines: entities.filter((e) => e.polyline).length,
draped:
entities.every(
(e) =>
!e.polygon ||
e.polygon.classificationType?.getValue() ===
Cesium.ClassificationType.BOTH,
) &&
entities.every(
(e) => !e.polyline || e.polyline.clampToGround?.getValue(),
),
primitiveIds: ids,
ready: viewer.dataSourceDisplay.ready,
};
},
};
});
const settle = () =>
page
.waitForFunction(
() => {
const { scene } = window.__godsEyeView.viewer;
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (p?.show && typeof p.tilesLoaded === 'boolean' && !p.tilesLoaded)
return false;
}
return true;
},
{ timeout: 60_000, polling: 500 },
)
.catch(() => {});
async function shot(name) {
await page.evaluate(
() =>
new Promise((resolve) =>
requestAnimationFrame(() => requestAnimationFrame(resolve)),
),
);
const filename = `admin-outlines-${name}.png`;
await page.screenshot({ path: path.join(shots, filename) });
result.screenshots.push(filename);
}
async function move(name, view, duration = 2.5) {
marker = `move:${name}`;
const [, times] = await Promise.all([
page.evaluate((v, d) => window.__qaOutlines.fly(v, d), view, duration),
page.evaluate((ms) => window.__qaOutlines.frameTimes(ms), duration * 900),
]);
result.steps.push({ step: `move:${name}`, frameTimes: stats(times) });
}
/** Annotate like the voice tool does, then watch the mark while still. */
async function outline(name, target, { flyTo = false } = {}) {
marker = `outline:${name}`;
const lookupsBefore = result.lookups.length;
const errorsBefore = result.consoleErrors.length + result.pageErrors.length;
const timing = await page.evaluate(
async (target, flyTo) => {
const started = performance.now();
const response = await window.__godsEyeView.annotations.annotate(
[{ type: 'area', target, label: target, footprint: true }],
{ flyTo },
);
const resolvedMs = performance.now() - started;
// "Drawn" = the first rendered frame after the entity visualizers have
// built the mark's primitives (each tick updates them before render).
const { viewer } = window.__godsEyeView;
await new Promise((resolve) => {
const off = viewer.scene.postRender.addEventListener(() => {
if (!viewer.dataSourceDisplay.ready) return;
off();
resolve();
});
});
const drawnMs = performance.now() - started;
const mark = window.__godsEyeView.annotations
.list()
.find((m) => m.id === response.ids?.[0]);
return {
response: response.results?.[0] ?? null,
resolvedMs,
drawnMs,
parts: mark?.polygons?.length ?? (mark?.ring ? 1 : 0),
holes: (mark?.polygons ?? []).reduce(
(n, part) => n + part.length - 1,
0,
),
ringVertices: mark?.ring?.length ?? 0,
pending: Boolean(mark?.pendingOutline),
};
},
target,
flyTo,
);
if (flyTo) await new Promise((r) => setTimeout(r, 2200));
await settle();
// Hold still: the drawn mark must not rebuild or grow.
marker = `still:${name}`;
const before = await page.evaluate(() => window.__qaOutlines.snapshot());
const stillStart = frames.length;
const times = await page.evaluate(
(ms) => window.__qaOutlines.frameTimes(ms),
STILL_MS,
);
const after = await page.evaluate(() => window.__qaOutlines.snapshot());
const step = {
step: `outline:${name}`,
target,
...timing,
entities: after.entities,
polygons: after.polygons,
polylines: after.polylines,
draped: after.draped,
stableWhileStill:
before.entities === after.entities &&
JSON.stringify(before.primitiveIds) ===
JSON.stringify(after.primitiveIds),
stillFrameTimes: stats(times),
stillFrames: [stillStart, frames.length],
lookups: result.lookups.slice(lookupsBefore),
errors:
result.consoleErrors.length + result.pageErrors.length - errorsBefore,
};
result.steps.push(step);
console.log(
`${name}: resolved ${timing.resolvedMs.toFixed(0)} ms, drawn ${timing.drawnMs.toFixed(0)} ms, ` +
`${timing.parts} part(s), ${timing.holes} hole(s), via ${timing.response?.resolvedVia}; ` +
`still p50/p95 ${step.stillFrameTimes.p50}/${step.stillFrameTimes.p95} ms; ` +
`stable=${step.stableWhileStill}`,
);
assert.equal(timing.response?.ok, true, `${name}: annotation drawn`);
assert.equal(timing.response.outline, true, `${name}: outline, not a pin`);
assert.equal(timing.response.resolvedVia, 'bundled', `${name}: bundled`);
assert.equal(timing.pending, false, `${name}: nothing pending`);
assert.ok(
timing.drawnMs < OUTLINE_BUDGET_MS,
`${name}: drawn in ${timing.drawnMs} ms`,
);
assert.equal(step.draped, true, `${name}: draped on terrain/tiles`);
assert.equal(step.stableWhileStill, true, `${name}: no redraw when still`);
assert.deepEqual(step.lookups, [], `${name}: no network lookups`);
assert.equal(step.errors, 0, `${name}: no console errors`);
return step;
}
// Start from a clean board and the default layers, as a user would.
await page.evaluate(() => window.__godsEyeView.annotations.clear?.());
await page.evaluate(() =>
window.__qaOutlines.fly(
{ lat: 25.5, lon: -99.6, height: 2_600_000, pitch: -80 },
0,
),
);
await settle();
if (recordVideo) {
await client.send('Page.startScreencast', {
format: 'jpeg',
quality: 72,
maxWidth: 1280,
maxHeight: 800,
everyNthFrame: 1,
});
recording = true;
}
await outline('texas', 'Texas');
await shot('texas');
await move('pan-west', {
lat: 25.2,
lon: -101.5,
height: 2_300_000,
pitch: -75,
});
await move('zoom-central-texas', {
lat: 28.8,
lon: -98.2,
height: 700_000,
pitch: -65,
});
await move('zoom-austin', {
lat: 29.75,
lon: -97.8,
height: 120_000,
pitch: -55,
});
await settle();
await outline('travis', 'Travis County, Texas');
await shot('travis');
await move('pan-williamson', {
lat: 30.05,
lon: -97.65,
height: 110_000,
pitch: -55,
});
await settle();
await outline('williamson', 'Williamson County, Texas');
// Close over the shared Travis/Williamson line near Pflugerville: the
// outline must lie on the photoreal mesh, not float above or sink into it.
const border = await page.evaluate(() => {
const travis = window.__godsEyeView.annotations
.list()
.find((m) => m.label === 'Travis County, Texas');
let best = null;
for (const [lon, lat] of travis.ring) {
const d = (lat - 30.46) ** 2 + (lon + 97.62) ** 2;
if (!best || d < best.d) best = { lon, lat, d };
}
return best;
});
await move(
'drape-close',
{ lat: border.lat - 0.035, lon: border.lon, height: 2500, pitch: -40 },
3,
);
await settle();
await page.evaluate((ms) => new Promise((r) => setTimeout(r, ms)), 2500);
await shot('drape-close');
await move('zoom-out-both', {
lat: 29.55,
lon: -97.7,
height: 260_000,
pitch: -60,
});
await settle();
await page.evaluate((ms) => new Promise((r) => setTimeout(r, ms)), 1500);
await shot('both-counties');
await outline('bavaria', 'Bavaria', { flyTo: true });
await shot('bavaria');
await move(
'orbit-bavaria',
{ lat: 46.6, lon: 11.9, height: 650_000, heading: 20, pitch: -55 },
3,
);
await settle();
await shot('bavaria-orbit');
// A hole (Berlin inside Brandenburg) and islands (Hawaii) draw as such.
const brandenburg = await outline('brandenburg', 'Brandenburg', {
flyTo: true,
});
assert.ok(brandenburg.holes >= 1, 'Brandenburg keeps the Berlin hole');
await shot('brandenburg');
const hawaii = await outline('hawaii', 'Hawaii', { flyTo: true });
assert.ok(hawaii.parts >= 7, 'Hawaii keeps its islands');
await shot('hawaii');
marker = 'end';
for (const [name, target, lat, lon, height] of [
['scotland', 'Scotland', 57, -4, 800000],
['england', 'England', 52.5, -1.5, 800000],
['wales', 'Wales', 52.3, -3.7, 420000],
['northern-ireland', 'Northern Ireland', 54.7, -6.7, 300000],
['switzerland', 'Switzerland', 46.8, 8.2, 500000],
['iran', 'Iran', 32, 54, 2200000],
['france', 'France', 46.6, 2.2, 1600000],
['japan', 'Japan', 37, 138, 2200000],
['georgia-state', 'Georgia', 32.6, -83.4, 650000],
['georgia-country', 'Georgia', 42, 43.6, 700000],
['georgia-qualified', 'the country of Georgia', 42, 43.6, 700000],
]) {
await page.evaluate(() => window.__godsEyeView.annotations.clear());
await move(name, { lat, lon, height, pitch: -90 }, 1);
await outline(name, target);
await shot(name);
const rendered = await page.evaluate(() => window.__qaOutlines.snapshot());
assert.ok(rendered.polygons > 0, `${target} polygon renders`);
}
if (recordVideo) {
recording = false;
await client.send('Page.stopScreencast');
}
result.totals = {
forbiddenRequests: result.forbiddenRequests.length,
consoleErrors: result.consoleErrors.length,
pageErrors: result.pageErrors.length,
};
assert.deepEqual(
result.forbiddenRequests,
[],
'no Overpass or nominatim.openstreetmap.org requests',
);
// The packs load on the first outline that needs them, never at start-up.
assert.deepEqual(
result.packRequests.map((r) => [r.step, path.basename(r.pack)]),
[
['outline:texas', 'states_provinces.json'],
['outline:travis', 'counties.json'],
['outline:scotland', 'countries.json'],
],
'packs load lazily, once each',
);
assert.deepEqual(result.pageErrors, [], 'no page errors');
assert.deepEqual(result.consoleErrors, [], 'no console errors');
} catch (error) {
result.failure = error.message;
console.error(error);
process.exitCode = 1;
if (page)
await page
.screenshot({ path: path.join(shots, 'admin-outlines-failure.png') })
.catch(() => {});
} finally {
await browser.close();
}
if (recordVideo && frames.length > 1) {
// Frame-to-frame change while the camera is still: steady pulses read as
// small, even values; a flicker (a part vanishing and returning) as a spike.
for (const step of result.steps.filter((s) => s.stillFrames)) {
const [start, end] = step.stillFrames;
const diffs = [];
let previous = null;
for (const frame of frames.slice(start, end)) {
const pixels = await sharp(frame.file)
.resize(160, 100, { fit: 'fill' })
.greyscale()
.raw()
.toBuffer();
if (previous) {
let sum = 0;
let max = 0;
for (let i = 0; i < pixels.length; i++) {
const d = Math.abs(pixels[i] - previous[i]);
sum += d;
if (d > max) max = d;
}
diffs.push(sum / pixels.length);
}
previous = pixels;
}
step.stillFrameDiff = {
frames: end - start,
meanAbsDiff: stats(diffs, 3),
};
}
const list = [];
for (let i = 0; i < frames.length; i++) {
const duration =
i + 1 < frames.length ? frames[i + 1].t - frames[i].t : FRAME_INTERVAL_S;
list.push(`file '${frames[i].file}'`, `duration ${duration.toFixed(4)}`);
}
list.push(`file '${frames.at(-1).file}'`);
const listFile = path.join(framesDir, 'frames.txt');
await fs.writeFile(listFile, list.join('\n') + '\n');
const video = path.join(shots, 'admin-outlines-journey.mp4');
execFileSync('ffmpeg', [
'-y',
'-loglevel',
'error',
'-f',
'concat',
'-safe',
'0',
'-i',
listFile,
'-vf',
'fps=24,format=yuv420p',
'-c:v',
'libx264',
'-crf',
'23',
video,
]);
const span = frames.at(-1).t - frames[0].t;
result.video = {
file: path.relative(process.cwd(), video),
frames: frames.length,
seconds: Number(span.toFixed(1)),
capturedFps: Number(((frames.length - 1) / span).toFixed(1)),
};
console.log('Video:', result.video);
await fs.rm(framesDir, { recursive: true, force: true });
}
await fs.writeFile(
path.join(shots, 'admin-outlines-result.json'),
JSON.stringify(result, null, 2),
);
console.table(
result.steps.map((s) => ({
step: s.step,
resolvedMs: s.resolvedMs?.toFixed(0),
drawnMs: s.drawnMs?.toFixed(0),
parts: s.parts,
stable: s.stableWhileStill,
p50: (s.stillFrameTimes || s.frameTimes)?.p50,
p95: (s.stillFrameTimes || s.frameTimes)?.p95,
stillDiffP95: s.stillFrameDiff?.meanAbsDiff.p95,
stillDiffMax: s.stillFrameDiff?.meanAbsDiff.max,
})),
);
console.log(process.exitCode ? 'FAILED' : 'PASSED');
function stats(values, digits = 1) {
if (!values.length) return { n: 0 };
const sorted = [...values].sort((a, b) => a - b);
const at = (q) =>
Number(
sorted[
Math.min(sorted.length - 1, Math.floor(q * sorted.length))
].toFixed(digits),
);
return { n: values.length, p50: at(0.5), p95: at(0.95), max: at(1) };
}
+588
View File
@@ -0,0 +1,588 @@
#!/usr/bin/env node
/**
* qa-alpr-journey.mjs — ALPR cameras on a realistic Austin camera journey.
*
* Run: node scripts/qa-alpr-journey.mjs http://localhost:4173 [--headful]
* [--passes off,on,on] [--out qa-shots/alpr-journey] [--no-assert]
*
* Drives the same eased camera path several times: a neighbourhood by Camp
* Mabry (~2 km across), two 600 m pans, a small zoom out and back, downtown
* (~6 km), the whole city (~30 km) and back to the neighbourhood. The first
* pass runs with Mapped ALPR Cameras off (it also warms the photoreal tiles),
* then ALPR is switched on mid-journey from its layer row and the path repeats
* cold and warm.
*
* Per stop it records: time from the camera stopping to cameras shown (or an
* explicit "zoom in" state), the camera count and row status, rAF interval and
* Cesium render CPU time (mean / p50 / p95) while moving and while holding, and
* marker stability while the camera is still (camera-set changes after the stop
* has settled). Asserts, unless --no-assert: neighbourhood and downtown show
* cameras within 2 s on the warm pass; the whole-city stop shows cameras or a
* zoom-in prompt and never ends a hold still loading; at most one visible
* marker update per stop and none after 3 s while still; warm-pass frame p50/p95 within 10% of the ALPR-off pass (plus a
* 2 ms floor for timer noise); zero console errors.
*
* Rendered-state checks read the recorded frames, not entity state: ALPR-cyan
* pixels per frame during each hold, minus the ALPR-off pass at the same stop,
* give when markers actually reached the screen. Cameras projected on screen
* must be painted, no later than 700 ms after the layer state settled (a
* missing render request fails here), within 2 s of the stop on the warm pass;
* a view whose loaded cameras are all off screen must say "None on screen".
* At the end of each hold the screen is compared with the same scene after
* forcing four frames; any marker difference is a missing render request.
* --self-test-drop-renders disables explicit render requests during the ALPR
* passes to prove those checks fail.
*
* Writes <out>/result.json, <out>/clip.mp4 (CDP screencast with real frame
* timing) and per-stop screenshots. Needs ffmpeg on PATH.
*/
import fs from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
import sharp from 'sharp';
import {
createScreencast,
frameStats,
installFrameProbe,
moveCamera,
pageNow,
readFrames,
} from './qa-journey-recorder.mjs';
const argv = process.argv.slice(2);
if (argv.includes('--help') || argv.includes('-h')) {
console.log(
'Usage: node scripts/qa-alpr-journey.mjs <dev-server-url> [--headful] [--passes off,on,on] [--out dir] [--no-assert] [--self-test-drop-renders]',
);
process.exit(0);
}
const option = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] ? argv[i + 1] : fallback;
};
const appUrl =
argv.find((arg) => /^https?:/.test(arg)) || 'http://localhost:4173';
const outDir = path.resolve(option('--out', 'qa-shots/alpr-journey'));
const passes = option('--passes', 'off,on,on').split(',');
const assertOn = !argv.includes('--no-assert');
const dropRenders = argv.includes('--self-test-drop-renders');
fs.mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const cyanCount = async (file) => {
const { data, info } = await sharp(file)
.resize({ width: 640 })
.raw()
.toBuffer({ resolveWithObject: true });
let n = 0;
for (let i = 0; i < data.length; i += info.channels) {
const r = data[i];
const g = data[i + 1];
const b = data[i + 2];
if (r < 140 && g > 160 && b > 205 && b - r > 80) n += 1;
}
return n;
};
/** Cyan pixels (at 640 px width) that mean at least one badge is painted. */
const MIN_MARKER_PIXELS = 12;
// Camp Mabry neighbourhood (Tarrytown / Bryker Woods), downtown, whole city.
const N = {
lat: 30.2985,
lon: -97.7655,
height: 1500,
heading: 20,
pitch: -55,
};
const east = (view, m) => ({
...view,
lon: view.lon + m / (111_320 * Math.cos((view.lat * Math.PI) / 180)),
});
const north = (view, m) => ({ ...view, lat: view.lat + m / 111_320 });
const STOPS = [
{ name: 'neighbourhood', view: N, move: 0, hold: 6, scale: 'neighbourhood' },
{
name: 'pan-east-600m',
view: east(N, 600),
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'pan-north-600m',
view: north(east(N, 600), 600),
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'zoom-out-3km',
view: { ...north(east(N, 600), 600), height: 3000 },
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'zoom-back-in',
view: north(east(N, 600), 600),
move: 2.5,
hold: 5,
scale: 'neighbourhood',
},
{
name: 'downtown-6km',
view: { lat: 30.262, lon: -97.7431, height: 4500, heading: 0, pitch: -60 },
move: 3,
hold: 7,
scale: 'downtown',
},
{
name: 'city-30km',
view: { lat: 30.2, lon: -97.75, height: 24000, heading: 0, pitch: -65 },
move: 3,
hold: 8,
scale: 'city',
},
{
name: 'back-to-neighbourhood',
view: N,
move: 3,
hold: 6,
scale: 'neighbourhood',
},
];
const browser = await puppeteer.launch({
headless: !argv.includes('--headful'),
protocolTimeout: 300_000,
args: [
'--no-sandbox',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--disable-backgrounding-occluded-windows',
],
});
const result = {
url: appUrl,
startedAt: new Date().toISOString(),
passes: [],
consoleErrors: [],
failures: [],
};
const fail = (message) => {
result.failures.push(message);
console.log(`[FAIL] ${message}`);
};
let page;
try {
page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
page.on('console', (message) => {
if (message.type() === 'error') result.consoleErrors.push(message.text());
});
page.on('pageerror', (error) => result.consoleErrors.push(error.message));
const url = new URL(appUrl);
url.searchParams.set('welcome', '0');
await page.goto(url.href, { waitUntil: 'domcontentloaded', timeout: 90_000 });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120_000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await installFrameProbe(page);
// ALPR watcher: row state every 100 ms and the rendered camera set (ids and
// visibility) as a cheap hash, so structural churn while still is visible.
await page.evaluate(() => {
const samples = [];
window.__gevAlprSamples = samples;
setInterval(() => {
const entry = window.__godsEyeView.dataManager.layers.get('alpr-cameras');
const stats = entry?.module?.getStats?.() || {};
// Layers that publish a render revision are measured by it; otherwise
// hash the rendered entity ids and their visibility.
let hash = 0;
let shown = 0;
if (Number.isFinite(stats.renderRevision)) {
hash = stats.renderRevision;
shown = stats.shown ?? stats.count ?? 0;
} else {
const entities =
entry?.module && entry.enabled
? window.__godsEyeView.viewer.dataSources.getByName(
'alpr-cameras',
)[0]?.entities.values || []
: [];
for (const entity of entities) {
if (entity.show === false) continue;
shown += 1;
const id = String(entity.id);
for (let i = 0; i < id.length; i++)
hash = (hash * 31 + id.charCodeAt(i)) | 0;
}
}
samples.push({
t: performance.now(),
enabled: Boolean(entry?.enabled),
count: stats.count ?? 0,
shown,
hash,
loading: Boolean(stats.loading),
status: stats.status || null,
label: stats.loadingLabel || '',
onScreen: stats.onScreen ?? null,
});
if (samples.length > 20_000) samples.splice(0, 5_000);
}, 100);
});
const recorder = await createScreencast(page, outDir);
await recorder.start();
async function waitTiles(maxMs = 20_000) {
await page
.waitForFunction(
() => {
const { scene } = window.__godsEyeView.viewer;
if (scene.globe.show) return scene.globe.tilesLoaded;
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (p.show && p.tilesLoaded === false) return false;
}
return true;
},
{ timeout: maxMs, polling: 250 },
)
.catch(() => {});
}
for (const [passIndex, mode] of passes.entries()) {
const alpr = mode === 'on';
const passName = `${passIndex + 1}-${mode}${alpr && passes.slice(0, passIndex).includes('on') ? '-warm' : alpr ? '-cold' : ''}`;
console.log(`Pass ${passName}`);
await moveCamera(page, STOPS[0].view, 0);
await waitTiles();
if (
alpr !==
(await page.evaluate(() =>
window.__godsEyeView.dataManager.isEnabled('alpr-cameras'),
))
) {
// The user's switch: the layer row, not a private API.
await page.evaluate(
(enabled) =>
window.__godsEyeView.dataManager.setEnabled('alpr-cameras', enabled, {
origin: 'user',
}),
alpr,
);
}
if (alpr && dropRenders)
// Self-test: drop explicit render requests so markers reach the screen
// only with the next camera move. The rendered-pixel checks must fail.
await page.evaluate(() => {
window.__godsEyeView.viewer.scene.requestRender = () => {};
});
const pass = { name: passName, alpr, stops: [] };
for (const stop of STOPS) {
const moveStart = await pageNow(page);
if (stop.move) await moveCamera(page, stop.view, stop.move);
else await moveCamera(page, stop.view, 0);
const holdStart = await pageNow(page);
const holdStartEpoch = await page.evaluate(
() => performance.timeOrigin + performance.now(),
);
await sleep(stop.hold * 1000);
const holdEnd = await pageNow(page);
// Render truth: what the screen shows at the end of the hold versus the
// same scene after forcing a few frames. A difference means the layer
// changed state without asking for a render.
let staleFrame = null;
if (alpr) {
const shown = await cyanCount(await page.screenshot({ type: 'png' }));
await page.evaluate(async () => {
const { scene } = window.__godsEyeView.viewer;
const mode = scene.requestRenderMode;
scene.requestRenderMode = false;
for (let i = 0; i < 4; i++)
await new Promise((resolve) => requestAnimationFrame(resolve));
scene.requestRenderMode = mode;
});
const forced = await cyanCount(await page.screenshot({ type: 'png' }));
staleFrame = {
shown,
forced,
stale: Math.abs(forced - shown) > Math.max(8, forced * 0.15),
};
}
const moving = frameStats(await readFrames(page, moveStart, holdStart));
const holding = frameStats(await readFrames(page, holdStart, holdEnd));
const samples = await page.evaluate(
(a, b) => window.__gevAlprSamples.filter((s) => s.t >= a && s.t < b),
holdStart,
holdEnd,
);
// Time to cameras: the last visible change (or the end of loading) after
// the camera stopped. More than one visible change, or any change after
// 3 s, is flicker: a still camera must see one clean update at most.
let visualChanges = 0;
const changeLog = [];
let lastChangeAt = null;
let loadingEndAt = null;
let wasLoading = false;
let previous = null;
for (const s of samples) {
if (
previous &&
(s.hash !== previous.hash || s.shown !== previous.shown)
) {
visualChanges += 1;
lastChangeAt = s.t;
changeLog.push({
atMs: Math.round(s.t - holdStart),
shown: s.shown,
loading: s.loading,
});
}
if (s.loading) wasLoading = true;
else if (wasLoading && loadingEndAt === null) loadingEndAt = s.t;
previous = s;
}
const first = samples[0] || {};
const settledAt = Math.max(
lastChangeAt ?? holdStart,
loadingEndAt ?? (first.loading ? holdEnd : holdStart),
);
const end = samples.at(-1) || {};
const row = {
stop: stop.name,
scale: stop.scale,
holdEpochMs: [holdStartEpoch, holdStartEpoch + (holdEnd - holdStart)],
onScreen: end.onScreen ?? null,
staleFrame,
timeToCamerasMs: alpr ? Math.round(settledAt - holdStart) : null,
cameras: end.count ?? 0,
shown: end.shown ?? 0,
status: end.status,
label: end.label,
loadingAtHoldEnd: Boolean(end.loading),
visualChanges,
changeLog,
lateChanges: samples.filter(
(sample, i) =>
i > 0 &&
sample.t > holdStart + 3000 &&
(sample.hash !== samples[i - 1].hash ||
sample.shown !== samples[i - 1].shown),
).length,
moving,
holding,
};
pass.stops.push(row);
console.log(
` ${stop.name.padEnd(22)} cams ${String(row.cameras).padStart(5)} t ${String(row.timeToCamerasMs ?? '-').padStart(5)} ms frame p50/p95 ${holding.frameP50}/${holding.frameP95} render p50/p95 ${holding.renderP50}/${holding.renderP95} changes ${visualChanges}/${row.lateChanges} ${row.status || ''} ${row.label || ''}`,
);
if (passIndex === passes.length - 1) {
await page.screenshot({
path: path.join(outDir, `${passName}-${stop.name}.png`),
});
}
}
result.passes.push(pass);
}
const clip = await recorder.stop();
result.clip = clip;
// Clip-relative hold windows, so a reviewer can jump to each stop.
const clipStart = recorder.frames[0]?.timestamp * 1000;
for (const pass of result.passes)
for (const stop of pass.stops)
stop.clipSeconds = stop.holdEpochMs.map(
(ms) => Math.round((ms - clipStart) / 100) / 10,
);
console.log('clip', clip);
// Rendered-state check from the recorded frames themselves: count ALPR-cyan
// pixels per frame during each hold, subtract the ALPR-off pass at the same
// stop, and find when the markers actually became visible on screen. Entity
// state can be ready while nothing is painted (a missing render request),
// which this catches and the in-page samples cannot.
const framesIn = ([a, b]) =>
recorder.frames.filter(
(frame) => frame.timestamp * 1000 >= a && frame.timestamp * 1000 < b,
);
const median = (list) =>
list.length ? [...list].sort((x, y) => x - y)[list.length >> 1] : 0;
const offPass = result.passes.find((p) => !p.alpr);
for (const pass of result.passes) {
for (const [index, stop] of pass.stops.entries()) {
const frames = framesIn(stop.holdEpochMs);
const counts = [];
for (const frame of frames)
counts.push({
atMs: Math.round(frame.timestamp * 1000 - stop.holdEpochMs[0]),
cyan: await cyanCount(frame.file),
});
stop.pixelTimeline = counts;
if (!pass.alpr) {
stop.cyanBaseline = median(counts.map((c) => c.cyan));
continue;
}
const baseline = offPass?.stops[index]?.cyanBaseline ?? 0;
const tail = counts.filter(
(c) => c.atMs >= stop.holdEpochMs[1] - stop.holdEpochMs[0] - 1000,
);
const final = median(tail.map((c) => c.cyan)) - baseline;
stop.renderedMarkerPixels = final;
let visibleAt = null;
if (final >= MIN_MARKER_PIXELS) {
for (let i = counts.length - 1; i >= 0; i--) {
if (counts[i].cyan - baseline >= final * 0.8)
visibleAt = counts[i].atMs;
else break;
}
}
stop.renderedVisibleAtMs = visibleAt;
console.log(
` ${pass.name} ${stop.stop.padEnd(22)} on screen ${stop.onScreen ?? '-'} marker px ${final} visible at ${visibleAt ?? '-'} ms (state ${stop.timeToCamerasMs} ms) end-of-hold px ${stop.staleFrame?.shown} / forced ${stop.staleFrame?.forced}`,
);
}
}
// Aggregate and assert.
const aggregate = (pass, phase) => {
const frames = pass.stops.map((s) => s[phase]);
const weighted = (key) => {
const total = frames.reduce((n, f) => n + (f.frames || 0), 0);
return (
Math.round(
(frames.reduce((n, f) => n + (f[key] || 0) * (f.frames || 0), 0) /
Math.max(1, total)) *
10,
) / 10
);
};
return {
frameP50: weighted('frameP50'),
frameP95: weighted('frameP95'),
renderP50: weighted('renderP50'),
renderP95: weighted('renderP95'),
longFrames: frames.reduce((n, f) => n + f.longFrames, 0),
};
};
result.summary = result.passes.map((pass) => ({
pass: pass.name,
moving: aggregate(pass, 'moving'),
holding: aggregate(pass, 'holding'),
}));
console.table(
result.summary.map((s) => ({
pass: s.pass,
'move p50/p95': `${s.moving.frameP50}/${s.moving.frameP95}`,
'hold p50/p95': `${s.holding.frameP50}/${s.holding.frameP95}`,
'render hold p50/p95': `${s.holding.renderP50}/${s.holding.renderP95}`,
'long frames': s.moving.longFrames + s.holding.longFrames,
})),
);
if (assertOn) {
const off = result.summary.find((s) => s.pass.includes('off'));
const warm = result.summary.find((s) => s.pass.endsWith('warm'));
const warmPass = result.passes.find((p) => p.name.endsWith('warm'));
const onPasses = result.passes.filter((p) => p.alpr);
if (off && warm) {
for (const phase of ['moving', 'holding'])
for (const key of ['frameP50', 'frameP95'])
if (warm[phase][key] > off[phase][key] * 1.1 + 2)
fail(
`${phase} ${key} ${warm[phase][key]} ms with ALPR vs ${off[phase][key]} ms without`,
);
}
for (const stop of warmPass?.stops || []) {
if (
stop.scale !== 'city' &&
!(
stop.timeToCamerasMs !== null &&
stop.timeToCamerasMs <= 2000 &&
stop.cameras > 0
)
)
fail(
`${stop.stop}: cameras after ${stop.timeToCamerasMs} ms (${stop.cameras}) on the warm pass`,
);
}
for (const pass of onPasses)
for (const stop of pass.stops) {
if (stop.staleFrame?.stale)
fail(
`${pass.name} ${stop.stop}: screen showed ${stop.staleFrame.shown} marker px, ${stop.staleFrame.forced} after forcing a render`,
);
// Cameras on screen must be painted, and painted when state says so.
if (stop.onScreen > 0 && stop.renderedMarkerPixels < MIN_MARKER_PIXELS)
fail(
`${pass.name} ${stop.stop}: ${stop.onScreen} cameras on screen but no markers rendered`,
);
if (
stop.renderedVisibleAtMs !== null &&
stop.renderedVisibleAtMs > stop.timeToCamerasMs + 700
)
fail(
`${pass.name} ${stop.stop}: markers rendered ${stop.renderedVisibleAtMs} ms after the stop, ${stop.timeToCamerasMs} ms after state`,
);
if (
pass.name.endsWith('warm') &&
stop.scale !== 'city' &&
stop.renderedVisibleAtMs !== null &&
stop.renderedVisibleAtMs > 2000
)
fail(
`${pass.name} ${stop.stop}: markers visible only after ${stop.renderedVisibleAtMs} ms`,
);
if (
stop.cameras > 0 &&
stop.onScreen === 0 &&
!/none on screen/i.test(stop.label)
)
fail(
`${pass.name} ${stop.stop}: no camera on screen and no explicit state`,
);
if (stop.loadingAtHoldEnd)
fail(`${pass.name} ${stop.stop}: still loading when the hold ended`);
if (
stop.scale === 'city' &&
!(stop.cameras > 0 || /zoom in/i.test(stop.label))
)
fail(
`${pass.name} ${stop.stop}: neither cameras nor a zoom-in prompt`,
);
if (stop.visualChanges > 1 || stop.lateChanges > 0)
fail(
`${pass.name} ${stop.stop}: ${stop.visualChanges} marker changes (${stop.lateChanges} after 3 s) while the camera was still`,
);
}
const errors = result.consoleErrors.filter(
(text) => !/favicon/i.test(text),
);
if (errors.length)
fail(`console errors: ${errors.slice(0, 3).join(' | ')}`);
}
} catch (error) {
fail(error.stack || error.message);
} finally {
fs.writeFileSync(
path.join(outDir, 'result.json'),
JSON.stringify(result, null, 1),
);
await browser.close();
}
console.log(
`${result.failures.length ? 'FAIL' : 'PASS'} — ${path.join(outDir, 'result.json')}`,
);
process.exit(result.failures.length ? 1 : 0);
+92
View File
@@ -1943,6 +1943,98 @@ try {
}),
);
// Cyber relocates its collapsed briefing header into the expanded utility
// lane. Exercise actual height changes without reload and measure both cards.
await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
manager._setHudVariant('cyber');
manager.hud.setMode('on');
manager._setCockpitDisclosure('display', false);
manager._setCockpitDisclosure('radio', true);
});
const cyberRadioOrder = await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
manager.cockpitView.syncSignalLayout();
const display = document
.getElementById('cockpit-display-toggle-btn')
.closest('.cockpit-utility-control');
const radio = document
.getElementById('cockpit-radio-toggle-btn')
.closest('.cockpit-utility-control');
const displayRect = display.getBoundingClientRect();
const radioRect = radio.getBoundingClientRect();
return {
display: displayRect.toJSON(),
radio: radioRect.toJSON(),
displayExpanded: display.classList.contains('is-expanded'),
radioExpanded: radio.classList.contains('is-expanded'),
gap: radioRect.top - displayRect.bottom,
};
});
check(
'Cyber expanding Radio preserves the authored Display-then-Radio order',
!cyberRadioOrder.displayExpanded
&& cyberRadioOrder.radioExpanded
&& cyberRadioOrder.display.height > 0
&& cyberRadioOrder.radio.height > 0
&& cyberRadioOrder.gap >= 7,
JSON.stringify(cyberRadioOrder),
);
await page.screenshot({ path: path.join(shotsDir, 'cyber-radio-order.png') });
await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
manager._setCockpitDisclosure('radio', false);
manager._setCockpitDisclosure('display', true);
});
const cyberStyleStates = await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
const states = [];
for (const style of ['normal', 'surveillance', 'thermal', 'retro']) {
document.querySelector(`.style-btn[data-style="${style}"]`).click();
states.push({
requestedStyle: style,
activeStyle: manager.activeStyle,
hud: manager.hud.getVariant(),
theme: document.documentElement.dataset.uiTheme,
selector: document.getElementById('hud-layout-select').value,
savedHud: manager.getVisualState().hud.variant,
});
}
return states;
});
check('Cockpit visual-style controls retain Cyber HUD, selector, shell and saved layout',
cyberStyleStates.every(state => state.activeStyle === state.requestedStyle
&& state.hud === 'cyber' && state.theme === 'cyber'
&& state.selector === 'cyber' && state.savedHud === 'cyber'),
JSON.stringify(cyberStyleStates));
for (const height of [987, 720, 640, 900]) {
await page.setViewport({ width: 1440, height, deviceScaleFactor: 1 });
await new Promise((resolve) => setTimeout(resolve, 400));
const geometry = await page.evaluate(() => {
const signal = document.getElementById('cockpit-signal-stream');
const utility = document.getElementById('cockpit-utility-controls');
const expanded = utility.querySelector('.cockpit-utility-control.is-expanded');
const s = signal.getBoundingClientRect();
const u = expanded.getBoundingClientRect();
const toggle = document.getElementById('cockpit-signal-toggle');
const t = toggle.getBoundingClientRect();
const hit = document.elementFromPoint(t.x + t.width / 2, t.y + t.height / 2);
return { height: innerHeight, theme: document.documentElement.dataset.uiTheme,
collapsed: signal.dataset.collapsed, signal: s.toJSON(), utility: u.toJSON(),
reachable: Boolean(hit && toggle.contains(hit)),
contained: u.bottom <= s.top - 7 && s.bottom < innerHeight,
};
});
check(`Cyber collapsed Live Signals remains visible and reachable after resize to ${height}px`,
geometry.theme === 'cyber' && geometry.collapsed === 'true'
&& geometry.signal.width > 0 && geometry.signal.height > 0
&& geometry.reachable && geometry.contained, JSON.stringify(geometry));
if (height === 640 || height === 900) {
await page.screenshot({ path: path.join(shotsDir, `cyber-resize-${height}.png`) });
}
}
await page.evaluate(() => window.__godsEyeView.styleManager._setHudVariant('operator'));
const signalTransition = await page.evaluate(() => {
const manager = window.__godsEyeView.styleManager;
const cockpit = manager.cockpitView;
+590
View File
@@ -0,0 +1,590 @@
#!/usr/bin/env node
/**
* qa-context-arrivals.mjs — search arrivals above the surface, and Contacts
* listing mapped installations around a moving tracked aircraft.
*
* Run: node scripts/qa-context-arrivals.mjs http://localhost:4173 [--headful]
* [--out qa-shots/context-arrivals] [--only arrivals|contacts]
*
* Arrivals: types "Camp Mabry" and "39.7392, -104.9903" (downtown Denver)
* into the location search, waits for the flight and the photoreal tiles,
* orbits a little like a user looking around, and asserts the camera sits at
* least 60 m above the rendered surface under it and under the view centre.
*
* Contacts: serves two synthetic aircraft through the page's own
* /api/opensky response (one flying north over Camp Mabry at 2,500 m, one
* near Fort Cavazos) and an empty military feed, starts from a wide Texas
* view, and presses CONTACTS as a user would. Asserts the Mapped
* installations row lists at least one site with a distance within 15 s,
* never falls back to "?" while the aircraft moves (30 s), keeps its site
* entities (no rebuilds while the view is still), still lists sites in
* Cockpit, and that SEARCH NEARBY SITES reports what it found. Then re-enters
* Contacts on the Fort Cavazos aircraft and asserts a site within 30 km.
* Requires zero console errors.
*
* Writes <out>/result.json, screenshots and <out>/clip.mp4 (CDP screencast
* with real frame timing; needs ffmpeg).
*/
import fs from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
import { createScreencast, moveCamera } from './qa-journey-recorder.mjs';
const argv = process.argv.slice(2);
if (argv.includes('--help') || argv.includes('-h')) {
console.log(
'Usage: node scripts/qa-context-arrivals.mjs <dev-server-url> [--headful] [--out dir] [--only arrivals|contacts]',
);
process.exit(0);
}
const option = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] ? argv[i + 1] : fallback;
};
const appUrl =
argv.find((arg) => /^https?:/.test(arg)) || 'http://localhost:4173';
const outDir = path.resolve(option('--out', 'qa-shots/context-arrivals'));
const only = option('--only', null);
fs.mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const result = {
url: appUrl,
arrivals: [],
contacts: {},
consoleErrors: [],
failures: [],
};
const check = (name, ok, detail = '') => {
console.log(
`[${ok ? 'PASS' : 'FAIL'}] ${name}${detail ? ` — ${detail}` : ''}`,
);
if (!ok) result.failures.push(`${name}${detail ? ` — ${detail}` : ''}`);
};
const browser = await puppeteer.launch({
headless: !argv.includes('--headful'),
protocolTimeout: 300_000,
args: [
'--no-sandbox',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--disable-backgrounding-occluded-windows',
],
});
async function openPage({ aircraft = false } = {}) {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
page.on('console', (message) => {
if (message.type() === 'error') result.consoleErrors.push(message.text());
});
page.on('pageerror', (error) => result.consoleErrors.push(error.message));
page.on('response', (response) => {
if (response.status() < 400) return;
const url = new URL(response.url());
// Track/enrichment lookups for the synthetic ICAOs have no upstream record.
const synthetic = /aaa05\d/i.test(url.search + url.pathname);
(synthetic
? (result.expectedMisses ??= [])
: (result.httpErrors ??= [])
).push(`${response.status()} ${url.pathname}${url.search}`);
});
await page.evaluateOnNewDocument((aircraft) => {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
if (!aircraft) return;
// Synthetic traffic through the app's own feed endpoints: one aircraft
// flying north over Camp Mabry, one loitering near Fort Cavazos.
const started = Date.now();
const realFetch = window.fetch.bind(window);
window.fetch = (input, init) => {
const raw =
typeof input === 'string' || input instanceof URL
? String(input)
: input?.url;
let url;
try {
url = new URL(raw, location.href);
} catch {
return realFetch(input, init);
}
if (url.origin === location.origin && url.pathname === '/api/opensky') {
const now = Math.floor(Date.now() / 1000);
const seconds = (Date.now() - started) / 1000;
const lat = 30.29 + (seconds * 120) / 111_320;
const json = {
time: now,
states: [
[
'aaa051',
'QA051',
'Synthetic',
now,
now,
-97.765,
lat,
2500,
false,
120,
0,
0,
null,
2500,
null,
false,
0,
],
[
'aaa052',
'QA052',
'Synthetic',
now,
now,
-97.72,
31.12,
3000,
false,
90,
270,
0,
null,
3000,
null,
false,
0,
],
],
};
return Promise.resolve(Response.json(json));
}
if (url.origin === location.origin && url.pathname === '/api/adsblol/mil')
return Promise.resolve(
Response.json({ ac: [], now: Date.now(), total: 0 }),
);
return realFetch(input, init);
};
}, aircraft);
const url = new URL(appUrl);
url.searchParams.set('welcome', '0');
await page.goto(url.href, { waitUntil: 'domcontentloaded', timeout: 90_000 });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120_000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
return page;
}
async function waitTiles(page, maxMs = 30_000) {
await page
.waitForFunction(
() => {
const { scene } = window.__godsEyeView.viewer;
if (scene.globe.show) return scene.globe.tilesLoaded;
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (p.show && p.tilesLoaded === false) return false;
}
return true;
},
{ timeout: maxMs, polling: 250 },
)
.catch(() => {});
}
/** Camera height above the rendered surface under it and under the view centre. */
function clearance(page) {
return page.evaluate(() => {
const { viewer } = window.__godsEyeView;
const { scene, camera } = viewer;
const carto = camera.positionCartographic.clone();
const sample = (c) => {
try {
const h = scene.sampleHeight(c);
return Number.isFinite(h) ? h : null;
} catch {
return null;
}
};
const under = sample(carto);
let centre = null;
const canvas = scene.canvas;
const hit = scene.pickPosition?.({
x: canvas.clientWidth / 2,
y: canvas.clientHeight / 2,
});
if (hit) centre = scene.globe.ellipsoid.cartesianToCartographic(hit);
const centreSurface = centre ? sample(centre) : null;
const surface = Math.max(under ?? -Infinity, centreSurface ?? -Infinity);
return {
lat: (carto.latitude * 180) / Math.PI,
lon: (carto.longitude * 180) / Math.PI,
cameraM: Math.round(carto.height),
surfaceUnderM: under === null ? null : Math.round(under),
surfaceCentreM: centreSurface === null ? null : Math.round(centreSurface),
clearanceM: Number.isFinite(surface)
? Math.round(carto.height - surface)
: null,
pitchDeg: Math.round((camera.pitch * 180) / Math.PI),
};
});
}
async function arrivals() {
const page = await openPage();
const dir = path.join(outDir, 'arrivals');
fs.mkdirSync(dir, { recursive: true });
const recorder = await createScreencast(page, dir);
await recorder.start();
for (const query of ['Camp Mabry', '39.7392, -104.9903']) {
await page.evaluate(() =>
document.getElementById('search-toggle')?.click(),
);
await page.focus('#location-search');
await page.evaluate(() => {
document.getElementById('location-search').value = '';
});
await page.type('#location-search', query, { delay: 40 });
await page.keyboard.press('Enter');
// Wait for the flight to land: the camera still for 1.5 s.
const landed = Date.now();
let last = null;
let stillSince = null;
while (Date.now() - landed < 30_000) {
const now = await page.evaluate(() => {
const c = window.__godsEyeView.viewer.camera.positionWC;
return [c.x, c.y, c.z];
});
const moved =
!last ||
Math.hypot(now[0] - last[0], now[1] - last[1], now[2] - last[2]) > 0.5;
last = now;
if (moved) stillSince = null;
else stillSince ??= Date.now();
if (
Date.now() - landed > 4000 &&
stillSince &&
Date.now() - stillSince > 1500
)
break;
await sleep(250);
}
await waitTiles(page);
await sleep(2500); // the ground guard measures after tiles stream in
const arrived = await clearance(page);
const slug = query.replace(/\W+/g, '-').replace(/^-|-$/g, '').toLowerCase();
await page.screenshot({ path: path.join(dir, `arrival-${slug}.png`) });
// Look around: a gentle orbit, then check the eye still clears the ground.
const view = await page.evaluate(() => {
const c = window.__godsEyeView.viewer.camera;
return {
lat: (c.positionCartographic.latitude * 180) / Math.PI,
lon: (c.positionCartographic.longitude * 180) / Math.PI,
height: c.positionCartographic.height,
heading: (c.heading * 180) / Math.PI,
pitch: (c.pitch * 180) / Math.PI,
};
});
await moveCamera(page, { ...view, heading: view.heading + 25 }, 2.5);
await sleep(1500);
const lookedAround = await clearance(page);
result.arrivals.push({ query, arrived, lookedAround });
check(
`${query}: arrival clears the rendered surface`,
arrived.clearanceM !== null && arrived.clearanceM >= 60,
JSON.stringify(arrived),
);
}
result.arrivalClip = await recorder.stop();
await page.close();
}
async function contacts() {
const page = await openPage({ aircraft: true });
const dir = path.join(outDir, 'contacts');
fs.mkdirSync(dir, { recursive: true });
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('flights', true, {
origin: 'user',
}),
);
// A wide Texas view: installation viewports are unbounded here.
await moveCamera(
page,
{ lat: 30.35, lon: -97.76, height: 900_000, pitch: -90 },
0,
);
await sleep(3000);
// In-page probe: row text and installation entity churn every 250 ms.
await page.evaluate(() => {
const samples = [];
window.__gevContextSamples = samples;
let created = 0;
const seen = new WeakSet();
setInterval(() => {
const row = [
...document.querySelectorAll(
'#military-awareness-panel .military-awareness-row',
),
].find(
(candidate) =>
candidate.querySelector('strong')?.textContent?.trim() ===
'Mapped installations',
);
const ds = window.__godsEyeView.viewer.dataSources.getByName(
'military-installations',
)[0];
for (const entity of ds?.entities.values || [])
if (!seen.has(entity)) {
seen.add(entity);
created += 1;
}
samples.push({
t: performance.now(),
count: row?.querySelector('b')?.textContent?.trim() ?? null,
text: row?.innerText?.replace(/\s+/g, ' ').trim() ?? null,
installations: [
...(row?.querySelectorAll('button[data-awareness-id]') || []),
].map((button) => ({
id: button.dataset.awarenessId,
name: button.childNodes[0]?.textContent?.trim(),
distance: button.querySelector('span')?.textContent?.trim(),
})),
entities: ds?.entities.values.length ?? 0,
created,
tracked:
window.__godsEyeView.viewer.trackedEntity?.gevTrackedId ?? null,
cockpit: document.body.classList.contains('cockpit-mode'),
});
if (samples.length > 5000) samples.splice(0, 1000);
}, 250);
});
const recorder = await createScreencast(page, dir);
await recorder.start();
const t0 = await page.evaluate(() => performance.now());
await page.evaluate(() =>
document.getElementById('global-context-flights-btn').click(),
);
await page
.waitForFunction(
() =>
window.__godsEyeView.viewer.trackedEntity?.gevTrackedId ===
'flights:aaa051',
{ timeout: 60_000 },
)
.catch(() => {});
const readSamples = (from) =>
page.evaluate(
(a) => window.__gevContextSamples.filter((s) => s.t >= a),
from,
);
const numeric = (s) => /^\d+/.test(String(s.count ?? ''));
// Up to 15 s for the first answer, then watch 30 s of flight.
const deadline = Date.now() + 15_000;
let first = null;
while (Date.now() < deadline && !first) {
first = (await readSamples(t0)).find(
(s) =>
numeric(s) &&
Number.parseInt(s.count, 10) >= 1 &&
/\d+(\.\d+)? k?m\b/.test(s.text),
);
if (!first) await sleep(500);
}
check(
'Contacts lists a mapped installation with a distance',
Boolean(first),
first
? `${Math.round(first.t - t0)} ms: ${first.text}`
: 'row never answered',
);
await page.screenshot({ path: path.join(dir, 'contacts-camp-mabry.png') });
await sleep(30_000);
const flight = (await readSamples(first?.t ?? t0)).filter(
(s) => s.tracked === 'flights:aaa051',
);
const reverted = flight.filter((s) => !numeric(s));
const createdDuring = flight.length
? flight.at(-1).created - flight[0].created
: 0;
result.contacts.mabry = {
firstAnswerMs: first ? Math.round(first.t - t0) : null,
firstText: first?.text ?? null,
lastText: flight.at(-1)?.text ?? null,
samples: flight.length,
revertedSamples: reverted.length,
entities: flight.at(-1)?.entities ?? 0,
entitiesCreatedWhileTracking: createdDuring,
};
check(
'the row never falls back to "?" while the aircraft moves',
reverted.length === 0,
`${reverted.length}/${flight.length}`,
);
check(
'site entities are not rebuilt while tracking',
createdDuring === 0,
`${createdDuring} created`,
);
// Cockpit: the same context must hold with a horizon view.
const cockpit = await page.evaluate(() => {
const view = window.__godsEyeView.styleManager.cockpitView;
view.syncEntry();
return view.enter();
});
await sleep(12_000);
const inCockpit = (
await readSamples((await page.evaluate(() => performance.now())) - 8000)
).filter((s) => s.cockpit);
const scope = await page.evaluate(
() =>
document.getElementById('cockpit-context-subject')?.textContent || null,
);
await page.screenshot({ path: path.join(dir, 'cockpit-camp-mabry.png') });
result.contacts.cockpit = {
entered: cockpit,
samples: inCockpit.length,
last: inCockpit.at(-1)?.text ?? null,
scope,
};
check(
'Cockpit scope names the installation window',
/INSTALLATIONS WITHIN \d+ KM/.test(scope || ''),
scope || 'no scope',
);
check(
'Cockpit keeps mapped installations listed',
cockpit !== false &&
inCockpit.length > 0 &&
inCockpit.every((s) => numeric(s) && Number.parseInt(s.count, 10) >= 1),
inCockpit.at(-1)?.text || 'no cockpit samples',
);
await page.evaluate(() =>
window.__godsEyeView.styleManager.cockpitView.exit?.(),
);
await sleep(2000);
// SEARCH NEARBY SITES, as the user presses it.
await page.evaluate(() => {
document.getElementById('toast').textContent = '';
document.getElementById('installations-search-btn').click();
});
await page
.waitForFunction(
() => document.getElementById('toast').textContent.trim().length > 0,
{ timeout: 20_000 },
)
.catch(() => {});
const toast = await page.evaluate(() =>
document.getElementById('toast').textContent.trim(),
);
await sleep(1500);
const afterSearch = (
await readSamples((await page.evaluate(() => performance.now())) - 1000)
).at(-1);
result.contacts.search = { toast, row: afterSearch?.text ?? null };
check(
'SEARCH NEARBY SITES reports what it found and the row still lists sites',
/mapped sites? within \d+ km/i.test(toast) && numeric(afterSearch || {}),
`toast "${toast}" · ${afterSearch?.text}`,
);
// Fort Cavazos: leave Contacts, track the other aircraft, re-enter.
await page.evaluate(() =>
document.getElementById('global-context-flights-btn').click(),
);
await sleep(2000);
await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('flights')
.module.trackById('aaa052'),
);
await sleep(3000);
const t1 = await page.evaluate(() => performance.now());
await page.evaluate(() =>
document.getElementById('global-context-flights-btn').click(),
);
let cavazos = null;
const cavazosDeadline = Date.now() + 20_000;
while (Date.now() < cavazosDeadline) {
const latest = (await readSamples(t1))
.filter((s) => s.tracked === 'flights:aaa052' && numeric(s))
.at(-1);
const site = latest?.installations.find(
(record) => record.name === 'Fort Hood',
);
const nearest = site?.distance?.match(/^(\d+(?:\.\d+)?) (k?m)$/);
const km = nearest
? Number(nearest[1]) / (nearest[2] === 'm' ? 1000 : 1)
: null;
if (km !== null && km <= 30) {
cavazos = {
text: latest.text,
nearestKm: km,
site,
afterMs: Math.round(latest.t - t1),
};
break;
}
await sleep(500);
}
await sleep(2000);
await page.screenshot({ path: path.join(dir, 'contacts-fort-cavazos.png') });
result.contacts.cavazos = cavazos;
check(
'Fort Cavazos contact lists Fort Hood within 30 km',
Boolean(cavazos),
JSON.stringify(cavazos),
);
result.contactsClip = await recorder.stop();
await page.close();
}
try {
if (only !== 'contacts') await arrivals();
if (only !== 'arrivals') await contacts();
// A 404 for a synthetic aircraft's track logs "Failed to load resource";
// count those only when every failed response was such an expected miss.
const resourceErrors = result.consoleErrors.filter((text) =>
/Failed to load resource/.test(text),
).length;
const errors = result.consoleErrors.filter(
(text) =>
!/favicon/i.test(text) &&
!(
/Failed to load resource/.test(text) &&
!(result.httpErrors || []).length &&
resourceErrors <= (result.expectedMisses || []).length
),
);
check(
'zero console errors',
errors.length === 0,
errors.slice(0, 3).join(' | '),
);
check(
'zero failed app responses',
!(result.httpErrors || []).length,
(result.httpErrors || []).slice(0, 3).join(' | '),
);
} catch (error) {
check('harness completed', false, error.stack || error.message);
} finally {
fs.writeFileSync(
path.join(outDir, 'result.json'),
JSON.stringify(result, null, 1),
);
await browser.close();
}
console.log(
`${result.failures.length ? 'FAIL' : 'PASS'} — ${path.join(outDir, 'result.json')}`,
);
process.exit(result.failures.length ? 1 : 0);
+364
View File
@@ -0,0 +1,364 @@
#!/usr/bin/env node
/** Real pointer/keyboard acceptance using a synthetic, self-authored feature pack. */
import fs from 'node:fs';
import puppeteer from 'puppeteer';
const output = 'qa-shots/director-interactions';
fs.mkdirSync(output, { recursive: true });
const browser = await puppeteer.launch({
headless: true,
args: ['--no-sandbox', '--disable-dev-shm-usage'],
});
const page = await browser.newPage();
let failures = 0;
const errors = [];
const check = (name, ok) => {
console.log(`[${ok ? 'PASS' : 'FAIL'}] ${name}`);
if (!ok) failures++;
};
page.on('pageerror', (e) => errors.push(e.message));
await page.setRequestInterception(true);
page.on('request', (request) => {
if (new URL(request.url()).pathname === '/scene-assets/qa/actions.json')
void request
.respond({
status: 200,
contentType: 'application/geo+json',
body: JSON.stringify({
type: 'FeatureCollection',
features: [
{
type: 'Feature',
id: 'point',
geometry: { type: 'Point', coordinates: [-97.742, 30.2672, 600] },
},
],
}),
})
.catch(() => {});
else void request.continue().catch(() => {});
});
const action = (id) => `[data-director-action="${id}"]`;
const load = () =>
page.evaluate(() =>
window.__godsEyeView.sceneDirector.loadShot('interactive', 'one', {
flyDuration: 0.2,
}),
);
try {
await page.setViewport({ width: 1440, height: 900 });
await page.goto(
`${process.env.QA_BASE_URL || 'http://localhost:4173'}/?welcome=0`,
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() =>
window.__godsEyeView?.sceneDirector &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 60000 },
);
await page.evaluate(async () => {
const d = window.__godsEyeView.sceneDirector;
const camera = {
lat: 30.2672,
lon: -97.742,
alt: 4000,
heading: 0,
pitch: -90,
roll: 0,
altitudeReference: 'ellipsoid',
};
const target = { packId: 'data', featureId: 'point' };
const interaction = (id, label, action) => ({ id, label, target, action });
window.__interactionProject = {
version: 6,
scenes: [
{
id: 'interactive',
title: 'Interactive fixture',
anchors: [
{
id: 'view',
lat: 30.27,
lon: -97.74,
alt: 5000,
altitudeReference: 'ellipsoid',
},
],
dataPacks: [
{
id: 'data',
version: 1,
format: 'geojson',
source: { adapter: 'assets', path: 'qa/actions.json' },
attribution: { text: 'Synthetic fixture', license: 'CC0-1.0' },
placement: { altitudeReference: 'ellipsoid' },
},
],
shots: [
{
id: 'one',
title: 'Choose an action',
camera,
durationSec: 0.2,
holdSec: 0,
move: {
from: { ...camera, alt: 4100, altitudeReference: 'ellipsoid' },
easing: 'linear',
},
visual: { style: 'normal', mapStack: 'photoreal' },
layers: { traffic: false },
dataPackIds: ['data'],
interactions: [
interaction('card', 'Read source', {
type: 'card',
text: '<b>Plain text only</b>',
url: 'https://example.org/source',
}),
interaction('focus', 'Focus anchor', {
type: 'focus',
anchorId: 'view',
}),
interaction('layer', 'Show traffic', {
type: 'layer',
layerId: 'traffic',
enabled: true,
}),
interaction('next', 'Next shot', {
type: 'shot',
shotId: 'two',
}),
],
},
{
id: 'two',
title: 'Next',
camera: { ...camera, heading: 30 },
durationSec: 0.2,
holdSec: 0,
layers: { traffic: false },
},
],
},
],
};
await d.importProjectFile(
new File([JSON.stringify(window.__interactionProject)], 'actions.json'),
);
});
check(
'import installs no action handlers or UI',
await page.evaluate(
() =>
!window.__godsEyeView.sceneDirector.getInteractionState().active &&
!document.querySelector('[data-director-interactions]'),
),
);
await load();
check(
'settled LOAD activates four actions',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getInteractionState().count === 4,
),
);
await page.waitForFunction(
() => window.__godsEyeView.tileset?.tilesLoaded === true,
{ timeout: 60000 },
);
await new Promise((r) => setTimeout(r, 2000));
const point = await page.evaluate(async () => {
const d = window.__godsEyeView.sceneDirector;
const entity = d.viewer.entities.values.find(
(e) => e.name === 'data / point',
);
const pos = entity.position.getValue(d.viewer.clock.currentTime);
const screen = d.viewer.scene.cartesianToCanvasCoordinates(pos);
const rect = d.viewer.scene.canvas.getBoundingClientRect();
return { x: screen.x + rect.left, y: screen.y + rect.top };
});
await page.mouse.click(point.x, point.y);
check(
'real globe pick selects the feature and focuses its keyboard action',
await page.evaluate(
() =>
document.activeElement?.dataset.directorAction === 'card' &&
document
.querySelector('[data-director-interactions] [role=status]')
.textContent.includes('point'),
),
);
await page.keyboard.press('Enter');
check(
'Enter shows literal text and a safe source link',
await page.evaluate(() => {
const card = document.querySelector('[data-director-action-card]');
return (
card?.textContent.includes('<b>Plain text only</b>') &&
!card.querySelector('b') &&
card.querySelector('a')?.rel === 'noopener noreferrer'
);
}),
);
await page.screenshot({ path: `${output}/selected.png` });
fs.writeFileSync(
`${output}/report.json`,
JSON.stringify(
await page.evaluate(() => ({
tilesSettled: window.__godsEyeView.tileset?.tilesLoaded === true,
})),
null,
2,
),
);
await page.keyboard.press('Tab');
await page.keyboard.press('Space');
check(
'Tab/Space focuses the authored anchor through camera ownership',
await page.evaluate(
() =>
Math.abs(
window.__godsEyeView.sceneDirector.viewer.camera.positionCartographic
.height - 5000,
) < 1,
),
);
await page.screenshot({ path: `${output}/anchor.png` });
const sourceBase = process.env.QA_SOURCE_BASE || '/src';
// Resolve ownership from the same module graph as the active application.
await page.evaluate(async (base) => {
window.__inputOwner = await import(`${base}/data/inputOwnership.js`);
window.__lease = window.__inputOwner.claimPointer('qa-draw');
}, sourceBase);
await page.click(action('card'));
check(
'drawing ownership refuses actions',
await page.evaluate(() =>
document
.querySelector('[data-director-interactions] [role=status]')
.textContent.includes('unavailable'),
),
);
await page.evaluate(() => window.__inputOwner.releasePointer(window.__lease));
// Hold the actual manager admission call to exercise Stop before an enable settles.
await page.evaluate(() => {
const d = window.__godsEyeView.sceneDirector,
m = d.dataManager;
window.__originalEnable = m.setEnabled;
m.setEnabled = function (id, enabled, options) {
if (id !== 'traffic' || !enabled)
return window.__originalEnable.call(this, id, enabled, options);
window.__actionSignal = options.signal;
return new Promise((resolve) => {
window.__lateAction = () => resolve(true);
options.signal.addEventListener('abort', () => resolve(false), {
once: true,
});
});
};
});
await page.click(action('layer'));
await page.waitForFunction(() => window.__actionSignal);
await page.evaluate(() => window.__godsEyeView.sceneDirector.stopScene());
check(
'Stop aborts pending layer admission and removes handlers/cards/selection',
await page.evaluate(
() =>
window.__actionSignal.aborted &&
!window.__godsEyeView.sceneDirector.getInteractionState().active &&
!document.querySelector('[data-director-interactions]'),
),
);
await page.evaluate(() => {
window.__lateAction();
window.__godsEyeView.sceneDirector.dataManager.setEnabled =
window.__originalEnable;
});
await load();
await page.evaluate(() =>
window.__godsEyeView.sceneDirector.seekScene('interactive', 0.1),
);
check(
'same-shot seek restores pack geometry and resets action state',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getInteractionState().count === 4 &&
window.__godsEyeView.sceneDirector.getInteractionState().selected ===
null &&
window.__godsEyeView.sceneDirector.getDataPackState().count === 1,
),
);
await page.evaluate(() => {
window.__oldActionButton = document.querySelector(
'[data-director-action=card]',
);
});
await page.click(action('next'));
await page.waitForFunction(
() => window.__godsEyeView.sceneDirector._selectedShotId === 'two',
);
check(
'explicit shot transition replaces resources and removes actions',
await page.evaluate(
() =>
!window.__godsEyeView.sceneDirector.getInteractionState().active &&
window.__godsEyeView.sceneDirector.getDataPackState().count === 0,
),
);
await load();
await page.evaluate(() => window.__oldActionButton.click());
check(
'detached buttons cannot invoke replacement actions with the same ID',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getInteractionState().selected ===
null &&
!document.querySelector('[data-director-action-card]').textContent,
),
);
await page.evaluate(() => {
window.__godsEyeView.sceneDirector._interactionTransitions = 64;
});
await page.click(action('next'));
check(
'bounded transition chain cannot take another branch',
await page.evaluate(
() => window.__godsEyeView.sceneDirector._selectedShotId === 'one',
),
);
await load();
await page.evaluate(async () => {
const p = structuredClone(window.__interactionProject);
p.scenes[0].shots[0].interactions[0].target.featureId = 'missing';
const d = window.__godsEyeView.sceneDirector;
await d.importProjectFile(new File([JSON.stringify(p)], 'missing.json'));
await d.loadShot('interactive', 'one', { flyDuration: 0.2 });
});
check(
'unknown loaded feature refuses the whole action set',
await page.evaluate(
() =>
!window.__godsEyeView.sceneDirector.getInteractionState().active &&
!document.querySelector('[data-director-interactions]'),
),
);
await page.evaluate(async () => {
const d = window.__godsEyeView.sceneDirector;
await d.importProjectFile(
new File([JSON.stringify(window.__interactionProject)], 'restore.json'),
);
await d.loadShot('interactive', 'one', { flyDuration: 0.2 });
await d.destroy();
});
check(
'teardown releases interactive resources',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getInteractionState().count === 0 &&
!document.querySelector('[data-director-interactions]'),
),
);
check('no uncaught browser errors', errors.length === 0);
} finally {
await browser.close();
}
process.exitCode = failures ? 1 : 0;
+482
View File
@@ -0,0 +1,482 @@
#!/usr/bin/env node
/** Sharing/authoring acceptance with local synthetic assets and real installed controls. */
import fs from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const output = path.resolve('qa-shots/director-sharing');
fs.mkdirSync(output, { recursive: true });
const geometry = {
type: 'FeatureCollection',
features: [
{
type: 'Feature',
id: 'point',
geometry: { type: 'Point', coordinates: [-97.742, 30.2672, 500] },
},
],
};
fs.writeFileSync(path.join(output, 'data.geojson'), JSON.stringify(geometry));
const camera = {
lat: 30.2672,
lon: -97.742,
alt: 4000,
heading: 0,
pitch: -90,
roll: 0,
};
const project = {
version: 6,
scenes: [
{
id: 'share',
title: 'Sharing fixture',
dataPacks: [
{
id: 'data',
version: 1,
format: 'geojson',
source: { adapter: 'assets', path: 'example/data.geojson' },
attribution: { text: 'Synthetic <b>fixture</b>', license: 'CC0-1.0' },
placement: { altitudeReference: 'ellipsoid' },
},
],
shots: [
{
id: 'shot',
title: 'First',
camera,
durationSec: 0.3,
holdSec: 0,
visual: { style: 'normal', mapStack: 'photoreal' },
layers: {},
dataPackIds: ['data'],
interactions: [
{
id: 'info',
label: 'Read fixture',
target: { packId: 'data', featureId: 'point' },
action: { type: 'card', text: 'Synthetic scene note' },
},
],
},
],
},
{ id: 'second', title: 'Other scene', shots: [] },
],
};
const file = path.join(output, 'project.json');
fs.writeFileSync(file, JSON.stringify(project));
const browser = await puppeteer.launch({
headless: true,
args: ['--no-sandbox', '--disable-dev-shm-usage'],
});
const page = await browser.newPage();
let failures = 0;
const errors = [],
requests = [];
const check = (name, ok) => {
console.log(`[${ok ? 'PASS' : 'FAIL'}] ${name}`);
if (!ok) failures++;
};
page.on('pageerror', (e) => errors.push(e.message));
page.on('request', (r) => {
if (new URL(r.url()).pathname.startsWith('/scene-assets/'))
requests.push(r.url());
});
async function click(label) {
const handle = await page.evaluateHandle(
(label) =>
[...document.querySelectorAll('button')].find(
(b) => b.textContent === label,
),
label,
);
await handle.asElement().click();
await handle.dispose();
}
async function value(label, text) {
await page.$eval(
`[aria-label="${label}"]`,
(node, text) => {
node.value = text;
node.dispatchEvent(new Event('input', { bubbles: true }));
},
text,
);
}
try {
await page.setViewport({ width: 1440, height: 900 });
await page.goto(
`${process.env.QA_BASE_URL || 'http://localhost:4173'}/?welcome=0`,
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() =>
window.__godsEyeView?.sceneDirector &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 60000 },
);
await page.click('[data-collapse-target="scene-panel"]');
const before = await page.evaluate(() =>
JSON.stringify(window.__godsEyeView.sceneDirector._project),
);
const input = await page.$('#scene-import-file');
await input.uploadFile(file);
await page.waitForSelector('[data-director-apply-import]');
check(
'preview parses metadata without acquiring assets or replacing the current project',
requests.length === 0 &&
(await page.evaluate(
(before) =>
JSON.stringify(window.__godsEyeView.sceneDirector._project) ===
before,
before,
)),
);
check(
'preview renders author attribution as literal text',
await page.evaluate(
() =>
document
.querySelector('[data-director-dialog]')
.textContent.includes('Synthetic <b>fixture</b>') &&
!document.querySelector('[data-director-dialog] b'),
),
);
await page.keyboard.press('Escape');
check(
'Escape cancels preview without altering saved work',
await page.evaluate(
(before) =>
!document.querySelector('[data-director-dialog]') &&
JSON.stringify(window.__godsEyeView.sceneDirector._project) === before,
before,
),
);
await input.uploadFile(file);
await page.waitForSelector('[data-director-apply-import]');
await page.click('[data-director-apply-import]');
await page.waitForFunction(
() => !document.querySelector('[data-director-dialog]'),
);
check(
'Apply import replaces metadata but does not start playback or fetch a pack',
requests.length === 0 &&
(await page.evaluate(
() =>
window.__godsEyeView.sceneDirector._project.scenes.length === 2 &&
!window.__godsEyeView.sceneDirector.running,
)),
);
await click('EDIT DETAILS');
await click('Capture camera as anchor');
await page.waitForFunction(() =>
document
.querySelector('[data-director-dialog] [role=status]')
.textContent.includes('Anchor added'),
);
await click('Set move start to current camera');
await page.waitForFunction(() =>
document
.querySelector('[data-director-dialog] [role=status]')
.textContent.includes('Move added'),
);
const draft = await page.$eval(
'[aria-label="Shot camera, timing, packs and actions"]',
(n) => n.value,
);
const bad = JSON.parse(draft);
bad.camera.lat = 91;
await value('Shot camera, timing, packs and actions', JSON.stringify(bad));
const saved = await page.evaluate(() =>
localStorage.getItem('godsEyeView.sceneProject.v2'),
);
await page.click('[data-director-apply-details]');
await page.waitForFunction(() =>
document
.querySelector('[data-director-dialog] [role=status]')
.textContent.includes('camera.lat'),
);
check(
'invalid draft preserves saved bytes and existing authored content',
await page.evaluate(
(saved) => localStorage.getItem('godsEyeView.sceneProject.v2') === saved,
saved,
),
);
await value('Shot camera, timing, packs and actions', draft);
await page.click('[data-director-apply-details]');
await page.waitForFunction(
() => !document.querySelector('[data-director-dialog]'),
);
check(
'author controls preserve selected shot, pack attribution and action definitions',
await page.evaluate(() => {
const d = window.__godsEyeView.sceneDirector,
s = d._getSelectedScene();
return (
d._selectedShotId === 'shot' &&
s.anchors.length === 1 &&
s.shots[0].move.easing === 'cubic-in-out' &&
s.shots[0].interactions.length === 1 &&
s.dataPacks[0].attribution.license === 'CC0-1.0'
);
}),
);
await page.evaluate(() => {
window.__shareExports = [];
const create = URL.createObjectURL.bind(URL);
URL.createObjectURL = (blob) => {
if (blob.type === 'application/json')
window.__shareExports.push(blob.text());
return create(blob);
};
});
await click('SHARE SCENE');
await click('Download scene JSON');
const exported = await page.evaluate(async () =>
JSON.parse(await window.__shareExports.at(-1)),
);
check(
'scene JSON exports only the selected scene and keeps its credit',
exported.scenes.length === 1 &&
exported.scenes[0].dataPacks[0].attribution.license === 'CC0-1.0',
);
await click('Download asset bundle');
await page.waitForFunction(() =>
document
.querySelector('[data-director-dialog] [role=status]')
.textContent.includes('Could not complete'),
);
check(
'missing explicit files produce no partial download or network fallback',
requests.length === 0 &&
(await page.evaluate(() => window.__shareExports.length === 1)),
);
const files = await page.$('[aria-label="Choose data-pack files"]');
await files.uploadFile(path.join(output, 'data.geojson'));
await click('Download asset bundle');
await page.waitForFunction(
() =>
document.querySelector('[data-director-dialog] [role=status]')
.textContent === 'Asset bundle downloaded',
);
const bundle = await page.evaluate(
async () => await window.__shareExports.at(-1),
);
const bundleFile = path.join(output, 'scene.gevbundle.json');
fs.writeFileSync(bundleFile, bundle);
check(
'bundle contains the selected bytes and preserves attribution',
JSON.parse(bundle).assets.length === 1 &&
JSON.parse(bundle).project.scenes[0].dataPacks[0].attribution.license ===
'CC0-1.0',
);
await page.keyboard.press('Escape');
await input.uploadFile(bundleFile);
await page.waitForSelector('[data-director-apply-import]');
check(
'bundle preview verifies bytes and explains session-only storage',
await page.evaluate(
() =>
document
.querySelector('[data-director-dialog]')
.textContent.includes('bundled bytes verified') &&
window.__godsEyeView.sceneDirector.getSharingState().assets.count === 0,
),
);
await page.screenshot({ path: path.join(output, 'preview.png') });
await page.click('[data-director-apply-import]');
await page.waitForFunction(
() => !document.querySelector('[data-director-dialog]'),
);
await page.evaluate(() =>
window.__godsEyeView.sceneDirector.loadShot('share', 'shot', {
flyDuration: 0.3,
}),
);
check(
'imported bundle renders and enables feature actions with zero asset requests',
requests.length === 0 &&
(await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getDataPackState().count === 1 &&
window.__godsEyeView.sceneDirector.getInteractionState().count === 1,
)),
);
await page.waitForFunction(
() => window.__godsEyeView.tileset?.tilesLoaded === true,
{ timeout: 60000 },
);
await page.screenshot({ path: path.join(output, 'rendered.png') });
fs.writeFileSync(
path.join(output, 'report.json'),
JSON.stringify(
await page.evaluate(() => ({
tilesSettled: window.__godsEyeView.tileset?.tilesLoaded === true,
})),
),
);
await page.evaluate(() => window.__godsEyeView.sceneDirector.stopScene());
check(
'Stop releases rendered resources while retaining reusable project bytes',
await page.evaluate(() => {
const d = window.__godsEyeView.sceneDirector;
return (
d.getDataPackState().count === 0 &&
d.getSharingState().assets.count === 1
);
}),
);
await page.evaluate(() =>
window.__godsEyeView.sceneDirector.seekScene('share', 0),
);
check(
'seek reloads the bundle without a network fallback',
requests.length === 0 &&
(await page.evaluate(
() => window.__godsEyeView.sceneDirector.getDataPackState().count === 1,
)),
);
await click('EDIT DETAILS');
await page.evaluate(() => {
window.__godsEyeView.sceneDirector._project.scenes[0].title = 'Newer edit';
});
await page.click('[data-director-apply-details]');
check(
'stale author draft cannot overwrite a newer edit',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector._project.scenes[0].title ===
'Newer edit',
),
);
await page.keyboard.press('Escape');
await click('SHARE SCENE');
await page.setViewport({ width: 390, height: 844 });
check(
'modal remains within a narrow viewport',
await page.$eval('[data-director-dialog]', (n) => {
const r = n.getBoundingClientRect();
return (
r.left >= 0 &&
r.right <= innerWidth &&
r.top >= 0 &&
r.bottom <= innerHeight
);
}),
);
await page.screenshot({ path: path.join(output, 'narrow.png') });
await page.keyboard.press('Escape');
const pending = await page.evaluate(async () => {
const d = window.__godsEyeView.sceneDirector;
let finish;
const work = d._sharing.preview({
name: 'pending.json',
text: () => new Promise((r) => (finish = r)),
});
d._sharing.close();
await work;
finish(JSON.stringify({ version: 6, scenes: [] }));
return !d.getSharingState().open;
});
check(
'cancel settles a pending file read and suppresses its late completion',
pending,
);
const cancelledApply = await page.evaluate(async () => {
const d = window.__godsEyeView.sceneDirector;
const before = JSON.stringify(d._project);
const saved = JSON.stringify({ ...localStorage });
let finish;
const pendingWork = new Promise((resolve) => (finish = resolve));
d._pendingWork.add(pendingWork);
const owner = new AbortController();
const work = d.importProjectFile(
{ name: 'cancelled.json' },
{
prepared: { project: { version: 6, scenes: [] }, assets: new Map() },
expectedProject: JSON.stringify(d._project, null, 2),
signal: owner.signal,
},
);
owner.abort();
finish();
const applied = await work;
d._pendingWork.delete(pendingWork);
return (
applied === false &&
JSON.stringify(d._project) === before &&
JSON.stringify({ ...localStorage }) === saved &&
d.getSharingState().assets.count === 1
);
});
check(
'cancelling Apply during pending scene cleanup preserves saved project and bundle bytes',
cancelledApply,
);
await click('EDIT DETAILS');
await page.evaluate(() => {
const fields = [
...document.querySelectorAll('[data-director-dialog] textarea'),
];
const scene = JSON.parse(fields[0].value),
shot = JSON.parse(fields[1].value);
scene.dataPacks = [];
shot.dataPackIds = [];
shot.interactions = [];
fields[0].value = JSON.stringify(scene);
fields[1].value = JSON.stringify(shot);
});
await page.click('[data-director-apply-details]');
await page.waitForFunction(
() => !document.querySelector('[data-director-dialog]'),
);
check(
'removing a pack through authoring releases its retained bytes',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getSharingState().assets.count === 0,
),
);
await page.evaluate(
(text) =>
window.__godsEyeView.sceneDirector.importProjectFile(
new File([text], 'again.gevbundle.json'),
),
bundle,
);
await page.evaluate(() =>
window.__godsEyeView.sceneDirector.importProjectFile(
new File([JSON.stringify({ version: 6, scenes: [] })], 'empty.json'),
),
);
check(
'replacement releases stored bundle bytes',
await page.evaluate(
() =>
window.__godsEyeView.sceneDirector.getSharingState().assets.count === 0,
),
);
await page.evaluate(async () => {
const d = window.__godsEyeView.sceneDirector;
void d._sharing.preview({
name: 'pending.json',
text: () => new Promise(() => {}),
});
await d.destroy();
});
check(
'teardown removes pending dialogs and authoring controls',
await page.evaluate(
() =>
!document.querySelector('[data-director-dialog]') &&
!document.querySelector('[data-director-authoring]') &&
!window.__godsEyeView.sceneDirector.getSharingState().open,
),
);
check('no uncaught browser errors', errors.length === 0);
} finally {
await browser.close();
}
process.exitCode = failures ? 1 : 0;
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env node
/** Persistent inline attribution through actual OSM layer display lifetimes. */
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import puppeteer from 'puppeteer';
import { moveCamera } from './qa-journey-recorder.mjs';
const url = process.argv.find((a) => /^https?:/.test(a));
if (!url) throw new Error('Supply the running server URL');
const dir = process.env.QA_POLISH_DIR || 'qa-shots/final-polish';
await fs.mkdir(dir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const result = { states: [], errors: [] };
const browser = await puppeteer.launch({
headless: true,
args: ['--use-angle=metal'],
});
try {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
await page.goto(`${url}/?welcome=0`, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.styleManager.setDetection({ enabled: false });
});
await moveCamera(
page,
{ lon: -97.765, lat: 30.3125, height: 3500, pitch: -80 },
0,
);
const toggle = (id, value) =>
page.evaluate(
(id, value) => window.__godsEyeView.dataManager.setEnabled(id, value),
id,
value,
);
const text = () => page.$eval('#cesium-credits', (e) => e.innerText);
const shot = async (name, osm, tiles) => {
await sleep(300);
const state = await page.$eval('#cesium-credits', (e) => {
const nodes = [
...e.querySelectorAll('a,img,.cesium-credit-expand-link'),
].filter((n) =>
n.checkVisibility({ visibilityProperty: true, opacityProperty: true }),
);
return {
text: e.innerText,
rects: nodes
.map((n) => {
const b = n.getBoundingClientRect();
return {
text: n.textContent || n.alt || 'image',
x: b.x,
y: b.y,
width: b.width,
height: b.height,
reachable: n.contains(
document.elementFromPoint(
b.x + b.width / 2,
b.y + b.height / 2,
),
),
};
})
.filter((b) => b.width > 0 && b.height > 0),
};
});
assert.equal(state.text.includes('© OpenStreetMap'), osm);
assert.equal(state.text.includes('© OpenMapTiles'), tiles);
assert.equal(
(state.text.match(/© OpenStreetMap/g) || []).length,
osm ? 1 : 0,
);
assert.ok(state.rects.length >= 3);
const centers = state.rects.map((b) => b.y + b.height / 2);
assert.ok(
Math.max(...centers) - Math.min(...centers) < 12,
`${name}: credits stay on one desktop row`,
);
assert.ok(
state.rects.every(
(b) => b.x >= 0 && b.x + b.width <= page.viewport().width,
),
'no credit clipped',
);
assert.ok(
state.rects.every((b) => b.reachable),
'all credits and the attribution link remain clear of controls',
);
await page.screenshot({ path: `${dir}/after-${name}.png` });
await page.screenshot({
path: `${dir}/after-${name}-row.png`,
clip: { x: 16, y: 750, width: 1150, height: 150 },
});
result.states.push({ name, ...state });
};
await sleep(1600);
await shot('nothing', false, false);
await toggle('alpr-cameras', true);
await page.waitForFunction(
() =>
document
.querySelector('#cesium-credits')
.innerText.includes('© OpenStreetMap'),
{ timeout: 60000 },
);
await shot('alpr', true, false);
await sleep(6000);
await shot('alpr-persistent', true, false);
await toggle('traffic', true);
await page.waitForFunction(
() =>
document
.querySelector('#cesium-credits')
.innerText.includes('© OpenMapTiles'),
{ timeout: 60000 },
);
await shot('alpr-and-traffic', true, true);
await toggle('alpr-cameras', false);
await shot('traffic', true, true);
for (const width of [1280, 1920]) {
await page.setViewport({ width, height: 900 });
await sleep(500);
await shot(`traffic-${width}`, true, true);
}
await page.setViewport({ width: 1440, height: 900 });
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(true),
);
await shot('traffic-clean-ui', true, true);
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(false),
);
await toggle('traffic', false);
await shot('nothing-restored', false, false);
assert.deepEqual(result.errors, []);
console.log(JSON.stringify(result, null, 2));
} finally {
await fs.writeFile(`${dir}/credits.json`, JSON.stringify(result, null, 2));
await browser.close();
}
+412
View File
@@ -0,0 +1,412 @@
#!/usr/bin/env node
/** Mapped installations: isolated camera navigation, cached revisits and Contacts release. */
import fs from 'node:fs';
import puppeteer from 'puppeteer';
import { moveCamera } from './qa-journey-recorder.mjs';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-installation-navigation.mjs <url> [--baseline]',
);
process.exit(0);
}
const baseline = args.includes('--baseline');
const result = { views: [], errors: [], forbidden: [], failures: [] };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const browser = await puppeteer.launch({
headless: true,
args: ['--use-angle=metal', '--disable-background-timer-throttling'],
protocolTimeout: 120000,
});
try {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
const tiles = [];
page.on('request', (r) => {
const u = new URL(r.url());
if (
u.hostname === 'tiles.openfreemap.org' &&
/\/\d+\/\d+\/\d+(\.pbf)?$/.test(u.pathname)
)
tiles.push(u.pathname);
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
result.forbidden.push(u.hostname);
});
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
await page.goto(
args.find((a) => /^https?:/.test(a)),
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
window.installationEvents = { changed: 0, end: 0 };
window.installationEntityRefs = new Map();
g.viewer.camera.changed.addEventListener(
() => window.installationEvents.changed++,
);
g.viewer.camera.moveEnd.addEventListener(
() => window.installationEvents.end++,
);
});
let low = 60000,
high = 400000;
for (let i = 0; i < 9; i++) {
const height = (low + high) / 2;
await moveCamera(page, { lat: 32.72, lon: -117.16, height, pitch: -90 }, 0);
const fits = await page.evaluate(async () => {
const { installationTileZoom } =
await import('/src/layers/installations/source.js');
const r = window.__godsEyeView.viewer.camera.computeViewRectangle();
const d = 180 / Math.PI;
return (
r &&
installationTileZoom({
west: r.west * d,
east: r.east * d,
south: r.south * d,
north: r.north * d,
}) !== null
);
});
if (fits) low = height;
else high = height;
}
const widestHeight = Math.floor(low);
result.widestHeight = widestHeight;
result.firstTooWideHeight = Math.ceil(high);
let enabled = false;
const places = [
['san-diego', 32.72, -117.16],
['pendleton', 33.33, -117.4],
['miramar', 32.87, -117.14],
['san-diego-return', 32.72, -117.16],
];
for (const height of [
5000,
20000,
60000,
150000,
widestHeight,
400000,
5000,
]) {
for (const [name, lat, lon] of height >= 150000
? places.slice(0, 1)
: places) {
const before = tiles.length;
const previous = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().lastUpdate,
);
await moveCamera(
page,
{ lat, lon, height, pitch: -90 },
enabled ? 0.6 : 0,
);
const stop = Date.now();
if (!enabled) {
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'military-installations',
true,
),
);
enabled = true;
}
await page
.waitForFunction(
(prev, wide) => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return (
!s.loading &&
(s.lastUpdate > prev || (wide && s.status === 'zoom-in'))
);
},
{ timeout: 6000 },
previous,
height === 400000,
)
.catch(() => {});
const snapshot = await page.evaluate(() => {
const g = window.__godsEyeView;
const ds = g.viewer.dataSources.getByName('military-installations')[0];
const points = ds.entities.values.filter((e) => e.point);
const stable = points.every(
(e) =>
!window.installationEntityRefs.has(e.id) ||
window.installationEntityRefs.get(e.id) === e,
);
window.installationEntityRefs = new Map(points.map((e) => [e.id, e]));
return {
...g.dataManager.layers
.get('military-installations')
.module.getStats(),
stable,
ids: points.map((e) => e.id),
events: { ...window.installationEvents },
};
});
const row = {
name,
height,
ms: Date.now() - stop,
tiles: tiles.length - before,
updated: snapshot.lastUpdate > previous,
...snapshot,
};
if (
(!row.stable && result.views.at(-1)?.height === height) ||
new Set(row.ids).size !== row.ids.length
)
result.failures.push(
`${name}/${height}: unstable or duplicate site entities`,
);
if (name.endsWith('return')) {
const first = result.views.find(
(v) => v.name === 'san-diego' && v.height === height,
);
if (
first &&
JSON.stringify([...first.ids].sort()) !==
JSON.stringify([...row.ids].sort())
)
result.failures.push(
`${name}/${height}: changed site ids on revisit`,
);
}
result.views.push(row);
console.log(JSON.stringify(row));
if (!row.updated && row.status !== 'zoom-in')
result.failures.push(`${name}/${height}: no reload`);
if (row.ms > 1500 && row.status !== 'zoom-in')
result.failures.push(`${name}/${height}: ${row.ms} ms`);
if (name.endsWith('return') && row.tiles)
result.failures.push(`${name}/${height}: revisit fetched ${row.tiles}`);
}
}
for (const gesture of [
'wheel',
'keyboard',
'contacts-close',
'contacts-clear',
]) {
let previous = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().lastUpdate,
);
if (gesture === 'wheel') {
await page.mouse.move(640, 400);
await page.mouse.wheel({ deltaY: -350 });
} else if (gesture === 'keyboard') {
await page.evaluate(() =>
document.getElementById('search-toggle').click(),
);
await page.focus('#location-search');
await page.evaluate(() => {
document.getElementById('location-search').value = '';
});
await page.type('#location-search', '33.33, -117.40');
await page.keyboard.press('Enter');
await sleep(4500);
} else {
await moveCamera(
page,
{ lat: 32.87, lon: -117.14, height: 5000, pitch: -90 },
0.6,
);
await page.waitForFunction(
() =>
window.__godsEyeView.viewer.dataSources
.getByName('military-installations')[0]
.entities.values.some((e) => e.point),
{ timeout: 5000 },
);
await page.evaluate(() => {
const button = document.getElementById('global-context-flights-btn');
if (button.getAttribute('aria-selected') !== 'true') button.click();
});
await page.waitForFunction(
() => {
const g = window.__godsEyeView;
return (
g.dataManager.isEffectivelyEnabled('military-awareness') &&
!g.dataManager.layers.get('military-awareness').module.getParams()
.passive
);
},
{ timeout: 15000 },
);
await sleep(1000);
await page.evaluate(async () => {
const g = window.__godsEyeView;
const ds = g.viewer.dataSources.getByName('military-installations')[0];
const id = ds.entities.values.find((e) => e.point).id;
window.installationSubjectId = id;
await g.dataManager.layers
.get('military-awareness')
.module.focusTarget('military-installations', id);
});
await page.waitForFunction(
() => {
const a =
window.__godsEyeView.dataManager.layers.get(
'military-awareness',
).module;
const subject = a.getContextSnapshot()?.subject;
return (
subject?.layerId === 'military-installations' &&
subject.id === window.installationSubjectId &&
!a._getAwarenessNavigationStateForTest().pendingSelectionKey
);
},
{ timeout: 15000 },
);
await page.waitForFunction(
() => !window.__godsEyeView.viewer.camera._currentFlight,
{ timeout: 10000 },
);
await page.waitForFunction(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().coverage.kind === 'subject',
{ timeout: 10000 },
);
await page.waitForFunction(
() =>
document
.getElementById('global-context-flights-btn')
.getAttribute('aria-busy') === 'false',
{ timeout: 30000 },
);
const anchored = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats(),
);
if (anchored.coverage.kind !== 'subject')
result.failures.push(`${gesture}: Contacts did not acquire subject`);
result.contextClear = await page.evaluate(async (clear) => {
if (clear) {
const { clearSelectedEntityContextForLayer, getContextStore } =
await import('/src/data/contextStore.js');
const store = getContextStore();
const before = {
selectedId: store.selectedEntityId,
record: store.entities.get(store.selectedEntityId)?.id,
subject: window.__godsEyeView.dataManager.layers
.get('military-awareness')
.module.getContextSnapshot()?.subject,
};
clearSelectedEntityContextForLayer('military-installations');
return before;
} else document.getElementById('global-context-flights-btn').click();
}, gesture === 'contacts-clear');
if (gesture === 'contacts-clear') {
await page.waitForFunction(
() =>
!window.__godsEyeView.dataManager.layers
.get('military-awareness')
.module.getContextSnapshot(),
{ timeout: 5000 },
);
} else {
await page.waitForFunction(
() => {
const button = document.getElementById(
'global-context-flights-btn',
);
return (
button.getAttribute('aria-selected') === 'false' &&
button.getAttribute('aria-busy') === 'false'
);
},
{ timeout: 30000 },
);
}
await page.waitForFunction(
() => !window.__godsEyeView.viewer.camera._currentFlight,
{ timeout: 10000 },
);
await sleep(500);
previous = await page.evaluate(
() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats().lastUpdate,
);
await moveCamera(
page,
{ lat: 33.33, lon: -117.4, height: 5000, pitch: -90 },
0.6,
);
}
await page
.waitForFunction(
(prev) => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return (
!s.loading && s.lastUpdate > prev && s.coverage.kind === 'viewport'
);
},
{ timeout: 6000 },
previous,
)
.catch(() => result.failures.push(`${gesture}: no viewport reload`));
const row = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats(),
);
if (gesture.startsWith('contacts') && row.count !== 1)
result.failures.push(
`${gesture}: destination count ${row.count}, expected 1`,
);
result.views.push({ name: gesture, ...row });
console.log(JSON.stringify({ name: gesture, ...row }));
}
await page.screenshot({ path: 'qa-shots/installations-navigation.png' });
} finally {
fs.mkdirSync('qa-shots', { recursive: true });
fs.writeFileSync(
`qa-shots/installations-navigation-${baseline ? 'before' : 'after'}.json`,
JSON.stringify(result, null, 2),
);
await browser.close();
}
if (
!baseline &&
(result.failures.length || result.errors.length || result.forbidden.length)
)
process.exitCode = 1;
+404
View File
@@ -0,0 +1,404 @@
/** Shared browser assertions for installation label persistence and selection. */
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import { moveCamera, createScreencast } from './qa-journey-recorder.mjs';
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
export async function checkInstallationPan(page, dir) {
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('local-datacenters', true),
);
await sleep(3000);
await page.evaluate(async () => {
const { getOverlayPaintRect } =
await import('/src/overlays/worldOverlay.js');
const g = window.__godsEyeView;
const installations = g.dataManager.layers.get(
'military-installations',
).module;
const centerIds = [];
for (let i = 0; i < g.viewer.dataSources.length; i++)
for (const e of g.viewer.dataSources.get(i).entities.values)
if (e.__localLayerId === 'local-datacenters') centerIds.push(e.id);
window.__polishPanFrames = [];
window.__stopPolishPan = g.viewer.scene.postRender.addEventListener(() => {
const sites = [],
centers = [];
installations.visitNamedMarkers((id) => {
if (getOverlayPaintRect('military-installations', id)) sites.push(id);
});
for (const id of centerIds)
if (getOverlayPaintRect('local-datacenters', id)) centers.push(id);
window.__polishPanFrames.push({ t: performance.now(), sites, centers });
});
});
await moveCamera(page, { lon: -99, lat: 39, height: 6500000, pitch: -90 }, 8);
const frames = await page.evaluate(() => {
window.__stopPolishPan();
return window.__polishPanFrames;
});
const count = (key) => {
let appeared = 0,
disappeared = 0,
reappeared = 0;
const seen = new Set(frames[0][key]);
for (let i = 1; i < frames.length; i++) {
const previous = new Set(frames[i - 1][key]),
next = new Set(frames[i][key]);
frames[i][`${key}Appeared`] = [...next].filter((id) => !previous.has(id));
frames[i][`${key}Disappeared`] = [...previous].filter(
(id) => !next.has(id),
);
for (const id of frames[i][`${key}Appeared`]) {
appeared++;
if (seen.has(id)) reappeared++;
seen.add(id);
}
disappeared += frames[i][`${key}Disappeared`].length;
}
const mean =
frames.reduce((sum, f) => sum + f[key].length, 0) / frames.length;
return {
appeared,
disappeared,
reappeared,
mean,
eventsPerLabel: (appeared + disappeared) / mean,
};
};
const result = {
frames: frames.length,
installations: count('sites'),
datacenters: count('centers'),
};
await fs.writeFile(
`${dir}/slow-pan-frames.json`,
JSON.stringify(frames, null, 2),
);
await fs.writeFile(`${dir}/slow-pan.json`, JSON.stringify(result, null, 2));
assert.ok(frames.length > 100);
assert.ok(
result.installations.mean >= 5 && result.datacenters.mean >= 5,
'both real layers contribute visible labels on the same pan',
);
assert.ok(
frames.every((f) => f.sites.length <= 24),
'shared-domain budget cannot exceed the installation title cap',
);
assert.ok(
result.installations.eventsPerLabel <=
result.datacenters.eventsPerLabel + 0.1,
'installation label transitions per label match Data Centers on the same slow pan',
);
assert.ok(
result.installations.reappeared <= result.datacenters.reappeared,
'installation labels do not flicker off and back on more than Data Centers',
);
await page.screenshot({ path: `${dir}/slow-pan-side-by-side.png` });
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('local-datacenters', false),
);
return result;
}
export async function checkInstallationSelection(
page,
dir,
{ video = true } = {},
) {
const result = [];
const recorder = video
? await createScreencast(page, `${dir}/selection-orbit`, { fps: 24 })
: null;
await recorder?.start();
try {
for (const site of [
{
key: 'mabry',
name: 'Camp Mabry',
lon: -97.765,
lat: 30.3125,
height: 3500,
},
{
key: 'naval',
name: 'Naval Base San Diego',
lon: -117.119,
lat: 32.684,
height: 6500,
},
]) {
await moveCamera(page, { ...site, pitch: -80 }, 1.5);
await page.waitForFunction(
() => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return !s.loading && s.count > 0;
},
{ timeout: 60000 },
);
await sleep(1500);
const picked = await page.evaluate(async (site) => {
const C = await import('/node_modules/cesium/Build/Cesium/index.js');
const g = window.__godsEyeView,
layer = g.dataManager.layers.get('military-installations').module;
const record = layer
.getNearby(C.Cartesian3.fromDegrees(site.lon, site.lat), 100000, 1000)
.find((r) => r.name === site.name && r.footprints?.length);
if (!record) throw new Error(`No polygon for ${site.name}`);
let point;
layer.visitNamedMarkers((id, p) => {
if (id === record.id) point = p.position;
});
const screen = C.SceneTransforms.worldToWindowCoordinates(
g.viewer.scene,
point,
);
window.__polishSelectedId = record.id;
window.__polishTitlePaints = [];
const canvas = document.getElementById('world-overlay-canvas'),
ctx = canvas.getContext('2d');
const original = ctx.fillText;
const matches = (text) =>
String(text).toLowerCase() === site.name.toLowerCase();
ctx.fillText = function (text, ...args) {
if (matches(text))
window.__polishTitlePaints.push({ t: performance.now(), text });
return original.call(this, text, ...args);
};
window.__restorePolishPaint = () => {
ctx.fillText = original;
};
window.__polishSelectionFrames = [];
const { getSelectedEntityContext } =
await import('/src/data/contextStore.js');
const { getOverlayPaintRect, getWorldOverlayDiagnostics } =
await import('/src/overlays/worldOverlay.js');
const knownFills = new Set();
const fillEntities = () => {
const list = [];
for (let i = 0; i < g.viewer.dataSources.length; i++)
for (const e of g.viewer.dataSources.get(i).entities.values)
if (e.id.startsWith(`${record.id}:selected-fill:`)) list.push(e);
for (const e of list) knownFills.add(e);
return list;
};
// Geometry-instance ids prove the ground primitive actually built, not just an entity declaration.
const primitives = () => {
let count = 0;
const walk = (collection) => {
if (collection?._collectionsArray) {
for (const child of collection._collectionsArray) walk(child);
return;
}
for (let i = 0; i < (collection?.length || 0); i++) {
const p = collection.get(i);
if (
p.ready &&
typeof p.getGeometryInstanceAttributes === 'function'
) {
for (const e of knownFills) {
try {
const a = p.getGeometryInstanceAttributes(e);
if (a && (!a.show || a.show[0])) count++;
} catch {}
}
}
if (typeof p.get === 'function' || p._collectionsArray) walk(p);
}
};
walk(g.viewer.scene.groundPrimitives);
return count;
};
window.__polishFillProbe = () => ({
entities: fillEntities().length,
primitives: primitives(),
classification: fillEntities().map((e) =>
e.polygon.classificationType.getValue(),
),
selected: getSelectedEntityContext()?.id || null,
});
window.__stopPolishSelection =
g.viewer.scene.postRender.addEventListener(() => {
window.__polishSelectionFrames.push({
paints: window.__polishTitlePaints.splice(0).length,
own: Boolean(
getOverlayPaintRect('military-installations', record.id),
),
tracked: Boolean(
getOverlayPaintRect('tracked', `installations:${record.id}`),
),
selected: getSelectedEntityContext()?.id === record.id,
total:
getWorldOverlayDiagnostics().paintedBySource[
'military-installations'
] || 0,
});
});
return {
id: record.id,
x: screen.x,
y: screen.y,
parts: record.footprints.length,
};
}, site);
await page.screenshot({
path: `${dir}/after-${site.key}-unselected.png`,
});
await page.mouse.click(picked.x, picked.y);
await page
.waitForFunction(
() => {
const p = window.__polishFillProbe();
return p.selected === window.__polishSelectedId && p.primitives > 0;
},
{ timeout: 20000 },
)
.catch(async (error) => {
await page.screenshot({ path: `${dir}/selection-failure.png` });
const diagnostic = await page.evaluate(() => {
const g = window.__godsEyeView;
const describe = (p) => ({
type: p.constructor.name,
ready: p.ready,
length: p.length,
keys: Object.keys(p).filter((k) =>
/primitive|instance|collection/i.test(k),
),
ids: p._instanceIds?.map((id) => id.id || String(id)),
children:
typeof p.get === 'function'
? Array.from({ length: p.length }, (_, i) =>
describe(p.get(i)),
)
: p._collectionsArray?.map(describe),
});
return {
probe: window.__polishFillProbe(),
stats: g.dataManager.layers
.get('military-installations')
.module.getStats(),
primitives: describe(g.viewer.scene.groundPrimitives),
frames: window.__polishSelectionFrames.slice(-5),
};
});
await fs.writeFile(
`${dir}/selection-failure.json`,
JSON.stringify({ picked, ...diagnostic }, null, 2),
);
throw error;
});
await sleep(500);
const selected = await page.evaluate(() => window.__polishFillProbe());
await page.screenshot({ path: `${dir}/after-selection-${site.key}.png` });
// Close, oblique angles expose surface clipping and floating fills.
for (const heading of [30, 120]) {
await page.evaluate(
async (site, heading) => {
const C =
await import('/node_modules/cesium/Build/Cesium/index.js');
const g = window.__godsEyeView;
const target = C.Cartesian3.fromDegrees(
site.lon,
site.lat,
site.key === 'mabry' ? 180 : 10,
);
g.viewer.camera.lookAt(
target,
new C.HeadingPitchRange(
C.Math.toRadians(heading),
C.Math.toRadians(-30),
800,
),
);
g.viewer.camera.lookAtTransform(C.Matrix4.IDENTITY);
},
site,
heading,
);
await sleep(1700);
assert.ok(
(await page.evaluate(() => window.__polishFillProbe())).selected,
'orbit must preserve installation selection',
);
await page.screenshot({
path: `${dir}/after-${site.key}-orbit-${heading}.png`,
});
}
await moveCamera(page, { ...site, pitch: -80 }, 1);
await sleep(900);
const empty = await page.evaluate(() => {
const g = window.__godsEyeView;
for (let y = 250; y < 650; y += 75)
for (let x = 400; x < 850; x += 75) {
const pick = g.viewer.scene.pick({ x, y });
if (!pick?.id?.installationId && !pick?.id?.gevTrackedId)
return { x, y };
}
throw new Error('No empty map point');
});
await page.mouse.click(empty.x, empty.y);
await page.waitForFunction(
() =>
window.__polishFillProbe().selected === null &&
window.__polishFillProbe().entities === 0,
);
await sleep(600);
const deselected = await page.evaluate(() => {
window.__stopPolishSelection();
window.__restorePolishPaint();
return {
...window.__polishFillProbe(),
frames: window.__polishSelectionFrames,
};
});
assert.equal(selected.entities, picked.parts);
assert.equal(selected.primitives, picked.parts);
assert.ok(
selected.classification.every((c) => c === 1),
'Google 3D tile classification',
);
assert.equal(deselected.entities, 0);
assert.equal(deselected.primitives, 0);
const selectedFrames = deselected.frames.filter(
(f) => f.selected && f.own,
);
assert.ok(selectedFrames.length > 10);
assert.ok(
selectedFrames.every((f) => !f.tracked && f.paints === 1),
'selected record has exactly one painted overlay element',
);
assert.ok(
deselected.frames
.slice(-5)
.every((f) => !f.selected && f.own && !f.tracked && f.paints === 1),
'deselect restores the one ambient entry',
);
await page.screenshot({
path: `${dir}/after-${site.key}-deselected.png`,
});
result.push({ site, picked, selected, deselected });
}
} finally {
await fs.writeFile(
`${dir}/selection-checks.json`,
JSON.stringify(result, null, 2),
);
if (recorder)
await fs.writeFile(
`${dir}/selection-video.json`,
JSON.stringify(await recorder.stop(), null, 2),
);
}
return result.map(({ site, picked, selected, deselected }) => ({
site: site.name,
id: picked.id,
selected,
deselected: {
entities: deselected.entities,
primitives: deselected.primitives,
},
frames: deselected.frames.length,
}));
}
+223
View File
@@ -0,0 +1,223 @@
/**
* qa-journey-recorder.mjs — shared support for journey QA gates (not a gate).
*
* - `createScreencast(page, dir)` records CDP `Page.startScreencast` frames
* (every compositor frame) with their real timestamps and builds
* `<dir>/clip.mp4` through ffmpeg's concat demuxer with per-frame
* durations, so held frames last as long as they did on screen.
* - `installFrameProbe(page)` measures every animation frame in the page:
* the requestAnimationFrame interval and, for frames Cesium rendered, the
* CPU time between `scene.preUpdate` and `scene.postRender`.
* - `frameStats(frames)` reduces a probe window to count / mean / p50 / p95.
* - `moveCamera(page, view, seconds)` eases the camera like a drag or wheel
* zoom rather than teleporting it.
*/
import fs from 'node:fs';
import path from 'node:path';
import { execFileSync } from 'node:child_process';
/** Percentile of a numeric list (nearest rank); NaN for an empty list. */
export function percentile(values, p) {
if (!values.length) return Number.NaN;
const sorted = [...values].sort((a, b) => a - b);
return sorted[
Math.min(sorted.length - 1, Math.ceil((p / 100) * sorted.length) - 1)
];
}
/** Summarize probe frames: rAF interval and Cesium render CPU time (ms). */
export function frameStats(frames) {
const intervals = frames.map((f) => f.dt).filter(Number.isFinite);
const renders = frames.map((f) => f.renderMs).filter(Number.isFinite);
const round = (v) => (Number.isFinite(v) ? Math.round(v * 10) / 10 : null);
const mean = (list) =>
list.length ? list.reduce((a, b) => a + b, 0) / list.length : Number.NaN;
return {
frames: intervals.length,
frameMean: round(mean(intervals)),
frameP50: round(percentile(intervals, 50)),
frameP95: round(percentile(intervals, 95)),
longFrames: intervals.filter((v) => v > 50).length,
renderedFrames: renders.length,
renderMean: round(mean(renders)),
renderP50: round(percentile(renders, 50)),
renderP95: round(percentile(renders, 95)),
};
}
/** Install the in-page frame probe once per document; returns nothing. */
export async function installFrameProbe(page) {
await page.evaluate(() => {
if (window.__gevFrameProbe) return;
const { viewer } = window.__godsEyeView;
const probe = { frames: [], started: 0 };
window.__gevFrameProbe = probe;
let last = performance.now();
let renderStart = null;
let pendingRender = null;
viewer.scene.preUpdate.addEventListener(() => {
renderStart = performance.now();
});
viewer.scene.postRender.addEventListener(() => {
if (renderStart !== null) pendingRender = performance.now() - renderStart;
renderStart = null;
});
const tick = (now) => {
probe.frames.push({ t: now, dt: now - last, renderMs: pendingRender });
if (probe.frames.length > 40_000) probe.frames.splice(0, 10_000);
pendingRender = null;
last = now;
requestAnimationFrame(tick);
};
requestAnimationFrame(tick);
});
}
/** Read probe frames with t in [t0, t1) (page performance.now() times). */
export async function readFrames(page, t0, t1 = Infinity) {
return page.evaluate(
(a, b) =>
window.__gevFrameProbe.frames.filter(
(f) => f.t >= a && f.t < (b ?? Infinity),
),
t0,
Number.isFinite(t1) ? t1 : null,
);
}
/** Page clock in ms (performance.now()). */
export function pageNow(page) {
return page.evaluate(() => performance.now());
}
/**
* Ease the camera to a view over `seconds`, resolving when it lands.
* @param {import('puppeteer').Page} page Page.
* @param {{lat:number, lon:number, height:number, heading?:number, pitch?:number}} view
* Destination (degrees, metres).
* @param {number} seconds Flight duration; 0 jumps.
*/
export async function moveCamera(page, view, seconds) {
await page.evaluate(
(v, duration) =>
new Promise((resolve) => {
const { viewer } = window.__godsEyeView;
const C = viewer.camera;
C.cancelFlight();
C.flyTo({
destination: viewer.scene.globe.ellipsoid.cartographicToCartesian({
latitude: (v.lat * Math.PI) / 180,
longitude: (v.lon * Math.PI) / 180,
height: v.height,
}),
orientation: {
heading: ((v.heading ?? 0) * Math.PI) / 180,
pitch: ((v.pitch ?? -45) * Math.PI) / 180,
roll: 0,
},
duration,
// Straight, eased motion like a drag or wheel zoom, not an arc.
maximumHeight: Math.max(v.height, C.positionCartographic.height),
complete: resolve,
cancel: resolve,
});
}),
view,
seconds,
);
}
/**
* Record the page with CDP screencast frames and real per-frame timing.
* @param {import('puppeteer').Page} page Page to record.
* @param {string} dir Output directory (frames + clip.mp4).
* @param {{maxWidth?: number, maxHeight?: number, quality?: number, fps?: number|null}} [options]
*/
export async function createScreencast(page, dir, options = {}) {
const {
maxWidth = 1280,
maxHeight = 800,
quality = 70,
fps = null,
} = options;
const framesDir = path.join(dir, 'frames');
fs.rmSync(framesDir, { recursive: true, force: true });
fs.mkdirSync(framesDir, { recursive: true });
const cdp = await page.createCDPSession();
const frames = [];
let recording = false;
cdp.on('Page.screencastFrame', async (frame) => {
const file = path.join(
framesDir,
`f${String(frames.length).padStart(5, '0')}.jpg`,
);
if (recording) {
fs.writeFileSync(file, Buffer.from(frame.data, 'base64'));
frames.push({ file, timestamp: frame.metadata.timestamp });
}
await cdp
.send('Page.screencastFrameAck', { sessionId: frame.sessionId })
.catch(() => {});
});
return {
frames,
async start() {
recording = true;
await cdp.send('Page.startScreencast', {
format: 'jpeg',
quality,
maxWidth,
maxHeight,
everyNthFrame: 1,
});
},
/** Stop, encode with real timing, and return clip facts. */
async stop() {
recording = false;
await cdp.send('Page.stopScreencast').catch(() => {});
await cdp.detach().catch(() => {});
if (frames.length < 2) return { clip: null, frames: frames.length };
const lines = [];
for (let i = 0; i < frames.length; i++) {
const next = frames[i + 1]?.timestamp;
const duration = next
? Math.min(2, Math.max(0.001, next - frames[i].timestamp))
: 0.05;
lines.push(`file '${path.resolve(frames[i].file)}'`);
lines.push(`duration ${duration.toFixed(4)}`);
}
// The concat demuxer ignores the last duration unless the file repeats.
lines.push(`file '${path.resolve(frames.at(-1).file)}'`);
const list = path.join(dir, 'frames.txt');
fs.writeFileSync(list, `${lines.join('\n')}\n`);
const clip = path.join(dir, 'clip.mp4');
execFileSync('ffmpeg', [
'-y',
'-loglevel',
'error',
'-f',
'concat',
'-safe',
'0',
'-i',
list,
'-fps_mode',
fps ? 'cfr' : 'vfr',
'-c:v',
'libx264',
'-pix_fmt',
'yuv420p',
'-vf',
`scale=trunc(iw/2)*2:trunc(ih/2)*2${fps ? `,fps=${fps}` : ''}`,
clip,
]);
const seconds = frames.at(-1).timestamp - frames[0].timestamp;
return {
clip,
frames: frames.length,
seconds: Math.round(seconds * 10) / 10,
fps: Math.round((frames.length / Math.max(seconds, 0.001)) * 10) / 10,
};
},
};
}
+15 -11
View File
@@ -23,13 +23,17 @@ try {
`${process.env.QA_BASE_URL || 'http://localhost:4173'}/?welcome=0`,
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() => window.__godsEyeView?.dataManager?._layerPanel,
{ timeout: 60000 },
);
await page.waitForFunction(() => window.__godsEyeView?.dataManager, {
timeout: 60000,
});
const results = await page.evaluate(async () => {
const manager = window.__godsEyeView.dataManager;
const container = manager._toggleContainer;
const entry = document.querySelector(
'script[type="module"][src*="/src/main.js"]',
);
const { application } = await import(entry.src);
const { presentation } = application.getComponents().data;
const container = presentation.panel._toggleContainer;
const id = 'qa-panel-lifecycle';
let listener = null;
let enabled = 0;
@@ -54,7 +58,7 @@ try {
});
const result = [];
try {
manager.buildTogglePanel(container);
presentation.mount(container);
const row = () => container.querySelector(`[data-layer-id="${id}"]`);
const first = row().querySelector('.data-toggle-btn');
result.push([
@@ -66,7 +70,7 @@ try {
'row subscription is installed',
typeof listener === 'function',
]);
manager.buildTogglePanel(container);
presentation.mount(container);
first.click();
await Promise.resolve();
result.push([
@@ -97,7 +101,7 @@ try {
]);
await manager.setEnabled(id, true);
const final = row().querySelector('.data-toggle-btn');
manager._layerPanel.destroy();
presentation._panel.destroy();
final.click();
await Promise.resolve();
result.push([
@@ -109,10 +113,10 @@ try {
manager.isEnabled(id),
]);
} finally {
manager._layerPanel?.destroy();
manager._layerPanel = null;
presentation._panel?.destroy();
presentation._panel = null;
await window.__gevQaUnregisterLayer(manager, id);
manager.buildTogglePanel(container);
presentation.mount(container);
}
result.push([
'ordinary layer rows return after reassembly',
+337
View File
@@ -0,0 +1,337 @@
#!/usr/bin/env node
/**
* End-to-end share/reload proof for the first two-character token. The QA
* layer and an exhausted-digit reservation scenario are injected into
* dev-server responses in this isolated browser only. Neither belongs in a
* production bundle or ledger.
*
* Run against a local Vite server: node scripts/qa-layer-token-twochar.mjs
*/
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import puppeteer from 'puppeteer';
import {
LAYER_STATE_TOKEN_RESERVATIONS,
nextLayerStateToken,
parseLayerStateTokenReservations,
validateLayerStateAllocations,
} from '../src/data/layerState.js';
const base = new URL(process.env.QA_BASE_URL || 'http://127.0.0.1:4173');
if (!['127.0.0.1', 'localhost'].includes(base.hostname))
throw new Error('The QA fixture requires a loopback-only dev server');
const origin = base.origin;
const fixtureId = 'qa-twochar-fixture';
const injections = { codec: 0, catalog: 0 };
const errors = [];
const reservations = JSON.parse(
await readFile(
new URL('../src/data/layerStateTokenReservations.json', import.meta.url),
'utf8',
),
);
assert.equal(
reservations.some(([, token]) => token === '00'),
false,
'00 must remain unallocated in the production ledger for this QA fixture',
);
const productionTokens = new Set(reservations.map(([, token]) => token));
const priorDigitRows = [...'0123456789']
.filter((digit) => !productionTokens.has(digit))
.map((digit) => [`qa-prior-digit-${digit}`, digit]);
const priorState = parseLayerStateTokenReservations([
...reservations,
...priorDigitRows,
]);
assert.equal(nextLayerStateToken(priorState), '00');
assert.equal(
validateLayerStateAllocations(
LAYER_STATE_TOKEN_RESERVATIONS,
parseLayerStateTokenReservations([
...reservations,
...priorDigitRows,
[fixtureId, '00'],
]),
),
true,
);
console.log(
'PASS: isolated fixture exhausts prior digits before allocating 00',
);
for (const source of [
'../src/data/layerState.js',
'../src/app/constructCatalog.js',
]) {
assert.equal(
(await readFile(new URL(source, import.meta.url), 'utf8')).includes(
fixtureId,
),
false,
`${fixtureId} must not exist in production source`,
);
}
function replaceOnce(source, from, to, label) {
const first = source.indexOf(from);
if (first < 0 || source.indexOf(from, first + from.length) >= 0)
throw new Error(`QA fixture anchor changed: ${label}`);
return source.replace(from, to);
}
function injectCodec(source) {
const fixtureRows = JSON.stringify([...priorDigitRows, [fixtureId, '00']]);
let result = replaceOnce(
source,
'parseLayerStateTokenReservations(reservationRows)',
`parseLayerStateTokenReservations([...reservationRows, ...${fixtureRows}])`,
'reservation',
);
result = replaceOnce(
result,
']);\n\nexport const REGISTERED_LAYER_IDS',
` Object.freeze({ id: '${fixtureId}', token: '00', disposition: 'enabled-only' }),\n]);\n\nexport const REGISTERED_LAYER_IDS`,
'registry',
);
injections.codec += 1;
return result;
}
function injectCatalog(source) {
const result = replaceOnce(
source,
' createBhoteKoshiEventLayer(),',
` {
id: '${fixtureId}',
name: 'Synthetic two-character token',
icon: '◆',
source: 'QA-only fixture',
async init() { return true; },
async enable() { return true; },
async update() { return true; },
async disable() { return true; },
async destroy() { return true; },
getStats() { return { count: 0 }; },
},
createBhoteKoshiEventLayer(),`,
'catalog',
);
injections.catalog += 1;
return result;
}
const browser = await puppeteer.launch({
headless: true,
args: [
'--no-sandbox',
'--disable-dev-shm-usage',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
],
});
async function newPage() {
const context = await browser.createBrowserContext();
const page = await context.newPage();
page.on('pageerror', (error) => errors.push(error.message));
await page.setViewport({ width: 1280, height: 800 });
await page.setCacheEnabled(false);
await page.evaluateOnNewDocument(() => {
window.__qaRestoreEvents = [];
window.addEventListener('gev:initial-share-restore-settled', (event) => {
window.__qaRestoreEvents.push(event.detail);
});
});
await page.setRequestInterception(true);
page.on('request', async (request) => {
try {
const url = new URL(request.url());
if (url.origin !== origin) return await request.abort();
const transform =
url.pathname === '/src/data/layerState.js'
? injectCodec
: url.pathname === '/src/app/constructCatalog.js'
? injectCatalog
: null;
if (!transform) return await request.continue();
const response = await fetch(request.url());
assert.equal(response.status, 200, `dev module ${url.pathname}`);
await request.respond({
status: 200,
contentType: 'application/javascript',
body: transform(await response.text()),
});
} catch (error) {
errors.push(String(error?.stack || error));
await request.abort().catch(() => {});
}
});
return { context, page };
}
async function load(page, url) {
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60_000 });
await page.waitForFunction(
(id) => window.__godsEyeView?.dataManager?.layers?.has(id),
{ timeout: 60_000 },
fixtureId,
);
}
async function loadedState(page) {
return page.evaluate((id) => {
const manager = window.__godsEyeView.dataManager;
return {
enabled: manager.isEnabled(id),
flights: manager.isEnabled('flights'),
localStorage: localStorage.getItem('gev:layer-state:v2'),
restoreEvents: window.__qaRestoreEvents,
hash: location.hash,
};
}, fixtureId);
}
async function waitForRestore(page) {
await page.waitForFunction(() => window.__qaRestoreEvents.length > 0, {
timeout: 60_000,
});
return loadedState(page);
}
function withLayers(link, layers) {
const url = new URL(link);
const params = new URLSearchParams(url.hash.slice(1));
params.set('l', layers);
url.hash = params.toString();
return url.href;
}
const contexts = [];
try {
const sender = await newPage();
contexts.push(sender.context);
await load(sender.page, `${origin}/?welcome=0`);
const toggled = await sender.page.evaluate(
(id) =>
window.__godsEyeView.dataManager.setEnabled(id, true, { origin: 'user' }),
fixtureId,
);
assert.equal(toggled, true, 'synthetic layer enables through manager');
await sender.page.waitForFunction(
() => new URLSearchParams(location.hash.slice(1)).get('l') === '00',
{ timeout: 15_000 },
);
const sharedUrl = sender.page.url();
const authored = new URLSearchParams(new URL(sharedUrl).hash.slice(1));
assert.equal(authored.get('v'), '2');
assert.equal(authored.get('l'), '00');
assert.ok(authored.has('lat') && authored.has('lon'));
console.log('PASS: real user-origin enable encoded 00 in a full share URL');
const recipient = await newPage();
contexts.push(recipient.context);
await load(recipient.page, sharedUrl);
const restored = await waitForRestore(recipient.page);
assert.equal(restored.enabled, true);
assert.equal(restored.localStorage, null);
assert.equal(restored.restoreEvents.at(-1)?.status, 'settled');
assert.ok(
restored.restoreEvents
.at(-1)
?.layers?.some(
(layer) =>
layer.layerId === fixtureId &&
layer.targetEnabled === true &&
layer.succeeded === true,
),
'share coordinator restored the synthetic layer',
);
console.log(
'PASS: cold recipient reload restored the 00 layer without writing local preferences',
);
await recipient.page.reload({
waitUntil: 'domcontentloaded',
timeout: 60_000,
});
await recipient.page.waitForFunction(
(id) => window.__godsEyeView?.dataManager?.layers?.has(id),
{ timeout: 60_000 },
fixtureId,
);
const reloaded = await waitForRestore(recipient.page);
assert.equal(reloaded.enabled, true);
assert.equal(reloaded.localStorage, null);
assert.equal(new URLSearchParams(reloaded.hash.slice(1)).get('l'), '00');
console.log('PASS: a full recipient reload restores the same 00 layer state');
const disabled = await recipient.page.evaluate(
(id) =>
window.__godsEyeView.dataManager.setEnabled(id, false, {
origin: 'user',
}),
fixtureId,
);
assert.equal(disabled, true);
await recipient.page.waitForFunction(
() => new URLSearchParams(location.hash.slice(1)).get('l') === '',
{ timeout: 15_000 },
);
assert.equal((await loadedState(recipient.page)).enabled, false);
console.log('PASS: explicit OFF removes 00 from the authored share URL');
const legacy = await newPage();
contexts.push(legacy.context);
await load(legacy.page, withLayers(sharedUrl, 'f'));
const legacyState = await waitForRestore(legacy.page);
assert.equal(legacyState.flights, true);
assert.equal(legacyState.enabled, false);
assert.equal(legacyState.restoreEvents.at(-1)?.status, 'settled');
console.log('PASS: a legacy one-character token still restores');
const mixed = await newPage();
contexts.push(mixed.context);
await load(mixed.page, withLayers(sharedUrl, 'f.00'));
const mixedState = await waitForRestore(mixed.page);
assert.equal(mixedState.flights, true);
assert.equal(mixedState.enabled, true);
console.log('PASS: mixed legacy and two-character tokens restore together');
const malformed = await newPage();
contexts.push(malformed.context);
await load(malformed.page, withLayers(sharedUrl, 'f..00'));
const malformedState = await waitForRestore(malformed.page);
assert.equal(malformedState.flights, false);
assert.equal(malformedState.enabled, false);
assert.equal(malformedState.localStorage, null);
console.log(
'PASS: malformed layer list is rejected without partial restoration',
);
const repeated = await newPage();
contexts.push(repeated.context);
const repeatedUrl = new URL(withLayers(sharedUrl, 'f'));
const repeatedParams = new URLSearchParams(repeatedUrl.hash.slice(1));
repeatedParams.append('l', '00');
repeatedUrl.hash = repeatedParams.toString();
await load(repeated.page, repeatedUrl.href);
const repeatedState = await waitForRestore(repeated.page);
assert.equal(repeatedState.flights, false);
assert.equal(repeatedState.enabled, false);
assert.equal(repeatedState.localStorage, null);
console.log('PASS: repeated layer fields reject the entire share payload');
assert.ok(injections.codec >= contexts.length);
assert.ok(injections.catalog >= contexts.length);
assert.deepEqual(errors, []);
console.log(
'PASS: QA-only codec and catalog injections reached every isolated session',
);
console.log(
'Synthetic end-to-end evidence only; production 00 allocation remains pending.',
);
console.log('RESULT: 10 passed, 0 failed, 0 skipped');
} finally {
await Promise.all(contexts.map((context) => context.close().catch(() => {})));
await browser.close();
}
+485
View File
@@ -0,0 +1,485 @@
#!/usr/bin/env node
/** Named military areas: continent performance, close identities and a recorded zoom. */
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import puppeteer from 'puppeteer';
import {
checkInstallationPan,
checkInstallationSelection,
} from './qa-installation-polish.mjs';
import {
createScreencast,
installFrameProbe,
readFrames,
frameStats,
pageNow,
moveCamera,
} from './qa-journey-recorder.mjs';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log('Usage: node scripts/qa-military-names.mjs <url> [--no-video]');
process.exit(0);
}
const url = args.find((a) => /^https?:/.test(a));
if (!url) throw new Error('Supply the running server URL');
const dir = process.env.QA_NAMES_DIR || 'qa-shots/military-names';
await fs.mkdir(dir, { recursive: true });
const result = {
views: [],
sites: [],
errors: [],
forbidden: [],
packs: [],
video: null,
};
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const browser = await puppeteer.launch({
headless: true,
protocolTimeout: 180000,
args: [
'--use-angle=metal',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
let recorder;
try {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
page.on('request', (request) => {
const u = new URL(request.url());
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
result.forbidden.push(u.origin);
if (u.pathname.endsWith('/osm_military_names/names.json'))
result.packs.push(u.pathname);
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
await page.goto(url + (url.includes('?') ? '&' : '?') + 'welcome=0', {
waitUntil: 'domcontentloaded',
});
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
result.page = await page.evaluate(() => ({
url: location.href,
title: document.title,
canvas: Boolean(document.querySelector('#cesiumContainer canvas')),
errorOverlay: Boolean(document.querySelector('vite-error-overlay')),
}));
assert.equal(new URL(result.page.url).origin, new URL(url).origin);
assert.ok(
result.page.title && result.page.canvas && !result.page.errorOverlay,
);
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.styleManager.setDetection({ enabled: false });
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
});
assert.equal(result.packs.length, 0, 'names pack stays unloaded at startup');
result.renderer = await page.evaluate(() => {
const gl = window.__godsEyeView.viewer.scene.context._gl;
const ext = gl.getExtension('WEBGL_debug_renderer_info');
return ext
? gl.getParameter(ext.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER);
});
assert.doesNotMatch(result.renderer, /swiftshader|software/i);
await installFrameProbe(page);
const stats = () =>
page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats(),
);
const settle = async () => {
await sleep(500);
await page.waitForFunction(
() => {
const s = window.__godsEyeView.dataManager.layers
.get('military-installations')
.module.getStats();
return !s.loading && s.count > 0 && s.status === 'ready';
},
{ timeout: 60000 },
);
await sleep(1200);
};
const us = { lon: -100, lat: 39, height: 6500000, pitch: -90 };
await moveCamera(page, us, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('military-installations', true),
);
await settle();
result.credit = await page.evaluate(() => {
const el = document.querySelector('#cesium-credits');
return {
text: el?.innerText || '',
visible: Boolean(el && getComputedStyle(el).display !== 'none'),
};
});
assert.match(result.credit.text, /© OpenStreetMap/);
assert.doesNotMatch(
result.credit.text,
/© OpenMapTiles/,
'wide bundled names use no vector tiles',
);
assert.ok(result.credit.visible);
await page.screenshot({ path: `${dir}/credit.png` });
for (const [name, view] of [
['us', us],
['europe', { lon: 12, lat: 50, height: 4500000, pitch: -90 }],
]) {
await moveCamera(page, view, 2);
await settle();
const before = await stats();
const overlaps = await page.evaluate(async () => {
const { getOverlayPaintRect } =
await import('/src/overlays/worldOverlay.js');
const layer = window.__godsEyeView.dataManager.layers.get(
'military-installations',
).module;
const rects = [];
layer.visitNamedMarkers((id) => {
const rect = getOverlayPaintRect('military-installations', id);
if (rect) rects.push(rect);
});
let overlaps = 0;
for (let a = 0; a < rects.length; a++)
for (let b = a + 1; b < rects.length; b++) {
const x = rects[a],
y = rects[b];
if (
x.x < y.x + y.w &&
y.x < x.x + x.w &&
x.y < y.y + y.h &&
y.y < x.y + x.h
)
overlaps++;
}
return overlaps;
});
// Incumbents retain their placements during camera motion, as for Data Centers.
assert.ok(
before.namedInView > 0 &&
before.namedMarkers > 0 &&
before.pointsOnScreen > 0,
);
assert.ok(
before.labelsOnScreen > 0 && before.labelsOnScreen <= before.labelCap,
);
await page.evaluate(async () => {
// Use the governor's hold so an asynchronous idle transition cannot
// overwrite a raw scene flag during the sustained-render measurement.
const { holdContinuousRender } = await import('/src/renderGovernor.js');
holdContinuousRender('qa-military-names-fps');
});
const t0 = await pageNow(page);
await sleep(3500);
const frames = await readFrames(page, t0);
await page.evaluate(async () => {
const { releaseContinuousRender } =
await import('/src/renderGovernor.js');
releaseContinuousRender('qa-military-names-fps');
});
const metrics = frameStats(frames);
const seconds = frames.reduce((sum, frame) => sum + frame.dt, 0) / 1000;
const fps = metrics.renderedFrames / seconds;
const rafFps = metrics.frames / seconds;
result.views.push({ name, ...before, overlaps, fps, rafFps, ...metrics });
console.log(name, JSON.stringify(result.views.at(-1)));
await page.screenshot({ path: `${dir}/${name}.png` });
assert.ok(fps >= 55, `${name} fps ${fps.toFixed(1)} < 55`);
}
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(true),
);
await sleep(300);
result.cleanUi = await page.evaluate(async () => {
const host = await import('/src/overlays/worldOverlay.js');
const root = document.getElementById('world-overlay-root');
return {
active: document.body.classList.contains('ui-clean-view'),
visible: root.checkVisibility({
visibilityProperty: true,
opacityProperty: true,
}),
labels:
host.getWorldOverlayDiagnostics().paintedBySource[
'military-installations'
] || 0,
};
});
assert.ok(
result.cleanUi.active &&
result.cleanUi.visible &&
result.cleanUi.labels > 0,
);
await page.screenshot({ path: `${dir}/clean-ui.png` });
await page.evaluate(() =>
window.__godsEyeView.styleManager.setCleanView(false),
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'military-installations',
false,
),
);
result.disabled = await page.evaluate(async () => {
const host = await import('/src/overlays/worldOverlay.js');
return host.getWorldOverlayDiagnostics().entriesBySource[
'military-installations'
];
});
assert.equal(result.disabled, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('military-installations', true),
);
await moveCamera(page, us, 1);
await settle();
result.slowPan = await checkInstallationPan(page, dir);
await moveCamera(page, us, 1);
await settle();
if (!args.includes('--no-video')) {
recorder = await createScreencast(page, `${dir}/us-to-san-diego`, {
fps: 24,
});
await recorder.start();
}
await page.evaluate(async () => {
const C = await import('/node_modules/cesium/Build/Cesium/index.js');
const { getOverlayPaintRect } =
await import('/src/overlays/worldOverlay.js');
const g = window.__godsEyeView;
const layer = g.dataManager.layers.get('military-installations').module;
const overlay = document.createElement('div');
overlay.id = 'installation-label-probe';
overlay.style.cssText =
'position:fixed;top:60px;left:360px;z-index:99999;background:#07121cdd;color:#fff;padding:8px 12px;font:13px monospace;pointer-events:none';
document.body.append(overlay);
const screen = new C.Cartesian2();
window.__installationFrames = [];
window.__stopInstallationFrames =
g.viewer.scene.postRender.addEventListener(() => {
const stats = layer.getStats();
const markers = [],
labels = [],
rects = [];
layer.visitNamedMarkers((id, point) => {
const rect = getOverlayPaintRect('military-installations', id);
if (rect) rects.push(rect);
const p = C.SceneTransforms.worldToWindowCoordinates(
g.viewer.scene,
point.position,
screen,
);
if (
point.show &&
p &&
p.x >= 0 &&
p.y >= 0 &&
p.x < 1280 &&
p.y < 800
) {
markers.push(id);
if (rect) labels.push(id);
}
});
let overlaps = 0;
for (let a = 0; a < rects.length; a++)
for (let b = a + 1; b < rects.length; b++) {
const x = rects[a],
y = rects[b];
if (
x.x < y.x + y.w &&
y.x < x.x + x.w &&
x.y < y.y + y.h &&
y.y < x.y + x.h
)
overlaps++;
}
const altitude = g.viewer.camera.positionCartographic.height;
window.__installationFrames.push({
t: performance.now(),
altitude,
markers,
labels,
paintedLabels: stats.labelsOnScreen,
overlaps,
loading: stats.loading,
wide: stats.wide,
});
overlay.textContent = `${(altitude / 1000).toFixed(1)} km | markers ${markers.length} | labels ${labels.length} / 24`;
});
});
const clipStart = await pageNow(page);
await sleep(1200);
for (const view of [
{ lon: -116, lat: 35, height: 1500000, pitch: -85 },
{ lon: -117.2, lat: 33, height: 200000, pitch: -80 },
{ lon: -117.1, lat: 32.86, height: 18000, pitch: -80 },
]) {
await moveCamera(page, view, 3);
await sleep(800);
}
await settle();
await sleep(1500);
result.clipFrames = await readFrames(page, clipStart);
result.labelFrames = await page.evaluate(() => {
window.__stopInstallationFrames();
document.getElementById('installation-label-probe').remove();
return window.__installationFrames;
});
await fs.writeFile(
`${dir}/label-frames.json`,
JSON.stringify(result.labelFrames, null, 2),
);
assert.ok(result.labelFrames.length > 100);
assert.ok(
result.labelFrames.every(
(frame) =>
frame.markers.length > 0 &&
frame.labels.length > 0 &&
frame.paintedLabels > 0 &&
frame.paintedLabels <= 24,
),
'zoom never loses all visible named markers or labels',
);
if (recorder) {
result.video = await recorder.stop();
recorder = null;
}
result.selection = await checkInstallationSelection(page, dir, {
video: !args.includes('--no-video'),
});
const sites = [
['mabry', 'Camp Mabry', -97.765, 30.3125, 3500],
['pendleton', 'Marine Corps Base Camp Pendleton', -117.4, 33.33, 25000],
['miramar', 'Marine Corps Air Station Miramar', -117.14, 32.87, 18000],
['ramstein', 'Ramstein Air Base', 7.6, 49.44, 7000],
['lakenheath', 'RAF Lakenheath', 0.56, 52.41, 7000],
['hood', 'Fort Hood', -97.78, 31.135, 30000],
];
for (const [key, expected, lon, lat, height] of sites) {
await moveCamera(page, { lon, lat, height, pitch: -80 }, 1.5);
await settle();
const snapshot = await page.evaluate(
(lat, lon) => {
const g = window.__godsEyeView;
const layer = g.dataManager.layers.get('military-installations').module;
const position = g.viewer.scene.globe.ellipsoid.cartographicToCartesian(
{
longitude: (lon * Math.PI) / 180,
latitude: (lat * Math.PI) / 180,
height: 0,
},
);
return {
stats: layer.getStats(),
records: layer.getNearby(position, 100000, 1000).map((r) => ({
id: r.id,
name: r.name,
memberNames: r.memberNames || [],
hasPolygon: Boolean(r.footprints?.length),
})),
};
},
lat,
lon,
);
result.sites.push({ key, expected, ...snapshot });
assert.equal(
new Set(snapshot.records.map((r) => r.id)).size,
snapshot.records.length,
);
assert.ok(
snapshot.stats.labelsOnScreen > 0 &&
snapshot.stats.labelsOnScreen <= snapshot.stats.labelCap,
);
assert.ok(
snapshot.records.some((r) => r.name === expected && r.hasPolygon),
`${key}: missing named polygon ${expected}; got ${snapshot.records.map((r) => r.name).join(', ')}`,
);
await page.screenshot({ path: `${dir}/${key}.png` });
console.log(key, expected);
}
await page.evaluate(() => {
if (
document
.getElementById('global-context-panel')
.classList.contains('collapsed')
)
document
.querySelector('[data-collapse-target="global-context-panel"]')
.click();
});
await page.waitForSelector('#global-context-flights-btn', { visible: true });
await page.evaluate(() => {
const button = document.getElementById('global-context-flights-btn');
if (button.getAttribute('aria-selected') !== 'true') button.click();
});
await page.waitForFunction(
() => {
const state = window.__godsEyeView.styleManager.getContextModeState();
return state.mode === 'flights' && !state.changing;
},
{ timeout: 60000 },
);
result.card = await page.evaluate(() => {
const g = window.__godsEyeView;
return g.dataManager.layers
.get('military-installations')
.module.focusById('osm:military:r13529728');
});
assert.equal(result.card, true);
await page.waitForFunction(
() =>
document
.getElementById('military-awareness-panel')
?.textContent.includes('Named areas') &&
document
.getElementById('military-awareness-panel')
.checkVisibility({ visibilityProperty: true, opacityProperty: true }),
{ timeout: 15000 },
);
result.cardText = await page.$eval(
'#military-awareness-panel',
(el) => el.textContent,
);
assert.match(result.cardText, /Fort Hood/);
assert.match(result.cardText, /TA-25/);
await sleep(1800);
await page.$eval(
'#military-awareness-panel .military-awareness-names',
(el) => {
el.open = true;
},
);
await page.screenshot({ path: `${dir}/member-names-card.png` });
assert.equal(result.packs.length, 1, 'single lazy asset fetch');
assert.deepEqual(result.forbidden, []);
assert.deepEqual(result.errors, []);
} finally {
if (recorder)
result.video = await recorder.stop().catch((e) => ({ error: e.message }));
await fs.writeFile(`${dir}/result.json`, JSON.stringify(result, null, 2));
await browser.close();
}
+204
View File
@@ -0,0 +1,204 @@
#!/usr/bin/env node
// Composed-app media policy/lifecycle checks. Pinokio UA is a policy fixture,
// not native-host acceptance; the YouTube SDK clock below is deterministic.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const output = path.resolve(
process.env.QA_OUTPUT_DIR || 'qa-shots/nepal-media',
);
fs.mkdirSync(output, { recursive: true });
const browser = await puppeteer.launch({
headless: process.env.QA_HEADFUL !== '1',
args: process.env.QA_HEADFUL === '1' ? ['--use-angle=metal'] : [],
});
let assertions = 0;
const check = (name, value) => {
assert.ok(value, name);
assertions++;
console.log(`[PASS] ${name}`);
};
try {
for (const pinokio of [true, false]) {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 900 });
if (pinokio)
await page.setUserAgent((await browser.userAgent()) + ' Pinokio/8.0.40');
const popups = [];
page.on('popup', (popup) => popups.push(popup.url()));
// Do not depend on provider availability to prove our startup/trim/exit handoff.
await page.setRequestInterception(true);
page.on('request', (request) => {
if (/youtube(?:-nocookie)?\.com\/embed\//.test(request.url())) {
void request.respond({
status: 200,
contentType: 'text/html',
body: '<p>Player clock fixture</p>',
});
} else void request.continue();
});
await page.evaluateOnNewDocument(() => {
window.__mediaFixture = { starts: [], stops: [] };
window.YT = {
Player: class {
constructor(_frame, { events }) {
this.events = events;
this.began = null;
this.stoppedAt = null;
this.timer = setTimeout(
() => events.onReady({ target: this }),
1200,
);
}
mute() {}
playVideo() {
if (this.began != null) return;
this.began = Date.now();
window.__mediaFixture.starts.push(this.began);
this.events.onStateChange({ target: this, data: 1 });
}
getCurrentTime() {
return this.began == null
? 0
: ((this.stoppedAt || Date.now()) - this.began) / 1000;
}
getPlayerState() {
return this.began == null ? -1 : this.stoppedAt ? 2 : 1;
}
pauseVideo() {
if (this.began != null && !this.stoppedAt) {
this.stoppedAt = Date.now();
window.__mediaFixture.stops.push(this.getCurrentTime());
}
}
destroy() {
clearTimeout(this.timer);
this.pauseVideo();
}
},
};
});
await page.goto(
`${process.env.QA_BASE_URL || 'http://localhost:4173'}/?welcome=0`,
{ waitUntil: 'domcontentloaded' },
);
await page.waitForFunction(
() =>
window.__godsEyeView?.sceneDirector &&
document.querySelector('#loading-screen')?.classList.contains('hidden'),
{ timeout: 60000 },
);
const result = await page.evaluate(async (pinokio) => {
const d = window.__godsEyeView.sceneDirector;
const scene = d._project.scenes.find((s) => /Nepal/.test(s.title));
const shot = scene.shots.find((s) => /Mailung Bazzar/.test(s.title));
const following = scene.shots[scene.shots.indexOf(shot) + 1];
const previous = scene.shots[scene.shots.indexOf(shot) - 1];
const frames = () =>
[...document.querySelectorAll('iframe')].map((f) => f.src);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
await d.loadShot(scene.id, scene.shots[0].id, { flyDuration: 0.1 });
const openingFrames = frames();
await d.loadShot(scene.id, shot.id, { flyDuration: 0.1 });
const passiveFrames = frames();
const started = Date.now();
const run = d.startScene(scene.id, {
single: true,
afterShotId: previous.id,
preview: false,
});
let advanced = null;
while (Date.now() - started < 25000) {
if (d._selectedShotId === following.id) {
advanced = Date.now() - started;
break;
}
if (!d.running) break;
await sleep(100);
}
const runningAtAdvance = d.running;
const clipStops = [...window.__mediaFixture.stops];
const status = document.querySelector('#scene-status')?.textContent;
d.stopScene();
await run;
await sleep(250);
const stoppedFrames = frames();
let replayAutoplay = false;
let replacementSilent = false;
if (!pinokio) {
await d.replayShot(scene.id, shot.id);
replayAutoplay = frames().some((src) => /autoplay=1/.test(src));
await d.loadShot(scene.id, scene.shots[0].id, { flyDuration: 0.1 });
const starts = window.__mediaFixture.starts.length;
await sleep(1600);
replacementSilent =
frames().length === 0 &&
starts === window.__mediaFixture.starts.length;
}
const gl = d.viewer.scene.context._gl;
const info = gl.getExtension('WEBGL_debug_renderer_info');
return {
openingFrames,
passiveFrames,
advanced,
runningAtAdvance,
status,
clipStops,
stoppedFrames,
replayAutoplay,
replacementSilent,
renderer: info
? gl.getParameter(info.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER),
};
}, pinokio);
console.log(JSON.stringify({ pinokioPolicyFixture: pinokio, ...result }));
check(
'opening shot creates no unrelated provider frames',
result.openingFrames.length === 0,
);
check(
'passive LOAD never requests autoplay',
result.passiveFrames.every((src) => !/autoplay=1/.test(src)),
);
check(
'actual next-shot identity reached while scene remains running',
result.advanced > 0 && result.runningAtAdvance,
);
check('no scene timeout', !/timed out|Error:/i.test(result.status || ''));
check('Stop removes provider resources', result.stoppedFrames.length === 0);
check('no automatic external window', popups.length === 0);
if (pinokio) {
check(
'suppressed providers allocate no iframe',
result.passiveFrames.length === 0,
);
check(
'fallback keeps authored dwell',
result.advanced >= 7500 && result.advanced < 18000,
);
} else {
check(
'supported clip follows seven-second source clock',
result.clipStops.some((sec) => sec >= 7 && sec < 7.6),
);
check(
'explicit Play Shot still requests autoplay',
result.replayAutoplay,
);
check(
'replacement disowns pending player startup',
result.replacementSilent,
);
}
await page.screenshot({
path: path.join(output, pinokio ? 'pinokio-policy.png' : 'chrome.png'),
});
await page.close();
}
console.log(`PASS: ${assertions} assertions`);
} finally {
await browser.close();
}
+714
View File
@@ -0,0 +1,714 @@
#!/usr/bin/env node
/**
* Traffic-warm OpenFreeMap outline acceptance.
*
* Each sample opens a fresh application page, loads the Texas Capitol z14 tile
* through Street Traffic, then circles a building from that same tile. It
* records application fetches separately from browser transfer bytes,
* body-read/decode/extraction/install measures, and Long Tasks. A second ask
* on the page proves the outline source's own consumer-cache path. A separate
* page holds traffic's target response while the outline consumer subscribes,
* proving that overlap still produces one application fetch and one decode.
*
* Acceptance is declared before measurement: cold outline decode, extraction,
* and install must fit one 60 Hz frame at p95 (16.7 ms), no sample may create a
* >=50 ms Long Task, and neither the first nor second outline ask may fetch or
* decode the traffic-warm tile again. The target tile's traffic-idle outline
* projection must also fit one frame; unrelated traffic activation Long Tasks
* are reported separately as baseline evidence.
*/
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import path from 'node:path';
import puppeteer from 'puppeteer';
const args = process.argv.slice(2);
const valueAfter = (flag, fallback) => {
const index = args.indexOf(flag);
return index >= 0 && args[index + 1] ? args[index + 1] : fallback;
};
if (args.includes('--help') || args.includes('-h')) {
console.log(
'Usage: node scripts/qa-openfreemap-same-tile.mjs [--url http://localhost:4173] [--runs 5] [--headful] [--out qa-shots/openfreemap-same-tile.json]',
);
process.exit(0);
}
const APP_URL = valueAfter('--url', 'http://localhost:4173');
const RUNS = Number(valueAfter('--runs', '5'));
const OUTPUT = path.resolve(
valueAfter('--out', 'qa-shots/openfreemap-same-tile.json'),
);
const FRAME_BUDGET_MS = 16.7;
const LONG_TASK_MS = 50;
const TARGET = Object.freeze({
latitude: 30.27472,
longitude: -97.74035,
z: 14,
x: 3743,
y: 6745,
});
const TARGET_PATH = `/${TARGET.z}/${TARGET.x}/${TARGET.y}.pbf`;
if (!Number.isInteger(RUNS) || RUNS < 3 || RUNS > 20)
throw new Error('--runs must be an integer from 3 through 20');
const targetUrl = new URL(APP_URL);
if (!['http:', 'https:'].includes(targetUrl.protocol))
throw new Error('--url must be HTTP(S)');
const percentile = (values, p) => {
if (!values.length) return null;
const sorted = values.slice().sort((a, b) => a - b);
return sorted[Math.ceil(p * sorted.length) - 1];
};
const round = (value) =>
value == null ? null : Number(Number(value).toFixed(3));
const measures = (entries, name) =>
entries.filter((entry) => entry.name === name).map((entry) => entry.ms);
const sum = (values) => values.reduce((total, value) => total + value, 0);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
await fs.mkdir(path.dirname(OUTPUT), { recursive: true });
const browser = await puppeteer.launch({
headless: args.includes('--headful') ? false : 'new',
protocolTimeout: 240_000,
...(process.env.PUPPETEER_EXECUTABLE_PATH
? { executablePath: process.env.PUPPETEER_EXECUTABLE_PATH }
: {}),
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--use-gl=angle',
...(args.includes('--headful')
? []
: ['--use-angle=swiftshader', '--enable-unsafe-swiftshader']),
'--disable-dev-shm-usage',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--window-size=1440,1000',
],
});
const report = {
appUrl: APP_URL,
target: TARGET,
targetPath: TARGET_PATH,
acceptance: {
coldCpuP95Ms: FRAME_BUDGET_MS,
maxLongTaskMs: LONG_TASK_MS,
trafficIdleProjectionMaxMs: FRAME_BUDGET_MS,
coldApplicationFetches: 0,
coldOutlineTransferBytes: 0,
coldDecodes: 0,
warmApplicationFetches: 0,
warmDecodes: 0,
overlapApplicationFetches: 1,
overlapDecodes: 1,
},
renderer: null,
samples: [],
errors: [],
};
function relevantMeasures(raw) {
return raw
.filter(
(entry) =>
entry.name.startsWith('roads:') || entry.name.startsWith('outlines:'),
)
.map((entry) => ({
name: entry.name,
ms: round(entry.duration),
at: round(entry.startTime),
detail: entry.detail || {},
}));
}
async function beginOutline(page, label) {
await page.evaluate(
({ label, target }) => {
performance.clearMeasures();
window.__qaOutlineLongTasks.length = 0;
window.__qaOutlineStarted = performance.now();
window.__qaOutlinePromise = Promise.resolve(
window.__gevVoiceCommands.runner('annotate_map', {
annotations: [
{
type: 'area',
target: 'this building',
label,
entityKind: 'building',
latitude: target.latitude,
longitude: target.longitude,
},
],
}),
);
},
{ label, target: TARGET },
);
}
async function settleOutline(page, label) {
return page.evaluate(async (label) => {
const result = await window.__qaOutlinePromise;
let annotation = null;
const deadline = performance.now() + 30_000;
while (performance.now() < deadline) {
annotation = (window.__gevAnnotations?.list?.() || []).find(
(item) => item.label === label,
);
if (annotation && !annotation.pendingOutline) break;
await new Promise((resolve) => setTimeout(resolve, 25));
}
if (!annotation || annotation.pendingOutline)
throw new Error(`${label}: outline did not settle`);
const { scene } = window.__godsEyeView.viewer;
await new Promise((resolve) => {
const timer = setTimeout(resolve, 1500);
const remove = scene.postRender.addEventListener(() => {
clearTimeout(timer);
remove();
resolve();
});
scene.requestRender();
});
await new Promise((resolve) =>
requestAnimationFrame(() => requestAnimationFrame(resolve)),
);
return {
commandToRenderMs: performance.now() - window.__qaOutlineStarted,
result,
annotation: {
pending: Boolean(annotation.pendingOutline),
unavailable: Boolean(annotation.outlineUnavailable),
synthesized: Boolean(annotation.synthesized),
footprintKind: annotation.footprintKind || null,
ringPoints: annotation.ring?.length || 0,
parts: annotation.polygons?.length || (annotation.ring ? 1 : 0),
},
measures: performance.getEntriesByType('measure').map((entry) => ({
name: entry.name,
duration: entry.duration,
startTime: entry.startTime,
detail: entry.detail,
})),
longTasks: window.__qaOutlineLongTasks.slice(),
};
}, label);
}
async function driveOutline(page, label) {
await beginOutline(page, label);
return settleOutline(page, label);
}
try {
for (let run = 0; run <= RUNS; run += 1) {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 1000 });
let overlapMode = false;
let overlapApplicationRequests = 0;
let heldTargetRequest = null;
let resolveHeldTarget;
const heldTarget = new Promise((resolve) => {
resolveHeldTarget = resolve;
});
if (run === 0) {
await page.setRequestInterception(true);
page.on('request', (request) => {
let isTarget = false;
try {
isTarget = new URL(request.url()).pathname.endsWith(TARGET_PATH);
} catch {
// Invalid URLs are passed through for the browser to handle.
}
if (overlapMode && isTarget) {
overlapApplicationRequests += 1;
if (!heldTargetRequest) {
heldTargetRequest = request;
resolveHeldTarget();
return;
}
}
void request.continue();
});
}
await page.evaluateOnNewDocument(() => {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
window.__qaOutlineLongTasks = [];
try {
new PerformanceObserver((list) => {
for (const entry of list.getEntries())
window.__qaOutlineLongTasks.push({
at: entry.startTime,
ms: entry.duration,
});
}).observe({ type: 'longtask', buffered: false });
window.__qaLongTaskSupported = true;
} catch {
window.__qaLongTaskSupported = false;
}
});
let phase = 'startup';
const network = [];
const pending = new Map();
const client = await page.createCDPSession();
await client.send('Network.enable');
client.on('Network.requestWillBeSent', (event) => {
let url;
try {
url = new URL(event.request.url);
} catch {
return;
}
if (
url.hostname !== 'tiles.openfreemap.org' ||
!url.pathname.endsWith('.pbf')
)
return;
const entry = {
requestId: event.requestId,
phase,
url: url.origin + url.pathname,
targetTile: url.pathname.endsWith(TARGET_PATH),
cached: false,
transferBytes: 0,
responseBytes: 0,
finished: false,
};
pending.set(event.requestId, entry);
network.push(entry);
});
client.on('Network.requestServedFromCache', ({ requestId }) => {
const entry = pending.get(requestId);
if (entry) entry.cached = true;
});
client.on('Network.responseReceived', ({ requestId, response }) => {
const entry = pending.get(requestId);
if (!entry) return;
entry.status = response.status;
entry.cached ||= Boolean(
response.fromDiskCache ||
response.fromServiceWorker ||
response.fromPrefetchCache,
);
});
client.on('Network.dataReceived', ({ requestId, dataLength }) => {
const entry = pending.get(requestId);
if (entry) entry.responseBytes += dataLength;
});
client.on('Network.loadingFinished', ({ requestId, encodedDataLength }) => {
const entry = pending.get(requestId);
if (!entry) return;
entry.transferBytes = entry.cached ? 0 : encodedDataLength;
entry.finished = true;
pending.delete(requestId);
});
client.on('Network.loadingFailed', ({ requestId, errorText }) => {
const entry = pending.get(requestId);
if (!entry) return;
entry.failure = errorText;
entry.finished = true;
pending.delete(requestId);
});
const pageErrors = [];
page.on('pageerror', (error) => pageErrors.push(String(error.message)));
const navigation = new URL(APP_URL);
navigation.searchParams.set('welcome', '0');
navigation.searchParams.set('trafficDebug', '1');
navigation.searchParams.set('trafficRoads', 'osm');
await page.goto(navigation.href, {
waitUntil: 'domcontentloaded',
timeout: 90_000,
});
await page.waitForFunction(
() =>
window.__godsEyeView?.viewer &&
window.__godsEyeView?.dataManager &&
window.__gevVoiceCommands?.runner &&
window.__gevAnnotations &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120_000, polling: 250 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.evaluate(() =>
window.__godsEyeView.styleManager.setDetection({ enabled: false }),
);
await page.evaluate(async (target) => {
const { viewer, dataManager } = window.__godsEyeView;
await dataManager.setEnabled('traffic', false);
viewer.camera.cancelFlight();
viewer.camera.setView({
destination: window.__CESIUM__.Cartesian3.fromDegrees(
target.longitude,
target.latitude,
700,
),
orientation: {
heading: 0,
pitch: (-65 * Math.PI) / 180,
roll: 0,
},
});
viewer.camera.moveEnd.raiseEvent();
window.__gevAnnotations.clear();
performance.clearMeasures();
}, TARGET);
await sleep(750);
if (run === 0) {
phase = 'overlap';
overlapMode = true;
await page.evaluate(() => {
window.__qaOutlineLongTasks.length = 0;
performance.clearMeasures();
});
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
await Promise.race([
heldTarget,
sleep(30_000).then(() => {
throw new Error('overlap phase did not intercept the target tile');
}),
]);
await beginOutline(page, 'QA Capitol overlap');
await sleep(100);
await heldTargetRequest.continue();
const overlap = await settleOutline(page, 'QA Capitol overlap');
await sleep(750);
const trafficStats = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('traffic')
.module.getStats(),
);
assert.equal(trafficStats.error, null, 'overlap traffic source error');
overlap.measures = relevantMeasures(overlap.measures);
overlap.network = network.filter(
(entry) => entry.phase === 'overlap' && entry.targetTile,
);
const overlapDecodes = overlap.measures
.filter(
(entry) =>
entry.name === 'roads:decode' &&
String(entry.detail?.key).endsWith(
`:${TARGET.z}/${TARGET.x}/${TARGET.y}`,
),
)
.map((entry) => entry.ms);
assert.equal(
overlapApplicationRequests,
1,
'overlapping traffic and outline consumers issue one application request',
);
assert.equal(
overlap.network.length,
1,
'overlapping traffic and outline consumers emit one target network request',
);
assert.equal(
overlapDecodes.length,
1,
'overlapping traffic and outline consumers decode once',
);
assert.equal(overlap.annotation.unavailable, false);
assert.equal(overlap.annotation.footprintKind, 'building');
report.overlap = {
applicationFetches: overlapApplicationRequests,
targetNetworkRequests: overlap.network.length,
transferBytes: sum(
overlap.network.map((entry) => entry.transferBytes),
),
decodes: overlapDecodes.length,
decodeMs: round(sum(overlapDecodes)),
longTasks: overlap.longTasks.length,
};
await page.close();
continue;
}
phase = 'traffic';
await page.evaluate(() => {
window.__qaOutlineLongTasks.length = 0;
performance.clearMeasures();
});
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
await page.waitForFunction(
() => {
const stats = window.__godsEyeView.dataManager.layers
.get('traffic')
.module.getStats();
return stats.count > 0 && !stats.loading && !stats.error;
},
{ timeout: 180_000, polling: 250 },
);
await sleep(1500);
const trafficMeasures = relevantMeasures(
await page.evaluate(() =>
performance.getEntriesByType('measure').map((entry) => ({
name: entry.name,
duration: entry.duration,
startTime: entry.startTime,
detail: entry.detail,
})),
),
);
const trafficLongTasks = await page.evaluate(() =>
window.__qaOutlineLongTasks.slice(),
);
const trafficTarget = network.filter(
(entry) => entry.phase === 'traffic' && entry.targetTile,
);
const targetTile = `${TARGET.z}/${TARGET.x}/${TARGET.y}`;
const targetProjectionMeasures = trafficMeasures.filter(
(entry) =>
entry.detail?.tile === targetTile &&
['outlines:street-extract', 'outlines:polygon-extract'].includes(
entry.name,
),
);
const targetProjectionMs = sum(
targetProjectionMeasures.map((entry) => entry.ms),
);
assert.ok(
trafficTarget.length >= 1,
`sample ${run}: traffic did not load ${TARGET_PATH}`,
);
assert.equal(
targetProjectionMeasures.length,
2,
`sample ${run}: traffic did not prepare the target outline projection`,
);
assert.ok(
targetProjectionMs <= FRAME_BUDGET_MS,
`sample ${run}: traffic-idle target projection ${targetProjectionMs} ms exceeds ${FRAME_BUDGET_MS} ms`,
);
phase = 'outline-cold';
const cold = await driveOutline(page, `QA Capitol cold ${run}`);
await sleep(750);
cold.measures = relevantMeasures(cold.measures);
cold.network = network.filter(
(entry) => entry.phase === 'outline-cold' && entry.targetTile,
);
cold.longTasks = cold.longTasks.map((entry) => ({
at: round(entry.at),
ms: round(entry.ms),
}));
await page.evaluate(() => window.__gevAnnotations.clear());
phase = 'outline-warm';
const warm = await driveOutline(page, `QA Capitol warm ${run}`);
await sleep(750);
warm.measures = relevantMeasures(warm.measures);
warm.network = network.filter(
(entry) => entry.phase === 'outline-warm' && entry.targetTile,
);
warm.longTasks = warm.longTasks.map((entry) => ({
at: round(entry.at),
ms: round(entry.ms),
}));
const supported = await page.evaluate(() => window.__qaLongTaskSupported);
assert.equal(supported, true, `sample ${run}: Long Tasks unsupported`);
assert.deepEqual(pageErrors, [], `sample ${run}: browser errors`);
assert.equal(cold.annotation.unavailable, false);
assert.equal(cold.annotation.synthesized, false);
assert.equal(cold.annotation.footprintKind, 'building');
assert.ok(cold.annotation.ringPoints >= 4);
assert.equal(warm.annotation.footprintKind, 'building');
assert.ok(warm.annotation.ringPoints >= 4);
const coldDecodes = measures(cold.measures, 'roads:decode');
const coldInstalls = measures(cold.measures, 'outlines:install');
const warmDecodes = measures(warm.measures, 'roads:decode');
assert.equal(
cold.network.length,
0,
'shared decoded tile performs no second application fetch',
);
assert.equal(sum(cold.network.map((entry) => entry.transferBytes)), 0);
assert.equal(coldDecodes.length, 0, 'shared decoded tile is not parsed again');
assert.equal(
measures(cold.measures, 'outlines:street-extract').length,
0,
'traffic idle preparation owns street extraction',
);
assert.equal(
measures(cold.measures, 'outlines:polygon-extract').length,
0,
'traffic idle preparation owns polygon extraction',
);
assert.ok(coldInstalls.length >= 1, 'cold outline installs geometry');
assert.equal(warm.network.length, 0, 'decoded-cache hit performs no fetch');
assert.equal(warmDecodes.length, 0, 'decoded-cache hit performs no decode');
report.renderer ||= await page.evaluate(() => {
const gl = window.__godsEyeView.viewer.scene.context._gl;
const debug = gl.getExtension('WEBGL_debug_renderer_info');
return debug
? gl.getParameter(debug.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER);
});
if (args.includes('--headful')) {
assert.doesNotMatch(
String(report.renderer),
/swiftshader|software rasterizer|llvmpipe/i,
`sample ${run}: headful evidence requires a hardware renderer`,
);
}
report.samples.push({
run,
pageErrors,
traffic: {
applicationFetches: trafficTarget.length,
transferBytes: sum(trafficTarget.map((entry) => entry.transferBytes)),
decodes: measures(trafficMeasures, 'roads:decode').length,
targetProjectionMs: round(targetProjectionMs),
longTasks: trafficLongTasks.map((entry) => ({
at: round(entry.at),
ms: round(entry.ms),
})),
measures: trafficMeasures,
network: trafficTarget,
},
cold: {
...cold,
applicationFetches: cold.network.length,
transferBytes: sum(cold.network.map((entry) => entry.transferBytes)),
decodeMs: round(sum(coldDecodes)),
bodyReadMs: round(sum(measures(cold.measures, 'roads:tile-body-read'))),
streetExtractMs: round(
sum(measures(cold.measures, 'outlines:street-extract')),
),
polygonExtractMs: round(
sum(measures(cold.measures, 'outlines:polygon-extract')),
),
installMs: round(sum(coldInstalls)),
mainThreadCpuMs: round(
sum(coldDecodes) +
sum(measures(cold.measures, 'outlines:street-extract')) +
sum(measures(cold.measures, 'outlines:polygon-extract')) +
sum(coldInstalls),
),
},
warm: {
...warm,
applicationFetches: warm.network.length,
transferBytes: sum(warm.network.map((entry) => entry.transferBytes)),
decodeMs: round(sum(warmDecodes)),
installMs: round(sum(measures(warm.measures, 'outlines:install'))),
},
});
await page.close();
}
const coldCpu = report.samples.map((sample) => sample.cold.mainThreadCpuMs);
const outlineLongTasks = report.samples.flatMap((sample) => [
...sample.cold.longTasks,
...sample.warm.longTasks,
]);
const trafficLongTasks = report.samples.flatMap(
(sample) => sample.traffic.longTasks,
);
report.summary = {
samples: report.samples.length,
overlapApplicationFetches: report.overlap?.applicationFetches ?? null,
overlapDecodes: report.overlap?.decodes ?? null,
coldCpuP50Ms: round(percentile(coldCpu, 0.5)),
coldCpuP95Ms: round(percentile(coldCpu, 0.95)),
coldCpuMaxMs: round(Math.max(...coldCpu)),
coldCommandToRenderP95Ms: round(
percentile(
report.samples.map((sample) => sample.cold.commandToRenderMs),
0.95,
),
),
coldApplicationFetches: sum(
report.samples.map((sample) => sample.cold.applicationFetches),
),
coldTransferBytes: sum(
report.samples.map((sample) => sample.cold.transferBytes),
),
coldDecodes: sum(
report.samples.map(
(sample) => measures(sample.cold.measures, 'roads:decode').length,
),
),
warmApplicationFetches: sum(
report.samples.map((sample) => sample.warm.applicationFetches),
),
warmDecodes: sum(
report.samples.map(
(sample) => measures(sample.warm.measures, 'roads:decode').length,
),
),
trafficIdleProjectionMaxMs: round(
Math.max(
...report.samples.map((sample) => sample.traffic.targetProjectionMs),
),
),
trafficBaselineLongTasks: trafficLongTasks.length,
trafficBaselineMaxLongTaskMs: round(
trafficLongTasks.length
? Math.max(...trafficLongTasks.map((entry) => entry.ms))
: 0,
),
longTasks: outlineLongTasks.length,
maxLongTaskMs: round(
outlineLongTasks.length
? Math.max(...outlineLongTasks.map((entry) => entry.ms))
: 0,
),
};
assert.ok(
report.summary.coldCpuP95Ms <= FRAME_BUDGET_MS,
`cold decode+extract+install p95 ${report.summary.coldCpuP95Ms} ms exceeds ${FRAME_BUDGET_MS} ms`,
);
assert.ok(
report.summary.trafficIdleProjectionMaxMs <= FRAME_BUDGET_MS,
'traffic-idle outline projection exceeded one frame',
);
assert.equal(report.summary.longTasks, 0, 'outline ask created a Long Task');
assert.equal(report.summary.coldTransferBytes, 0);
assert.equal(report.summary.coldApplicationFetches, 0);
assert.equal(report.summary.coldDecodes, 0);
assert.equal(report.summary.warmApplicationFetches, 0);
assert.equal(report.summary.warmDecodes, 0);
assert.equal(report.summary.overlapApplicationFetches, 1);
assert.equal(report.summary.overlapDecodes, 1);
report.passed = true;
console.table(
report.samples.map((sample) => ({
run: sample.run,
trafficPrepMs: sample.traffic.targetProjectionMs,
appFetches: sample.cold.applicationFetches,
transferBytes: sample.cold.transferBytes,
bodyReadMs: sample.cold.bodyReadMs,
decodeMs: sample.cold.decodeMs,
streetMs: sample.cold.streetExtractMs,
polygonMs: sample.cold.polygonExtractMs,
installMs: sample.cold.installMs,
cpuMs: sample.cold.mainThreadCpuMs,
renderMs: round(sample.cold.commandToRenderMs),
longTasks: sample.cold.longTasks.length,
})),
);
console.log('OpenFreeMap same-tile acceptance passed:', report.summary);
} catch (error) {
report.passed = false;
report.failure = String(error?.stack || error);
console.error(error);
process.exitCode = 1;
} finally {
await fs.writeFile(OUTPUT, `${JSON.stringify(report, null, 2)}\n`);
await browser.close();
}
File diff suppressed because it is too large. Load diff
+9 -3
View File
@@ -374,7 +374,7 @@ async function main() {
enabled: true,
uncertain: true,
});
gev.dataManager._refreshTogglePanel();
gev.dataManager.refreshLayerStats();
const staticResult = module.setTuningStatic(true);
const filterBefore = module.getUIState().filter;
const filterControl = document.getElementById('radio-filter');
@@ -447,7 +447,7 @@ async function main() {
enabled: true,
uncertain: false,
});
gev.dataManager._refreshTogglePanel();
gev.dataManager.refreshLayerStats();
module.cancelTuning();
camera.flyTo = originalFlyTo;
return result;
@@ -4048,7 +4048,13 @@ async function main() {
const { getOverlayPaintRect, getWorldOverlayDiagnostics } = await import('/src/overlays/worldOverlay.js');
const expectedEntryId = `selected:${selected.id}`;
let painted = null;
for (let attempt = 0; attempt < 20 && !painted?.rect; attempt += 1) {
// Radio's horizon pass reveals the selected entity on its own timer
// after the camera jump, so a stationary view waits for both to settle.
for (
let attempt = 0;
attempt < 20 && !(painted?.rect && selectedEntity?.show === true);
attempt += 1
) {
painted = await new Promise((resolve) => {
let removePostRender = null;
const timeout = setTimeout(() => {
File diff suppressed because it is too large. Load diff
+22 -4
View File
@@ -154,7 +154,20 @@ try {
const projectFile = path.join(shots, 'project.json');
fs.writeFileSync(projectFile, JSON.stringify(fixture));
const input = await page.$('#scene-import-file');
const beforePreview = await page.evaluate(() =>
JSON.stringify(window.__godsEyeView.sceneDirector._project),
);
await input.uploadFile(projectFile);
await page.waitForSelector('[data-director-apply-import]');
check(
'Import preview leaves the current project untouched',
await page.evaluate(
(saved) =>
JSON.stringify(window.__godsEyeView.sceneDirector._project) === saved,
beforePreview,
),
);
await page.click('[data-director-apply-import]');
await page.waitForFunction(
() =>
document.getElementById('scene-status').textContent ===
@@ -262,8 +275,8 @@ try {
await input.uploadFile(badFile);
await page.waitForFunction(() =>
document
.getElementById('scene-status')
.textContent.includes('invalid JSON'),
.querySelector('[data-director-dialog] [role=status]')
?.textContent.includes('invalid JSON'),
);
check(
'Invalid import reports failure and preserves the current project',
@@ -281,7 +294,9 @@ try {
fs.writeFileSync(futureFile, JSON.stringify({ version: 99, scenes: [] }));
await input.uploadFile(futureFile);
await page.waitForFunction(() =>
document.getElementById('scene-status').textContent.includes('$.version'),
document
.querySelector('[data-director-dialog] [role=status]')
?.textContent.includes('$.version'),
);
check(
'Unsupported versions leave authored state and saved bytes unchanged',
@@ -299,7 +314,9 @@ try {
fs.writeFileSync(malformedFile, JSON.stringify(malformed));
await input.uploadFile(malformedFile);
await page.waitForFunction(() =>
document.getElementById('scene-status').textContent.includes('camera.lat'),
document
.querySelector('[data-director-dialog] [role=status]')
?.textContent.includes('camera.lat'),
);
check(
'Invalid camera field identifies its path without replacing the project',
@@ -308,6 +325,7 @@ try {
savedBefore,
),
);
await page.keyboard.press('Escape');
await page.screenshot({ path: path.join(shots, 'desktop.png') });
await page.setViewport({ width: 390, height: 844 });
await page.screenshot({ path: path.join(shots, 'narrow.png') });
+362
View File
@@ -0,0 +1,362 @@
#!/usr/bin/env node
/**
* Keyless terrain throttling probe.
*
* On the keyless map source the browser fetches Re:Earth quantized-mesh tiles
* (`terrain.reearth.land/cesium-mesh/ellipsoid/{z}/{x}/{y}.terrain`) directly.
* A zoom-out burst asks for a few dozen shallow tiles at once and the upstream
* answers HTTP 429 for most of them. This harness drives that burst, counts the
* throttled responses, keeps the first 429's headers as evidence, and then
* checks whether every throttled tile eventually loaded — the contract the
* client-side retry in `src/maps/terrain.js` provides.
*
* Run against a keyless dev server (no Google or ion credentials):
* env -u GOOGLE_MAPS_API_KEY -u CESIUM_ION_TOKEN npm run dev -- --port 4180
* QA_BASE_URL=http://localhost:4180 node scripts/qa-terrain-429.mjs
*
* Flags: `--mode zoomOut|wheel` (default zoomOut), `--notches N` (default 12),
* `--settle MS` (default 25000), `--rounds N` (default 1), `--inject N`
* (answer the first N tile requests of each burst with a synthetic 429),
* `--headful`.
* Exits 1 when a throttled tile never recovered or a terrain tile is left in
* Cesium's FAILED state.
*/
import fs from 'node:fs';
import puppeteer from 'puppeteer';
const appUrl = process.env.QA_BASE_URL || 'http://localhost:4173';
const argv = process.argv.slice(2);
const flag = (name, fallback) => {
const index = argv.indexOf(`--${name}`);
return index >= 0 && argv[index + 1] !== undefined
? argv[index + 1]
: fallback;
};
const mode = flag('mode', 'zoomOut');
const notches = Number(flag('notches', 12));
const settleMs = Number(flag('settle', 25000));
const rounds = Number(flag('rounds', 1));
// The upstream throttle is load-dependent and does not fire on demand. With
// `--inject N`, the first N tile requests of each burst are answered 429 by an
// XMLHttpRequest shim inside the page so the recovery contract can be proven
// deterministically; real 429s are still counted and kept as evidence. (CDP
// request interception is not used: it stalls Cesium's request scheduler.)
const inject = Number(flag('inject', 0));
const headful = argv.includes('--headful');
const TERRAIN_HOST = 'terrain.reearth.land';
const TILE_PATTERN = /\/cesium-mesh\/ellipsoid\/\d+\/\d+\/\d+\.terrain/;
const executablePath =
process.env.PUPPETEER_EXECUTABLE_PATH ||
(await puppeteer.executablePath().catch(() => null));
if (!executablePath || !fs.existsSync(executablePath)) {
throw new Error('Puppeteer Chrome for Testing is unavailable');
}
const browser = await puppeteer.launch({
headless: headful ? false : 'new',
executablePath,
args: [
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
'--no-sandbox',
],
});
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
/** @type {Map<string, {statuses: number[], failures: number}>} per tile URL. */
const tiles = new Map();
const evidence = [];
let layerJsonSeen = false;
let recording = false;
let requestsInWindow = 0;
let throttledInWindow = 0;
const tileEntry = (url) => {
if (!tiles.has(url)) tiles.set(url, { statuses: [], failures: 0 });
return tiles.get(url);
};
if (inject > 0) {
// Cesium fetches terrain through XMLHttpRequest and reads status, response
// and getAllResponseHeaders() in its onload handler; shadowing those on the
// instance is enough to make it see a throttled reply.
await page.evaluateOnNewDocument((pattern) => {
const tilePattern = new RegExp(pattern);
const shim = { remaining: 0, injected: [] };
window.__qaTerrainThrottle = shim;
const open = XMLHttpRequest.prototype.open;
const send = XMLHttpRequest.prototype.send;
XMLHttpRequest.prototype.open = function (method, url, ...rest) {
this.__qaUrl = String(url);
return open.call(this, method, url, ...rest);
};
XMLHttpRequest.prototype.send = function (...args) {
if (!(shim.remaining > 0 && tilePattern.test(this.__qaUrl || ''))) {
return send.apply(this, args);
}
shim.remaining -= 1;
shim.injected.push(this.__qaUrl);
const xhr = this;
setTimeout(() => {
for (const [key, value] of [
['status', 429],
['readyState', 4],
['response', null],
]) {
Object.defineProperty(xhr, key, { value, configurable: true });
}
xhr.getAllResponseHeaders = () => 'retry-after: 1\r\n';
xhr.onload?.(new Event('load'));
}, 30);
return undefined;
};
}, TILE_PATTERN.source);
}
page.on('response', async (response) => {
const url = response.url();
if (!url.includes(TERRAIN_HOST)) return;
if (url.endsWith('layer.json')) layerJsonSeen = true;
if (!TILE_PATTERN.test(url)) return;
const status = response.status();
tileEntry(url).statuses.push(status);
if (recording) requestsInWindow += 1;
if (status !== 429) return;
if (recording) throttledInWindow += 1;
if (evidence.length < 3) {
let body = '';
try {
body = (await response.text()).slice(0, 160);
} catch {
/* body unavailable */
}
evidence.push({ url, status, headers: response.headers(), body });
}
});
page.on('requestfailed', (request) => {
const url = request.url();
if (!TILE_PATTERN.test(url)) return;
tileEntry(url).failures += 1;
if (recording) requestsInWindow += 1;
});
const consoleErrors = [];
page.on('pageerror', (error) => consoleErrors.push(error.message));
page.on('console', (message) => {
if (message.type() === 'error') consoleErrors.push(message.text());
});
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const waitFor = async (predicate, timeoutMs, label) => {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (await predicate()) return;
await sleep(250);
}
throw new Error(`timed out waiting for ${label}`);
};
const setCamera = (lon, lat, height) =>
page.evaluate(
({ lon, lat, height }) => {
const viewer = window.__godsEyeView.viewer;
// The app's own startup flight would otherwise keep steering the camera
// through the burst and turn the zoom-out into a zoom-in.
viewer.camera.cancelFlight();
const cartographic =
viewer.camera.positionCartographic.constructor.fromDegrees(
lon,
lat,
height,
);
viewer.camera.setView({
destination:
viewer.scene.globe.ellipsoid.cartographicToCartesian(cartographic),
orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 },
});
window.__godsEyeView.requestRender?.();
},
{ lon, lat, height },
);
const cameraHeight = () =>
page.evaluate(
() => window.__godsEyeView.viewer.camera.positionCartographic.height,
);
const globeState = () =>
page.evaluate(() => {
const viewer = window.__godsEyeView.viewer;
const surface = viewer.scene.globe._surface;
let failed = 0;
let loaded = 0;
const visit = (tile) => {
if (!tile) return;
const state = tile.data?.terrainState;
if (state === 0) failed += 1;
else if (state === 6) loaded += 1;
for (const child of tile._children || []) visit(child);
};
for (const tile of surface._levelZeroTiles || []) visit(tile);
return {
failed,
loaded,
tilesLoaded: viewer.scene.globe.tilesLoaded,
provider: viewer.terrainProvider?.constructor?.name,
};
});
const burst = async () => {
if (mode === 'zoomOut') {
for (let i = 0; i < notches; i += 1) {
await page.evaluate(() => {
const viewer = window.__godsEyeView.viewer;
viewer.camera.zoomOut(viewer.camera.positionCartographic.height * 0.6);
window.__godsEyeView.requestRender?.();
});
await sleep(40);
}
return;
}
await page.mouse.move(640, 400);
for (let i = 0; i < notches; i += 1) {
await page.mouse.wheel({ deltaY: 200 });
await page.evaluate(() => window.__godsEyeView.requestRender?.());
await sleep(40);
}
};
try {
await page.goto(appUrl, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForFunction(() => Boolean(window.__godsEyeView?.viewer), {
timeout: 60000,
});
await waitFor(() => layerJsonSeen, 60000, 'the Re:Earth layer.json fetch');
await waitFor(
async () => (await globeState()).provider === 'CesiumTerrainProvider',
30000,
'the keyless terrain provider',
);
const summaries = [];
for (let round = 1; round <= rounds; round += 1) {
// Re-apply the view until it sticks: a still-running startup flight can
// overwrite the first setView on the very next frame.
for (let attempt = 0; attempt < 5; attempt += 1) {
await setCamera(-97.7431, 30.2672, 40000);
await sleep(1500);
const height = await cameraHeight();
if (Math.abs(height - 40000) < 1000) break;
}
await waitFor(
async () => (await globeState()).tilesLoaded,
30000,
'the 40 km view to settle',
).catch(() => {});
await sleep(2000);
const settledHeight = await cameraHeight();
if (Math.abs(settledHeight - 40000) > 1000) {
throw new Error(
`camera did not hold the 40 km start view (at ${Math.round(settledHeight)} m)`,
);
}
const before = await globeState();
tiles.clear();
requestsInWindow = 0;
throttledInWindow = 0;
if (inject > 0) {
await page.evaluate((count) => {
window.__qaTerrainThrottle.remaining = count;
window.__qaTerrainThrottle.injected = [];
}, inject);
}
recording = true;
const startHeight = await cameraHeight();
await burst();
const endHeight = await cameraHeight();
await sleep(settleMs);
recording = false;
// Injected throttles never reach the network, so fold them in from the
// page: a later real 200 for the same tile is the retry succeeding.
const injected =
inject > 0
? await page.evaluate(() => window.__qaTerrainThrottle.injected)
: [];
for (const url of injected) tileEntry(url).statuses.unshift(429);
const throttled = [...tiles.entries()].filter(([, entry]) =>
entry.statuses.includes(429),
);
const recovered = throttled.filter(([, entry]) =>
entry.statuses.includes(200),
);
const unrecovered = throttled.filter(
([, entry]) => !entry.statuses.includes(200),
);
const after = await globeState();
summaries.push({
round,
mode,
notches,
startHeightKm: Math.round(startHeight / 1000),
endHeightKm: Math.round(endHeight / 1000),
tileResponses: requestsInWindow,
throttledResponses: throttledInWindow,
injectedThrottles: injected.length,
throttledTiles: throttled.length,
recoveredTiles: recovered.length,
unrecoveredTiles: unrecovered.length,
failedTilesBefore: before.failed,
failedTilesAfter: after.failed,
loadedTilesAfter: after.loaded,
});
console.log(JSON.stringify(summaries.at(-1)));
if (unrecovered.length) {
console.log(
' unrecovered: ' +
unrecovered
.slice(0, 8)
.map(
([url, entry]) =>
`${new URL(url).pathname} [${entry.statuses.join(',')}]`,
)
.join(' · '),
);
}
}
if (evidence.length) {
console.log('first 429 evidence:');
for (const item of evidence) {
const headers = Object.fromEntries(
Object.entries(item.headers).filter(([name]) =>
/^(cf-|retry-after|server|content-type|date|x-ratelimit|ratelimit|access-control)/i.test(
name,
),
),
);
console.log(
` ${new URL(item.url).pathname} ${JSON.stringify(headers)} body=${JSON.stringify(item.body)}`,
);
}
}
const failures = summaries.filter(
(summary) => summary.unrecoveredTiles > 0 || summary.failedTilesAfter > 0,
);
const terrainErrors = consoleErrors.filter((text) =>
/terrain tile/i.test(text),
);
if (terrainErrors.length) {
console.log(`terrain console errors: ${terrainErrors.length}`);
console.log(` ${terrainErrors[0].slice(0, 200)}`);
}
console.log(
failures.length
? `FAIL — ${failures.length}/${summaries.length} round(s) left throttled terrain tiles unloaded`
: `PASS — every throttled terrain tile recovered in ${summaries.length} round(s)`,
);
process.exitCode = failures.length ? 1 : 0;
} finally {
await browser.close();
}
+533
View File
@@ -0,0 +1,533 @@
#!/usr/bin/env node
/**
* Traffic motion acceptance: real-GPU Puppeteer, stable identities, stationary
* camera holds and timestamped CDP JPEGs encoded with ffmpeg. No tile mocks.
* Run: node scripts/qa-traffic-motion.mjs http://localhost:4186
* --baseline records the unmodified pre-fix implementation (object identities
* via WeakMap, never collection indices); it reports failures without exiting 1.
* --mode=tomtom|osm|hybrid limits a diagnostic run. --tag=name keeps captures.
* --journey follows a neighborhood-to-city route with layer toggles, then
* checks Capitol/greenbelt/parking/corridor views against a 1.5 s limit.
*/
import fs from 'node:fs';
import path from 'node:path';
import { execFileSync } from 'node:child_process';
import puppeteer from 'puppeteer';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-traffic-motion.mjs <url> [--baseline] [--mode=tomtom|osm|hybrid] [--tag=name] [--journey]',
);
process.exit(0);
}
const url = args.find((a) => /^https?:/.test(a));
if (!url) throw new Error('A running dev server URL is required');
const baseline = args.includes('--baseline');
const journey = args.includes('--journey');
const option = (key) =>
args.find((a) => a.startsWith(`--${key}=`))?.split('=')[1];
const modes = option('mode') ? [option('mode')] : ['tomtom', 'osm', 'hybrid'];
if (modes.some((m) => !['tomtom', 'osm', 'hybrid'].includes(m)))
throw new Error('Invalid road mode');
const root = path.resolve(
'qa-shots/motion',
option('tag') || (baseline ? 'before' : '.'),
);
fs.mkdirSync(root, { recursive: true });
const revision = execFileSync('git', ['rev-parse', 'HEAD'], {
encoding: 'utf8',
}).trim();
const browser = await puppeteer.launch({
headless: true,
protocolTimeout: 300000,
args: [
'--use-angle=metal',
'--window-size=1280,800',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
const results = [];
const delay = (ms) => new Promise((r) => setTimeout(r, ms));
try {
for (const mode of modes) {
const dir = path.join(root, mode);
fs.mkdirSync(dir, { recursive: true });
const page = await browser.newPage();
const errors = [],
forbidden = [],
logs = [],
tileRequests = [];
page.on('pageerror', (e) => errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') errors.push(m.text());
if (m.text().startsWith('[Data:Traffic]')) logs.push(m.text());
});
page.on('request', (r) => {
const u = new URL(r.url());
if (
u.hostname === 'tiles.openfreemap.org' &&
/\/\d+\/\d+\/\d+(?:\.pbf)?$/.test(u.pathname)
)
tileRequests.push(u.pathname);
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
forbidden.push(u.hostname);
});
await page.setViewport({ width: 1280, height: 800 });
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
const u = new URL(url);
u.searchParams.set('trafficRoads', mode);
u.searchParams.set('trafficDebug', '1');
await page.goto(u.href, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(() => {
const g = window.__godsEyeView;
g.styleManager.setDetection({ enabled: false });
for (const [id] of g.dataManager.layers)
g.dataManager.setEnabled(id, false);
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
});
const identity = await page.evaluate(() => ({
title: document.title,
text: document.body.innerText.length,
overlay: Boolean(document.querySelector('vite-error-overlay')),
}));
await page.evaluate((baseline) => {
const { viewer, dataManager } = window.__godsEyeView;
const layer = dataManager.layers.get('traffic').module;
if (!baseline && typeof layer.visitMotionDots !== 'function')
throw new Error('Traffic must expose stable dot identities');
const scene = viewer.scene;
const weakIds = new WeakMap();
let nextId = 0,
rebuilds = 0;
const collections = [];
function discoverCollections() {
for (let i = 0; i < scene.primitives.length; i++) {
const p = scene.primitives.get(i);
if (!Array.isArray(p._pointPrimitives) || collections.includes(p))
continue;
collections.push(p);
const removeAll = p.removeAll;
p.removeAll = function () {
if (this.length) rebuilds++;
return removeAll.call(this);
};
}
}
let previous = new Map(),
previousN = 0,
lastRebuilds = 0,
lastTime = 0;
const rows = [];
window.__motion = {
rows,
active: false,
moveEnds: [],
label: 'camera moving',
holdStart: null,
};
const clock = document.createElement('div');
clock.style.cssText =
'position:fixed;bottom:4px;left:38%;padding:3px 8px;background:#000c;color:#fff;font:12px monospace;z-index:999999;pointer-events:none';
document.body.append(clock);
viewer.camera.moveEnd.addEventListener(() =>
window.__motion.moveEnds.push(performance.now()),
);
scene.postRender.addEventListener(() => {
if (!window.__motion.active) return;
discoverCollections();
const now = performance.now(),
points = new Map();
clock.textContent =
`QA: ${window.__motion.label}` +
(window.__motion.holdStart === null
? ''
: ` | camera stopped +${((now - window.__motion.holdStart) / 1000).toFixed(1)}s`);
const visit = (id, roadId, point) => {
if (!point.show) return;
const s = scene.cartesianToCanvasCoordinates(point.position);
if (s && s.x >= 0 && s.y >= 0 && s.x <= 1280 && s.y <= 800)
points.set(id, { x: s.x, y: s.y, roadId });
};
let n = 0,
offRoad = 0;
if (!baseline && layer.visitMotionDots) {
layer.visitMotionDots((id, roadId, point, road) => {
if (point.show) {
const p = road?.roadProperties;
const allowed = road?.directFlow
? [
'Motorway',
'International road',
'Major road',
'Secondary road',
'Connecting road',
'Major local road',
'Local road',
'Minor local road',
].includes(p?.roadType)
: [
'motorway',
'trunk',
'primary',
'secondary',
'tertiary',
'minor',
].includes(p?.class) &&
!p?.service &&
!p?.subclass &&
!['no', 'private'].includes(p?.access) &&
p?.brunnel !== 'tunnel';
if (!allowed) offRoad++;
}
if (point.show) n++;
visit(id, roadId, point);
});
} else {
for (const coll of collections)
if (coll.show)
for (let i = 0; i < coll.length; i++) {
const p = coll.get(i);
if (!weakIds.has(p)) weakIds.set(p, ++nextId);
if (p.show) n++;
visit(weakIds.get(p), null, p);
}
}
let tracked = 0,
jumps = 0,
reassigned = 0,
births = 0,
deaths = 0;
for (const [id, p] of points) {
const prev = previous.get(id);
if (!prev) {
births++;
continue;
}
tracked++;
if (Math.hypot(p.x - prev.x, p.y - prev.y) > 30) jumps++;
if (p.roadId !== prev.roadId) reassigned++;
}
for (const id of previous.keys()) if (!points.has(id)) deaths++;
rows.push({
t: now,
loading: layer.getStats().loading,
syncing:
document
.getElementById('traffic-sync-chip')
?.classList.contains('visible') &&
document.getElementById('traffic-sync-progress')?.textContent ===
'...',
surfacePending: layer.getStats().surfacePending,
offRoad,
dt: now - lastTime,
n,
visible: points.size,
tracked,
jumps,
jumpFrac: tracked ? jumps / tracked : 0,
reassigned,
births,
deaths,
sizeChanged: n !== previousN,
rebuilds: rebuilds - lastRebuilds,
});
previous = points;
previousN = n;
lastRebuilds = rebuilds;
lastTime = now;
});
}, baseline);
const fly = (lat, lon, height, pitch, duration, heading = 20) =>
page.evaluate(
async (v) => {
const { viewer } = window.__godsEyeView;
window.__motion.label = 'camera moving';
window.__motion.holdStart = null;
viewer.camera.cancelFlight();
await new Promise((resolve) =>
viewer.camera.flyTo({
destination: viewer.scene.globe.ellipsoid.cartographicToCartesian(
{
latitude: (v.lat * Math.PI) / 180,
longitude: (v.lon * Math.PI) / 180,
height: v.height,
},
),
orientation: {
heading: (v.heading * Math.PI) / 180,
pitch: (v.pitch * Math.PI) / 180,
roll: 0,
},
duration: v.duration,
complete: resolve,
cancel: resolve,
}),
);
},
{ lat, lon, height, pitch, duration, heading },
);
const cdp = await page.createCDPSession();
let fi = 0;
const frames = [];
cdp.on('Page.screencastFrame', (f) => {
const name = `f${String(fi++).padStart(5, '0')}.jpg`;
fs.writeFileSync(path.join(dir, name), Buffer.from(f.data, 'base64'));
frames.push({ name, timestamp: f.metadata.timestamp });
cdp
.send('Page.screencastFrameAck', { sessionId: f.sessionId })
.catch(() => {});
});
await fly(
...(journey
? [30.25, -97.7497, 700, -40, 0, 10]
: [30.2685, -97.7425, 450, -35, 0]),
);
await cdp.send('Page.startScreencast', {
format: 'jpeg',
quality: 65,
maxWidth: 1280,
maxHeight: 800,
everyNthFrame: journey ? 1 : 3,
});
await page.evaluate(() => {
window.__motion.active = true;
});
const holds = [];
const hold = async (name, warm = false, holdMs = 10000, empty = false) => {
const start = await page.evaluate((label) => {
const start = performance.now();
window.__motion.label = label;
window.__motion.holdStart = start;
return start;
}, name);
await delay(holdMs);
const end = await page.evaluate(() => performance.now());
holds.push({
name,
warm,
start,
end,
empty,
settleMs: journey ? 1500 : warm ? 1000 : 3000,
});
await page.screenshot({ path: path.join(dir, `${name}.png`) });
console.log(`${mode}: captured ${name}`);
};
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
// Match motion-reference: begin the camera-motion scenario from a
// settled street view. Initial mesh acquisition is recorded in the clip
// and first-dot diagnostics, but is not a stationary motion hold.
if (journey) {
await delay(4000);
await hold('south-congress', true, 6000);
const step = async (name, view, ms = 6000, empty = false) => {
await fly(...view);
await hold(name, false, ms, empty);
};
await step('pan-north-400m', [30.2536, -97.7497, 700, -40, 2, 10], 4000);
await step('pan-east-500m', [30.2536, -97.7445, 700, -40, 2, 10], 4000);
await step('zoom-out-2.5km', [30.2536, -97.7445, 2500, -45, 2.5, 10]);
await step(
'downtown-2.5km',
[30.2672, -97.7431, 2500, -45, 2.5, 10],
5000,
);
await step('downtown-500m', [30.2672, -97.7431, 500, -35, 2.5, 20]);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('alpr-cameras', true),
);
await hold('enable-alpr', true, 6000);
await step('pan-west-600m', [30.2672, -97.7494, 500, -35, 2, 20], 5000);
await step('city-12km', [30.28, -97.75, 12000, -60, 3, 0], 8000, true);
await fly(30.3125, -97.765, 1000, -45, 3, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled(
'military-installations',
true,
),
);
await hold('camp-mabry-1km', false, 8000);
await step('camp-mabry-pan', [30.3125, -97.76, 1000, -45, 2, 30], 5000);
await step('greenbelt', [30.312, -97.775, 1000, -55, 3, 0], 6000);
await step('capitol', [30.2747, -97.7404, 500, -70, 3, 0], 6000);
await step('commercial-parking', [30.307, -97.735, 600, -60, 3, 0], 6000);
await step('corridor', [30.347, -97.799, 1000, -45, 3, 0], 6000);
} else {
await delay(12000);
await page.waitForFunction(
() => window.__motion.rows.some((r) => r.n > 0),
{ timeout: 1000 },
);
await hold('street-warm', true);
await fly(30.2685, -97.7425, 3000, -45, 1.5);
await hold('zoom-out');
await fly(30.2685, -97.72, 3000, -45, 1.5);
await hold('pan-east');
await fly(30.2685, -97.725, 600, -35, 1.5);
await hold('zoom-in');
await fly(-23.5505, -46.6333, 1250, -35, 1.5);
await hold('sao-paulo');
await fly(30.2685, -97.7425, 450, -35, 1.5);
await hold('austin-revisit', true);
}
await page.evaluate(() => {
window.__motion.active = false;
});
await cdp.send('Page.stopScreencast');
const rows = await page.evaluate(() => window.__motion.rows);
const surfacePhases = await page.evaluate(() =>
performance
.getEntriesByType('measure')
.filter(
(e) => e.name.startsWith('roads:') || e.name.startsWith('traffic:'),
)
.map((e) => ({
name: e.name,
start: e.startTime,
duration: e.duration,
detail: e.detail,
})),
);
const stats = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers.get('traffic').module.getStats(),
);
const phases = holds.map((h) => {
const all = rows.filter((r) => r.t >= h.start && r.t < h.end);
const settled = all.filter((r) => r.t >= h.start + h.settleMs);
const n = settled.map((r) => r.n);
const lastBusy = all.findLastIndex(
(r) => r.loading || r.syncing || r.sizeChanged || r.jumpFrac > 0.02,
);
// First observed settled frame after the last population/loading event.
const steadyMs =
lastBusy < 0
? 0
: Math.round((all[lastBusy + 1]?.t ?? h.end) - h.start);
const phase = {
mode,
phase: h.name,
steadyMs,
lateLoadingFrames: settled.filter((r) => r.loading).length,
offRoadDots: Math.max(0, ...all.map((r) => r.offRoad)),
settleMs: h.settleMs,
frames: settled.length,
rebuilds: all.reduce((s, r) => s + r.rebuilds, 0),
lateRebuilds: settled.reduce((s, r) => s + r.rebuilds, 0),
jumpFrames: settled.filter((r) => r.jumpFrac > 0.02).length,
reassigned: settled.reduce((s, r) => s + r.reassigned, 0),
sizeChanges: settled.filter((r) => r.sizeChanged).length,
minDots: Math.min(...n),
maxDots: Math.max(...n),
maxFrameMs: Math.round(Math.max(...settled.map((r) => r.dt))),
};
phase.passed =
phase.frames >= 20 &&
phase.rebuilds <= 1 &&
phase.lateRebuilds === 0 &&
phase.jumpFrames === 0 &&
phase.reassigned === 0 &&
phase.sizeChanges === 0 &&
(h.empty ? phase.maxDots === 0 : phase.minDots > 0) &&
(!journey ||
(phase.steadyMs <= 1500 &&
phase.offRoadDots === 0 &&
all.at(-1)?.surfacePending === 0));
return phase;
});
fs.writeFileSync(
path.join(dir, 'frames.ffconcat'),
'ffconcat version 1.0\n' +
frames
.map(
(f, i) =>
`file '${f.name}'\nduration ${Math.max(0.001, Math.min(2, (frames[i + 1]?.timestamp || f.timestamp + 0.1) - f.timestamp))}\n`,
)
.join(''),
);
execFileSync('ffmpeg', [
'-y',
'-loglevel',
'error',
'-safe',
'0',
'-f',
'concat',
'-i',
path.join(dir, 'frames.ffconcat'),
'-fps_mode',
'vfr',
'-c:v',
'libx264',
'-pix_fmt',
'yuv420p',
'-vf',
journey ? 'scale=1280:-2' : 'scale=960:-2',
path.join(dir, 'clip.mp4'),
]);
const result = {
mode,
revision,
baseline,
identity,
url: u.href,
phases,
errors,
forbidden,
tileRequests,
duplicateTiles: tileRequests.filter(
(p, i) => tileRequests.indexOf(p) !== i,
),
stats,
surfacePhases,
logs,
holds,
rows,
moveEnds: await page.evaluate(() => window.__motion.moveEnds),
clip: path.join(dir, 'clip.mp4'),
frames: frames.length,
};
result.passed =
phases.every((p) => p.passed) &&
!errors.length &&
!forbidden.length &&
result.duplicateTiles.length === 0 &&
!identity.overlay &&
identity.text > 100;
fs.writeFileSync(
path.join(dir, 'metrics.json'),
JSON.stringify(result, null, 2),
);
results.push(result);
console.table(phases);
await page.close();
}
} finally {
await browser.close();
fs.writeFileSync(
path.join(root, 'results.json'),
JSON.stringify(
results.map(({ rows, ...r }) => r),
null,
2,
),
);
}
if (!baseline && results.some((r) => !r.passed)) process.exitCode = 1;
+315
View File
@@ -0,0 +1,315 @@
#!/usr/bin/env node
/** Reticle coverage on oblique arrivals: source requests, toggle oracle and zoom revisits. */
import fs from 'node:fs';
import puppeteer from 'puppeteer';
import {
moveCamera,
createScreencast,
installFrameProbe,
} from './qa-journey-recorder.mjs';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-traffic-oblique.mjs <url> [--baseline] [--mode=hybrid|osm|tomtom]',
);
process.exit(0);
}
const baseline = args.includes('--baseline');
const modeArg = args.find((a) => a.startsWith('--mode='))?.split('=')[1];
const modes = modeArg ? [modeArg] : ['tomtom', 'osm', 'hybrid'];
const out = `qa-shots/oblique-${baseline ? 'before' : 'after'}`;
fs.mkdirSync(out, { recursive: true });
const result = { views: [], errors: [], forbidden: [], failures: [] };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const browser = await puppeteer.launch({
headless: true,
protocolTimeout: 180000,
args: [
'--use-angle=metal',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
],
});
try {
for (const mode of modes) {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
const counts = { ofm: 0, tomtom: 0 };
page.on('request', (r) => {
const u = new URL(r.url());
if (
u.hostname === 'tiles.openfreemap.org' &&
/\/\d+\/\d+\/\d+(\.pbf)?$/.test(u.pathname)
)
counts.ofm++;
if (u.pathname.startsWith('/api/tomtom/flow/')) counts.tomtom++;
if (
/overpass/i.test(u.hostname) ||
u.hostname === 'nominatim.openstreetmap.org'
)
result.forbidden.push(u.hostname);
});
page.on('pageerror', (e) => result.errors.push(e.message));
page.on('console', (m) => {
if (m.type() === 'error') result.errors.push(m.text());
});
await page.evaluateOnNewDocument(() =>
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed'),
);
const url = new URL(args.find((a) => /^https?:/.test(a)));
url.searchParams.set('trafficRoads', mode);
await page.goto(url.href, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(
() =>
window.__godsEyeView?.dataManager &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120000 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.keyboard.press('Escape');
await page.evaluate(async () => {
const g = window.__godsEyeView;
for (const [id] of g.dataManager.layers)
await g.dataManager.setEnabled(id, false);
g.styleManager.setDetection({ enabled: false });
g.viewer.scene.screenSpaceCameraController.enableCollisionDetection = false;
});
const snap = () =>
page.evaluate(() => {
const { viewer, dataManager } = window.__godsEyeView;
let upper = 0,
lower = 0,
near = 0;
const camera = viewer.camera.positionWC;
dataManager.layers
.get('traffic')
.module.visitMotionDots((id, key, p) => {
if (!p.show) return;
const s = viewer.scene.cartesianToCanvasCoordinates(p.position);
if (s && s.x >= 240 && s.x <= 1040 && s.y >= 0 && s.y <= 800) {
if (s.y < 400) upper++;
else lower++;
}
});
return {
upper,
lower,
...dataManager.layers.get('traffic').module.getStats(),
};
});
const settle = async () => {
await sleep(700);
await page
.waitForFunction(
() => {
const s = window.__godsEyeView.dataManager.layers
.get('traffic')
.module.getStats();
return !s.loading;
},
{ timeout: 15000 },
)
.catch(() => {});
await sleep(1200);
return snap();
};
for (const [name, lat, lon, height] of [
['dubai', 24.964, 55.037, 276],
['austin', 30.2672, -97.7431, 300],
['london', 51.507, -0.128, 300],
]) {
const before = { ...counts };
await moveCamera(page, { lat, lon, height: 3000, pitch: -45 }, 0);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
await moveCamera(page, { lat, lon, height, pitch: -15 }, 1.4);
const settled = await settle();
const load = {
ofm: counts.ofm - before.ofm,
tomtom: counts.tomtom - before.tomtom,
};
await page.screenshot({ path: `${out}/${name}-${mode}.png` });
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', false),
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
const toggled = await settle();
const oracle =
Math.abs(settled.upper - toggled.upper) / Math.max(1, toggled.upper);
const preZoom = { ...counts };
await moveCamera(page, { lat, lon, height: 3000, pitch: -15 }, 0.7);
await settle();
const atWide = { ...counts };
await moveCamera(page, { lat, lon, height, pitch: -15 }, 0.7);
await settle();
const revisit = {
ofm: counts.ofm - atWide.ofm,
tomtom: counts.tomtom - atWide.tomtom,
};
const row = {
mode,
name,
load,
settled,
toggled,
oracle,
revisit,
zoomOut: {
ofm: atWide.ofm - preZoom.ofm,
tomtom: atWide.tomtom - preZoom.tomtom,
},
};
result.views.push(row);
console.log(JSON.stringify(row));
if (oracle > 0.15)
result.failures.push(`${name}/${mode}: upper oracle ${oracle}`);
if (revisit.ofm || revisit.tomtom)
result.failures.push(`${name}/${mode}: revisit requests`);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', false),
);
}
const locationView = { lat: 24.973, lon: 55.037 };
await moveCamera(
page,
{ lat: 24.964, lon: 55.037, height: 3000, pitch: -45 },
0,
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
const dir = `${out}/dubai-${mode}-journey`;
const recording =
mode === 'hybrid' ? await createScreencast(page, dir, { fps: 24 }) : null;
await installFrameProbe(page);
await page.evaluate(() => {
const { viewer, dataManager } = window.__godsEyeView;
window.__obliqueFrames = [];
window.__obliqueMoving = true;
viewer.scene.postRender.addEventListener(() => {
const s = dataManager.layers.get('traffic').module.getStats();
let upper = 0,
lower = 0,
near = 0;
const camera = viewer.camera.positionWC;
dataManager.layers
.get('traffic')
.module.visitMotionDots((id, key, p) => {
if (!p.show) return;
const q = viewer.scene.cartesianToCanvasCoordinates(p.position);
if (q && q.x >= 240 && q.x <= 1040 && q.y >= 0 && q.y <= 800) {
if (q.y < 400) upper++;
else lower++;
const dx = p.position.x - camera.x,
dy = p.position.y - camera.y,
dz = p.position.z - camera.z;
if (dx * dx + dy * dy + dz * dz <= 1500 * 1500) near++;
}
});
window.__obliqueFrames.push({
t: performance.now(),
moving: window.__obliqueMoving,
count: s.count,
loading: s.loading,
upper,
lower,
near,
publishes: s.motion.publishes,
rebuilds: s.motion.rebuilds,
});
});
});
await recording?.start();
await page.evaluate(async (v) => {
const { flyToLandmark } = await import('/src/locations.js');
await new Promise((resolve) =>
flyToLandmark(window.__godsEyeView.viewer, v.lat, v.lon, {
range: 1000,
pitch: -15,
heading: 0,
duration: 1.6,
buildingHeight: 0,
onComplete: resolve,
onCancel: resolve,
}),
);
window.__obliqueMoving = false;
}, locationView);
const arrived = await settle();
await page.evaluate(() => {
window.__obliqueMoving = true;
});
await moveCamera(
page,
{ lat: 24.964, lon: 55.042, height: 300, pitch: -15 },
1.2,
);
await page.evaluate(() => {
window.__obliqueMoving = false;
});
const panned = await settle();
await sleep(1800);
const frames = await page.evaluate(() => window.__obliqueFrames);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', false),
);
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
const fresh = await settle();
const oracle =
Math.abs(panned.upper - fresh.upper) / Math.max(1, fresh.upper);
const stops = frames.flatMap((f, i) =>
i && !f.moving && frames[i - 1].moving ? [i] : [],
);
const nearSettles = stops.map((start, index) => {
let end = start + 1;
while (end < frames.length && !frames[end].moving) end++;
const phase = frames.slice(start, end),
target = phase.at(-1).near;
const reached = phase.find((f) => f.near >= target * 0.85);
const ms = reached ? Math.round(reached.t - phase[0].t) : Infinity;
if (target && ms > 1100)
result.failures.push(
`dubai-location-pan/${mode}/${index}: near settle ${ms} ms`,
);
return {
phase: index === 0 ? 'arrival' : 'pan',
ms,
target,
first: phase[0].near,
};
});
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(`${dir}/metrics.json`, JSON.stringify(frames));
const clip = await recording?.stop();
result.views.push({
mode,
name: 'dubai-location-pan',
nearSettles,
arrived,
settled: panned,
toggled: fresh,
oracle,
clip,
});
if (oracle > 0.15)
result.failures.push(
`dubai-location-pan/${mode}: upper oracle ${oracle}`,
);
await page.close();
}
} finally {
fs.writeFileSync(`${out}/result.json`, JSON.stringify(result, null, 2));
await browser.close();
}
if (
!baseline &&
(result.failures.length || result.errors.length || result.forbidden.length)
)
process.exitCode = 1;
+411
View File
@@ -0,0 +1,411 @@
#!/usr/bin/env node
/**
* qa-voice-bench.mjs — provider-neutral TEXT-mode voice-routing benchmark.
*
* Every provider gets the same prompt: production's realtimeInstructions()
* and GEV_REALTIME_TOOLS (the Realtime adapter uses the exact session body the
* token handler mints). Two suites, reported separately:
* core the qa-voice-routing phrase table, graded with its semantics
* (plus benchmark-only typed-argument checks)
* coverage the newer-layers coverage probe, graded correct /
* honest-refusal / hallucinated-capability / wrong-tool
* dialogue two-turn dialogues in one context; turn 2 depends on turn 1
* capability area handles / imagery / OSM / point-and-ask phrases (and two
* dialogues) for the tools added after 4b56d0e9
* Coverage rows are graded twice: v1 (original rubric, 4b56d0e9 tool schema)
* and v2 (rubric for the current tool surface).
*
* Methodology: a fresh context per phrase (new Realtime/Live session, or a
* stateless chat request); stub tool results; continuations until the model
* stops calling tools (max 4). No app page is involved.
*
* Usage:
* node scripts/qa-voice-bench.mjs --provider openai-realtime --model gpt-realtime-2 --reps 2
* node scripts/qa-voice-bench.mjs --provider gemini-live --model gemini-3.8-live
* node scripts/qa-voice-bench.mjs --provider ollama --model gemma4:e4b
* node scripts/qa-voice-bench.mjs --provider openai-chat --base http://localhost:1234/v1 --model <id>
* node scripts/qa-voice-bench.mjs --summarize --out qa-shots/voice-bench/<run>
* Common flags: --out <dir> (reuse one dir to collect several models),
* --suite core,coverage --only <substr> --budget <usd, cloud total>
* --variant <label> --temperature <t> --num-ctx <n> --think --note <text>
* --reps <n> --start-rep <k> (add repetitions k..k+n-1 to an existing run)
* --replace (supersede earlier runs of this model for the suites run)
* --compare <run dir> (add a before → after table against another run)
*
* Output (gitignored): <out>/results.jsonl, summary.json, summary.md.
* Keys come from the environment or the macOS Keychain at runtime only.
*/
import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';
import { execFileSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import {
CORE_PHRASES, COVERAGE_PROBE, DIALOGUES, CAPABILITY_EXTRAS,
dialogueResult, coverageRubric,
} from './voice-bench/phrases.mjs';
import { gradeCore, gradeCoverage, gradeDialogue, expectLabel } from './voice-bench/grade.mjs';
import { stubResult } from './voice-bench/stub.mjs';
import { summarize, renderMarkdown, renderComparison } from './voice-bench/report.mjs';
import { GEV_REALTIME_TOOLS, realtimeInstructions } from './voice-bench/productionConfig.mjs';
import { chatNormalizationDiff } from './voice-bench/toolFormats.mjs';
import { createOpenAiRealtimeAdapter } from './voice-bench/adapters/openaiRealtime.mjs';
import { createGeminiLiveAdapter } from './voice-bench/adapters/geminiLive.mjs';
import { createOpenAiChatAdapter, createOllamaAdapter } from './voice-bench/adapters/localChat.mjs';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const argv = process.argv.slice(2);
const opt = (flag, fallback) => {
const i = argv.indexOf(flag);
return i >= 0 && argv[i + 1] !== undefined ? argv[i + 1] : fallback;
};
const has = (flag) => argv.includes(flag);
const PROVIDER = opt('--provider', null);
const MODEL = opt('--model', null);
const CLOUD = PROVIDER === 'openai-realtime' || PROVIDER === 'gemini-live';
const REPS = Number(opt('--reps', CLOUD ? '2' : '1'));
const START_REP = Number(opt('--start-rep', '1'));
const SUITES = opt('--suite', 'core,coverage,dialogue,capability').split(',');
const ONLY = opt('--only', null);
const BUDGET = Number(opt('--budget', '30'));
const VARIANT = opt('--variant', null);
const OUT = path.resolve(ROOT, opt('--out', path.join('qa-shots', 'voice-bench', new Date().toISOString().replace(/[:.]/g, '-'))));
const LEDGER = path.join(ROOT, 'qa-shots', 'voice-bench', 'cloud-spend.json');
fs.mkdirSync(OUT, { recursive: true });
const RESULTS = path.join(OUT, 'results.jsonl');
const NOTES = path.join(OUT, 'notes.json');
function readRows() {
if (!fs.existsSync(RESULTS)) return [];
// Several runs may append to one results file (a cloud run beside a local
// one); skip a line another process is still writing.
return fs.readFileSync(RESULTS, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
try { return [JSON.parse(l)]; } catch { return []; }
});
}
function appendRow(row) {
fs.appendFileSync(RESULTS, `${JSON.stringify(row)}\n`);
}
function readNotes() {
try { return JSON.parse(fs.readFileSync(NOTES, 'utf8')); } catch { return []; }
}
/**
* Re-grade stored turns with the CURRENT phrase tables and grader, so a
* grader fix applies to every model already in the results file. The raw
* calls and text in results.jsonl are the source of truth; `gradeAtRun`
* keeps the verdict recorded at run time.
*/
function regrade(rows) {
const core = new Map(CORE_PHRASES.map((p) => [p.phrase, p]));
const cov = new Map(COVERAGE_PROBE.map((c) => [c.id, c]));
const dia = new Map(DIALOGUES.map((d) => [d.id, d]));
const cap = new Map(CAPABILITY_ITEMS.map((c) => [c.id, c]));
// A run started with --replace supersedes earlier runs of the same
// provider/model/variant for the suites it covers.
const key = (r) => `${r.provider}:${r.model}#${r.variant || ''}`;
const runs = rows.filter((r) => r.type === 'run');
const superseded = new Set();
for (const r of runs.filter((x) => x.replace)) {
for (const older of runs) {
if (older.at < r.at && key(older) === key(r)) superseded.add(`${older.runId}|${(r.suites || []).join(',')}`);
}
}
const isSuperseded = (row) => [...superseded].some((s) => {
const [runId, suites] = s.split('|');
return runId === row.runId && suites.split(',').includes(row.suite);
});
return rows.filter((row) => row.type !== 'turn' || !isSuperseded(row)).map((row) => {
if (row.type !== 'turn') return row;
if (row.suite === 'dialogue' && dia.has(row.phraseId)) {
return { ...row, gradeAtRun: row.grade, grade: gradeDialogue(dia.get(row.phraseId), [row.turn1?.calls || [], row.calls || []]) };
}
if (row.suite === 'core' && core.has(row.phraseId)) {
return { ...row, gradeAtRun: row.grade, grade: gradeCore(core.get(row.phraseId), row.calls || []) };
}
if (row.suite === 'coverage' && cov.has(row.phraseId)) {
const item = cov.get(row.phraseId);
return {
...row,
gradeAtRun: row.grade,
grade: gradeCoverage(item, row.calls || [], row.text, BASELINE_TOOLS),
gradeV2: gradeCoverage(coverageRubric(item, 'v2'), row.calls || [], row.text, GEV_REALTIME_TOOLS),
};
}
if (row.suite === 'capability' && cap.has(row.phraseId)) {
const c = cap.get(row.phraseId);
const grade = c.dialogue ? gradeDialogue(c.dialogue, [row.turn1?.calls || [], row.calls || []]) : gradeCore(c.phrase, row.calls || []);
return { ...row, gradeAtRun: row.grade, grade };
}
return row;
});
}
function readRowsFrom(dir) {
const file = path.join(dir, 'results.jsonl');
if (!fs.existsSync(file)) return [];
return fs.readFileSync(file, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
try { return [JSON.parse(l)]; } catch { return []; }
});
}
function writeSummary() {
const summary = summarize(regrade(readRows()));
// --compare <dir> (remembered in compare-with.txt): a before → after table
// against another run folder, both re-graded with the current grader.
const compareFile = path.join(OUT, 'compare-with.txt');
if (has('--compare')) fs.writeFileSync(compareFile, `${path.resolve(ROOT, opt('--compare', ''))}\n`);
let comparison = '';
if (fs.existsSync(compareFile)) {
const beforeDir = fs.readFileSync(compareFile, 'utf8').trim();
const before = summarize(regrade(readRowsFrom(beforeDir)));
const beforeHead = readRowsFrom(beforeDir).find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '4b56d0e9';
const afterHead = readRows().find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '?';
comparison = renderComparison(before, summary, {
beforeLabel: `${path.relative(ROOT, beforeDir)} (prompt/tools @ ${beforeHead})`,
afterLabel: `${path.relative(ROOT, OUT)} (prompt/tools @ ${afterHead})`,
});
summary.comparison = { beforeDir: path.relative(ROOT, beforeDir), before: before.models.map((m) => ({ key: m.key, coreFull: m.core.fullCallAccuracyPct, coverageV1Pct: m.coverage.acceptablePct, hcV1: m.coverage.counts['hallucinated-capability'], dialogueT2: m.dialogue.turn2FullPct, toolCallP50: m.latencyMs.toolCallP50, promptTokens: m.warmup?.promptTokens ?? m.tokens.inputPerRequestMean })) };
fs.writeFileSync(path.join(OUT, 'comparison.md'), `${comparison}\n`);
}
fs.writeFileSync(path.join(OUT, 'summary.json'), JSON.stringify(summary, null, 2));
const md = renderMarkdown(summary, { notes: readNotes(), payload: payloadChanges() });
fs.writeFileSync(path.join(OUT, 'summary.md'), comparison ? md.replace('## Routing accuracy', `${comparison}\n## Routing accuracy`) : md);
return summary;
}
/** Exactly what each provider receives differently from production's payload. */
function payloadChanges() {
const chat = chatNormalizationDiff(GEV_REALTIME_TOOLS)
.map((c) => `\`${c.path}\`: ${JSON.stringify(c.from)} → ${JSON.stringify(c.to)}`);
return [
`openai-realtime: none — the session body is captured from production's token handler (${GEV_REALTIME_TOOLS.length} tools, instructions, tool_choice auto, reasoning low, truncation). Only difference from a mic session: response.create asks for output_modalities ["text"] (as qa-voice-routing does).`,
'gemini-live: same instructions text as systemInstruction; the same tool schemas passed verbatim as functionDeclarations[].parametersJsonSchema (no keyword changes); tool_choice has no Live equivalent (AUTO is the default); responseModalities AUDIO + outputAudioTranscription (TEXT is rejected by the model).',
`ollama / openai-chat: same instructions as the system message; tools wrapped as {type:"function",function:{name,description,parameters}} with ${chat.length} schema change(s): ${chat.join('; ') || 'none'}. Descriptions, enums and required lists are unchanged; nothing is stringified.`,
];
}
function ledgerSpent() {
try { return JSON.parse(fs.readFileSync(LEDGER, 'utf8')).spentUsd || 0; } catch { return 0; }
}
function ledgerAdd(usd) {
const spent = ledgerSpent() + usd;
fs.mkdirSync(path.dirname(LEDGER), { recursive: true });
fs.writeFileSync(LEDGER, JSON.stringify({ spentUsd: spent, updatedAt: new Date().toISOString() }));
return spent;
}
const sh = (cmd, args) => {
try { return execFileSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000 }).trim(); } catch { return null; }
};
/** Snapshot of what else is using the machine (GPU-heavy servers, load). */
function machineSnapshot() {
const load = os.loadavg().map((x) => x.toFixed(2)).join(' ');
const ollamaPs = sh('ollama', ['ps']);
const procs = sh('ps', ['-Ao', 'pcpu=,rss=,comm=']) || '';
const heavy = procs.split('\n').map((l) => l.trim().split(/\s+/)).filter((p) => Number(p[0]) > 20)
.map((p) => `${path.basename(p.slice(2).join(' '))} ${p[0]}%cpu`).slice(0, 6);
const gpuServers = procs.split('\n').filter((l) => /ollama runner|llama-server|LM Studio|lms|mlx|whisper/i.test(l))
.map((l) => path.basename(l.trim().split(/\s+/).slice(2).join(' '))).slice(0, 6);
return {
at: new Date().toISOString(),
loadavg: load,
ollamaPs,
busyProcesses: heavy,
modelServers: gpuServers,
summary: `load ${load}; busy: ${heavy.join(', ') || 'none'}`,
};
}
// ── Ollama lifecycle ────────────────────────────────────────────────────────
const OLLAMA = opt('--base', 'http://localhost:11434').replace(/\/v1\/?$/, '');
async function ollamaUnloadAll() {
const ps = await fetch(`${OLLAMA}/api/ps`).then((r) => r.json()).catch(() => ({ models: [] }));
for (const m of ps.models || []) {
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: m.name, keep_alive: 0 }) }).then((r) => r.text()).catch(() => null);
}
return (ps.models || []).map((m) => m.name);
}
async function ollamaLoad(model, numCtx) {
const t = Date.now();
const res = await fetch(`${OLLAMA}/api/generate`, {
method: 'POST',
body: JSON.stringify({ model, keep_alive: '30m', options: { num_ctx: numCtx } }),
signal: AbortSignal.timeout(600_000),
});
const body = await res.json().catch(() => ({}));
if (!res.ok || body.error) throw new Error(`load failed: ${body.error || res.status}`);
return { wallMs: Date.now() - t, loadMs: body.load_duration ? body.load_duration / 1e6 : null };
}
// ── Main ────────────────────────────────────────────────────────────────────
function buildAdapter() {
const temperature = Number(opt('--temperature', '0.8'));
switch (PROVIDER) {
case 'openai-realtime':
return createOpenAiRealtimeAdapter({ model: MODEL });
case 'gemini-live':
return createGeminiLiveAdapter({ model: MODEL, schemaMode: opt('--gemini-schema', 'jsonSchema') });
case 'ollama':
return createOllamaAdapter({ model: MODEL, baseUrl: OLLAMA, temperature, numCtx: Number(opt('--num-ctx', '32768')), think: has('--think') });
case 'openai-chat':
return createOpenAiChatAdapter({ model: MODEL, baseUrl: opt('--base', 'http://localhost:1234/v1'), temperature });
default:
throw new Error(`unknown --provider ${PROVIDER}`);
}
}
const COLD_PHRASE = 'Take me to Tokyo';
/** Capability suite: single phrases (optional pointer context) and dialogues. */
const CAPABILITY_ITEMS = CAPABILITY_EXTRAS.map((c) => (c.turns
? { suite: 'capability', id: c.id, text: c.turns.join(' → '), dialogue: c }
: { suite: 'capability', id: c.context ? `${c.phrase} [pointer]` : c.phrase, text: c.phrase, phrase: c, context: c.context || null }));
/** The 4b56d0e9 tool schema, for grading coverage by the original yardstick. */
const BASELINE_TOOLS = JSON.parse(fs.readFileSync(path.join(ROOT, 'scripts', 'voice-bench', 'baseline-tools-4b56d0e9.json'), 'utf8'));
/** What prompt this run actually sent (so before/after runs are identifiable). */
function promptVersion() {
let head = null;
try { head = execFileSync('git', ['-C', ROOT, 'rev-parse', '--short', 'HEAD'], { encoding: 'utf8' }).trim(); } catch { /* not a checkout */ }
return {
gitHead: head,
tools: GEV_REALTIME_TOOLS.length,
toolJsonChars: JSON.stringify(GEV_REALTIME_TOOLS).length,
instructionsChars: realtimeInstructions().length,
};
}
function workList() {
const items = [];
if (SUITES.includes('core')) {
for (const p of CORE_PHRASES) items.push({ suite: 'core', id: p.phrase, text: p.phrase, phrase: p });
}
if (SUITES.includes('coverage')) {
for (const c of COVERAGE_PROBE) items.push({ suite: 'coverage', id: c.id, text: c.utterance, item: c });
}
if (SUITES.includes('dialogue')) {
for (const d of DIALOGUES) items.push({ suite: 'dialogue', id: d.id, text: d.turns.join(' → '), dialogue: d });
}
if (SUITES.includes('capability')) items.push(...CAPABILITY_ITEMS);
return items.filter((w) => !ONLY || w.text.toLowerCase().includes(ONLY.toLowerCase()));
}
async function main() {
if (has('--note')) {
const notes = readNotes();
notes.push(opt('--note', ''));
fs.writeFileSync(NOTES, JSON.stringify(notes, null, 2));
}
if (has('--summarize') || !PROVIDER) {
writeSummary();
console.log(`summary written to ${path.join(OUT, 'summary.md')}`);
return;
}
if (!MODEL) throw new Error('--model is required');
const adapter = buildAdapter();
await adapter.open();
let warmup = null;
if (PROVIDER === 'ollama') {
const unloaded = await ollamaUnloadAll();
const numCtx = Number(opt('--num-ctx', '32768'));
const load = await ollamaLoad(MODEL, numCtx);
// COLD turn: the first request after load pays the full-prompt prefill.
// It is recorded separately (warmup.cold*) and primes the prefix cache
// the way a live session's first turn would; every benchmark turn after
// it is steady state.
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
warmup = { unloaded, loadWallMs: load.wallMs, loadMs: load.loadMs, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
console.log(`warm-up: load ${Math.round(load.wallMs)}ms, prefill ${Math.round(w.local.prefillTokS || 0)} tok/s over ${w.local.promptEvalCount} tokens`);
if (w.errors.length) console.log(` warm-up errors: ${w.errors.join('; ')}`);
} else if (PROVIDER === 'openai-chat') {
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
warmup = { loadMs: Number(opt('--load-ms', 'NaN')) || null, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
console.log(`warm-up: ${w.tDoneMs}ms prefill≈${Math.round(w.local.prefillTokS || 0)} tok/s`);
if (w.errors.length) throw new Error(`warm-up failed, not benchmarking: ${w.errors[0]}`);
}
const runId = `${Date.now().toString(36)}`;
const machine = CLOUD ? null : machineSnapshot();
appendRow({ type: 'run', runId, at: new Date().toISOString(), provider: PROVIDER, model: MODEL, variant: VARIANT, reps: REPS, suites: SUITES, settings: adapter.settings, machine, warmup, replace: has('--replace'), prompt: promptVersion(), methodology: 'fresh context per phrase/dialogue; stub tool results (canned results for dialogue turn 1); ≤4 continuations' });
const items = workList();
let stopReason = null;
let turnCosts = [];
outer: for (let rep = START_REP; rep < START_REP + REPS; rep += 1) {
console.log(`\n== ${PROVIDER} ${MODEL}${VARIANT ? ` (${VARIANT})` : ''} — rep ${rep}/${REPS}, ${items.length} phrases`);
for (const w of items) {
if (CLOUD) {
const avg = turnCosts.length ? turnCosts.reduce((a, b) => a + b, 0) / turnCosts.length : 0.1;
if (ledgerSpent() + avg > BUDGET) { stopReason = `cloud budget $${BUDGET} reached (ledger $${ledgerSpent().toFixed(2)})`; break outer; }
}
let r;
let turn1 = null;
let grade;
if (w.dialogue) {
const rs = await adapter.runDialogue({
turns: w.dialogue.turns,
onCall: (c, turn) => dialogueResult(w.dialogue, c.name, turn) ?? stubResult('core', c, GEV_REALTIME_TOOLS),
});
const [a, b] = rs;
turn1 = { text: a.text.slice(0, 400), calls: a.calls, tFirstCallMs: a.tFirstCallMs, tDoneMs: a.tDoneMs, errors: a.errors };
r = {
...b,
errors: [...a.errors, ...b.errors],
timeout: a.timeout || b.timeout,
costUsd: a.costUsd + b.costUsd,
usage: Object.fromEntries(Object.keys(b.usage).map((k) => [k, (a.usage[k] || 0) + (b.usage[k] || 0)])),
setupMs: a.setupMs,
};
grade = gradeDialogue(w.dialogue, [a.calls, b.calls]);
} else {
r = await adapter.runTurn({ text: w.text, context: w.context || undefined, onCall: (c) => stubResult(w.suite, c, GEV_REALTIME_TOOLS) });
grade = w.suite === 'coverage' ? gradeCoverage(w.item, r.calls, r.text, GEV_REALTIME_TOOLS) : gradeCore(w.phrase, r.calls);
}
const row = {
type: 'turn', runId, provider: PROVIDER, model: MODEL, variant: VARIANT, cloud: CLOUD, rep, suite: w.suite,
phraseId: w.id, phrase: w.text,
expect: w.dialogue ? `t1 ${expectLabel(w.dialogue.first)} → t2 ${expectLabel(w.dialogue.second)}` : w.suite === 'coverage' ? w.item.expect : expectLabel(w.phrase),
context: w.context || undefined,
expectNone: Boolean(w.phrase?.expectNone), supported: w.item ? w.item.supported : undefined,
calls: r.calls, text: r.text.slice(0, 800), grade, turn1: turn1 || undefined,
tFirstTokenMs: r.tFirstTokenMs, tFirstCallMs: r.tFirstCallMs, tDoneMs: r.tDoneMs, setupMs: r.setupMs,
responses: r.responses, usage: r.usage, costUsd: r.costUsd, errors: r.errors, timeout: r.timeout,
local: r.local || undefined,
at: new Date().toISOString(),
};
appendRow(row);
if (CLOUD) { ledgerAdd(r.costUsd); turnCosts.push(r.costUsd); }
const verdict = w.suite === 'coverage' ? grade.verdict : (grade.ok ? 'PASS' : 'FAIL');
console.log(` [${verdict}] ${w.text} → ${turn1 ? `${turn1.calls.map((c) => c.name).join(',') || '(none)'} ⇒ ` : ''}${r.calls.map((c) => c.name).join(',') || '(none)'} tool@${r.tFirstCallMs ?? '—'}ms done@${r.tDoneMs}ms $${r.costUsd.toFixed(4)}${r.errors.length ? ` ERR ${r.errors[0]}` : ''}${r.timeout ? ' TIMEOUT' : ''}`);
if (r.errors.some((e) => /HTTP 4\d\d|close 100[78]|invalid|not found|model_not_found|does not exist/i.test(e)) && r.calls.length === 0 && !r.text) {
const same = readRows().filter((x) => x.type === 'turn' && x.runId === runId).slice(-3);
if (same.length === 3 && same.every((x) => x.errors?.length && !x.calls.length)) { stopReason = `3 consecutive hard errors: ${r.errors[0]}`; break outer; }
}
}
writeSummary();
}
await adapter.close();
if (PROVIDER === 'ollama' && !has('--keep-loaded')) {
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: MODEL, keep_alive: 0 }) }).catch(() => null);
}
if (stopReason) {
console.log(`STOPPED: ${stopReason}`);
appendRow({ type: 'stop', runId, provider: PROVIDER, model: MODEL, reason: stopReason, at: new Date().toISOString() });
}
const summary = writeSummary();
const mine = summary.models.find((m) => m.provider === PROVIDER && m.model === MODEL && (m.variant || null) === (VARIANT || null));
if (mine) console.log(`\n${mine.key}: core ${mine.core.accuracyPct}% | coverage acceptable ${mine.coverage.acceptablePct}% | p50 tool ${mine.latencyMs.toolCallP50}ms | $${mine.costUsd}`);
console.log(`results: ${RESULTS}\nsummary: ${path.join(OUT, 'summary.md')}\ncloud ledger: $${ledgerSpent().toFixed(3)}`);
}
main().catch((e) => {
console.error(e);
process.exitCode = 1;
});
File diff suppressed because it is too large. Load diff
+423
View File
@@ -0,0 +1,423 @@
#!/usr/bin/env node
/**
* qa-weather-journey — drive the Weather layers the way a person would and
* keep the evidence: numbered screenshots plus per-step layer diagnostics.
*
* Starts at the Earth view, opens Data Layers, switches Wind on from its real
* row button, cycles the wind fields/models, switches radar, satellite clouds,
* lightning and cyclones on, follows "View storm" and "View US radar", zooms
* into a regional view, a city view and a street view, replays history, and
* ends with real wheel/drag input. Every step records the enabled layers'
* diagnostics (imagery tiles loaded, shown observation time, wind geometry)
* and a 2 s `scene.postRender` count so a stalled renderer cannot pass as a
* healthy screenshot.
*
* Usage:
* node scripts/qa-weather-journey.mjs [--url http://localhost:4173]
* [--label google] [--headless] [--out qa-shots/weather-journey]
*
* Requires a running dev server. Screenshots and steps.json land in
* <out>/<label>/.
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg('--label', 'journey');
const outDir = path.resolve(arg('--out', 'qa-shots/weather-journey'), label);
mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const WEATHER = ['weather-radar', 'weather-satellite', 'weather-lightning'];
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
),
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--window-size=1400,1000',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const steps = [];
let index = 0;
try {
const page = await browser.newPage();
await page.setViewport({ width: 1400, height: 920, deviceScaleFactor: 2 });
page.on('pageerror', (e) => console.log(' [pageerror]', e.message));
// Non-2xx responses, counted per path so a burst of throttled tiles reads
// as one line instead of one console error per tile.
const failures = new Map();
page.on('response', (r) => {
const status = r.status();
if (status < 400) return;
const key = `${status} ${new URL(r.url()).pathname}`;
failures.set(key, (failures.get(key) || 0) + 1);
});
const reportFailures = () => {
if (!failures.size) return;
console.log(
' [http failures] ' +
[...failures].map(([k, n]) => `${k} ×${n}`).join(' · '),
);
failures.clear();
};
page.on('console', (m) => {
if (m.type() === 'error')
console.log(' [console.error]', m.text().slice(0, 200));
});
// The Data Layers panel remembers its collapsed state; start it open.
await page.evaluateOnNewDocument(() => {
try {
localStorage.setItem('godsEyeView.v6.panelCollapsed.data-panel', '0');
} catch {
/* storage may be unavailable */
}
});
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(12_000);
const diagnostics = () =>
page.evaluate(
({ WEATHER }) => {
const gev = window.__godsEyeView;
const dm = gev.dataManager;
const out = {
basemap: gev.tileset ? 'google-3d' : 'keyless-globe',
globeShown: gev.viewer.scene.globe.show,
heightM: Math.round(gev.viewer.camera.positionCartographic.height),
summaryHeight: Math.round(
document.querySelector('.weather-summary')?.getBoundingClientRect()
.height ?? 0,
),
};
for (const id of ['wind', ...WEATHER, 'weather-cyclones']) {
const entry = dm.layers.get(id);
if (!entry?.enabled) continue;
const d = entry.module?.getDiagnostics?.() || {};
const s = entry.module?.getStats?.() || {};
out[id] =
id === 'wind'
? {
mode: d.renderMode,
ready: d.gpu?.ready,
paths: d.gpu?.pathCount,
visibleCells: d.gpu?.visibleCells,
heightFade: d.gpu?.heightFade,
overlay: d.overlay,
imagery: d.imageryActive,
error: s.error,
}
: id === 'weather-cyclones'
? { count: s.count, error: s.error }
: {
time: d.time,
playing: d.playing,
loading: d.loading,
loadedTiles: d.loadedTiles,
frameLoadMs: d.frameLoadMs && Math.round(d.frameLoadMs),
error: d.error,
};
}
return out;
},
{ WEATHER },
);
const renders = (ms = 2000) =>
page.evaluate(
(ms) =>
new Promise((resolve) => {
const scene = window.__godsEyeView.viewer.scene;
let n = 0;
const off = scene.postRender.addEventListener(() => {
n += 1;
});
setTimeout(() => {
off();
resolve(n);
}, ms);
}),
ms,
);
const shot = async (name, note = '') => {
index += 1;
const file = `${String(index).padStart(2, '0')}-${name}.png`;
const d = await diagnostics();
const r = await renders(2000);
await page.screenshot({ path: path.join(outDir, file) });
reportFailures();
steps.push({ index, name, note, file, renders2s: r, ...d });
console.log(` ${file} renders/2s=${r} ${note} ${JSON.stringify(d)}`);
};
const clickRow = async (id) => {
const ok = await page.evaluate((id) => {
const btn = document.querySelector(
`[data-layer-id="${id}"] .data-toggle-btn`,
);
if (!btn) return false;
btn.scrollIntoView({ block: 'center' });
btn.click();
return true;
}, id);
if (!ok) console.log(` !! no row button for ${id}`);
};
const clickChip = async (id, chip) => {
const ok = await page.evaluate(
({ id, chip }) => {
const btn = document.querySelector(
`[data-layer-id="${id}"] [data-chip-id="${chip}"]`,
);
if (!btn || btn.disabled) return false;
btn.scrollIntoView({ block: 'center' });
btn.click();
return true;
},
{ id, chip },
);
if (!ok) console.log(` !! chip ${id}/${chip} missing or disabled`);
return ok;
};
const waitReady = (ids, timeout = 90_000) =>
page
.waitForFunction(
(ids) => {
const dm = window.__godsEyeView.dataManager;
for (const id of ids) {
const entry = dm.layers.get(id);
if (!entry?.enabled) return false;
const d = entry.module?.getDiagnostics?.() || {};
if (id === 'wind') {
if (d.renderMode === 'gpu-streamlines' && !d.gpu?.ready)
return false;
} else if (id !== 'weather-cyclones') {
if (d.loading || (!d.time && !d.error)) return false;
}
}
return true;
},
{ timeout, polling: 250 },
ids,
)
.catch(() =>
console.log(` !! ${ids.join(',')} not ready within ${timeout} ms`),
);
const fly = async (
lon,
lat,
height,
pitch = -90,
heading = 0,
duration = 2,
) => {
await page.evaluate(
({ lon, lat, height, pitch, heading, duration }) => {
const v = window.__godsEyeView.viewer;
const d2r = Math.PI / 180;
v.camera.cancelFlight();
v.camera.flyTo({
destination: v.scene.ellipsoid.cartographicToCartesian({
longitude: lon * d2r,
latitude: lat * d2r,
height,
}),
orientation: { heading: heading * d2r, pitch: pitch * d2r, roll: 0 },
duration,
});
},
{ lon, lat, height, pitch, heading, duration },
);
await sleep(duration * 1000 + 3000);
};
console.log(`\nqa-weather-journey · ${label} · ${url}`);
await page.evaluate(async () => {
const gev = window.__godsEyeView;
for (const [id, entry] of gev.dataManager.layers)
if (entry.enabled)
await gev.dataManager.setEnabled(id, false, { origin: 'user' });
document.querySelector('#data-panel')?.classList.remove('collapsed');
});
// 1. Earth view, clean.
await fly(-97.7431, 30.2672, 17_368_000);
await shot('globe-clean', 'Earth view, no layers');
// 2. Wind from its row, then each field, then ECMWF.
await clickRow('wind');
await waitReady(['wind']);
await sleep(1500);
await shot('globe-wind-trails', 'Wind on (trails, no field)');
await clickChip('wind', 'overlay-speed');
await sleep(2500);
await shot('globe-wind-speed', 'Wind speed field');
await clickChip('wind', 'overlay-temperature');
await sleep(2500);
await waitReady(['wind']);
await shot('globe-wind-temperature', 'Wind + 2 m temperature');
await clickChip('wind', 'overlay-pressure');
await sleep(2500);
await waitReady(['wind']);
await shot('globe-wind-pressure', 'Wind + MSL pressure');
await clickChip('wind', 'model-ifs');
await waitReady(['wind']);
await sleep(2500);
await shot('globe-wind-ecmwf-pressure', 'ECMWF IFS model');
await clickChip('wind', 'model-gfs');
await clickChip('wind', 'overlay-speed');
await waitReady(['wind']);
await sleep(1500);
// 3. Observed products.
for (const id of WEATHER) await clickRow(id);
await clickRow('weather-cyclones');
await waitReady([...WEATHER]);
await sleep(2500);
await shot(
'globe-all-weather',
'Wind speed + radar + regional IR + lightning + cyclones',
);
await clickChip('weather-satellite', 'clouds');
await waitReady(['weather-satellite']);
await sleep(2500);
await shot('globe-global-infrared', 'Satellite: global infrared mosaic');
await clickChip('weather-satellite', 'clouds-regional');
await waitReady(['weather-satellite']);
// 4. Follow a storm if there is one (advisories arrive a few seconds after enable).
await page
.waitForFunction(
() =>
!!document.querySelector(
'[data-layer-id="weather-cyclones"] .data-toggle-chip:not([data-chip-id="focus"]):not([data-chip-id="advisory"])',
),
{ timeout: 25_000, polling: 500 },
)
.catch(() => {});
const storm = await page.evaluate(() => {
const first = document.querySelector(
'[data-layer-id="weather-cyclones"] .data-toggle-chip:not([data-chip-id="focus"]):not([data-chip-id="advisory"])',
);
if (!first) return null;
first.click();
return first.textContent.trim();
});
if (storm) {
await sleep(1500);
if (await clickChip('weather-cyclones', 'focus')) {
await sleep(6000);
await shot('storm-view', `View storm: ${storm}`);
await page.evaluate(() =>
window.__godsEyeView.viewer.camera.zoomIn(400_000),
);
await sleep(4000);
await shot('storm-closer', 'Zoomed toward the storm');
}
} else console.log(' (no active cyclone advisories to follow)');
// 5. US radar, then regional → city → street over the same spot.
await clickChip('weather-radar', 'coverage');
await sleep(6000);
await shot('us-radar', 'View US radar');
await fly(-94.6, 37.5, 600_000);
await shot('regional-600km', 'Regional 600 km');
await fly(-94.6, 37.5, 120_000);
await shot('regional-120km', 'Regional 120 km');
await fly(-97.7431, 30.2672, 12_000, -60, 20);
await shot('city-12km', 'Austin 12 km oblique');
await fly(-97.7431, 30.2672, 1_200, -45, 20);
await shot('city-1200m', 'Austin 1.2 km oblique (the perf route pose)');
await fly(-97.7431, 30.2672, 350, -30, 60);
await shot('street-350m', 'Austin 350 m street level');
// 6. History playback at the city pose (panel behavior while frames swap).
await clickChip('weather-radar', 'play');
const heights = [];
for (let i = 0; i < 4; i++) {
await sleep(2200);
heights.push(
await page.evaluate(() =>
Math.round(
document.querySelector('.weather-summary')?.getBoundingClientRect()
.height ?? 0,
),
),
);
}
await shot(
'city-history-playing',
`History playing; summary heights ${heights.join('/')}`,
);
await clickChip('weather-radar', 'play');
await clickChip('weather-radar', 'latest');
// 7. Real input: wheel-zoom out, drag, wheel-zoom in.
await fly(-97.7431, 30.2672, 40_000, -55, 0, 1.5);
const canvas = await page.$(
'canvas.cesium-widget-canvas, .cesium-widget canvas',
);
const box = canvas
? await canvas.boundingBox()
: { x: 700, y: 460, width: 0, height: 0 };
const cx = box.x + (box.width || 0) / 2;
const cy = box.y + (box.height || 0) / 2;
await page.mouse.move(cx, cy);
for (let i = 0; i < 12; i++) {
await page.mouse.wheel({ deltaY: 240 });
await sleep(120);
}
await sleep(3000);
await shot('wheel-out', 'Wheel zoom out ×12');
await page.mouse.move(cx - 200, cy);
await page.mouse.down();
for (let i = 1; i <= 20; i++) {
await page.mouse.move(cx - 200 + i * 20, cy + i * 4, { steps: 2 });
await sleep(30);
}
await page.mouse.up();
await sleep(2500);
await shot('drag-pan', 'Drag pan');
for (let i = 0; i < 10; i++) {
await page.mouse.wheel({ deltaY: -240 });
await sleep(120);
}
await sleep(4000);
await shot('wheel-in', 'Wheel zoom in ×10');
// 8. Back to Earth with everything on.
await fly(-97.7431, 30.2672, 17_368_000, -90, 0, 2.5);
await shot('globe-return', 'Return to Earth view');
writeFileSync(
path.join(outDir, 'steps.json'),
JSON.stringify({ label, url, steps }, null, 2),
);
console.log(`\nWrote ${steps.length} steps to ${outDir}`);
} finally {
await browser.close();
}
+568
View File
@@ -0,0 +1,568 @@
#!/usr/bin/env node
/**
* qa-weather-perf — matched-route throughput and idle-work measurement for the
* Weather layers (Wind, Rain radar, Satellite clouds, Lightning density).
*
* This is a measurement harness, not a pass/fail gate. It reproduces one fixed
* camera route per weather state so two builds (or two basemaps) can be
* compared on the same numbers:
*
* park 10 km over Austin → 3 s flight to 1.2 km (heading 20°, pitch −45°)
* → 6 s close idle → 8 s orbit at 1.8 km range → 3 s flight back to the
* 17,368 km globe view → 6 s globe idle.
*
* Route windows hold render demand constant (`scene.requestRender()` on every
* animation frame) so the idle-render governor cannot masquerade as low FPS.
* FPS counts `scene.postRender` fires, and every window asserts the canvas
* size did not change, because a stopped renderer with a free-running rAF
* reads as a clean 60 fps otherwise.
*
* The separate idle test restores the normal governor, parks the camera at
* the 1.2 km pose with Wind off, and counts scene redraws, DOM mutations
* inside `.weather-summary`, and `.weather-summary` height changes over
* 8-second windows for: weather off, history paused, history playing.
*
* Usage:
* node scripts/qa-weather-perf.mjs [--url http://localhost:4173]
* [--label google-baseline] [--states off,wind,...] [--repeat 1]
* [--headless] [--no-route] [--no-idle] [--out qa-shots/weather-perf]
*
* States: off · history-paused · history-playing · wind · wind+history-paused
* · wind+history-playing. "history" is radar + regional clouds + lightning
* together; "wind" is animated GFS wind with the Speed field selected.
* Requires a running dev server. Screenshots and results.json land in
* <out>/<label>/.
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
// The first-launch chooser would otherwise sit over the globe; `?welcome=0`
// is the app's own suppression flag (src/firstRunExperience.js).
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg(
'--label',
new Date().toISOString().slice(0, 19).replace(/[:T]/g, '-'),
);
const ALL_STATES = [
'off',
'history-paused',
'history-playing',
'wind',
'wind+history-paused',
'wind+history-playing',
];
const states = arg('--states', ALL_STATES.join(','))
.split(',')
.map((s) => s.trim())
.filter(Boolean);
for (const s of states)
if (!ALL_STATES.includes(s)) {
console.error(`unknown state "${s}"; choose from ${ALL_STATES.join(', ')}`);
process.exit(2);
}
const repeat = Math.max(1, Number(arg('--repeat', '1')) || 1);
// `--scale 2` renders at 2× resolutionScale (4× the pixels) so a GPU that idles
// at 60 fps natively still exposes per-frame cost differences.
const scale = Number(arg('--scale', '1')) || 1;
const outDir = path.resolve(arg('--out', 'qa-shots/weather-perf'), label);
mkdirSync(outDir, { recursive: true });
const WEATHER = ['weather-radar', 'weather-satellite', 'weather-lightning'];
const AUSTIN = { lon: -97.7431, lat: 30.2672 };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const chromePath =
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
);
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath: chromePath,
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--window-size=1200,960',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const results = [];
let reportFailures = null;
const record = (row) => {
results.push(row);
reportFailures?.();
const {
phase,
state,
run,
window: win,
fps,
p95,
longTasks,
mutations,
heightChanges,
canvasStable,
} = row;
console.log(
` ${phase.padEnd(5)} ${state.padEnd(22)} r${run} ${win.padEnd(12)} ` +
`fps=${String(fps).padStart(5)} p95=${String(p95).padStart(6)}ms ` +
`long=${String(longTasks).padStart(2)} dom=${String(mutations).padStart(4)} ` +
`h∆=${String(heightChanges).padStart(2)}${canvasStable ? '' : ' CANVAS CHANGED'}`,
);
};
try {
const page = await browser.newPage();
await page.setViewport({ width: 1200, height: 886, deviceScaleFactor: 2 });
page.on('pageerror', (e) => console.log(' [pageerror]', e.message));
// Non-2xx responses, counted per path so a burst of throttled tiles reads
// as one line instead of one console error per tile.
const failures = new Map();
page.on('response', (r) => {
const status = r.status();
if (status < 400) return;
const key = `${status} ${new URL(r.url()).pathname}`;
failures.set(key, (failures.get(key) || 0) + 1);
});
reportFailures = () => {
if (!failures.size) return;
console.log(
' [http failures] ' +
[...failures].map(([k, n]) => `${k} ×${n}`).join(' · '),
);
failures.clear();
};
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(15_000);
if (scale !== 1)
await page.evaluate((scale) => {
window.__godsEyeView.viewer.resolutionScale = scale;
window.__godsEyeView.viewer.scene.requestRender();
}, scale);
const env = await page.evaluate(() => {
const gev = window.__godsEyeView;
const scene = gev.viewer.scene;
return {
basemap: gev.tileset ? 'google-3d' : 'keyless-globe',
globeShown: scene.globe.show,
canvas: `${scene.canvas.width}x${scene.canvas.height}`,
dpr: devicePixelRatio,
ua:
navigator.userAgent.match(/Chrome\/[\d.]+/)?.[0] || navigator.userAgent,
msaa: scene.msaaSamples,
resolutionScale: gev.viewer.resolutionScale,
};
});
console.log(`\nqa-weather-perf · ${label}\n url=${url}`);
console.log(` ${JSON.stringify(env)}`);
// Park deterministically, disable every layer, and instrument once.
await page.evaluate(async () => {
const gev = window.__godsEyeView;
const v = gev.viewer;
v.camera.cancelFlight();
for (const [id, entry] of gev.dataManager.layers)
if (entry.enabled) {
try {
await gev.dataManager.setEnabled(id, false, { origin: 'user' });
} catch {
/* counted through the layer, not here */
}
}
const P = (window.__gevWeatherPerf = {
renders: 0,
longTasks: 0,
longTaskMs: 0,
mutations: 0,
});
v.scene.postRender.addEventListener(() => {
P.renders += 1;
});
try {
new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
P.longTasks += 1;
P.longTaskMs += entry.duration;
}
}).observe({ type: 'longtask', buffered: false });
} catch {
/* longtask unsupported → stays 0 */
}
const summary = document.querySelector('.weather-summary');
if (summary)
new MutationObserver((records) => {
P.mutations += records.length;
}).observe(summary, {
subtree: true,
childList: true,
attributes: true,
characterData: true,
});
});
const pose = (which) =>
page.evaluate(
({ which, AUSTIN }) => {
const C = window.Cesium || null;
const v = window.__godsEyeView.viewer;
v.camera.cancelFlight();
v.camera.lookAtTransform?.(
(C || v.camera.constructor).Matrix4?.IDENTITY ??
v.scene.camera.transform,
);
const ell = v.scene.ellipsoid || v.scene.globe.ellipsoid;
const d2r = Math.PI / 180;
const destination = ell.cartographicToCartesian({
longitude: AUSTIN.lon * d2r,
latitude: AUSTIN.lat * d2r,
height:
which === 'overhead'
? 10_000
: which === 'close'
? 1_200
: 17_368_000,
});
v.camera.setView({
destination,
orientation:
which === 'close'
? { heading: 20 * d2r, pitch: -45 * d2r, roll: 0 }
: { heading: 0, pitch: -90 * d2r, roll: 0 },
});
},
{ which, AUSTIN },
);
/** Start a 3 s flight without awaiting it; the caller measures alongside. */
const fly = (which) =>
page.evaluate(
({ which, AUSTIN }) => {
const v = window.__godsEyeView.viewer;
const ell = v.scene.ellipsoid || v.scene.globe.ellipsoid;
const d2r = Math.PI / 180;
v.camera.flyTo({
destination: ell.cartographicToCartesian({
longitude: AUSTIN.lon * d2r,
latitude: AUSTIN.lat * d2r,
height: which === 'close' ? 1_200 : 17_368_000,
}),
orientation:
which === 'close'
? { heading: 20 * d2r, pitch: -45 * d2r, roll: 0 }
: { heading: 0, pitch: -90 * d2r, roll: 0 },
duration: 3,
});
},
{ which, AUSTIN },
);
/** Start an 8 s orbit at 1.8 km range around the Austin anchor (not awaited). */
const orbit = () =>
page.evaluate(
({ AUSTIN }) => {
const v = window.__godsEyeView.viewer;
const camera = v.camera;
const ell = v.scene.ellipsoid || v.scene.globe.ellipsoid;
const d2r = Math.PI / 180;
const center = ell.cartographicToCartesian({
longitude: AUSTIN.lon * d2r,
latitude: AUSTIN.lat * d2r,
height: 0,
});
const HPR = camera.constructor?.HeadingPitchRange
? camera.constructor.HeadingPitchRange
: null;
const t0 = performance.now();
let heading = 20 * d2r;
const step = (now) => {
const elapsed = (now - t0) / 1000;
heading = 20 * d2r + (elapsed / 8) * Math.PI * 2;
const hpr = HPR
? new HPR(heading, -45 * d2r, 1800)
: { heading, pitch: -45 * d2r, range: 1800 };
camera.lookAt(center, hpr);
if (elapsed < 8) requestAnimationFrame(step);
else camera.lookAtTransform(camera.transform.constructor.IDENTITY);
};
requestAnimationFrame(step);
},
{ AUSTIN },
);
/** Measure one window; `hold` keeps render demand constant. */
const measure = (ms, hold) =>
page.evaluate(
({ ms, hold }) =>
new Promise((resolve) => {
const gev = window.__godsEyeView;
const scene = gev.viewer.scene;
const P = window.__gevWeatherPerf;
const canvas = scene.canvas;
const w0 = canvas.width;
const h0 = canvas.height;
const r0 = P.renders;
const l0 = P.longTasks;
const lm0 = P.longTaskMs;
const m0 = P.mutations;
const summary = document.querySelector('.weather-summary');
const intervals = [];
const heights = [];
let lastH = -1;
let rafs = 0;
let last = performance.now();
const t0 = last;
const finish = (now) => {
const seconds = (now - t0) / 1000;
intervals.sort((a, b) => a - b);
const q = (p) =>
intervals[
Math.min(intervals.length - 1, Math.floor(p * intervals.length))
] ?? 0;
const hs = heights.map(([, h]) => h);
resolve({
seconds: Number(seconds.toFixed(2)),
renders: P.renders - r0,
fps: Number(((P.renders - r0) / seconds).toFixed(1)),
rafHz: Number((rafs / seconds).toFixed(1)),
p50: Number(q(0.5).toFixed(1)),
p95: Number(q(0.95).toFixed(1)),
max: Number((intervals.at(-1) ?? 0).toFixed(1)),
longTasks: P.longTasks - l0,
longTaskMs: Math.round(P.longTaskMs - lm0),
mutations: P.mutations - m0,
heightChanges: Math.max(0, heights.length - 1),
heightMin: hs.length ? Math.min(...hs) : null,
heightMax: hs.length ? Math.max(...hs) : null,
canvasStable: canvas.width === w0 && canvas.height === h0,
canvas: `${w0}x${h0}`,
});
};
const tick = (now) => {
rafs += 1;
intervals.push(now - last);
last = now;
if (hold) scene.requestRender();
if (summary && !summary.hidden) {
const h =
Math.round(summary.getBoundingClientRect().height * 2) / 2;
if (h !== lastH) {
heights.push([Math.round(now - t0), h]);
lastH = h;
}
}
if (now - t0 < ms) requestAnimationFrame(tick);
else finish(now);
};
requestAnimationFrame(tick);
}),
{ ms, hold },
);
const diagnostics = () =>
page.evaluate(
({ WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
const out = {};
for (const id of ['wind', ...WEATHER]) {
const entry = dm.layers.get(id);
if (!entry?.enabled) continue;
const d = entry.module?.getDiagnostics?.() || {};
out[id] =
id === 'wind'
? {
mode: d.renderMode,
ready: d.gpu?.ready,
paths: d.gpu?.pathCount,
vertices: d.gpu?.vertexCount,
buildMs: d.gpu?.buildMs && Math.round(d.gpu.buildMs),
cells: d.gpu?.cellCount,
visibleCells: d.gpu?.visibleCells,
visibleVertices: d.gpu?.visibleVertexCount,
heightFade: d.gpu?.heightFade,
}
: {
time: d.time,
playing: d.playing,
loading: d.loading,
loadedTiles: d.loadedTiles,
imageryCount: d.imageryCount,
frameLoadMs: d.frameLoadMs && Math.round(d.frameLoadMs),
error: d.error,
suspended: d.suspended ?? d.globeHidden ?? null,
};
}
return out;
},
{ WEATHER },
);
/** Enable the layers a state names, wait for first imagery/geometry, then set playback. */
const applyState = async (state) => {
const wantWind = state.includes('wind');
const wantHistory = state.includes('history');
const wantPlaying = state.includes('playing');
const started = Date.now();
await page.evaluate(
async ({ wantWind, wantHistory, WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
const set = async (id, on) => {
const entry = dm.layers.get(id);
if (!entry || Boolean(entry.enabled) === on) return;
await dm.setEnabled(id, on, { origin: 'user' });
};
await set('wind', wantWind);
if (wantWind)
dm.setLayerParams('wind', { overlay: 'speed' }, { origin: 'user' });
for (const id of WEATHER) await set(id, wantHistory);
},
{ wantWind, wantHistory, WEATHER },
);
let ready = true;
await page
.waitForFunction(
({ wantWind, wantHistory, WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
if (wantWind) {
const d = dm.layers.get('wind')?.module?.getDiagnostics?.();
if (!d) return false;
if (d.renderMode === 'gpu-streamlines' && !d.gpu?.ready)
return false;
}
if (wantHistory)
for (const id of WEATHER) {
const d = dm.layers.get(id)?.module?.getDiagnostics?.();
if (!d || d.loading || (!d.time && !d.error)) return false;
}
return true;
},
{ timeout: 90_000, polling: 250 },
{ wantWind, wantHistory, WEATHER },
)
.catch(() => {
ready = false;
});
const readyMs = Date.now() - started;
await page.evaluate(
({ wantPlaying, WEATHER }) => {
const dm = window.__godsEyeView.dataManager;
for (const id of WEATHER) {
const module = dm.layers.get(id)?.module;
if (!module || !dm.layers.get(id)?.enabled) continue;
const playing = Boolean(module.getDiagnostics?.().playing);
if (playing !== wantPlaying)
dm.setLayerParams(id, { play: true }, { origin: 'user' });
}
},
{ wantPlaying, WEATHER },
);
await sleep(2_500);
return { ready, readyMs, layers: await diagnostics() };
};
const shot = (name) =>
page.screenshot({ path: path.join(outDir, `${name}.png`) });
// ── Matched route ────────────────────────────────────────────────────────
if (!flag('--no-route')) {
console.log('\nRoute windows (continuous-render hold):');
for (let run = 1; run <= repeat; run++) {
const order = run % 2 === 1 ? states : [...states].reverse();
for (const state of order) {
await pose('overhead');
const setup = await applyState(state);
console.log(
` setup ${state}: ready=${setup.ready} in ${setup.readyMs}ms ${JSON.stringify(setup.layers)}`,
);
const base = { phase: 'route', state, run, setup };
await fly('close');
record({ ...base, window: 'zoom-in', ...(await measure(6_000, true)) });
record({
...base,
window: 'close-idle',
...(await measure(6_000, true)),
});
await shot(`${state}-r${run}-close`);
await orbit();
record({ ...base, window: 'orbit', ...(await measure(8_000, true)) });
await fly('globe');
record({
...base,
window: 'zoom-out',
...(await measure(6_000, true)),
});
record({
...base,
window: 'globe-idle',
...(await measure(6_000, true)),
});
await shot(`${state}-r${run}-globe`);
console.log(` after ${state}: ${JSON.stringify(await diagnostics())}`);
}
}
}
// ── Idle test (normal governor) ──────────────────────────────────────────
if (!flag('--no-idle')) {
console.log('\nIdle windows (normal governor, fixed 1.2 km pose):');
for (const state of ['off', 'history-paused', 'history-playing', 'wind']) {
await pose('close');
const setup = await applyState(state);
await sleep(4_000);
for (let run = 1; run <= 2; run++)
record({
phase: 'idle',
state,
run,
window: 'fixed-8s',
setup,
...(await measure(8_000, false)),
});
await shot(`idle-${state}`);
}
await applyState('off');
}
writeFileSync(
path.join(outDir, 'results.json'),
JSON.stringify({ label, url, env, results }, null, 2),
);
const lines = [
`# qa-weather-perf · ${label}`,
'',
`url: ${url} · ${env.basemap} · canvas ${env.canvas} @ DPR ${env.dpr} · ${env.ua}`,
'',
'| phase | state | run | window | fps | p50 ms | p95 ms | max ms | long tasks | long ms | dom mut | height ∆ | height min–max |',
'|---|---|---|---|---|---|---|---|---|---|---|---|---|',
...results.map(
(r) =>
`| ${r.phase} | ${r.state} | ${r.run} | ${r.window} | ${r.fps}${r.canvasStable ? '' : ' ⚠'} | ${r.p50} | ${r.p95} | ${r.max} | ${r.longTasks} | ${r.longTaskMs} | ${r.mutations} | ${r.heightChanges} | ${r.heightMin ?? '—'}–${r.heightMax ?? '—'} |`,
),
];
writeFileSync(path.join(outDir, 'summary.md'), lines.join('\n') + '\n');
console.log(`\nWrote ${path.join(outDir, 'results.json')} and summary.md`);
} finally {
await browser.close();
}
+203
View File
@@ -0,0 +1,203 @@
/**
* Weather history-swap probe: does the scene keep drawing while a draped
* weather frame is replaced? Runs against a live dev server, turns on one
* observed product at the Earth view, steps Earlier and plays history while
* screenshotting, then measures the black fraction inside the globe disc of
* every capture. Writes <out>/<label>/summary.json.
*
* node scripts/qa-weather-swap.mjs --url http://localhost:4173 --label google
* [--layer weather-satellite] [--product clouds] [--headless]
*/
import {
existsSync,
mkdirSync,
readdirSync,
readFileSync,
writeFileSync,
} from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg('--label', 'swap');
const layerId = arg('--layer', 'weather-satellite');
const product = arg(
'--product',
layerId === 'weather-satellite' ? 'clouds' : null,
);
const outDir = path.resolve(arg('--out', 'qa-shots/weather-swap'), label);
mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
// Globe disc in CSS pixels for the 1400×920 viewport at the Earth view.
const DISC = { cx: 700, cy: 460, r: 300 };
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
),
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--window-size=1400,1000',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const frames = [];
try {
const page = await browser.newPage();
await page.setViewport({ width: 1400, height: 920, deviceScaleFactor: 1 });
page.on('pageerror', (e) => console.log(' [pageerror]', e.message));
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(12_000);
await page.evaluate(
async ({ layerId, product }) => {
const gev = window.__godsEyeView;
const dm = gev.dataManager;
for (const [id, e] of dm.layers)
if (e.enabled) await dm.setEnabled(id, false, { origin: 'user' });
const v = gev.viewer;
const d = Math.PI / 180;
v.camera.setView({
destination: v.scene.ellipsoid.cartographicToCartesian({
longitude: -80 * d,
latitude: 25 * d,
height: 17_368_000,
}),
orientation: { heading: 0, pitch: -90 * d, roll: 0 },
});
await dm.setEnabled(layerId, true, { origin: 'user' });
if (product) dm.setLayerParams(layerId, { product }, { origin: 'user' });
},
{ layerId, product },
);
await page
.waitForFunction(
({ layerId, product }) => {
const d = window.__godsEyeView.dataManager.layers
.get(layerId)
?.module?.getDiagnostics?.();
return d && !d.loading && d.time && (!product || d.product === product);
},
{ timeout: 90_000 },
{ layerId, product },
)
.catch(() => console.log(' !! first frame not ready'));
await sleep(3000);
const diag = () =>
page.evaluate((layerId) => {
const d = window.__godsEyeView.dataManager.layers
.get(layerId)
.module.getDiagnostics();
return {
time: d.time,
loading: d.loading,
pending: d.pendingTiles,
loaded: d.loadedTiles,
imagery: d.imageryCount,
mosaic: d.mosaic ?? null,
};
}, layerId);
const capture = async (name) => {
const file = `${name}.png`;
await page.screenshot({ path: path.join(outDir, file) });
frames.push({ file, ...(await diag()) });
};
await capture('00-latest');
const step = (params) =>
page.evaluate(
({ layerId, params }) =>
window.__godsEyeView.dataManager.setLayerParams(layerId, params, {
origin: 'user',
}),
{ layerId, params },
);
await step({ step: -1 });
for (let i = 1; i <= 15; i++) {
await sleep(400);
await capture(`earlier-${String(i).padStart(2, '0')}`);
}
await step({ play: true });
for (let i = 1; i <= 20; i++) {
await sleep(500);
if (i % 2 === 0) await capture(`play-${String(i).padStart(2, '0')}`);
}
await step({ play: true });
// Black fraction inside the globe disc, measured in a blank page.
const meter = await browser.newPage();
for (const frame of frames) {
const b64 = readFileSync(path.join(outDir, frame.file)).toString('base64');
const black = await meter.evaluate(
async ({ b64, DISC }) => {
const img = new Image();
img.src = `data:image/png;base64,${b64}`;
await img.decode();
const c = document.createElement('canvas');
c.width = img.naturalWidth;
c.height = img.naturalHeight;
const ctx = c.getContext('2d');
ctx.drawImage(img, 0, 0);
const s = img.naturalWidth / 1400;
const d = ctx.getImageData(0, 0, c.width, c.height).data;
const rr = (DISC.r * s) ** 2;
let n = 0;
let dark = 0;
for (let y = 0; y < c.height; y += 2)
for (let x = 0; x < c.width; x += 2) {
if ((x - DISC.cx * s) ** 2 + (y - DISC.cy * s) ** 2 > rr) continue;
n++;
const i = (y * c.width + x) * 4;
if (d[i] < 18 && d[i + 1] < 18 && d[i + 2] < 18) dark++;
}
return dark / n;
},
{ b64, DISC },
);
frame.black = Number(black.toFixed(4));
console.log(
`${frame.file}\t${(black * 100).toFixed(1)}%\t${frame.time}\tpending=${frame.pending}`,
);
}
const blackFrames = frames.filter((f) => f.black > 0.15).length;
const distinctTimes = new Set(frames.map((f) => f.time).filter(Boolean)).size;
const summary = {
url: rawUrl,
label,
layerId,
product,
frames: frames.length,
blackFrames,
maxBlack: Math.max(...frames.map((f) => f.black)),
distinctTimes,
capturedAt: new Date().toISOString(),
};
writeFileSync(
path.join(outDir, 'summary.json'),
JSON.stringify({ summary, frames }, null, 2),
);
console.log(
`frames=${frames.length} black(>15%)=${blackFrames} max=${(summary.maxBlack * 100).toFixed(1)}% distinctTimes=${distinctTimes}`,
);
} finally {
await browser.close();
}
+341
View File
@@ -0,0 +1,341 @@
/**
* Weather teardown and cross-layer regression probe.
*
* Against a running dev server: records a resource baseline, turns every
* weather layer on and exercises it (history playback, a wind reading, a
* storm selection), turns them all off, and asserts the scene is back to the
* baseline (imagery layers on the globe and the tileset, primitives, ground
* primitives, data sources, entities, camera/scene listeners, the weather
* panel, the wind reading marker). Then it enables a set of non-weather
* layers one at a time and samples the render rate, long tasks and console
* errors for each, and repeats the same non-weather sequence in a control
* page that never had weather on. Writes <out>/<label>/results.json.
*
* node scripts/qa-weather-teardown.mjs --url http://localhost:4173 --label google
*/
import { existsSync, mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import puppeteer from 'puppeteer';
const argv = process.argv.slice(2);
const arg = (name, fallback) => {
const i = argv.indexOf(name);
return i >= 0 && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--')
? argv[i + 1]
: fallback;
};
const flag = (name) => argv.includes(name);
const rawUrl = arg('--url', 'http://localhost:4173');
const url = rawUrl.includes('welcome=')
? rawUrl
: `${rawUrl}${rawUrl.includes('?') ? '&' : '?'}welcome=0`;
const label = arg('--label', 'teardown');
const outDir = path.resolve(arg('--out', 'qa-shots/weather-teardown'), label);
mkdirSync(outDir, { recursive: true });
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const WEATHER = [
'wind',
'weather-radar',
'weather-satellite',
'weather-lightning',
'weather-cyclones',
];
const browser = await puppeteer.launch({
headless: flag('--headless') ? 'new' : false,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
['/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'].find((p) =>
existsSync(p),
),
protocolTimeout: 300_000,
defaultViewport: null,
args: [
'--no-sandbox',
'--window-size=1400,1000',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-background-timer-throttling',
],
});
const boot = async () => {
const page = await browser.newPage();
await page.setViewport({ width: 1400, height: 920, deviceScaleFactor: 1 });
const errors = [];
page.on('pageerror', (e) =>
errors.push(`pageerror: ${e.message}`.slice(0, 200)),
);
page.on('console', (m) => {
if (m.type() === 'error') errors.push(`console: ${m.text()}`.slice(0, 200));
});
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => !!window.__godsEyeView?.viewer, {
timeout: 120_000,
});
await sleep(10_000);
await page.evaluate(async () => {
const gev = window.__godsEyeView;
const dm = gev.dataManager;
for (const [id, e] of dm.layers)
if (e.enabled) await dm.setEnabled(id, false, { origin: 'user' });
const v = gev.viewer;
const d = Math.PI / 180;
v.camera.setView({
destination: v.scene.ellipsoid.cartographicToCartesian({
longitude: -95 * d,
latitude: 32 * d,
height: 4_500_000,
}),
orientation: { heading: 0, pitch: -90 * d, roll: 0 },
});
});
await sleep(3000);
return { page, errors };
};
const snapshot = (page) =>
page.evaluate(() => {
const gev = window.__godsEyeView;
const v = gev.viewer;
const s = v.scene;
const t = gev.tileset;
const dm = gev.dataManager;
const weather = {};
for (const id of [
'weather-radar',
'weather-satellite',
'weather-lightning',
]) {
const d = dm.layers.get(id)?.module?.getDiagnostics?.();
if (d)
weather[id] = {
imagery: d.imageryCount,
mosaics: d.cache?.mosaics ?? 0,
prefetching: !!d.cache?.prefetching,
};
}
const wind = dm.layers.get('wind')?.module?.getDiagnostics?.() || {};
return {
globeImagery: v.imageryLayers.length,
tilesetImagery: t?.imageryLayers?.length ?? null,
primitives: s.primitives.length,
groundPrimitives: s.groundPrimitives.length,
dataSources: v.dataSources.length,
entities: v.entities.values.length,
listeners: {
postRender: s.postRender.numberOfListeners,
preRender: s.preRender.numberOfListeners,
postUpdate: s.postUpdate.numberOfListeners,
moveEnd: v.camera.moveEnd.numberOfListeners,
moveStart: v.camera.moveStart.numberOfListeners,
changed: v.camera.changed.numberOfListeners,
tick: v.clock.onTick.numberOfListeners,
},
dom: {
weatherPanelHidden:
document.getElementById('weather-panel')?.hidden ?? null,
weatherCards: document.querySelectorAll('.weather-card').length,
windReading: !!document.querySelector(
'.gev-wind-reading:not([hidden])',
),
},
weather,
windCells: wind.gpu?.pathCount ?? wind.paths ?? null,
enabled: [...dm.layers].filter(([, e]) => e.enabled).map(([id]) => id),
};
});
const sample = async (page, seconds) => {
await page.evaluate(() => {
window.__qaRenders = 0;
window.__qaLong = { count: 0, ms: 0 };
const s = window.__godsEyeView.viewer.scene;
window.__qaOff = s.postRender.addEventListener(() => window.__qaRenders++);
window.__qaObs = new PerformanceObserver((list) => {
for (const e of list.getEntries()) {
window.__qaLong.count++;
window.__qaLong.ms += e.duration;
}
});
window.__qaObs.observe({ entryTypes: ['longtask'] });
window.__qaRaf = () => {
s.requestRender();
window.__qaRafId = requestAnimationFrame(window.__qaRaf);
};
window.__qaRaf();
});
await sleep(seconds * 1000);
return page.evaluate((seconds) => {
cancelAnimationFrame(window.__qaRafId);
window.__qaOff();
window.__qaObs.disconnect();
return {
fps: +(window.__qaRenders / seconds).toFixed(1),
longTasks: window.__qaLong.count,
longMs: Math.round(window.__qaLong.ms),
};
}, seconds);
};
const setLayer = (page, id, on) =>
page.evaluate(
async ({ id, on }) => {
const dm = window.__godsEyeView.dataManager;
const e = dm.layers.get(id);
if (e && Boolean(e.enabled) !== on)
await dm.setEnabled(id, on, { origin: 'user' });
return !!e;
},
{ id, on },
);
const diff = (a, b) => {
const out = {};
const walk = (x, y, prefix) => {
for (const key of Object.keys(x)) {
const p = prefix ? `${prefix}.${key}` : key;
if (x[key] && typeof x[key] === 'object' && !Array.isArray(x[key]))
walk(x[key], y?.[key] || {}, p);
else if (JSON.stringify(x[key]) !== JSON.stringify(y?.[key]))
out[p] = [x[key], y?.[key]];
}
};
walk(a, b, '');
return out;
};
const results = { url: rawUrl, label, startedAt: new Date().toISOString() };
try {
// ── Run A: weather on → exercised → off → baseline check → other layers ──
const a = await boot();
const others = await a.page.evaluate(() =>
[...window.__godsEyeView.dataManager.layers.keys()].filter((id) =>
/^flights$|^satellites$|vessel|firms|earthquake|cables|^military$/.test(
id,
),
),
);
console.log('other layers under test:', others.join(', '));
const baseline = await snapshot(a.page);
console.log('baseline', JSON.stringify(baseline));
for (const id of WEATHER) await setLayer(a.page, id, true);
await a.page
.waitForFunction(
() =>
['weather-radar', 'weather-satellite', 'weather-lightning'].every(
(id) => {
const d = window.__godsEyeView.dataManager.layers
.get(id)
?.module?.getDiagnostics?.();
return d && !d.loading && d.time;
},
),
{ timeout: 120_000 },
)
.catch(() => console.log(' !! weather not ready'));
await sleep(2000);
// exercise: history playback, wind reading, storm selection, product switch
await a.page.evaluate(() => {
const dm = window.__godsEyeView.dataManager;
dm.setLayerParams('weather-radar', { play: true }, { origin: 'user' });
dm.setLayerParams('wind', { overlay: 'speed' }, { origin: 'user' });
dm.setLayerParams('wind', { inspect: true }, { origin: 'user' });
dm.setLayerParams(
'weather-satellite',
{ product: 'clouds' },
{ origin: 'user' },
);
});
await sleep(8000);
await a.page.evaluate(() => {
const dm = window.__godsEyeView.dataManager;
const storms =
dm.layers.get('weather-cyclones')?.module?.getRowControls?.().list
?.items || [];
if (storms[1])
dm.setLayerParams('weather-cyclones', storms[1].params, {
origin: 'user',
});
dm.setLayerParams('weather-radar', { play: true }, { origin: 'user' });
});
await sleep(4000);
const weatherOn = await snapshot(a.page);
console.log('weather on', JSON.stringify(weatherOn));
results.weatherOnSample = await sample(a.page, 5);
for (const id of WEATHER) await setLayer(a.page, id, false);
await sleep(4000);
const afterOff = await snapshot(a.page);
const leak = diff(
{ ...baseline, enabled: undefined, weather: undefined },
{ ...afterOff, enabled: undefined, weather: undefined },
);
results.baseline = baseline;
results.afterOff = afterOff;
results.leak = leak;
console.log('after off', JSON.stringify(afterOff));
console.log(
Object.keys(leak).length
? `!! DIFFERENCES after teardown: ${JSON.stringify(leak)}`
: 'teardown: scene back to baseline',
);
results.afterWeather = [];
for (const id of others) {
const ok = await setLayer(a.page, id, true);
await sleep(9000);
const m = await sample(a.page, 5);
const snap = await snapshot(a.page);
results.afterWeather.push({
id,
ok,
...m,
primitives: snap.primitives,
dataSources: snap.dataSources,
});
console.log(
`after weather + ${id}: fps=${m.fps} long=${m.longTasks}/${m.longMs}ms`,
);
}
results.errorsA = a.errors;
await a.page.screenshot({ path: path.join(outDir, 'a-others-on.png') });
await a.page.close();
// ── Run B: control (same others, never had weather) ──
const b = await boot();
results.control = [];
for (const id of others) {
const ok = await setLayer(b.page, id, true);
await sleep(9000);
const m = await sample(b.page, 5);
const snap = await snapshot(b.page);
results.control.push({
id,
ok,
...m,
primitives: snap.primitives,
dataSources: snap.dataSources,
});
console.log(
`control + ${id}: fps=${m.fps} long=${m.longTasks}/${m.longMs}ms`,
);
}
results.errorsB = b.errors;
await b.page.screenshot({ path: path.join(outDir, 'b-control.png') });
await b.page.close();
console.log(
'\nerrors (weather run):',
results.errorsA.length ? results.errorsA : 'none',
);
console.log(
'errors (control run):',
results.errorsB.length ? results.errorsB : 'none',
);
writeFileSync(
path.join(outDir, 'results.json'),
JSON.stringify(results, null, 2),
);
console.log(`wrote ${path.join(outDir, 'results.json')}`);
} finally {
await browser.close();
}
+44
View File
@@ -0,0 +1,44 @@
/** Real-canvas wind regression. Run against this candidate's dev server. */
import assert from 'node:assert/strict';
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch({ executablePath: process.env.PUPPETEER_EXECUTABLE_PATH || await puppeteer.executablePath(), headless: false, args: ['--no-sandbox', '--disable-dev-shm-usage'] });
try {
const page = await browser.newPage();
await page.goto(`${process.env.QA_BASE_URL || 'http://localhost:4173'}/`, { waitUntil: 'domcontentloaded' });
const result = await page.evaluate(async () => {
const { createWindRendering } = await import('/src/layers/wind/rendering.js');
const raf = window.requestAnimationFrame; const caf = window.cancelAnimationFrame;
const callbacks = new Map(); let sequence = 0;
window.requestAnimationFrame = callback => { callbacks.set(++sequence, callback); return sequence; };
window.cancelAnimationFrame = id => callbacks.delete(id);
const container = document.createElement('div');
container.style.cssText = 'position:fixed;left:0;top:0;width:800px;height:600px;z-index:9999;background:#102030';
document.body.appendChild(container);
const sceneCanvas = { clientWidth: 800, clientHeight: 600 };
const viewer = { scene: { canvas: sceneCanvas, camera: {} }, isDestroyed: () => false };
const cesium = { Cartesian3: { fromDegrees: (lon,lat) => ({ lon,lat }) }, SceneTransforms: { worldToWindowCoordinates: (scene, point) => ({ x: 400 + point.lon, y: 300 + point.lat }) } };
const rendering = createWindRendering({ cesium, container, getViewer: () => viewer });
try {
rendering.attach(); rendering.start();
const idleCallbacks = callbacks.size;
rendering.setField({ grid: { nx:1,ny:1,lo1:0,la1:90,dx:360,dy:180 }, u: new Float32Array([10]), v: new Float32Array([0]) });
const canvas = container.querySelector('canvas'); const context = canvas.getContext('2d');
const tick = time => { const [id, callback] = callbacks.entries().next().value; callbacks.delete(id); callback(time); };
tick(16); // First camera signature clears the surface.
context.fillStyle = '#ff0000'; context.fillRect(10,10,5,5);
tick(32);
const alpha = context.getImageData(12,12,1,1).data[3];
const count = rendering.getParticleCount();
sceneCanvas.clientWidth = 320; sceneCanvas.clientHeight = 240;
tick(48);
const resizedCount = rendering.getParticleCount();
rendering.stop(); rendering.clear();
return { idleCallbacks, alpha, count, resizedCount, pending: callbacks.size, afterClear: context.getImageData(12,12,1,1).data[3] };
} finally { rendering.destroy(); container.remove(); window.requestAnimationFrame = raf; window.cancelAnimationFrame = caf; }
});
assert.equal(result.idleCallbacks, 0);
assert.ok(result.alpha > 200, 'unchanged canvas dimensions preserve existing trail pixels');
assert.ok(result.resizedCount < result.count);
assert.equal(result.pending, 0); assert.equal(result.afterClear, 0);
console.log(JSON.stringify(result));
} finally { await browser.close(); }
+39 -5
View File
@@ -8,6 +8,8 @@ import { projectRoot } from './project-root.mjs';
import { selectMapStartupRoute } from '../src/mapStartup.js';
const ROOT = projectRoot(import.meta.url);
const OPENSKY_AUTH_MODE_DEFAULT = 'oauth';
const OPENSKY_AUTH_MODES = new Set(['basic', 'oauth', 'auto', 'anon']);
export const CREDENTIALS = Object.freeze([
{ name: 'GOOGLE_MAPS_API_KEY', label: 'Google Maps', keychain: [['google-maps-api', 'api-key'], ['google-maps-api', 'default'], ['google-maps-api', 'key']] },
@@ -101,6 +103,21 @@ export function readDoctorDotenvValue(
return String(values[key] ?? '');
}
/** Resolve the OpenSky auth mode using the same environment and dotenv precedence as the launcher. */
export function resolveOpenSkyAuthMode({
authoritativeEnvironment = false,
environment = process.env,
rootDir = ROOT,
} = {}) {
const environmentDefinesMode = Object.prototype.hasOwnProperty.call(environment, 'OPENSKY_AUTH_MODE');
const environmentMode = String(environment.OPENSKY_AUTH_MODE || '').trim();
const dotenvMode = authoritativeEnvironment && environmentDefinesMode
? ''
: readDoctorDotenvValue('OPENSKY_AUTH_MODE', rootDir);
const mode = (environmentMode || dotenvMode).trim().toLowerCase();
return OPENSKY_AUTH_MODES.has(mode) ? mode : OPENSKY_AUTH_MODE_DEFAULT;
}
function hasKeychainItem(service, account) {
if (process.platform !== 'darwin') return false;
const result = spawnSync('security', [
@@ -128,8 +145,25 @@ export function resolveCredential(spec, {
return { configured: false, source: null };
}
export function buildCapabilitySummary(credentials) {
export function buildCapabilitySummary(
credentials,
{ openSkyAuthMode = OPENSKY_AUTH_MODE_DEFAULT } = {},
) {
const configured = (name) => credentials[name]?.configured === true;
const hasOAuthCredentials = configured('OPENSKY_CLIENT_ID') && configured('OPENSKY_CLIENT_SECRET');
const requestedOpenSkyMode = String(openSkyAuthMode || '').trim().toLowerCase();
const openSkyMode = OPENSKY_AUTH_MODES.has(requestedOpenSkyMode)
? requestedOpenSkyMode
: OPENSKY_AUTH_MODE_DEFAULT;
const flights = openSkyMode === 'anon'
? 'OpenSky keyless anonymous access (rate-limited)'
: openSkyMode === 'basic'
? 'OpenSky Basic mode selected (credential presence and validity not verified)'
: openSkyMode === 'auto'
? 'OpenSky auto mode selected (runtime credential choice and validity not verified)'
: hasOAuthCredentials
? 'OpenSky OAuth credentials present (runtime mode and validity not verified)'
: 'OpenSky keyless anonymous access (rate-limited)';
const route = selectMapStartupRoute({
googleApiKey: configured('GOOGLE_MAPS_API_KEY') ? 'configured' : '',
cesiumToken: configured('CESIUM_ION_TOKEN') ? 'configured' : '',
@@ -140,9 +174,7 @@ export function buildCapabilitySummary(credentials) {
: route === 'google-ion'
? 'Google Photorealistic 3D Tiles through Cesium ion; Bing and world-terrain stacks available'
: 'Esri World Imagery (keyless satellite basemap) with keyless terrain',
flights: configured('OPENSKY_CLIENT_ID') && configured('OPENSKY_CLIENT_SECRET')
? 'OpenSky OAuth credentials present (runtime mode and validity not verified)'
: 'OpenSky OAuth credentials not configured',
flights,
voice: configured('OPENAI_API_KEY') ? 'available' : 'off until an OpenAI key is added',
vessels: configured('AISSTREAM_API_KEY') ? 'live AISStream feed' : 'off until an AISStream key is added',
fires: configured('FIRMS_MAP_KEY') ? 'live NASA FIRMS feed' : 'off until a FIRMS key is added',
@@ -173,7 +205,9 @@ export function inspectSetup({ includeKeychain = true, authoritativeEnvironment
: { available: false, version: null },
dependenciesInstalled,
credentials,
capabilities: buildCapabilitySummary(credentials),
capabilities: buildCapabilitySummary(credentials, {
openSkyAuthMode: resolveOpenSkyAuthMode({ authoritativeEnvironment, rootDir }),
}),
};
}
+44
View File
@@ -88,6 +88,7 @@
* --source-base <path> Browser module prefix (default /src)
* --url <url> App URL (default http://localhost:4173)
* --headful Show the browser (debugging)
* --offline-imagery Use bundled texture for known Esri/OSM tile images only
* --keep-open Leave the browser open after the run (debugging)
*/
@@ -110,6 +111,7 @@ const SOURCE_BASE = getOpt('--source-base', '/src').replace(/\/$/, '');
const APP_URL = getOpt('--url', 'http://localhost:4173');
const APP_ORIGIN = new URL(APP_URL).origin;
const HEADFUL = getFlag('--headful');
const OFFLINE_IMAGERY = getFlag('--offline-imagery');
const KEEP_OPEN = getFlag('--keep-open');
const CHROME_EXECUTABLE_CANDIDATES = [
@@ -201,6 +203,15 @@ async function main() {
console.log(` App URL : ${APP_URL}`);
console.log(` Mode : ${HEADFUL ? 'headful' : 'headless'}\n`);
// Tracking does not assert imagery geography. This optional fixture leaves
// provider metadata, terrain, models and every tracking assertion untouched.
// Read before launching so a missing bundled asset fails explicitly.
const offlineImageryTile = OFFLINE_IMAGERY
? fs.readFileSync(new URL('../node_modules/cesium/Build/Cesium/Assets/Textures/NaturalEarthII/0/0/0.jpg', import.meta.url))
: null;
let offlineImageryRequests = 0;
if (OFFLINE_IMAGERY) console.log(' Imagery : bundled NaturalEarthII fixture (tile images only)');
// Verify the dev server is up before launching a browser.
try {
const res = await fetch(APP_URL, { method: 'GET' });
@@ -256,6 +267,25 @@ async function main() {
await page.setRequestInterception(true);
page.on('request', (request) => {
const url = new URL(request.url());
// Reuse the one existing interceptor; do not install competing handlers.
// Exact HTTPS origins and image paths only: metadata, feature queries,
// other providers, terrain and local/model resources continue normally.
const knownImageryTile = request.method() === 'GET' && (
(url.origin === 'https://services.arcgisonline.com'
&& /^\/ArcGIS\/rest\/services\/World_Imagery\/MapServer\/tile\/\d+\/\d+\/\d+$/.test(url.pathname))
|| (url.origin === 'https://tile.openstreetmap.org'
&& /^\/\d+\/\d+\/\d+\.png$/.test(url.pathname))
);
if (OFFLINE_IMAGERY && knownImageryTile) {
offlineImageryRequests++;
request.respond({
status: 200,
contentType: 'image/jpeg',
headers: { 'Access-Control-Allow-Origin': '*' },
body: offlineImageryTile,
});
return;
}
if (url.origin === APP_ORIGIN && url.pathname === '/api/openai/hud-summary') {
request.respond({
status: 200,
@@ -365,6 +395,19 @@ async function main() {
lastMessageAt: null,
}));
}
// Contacts also enables mapped installations. This tracking harness
// does not test installation acquisition; a valid empty viewport keeps
// public Overpass outages out of aircraft/satellite tracking results.
// Preserve the real response shape and all console-error assertions.
if (isAppRequest && url.pathname === '/api/military-installations') {
return Promise.resolve(jsonResponse({
elements: [],
saturated: false,
elementCap: 700,
retrievedAt: new Date().toISOString(),
status: 'ready',
}));
}
// CelesTrak TLE groups. Served from a fixed two-satellite catalog so
// the satellite assertions are deterministic and the run never depends
// on an upstream that rate-limits (it was answering 403 the night this
@@ -4082,6 +4125,7 @@ async function main() {
finishAndExit();
} finally {
if (OFFLINE_IMAGERY) console.log(` Bundled imagery tiles served: ${offlineImageryRequests}`);
if (!KEEP_OPEN) {
await browser.close();
} else {
+179
View File
@@ -0,0 +1,179 @@
/**
* Gemini Live adapter — BidiGenerateContent over WebSocket.
*
* Same prompt as production (system instruction = realtimeInstructions(),
* tools = GEV_REALTIME_TOOLS as functionDeclarations with the production JSON
* schema passed verbatim via parametersJsonSchema). The native-audio Live
* model rejects a TEXT response modality (close 1007), so the session
* requests AUDIO with output transcription on; tool calls arrive as
* `toolCall` messages and the transcript stands in for the final words.
*
* Fresh session per phrase / dialogue. Each toolCall gets stub
* `functionResponses`; the model continues on its own. A turn ends at a
* `turnComplete` whose generation had no toolCall, or after 5 tool rounds
* (mirrors the Realtime adapter's continuation cap), or 12 s of silence after
* a tool round.
*/
import WebSocket from 'ws';
import { providerKey } from '../keys.mjs';
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
import { toGeminiDeclarations } from '../toolFormats.mjs';
import { geminiUsageCostUsd } from '../pricing.mjs';
import { newTurnResult } from './openaiRealtime.mjs';
const ENDPOINT = 'wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent';
const MAX_TOOL_ROUNDS = 5;
const IDLE_AFTER_TOOL_MS = 12_000;
export function createGeminiLiveAdapter({ model, schemaMode = 'jsonSchema', turnTimeoutMs = 60_000 }) {
const instructions = realtimeInstructions();
const functionDeclarations = toGeminiDeclarations(GEV_REALTIME_TOOLS, { mode: schemaMode });
const adapter = {
provider: 'gemini-live',
model,
settings: {
responseModalities: ['AUDIO'],
outputAudioTranscription: true,
schemaMode,
temperature: 'server default (not set; production has no Gemini config)',
thinking: 'server default',
input: 'realtimeInput.text',
context: 'fresh session per phrase / dialogue',
},
async open() {},
async close() {},
async runTurn({ text, onCall, context }) {
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
},
runDialogue({ turns, onCall, contexts = [] }) {
return new Promise((resolveAll) => {
const tSetup = Date.now();
const ws = new WebSocket(`${ENDPOINT}?key=${providerKey('gemini')}`);
const results = [];
let setupMs = null;
let cur = null;
let closed = false;
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
const shutdown = (error) => {
if (closed) return;
closed = true;
if (cur) {
clearTimeout(cur.timer);
clearTimeout(cur.idle);
if (error) cur.r.errors.push(error);
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
cur = null;
}
while (results.length < turns.length) {
const r = newTurnResult();
r.errors.push(error || 'dialogue aborted');
results.push(r);
}
try { ws.terminate(); } catch { /* noop */ }
resolveAll(results);
};
const startTurn = (turn) => {
const r = newTurnResult();
r.setupMs = turn === 0 ? setupMs : 0;
cur = { r, t0: Date.now(), turn, toolRounds: 0, toolCallThisGeneration: false, idle: null };
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
// Live has no system-role mid-session item; a context item (pointer_context)
// rides as a JSON line ahead of the user's words in the same text turn.
const ctx = contexts[turn] ? `${JSON.stringify(contexts[turn])}\n` : '';
send({ realtimeInput: { text: `${ctx}${turns[turn]}` } });
};
const endTurn = () => {
if (!cur) return;
clearTimeout(cur.timer);
clearTimeout(cur.idle);
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
const next = cur.turn + 1;
const timedOut = cur.r.timeout;
cur = null;
if (next < turns.length && !timedOut) startTurn(next);
else shutdown();
};
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
ws.on('open', () => {
send({
setup: {
model: `models/${model}`,
generationConfig: { responseModalities: ['AUDIO'] },
systemInstruction: { parts: [{ text: instructions }] },
tools: [{ functionDeclarations }],
outputAudioTranscription: {},
},
});
});
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
ws.on('close', (code, reason) => {
shutdown(results.length < turns.length ? `close ${code}: ${reason.toString().slice(0, 200)}` : undefined);
});
ws.on('message', (raw) => {
let m;
try { m = JSON.parse(raw.toString()); } catch { return; }
if (m.setupComplete) {
if (setupMs === null) {
setupMs = Date.now() - tSetup;
startTurn(0);
}
return;
}
if (!cur) return;
clearTimeout(cur.idle);
const r = cur.r;
const lastAt = Date.now();
const sc = m.serverContent;
if (sc?.modelTurn?.parts?.length) {
mark('tFirstTokenMs');
for (const p of sc.modelTurn.parts) if (typeof p.text === 'string' && !p.thought) r.text += p.text;
}
if (sc?.outputTranscription?.text) {
mark('tFirstTokenMs');
r.text += sc.outputTranscription.text;
}
if (m.toolCall?.functionCalls?.length) {
mark('tFirstTokenMs');
mark('tFirstCallMs');
cur.toolCallThisGeneration = true;
cur.toolRounds += 1;
const functionResponses = m.toolCall.functionCalls.map((fc) => {
const c = { name: fc.name, args: fc.args || {}, atMs: Date.now() - cur.t0 };
r.calls.push(c);
return { id: fc.id, name: fc.name, response: onCall(c, cur.turn) };
});
send({ toolResponse: { functionResponses } });
}
if (m.usageMetadata) {
const u = m.usageMetadata;
r.usage.inputTokens += u.promptTokenCount || 0;
r.usage.cachedTokens += u.cachedContentTokenCount || 0;
r.usage.outputTokens += u.responseTokenCount || 0;
r.usage.reasoningTokens += u.thoughtsTokenCount || 0;
r.costUsd += geminiUsageCostUsd(model, u);
}
if (m.goAway) r.errors.push('goAway');
if (sc?.turnComplete) {
r.responses += 1;
if (cur.toolCallThisGeneration && cur.toolRounds < MAX_TOOL_ROUNDS) {
cur.toolCallThisGeneration = false;
if (r.text && !r.text.endsWith(' | ')) r.text += ' | ';
// The follow-up generation normally starts within a second; if
// the model has nothing more to say, end the turn at its last
// message rather than charging the wait to its latency.
const c = cur;
c.idle = setTimeout(() => { if (cur === c) { c.r.tDoneMs = lastAt - c.t0; endTurn(); } }, IDLE_AFTER_TOOL_MS);
return;
}
endTurn();
}
});
});
},
};
return adapter;
}
+263
View File
@@ -0,0 +1,263 @@
/**
* Local-model adapters: stateless chat requests, so every phrase is a fresh
* context by construction. Same system prompt (realtimeInstructions()) and
* the production tools converted with toChatTools() (schema-normalized).
*
* openai-chat any OpenAI-compatible server (LM Studio :1234/v1, Ollama
* :11434/v1). Streams SSE; timing is client-side.
* ollama Ollama's native /api/chat. Same message and tool formats as
* Chat Completions, but it can set num_ctx (the OpenAI shim
* cannot — the ~15k-token prompt would be silently truncated
* at a small default context) and turn thinking off, and it
* reports server-side prefill/decode/load timings.
*
* Both mirror the Realtime loop: stub tool results, then continue until a
* response has no tool calls, up to 4 continuations.
*/
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
import { toChatTools } from '../toolFormats.mjs';
const MAX_CONTINUATIONS = 4;
function emptyResult() {
return {
calls: [], text: '', errors: [], responses: 0,
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: 0, timeout: false,
local: { prefillTokS: null, decodeTokS: null, promptEvalCount: null, evalCount: null, loadMs: null, firstResponseMs: null },
};
}
function parseArgs(raw) {
if (raw && typeof raw === 'object') return raw;
try { return JSON.parse(raw || '{}'); } catch { return { __unparsed: String(raw).slice(0, 400) }; }
}
async function* lines(body) {
const decoder = new TextDecoder();
let buf = '';
for await (const chunk of body) {
buf += decoder.decode(chunk, { stream: true });
let i;
while ((i = buf.indexOf('\n')) >= 0) {
yield buf.slice(0, i);
buf = buf.slice(i + 1);
}
}
if (buf.trim()) yield buf;
}
/**
* Shared multi-response loop over one or more user turns in ONE growing
* message history. `request(messages, r, t0, first, deadline)` performs one
* streamed completion and returns `{ content, toolCalls:[{id,name,args}] }`.
*/
async function runChatDialogue({ turns, onCall, request, turnTimeoutMs, contexts = [] }) {
const messages = [{ role: 'system', content: realtimeInstructions() }];
const results = [];
for (let turn = 0; turn < turns.length; turn += 1) {
const r = emptyResult();
if (contexts[turn]) messages.push({ role: 'system', content: JSON.stringify(contexts[turn]) });
messages.push({ role: 'user', content: turns[turn] });
const t0 = Date.now();
const deadline = t0 + turnTimeoutMs;
try {
for (let round = 0; round <= MAX_CONTINUATIONS; round += 1) {
if (Date.now() > deadline) { r.timeout = true; break; }
const out = await request(messages, r, t0, round === 0, deadline);
r.responses += 1;
if (out.content) r.text += (r.text ? ' | ' : '') + out.content.trim();
if (!out.toolCalls.length) {
messages.push({ role: 'assistant', content: out.content || '' });
break;
}
messages.push({ role: 'assistant', content: out.content || '', tool_calls: out.toolCalls.map((tc) => tc.wire) });
for (const tc of out.toolCalls) {
const c = { name: tc.name, args: tc.args, atMs: tc.atMs };
r.calls.push(c);
messages.push({ role: 'tool', tool_call_id: tc.id, tool_name: tc.name, name: tc.name, content: JSON.stringify(onCall(c, turn)) });
}
}
} catch (e) {
if (e?.name === 'TimeoutError' || e?.name === 'AbortError') r.timeout = true;
else r.errors.push(String(e?.message || e).slice(0, 300));
}
r.tDoneMs = Date.now() - t0;
results.push(r);
if (r.timeout || r.errors.length) {
while (results.length < turns.length) {
const skipped = emptyResult();
skipped.errors.push('dialogue aborted after an earlier turn failed');
results.push(skipped);
}
break;
}
}
return results;
}
// ── OpenAI-compatible (SSE) ─────────────────────────────────────────────────
export function createOpenAiChatAdapter({ model, baseUrl, temperature = 0.8, extraBody = {}, turnTimeoutMs = 180_000, apiKey = 'local' }) {
const tools = toChatTools(GEV_REALTIME_TOOLS);
const request = async (messages, r, t0, first, deadline) => {
const res = await fetch(`${baseUrl.replace(/\/$/, '')}/chat/completions`, {
method: 'POST',
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
body: JSON.stringify({
model, messages, tools, tool_choice: 'auto', temperature, stream: true,
stream_options: { include_usage: true }, ...extraBody,
}),
});
if (!res.ok) throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`);
const tStart = Date.now();
let tFirst = null;
let content = '';
const byIndex = new Map();
let usage = null;
let firstCallDoneAt = null;
for await (const line of lines(res.body)) {
if (!line.startsWith('data:')) continue;
const data = line.slice(5).trim();
if (data === '[DONE]') break;
let m;
try { m = JSON.parse(data); } catch { continue; }
if (m.usage) usage = m.usage;
const choice = m.choices?.[0];
const d = choice?.delta || {};
const now = Date.now();
if ((d.content || d.reasoning_content || d.reasoning || d.tool_calls?.length) && tFirst === null) {
tFirst = now;
if (r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
}
if (d.content) content += d.content;
for (const tc of d.tool_calls || []) {
const idx = tc.index ?? byIndex.size;
if (idx > 0 && byIndex.has(0) && firstCallDoneAt === null) firstCallDoneAt = now;
const cur = byIndex.get(idx) || { id: tc.id || `call_${idx}`, name: '', argText: '' };
if (tc.id) cur.id = tc.id;
if (tc.function?.name) cur.name += tc.function.name;
if (tc.function?.arguments) cur.argText += typeof tc.function.arguments === 'string' ? tc.function.arguments : JSON.stringify(tc.function.arguments);
byIndex.set(idx, cur);
}
if (choice?.finish_reason && byIndex.size && firstCallDoneAt === null) firstCallDoneAt = now;
}
const tEnd = Date.now();
if (byIndex.size && firstCallDoneAt === null) firstCallDoneAt = tEnd;
if (byIndex.size && r.tFirstCallMs === null) r.tFirstCallMs = firstCallDoneAt - t0;
if (usage) {
r.usage.inputTokens += usage.prompt_tokens || 0;
r.usage.cachedTokens += usage.prompt_tokens_details?.cached_tokens || 0;
r.usage.outputTokens += usage.completion_tokens || 0;
r.usage.reasoningTokens += usage.completion_tokens_details?.reasoning_tokens || 0;
}
if (first) {
r.local.firstResponseMs = tEnd - tStart;
if (usage && tFirst) {
r.local.promptEvalCount = usage.prompt_tokens ?? null;
r.local.evalCount = usage.completion_tokens ?? null;
// Client-side approximations: prefill ≈ prompt / (request → first token);
// decode ≈ (completion − 1) / (first token → end of stream).
r.local.prefillTokS = usage.prompt_tokens ? usage.prompt_tokens / Math.max(0.001, (tFirst - tStart) / 1000) : null;
r.local.decodeTokS = usage.completion_tokens > 1 && tEnd > tFirst ? (usage.completion_tokens - 1) / ((tEnd - tFirst) / 1000) : null;
r.local.timingSource = 'client';
}
}
const toolCalls = [...byIndex.values()].map((tc) => ({
id: tc.id, name: tc.name, args: parseArgs(tc.argText), atMs: firstCallDoneAt - t0,
wire: { id: tc.id, type: 'function', function: { name: tc.name, arguments: tc.argText || '{}' } },
}));
return { content, toolCalls };
};
return {
provider: 'openai-chat',
model,
settings: { baseUrl, temperature, tool_choice: 'auto', toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase', ...extraBody },
async open() {},
async close() {},
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
};
}
// ── Ollama native (/api/chat, NDJSON) ───────────────────────────────────────
export function createOllamaAdapter({ model, baseUrl = 'http://localhost:11434', temperature = 0.8, numCtx = 32768, think = false, keepAlive = '30m', turnTimeoutMs = 300_000 }) {
const tools = toChatTools(GEV_REALTIME_TOOLS);
let thinkParam = think;
const request = async (messages, r, t0, first, deadline) => {
const wireMessages = messages.map((m) => {
if (m.role === 'assistant' && m.tool_calls) {
return { role: 'assistant', content: m.content, tool_calls: m.tool_calls.map((tc) => ({ function: { name: tc.function.name, arguments: parseArgs(tc.function.arguments) } })) };
}
if (m.role === 'tool') return { role: 'tool', tool_name: m.tool_name, content: m.content };
return m;
});
const body = { model, messages: wireMessages, tools, stream: true, keep_alive: keepAlive, options: { temperature, num_ctx: numCtx } };
if (thinkParam !== null) body.think = thinkParam;
const res = await fetch(`${baseUrl}/api/chat`, {
method: 'POST',
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
if (!res.ok) {
const errText = await res.text();
if (/does not support thinking/i.test(errText) && thinkParam !== null) {
thinkParam = null;
return request(messages, r, t0, first, deadline);
}
throw new Error(`HTTP ${res.status}: ${errText.slice(0, 300)}`);
}
let content = '';
const toolCalls = [];
let final = null;
for await (const line of lines(res.body)) {
if (!line.trim()) continue;
let m;
try { m = JSON.parse(line); } catch { continue; }
if (m.error) throw new Error(m.error);
const now = Date.now();
const msg = m.message || {};
if ((msg.content || msg.thinking || msg.tool_calls?.length) && r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
if (msg.content) content += msg.content;
for (const tc of msg.tool_calls || []) {
if (r.tFirstCallMs === null) r.tFirstCallMs = now - t0;
const i = `${messages.length}_${toolCalls.length}`;
const args = parseArgs(tc.function?.arguments);
toolCalls.push({
id: `call_${i}`, name: tc.function?.name, args, atMs: now - t0,
wire: { id: `call_${i}`, type: 'function', function: { name: tc.function?.name, arguments: JSON.stringify(args) } },
});
}
if (m.done) final = m;
}
if (final) {
r.usage.inputTokens += final.prompt_eval_count || 0;
r.usage.outputTokens += final.eval_count || 0;
if (first) {
const ns = 1e9;
r.local.promptEvalCount = final.prompt_eval_count ?? null;
r.local.evalCount = final.eval_count ?? null;
r.local.prefillTokS = final.prompt_eval_duration ? final.prompt_eval_count / (final.prompt_eval_duration / ns) : null;
r.local.decodeTokS = final.eval_duration ? final.eval_count / (final.eval_duration / ns) : null;
r.local.loadMs = final.load_duration ? final.load_duration / 1e6 : null;
r.local.firstResponseMs = final.total_duration ? final.total_duration / 1e6 : null;
r.local.timingSource = 'ollama';
}
}
return { content, toolCalls };
};
return {
provider: 'ollama',
model,
get settings() {
return { baseUrl, api: '/api/chat', temperature, num_ctx: numCtx, think: thinkParam === null ? 'unsupported by model (param omitted)' : thinkParam, keep_alive: keepAlive, toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase' };
},
async open() {},
async close() {},
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
};
}
@@ -0,0 +1,167 @@
/**
* OpenAI Realtime adapter — text turns over the Realtime WebSocket.
*
* Each dialogue (one phrase, or a multi-turn dialogue) opens a FRESH session
* configured with production's exact session body (captured from the token
* handler) and sends its user text items one turn at a time. Each turn
* mirrors qa-voice-routing's loop: every function call gets a stub output, and
* a continuation response is requested after `response.done` (never while a
* response is active), up to 4 continuations.
*/
import WebSocket from 'ws';
import { providerKey } from '../keys.mjs';
import { captureProductionSession } from '../productionConfig.mjs';
import { openAiUsageCostUsd } from '../pricing.mjs';
const MAX_CONTINUATIONS = 4;
export function newTurnResult() {
return {
calls: [], text: '', errors: [], responses: 0,
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: null, timeout: false,
};
}
export function createOpenAiRealtimeAdapter({ model, turnTimeoutMs = 45_000 }) {
let session = null;
const adapter = {
provider: 'openai-realtime',
model,
settings: {},
async open() {
session = await captureProductionSession(model);
adapter.settings = {
reasoning: session.reasoning,
truncation: session.truncation,
tool_choice: session.tool_choice,
temperature: 'not settable (GA Realtime session has no temperature field)',
output_modalities: ['text'],
context: 'fresh session per phrase / dialogue',
};
},
async close() {},
async runTurn({ text, onCall, context }) {
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
},
runDialogue({ turns, onCall, contexts = [] }) {
return new Promise((resolveAll) => {
const tSetup = Date.now();
const ws = new WebSocket(`wss://api.openai.com/v1/realtime?model=${encodeURIComponent(model)}`, {
headers: { Authorization: `Bearer ${providerKey('openai')}` },
});
const results = [];
let setupMs = null;
let cur = null; // { r, t0, continuations, needContinuation, turn, timer }
let closed = false;
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
const shutdown = (error) => {
if (closed) return;
closed = true;
if (cur) {
clearTimeout(cur.timer);
if (error) cur.r.errors.push(error);
cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
cur = null;
}
while (results.length < turns.length) {
const r = newTurnResult();
r.errors.push(error || 'dialogue aborted');
results.push(r);
}
try { ws.terminate(); } catch { /* noop */ }
resolveAll(results);
};
const startTurn = (turn) => {
const r = newTurnResult();
r.setupMs = turn === 0 ? setupMs : 0;
cur = { r, t0: Date.now(), continuations: 0, needContinuation: false, turn };
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
// Optional inert system item first (the app's pointer_context shape).
if (contexts[turn]) send({ type: 'conversation.item.create', item: { type: 'message', role: 'system', content: [{ type: 'input_text', text: JSON.stringify(contexts[turn]) }] } });
send({ type: 'conversation.item.create', item: { type: 'message', role: 'user', content: [{ type: 'input_text', text: turns[turn] }] } });
send({ type: 'response.create', response: { output_modalities: ['text'] } });
};
const endTurn = () => {
if (!cur) return;
clearTimeout(cur.timer);
cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
const next = cur.turn + 1;
const timedOut = cur.r.timeout;
cur = null;
if (next < turns.length && !timedOut) startTurn(next);
else shutdown();
};
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
ws.on('close', () => shutdown(results.length < turns.length ? 'socket closed' : undefined));
ws.on('message', (raw) => {
let m;
try { m = JSON.parse(raw.toString()); } catch { return; }
if (m.type === 'session.created') {
send({ type: 'session.update', session });
return;
}
if (m.type === 'session.updated' && setupMs === null) {
setupMs = Date.now() - tSetup;
startTurn(0);
return;
}
if (m.type === 'error') {
if (!cur) { shutdown(m.error?.message || 'realtime error'); return; }
cur.r.errors.push(m.error?.message || 'realtime error');
return;
}
if (!cur) return;
const r = cur.r;
if (
m.type === 'response.output_text.delta' ||
m.type === 'response.function_call_arguments.delta' ||
m.type === 'response.output_audio_transcript.delta'
) mark('tFirstTokenMs');
if (m.type === 'response.output_text.delta' || m.type === 'response.output_audio_transcript.delta') {
r.text += m.delta || '';
}
if (m.type === 'response.output_item.done' && m.item?.type === 'function_call') {
mark('tFirstTokenMs');
mark('tFirstCallMs');
let args = {};
try { args = JSON.parse(m.item.arguments || '{}'); } catch { args = { __unparsed: m.item.arguments }; }
const c = { name: m.item.name, args, atMs: Date.now() - cur.t0 };
r.calls.push(c);
send({
type: 'conversation.item.create',
item: { type: 'function_call_output', call_id: m.item.call_id, output: JSON.stringify(onCall(c, cur.turn)) },
});
cur.needContinuation = true;
}
if (m.type === 'response.done') {
r.responses += 1;
const u = m.response?.usage;
if (u) {
r.usage.inputTokens += u.input_tokens || 0;
r.usage.cachedTokens += u.input_token_details?.cached_tokens || 0;
r.usage.outputTokens += u.output_tokens || 0;
r.usage.reasoningTokens += u.output_token_details?.reasoning_tokens || 0;
r.costUsd += openAiUsageCostUsd(model, u);
}
if (m.response?.status === 'failed') r.errors.push(JSON.stringify(m.response?.status_details).slice(0, 200));
if (cur.needContinuation && cur.continuations < MAX_CONTINUATIONS) {
cur.needContinuation = false;
cur.continuations += 1;
if (r.text) r.text += ' | ';
send({ type: 'response.create', response: { output_modalities: ['text'] } });
return;
}
endTurn();
}
});
});
},
};
return adapter;
}
@@ -0,0 +1 @@
[{"type":"function","name":"fly_to_location","description":"Fly the God's Eye View camera to a known city, geocoded country/region/city/landmark, or explicit WGS84 coordinate. Countries/cities frame the whole place; landmarks/buildings use close framing.","parameters":{"type":"object","additionalProperties":false,"properties":{"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID. Use when the requested place matches one of these cities."},"query":{"type":"string","description":"Plain place search query, e.g. \"London\", \"Eiffel Tower\", or \"Dubai Marina\"."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180},"viewMode":{"type":"string","enum":["close","overview"],"description":"Optional framing intent. Usually omit this; GEV infers whole-place framing for countries/cities and close framing for landmarks."},"rangeM":{"type":"number","minimum":100,"maximum":20000000,"description":"Optional camera range from the target in meters. Omit it for automatic whole-country/whole-city or close-landmark framing; provide it only when the user explicitly requests a numeric height or distance."},"waitForArrival":{"type":"boolean","description":"Set true when a later tool depends on the destination viewport. The result then waits for the camera flight and returns arrived=true; cancellation returns ok=false."}}}},{"type":"function","name":"select_nearest_aircraft","description":"Atomically fly to a place, wait for arrival, enable and load Flights or Military Flights in that viewport, exclude on-ground records, and select/follow the nearest airborne aircraft. Healthy fallback feeds remain usable and are reported in the result. This does not open Contacts or Cockpit.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military"],"description":"Aircraft layer to enable and search. Use flights unless the user explicitly asks for military aircraft."},"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID when the place matches one of these cities."},"locationQuery":{"type":"string","maxLength":160,"description":"Free-form destination when no locationId matches."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180}},"required":["layerId"]}},{"type":"function","name":"adjust_camera_zoom","description":"Move the current Cesium camera closer to or farther from what it is presently looking at. Use for relative zoom requests without changing location.","parameters":{"type":"object","additionalProperties":false,"properties":{"direction":{"type":"string","enum":["in","out"]},"amount":{"type":"string","enum":["little","medium","lot"],"description":"Use little for phrases like \"a bit\" or \"a little\", medium for ordinary zoom requests, and lot for \"way out/in\"."}},"required":["direction","amount"]}},{"type":"function","name":"zoom_to_globe","description":"Pull the camera out to an ABSOLUTE full-Earth globe view (~18,000 km altitude, the whole planet in frame), keeping the current region centered. Use for \"globe view\", \"whole earth\", \"see the planet\", \"zoom all the way out\". Never use adjust_camera_zoom for these — its relative steps cannot reach the globe.","parameters":{"type":"object","additionalProperties":false,"properties":{}}},{"type":"function","name":"set_layer_visibility","description":"Enable or disable one registered God's Eye View data layer.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","description":"Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.","enum":["flights","military","earthquakes","satellites","rocket-launches","traffic","cctv","radio","bikeshare","ais-live-vessels","local-datacenters","local-dams","telegeography-submarine-cables","local-firms","alpr-cameras","local-adsb"]},"enabled":{"type":"boolean"}},"required":["layerId","enabled"]}},{"type":"function","name":"show_data_layers_menu","description":"Open the data layers dropdown/menu and optionally scroll to a specific layer row without toggling it.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military","earthquakes","satellites","traffic","ccLine truncated
+281
View File
@@ -0,0 +1,281 @@
/**
* Pure grading for voice-routing runs. No I/O, no network.
*
* `gradeCore` reproduces the pass/fail semantics of the routing layer in
* `scripts/qa-voice-routing.mjs` (which now calls it), plus a few extra
* benchmark signals. `gradeCoverage` sorts coverage-probe turns into
* correct / honest-refusal / hallucinated-capability / wrong-tool.
*/
/**
* Spot-check `expected` against `actual` args: substring match (case
* insensitive) for strings, exact match for everything else. An array value
* is matched through its comma-joined string form, so `{ layers: 'earthquakes' }`
* matches `layers: ['earthquakes']`.
*/
export function matchArgs(expected, actual) {
return Object.entries(expected).every(([k, v]) => {
const got = actual?.[k];
if (typeof v === 'string') return String(got ?? '').toLowerCase().includes(v.toLowerCase());
return got === v;
});
}
/** Build a call matcher: tool name plus optional arg spot-check. */
export function call(name, args) {
const matcher = (c) => c?.name === name && (!args || matchArgs(args, c.args));
matcher.toolName = name;
return matcher;
}
/** Build a call matcher from a tool name and a predicate over its args. */
export function where(name, predicate) {
const matcher = (c) => c?.name === name && Boolean(predicate(c.args || {}));
matcher.toolName = name;
return matcher;
}
/** Human label for a phrase's expectation. */
export function expectLabel(phrase) {
if (phrase.expectNone) return '(no tool)';
if (Array.isArray(phrase.expect)) return phrase.expect.join('+');
if (phrase.expect?.oneOf) return phrase.expect.oneOf.join('|');
return String(phrase.expect);
}
/** Every tool name that satisfies a phrase (union for oneOf / arrays). */
export function expectedToolNames(phrase) {
if (phrase.expectNone) return [];
if (Array.isArray(phrase.expect)) return [...phrase.expect];
if (phrase.expect?.oneOf) return [...phrase.expect.oneOf];
return [phrase.expect];
}
/**
* Grade one core-suite turn.
*
* `ok` is exactly the qa-voice-routing verdict: expectNone passes only with no
* calls; an array needs every tool; oneOf needs any; a string needs that tool;
* then `args` / `argsByTool` spot checks. The rest are benchmark extras:
* toolOk tool-name routing alone (before the args check)
* argsOk null when the phrase has no arg spot check
* firstCallOk the FIRST call was an expected tool (what a user sees first)
* falsePositive a conversational turn produced a tool call
* extraCalls calls to tools outside the expected set
* typedArgsOk benchmark-only typed-argument check (`typedArgs`: exact
* enum values, layer ids, numeric thresholds, sort order);
* null when the phrase has none. Not part of `ok`.
* fullCallOk toolOk and every arg check (args, argsByTool, typedArgs)
*/
export function gradeCore(phrase, calls) {
const names = calls.map((c) => c.name);
if (phrase.expectNone) {
return {
ok: names.length === 0,
toolOk: names.length === 0,
fullCallOk: names.length === 0,
argsOk: null,
typedArgsOk: null,
firstCallOk: names.length === 0,
falsePositive: names.length > 0,
extraCalls: names,
detail: names.length ? `unexpected calls: ${names.join(',')}` : 'clean conversational turn',
};
}
const expected = expectedToolNames(phrase);
let toolOk;
if (Array.isArray(phrase.expect)) toolOk = phrase.expect.every((n) => names.includes(n));
else if (phrase.expect?.oneOf) toolOk = names.some((n) => phrase.expect.oneOf.includes(n));
else toolOk = names.includes(phrase.expect);
let detail = `called: ${names.join(',') || '(none)'}`;
let argsOk = null;
if (phrase.args) {
const target = calls.find((c) => (Array.isArray(phrase.expect) ? true : expected.includes(c.name)));
argsOk = Boolean(target) && matchArgs(phrase.args, target.args);
if (toolOk && !argsOk) detail += ` args mismatch: ${JSON.stringify(target?.args)}`;
}
if (phrase.argsByTool) {
let allOk = true;
for (const [toolName, expectedArgs] of Object.entries(phrase.argsByTool)) {
const target = calls.find((c) => c.name === toolName);
if (!target || !matchArgs(expectedArgs, target.args)) {
allOk = false;
if (toolOk) detail += ` ${toolName} args mismatch: ${JSON.stringify(target?.args)}`;
}
}
argsOk = argsOk === null ? allOk : argsOk && allOk;
}
const typedArgsOk = phrase.typedArgs ? phrase.typedArgs.every((m) => calls.some((c) => m(c))) : null;
if (toolOk && typedArgsOk === false) detail += ` typed args miss: ${JSON.stringify(calls.map((c) => c.args)).slice(0, 200)}`;
return {
ok: toolOk && argsOk !== false,
toolOk,
fullCallOk: toolOk && argsOk !== false && typedArgsOk !== false,
argsOk,
typedArgsOk,
firstCallOk: names.length > 0 && expected.includes(names[0]),
falsePositive: false,
extraCalls: names.filter((n) => !expected.includes(n)),
detail,
};
}
/**
* Grade a two-turn dialogue from each turn's calls. The headline `ok` is
* turn 2's full-call accuracy (tool + typed args), since turn 2 is the one
* that depends on context; turn 1 is reported alongside.
*/
export function gradeDialogue(dialogue, turnCalls) {
const turn1 = gradeCore(dialogue.first, turnCalls[0] || []);
const turn2 = gradeCore(dialogue.second, turnCalls[1] || []);
return {
ok: turn2.fullCallOk,
toolOk: turn2.toolOk,
fullCallOk: turn2.fullCallOk,
turn1Ok: turn1.fullCallOk,
turn1,
turn2,
detail: `t1 ${turn1.detail} | t2 ${turn2.detail}`,
};
}
// ── JSON-schema argument validation (the subset GEV tools use) ─────────────
const TYPE_CHECKS = {
string: (v) => typeof v === 'string',
number: (v) => typeof v === 'number' && Number.isFinite(v),
integer: (v) => Number.isInteger(v),
boolean: (v) => typeof v === 'boolean',
object: (v) => v !== null && typeof v === 'object' && !Array.isArray(v),
array: (v) => Array.isArray(v),
null: (v) => v === null,
};
/**
* Validate `value` against a JSON schema (type / enum / anyOf / required /
* properties / additionalProperties:false / items / min/max bounds).
* Returns a list of human-readable violations; empty means valid.
*/
export function validateArgs(schema, value, where = 'args') {
const errors = [];
if (!schema || typeof schema !== 'object') return errors;
if (schema.type) {
const types = Array.isArray(schema.type) ? schema.type : [schema.type];
if (!types.some((t) => TYPE_CHECKS[t]?.(value))) {
errors.push(`${where} should be ${types.join('|')}`);
return errors;
}
}
if (schema.enum && !schema.enum.includes(value)) {
errors.push(`${where}=${JSON.stringify(value)} is not one of ${schema.enum.join(', ')}`);
}
if (typeof value === 'string') {
if (typeof schema.minLength === 'number' && value.length < schema.minLength)
errors.push(`${where} needs at least ${schema.minLength} characters`);
if (typeof schema.pattern === 'string' && !new RegExp(schema.pattern).test(value))
errors.push(`${where} does not match ${schema.pattern}`);
}
if (typeof value === 'number') {
if (typeof schema.minimum === 'number' && value < schema.minimum) errors.push(`${where} below ${schema.minimum}`);
if (typeof schema.maximum === 'number' && value > schema.maximum) errors.push(`${where} above ${schema.maximum}`);
}
if (Array.isArray(value)) {
if (typeof schema.minItems === 'number' && value.length < schema.minItems) errors.push(`${where} needs ≥${schema.minItems} items`);
if (schema.items) value.forEach((item, i) => errors.push(...validateArgs(schema.items, item, `${where}[${i}]`)));
}
if (TYPE_CHECKS.object(value) && (schema.properties || schema.required)) {
for (const key of schema.required || []) {
if (!(key in value)) errors.push(`${where}.${key} is required`);
}
for (const [key, v] of Object.entries(value)) {
const sub = schema.properties?.[key];
if (!sub) {
if (schema.additionalProperties === false) errors.push(`${where}.${key} is not a known argument`);
continue;
}
errors.push(...validateArgs(sub, v, `${where}.${key}`));
}
}
if (Array.isArray(schema.anyOf)) {
const branches = schema.anyOf.map((branch) =>
validateArgs(branch, value, where),
);
if (!branches.some((branchErrors) => branchErrors.length === 0))
errors.push(`${where} does not match any allowed shape`);
}
return errors;
}
/** Validate a tool call against the Realtime tool list. */
export function validateCall(tools, c) {
const tool = tools.find((t) => t.name === c.name);
if (!tool) return [`unknown tool ${c.name}`];
return validateArgs(tool.parameters, c.args ?? {});
}
// ── Coverage probe grading ─────────────────────────────────────────────────
/** Tools that only read state; they never count as acting on the request. */
export const LOOKUP_TOOLS = Object.freeze([
'get_entity_context',
'get_current_view_state',
'show_data_layers_menu',
'set_panel_open',
]);
export const REFUSAL_RE =
/\b(can(?:no|')t|cannot|unable|not able|don'?t have|do not have|doesn'?t have|does not have|isn'?t (?:available|supported|something)|aren'?t (?:available|supported)|not (?:available|supported|possible|yet)|no (?:access|way|tool|data|layer|feed|live|transit|weather|wind|cyclone|radar|lightning|receiver)|n'?t (?:connected|see any|find)|(?:isn'?t|is no|are no) (?:an? )?(?:\w+ ){0,2}(?:layer|feed|tool|data source|integration)s?|outside what i can|beyond what i can|only (?:have|support|show|cover)|unfortunately|sorry)\b/i;
/**
* Grade one coverage-probe turn.
*
* Order matters:
* 1. a call pattern in `correct` → correct
* 2. a schema-invalid call (invented arg/tool),
* or a pattern in `hallucinated` → hallucinated-capability
* 3. any other acting call (not lookup/neutral) → wrong-tool
* 4. no acting call: text matches `claimText`
* and no refusal wording → hallucinated-capability
* 5. no acting call: refusal wording or a question → honest-refusal
* 6. no acting call, no refusal → hallucinated-capability
* (it answered as if it had the data)
*
* `acceptable` = correct, or honest-refusal on an unsupported item.
*/
export function gradeCoverage(item, calls, text, tools = []) {
// Models write typographic apostrophes ("can’t"); fold them before matching.
const finalText = String(text || '').replace(/[\u2018\u2019\u02bc]/g, "'");
const verdict = (v, reason) => ({
verdict: v,
reason,
acceptable: v === 'correct' || (v === 'honest-refusal' && !item.supported),
});
for (const alternative of item.correct || []) {
if (alternative.every((m) => calls.some((c) => m(c)))) {
return verdict('correct', `matched ${alternative.map((m) => m.toolName || 'pattern').join('+')}`);
}
}
if (tools.length) {
for (const c of calls) {
const errors = validateCall(tools, c);
if (errors.length) return verdict('hallucinated-capability', `invalid ${c.name}: ${errors[0]}`);
}
}
for (const m of item.hallucinated || []) {
const hit = calls.find((c) => m(c));
if (hit) return verdict('hallucinated-capability', `faked capability via ${hit.name}(${JSON.stringify(hit.args).slice(0, 80)})`);
}
const neutral = new Set([...LOOKUP_TOOLS, ...(item.neutral || [])]);
const acting = calls.filter((c) => !neutral.has(c.name));
if (acting.length) return verdict('wrong-tool', `called ${acting.map((c) => c.name).join(',')}`);
const refused = REFUSAL_RE.test(finalText);
if (item.claimText && item.claimText.test(finalText) && !refused) {
return verdict('hallucinated-capability', 'asserted data it cannot have');
}
if (refused || /\?\s*$/.test(finalText.trim())) return verdict('honest-refusal', 'refused or asked');
if (!finalText.trim().replace(/\|/g, '').trim()) {
return verdict('wrong-tool', calls.length ? `only lookups/neutral calls (${calls.map((c) => c.name).join(',')}) and no answer` : 'no tool call and no answer');
}
return verdict('hallucinated-capability', 'answered without a tool or a refusal');
}
+35
View File
@@ -0,0 +1,35 @@
/**
* Runtime key lookup for the voice benchmark. Keys are read from the
* environment, then the macOS Keychain, at call time. They are never written
* to results, logs, or files.
*/
import { execFileSync } from 'node:child_process';
const KEYCHAIN = {
openai: { env: 'OPENAI_API_KEY', service: 'openai-api', account: 'api-key' },
gemini: { env: 'GEMINI_API_KEY', service: 'gemini-api', account: 'api-key' },
};
const cache = new Map();
/** Resolve a provider key from the environment, then the Keychain. */
export function providerKey(provider) {
if (cache.has(provider)) return cache.get(provider);
const spec = KEYCHAIN[provider];
if (!spec) throw new Error(`unknown key provider: ${provider}`);
let value = process.env[spec.env] || '';
if (!value && process.platform === 'darwin') {
try {
value = execFileSync(
'security',
['find-generic-password', '-s', spec.service, '-a', spec.account, '-w'],
{ encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] },
).trim();
} catch {
value = '';
}
}
if (!value) throw new Error(`${spec.env} is not available (env or Keychain)`);
cache.set(provider, value);
return value;
}
+600
View File
@@ -0,0 +1,600 @@
/**
* Shared voice-routing phrase tables.
*
* CORE_PHRASES is the routing table used by `scripts/qa-voice-routing.mjs`
* (layer 1) and the core suite of `scripts/qa-voice-bench.mjs`.
*
* `expect` tool name the model must call (string), every tool in an
* array, or `{ oneOf: [...] }` for documented acceptable variance.
* `expectNone` conversational turns that must NOT call tools.
* `args` spot-check subset matched against the first matching call's
* args (substring match for strings, exact for booleans/numbers).
* `argsByTool` the same spot check, keyed by tool, for multi-intent phrases.
* `typedArgs` benchmark-only matchers for typed arguments (exact enum
* values, layer ids, thresholds, sort direction). They feed the
* benchmark's full-call accuracy and never change the
* qa-voice-routing verdict.
*
* COVERAGE_PROBE is the newer-layers coverage probe: utterances about layers
* and questions that today's tool surface only partly supports. Each item is
* graded into correct / honest-refusal / hallucinated-capability / wrong-tool
* by `gradeCoverage` in ./grade.mjs.
*/
import { call, where } from './grade.mjs';
export const CORE_PHRASES = Object.freeze([
// — navigation & framing —
{ phrase: 'Take me to Tokyo', expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => a.locationId === 'tokyo' || /tokyo/i.test(a.query || ''))] },
{ phrase: 'Fly to the Golden Gate Bridge', expect: 'fly_to_location' },
{ phrase: 'Go to Sixth Street in Austin', expect: 'fly_to_location' },
{ phrase: 'Show me the Alps from above', expect: { oneOf: ['fly_to_location', 'frame_overhead'] } },
{ phrase: 'Zoom in a bit', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in' && a.amount === 'little')] },
{ phrase: 'Zoom out a little', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'out' && a.amount === 'little')] },
{ phrase: 'Zoom out to a globe view', expect: 'zoom_to_globe' },
{ phrase: 'Show me the whole earth', expect: 'zoom_to_globe' },
{ phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead', typedArgs: [where('frame_overhead', (a) => a.target === 'flights')] },
// — the satellites trap: data layer, never basemap —
{ phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } },
{ phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'satellites' && a.enabled === false)] },
{ phrase: 'Switch to Bing aerial', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'bing-aerial')] },
{ phrase: 'Switch the basemap to OSM', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'osm')] },
// — layers —
{ phrase: 'Turn on the flights layer', expect: 'set_layer_visibility', args: { layerId: 'flights' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)] },
{ phrase: 'Show me live vessels', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'ais-live-vessels' && a.enabled === true)] },
{ phrase: 'Turn on the fires layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-firms' && a.enabled === true)] },
{ phrase: 'Turn on street traffic', expect: 'set_layer_visibility', args: { layerId: 'traffic' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'traffic' && a.enabled === true)] },
{ phrase: 'Open the data layers menu', expect: 'show_data_layers_menu' },
{ phrase: 'Show me the datacenter layers', expect: 'show_data_layers_menu', typedArgs: [where('show_data_layers_menu', (a) => a.layerId === 'local-datacenters')] },
{ phrase: 'Turn on the datacenters layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-datacenters' && a.enabled === true)] },
// — visual styles & post-fx —
{ phrase: 'Give me night vision', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance')] },
{ phrase: 'Switch to thermal view', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
{ phrase: 'Back to the normal look', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
{ phrase: 'Turn on bloom', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.bloom?.enabled === true)] },
{ phrase: 'Sharpen the image a touch', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.sharpen && a.sharpen.enabled !== false)] },
// — HUD / detection / panels —
{ phrase: 'Turn the HUD off', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.visible === 'off')] },
{ phrase: 'Switch to the tactical layout', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.layout === 'tactical')] },
{ phrase: 'Turn on detection', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.enabled !== false)] },
{ phrase: 'Set detection density to fifty percent', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.densityPct === 50 || a.mode === 'balanced')] },
// — context questions —
{ phrase: 'What am I looking at right now?', expect: 'get_entity_context' },
{ phrase: 'What city is this below us?', expect: 'get_entity_context' },
{ phrase: 'Is there anything interesting in view?', expect: 'get_entity_context' },
// — tracking —
// Context-free text turns may reasonably look before tracking; either
// routing is correct (production sessions always carry screen context).
{ phrase: 'Track that plane', expect: { oneOf: ['track_entity', 'get_entity_context'] } },
// The session instructions route "nearest" through analyst_query first
// (sortBy distance, limit 1); stubbed results make that first hop the whole
// observable turn.
{ phrase: 'Follow the nearest aircraft', expect: { oneOf: ['track_entity', 'get_entity_context', 'analyst_query'] } },
{ phrase: 'Stop tracking', expect: 'stop_tracking' },
// — CCTV / scenes / ISS —
{ phrase: 'Show me the nearest traffic camera', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'nearest')] },
{ phrase: 'Turn on the camera viewsheds', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'viewshed' && a.enabled !== false)] },
{ phrase: 'Play a news radio station near Austin', expect: 'control_radio', args: { action: 'select', category: 'news', locationId: 'austin' }, typedArgs: [where('control_radio', (a) => a.action === 'select' && a.category === 'news' && a.locationId === 'austin')] },
{ phrase: 'Turn on the radio', expect: 'control_radio', args: { action: 'play' }, typedArgs: [where('control_radio', (a) => a.action === 'play')] },
{ phrase: 'Set the radio volume to thirty percent', expect: 'control_radio', args: { action: 'volume', volumePct: 30 } },
{ phrase: 'Pause the radio', expect: 'control_radio', args: { action: 'pause' }, typedArgs: [where('control_radio', (a) => a.action === 'pause')] },
{ phrase: 'Stop the radio', expect: 'control_radio', args: { action: 'stop' }, typedArgs: [where('control_radio', (a) => a.action === 'stop')] },
{ phrase: 'When does the ISS pass over next?', expect: 'next_iss_pass' },
// — annotations —
{ phrase: 'Annotate the Texas State Capitol and its grounds', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Outline the state of Texas', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Outline Lady Bird Lake', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Draw the walking route from the Capitol to Zilker Park', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'route' && (x.mode ?? 'walking') === 'walking'))] },
{ phrase: 'How far is the Eiffel Tower from the Louvre?', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'arrow'))] },
{ phrase: 'Clear the map', expect: 'clear_annotations' },
// — multi-intent (assert ALL tools fire before speech) —
{
phrase: 'Switch to night vision and turn on the flights layer',
expect: ['set_visual_style', 'set_layer_visibility'],
typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance'), where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)],
},
{
phrase: 'Turn off the HUD and take me to Paris',
expect: ['set_hud', 'fly_to_location'],
typedArgs: [where('set_hud', (a) => a.visible === 'off'), where('fly_to_location', (a) => a.locationId === 'paris' || /paris/i.test(a.query || ''))],
},
{
phrase: 'Go to full planet view and then turn on the radio',
expect: ['zoom_to_globe', 'control_radio'],
argsByTool: { control_radio: { action: 'play' } },
},
// — camera verbs (tools #23/#24) —
{ phrase: 'Orbit around this area slowly', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'orbit' && a.speed === 'slow')] },
{ phrase: 'Pan left a bit', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'pan' && a.direction === 'left')] },
{ phrase: 'Stop moving the camera', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'stop')] },
{ phrase: 'Fly the route we just drew', expect: 'fly_route' },
// — analyst queries (tool #22) —
{ phrase: 'How many flights are over Texas right now?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('flights') && a.scope?.kind === 'region' && /texas/i.test(a.scope?.name || ''))] },
{ phrase: 'Which ships are headed to Oakland?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('ais-live-vessels') && (a.filters || []).some((f) => f.field === 'destination' && /oakland/i.test(String(f.value))))] },
{ phrase: 'What is the biggest fire near Los Angeles?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('local-firms') && ['frp', 'acres'].includes(a.sortBy) && a.sortDir !== 'asc')] },
{ phrase: 'Is anything flying above forty thousand feet?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).some((l) => l === 'flights' || l === 'military') && (a.filters || []).some((f) => f.field === 'altitudeM' && (f.op === 'gt' || f.op === 'gte') && Math.abs(Number(f.value) - 12192) <= 50))] },
// — negative controls: conversation must NOT tool-call —
{ phrase: 'How is your evening going?', expectNone: true },
{ phrase: 'Tell me a fun fact about maps', expectNone: true },
]);
// ── Coverage probe ──────────────────────────────────────────────────────────
//
// Text-mode port of the voice-coverage routing probe. The live probe ran each
// utterance against a real app page (setup steps, real tool results); this
// benchmark has no app, so every item states the scene it assumes in `scene`
// and the model sees only stub tool results. Grading therefore judges the
// ROUTING DECISION and the honesty of the final words, not app behaviour.
//
// supported true when today's tools can satisfy the ask (a refusal is a
// miss); false when the best possible outcome is an honest
// refusal, optionally after a neutral helper call.
// correct call patterns that satisfy the ask (any alternative).
// neutral extra tools that neither help nor hurt (navigation to the
// named place, enabling a related layer that exists).
// hallucinated call patterns that fake the capability (e.g. toggling the
// satellites DATA layer for "recent satellite imagery").
// claimText final-text pattern that asserts data the model cannot have.
//
// Any call whose arguments violate the tool schema (an invented layer id) is
// graded hallucinated-capability regardless of the item.
//
// Two rubrics, so a before/after comparison stays honest:
// v1 the original rubric, graded against the tool schema of 4b56d0e9
// (baseline-tools-4b56d0e9.json) — "by the old yardstick".
// v2 `item.v2` overrides (supported / correct / hallucinated / neutral),
// graded against the CURRENT production tools after the voice layer
// manifest and point-and-ask landed. Use
// coverageRubric(item, 'v2') to get the merged item.
const hasRoute = (c) =>
c.name === 'annotate_map' &&
Array.isArray(c.args?.annotations) &&
c.args.annotations.some((a) => a?.type === 'route');
/** The coverage item as graded under rubric 'v1' (original) or 'v2' (current tools). */
export function coverageRubric(item, version = 'v1') {
if (version !== 'v2' || !item.v2) return item;
return { ...item, ...item.v2 };
}
export const COVERAGE_PROBE = Object.freeze([
{
id: 'quakes-m5-week',
v2: { supported: true },
utterance: 'Show me earthquakes above magnitude 5 this week',
scene: 'globe, no layers on',
expect: 'enable earthquakes + analyst_query magnitude>=5; must say feed covers last 24h only (no week)',
supported: true,
correct: [
[call('analyst_query', { layers: 'earthquakes' })],
[call('set_layer_visibility', { layerId: 'earthquakes', enabled: true })],
],
neutral: ['zoom_to_globe'],
},
{
id: 'trains-london',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })], [call('set_layer_visibility', { layerId: 'transit', enabled: true })]], hallucinated: [] },
utterance: 'How many trains are moving in London right now?',
scene: 'London',
expect: 'honest "no transit data for London / transit not queryable" — no invented count',
supported: false,
neutral: ['fly_to_location'],
hallucinated: [call('analyst_query')],
claimText: /\b\d+\s+(trains?|services)\b/i,
},
{
id: 'transit-on',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'transit', enabled: true })]] },
utterance: 'Turn on transit',
scene: 'Austin',
expect: 'set_layer_visibility transit (no transit id in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('set_layer_visibility', { layerId: 'traffic', enabled: true })],
},
{
id: 'ship-fastest-singapore',
v2: { supported: true },
utterance: 'Which ship near Singapore is fastest?',
scene: 'Singapore Strait, vessels layer on',
expect: 'analyst_query ais-live-vessels near Singapore sortBy speedKts desc',
supported: true,
correct: [[call('analyst_query', { layers: 'ais-live-vessels' })]],
neutral: ['fly_to_location', 'set_layer_visibility'],
},
{
id: 'recent-imagery',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'recent-imagery', enabled: true })]] },
utterance: 'Show recent satellite imagery of this area',
scene: 'Austin',
expect: 'enable recent-imagery for current view (no tool today) — must NOT toggle satellites data layer or silently switch basemap',
supported: false,
hallucinated: [
call('set_layer_visibility', { layerId: 'satellites' }),
call('set_map_stack'),
call('frame_overhead', { target: 'satellites' }),
],
},
{
id: 'wind-here',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'wind', enabled: true })]] },
utterance: "What's the wind doing here?",
scene: 'Austin',
expect: 'enable/inspect wind layer at view centre (no tool today) — honest otherwise',
supported: false,
claimText: /\b\d+\s*(mph|km\/?h|knots?|kts|m\/s|meters per second|miles per hour)\b/i,
},
{
id: 'cyclones-active',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'weather-cyclones' })], [call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true })]] },
utterance: 'Are there any cyclones active right now?',
scene: 'globe',
expect: 'enable cyclones + list active storms (no tool today) — honest otherwise',
supported: false,
claimText: /\b(there (are|is)|currently)\b[^.?!]*\b(cyclones?|storms?|hurricanes?|typhoons?)\b/i,
},
{
id: 'fires-in-drawn-area',
// There is no drawn-area scope yet; the prompt says to say so. Any count
// presented as inside the drawn area is a faked capability.
v2: {
supported: false,
correct: [],
hallucinated: [where('analyst_query', (a) => (a.layers || []).includes('local-firms'))],
},
utterance: 'Count the fires inside the area I drew',
scene: 'Los Angeles, fires layer on, an area drawn around Angeles National Forest',
expect: 'analyst_query local-firms scoped to the drawn polygon (no polygon scope today)',
supported: true,
correct: [[call('analyst_query', { layers: 'local-firms' })]],
},
{
id: 'local-adsb-on',
v2: { supported: true },
utterance: 'Turn on the local ADS-B layer',
scene: 'globe',
expect: 'set_layer_visibility local-adsb; narrate receiver state honestly',
supported: true,
correct: [[call('set_layer_visibility', { layerId: 'local-adsb', enabled: true })]],
},
{
id: 'bikes-near-me',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'bikeshare' })]] },
utterance: 'How many bikes are available near me?',
scene: 'New York, bikeshare layer on',
expect: 'count available bikes at nearby stations (bikeshare not in analyst today) — honest otherwise',
supported: false,
neutral: ['set_layer_visibility'],
claimText: /\b\d+\s+(available\s+)?(bikes?|bicycles?)\b/i,
},
{
id: 'rocket-launch-track',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })], [call('set_context_mode', { mode: 'space-missions' })], [call('set_context_mode', { mode: 'missions' })], [call('analyst_query', { layers: 'rocket-launches' })]] },
utterance: 'Track the next rocket launch',
scene: 'globe',
expect: 'enable rocket-launches / space-missions and select next launch (track_entity cannot) — honest otherwise',
supported: true,
correct: [
[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })],
[call('set_context_mode', { mode: 'space-missions' })],
[call('set_context_mode', { mode: 'missions' })],
],
hallucinated: [call('track_entity')],
},
{
id: 'weather-radar-on',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-radar', enabled: true })]] },
utterance: 'Turn on the weather radar',
scene: 'Austin',
expect: 'set_layer_visibility weather-radar (not in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('set_map_stack'), call('set_visual_style')],
},
{
id: 'lightning',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-lightning', enabled: true })]] },
utterance: 'Show me lightning strikes',
scene: 'globe',
expect: 'set_layer_visibility weather-lightning (not in the voice enum today) — honest otherwise',
supported: false,
},
{
id: 'military-bases',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'military-installations', enabled: true })], [call('analyst_query', { layers: 'military-installations' })]], hallucinated: [] },
utterance: 'Show military bases near here',
scene: 'San Diego',
expect: 'set_layer_visibility military-installations (not in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('annotate_map')],
},
{
id: 'driving-directions',
v2: { supported: true, correct: [[hasRoute], [call('set_layer_visibility', { layerId: 'directions', enabled: true })]] },
utterance: 'Give me driving directions from the Texas Capitol to the Austin airport',
scene: 'Austin',
expect: 'annotate_map route driving (or directions layer)',
supported: true,
correct: [[hasRoute]],
neutral: ['fly_to_location'],
},
{
id: 'satellites-overhead-count',
v2: { supported: true },
utterance: 'How many satellites are overhead right now?',
scene: 'globe, satellites layer on',
expect: 'analyst_query satellites (control: supported)',
supported: true,
correct: [
[call('analyst_query', { layers: 'satellites' })],
[call('frame_overhead', { target: 'satellites' })],
],
neutral: ['set_layer_visibility'],
},
{
id: 'biggest-quake-today',
v2: { supported: true },
utterance: "What's the biggest earthquake today?",
scene: 'globe, earthquakes layer on',
expect: 'analyst_query earthquakes sortBy magnitude desc (control: supported)',
supported: true,
correct: [[call('analyst_query', { layers: 'earthquakes' })]],
neutral: ['set_layer_visibility'],
},
{
id: 'flock-on',
v2: { supported: true },
utterance: 'Turn on the Flock cameras',
scene: 'Austin',
expect: 'set_layer_visibility alpr-cameras (control: supported)',
supported: true,
correct: [[call('set_layer_visibility', { layerId: 'alpr-cameras', enabled: true })]],
hallucinated: [call('control_cctv')],
},
{
id: 'alpr-count',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'alpr-cameras' })]] },
utterance: 'How many license plate cameras are around here?',
scene: 'Austin, ALPR layer on',
expect: 'count alpr-cameras in view (not in analyst today) — honest otherwise',
supported: false,
neutral: ['set_layer_visibility'],
claimText: /\b\d+\s+(license|plate|alpr|flock|cameras?)\b/i,
},
{
id: 'nearest-bus',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })]] },
utterance: 'Which bus is closest to the center of the screen?',
scene: 'Austin, transit layer on',
expect: 'nearest transit vehicle (no transit query/context today) — honest otherwise',
supported: false,
claimText: /\b(route|bus|line)\s+#?\d+\b/i,
},
{
id: 'weather-replay',
v2: { supported: false, hallucinated: [call('control_scene')] },
utterance: 'Replay the last hour of weather radar',
scene: 'globe',
expect: 'weather clock replay (no time tool today) — honest refusal',
supported: false,
},
{
id: 'receiver-count',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'local-adsb' })]], hallucinated: [call('frame_overhead')] },
utterance: 'How many aircraft is my receiver picking up?',
scene: 'globe, no receiver connected',
expect: 'count local-adsb (not in analyst) — honest: no receiver connected',
supported: false,
neutral: ['set_layer_visibility'],
hallucinated: [call('analyst_query'), call('frame_overhead')],
claimText: /\b\d+\s+(aircraft|planes?|contacts?)\b/i,
},
{
id: 'nearest-traffic-camera',
v2: { supported: true },
utterance: 'Show me the nearest traffic camera',
scene: 'Austin',
expect: 'control_cctv nearest (control: supported)',
supported: true,
correct: [[call('control_cctv', { action: 'nearest' })]],
neutral: ['set_layer_visibility'],
},
{
id: 'hurricane-strongest',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true }), call('analyst_query', { layers: 'weather-cyclones' })], [call('analyst_query', { layers: 'weather-cyclones' })]], hallucinated: [] },
utterance: 'Turn on the hurricane tracks and fly me to the strongest storm',
scene: 'globe',
expect: 'enable cyclones + fly to strongest (no tool today) — honest otherwise',
supported: false,
hallucinated: [call('fly_to_location')],
},
]);
// ── Multi-turn dialogues ───────────────────────────────────────────────────
//
// Two-turn dialogues where turn 2 only makes sense given turn 1. Both turns
// run in ONE session/context. Turn 1's tools get the plausible canned results
// in `results` (anything not listed gets the neutral stub), so turn 2 has
// something real to refer back to. `first` and `second` use the CORE_PHRASES
// grading shape (expect / oneOf / expectNone / typedArgs); the headline score
// is turn 2's full-call accuracy.
const flightsTexas = {
ok: true,
count: 212,
scopeLabel: '212 flights over Texas',
feedProvenance: { overall: 'nominal' },
examples: [
{ callsign: 'AAL1402', operator: 'American Airlines', altitudeM: 10972, military: false },
{ callsign: 'SWA2210', operator: 'Southwest Airlines', altitudeM: 11278, military: false },
{ callsign: 'RCH451', operator: 'US Air Force', altitudeM: 9144, military: true },
],
coverage: 'Counts cover data loaded by enabled layers.',
};
const shipsOakland = {
ok: true,
count: 4,
scopeLabel: '4 ships headed to Oakland',
feedProvenance: { overall: 'nominal' },
examples: [
{ name: 'MAERSK KENSINGTON', destination: 'OAKLAND', speedKts: 14.2, distanceKm: 180 },
{ name: 'EVER LOVELY', destination: 'OAKLAND', speedKts: 16.8, distanceKm: 410 },
{ name: 'APL SINGAPURA', destination: 'OAKLAND', speedKts: 12.1, distanceKm: 62 },
],
};
export const DIALOGUES = Object.freeze([
{
id: 'flights-then-military',
turns: ['How many flights are over Texas right now?', 'Now just the military ones'],
results: { analyst_query: [flightsTexas, { ...flightsTexas, count: 9, scopeLabel: '9 military aircraft over Texas', examples: [flightsTexas.examples[2]] }] },
first: { expect: 'analyst_query' },
second: {
expect: 'analyst_query',
typedArgs: [where('analyst_query', (a) => a.followUp === true || (a.layers || []).includes('military') || (a.filters || []).some((f) => f.field === 'military'))],
},
},
{
id: 'quakes-then-biggest',
turns: ['Turn on the earthquakes layer', 'Take me to the biggest one'],
results: {
set_layer_visibility: [{ ok: true, layerId: 'earthquakes', enabled: true, feedState: 'nominal', count: 143 }],
analyst_query: [null, { ok: true, count: 143, results: [{ id: 'us7000abcd', place: 'Kermadec Islands', magnitude: 6.4, latitude: -29.9, longitude: -177.8 }] }],
},
first: { expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'earthquakes' && a.enabled === true)] },
second: {
expect: { oneOf: ['track_entity', 'analyst_query'] },
typedArgs: [(c) =>
(c.name === 'track_entity' && /quake|biggest|largest|strongest|kermadec/i.test(c.args?.query || '')) ||
(c.name === 'analyst_query' && (c.args?.layers || []).includes('earthquakes') && c.args?.sortBy === 'magnitude' && c.args?.sortDir !== 'asc')],
},
},
{
id: 'track-then-stop',
turns: ['Track UAL428', 'Stop'],
results: {
track_entity: [{ ok: true, tracking: 'UAL428', layerId: 'flights' }],
move_camera: [null, { ok: true, stopped: false, tracking: true, note: 'No camera motion was active; an entity is being tracked.' }],
},
first: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => /UAL\s*428/i.test(a.query || ''))] },
second: { expect: 'stop_tracking' },
},
{
id: 'paris-then-closer',
turns: ['Take me to Paris', 'Get a little closer'],
results: { fly_to_location: [{ ok: true, arrived: true, place: 'Paris, France' }] },
first: { expect: 'fly_to_location' },
second: { expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in')] },
},
{
id: 'thermal-then-back',
turns: ['Switch to thermal view', 'Actually, put it back the way it was'],
results: { set_visual_style: [{ ok: true, style: 'thermal', previousStyle: 'normal' }] },
first: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
second: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
},
{
id: 'radio-then-volume',
turns: ['Turn on the radio', 'Turn it down to twenty percent'],
results: { control_radio: [{ ok: true, prepared: true, action: 'play', station: 'KUT 90.5 Austin' }] },
first: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'play')] },
second: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'volume' && a.volumePct === 20)] },
},
{
id: 'mark-then-route',
turns: ['Mark the Texas Capitol on the map', 'Now draw the walking route from there to Zilker Park'],
results: { annotate_map: [{ ok: true, placed: [{ label: 'Texas State Capitol', type: 'pin' }] }] },
first: { expect: 'annotate_map' },
second: {
expect: 'annotate_map',
typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) =>
x.type === 'route' && (x.mode ?? 'walking') === 'walking' && /capitol/i.test(x.points?.[0]?.target || '')))],
},
},
{
id: 'ships-then-closest',
turns: ['Which ships are headed to Oakland?', 'Which of those is closest?'],
results: { analyst_query: [shipsOakland, { ...shipsOakland, count: 1, examples: [shipsOakland.examples[2]] }] },
first: { expect: 'analyst_query' },
second: { expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => a.followUp === true && a.sortBy === 'distance')] },
},
{
id: 'flights-then-thanks',
turns: ['Turn on the flights layer', "Thanks, that's perfect"],
results: { set_layer_visibility: [{ ok: true, layerId: 'flights', enabled: true, feedState: 'nominal', count: 1840 }] },
first: { expect: 'set_layer_visibility' },
second: { expectNone: true },
},
]);
/** Canned result for a dialogue tool call, or null for the neutral stub. */
export function dialogueResult(dialogue, toolName, turn) {
const byTurn = dialogue.results?.[toolName];
if (!byTurn) return null;
return byTurn[turn] ?? null;
}
// ── New capabilities (point-and-ask, layer reach) ─────────────────────────
//
// ROUTING_PHRASES is the qa-voice-routing table, so that harness runs exactly
// its own table. The benchmark runs CAPABILITY_EXTRAS as the separate
// `capability` suite so the core suite stays comparable across tool versions.
/** The full qa-voice-routing table. */
export const ROUTING_PHRASES = CORE_PHRASES;
// Pointer context items in the app's own format (src/voice/pointerContext.js
// output), sent as an inert system item just before the user's words.
const AIRCRAFT_POINTER = {
type: 'pointer_context', at: 'speech_start',
target: { kind: 'aircraft', layer: 'flights', label: 'UPS793' },
ground: { lat: 30.3121, lon: -97.6013 }, screen: { x: 0.608, y: 0.408 },
};
const GROUND_POINTER = {
type: 'pointer_context', at: 'speech_start',
ground: { lat: 30.2849, lon: -97.7341 }, screen: { x: 0.427, y: 0.547 },
};
const isPointer = (v) => String(v || '').toLowerCase() === 'pointer';
const flightsInView = {
ok: true,
count: 3,
scopeLabel: 'in view',
items: [
{ n: 1, id: 'a1b2c3', callsign: 'UAL428', layerKey: 'flights', lat: 30.2, lon: -97.7 },
{ n: 2, id: 'a4b5c6', callsign: 'SWA1201', layerKey: 'flights', lat: 30.3, lon: -97.6 },
{ n: 3, id: 'a7b8c9', callsign: 'AAL88', layerKey: 'flights', lat: 30.1, lon: -97.8 },
],
};
/**
* Extra single-turn phrases (with an optional pointer `context`) and two-turn
* dialogues for the new capabilities. Items with `turns` are graded like
* DIALOGUES (turn 2 full-call); the rest like CORE_PHRASES.
*/
export const CAPABILITY_EXTRAS = Object.freeze([
{ phrase: 'Tell me about this', context: AIRCRAFT_POINTER, expect: 'get_entity_context', typedArgs: [where('get_entity_context', (a) => isPointer(a.scope))] },
{ phrase: 'How many flights are around here?', context: GROUND_POINTER, expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => isPointer(a.scope?.kind))] },
{ phrase: 'Take me there', context: GROUND_POINTER, expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => isPointer(a.query))] },
{ phrase: 'Track that one', context: AIRCRAFT_POINTER, expect: 'track_entity', typedArgs: [where('track_entity', (a) => isPointer(a.query))] },
{ phrase: 'Turn on the weather radar', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'weather-radar' && a.enabled === true)] },
{ phrase: 'Are there any hurricanes right now?', expect: { oneOf: ['analyst_query', 'set_layer_visibility'] }, typedArgs: [(c) => (c.args?.layers || []).includes('weather-cyclones') || c.args?.layerId === 'weather-cyclones'] },
{ phrase: 'Which bus is closest to the middle of the screen?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('transit'))] },
{
id: 'flights-then-second',
turns: ['How many flights are in view?', 'Track the second one'],
results: { analyst_query: [flightsInView] },
first: { expect: 'analyst_query' },
second: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => Number(a.referent) === 2 || /SWA\s*1201/i.test(a.query || ''))] },
},
]);
+58
View File
@@ -0,0 +1,58 @@
/**
* Cost estimation for benchmark turns (USD). Pure.
*
* OpenAI Realtime: the app's own rate table (`src/voice/voiceCost.js`).
* `gpt-realtime-2.1` has no registry entry, so it resolves to the priciest
* known table (gpt-realtime-2) — which matches OpenAI's published 2.1 rates
* as read on 2026-09-23 ($4 / $0.40 cached / $24 per 1M text tokens).
*
* Gemini Live: https://ai.google.dev/gemini-api/docs/pricing, read 2026-09-23,
* gemini-3.8-live paid tier, USD per 1M tokens. Thinking tokens bill as text
* output. The native-audio Live model rejects TEXT output, so the benchmark
* requests AUDIO and the spoken confirmation bills at the audio-output rate.
*/
import { estimateUsageCostUsd, resolveVoiceModelById } from '../../src/voice/voiceCost.js';
export const GEMINI_RATES = Object.freeze({
'gemini-3.8-live': Object.freeze({
textInput: 0.75,
audioInput: 3.0,
textOutput: 4.5,
audioOutput: 12.0,
verifiedOn: '2026-09-23',
}),
});
/** Cost of one Realtime `response.usage` for `model`. */
export function openAiUsageCostUsd(model, usage) {
return estimateUsageCostUsd(usage, resolveVoiceModelById(model).rates);
}
const modalityCount = (details, modality) =>
(Array.isArray(details) ? details : [])
.filter((d) => String(d?.modality || '').toUpperCase() === modality)
.reduce((sum, d) => sum + (Number(d?.tokenCount) || 0), 0);
/**
* Cost of one Gemini Live `usageMetadata`. Unattributed prompt tokens bill
* as audio input and unattributed response tokens as audio output — the
* conservative direction for a spend cap.
*/
export function geminiUsageCostUsd(model, usage) {
const rates = GEMINI_RATES[model] || GEMINI_RATES['gemini-3.8-live'];
if (!usage) return 0;
const prompt = Number(usage.promptTokenCount) || 0;
const response = Number(usage.responseTokenCount) || 0;
const thoughts = Number(usage.thoughtsTokenCount) || 0;
const textIn = modalityCount(usage.promptTokensDetails, 'TEXT');
const audioIn = Math.max(0, prompt - textIn);
const textOut = modalityCount(usage.responseTokensDetails, 'TEXT');
const audioOut = Math.max(0, response - textOut);
const usd =
(textIn * rates.textInput +
audioIn * rates.audioInput +
(textOut + thoughts) * rates.textOutput +
audioOut * rates.audioOutput) /
1_000_000;
return Number.isFinite(usd) && usd > 0 ? usd : 0;
}
+36
View File
@@ -0,0 +1,36 @@
/**
* Capture the exact Realtime session config production mints.
*
* Rather than re-typing the session fields, this drives the production token
* handler (`server/providers/openai/realtime.js`) with a fake upstream fetch
* and returns the body it would have POSTed to OpenAI. Instructions, tools,
* tool_choice, reasoning, and truncation therefore cannot drift from what a
* real GEV mic session receives.
*/
import { createRealtimeTokenHandler } from '../../server/providers/openai/realtime.js';
import { realtimeInstructions } from '../../server/providers/openai/instructions.js';
import { GEV_REALTIME_TOOLS } from '../../server/providers/openai/tools.js';
export { realtimeInstructions, GEV_REALTIME_TOOLS };
/** Return production's `{ session: {...} }` client-secret body for `model`. */
export async function captureProductionSession(model) {
let captured = null;
const handler = createRealtimeTokenHandler({
resolveApiKey: () => 'bench-placeholder',
models: { standard: model, mini: model },
fetchImpl: async (_url, init) => {
captured = JSON.parse(init.body);
return new Response('{}', { status: 200, headers: { 'content-type': 'application/json' } });
},
});
const res = {
statusCode: 0,
headers: {},
setHeader(k, v) { this.headers[k] = v; },
end() {},
};
await handler({ method: 'POST', url: '/api/realtime/token', headers: {}, socket: {} }, res);
if (!captured?.session) throw new Error('production token handler did not produce a session config');
return captured.session;
}
+272
View File
@@ -0,0 +1,272 @@
/**
* Pure aggregation of benchmark rows into summary.json / summary.md.
* Rows: `{ type:'run', ... }` run metadata and `{ type:'turn', ... }` turns.
*/
export function percentile(values, p) {
const xs = values.filter((v) => typeof v === 'number' && Number.isFinite(v)).sort((a, b) => a - b);
if (!xs.length) return null;
const idx = Math.min(xs.length - 1, Math.max(0, Math.ceil((p / 100) * xs.length) - 1));
return xs[idx];
}
const pct = (n, d) => (d ? Math.round((1000 * n) / d) / 10 : null);
const sum = (xs) => xs.reduce((a, b) => a + (Number(b) || 0), 0);
const mean = (xs) => {
const v = xs.filter((x) => typeof x === 'number' && Number.isFinite(x));
return v.length ? sum(v) / v.length : null;
};
const round = (x, d = 0) => (x === null || x === undefined ? null : Math.round(x * 10 ** d) / 10 ** d);
export const COVERAGE_VERDICTS = Object.freeze(['correct', 'honest-refusal', 'hallucinated-capability', 'wrong-tool']);
export function modelKey(row) {
return `${row.provider}:${row.model}${row.variant ? `#${row.variant}` : ''}`;
}
/** Aggregate turn rows into one summary entry per provider×model(×variant). */
export function summarize(rows) {
const runs = rows.filter((r) => r.type === 'run');
const turns = rows.filter((r) => r.type === 'turn');
const keys = [...new Set(turns.map(modelKey))];
const models = keys.map((key) => {
const mine = turns.filter((t) => modelKey(t) === key);
const core = mine.filter((t) => t.suite === 'core');
const cov = mine.filter((t) => t.suite === 'coverage');
const dia = mine.filter((t) => t.suite === 'dialogue');
const capAll = mine.filter((t) => t.suite === 'capability');
// Latency percentiles stay on the core + coverage phrase sets so runs of
// different builds remain comparable.
const single = mine.filter((t) => t.suite === 'core' || t.suite === 'coverage');
const covV2 = cov.filter((t) => t.gradeV2);
const reps = [...new Set(core.map((t) => t.rep))].sort((a, b) => a - b);
const withCall = single.filter((t) => t.tFirstCallMs !== null && t.tFirstCallMs !== undefined);
const typed = core.filter((t) => t.grade?.typedArgsOk !== null && t.grade?.typedArgsOk !== undefined);
const diaFailures = dia.filter((t) => !t.grade?.ok).map((t) => ({
id: t.phraseId, rep: t.rep, expect: t.expect,
turn1: (t.turn1?.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
turn2: (t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
}));
const coverageV2Counts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, covV2.filter((t) => t.gradeV2?.verdict === v).length]));
const capFailures = {};
for (const t of capAll.filter((x) => !x.grade?.fullCallOk)) {
const k = t.phraseId;
capFailures[k] = capFailures[k] || { id: k, expect: t.expect, fails: 0, of: capAll.filter((x) => x.phraseId === k).length, called: [] };
capFailures[k].fails += 1;
capFailures[k].called.push((t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 70)})`).join(', ') || '(none)');
}
const coverageCounts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, cov.filter((t) => t.grade?.verdict === v).length]));
const coreFailures = {};
for (const t of core.filter((x) => !x.grade?.ok)) {
const k = t.phrase;
coreFailures[k] = coreFailures[k] || { phrase: k, expect: t.expect, fails: 0, of: core.filter((x) => x.phrase === k).length, called: [] };
coreFailures[k].fails += 1;
coreFailures[k].called.push((t.calls || []).map((c) => c.name).join(',') || '(none)');
}
const covByItem = {};
for (const t of cov) {
covByItem[t.phraseId] = covByItem[t.phraseId] || { id: t.phraseId, utterance: t.phrase, supported: t.supported, verdicts: [] };
covByItem[t.phraseId].verdicts.push(t.grade?.verdict);
}
const run = runs.filter((r) => modelKey(r) === key).at(-1) || {};
const local = mine.filter((t) => t.local);
return {
key,
provider: mine[0].provider,
model: mine[0].model,
variant: mine[0].variant || null,
settings: run.settings || null,
machine: run.machine || null,
warmup: run.warmup || null,
turns: mine.length,
core: {
n: core.length,
reps: reps.length,
accuracyPct: pct(core.filter((t) => t.grade?.ok).length, core.length),
toolAccuracyPct: pct(core.filter((t) => t.grade?.toolOk).length, core.length),
fullCallAccuracyPct: pct(core.filter((t) => t.grade?.fullCallOk).length, core.length),
typedArgsAccuracyPct: pct(typed.filter((t) => t.grade?.typedArgsOk).length, typed.length),
typedArgsN: typed.length,
argsAccuracyPct: pct(core.filter((t) => t.grade?.argsOk === true).length, core.filter((t) => t.grade?.argsOk !== null && t.grade?.argsOk !== undefined).length),
firstCallAccuracyPct: pct(core.filter((t) => t.grade?.firstCallOk).length, core.length),
falsePositives: core.filter((t) => t.grade?.falsePositive).length,
negativeTurns: core.filter((t) => t.expectNone).length,
perRepAccuracyPct: reps.map((rep) => {
const r = core.filter((t) => t.rep === rep);
return pct(r.filter((t) => t.grade?.ok).length, r.length);
}),
failures: Object.values(coreFailures).sort((a, b) => b.fails - a.fails),
},
coverage: {
n: cov.length,
counts: coverageCounts,
acceptablePct: pct(cov.filter((t) => t.grade?.acceptable).length, cov.length),
supportedCorrectPct: pct(cov.filter((t) => t.supported && t.grade?.verdict === 'correct').length, cov.filter((t) => t.supported).length),
unsupportedHonestPct: pct(cov.filter((t) => !t.supported && t.grade?.acceptable).length, cov.filter((t) => !t.supported).length),
items: Object.values(covByItem),
},
coverageV2: {
n: covV2.length,
counts: coverageV2Counts,
acceptablePct: pct(covV2.filter((t) => t.gradeV2?.acceptable).length, covV2.length),
},
capability: {
n: capAll.length,
toolPct: pct(capAll.filter((t) => t.grade?.toolOk).length, capAll.length),
fullPct: pct(capAll.filter((t) => t.grade?.fullCallOk).length, capAll.length),
failures: Object.values(capFailures).sort((a, b) => b.fails - a.fails),
},
prompt: run.prompt || null,
dialogue: {
n: dia.length,
turn2FullPct: pct(dia.filter((t) => t.grade?.fullCallOk).length, dia.length),
turn2ToolPct: pct(dia.filter((t) => t.grade?.toolOk).length, dia.length),
turn1FullPct: pct(dia.filter((t) => t.grade?.turn1Ok).length, dia.length),
turn2ToolCallP50: percentile(dia.map((t) => t.tFirstCallMs), 50),
failures: diaFailures,
},
latencyMs: {
scope: 'single-turn phrases (core + coverage); send → event, text mode',
firstTokenP50: percentile(single.map((t) => t.tFirstTokenMs), 50),
firstTokenP90: percentile(single.map((t) => t.tFirstTokenMs), 90),
toolCallP50: percentile(withCall.map((t) => t.tFirstCallMs), 50),
toolCallP90: percentile(withCall.map((t) => t.tFirstCallMs), 90),
turnDoneP50: percentile(single.map((t) => t.tDoneMs), 50),
turnDoneP90: percentile(single.map((t) => t.tDoneMs), 90),
setupP50: percentile(single.map((t) => t.setupMs), 50),
},
tokens: {
input: sum(mine.map((t) => t.usage?.inputTokens)),
cached: sum(mine.map((t) => t.usage?.cachedTokens)),
output: sum(mine.map((t) => t.usage?.outputTokens)),
reasoning: sum(mine.map((t) => t.usage?.reasoningTokens)),
inputPerTurnMean: round(mean(mine.map((t) => t.usage?.inputTokens))),
inputPerRequestMean: round(mean(single.filter((t) => t.responses > 0 && t.usage?.inputTokens).map((t) => t.usage.inputTokens / t.responses))),
},
costUsd: round(sum(mine.map((t) => t.costUsd)), 4),
costPerTurnUsd: mine.length ? round(sum(mine.map((t) => t.costUsd)) / mine.length, 5) : null,
errors: mine.filter((t) => t.errors?.length).length,
timeouts: mine.filter((t) => t.timeout).length,
local: local.length
? {
prefillTokSMean: round(mean(local.map((t) => t.local.prefillTokS)), 1),
prefillTokSP50: round(percentile(local.map((t) => t.local.prefillTokS), 50), 0),
decodeTokSMean: round(mean(local.map((t) => t.local.decodeTokS)), 1),
decodeTokSP50: round(percentile(local.map((t) => t.local.decodeTokS), 50), 1),
decodeTokSP90: round(percentile(local.map((t) => t.local.decodeTokS), 90), 1),
}
: null,
};
});
return {
generatedAt: new Date().toISOString(),
cloudSpendUsd: round(sum(turns.filter((t) => t.cloud).map((t) => t.costUsd)), 4),
models,
};
}
const machineBrief = (machine) => {
if (!machine) return '—';
const load1 = String(machine.loadavg || '').split(' ')[0] || '?';
const busy = (machine.busyProcesses || []).filter((p) => Number(p.match(/([\d.]+)%cpu/)?.[1]) >= 100)
.map((p) => p.replace(/ Helper \(GPU\)/, ' GPU').replace(/Google Chrome for Testing/, 'Chrome-for-Testing'));
return `load ${load1}${busy.length ? `; ${busy.join(', ')}` : ''}`;
};
const fmtCounts = (block) => (block?.n
? `${block.counts.correct} / ${block.counts['honest-refusal']} / ${block.counts['hallucinated-capability']} / ${block.counts['wrong-tool']} (n=${block.n})`
: '—');
const fmtMs = (v) => (v === null || v === undefined ? '—' : v >= 10_000 ? `${(v / 1000).toFixed(1)}s` : `${Math.round(v)}ms`);
const fmtPct = (v) => (v === null || v === undefined ? '—' : `${v}%`);
/** Render summary.md from a summarize() result plus free-form notes. */
export function renderMarkdown(summary, { title = 'GEV voice-routing benchmark', notes = [], payload = [] } = {}) {
const out = [`# ${title}`, '', `Generated ${summary.generatedAt}. Cloud spend so far: **$${summary.cloudSpendUsd?.toFixed(2)}**.`, ''];
out.push('> **What this measures.** Tool CHOICE for text turns sent over each provider\'s API with production\'s instructions and tools, with stubbed (always-successful) tool results — the same method as `qa-voice-routing.mjs` layer 1. It is **not** end-to-end spoken accuracy (no speech recognition, no audio, no screenshot/scene context) and **not** spoken latency: every latency below is "send text → event", before any audio would be synthesized or played.', '');
out.push('## Routing accuracy', '');
out.push('| model | core tool-name | core full-call (tool + all arg checks) | typed args (n) | harness verdict (legacy) | first call right | false +ve | per-rep harness verdict | dialogue turn-2 full-call | dialogue turn-1 | coverage v1 C / HR / HC / WT | coverage v1 acceptable | coverage v2 C / HR / HC / WT | coverage v2 acceptable | capability tool / full-call |');
out.push('|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|');
for (const m of summary.models) {
const c = m.coverage.counts;
out.push(`| ${m.key} | ${fmtPct(m.core.toolAccuracyPct)} | ${fmtPct(m.core.fullCallAccuracyPct)} | ${fmtPct(m.core.typedArgsAccuracyPct)} (${m.core.typedArgsN}) | ${fmtPct(m.core.accuracyPct)} (n=${m.core.n}) | ${fmtPct(m.core.firstCallAccuracyPct)} | ${m.core.falsePositives}/${m.core.negativeTurns} | ${m.core.perRepAccuracyPct.map(fmtPct).join(', ')} | ${fmtPct(m.dialogue.turn2FullPct)} (n=${m.dialogue.n}) | ${fmtPct(m.dialogue.turn1FullPct)} | ${c.correct} / ${c['honest-refusal']} / ${c['hallucinated-capability']} / ${c['wrong-tool']} (n=${m.coverage.n}) | ${fmtPct(m.coverage.acceptablePct)} | ${fmtCounts(m.coverageV2)} | ${fmtPct(m.coverageV2.acceptablePct)} | ${m.capability.n ? `${fmtPct(m.capability.toolPct)} / ${fmtPct(m.capability.fullPct)} (n=${m.capability.n})` : '—'} |`);
}
out.push('', 'Core tool-name = right tool(s) called. Full-call = right tool(s) AND every argument check (the legacy `args` spot checks plus the benchmark\'s typed checks: exact layer ids, enum values, numeric thresholds, sort direction). Legacy harness verdict = exactly what `qa-voice-routing.mjs` would report. Coverage: C = correct, HR = honest refusal, HC = hallucinated capability, WT = wrong tool; "acceptable" = correct, or an honest refusal where today\'s tools cannot do it.', '');
out.push('## Latency and cost', '');
out.push('| model | p50 send → first token (text) | p50 send → tool call (text) | p90 send → tool call (text) | p50 send → turn done (text) | dialogue turn-2 p50 → tool call | session setup p50 | input tok/turn | $ total | $/turn |');
out.push('|---|---|---|---|---|---|---|---|---|---|');
for (const m of summary.models) {
out.push(`| ${m.key} | ${fmtMs(m.latencyMs.firstTokenP50)} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${fmtMs(m.latencyMs.turnDoneP50)} | ${fmtMs(m.dialogue.turn2ToolCallP50)} | ${fmtMs(m.latencyMs.setupP50)} | ${m.tokens.inputPerTurnMean ?? '—'} | ${m.costUsd?.toFixed(3)} | ${m.costPerTurnUsd ?? '—'} |`);
}
out.push('', 'Latencies are over single-turn phrases (core + coverage). "Turn done" includes the continuation response after the stubbed tool result. Cloud session setup (WebSocket + session config) is excluded from every latency and shown separately.', '');
const locals = summary.models.filter((m) => m.local);
if (locals.length) {
out.push('## Local runtime (cold vs warm)', '');
out.push('| model | load (wall) | COLD: first request after load → tool call | cold prefill tok/s (full prompt) | prompt tokens | WARM p50 → tool call | WARM p90 → tool call | warm per-turn prefill tok/s p50 | decode tok/s p50 / p90 | machine at start |');
out.push('|---|---|---|---|---|---|---|---|---|---|');
for (const m of locals) {
const w = m.warmup || {};
out.push(`| ${m.key} | ${fmtMs(w.loadWallMs ?? w.loadMs)} | ${fmtMs(w.coldToolCallMs ?? w.warmupTurnMs)}${w.coldToolCallMs == null ? ' (turn, no tool)' : ''} | ${w.prefillTokS ? Math.round(w.prefillTokS) : '—'} | ${w.promptTokens ?? '—'} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${m.local.prefillTokSP50 ?? '—'} | ${m.local.decodeTokSP50 ?? '—'} / ${m.local.decodeTokSP90 ?? '—'} | ${machineBrief(m.machine)} |`);
}
out.push('', 'COLD = the first request after the model is loaded (pays the full ~11k-token prompt prefill). WARM = every later turn. "Warm per-turn prefill tok/s" is prompt tokens ÷ prompt-eval time as the runtime reports it: a value in the tens of thousands means the cached prompt prefix was reused and only the new user text was prefilled; a value near the cold rate means the WHOLE prompt was re-prefilled on every turn. A real voice session pays the cold cost once per model load. "Machine at start" = 1-minute load average and processes above 100% CPU when the run began (other GPU/CPU users on the machine inflate local timings).', '');
}
if (payload.length) {
out.push('## Payload changes per provider', '');
for (const p of payload) out.push(`- ${p}`);
out.push('');
}
out.push('## Failures by model', '');
for (const m of summary.models) {
out.push(`### ${m.key}`, '');
out.push(`Settings: \`${JSON.stringify(m.settings)}\``, '');
if (m.errors || m.timeouts) out.push(`Errors: ${m.errors} turns, timeouts: ${m.timeouts}.`, '');
if (m.core.failures.length) {
out.push('Core misses (legacy verdict):', '');
for (const f of m.core.failures) out.push(`- "${f.phrase}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ')}`);
out.push('');
}
if (m.capability.failures.length) {
out.push('Capability misses (full call):', '');
for (const f of m.capability.failures) out.push(`- "${f.id}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ').slice(0, 400)}`);
out.push('');
}
if (m.dialogue.failures.length) {
out.push('Dialogue misses (turn 2):', '');
for (const f of m.dialogue.failures) out.push(`- ${f.id} rep ${f.rep} (expect ${f.expect}) — t1: ${f.turn1} ⇒ t2: ${f.turn2}`);
out.push('');
}
const badCov = m.coverage.items.filter((i) => i.verdicts.some((v) => v === 'hallucinated-capability' || v === 'wrong-tool' || (i.supported && v !== 'correct')));
if (badCov.length) {
out.push('Coverage misses:', '');
for (const i of badCov) out.push(`- "${i.utterance}" (${i.supported ? 'supported' : 'unsupported'}) — ${i.verdicts.join(', ')}`);
out.push('');
}
}
if (notes.length) {
out.push('## Notes', '');
for (const n of notes) out.push(`- ${n}`);
out.push('');
}
return out.join('\n');
}
/**
* Before → after table for two summarize() results (matched by model key).
* Coverage "before" is the v1 rubric (what the old tools could do); "after"
* shows both v1 (same yardstick) and v2 (the current tool surface).
*/
export function renderComparison(before, after, { beforeLabel = 'before', afterLabel = 'after' } = {}) {
const out = ['## Before → after', '', `Before = ${beforeLabel}; after = ${afterLabel}. Core accuracy = tool + all argument checks (full call). Coverage OK % = correct, or an honest refusal where the tools of that build cannot do it. HC = hallucinated-capability count. p50 = send → tool call (text). Prompt tokens = mean input tokens per model request (cloud: as billed, before cache discounts; local: the cold full-prompt count).`, ''];
out.push('| model | core full-call | coverage OK % (v1 → v1 / v2) | HC (v1 → v1 / v2) | dialogue turn-2 | capability full-call (after) | p50 → tool call | prompt tokens |');
out.push('|---|---|---|---|---|---|---|---|');
const hc = (block) => (block?.n ? block.counts['hallucinated-capability'] : null);
const promptTok = (m) => (m?.warmup?.promptTokens ?? m?.tokens?.inputPerRequestMean ?? null);
for (const a of after.models) {
const b = before.models.find((x) => x.key === a.key);
const arrow = (x, y, f) => `${f(x)} → ${f(y)}`;
const pctF = (v) => (v === null || v === undefined ? '—' : `${v}%`);
const msF = (v) => (v === null || v === undefined ? '—' : `${Math.round(v)}ms`);
const num = (v) => (v === null || v === undefined ? '—' : String(v));
out.push(`| ${a.key} | ${arrow(b?.core.fullCallAccuracyPct, a.core.fullCallAccuracyPct, pctF)} | ${pctF(b?.coverage.acceptablePct)} → ${pctF(a.coverage.acceptablePct)} / ${pctF(a.coverageV2.acceptablePct)} | ${num(hc(b?.coverage))} → ${num(hc(a.coverage))} / ${num(hc(a.coverageV2))} | ${arrow(b?.dialogue.turn2FullPct, a.dialogue.turn2FullPct, pctF)} | ${pctF(a.capability.fullPct)} | ${arrow(b?.latencyMs.toolCallP50, a.latencyMs.toolCallP50, msF)} | ${arrow(promptTok(b), promptTok(a), num)} |`);
}
out.push('');
return out.join('\n');
}
+21
View File
@@ -0,0 +1,21 @@
/**
* Stub tool results fed back to the model after each call. Pure.
*
* core suite: the neutral `{ ok:true }` stub qa-voice-routing has always
* used, so continuations behave exactly as in that harness.
* coverage suite: the same stub, except a call whose arguments violate the
* tool schema gets `{ ok:false, error }` — what the app's runner returns
* for an unknown layer id — so the model's follow-up words can be judged
* for honesty.
*/
import { validateCall } from './grade.mjs';
export const STUB_OK = Object.freeze({ ok: true, note: 'qa-harness stub result' });
export function stubResult(suite, c, tools) {
if (suite === 'coverage') {
const errors = validateCall(tools, c);
if (errors.length) return { ok: false, error: `Invalid arguments: ${errors.join('; ')}` };
}
return { ...STUB_OK };
}
+119
View File
@@ -0,0 +1,119 @@
/**
* Pure converters from the production Realtime tool list
* (`{ type:'function', name, description, parameters }`) to other providers'
* tool formats. Descriptions and schemas are carried over unchanged except
* where a target cannot represent them; every such change is listed below.
*/
const PERMISSIVE_SCALAR = Object.freeze(['string', 'number', 'boolean']);
const clone = (value) => JSON.parse(JSON.stringify(value));
/**
* Normalize a JSON schema for OpenAI-compatible local servers.
*
* Only one change: a property schema with no `type` (and no enum / anyOf /
* oneOf / $ref / const that would imply one) gets a permissive scalar union.
* `analyst_query.filters[].value` is `{}` in production — valid JSON Schema,
* but chat templates that print `type` for every property (Gemma's among
* them) crash on it.
*/
export function normalizeSchemaForChat(schema) {
const out = clone(schema);
const visit = (node, isRoot) => {
if (!node || typeof node !== 'object' || Array.isArray(node)) return;
const implied = ['type', 'enum', 'anyOf', 'oneOf', 'allOf', '$ref', 'const'].some((k) => k in node);
if (!isRoot && !implied) node.type = [...PERMISSIVE_SCALAR];
if (node.properties) for (const sub of Object.values(node.properties)) visit(sub, false);
if (node.items) visit(node.items, false);
for (const key of ['anyOf', 'oneOf', 'allOf']) if (Array.isArray(node[key])) node[key].forEach((s) => visit(s, false));
};
visit(out, true);
return out;
}
/**
* Every schema location normalizeSchemaForChat changes, as
* `{ tool, path, from, to }` — logged in summary.md so the per-provider
* payload difference is explicit rather than implied.
*/
export function chatNormalizationDiff(realtimeTools) {
const changes = [];
const walk = (before, after, tool, where) => {
if (!before || typeof before !== 'object' || Array.isArray(before)) return;
if (JSON.stringify(before.type) !== JSON.stringify(after.type)) {
changes.push({ tool, path: where, from: before.type === undefined ? '(untyped)' : before.type, to: after.type });
}
for (const [k, v] of Object.entries(before.properties || {})) walk(v, after.properties[k], tool, `${where}.${k}`);
if (before.items) walk(before.items, after.items, tool, `${where}[]`);
};
for (const tool of realtimeTools) walk(tool.parameters, normalizeSchemaForChat(tool.parameters), tool.name, tool.name);
return changes;
}
/** Realtime tools → Chat Completions tools (`{type:'function', function:{...}}`). */
export function toChatTools(realtimeTools, { normalize = true } = {}) {
return realtimeTools.map((tool) => ({
type: 'function',
function: {
name: tool.name,
description: tool.description,
parameters: normalize ? normalizeSchemaForChat(tool.parameters) : clone(tool.parameters),
},
}));
}
// Keywords the Gemini OpenAPI-subset `Schema` does not accept.
const GEMINI_DROP = new Set(['additionalProperties', '$schema', 'default', 'examples', '$id', 'const']);
/**
* Convert a JSON schema to the Gemini OpenAPI-subset `Schema`.
* Drops unsupported keywords (additionalProperties, $schema, default, ...);
* an untyped property becomes `anyOf` string|number|boolean; a type array
* becomes `anyOf` of single types.
*/
export function toGeminiSchema(schema) {
if (!schema || typeof schema !== 'object' || Array.isArray(schema)) return schema;
const out = {};
for (const [key, value] of Object.entries(schema)) {
if (GEMINI_DROP.has(key)) continue;
if (key === 'properties') {
out.properties = Object.fromEntries(Object.entries(value).map(([k, v]) => [k, toGeminiSchema(v)]));
} else if (key === 'items') {
out.items = toGeminiSchema(value);
} else if (key === 'anyOf' || key === 'oneOf') {
out.anyOf = value.map((s) => toGeminiSchema(s));
} else {
out[key] = clone(value);
}
}
if (Array.isArray(out.type)) {
const types = out.type;
delete out.type;
out.anyOf = types.map((t) => ({ type: t }));
}
const implied = ['type', 'enum', 'anyOf'].some((k) => k in out);
if (!implied) out.anyOf = PERMISSIVE_SCALAR.map((t) => ({ type: t }));
return out;
}
/**
* Realtime tools → Gemini `functionDeclarations`.
* mode 'jsonSchema' (default): pass the production schema verbatim through
* `parametersJsonSchema` — full JSON Schema, closest to prompt parity.
* mode 'openapi': convert to the OpenAPI-subset `parameters` field.
* A tool with no properties gets no parameters field in openapi mode (Gemini
* rejects an empty OBJECT schema there).
*/
export function toGeminiDeclarations(realtimeTools, { mode = 'jsonSchema' } = {}) {
return realtimeTools.map((tool) => {
const decl = { name: tool.name, description: tool.description };
const params = tool.parameters;
if (mode === 'jsonSchema') {
decl.parametersJsonSchema = clone(params);
} else if (params && Object.keys(params.properties || {}).length) {
decl.parameters = toGeminiSchema(params);
}
return decl;
});
}
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env node
/**
* Summarize voice turn spans from the Realtime debug log.
*
* Every closed voice turn writes one `turn.span` record (see
* src/voice/turnMetrics.js). This prints each span and p50/p90 figures for
* end-of-speech → first audio, the longest silence, tool time, preamble rate,
* spoken words and hedge hits.
*
* Usage:
* node scripts/voice-turn-spans.mjs [log.jsonl] [--since <ISO time>] [--session <id>] [--json]
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const args = process.argv.slice(2);
const option = (flag) => {
const index = args.indexOf(flag);
return index >= 0 ? args[index + 1] : null;
};
const positional = args.filter(
(arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--'),
);
const logPath =
positional[0] || path.join(root, '.gev-logs', 'realtime-conversations.jsonl');
const since = option('--since') ? Date.parse(option('--since')) : null;
const session = option('--session');
const spans = [];
for (const line of fs.readFileSync(logPath, 'utf8').split('\n')) {
if (!line.includes('"turn.span"')) continue;
let record;
try {
record = JSON.parse(line);
} catch {
continue;
}
if (record.event !== 'turn.span') continue;
if (since && Date.parse(record.timestamp) < since) continue;
if (session && record.sessionId !== session) continue;
spans.push({
at: record.timestamp,
session: record.sessionId,
...record.payload,
});
}
const quantile = (values, q) => {
const sorted = values.filter(Number.isFinite).sort((a, b) => a - b);
if (!sorted.length) return null;
return sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))];
};
const withTools = spans.filter((span) => span.tools?.length);
const slowToolTurns = withTools.filter((span) =>
span.tools.some((tool) => tool.ms >= 700),
);
const summary = {
turns: spans.length,
toolTurns: withTools.length,
firstAudioMs: {
p50: quantile(spans.map((s) => s.first_audio_ms), 0.5),
p90: quantile(spans.map((s) => s.first_audio_ms), 0.9),
},
maxSilenceMs: {
p50: quantile(spans.map((s) => s.max_silence_ms), 0.5),
p90: quantile(spans.map((s) => s.max_silence_ms), 0.9),
},
toolMs: {
p50: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.5),
p90: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.9),
},
preambleRate: withTools.length
? withTools.filter((s) => s.preamble).length / withTools.length
: null,
slowToolPreambleOrProgressRate: slowToolTurns.length
? slowToolTurns.filter((s) => s.preamble || s.progress_lines > 0).length /
slowToolTurns.length
: null,
answerWords: {
p50: quantile(spans.map((s) => s.answer_words), 0.5),
p90: quantile(spans.map((s) => s.answer_words), 0.9),
},
spokenWordsTotal: spans.reduce((sum, s) => sum + (s.spoken_words || 0), 0),
hedgeHits: spans.reduce((sum, s) => sum + (s.hedge_hits?.length || 0), 0),
};
if (args.includes('--json')) {
console.log(JSON.stringify({ summary, spans }, null, 2));
} else {
for (const span of spans) {
const tools = (span.tools || [])
.map((tool) => `${tool.name}:${tool.ms}ms`)
.join(',');
console.log(
[
span.at,
`first=${span.first_audio_ms ?? '-'}ms`,
`maxSilence=${span.max_silence_ms ?? '-'}ms`,
`preamble=${span.preamble ? 'y' : 'n'}`,
`progress=${span.progress_lines}`,
`words=${span.answer_words}/${span.spoken_words}`,
`hedges=${span.hedge_hits?.length || 0}`,
tools || '(no tools)',
span.answer_text ? `"${span.answer_text.slice(0, 80)}"` : '',
].join(' '),
);
}
console.log(JSON.stringify(summary, null, 2));
}
+43
View File
@@ -0,0 +1,43 @@
{
"reviewedSources": {
"zs": {
"name": "Zetashapes",
"registryCommit": "e17af153c144fe6e08aa8db2607b4bf8aecd448d",
"registrySha256": "d8a5dd873d26eee11583de4438185c046dec33848d5bde27047ca4bff44d9c39",
"rawLicenseType": "Public domain",
"licenseTextSha256": "91757f96f74f6dfb4a7206486b78586da0d039e9611e224fc793c7167dcfe615",
"decision": "Approved from the pinned WOF registry text: no added restrictions beyond public-domain US TIGER/Line data and Flickr-derived data; retain both upstream credits."
}
},
"licenses": [
"Attribution 3.0 Unported",
"CC BY",
"CC BY (equivalent)",
"CC BY 2.0",
"CC BY 3.0",
"CC BY 3.0 US",
"CC BY 4.0",
"CC-BY",
"CC0",
"CC0 (equivalent)",
"CC0 1.0 Universal",
"ODC-By, v1.0",
"ODC-By v1.0",
"OGL - Canada",
"OGL - City of New Westminster",
"OGL - City of Surrey",
"Open Government Licence v3.0",
"Open Government Licence - British Columbia, v2.0",
"Open Government Licence - Canada",
"Open Government Licence - City of Calgary",
"Open Government Licence - City of Regina",
"Open Government Licence - Edmonton",
"Open Government Licence - Ontario, v1.0",
"Open Government License - British Columbia, v2.0",
"Open Government License - Winnipeg",
"PDDL 1.0",
"Public domain",
"Public Domain",
"Singapore Open Data Licence v1"
]
}
+6 -2
View File
@@ -45,7 +45,9 @@ export function trackBackfillProxies() {
RESPONSE_CAP_BYTES,
);
let body;
let status = upstream.status;
if (tooLarge) {
status = 502;
body = JSON.stringify({ error: 'Upstream track response too large' });
} else if (!upstream.ok) {
// Sanitize upstream error surface; status code is signal enough
@@ -53,8 +55,10 @@ export function trackBackfillProxies() {
} else {
body = text;
}
cachePut(key, { at: Date.now(), status: upstream.status, body });
res.statusCode = upstream.status;
// Cache the 502 like any upstream error: /tracks/all costs OpenSky
// credits, so a retry inside the window must not re-download the body.
cachePut(key, { at: Date.now(), status, body });
res.statusCode = status;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'no-store');
res.end(body);
+132 -4
View File
@@ -17,6 +17,7 @@ import {
CCTV_MAX_SOURCES_CEILING,
} from './cctv/constants.js';
import { sanitizeCctvRangeHeader } from './cctv/range.js';
import { createHlsPuller } from './cctv/stream.js';
import { googleServerApiKey } from './places/google-key.js';
export { CCTV_FRAME_FETCH_TIMEOUT_MS, fetchCctvImageFromUpstream };
/**
@@ -40,6 +41,8 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
* CCTV_MAX_SOURCES ceiling so health/status observability is never evicted
* for any catalog the proxy can actually serve. */
const HEALTH_MAX_ENTRIES = CCTV_MAX_SOURCES_CEILING;
/** Live HLS strategies (see ./cctv/stream.js). Shared across dev and preview. */
const puller = createHlsPuller();
/** Update the health entry for a camera, evicting the oldest entry if at capacity. */
const setHealth = (cameraId, patch) => {
@@ -127,6 +130,9 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
};
const installMiddleware = (server) => {
server.httpServer?.on('close', () => {
puller.shutdown();
});
server.middlewares.use('/api/cctv', async (req, res) => {
try {
const sources = await getCctvSources();
@@ -194,12 +200,124 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
}
if (url.pathname.startsWith('/media/')) {
const cameraId =
decodeURIComponent(url.pathname.replace('/media/', '').trim()) ||
'camera';
const match = /^\/media\/([^/]+)(?:\/(seg_(\d+)\.ts))?$/.exec(
url.pathname,
);
if (!match) {
res.writeHead(404);
res.end();
return;
}
const cameraId = decodeURIComponent(match[1]);
const source = sourceById.get(cameraId);
const mediaUrl = source?.url || '';
const feedType = normalizeFeedType(source?.feedType || 'image');
const leaseId = url.searchParams.get('lease');
if (feedType === 'hls' && !/^[a-f0-9-]{36}$/i.test(leaseId || '')) {
res.writeHead(400);
res.end();
return;
}
if (req.method === 'DELETE') {
if (leaseId) puller.release(cameraId, leaseId);
res.writeHead(204);
res.end();
return;
}
if (req.method !== 'GET') {
res.writeHead(405);
res.end();
return;
}
if (feedType === 'hls') {
if (
!/^https?:\/\//i.test(mediaUrl) ||
!/\.m3u8(?:\?|$)/i.test(mediaUrl)
) {
res.writeHead(503, { 'Content-Type': 'application/json' });
res.end(
JSON.stringify({
error:
'This stream requires an unsupported transport; use the frame fallback',
}),
);
return;
}
if (match[2]) {
const body = puller.getSegment(
cameraId,
url.searchParams.get('session'),
Number(match[3]),
leaseId,
);
res.writeHead(body ? 200 : 404, {
'Content-Type': 'video/mp2t',
'Cache-Control': 'no-store',
});
res.end(body || undefined);
return;
}
const downstream = watchDownstreamClose(res);
let entry;
const cancelPending = () => {
if (entry) puller.release(cameraId, leaseId);
};
try {
entry = await puller.ensure(cameraId, mediaUrl, leaseId);
if (downstream.closed) {
cancelPending();
return;
}
downstream.signal.addEventListener('abort', cancelPending, {
once: true,
});
if (!(await puller.waitReady(entry, downstream.signal)))
throw new Error('Stream unavailable');
const playlist = await puller.buildPlaylist(
entry,
cameraId,
leaseId,
);
if (downstream.closed) return;
if (!playlist) throw new Error('Stream unavailable');
setHealth(cameraId, {
status: 'ok',
sourceKind: 'live',
label: source?.provider || 'Configured source',
message: 'Live HLS connected',
});
res.writeHead(200, {
'Content-Type': 'application/vnd.apple.mpegurl',
'Cache-Control': 'no-store',
'X-CCTV-Source': 'hls-pull',
'X-CCTV-Session': entry.token,
});
res.end(playlist);
} catch {
setHealth(cameraId, {
status: 'degraded',
sourceKind: 'fallback',
label: source?.provider || 'Configured source',
message: 'Live HLS unavailable',
});
if (!downstream.closed) {
res.writeHead(503, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
'Retry-After': '2',
});
res.end(JSON.stringify({ error: 'Live stream unavailable' }));
}
} finally {
downstream.signal.removeEventListener('abort', cancelPending);
}
return;
}
if (match[2] || req.method !== 'GET') {
res.writeHead(404);
res.end();
return;
}
if (!mediaUrl || !/^https?:\/\//i.test(mediaUrl)) {
setHealth(cameraId, {
@@ -308,11 +426,21 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
}
const timedOut =
error?.name === 'AbortError' || error?.name === 'TimeoutError';
// GET /api/cctv/health serializes `message`, and the CCTV panel
// renders it as a status label, so the raw error would leave the
// server by a different door than the sanitized body below and
// land on screen.
//
// The log line drops the text too, unlike the catch at the bottom
// of this file: a media-fetch failure names the camera's upstream
// host, and these lines get pasted into issues. The status codes
// below carry the diagnosis — 504 for a timeout, 502 otherwise.
console.warn('[CCTV Proxy] media fetch failed');
setHealth(cameraId, {
status: 'degraded',
sourceKind: 'upstream',
label: source?.provider || 'Configured source',
message: error?.message || 'Media fetch failed',
message: 'Media fetch failed',
});
res.writeHead(timedOut ? 504 : 502, {
'Content-Type': 'application/json',
+6
View File
@@ -17,6 +17,7 @@ import {
loadWarendorfSourcesFromCatalog,
loadNswSourcesFromOpenData,
loadCalgarySourcesFromOpenData,
loadDelDOTSourcesFromOpenData,
} from './sources.js';
/** Env kill switch: unset or anything but "0" means enabled. */
@@ -86,6 +87,11 @@ const LIVE_PACKS = [
enabled: () => envEnabled('CCTV_CALGARY_ENABLED'),
load: loadCalgarySourcesFromOpenData,
},
{
name: 'deldot',
enabled: () => envEnabled('CCTV_DELDOT_ENABLED'),
load: loadDelDOTSourcesFromOpenData,
},
];
/**
* Load CCTV sources from a local JSON file (CCTV_SOURCES_FILE env or default).
+13
View File
@@ -236,6 +236,14 @@ export const CALGARY_DOWNTOWN = { lat: 51.0461, lon: -114.0626 };
* body cannot be buffered without limit. */
export const CALGARY_MAX_CATALOG_BYTES = 4 * 1024 * 1024;
/** DelDOT CCTV: one keyless statewide JSON catalog; live video via RTMP-over-HTTP (rtmpt:80). */
export const DELDOT_CCTV_URL = 'https://tmc.deldot.gov/json/videocamera.json';
export const DEFAULT_DELDOT_MAX_SOURCES = 300;
export const DELDOT_ANCHORS = [
{ lat: 39.7459, lon: -75.5466 }, // Wilmington (New Castle)
{ lat: 39.1582, lon: -75.5244 }, // Dover (Kent)
{ lat: 38.6906, lon: -75.3877 }, // Georgetown (Sussex)
];
/** Camera CATALOGS change rarely; 15 min keeps multi-megabyte upstream list refetches (Austin rows.json + 4 Caltrans districts + TfL + Ontario 511) infrequent. Frames are fetched per-request and are unaffected. */
export const CCTV_SOURCE_CACHE_MS = 15 * 60 * 1000;
/** Per-provider catalog-fetch timeout. Bounds the worst-case refresh so one
@@ -253,5 +261,10 @@ export const CCTV_FRAME_MAX_BODY_BYTES = 16 * 1024 * 1024;
/** Deadline for upstream response headers; live bodies keep streaming afterward. */
export const CCTV_MEDIA_FETCH_TIMEOUT_MS = 15 * 1000;
/** Silence a live body may carry before the relay gives up on it. Twice the
* header deadline, because a camera that is merely slow between frames is far
* more common than one that has died mid-stream, and a viewer would rather
* wait than be dropped. */
export const CCTV_MEDIA_IDLE_TIMEOUT_MS = 30 * 1000;
/** Declared size ceiling for fixed media responses. */
export const CCTV_MEDIA_MAX_BODY_BYTES = 64 * 1024 * 1024;
+45 -1
View File
@@ -4,6 +4,7 @@ import {
CCTV_FRAME_FETCH_TIMEOUT_MS,
CCTV_FRAME_MAX_BODY_BYTES,
CCTV_MEDIA_FETCH_TIMEOUT_MS,
CCTV_MEDIA_IDLE_TIMEOUT_MS,
CCTV_MEDIA_MAX_BODY_BYTES,
NSW_IMAGE_ORIGIN,
NSW_IMAGE_USER_AGENT,
@@ -103,11 +104,17 @@ export function toReadable(body) {
* @param {Response} upstream - fetch() Response object.
* @param {object} [opts]
* @param {string} [opts.sourceHeader='upstream'] - Value for X-CCTV-Source header.
* @param {number} [opts.idleTimeoutMs=CCTV_MEDIA_IDLE_TIMEOUT_MS] - Silence
* allowed between upstream chunks before the stream is released. Injectable
* only to keep the deadline unit-testable.
*/
export async function proxyMediaResponse(
res,
upstream,
{ sourceHeader = 'upstream' } = {},
{
sourceHeader = 'upstream',
idleTimeoutMs = CCTV_MEDIA_IDLE_TIMEOUT_MS,
} = {},
) {
const contentType =
upstream.headers.get('content-type') || 'application/octet-stream';
@@ -153,7 +160,20 @@ export async function proxyMediaResponse(
return;
}
// The header deadline only covers the wait for a response line. Past that an
// upstream can hold the connection open and send nothing at all, and the
// relay would wait on it for as long as the camera host cared to. The
// deadline below measures the gap between upstream chunks rather than the
// life of the stream, so a feed that keeps delivering keeps its connection.
let idleTimer = null;
const clearIdleDeadline = () => {
if (!idleTimer) return;
clearTimeout(idleTimer);
idleTimer = null;
};
stream.on('error', () => {
clearIdleDeadline();
if (!res.writableEnded) res.end();
});
@@ -164,6 +184,7 @@ export async function proxyMediaResponse(
const releaseUpstream = () => {
if (released) return;
released = true;
clearIdleDeadline();
stream.unpipe(res);
// Destroying the Node stream cancels the web body it wraps; the direct
// cancel covers a body that was never wrapped, and rejects harmlessly when
@@ -176,14 +197,37 @@ export async function proxyMediaResponse(
/* already closed */
}
};
const armIdleDeadline = () => {
clearIdleDeadline();
idleTimer = setTimeout(onIdleDeadline, idleTimeoutMs);
idleTimer.unref?.();
};
const onIdleDeadline = () => {
// A viewer who cannot keep up pauses the pipe, and no upstream bytes arrive
// while it is paused. That is a slow client rather than a dead camera, so
// it gets the deadline again instead of a teardown.
if (res.writableNeedDrain) {
armIdleDeadline();
return;
}
releaseUpstream();
if (!res.writableEnded) res.end();
};
res.once('close', () => {
clearIdleDeadline();
if (!res.writableEnded) releaseUpstream();
});
res.once('error', releaseUpstream);
stream.once('end', () => {
clearIdleDeadline();
released = true;
});
armIdleDeadline();
stream.pipe(res);
// Attached after the pipe because a data listener resumes the stream, and
// flowing before the destination is attached would spill chunks nobody
// forwards. Only bytes from upstream renew the deadline.
stream.on('data', armIdleDeadline);
}
/**
+107
View File
@@ -56,6 +56,9 @@ import {
CALGARY_DOWNTOWN,
CALGARY_MAX_CATALOG_BYTES,
CCTV_SOURCE_FETCH_TIMEOUT_MS,
DELDOT_CCTV_URL,
DEFAULT_DELDOT_MAX_SOURCES,
DELDOT_ANCHORS,
} from './constants.js';
import {
toFiniteNumber,
@@ -1594,3 +1597,107 @@ export async function loadCalgarySourcesFromOpenData() {
return [];
}
}
export async function loadDelDOTSourcesFromOpenData() {
try {
const resp = await fetch(DELDOT_CCTV_URL, {
headers: { Accept: 'application/json' },
signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS),
redirect: 'error',
});
if (!resp.ok) {
console.warn('[CCTV] DelDOT source download failed:', resp.status);
return [];
}
const payload = await readResponseJsonCapped(resp, 2 * 1024 * 1024);
const rows = Array.isArray(payload?.videoCameras)
? payload.videoCameras
: [];
if (!rows.length) return [];
const cameras = [];
for (const row of rows) {
if (String(row?.status).toLowerCase() !== 'active') continue;
const lat = toFiniteNumber(row?.lat);
const lon = toFiniteNumber(row?.lon);
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
// Delaware bounding box — a bad upstream coord can't place a camera out of state.
if (lat < 38.4 || lat > 39.9 || lon < -75.85 || lon > -75.0) continue;
// Prefer the catalog's HTTPS HLS transport; no subprocess is required.
let stream;
try {
stream = new URL(String(row?.urls?.m3u8s || ''));
} catch {
continue;
}
if (
stream.origin !== 'https://video.deldot.gov' ||
stream.username ||
stream.password ||
!/^\/live\/[A-Za-z0-9_.-]+\/playlist\.m3u8$/.test(stream.pathname)
)
continue;
const streamUrl = stream.href;
const id = String(row?.id || '').trim();
if (!/^[A-Za-z0-9_-]{1,80}$/.test(id)) continue;
const cameraId = `deldot-${id.toLowerCase()}`;
const title = String(row?.title || '').trim();
// Title carries an unambiguous travel token for ~1 in 5 cameras
// ("US 13 SB @ ..."). directionToHeading (allowBare=false) matches only
// NB/SB/EB/WB, never a bare "W"/"N" street prefix — "W NORTH ST" yields
// no heading and falls back.
const heading = directionToHeading(title);
const hasHeading = Number.isFinite(heading);
cameras.push({
id: cameraId,
name: title || `DelDOT ${id}`,
city: String(row?.county ? `${row.county} County` : 'Delaware'),
cityId: `deldot-${String(row?.county || 'de')
.toLowerCase()
.replace(/\s+/g, '-')}`,
provider: 'DelDOT',
lat,
lon,
headingDeg: hasHeading ? heading : fallbackHeadingFromId(cameraId),
headingConfidence: hasHeading ? 'high' : 'low',
// Fabricated RAW PRIOR poses (same personalities as Austin/Caltrans);
// the client ground-snap + manual calibration own the truth.
pitchDeg: hasHeading ? -24 : -18,
fovDeg: hasHeading ? 56 : 44,
rangeM: hasHeading ? 210 : 145,
mountHeightM: hasHeading ? 10 : 8,
groundElevationM: 10, // Estimated prior; client ground resolution owns placement.
feedType: 'hls',
url: streamUrl,
snapshotUrl: '',
sourceKind: 'deldot-open-data',
license: 'Public DelDOT traffic camera',
});
}
const unique = Array.from(
new Map(cameras.map((camera) => [camera.id, camera])).values(),
);
const maxRaw = Number(
process.env.CCTV_DELDOT_MAX_SOURCES || DEFAULT_DELDOT_MAX_SOURCES,
);
const maxCount = Number.isFinite(maxRaw)
? Math.max(8, Math.min(400, Math.floor(maxRaw)))
: DEFAULT_DELDOT_MAX_SOURCES;
const prioritized = prioritizeSources(unique, maxCount, DELDOT_ANCHORS);
console.log(
`[CCTV] Loaded DelDOT camera sources: ${unique.length} Active (using nearest ${prioritized.length})`,
);
return prioritized;
} catch (error) {
console.warn(
'[CCTV] DelDOT source download error:',
error?.message || error,
);
return [];
}
}
+348
View File
@@ -0,0 +1,348 @@
/** Bounded, demand-owned HLS sessions. No subprocess, disk cache or idle sweep. */
import { randomUUID } from 'node:crypto';
export const HLS_LIMITS = Object.freeze({
sessions: 2,
leasesPerSession: 8,
playlistBytes: 256 * 1024,
segmentBytes: 4 * 1024 * 1024,
sessionBytes: 24 * 1024 * 1024,
segments: 12,
pollMs: 2000,
idleMs: 15000,
timeoutMs: 10000,
readyMs: 12000,
});
/** Refuse redirects; count actual streamed bytes, including chunked responses. */
export async function fetchHlsBytes(
url,
{
fetchImpl = fetch,
signal,
maxBytes,
timeoutMs = HLS_LIMITS.timeoutMs,
} = {},
) {
const controller = new AbortController();
const abort = () => controller.abort();
if (signal?.aborted) controller.abort();
else signal?.addEventListener('abort', abort, { once: true });
const timer = setTimeout(abort, timeoutMs);
let reader;
let response;
try {
controller.signal.throwIfAborted();
response = await fetchImpl(url, {
redirect: 'error',
signal: controller.signal,
headers: { 'User-Agent': 'gods-eye-view-cctv-proxy/1.0' },
});
if (
!response.ok ||
response.redirected ||
(response.url && new URL(response.url).href !== new URL(url).href)
)
throw new Error('HLS upstream refused');
if (Number(response.headers.get('content-length')) > maxBytes)
throw new Error('HLS response too large');
reader = response.body?.getReader?.();
if (!reader) throw new Error('HLS response is not streamable');
const chunks = [];
let length = 0;
for (;;) {
controller.signal.throwIfAborted();
const { done, value } = await reader.read();
controller.signal.throwIfAborted();
if (done) break;
length += value.byteLength;
if (length > maxBytes) throw new Error('HLS response too large');
chunks.push(Buffer.from(value));
}
return Buffer.concat(chunks, length);
} finally {
controller.abort();
if (reader) await reader.cancel().catch(() => {});
else await response?.body?.cancel().catch(() => {});
reader?.releaseLock();
clearTimeout(timer);
signal?.removeEventListener('abort', abort);
}
}
export function sameOriginHlsUrl(value, base) {
const url = new URL(value, base);
if (
!['https:', 'http:'].includes(url.protocol) ||
url.origin !== new URL(base).origin ||
url.username ||
url.password
)
throw new Error('HLS reference escapes registered origin');
return url.href;
}
/** Only clear MPEG-TS media playlists: encrypted, fMP4 and byte ranges fail closed. */
export function parseHlsMedia(text, base, limit = HLS_LIMITS.segments) {
if (
!text.startsWith('#EXTM3U') ||
/#EXT-X-(?:KEY|MAP|BYTERANGE|I-FRAMES-ONLY)/.test(text)
)
throw new Error('Unsupported HLS playlist');
let seq = 0;
let duration = null;
let discontinuity = false;
const segments = [];
for (const raw of text.split('\n')) {
const line = raw.trim();
if (line.startsWith('#EXT-X-MEDIA-SEQUENCE:')) {
seq = Number(line.slice(22));
if (!Number.isSafeInteger(seq) || seq < 0)
throw new Error('Invalid media sequence');
} else if (line === '#EXT-X-DISCONTINUITY') {
discontinuity = true;
} else if (line.startsWith('#EXTINF:')) {
duration = Number.parseFloat(line.slice(8));
if (!Number.isFinite(duration) || duration <= 0 || duration > 60)
throw new Error('Invalid segment duration');
} else if (line && !line.startsWith('#')) {
if (duration === null || !Number.isSafeInteger(seq))
throw new Error('Invalid HLS segment');
const uri = sameOriginHlsUrl(line, base);
if (!new URL(uri).pathname.endsWith('.ts'))
throw new Error('Only MPEG-TS segments are supported');
segments.push({ seq: seq++, duration, uri, discontinuity });
discontinuity = false;
duration = null;
}
}
return segments.slice(-limit);
}
export function createHlsPuller({
fetchImpl = fetch,
limits = HLS_LIMITS,
} = {}) {
const active = new Map();
let closed = false;
const stop = (cameraId, token) => {
const entry = active.get(cameraId);
if (!entry || (token && token !== entry.token)) return;
active.delete(cameraId);
entry.stopping = true;
entry.controller.abort();
clearTimeout(entry.timer);
for (const lease of entry.leases.values()) clearTimeout(lease.timer);
entry.leases.clear();
entry.segments.clear();
entry.upstream.clear();
entry.bytes = 0;
};
const release = (cameraId, leaseId) => {
const entry = active.get(cameraId);
const lease = entry?.leases.get(leaseId);
if (!lease) return;
clearTimeout(lease.timer);
entry.leases.delete(leaseId);
if (!entry.leases.size) stop(cameraId, entry.token);
};
const touch = (entry, leaseId, create = false) => {
let lease = entry.leases.get(leaseId);
if (!lease) {
if (!create || entry.leases.size >= limits.leasesPerSession)
throw new Error('HLS lease capacity reached');
lease = {};
entry.leases.set(leaseId, lease);
}
clearTimeout(lease.timer);
lease.timer = setTimeout(
() => release(entry.cameraId, leaseId),
limits.idleMs,
);
lease.timer.unref?.();
};
const read = (entry, url, maxBytes) =>
fetchHlsBytes(url, {
fetchImpl,
signal: entry.controller.signal,
maxBytes,
timeoutMs: limits.timeoutMs,
});
const poll = async (entry) => {
try {
let base = entry.chunklistUrl || entry.url;
let text = (await read(entry, base, limits.playlistBytes)).toString(
'utf8',
);
if (!text.includes('#EXTINF:')) {
const variant = text
.split('\n')
.map((line) => line.trim())
.find((line) => line && !line.startsWith('#'));
if (!variant || !text.includes('#EXT-X-STREAM-INF:'))
throw new Error('Missing HLS variant');
base = sameOriginHlsUrl(variant, entry.url);
text = (await read(entry, base, limits.playlistBytes)).toString('utf8');
}
entry.chunklistUrl = base;
const segments = parseHlsMedia(text, base, limits.segments);
const newest = segments.at(-1)?.seq ?? -1;
const restarted =
newest < entry.upstreamNewest ||
segments.some((segment) => {
const prior = entry.upstream.get(segment.seq);
return prior && prior !== segment.uri;
});
if (restarted) {
entry.upstream.clear();
entry.pendingDiscontinuity = true;
}
entry.upstreamNewest = newest;
for (const segment of segments) {
if (entry.stopping) return;
if (entry.upstream.has(segment.seq)) continue;
const body = await read(entry, segment.uri, limits.segmentBytes);
if (entry.stopping) return;
// Evict BEFORE retaining the incoming segment. One bounded download may
// exist in addition to the cache; no unbounded disk output is possible.
while (
entry.segments.size &&
(entry.bytes + body.length > limits.sessionBytes ||
entry.segments.size >= limits.segments)
) {
const oldest = entry.segments.keys().next().value;
const removed = entry.segments.get(oldest);
entry.bytes -= removed.body.length;
if (removed.discontinuity) entry.discontinuitiesRemoved++;
entry.segments.delete(oldest);
}
if (body.length > limits.sessionBytes)
throw new Error('Segment exceeds session budget');
const seq = entry.nextSeq++;
entry.segments.set(seq, {
...segment,
seq,
body,
discontinuity: segment.discontinuity || entry.pendingDiscontinuity,
});
entry.pendingDiscontinuity = false;
entry.upstream.set(segment.seq, segment.uri);
while (entry.upstream.size > limits.segments * 2)
entry.upstream.delete(entry.upstream.keys().next().value);
entry.bytes += body.length;
}
entry.failures = 0;
} catch {
entry.chunklistUrl = null;
entry.failures++;
if (entry.failures >= 3) {
stop(entry.cameraId, entry.token);
return;
}
} finally {
if (!entry.stopping) {
entry.timer = setTimeout(() => {
entry.polling = poll(entry);
}, limits.pollMs);
entry.timer.unref?.();
}
}
};
const ensure = async (cameraId, url, leaseId = 'legacy') => {
if (closed) throw new Error('HLS service closed');
sameOriginHlsUrl(url, url);
let entry = active.get(cameraId);
if (entry && entry.url !== url) {
stop(cameraId);
entry = null;
}
if (entry) {
touch(entry, leaseId, true);
return entry;
}
// Reserve synchronously, before any await: concurrent creation is singleflight.
if (active.size >= limits.sessions)
throw new Error('HLS session capacity reached');
entry = {
cameraId,
url,
token: randomUUID(),
controller: new AbortController(),
segments: new Map(),
bytes: 0,
stopping: false,
failures: 0,
nextSeq: 0,
upstream: new Map(),
leases: new Map(),
upstreamNewest: -1,
pendingDiscontinuity: false,
discontinuitiesRemoved: 0,
};
active.set(cameraId, entry);
touch(entry, leaseId, true);
entry.polling = poll(entry);
return entry;
};
const waitReady = async (entry, signal) => {
const until = Date.now() + limits.readyMs;
while (!entry.stopping && !signal?.aborted && Date.now() < until) {
if (entry.segments.size >= 2) return true;
await new Promise((resolve) => setTimeout(resolve, 50));
}
return false;
};
const buildPlaylist = async (entry, cameraId, leaseId = 'legacy') => {
if (entry.stopping || entry.segments.size < 2) return null;
touch(entry, leaseId);
const segments = [...entry.segments.values()].sort((a, b) => a.seq - b.seq);
const lines = [
'#EXTM3U',
'#EXT-X-VERSION:3',
`#EXT-X-TARGETDURATION:${Math.ceil(Math.max(...segments.map((s) => s.duration)))}`,
`#EXT-X-MEDIA-SEQUENCE:${segments[0].seq}`,
`#EXT-X-DISCONTINUITY-SEQUENCE:${entry.discontinuitiesRemoved}`,
];
let previous;
for (const segment of segments) {
if (
segment.discontinuity ||
(previous !== undefined && segment.seq !== previous + 1)
)
lines.push('#EXT-X-DISCONTINUITY');
lines.push(
`#EXTINF:${segment.duration.toFixed(3)},`,
`/api/cctv/media/${encodeURIComponent(cameraId)}/seg_${segment.seq}.ts?session=${entry.token}&lease=${encodeURIComponent(leaseId)}`,
);
previous = segment.seq;
}
return lines.join('\n') + '\n';
};
const getSegment = (cameraId, token, seq, leaseId = 'legacy') => {
const entry = active.get(cameraId);
if (!entry || entry.token !== token || !entry.leases.has(leaseId))
return null;
const body = entry.segments.get(seq)?.body;
if (body) touch(entry, leaseId);
return body || null;
};
const shutdown = async () => {
closed = true;
const entries = [...active.values()];
for (const entry of entries) stop(entry.cameraId);
await Promise.allSettled(entries.map((entry) => entry.polling));
};
return {
ensure,
waitReady,
buildPlaylist,
getSegment,
stop,
release,
shutdown,
stats: () => ({
sessions: active.size,
bytes: [...active.values()].reduce((total, e) => total + e.bytes, 0),
}),
};
}
+15 -3
View File
@@ -17,17 +17,29 @@ async function readRequestBodyCapped(req, maxBytes) {
function readRequestBody(req, maxBytes = 1024 * 1024) {
return new Promise((resolve, reject) => {
let total = 0;
let bodyTooLarge = false;
const chunks = [];
req.on('data', (chunk) => {
// Keep draining after the cap without retaining more bytes. Destroying
// the request also destroys its socket, so the handler cannot deliver
// the fixed error response it constructs for BODY_TOO_LARGE.
if (bodyTooLarge) return;
total += chunk.length;
if (total > maxBytes) {
reject(new Error(`Request body exceeds ${maxBytes} bytes`));
req.destroy();
// Same marker readRequestBodyCapped sets, so a caller can tell an
// oversized body from a genuine failure without matching on text.
bodyTooLarge = true;
chunks.length = 0;
const error = new Error(`Request body exceeds ${maxBytes} bytes`);
error.code = 'BODY_TOO_LARGE';
reject(error);
return;
}
chunks.push(chunk);
});
req.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
req.on('end', () => {
if (!bodyTooLarge) resolve(Buffer.concat(chunks).toString('utf8'));
});
req.on('error', reject);
});
}
+416
View File
@@ -0,0 +1,416 @@
import { readResponseTextCapped } from './common/http.js';
const STATUS_URL = 'https://www.nhc.noaa.gov/CurrentStorms.json';
const GIS =
'https://mapservices.weather.noaa.gov/tropical/rest/services/tropical/NHC_tropical_weather_summary/MapServer';
const HOUR = 3600_000;
const COVERAGE =
'Atlantic and eastern/central North Pacific; not worldwide cyclone coverage.';
const LAYERS = [
{ id: 5, cap: 512 * 1024, count: 500, kind: 'points' },
{ id: 6, cap: 512 * 1024, count: 32, kind: 'track' },
{ id: 7, cap: 2 * 1024 * 1024, count: 32, kind: 'cone' },
];
function invalid() {
return new Error('invalid_cyclone_data');
}
function text(value, max = 80) {
if (
typeof value !== 'string' ||
!value.trim() ||
value.length > max ||
/[\u0000-\u001f<>]/.test(value)
)
throw invalid();
return value.trim();
}
function advisory(value) {
if (typeof value !== 'string' || !/^\d{1,3}[A-Z]?$/i.test(value))
throw invalid();
return value.replace(/^0+(?=\d)/, '').toUpperCase();
}
function number(value, min, max) {
if (
value === null ||
value === undefined ||
value === '' ||
!['string', 'number'].includes(typeof value)
)
return null;
const n = Number(value);
return Number.isFinite(n) && n >= min && n <= max ? n : null;
}
function iso(value, now) {
if (
typeof value !== 'string' ||
!/^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d(?:\.\d{3})?Z$/.test(value)
)
throw invalid();
const n = Date.parse(value);
if (
!Number.isFinite(n) ||
n > now + 5 * 60_000 ||
now - n > 12 * HOUR ||
new Date(n).toISOString().replace('.000Z', 'Z') !==
value.replace('.000Z', 'Z')
)
throw invalid();
return new Date(n).toISOString();
}
function position(coordinates) {
if (
!Array.isArray(coordinates) ||
coordinates.length !== 2 ||
!coordinates.every((n) => typeof n === 'number' && Number.isFinite(n)) ||
Math.abs(coordinates[0]) > 180 ||
Math.abs(coordinates[1]) > 90
)
throw invalid();
return { longitude: coordinates[0], latitude: coordinates[1] };
}
function officialLink(value) {
if (typeof value !== 'string' || value.length > 256) return null;
try {
const url = new URL(value);
return url.origin === 'https://www.nhc.noaa.gov' &&
!url.username &&
!url.password &&
!url.search &&
!url.hash &&
/^\/text\/[A-Z0-9]+\.shtml$/.test(url.pathname)
? url.href
: null;
} catch {
return null;
}
}
export function parseCycloneStatus(payload, now = Date.now()) {
if (!Array.isArray(payload?.activeStorms) || payload.activeStorms.length > 32)
throw invalid();
const seen = new Set();
return payload.activeStorms.map((raw) => {
if (
typeof raw?.id !== 'string' ||
!/^(?:al|ep|cp)\d{6}$/.test(raw.id) ||
seen.has(raw.id)
)
throw invalid();
seen.add(raw.id);
const forecast = raw.forecastAdvisory;
const issuedAt = iso(forecast?.issuance, now);
const advisoryNumber = advisory(forecast?.advNum);
const positionAt = iso(raw.lastUpdate, now);
return {
id: raw.id,
name: text(raw.name),
classification: text(raw.classification, 16),
basin: raw.id.slice(0, 2).toUpperCase(),
position: position([raw.longitudeNumeric, raw.latitudeNumeric]),
positionAt,
advisoryNumber,
issuedAt,
windKt: number(raw.intensity, 0, 300),
pressureHpa: number(raw.pressure, 800, 1100),
movement: {
directionDegrees: number(raw.movementDir, 0, 360),
speedKt: number(raw.movementSpeed, 0, 200),
},
advisoryUrl: officialLink(forecast.url),
outlookUrl: raw.id.startsWith('al')
? 'https://www.nhc.noaa.gov/gtwo.php?basin=atlc&fdays=7'
: raw.id.startsWith('cp')
? 'https://www.nhc.noaa.gov/gtwo.php?basin=cpac&fdays=7'
: 'https://www.nhc.noaa.gov/gtwo.php?basin=epac&fdays=7',
geometryStatus: 'pending',
geometryAdvisoryNumber: null,
forecastPoints: [],
track: null,
cone: null,
};
});
}
function geometry(raw, kind, budget) {
const types =
kind === 'points'
? ['Point']
: kind === 'track'
? ['LineString', 'MultiLineString']
: ['Polygon', 'MultiPolygon'];
if (!types.includes(raw?.type)) throw invalid();
const point = (value) => {
position(value);
if (++budget.count > 25_000) throw invalid();
return [...value];
};
const line = (value, ring = false) => {
if (
!Array.isArray(value) ||
value.length < (ring ? 4 : 2) ||
value.length > 10_000
)
throw invalid();
const result = value.map(point);
if (
ring &&
(result[0][0] !== result.at(-1)[0] || result[0][1] !== result.at(-1)[1])
)
throw invalid();
return result;
};
const list = (value, read) => {
if (!Array.isArray(value) || !value.length || value.length > 128)
throw invalid();
return value.map(read);
};
const polygon = (value) => list(value, (ring) => line(ring, true));
const coordinates =
raw.type === 'Point'
? point(raw.coordinates)
: raw.type === 'LineString'
? line(raw.coordinates)
: raw.type === 'MultiLineString'
? list(raw.coordinates, (part) => line(part))
: raw.type === 'Polygon'
? polygon(raw.coordinates)
: list(raw.coordinates, polygon);
return { type: raw.type, coordinates };
}
/** Never relabel earlier GIS geometry with the newer status advisory. */
export function attachCycloneGeometry(storms, collections) {
const budget = { count: 0 };
const parsed = LAYERS.map((spec, index) => {
const payload = collections[index];
if (
payload?.type !== 'FeatureCollection' ||
payload.exceededTransferLimit ||
!Array.isArray(payload.features) ||
payload.features.length > spec.count
)
throw invalid();
return payload.features.map((feature) => {
if (feature?.type !== 'Feature') throw invalid();
const p = feature.properties;
const source =
typeof p?.idp_source === 'string' &&
p.idp_source.match(
/^((?:al|ep|cp)\d{6})-(\d{1,3}[a-z]?)_5day_(pts|lin|pgn)$/i,
);
if (!source || source[3].toLowerCase() !== ['pts', 'lin', 'pgn'][index])
throw invalid();
const adv = advisory(p.advisnum);
if (adv !== advisory(source[2])) throw invalid();
return {
id: source[1].toLowerCase(),
advisoryNumber: adv,
geometry: geometry(feature.geometry, spec.kind, budget),
tauHours: number(p.tau, 0, 168),
windKt: number(p.maxwind, 0, 300),
gustKt: number(p.gust, 0, 350),
};
});
});
return storms.map((storm) => {
const groups = parsed.map((items) =>
items.filter(
(item) =>
item.id === storm.id && item.advisoryNumber === storm.advisoryNumber,
),
);
if (groups[1].length > 1 || groups[2].length > 1) throw invalid();
if (!groups[0].length || groups[1].length !== 1 || groups[2].length !== 1)
return storm;
const taus = new Set();
const forecastPoints = groups[0]
.map((item) => {
if (item.tauHours === null || taus.has(item.tauHours)) throw invalid();
taus.add(item.tauHours);
return {
position: position(item.geometry.coordinates),
tauHours: item.tauHours,
windKt: item.windKt,
gustKt: item.gustKt,
};
})
.sort((a, b) => a.tauHours - b.tauHours);
return {
...storm,
geometryStatus: 'current',
geometryAdvisoryNumber: storm.advisoryNumber,
forecastPoints,
track: groups[1][0].geometry,
cone: groups[2][0].geometry,
};
});
}
/** Fixed official endpoints; one shared bounded refresh, no user destinations. */
export function cycloneProxy({
fetchImpl = fetch,
now = () => Date.now(),
timeoutMs = 12_000,
} = {}) {
let cache = null;
let operation = null;
let attemptedAt = -Infinity;
async function upstream(url, cap, signal) {
signal.throwIfAborted();
const response = await fetchImpl(url, {
signal,
redirect: 'error',
headers: {
Accept: 'application/geo+json,application/json',
'User-Agent': 'Gods Eye View (public NOAA weather context)',
},
});
if (!response.ok) {
await response.body?.cancel();
throw new Error('cyclone_upstream_unavailable');
}
const result = JSON.parse(
await readResponseTextCapped(response, cap, signal),
);
signal.throwIfAborted();
return result;
}
async function refresh(signal) {
let storms = parseCycloneStatus(
await upstream(STATUS_URL, 128 * 1024, signal),
now(),
);
if (storms.length) {
try {
const results = await Promise.allSettled(
LAYERS.map(async (spec) => {
const url = new URL(`${GIS}/${spec.id}/query`);
url.search = new URLSearchParams({
where: '1=1',
outFields:
spec.kind === 'points'
? 'idp_source,advisnum,tau,maxwind,gust'
: 'idp_source,advisnum',
outSR: '4326',
resultRecordCount: String(spec.count),
geometryPrecision: '4',
f: 'geojson',
}).toString();
return upstream(url.href, spec.cap, signal);
}),
);
if (results.some((result) => result.status === 'rejected'))
throw invalid();
storms = attachCycloneGeometry(
storms,
results.map((result) => result.value),
);
} catch {
storms = storms.map((storm) => ({
...storm,
geometryStatus: 'unavailable',
}));
}
}
signal.throwIfAborted();
cache = { storms, fetchedAt: now() };
return cache;
}
async function acquire(signal) {
signal.throwIfAborted();
if (cache && now() - cache.fetchedAt < 300_000) return cache;
if (operation?.controller.signal.aborted) operation = null;
if (!operation) {
if (now() - attemptedAt < 60_000) throw new Error('cyclone_retry_later');
attemptedAt = now();
const controller = new AbortController();
const owned = { controller, waiters: 0 };
const timer = setTimeout(() => controller.abort(), timeoutMs);
owned.promise = refresh(controller.signal).finally(() => {
clearTimeout(timer);
if (operation === owned) operation = null;
});
operation = owned;
}
const owned = operation;
if (owned.waiters >= 32)
throw Object.assign(new Error('cyclone_busy'), { status: 429 });
owned.waiters++;
let abort;
const cancelled = new Promise((_, reject) => {
abort = () => reject(signal.reason ?? new Error('cancelled'));
signal.addEventListener('abort', abort, { once: true });
});
try {
return await Promise.race([owned.promise, cancelled]);
} finally {
signal.removeEventListener('abort', abort);
if (--owned.waiters === 0 && operation === owned) {
owned.controller.abort();
if (signal.aborted) attemptedAt = -Infinity;
}
}
}
function describe(value, stale = false) {
return {
schemaVersion: 1,
source: 'NOAA NHC / CPHC',
attribution:
'NOAA/NWS National Hurricane Center / Central Pacific Hurricane Center',
coverage: COVERAGE,
fetchedAt: value?.fetchedAt ?? null,
stale: stale || !value,
unavailable: !value,
reason: !value
? 'Cyclone data unavailable'
: stale
? 'Cached cyclone advisory; upstream unavailable'
: null,
storms: value?.storms ?? [],
};
}
async function handler(req, res) {
const controller = new AbortController();
const close = () => controller.abort();
res.once?.('close', close);
const json = (status, value) => {
if (controller.signal.aborted) return;
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(status === 429 ? { 'Retry-After': '2' } : {}),
});
res.end(JSON.stringify(value));
};
try {
if (req.method !== 'GET')
return json(405, { error: 'method_not_allowed' });
if (req.url !== '/' && req.url !== '')
return json(400, { error: 'invalid_cyclone_query' });
try {
json(200, describe(await acquire(controller.signal)));
} catch (error) {
if (error.status === 429) return json(429, { error: 'cyclone_busy' });
const usable =
cache &&
now() - cache.fetchedAt <= 12 * HOUR &&
cache.storms.every(
(storm) => now() - Date.parse(storm.issuedAt) <= 12 * HOUR,
);
json(200, describe(usable ? cache : null, true));
}
} finally {
res.removeListener?.('close', close);
}
}
return {
name: 'cyclones',
configureServer({ middlewares }) {
middlewares.use('/api/cyclones', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/cyclones', handler);
},
};
}
+232
View File
@@ -0,0 +1,232 @@
import { normalizeFirePerimeterSnapshot } from '../../src/layers/perimeters/records.js';
import {
readResponseJsonCapped,
readResponseTextCapped,
coalesceProxyRequest,
} from './common/http.js';
import { makeRateLimiter, clientKey } from './common/rate-limit.js';
// NIFC WFIGS current interagency fire perimeters (public, keyless).
// maxAllowableOffset trades ~100 m of boundary fidelity for a payload small
// enough to refresh continuously (~1 MB for a typical fire season).
const API_URL =
'https://services3.arcgis.com/T4QMspbfLg3qTGWY/arcgis/rest/services/' +
'WFIGS_Interagency_Perimeters_Current/FeatureServer/0/query?' +
new URLSearchParams({
where: '1=1',
outFields: [
'poly_IncidentName',
'attr_UniqueFireIdentifier',
'attr_IncidentSize',
'attr_PercentContained',
'attr_POOState',
'attr_IncidentTypeCategory',
'attr_FireDiscoveryDateTime',
'poly_DateCurrent',
'attr_FireCause',
'attr_FireBehaviorGeneral',
'attr_TotalIncidentPersonnel',
'attr_POOCounty',
'attr_EstimatedCostToDate',
'attr_IncidentComplexityLevel',
'attr_CpxName',
].join(','),
maxAllowableOffset: '0.001',
outSR: '4326',
f: 'geojson',
}).toString();
// The service caps a single response at its maxRecordCount (2000); a peak
// season can exceed that, so follow exceededTransferLimit with offset pages.
const MAX_PAGES = 5;
const exceededTransferLimit = (payload) =>
payload?.exceededTransferLimit === true ||
payload?.properties?.exceededTransferLimit === true;
const MIB = 1024 * 1024;
/** Fixed-origin, bounded WFIGS and InciWeb routes for dev and preview. */
export function firePerimetersProxy({
fetchImpl = (...args) => globalThis.fetch(...args),
now = () => Date.now(),
} = {}) {
const cache = new Map();
const publications = new Map();
const inFlight = new Map();
const allow = makeRateLimiter({ windowMs: 60_000, max: 60, globalMax: 1200 });
async function upstream(url, cap, timeout, options = {}) {
const signal = AbortSignal.timeout(timeout);
const response = await fetchImpl(url, {
...options,
signal,
redirect: 'error',
});
if (!response.ok) {
await response.body?.cancel();
throw new Error('upstream_unavailable');
}
return readResponseJsonCapped(response, cap, signal);
}
async function fetchPerimeters() {
const features = [];
for (let page = 0; page < MAX_PAGES; page++) {
const url =
page === 0 ? API_URL : `${API_URL}&resultOffset=${features.length}`;
const payload = await upstream(url, 16 * MIB, 30_000);
if (!Array.isArray(payload?.features))
throw new Error('invalid_snapshot');
features.push(...payload.features);
if (!exceededTransferLimit(payload) || !payload.features.length) break;
}
const rows = normalizeFirePerimeterSnapshot({ features });
return { fetchedAt: now(), rows };
}
async function fetchIndex() {
const rows = await upstream(
'https://inciweb.wildfire.gov/api/single-publication/',
4 * MIB,
20_000,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title: '' }),
},
);
if (!Array.isArray(rows)) throw new Error('invalid_index');
return rows;
}
// InciWeb's publication API was retired; incident pages carry origin and update times.
async function fetchPublication(id) {
const signal = AbortSignal.timeout(15_000);
let response = await fetchImpl(`https://inciweb.wildfire.gov/node/${id}`, {
signal,
redirect: 'manual',
});
if ([301, 302, 303, 307, 308].includes(response.status)) {
await response.body?.cancel();
const location = response.headers.get('location');
if (!location || /\.\.|[?%\\]/.test(location))
throw new Error('invalid_redirect');
const url = new URL(location, 'https://inciweb.wildfire.gov/');
if (
url.hostname !== 'inciweb.wildfire.gov' ||
!/^\/[a-z0-9-]+(?:\/[a-z0-9-]+)?$/.test(url.pathname) ||
!['http:', 'https:'].includes(url.protocol) ||
url.port ||
url.username ||
url.password
)
throw new Error('invalid_redirect');
url.protocol = 'https:';
url.search = '';
url.hash = '';
response = await fetchImpl(url.href, { signal, redirect: 'error' });
}
if (response.status !== 200) {
await response.body?.cancel();
throw new Error('upstream_unavailable');
}
const html = await readResponseTextCapped(response, 2 * MIB, signal);
const changed = Date.parse(
html.match(/<meta\s+property="og:updated_time"\s+content="([^"]+)"/)?.[1],
);
const created = Date.parse(
html.match(
/Date of Origin<\/th>\s*<td[^>]*>\s*<time\s+datetime="([^"]+)"/,
)?.[1],
);
const createdMs = Number.isFinite(created) ? created : null;
const changedMs = Number.isFinite(changed) ? changed : null;
if (createdMs === null && changedMs === null)
throw new Error('invalid_incident_page');
return { createdMs, changedMs };
}
async function acquire(key, ttl, load, store) {
const previous = store.get(key);
if (previous && now() - previous.savedAt < ttl)
return { value: previous.value, stale: false };
// Bound distinct concurrent publication requests as well as retained entries.
if (!inFlight.has(key) && inFlight.size >= 256)
throw Object.assign(new Error('busy'), { status: 429 });
try {
const { promise } = coalesceProxyRequest(inFlight, key, async () => {
const value = await load();
store.delete(key);
store.set(key, { value, savedAt: now() });
if (store === publications && store.size > 256)
store.delete(store.keys().next().value);
return value;
});
return { value: await promise, stale: false };
} catch (error) {
if (previous) return { value: previous.value, stale: true };
throw error;
}
}
async function handler(req, res) {
const json = (status, value, stale = false) => {
if (res.destroyed) return;
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(status === 405 ? { Allow: 'GET' } : {}),
...(status === 429 ? { 'Retry-After': '60' } : {}),
...(stale ? { 'X-Data-Stale': 'true' } : {}),
});
res.end(JSON.stringify(value));
};
if (req.method !== 'GET') return json(405, { error: 'method_not_allowed' });
const path = (req.url || '/').split('?')[0];
let key,
ttl,
load,
store = cache;
if (path === '/' || path === '') {
key = 'perimeters';
ttl = 300_000;
load = fetchPerimeters;
} else if (path === '/inciweb/index') {
key = 'index';
ttl = 3_600_000;
load = fetchIndex;
} else if (path.startsWith('/inciweb/publication/')) {
const id = path.slice('/inciweb/publication/'.length);
if (!/^\d{1,9}$/.test(id))
return json(400, { error: 'invalid_publication_id' });
key = `publication:${id}`;
ttl = 1_800_000;
load = () => fetchPublication(id);
store = publications;
} else return json(404, { error: 'unknown_route' });
if (!allow(clientKey(req))) return json(429, { error: 'rate_limited' });
try {
const { value, stale } = await acquire(key, ttl, load, store);
json(
200,
key === 'perimeters' && stale ? { ...value, stale: true } : value,
stale,
);
} catch (error) {
json(error.status === 429 ? 429 : 502, {
error: 'fire_perimeters_unavailable',
});
}
}
return {
name: 'fire-perimeters',
configureServer({ middlewares }) {
middlewares.use('/api/fire-perimeters', handler);
},
configurePreviewServer({ middlewares }) {
middlewares.use('/api/fire-perimeters', handler);
},
};
}
+11 -5
View File
@@ -8,10 +8,11 @@ import { filterTrailing24h, parseFirmsCsv } from '../../src/data/firmsCsv.js';
* Upstream: https://firms.modaps.eosdis.nasa.gov/api/area/csv/{KEY}/{SOURCE}/world/2
*
* Merges three VIIRS NRT sources (NOAA-20, NOAA-21, Suomi-NPP — independent
* satellites, no cross-source dedup) fetched sequentially with `days=2`
* (`days=1` means "current UTC day", nearly empty just after 00:00Z) and
* clamps to the trailing 24 h via src/data/firmsCsv.js. FIRMS quota is
* 5,000 transactions / 10 min per MAP_KEY, so the cache is the point:
* satellites, no cross-source dedup) plus MODIS_NRT (combined Terra+Aqua,
* ~1 km, numeric 0-100 confidence), fetched sequentially (quota courtesy)
* with `days=2` (`days=1` means "current UTC day", nearly empty just after
* 00:00Z) and clamps to the trailing 24 h via src/data/firmsCsv.js. FIRMS
* quota is 5,000 transactions / 10 min per MAP_KEY, so the cache is the point:
* TTL 30 min, single-flight refresh, serve-stale-on-failure, and a
* fresh-enough disk cache (.gev-cache/firms.json) prevents ANY upstream
* fetch across dev-server restarts. Pattern mirrors celestrakProxy.
@@ -28,7 +29,12 @@ import { filterTrailing24h, parseFirmsCsv } from '../../src/data/firmsCsv.js';
export function firmsProxy() {
const TTL_MS = 30 * 60_000;
const STATUS_TTL_MS = 5 * 60_000;
const SOURCES = ['VIIRS_NOAA20_NRT', 'VIIRS_NOAA21_NRT', 'VIIRS_SNPP_NRT'];
const SOURCES = [
'VIIRS_NOAA20_NRT',
'VIIRS_NOAA21_NRT',
'VIIRS_SNPP_NRT',
'MODIS_NRT',
];
const CACHE_DIR = path.join(process.cwd(), '.gev-cache');
const CACHE_PATH = path.join(CACHE_DIR, 'firms.json');
+521
View File
@@ -0,0 +1,521 @@
import { lookup as dnsLookup } from 'node:dns/promises';
import http from 'node:http';
import https from 'node:https';
import { Readable } from 'node:stream';
import { coalesceProxyRequest, readResponseTextCapped } from './common/http.js';
import { isLocalIpv4, parseTapAddress } from '../../src/data/tapAddress.js';
import { normalizeDump1090Aircraft } from '../../src/sources/adsbRecords.js';
/**
* Local decoder feeds for the Local ADS-B layer.
*
* A user who runs dump1090 / readsb / tar1090 (1090 MHz) or dump978-fa +
* skyaware978 (978 MHz UAT) exposes an `aircraft.json` over HTTP. This route
* reads those documents and returns them as Local ADS-B records.
*
* The feed list comes only from the operator's `LOCAL_RECEIVER_FEEDS`
* environment value; the browser never supplies an address. Every host must
* pass the shared receiver-tap address rule (`src/data/tapAddress.js`:
* loopback, RFC1918, `localhost`, `*.local`), the scheme must be http(s) and
* the path must end in `aircraft.json`. Invalid entries are logged at startup,
* reported as `invalid` and never fetched.
*
* A name (`localhost`, `*.local`) is resolved before every read; each address
* it resolves to must itself be loopback or RFC1918, and the connection is
* pinned to exactly those validated addresses, so a later re-resolution
* cannot redirect it anywhere else.
*/
export const LOCAL_RECEIVERS_ROUTE = '/api/local-receivers/aircraft';
export const LOCAL_RECEIVER_FEEDS_ENV = 'LOCAL_RECEIVER_FEEDS';
export const LOCAL_RECEIVER_BANDS = Object.freeze(['1090', '978']);
export const LOCAL_RECEIVER_TIMEOUT_MS = 2_000;
export const LOCAL_RECEIVER_MAX_BODY_BYTES = 2 * 1024 * 1024;
export const LOCAL_RECEIVER_CACHE_MS = 1_000;
/** A feed whose own `now` is older than this is reported stale. */
export const LOCAL_RECEIVER_STALE_MS = 10_000;
const MAX_FEEDS = 8;
const LOG_PREFIX = '[local-receivers]';
const BAND_LABELS = Object.freeze({ 1090: '1090 MHz', 978: '978 MHz UAT' });
/** Statuses whose `Response` must be constructed without a body. */
const NULL_BODY_STATUSES = new Set([101, 103, 204, 205, 304]);
function defaultPort(protocol) {
return protocol === 'https:' ? 443 : 80;
}
/**
* Validate one `band=url` entry.
* @param {string} entry Raw entry text.
* @returns {{band:string, url:string}|{band:string|null, reason:string}}
*/
function parseEntry(entry) {
const separator = entry.indexOf('=');
if (separator <= 0) return { band: null, reason: 'expected band=url' };
const band = entry.slice(0, separator).trim();
const rawUrl = entry.slice(separator + 1).trim();
if (!LOCAL_RECEIVER_BANDS.includes(band))
return { band: null, reason: 'band must be 1090 or 978' };
let url;
try {
url = new URL(rawUrl);
} catch {
return { band, reason: 'not a valid URL' };
}
if (url.protocol !== 'http:' && url.protocol !== 'https:')
return { band, reason: 'scheme must be http or https' };
if (url.username || url.password)
return { band, reason: 'credentials are not allowed in the URL' };
if (url.search || url.hash || rawUrl.includes('?') || rawUrl.includes('#'))
return { band, reason: 'query and fragment are not allowed' };
const port = url.port ? Number(url.port) : defaultPort(url.protocol);
const address = parseTapAddress(`${url.hostname}:${port}`);
if (!address)
return {
band,
reason: 'host must be loopback, RFC1918, localhost or *.local',
};
if (!/(?:^|\/)aircraft\.json$/.test(url.pathname))
return { band, reason: 'path must end in aircraft.json' };
return {
band,
url: `${url.protocol}//${address.host}:${address.port}${url.pathname}`,
};
}
/**
* Parse `LOCAL_RECEIVER_FEEDS` (`band=url[,band=url…]`).
*
* Labels name the band only (plus an ordinal when a band repeats), so feed
* addresses never reach the browser.
* @param {string|undefined} value Environment value.
* @returns {{configured:boolean, feeds:object[]}} Each feed is
* `{ id, band, label, url }` when valid or `{ id, band, label, reason }`
* when invalid (never fetched).
*/
export function parseLocalReceiverFeeds(value) {
const entries = String(value ?? '')
.split(',')
.map((entry) => entry.trim())
.filter(Boolean);
const feeds = [];
const perBand = new Map();
entries.forEach((entry, index) => {
const parsed =
index < MAX_FEEDS
? parseEntry(entry)
: { band: null, reason: `at most ${MAX_FEEDS} feeds are read` };
const band = parsed.band;
const ordinal = (perBand.get(band) || 0) + 1;
perBand.set(band, ordinal);
const base = band ? BAND_LABELS[band] : `entry ${index + 1}`;
const label = band && ordinal > 1 ? `${base} #${ordinal}` : base;
feeds.push({
id: `feed-${index + 1}`,
band,
label,
...(parsed.url ? { url: parsed.url } : { reason: parsed.reason }),
});
});
// Ordinals are only needed when a band actually repeats.
for (const feed of feeds) {
if (feed.band && perBand.get(feed.band) > 1 && !feed.label.includes('#'))
feed.label = `${feed.label} #1`;
}
return { configured: feeds.length > 0, feeds };
}
class FeedError extends Error {
constructor(code) {
super(code);
this.code = code;
}
}
/**
* Whether a resolved address satisfies the receiver-tap rule: loopback or
* RFC1918 IPv4, the IPv6 loopback, or an IPv4-mapped form of the former.
* @param {string} address Resolved IP address.
* @returns {boolean}
*/
export function isLocalReceiverAddress(address) {
const value = String(address || '').toLowerCase();
if (value === '::1') return true;
const mapped = /^::ffff:(\d{1,3}(?:\.\d{1,3}){3})$/.exec(value);
return isLocalIpv4(mapped ? mapped[1] : value);
}
/**
* Resolve a feed host name and require every address to be local.
* @param {string} hostname `localhost` or a `*.local` name.
* @param {typeof dnsLookup} lookupImpl `dns.promises.lookup`-compatible.
* @returns {Promise<Array<{address:string, family:number}>>}
*/
export async function resolveLocalReceiverAddresses(hostname, lookupImpl) {
let resolved;
try {
resolved = await lookupImpl(hostname, { all: true, verbatim: true });
} catch {
throw new FeedError('UNRESOLVED');
}
const addresses = (Array.isArray(resolved) ? resolved : [resolved])
.map((row) => ({
address: String(row?.address || ''),
family:
Number(row?.family) || (String(row?.address).includes(':') ? 6 : 4),
}))
.filter((row) => row.address);
if (!addresses.length) throw new FeedError('UNRESOLVED');
if (addresses.some((row) => !isLocalReceiverAddress(row.address)))
throw new FeedError('FORBIDDEN_ADDRESS');
return addresses;
}
/**
* Settle with `promise`, or reject with an AbortError as soon as `signal`
* aborts; a later settlement of `promise` is ignored.
*/
function untilAborted(promise, signal) {
return new Promise((resolve, reject) => {
const onAbort = () => {
const error = new Error('aborted');
error.name = 'AbortError';
reject(error);
};
// Handlers are attached first so a late rejection is never unhandled.
promise.then(
(value) => {
signal.removeEventListener('abort', onAbort);
resolve(value);
},
(error) => {
signal.removeEventListener('abort', onAbort);
reject(error);
},
);
if (signal.aborted) onAbort();
else signal.addEventListener('abort', onAbort, { once: true });
});
}
/** A `net`-style lookup that only ever answers the validated addresses. */
function pinnedLookup(addresses) {
return (_hostname, options, callback) => {
const done = typeof options === 'function' ? options : callback;
if (options?.all)
done(
null,
addresses.map((row) => ({ ...row })),
);
else done(null, addresses[0].address, addresses[0].family);
};
}
/**
* Default transport. With `options.lookup` (a pinned feed name) the request
* is made with node:http(s) so the socket connects only to the validated
* addresses (TLS still verifies the name); otherwise it is a plain fetch of
* an IP-literal URL. Redirects are never followed either way.
* @param {string} url
* @param {object} options
* @returns {Promise<Response>}
*/
function fetchLocalReceiverFeed(url, options) {
if (typeof options?.lookup !== 'function') return fetch(url, options);
const client = new URL(url).protocol === 'https:' ? https : http;
return new Promise((resolve, reject) => {
const request = client.request(
url,
{
method: 'GET',
headers: options.headers,
signal: options.signal,
lookup: options.lookup,
// A fresh socket per read: a pooled one could outlive the pin.
agent: false,
},
(response) => {
// Nothing thrown here may escape: this callback runs outside the
// promise, so an exception would be uncaught and crash the server.
try {
const status = response.statusCode;
// A `Response` only represents final statuses 200–599.
if (!Number.isInteger(status) || status < 200 || status > 599)
throw new FeedError('HTTP_STATUS');
const headers = new Headers();
for (const [name, value] of Object.entries(response.headers)) {
if (Array.isArray(value))
value.forEach((item) => headers.append(name, item));
else if (value !== undefined) headers.set(name, String(value));
}
const nullBody = NULL_BODY_STATUSES.has(status);
const upstream = new Response(
nullBody ? null : Readable.toWeb(response),
{ status, statusText: response.statusMessage || '', headers },
);
if (nullBody) response.destroy();
resolve(upstream);
} catch (error) {
response.destroy();
reject(error);
}
},
);
request.on('error', reject);
request.end();
});
}
/**
* Fetch one validated feed document without following redirects and with the
* body cap enforced while it streams.
* @param {string} url Validated feed URL.
* @param {object} options
* @returns {Promise<object>} Parsed JSON document.
*/
export async function fetchLocalReceiverDocument(
url,
{
fetchImpl = fetchLocalReceiverFeed,
lookupImpl = dnsLookup,
timeoutMs = LOCAL_RECEIVER_TIMEOUT_MS,
maxBytes = LOCAL_RECEIVER_MAX_BODY_BYTES,
} = {},
) {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
let upstream;
try {
const { hostname } = new URL(url);
// IP literals were validated when the feed list was parsed; a name is
// validated by what it resolves to now, and the socket is pinned there.
// Resolution runs against the same deadline as the read: a stalled
// lookup fails this feed instead of holding the whole snapshot.
const lookup = isLocalIpv4(hostname)
? undefined
: pinnedLookup(
await untilAborted(
resolveLocalReceiverAddresses(hostname, lookupImpl),
controller.signal,
),
);
upstream = await fetchImpl(url, {
method: 'GET',
headers: { Accept: 'application/json' },
redirect: 'manual',
signal: controller.signal,
...(lookup ? { lookup } : {}),
});
if (upstream.status >= 300 && upstream.status < 400)
throw new FeedError('REDIRECT_REFUSED');
// Opaque redirect responses from `redirect: 'manual'` report status 0.
if (upstream.type === 'opaqueredirect')
throw new FeedError('REDIRECT_REFUSED');
if (!upstream.ok) throw new FeedError('HTTP_STATUS');
const text = await readResponseTextCapped(
upstream,
maxBytes,
controller.signal,
);
let json;
try {
json = JSON.parse(text);
} catch {
throw new FeedError('BAD_JSON');
}
if (!json || typeof json !== 'object' || !Array.isArray(json.aircraft))
throw new FeedError('BAD_DOCUMENT');
return json;
} catch (error) {
controller.abort();
if (upstream?.body && !upstream.body.locked)
void upstream.body.cancel().catch(() => {});
if (error instanceof FeedError) throw error;
if (error?.code === 'RESPONSE_TOO_LARGE')
throw new FeedError('RESPONSE_TOO_LARGE');
if (error?.name === 'AbortError' || controller.signal.aborted)
throw new FeedError('TIMEOUT');
throw new FeedError('UNREACHABLE');
} finally {
clearTimeout(timeoutId);
}
}
/**
* Read one feed and describe it.
* @returns {Promise<{status:object, records:object[]}>}
*/
async function readFeed(
feed,
{ fetchImpl, lookupImpl, now, timeoutMs, maxBytes, log },
) {
const summary = { band: feed.band, label: feed.label };
if (!feed.url)
return {
status: { ...summary, status: 'invalid', aircraft: 0, ageMs: null },
records: [],
};
let json;
try {
json = await fetchLocalReceiverDocument(feed.url, {
fetchImpl,
lookupImpl,
timeoutMs,
maxBytes,
});
} catch (error) {
log(feed, 'unreachable', error.code);
return {
status: { ...summary, status: 'unreachable', aircraft: 0, ageMs: null },
records: [],
};
}
const receivedAt = now();
const feedNowS = Number(json.now);
// The feed's own clock decides staleness. A clock ahead of ours reads as
// age 0; a document without `now` is treated as current.
const ageMs = Number.isFinite(feedNowS)
? Math.max(0, Math.round(receivedAt - feedNowS * 1000))
: 0;
const status = ageMs > LOCAL_RECEIVER_STALE_MS ? 'stale' : 'live';
log(feed, status);
// Anchor record ages at the document time so a stale feed ages out
// naturally, without trusting a receiver clock that runs ahead of ours.
const records = normalizeDump1090Aircraft(json, receivedAt - ageMs, {
band: feed.band,
});
return {
status: { ...summary, status, aircraft: records.length, ageMs },
records,
};
}
/**
* Build the route handler. Exposed for tests; the plugin wraps it.
* @param {object} [options]
* @param {string} [options.feedsValue] `LOCAL_RECEIVER_FEEDS` value.
* @param {typeof fetch} [options.fetchImpl] Receives `options.lookup` when
* the feed host is a name; the connection must resolve only through it.
* @param {Function} [options.lookupImpl] `dns.promises.lookup`-compatible.
* @param {() => number} [options.now]
* @param {{warn:Function, info?:Function}} [options.logger]
* @returns {{handle:Function, config:object}}
*/
export function createLocalReceiversHandler({
feedsValue = process.env[LOCAL_RECEIVER_FEEDS_ENV],
fetchImpl = fetchLocalReceiverFeed,
lookupImpl = dnsLookup,
now = Date.now,
logger = console,
timeoutMs = LOCAL_RECEIVER_TIMEOUT_MS,
maxBytes = LOCAL_RECEIVER_MAX_BODY_BYTES,
cacheMs = LOCAL_RECEIVER_CACHE_MS,
} = {}) {
const config = parseLocalReceiverFeeds(feedsValue);
for (const feed of config.feeds) {
if (feed.reason)
logger.warn(
`${LOG_PREFIX} ${LOCAL_RECEIVER_FEEDS_ENV} ${feed.id} (${feed.label}) rejected: ${feed.reason}. It will not be fetched.`,
);
}
const lastStatus = new Map();
// Log a feed only when its status changes, so a feed that is down does not
// write a line every second.
const log = (feed, status, code) => {
if (lastStatus.get(feed.id) === status) return;
lastStatus.set(feed.id, status);
if (status === 'live') logger.info?.(`${LOG_PREFIX} ${feed.label} live`);
else
logger.warn(
`${LOG_PREFIX} ${feed.label} ${status}${code ? ` (${code})` : ''}`,
);
};
const inFlight = new Map();
let cached = null;
async function snapshot() {
const results = await Promise.all(
config.feeds.map((feed) =>
readFeed(feed, {
fetchImpl,
lookupImpl,
now,
timeoutMs,
maxBytes,
log,
}),
),
);
return {
configured: true,
generatedAt: now(),
feeds: results.map((result) => result.status),
records: results.flatMap((result) => result.records),
};
}
async function payload() {
if (!config.configured)
return { configured: false, feeds: [], records: [] };
if (cached && now() - cached.at < cacheMs) return cached.payload;
const { promise } = coalesceProxyRequest(inFlight, 'all', snapshot);
const result = await promise;
cached = { at: now(), payload: result };
return result;
}
async function handle(req, res) {
if (req.method !== 'GET' && req.method !== 'HEAD') {
res.writeHead(405, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
return;
}
try {
const body = await payload();
res.writeHead(200, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify(body));
} catch {
res.writeHead(502, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(JSON.stringify({ error: 'Local receiver feeds unavailable' }));
}
}
return { handle, config };
}
/**
* Vite plugin mounting `GET /api/local-receivers/aircraft`.
*
* The environment is read when the server starts, so construction alone
* neither parses configuration nor fetches anything.
* @param {object} [options] Passed to {@link createLocalReceiversHandler}.
* @returns {import('vite').Plugin}
*/
export function localReceiversProxy(options = {}) {
const install = (server) => {
const { handle } = createLocalReceiversHandler(options);
server.middlewares.use(LOCAL_RECEIVERS_ROUTE, (req, res, next) => {
const path = new URL(req.url || '/', 'http://localhost').pathname;
if (path !== '/' && path !== '') {
if (typeof next === 'function') return next();
return undefined;
}
return handle(req, res);
});
};
return {
name: 'local-receivers-proxy',
configureServer: install,
configurePreviewServer: install,
};
}
+12 -2
View File
@@ -7,12 +7,13 @@ import { adsbdbProxy } from './aircraft/enrichment.js';
import { overpassProxy } from './overpass.js';
import { militaryInstallationsProxy } from './military-installations.js';
import { regionalBriefProxy } from './regional/briefing.js';
import { geocodeProxy } from './regional/place.js';
import { geocodeProxy, NOMINATIM_CACHE_DIR } from './regional/place.js';
import { weatherEffectsProxy } from './regional/weather-effects.js';
import { cctvProxy } from './cctv.js';
import { defaultSourceRoot } from './common/source-root.js';
import { radioBrowserProxy } from './radio.js';
import { gbfsProxy } from './gbfs.js';
import { localReceiversProxy } from './local-receivers.js';
import { transitProxy } from './transit.js';
import { adsbLolProxy } from './aircraft/adsb-lol.js';
import { aisLiveProxy } from './vessels/ais-live.js';
@@ -20,6 +21,10 @@ import { trackBackfillProxies } from './aircraft/tracks.js';
import { openAiRealtimeProxy } from './openai.js';
import { googlePlacesContextProxy } from './places.js';
import { keySetupEndpoint } from '../standalone/key-setup.js';
import { weatherProxy } from './weather.js';
import { firePerimetersProxy } from './firePerimeters.js';
import { cycloneProxy } from './cyclones.js';
import { windProxy } from './wind.js';
/** Construct the local provider plugins in their established order. */
function localProviderPlugins() {
@@ -34,17 +39,22 @@ function localProviderPlugins() {
overpassProxy(),
militaryInstallationsProxy(),
regionalBriefProxy(),
geocodeProxy(),
geocodeProxy({ storageDir: NOMINATIM_CACHE_DIR }),
weatherEffectsProxy(),
cctvProxy({ sourceRoot: defaultSourceRoot }),
radioBrowserProxy(),
gbfsProxy(),
localReceiversProxy(),
transitProxy(),
adsbLolProxy(),
aisLiveProxy(),
trackBackfillProxies(),
openAiRealtimeProxy(),
googlePlacesContextProxy(),
windProxy(),
weatherProxy(),
cycloneProxy(),
firePerimetersProxy(),
keySetupEndpoint(),
];
}
+40 -9
View File
@@ -5,7 +5,11 @@ import {
MILITARY_INSTALLATION_DISK_TTL_MS,
MILITARY_INSTALLATION_STALE_MS,
} from './military-installations/constants.js';
import { fetchOverpassPayload } from './overpass/transport.js';
import {
fetchOverpassPayload,
overpassNotConfigured,
} from './overpass/transport.js';
import { resolveOverpassUpstreams } from './overpass/constants.js';
import {
_militaryInstallationCache,
trimMilitaryInstallationCache,
@@ -45,6 +49,7 @@ function militaryInstallationsProxy() {
throw Object.assign(
new Error('Mapped installation upstream unavailable'),
{
retryAfterMs: upstream.retryAfterMs,
installationReason: upstream.rateLimited
? 'rate_limited'
: upstream.status === 504
@@ -83,14 +88,6 @@ function militaryInstallationsProxy() {
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
return;
}
if (!_militaryInstallationsRateLimiter(clientKey(req))) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Retry-After': '5',
});
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
return;
}
const url = new URL(req.url, 'http://localhost');
const requested = validMilitaryInstallationBox(url.searchParams);
if (!requested) {
@@ -117,6 +114,36 @@ function militaryInstallationsProxy() {
: militaryInstallationCacheKey(box);
const now = Date.now();
const cached = _militaryInstallationCache.get(key);
if (!resolveOverpassUpstreams().length) {
const stale =
cached || (await readMilitaryInstallationDisk(key, Infinity));
const disabled = overpassNotConfigured();
res.writeHead(200, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
'X-Military-Installations': stale ? 'STALE' : 'DISABLED',
});
res.end(
stale
? JSON.stringify({
...stale.payload,
status: 'stale',
retrievedAt:
stale.payload.retrievedAt ||
new Date(stale.cachedAt).toISOString(),
})
: disabled.body,
);
return;
}
if (!_militaryInstallationsRateLimiter(clientKey(req))) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Retry-After': '5',
});
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
return;
}
const preflight = await resolveMilitaryInstallationTier({
cacheKey: key,
memoryCache: _militaryInstallationCache,
@@ -176,6 +203,9 @@ function militaryInstallationsProxy() {
return;
}
res.writeHead(503, {
...(error?.retryAfterMs
? { 'Retry-After': String(Math.ceil(error.retryAfterMs / 1000)) }
: {}),
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
@@ -183,6 +213,7 @@ function militaryInstallationsProxy() {
JSON.stringify({
error: 'Mapped installation context is temporarily unavailable',
reason: militaryInstallationFailureReason(error),
retryAfterMs: error?.retryAfterMs,
}),
);
}
+4
View File
@@ -16,6 +16,9 @@ const OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT = 0.5;
const OPENAI_HUD_SUMMARY_MODEL_DEFAULT = 'gpt-5-nano';
/** Input transcription for voice captions; OPENAI_REALTIME_TRANSCRIBE_MODEL=off disables it. */
const OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT = 'gpt-4o-mini-transcribe';
export {
OPENAI_HUD_SUMMARY_MODEL_DEFAULT,
OPENAI_REALTIME_MODEL_MINI_DEFAULT,
@@ -24,4 +27,5 @@ export {
OPENAI_REALTIME_REASONING_DEFAULT,
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
};
+111 -18
View File
@@ -1,16 +1,98 @@
import { defaultSourceRoot } from '../common/source-root.js';
import path from 'node:path';
import { readRequestBody } from '../common/request.js';
import fs from 'node:fs';
import { promises as fsp } from 'node:fs';
import { makeRateLimiter, clientKey } from '../common/rate-limit.js';
/** Cap on one request body. */
const REALTIME_DEBUG_LOG_MAX_BYTES = 8 * 1024 * 1024;
function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
const REALTIME_DEBUG_LOG_DIR = path.join(sourceRoot, '.gev-logs');
const REALTIME_DEBUG_LOG_FILE = path.join(
REALTIME_DEBUG_LOG_DIR,
'realtime-conversations.jsonl',
);
/**
* Ceiling for the live log before it rotates. One generation is kept
* (`…jsonl.1`), so the sink is bounded at twice this however long a session
* runs. REALTIME_DEBUG_LOG_MAX_BYTES caps a single request; it never bounded
* the file those requests accumulate into.
*/
const REALTIME_DEBUG_LOG_MAX_FILE_BYTES = 32 * 1024 * 1024;
/**
* Per-IP write ceiling. Always on, unlike the opt-in limiter the cost-bearing
* OpenAI routes share: throttling those by default would change what a user
* spends, while this sink spends disk and event-loop time. A voice session
* writes far below this, so the cap needs no configuration to stay invisible.
*/
const REALTIME_DEBUG_LOG_MAX_PER_MIN = 120;
/**
* Fields that carry what people said or heard: transcripts, typed and spoken
* text, streamed deltas and the span's quoted replies. They are dropped from
* the persisted log unless content logging is explicitly enabled
* (GEV_VOICE_LOG_CONTENT=1), so timing and tool diagnostics never retain
* speech by default.
*/
const VOICE_CONTENT_FIELDS = new Set([
'transcript',
'text',
'delta',
'answer_text',
'preamble_text',
]);
/** Replace spoken or typed content with its length. */
function omitVoiceContent(value, depth = 0) {
if (depth > 12 || value === null || typeof value !== 'object') return value;
if (Array.isArray(value))
return value.map((item) => omitVoiceContent(item, depth + 1));
const output = {};
for (const [key, item] of Object.entries(value)) {
output[key] =
VOICE_CONTENT_FIELDS.has(key) && typeof item === 'string'
? `[omitted ${item.length} chars]`
: omitVoiceContent(item, depth + 1);
}
return output;
}
function createDebugLogHandler({
sourceRoot = defaultSourceRoot,
includeContent = process.env.GEV_VOICE_LOG_CONTENT === '1',
} = {}) {
const logDir = path.join(sourceRoot, '.gev-logs');
const logFile = path.join(logDir, 'realtime-conversations.jsonl');
const allow = makeRateLimiter({
windowMs: 60_000,
max: REALTIME_DEBUG_LOG_MAX_PER_MIN,
globalMax: 400,
});
// Appends are chained rather than fired concurrently because the rotation
// check is a read-then-write on file size: run in parallel, every caller
// stats a file still under the ceiling, none of them rotates, and the log
// grows past it. The records themselves are O_APPEND and never interleave.
let queue = Promise.resolve();
function append(line) {
const done = queue.then(async () => {
await fsp.mkdir(logDir, { recursive: true });
const size = await fsp.stat(logFile).then(
(stat) => stat.size,
() => 0,
);
if (size + Buffer.byteLength(line) > REALTIME_DEBUG_LOG_MAX_FILE_BYTES) {
// Node does not replace an existing destination on Windows. Remove the
// retained generation explicitly so every rotation behaves the same on
// supported platforms rather than failing after the first 64 MB.
await fsp.rm(`${logFile}.1`, { force: true });
await fsp.rename(logFile, `${logFile}.1`);
}
await fsp.appendFile(logFile, line);
});
// The caller still sees its own rejection, but one failed append must not
// poison the chain for every request queued behind it.
queue = done.catch(() => {});
return done;
}
return async (req, res) => {
if (req.method !== 'POST') {
res.statusCode = 405;
@@ -19,12 +101,20 @@ function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
return;
}
if (!allow(clientKey(req))) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Retry-After': '60',
});
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
return;
}
try {
const body = await readRequestBody(req, REALTIME_DEBUG_LOG_MAX_BYTES);
const record = JSON.parse(body || '{}');
fs.mkdirSync(REALTIME_DEBUG_LOG_DIR, { recursive: true });
fs.appendFileSync(
REALTIME_DEBUG_LOG_FILE,
const parsed = JSON.parse(body || '{}');
const record = includeContent ? parsed : omitVoiceContent(parsed);
await append(
`${JSON.stringify({
loggedAt: new Date().toISOString(),
...record,
@@ -33,15 +123,18 @@ function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
res.statusCode = 204;
res.end();
} catch (error) {
res.statusCode = 400;
// A malformed record or an oversized body is the caller's fault; a failed
// write is ours, and only the second is worth a log line. Neither answer
// carries the error text, which for an fs failure is an errno and an
// absolute path.
const callerFault =
error instanceof SyntaxError || error?.code === 'BODY_TOO_LARGE';
if (!callerFault) console.warn('[realtime-debug-log] append failed');
res.statusCode = callerFault ? 400 : 500;
res.setHeader('Content-Type', 'application/json');
res.end(
JSON.stringify({
error: error?.message || 'Failed to write Realtime debug log',
}),
);
res.end(JSON.stringify({ error: 'Failed to write Realtime debug log' }));
}
};
}
export { createDebugLogHandler };
export { createDebugLogHandler, omitVoiceContent };
+13 -13
View File
@@ -1,4 +1,7 @@
import { keylessHudSummaryResponse } from '../../../src/hudSummaryResponse.js';
import {
HUD_SUMMARY_INSTRUCTIONS,
keylessHudSummaryResponse,
} from '../../../src/hudSummaryResponse.js';
import { enforceOptInRateLimit, openAiRateLimiter } from './rate-limit.js';
import { readRequestBody } from '../common/request.js';
import { OPENAI_HUD_SUMMARY_MODEL_DEFAULT } from './constants.js';
@@ -61,13 +64,7 @@ async function handleHudSummary(req, res) {
model:
process.env.OPENAI_HUD_SUMMARY_MODEL ||
OPENAI_HUD_SUMMARY_MODEL_DEFAULT,
instructions: [
"Write one concise intelligence-HUD summary for God's Eye View.",
'Use only the supplied place, street, nearby-place, and enabled-layer text labels.',
'Prefer the clearest named place and include a relevant enabled layer only when useful.',
'Do not infer from coordinates or invent a place.',
'Output exactly five words with no title, punctuation, markdown, or introductory phrase.',
].join(' '),
instructions: HUD_SUMMARY_INSTRUCTIONS,
input: JSON.stringify(context),
reasoning: { effort: 'minimal' },
max_output_tokens: 100,
@@ -78,20 +75,23 @@ async function handleHudSummary(req, res) {
res.statusCode = response.ok && summary ? 200 : response.status || 502;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'no-store');
if (!response.ok)
console.warn(`[hud-summary] upstream HTTP ${response.status}`);
res.end(
JSON.stringify({
summary: summary || null,
error: response.ok
? null
: data.error?.message || 'OpenAI HUD summary request failed',
// Never relay `data.error.message`: that is OpenAI's own wording, and
// it carries request ids, organization hints and quota phrasing.
error: response.ok ? null : 'OpenAI HUD summary request failed',
}),
);
} catch (error) {
} catch {
console.warn('[hud-summary] request failed');
res.statusCode = 502;
res.setHeader('Content-Type', 'application/json');
res.end(
JSON.stringify({
error: error?.message || 'OpenAI HUD summary request failed',
error: 'OpenAI HUD summary request failed',
}),
);
}
+19 -24
View File
@@ -1,5 +1,5 @@
const DEFAULT_ANNOTATION_GUIDANCE =
'Use a single annotate_map call with several annotations when you are describing multiple related places at once. Set flyTo true only when the user is not already looking at the place; if every mark in a call lands off-screen the app auto-frames them, so when unsure leave flyTo false. Do NOT say out loud that you are drawing, highlighting, or annotating — just speak naturally about the places while the marks appear. ANNOTATIONS ACCUMULATE AND PERSIST — keep adding marks as you explore; you can fly around, change topic, and jump between far-apart places and the marks STAY, so the user can build up the map and show people things. Do NOT clear on your own initiative: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
'Put related places in one annotate_map call. Set flyTo true only when the user is not already looking at the place (off-screen marks auto-frame, so when unsure leave it false). Do not say you are drawing or annotating — talk about the places while the marks appear. Marks ACCUMULATE AND PERSIST across flights and topics so the user can build up the map: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
function realtimeInstructions(
annotationGuidance = DEFAULT_ANNOTATION_GUIDANCE,
@@ -13,25 +13,23 @@ function realtimeInstructions(
'For requests to open, show, reveal, or focus a menu/panel, call set_panel_open or show_data_layers_menu. "Open Context" means only set_panel_open{panelId:"global-context-panel",open:true}; it does not activate a Context sub-mode. "Open Contacts" means set_context_mode{mode:"contacts"}; that action expands the parent Context panel before activating Contacts.',
'For requests like "show me the datacenter layers", open the data layers menu and focus the matching layer row; do not enable the layer unless the user asks to turn it on.',
'For questions like "what am I looking at?", "what is in view?", "what is this?", "that selected thing", nearby datacenter, dam, cable, ship, or current view contents, call get_entity_context first, then answer from the returned scene/entity context.',
'For "what is this aircraft?" answers, read the callsign, operator, registration, type, and route only from get_entity_context selected.properties. Treat route, routeOrigin, and routeDestination as the only authoritative route fields. Every aircraft identity answer MUST explicitly cover operator, type, and route. When a route is present, repeat its endpoint codes exactly; do not expand airport codes into city names. For a missing field say exactly "Operator details are unavailable", "Aircraft type is unavailable", or "Route details are unavailable" as applicable. Never silently omit missing enrichment or infer it from the callsign.',
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking automatically stops the tracking (the result says so): mention it briefly.',
'For "what is this aircraft?", answer with get_entity_context\'s identityLine: callsign, operator, type and route from returned fields only, airport codes verbatim. Other questions about the selection answer from its properties. Never infer them from the callsign.',
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking stops the tracking.',
'For camera-motion requests — "orbit around this", "pan left", "tilt up", "stop moving" — call move_camera. For "fly the route" over a drawn route, call fly_route. Confirm with the RESULTING state ("Orbiting slowly", "Flying the route").',
'analyst_query ANSWERS questions; it never moves the camera or starts tracking. For requests to FOLLOW or TRACK a specific aircraft/ship, call track_entity (get_entity_context first when the target is ambiguous), never analyst_query as the final or only action. For "follow/track the nearest aircraft", first call analyst_query with the aircraft layer(s), sortBy=distance, and limit=1, then call track_entity with the returned aircraft identity in the same turn. The lookup alone does not fulfill a follow/track command.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data: report the returned feed source briefly, never call it an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about layer data — how many / which / fastest / highest / biggest / nearest flights, ships, fires, or earthquakes ("how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Narrate the count plus two or three notable examples by name, and reflect the result\'s coverage note honestly: the answer covers data loaded by enabled layers, not the whole world. If the needed layer is disabled, say so and offer to enable it. For follow-ups about the same set ("which of THOSE is closest?"), call analyst_query with followUp=true and only the new filter/sort.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — those are the exact numbers on the user\'s panel. set_context_mode, analyst_query, and get_current_view_state carry it after Contacts settles. For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft; do not answer from a pre-Contacts analyst query. analyst_query\'s own count measures currently-loaded records and is usually lower; never give it as the window count. CENTER PRECEDENCE for a nearby/how-many ask, in order: an explicit place in the question ("over Texas", "near Austin") always wins and ignores Contacts state; else the CONTACTS SUBJECT when Contacts is active and has one — a selected datacenter, dam, fire, or cable does NOT silently become the center; else an entity the user explicitly names ("around this datacenter"); else the current view, said aloud ("nothing is selected, so this is the current view"). With Contacts active but NO subject yet, use the view and say so; never read an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two different numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge ("a few", "less than a dozen"): if a tool returns 46, say 46. (5) When it matters, add once: counts cover loaded data, and the flights layer loads where you look.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data, never an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about loaded layer data — how many / which / fastest / highest / biggest / nearest / next flights, ships, fires, earthquakes, satellites, launches, storms, transit, bikes, bases, datacenters or dams ("how many flights over Texas", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Always set scope by WHERE: "around here" or "near this" while pointing is kind pointer, never view. Filter on the attribute the user names ("MODIS" → sensor eq MODIS). Say the count with its scope in a few words ("37 loaded flights over Texas") plus one or two examples by name. If the needed layer is off, say so and offer to turn it on. "Which of THOSE…" about the last answer → followUp=true with only the new filter/sort.',
'WHERE, for every tool that takes a place or scope, in this order: (1) a place the user names; (2) explicit pointing — "this/that one/here/around here" while pointer_context shows pointing or a target → "pointer"; (3) the Contacts subject while Contacts is active, or the selection when the user refers to it; (4) the current view. A selected datacenter, dam, fire or cable never silently becomes the centre.',
'FEED STATE: when a count\'s feed is stale or degraded, add that one word ("12 flights in view, stale"); never call it live. If unavailable, give no number: say the feed is unavailable. Fallback feeds are valid data; name them only if asked.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — the exact numbers on the user\'s panel (set_context_mode, analyst_query and get_current_view_state carry it once Contacts settles). For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft, not from a pre-Contacts analyst query; analyst_query\'s own count measures loaded records and is usually lower — never give it as the window count. The centre follows WHERE; with Contacts active but no subject yet, use the view, never an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge: if a tool returns 46, say 46.',
'While Cockpit is active, navigate with control_cockpit (next/previous, optionally targetLayer or aircraftClass). track_entity and fly_to_location are REFUSED by design while Cockpit owns the camera — that refusal is correct, not an error to retry. To go somewhere else, exit Cockpit first. control_cockpit enter establishes Contacts itself, so do not call set_context_mode before or after it.',
'When the target layer is unknown, OMIT layerId in track_entity so it searches all enabled layers. Passing the wrong layerId ("flights" for a military contact) returns "Nothing matched" even though the contact is loaded.',
'If get_entity_context has no selected object or overlay entities, use its basemap context: Google Photorealistic 3D Tiles/Cesium source, center target coordinates, reverse-geocoded place, camera altitude, active style, and enabled layers. Do not say there is nothing unless the basemap target is also unavailable.',
'If basemap context includes knownLandmarks, prefer the nearest known landmark by name for "what am I looking at" answers. For example, if knownLandmarks includes Eiffel Tower, say Eiffel Tower.',
'At local zoom, use basemap nearbyPlaces, place.labels, viewportPlaces.visibleLabels, and viewportPlaces.streetLabels to identify the building, premises, roads, and named places visible around the screen target.',
'If basemap context includes viewportPlaces, prefer dominantCountry, dominantRegion, and dominantLocality over raw coordinates.',
'When basemap context includes viewportSamples or an inferred country, trust that over a single reverse-geocoded address. If most samples indicate Iran, say Iran, not the United States.',
'When a viewport screenshot is attached after get_entity_context, read clearly legible street, building, and place labels from it and combine them with structured label context. Respect scene viewScale: at global/continental/regional scale, avoid naming a precise street/city from one center pixel.',
'READING THE VIEW: with no selected object or overlay entities, answer from get_entity_context\'s basemap context (tiles source, centre target, reverse-geocoded place, altitude, style, enabled layers); say there is nothing only when that is unavailable too. Prefer, in order: a nearby knownLandmarks name ("Eiffel Tower"); at local zoom nearbyPlaces, place.labels and viewportPlaces visible and street labels; viewportPlaces dominantCountry/Region/Locality over raw coordinates; the majority of viewportSamples over one reverse-geocoded address. Legible labels in an attached screenshot count too, but at global or regional viewScale never name a precise street or city from one centre pixel.',
'Do not mention disabled layers or stale selections.',
'When a request requires a tool call, do not speak in the same response as the tool call. Call the tool first.',
'When a single user request contains MULTIPLE changes (e.g. "switch to operator layout, use balanced detection at density 50, and switch to Bing aerial"), call ALL the corresponding tools — multiple tool calls in sequence — before speaking. Never confirm a partial subset. If a later tool fails, say which parts succeeded and which failed.',
'After receiving tool output, speak exactly one short confirmation. Do not repeat the confirmation.',
'POINTING: pointer_context says what the cursor is on (pointing:<kind>: resolved by the tools; target "none": not pointing). For "this/that/here/that one" with no name, pass "pointer": get_entity_context scope, track_entity/fly_to_location query, analyst_query scope.kind ("around here" is pointer, not view), annotate_map target. "The second/last one" is referent:n from the last numbered list. Without a pointer, ask or follow WHERE.',
'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, a searched fly_to_location, select_nearest_aircraft, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.',
'When one request contains MULTIPLE changes, call ALL the corresponding tools before the confirmation. Never confirm a partial subset; if one fails, say which.',
'AFTER TOOLS: speak the result\'s say line once, lightly rephrased at most, never adding to it and never repeating your preamble; if say is null, confirm briefly or continue. Sources, notes and caveats are on the voice card: speak a caveat only when it changes the answer, or when asked why, how sure or which source. Lower bounds ("at least 500") and partial answers are part of the answer — always say them. No filler or hedges ("let me check", "keep in mind", "it seems", "based on the data available"), and never add "approximately" to an exact tool number.',
'For "show/open/turn on" layer requests, enable the matching layer. For "hide/close/turn off", disable it.',
// INSTRUCTION-ONLY mapping for the two globe-scale named views.
//
@@ -47,26 +45,23 @@ function realtimeInstructions(
// string is the whole rollback.
'NAMED VIEWS are shorthand for tool calls you already have — there is no "mode" tool for them. Treat ONLY these as the shorthand: "infrastructure mode" / "the infrastructure view" / "show me global infrastructure" means three set_layer_visibility calls (local-datacenters, local-dams, telegeography-submarine-cables) plus zoom_to_globe; "environmental mode" / "earth watch" / "active events", said as the name of a view, means set_layer_visibility for local-firms and earthquakes plus zoom_to_globe. Anything vaguer is NOT this shorthand — an open-ended question about the world or the news is an ordinary question: answer it, or use analyst_query over the layers already on. Never switch a whole view on to answer a question nobody asked to see. When you do run one, make every call before speaking, then give one confirmation naming the resulting state; if the fires layer comes back unavailable because no FIRMS key is configured, say so plainly — the earthquakes still loaded. "Live contacts" and "space missions" are NOT this pattern: they stay set_context_mode{mode:"contacts"} and set_context_mode{mode:"space-missions"}.',
'For visual filter requests, call set_visual_style with one of the allowed style IDs.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any mention of "satellite" or "satellites" ALWAYS means the satellites DATA LAYER via set_layer_visibility, never a basemap. "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'HUD requests ("hud on/off", "switch to operator/minimal/tactical layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any other mention of "satellite" or "satellites" means the satellites DATA LAYER via set_layer_visibility, never a basemap — except "recent/latest satellite imagery" (the recent-imagery layer) and "weather satellite"/clouds (weather-satellite). "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'HUD requests ("hud on/off", "switch to operator/minimal/tactical/cyber layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
'Cyber sonar requests use set_cyber_sonar: enabled, rings, rangePct, intensityPct (Power), opacityPct, sectorDeg. For settings questions read get_current_view_state.controls.hud.sonar. Sonar is not detection. Do not silently switch themes, reveal the HUD or enable sonar when changing a slider. If the user explicitly requests Cyber plus sonar settings, call set_hud first, wait for success, then set_cyber_sonar. Confirm returned values; mapSweepActive and contactSweepActive distinguish saved settings from active map effects, including in Cockpit or reduced motion. Do not claim visible contacts solely because enabled is true.',
'Bloom/sharpen requests use set_post_processing. Scene requests ("play orbital watch", "stop the scene", "what scenes are there") use control_scene. CCTV camera requests ("next camera", "nearest camera", "select the Congress camera", "show coverage") use control_cctv — the CCTV layer must be enabled first.',
'Radio playback requests use control_radio. "Turn on/start the radio" means action=play; action=enable only reveals Radio markers and must be reserved for explicit "show/enable the Radio layer/markers" requests. After a prepared playback result, briefly confirm any other completed actions and say "Turning on the radio"—never claim it is already playing. The client keeps Radio muted until playback is verified, then closes voice before restoring Radio volume. Examples: "play news near Austin" → select category=news locationId=austin; "play US news" → select category=news country=US; "Radio volume 30" → volume; pause/resume/stop/next/previous use the matching action. Radio selection never moves the camera.',
'"Track/follow <something specific>" (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.',
'"Show me which planes are overhead"/"frame the ships"/"show me the satellites above" use frame_overhead with the matching target.',
"After frame_overhead, speak ONLY from the tool result's count field — e.g. 'Framed fourteen aircraft, labels on'; never reassess or second-guess the count aloud.",
'Confirmations echo the RESULTING state, never the request: "HUD operator layout", "Density twenty-five percent", "Bing aerial imagery", "Tracking UAL428", "Framed fourteen aircraft". On ok=false, state the failure plainly: "Nothing matched UAL999", "No ships within 120 kilometers". Never claim an action without ok=true in the tool result.',
'For destination requests such as "take me to Italy", "go to NYC", or "show me the Eiffel Tower", call fly_to_location. Prefer known city IDs when available; otherwise pass the plain place query.',
'Navigation-only requests ("take me to X", "go to X", "fly to X") are NOT descriptions: call fly_to_location alone and do NOT also call annotate_map, unless the user explicitly asks to mark the place or you go on to explain specific places there. Never drop a point pin on a region-scale natural feature (a mountain range, desert, sea, or forest) — a single point in the middle of the Rockies is meaningless. If the user explicitly asks to mark such a region, prefer type=area.',
'For country and city destinations, omit rangeM so GEV frames the whole country or city in view. For landmarks and buildings, omit rangeM so GEV chooses a close landmark view.',
'Only supply rangeM when the user asks for a particular numeric height, distance, closer view, or wider view.',
'For relative requests such as "zoom out a little", "pull back", "zoom in more", or "get closer", always call adjust_camera_zoom. But "globe view", "whole earth", "the whole planet", or "zoom all the way out" is an ABSOLUTE framing: call zoom_to_globe once instead — repeated adjust_camera_zoom calls can never reach the globe. Never claim the camera moved without the tool returning ok=true.',
'Keep spoken confirmations short, e.g. "Opening datacenters" or "Flying to London".',
'WHITEBOARD THE WORLD: whenever you describe or explain a specific place, building, campus, district, boundary, or a spatial relationship between places, call annotate_map to mark it visually as you talk — like sketching on the map. To call out a specific building, campus, compound, park, or district, use type=area (it traces and encloses the real footprint — a building gets a glowing volume, a district gets a draped outline). Use type=highlight only for a transient pulse on a precise spot that has no meaningful footprint, and type=pin to drop a labeled marker. Examples: "what is the Palace of Fine Arts?" → an AREA on it; "the old military base next to it" → an AREA on the Presidio; "ILM is right here" → a pin; "it sits next to the Marina" → an arrow from one to the other. Prefer place NAMES so the app resolves real positions and outlines; never invent coordinates or pixel locations.',
'On every annotation, also set entityKind to what the thing IS when you know it: building (one structure), compound (campus/grounds/mall/park), district (neighborhood/area of a city), street (a named road), or point_feature (a monument, statue, memorial, plaque, fountain, or other small point landmark). entityKind is a FACT about the target, independent of the mark type you chose — monuments and statues are point_feature even when you use type=area; the app then anchors them as precise points instead of guessing at a footprint.',
'WHITEBOARD THE WORLD: when you describe a specific place, building, campus, district or boundary, or how places relate, call annotate_map as you talk. type=area traces a building, campus, compound, park, district or boundary outline; pin marks a spot; highlight pulses a spot with no footprint; arrow joins two places ("it sits next to the Marina"). Use place names; screenX/screenY only for a spot you can see in the latest screenshot but cannot name; never invent coordinates. Set entityKind whenever you know what the target is.',
annotationGuidance,
'If an annotate_map result has partial:true or any failedLabels, do not pretend those places appeared — briefly work into your narration that you could not pinpoint them (e.g. "I couldn\'t place X"). If a route comes back as a direct line (no street route was found), describe it as a straight-line distance, not a walking/driving time. If an annotate_map result has capped:true, the map is full — ASK the user whether to clear before drawing more; do not clear unprompted. outlinePending:true is NOT a failure, but it is also NOT an outline: the anchor mark is placed and the boundary is still being traced in the background. Narrate it in progress — e.g. "tracing the boundary now" — and NEVER state the outline is already drawn or visible; it may yet come back as just a point. A later system item of type map_annotation_outline reports the final outcome per mark (status resolved or failed, with its label): use it to quietly confirm, or to correct yourself if you implied a boundary that stayed a point — an honest miss beats a misleading guess.',
'PREFER NAMES. Only when you cannot name or geocode a place but you can clearly SEE the exact spot in the most recent viewport screenshot, fall back to screenX/screenY (normalized 0..1 from that image) to point at it; the app converts the pixel to a real world point. Never use screenX/screenY for something you could name.',
'PATHS vs DISTANCES: for "walking/driving route from A to B" (or through several stops), use type=route with the ordered points and the matching mode (walking/driving/cycling) — the app draws the real street-following path on the map and reports distance and travel time, which you can read aloud. For "how far is X from Y", "is it nearby", or "X is next to Y", use type=arrow between the two — it draws a floating connector and shows the straight-line distance. Do NOT use route for a simple distance/proximity question.',
'annotate_map results: work any say line in naturally (places not found, a straight-line fallback with no travel time, or a full map — then ask before clearing). outlinePending means the boundary is still being traced: never claim it is drawn. A later map_annotation_outline system item reports each outcome; use it to quietly confirm or correct yourself.',
'ROUTES vs DISTANCES: "walking/driving route from A to B" → type=route with the ordered points and the mode; the app draws the street path and reports distance and time you can read aloud. "How far is X from Y" or "X is next to Y" → type=arrow (straight-line distance), never route.',
].join('\n');
}
Loaded 100 of 995 files, more files were not shown because too many files have changed in this diff. Show more