diff --git a/CHANGELOG.md b/CHANGELOG.md index 58103d04d..c2f35af2a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,10 @@ # Changelog +- Add Global (RainViewer) beside US (NOAA) in Rain radar, with shared observed + history and share-link region selection. Validated, rate-limited Web Mercator + tiles draw on globe basemaps and compose into reprojected images on 3D Tiles + shells; coverage follows national radar networks, with 2 hours of history. + - Public Overpass instances are no longer used by default. Street Traffic roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen on the layer row (TomTom / OSM / Hybrid) or with `?trafficRoads=`. With a diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index ab059cebb..b10049833 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -18,6 +18,7 @@ How to read this: | **NOAA GFS (wind)** (`noaa-gfs-bdp-pds.s3.amazonaws.com`) | Global 10 m wind, optional 2 m temperature and mean sea-level pressure for Wind. Keyless; latest 6-hourly 0.25° cycle, byte-range GRIB2 reads, cached for an hour | U.S. public domain (NOAA); keyless via NOAA Open Data on AWS | "NOAA Global Forecast System (GFS)" (courtesy; not an endorsement) | | **ECMWF IFS (wind)** (`data.ecmwf.int/forecasts`, ECMWF Open Data) | The alternative Wind model: 10 m wind, optional 2 m temperature and mean sea-level pressure. Keyless; latest 6-hourly 0.25° run, byte-range GRIB2 reads, cached for an hour | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) plus the [ECMWF Terms of Use](https://apps.ecmwf.int/datasets/licences/general/) | "This service is based on data and products of the European Centre for Medium-Range Weather Forecasts (ECMWF)", with the CC BY 4.0 link, the modification notice and ECMWF's liability disclaimer — shown in-app | | **NOAA nowCOAST observed weather** (`nowcoast.noaa.gov/geoserver/observations/{weather_radar,satellite}/ows`, WMS 1.1.1) | Rain radar (MRMS base reflectivity, contiguous US, about 4-minute updates) and Satellite clouds (GOES-19/18 Band 14 regional infrared, about 5-minute updates; global infrared mosaic, hourly). Keyless; layer metadata refreshed every 2 minutes, exact-time images cached for up to 24 hours | [NOAA disclaimer](https://oceanservice.noaa.gov/disclaimer.html) | "NOAA nowCOAST · NWS/OAR MRMS radar; NESDIS GOES and global satellite partners" | +| **RainViewer** (`api.rainviewer.com`, `tilecache.rainviewer.com`) | Rain radar, Global: past 2 hours of radar composite tiles at 10-minute steps, zoom ≤ 7. Keyless; manifest cached 5 minutes, tiles cached like other weather tiles, upstream limited to 80 tile requests a minute | [RainViewer API terms](https://www.rainviewer.com/api.html) | RainViewer (rainviewer.com), linked in-app | | **NOAA nowCOAST lightning density** (`nowcoast.noaa.gov/geoserver/observations/lightning_detection/ows`) | Lightning density: 15-minute accumulated strike density on an approximately 8 km grid, Americas and Pacific. Keyless; layer metadata refreshed every 10 minutes, exact-time images cached for up to 24 hours | NOAA Level-5 derived product from Vaisala NLDN/GLD360; the [product description](https://ocean.weather.gov/lightning/lightning_pdd.php) permits public distribution of this density product, not of raw Vaisala detections | "NOAA/NWS nowCOAST; derived from Vaisala NLDN/GLD360" | | **NOAA NHC / CPHC advisories** (`www.nhc.noaa.gov/CurrentStorms.json` + the `mapservices.weather.noaa.gov` tropical weather summary MapServer) | Cyclone advisories: storm positions, forecast tracks, points and cones for the Atlantic and eastern/central North Pacific. Keyless through `/api/cyclones`; cached for 5 minutes | [NWS public-data terms](https://www.weather.gov/disclaimer); no endorsement implied | "NOAA/NWS NHC / CPHC" | | **OpenStreetMap** | Map and place data (roads, military areas and names, ALPR, place search, outlines); via vector tiles, hourly ALPR extract, bundled data, Nominatim and optional self-configured Overpass | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/) | © OpenStreetMap contributors | diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 22ce1eb3b..d193f0444 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -93,6 +93,14 @@ catalog construction owns each instance. Wind animates one forecast without advancing forecast time. Separate Weather observation layers provide radar and satellite history; none claims measured cloud volume. +Rain radar defaults to **US (NOAA)** MRMS; **Global (RainViewer)** selects a keyless +composite where national radars exist, with the past 2 hours at 10-minute steps +on the shared observed timeline, and the region choice persists in share links. +RainViewer uses pinned, validated 256 px Web Mercator tiles through zoom 7, +5-minute metadata caching and a server-wide 80-tile-request rolling-minute limit; +gaps do not mean no rain. Globe hosts draw those tiles directly; on 3D Tiles, +the 6.2 km shell composes at most 24 tiles per full frame or detail window and +reprojects them to an equirectangular image. Weather imagery drapes onto the globe basemap. On photorealistic 3D Tiles each observed product is instead a raised, translucent shell: one rectangle over the product bounds at a fixed height (global infrared 5.5 km, regional infrared @@ -105,10 +113,10 @@ other map content draws over the shells, and shells are not pickable. A map-source change tears down one renderer and restages the shown observation on the other. A map without an imagery host pauses history while metadata refresh continues, then resumes when a host returns. -Globe tiled products use 256 px tiles to maximum level 6. The tile proxy accepts +NOAA globe tiled products use 256 px tiles to maximum level 6. The tile proxy accepts size=256 (default), 512 or 1024 and keys cached bytes by size, retaining the 24-hour immutable response and eight upstream slots. The image proxy serves every -product as one whole-extent PNG at its advertised bounds, up to 4096×2048 for +NOAA product as one whole-extent PNG at its advertised bounds, up to 4096×2048 for radar, regional infrared and lightning and 2048×1024 for global infrared (`size=W×H`, default the maximum), capped at 16 MiB and cached by product, time and size. NOAA returned each of these sizes from a single request in a live @@ -127,7 +135,7 @@ full-extent image covers it. A new frame swaps the full-extent image first and keeps the previous detail image until the new one decodes, over at most one newer frame; a moved window hides until its image arrives. Global infrared contrast depends on the requested extent, so it keeps one image. The -image proxy accepts `bbox=west,south,east,north` for every product: inside the +image proxy accepts `bbox=west,south,east,north` for every NOAA product: inside the product bounds, 2:1 within 1 %, rounded to 0.25°, up to 4096×2048 (the default), cached by product, time, size and bbox. Diagnostics report `shell.detail` (`bbox`, `size`, `ready`, `enabled`). diff --git a/docs/KNOWN-ISSUES.md b/docs/KNOWN-ISSUES.md index e41623cc6..ac4361fb6 100644 --- a/docs/KNOWN-ISSUES.md +++ b/docs/KNOWN-ISSUES.md @@ -81,9 +81,10 @@ Status: Open (owner-accepted 2026-07-08, documented) ### Weather layers: coverage and meaning Status: Open (source limits, by design) -- **Rain radar covers the contiguous United States only.** It shows MRMS - reflectivity in dBZ, not rainfall rate or a forecast; a gap in coverage does not - mean no precipitation. +- **Rain radar:** the US option is MRMS for the contiguous United States; the + Global option is RainViewer’s composite where national radars exist, past + 2 hours only. Both show reflectivity in dBZ, not rainfall rate or a forecast; + gaps are not “no rain”. - **Lightning density is a ground-network grid, not GLM flashes.** It is NOAA's 15-minute density product derived from Vaisala NLDN/GLD360 on an approximately 8 km grid, for the Americas and Pacific only (110°E across the dateline to 0°, diff --git a/scripts/format-scope.json b/scripts/format-scope.json index cd836a5cb..b4e93ec8d 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -864,5 +864,7 @@ "src/layers/traffic/motion.test.mjs", "scripts/qa-journey-recorder.mjs", "scripts/qa-alpr-journey.mjs", - "scripts/qa-context-arrivals.mjs" + "scripts/qa-context-arrivals.mjs", + "src/data/weatherRainviewer.test.mjs", + "src/layers/weather/mercatorMosaic.test.mjs" ] diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index 276b9e0db..0bdb44170 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -1953,6 +1953,7 @@ "src/layers/weather/shellRendering.js", "src/layers/weather/infraredAlpha.js", "src/layers/weather/infraredImage.js", + "src/layers/weather/mercatorMosaic.js", "src/layers/weather/source.js", "src/director/clock.js", "src/director/timeline.js", @@ -2465,6 +2466,7 @@ "src/layers/weather/shellRendering.js", "src/layers/weather/infraredAlpha.js", "src/layers/weather/infraredImage.js", + "src/layers/weather/mercatorMosaic.js", "src/layers/weather/source.js", "src/layers/weather/imageryOrder.js", "src/layers/weather/imageryHost.js", diff --git a/server/providers/rainViewerGovernor.js b/server/providers/rainViewerGovernor.js new file mode 100644 index 000000000..864da8006 --- /dev/null +++ b/server/providers/rainViewerGovernor.js @@ -0,0 +1,12 @@ +/** Admit at most 80 requests in a rolling minute; return retry seconds or zero. + * Call only at the upstream boundary, after cache lookup and coalescing. */ +export function createRainViewerGovernor() { + const attempts = []; + return (now) => { + while (attempts.length && now - attempts[0] >= 60_000) attempts.shift(); + if (attempts.length >= 80) + return Math.max(1, Math.ceil((60_000 - (now - attempts[0])) / 1000)); + attempts.push(now); + return 0; + }; +} diff --git a/server/providers/weather.js b/server/providers/weather.js index 2302359ed..120edef4b 100644 --- a/server/providers/weather.js +++ b/server/providers/weather.js @@ -1,7 +1,11 @@ +import { createRainViewerGovernor } from './rainViewerGovernor.js'; import { readResponseTextCapped } from './common/http.js'; import { readWindBody as readBytesCapped } from '../../src/sources/windBody.js'; const BASE = 'https://nowcoast.noaa.gov/geoserver/observations/'; +const RAINVIEWER_MANIFEST = + 'https://api.rainviewer.com/public/weather-maps.json'; +const RAINVIEWER_TILES = 'https://tilecache.rainviewer.com'; const HOUR = 3600_000; const PRODUCTS = Object.freeze({ lightning: Object.freeze({ @@ -28,6 +32,25 @@ const PRODUCTS = Object.freeze({ 'Observed MRMS radar base reflectivity (dBZ), approximately 1 km and 4-minute updates; not a rainfall forecast.', image: Object.freeze({ width: 4096, height: 2048 }), }), + 'radar-global': Object.freeze({ + service: 'rainviewer', + title: 'Global radar reflectivity', + coverage: 'Global · where radars exist', + description: + 'RainViewer global radar composite (dBZ). Coverage follows national radar networks; a gap does not mean no rain.', + source: 'RainViewer', + attribution: 'RainViewer', + metadataTtlMs: 300_000, + tilingScheme: 'web-mercator', + maxLevel: 7, + bounds: Object.freeze({ + west: -180, + south: -85.0511, + east: 180, + north: 85.0511, + }), + image: Object.freeze({ width: 2048, height: 1024 }), + }), clouds: Object.freeze({ service: 'satellite', layer: 'global_longwave_imagery_mosaic', @@ -169,6 +192,52 @@ export function parseWeatherCapabilities(xml, product, nowMs = Date.now()) { return { bounds, times: recent.slice(-13), allowedTimes: recent }; } +/** Accept only observed frames with bounded paths; never trust the manifest host. */ +export function parseRainViewerManifest(text) { + const manifest = JSON.parse(text); + if (manifest?.version !== '2.0' || !Array.isArray(manifest.radar?.past)) + throw failure('invalid_weather_metadata'); + const frames = new Map(); + for (const frame of manifest.radar.past) { + if ( + !Number.isSafeInteger(frame?.time) || + frame.time <= 0 || + typeof frame.path !== 'string' || + !/^\/v2\/radar\/[A-Za-z0-9]{1,64}$/.test(frame.path) + ) + continue; + const date = new Date(frame.time * 1000); + if (!Number.isFinite(date.getTime()) || date.getUTCFullYear() > 9999) + continue; + frames.set(date.toISOString(), frame.path); + } + const times = [...frames.keys()].sort().slice(-13); + if (!times.length) throw failure('invalid_weather_metadata'); + return { + bounds: PRODUCTS['radar-global'].bounds, + times, + allowedTimes: times, + framePaths: Object.fromEntries( + times.map((time) => [time, frames.get(time)]), + ), + }; +} + +/** Web Mercator XYZ has one square root, unlike NOAA's geographic grid. */ +export function rainViewerTileCoordinates(z, x, y) { + if ( + ![z, x, y].every(Number.isInteger) || + z < 0 || + z > 7 || + x < 0 || + y < 0 || + x >= 2 ** z || + y >= 2 ** z + ) + throw failure('invalid_weather_tile', 400); + return [z, x, y]; +} + /** A detail window `west,south,east,north` in degrees, rounded to 0.25° so cache * keys repeat, with a 2:1 aspect within 1 %; null when absent. Containment in * the product bounds is checked once the metadata is known. */ @@ -234,12 +303,13 @@ function validatePng(bytes, width, height) { return buffer; } -/** Fixed NOAA observed-weather metadata and WMS tiles; no client-supplied destinations. */ +/** Fixed observed-weather metadata and tiles; no client-supplied destinations. */ export function weatherProxy({ fetchImpl = fetch, now = () => Date.now(), timeoutMs = 12_000, } = {}) { + const admitRainViewerTile = createRainViewerGovernor(); const metadata = new Map(); const attempts = new Map(); const operations = new Map(); @@ -310,12 +380,18 @@ export function weatherProxy({ if (!clientSignal.aborted) operation.waiters--; } } - async function upstream(url, signal, image = null) { + async function upstream(url, signal, image = null, json = false) { signal.throwIfAborted(); const response = await fetchImpl(url, { signal, redirect: 'error', - headers: { Accept: image ? 'image/png' : 'application/xml,text/xml' }, + headers: { + Accept: image + ? 'image/png' + : json + ? 'application/json' + : 'application/xml,text/xml', + }, }); if ( !response.ok || @@ -331,7 +407,11 @@ export function weatherProxy({ image.width, image.height, ) - : await readResponseTextCapped(response, 512 * 1024, signal); + : await readResponseTextCapped( + response, + (json ? 256 : 512) * 1024, + signal, + ); signal.throwIfAborted(); return value; } @@ -348,14 +428,19 @@ export function weatherProxy({ ) throw failure('weather_upstream_unavailable'); attempts.set(product, now()); - const url = `${BASE}${spec.service}/ows?service=WMS&version=1.3.0&request=GetCapabilities`; - const xml = await shared( + const globalRadar = product === 'radar-global'; + const url = globalRadar + ? RAINVIEWER_MANIFEST + : `${BASE}${spec.service}/ows?service=WMS&version=1.3.0&request=GetCapabilities`; + const text = await shared( `metadata:${spec.service}`, - (activeSignal) => upstream(url, activeSignal), + (activeSignal) => upstream(url, activeSignal, null, globalRadar), signal, ); const value = { - ...parseWeatherCapabilities(xml, product, now()), + ...(globalRadar + ? parseRainViewerManifest(text) + : parseWeatherCapabilities(text, product, now())), fetchedAt: now(), }; metadata.set(product, value); @@ -388,7 +473,7 @@ export function weatherProxy({ title: spec.title, coverage: spec.coverage, description: spec.description, - source: 'NOAA nowCOAST', + source: spec.source ?? 'NOAA nowCOAST', attribution: spec.attribution ?? 'NOAA/NWS/NESDIS nowCOAST', bounds: value?.bounds ?? null, times: value?.times ?? [], @@ -404,22 +489,23 @@ export function weatherProxy({ ? 'Cached weather metadata; upstream unavailable' : null, tileSize: 256, - maxLevel: 6, - tilingScheme: 'geographic', + maxLevel: spec.maxLevel ?? 6, + tilingScheme: spec.tilingScheme ?? 'geographic', tileTemplate: time ? `/api/weather/tile?product=${product}&time=${encodeURIComponent(time)}&z={z}&x={x}&y={y}` : null, - imageUrl: time - ? `/api/weather/image?product=${product}&time=${encodeURIComponent(time)}` - : null, + imageUrl: + time && product !== 'radar-global' + ? `/api/weather/image?product=${product}&time=${encodeURIComponent(time)}` + : null, imageSize: { ...spec.image }, }; } - function json(res, status, body) { + function json(res, status, body, retryAfter = 2) { res.writeHead(status, { 'Content-Type': 'application/json', 'Cache-Control': 'no-store', - ...(status === 429 ? { 'Retry-After': '2' } : {}), + ...(status === 429 ? { 'Retry-After': String(retryAfter) } : {}), }); res.end(JSON.stringify(body)); } @@ -451,6 +537,9 @@ export function weatherProxy({ if (!Object.hasOwn(PRODUCTS, product)) return json(res, 400, { error: 'unknown_weather_product' }); const wholeImage = url.pathname === '/image'; + const globalRadar = product === 'radar-global'; + if (wholeImage && globalRadar) + throw failure('whole_image_unsupported', 400); const detailBox = wholeImage ? weatherImageBbox(url.searchParams.get('bbox')) : null; @@ -462,7 +551,7 @@ export function weatherProxy({ wholeImage ? !IMAGE_SIZES.includes(size) || Number.parseInt(size, 10) > largest.width - : !['256', '512', '1024'].includes(size) + : !(globalRadar ? ['256'] : ['256', '512', '1024']).includes(size) ) throw failure( wholeImage @@ -487,7 +576,9 @@ export function weatherProxy({ if (coords?.some((value) => !/^(?:0|[1-9]\d{0,2})$/.test(value ?? ''))) throw failure('invalid_weather_tile', 400); const tileBounds = coords - ? weatherTileBounds(...coords.map(Number)) + ? globalRadar + ? rainViewerTileCoordinates(...coords.map(Number)) + : weatherTileBounds(...coords.map(Number)) : null; const time = url.searchParams.get('time'); if (!time || observationTime(time) !== time) @@ -495,7 +586,9 @@ export function weatherProxy({ const value = await getMetadata(product, controller.signal); controller.signal.throwIfAborted(); if ( - !value.allowedTimes.includes(time) || + (globalRadar + ? !Object.hasOwn(value.framePaths, time) + : !value.allowedTimes.includes(time)) || now() - Date.parse(time) > 24 * HOUR ) throw failure('unknown_weather_time', 400); @@ -543,25 +636,39 @@ export function weatherProxy({ if (now() - (tileFailures.get(key) ?? -Infinity) < 30_000) throw failure('weather_upstream_unavailable'); const spec = PRODUCTS[product]; - const upstreamUrl = new URL(`${BASE}${spec.service}/ows`); - upstreamUrl.search = new URLSearchParams({ - service: 'WMS', - version: '1.1.1', - request: 'GetMap', - layers: spec.layer, - styles: spec.style, - srs: 'EPSG:4326', - bbox: bbox.join(','), - width: String(imageShape.width), - height: String(imageShape.height), - format: 'image/png', - transparent: 'true', - time, - }).toString(); + const upstreamUrl = new URL( + globalRadar + ? `${RAINVIEWER_TILES}${value.framePaths[time]}/256/${coords.join('/')}/2/1_1.png` + : `${BASE}${spec.service}/ows`, + ); + if (!globalRadar) + upstreamUrl.search = new URLSearchParams({ + service: 'WMS', + version: '1.1.1', + request: 'GetMap', + layers: spec.layer, + styles: spec.style, + srs: 'EPSG:4326', + bbox: bbox.join(','), + width: String(imageShape.width), + height: String(imageShape.height), + format: 'image/png', + transparent: 'true', + time, + }).toString(); try { const bytes = await shared( `image:${key}`, - (signal) => upstream(upstreamUrl.href, signal, imageShape), + (signal) => { + if (globalRadar) { + const retryAfter = admitRainViewerTile(now()); + if (retryAfter) + throw Object.assign(failure('weather_busy', 429), { + retryAfter, + }); + } + return upstream(upstreamUrl.href, signal, imageShape); + }, controller.signal, ); rememberTile(key, bytes); @@ -593,6 +700,7 @@ export function weatherProxy({ ? error.code : 'weather_upstream_unavailable', }, + error.retryAfter, ); } finally { res.removeListener?.('close', close); diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index f9fc23b70..16a645bc6 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -271,6 +271,10 @@ export const DATA_CREDITS = [ 'Re:Earth Terrain / ' + 'Mapterhorn (CC BY 4.0) / EGM2008 (NGA)', }, + { + key: 'rainviewer', + html: 'Global rain radar: RainViewer', + }, { key: 'weather-noaa', html: 'Observed weather: NOAA nowCOAST · NWS/OAR MRMS radar; NESDIS GOES and global satellite partners. Source disclaimer.', diff --git a/src/data/layerState.js b/src/data/layerState.js index eccc555f6..129ae9fba 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -312,6 +312,10 @@ const OPTION_GROUPS = Object.freeze({ }), ]), 'weather-radar': Object.freeze([ + enumOption('product', 'p', 'radar', ['radar', 'radar-global'], { + radar: 'r', + 'radar-global': 'g', + }), enumOption('opacity', 'o', 'strong', ['light', 'strong'], { light: 'l', strong: 's', diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index 1b504a439..764718a25 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -2642,3 +2642,32 @@ test('fire perimeters uses digit 2 without colliding with wind or recent imagery decoded, ); }); + +test('RainViewer radar choice round trips on the existing v token and old links default to NOAA', () => { + const entry = LAYER_STATE_REGISTRY.find(({ id }) => id === 'weather-radar'); + assert.equal(entry.token, 'v'); + assert.equal(entry.optionOwner, 'weather-radar'); + for (const product of ['radar', 'radar-global']) { + const state = normalizeLayerState({ + enabledLayerIds: ['weather-radar'], + options: { 'weather-radar': { product, opacity: 'light', play: true } }, + }); + const decoded = decodeLayerStateParams(new URLSearchParams(encode(state))); + assert.deepEqual(decoded, state); + assert.equal(decoded.options['weather-radar'].product, product); + assert.equal( + Object.hasOwn(decoded.options['weather-radar'], 'play'), + false, + ); + } + const old = normalizeLayerState({ + enabledLayerIds: ['weather-radar'], + options: { 'weather-radar': { opacity: 'light' } }, + }); + assert.equal(old.options['weather-radar'].product, 'radar'); + assert.equal( + normalizeLayerState({ options: { 'weather-radar': { product: 'clouds' } } }) + .options['weather-radar'].product, + 'radar', + ); +}); diff --git a/src/data/weatherRainviewer.test.mjs b/src/data/weatherRainviewer.test.mjs new file mode 100644 index 000000000..757727440 --- /dev/null +++ b/src/data/weatherRainviewer.test.mjs @@ -0,0 +1,263 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { EventEmitter } from 'node:events'; +import { + weatherProxy, + parseRainViewerManifest, +} from '../../server/providers/weather.js'; +import { createRainViewerGovernor } from '../../server/providers/rainViewerGovernor.js'; + +const seconds = 1790215200; +const TIME = new Date(seconds * 1000).toISOString(); +const frame = (time = seconds, path = '/v2/radar/47d5f2b2ef80') => ({ + time, + path, +}); +const manifest = (past = [frame()]) => ({ + version: '2.0', + host: 'http://127.0.0.1/evil', + radar: { past, nowcast: [frame(seconds + 600)] }, +}); +function png(width = 256) { + const bytes = Buffer.alloc(33); + Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]).copy(bytes); + bytes.writeUInt32BE(13, 8); + bytes.write('IHDR', 12); + bytes.writeUInt32BE(width, 16); + bytes.writeUInt32BE(256, 20); + return new Response(bytes, { headers: { 'Content-Type': 'image/png' } }); +} +function install(options = {}) { + let handler; + weatherProxy({ now: () => seconds * 1000, ...options }).configureServer({ + middlewares: { + use(_path, fn) { + handler = fn; + }, + }, + }); + return async (url) => { + const res = new EventEmitter(); + res.writeHead = (status, headers) => + Object.assign(res, { status, headers }); + res.end = (body) => { + res.body = body; + }; + await handler({ url, method: 'GET' }, res); + return res; + }; +} +const tile = (extra = {}) => + '/tile?' + + new URLSearchParams({ + product: 'radar-global', + time: TIME, + z: 7, + x: 0, + y: 0, + ...extra, + }); +const body = (res) => JSON.parse(res.body); + +test('RainViewer parses sorted observed frames only, drops malformed frames and bounds the fresh map', () => { + const past = Array.from({ length: 20 }, (_, i) => + frame(seconds - i * 600, `/v2/radar/frame${i}`), + ); + past.push( + null, + {}, + frame(0), + frame(-1), + frame('1790215200'), + frame(1.5), + frame(1e30), + frame(seconds, '/v2/radar/../escape'), + frame(seconds, 'https://evil.test'), + frame(seconds, '/v2/radar/' + 'a'.repeat(65)), + frame(seconds, '/v2/radar/abc?x=1'), + ); + const parsed = parseRainViewerManifest(JSON.stringify(manifest(past))); + assert.equal(parsed.times.length, 13); + assert.deepEqual(parsed.times, [...parsed.times].sort()); + assert.equal(parsed.times.at(-1), TIME); + assert.equal(Object.keys(parsed.framePaths).length, 13); + assert.equal(parsed.framePaths[TIME], '/v2/radar/frame0'); + assert.throws(() => parseRainViewerManifest(JSON.stringify(manifest([])))); + assert.throws(() => parseRainViewerManifest('{}')); +}); + +test('RainViewer pins both URLs, uses PNG validation, caches tiles and refreshes a replaced frame map', async () => { + let clock = seconds * 1000; + let past = [frame()]; + const calls = []; + const request = install({ + now: () => clock, + fetchImpl: async (url, options) => { + calls.push({ url, options }); + return url.includes('weather-maps.json') + ? Response.json(manifest(past)) + : png(); + }, + }); + const first = body(await request('/manifest?product=radar-global')); + assert.equal(first.title, 'Global radar reflectivity'); + assert.equal(first.tilingScheme, 'web-mercator'); + assert.equal(first.maxLevel, 7); + assert.equal(first.attribution, 'RainViewer'); + assert.equal(first.imageUrl, null); + assert.equal( + calls[0].url, + 'https://api.rainviewer.com/public/weather-maps.json', + ); + const response = await request(tile()); + assert.equal(response.status, 200); + assert.equal( + response.headers['Cache-Control'], + 'public, max-age=86400, immutable', + ); + assert.equal( + calls[1].url, + 'https://tilecache.rainviewer.com/v2/radar/47d5f2b2ef80/256/7/0/0/2/1_1.png', + ); + assert.equal(calls[1].options.redirect, 'error'); + assert.deepEqual((await request(tile())).body, response.body); + assert.equal(calls.length, 2); + clock += 299_999; + await request('/manifest?product=radar-global'); + assert.equal(calls.length, 2); + clock++; + past = [frame(seconds + 600, '/v2/radar/new')]; + await request('/manifest?product=radar-global'); + const old = await request(tile()); + assert.equal(old.status, 400); + assert.equal(body(old).error, 'unknown_weather_time'); + assert.equal( + calls.length, + 3, + 'expired map entries cannot serve even a cached tile', + ); +}); + +test('RainViewer rejects invalid coordinates, sizes, destinations and whole images before upstream', async () => { + let calls = 0; + const request = install({ + fetchImpl: async () => { + calls++; + throw new Error(); + }, + }); + for (const coords of [ + { z: 8 }, + { x: 128 }, + { y: 128 }, + { z: 0, x: 1 }, + { z: 0, y: 1 }, + ...['z', 'x', 'y'].flatMap((key) => + ['1e3', '0x1', '-0', '-1'].map((value) => ({ [key]: value })), + ), + { size: 512 }, + { size: 1024 }, + { size: '0256' }, + { host: 'https://evil.test' }, + ]) { + assert.equal( + (await request(tile(coords))).status, + 400, + JSON.stringify(coords), + ); + } + const image = await request('/image?product=radar-global&time=' + TIME); + assert.equal(image.status, 400); + assert.equal(body(image).error, 'whole_image_unsupported'); + assert.equal(calls, 0); +}); + +test('RainViewer metadata retains stale observations and enforces capped reads and 30-second failure backoff', async () => { + let clock = seconds * 1000, + calls = 0; + const request = install({ + now: () => clock, + fetchImpl: async () => { + calls++; + if (calls === 1) return Response.json(manifest()); + throw new Error('offline'); + }, + }); + await request('/manifest?product=radar-global'); + clock += 300_000; + assert.equal( + body(await request('/manifest?product=radar-global')).stale, + true, + ); + clock += 29_999; + assert.equal( + body(await request('/manifest?product=radar-global')).latest, + TIME, + ); + assert.equal(calls, 2); + clock++; + await request('/manifest?product=radar-global'); + assert.equal(calls, 3); + for (const declared of [false, true]) { + const capped = install({ + fetchImpl: async () => + new Response(' '.repeat(256 * 1024 + 1), { + headers: declared ? { 'Content-Length': String(256 * 1024 + 1) } : {}, + }), + }); + assert.equal( + body(await capped('/manifest?product=radar-global')).unavailable, + true, + ); + } +}); + +test('RainViewer governor is rolling, bounded and supplies the earliest retry time', () => { + const admit = createRainViewerGovernor(); + for (let i = 0; i < 80; i++) assert.equal(admit(i * 10), 0); + assert.equal(admit(1000), 59); + assert.equal(admit(59_999), 1); + assert.equal(admit(60_000), 0); + assert.equal(admit(60_000), 1); + assert.equal(admit(120_000), 0); +}); + +test('RainViewer server governor stops at 80 upstream tiles across times, admits cache hits and resumes after a minute', async () => { + let clock = seconds * 1000, + tiles = 0; + const request = install({ + now: () => clock, + fetchImpl: async (url) => { + if (url.includes('weather-maps.json')) return Response.json(manifest()); + tiles++; + return png(); + }, + }); + for (let x = 0; x < 80; x++) + assert.equal((await request(tile({ x }))).status, 200); + const blocked = await request(tile({ x: 80 })); + assert.equal(blocked.status, 429); + assert.equal(blocked.headers['Retry-After'], '60'); + assert.equal(tiles, 80); + assert.equal((await request(tile())).status, 200); + assert.equal(tiles, 80); + clock += 60_000; + assert.equal((await request(tile({ x: 80 }))).status, 200); + assert.equal(tiles, 81); +}); + +test('RainViewer rejects non-PNG and wrong PNG dimensions', async () => { + for (const response of [ + () => png(512), + () => new Response('bad'), + () => new Response('bad', { headers: { 'Content-Type': 'image/png' } }), + ]) { + const request = install({ + fetchImpl: async (url) => + url.includes('weather-maps.json') + ? Response.json(manifest()) + : response(), + }); + assert.equal((await request(tile())).status, 503); + } +}); diff --git a/src/layers/weather/index.js b/src/layers/weather/index.js index 8d6d331eb..363638888 100644 --- a/src/layers/weather/index.js +++ b/src/layers/weather/index.js @@ -18,6 +18,17 @@ const STOPS = [ [70, '#ff00ff'], [80, '#05ede0'], ]; +const RAINVIEWER_STOPS = [ + [10, '#cec087'], + [20, '#00a3e0'], + [30, '#005588'], + [35, '#ffee00'], + [40, '#ffaa00'], + [45, '#ff4400'], + [50, '#c10000'], + [55, '#ffaaff'], + [65, '#ffffff'], +]; const LIGHTNING_STOPS = [ ['0.1', '#FFFFCC'], ['1', '#FFA400'], @@ -255,7 +266,9 @@ export function createWeatherLayer({ ? 'Lightning density' : 'Satellite clouds', icon: radar ? '◉' : lightning ? 'ϟ' : '☁', - source: 'NOAA nowCOAST · OBSERVED', + get source() { + return `${product === 'radar-global' ? 'RainViewer' : 'NOAA nowCOAST'} · OBSERVED`; + }, updateInterval: lightning ? 600_000 : 120_000, init(nextViewer) { viewer = nextViewer; @@ -392,8 +405,9 @@ export function createWeatherLayer({ rendering?.setAlpha(opacity === 'light' ? 0.4 : satellite ? 0.7 : 0.8); } if ( - satellite && - ['clouds', 'clouds-regional'].includes(params.product) && + ((satellite && + ['clouds', 'clouds-regional'].includes(params.product)) || + (radar && ['radar', 'radar-global'].includes(params.product))) && params.product !== product ) { product = params.product; @@ -475,7 +489,7 @@ export function createWeatherLayer({ }, getParams() { return radar - ? { opacity } + ? { product, opacity } : satellite ? { product, opacity, infrared } : { product, opacity }; @@ -539,12 +553,16 @@ export function createWeatherLayer({ readout: true, summary: { label: radar - ? 'Rain radar · US' + ? product === 'radar-global' + ? 'Rain radar · Global' + : 'Rain radar · US' : lightning ? 'Lightning density · 15 min' : 'Satellite clouds', coverage: radar - ? 'CONUS' + ? product === 'radar-global' + ? 'Global · where radars exist' + : 'CONUS' : lightning ? 'Americas + Pacific' : product === 'clouds' @@ -574,6 +592,21 @@ export function createWeatherLayer({ units: radar ? 'dBZ' : lightning ? 'strikes/km²/min ×10³' : '', }, chips: [ + ...(radar + ? [ + ['radar', 'US (NOAA)'], + ['radar-global', 'Global (RainViewer)'], + ].map(([value, label]) => ({ + id: value, + label, + active: product === value, + params: { product: value }, + title: + value === 'radar' + ? 'NOAA MRMS, contiguous US, about 4-minute updates' + : 'RainViewer global composite from national radar networks, past 2 hours', + })) + : []), ...(satellite ? [ ['clouds-regional', 'N. America'], @@ -618,7 +651,9 @@ export function createWeatherLayer({ { id: 'coverage', label: radar - ? 'View US radar' + ? product === 'radar-global' + ? 'View coverage' + : 'View US radar' : lightning ? 'View Americas & Pacific' : 'View coverage', @@ -628,7 +663,12 @@ export function createWeatherLayer({ ], legend: radar || lightning - ? (lightning ? LIGHTNING_STOPS : STOPS).map(([label, color]) => ({ + ? (lightning + ? LIGHTNING_STOPS + : product === 'radar-global' + ? RAINVIEWER_STOPS + : STOPS + ).map(([label, color]) => ({ label: String(label), color, blurb: lightning @@ -638,14 +678,26 @@ export function createWeatherLayer({ : [], info: hostHidden ? hostStatus - : `${radar ? 'RADAR REFLECTIVITY · dBZ' : lightning ? 'LIGHTNING DENSITY · 15 min accumulation' : product === 'clouds' ? 'GLOBAL INFRARED · hourly' : 'GOES INFRARED · ~5 min'}\n${time ? `${followLatest ? 'Latest observation' : 'History'}: ${utc(time)}\n${lag}${current && !followLatest ? ` · frame ${index + 1}/${times.length}` : ''}${loading ? ' · loading' : ''}` : `Observation: unavailable${loading ? ' · loading' : ''}`}${missing ? `\n${missing}` : ''}${manifest?.stale ? '\nSTALE · cached source metadata' : ''}${error || diagnostic?.error ? '\n' + (error || diagnostic.error) : ''}\n${radar ? 'Contiguous US · gaps ≠ no rain' : lightning ? 'Americas + Pacific · not individual strikes\nColor: strikes/km²/min ×10³' : product === 'clouds' ? '60°S–60°N · typically 2–3 h delayed' : 'North America · infrared imagery'}${outside ? '\nMap center is outside source coverage' : ''}${motion?.matches ? (clock ? '\nReduced motion · history playback unavailable' : '\nReduced motion · manual history available') : ''}`, - infoTitle: lightning - ? 'NOAA/NWS 15-minute lightning density derived from Vaisala NLDN/GLD360. Coverage 110°E across the Pacific/Americas to 0°, 25°S–80°N. Not a live strike count, global coverage or a safety warning.' - : radar - ? 'NOAA MRMS radar echoes indicate precipitation patterns, not rain rate, a storm warning or a future forecast. Native source approximately 1 km; display is limited to level 6. Frames use exact advertised observation times.' - : 'GOES-19/18 longwave infrared Band 14 regional; NESDIS global longwave mosaic. Clouds only dims everything but bright, cold cloud tops; a brightness filter, not a cloud mask. Coverage and freshness differ by region.', + : `${radar ? 'RADAR REFLECTIVITY · dBZ' : lightning ? 'LIGHTNING DENSITY · 15 min accumulation' : product === 'clouds' ? 'GLOBAL INFRARED · hourly' : 'GOES INFRARED · ~5 min'}\n${time ? `${followLatest ? 'Latest observation' : 'History'}: ${utc(time)}\n${lag}${current && !followLatest ? ` · frame ${index + 1}/${times.length}` : ''}${loading ? ' · loading' : ''}` : `Observation: unavailable${loading ? ' · loading' : ''}`}${missing ? `\n${missing}` : ''}${manifest?.stale ? '\nSTALE · cached source metadata' : ''}${error || diagnostic?.error ? '\n' + (error || diagnostic.error) : ''}\n${radar ? (product === 'radar-global' ? 'Global · where radars exist · gaps ≠ no rain' : 'Contiguous US · gaps ≠ no rain') : lightning ? 'Americas + Pacific · not individual strikes\nColor: strikes/km²/min ×10³' : product === 'clouds' ? '60°S–60°N · typically 2–3 h delayed' : 'North America · infrared imagery'}${outside ? '\nMap center is outside source coverage' : ''}${motion?.matches ? (clock ? '\nReduced motion · history playback unavailable' : '\nReduced motion · manual history available') : ''}`, + infoTitle: + product === 'radar-global' + ? 'RainViewer global radar composite (dBZ). Coverage follows national radar networks; a gap does not mean no rain.' + : lightning + ? 'NOAA/NWS 15-minute lightning density derived from Vaisala NLDN/GLD360. Coverage 110°E across the Pacific/Americas to 0°, 25°S–80°N. Not a live strike count, global coverage or a safety warning.' + : radar + ? 'NOAA MRMS radar echoes indicate precipitation patterns, not rain rate, a storm warning or a future forecast. Native source approximately 1 km; display is limited to level 6. Frames use exact advertised observation times.' + : 'GOES-19/18 longwave infrared Band 14 regional; NESDIS global longwave mosaic. Clouds only dims everything but bright, cold cloud tops; a brightness filter, not a cloud mask. Coverage and freshness differ by region.', }; controls.summary.settings = [ + ...(radar + ? [ + { + id: 'region', + label: 'REGION', + chips: controls.chips.filter(({ params }) => params.product), + }, + ] + : []), ...(satellite ? [ { @@ -682,7 +734,7 @@ export function createWeatherLayer({ loading, error: error || rendering?.getDiagnostics().error || null, stale: Boolean(manifest?.stale || observationDelayed()), - source: 'NOAA nowCOAST', + source: product === 'radar-global' ? 'RainViewer' : 'NOAA nowCOAST', observedAt: shownTime(), }; }, diff --git a/src/layers/weather/infraredImage.js b/src/layers/weather/infraredImage.js index ccbb8e991..f7c2c50cb 100644 --- a/src/layers/weather/infraredImage.js +++ b/src/layers/weather/infraredImage.js @@ -1,5 +1,6 @@ import { readResponseBytesCapped } from '../../sources/httpBody.js'; import { weatherImageUrl } from './source.js'; +import { composeMercatorImage } from './mercatorMosaic.js'; import { infraredAlpha } from './infraredAlpha.js'; export const MAX_MOSAIC_BYTES = 4 * 1024 * 1024; @@ -76,6 +77,20 @@ export async function acquireWeatherImage( onFetched = () => {}, }, ) { + if (product === 'radar-global') { + const result = await composeMercatorImage({ + product, + time, + bbox, + size, + fetchImpl, + createCanvas, + decodeImage, + signal, + }); + onFetched(); + return result; + } const response = await fetchImpl(weatherImageUrl(product, time, size, bbox), { signal, }); diff --git a/src/layers/weather/mercatorMosaic.js b/src/layers/weather/mercatorMosaic.js new file mode 100644 index 000000000..2349b9931 --- /dev/null +++ b/src/layers/weather/mercatorMosaic.js @@ -0,0 +1,184 @@ +import { readResponseBytesCapped } from '../../sources/httpBody.js'; +import { weatherTileUrl, WEATHER_IMAGE_SIZES } from './source.js'; +import { decodeInfraredImage } from './infraredImage.js'; + +export const MERCATOR_LATITUDE = 85.0511; +export const RAINVIEWER_BOUNDS = Object.freeze({ + west: -180, + south: -MERCATOR_LATITUDE, + east: 180, + north: MERCATOR_LATITUDE, +}); + +/** Clamp a non-wrapping geographic window to the Mercator domain. */ +export function mercatorBbox(bbox = RAINVIEWER_BOUNDS) { + const box = { ...bbox }; + if ( + !['west', 'south', 'east', 'north'].every((key) => + Number.isFinite(box[key]), + ) || + box.west < -180 || + box.east > 180 || + box.west >= box.east + ) + throw new Error('Invalid Mercator window'); + box.south = Math.max(-MERCATOR_LATITUDE, box.south); + box.north = Math.min(MERCATOR_LATITUDE, box.north); + if (box.south >= box.north) throw new Error('Invalid Mercator window'); + return box; +} + +/** Normalized Web Mercator row, increasing southward. */ +export function latitudeToMercatorRow(latitude) { + const radians = + (Math.max(-MERCATOR_LATITUDE, Math.min(MERCATOR_LATITUDE, latitude)) * + Math.PI) / + 180; + return (1 - Math.asinh(Math.tan(radians)) / Math.PI) / 2; +} + +/** Latitude of a normalized Web Mercator row. */ +export function mercatorRowToLatitude(row) { + return (Math.atan(Math.sinh(Math.PI * (1 - 2 * row))) * 180) / Math.PI; +} + +/** Inclusive XYZ tile range; an exact east/south edge does not add a tile. */ +export function mercatorTileRange(bbox, z) { + const b = mercatorBbox(bbox); + const n = 2 ** z; + const edge = (value) => Math.max(0, Math.min(n - 1, value)); + const west = edge(Math.floor(((b.west + 180) / 360) * n)); + const east = edge(Math.ceil(((b.east + 180) / 360) * n) - 1); + const north = edge(Math.floor(latitudeToMercatorRow(b.north) * n)); + const south = edge(Math.ceil(latitudeToMercatorRow(b.south) * n) - 1); + return { + west, + east, + north, + south, + count: (east - west + 1) * (south - north + 1), + }; +} + +/** Highest permitted zoom that fits both the tile and horizontal pixel budgets. */ +export function chooseMercatorZoom(bbox, size, maxTiles = 24) { + const b = mercatorBbox(bbox); + for (let z = 7; z >= 0; z--) { + if ( + mercatorTileRange(b, z).count <= maxTiles && + ((b.east - b.west) / 360) * 256 * 2 ** z <= 2 * size.width + ) + return z; + } + throw new Error('Mercator image budget too small'); +} + +/** Source pixel row at the centre of an equirectangular output row. */ +export function mercatorSourceRow(row, height, bbox, z, tileNorth) { + const latitude = + bbox.north - ((row + 0.5) / height) * (bbox.north - bbox.south); + return latitudeToMercatorRow(latitude) * 256 * 2 ** z - tileNorth * 256; +} + +/** Compose bounded same-origin XYZ tiles and reproject with one strip per row. */ +export async function composeMercatorImage({ + product = 'radar-global', + time, + bbox = RAINVIEWER_BOUNDS, + size = WEATHER_IMAGE_SIZES['radar-global'], + fetchImpl = (...args) => globalThis.fetch(...args), + createCanvas = () => document.createElement('canvas'), + decodeImage = decodeInfraredImage, + signal = new AbortController().signal, + maxTiles = 24, + concurrency = 6, +}) { + signal.throwIfAborted(); + if ( + product !== 'radar-global' || + !Number.isInteger(concurrency) || + concurrency < 1 || + !Number.isInteger(maxTiles) || + maxTiles < 1 || + !Number.isInteger(size.width) || + !Number.isInteger(size.height) || + size.width < 1 || + size.height < 1 || + size.width > 4096 || + size.height > 2048 + ) + throw new Error('Invalid Mercator image options'); + const box = mercatorBbox(bbox ?? RAINVIEWER_BOUNDS); + const z = chooseMercatorZoom(box, size, maxTiles); + const range = mercatorTileRange(box, z); + const mosaic = createCanvas(); + mosaic.width = (range.east - range.west + 1) * 256; + mosaic.height = (range.south - range.north + 1) * 256; + const context = mosaic.getContext('2d'); + const template = weatherTileUrl(product, time); + const tiles = []; + for (let y = range.north; y <= range.south; y++) + for (let x = range.west; x <= range.east; x++) tiles.push({ x, y }); + let next = 0, + loaded = 0, + decodeMs = 0; + async function worker() { + while (next < tiles.length && !signal.aborted) { + const { x, y } = tiles[next++]; + let image; + try { + const url = template + .replace('{z}', z) + .replace('{x}', x) + .replace('{y}', y); + const response = await fetchImpl(url, { signal }); + if (!response.ok) throw new Error('Weather tile unavailable'); + const bytes = await readResponseBytesCapped(response, 1024 * 1024); + signal.throwIfAborted(); + const started = performance.now(); + image = await decodeImage( + new Blob([bytes], { type: 'image/png' }), + signal, + ); + signal.throwIfAborted(); + if (image.width !== 256 || image.height !== 256) + throw new Error('Invalid weather tile dimensions'); + context.drawImage( + image, + (x - range.west) * 256, + (y - range.north) * 256, + ); + decodeMs += performance.now() - started; + loaded++; + } catch { + // Missing tiles stay transparent; cancellation is checked after all workers settle. + } finally { + image?.close?.(); + } + } + } + await Promise.all( + Array.from({ length: Math.min(concurrency, tiles.length) }, worker), + ); + signal.throwIfAborted(); + if (!loaded) throw new Error('Weather tiles unavailable'); + const texture = createCanvas(); + texture.width = size.width; + texture.height = size.height; + const output = texture.getContext('2d'); + const scale = 256 * 2 ** z; + const sx = ((box.west + 180) / 360) * scale - range.west * 256; + const sw = ((box.east - box.west) / 360) * scale; + const started = performance.now(); + for (let row = 0; row < size.height; row++) { + const sy = Math.max( + 0, + Math.min( + mosaic.height - 1, + mercatorSourceRow(row, size.height, box, z, range.north) - 0.5, + ), + ); + output.drawImage(mosaic, sx, sy, sw, 1, 0, row, size.width, 1); + } + return { texture, decodeMs: decodeMs + performance.now() - started }; +} diff --git a/src/layers/weather/mercatorMosaic.test.mjs b/src/layers/weather/mercatorMosaic.test.mjs new file mode 100644 index 000000000..a81665239 --- /dev/null +++ b/src/layers/weather/mercatorMosaic.test.mjs @@ -0,0 +1,220 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + composeMercatorImage, + mercatorBbox, + mercatorTileRange, + chooseMercatorZoom, + latitudeToMercatorRow, + mercatorRowToLatitude, + mercatorSourceRow, + RAINVIEWER_BOUNDS, +} from './mercatorMosaic.js'; +import { acquireWeatherImage } from './infraredImage.js'; +import { validateWeatherSnapshot } from './source.js'; + +const time = '2026-09-23T12:00:00.000Z'; +const near = (a, b) => assert.ok(Math.abs(a - b) < 1e-8, `${a} ≈ ${b}`); +function canvas() { + const value = { draws: [] }; + value.getContext = () => ({ drawImage: (...args) => value.draws.push(args) }); + return value; +} +const options = { + product: 'radar-global', + time, + createCanvas: canvas, + fetchImpl: async () => new Response(new Uint8Array([1])), + decodeImage: async () => ({ width: 256, height: 256 }), +}; + +test('Mercator maths preserve XYZ edges, latitude direction and zoom budgets', () => { + near(latitudeToMercatorRow(0), 0.5); + near(latitudeToMercatorRow(60), 0.2903996408605086); + near(latitudeToMercatorRow(-60), 0.7096003591394914); + for (const lat of [-85, -60, 0, 60, 85]) + near(mercatorRowToLatitude(latitudeToMercatorRow(lat)), lat); + assert.deepEqual( + mercatorBbox({ ...RAINVIEWER_BOUNDS, north: 90, south: -90 }), + RAINVIEWER_BOUNDS, + ); + assert.deepEqual(mercatorTileRange(RAINVIEWER_BOUNDS, 1), { + west: 0, + east: 1, + north: 0, + south: 1, + count: 4, + }); + assert.deepEqual( + mercatorTileRange({ west: -180, east: 0, north: 60, south: 0 }, 1), + { west: 0, east: 0, north: 0, south: 0, count: 1 }, + ); + assert.equal(chooseMercatorZoom(RAINVIEWER_BOUNDS, { width: 2048 }), 2); + assert.equal(chooseMercatorZoom(RAINVIEWER_BOUNDS, { width: 128 }), 0); + const detail = { west: 10, east: 11, south: 40, north: 41 }; + assert.equal(chooseMercatorZoom(detail, { width: 4096 }), 7); + assert.throws(() => mercatorBbox({ ...detail, east: -10 })); +}); + +test('2×2 fake tiles compose north-up and row strips reproject the equator and ±60°', async () => { + const urls = [], + images = []; + const bbox = { west: -180, east: 180, north: 60.5, south: -60.5 }; + const { texture } = await composeMercatorImage({ + ...options, + bbox, + size: { width: 512, height: 121 }, + maxTiles: 4, + fetchImpl: async (url) => { + urls.push(url); + return new Response('tile'); + }, + decodeImage: async () => { + const image = { + width: 256, + height: 256, + close() { + this.closed = true; + }, + }; + images.push(image); + return image; + }, + }); + assert.equal(urls.length, 4); + assert.ok( + urls.every( + (url) => + url.startsWith('/api/weather/tile?product=radar-global&') && + url.includes('&z=1&'), + ), + ); + const mosaic = texture.draws[0][0]; + assert.equal(mosaic.width, 512); + assert.equal(mosaic.height, 512); + assert.deepEqual( + mosaic.draws.map((draw) => draw.slice(1)).sort(), + [ + [0, 0], + [0, 256], + [256, 0], + [256, 256], + ].sort(), + ); + assert.equal(texture.draws.length, 121); + for (const [row, latitude] of [ + [0, 60], + [60, 0], + [120, -60], + ]) { + near( + mercatorSourceRow(row, 121, bbox, 1, 0), + latitudeToMercatorRow(latitude) * 512, + ); + const [, sx, sy, sw, sh, dx, dy, dw, dh] = texture.draws[row]; + near(sy, latitudeToMercatorRow(latitude) * 512 - 0.5); + assert.deepEqual([sx, sw, sh, dx, dy, dw, dh], [0, 512, 1, 0, row, 512, 1]); + } + assert.ok(images.every((image) => image.closed)); +}); + +test('mosaic limits concurrency, tolerates partial failure, releases decoded images and rejects total failure', async () => { + let active = 0, + peak = 0, + calls = 0, + closed = 0; + const result = await composeMercatorImage({ + ...options, + maxTiles: 4, + concurrency: 2, + fetchImpl: async () => { + active++; + peak = Math.max(peak, active); + const n = ++calls; + await new Promise((resolve) => setImmediate(resolve)); + active--; + return n === 1 ? new Response('', { status: 503 }) : new Response('tile'); + }, + decodeImage: async () => ({ + width: 256, + height: 256, + close() { + closed++; + }, + }), + }); + assert.equal(peak, 2); + assert.equal(closed, 3); + assert.equal(result.texture.draws[0][0].draws.length, 3); + await assert.rejects( + composeMercatorImage({ + ...options, + fetchImpl: async () => new Response('', { status: 429 }), + }), + /tiles unavailable/, + ); + await assert.rejects( + composeMercatorImage({ + ...options, + signal: AbortSignal.abort(), + fetchImpl: () => assert.fail('aborted'), + }), + { name: 'AbortError' }, + ); + const controller = new AbortController(); + closed = 0; + await assert.rejects( + composeMercatorImage({ + ...options, + signal: controller.signal, + concurrency: 1, + decodeImage: async () => { + controller.abort(); + return { + width: 256, + height: 256, + close() { + closed++; + }, + }; + }, + }), + { name: 'AbortError' }, + ); + assert.equal(closed, 1); +}); + +test('whole and detail RainViewer images use composition and strict source metadata', async () => { + for (const bbox of [null, { west: 10, east: 16, south: 40, north: 43 }]) { + const result = await acquireWeatherImage('radar-global', time, { + ...options, + bbox, + size: { width: 2048, height: 1024 }, + }); + assert.equal(result.texture.width, 2048); + assert.equal(result.texture.height, 1024); + } + const snapshot = { + schemaVersion: 1, + product: 'radar-global', + bounds: RAINVIEWER_BOUNDS, + times: [time], + latest: time, + tileSize: 256, + maxLevel: 7, + tilingScheme: 'web-mercator', + }; + assert.equal(validateWeatherSnapshot(snapshot, 'radar-global'), snapshot); + for (const patch of [ + { maxLevel: 6 }, + { maxLevel: 8 }, + { tilingScheme: 'geographic' }, + { bounds: { ...RAINVIEWER_BOUNDS, north: 86 } }, + ]) + assert.throws(() => + validateWeatherSnapshot({ ...snapshot, ...patch }, 'radar-global'), + ); + assert.throws(() => + validateWeatherSnapshot({ ...snapshot, product: 'radar' }, 'radar'), + ); +}); diff --git a/src/layers/weather/rendering.js b/src/layers/weather/rendering.js index 8d195f302..4464b94bd 100644 --- a/src/layers/weather/rendering.js +++ b/src/layers/weather/rendering.js @@ -88,7 +88,10 @@ function createGlobeRendering({ ? cesium.Rectangle.intersection(bounds, view) : bounds; if (!coverage) return []; - const scheme = new cesium.GeographicTilingScheme(); + const scheme = + snapshot.tilingScheme === 'web-mercator' + ? new cesium.WebMercatorTilingScheme() + : new cesium.GeographicTilingScheme(); const template = weatherTileUrl(snapshot.product, time, { size: profile(snapshot.product).tileSize, }); @@ -248,7 +251,7 @@ function createGlobeRendering({ priority: snapshot.product === 'lightning' ? 3 - : snapshot.product === 'radar' + : ['radar', 'radar-global'].includes(snapshot.product) ? 2 : 1, product: snapshot.product, @@ -324,21 +327,26 @@ function createGlobeRendering({ viewer.scene.requestRender(); return; } - const tilingScheme = new cesium.GeographicTilingScheme( - global - ? { - rectangle, - numberOfLevelZeroTilesX: 2, - numberOfLevelZeroTilesY: 1, - } - : undefined, - ); + const mercator = snapshot.tilingScheme === 'web-mercator'; + const tilingScheme = mercator + ? new cesium.WebMercatorTilingScheme() + : new cesium.GeographicTilingScheme( + global + ? { + rectangle, + numberOfLevelZeroTilesX: 2, + numberOfLevelZeroTilesY: 1, + } + : undefined, + ); const credit = new cesium.Credit( - snapshot.product === 'lightning' - ? 'NOAA/NWS lightning density · derived from Vaisala NLDN/GLD360' - : snapshot.product === 'radar' - ? 'NOAA nowCOAST · NWS/OAR MRMS' - : 'NOAA nowCOAST · NESDIS GOES / global satellite partners', + snapshot.product === 'radar-global' + ? 'RainViewer' + : snapshot.product === 'lightning' + ? 'NOAA/NWS lightning density · derived from Vaisala NLDN/GLD360' + : snapshot.product === 'radar' + ? 'NOAA nowCOAST · NWS/OAR MRMS' + : 'NOAA nowCOAST · NESDIS GOES / global satellite partners', false, ); const provider = global @@ -360,7 +368,9 @@ function createGlobeRendering({ rectangle, tileWidth: nextProfile.tileSize, tileHeight: nextProfile.tileSize, - maximumLevel: nextProfile.maximumLevel, + maximumLevel: mercator + ? Math.min(nextProfile.maximumLevel, 7) + : nextProfile.maximumLevel, enablePickFeatures: false, credit, }); diff --git a/src/layers/weather/shellRendering.js b/src/layers/weather/shellRendering.js index cc8b32035..75e6950cc 100644 --- a/src/layers/weather/shellRendering.js +++ b/src/layers/weather/shellRendering.js @@ -8,6 +8,7 @@ export const WEATHER_SHELL_HEIGHTS = Object.freeze({ clouds: 5_500, 'clouds-regional': 5_800, radar: 6_200, + 'radar-global': 6_200, lightning: 6_600, }); // Decoded canvases per renderer, full-extent and detail images alike: the shown diff --git a/src/layers/weather/shellRendering.test.mjs b/src/layers/weather/shellRendering.test.mjs index ad65dcef2..a408959d7 100644 --- a/src/layers/weather/shellRendering.test.mjs +++ b/src/layers/weather/shellRendering.test.mjs @@ -126,6 +126,7 @@ test('shell heights stack every product with lightning highest', () => { clouds: 5_500, 'clouds-regional': 5_800, radar: 6_200, + 'radar-global': 6_200, lightning: 6_600, }); assert.equal(WEATHER_SHELL_CACHE_BYTES, 128 * 1024 * 1024); @@ -1502,3 +1503,37 @@ test('global infrared keeps one full-extent image; detail images halve for small }); small.shell.clear(); }); + +test('RainViewer shell composes full and detail textures on one raised surface', async () => { + const view = camera(10, 40, 12, 41); + const h = harness({ + product: 'radar-global', + camera: view, + decodeImage: async () => ({ width: 256, height: 256, close() {} }), + }); + const global = snapshot('radar-global', { + bounds: { west: -180, south: -85.0511, east: 180, north: 85.0511 }, + }); + const pending = h.shell.setFrame(global, times[0]); + for (let i = 0; i < 10; i++) { + await flush(); + h.render(); + } + assert.equal(await pending, true); + for (let i = 0; i < 10; i++) { + await flush(); + h.render(); + } + assert.equal(h.primitives().length, 1); + assert.equal(h.material().uniforms.image.width, 2048); + assert.equal(h.material().uniforms.detail.width, 4096); + assert.ok(h.fetches.length > 16 && h.fetches.length <= 40); + assert.ok( + h.fetches.every(({ url }) => + url.startsWith('/api/weather/tile?product=radar-global&'), + ), + ); + assert.equal(h.shell.getDiagnostics().shell.height, 6200); + h.shell.clear(); + assert.equal(h.primitives().length, 0); +}); diff --git a/src/layers/weather/source.js b/src/layers/weather/source.js index ec92cd5a2..ee9171952 100644 --- a/src/layers/weather/source.js +++ b/src/layers/weather/source.js @@ -2,6 +2,7 @@ import { readResponseJsonCapped } from '../../sources/httpBody.js'; export const WEATHER_PRODUCTS = Object.freeze([ 'radar', + 'radar-global', 'clouds', 'clouds-regional', 'lightning', @@ -20,8 +21,8 @@ export function validateWeatherSnapshot(value, product) { ) || bounds.west < -180 || bounds.east > 180 || - bounds.south < -90 || - bounds.north > 90 || + bounds.south < (product === 'radar-global' ? -85.0511 : -90) || + bounds.north > (product === 'radar-global' ? 85.0511 : 90) || bounds.west >= bounds.east || bounds.south >= bounds.north || !Array.isArray(times) || @@ -37,8 +38,9 @@ export function validateWeatherSnapshot(value, product) { ) || value.latest !== times.at(-1) || value.tileSize !== 256 || - value.maxLevel !== 6 || - value.tilingScheme !== 'geographic' + value.maxLevel !== (product === 'radar-global' ? 7 : 6) || + value.tilingScheme !== + (product === 'radar-global' ? 'web-mercator' : 'geographic') ) throw new Error('Malformed weather manifest'); return value; @@ -46,6 +48,7 @@ export function validateWeatherSnapshot(value, product) { /** Largest whole-extent image per product; also the proxy default size. */ export const WEATHER_IMAGE_SIZES = Object.freeze({ + 'radar-global': Object.freeze({ width: 2048, height: 1024 }), radar: Object.freeze({ width: 4096, height: 2048 }), 'clouds-regional': Object.freeze({ width: 4096, height: 2048 }), clouds: Object.freeze({ width: 2048, height: 1024 }), @@ -94,7 +97,10 @@ export function weatherImageUrl( export function weatherTileUrl(product, time, { size } = {}) { if (!WEATHER_PRODUCTS.includes(product) || !Number.isFinite(Date.parse(time))) throw new Error('Invalid weather frame'); - if (size !== undefined && ![256, 512, 1024].includes(size)) + if ( + size !== undefined && + !(product === 'radar-global' ? [256] : [256, 512, 1024]).includes(size) + ) throw new Error('Invalid weather tile size'); // Construct locally; never accept a manifest-provided host or template. return `/api/weather/tile?product=${product}&time=${encodeURIComponent(time)}&z={z}&x={x}&y={y}${size === undefined ? '' : `&size=${size}`}`; diff --git a/src/layers/weather/weather.test.mjs b/src/layers/weather/weather.test.mjs index a80ad6560..53a852ed0 100644 --- a/src/layers/weather/weather.test.mjs +++ b/src/layers/weather/weather.test.mjs @@ -1742,7 +1742,7 @@ test('one row steps every registered observation; missing frames hide and latest }); assert.deepEqual( radar.layer.getParams(), - { opacity: 'strong' }, + { product: 'radar', opacity: 'strong' }, 'history is never serialized', ); satellite.layer.disable(); @@ -1889,7 +1889,11 @@ test('observed descriptors keep configuration only and label satellite clouds by assert.equal(controls.summary.sections, undefined); assert.deepEqual( controls.summary.settings.map(({ label }) => label), - id === 'weather-satellite' ? ['REGION', 'IMAGE', 'OPACITY'] : ['OPACITY'], + id === 'weather-satellite' + ? ['REGION', 'IMAGE', 'OPACITY'] + : id === 'weather-radar' + ? ['REGION', 'OPACITY'] + : ['OPACITY'], ); assert.deepEqual( controls.summary.settings.flatMap(({ chips }) => chips), @@ -2253,3 +2257,163 @@ for (const product of ['radar', 'clouds-regional', 'lightning']) { assert.equal(h.viewer.camera.moveEnd.size, 0); }); } + +test('global radar uses Web Mercator provider, linked credit and square-grid prefetch', async () => { + const urls = []; + const h = renderingHarness({ + fetchImpl: async (url) => { + urls.push(url); + return mockResponse(); + }, + }); + h.cesium.WebMercatorTilingScheme = Cesium.WebMercatorTilingScheme; + const globalRadar = { + ...snapshot, + product: 'radar-global', + tilingScheme: 'web-mercator', + maxLevel: 7, + bounds: { west: -180, south: -85.0511, east: 180, north: 85.0511 }, + }; + const pending = h.rendering.setFrame(globalRadar, times[0]); + const provider = h.providers[0].options; + assert.ok(provider.tilingScheme instanceof Cesium.WebMercatorTilingScheme); + assert.equal(provider.maximumLevel, 6); + assert.equal(provider.tileWidth, 256); + assert.match(provider.credit.html, /href="https:\/\/www.rainviewer.com"/); + h.settle(); + assert.equal(await pending, true); + assert.equal(await h.rendering.prefetch(globalRadar, times[1]), true); + const xyz = urls.map((url) => { + const params = new URL(url, 'https://example.test').searchParams; + return ['z', 'x', 'y'].map((key) => Number(params.get(key))); + }); + assert.deepEqual(xyz, [ + [0, 0, 0], + [1, 0, 0], + [1, 1, 0], + [1, 0, 1], + [1, 1, 1], + ]); + h.rendering.clear(); +}); + +test('radar region switch cancels old work, retains clock target and advertises RainViewer semantics', async () => { + const clock = createWeatherClock(); + const products = []; + const h = layerHarness({ + clock, + feed: { + getSnapshot: async ({ product }) => { + products.push(product); + return { + ...snapshot, + product, + bounds: + product === 'radar-global' + ? { west: -180, south: -85.0511, east: 180, north: 85.0511 } + : snapshot.bounds, + }; + }, + }, + }); + const initial = h.layer.update(); + await flush(); + const old = h.stages.at(-1); + h.layer.setParams({ product: 'radar-global' }); + await flush(); + assert.equal(old.signal.aborted, true); + assert.deepEqual(products, ['radar', 'radar-global']); + h.stages.at(-1).finish(); + await flush(); + await initial; + assert.equal(h.layer.getParams().product, 'radar-global'); + const controls = h.layer.getRowControls(); + assert.equal(controls.summary.coverage, 'Global · where radars exist'); + assert.equal(controls.summary.maxGapMinutes, 30); + assert.deepEqual( + controls.summary.settings[0].chips.map(({ label }) => label), + ['US (NOAA)', 'Global (RainViewer)'], + ); + assert.deepEqual( + controls.legend.map(({ color }) => color), + [ + '#cec087', + '#00a3e0', + '#005588', + '#ffee00', + '#ffaa00', + '#ff4400', + '#c10000', + '#ffaaff', + '#ffffff', + ], + ); + assert.match(controls.infoTitle, /gap does not mean no rain/); + assert.equal(h.layer.getStats().source, 'RainViewer'); + const history = clock.setTarget(times[1]); + await flush(); + h.stages.at(-1).finish(); + await history; + assert.equal(h.layer.getDiagnostics().time, times[1]); + h.layer.setParams({ product: 'radar' }); + await flush(); + h.stages.at(-1).finish(); + await flush(); + assert.equal(clock.getState().target, times[1]); + assert.equal(h.layer.getRowControls().summary.coverage, 'CONUS'); + h.layer.destroy(); + clock.destroy(); +}); + +test('global radar coverage status follows the active bounds over land and ocean', async () => { + let location = Cesium.Cartesian3.fromDegrees(0, 51); + let shown = null; + const latest = new Date().toISOString(); + const layer = createWeatherLayer({ + documentRef: null, + eventTarget: null, + matchMedia: null, + feed: { + getSnapshot: async ({ product }) => ({ + product, + times: [latest], + latest, + bounds: + product === 'radar-global' + ? { west: -180, south: -85.0511, east: 180, north: 85.0511 } + : snapshot.bounds, + }), + }, + createRendering: () => ({ + rehome() {}, + clear() { + shown = null; + }, + setAlpha() {}, + async setFrame(_snapshot, time) { + shown = time; + return true; + }, + getDiagnostics: () => ({ time: shown }), + }), + }); + layer.init({ + camera: { pickEllipsoid: () => location }, + scene: { + canvas: { clientWidth: 800, clientHeight: 600 }, + globe: { ellipsoid: Cesium.Ellipsoid.WGS84 }, + }, + }); + layer.enable(); + await layer.update(); + assert.equal( + layer.getRowControls().summary.status, + 'Map center outside coverage', + ); + layer.setParams({ product: 'radar-global' }); + await flush(); + assert.equal(layer.getRowControls().summary.status, null); + location = Cesium.Cartesian3.fromDegrees(-150, -30); + assert.equal(layer.getRowControls().summary.status, null); + layer.destroy(); +});