diff --git a/CHANGELOG.md b/CHANGELOG.md index b735c8745..3b8282ef0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -165,6 +165,8 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md ## [Unreleased] +- Add ECMWF IFS model selection to Wind (#464, thanks @beneduzi), with model-scoped forecast-step caches, cancellation of replaced requests, and separate issue/valid timestamps. + - Give application request services, terrain/floor caches and annotation lookup state explicit owners and cancellation; share them across controls, layers and voice. - Construct application layers from explicit sources, with standalone provider selection and catalog-owned aircraft classification; controls and voice queries use those instances. @@ -283,6 +285,12 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md Visible animation, detection membership, history storage and proxy requests are bounded. Share links carry Transit as token `j`. +- Add a keyless **Wind** layer from NOAA GFS 10 m wind (#459, thanks @beneduzi). The `/api/wind` proxy + byte-range fetches only the UGRD/VGRD GRIB2 messages from the public AWS bucket, + decodes them with ecCodes (WASM), and serves a compact Float32 U/V grid; the + client renders nullschool-style animated particles in a canvas overlay that + follows the Cesium camera and skips globe-occluded points. Forecast, not + observations. Requires Node ≥24 for the WASM decoder. - Add Ontario 511 as a keyless CCTV source pack, including Kitchener-area highway cameras, with server-registered still URLs and attribution. - CCTV Mesh adds Finland: Fintraffic road weather cameras, keyless, nationwide, 300 by default. Each camera view of a station is placed separately; ambient stills refresh on the source's 10-minute cadence (the active camera keeps the usual 10-second refresh). diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index b246a305d..5676c3bb0 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -15,6 +15,7 @@ How to read this: | Source | Used for | License / terms | Attribution | | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| **NOAA GFS (wind)** | Global 10 m wind field for the animated Wind layer | U.S. public domain (NOAA); keyless via NOAA Open Data on AWS | "NOAA Global Forecast System (GFS)" (courtesy; not an endorsement) | | **OpenStreetMap ALPR camera locations** (including DeFlock community mapping) | Optional mapped automatic license-plate-reader camera layer | [ODbL 1.0](https://opendatacommons.org/licenses/odbl/1-0/); commercial use permitted with applicable attribution and database share-alike obligations | [© OpenStreetMap contributors](https://www.openstreetmap.org/copyright); [DeFlock](https://deflock.org) community mapping | | **Google Map Tiles API** (Photorealistic 3D Tiles) + Places/Geocoding | The 3D globe, voice scene context, and on-demand nearby installation search | Google Maps Platform ToS (proprietary, your own key + billing) | "Google" / "Google Maps" logo — **shown in-app**, required | | **OpenSky Network** | Primary worldwide live-flight snapshot | Non-commercial research/education license | Schäfer et al., _"Bringing Up OpenSky"_, IPSN 2014 + opensky-network.org | @@ -153,6 +154,24 @@ transaction quota. Requires a free `FIRMS_MAP_KEY` (https://firms.modaps.eosdis.nasa.gov/api/map_key/); the layer is empty without it. The former bundled 2026-05-25 snapshot was removed 2026-07-16. +### ECMWF IFS wind + +Wind also offers ECMWF IFS 10 m forecasts from the keyless [ECMWF Open Data](https://www.ecmwf.int/en/forecasts/datasets/open-data) service. The proxy reads its JSON Lines inventory and fetches only 10u/10v GRIB messages, resampling the forecast to the display grid. Data is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The [ECMWF Terms of Use](https://apps.ecmwf.int/datasets/licences/general/) also apply. In-app attribution identifies this service as based on ECMWF data and products, links the CC BY 4.0 licence, identifies resampling and animation as modifications, and retains the required ECMWF liability disclaimer. Both model issue time and the selected forecast valid time appear in the layer row. These are model forecasts, not observations. + +### NOAA GFS wind + +The optional **Wind** layer animates the global 10 m wind field from NOAA's +Global Forecast System (GFS). The `/api/wind` server-side proxy selects the +latest available 0.25° cycle, reads its `.idx` inventory, byte-range fetches only +the `UGRD`/`VGRD` 10 m GRIB2 messages (≈1 MB each instead of the whole ~500 MB +file), decodes them with ecCodes (WASM), resamples to a compact grid (1° by +default), and serves a manifest plus a Float32 U/V payload. It is keyless, cached +per cycle for an hour, and is a **forecast, not an observation**: the particles +show model flow, not measured wind. Rendering is a 2D canvas particle overlay +advected by the field and projected through the Cesium camera; globe-occluded +particles are skipped. NOAA GFS data is U.S. public domain; the credit above is a +courtesy and does not imply endorsement. + ### Natural Earth physical regions (`natural_earth/`) Curated from the **Natural Earth 10m physical vectors** (https://www.naturalearthdata.com/ — diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 60e41fcbd..3eebcbb9d 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1,5 +1,7 @@ # God's Eye View Current State +Wind is an optional GFS 10 m forecast overlay. Source acquisition has deadline and body budgets, disconnect cancellation and one-minute failure backoff. The row distinguishes model issue time from forecast valid time. Animation starts only with a field, stops on disable/destruction, preserves trails between unchanged canvas dimensions and scales from 200 to 4,000 particles with viewport area. Application catalog construction owns each instance. + Search framing and annotations request semantic map features from an explicit source. The Overpass adapter owns bounded queries, member/tag decoding and request deadlines; callers retain candidate ranking, outline caching, deferred retries and scene placement. Empty, transient and throttled outcomes remain distinct. Traffic sources return road records and installation sources return mapped records with freshness/saturation metadata, so their layers no longer decode upstream elements. ALPR already normalizes its records in the source. Default providers, footprints, road directions, exact-viewport retries and source attribution are unchanged. Nepal media preloads survive repeated camera-flight updates, but Stop, event disable and replacement remove abandoned frames and revoke pending Facebook sessions. Fallback evidence-card clicks respect the shared drawing-tool pointer lease. @@ -2442,6 +2444,7 @@ its criteria cannot be silently ignored. | Dams ▰ | OpenInfraMap/OSM extract (bundled) | `src/data/localLayers.js` | — | static | | Submarine Cables ◠ | TeleGeography public map (bundled) | `src/data/telegeographySubmarineCables.js` | — | static | | FIRMS Active Fires ▲ | NASA FIRMS live (VIIRS ×3 NRT, trailing 24h) | `src/data/firmsHeatmap.js` | `/api/firms` (`FIRMS_MAP_KEY`) | 10 min (proxy TTL 30 min) | +| Wind 🌬 | NOAA GFS 10 m wind (keyless, 0.25°→1° grid; animated particles) | `src/data/wind.js` | `/api/wind` | 1 h (forecast cycle) | Directions is a keyless front end to the routing the voice agent already uses. Its row chips are the whole interface: DRIVE / WALK / BIKE pick the @@ -3825,3 +3828,7 @@ frame. hls.js is imported only when an HLS camera starts. Two live sessions may coexist with at most eight independent consumer leases each; no-access lease expiry is 15 seconds. Closing one consumer leaves other consumers running. Encrypted/fMP4/byte-range playlists fail closed. Performance and live agency reliability require recorded validation. + +## Wind forecast models + +Wind supports selectable GFS/IFS models with persisted model choice. Each fetch chooses the available cycle and forecast step nearest the current time; rows show both issue and valid UTC timestamps. Grid identities include model, cycle and forecast step. Per-model caches retain the last two issued grids for manifest/grid rollover, coalesce concurrent requests, retain last-good data on failures and retry no faster than once per minute. Model changes clear the old field immediately, cancel old acquisition and ignore superseded results. diff --git a/package-lock.json b/package-lock.json index f854563a4..d98116fb4 100644 --- a/package-lock.json +++ b/package-lock.json @@ -10,6 +10,7 @@ "license": "MIT", "dependencies": { "@mapbox/vector-tile": "^3.0.0", + "@meri-imperiumi/eccodes-wasm": "^2.48.2", "cesium": "^1.124.0", "egm96-universal": "^1.1.1", "hls.js": "^1.7.3", @@ -1077,6 +1078,24 @@ "pbf": "^5.0.0" } }, + "node_modules/@meri-imperiumi/eccodes-wasm": { + "version": "2.48.2", + "resolved": "https://registry.npmjs.org/@meri-imperiumi/eccodes-wasm/-/eccodes-wasm-2.48.2.tgz", + "integrity": "sha512-Z7rD3xInomPG7FS97ORD1UGKAdgeeRP9ofa9k4m8d+k2sDfDQeAjTzvs9UOndyaZovaSbEQeIoNX8gTmQNhiSA==", + "cpu": [ + "x64", + "arm64" + ], + "license": "Apache-2.0", + "os": [ + "darwin", + "linux", + "win32" + ], + "engines": { + "node": ">=24.0.0" + } + }, "node_modules/@puppeteer/browsers": { "version": "3.2.2", "resolved": "https://registry.npmjs.org/@puppeteer/browsers/-/browsers-3.2.2.tgz", diff --git a/package.json b/package.json index 79eb7a0aa..9a8bb2b99 100644 --- a/package.json +++ b/package.json @@ -48,6 +48,7 @@ }, "dependencies": { "@mapbox/vector-tile": "^3.0.0", + "@meri-imperiumi/eccodes-wasm": "^2.48.2", "cesium": "^1.124.0", "egm96-universal": "^1.1.1", "hls.js": "^1.7.3", diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index b69e8ab10..03e654264 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -1212,6 +1212,12 @@ "src/data/contactMatch.js", "src/data/contextStore.js", "src/data/dataCredits.js", + "src/data/wind.js", + "src/layers/wind/index.js", + "src/layers/wind/model.js", + "src/layers/wind/rendering.js", + "src/layers/wind/source.js", + "src/sources/windBody.js", "src/data/detection.js", "src/data/detectionCohort.js", "src/data/detectionDraw.js", @@ -1727,6 +1733,9 @@ "src/annotations/resolver.js", "src/app/catalog.js", "src/app/constructCatalog.js", + "src/layers/wind/index.js", + "src/layers/wind/model.js", + "src/layers/wind/rendering.js", "src/app/layers/aisLiveVessels.js", "src/app/layers/alprCameras.js", "src/app/layers/bikeshare.js", diff --git a/scripts/qa-wind-canvas.mjs b/scripts/qa-wind-canvas.mjs new file mode 100644 index 000000000..f29f60e3e --- /dev/null +++ b/scripts/qa-wind-canvas.mjs @@ -0,0 +1,44 @@ +/** Real-canvas wind regression. Run against this candidate's dev server. */ +import assert from 'node:assert/strict'; +import puppeteer from 'puppeteer'; +const browser = await puppeteer.launch({ executablePath: process.env.PUPPETEER_EXECUTABLE_PATH || await puppeteer.executablePath(), headless: false, args: ['--no-sandbox', '--disable-dev-shm-usage'] }); +try { + const page = await browser.newPage(); + await page.goto(`${process.env.QA_BASE_URL || 'http://localhost:4173'}/`, { waitUntil: 'domcontentloaded' }); + const result = await page.evaluate(async () => { + const { createWindRendering } = await import('/src/layers/wind/rendering.js'); + const raf = window.requestAnimationFrame; const caf = window.cancelAnimationFrame; + const callbacks = new Map(); let sequence = 0; + window.requestAnimationFrame = callback => { callbacks.set(++sequence, callback); return sequence; }; + window.cancelAnimationFrame = id => callbacks.delete(id); + const container = document.createElement('div'); + container.style.cssText = 'position:fixed;left:0;top:0;width:800px;height:600px;z-index:9999;background:#102030'; + document.body.appendChild(container); + const sceneCanvas = { clientWidth: 800, clientHeight: 600 }; + const viewer = { scene: { canvas: sceneCanvas, camera: {} }, isDestroyed: () => false }; + const cesium = { Cartesian3: { fromDegrees: (lon,lat) => ({ lon,lat }) }, SceneTransforms: { worldToWindowCoordinates: (scene, point) => ({ x: 400 + point.lon, y: 300 + point.lat }) } }; + const rendering = createWindRendering({ cesium, container, getViewer: () => viewer }); + try { + rendering.attach(); rendering.start(); + const idleCallbacks = callbacks.size; + rendering.setField({ grid: { nx:1,ny:1,lo1:0,la1:90,dx:360,dy:180 }, u: new Float32Array([10]), v: new Float32Array([0]) }); + const canvas = container.querySelector('canvas'); const context = canvas.getContext('2d'); + const tick = time => { const [id, callback] = callbacks.entries().next().value; callbacks.delete(id); callback(time); }; + tick(16); // First camera signature clears the surface. + context.fillStyle = '#ff0000'; context.fillRect(10,10,5,5); + tick(32); + const alpha = context.getImageData(12,12,1,1).data[3]; + const count = rendering.getParticleCount(); + sceneCanvas.clientWidth = 320; sceneCanvas.clientHeight = 240; + tick(48); + const resizedCount = rendering.getParticleCount(); + rendering.stop(); rendering.clear(); + return { idleCallbacks, alpha, count, resizedCount, pending: callbacks.size, afterClear: context.getImageData(12,12,1,1).data[3] }; + } finally { rendering.destroy(); container.remove(); window.requestAnimationFrame = raf; window.cancelAnimationFrame = caf; } + }); + assert.equal(result.idleCallbacks, 0); + assert.ok(result.alpha > 200, 'unchanged canvas dimensions preserve existing trail pixels'); + assert.ok(result.resizedCount < result.count); + assert.equal(result.pending, 0); assert.equal(result.afterClear, 0); + console.log(JSON.stringify(result)); +} finally { await browser.close(); } diff --git a/server/providers/local.js b/server/providers/local.js index 68f61a3cb..deb746b7a 100644 --- a/server/providers/local.js +++ b/server/providers/local.js @@ -20,6 +20,7 @@ import { trackBackfillProxies } from './aircraft/tracks.js'; import { openAiRealtimeProxy } from './openai.js'; import { googlePlacesContextProxy } from './places.js'; import { keySetupEndpoint } from '../standalone/key-setup.js'; +import { windProxy } from './wind.js'; /** Construct the local provider plugins in their established order. */ function localProviderPlugins() { @@ -45,6 +46,7 @@ function localProviderPlugins() { trackBackfillProxies(), openAiRealtimeProxy(), googlePlacesContextProxy(), + windProxy(), keySetupEndpoint(), ]; } diff --git a/server/providers/wind.js b/server/providers/wind.js new file mode 100644 index 000000000..42b5e37cb --- /dev/null +++ b/server/providers/wind.js @@ -0,0 +1,176 @@ +import { fetchGfsWind } from './wind/gfs.js'; +import { fetchIfsWind } from './wind/ifs.js'; +import { decodeWindGribMessage } from './wind/decode.js'; + +/** Serve bounded, single-flight, per-model forecast snapshots from fixed providers. */ +export function windProxy({ + fetchImpl = fetch, + now = () => Date.now(), + decodeImpl = decodeWindGribMessage, + targetDx = 1, + ttlMs = 3600_000, + timeoutMs = 40_000, + models = { gfs: fetchGfsWind, ifs: fetchIfsWind }, +} = {}) { + const caches = new Map(); + const loadings = new Map(); + const grids = new Map(); + const attempts = new Map(); + const unavailable = (model) => ({ + manifest: { + model, + schemaVersion: 1, + unavailable: true, + stale: true, + reason: 'Wind upstream unavailable', + }, + }); + const sendJson = (res, value, status = 200) => { + res.writeHead(status, { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store', + }); + res.end(JSON.stringify(value)); + }; + function refresh(model) { + const controller = new AbortController(); + const operation = { controller, waiters: 0, promise: null }; + loadings.set(model, operation); + attempts.set(model, now()); + operation.promise = (async () => { + const timer = setTimeout(() => controller.abort(), timeoutMs); + try { + const value = await models[model]({ + fetchImpl, + now, + decodeImpl, + targetDx, + signal: controller.signal, + }); + controller.signal.throwIfAborted(); + const { grid, cycle } = value; + if (!grid.u.every(Number.isFinite) || !grid.v.every(Number.isFinite)) + throw new Error('Invalid wind grid'); + const id = `${model}-${cycle.date}-${cycle.hour}-f${cycle.forecastHour || 0}-${targetDx}`; + const manifest = { + schemaVersion: 1, + model, + cycle, + fetchedAt: now(), + level: value.level, + units: value.units, + grid: { + nx: grid.nx, + ny: grid.ny, + lo1: grid.lo1, + la1: grid.la1, + dx: grid.dx, + dy: grid.dy, + }, + stale: false, + unavailable: false, + reason: null, + gridUrl: `/api/wind/grid/${id}.bin?model=${model}`, + }; + const state = { id, grid, manifest, fetchedAt: now() }; + caches.set(model, state); + // Retain the previous issued grid as well to cover a manifest/grid rollover. + const history = grids.get(model) || new Map(); + history.set(id, state); + while (history.size > 2) history.delete(history.keys().next().value); + grids.set(model, history); + return state; + } catch { + if (controller.signal.aborted && operation.waiters === 0) { + attempts.delete(model); + return unavailable(model); + } + const old = caches.get(model); + if (old) { + old.manifest = { + ...old.manifest, + stale: true, + reason: 'Wind upstream unavailable', + }; + return old; + } + return unavailable(model); + } finally { + clearTimeout(timer); + controller.abort(); // Cancel a sibling request if the other component failed. + if (loadings.get(model) === operation) loadings.delete(model); + } + })(); + return operation; + } + const handler = async (req, res) => { + const url = new URL(req.url, 'http://localhost'); + const model = url.searchParams.get('model') || 'gfs'; + if (req.method !== 'GET') + return sendJson(res, { error: 'method_not_allowed' }, 405); + if (!['gfs', 'ifs'].includes(model) || !Object.hasOwn(models, model)) + return sendJson(res, { error: 'unknown_model' }, 400); + if (url.pathname.startsWith('/grid/')) { + const id = url.pathname.slice(6).replace(/\.bin$/, ''); + const state = + url.pathname === `/grid/${id}.bin` && grids.get(model)?.get(id); + if (!state) return sendJson(res, { error: 'unknown_grid' }, 404); + const bytes = Buffer.concat([ + Buffer.from( + state.grid.u.buffer, + state.grid.u.byteOffset, + state.grid.u.byteLength, + ), + Buffer.from( + state.grid.v.buffer, + state.grid.v.byteOffset, + state.grid.v.byteLength, + ), + ]); + res.writeHead(200, { + 'Content-Type': 'application/octet-stream', + 'Cache-Control': 'public, max-age=3600, immutable', + }); + return res.end(bytes); + } + if (!['/', '/manifest', '/status'].includes(url.pathname)) + return sendJson(res, { error: 'not_found' }, 404); + let state = caches.get(model); + if (!state || now() - state.fetchedAt >= ttlMs) { + let operation = loadings.get(model); + if (!operation && now() - (attempts.get(model) ?? -Infinity) >= 60_000) + operation = refresh(model); + if (operation) { + let disconnected = false; + operation.waiters += 1; + const close = () => { + disconnected = true; + if (--operation.waiters === 0) operation.controller.abort(); + }; + res.once?.('close', close); + try { + state = await operation.promise; + } finally { + res.removeListener?.('close', close); + if (!disconnected) operation.waiters -= 1; + } + if (disconnected) return; + } + } + const manifest = state?.manifest || unavailable(model).manifest; + if (url.pathname === '/status') { + const { gridUrl, ...status } = manifest; + return sendJson(res, status); + } + return sendJson(res, manifest); + }; + return { + name: 'wind', + configureServer({ middlewares }) { + middlewares.use('/api/wind', handler); + }, + configurePreviewServer({ middlewares }) { + middlewares.use('/api/wind', handler); + }, + }; +} diff --git a/server/providers/wind/catalog.js b/server/providers/wind/catalog.js new file mode 100644 index 000000000..445953f7f --- /dev/null +++ b/server/providers/wind/catalog.js @@ -0,0 +1,34 @@ +export const GFS_BUCKET = 'noaa-gfs-bdp-pds'; + +/** Build the public GFS object key. */ +export function gfsObjectKey({ date, hour, forecastHour = 0 }) { + const hh = String(hour).padStart(2, '0'); + return `gfs.${date}/${hh}/atmos/gfs.t${hh}z.pgrb2.0p25.f${String(forecastHour).padStart(3, '0')}`; +} + +/** Select the latest GFS cycle that should be available. */ +export function selectLatestGfsCycle( + nowMs, + { availabilityLagMs = 5 * 3600_000 } = {}, +) { + const d = new Date(nowMs - availabilityLagMs); + const date = new Date( + Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()), + ); + const hour = Math.floor(d.getUTCHours() / 6) * 6; + date.setUTCHours(hour); + return { date: date.toISOString().slice(0, 10).replaceAll('-', ''), hour }; +} + +/** + * Round a time offset to the nearest published GFS forecast step so the field + * shown is the one valid closest to now, not always the analysis (f000). + * GFS 0.25° is hourly through f120, then every 3 h through f384. + * @param {number} hours - Hours elapsed since the cycle run time. + * @returns {number} A valid forecast step. + */ +export function nearestGfsStep(hours) { + const value = Math.max(0, Number.isFinite(hours) ? hours : 0); + const step = value <= 120 ? Math.round(value) : Math.round(value / 3) * 3; + return Math.max(0, Math.min(384, step)); +} diff --git a/server/providers/wind/decode.js b/server/providers/wind/decode.js new file mode 100644 index 000000000..dbc756404 --- /dev/null +++ b/server/providers/wind/decode.js @@ -0,0 +1,51 @@ +/** Lazy shared ecCodes WASM instance, created only for an active wind request. */ +let eccodesPromise = null; +function loadEccodes() { + eccodesPromise ??= import('@meri-imperiumi/eccodes-wasm') + .then((module) => module.createEccodes()) + .catch((error) => { + eccodesPromise = null; + throw error; + }); + return eccodesPromise; +} + +/** Decode one bounded message in WASM memory without mounting the host filesystem. */ +export async function decodeWindGribMessage(buffer, { eccodesModule } = {}) { + if (buffer.byteLength > 8 * 1024 * 1024) + throw new Error('Wind GRIB exceeds byte budget'); + const eccodes = eccodesModule ?? (await loadEccodes()); + // Everything after this await is synchronous. A shared virtual file is safe + // across U/V and model requests and its storage is released after each decode. + const file = '/gev-wind.grib2'; + let handle = null; + try { + eccodes.writeFile(file, buffer); + handle = eccodes.openGrib(file); + const ni = handle.getLong('Ni'); + const nj = handle.getLong('Nj'); + if ( + !Number.isInteger(ni) || + !Number.isInteger(nj) || + ni < 1 || + nj < 1 || + ni * nj > 2_000_000 + ) + throw new Error('Invalid wind GRIB geometry'); + return { + ni, + nj, + lo1: handle.getDouble('longitudeOfFirstGridPointInDegrees'), + la1: handle.getDouble('latitudeOfFirstGridPointInDegrees'), + di: handle.getDouble('iDirectionIncrementInDegrees'), + dj: handle.getDouble('jDirectionIncrementInDegrees'), + values: handle.getDoubleArray('values'), + shortName: handle.getString('shortName'), + level: handle.getLong('level'), + units: handle.getString('units'), + }; + } finally { + handle?.delete(); + eccodes.writeFile(file, new Uint8Array(0)); + } +} diff --git a/server/providers/wind/gfs.js b/server/providers/wind/gfs.js new file mode 100644 index 000000000..403728ba4 --- /dev/null +++ b/server/providers/wind/gfs.js @@ -0,0 +1,165 @@ +import { readWindBody } from '../../../src/sources/windBody.js'; +/** + * Parse a GFS `.idx` inventory into ordered message records. + * + * Each line is `msgNumber:byteOffset:d=YYYYMMDDHH:VAR:LEVEL:...:`. Malformed + * lines are skipped; a payload with no valid lines is rejected so callers never + * treat an HTML error page as an empty inventory. + * + * @param {string} text - Raw `.idx` body. + * @returns {{ messages: Array<{index: number, offset: number, variable: string, level: string}> }} + */ +export function parseGfsIdx(text) { + const messages = []; + for (const line of String(text).split(/\r?\n/)) { + if (!line) continue; + const parts = line.split(':'); + if ( + parts.length < 6 || + !/^\d+$/.test(parts[0]) || + !/^\d+$/.test(parts[1]) || + !parts[3] || + !parts[4] + ) + continue; + messages.push({ + index: Number(parts[0]), + offset: Number(parts[1]), + variable: parts[3], + level: parts[4], + }); + } + if (!messages.length) throw new Error('malformed GFS index'); + return { messages }; +} + +/** + * Byte ranges for the two 10 m wind components. A message runs from its own + * offset to one byte before the next message in file order. + * + * @param {{messages: Array}} parsed - {@link parseGfsIdx} result. + * @param {{level?: string}} [options] + * @returns {{u: {start: number, end: number}, v: {start: number, end: number}}} + */ +export function windMessageRanges( + parsed, + { level = '10 m above ground' } = {}, +) { + const range = (variable) => { + const index = parsed.messages.findIndex( + (message) => message.variable === variable && message.level === level, + ); + const message = parsed.messages[index]; + const next = parsed.messages[index + 1]; + if (!message || !next) throw new Error(`missing ${variable} at ${level}`); + return { start: message.offset, end: next.offset - 1 }; + }; + return { u: range('UGRD'), v: range('VGRD') }; +} + +/** Fetch a bounded text response as a Buffer. */ +export async function fetchText({ url, fetchImpl = fetch, signal }) { + const response = await fetchImpl(url, { signal, redirect: 'error' }); + if (!response.ok) { + await response.body?.cancel(); + throw new Error('Wind upstream unavailable'); + } + return Buffer.from(await readWindBody(response, 2 * 1024 * 1024, signal)); +} + +/** + * Fetch one bounded byte range. Rejects oversized ranges, non-206 responses + * (except a whole-object 200 starting at zero), and short bodies. + */ +export async function fetchRange({ + url, + start, + end, + fetchImpl = fetch, + signal, + maxBytes = 8 * 1024 * 1024, +}) { + if ( + !Number.isSafeInteger(start) || + !Number.isSafeInteger(end) || + start < 0 || + end < start || + end - start + 1 > maxBytes + ) + throw new Error('range too large'); + const response = await fetchImpl(url, { + signal, + redirect: 'error', + headers: { Range: `bytes=${start}-${end}` }, + }); + const validStatus = + response.status === 206 || (response.status === 200 && start === 0); + if (!validStatus || !response.ok) { + await response.body?.cancel(); + throw new Error('Wind upstream unavailable'); + } + const buffer = Buffer.from( + await readWindBody(response, end - start + 1, signal), + ); + if (buffer.length !== end - start + 1) + throw new Error('invalid range length'); + return buffer; +} + +import { + GFS_BUCKET, + gfsObjectKey, + selectLatestGfsCycle, + nearestGfsStep, +} from './catalog.js'; +import { decodeWindGribMessage } from './decode.js'; +import { resampleWindGrid } from './grid.js'; + +/** Fetch and decode the GFS 10 m wind field valid closest to now. */ +export async function fetchGfsWind({ + fetchImpl = fetch, + now = () => Date.now(), + targetDx = 1, + decodeImpl = decodeWindGribMessage, + signal, +} = {}) { + const nowMs = now(); + const cycle = selectLatestGfsCycle(nowMs); + const runMs = Date.UTC( + Number(cycle.date.slice(0, 4)), + Number(cycle.date.slice(4, 6)) - 1, + Number(cycle.date.slice(6, 8)), + cycle.hour, + ); + // Pick the forecast step whose valid time is closest to now, so the layer + // shows the freshest field the cycle offers instead of the analysis. + const forecastHour = nearestGfsStep((nowMs - runMs) / 3600_000); + const base = `https://${GFS_BUCKET}.s3.amazonaws.com/${gfsObjectKey({ ...cycle, forecastHour })}`; + const index = await fetchText({ url: `${base}.idx`, fetchImpl, signal }); + const ranges = windMessageRanges(parseGfsIdx(index.toString())); + const [uBuffer, vBuffer] = await Promise.all([ + fetchRange({ url: base, ...ranges.u, fetchImpl, signal }), + fetchRange({ url: base, ...ranges.v, fetchImpl, signal }), + ]); + const [u, v] = await Promise.all([decodeImpl(uBuffer), decodeImpl(vBuffer)]); + const grid = resampleWindGrid({ + u: u.values, + v: v.values, + ni: u.ni, + nj: u.nj, + lo1: u.lo1, + la1: u.la1, + di: u.di, + dj: u.dj, + dx: targetDx, + dy: targetDx, + }); + const runIso = new Date(runMs).toISOString(); + const validIso = new Date(runMs + forecastHour * 3600_000).toISOString(); + return { + cycle: { ...cycle, forecastHour, runIso, validIso }, + level: '10 m above ground', + units: 'm/s', + grid, + }; +} diff --git a/server/providers/wind/grid.js b/server/providers/wind/grid.js new file mode 100644 index 000000000..c610fee17 --- /dev/null +++ b/server/providers/wind/grid.js @@ -0,0 +1,68 @@ +/** + * Resample a GRIB wind grid into a compact equirectangular grid. + * + * Source grids are north-to-south (`la1` at the top, `dj` increasing toward the + * south) and start at `lo1`. Longitude wraps across the seam; latitude clamps. + * + * @param {{u: ArrayLike, v: ArrayLike, ni: number, nj: number, + * lo1: number, la1: number, di: number, dj: number, dx?: number, dy?: number}} input + * @returns {{nx: number, ny: number, lo1: number, la1: number, dx: number, + * dy: number, u: Float32Array, v: Float32Array}} + */ +export function resampleWindGrid({ + u, + v, + ni, + nj, + lo1, + la1, + di, + dj, + dx = 1, + dy = 1, +}) { + if ( + !Number.isInteger(ni) || + !Number.isInteger(nj) || + ni < 1 || + nj < 1 || + ni * nj > 2_000_000 || + u?.length !== ni * nj || + v?.length !== ni * nj || + ![lo1, la1, di, dj, dx, dy].every(Number.isFinite) || + di <= 0 || + dj <= 0 || + dx < 0.25 || + dy < 0.25 || + dx > 180 || + dy > 180 + ) + throw new Error('Invalid wind grid geometry'); + const nx = Math.round(360 / dx); + const ny = Math.round(180 / dy) + 1; + const outU = new Float32Array(nx * ny); + const outV = new Float32Array(nx * ny); + const sample = (source, lon, lat) => { + const x = ((lon - lo1) / di + ni) % ni; + const y = Math.max(0, Math.min(nj - 1, (la1 - lat) / dj)); + const x0 = Math.floor(x); + const x1 = (x0 + 1) % ni; + const y0 = Math.floor(y); + const y1 = Math.min(nj - 1, y0 + 1); + const fx = x - x0; + const fy = y - y0; + const top = source[y0 * ni + x0] * (1 - fx) + source[y0 * ni + x1] * fx; + const bottom = source[y1 * ni + x0] * (1 - fx) + source[y1 * ni + x1] * fx; + return top * (1 - fy) + bottom * fy; + }; + for (let y = 0; y < ny; y += 1) { + for (let x = 0; x < nx; x += 1) { + const index = y * nx + x; + const lon = x * dx; + const lat = 90 - y * dy; + outU[index] = sample(u, lon, lat); + outV[index] = sample(v, lon, lat); + } + } + return { nx, ny, lo1: 0, la1: 90, dx, dy, u: outU, v: outV }; +} diff --git a/server/providers/wind/ifs.js b/server/providers/wind/ifs.js new file mode 100644 index 000000000..648dba43a --- /dev/null +++ b/server/providers/wind/ifs.js @@ -0,0 +1,148 @@ +import { fetchText, fetchRange } from './gfs.js'; +import { decodeWindGribMessage } from './decode.js'; +import { resampleWindGrid } from './grid.js'; + +/** Keyless ECMWF Open Data root for real-time IFS forecasts. */ +export const IFS_BASE = 'https://data.ecmwf.int/forecasts'; + +/** + * Select the latest IFS cycle that should be available. + * @param {number} nowMs - Reference epoch milliseconds. + * @param {{availabilityLagMs?: number}} [options] + * @returns {{date: string, hour: number}} + */ +export function selectLatestIfsCycle( + nowMs, + { availabilityLagMs = 6 * 3600_000 } = {}, +) { + const shifted = new Date(nowMs - availabilityLagMs); + return { + date: shifted.toISOString().slice(0, 10).replaceAll('-', ''), + hour: Math.floor(shifted.getUTCHours() / 6) * 6, + }; +} + +/** + * Build the IFS GRIB2 and `.index` URLs for one cycle and step. + * @param {{date: string, hour: number, step?: number}} cycle + * @returns {{grib: string, index: string}} + */ +export function ifsObjectUrls({ date, hour, step = 0 }) { + const hh = String(hour).padStart(2, '0'); + const stem = `${IFS_BASE}/${date}/${hh}z/ifs/0p25/oper/${date}${hh}0000-${step}h-oper-fc`; + // The inventory sits beside the GRIB2 as `.index`, NOT `.index`. + return { grib: `${stem}.grib2`, index: `${stem}.index` }; +} + +/** + * Parse the IFS JSON Lines inventory into `{param, offset, length}` records. + * Malformed lines are skipped; a payload with no valid lines is rejected. + * @param {string} text - Raw `.index` body. + * @returns {Array<{param: string, offset: number, length: number}>} + */ +export function parseIfsIndex(text) { + const entries = []; + for (const line of String(text).split(/\r?\n/)) { + if (!line) continue; + let value; + try { + value = JSON.parse(line); + } catch { + continue; // non-JSON line in the index + } + if ( + value?.param && + Number.isFinite(value._offset) && + Number.isFinite(value._length) + ) + entries.push({ + param: value.param, + offset: value._offset, + length: value._length, + }); + } + if (!entries.length) throw new Error('malformed IFS index'); + return entries; +} + +/** + * Byte ranges for the IFS 10 m wind components. + * @param {Array<{param: string, offset: number, length: number}>} entries + * @returns {{u: {start: number, end: number}, v: {start: number, end: number}}} + */ +export function ifsWindRanges(entries) { + const range = (param) => { + const item = entries.find((entry) => entry.param === param); + if (!item) throw new Error(`missing ${param}`); + return { start: item.offset, end: item.offset + item.length - 1 }; + }; + return { u: range('10u'), v: range('10v') }; +} + +/** + * Round a time offset to the nearest published IFS forecast step. IFS 0.25° + * oper is 3-hourly through f144, then 6-hourly to f360 for 00/12z. + * @param {number} hours - Hours elapsed since the cycle run time. + * @param {number} cycleHour - The cycle hour (0, 6, 12 or 18). + * @returns {number} A valid forecast step. + */ +export function nearestIfsStep(hours, cycleHour) { + const max = cycleHour === 6 || cycleHour === 18 ? 144 : 360; + const value = Math.max(0, Number.isFinite(hours) ? hours : 0); + const step = + value <= 144 ? Math.round(value / 3) * 3 : Math.round(value / 6) * 6; + return Math.max(0, Math.min(max, step)); +} + +/** + * Fetch and decode the IFS 10 m wind field valid closest to now. + * @param {{fetchImpl?: Function, now?: Function, targetDx?: number, + * decodeImpl?: Function}} [options] + * @returns {Promise<{cycle: object, level: string, units: string, grid: object}>} + */ +export async function fetchIfsWind({ + fetchImpl = fetch, + now = () => Date.now(), + targetDx = 1, + decodeImpl = decodeWindGribMessage, + signal, +} = {}) { + const nowMs = now(); + const cycle = selectLatestIfsCycle(nowMs); + const runMs = Date.UTC( + Number(cycle.date.slice(0, 4)), + Number(cycle.date.slice(4, 6)) - 1, + Number(cycle.date.slice(6, 8)), + cycle.hour, + ); + // Show the field valid closest to now rather than always the analysis. + const forecastHour = nearestIfsStep((nowMs - runMs) / 3600_000, cycle.hour); + const urls = ifsObjectUrls({ ...cycle, step: forecastHour }); + const index = await fetchText({ url: urls.index, fetchImpl, signal }); + const ranges = ifsWindRanges(parseIfsIndex(index.toString())); + const [uBuffer, vBuffer] = await Promise.all([ + fetchRange({ url: urls.grib, ...ranges.u, fetchImpl, signal }), + fetchRange({ url: urls.grib, ...ranges.v, fetchImpl, signal }), + ]); + const [u, v] = await Promise.all([decodeImpl(uBuffer), decodeImpl(vBuffer)]); + const grid = resampleWindGrid({ + u: u.values, + v: v.values, + ni: u.ni, + nj: u.nj, + lo1: u.lo1, + la1: u.la1, + di: u.di, + dj: u.dj, + dx: targetDx, + dy: targetDx, + }); + const runIso = new Date(runMs).toISOString(); + const validIso = new Date(runMs + forecastHour * 3600_000).toISOString(); + return { + cycle: { ...cycle, forecastHour, runIso, validIso }, + level: '10 m above ground', + units: 'm/s', + grid, + }; +} diff --git a/src/app/constructCatalog.js b/src/app/constructCatalog.js index 47ffe5658..fc1e4a4d4 100644 --- a/src/app/constructCatalog.js +++ b/src/app/constructCatalog.js @@ -1,3 +1,4 @@ +import { createWindLayer } from '../layers/wind/index.js'; import { createLayerCatalog } from './catalog.js'; import { LAYER_STATE_REGISTRY } from '../data/layerState.js'; import { createMilitaryRegistry } from '../layers/aircraft/classification.js'; @@ -42,6 +43,7 @@ const SOURCE_METHODS = Object.freeze({ launches: ['getLaunches', 'getActiveTle'], alpr: ['fetch'], firms: ['getSnapshot'], + wind: ['getSnapshot'], earthquakes: ['getSnapshot'], cables: ['fetch'], }); @@ -128,6 +130,7 @@ export function createApplicationCatalog({ vessels, installations, }), + createWindLayer({ feed: sources.wind }), ...createInfrastructureLayers(localGeoJsonServices), createApplicationCables({ source: sources.cables }), createApplicationFirms({ diff --git a/src/app/constructCatalog.test.mjs b/src/app/constructCatalog.test.mjs index 9de176605..bbf45acc9 100644 --- a/src/app/constructCatalog.test.mjs +++ b/src/app/constructCatalog.test.mjs @@ -39,7 +39,7 @@ test('catalogs construct distinct layers and classification from their supplied signal: b.signal, surface: fixtureSurface(b.signal), }); - assert.equal(first.layers.length, 21); + assert.equal(first.layers.length, 22); assert.ok(first.get('transit')); assert.ok(first.get('bhote-koshi-2026')); assert.ok(first.get('bhote-koshi-locator')); diff --git a/src/app/sources.js b/src/app/sources.js index ced314e6a..a1e7d1eac 100644 --- a/src/app/sources.js +++ b/src/app/sources.js @@ -10,6 +10,7 @@ import { configureInstallationSource } from '../data/militaryInstallations.js'; import { configureSatelliteSource } from '../data/satellites.js'; import { configureLaunchSource } from '../data/rocketLaunches.js'; import { configureFirmsSource } from '../data/firmsHeatmap.js'; +import { configureWindSource } from '../data/wind.js'; import { configureMilitaryRegistrySource } from '../data/militaryRegistry.js'; const configure = { alpr: configureAlprSource, @@ -21,6 +22,7 @@ const configure = { satellites: configureSatelliteSource, launches: configureLaunchSource, firms: configureFirmsSource, + wind: configureWindSource, }; /** Configure sources before any registration or state restoration starts. */ export function configureApplicationSources({ diff --git a/src/data/dataCredits.js b/src/data/dataCredits.js index d228e29c4..3505dee3c 100644 --- a/src/data/dataCredits.js +++ b/src/data/dataCredits.js @@ -205,6 +205,22 @@ export const DATA_CREDITS = [ 'Re:Earth Terrain / ' + 'Mapterhorn (CC BY 4.0) / EGM2008 (NGA)', }, + { + key: 'wind-gfs', + html: + 'Wind: resampled NOAA Global Forecast System (GFS) 10 m wind via ' + + 'NOAA Open Data on AWS ' + + '(U.S. public domain; forecast, not observations)', + }, + { + key: 'ecmwf-ifs', + html: + 'This service is based on data and products of the European Centre for Medium-Range Weather Forecasts (ECMWF). ' + + 'ECMWF Open Data · ' + + 'CC BY 4.0. ' + + 'Modified: resampled 10 m IFS forecast vectors and animated display. ' + + 'ECMWF does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.', + }, // ── Bundled snapshots ─────────────────────────────────────────── { key: 'datacenters', diff --git a/src/data/layerState.js b/src/data/layerState.js index 68eebd777..5575a12db 100644 --- a/src/data/layerState.js +++ b/src/data/layerState.js @@ -205,6 +205,9 @@ function integerOption(key, token, defaultValue) { } const OPTION_GROUPS = Object.freeze({ + wind: Object.freeze([ + enumOption('model', 'm', 'gfs', ['gfs', 'ifs'], { gfs: 'g', ifs: 'i' }), + ]), flights: Object.freeze([ // Owner directive 2026-08-22: the fleet's 3D models are DEFAULT-ON in // PROXIMITY mode. Proximity is itself the altitude/count gate — models only @@ -384,6 +387,12 @@ export const LAYER_STATE_REGISTRY = Object.freeze([ }), Object.freeze({ id: 'traffic', token: 't', disposition: 'enabled-only' }), Object.freeze({ id: 'transit', token: 'j', disposition: 'enabled-only' }), + Object.freeze({ + id: 'wind', + token: 'k', + disposition: 'enabled+options', + optionOwner: 'wind', + }), ]); export const REGISTERED_LAYER_IDS = Object.freeze( diff --git a/src/data/layerState.test.mjs b/src/data/layerState.test.mjs index 577027069..1a3c62e82 100644 --- a/src/data/layerState.test.mjs +++ b/src/data/layerState.test.mjs @@ -158,8 +158,8 @@ function encode(state) { test('production registry is exact, canonical, and rejects incomplete contracts', async () => { assert.equal(validateLayerStateRegistry(), true); - assert.equal(REGISTERED_LAYER_IDS.length, 21); - assert.equal(new Set(REGISTERED_LAYER_IDS).size, 21); + assert.equal(REGISTERED_LAYER_IDS.length, 22); + assert.equal(new Set(REGISTERED_LAYER_IDS).size, 22); assert.ok(REGISTERED_LAYER_IDS.includes('transit')); assert.deepEqual(REGISTERED_LAYER_IDS, [...REGISTERED_LAYER_IDS].sort()); assert.throws( diff --git a/src/data/wind.js b/src/data/wind.js new file mode 100644 index 000000000..38bc10688 --- /dev/null +++ b/src/data/wind.js @@ -0,0 +1,17 @@ +import { createSourceSlot } from '../sources/sourceSlot.js'; +import { createWindLayer as createLayer } from '../layers/wind/index.js'; +import { createWindSource } from '../layers/wind/source.js'; + +const sourceSlot = createSourceSlot( + createWindSource(), + ['getSnapshot'], + 'Wind source', +); +export const configureWindSource = sourceSlot.configure; + +/** Create a wind layer using the configured application source by default. */ +export function createWindLayer(options = {}) { + return createLayer({ ...options, feed: options.feed ?? sourceSlot.source }); +} + +export default createWindLayer(); diff --git a/src/data/windDecode.test.mjs b/src/data/windDecode.test.mjs new file mode 100644 index 000000000..173ff97eb --- /dev/null +++ b/src/data/windDecode.test.mjs @@ -0,0 +1,9 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { decodeWindGribMessage } from '../../server/providers/wind/decode.js'; +test('decoder uses bounded virtual storage and releases both handle and message on errors', async () => { + const writes = []; let deleted = false; + const eccodesModule = { writeFile: (file, bytes) => writes.push(bytes.length), openGrib: () => ({ getLong: () => 1e9, delete: () => { deleted = true; } }) }; + await assert.rejects(decodeWindGribMessage(new Uint8Array(20), { eccodesModule }), /geometry/); + assert.deepEqual(writes, [20,0]); assert.equal(deleted, true); +}); diff --git a/src/data/windGfs.test.mjs b/src/data/windGfs.test.mjs new file mode 100644 index 000000000..e5387f115 --- /dev/null +++ b/src/data/windGfs.test.mjs @@ -0,0 +1,21 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { gfsObjectKey, selectLatestGfsCycle } from '../../server/providers/wind/catalog.js'; +import { parseGfsIdx, windMessageRanges, fetchRange } from '../../server/providers/wind/gfs.js'; +test('catalog and index helpers', async () => { + assert.equal(gfsObjectKey({ date: '20260914', hour: 6, forecastHour: 3 }), 'gfs.20260914/06/atmos/gfs.t06z.pgrb2.0p25.f003'); + assert.deepEqual(selectLatestGfsCycle(Date.UTC(2026, 8, 15, 4)), { date: '20260914', hour: 18 }); + const p = parseGfsIdx('1:10:d=x:UGRD:10 m above ground:anl:\n2:20:d=x:VGRD:10 m above ground:anl:\n3:30:d=x:X:y:z'); + assert.deepEqual(windMessageRanges(p), { u: { start: 10, end: 19 }, v: { start: 20, end: 29 } }); + const b = await fetchRange({ url: 'x', start: 0, end: 2, fetchImpl: async () => new Response(new Uint8Array([1, 2, 3]), { status: 200 }) }); + assert.deepEqual([...b], [1, 2, 3]); + assert.throws(() => windMessageRanges({ messages: p.messages.slice(0, 1) })); +}); + +test('wind range fetch rejects invalid offsets and caps chunked bodies', async () => { + await assert.rejects(fetchRange({ url: 'x', start: -1, end: 2 }), /range too large/); + let cancelled = false; + const body = new ReadableStream({ pull(controller) { controller.enqueue(new Uint8Array(10)); }, cancel() { cancelled = true; } }); + await assert.rejects(fetchRange({ url: 'x', start: 0, end: 2, fetchImpl: async () => new Response(body, { status: 206 }) }), /budget/); + assert.equal(cancelled, true); +}); diff --git a/src/data/windGrid.test.mjs b/src/data/windGrid.test.mjs new file mode 100644 index 000000000..0a57f501b --- /dev/null +++ b/src/data/windGrid.test.mjs @@ -0,0 +1,9 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { resampleWindGrid } from '../../server/providers/wind/grid.js'; + +test('resamples and wraps a wind grid', () => { + const r = resampleWindGrid({ ni: 4, nj: 3, lo1: 0, la1: 90, di: 90, dj: 90, u: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], v: new Array(12).fill(1), dx: 90, dy: 90 }); + assert.deepEqual([r.nx, r.ny], [4, 3]); + assert.equal(r.u[0], 0); assert.equal(r.u[3], 3); assert.equal(r.u[4], 4); +}); diff --git a/src/data/windIfs.test.mjs b/src/data/windIfs.test.mjs new file mode 100644 index 000000000..25c21ba38 --- /dev/null +++ b/src/data/windIfs.test.mjs @@ -0,0 +1,12 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { ifsObjectUrls, parseIfsIndex, ifsWindRanges, nearestIfsStep, selectLatestIfsCycle } from '../../server/providers/wind/ifs.js'; +import { nearestGfsStep } from '../../server/providers/wind/catalog.js'; +test('IFS inventory selects only 10 m components and distinct index suffix', () => { + const urls = ifsObjectUrls({ date: '20260915', hour: 6, step: 9 }); + assert.match(urls.index, /9h-oper-fc\.index$/); + assert.equal(urls.index.includes('.grib2.index'), false); + assert.deepEqual(ifsWindRanges(parseIfsIndex('{"param":"10u","_offset":20,"_length":10}\n{"param":"10v","_offset":40,"_length":15}')), { u: {start:20,end:29}, v:{start:40,end:54} }); + assert.deepEqual(selectLatestIfsCycle(Date.UTC(2026,8,15,4)), { date:'20260914', hour:18 }); + assert.equal(nearestIfsStep(8,6),9); assert.equal(nearestIfsStep(200,6),144); assert.equal(nearestIfsStep(151,0),150); assert.equal(nearestGfsStep(8.2),8); +}); diff --git a/src/data/windProxy.test.mjs b/src/data/windProxy.test.mjs new file mode 100644 index 000000000..d60d2a33b --- /dev/null +++ b/src/data/windProxy.test.mjs @@ -0,0 +1,253 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; + +import { windProxy } from '../../server/providers/wind.js'; + +/** Boot a mounted provider middleware without a real server. */ +function install(plugin) { + const routes = new Map(); + plugin.configureServer({ + middlewares: { + use(route, handler) { + routes.set(route, handler); + }, + }, + }); + assert.equal(routes.size, 1); + const handler = [...routes.values()][0]; + return async (url = '/', method = 'GET') => { + const res = { + statusCode: 200, + headers: {}, + headersSent: false, + writeHead(status, headers) { + this.statusCode = status; + for (const [name, value] of Object.entries(headers || {})) + this.headers[name.toLowerCase()] = value; + this.headersSent = true; + }, + end(body) { + this.body = body; + this.headersSent = true; + }, + }; + await handler({ url, method }, res); + return res; + }; +} + +const IDX = [ + '1:0:d=2026091400:UGRD:10 m above ground:anl:', + '2:5:d=2026091400:VGRD:10 m above ground:anl:', + '3:10:d=2026091400:HGT:surface:anl:', +].join('\n'); + +const DECODED_U = { + ni: 4, + nj: 3, + lo1: 0, + la1: 90, + di: 90, + dj: 90, + values: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], +}; +const DECODED_V = { ...DECODED_U, values: [21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32] }; + +/** Resampled-grid metadata shared by the injected model fakes. */ +const GRID = { nx: 4, ny: 3, lo1: 0, la1: 90, dx: 90, dy: 90 }; + +/** Fetch double: `.idx` text, and range bodies of the requested length. */ +function makeFetch({ fail = false, counter } = {}) { + return async (url, options = {}) => { + if (counter) counter.calls += 1; + if (fail) throw new Error('gfs upstream down'); + if (String(url).endsWith('.idx')) return new Response(IDX, { status: 200 }); + const match = /bytes=(\d+)-(\d+)/.exec(options.headers?.Range || ''); + const length = Number(match[2]) - Number(match[1]) + 1; + return new Response(new Uint8Array(length), { status: 206 }); + }; +} + +function proxy(options = {}) { + return windProxy({ + now: () => Date.UTC(2026, 8, 14, 12), + targetDx: 90, + fetchImpl: makeFetch(), + decodeImpl: async (buffer) => + buffer[0] === undefined ? DECODED_U : DECODED_U, + ...options, + }); +} + +test('wind manifest describes the GFS cycle and resampled grid', async () => { + const request = install(proxy()); + const res = await request('/'); + assert.equal(res.statusCode, 200); + const body = JSON.parse(res.body); + assert.equal(body.schemaVersion, 1); + assert.equal(body.model, 'gfs'); + assert.equal(body.cycle.hour, 6); + assert.equal(body.cycle.date, '20260914'); + assert.equal(body.units, 'm/s'); + assert.deepEqual(body.grid, { nx: 4, ny: 3, lo1: 0, la1: 90, dx: 90, dy: 90 }); + assert.match(body.gridUrl, /^\/api\/wind\/grid\/gfs-20260914-6-f6-90\.bin\?model=gfs$/); + assert.equal(body.stale, false); +}); + +test('wind grid route returns Float32 U then V', async () => { + let u = 0; + const request = install( + proxy({ + decodeImpl: async () => { + u += 1; + return u === 1 ? DECODED_U : DECODED_V; + }, + }), + ); + const manifest = JSON.parse((await request('/')).body); + const res = await request(manifest.gridUrl.replace('/api/wind', '')); + assert.equal(res.statusCode, 200); + assert.equal(res.headers['content-type'], 'application/octet-stream'); + assert.equal(res.body.length, 4 * 3 * 4 * 2); + const floats = new Float32Array( + res.body.buffer, + res.body.byteOffset, + res.body.length / 4, + ); + assert.deepEqual([...floats.slice(0, 12)], DECODED_U.values); + assert.deepEqual([...floats.slice(12)], DECODED_V.values); +}); + +test('wind status omits the grid url', async () => { + const request = install(proxy()); + await request('/'); + const body = JSON.parse((await request('/status')).body); + assert.equal('gridUrl' in body, false); + assert.equal(body.model, 'gfs'); +}); + +test('wind refresh is cached within the TTL and refetches after it', async () => { + const counter = { calls: 0 }; + let clock = Date.UTC(2026, 8, 14, 12); + const request = install( + proxy({ now: () => clock, fetchImpl: makeFetch({ counter }) }), + ); + await request('/'); + const afterFirst = counter.calls; + assert.equal(afterFirst, 3, 'one idx plus two range fetches'); + await request('/'); + assert.equal(counter.calls, afterFirst, 'a second request inside the TTL must not refetch'); + clock += 2 * 3600_000; + await request('/'); + assert.equal(counter.calls, afterFirst + 3, 'a request past the TTL refetches'); +}); + +test('wind serves last-good with stale on upstream failure', async () => { + let failing = false; + let clock = Date.UTC(2026, 8, 14, 12); + const request = install( + proxy({ + now: () => clock, + fetchImpl: async (...args) => { + if (failing) throw new Error('gfs upstream down'); + return makeFetch()(...args); + }, + }), + ); + const good = JSON.parse((await request('/')).body); + failing = true; + clock += 2 * 3600_000; + const degraded = JSON.parse((await request('/')).body); + assert.equal(degraded.stale, true); + assert.equal(degraded.reason, 'Wind upstream unavailable'); + const frame = await request(good.gridUrl.replace('/api/wind', '')); + assert.equal(frame.statusCode, 200, 'last-good grid stays addressable'); +}); + +test('wind returns a JSON 404 for an unknown grid', async () => { + const request = install(proxy()); + await request('/'); + const res = await request('/grid/nope.bin'); + assert.equal(res.statusCode, 404); + assert.deepEqual(JSON.parse(res.body), { error: 'unknown_grid' }); +}); + +test('wind grid ids and URLs are model-scoped', async () => { + const request = install( + proxy({ + models: { + gfs: async () => ({ cycle: { date: '20260914', hour: 6 }, level: 'x', units: 'm/s', grid: { ...GRID, u: new Float32Array(12), v: new Float32Array(12) } }), + ifs: async () => ({ cycle: { date: '20260914', hour: 0 }, level: 'x', units: 'm/s', grid: { ...GRID, u: new Float32Array(12), v: new Float32Array(12) } }), + }, + }), + ); + const gfs = JSON.parse((await request('/')).body); + const ifs = JSON.parse((await request('/?model=ifs')).body); + assert.match(gfs.gridUrl, /gfs-20260914-6-f0-90\.bin\?model=gfs$/); + assert.match(ifs.gridUrl, /ifs-20260914-0-f0-90\.bin\?model=ifs$/); + // Each model's grid resolves through its own model parameter. + assert.equal((await request(gfs.gridUrl.replace('/api/wind', ''))).statusCode, 200); + assert.equal((await request(ifs.gridUrl.replace('/api/wind', ''))).statusCode, 200); +}); + +test('wind rejects a non-finite grid instead of caching it', async () => { + const bad = new Float32Array(12); + bad[3] = Number.POSITIVE_INFINITY; + const request = install( + proxy({ + models: { + gfs: async () => ({ cycle: { date: '20260914', hour: 6 }, level: 'x', units: 'm/s', grid: { ...GRID, u: bad, v: new Float32Array(12) } }), + }, + }), + ); + const body = JSON.parse((await request('/')).body); + assert.equal(body.unavailable, true); + assert.equal(body.reason, 'Wind upstream unavailable'); +}); + +test('wind rejects unknown models, paths and methods before acquisition', async () => { + let calls = 0; + const request = install(proxy({ models: { gfs: async () => { calls++; throw new Error(); } } })); + assert.equal((await request('/?model=toString')).statusCode, 400); + assert.equal((await request('/secret')).statusCode, 404); + assert.equal((await request('/manifest', 'POST')).statusCode, 405); + assert.equal((await request('/grid/unknown.bin')).statusCode, 404); + assert.equal(calls, 0); +}); + +test('wind caches forecast steps separately and keeps a prior issued grid readable', async () => { + let clock = Date.UTC(2026, 8, 14, 12); let step = 6; + const request = install(proxy({ now: () => clock, models: { gfs: async () => ({ cycle: { date: '20260914', hour: 6, forecastHour: step }, level:'10 m', units:'m/s', grid: { ...GRID, u: new Float32Array(12).fill(step), v: new Float32Array(12) } }) } })); + const first = JSON.parse((await request('/')).body); + clock += 3600_000; step = 7; + const second = JSON.parse((await request('/')).body); + assert.notEqual(first.gridUrl, second.gridUrl); + assert.equal((await request(first.gridUrl.replace('/api/wind',''))).statusCode, 200); +}); + +test('wind failure backoff prevents unbounded repeated upstream acquisition', async () => { + let calls = 0; + const request = install(proxy({ models: { gfs: async () => { calls++; throw new Error('private upstream detail'); } } })); + const results = await Promise.all([request('/'), request('/')]); + assert.equal(calls, 1); + assert.equal(JSON.parse(results[0].body).reason, 'Wind upstream unavailable'); + await request('/'); assert.equal(calls, 1); +}); + +test('shared upstream work survives one disconnect and aborts when the final client leaves', async () => { + const { EventEmitter } = await import('node:events'); + let handler; let signal; let loads = 0; + const plugin = proxy({ models: { gfs: ({ signal: next }) => { + loads++; signal = next; + return new Promise((resolve, reject) => next.addEventListener('abort', () => reject(next.reason), { once: true })); + } } }); + plugin.configureServer({ middlewares: { use: (_path, value) => { handler = value; } } }); + const first = new EventEmitter(); const second = new EventEmitter(); + const a = handler({ url: '/manifest', method: 'GET' }, first); + const b = handler({ url: '/manifest', method: 'GET' }, second); + assert.equal(loads, 1); + first.emit('close'); assert.equal(signal.aborted, false); + second.emit('close'); assert.equal(signal.aborted, true); + await Promise.all([a,b]); + assert.equal(first.listenerCount('close'), 0); assert.equal(second.listenerCount('close'), 0); +}); diff --git a/src/layers/wind/index.js b/src/layers/wind/index.js new file mode 100644 index 000000000..588e171dc --- /dev/null +++ b/src/layers/wind/index.js @@ -0,0 +1,182 @@ +import * as Cesium from 'cesium'; +import { createWindRendering } from './rendering.js'; + +/** Format a forecast timestamp explicitly in UTC. */ +export function formatWindValidTime(value) { + const ms = Date.parse(value); + return Number.isFinite(ms) + ? `${new Date(ms).toISOString().slice(0, 16).replace('T', ' ')} UTC` + : null; +} + +/** Forecast issue time is source age; valid time is reported separately. */ +export function windStats(manifest) { + const run = Date.parse(manifest?.cycle?.runIso); + return { + count: manifest?.grid ? manifest.grid.nx * manifest.grid.ny : 0, + lastUpdate: Number.isFinite(run) ? run : null, + error: + manifest?.reason || (manifest?.unavailable ? 'Wind unavailable' : null), + }; +} + +/** Construct one wind layer with an explicit source and owned animation. */ +export function createWindLayer({ + feed, + cesium = Cesium, + container, + createRendering = createWindRendering, +} = {}) { + if (typeof feed?.getSnapshot !== 'function') + throw new TypeError('Wind requires a snapshot source'); + let viewer = null; + let request = null; + let enabled = false; + let rendering = null; + let manifest = null; + let error = null; + let loading = false; + let model = 'gfs'; + let generation = 0; + let rowControlsListener = null; + const notify = () => rowControlsListener?.(); + const layer = { + id: 'wind', + name: 'Wind', + icon: '🌬', + source: 'GFS / ECMWF IFS · FORECAST', + updateInterval: 3600_000, + init(nextViewer) { + viewer = nextViewer; + rendering = createRendering({ + cesium, + container: container ?? nextViewer.container, + getViewer: () => viewer, + }); + rendering.attach(); + }, + enable() { + enabled = true; + rendering?.start(); + }, + disable() { + enabled = false; + generation += 1; + request?.abort(); + request = null; + loading = false; + rendering?.stop(); + rendering?.clear(); + }, + async update(nextViewer, { signal } = {}) { + if (!enabled) return false; + request?.abort(); + const controller = new AbortController(); + const abort = () => controller.abort(signal.reason); + signal?.addEventListener('abort', abort, { once: true }); + request = controller; + generation += 1; + loading = true; + notify(); + try { + signal?.throwIfAborted(); + const snapshot = await feed.getSnapshot({ + signal: controller.signal, + model, + }); + if (!enabled || controller.signal.aborted || request !== controller) + return false; + manifest = snapshot; + error = null; + if (snapshot.unavailable) { + rendering.stop(); + rendering.clear(); + } else { + rendering.setField(snapshot); + rendering.start(); + } + return true; + } catch (cause) { + if (controller.signal.aborted || request !== controller) return false; + error = cause?.message || 'Wind source unavailable'; + return true; + } finally { + signal?.removeEventListener('abort', abort); + if (request === controller) { + request = null; + loading = false; + notify(); + } + } + }, + setParams(params = {}) { + if (!['gfs', 'ifs'].includes(params.model) || params.model === model) + return; + model = params.model; + request?.abort(); + request = null; + manifest = null; + error = null; + loading = enabled; + rendering?.stop(); + rendering?.clear(); + const current = ++generation; + notify(); + if (enabled) + queueMicrotask(() => { + if (enabled && generation === current) void layer.update(viewer); + }); + }, + getParams() { + return { model }; + }, + getRowControls() { + const valid = formatWindValidTime(manifest?.cycle?.validIso); + const run = formatWindValidTime(manifest?.cycle?.runIso); + return { + chips: ['gfs', 'ifs'].map((value) => ({ + id: `model-${value}`, + label: value.toUpperCase(), + active: model === value, + params: { model: value }, + title: + value === 'ifs' + ? 'ECMWF IFS 10 m forecast' + : 'NOAA GFS 10 m forecast', + })), + legend: [ + { label: '0', color: '#1e3a8a' }, + { label: '10', color: '#22d3ee' }, + { label: '20', color: '#fbbf24' }, + { label: '30+ m/s', color: '#ef4444' }, + ], + info: `${model === 'ifs' ? 'ECMWF IFS' : 'GFS'} forecast · Valid: ${valid || 'Unavailable'} · Issued: ${run || 'Unavailable'}${manifest?.stale ? ' · STALE' : ''}`, + }; + }, + setRowControlsListener(listener) { + rowControlsListener = typeof listener === 'function' ? listener : null; + }, + destroy() { + layer.disable(); + rendering?.destroy(); + rendering = null; + viewer = null; + rowControlsListener = null; + }, + getStats() { + return { + ...windStats(manifest), + loading, + model: model.toUpperCase(), + source: model === 'ifs' ? 'ECMWF IFS' : 'NOAA GFS', + validTime: + formatWindValidTime(manifest?.cycle?.validIso) || 'Unavailable', + error: error || windStats(manifest).error, + }; + }, + getParticleCount() { + return rendering?.getParticleCount() || 0; + }, + }; + return layer; +} diff --git a/src/layers/wind/index.test.mjs b/src/layers/wind/index.test.mjs new file mode 100644 index 000000000..4b6ece1c6 --- /dev/null +++ b/src/layers/wind/index.test.mjs @@ -0,0 +1,51 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createWindLayer, formatWindValidTime, windStats } from './index.js'; + +const snapshot = (model) => ({ model, grid: { nx: 2, ny: 2 }, cycle: { runIso: '2026-09-14T12:00:00Z', validIso: '2026-09-14T18:00:00Z' } }); +function harness(feed) { + const calls = []; + const rendering = Object.fromEntries(['attach','start','stop','clear','destroy','setField'].map(name => [name, (...args) => calls.push([name, ...args])])); + const layer = createWindLayer({ feed, createRendering: () => rendering }); + layer.init({ container: {} }); layer.enable(); + return { layer, calls }; +} +test('forecast valid time and source age stay distinct', () => { + assert.equal(formatWindValidTime('invalid'), null); + assert.equal(formatWindValidTime('2026-01-05T06:30:00Z'), '2026-01-05 06:30 UTC'); + assert.equal(windStats(snapshot('gfs')).lastUpdate, Date.parse('2026-09-14T12:00:00Z')); +}); +test('switching models clears old data and ignores an abort-insensitive late source', async () => { + const pending = []; + const { layer, calls } = harness({ getSnapshot: args => new Promise(resolve => pending.push({ ...args, resolve })) }); + const first = layer.update(); + layer.setParams({ model: 'ifs' }); + await Promise.resolve(); + assert.equal(pending.length, 2); + assert.equal(pending[0].signal.aborted, true); + assert.equal(layer.getStats().count, 0); + assert.equal(layer.getStats().model, 'IFS'); + pending[0].resolve(snapshot('gfs')); + await first; + assert.equal(calls.filter(([name]) => name === 'setField').length, 0); + pending[1].resolve(snapshot('ifs')); + await Promise.resolve(); await Promise.resolve(); + assert.equal(calls.find(([name]) => name === 'setField')[1].model, 'ifs'); + assert.match(layer.getRowControls().info, /IFS forecast.*Valid: 2026-09-14 18:00 UTC.*Issued: 2026-09-14 12:00 UTC/); + layer.destroy(); +}); +test('external and owned cancellation both cancel source work; queued switches stop on disable', async () => { + let signal; + const { layer } = harness({ getSnapshot: args => { signal = args.signal; return new Promise(resolve => signal.addEventListener('abort', () => resolve(snapshot('gfs')), { once: true })); } }); + const external = new AbortController(); + const update = layer.update(null, { signal: external.signal }); + layer.disable(); + assert.equal(signal.aborted, true); assert.equal(external.signal.aborted, false); + assert.equal(await update, false); + layer.enable(); + const other = layer.update(null, { signal: external.signal }); + external.abort(); assert.equal(signal.aborted, true); await other; + layer.setParams({ model: 'ifs' }); layer.disable(); + await Promise.resolve(); assert.equal(layer.getStats().loading, false); + layer.destroy(); +}); diff --git a/src/layers/wind/model.js b/src/layers/wind/model.js new file mode 100644 index 000000000..efff055bd --- /dev/null +++ b/src/layers/wind/model.js @@ -0,0 +1,78 @@ +/** Normalize a longitude into the half-open world interval. */ +export function normalizeLongitude(lon) { + return ((((lon + 180) % 360) + 360) % 360) - 180; +} + +/** Sample a north-to-south, seam-wrapped wind grid bilinearly. */ +export function sampleWind(field, lon, lat) { + if (!Number.isFinite(lon) || !Number.isFinite(lat)) return { u: 0, v: 0 }; + const { u, v, nx, ny, lo1, la1, dx, dy } = field; + const x = ((normalizeLongitude(lon - lo1) + 360) % 360) / dx; + const y = (la1 - lat) / dy; + const x0 = Math.floor(x); + const tx = x - x0; + const yClamped = Math.max(0, Math.min(ny - 1, y)); + const y0 = Math.floor(yClamped); + const y1 = Math.min(ny - 1, y0 + 1); + const ty = yClamped - y0; + const column = (index) => ((index % nx) + nx) % nx; + const value = (array) => { + const a = array[y0 * nx + column(x0)]; + const b = array[y0 * nx + column(x0 + 1)]; + const c = array[y1 * nx + column(x0)]; + const d = array[y1 * nx + column(x0 + 1)]; + return (a * (1 - tx) + b * tx) * (1 - ty) + (c * (1 - tx) + d * tx) * ty; + }; + return { u: value(u), v: value(v) }; +} + +/** Return metres represented by one longitude degree at a latitude. */ +export function metersPerDegreeLon(lat) { + return 111320 * Math.cos((lat * Math.PI) / 180); +} + +/** Calculate wind magnitude. */ +export function windSpeed(u, v) { + return Math.hypot(u, v); +} + +/** Map wind magnitude to a fixed CSS colour ramp (calm blue → strong red). */ +export function windColor(speed, { max = 30 } = {}) { + const stops = [ + '#1e3a8a', + '#2563eb', + '#22d3ee', + '#34d399', + '#fbbf24', + '#f97316', + '#ef4444', + ]; + const t = Math.max(0, Math.min(1, speed / max)); + const position = t * (stops.length - 1); + const index = Math.min(stops.length - 2, Math.floor(position)); + const fraction = position - index; + if (t === 1) return stops.at(-1); + const parse = (color) => + color.match(/\w\w/g).map((part) => parseInt(part, 16)); + const a = parse(stops[index]); + const b = parse(stops[index + 1]); + return `rgb(${a.map((value, i) => Math.round(value + (b[i] - value) * fraction)).join(', ')})`; +} + +/** Advance a particle by a wind vector, mutating and returning it. */ +export function advectParticle( + particle, + wind, + dtSeconds, + { speedScale = 1 } = {}, +) { + particle.lon = normalizeLongitude( + particle.lon + + (wind.u * dtSeconds * speedScale) / metersPerDegreeLon(particle.lat), + ); + particle.lat = Math.max( + -89, + Math.min(89, particle.lat + (wind.v * dtSeconds * speedScale) / 111320), + ); + return particle; +} diff --git a/src/layers/wind/model.test.mjs b/src/layers/wind/model.test.mjs new file mode 100644 index 000000000..59d86efbb --- /dev/null +++ b/src/layers/wind/model.test.mjs @@ -0,0 +1,25 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { advectParticle, sampleWind, windColor } from './model.js'; + +const field = { u: Float32Array.from([1, 2, 3, 4]), v: Float32Array.from([5, 6, 7, 8]), nx: 2, ny: 2, lo1: 0, la1: 90, dx: 180, dy: 90 }; + +test('sampleWind interpolates, wraps, clamps, and rejects non-finite input', () => { + assert.deepEqual(sampleWind(field, 0, 90), { u: 1, v: 5 }); + assert.deepEqual(sampleWind(field, 90, 45), { u: 2.5, v: 6.5 }); + assert.deepEqual(sampleWind(field, 359.99, 90), sampleWind(field, -0.01, 90)); + assert.deepEqual(sampleWind(field, 0, 100), { u: 1, v: 5 }); + assert.deepEqual(sampleWind(field, Infinity, 0), { u: 0, v: 0 }); +}); + +test('advectParticle moves and constrains particles', () => { + const particle = { lon: 179, lat: 88 }; + advectParticle(particle, { u: 10, v: 100 }, 3600); + assert.ok(particle.lon < 0); + assert.equal(particle.lat, 89); +}); + +test('windColor returns a monotonic ramp string', () => { + assert.equal(typeof windColor(0), 'string'); + assert.notEqual(windColor(0), windColor(30)); +}); diff --git a/src/layers/wind/rendering.js b/src/layers/wind/rendering.js new file mode 100644 index 000000000..39d035105 --- /dev/null +++ b/src/layers/wind/rendering.js @@ -0,0 +1,249 @@ +import { advectParticle, sampleWind, windColor, windSpeed } from './model.js'; + +/** + * Canvas particle renderer for a global wind field. + * + * Particles are advected in lon/lat by the sampled field and projected to the + * screen through Cesium each frame. Particles hidden by the globe are skipped + * with an `EllipsoidalOccluder`, and trails fade into a translucent backdrop. + * + * @param {{cesium: object, container: HTMLElement, getViewer: Function}} options + */ +export function createWindRendering({ cesium, container, getViewer } = {}) { + let canvas = null; + let context = null; + let field = null; + let particles = []; + let frame = null; + let running = false; + let lastTime = 0; + let cssWidth = 1; + let cssHeight = 1; + let warned = false; + let cameraSignature = null; + + const randomParticle = () => ({ + lon: Math.random() * 360 - 180, + lat: Math.random() * 178 - 89, + age: Math.random() * 180, + }); + + /** + * Spawn a particle inside the current view. Prefers the camera's visible + * rectangle (robust at any zoom), then ellipsoid picking, then a whole-globe + * fallback. Seeding globally would leave almost nothing on screen at regional + * zoom, which reads as "the layer is broken". + */ + const spawnParticle = (scene) => { + const camera = scene?.camera; + if (camera?.computeViewRectangle && cesium.Ellipsoid && cesium.Math) { + try { + const rect = camera.computeViewRectangle(cesium.Ellipsoid.WGS84); + if (rect) { + const west = cesium.Math.toDegrees(rect.west); + let east = cesium.Math.toDegrees(rect.east); + const south = cesium.Math.toDegrees(rect.south); + const north = cesium.Math.toDegrees(rect.north); + if (east < west) east += 360; // rectangle crosses the antimeridian + const lon = west + Math.random() * (east - west); + const lat = south + Math.random() * (north - south); + return { + lon: ((((lon + 180) % 360) + 360) % 360) - 180, + lat: Math.max(-89, Math.min(89, lat)), + age: Math.random() * 30, + }; + } + } catch { + /* fall through to the global fallback */ + } + } + return randomParticle(); + }; + + const budget = () => + Math.max(200, Math.min(4000, Math.floor((cssWidth * cssHeight) / 300))); + + const seed = () => { + particles = []; + const limit = budget(); + const scene = getViewer?.()?.scene; + for (let i = 0; i < limit; i += 1) particles.push(spawnParticle(scene)); + }; + + const resize = (viewer) => { + const ratio = Math.min(2, globalThis.devicePixelRatio || 1); + const sceneCanvas = viewer?.scene?.canvas; + cssWidth = sceneCanvas?.clientWidth || container?.clientWidth || 1; + cssHeight = sceneCanvas?.clientHeight || container?.clientHeight || 1; + const width = Math.floor(cssWidth * ratio); + const height = Math.floor(cssHeight * ratio); + // Assigning either dimension clears a real canvas, even if unchanged. + if (canvas.width === width && canvas.height === height) return; + canvas.width = width; + canvas.height = height; + canvas.style.width = `${cssWidth}px`; + canvas.style.height = `${cssHeight}px`; + context.setTransform?.(ratio, 0, 0, ratio, 0, 0); + }; + + /** Project lon/lat to CSS pixels, or null when hidden by the globe. */ + const project = (scene, occluder, lon, lat) => { + if (!Number.isFinite(lon) || !Number.isFinite(lat)) return null; + const cartesian = cesium.Cartesian3.fromDegrees(lon, lat, 0); + if (occluder && !occluder.isPointVisible(cartesian)) return null; + return cesium.SceneTransforms.worldToWindowCoordinates(scene, cartesian); + }; + + const draw = (time) => { + frame = null; + if (!running) return; + const viewer = getViewer?.(); + if (!viewer || viewer.isDestroyed?.()) return; + try { + resize(viewer); + if (particles.length !== budget()) seed(); + const camera = viewer.scene.camera; + const signature = [ + camera?.positionWC?.x, + camera?.positionWC?.y, + camera?.positionWC?.z, + camera?.heading, + camera?.pitch, + camera?.roll, + ].join(','); + if (signature !== cameraSignature) { + context.clearRect(0, 0, cssWidth, cssHeight); + cameraSignature = signature; + } + const dt = lastTime + ? Math.min(1, Math.max(0, (time - lastTime) / 1000)) + : 0.016; + lastTime = time; + // Fade previous trails by ERASING them (destination-out). A translucent + // black fill would instead accumulate alpha and saturate the overlay. + context.globalCompositeOperation = 'destination-out'; + context.fillStyle = 'rgba(0, 0, 0, 0.02)'; + context.fillRect(0, 0, cssWidth, cssHeight); + if (!field) return; + const scene = viewer.scene; + // Real wind (m/s) is invisible at globe scale, so the advection is + // exaggerated in proportion to camera height: the same visual speed is + // legible whether the camera is at street level or over a whole hemisphere. + const cameraHeight = scene.camera?.positionCartographic?.height ?? 1e6; + const speedScale = Math.max(1, Math.min(4000, cameraHeight / 400)); + let occluder = null; + try { + if ( + cesium.EllipsoidalOccluder && + cesium.Ellipsoid && + scene.camera?.positionWC + ) + occluder = new cesium.EllipsoidalOccluder( + cesium.Ellipsoid.WGS84, + scene.camera.positionWC, + ); + } catch { + occluder = null; // fall back to frustum-only culling + } + context.lineWidth = 1.8; + context.globalAlpha = 0.9; + // Normal (non-additive) blending: additive strokes saturate to white when + // many trails overlap over a dark globe. + context.globalCompositeOperation = 'source-over'; + for (const particle of particles) { + const wind = sampleWind(field, particle.lon, particle.lat); + const before = { lon: particle.lon, lat: particle.lat }; + advectParticle(particle, wind, dt, { speedScale }); + particle.age += dt; + // A NaN/Infinity position (polar divide-by-zero or a bad grid value) + // must never reach Cesium: it throws and would abort the whole frame. + if (!Number.isFinite(particle.lon) || !Number.isFinite(particle.lat)) { + Object.assign(particle, spawnParticle(scene)); + continue; + } + const point = project(scene, occluder, particle.lon, particle.lat); + const offscreen = + !point || + point.x < 0 || + point.y < 0 || + point.x > cssWidth || + point.y > cssHeight; + if (particle.age > 240 || offscreen) { + Object.assign(particle, spawnParticle(scene)); + continue; + } + const previous = project(scene, occluder, before.lon, before.lat); + if (!previous) continue; + const speed = windSpeed(wind.u, wind.v); + context.strokeStyle = windColor(speed); + context.beginPath(); + context.moveTo(previous.x, previous.y); + context.lineTo(point.x, point.y); + context.stroke(); + } + context.globalAlpha = 1; + } catch (error) { + if (!warned) { + console.warn('[Data:Wind] frame failed:', error?.message || error); + warned = true; + } + } + frame = globalThis.requestAnimationFrame(draw); + }; + + return { + /** Create and append the overlay canvas. */ + attach() { + if (canvas) return; + canvas = document.createElement('canvas'); + canvas.dataset.gevWind = '1'; + canvas.style.position = 'absolute'; + canvas.style.inset = '0'; + canvas.style.pointerEvents = 'none'; + canvas.style.zIndex = '1'; + context = canvas.getContext('2d'); + container.appendChild(canvas); + }, + /** Install a wind field and (re)seed particles. */ + setField(next) { + field = next?.grid ? { ...next.grid, u: next.u, v: next.v } : next; + if (canvas) { + resize(getViewer?.()); + seed(); + if (running && frame === null) + frame = globalThis.requestAnimationFrame(draw); + } + }, + /** Start the animation loop. Safe to call before a field is installed. */ + start() { + if (running || !canvas) return; + running = true; + lastTime = 0; + if (field) frame = globalThis.requestAnimationFrame(draw); + }, + /** Stop the animation loop. */ + stop() { + running = false; + if (frame !== null) globalThis.cancelAnimationFrame(frame); + frame = null; + }, + /** Clear the canvas and particles without removing the canvas. */ + clear() { + field = null; + particles = []; + context?.clearRect(0, 0, cssWidth, cssHeight); + }, + /** Stop and remove the overlay canvas. Idempotent. */ + destroy() { + this.stop(); + this.clear(); + canvas?.remove(); + canvas = null; + context = null; + }, + /** Test seam: the current particle count. */ + getParticleCount() { + return particles.length; + }, + }; +} diff --git a/src/layers/wind/rendering.test.mjs b/src/layers/wind/rendering.test.mjs new file mode 100644 index 000000000..d477d94c5 --- /dev/null +++ b/src/layers/wind/rendering.test.mjs @@ -0,0 +1,136 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; + +import { createWindRendering } from './rendering.js'; + +/** Build a fake browser/Cesium environment for the overlay. */ +function harness({ occluded = false } = {}) { + const strokes = []; + const context = { + setTransform() {}, + clearRect() {}, + fillRect() {}, + beginPath() {}, + moveTo() {}, + lineTo() {}, + stroke() { + strokes.push(true); + }, + }; + const canvas = { + style: {}, + dataset: {}, + width: 0, + height: 0, + clientWidth: 800, + clientHeight: 600, + getContext: () => context, + remove() { + this.removed = true; + }, + }; + const container = { + clientWidth: 800, + clientHeight: 600, + appendChild(node) { + this.node = node; + }, + }; + const callbacks = []; + globalThis.document = { createElement: () => canvas }; + globalThis.requestAnimationFrame = (callback) => { + callbacks.push(callback); + return callbacks.length; + }; + globalThis.cancelAnimationFrame = () => { + globalThis.cancelled = true; + }; + const viewer = { + container, + scene: { canvas, camera: { positionWC: {} } }, + isDestroyed: () => false, + }; + const cesium = { + Cartesian3: { fromDegrees: (lon, lat) => ({ lon, lat }) }, + Ellipsoid: { WGS84: {} }, + EllipsoidalOccluder: class { + isPointVisible() { + return !occluded; + } + }, + SceneTransforms: { + worldToWindowCoordinates: () => ({ x: 100, y: 100 }), + }, + }; + const rendering = createWindRendering({ cesium, container, getViewer: () => viewer }); + return { rendering, canvas, container, callbacks, strokes }; +} + +const FIELD = { + u: Float32Array.from([10]), + v: Float32Array.from([0]), + nx: 1, + ny: 1, + lo1: 0, + la1: 90, + dx: 360, + dy: 180, +}; + +test('wind rendering owns the canvas and particle lifecycle', () => { + const { rendering, canvas, callbacks, strokes } = harness(); + rendering.attach(); + assert.equal(typeof canvas.getContext('2d'), 'object'); + rendering.setField(FIELD); + assert.ok(rendering.getParticleCount() >= 200); + rendering.start(); + callbacks.shift()(16); + assert.ok(strokes.length > 0, 'a driven frame draws trails'); + rendering.stop(); + assert.equal(globalThis.cancelled, true); + rendering.clear(); + assert.equal(rendering.getParticleCount(), 0); + rendering.destroy(); + rendering.destroy(); + assert.equal(canvas.removed, true); +}); + +test('wind rendering starts before a field and still draws once one arrives', () => { + const { rendering, callbacks, strokes } = harness(); + rendering.attach(); + rendering.start(); + assert.equal(callbacks.length, 0, 'no animation work before a field arrives'); + rendering.setField(FIELD); + callbacks.shift()(32); + assert.ok(strokes.length > 0, 'draws once the field is installed'); + rendering.stop(); +}); + +test('wind rendering skips particles hidden by the globe', () => { + const { rendering, callbacks, strokes } = harness({ occluded: true }); + rendering.attach(); + rendering.setField(FIELD); + rendering.start(); + callbacks.shift()(16); + assert.equal(strokes.length, 0, 'no trail is drawn for an occluded point'); + rendering.stop(); +}); + +// HTML canvas dimension setters erase pixels even on an identical assignment. +test('steady frames preserve canvas dimensions and resize adjusts the particle budget', () => { + const { rendering, canvas, callbacks } = harness(); + let resets = 0; let width = 0; let height = 0; + Object.defineProperties(canvas, { + width: { get: () => width, set: value => { width = value; resets++; } }, + height: { get: () => height, set: value => { height = value; resets++; } }, + }); + rendering.attach(); rendering.setField({ grid: FIELD, u: FIELD.u, v: FIELD.v }); rendering.start(); + const initial = resets; + callbacks.shift()(16); callbacks.shift()(32); + assert.equal(resets, initial, 'steady frames must not invoke canvas dimension setters'); + canvas.clientWidth = 320; canvas.clientHeight = 240; + callbacks.shift()(48); + assert.equal(resets, initial + 2); + assert.ok(rendering.getParticleCount() < 500, 'small view uses a smaller budget'); + rendering.destroy(); +}); diff --git a/src/layers/wind/source.js b/src/layers/wind/source.js new file mode 100644 index 000000000..d8ce606d0 --- /dev/null +++ b/src/layers/wind/source.js @@ -0,0 +1,76 @@ +import { readWindBody } from '../../sources/windBody.js'; + +/** Create a bounded, cancellable source for the same-origin wind provider. */ +export function createWindSource({ + fetchImpl = (...args) => globalThis.fetch(...args), + timeoutMs = 45_000, +} = {}) { + return { + async getSnapshot({ signal, model = 'gfs' } = {}) { + if (!['gfs', 'ifs'].includes(model)) + throw new Error('Unknown wind model'); + const controller = new AbortController(); + const abort = () => controller.abort(signal.reason); + signal?.addEventListener('abort', abort, { once: true }); + const timer = setTimeout( + () => controller.abort(new Error('Wind request timed out')), + timeoutMs, + ); + const active = controller.signal; + try { + signal?.throwIfAborted(); + const response = await fetchImpl(`/api/wind/manifest?model=${model}`, { + signal: active, + cache: 'no-store', + redirect: 'error', + }); + if (!response.ok) throw new Error(`Wind HTTP ${response.status}`); + const manifest = JSON.parse( + new TextDecoder().decode( + await readWindBody(response, 16_384, active), + ), + ); + if (manifest?.unavailable) return manifest; + const grid = manifest?.grid; + if ( + manifest?.model !== model || + !grid || + !Number.isInteger(grid.nx) || + !Number.isInteger(grid.ny) || + grid.nx < 1 || + grid.ny < 1 || + grid.nx * grid.ny > 1_000_000 || + ![grid.lo1, grid.la1, grid.dx, grid.dy].every(Number.isFinite) || + grid.dx <= 0 || + grid.dy <= 0 || + Math.abs(grid.nx * grid.dx - 360) > 0.01 || + !new RegExp( + `^/api/wind/grid/${model}-[\\w.-]+\\.bin\\?model=${model}$`, + ).test(manifest.gridUrl) + ) + throw new Error('Malformed wind manifest'); + const count = grid.nx * grid.ny; + const gridResponse = await fetchImpl(manifest.gridUrl, { + signal: active, + redirect: 'error', + }); + if (!gridResponse.ok) + throw new Error(`Wind HTTP ${gridResponse.status}`); + const bytes = await readWindBody(gridResponse, count * 8, active); + if (bytes.byteLength !== count * 8) + throw new Error('Malformed wind grid'); + const values = new Float32Array(bytes.buffer); + if (!values.every(Number.isFinite)) + throw new Error('Malformed wind grid'); + return { + ...manifest, + u: values.slice(0, count), + v: values.slice(count), + }; + } finally { + clearTimeout(timer); + signal?.removeEventListener('abort', abort); + } + }, + }; +} diff --git a/src/layers/wind/source.test.mjs b/src/layers/wind/source.test.mjs new file mode 100644 index 000000000..05e69fd69 --- /dev/null +++ b/src/layers/wind/source.test.mjs @@ -0,0 +1,25 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createWindSource } from './source.js'; +const manifest = { model: 'gfs', grid: { nx: 2, ny: 1, lo1: 0, la1: 90, dx: 180, dy: 180 }, gridUrl: '/api/wind/grid/gfs-test.bin?model=gfs' }; +const response = (value) => new Response(typeof value === 'object' && !(value instanceof ArrayBuffer) ? JSON.stringify(value) : value); +test('wind source splits a bounded field and preserves unavailable responses', async () => { + const source = createWindSource({ fetchImpl: async (url) => response(url.includes('manifest') ? manifest : Float32Array.from([1,2,3,4]).buffer) }); + const field = await source.getSnapshot(); + assert.deepEqual([...field.u], [1,2]); assert.deepEqual([...field.v], [3,4]); + const unavailable = createWindSource({ fetchImpl: async () => response({ unavailable: true }) }); + assert.equal((await unavailable.getSnapshot()).unavailable, true); +}); +test('wind source rejects foreign URLs, excessive grids and non-finite values', async () => { + for (const bad of [{ ...manifest, gridUrl: 'https://example.com/exfil' }, { ...manifest, grid: { ...manifest.grid, nx: 1e9 } }]) { + let calls = 0; + const source = createWindSource({ fetchImpl: async () => { calls++; return response(bad); } }); + await assert.rejects(source.getSnapshot(), /Malformed/); assert.equal(calls, 1); + } + const source = createWindSource({ fetchImpl: async (url) => response(url.includes('manifest') ? manifest : Float32Array.from([NaN,2,3,4]).buffer) }); + await assert.rejects(source.getSnapshot(), /Malformed/); +}); +test('wind source timeout aborts the underlying fetch', async () => { + const source = createWindSource({ timeoutMs: 5, fetchImpl: (_, { signal }) => new Promise((resolve,reject) => signal.addEventListener('abort', () => reject(signal.reason), { once:true })) }); + await assert.rejects(source.getSnapshot(), /timed out/); +}); diff --git a/src/sources/windBody.js b/src/sources/windBody.js new file mode 100644 index 000000000..0aa1a0e22 --- /dev/null +++ b/src/sources/windBody.js @@ -0,0 +1,40 @@ +/** Read a fetch body under a hard byte budget, including chunked responses. */ +export async function readWindBody(response, maxBytes, signal) { + const fail = () => new Error('Wind response exceeds byte budget'); + if (Number(response.headers?.get('content-length')) > maxBytes) { + await response.body?.cancel(); + throw fail(); + } + const reader = response.body?.getReader(); + if (!reader) throw new Error('Wind response has no readable body'); + const chunks = []; + let size = 0; + const cancel = () => { + void reader.cancel().catch(() => {}); + }; + signal?.addEventListener('abort', cancel, { once: true }); + try { + while (true) { + signal?.throwIfAborted(); + const { done, value } = await reader.read(); + signal?.throwIfAborted(); + if (done) break; + size += value.byteLength; + if (size > maxBytes) throw fail(); + chunks.push(value); + } + const bytes = new Uint8Array(size); + let offset = 0; + for (const chunk of chunks) { + bytes.set(chunk, offset); + offset += chunk.byteLength; + } + return bytes; + } catch (error) { + cancel(); + throw error; + } finally { + signal?.removeEventListener('abort', cancel); + reader.releaseLock(); + } +} diff --git a/src/standalone/layerSources.js b/src/standalone/layerSources.js index f0bff95cc..9596943a9 100644 --- a/src/standalone/layerSources.js +++ b/src/standalone/layerSources.js @@ -12,6 +12,7 @@ import { createInstallationSource } from '../layers/installations/source.js'; import { createSatelliteSource } from '../layers/satellites/source.js'; import { createLaunchSource } from '../layers/launches/source.js'; import { createOverpassAlprSource } from '../layers/alpr/source.js'; +import { createWindSource } from '../layers/wind/source.js'; import { createFirmsSource } from '../layers/firms/source.js'; import { createReferenceSources } from '../sources/reference.js'; export { createReferenceSources as createStandaloneReferenceSources } from '../sources/reference.js'; @@ -35,5 +36,6 @@ export function createStandaloneLayerSources() { launches: createLaunchSource(), alpr: createOverpassAlprSource(), firms: createFirmsSource(), + wind: createWindSource(), }; } diff --git a/src/ui/layerPanel.js b/src/ui/layerPanel.js index d213cdc87..918e448cb 100644 --- a/src/ui/layerPanel.js +++ b/src/ui/layerPanel.js @@ -297,11 +297,18 @@ export class LayerPanel { const chips = controls?.chips || []; const legend = controls?.legend || []; this._syncRowList(listContainer, controls?.list || null); - container.hidden = chips.length === 0 && legend.length === 0; + container.hidden = + chips.length === 0 && legend.length === 0 && !controls?.info; for (const node of [...container.children]) { if ( - String(node.className).split(/\s+/).includes('data-toggle-legend-item') + String(node.className) + .split(/\s+/) + .some((name) => + ['data-toggle-legend-item', 'data-toggle-controls-info'].includes( + name, + ), + ) ) node.remove(); } @@ -338,10 +345,20 @@ export class LayerPanel { swatch.className = 'data-toggle-legend-swatch'; swatch.style.background = item.color; const text = document.createElement('span'); - text.textContent = `${item.label} ${this._formatCount(item.count)}`; + text.textContent = + item.count == null + ? String(item.label) + : `${item.label} ${this._formatCount(item.count)}`; entry.append(swatch, text); container.appendChild(entry); } + if (controls?.info) { + const info = document.createElement('div'); + info.className = 'data-toggle-controls-info'; + info.textContent = String(controls.info); + if (controls.infoTitle) info.title = controls.infoTitle; + container.appendChild(info); + } } /** diff --git a/src/ui/styles/radio.css b/src/ui/styles/radio.css index 7b2ce0a69..955a97b58 100644 --- a/src/ui/styles/radio.css +++ b/src/ui/styles/radio.css @@ -888,3 +888,12 @@ opacity: 0.75; font-style: italic; } + +.data-toggle-controls-info { + flex-basis: 100%; + font-family: var(--font-mono); + font-size: 9px; + color: var(--text-secondary); + padding-top: 3px; + overflow-wrap: anywhere; +}