Compare commits

..
Author SHA1 Message Date
Mazeyar Moeini FeizabadiandClaude Fable 5.2 5dfa7ca32d docs(changelog): add the #1717 fragment
Co-Authored-By: Claude Fable 5.2 <[email protected]>
2026-11-13 17:18:40 +01:00
Mazeyar Moeini FeizabadiandClaude Fable 5.2 ef6c30d4c3 perf(wind): decode forecast GRIB in a worker and reuse the cycle's wind
Each /api/wind refresh decoded two or three 0.25 degree GRIB2 messages with
ecCodes (WASM), validated ~1.04 M doubles per message and resampled them, all
synchronously on the server thread. Measured with cached upstream bytes: 43
to 164 ms of event-loop block per warm refresh, 230 to 381 ms for the first
one in a process, 316 to 974 ms when all six keys refresh together.

Decode, validation and resampling now run on one lazily started worker
thread, which transfers back only the display-grid samples. The server keeps
the sampled U/V of the last forecast object per model (1.04 MB each), so an
overlay change, or a refresh within the same forecast step, fetches and
decodes only the scalar. Manifests and grid bodies are byte-identical to
before for the same upstream bytes.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
2026-11-13 17:15:20 +01:00
17 changed files with 903 additions and 882 deletions

No files matched your search

+5 -2
View File
@@ -564,8 +564,11 @@ Global Forecast System (GFS). The `/api/wind` server-side proxy selects the
latest available 0.25° cycle, reads its `.idx` inventory, byte-range fetches only
the `UGRD`/`VGRD` 10 m GRIB2 messages, optionally adding `TMP` at 2 m or
`PRMSL` at mean sea level from the same run and forecast time. It decodes the
selected messages with ecCodes (WASM), resamples to a compact grid (1° by
default), and serves a manifest plus Float32 U/V and optional scalar values. It is keyless, cached
selected messages with ecCodes (WASM) and resamples them to a compact grid (1° by
default) on a worker thread, so a refresh does not stall other API routes, and
serves a manifest plus Float32 U/V and optional scalar values. The sampled wind
of the current forecast file is kept, so choosing a color field fetches only
that field. It is keyless, cached
per cycle for an hour, and is a **forecast, not an observation**: the particles
show model flow, not measured wind. The renderer bakes bounded curves through
the sampled field and animates their phase on the GPU, with a canvas fallback.
+4
View File
@@ -0,0 +1,4 @@
- The Wind server no longer blocks every other request while it decodes a
forecast. Decoding runs on a worker thread (up to 400 ms of blocking down to
about 4 ms), and choosing an overlay after wind has loaded fetches only the
overlay's field (mazy1998, #1717).
@@ -1,4 +0,0 @@
- The detection overlay no longer rebuilds every bracket on a paint where the
view and every contact are unchanged, which cuts frame work by a fifth to a
third at a whole-globe view of Live Flights with a still camera (mazy1998,
#1719).
-22
View File
@@ -7459,28 +7459,6 @@ apply to `public/` assets or built-preview serving.
holds the way it stripped detection's would take bracket promptness with it, silently.
- Known, pre-existing, and deliberately out of scope here: the overlay's backing store does not
re-derive on a DPR change mid-session (`worldOverlay` sizing — untouched by this work).
- **A detection paint reuses the sweep before it when nothing it reads has changed
(`src/data/detectionPaintCache.js`).** The sweep in `_drawOverlay` (project every contact, fade
it, append its bracket, build callout candidates) depends only on the view and on the contacts.
Each paint still asks every layer for its contacts, then compares them with what the last sweep
was built from: the same objects in the same order, the same position coordinates, and the same
value in every field the sweep reads (`PAINTED_OBJECT_FIELDS`), under the same view (profile,
style, canvas size, keyhole, camera position, view-projection matrix, UI rectangles; see
`_paintViewOf`). On a match the kept bracket paths, counts and callout candidates are used again
and only the strokes and callouts are repainted. No layer has to announce a change, so none can
leave a stale picture behind. A sweep that READS a new object field or a new view value must add
it to those two lists; `detectionHost.test.mjs` compares 41 frames against a full sweep, canvas
call for canvas call.
- Never reused or kept: a solve paint (the sweep gathers the cohort), any paint while the Cyber
sonar sweep or a Global Context echo is live (brackets change with the clock), and any sweep
that reached a depth-tested contact (its answer comes from the scene).
- Where it pays: a held camera with contacts that move in steps. Live Flights re-positions the
fleet every 80 ms, so at 60 fps about seven paints in ten reuse a sweep. Once a frame takes
longer than 80 ms every paint sees moved aircraft and none is reused; after eight such sweeps
the contacts are only recorded on every eighth, so the comparison costs nothing there.
Satellites move on every frame and are never reused.
- `getDetectionDiagnostics().paintReuseCount` (and `data-paint-reuse-count`) counts reused
paints since `initDetection`; `placementBuildCount` is 0 on a reused paint.
- The detection MODE BANNER (`DENSE VIS:… SRC:… DENS:…% ELASTIC …ms`) is
developer telemetry and is HIDDEN by default. It paints only under
`?detectDebug=1` (the `trafficDebug` convention), resolved once per
-2
View File
@@ -2690,7 +2690,6 @@
"src/data/detectionDraw.js",
"src/data/detectionPolicy.js",
"src/data/detectionOcclusion.js",
"src/data/detectionPaintCache.js",
"src/data/detectionPresentation.js",
"src/data/detectionRenderDemand.js",
"src/data/sceneOcclusionProbe.js",
@@ -4373,7 +4372,6 @@
"src/data/detectionRenderDemand.js",
"src/data/detectionPresentation.js",
"src/data/detectionOcclusion.js",
"src/data/detectionPaintCache.js",
"src/data/sceneOcclusionProbe.js",
"src/data/modelEligibility.js",
"src/app/layers/recentImagery.js",
+5 -2
View File
@@ -1,6 +1,6 @@
import { fetchGfsWind } from './wind/gfs.js';
import { fetchIfsWind } from './wind/ifs.js';
import { decodeWindGribMessage } from './wind/decode.js';
import { decodeWindGribInWorker } from './wind/decoder.js';
import { weatherScalarMetadata, weatherScalarError } from './wind/grid.js';
// Data, not UI copy: the client (src/layers/wind/index.js) marks this exact
@@ -11,7 +11,7 @@ const UPSTREAM_UNAVAILABLE = 'Wind upstream unavailable';
export function windProxy({
fetchImpl = fetch,
now = () => Date.now(),
decodeImpl = decodeWindGribMessage,
decodeImpl = decodeWindGribInWorker,
targetDx = 1,
ttlMs = 3600_000,
timeoutMs = 40_000,
@@ -22,6 +22,8 @@ export function windProxy({
const loadings = new Map();
const grids = new Map();
const attempts = new Map();
// Sampled U/V of the last forecast object per model (see loadWeatherSnapshot).
const windCache = new Map();
const unavailable = (model, overlay) => ({
manifest: {
model,
@@ -54,6 +56,7 @@ export function windProxy({
targetDx,
overlay,
signal: controller.signal,
windCache,
});
controller.signal.throwIfAborted();
const { grid, cycle } = value;
+6
View File
@@ -0,0 +1,6 @@
// Worker-thread entry for the wind provider, started by ./decoder.js. Node
// loads this file from disk, so it may only import files that ship beside it.
import { parentPort } from 'node:worker_threads';
import { serveWindDecodes } from './decode.js';
serveWindDecodes(parentPort);
+17
View File
@@ -1,3 +1,5 @@
import { sampleWindField } from './grid.js';
/** Lazy shared ecCodes WASM instance, created only for an active wind request. */
let eccodesPromise = null;
function loadEccodes() {
@@ -49,3 +51,18 @@ export async function decodeWindGribMessage(buffer, { eccodesModule } = {}) {
eccodes.writeFile(file, new Uint8Array(0));
}
}
/**
* Worker side of ./decoder.js: decode each posted message, validate and sample
* it to the display grid, and transfer back only the samples.
*/
export function serveWindDecodes(port, decodeImpl = decodeWindGribMessage) {
port.on('message', async ({ id, bytes, targetDx }) => {
try {
const field = sampleWindField(await decodeImpl(bytes), targetDx);
port.postMessage({ id, field }, [field.samples.buffer]);
} catch (error) {
port.postMessage({ id, error: String(error?.message ?? error) });
}
});
}
+108
View File
@@ -0,0 +1,108 @@
import { Worker } from 'node:worker_threads';
const WORKER_URL = new URL('./decode-worker.js', import.meta.url);
const MAX_MESSAGE_BYTES = 8 * 1024 * 1024;
/**
* Decode GRIB messages on one lazily started worker thread.
*
* ecCodes decodes a 0.25° message synchronously (~1.04 M doubles), and
* validating and resampling those values is synchronous too, so on the server
* thread each refresh stalled every other API route. The worker does all three
* and transfers back only the display-grid samples.
*
* The worker is referenced only while a job is pending, so it never keeps the
* process alive. A crash, an exit or a job that outlives `jobTimeoutMs`
* rejects every pending job and retires the worker; the next decode starts a
* new one.
*
* @param {{workerUrl?: URL, jobTimeoutMs?: number}} [options]
* @returns {{decode: Function, close: Function}}
*/
export function createWindDecoder({
workerUrl = WORKER_URL,
jobTimeoutMs = 30_000,
} = {}) {
let active = null;
let nextId = 1;
const retire = (state, error) => {
if (active === state) active = null;
const jobs = [...state.pending.values()];
state.pending.clear();
for (const job of jobs) job.reject(error);
void state.worker.terminate();
};
const start = () => {
const state = {
worker: new Worker(workerUrl),
pending: new Map(),
};
state.worker.unref();
state.worker.on('message', ({ id, field, error }) => {
const job = state.pending.get(id);
if (!job) return; // Already rejected by an abort.
if (error === undefined) job.resolve(field);
else job.reject(new Error(error));
});
state.worker.on('error', (cause) =>
retire(state, new Error('Wind decode worker failed', { cause })),
);
state.worker.on('exit', () =>
retire(state, new Error('Wind decode worker exited')),
);
return state;
};
/**
* Decode one message and sample it to `targetDx` (see `sampleWindField`).
* An aborted `signal` rejects at once; the worker finishes the bounded job
* and its result is dropped.
*/
const decode = (buffer, { targetDx = 1, signal } = {}) =>
new Promise((resolve, reject) => {
if (buffer.byteLength > MAX_MESSAGE_BYTES)
throw new Error('Wind GRIB exceeds byte budget');
signal?.throwIfAborted();
// A private copy: transferring the caller's (possibly pooled) Buffer
// would detach memory it still owns.
const bytes = new Uint8Array(buffer.byteLength);
bytes.set(buffer);
const state = (active ??= start());
const id = nextId++;
const settle = (finish) => (value) => {
if (state.pending.get(id) === job) state.pending.delete(id);
clearTimeout(timer);
signal?.removeEventListener('abort', abort);
if (state.pending.size === 0) state.worker.unref();
finish(value);
};
const job = { resolve: settle(resolve), reject: settle(reject) };
const abort = () => job.reject(signal.reason);
const timer = setTimeout(
() => retire(state, new Error('Wind decode timed out')),
jobTimeoutMs,
);
timer.unref();
state.pending.set(id, job);
state.worker.ref();
signal?.addEventListener('abort', abort, { once: true });
state.worker.postMessage({ id, bytes, targetDx }, [bytes.buffer]);
});
return {
decode,
/** Stop the worker; pending jobs reject and a later decode restarts it. */
close() {
if (active) retire(active, new Error('Wind decode worker closed'));
},
};
}
let shared = null;
/**
* Default decoder for the wind provider: one worker for the whole process,
* started by the first wind request, however many `windProxy` instances exist.
*/
export function decodeWindGribInWorker(buffer, options) {
shared ??= createWindDecoder();
return shared.decode(buffer, options);
}
+86 -20
View File
@@ -125,6 +125,7 @@ export async function loadOptionalScalar({
fetchImpl,
decodeImpl,
signal,
targetDx = 1,
timeoutMs = 12_000,
}) {
if (!range) return undefined;
@@ -140,7 +141,13 @@ export async function loadOptionalScalar({
fetchImpl,
signal: controller.signal,
});
const value = await decodeImpl(bytes);
// Sampling here keeps an unusable companion on the same wind-only path as
// an undecodable one. The decoder gets the signal so a timeout or a
// cancelled refresh stops waiting for a decode that runs off this thread.
const value = sampleWindField(
await decodeImpl(bytes, { targetDx, signal: controller.signal }),
targetDx,
);
controller.signal.throwIfAborted();
return value;
} catch {
@@ -158,17 +165,80 @@ import {
selectLatestGfsCycle,
nearestGfsStep,
} from './catalog.js';
import { decodeWindGribMessage } from './decode.js';
import { resampleWeatherSnapshot, weatherScalarMetadata } from './grid.js';
import { decodeWindGribInWorker } from './decoder.js';
import {
resampleWeatherSnapshot,
sampleWindField,
weatherScalarMetadata,
} from './grid.js';
/**
* Load U, V and the optional companion from one forecast object and resample
* them to the display grid.
*
* `windCache` (a Map owned by one `windProxy`) retains the sampled U/V of the
* last object loaded per model. A later request for the same object and byte
* ranges, such as an overlay change or a refresh within the same forecast
* step, then fetches and decodes only the companion. It holds two Float64
* display grids per model (2 x 65,160 x 8 B = 1.04 MB at 1 degree) and is
* written only after the pair has passed every wind check.
*/
export async function loadWeatherSnapshot({
model,
url,
ranges,
scalarRange,
overlay,
targetDx,
fetchImpl,
decodeImpl,
signal,
windCache,
}) {
const key = [
url,
`${ranges.u.start}-${ranges.u.end}`,
`${ranges.v.start}-${ranges.v.end}`,
targetDx,
].join('|');
const kept = windCache?.get(model);
const wind = kept?.key === key ? kept : null;
// All selected messages come from this one issue/valid-time object.
const load = async (range) =>
sampleWindField(
await decodeImpl(await fetchRange({ url, ...range, fetchImpl, signal }), {
targetDx,
signal,
}),
targetDx,
);
const [u, v, scalar] = await Promise.all([
wind ? wind.u : load(ranges.u),
wind ? wind.v : load(ranges.v),
loadOptionalScalar({
range: scalarRange,
url,
fetchImpl,
decodeImpl,
signal,
targetDx,
}),
]);
signal?.throwIfAborted();
const fields = resampleWeatherSnapshot({ u, v, scalar, overlay, targetDx });
if (!wind) windCache?.set(model, { key, u, v });
return fields;
}
/** Fetch and decode the GFS 10 m wind field valid closest to now. */
export async function fetchGfsWind({
fetchImpl = fetch,
now = () => Date.now(),
targetDx = 1,
decodeImpl = decodeWindGribMessage,
decodeImpl = decodeWindGribInWorker,
signal,
overlay = 'none',
windCache,
} = {}) {
weatherScalarMetadata(overlay);
const nowMs = now();
@@ -194,22 +264,18 @@ export async function fetchGfsWind({
/* A missing optional field does not discard valid wind. */
}
}
// All selected messages come from this one issue/valid-time object.
const load = async (range) =>
decodeImpl(await fetchRange({ url: base, ...range, fetchImpl, signal }));
const [u, v, scalar] = await Promise.all([
load(ranges.u),
load(ranges.v),
loadOptionalScalar({
range: scalarRange,
url: base,
fetchImpl,
decodeImpl,
signal,
}),
]);
signal?.throwIfAborted();
const fields = resampleWeatherSnapshot({ u, v, scalar, overlay, targetDx });
const fields = await loadWeatherSnapshot({
model: 'gfs',
url: base,
ranges,
scalarRange,
overlay,
targetDx,
fetchImpl,
decodeImpl,
signal,
windCache,
});
const runIso = new Date(runMs).toISOString();
const validIso = new Date(runMs + forecastHour * 3600_000).toISOString();
return {
+127 -35
View File
@@ -48,38 +48,133 @@ export function resampleWindGrid({
if (!Array.prototype.every.call(values, Number.isFinite))
throw new Error('Invalid wind grid values');
}
const geometry = { ni, nj, lo1, la1, di, dj };
const sampled = (values) => ({
...geometry,
dx,
dy,
samples: sampleValues(values, geometry, dx, dy),
});
return composeWindGrid({
u: sampled(u),
v: sampled(v),
...(scalar === undefined ? {} : { scalar: sampled(scalar) }),
scalarScale,
scalarOffset,
});
}
/** Display-grid shape for a target spacing, under the point budget. */
function displayShape(dx, dy) {
const nx = Math.round(360 / dx);
const ny = Math.round(180 / dy) + 1;
if (nx * ny > 1_000_000)
throw new Error('Resampled wind grid exceeds point budget');
const outU = new Float32Array(nx * ny);
const outV = new Float32Array(nx * ny);
const outScalar = scalar === undefined ? null : new Float32Array(nx * ny);
const sample = (source, lon, lat) => {
const x = ((lon - lo1) / di + ni) % ni;
const y = Math.max(0, Math.min(nj - 1, (la1 - lat) / dj));
const x0 = Math.floor(x);
const x1 = (x0 + 1) % ni;
const y0 = Math.floor(y);
const y1 = Math.min(nj - 1, y0 + 1);
const fx = x - x0;
const fy = y - y0;
const top = source[y0 * ni + x0] * (1 - fx) + source[y0 * ni + x1] * fx;
const bottom = source[y1 * ni + x0] * (1 - fx) + source[y1 * ni + x1] * fx;
return top * (1 - fy) + bottom * fy;
};
return { nx, ny };
}
/**
* Bilinearly sample one validated source field onto the display grid. The
* result stays in doubles so a caller can scale it before narrowing to
* Float32, exactly as if it had sampled, scaled and stored in one step.
*/
function sampleValues(source, { ni, nj, lo1, la1, di, dj }, dx, dy) {
const { nx, ny } = displayShape(dx, dy);
const samples = new Float64Array(nx * ny);
for (let y = 0; y < ny; y += 1) {
const lat = 90 - y * dy;
const sy = Math.max(0, Math.min(nj - 1, (la1 - lat) / dj));
const y0 = Math.floor(sy);
const y1 = Math.min(nj - 1, y0 + 1);
const fy = sy - y0;
for (let x = 0; x < nx; x += 1) {
const index = y * nx + x;
const lon = x * dx;
const lat = 90 - y * dy;
outU[index] = sample(u, lon, lat);
outV[index] = sample(v, lon, lat);
if (outScalar)
outScalar[index] =
sample(scalar, lon, lat) * scalarScale + scalarOffset;
const sx = ((lon - lo1) / di + ni) % ni;
const x0 = Math.floor(sx);
const x1 = (x0 + 1) % ni;
const fx = sx - x0;
const top = source[y0 * ni + x0] * (1 - fx) + source[y0 * ni + x1] * fx;
const bottom =
source[y1 * ni + x0] * (1 - fx) + source[y1 * ni + x1] * fx;
samples[y * nx + x] = top * (1 - fy) + bottom * fy;
}
}
return samples;
}
/**
* Validate one decoded GRIB field and reduce it to display-grid samples.
*
* This is the expensive per-message step after the decode (a 0.25° message is
* ~1.04 M doubles to check), so the decode worker runs it and transfers only
* the result. A field that already carries samples for `targetDx` passes
* through, which lets the same call normalise worker output, injected decoders
* and the retained wind of the current cycle.
*
* @param {object} field - `decodeWindGribMessage` output, or a sampled field.
* @param {number} [targetDx] - Display-grid spacing in degrees.
* @returns {{ni: number, nj: number, lo1: number, la1: number, di: number,
* dj: number, units: (string|undefined), dx: number, dy: number,
* samples: Float64Array}}
*/
export function sampleWindField(field, targetDx = 1) {
if (!field) throw new Error('Mismatched weather grid geometry');
if (
field.samples instanceof Float64Array &&
field.dx === targetDx &&
field.dy === targetDx
)
return field;
const { ni, nj, lo1, la1, di, dj, values, units } = field;
if (
!Number.isInteger(ni) ||
!Number.isInteger(nj) ||
ni < 1 ||
nj < 1 ||
ni * nj > 2_000_000 ||
values?.length !== ni * nj ||
![lo1, la1, di, dj, targetDx].every(Number.isFinite) ||
di <= 0 ||
dj <= 0 ||
targetDx < 0.25 ||
targetDx > 180
)
throw new Error('Invalid wind grid geometry');
if (!Array.prototype.every.call(values, Number.isFinite))
throw new Error('Invalid wind grid values');
const geometry = { ni, nj, lo1, la1, di, dj };
return {
...geometry,
units,
dx: targetDx,
dy: targetDx,
samples: sampleValues(values, geometry, targetDx, targetDx),
};
}
/** Narrow sampled components that share one display grid to Float32. */
function composeWindGrid({ u, v, scalar, scalarScale = 1, scalarOffset = 0 }) {
const { dx, dy } = u;
const { nx, ny } = displayShape(dx, dy);
const components = [u, v, ...(scalar ? [scalar] : [])];
if (
components.some(
(field) =>
field.dx !== dx ||
field.dy !== dy ||
!(field.samples instanceof Float64Array) ||
field.samples.length !== nx * ny,
)
)
throw new Error('Invalid wind grid geometry');
const outU = new Float32Array(u.samples);
const outV = new Float32Array(v.samples);
let outScalar = null;
if (scalar) {
outScalar = new Float32Array(nx * ny);
for (let index = 0; index < outScalar.length; index += 1)
outScalar[index] = scalar.samples[index] * scalarScale + scalarOffset;
}
for (const values of [outU, outV, ...(outScalar ? [outScalar] : [])]) {
if (!values.every(Number.isFinite))
throw new Error('Invalid resampled wind values');
@@ -130,20 +225,17 @@ export function resampleWeatherFields({
throw new Error('Mismatched weather grid geometry');
if (metadata && scalar.units !== (overlay === 'temperature' ? 'K' : 'Pa'))
throw new Error('Unexpected weather scalar units');
const grid = resampleWindGrid({
u: u.values,
v: v.values,
ni: u.ni,
nj: u.nj,
lo1: u.lo1,
la1: u.la1,
di: u.di,
dj: u.dj,
dx: targetDx,
dy: targetDx,
// Raw fields are validated and sampled here; worker output and the retained
// wind of this cycle already are, so only the cheap narrowing is left.
const [sampledU, sampledV, sampledScalar] = fields.map((field) =>
sampleWindField(field, targetDx),
);
const grid = composeWindGrid({
u: sampledU,
v: sampledV,
...(metadata
? {
scalar: scalar.values,
scalar: sampledScalar,
scalarScale: overlay === 'pressure' ? 0.01 : 1,
scalarOffset: overlay === 'temperature' ? -273.15 : 0,
}
+17 -22
View File
@@ -1,6 +1,6 @@
import { fetchText, fetchRange, loadOptionalScalar } from './gfs.js';
import { decodeWindGribMessage } from './decode.js';
import { resampleWeatherSnapshot, weatherScalarMetadata } from './grid.js';
import { fetchText, loadWeatherSnapshot } from './gfs.js';
import { decodeWindGribInWorker } from './decoder.js';
import { weatherScalarMetadata } from './grid.js';
/** Keyless ECMWF Open Data root for real-time IFS forecasts. */
const IFS_BASE = 'https://data.ecmwf.int/forecasts';
@@ -110,9 +110,10 @@ export async function fetchIfsWind({
fetchImpl = fetch,
now = () => Date.now(),
targetDx = 1,
decodeImpl = decodeWindGribMessage,
decodeImpl = decodeWindGribInWorker,
signal,
overlay = 'none',
windCache,
} = {}) {
weatherScalarMetadata(overlay);
const nowMs = now();
@@ -137,24 +138,18 @@ export async function fetchIfsWind({
/* A missing optional field does not discard valid wind. */
}
}
// All selected messages come from this one issue/valid-time object.
const load = async (range) =>
decodeImpl(
await fetchRange({ url: urls.grib, ...range, fetchImpl, signal }),
);
const [u, v, scalar] = await Promise.all([
load(ranges.u),
load(ranges.v),
loadOptionalScalar({
range: scalarRange,
url: urls.grib,
fetchImpl,
decodeImpl,
signal,
}),
]);
signal?.throwIfAborted();
const fields = resampleWeatherSnapshot({ u, v, scalar, overlay, targetDx });
const fields = await loadWeatherSnapshot({
model: 'ifs',
url: urls.grib,
ranges,
scalarRange,
overlay,
targetDx,
fetchImpl,
decodeImpl,
signal,
windCache,
});
const runIso = new Date(runMs).toISOString();
const validIso = new Date(runMs + forecastHour * 3600_000).toISOString();
return {
+11 -139
View File
@@ -51,7 +51,6 @@ import {
} from './detectionPolicy.js';
import { detectionBracketOpacity } from './detectionPresentation.js';
import { SurfaceOcclusion } from './detectionOcclusion.js';
import { createDetectionPaintCache } from './detectionPaintCache.js';
import { SceneOcclusionProbe } from './sceneOcclusionProbe.js';
import {
createCyberSonarSampler,
@@ -244,15 +243,6 @@ let _hostLane = null;
let _calloutLane = null;
let _transitBracketPaths = new Map();
let _transitBracketAlpha = 1;
/**
* What the last full sweep of the objects built, kept for the paints that
* follow it with the same view and the same objects. See `_drawOverlay`.
*/
const _paintCache = createDetectionPaintCache();
/** Scratch for `_paintViewOf`; the cache copies it when it keeps a paint. */
const _paintView = [];
/** Paints since `initDetection` that reused a kept sweep (diagnostics). */
let _paintReuseCount = 0;
/**
* Callouts solved by the sensor lane this frame, replayed by the callout lane.
* The array and its rows are POOLED — `_calloutCount` is the live length — so a
@@ -389,8 +379,6 @@ export function destroyDetection() {
_calloutLane?.unregister?.();
_calloutLane = null;
_transitBracketPaths.clear();
_paintCache.clear();
_paintReuseCount = 0;
_calloutCount = 0;
_viewer = null;
_layers = [];
@@ -655,7 +643,6 @@ function _publishDiagnostics() {
dataset.solveMs = String(_lastDiagnostics.solveMs ?? 0);
dataset.frameTotalMs = String(_lastDiagnostics.frameTotalMs ?? 0);
dataset.throttleSkipCount = String(_lastDiagnostics.throttleSkipCount ?? 0);
dataset.paintReuseCount = String(_lastDiagnostics.paintReuseCount ?? 0);
}
/**
@@ -731,7 +718,6 @@ function _applyModeState() {
_syncSurfaceVisibility();
_setLanesActive(false);
_labelArbiter.clear();
_paintCache.clear();
_surfaceOcclusion.clear();
_surfaceSuppressedKeys.clear();
_occlusionProbe.reset();
@@ -1273,53 +1259,15 @@ function _materializeCandidate(
};
}
/**
* Every value of the view that the object sweep in `_drawOverlay` and the
* callout candidates it builds read, in a fixed order, for
* `detectionPaintCache`. A value the sweep starts to read belongs here.
*/
function _paintViewOf(frame, keyholeOutsideOpacity, cyberMapActive) {
const view = _paintView;
const { keyhole, cameraPosition, viewProjectionMatrix, uiRects } = frame;
let n = 0;
view[n++] = _mode;
view[n++] = _theme;
view[n++] = _charWidth;
view[n++] = frame.width;
view[n++] = frame.height;
view[n++] = keyholeOutsideOpacity;
view[n++] = cyberMapActive;
view[n++] = keyhole?.centerX;
view[n++] = keyhole?.centerY;
view[n++] = keyhole?.radius;
view[n++] = keyhole?.featherPx;
view[n++] = cameraPosition?.x;
view[n++] = cameraPosition?.y;
view[n++] = cameraPosition?.z;
for (let i = 0; i < 16; i++) view[n++] = viewProjectionMatrix[i];
const rectCount = Array.isArray(uiRects) ? uiRects.length : -1;
view[n++] = rectCount;
for (let i = 0; i < rectCount; i++) {
const rect = uiRects[i];
view[n++] = rect?.x;
view[n++] = rect?.y;
view[n++] = rect?.w;
view[n++] = rect?.h;
}
view.length = n;
return view;
}
/**
* Paint detection against the shared host frame. Brackets remain broad and
* per-frame; rich placements rebuild only for selected/fading identities and
* for a bounded cohort on solve ticks.
*/
function _drawOverlay(frame) {
_transitBracketPaths.clear();
const { width, height } = frame;
if (!_ctx || width <= 0 || height <= 0) {
_transitBracketPaths.clear();
_paintCache.forget();
return {
didSolve: false,
solveMs: 0,
@@ -1361,8 +1309,6 @@ function _drawOverlay(frame) {
// callout lane repaints the last solve on frames the sensor lane skips,
// leaving rows here would strand the final callsigns on screen after the
// last data layer is switched off.
_transitBracketPaths.clear();
_paintCache.clear();
_calloutCount = 0;
_ctx.font = FONT;
_drawScanlines(width, height, now);
@@ -1415,9 +1361,6 @@ function _drawOverlay(frame) {
// every host lane. Detection retains the manual scalar projection below.
const occluder = frame.occluder;
const camPos = frame.cameraPosition;
const camX = camPos.x;
const camY = camPos.y;
const camZ = camPos.z;
// Answers to the occlusion tests started on earlier frames land here.
_surfaceOcclusion.beginFrame(now);
_occlusionProbe.beginFrame(_viewer?.scene, _surfaceOcclusion.now);
@@ -1447,7 +1390,7 @@ function _drawOverlay(frame) {
if (!_charWidth) _charWidth = _ctx.measureText('0000000000').width / 10 || 6;
const tiers = _theme.tiers || null;
const colorFor = (key) => (tiers && tiers[key]) || _theme.line;
let bracketPaths = new Map();
const bracketPaths = new Map();
// Global Context sonar echoes: a second bracket, banded by strength, around
// each flight or vessel a pulse front has just crossed.
// Resting pulses light nothing, so the per-contact echo work is skipped.
@@ -1472,41 +1415,14 @@ function _drawOverlay(frame) {
};
let visibleCount = 0;
let bracketOpacityCounts = { full: 0, partial: 0, hidden: 0 };
let aircraftBracketSectors = { left: 0, front: 0, right: 0 };
const bracketOpacityCounts = { full: 0, partial: 0, hidden: 0 };
const aircraftBracketSectors = { left: 0, front: 0, right: 0 };
let protectedVisibleCount = 0;
let candidateMap = new Map();
const candidateMap = new Map();
const cohortBuilders = shouldSolve ? new Map() : null;
const demandByLayer = shouldSolve ? new Map() : null;
let placementBuildCount = 0;
// The sweep below is the cost of this paint: every object projected, faded
// and given its bracket. What it builds depends only on the view and on the
// objects, so when both are exactly what the last sweep had, that sweep's
// paths, counts and callout candidates are used again and only the strokes
// and the callouts are repainted. Three things keep a paint from being
// reused or kept:
// - the Cyber sonar sweep and Global Context echoes change brackets with the
// clock alone;
// - a solve needs the sweep to gather its cohort;
// - a depth-tested contact gets its answer from the scene, not from here.
// A kept solve paint holds the solve's own candidates too, so the callouts a
// solve has just chosen are placed from the same map either way.
const paintKeepable = !sonarActive && !echoField;
const paintView = paintKeepable
? _paintViewOf(frame, keyholeOutsideOpacity, cyberMapActive)
: null;
const kept =
paintKeepable && !shouldSolve
? _paintCache.reuse(paintView, objects)
: null;
let depthTestedSwept = false;
let identityLayerId;
let renderSources;
let selectedSources;
if (!kept) _transitBracketPaths.clear();
const sweepCount = kept ? 0 : objects.length;
for (let i = 0; i < sweepCount; i++) {
for (let i = 0; i < objects.length; i++) {
const obj = objects[i];
if (!obj.position) continue;
// Skip objects occluded by the ellipsoid (behind the horizon)
@@ -1534,12 +1450,8 @@ function _drawOverlay(frame) {
let halfW;
let halfH;
if (obj.type === 'AIR') {
// Cartesian3.distance, without its two argument checks per object.
const dx = camX - px;
const dy = camY - py;
const dz = camZ - pz;
const bscale = nearFarScale(
Math.sqrt(dx * dx + dy * dy + dz * dz),
Cesium.Cartesian3.distance(camPos, obj.position),
BILL_NEAR,
BILL_NEAR_SCALE,
BILL_FAR,
@@ -1565,7 +1477,6 @@ function _drawOverlay(frame) {
// Something the scene drew in front of a depth-tested contact hides its
// bracket and callout too, on every frame, not only on solves.
if (obj.depthTested === true) {
depthTestedSwept = true;
const testedLayerId = obj._layerId || 'unknown';
const testedSourceId = obj.sourceId ?? obj.id ?? i;
const hidden = _surfaceOcclusion.isHidden(
@@ -1646,29 +1557,14 @@ function _drawOverlay(frame) {
if (!primary && !micro) continue;
const layerId = obj._layerId || 'unknown';
const sourceId = obj.sourceId ?? obj.id ?? i;
// Objects arrive a layer at a time, so the layer's sets are looked up once
// per run of objects, not once per object.
if (layerId !== identityLayerId) {
identityLayerId = layerId;
renderSources = renderIdentities.get(layerId);
selectedSources = selectedIdentities?.get(layerId);
}
const liveForRender = renderSources?.has(sourceId) || false;
const liveForRender = renderIdentities.get(layerId)?.has(sourceId) || false;
if (!shouldSolve && !liveForRender) continue;
obj._cohortSourceId = sourceId;
obj._cohortPriority = _semanticPriority(obj);
obj._cohortBand =
admissionAlpha >= 0.999 ? 8 : Math.floor(admissionAlpha * 8);
// The hash is of the identity alone, so it is worked out once per identity.
if (
obj._cohortHashLayerId !== layerId ||
obj._cohortHashSourceId !== sourceId
) {
obj._cohortHash = stableIdentityHash(layerId, sourceId);
obj._cohortHashLayerId = layerId;
obj._cohortHashSourceId = sourceId;
}
obj._cohortHash = stableIdentityHash(layerId, sourceId);
obj._candidateScreenX = sx;
obj._candidateScreenY = sy;
obj._candidateHalfW = halfW;
@@ -1682,7 +1578,8 @@ function _drawOverlay(frame) {
demandByLayer.set(layerId, (demandByLayer.get(layerId) || 0) + 1);
if (!cohortBuilders.has(layerId))
cohortBuilders.set(layerId, new BoundedCohort(256));
const incumbent = selectedSources?.has(sourceId) || false;
const incumbent =
selectedIdentities?.get(layerId)?.has(sourceId) || false;
cohortBuilders.get(layerId).consider(obj, incumbent);
}
@@ -1702,29 +1599,6 @@ function _drawOverlay(frame) {
}
// Start this frame's occlusion tests; nothing here waits for them.
_occlusionProbe.endFrame();
if (kept) {
_paintReuseCount++;
({
bracketPaths,
visibleCount,
bracketOpacityCounts,
aircraftBracketSectors,
protectedVisibleCount,
candidateMap,
} = kept);
} else if (paintKeepable && !depthTestedSwept) {
// `candidateMap` is kept by reference: a solve below adds to it.
_paintCache.keep(paintView, objects, {
bracketPaths,
visibleCount,
bracketOpacityCounts,
aircraftBracketSectors,
protectedVisibleCount,
candidateMap,
});
} else {
_paintCache.forget();
}
const altitude = _viewer?.camera?.positionCartographic?.height ?? 1e9;
const collectiveBudget = labelBudgetFor(altitude, _densityPct);
@@ -1908,7 +1782,6 @@ function _drawOverlay(frame) {
// drawing them cost it.
surfaceProbeTests: _occlusionProbe.lastFrameTests,
surfaceProbeMs: _occlusionProbe.lastFrameMs,
paintReuseCount: _paintReuseCount,
};
} else {
const selectedCount = _labelArbiter.selectedKeys.size;
@@ -1922,7 +1795,6 @@ function _drawOverlay(frame) {
_lastDiagnostics.aircraftBracketSectors = aircraftBracketSectors;
_lastDiagnostics.bracketPresentationOpacity = bracketPresentationOpacity;
_lastDiagnostics.protectedVisibleCount = protectedVisibleCount;
_lastDiagnostics.paintReuseCount = _paintReuseCount;
}
_drawScanlines(width, height, now);
-231
View File
@@ -1661,234 +1661,3 @@ test('a WebGL 1 scene reads depth synchronously and reports it once', () => {
env.cleanup();
}
});
// --- Reusing a paint -------------------------------------------------------
/**
* Contacts as the fleet layers return them: the same objects on every paint,
* changed in place. `fresh` returns equal copies instead, which no paint can
* reuse, so the two can be compared.
*/
function steadyContacts({ fresh = false } = {}) {
const objects = [
{ position: new Cesium.Cartesian3(-0.4, 0.2, 6_356_752), sourceId: 'air-1', id: 'AIRONE', metric: 'FL350', type: 'AIR' },
{ position: new Cesium.Cartesian3(0.4, -0.2, 6_356_752), sourceId: 'sat-1', id: 'SATONE', metric: '412KM', type: 'SAT' },
{ position: new Cesium.Cartesian3(0.1, 0.5, 6_356_752), sourceId: 'sea-1', id: 'SEAONE', metric: '12 kn', type: 'SEA' },
{ position: new Cesium.Cartesian3(-0.2, -0.5, 6_356_752), sourceId: 'mil-1', id: 'MILONE', metric: 'FL410', type: 'AIR', tier: 'military' },
];
let calls = 0;
return {
objects,
get calls() { return calls; },
layer: {
id: 'flights',
getDetectableObjects() {
calls += 1;
return fresh
? objects.map((object) => ({ ...object, position: Cesium.Cartesian3.clone(object.position) }))
: objects;
},
},
};
}
/** Everything painted since the trace was last emptied, with each path as its points. */
function takePainted(env) {
const painted = env.paintTrace.map((call) =>
call.map((value) => (value instanceof MockPath2D ? value.points.map((point) => [...point]) : value)));
env.paintTrace.length = 0;
return painted;
}
const bracketPointCount = (env) => env.detectionCtx.calls
.filter(([name]) => name === 'stroke')
.reduce((sum, [, path]) => sum + (path?.points?.length ?? 0), 0);
test('a paint with the same view and the same contacts reuses the last sweep, and paints exactly what a full sweep paints', () => {
// The same frames twice: once with contacts a paint can reuse, once with
// equal copies it cannot. Every canvas call must match, frame for frame.
const frames = (fresh) => {
const env = installEnvironment();
const contacts = steadyContacts({ fresh });
const painted = [];
try {
initWorldOverlay(env.viewer);
initDetection(env.viewer, [contacts.layer], () => {});
setMode('DENSE');
settleFrame(env);
painted.push(takePainted(env));
for (let frame = 1; frame <= 40; frame++) {
// Through the run: a contact moves, one is relabelled, one changes
// tier, and the camera turns.
if (frame === 9) contacts.objects[0].position.x += 0.05;
if (frame === 17) contacts.objects[1].metric = '398KM';
if (frame === 23) contacts.objects[2].tier = 'military';
if (frame === 31) env.viewer.camera.viewMatrix[12] = 0.03;
nextFrame(env);
painted.push(takePainted(env));
}
return { painted, reused: getDetectionDiagnostics().paintReuseCount };
} finally {
env.cleanup();
}
};
const reusing = frames(false);
const sweeping = frames(true);
assert.equal(sweeping.reused, 0, 'copies are swept on every paint');
// 40 frames 16 ms apart: a solve about every eighth, four changes, and the
// rest reuse the sweep before them.
assert.ok(reusing.reused >= 28, `${reusing.reused} of 40 paints reused a sweep`);
assert.equal(reusing.painted.length, sweeping.painted.length);
reusing.painted.forEach((frame, index) => {
assert.ok(frame.some(([target, name]) => target === 'detection' && name === 'stroke'), `frame ${index} strokes brackets`);
assert.deepEqual(frame, sweeping.painted[index], `frame ${index}`);
});
assert.ok(
reusing.painted.some((frame) => frame.some(([, name, text]) => name === 'fillText' && text === 'AIRONE')),
'callouts were painted during the run',
);
});
test('a reused paint builds no bracket paths and still strokes the kept ones', () => {
const env = installEnvironment();
const contacts = steadyContacts();
const builtPaths = [];
class CountingPath2D extends MockPath2D {
constructor() { super(); builtPaths.push(this); }
}
try {
globalThis.Path2D = CountingPath2D;
initWorldOverlay(env.viewer);
initDetection(env.viewer, [contacts.layer], () => {});
setMode('DENSE');
settleFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 0);
const built = builtPaths.length;
const points = builtPaths.reduce((sum, path) => sum + path.points.length, 0);
assert.ok(points >= 4 * 12, 'four brackets of twelve points were built');
env.detectionCtx.calls.length = 0;
nextFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 1);
assert.equal(builtPaths.length, built, 'no path was built');
assert.equal(builtPaths.reduce((sum, path) => sum + path.points.length, 0), points, 'no point was added to a kept path');
assert.equal(bracketPointCount(env), 4 * 12, 'the kept brackets are stroked');
assert.equal(getDetectionDiagnostics().visibleCount, 4);
assert.equal(getDetectionDiagnostics().placementBuildCount, 0, 'no placement was built on the reused paint');
} finally {
env.cleanup();
}
});
test('any change to a contact or to the view is swept on the very next paint', () => {
const changes = {
'a contact moves': (contacts) => { contacts.objects[0].position.y -= 0.01; },
'a contact gets a new position object': (contacts) => {
contacts.objects[0].position = new Cesium.Cartesian3(0.3, 0.3, 6_356_752);
},
'a contact is renamed': (contacts) => { contacts.objects[1].id = 'SATTWO'; },
'a contact reading changes': (contacts) => { contacts.objects[0].metric = 'FL360'; },
'a contact becomes the tracked one': (contacts) => { contacts.objects[0].skipLabel = true; },
'a contact changes tier': (contacts) => { contacts.objects[3].tier = undefined; },
'a contact changes type': (contacts) => { contacts.objects[2].type = 'VEH'; },
'a contact joins': (contacts) => {
contacts.objects.push({ position: new Cesium.Cartesian3(0.6, 0.6, 6_356_752), sourceId: 'air-2', id: 'AIRTWO', type: 'AIR' });
},
'a contact leaves': (contacts) => { contacts.objects.pop(); },
'a contact is replaced by an equal one': (contacts) => { contacts.objects[1] = { ...contacts.objects[1] }; },
'two contacts swap places in the list': (contacts) => { contacts.objects.reverse(); },
'the camera turns': (contacts, env) => { env.viewer.camera.viewMatrix[13] = 0.02; },
'the camera moves': (contacts, env) => {
env.viewer.camera.positionWC = new Cesium.Cartesian3(0, 100, 10_000_000);
},
'the style changes': () => { setDetectionStyle('thermal'); },
'the profile changes': () => { setMode('SPARSE'); },
};
for (const [name, change] of Object.entries(changes)) {
const env = installEnvironment();
const contacts = steadyContacts();
try {
initWorldOverlay(env.viewer);
initDetection(env.viewer, [contacts.layer], () => {});
setDetectionStyle('normal');
setMode('DENSE');
settleFrame(env);
nextFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 1, `${name}: steady paints reuse`);
change(contacts, env);
nextFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 1, `${name}: swept again`);
assert.equal(getDetectionDiagnostics().observationCount, contacts.objects.length, name);
} finally {
env.cleanup();
}
}
});
test('a depth-tested contact, the sonar sweep and echoes are swept on every paint', () => {
const run = ({ depthTested = false, sonar = false, echo = false } = {}) => {
const env = installEnvironment();
const contacts = steadyContacts();
if (depthTested) {
contacts.objects[0].type = 'VEH';
contacts.objects[0].depthTested = true;
}
if (sonar) {
env.document.documentElement = {
dataset: { uiTheme: 'cyber', cyberSonarOpacity: '35' },
style: { setProperty() {}, removeProperty() {} },
};
const hud = env.document.createElement('div');
hud.id = 'intel-hud';
hud.classList = { contains: (className) => className === 'active' };
env.document.body.appendChild(hud);
}
try {
if (echo)
setAwarenessEchoField({
center: Cesium.Cartesian3.fromDegrees(0, 89, 0),
radiusM: 250_000,
pulse: { count: 1000, periodMs: 1000, startFraction: 0 },
echo: { layerIds: ['flights'], fadeMs: 60_000, color: 'rgb(1, 2, 3)', spreadPx: 8, lineWidth: 3.25 },
});
initWorldOverlay(env.viewer);
initDetection(env.viewer, [contacts.layer], () => {});
setMode('DENSE');
settleFrame(env);
const sweeps = contacts.calls;
for (let i = 0; i < 30; i++) nextFrame(env);
assert.ok(contacts.calls > sweeps, 'frames painted');
return getDetectionDiagnostics().paintReuseCount;
} finally {
clearAwarenessEchoField();
env.cleanup();
}
};
assert.ok(run() > 15, 'the same contacts with none of these are reused');
assert.equal(run({ depthTested: true }), 0, 'depth-tested');
assert.equal(run({ sonar: true }), 0, 'sonar');
assert.equal(run({ echo: true }), 0, 'echo');
});
test('turning detection off lets go of the kept sweep', () => {
const env = installEnvironment();
const contacts = steadyContacts();
try {
initWorldOverlay(env.viewer);
initDetection(env.viewer, [contacts.layer], () => {});
setMode('DENSE');
settleFrame(env);
nextFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 1);
setMode('OFF');
setMode('DENSE');
// The first paint back is a solve, and the one after it reuses that.
nextFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 1);
nextFrame(env);
assert.equal(getDetectionDiagnostics().paintReuseCount, 2);
} finally {
env.cleanup();
}
});
-206
View File
@@ -1,206 +0,0 @@
/**
* @module detectionPaintCache
* @description Lets the detection overlay keep what one paint built (bracket
* paths, counts, callout candidates) and use it again on the next paint when
* that paint would build exactly the same thing.
*
* Nothing announces a change to this cache. Each paint hands over the values
* the sweep reads from the view and the objects the layers returned, and the
* cache compares them, value by value, with what the kept paint was built
* from. A layer therefore cannot leave a stale picture behind by forgetting to
* say it changed: an object that moved, appeared, left or was relabelled is
* seen here, and the sweep runs again.
*
* Pure: no Cesium, no DOM.
*/
/**
* The object fields the detection sweep and its callout candidates read.
* Position is compared separately, by coordinate. `sameFields` and
* `recordFields` below name each of these outright, in this order, because a
* field read by a name held in a variable costs several times as much, and
* this runs for every object on every paint.
*/
export const PAINTED_OBJECT_FIELDS = Object.freeze([
'_layerId',
'type',
'tier',
'skipLabel',
'id',
'metric',
'sourceId',
'bracketHalfWidth',
'bracketHalfHeight',
'depthTested',
'cardSourceId',
'cardEntryId',
]);
/**
* After this many sweeps in a row with no reuse between them, the objects are
* only recorded on every RECORD_STRIDE-th sweep. A scene whose objects move on
* every paint can never reuse one, so recording each of them would be work
* with no return; a scene that settles is picked up again within RECORD_STRIDE
* paints.
*/
export const SWEEPS_BEFORE_BACKOFF = 8;
export const RECORD_STRIDE = 8;
const FIELD_COUNT = PAINTED_OBJECT_FIELDS.length;
function sameFields(object, fields, f) {
return (
object._layerId === fields[f] &&
object.type === fields[f + 1] &&
object.tier === fields[f + 2] &&
object.skipLabel === fields[f + 3] &&
object.id === fields[f + 4] &&
object.metric === fields[f + 5] &&
object.sourceId === fields[f + 6] &&
object.bracketHalfWidth === fields[f + 7] &&
object.bracketHalfHeight === fields[f + 8] &&
object.depthTested === fields[f + 9] &&
object.cardSourceId === fields[f + 10] &&
object.cardEntryId === fields[f + 11]
);
}
/** Most fields are what they were at the last sweep; only the others are written. */
function recordFields(object, fields, f) {
let value;
if ((value = object._layerId) !== fields[f]) fields[f] = value;
if ((value = object.type) !== fields[f + 1]) fields[f + 1] = value;
if ((value = object.tier) !== fields[f + 2]) fields[f + 2] = value;
if ((value = object.skipLabel) !== fields[f + 3]) fields[f + 3] = value;
if ((value = object.id) !== fields[f + 4]) fields[f + 4] = value;
if ((value = object.metric) !== fields[f + 5]) fields[f + 5] = value;
if ((value = object.sourceId) !== fields[f + 6]) fields[f + 6] = value;
if ((value = object.bracketHalfWidth) !== fields[f + 7])
fields[f + 7] = value;
if ((value = object.bracketHalfHeight) !== fields[f + 8])
fields[f + 8] = value;
if ((value = object.depthTested) !== fields[f + 9]) fields[f + 9] = value;
if ((value = object.cardSourceId) !== fields[f + 10]) fields[f + 10] = value;
if ((value = object.cardEntryId) !== fields[f + 11]) fields[f + 11] = value;
}
/**
* @returns {{
* reuse: (view: Array, objects: Array<object>) => (object|null),
* keep: (view: Array, objects: Array<object>, built: object) => boolean,
* forget: () => void,
* clear: () => void,
* }}
*/
export function createDetectionPaintCache() {
let built = null;
let viewKept = [];
let count = 0;
let references = [];
let fields = [];
let coordinates = new Float64Array(0);
let positioned = new Uint8Array(0);
let sweepsSinceReuse = 0;
function sameView(view) {
if (view.length !== viewKept.length) return false;
for (let i = 0; i < view.length; i++)
if (view[i] !== viewKept[i]) return false;
return true;
}
function sameObjects(objects) {
if (objects.length !== count) return false;
for (let i = 0, c = 0, f = 0; i < count; i++, c += 3, f += FIELD_COUNT) {
const object = objects[i];
if (object !== references[i]) return false;
const position = object.position;
if (position) {
// A coordinate that is not a number never equals the kept one, so such
// an object is swept again on every paint.
if (
positioned[i] !== 1 ||
position.x !== coordinates[c] ||
position.y !== coordinates[c + 1] ||
position.z !== coordinates[c + 2]
)
return false;
} else if (positioned[i] !== 0) {
return false;
}
if (!sameFields(object, fields, f)) return false;
}
return true;
}
return {
/**
* What the kept paint built, when `view` and `objects` are what it was
* built from; otherwise null.
* @param {Array} view - Every value of the view the sweep reads, in a fixed order.
* @param {Array<object>} objects - The objects the layers returned this paint.
*/
reuse(view, objects) {
if (!built || !sameView(view) || !sameObjects(objects)) return null;
sweepsSinceReuse = 0;
return built;
},
/**
* Keep `result` as the result of sweeping `objects` under `view`.
* @returns {boolean} False when recording was skipped (see RECORD_STRIDE).
*/
keep(view, objects, result) {
sweepsSinceReuse++;
if (
sweepsSinceReuse > SWEEPS_BEFORE_BACKOFF &&
(sweepsSinceReuse - SWEEPS_BEFORE_BACKOFF) % RECORD_STRIDE !== 0
) {
built = null;
return false;
}
viewKept = view.slice();
count = objects.length;
if (positioned.length < count) {
// Room to spare, so a feed that grows by a few contacts a poll does
// not allocate on each of them.
const room = Math.ceil(count * 1.25);
coordinates = new Float64Array(room * 3);
positioned = new Uint8Array(room);
references = new Array(room).fill(undefined);
fields = new Array(room * FIELD_COUNT).fill(undefined);
}
for (let i = 0, c = 0, f = 0; i < count; i++, c += 3, f += FIELD_COUNT) {
const object = objects[i];
if (references[i] !== object) references[i] = object;
const position = object.position;
positioned[i] = position ? 1 : 0;
if (position) {
coordinates[c] = position.x;
coordinates[c + 1] = position.y;
coordinates[c + 2] = position.z;
}
recordFields(object, fields, f);
}
built = result;
return true;
},
/** Reuse nothing until the next `keep`. */
forget() {
built = null;
},
/** Forget the kept paint and let go of its objects. */
clear() {
built = null;
viewKept = [];
count = 0;
references = [];
fields = [];
coordinates = new Float64Array(0);
positioned = new Uint8Array(0);
sweepsSinceReuse = 0;
},
};
}
-197
View File
@@ -1,197 +0,0 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import {
createDetectionPaintCache,
PAINTED_OBJECT_FIELDS,
RECORD_STRIDE,
SWEEPS_BEFORE_BACKOFF,
} from './detectionPaintCache.js';
const contacts = () => [
{
position: { x: 1, y: 2, z: 3 },
_layerId: 'flights',
sourceId: 'a',
id: 'A',
metric: 'FL300',
type: 'AIR',
},
{
position: { x: 4, y: 5, z: 6 },
_layerId: 'vessels',
sourceId: 'b',
id: 'B',
type: 'SEA',
},
{ _layerId: 'cctv', sourceId: 'c', id: 'C', type: 'CAM' },
];
const view = () => ['DENSE', 800, 600, 0.25, 1, 0, 0, 1];
function kept() {
const cache = createDetectionPaintCache();
const objects = contacts();
const built = { paths: 'kept' };
assert.equal(cache.keep(view(), objects, built), true);
return { cache, objects, built };
}
test('the kept sweep is handed back for the same view and the same objects', () => {
const { cache, objects, built } = kept();
assert.equal(cache.reuse(view(), objects), built);
assert.equal(cache.reuse(view(), objects), built, 'and again');
// The same objects in a new list are still the same objects.
assert.equal(cache.reuse(view(), [...objects]), built);
});
test('nothing is handed back before a sweep is kept, or after it is forgotten', () => {
const cache = createDetectionPaintCache();
const objects = contacts();
assert.equal(cache.reuse(view(), objects), null);
cache.keep(view(), objects, {});
cache.forget();
assert.equal(cache.reuse(view(), objects), null);
cache.keep(view(), objects, {});
cache.clear();
assert.equal(cache.reuse(view(), objects), null);
});
test('any value of the view that differs is a miss', () => {
for (let i = 0; i < view().length; i++) {
const { cache, objects } = kept();
const changed = view();
changed[i] = typeof changed[i] === 'number' ? changed[i] + 1e-9 : 'SPARSE';
assert.equal(cache.reuse(changed, objects), null, `view value ${i}`);
}
const { cache, objects } = kept();
assert.equal(cache.reuse([...view(), 0], objects), null, 'one value more');
assert.equal(
cache.reuse(view().slice(0, -1), objects),
null,
'one value fewer',
);
});
test('a view value that is not a number never matches itself', () => {
const cache = createDetectionPaintCache();
const objects = contacts();
cache.keep([NaN], objects, {});
assert.equal(cache.reuse([NaN], objects), null);
});
test('the view is copied when kept, so the caller may write the next one over it', () => {
const cache = createDetectionPaintCache();
const objects = contacts();
const scratch = view();
const built = {};
cache.keep(scratch, objects, built);
scratch[1] = 1024;
assert.equal(cache.reuse(scratch, objects), null);
scratch[1] = 800;
assert.equal(cache.reuse(scratch, objects), built);
});
test('a coordinate changed in place is a miss, however small', () => {
for (const axis of ['x', 'y', 'z']) {
const { cache, objects } = kept();
objects[1].position[axis] += 1e-9;
assert.equal(cache.reuse(view(), objects), null, axis);
}
});
test('a position object swapped for an equal one is still the same paint', () => {
const { cache, objects, built } = kept();
objects[0].position = { x: 1, y: 2, z: 3 };
assert.equal(cache.reuse(view(), objects), built);
});
test('a position that appears, goes away or is not a number is a miss', () => {
let { cache, objects } = kept();
objects[2].position = { x: 0, y: 0, z: 0 };
assert.equal(cache.reuse(view(), objects), null, 'appears');
({ cache, objects } = kept());
objects[0].position = null;
assert.equal(cache.reuse(view(), objects), null, 'goes away');
cache = createDetectionPaintCache();
objects = contacts();
objects[0].position.x = NaN;
cache.keep(view(), objects, {});
assert.equal(cache.reuse(view(), objects), null, 'not a number');
});
test('every field the sweep reads is compared', () => {
for (const field of PAINTED_OBJECT_FIELDS) {
const { cache, objects, built } = kept();
const before = objects[0][field];
objects[0][field] = before === 'changed' ? 'again' : 'changed';
assert.equal(cache.reuse(view(), objects), null, field);
objects[0][field] = before;
assert.equal(cache.reuse(view(), objects), built, `${field} put back`);
}
});
test('a field the sweep does not read may change freely', () => {
const { cache, objects, built } = kept();
objects[0]._candidateScreenX = 412;
objects[0]._cohortHash = 7;
assert.equal(cache.reuse(view(), objects), built);
});
test('a different object, count or order is a miss', () => {
let { cache, objects } = kept();
assert.equal(cache.reuse(view(), contacts()), null, 'equal copies');
({ cache, objects } = kept());
assert.equal(cache.reuse(view(), objects.slice(0, 2)), null, 'one fewer');
assert.equal(cache.reuse(view(), [...objects, objects[0]]), null, 'one more');
({ cache, objects } = kept());
assert.equal(
cache.reuse(view(), [objects[1], objects[0], objects[2]]),
null,
'order',
);
});
test('keeping again replaces what was kept, for more or fewer objects', () => {
const { cache, objects, built } = kept();
const more = [...objects, ...contacts(), ...contacts()];
const builtMore = {};
cache.keep(view(), more, builtMore);
assert.equal(cache.reuse(view(), objects), null);
assert.equal(cache.reuse(view(), more), builtMore);
cache.keep(view(), objects, built);
assert.equal(cache.reuse(view(), more), null);
assert.equal(cache.reuse(view(), objects), built);
});
test('objects that change on every paint are recorded only now and then', () => {
const cache = createDetectionPaintCache();
const objects = contacts();
const recorded = [];
const sweep = () => {
objects[0].position.x += 1;
assert.equal(cache.reuse(view(), objects), null);
recorded.push(cache.keep(view(), objects, {}));
};
for (let i = 0; i < SWEEPS_BEFORE_BACKOFF; i++) sweep();
assert.ok(recorded.every(Boolean), 'the first sweeps are all recorded');
recorded.length = 0;
for (let i = 0; i < RECORD_STRIDE * 5; i++) sweep();
assert.equal(recorded.filter(Boolean).length, 5, 'then one in every stride');
// A skipped sweep leaves nothing behind to reuse by mistake.
const built = {};
let wasRecorded = cache.keep(view(), objects, built);
assert.equal(wasRecorded, false);
assert.equal(cache.reuse(view(), objects), null);
// Once the objects hold still, the next recorded sweep is reused, and
// every sweep after a reuse is recorded again.
while (!wasRecorded) wasRecorded = cache.keep(view(), objects, built);
assert.equal(cache.reuse(view(), objects), built);
for (let i = 0; i < SWEEPS_BEFORE_BACKOFF; i++)
assert.equal(cache.keep(view(), objects, built), true);
});
+517
View File
@@ -0,0 +1,517 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { spawnSync } from 'node:child_process';
import { createRequire } from 'node:module';
import { windProxy } from '../../server/providers/wind.js';
import { decodeWindGribMessage } from '../../server/providers/wind/decode.js';
// Imported per test so a missing worker decoder fails those tests, not the file.
const createWindDecoder = async (options) =>
(await import('../../server/providers/wind/decoder.js')).createWindDecoder(
options,
);
const require = createRequire(import.meta.url);
/** Whether this thread has loaded ecCodes (a CommonJS package). */
const ecCodesLoadedHere = () =>
Object.keys(require.cache).some((file) => file.includes('eccodes-wasm'));
const DECODE_MODULE = new URL(
'../../server/providers/wind/decode.js',
import.meta.url,
).href;
/**
* A worker that runs the provider's real serving loop over a stand-in for
* ecCodes. The first byte selects the behaviour, the second a synchronous busy
* time in tens of milliseconds, which is what a WASM decode looks like to the
* thread that runs it.
*/
const FAKE_WORKER = new URL(
`data:text/javascript,${encodeURIComponent(`
import { parentPort } from 'node:worker_threads';
import { serveWindDecodes } from ${JSON.stringify(DECODE_MODULE)};
serveWindDecodes(parentPort, (bytes) => {
if (bytes[0] === 1) throw new Error('bad message');
if (bytes[0] === 2) {
setTimeout(() => { throw new Error('crash'); });
return new Promise(() => {});
}
if (bytes[0] === 3) process.exit(1);
if (bytes[0] === 4) for (;;);
const until = Date.now() + (bytes[1] || 0) * 10;
while (Date.now() < until);
return {
ni: 4, nj: 3, lo1: 0, la1: 90, di: 90, dj: 90,
values: Float64Array.from({ length: 12 }, (_, i) => bytes[0] + i / 8),
units: 'm/s',
};
});
`)}`,
);
/** Run `work` while a 5 ms heartbeat records the longest main-thread stall. */
async function longestStall(work) {
let last = performance.now();
let gap = 0;
const beat = setInterval(() => {
const t = performance.now();
gap = Math.max(gap, t - last);
last = t;
}, 5);
try {
const started = performance.now();
const value = await work();
return { value, gap, elapsed: performance.now() - started };
} finally {
clearInterval(beat);
}
}
function install(plugin) {
let handler;
plugin.configureServer({
middlewares: { use: (_route, fn) => (handler = fn) },
});
return async (url = '/') => {
const res = {
writeHead(status) {
this.statusCode = status;
},
end(body) {
this.body = body;
},
};
await handler({ url, method: 'GET' }, res);
return res;
};
}
/** Manifest (minus fetchedAt) and grid bytes exactly as a client receives them. */
async function served(request, url) {
const { fetchedAt, ...manifest } = JSON.parse((await request(url)).body);
const grid = manifest.gridUrl
? (await request(manifest.gridUrl.replace('/api/wind', ''))).body
: null;
return { manifest, grid };
}
/** One GRIB2 message on a regular lat/lon grid with 16-bit simple packing. */
function syntheticGrib({ category, number, surface, level, values }) {
const [ni, nj] = [8, 5];
const u8 = (...bytes) => Buffer.from(bytes);
const u16 = (value) => {
const out = Buffer.alloc(2);
out.writeUInt16BE(value);
return out;
};
const u32 = (value) => {
const out = Buffer.alloc(4);
out.writeUInt32BE(value >>> 0);
return out;
};
const section = (id, ...parts) => {
const body = Buffer.concat(parts);
return Buffer.concat([u32(body.length + 5), u8(id), body]);
};
const reference = Buffer.alloc(4);
reference.writeFloatBE(Math.min(...values) * 10);
const packed = Buffer.alloc(values.length * 2);
values.forEach((value, index) =>
packed.writeUInt16BE(
Math.round(value * 10 - reference.readFloatBE()),
index * 2,
),
);
const rest = Buffer.concat([
section(1, u16(7), u16(0), u8(2, 1, 1), u16(2026), u8(9, 14, 6, 0, 0, 0, 1)),
section(
3,
u8(0),
u32(ni * nj),
u8(0, 0),
u16(0), // template 3.0, regular lat/lon
u8(6, 0),
u32(0),
u8(0),
u32(0),
u8(0),
u32(0),
u32(ni),
u32(nj),
u32(0),
u32(0xffffffff),
u32(90e6), // first point 90N 0E, last point 90S 315E, 45 degree steps
u32(0),
u8(48),
u32(90e6 | 0x80000000),
u32(315e6),
u32(45e6),
u32(45e6),
u8(0),
),
section(
4,
u16(0),
u16(0), // template 4.0
u8(category, number, 2, 0, 0),
u16(0),
u8(0, 1),
u32(0),
u8(surface, 0),
u32(level),
u8(255, 255),
u32(0xffffffff),
),
// Template 5.0: value = (reference + packed) / 10.
section(5, u32(ni * nj), u16(0), reference, u16(0), u16(1), u8(16, 0)),
section(6, u8(255)),
section(7, packed),
Buffer.from('7777'),
]);
return Buffer.concat([
Buffer.from('GRIB'),
u8(0, 0, 0, 2),
u32(0),
u32(16 + rest.length),
rest,
]);
}
const field = (seed, scale) =>
Array.from({ length: 40 }, (_, i) => seed + Math.sin(i * 1.7) * scale);
const MESSAGES = {
u: syntheticGrib({
category: 2,
number: 2,
surface: 103,
level: 10,
values: field(3, 14),
}),
v: syntheticGrib({
category: 2,
number: 3,
surface: 103,
level: 10,
values: field(-2, 9),
}),
tmp: syntheticGrib({
category: 0,
number: 0,
surface: 103,
level: 2,
values: field(281, 25),
}),
prmsl: syntheticGrib({
category: 3,
number: 1,
surface: 101,
level: 0,
values: field(101300, 2400),
}),
};
/** A GFS object holding the four synthetic messages, plus its `.idx`. */
function syntheticUpstream({ log = [], corrupt = () => false } = {}) {
const order = ['u', 'v', 'tmp', 'prmsl'];
const offsets = [0];
for (const name of order) offsets.push(offsets.at(-1) + MESSAGES[name].length);
const file = Buffer.concat(order.map((name) => MESSAGES[name]));
const idx = [
`1:${offsets[0]}:d=2026091406:UGRD:10 m above ground:6 hour fcst:`,
`2:${offsets[1]}:d=2026091406:VGRD:10 m above ground:6 hour fcst:`,
`3:${offsets[2]}:d=2026091406:TMP:2 m above ground:6 hour fcst:`,
`4:${offsets[3]}:d=2026091406:PRMSL:mean sea level:6 hour fcst:`,
`5:${offsets[4]}:d=2026091406:HGT:surface:6 hour fcst:`,
].join('\n');
const ranges = Object.fromEntries(
order.map((name, i) => [name, `bytes=${offsets[i]}-${offsets[i + 1] - 1}`]),
);
const fetchImpl = async (url, options = {}) => {
const range = options.headers?.Range;
log.push(range ?? 'idx');
if (!range) return new Response(idx, { status: 200 });
const [, start, end] = /bytes=(\d+)-(\d+)/.exec(range);
const name = order.find((key) => ranges[key] === range);
const body = Buffer.from(file.subarray(Number(start), Number(end) + 1));
if (corrupt(name)) body.fill(0);
return new Response(body, { status: 206 });
};
return { fetchImpl, ranges, log };
}
const NOON = Date.UTC(2026, 8, 14, 12);
const KEYS = ['/', '/?overlay=temperature', '/?overlay=pressure'];
test('a blocking decode runs off the server thread', async (t) => {
const decoder = await createWindDecoder({ workerUrl: FAKE_WORKER });
t.after(() => decoder.close());
// Warm the worker so its start-up is not counted, then block it for 300 ms.
await decoder.decode(Uint8Array.of(5, 0));
const { value, gap, elapsed } = await longestStall(() =>
decoder.decode(Uint8Array.of(7, 30), { targetDx: 90 }),
);
assert.ok(elapsed >= 280, `decode took ${elapsed} ms`);
assert.ok(gap < 150, `server thread stalled for ${gap} ms`);
// The worker validated and sampled the field; only the samples came back.
assert.equal(value.values, undefined);
assert.equal(value.dx, 90);
assert.ok(value.samples instanceof Float64Array);
assert.equal(value.samples.length, 4 * 3);
assert.equal(value.samples[0], 7);
});
test('the default wind provider decodes in the worker and matches the synchronous path byte for byte', async () => {
const outputs = [];
for (const options of [
{}, // default: ecCodes in the worker
{ decodeImpl: (bytes) => decodeWindGribMessage(bytes) }, // on this thread
]) {
const request = install(
windProxy({
now: () => NOON,
targetDx: 30,
fetchImpl: syntheticUpstream().fetchImpl,
...options,
}),
);
const out = [];
for (const key of KEYS) out.push(await served(request, key));
// The default path never brings ecCodes into this thread; the injected
// synchronous decoder does, which also proves the probe can see it.
assert.equal(ecCodesLoadedHere(), outputs.length === 1);
outputs.push(out);
}
const [worker, sync] = outputs;
for (let i = 0; i < KEYS.length; i += 1) {
assert.equal(worker[i].manifest.unavailable, false, KEYS[i]);
assert.deepEqual(worker[i].manifest, sync[i].manifest, KEYS[i]);
assert.ok(worker[i].grid.equals(sync[i].grid), `${KEYS[i]} grid bytes`);
}
assert.equal(worker[0].grid.length, 12 * 7 * 4 * 2);
assert.equal(worker[1].grid.length, 12 * 7 * 4 * 3);
assert.equal(worker[1].manifest.scalar.kind, 'temperature');
assert.equal(worker[2].manifest.scalar.kind, 'pressure');
});
test('partial and failed refreshes match the synchronous path through the worker', async () => {
for (const [bad, expected] of [
['tmp', { scalarError: 'Temperature field unavailable' }],
['u', { unavailable: true }],
]) {
const outputs = [];
for (const options of [
{},
{ decodeImpl: (bytes) => decodeWindGribMessage(bytes) },
]) {
const request = install(
windProxy({
now: () => NOON,
targetDx: 30,
fetchImpl: syntheticUpstream({ corrupt: (name) => name === bad })
.fetchImpl,
...options,
}),
);
outputs.push(await served(request, '/?overlay=temperature'));
}
const [worker, sync] = outputs;
assert.deepEqual(worker.manifest, sync.manifest);
for (const [name, value] of Object.entries(expected))
assert.equal(worker.manifest[name], value);
if (sync.grid) assert.ok(worker.grid.equals(sync.grid));
else assert.equal(worker.grid, null);
}
});
test('an overlay change within the same forecast object fetches and decodes only the scalar', async () => {
let clock = NOON;
let decodes = 0;
const upstream = syntheticUpstream();
const decodeImpl = (bytes) => {
decodes += 1;
return decodeWindGribMessage(bytes);
};
const request = install(
windProxy({
now: () => clock,
targetDx: 30,
fetchImpl: upstream.fetchImpl,
decodeImpl,
}),
);
const none = await served(request, '/');
assert.deepEqual(upstream.log, ['idx', upstream.ranges.u, upstream.ranges.v]);
assert.equal(decodes, 2);
upstream.log.length = 0;
const temperature = await served(request, '/?overlay=temperature');
assert.deepEqual(upstream.log, ['idx', upstream.ranges.tmp]);
assert.equal(decodes, 3);
upstream.log.length = 0;
const pressure = await served(request, '/?overlay=pressure');
assert.deepEqual(upstream.log, ['idx', upstream.ranges.prmsl]);
assert.equal(decodes, 4);
// Reused wind gives exactly what a server that fetched everything gives.
const fresh = install(
windProxy({
now: () => NOON,
targetDx: 30,
fetchImpl: syntheticUpstream().fetchImpl,
decodeImpl: (bytes) => decodeWindGribMessage(bytes),
}),
);
for (const [url, reused] of [
['/?overlay=temperature', temperature],
['/?overlay=pressure', pressure],
]) {
const full = await served(fresh, url);
assert.deepEqual(reused.manifest, full.manifest);
assert.ok(reused.grid.equals(full.grid));
}
assert.ok(
temperature.grid.subarray(0, none.grid.length).equals(none.grid),
'the overlay grid starts with the same U and V',
);
// The next forecast step is a different object: wind is fetched again.
clock += 3600_000;
upstream.log.length = 0;
await served(request, '/?overlay=temperature');
assert.deepEqual(upstream.log, [
'idx',
upstream.ranges.u,
upstream.ranges.v,
upstream.ranges.tmp,
]);
});
test('wind that failed its checks is never retained for a later overlay', async () => {
const upstream = syntheticUpstream({ corrupt: (name) => name === 'v' });
const request = install(
windProxy({
now: () => NOON,
targetDx: 30,
fetchImpl: upstream.fetchImpl,
decodeImpl: (bytes) => decodeWindGribMessage(bytes),
}),
);
assert.equal((await served(request, '/')).manifest.unavailable, true);
upstream.log.length = 0;
const overlay = await served(request, '/?overlay=temperature');
assert.equal(overlay.manifest.unavailable, true);
assert.ok(upstream.log.includes(upstream.ranges.u), 'U is fetched again');
});
test('a throwing, crashing, exiting or hung decode rejects its job and the next decode recovers', async (t) => {
const decoder = await createWindDecoder({
workerUrl: FAKE_WORKER,
jobTimeoutMs: 400,
});
t.after(() => decoder.close());
const ok = async () => {
const value = await decoder.decode(Uint8Array.of(9, 0), { targetDx: 90 });
assert.equal(value.samples[0], 9);
};
await ok();
// A decode error is reported for that job only; the worker stays up.
const [thrown, sibling] = await Promise.allSettled([
decoder.decode(Uint8Array.of(1, 0)),
decoder.decode(Uint8Array.of(9, 0), { targetDx: 90 }),
]);
assert.match(thrown.reason.message, /bad message/);
assert.equal(sibling.value.samples[0], 9);
// An uncaught error or an exit retires the worker; the next decode restarts it.
for (const [mode, pattern] of [
[2, /worker failed/],
[3, /worker exited/],
]) {
const results = await Promise.allSettled([
decoder.decode(Uint8Array.of(mode, 0)),
decoder.decode(Uint8Array.of(9, 0)),
]);
assert.match(results[0].reason.message, pattern);
// A job queued behind an exit dies with the worker; one queued behind an
// asynchronous crash may still finish first. Either way it settles.
if (mode === 3) assert.equal(results[1].status, 'rejected');
await ok();
}
// A decode that never returns is cut off instead of hanging its request.
await assert.rejects(decoder.decode(Uint8Array.of(4, 0)), /timed out/);
await ok();
// Cancelling stops the wait at once; the budget is checked before any copy.
const controller = new AbortController();
const slow = decoder.decode(Uint8Array.of(9, 20), {
signal: controller.signal,
});
controller.abort(new Error('cancelled'));
await assert.rejects(slow, /cancelled/);
await assert.rejects(
decoder.decode(new Uint8Array(8 * 1024 * 1024 + 1)),
/byte budget/,
);
await ok();
});
test('a worker crash during a refresh answers the request and the next refresh recovers', async (t) => {
const decoder = await createWindDecoder({ workerUrl: FAKE_WORKER });
t.after(() => decoder.close());
let clock = NOON;
let crash = true;
const request = install(
windProxy({
now: () => clock,
targetDx: 90,
decodeImpl: decoder.decode,
fetchImpl: async (url, options = {}) => {
if (!options.headers?.Range)
return new Response(
[
'1:0:d=2026091406:UGRD:10 m above ground:anl:',
'2:2:d=2026091406:VGRD:10 m above ground:anl:',
'3:4:d=2026091406:HGT:surface:anl:',
].join('\n'),
);
return new Response(Uint8Array.of(crash ? 3 : 6, 0), { status: 206 });
},
}),
);
const failed = await served(request, '/');
assert.equal(failed.manifest.unavailable, true);
crash = false;
clock += 60_000; // past the failure backoff
const recovered = await served(request, '/');
assert.equal(recovered.manifest.unavailable, false);
assert.equal(recovered.grid.length, 4 * 3 * 4 * 2);
});
test('closing one decoder rejects its jobs and leaves another working', async (t) => {
const a = await createWindDecoder({ workerUrl: FAKE_WORKER });
const b = await createWindDecoder({ workerUrl: FAKE_WORKER });
t.after(() => b.close());
const pending = a.decode(Uint8Array.of(9, 10));
a.close();
await assert.rejects(pending, /closed/);
assert.equal((await b.decode(Uint8Array.of(9, 0))).samples[0], 9);
});
test('an idle decode worker does not keep the process alive', () => {
const decoderUrl = new URL(
'../../server/providers/wind/decoder.js',
import.meta.url,
).href;
const script = `
import(${JSON.stringify(decoderUrl)}).then(async ({ createWindDecoder }) => {
const decoder = await createWindDecoder({ workerUrl: new URL(${JSON.stringify(FAKE_WORKER.href)}) });
const field = await decoder.decode(Uint8Array.of(9, 0));
console.log('decoded', field.samples.length);
});
`;
const child = spawnSync(process.execPath, ['-e', script], {
encoding: 'utf8',
timeout: 20_000,
});
assert.equal(child.status, 0, child.stderr);
assert.equal(child.stdout.trim(), 'decoded 65160');
});