Merge upstream/main into feature/catalonia-cctv-cameras

This commit is contained in:
ulihhhh committed 2026-09-25 12:01:51 +02:00
commit 21a86c05bd
24 files changed
+1261 -38

No files matched your search

+53
View File
@@ -1,5 +1,58 @@
# Changelog
- CCTV cameras whose bearing is a guess now say so. Packs mark bearings derived
from a hash of the camera id as `headingConfidence: 'low'`, but nothing read the
flag, so roughly 70% of a default catalog rendered like surveyed facings. The HUD
now reads `HDG n° (ESTIMATED)` and the coverage wireframe draws dashed; manual
calibrations and curated poses are never marked estimated (bassem chagra, #643).
- Report which upstream declined a Street Traffic road load. The layer row now
reads `Overpass rate-limited`, `Overpass timed out`, or
`Overpass refused the road query (HTTP 406)` instead of a general
"Road data temporarily unavailable", so a reader is not sent to check a
TomTom key when the public OpenStreetMap mirrors are the side that failed.
The proxy's own 502 (every mirror unreachable) and 503 (local limiter busy)
read `Overpass mirrors unreachable` and `Overpass temporarily unavailable`.
Failures the layer cannot classify keep the general line (daikaginza, #665).
- Bound the client terrain-height cache at 20 000 entries with least-recently-used eviction, so a long session no longer retains every coordinate it ever resolved. Consumer reads promote their entry and a batch still reports every point it resolved (Pedro Lobato, #594).
- Release CCTV media streams whose upstream falls silent after answering. The
15-second media deadline covered only the wait for response headers, so a
camera that replied and then stopped sending held both the proxy connection
and its upstream socket open for as long as the camera host allowed; the
declared `Content-Length` ceiling was the only body bound, and a chunked or
length-less body had none. A 30-second idle deadline now bounds the gap
between upstream chunks and releases a body that has gone silent. It is
rescheduled while the response is still waiting to drain, so a viewer on a
slow link is not mistaken for a dead camera; live feeds are unaffected
(Ethan Stoner, #688).
- Aircraft track backfill (`/api/opensky-track`, `/api/adsblol/trace`) now
answers 502 when the upstream body exceeds the 5 MB cap, instead of a 200
whose error body the client read as an empty track. The failure is cached
like other upstream errors, so retries inside the 60 s window do not spend
OpenSky credits (Raushankumar0720, #720, #722).
- Saving a key from Provider Settings works again on Macs where Nix or
Homebrew coreutils sit ahead of `/bin` on `PATH`. The credential hardener
now spawns Apple's `/bin/chmod -N` by absolute path; GNU `chmod` has no `-N`,
so the ACL strip failed closed and every save was refused (Arthur Bogaart, #694).
- Render on iPad and iPhone instead of stopping with "An error occurred while
rendering." Cesium's per-vertex model atmosphere binds shader `out`
parameters directly to varyings, which Apple's Metal/ANGLE backend cannot
link, so the program failed and the render loop was torn down. The stage is
now kept out of the pipeline on affected devices by clearing
`scene.fog.renderable`, which leaves `fog.enabled` — and the fog density that
drives 3D Tiles refinement — untouched. Detection is a WebGL2 link probe of
the same pattern, so a future driver fix restores the effect with no code
change, with iOS/iPadOS detection as a backstop. Sky atmosphere and the
ground-atmosphere fragment path route through locals and are unaffected;
affected devices lose distance fog on 3D tiles and on the globe basemaps.
`src/app/atmosphereCompat.test.mjs` pins the probe, the platform matrix and
the `renderable`-not-`enabled` choice (KnottyDyes, #705).
- Region scopes in voice analyst queries ("in the Gulf of Mexico", "over
the Alps") work again in the dev server: the bundled Natural Earth and
neighborhood packs are fetched as JSON in the browser
+48 -4
View File
@@ -701,6 +701,25 @@ snapshots, lifecycle actions and row descriptors. Remount and teardown remove
listeners and row subscriptions; obsolete completions do not repaint old rows.
The clear control presents busy state while its existing action owns the transaction.
## Model atmosphere on Apple Metal
Cesium's per-vertex model atmosphere is kept out of the pipeline on devices
whose driver cannot link it. `AtmosphereStageVS` binds shader `out` parameters
directly to varyings, which ANGLE's Metal backend rejects at link time, tearing
down the render loop on iPadOS and iOS. `ModelSceneGraph.configurePipeline`
attaches that stage only when `fog.enabled && fog.renderable`, so the viewer
clears `scene.fog.renderable` and leaves `fog.enabled` — and the fog density
that drives 3D Tiles screen-space-error scaling — intact. Sky atmosphere and the
ground-atmosphere fragment path route through locals and are unaffected.
Affected devices lose distance fog on 3D tiles and on globe basemaps (Esri,
Bing, keyless), since the globe fog shader uses the same `renderable` flag.
The probe releases its throwaway WebGL2 context immediately.
Detection is a WebGL2 link probe of the same out-parameter/varying pattern, so a
future driver fix restores the effect with no code change; iOS/iPadOS platform
detection is the backstop for when no probe context can be created. Applied
during viewer construction, before any tile builds a draw command.
## Map Source control ownership
Map Source controls own chip listeners, source-state subscriptions and selection
@@ -794,10 +813,12 @@ their `configured: false` response.
CCTV media waits at most 15 seconds for upstream response headers and returns
504 on timeout. Its timer stops when headers arrive, so live bodies can continue
streaming; body idle deadlines are separate from this header deadline. Error
responses are cancelled. Buffered snapshots have a 16 MiB streaming cap; an
oversized image remains an upstream miss and uses the normal fallback chain.
The existing declared media size ceiling remains 64 MiB.
streaming; a separate 30-second idle deadline then bounds the gap between
upstream chunks and releases a body that has gone silent. A response that is
still waiting to drain reschedules that deadline, so a slow client does not read
as a stalled upstream. Error responses are cancelled. Buffered snapshots have a
16 MiB streaming cap; an oversized image remains an upstream miss and uses the
normal fallback chain. The existing declared media size ceiling remains 64 MiB.
## GBFS upstream bounds
@@ -858,6 +879,22 @@ under `gods-eye-view/server/providers/`. Portable terrain mechanics, traffic til
math and GBFS source rules are available under `gods-eye-view/sources/`.
The browser layers and their rendering remain in their existing modules.
## Terrain height cache bound
The client terrain-height resolver's in-memory cache is bounded at 20 000
entries (`DEFAULT_MAX_CACHE_ENTRIES`, overridable per instance through
`createTerrainHeights({ maxCacheEntries })`). Coordinate rounding makes repeated
visits to one place share a key but does not bound how many distinct places a
session visits, and the cache previously had no eviction, so a long session
retained every coordinate it ever resolved. Eviction is least-recently-used:
a consumer read promotes its entry, so the cell being watched is not dropped for
having been resolved early. The bound sits far above a city-scale session, so
normal use never evicts and issues no additional proxy requests. A batch's
results are assembled from what that call resolved, so eviction during a large
batch cannot report a just-resolved point as unresolved. Re:Earth and
geoid-fallback entry semantics, the fallback cooldown and the abort-time flush
are unchanged.
## Landmark annotation identity
When a landmark geocode contains only address components, annotations retain
@@ -3347,6 +3384,12 @@ silently demoting every later lookup for the session.
- **The DENSE chip reports the dense LOAD, not the catalog param.** The param flips synchronously while the Starlink shell takes seconds to arrive over a chunked load, and CelesTrak 502s that feed regularly. So the chip reads `DENSE ···` (busy, disabled) while loading, ACTIVE only once dense points are actually on screen, and `DENSE ✕` with the reason on hover when the load fails — a failure also reverts `catalog` to `core`, drops any partial chunk, and leaves the chip clickable to retry. A load is judged by points added, not by HTTP status: a 200 carrying an empty body, a passed-through HTML error page, or only TLEs the core catalog already owns fails with the same revert semantics as a 502. Any explicit request for `core` clears a latched error even when the mode does not change, so a Space Missions restore of an already-core snapshot never leaves the user with a failure they did not cause. Because the load settles asynchronously, the layer pushes a re-render through the optional `setRowControlsListener()` hook; nothing else would repaint that row before the 5-minute catalog refresh, so the count and legend would otherwise sit stale.
- **A dependency owner takes the row with it.** Space Missions borrows this layer for TLE lookup with `showPoints:false`; while points are hidden the layer returns empty row controls, so the legend never describes an empty sky and the chip cannot accept a write that the owner's restore would silently revert.
- The detection-overlay record cache (`_detectionObjects`) is cleared with the catalog on every rebuild: it stamps id/class at creation only, and a rebuild can re-tag a satellite when a partial CelesTrak outage changes which group wins dedupe.
- **CCTV estimated bearings:** a camera whose pack marks `headingConfidence: 'low'` (the id-hash
`fallbackHeadingFromId` bearing) shows `HDG n° (ESTIMATED)` in the HUD and draws its coverage
wireframe dashed; colours, widths and active/idle emphasis are unchanged. A manual calibration
(`calSource: 'manual'`) or curated pose (`poseSource: 'curated'`) is never presented as
estimated, matching the CAL badge. Public camera state carries `headingConfidence` and
`headingEstimated` (`src/layers/cctv/headingConfidence.js`, #639).
- **FIRMS**: no ground clamping (zero 3D-tiles height sampling), ≤18 screen-decluttered ambient labels, click-to-inspect detail card, 2.5k/3k sprite budgets viewport-clipped by FRP.
- **CCTV v2 foundation:** a pitched
frustum wireframe (4 corner rays + far-cap rectangle) with a monitor plane at the frustum's
@@ -3446,6 +3489,7 @@ silently demoting every later lookup for the session.
- HTTP refusals such as 406 now rotate alongside the existing network, rate-limit, and runtime-error cases. A refusal from one mirror no longer prevents reaching healthy alternatives or persists under the seven-day road/month-long boundary cache TTLs. Concurrent identical queries share one mirror sequence; if it fails, both the initiating and joined callers can use the same last-good data.
- A refusal every mirror agrees on is still reported with the first mirror's status and body, so a genuinely malformed query says what upstream said — but only after every mirror has had the chance to answer it. `fetchOverpassPayload` takes injectable endpoints and fetch so the rotation is tested without a live mirror (`src/overpassProxy.test.mjs`).
- Every mirror is asked with a User-Agent that names the application, its version and the project address (`OVERPASS_USER_AGENT` in `server/providers/overpass/constants.js`), which is what the OSM API usage policy asks for. Mirrors may refuse a client they cannot identify; such a refusal is never treated as data and costs the fan-out that mirror, so the header is what keeps the list at full strength.
- When the fan-out has nothing left to offer, the traffic row says which upstream declined and how: `Overpass rate-limited`, `Overpass timed out`, or `Overpass refused the road query (HTTP 406)`; the proxy's own 502 (every mirror unreachable) and 503 (local limiter busy) read `Overpass mirrors unreachable` and `Overpass temporarily unavailable`. Street Traffic draws on two upstreams — OpenStreetMap for geometry, TomTom for flow — so a row that only said "road data unavailable" sent readers to check a TomTom key that was never the problem (#661). `roadRequestError` in `src/layers/traffic/source.js` owns the mapping and tags the error with its status. A refusal whose code is unreadable — the layer takes its source by injection, so an adapter may not supply one — reports `Overpass temporarily unavailable` rather than printing `HTTP undefined`, and anything the layer did not classify at all (a dropped connection, a malformed snapshot) still reads `Road data temporarily unavailable`, because a platform exception is not a sentence to put on a panel.
### Share-link v2 layer state (August 2026)
+2
View File
@@ -31,6 +31,7 @@
"src/app/mapLifecycle.test.mjs",
"src/app/startupCamera.test.mjs",
"src/app/viewer.js",
"src/app/atmosphereCompat.test.mjs",
"src/standalone/application.js",
"src/standalone/controls.js",
"src/standalone/data.js",
@@ -633,6 +634,7 @@
"src/data/surfaceServices.js",
"src/annotations/resolver.js",
"src/services/terrainHeights.js",
"src/services/terrainHeights.test.mjs",
"src/services/groundFloor.js",
"src/services/meshFloorSampler.js",
"src/services/groundSnap.js",
+5 -1
View File
@@ -20,7 +20,7 @@
},
"viewer": {
"exports": ["./application/viewer"],
"modules": ["src/app/viewer.js"],
"modules": ["src/app/viewer.js", "src/app/atmosphereCompat.js"],
"external": ["cesium"]
},
"vite-build": {
@@ -1072,6 +1072,7 @@
"src/layers/cctv/geometry.js",
"src/layers/cctv/geometryQueue.js",
"src/layers/cctv/ground.js",
"src/layers/cctv/headingConfidence.js",
"src/layers/cctv/health.js",
"src/layers/cctv/hover.js",
"src/layers/cctv/index.js",
@@ -1313,6 +1314,7 @@
"src/app/surfaceServices.js",
"src/app/tools.js",
"src/app/viewer.js",
"src/app/atmosphereCompat.js",
"src/bloom.js",
"src/camera.js",
"src/cameraVerbs.js",
@@ -1487,6 +1489,7 @@
"src/layers/cctv/geometry.js",
"src/layers/cctv/geometryQueue.js",
"src/layers/cctv/ground.js",
"src/layers/cctv/headingConfidence.js",
"src/layers/cctv/health.js",
"src/layers/cctv/hover.js",
"src/layers/cctv/index.js",
@@ -2126,6 +2129,7 @@
"src/layers/cctv/geometry.js",
"src/layers/cctv/geometryQueue.js",
"src/layers/cctv/ground.js",
"src/layers/cctv/headingConfidence.js",
"src/layers/cctv/health.js",
"src/layers/cctv/hover.js",
"src/layers/cctv/index.js",
+6 -2
View File
@@ -45,7 +45,9 @@ export function trackBackfillProxies() {
RESPONSE_CAP_BYTES,
);
let body;
let status = upstream.status;
if (tooLarge) {
status = 502;
body = JSON.stringify({ error: 'Upstream track response too large' });
} else if (!upstream.ok) {
// Sanitize upstream error surface; status code is signal enough
@@ -53,8 +55,10 @@ export function trackBackfillProxies() {
} else {
body = text;
}
cachePut(key, { at: Date.now(), status: upstream.status, body });
res.statusCode = upstream.status;
// Cache the 502 like any upstream error: /tracks/all costs OpenSky
// credits, so a retry inside the window must not re-download the body.
cachePut(key, { at: Date.now(), status, body });
res.statusCode = status;
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.setHeader('Cache-Control', 'no-store');
res.end(body);
+5
View File
@@ -302,5 +302,10 @@ export const CCTV_FRAME_MAX_BODY_BYTES = 16 * 1024 * 1024;
/** Deadline for upstream response headers; live bodies keep streaming afterward. */
export const CCTV_MEDIA_FETCH_TIMEOUT_MS = 15 * 1000;
/** Silence a live body may carry before the relay gives up on it. Twice the
* header deadline, because a camera that is merely slow between frames is far
* more common than one that has died mid-stream, and a viewer would rather
* wait than be dropped. */
export const CCTV_MEDIA_IDLE_TIMEOUT_MS = 30 * 1000;
/** Declared size ceiling for fixed media responses. */
export const CCTV_MEDIA_MAX_BODY_BYTES = 64 * 1024 * 1024;
+45 -1
View File
@@ -4,6 +4,7 @@ import {
CCTV_FRAME_FETCH_TIMEOUT_MS,
CCTV_FRAME_MAX_BODY_BYTES,
CCTV_MEDIA_FETCH_TIMEOUT_MS,
CCTV_MEDIA_IDLE_TIMEOUT_MS,
CCTV_MEDIA_MAX_BODY_BYTES,
NSW_IMAGE_ORIGIN,
NSW_IMAGE_USER_AGENT,
@@ -103,11 +104,17 @@ export function toReadable(body) {
* @param {Response} upstream - fetch() Response object.
* @param {object} [opts]
* @param {string} [opts.sourceHeader='upstream'] - Value for X-CCTV-Source header.
* @param {number} [opts.idleTimeoutMs=CCTV_MEDIA_IDLE_TIMEOUT_MS] - Silence
* allowed between upstream chunks before the stream is released. Injectable
* only to keep the deadline unit-testable.
*/
export async function proxyMediaResponse(
res,
upstream,
{ sourceHeader = 'upstream' } = {},
{
sourceHeader = 'upstream',
idleTimeoutMs = CCTV_MEDIA_IDLE_TIMEOUT_MS,
} = {},
) {
const contentType =
upstream.headers.get('content-type') || 'application/octet-stream';
@@ -153,7 +160,20 @@ export async function proxyMediaResponse(
return;
}
// The header deadline only covers the wait for a response line. Past that an
// upstream can hold the connection open and send nothing at all, and the
// relay would wait on it for as long as the camera host cared to. The
// deadline below measures the gap between upstream chunks rather than the
// life of the stream, so a feed that keeps delivering keeps its connection.
let idleTimer = null;
const clearIdleDeadline = () => {
if (!idleTimer) return;
clearTimeout(idleTimer);
idleTimer = null;
};
stream.on('error', () => {
clearIdleDeadline();
if (!res.writableEnded) res.end();
});
@@ -164,6 +184,7 @@ export async function proxyMediaResponse(
const releaseUpstream = () => {
if (released) return;
released = true;
clearIdleDeadline();
stream.unpipe(res);
// Destroying the Node stream cancels the web body it wraps; the direct
// cancel covers a body that was never wrapped, and rejects harmlessly when
@@ -176,14 +197,37 @@ export async function proxyMediaResponse(
/* already closed */
}
};
const armIdleDeadline = () => {
clearIdleDeadline();
idleTimer = setTimeout(onIdleDeadline, idleTimeoutMs);
idleTimer.unref?.();
};
const onIdleDeadline = () => {
// A viewer who cannot keep up pauses the pipe, and no upstream bytes arrive
// while it is paused. That is a slow client rather than a dead camera, so
// it gets the deadline again instead of a teardown.
if (res.writableNeedDrain) {
armIdleDeadline();
return;
}
releaseUpstream();
if (!res.writableEnded) res.end();
};
res.once('close', () => {
clearIdleDeadline();
if (!res.writableEnded) releaseUpstream();
});
res.once('error', releaseUpstream);
stream.once('end', () => {
clearIdleDeadline();
released = true;
});
armIdleDeadline();
stream.pipe(res);
// Attached after the pipe because a data listener resumes the stream, and
// flowing before the destination is attached would spill chunks nobody
// forwards. Only bytes from upstream renew the deadline.
stream.on('data', armIdleDeadline);
}
/**
+5 -1
View File
@@ -123,7 +123,11 @@ export function hardenCredentialFile(
if (platform !== 'win32') {
try {
if (platform === 'darwin') {
const aclRemoval = spawn('chmod', ['-N', filepath], {
// `-N` (strip the ACL) is Apple-only. Spawn /bin/chmod by absolute
// path, never by name: a Nix/Homebrew coreutils profile puts GNU chmod
// first on PATH, and GNU chmod rejects `-N` — which was treated as a
// hardening failure and refused every save on such machines.
const aclRemoval = spawn('/bin/chmod', ['-N', filepath], {
stdio: 'ignore',
});
if (!commandCompletedSuccessfully(aclRemoval)) return false;
+189
View File
@@ -0,0 +1,189 @@
/**
* Workaround for Apple's Metal/ANGLE shader translator rejecting Cesium's
* per-vertex model atmosphere.
*
* Cesium's `AtmosphereStageVS` calls a function whose `out` parameters are
* bound DIRECTLY to varyings:
*
* czm_computeGroundAtmosphereScattering(
* v_positionWC, lightDirection,
* v_atmosphereRayleighColor, v_atmosphereMieColor, v_atmosphereOpacity);
*
* ANGLE's Metal backend wraps each `out` argument in an `ANGLE_out(...)`
* helper that takes a `thread T &`. Vertex-output struct members live in
* `__metal_generic`, which cannot bind to `thread`, so the program fails to
* LINK (not compile) and Cesium tears the render loop down:
*
* RuntimeError: Program failed to link. ... MSL compilation error:
* reference to type 'thread float3' could not bind to an lvalue of type
* '__metal_generic float3'
*
* This is the only shader in Cesium that binds varyings to `out` params --
* SkyAtmosphere and the ground-atmosphere fragment path both route through
* locals first, so they are unaffected and stay on.
*
* Cesium attaches that stage in `ModelSceneGraph.configurePipeline` under a
* single condition:
*
* const fogRenderable = frameState.fog.enabled && frameState.fog.renderable;
* if (fogRenderable) modelPipelineStages.push(AtmospherePipelineStage);
*
* so clearing `scene.fog.renderable` keeps the broken stage out of the
* pipeline entirely -- the program is never built, so it cannot fail to link.
*
* We clear `renderable` rather than `enabled` deliberately:
* - `Fog.update` still computes fog density when only `renderable` is off,
* preserving the distance-based screen-space-error scaling that governs
* 3D Tiles refinement. Clearing `enabled` returns early and forfeits it.
* - `visualSettings.js` saves and restores `scene.fog.enabled` around IR
* boost. Parking our fix on `enabled` would let that restore resurrect
* the broken shader mid-session; `renderable` is orthogonal to it.
*
* @module app/atmosphereCompat
*/
/** Minimal vertex shader reproducing the exact out-param-to-varying bind. */
const PROBE_VERTEX_SHADER = `#version 300 es
precision highp float;
in vec3 a_position;
out vec3 v_probe;
void writeOut(out vec3 value) { value = vec3(1.0); }
void main() {
writeOut(v_probe);
gl_Position = vec4(a_position, 1.0);
}`;
const PROBE_FRAGMENT_SHADER = `#version 300 es
precision highp float;
in vec3 v_probe;
out vec4 fragColor;
void main() { fragColor = vec4(v_probe, 1.0); }`;
/**
* True when the user agent is iOS or iPadOS.
*
* iPadOS 13+ defaults to desktop-mode browsing and reports `MacIntel` with no
* iPad token, so it is identified by touch points instead -- real Macs report
* `maxTouchPoints` of 0, including Apple Silicon ones that also claim
* `MacIntel`. Brave/Chrome/Firefox on iOS are all WebKit underneath and share
* the same Metal backend, so matching the platform rather than the browser is
* what we want.
*/
export function isAppleMobilePlatform(navigatorLike = globalThis.navigator) {
if (!navigatorLike) return false;
const ua = String(navigatorLike.userAgent || '');
if (/\b(iPad|iPhone|iPod)\b/.test(ua)) return true;
// Desktop-mode iPadOS presents as a Mac. What separates it from a real Mac
// is touch: Macs report 0 touch points, including Apple Silicon ones that
// also claim `MacIntel`. `navigator.platform` is deprecated and may be
// empty, so the UA's Macintosh token is accepted as an equal signal rather
// than relying on platform alone.
const platform = String(navigatorLike.platform || '');
const macLike =
platform === 'MacIntel' ||
platform === 'MacARM' ||
/\bMacintosh\b/.test(ua);
const touchPoints = Number(navigatorLike.maxTouchPoints) || 0;
return macLike && touchPoints > 1;
}
/**
* Link the probe program and report whether the driver rejects the pattern.
*
* Returns `false` when WebGL2 is unavailable so that a missing context is
* never mistaken for a broken driver -- Cesium has its own handling for that,
* and the caller still falls back to platform detection.
*/
export function probeOutParamVaryingLinkFailure(gl) {
if (!gl) return false;
const vs = gl.createShader(gl.VERTEX_SHADER);
const fs = gl.createShader(gl.FRAGMENT_SHADER);
const program = gl.createProgram();
if (!vs || !fs || !program) return false;
try {
gl.shaderSource(vs, PROBE_VERTEX_SHADER);
gl.compileShader(vs);
gl.shaderSource(fs, PROBE_FRAGMENT_SHADER);
gl.compileShader(fs);
// A compile failure here is not the bug we are probing for: the driver
// rejects this pattern at LINK time, after both stages compile cleanly.
if (
!gl.getShaderParameter(vs, gl.COMPILE_STATUS) ||
!gl.getShaderParameter(fs, gl.COMPILE_STATUS)
)
return false;
gl.attachShader(program, vs);
gl.attachShader(program, fs);
gl.linkProgram(program);
return !gl.getProgramParameter(program, gl.LINK_STATUS);
} catch {
return false;
} finally {
gl.deleteShader(vs);
gl.deleteShader(fs);
gl.deleteProgram(program);
}
}
/** Build a throwaway WebGL2 context for the probe, or null if unavailable. */
function defaultProbeContext() {
try {
const canvas =
typeof OffscreenCanvas === 'function'
? new OffscreenCanvas(1, 1)
: globalThis.document?.createElement('canvas');
return canvas?.getContext('webgl2') ?? null;
} catch {
return null;
}
}
/**
* Decide whether the per-vertex model atmosphere must be kept out of the
* pipeline on this device.
*
* The probe is authoritative when it fires, so a future driver fix silently
* restores the effect without a code change. Platform detection is the
* backstop for the case where no context can be created -- on Apple mobile a
* link failure is fatal and unrecoverable, so we would rather lose fog than
* gamble the whole render loop on the probe having run.
*/
export function shouldDisableModelAtmosphere({
navigatorLike = globalThis.navigator,
createProbeContext = defaultProbeContext,
} = {}) {
const gl = createProbeContext();
if (gl) {
let linkFails;
try {
linkFails = probeOutParamVaryingLinkFailure(gl);
} finally {
// Browsers cap live WebGL contexts (Safari especially); release the
// throwaway probe context now rather than waiting for GC.
try {
gl.getExtension?.('WEBGL_lose_context')?.loseContext();
} catch {
// A context that cannot be released is left to GC.
}
}
if (linkFails) return true;
// Probe ran and the driver accepted the pattern. Apple mobile is still
// quarantined: the probe program is far simpler than Cesium's, and a
// false negative there costs the entire scene.
return isAppleMobilePlatform(navigatorLike);
}
return isAppleMobilePlatform(navigatorLike);
}
/**
* Keep Cesium's broken model-atmosphere stage out of the pipeline.
*
* Returns true when the workaround was applied. Safe to call on any scene;
* scenes without a fog object are left untouched.
*/
export function applyModelAtmosphereWorkaround(scene, options = {}) {
if (!scene?.fog) return false;
if (!shouldDisableModelAtmosphere(options)) return false;
scene.fog.renderable = false;
return true;
}
+221
View File
@@ -0,0 +1,221 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
isAppleMobilePlatform,
probeOutParamVaryingLinkFailure,
shouldDisableModelAtmosphere,
applyModelAtmosphereWorkaround,
} from './atmosphereCompat.js';
const IPAD_OS12 = {
userAgent:
'Mozilla/5.0 (iPad; CPU OS 12_5 like Mac OS X) AppleWebKit/605.1.15',
platform: 'iPad',
maxTouchPoints: 5,
};
// iPadOS 13+ desktop mode: no iPad token, reports as a Mac.
const IPAD_DESKTOP_MODE = {
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
platform: 'MacIntel',
maxTouchPoints: 5,
};
const MAC_DESKTOP = {
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
platform: 'MacIntel',
maxTouchPoints: 0,
};
const LINUX_DESKTOP = {
userAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/141',
platform: 'Linux x86_64',
maxTouchPoints: 0,
};
/** Minimal GL double. `linkOk: false` mimics the Metal translator's refusal. */
function fakeGl({ linkOk = true, compileOk = true, throwOn = null } = {}) {
const deleted = { shaders: 0, programs: 0 };
return {
deleted,
VERTEX_SHADER: 1,
FRAGMENT_SHADER: 2,
COMPILE_STATUS: 3,
LINK_STATUS: 4,
createShader: () => ({}),
createProgram: () => ({}),
shaderSource() {
if (throwOn === 'shaderSource') throw new Error('context lost');
},
compileShader() {},
attachShader() {},
linkProgram() {},
getShaderParameter: () => compileOk,
getProgramParameter: () => linkOk,
deleteShader: () => (deleted.shaders += 1),
deleteProgram: () => (deleted.programs += 1),
};
}
test('identifies iPadOS in both legacy and desktop-mode guises', () => {
assert.equal(isAppleMobilePlatform(IPAD_OS12), true);
assert.equal(isAppleMobilePlatform(IPAD_DESKTOP_MODE), true);
});
test('does not mistake a real Mac or a Linux desktop for Apple mobile', () => {
// Regression guard: the desktop-mode check keys on touch points, and a Mac
// reports the same MacIntel platform string with zero of them.
assert.equal(isAppleMobilePlatform(MAC_DESKTOP), false);
assert.equal(isAppleMobilePlatform(LINUX_DESKTOP), false);
});
test('treats a missing navigator as not-Apple rather than throwing', () => {
assert.equal(isAppleMobilePlatform(undefined), false);
assert.equal(isAppleMobilePlatform(null), false);
});
test('detects desktop-mode iPadOS without navigator.platform', () => {
// `navigator.platform` is deprecated and can be absent or stripped; the UA's
// Macintosh token plus touch points has to carry the detection on its own.
assert.equal(
isAppleMobilePlatform({
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
maxTouchPoints: 5,
}),
true,
);
// Same UA, no touch: a real Mac, which must stay untouched.
assert.equal(
isAppleMobilePlatform({
userAgent:
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15',
maxTouchPoints: 0,
}),
false,
);
});
test('probe reports failure only when the program fails to link', () => {
assert.equal(
probeOutParamVaryingLinkFailure(fakeGl({ linkOk: false })),
true,
);
assert.equal(
probeOutParamVaryingLinkFailure(fakeGl({ linkOk: true })),
false,
);
});
test('a compile failure is not reported as the link bug', () => {
// The Metal defect surfaces at link time; both stages compile cleanly.
// Treating a compile error as the bug would disable fog for the wrong reason.
assert.equal(
probeOutParamVaryingLinkFailure(
fakeGl({ compileOk: false, linkOk: false }),
),
false,
);
});
test('probe releases its GL objects on both paths', () => {
const ok = fakeGl({ linkOk: true });
probeOutParamVaryingLinkFailure(ok);
assert.equal(ok.deleted.shaders, 2);
assert.equal(ok.deleted.programs, 1);
const thrown = fakeGl({ throwOn: 'shaderSource' });
probeOutParamVaryingLinkFailure(thrown);
assert.equal(thrown.deleted.shaders, 2, 'shaders freed even when GL throws');
assert.equal(thrown.deleted.programs, 1);
});
test('probe treats a null context as healthy, not broken', () => {
assert.equal(probeOutParamVaryingLinkFailure(null), false);
});
test('a failing probe disables the stage on any platform', () => {
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: LINUX_DESKTOP,
createProbeContext: () => fakeGl({ linkOk: false }),
}),
true,
);
});
test('a passing probe still quarantines Apple mobile but frees desktops', () => {
const createProbeContext = () => fakeGl({ linkOk: true });
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: IPAD_DESKTOP_MODE,
createProbeContext,
}),
true,
);
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: LINUX_DESKTOP,
createProbeContext,
}),
false,
);
});
test('falls back to platform detection when no context can be made', () => {
const createProbeContext = () => null;
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: IPAD_OS12,
createProbeContext,
}),
true,
);
assert.equal(
shouldDisableModelAtmosphere({
navigatorLike: MAC_DESKTOP,
createProbeContext,
}),
false,
);
});
test('applies to the scene by clearing renderable, never enabled', () => {
// `enabled` must survive: Fog.update short-circuits when it is false, which
// forfeits the density used for 3D Tiles screen-space-error scaling, and
// visualSettings' IR boost saves/restores that same flag.
const scene = { fog: { enabled: true, renderable: true } };
const applied = applyModelAtmosphereWorkaround(scene, {
navigatorLike: IPAD_DESKTOP_MODE,
createProbeContext: () => null,
});
assert.equal(applied, true);
assert.equal(scene.fog.renderable, false);
assert.equal(scene.fog.enabled, true);
});
test('leaves unaffected devices completely untouched', () => {
const scene = { fog: { enabled: true, renderable: true } };
const applied = applyModelAtmosphereWorkaround(scene, {
navigatorLike: LINUX_DESKTOP,
createProbeContext: () => fakeGl({ linkOk: true }),
});
assert.equal(applied, false);
assert.equal(scene.fog.renderable, true);
});
test('tolerates a scene with no fog object', () => {
assert.equal(applyModelAtmosphereWorkaround({}, {}), false);
assert.equal(applyModelAtmosphereWorkaround(null, {}), false);
});
test('releases the throwaway probe context after probing', () => {
let lost = 0;
const gl = fakeGl({ linkOk: true });
gl.getExtension = (name) =>
name === 'WEBGL_lose_context' ? { loseContext: () => (lost += 1) } : null;
shouldDisableModelAtmosphere({
navigatorLike: LINUX_DESKTOP,
createProbeContext: () => gl,
});
assert.equal(lost, 1, 'probe context must not linger until GC');
});
+5
View File
@@ -1,4 +1,5 @@
import * as Cesium from 'cesium';
import { applyModelAtmosphereWorkaround } from './atmosphereCompat.js';
const PINCH_ZOOM_MULTIPLIER = 8;
const MAX_PINCH_PIXEL_DELTA = 120;
@@ -124,6 +125,10 @@ export function createApplicationViewer({ container, creditContainer }) {
});
try {
viewer.targetFrameRate = 60;
// Before any tile builds a draw command: Cesium's per-vertex model
// atmosphere fails to LINK on Apple's Metal backend and kills the
// render loop. See app/atmosphereCompat.js.
applyModelAtmosphereWorkaround(viewer.scene);
viewer.scene.globe.show = false;
viewer.scene.skyAtmosphere.show = true;
viewer.scene.skyAtmosphere.atmosphereLightIntensity = 18;
+116 -1
View File
@@ -8,8 +8,9 @@ import { test } from 'node:test';
import assert from 'node:assert/strict';
import http from 'node:http';
import net from 'node:net';
import { Readable } from 'node:stream';
import { Readable, Writable } from 'node:stream';
import { cctvProxy } from '../../server/providers/cctv.js';
import { proxyMediaResponse } from '../../server/providers/cctv/media.js';
import { sanitizeCctvRangeHeader } from '../../server/providers/cctv/range.js';
import { CCTV_MEDIA_MAX_BODY_BYTES as CAP } from '../../server/providers/cctv/constants.js';
@@ -586,3 +587,117 @@ test('two sequential seeks are served as two partial responses', async (t) => {
assert.equal(first.body, 'chunk-0');
assert.equal(second.body, 'chunk-100');
});
// ---------------------------------------------------------------------------
// The body idle deadline. proxyMediaResponse is driven directly here because
// the deadline is only injectable through its options; the route always passes
// the production value.
// ---------------------------------------------------------------------------
/**
* A response to pipe into, built on a real Writable so the backpressure the
* proxy reads is the stream's own rather than a flag the test sets. With
* `drains: false` the bytes are accepted and never flushed, which is what a
* viewer on a slow link looks like from the server side.
*/
function pipeTarget({ drains = true } = {}) {
const res = new Writable({
// One byte is enough to leave a target that never empties its buffer
// permanently in need of a drain.
highWaterMark: drains ? undefined : 1,
write(_chunk, _encoding, callback) {
if (drains) callback();
},
});
res.writeHead = (status, headers) => {
res.statusCode = status;
res.headers = headers || {};
};
return res;
}
/**
* An upstream that answers with headers, sends one chunk, and then either
* keeps producing every `everyMs` or goes silent. `released` reports the body's
* own cleanup: a cancelled fetch body lands in the web stream's cancel hook.
*/
function mediaUpstream({ everyMs = 0 } = {}) {
const state = { produced: 0, released: false };
let ticker = null;
const body = new ReadableStream({
start(controller) {
const send = () => {
state.produced += 1;
controller.enqueue(new Uint8Array([0xff, 0xd8, 0xff, 0xe0]));
};
send();
if (everyMs > 0) {
ticker = setInterval(send, everyMs);
ticker.unref();
}
},
cancel() {
state.released = true;
if (ticker) clearInterval(ticker);
},
});
state.upstream = {
ok: true,
status: 200,
headers: {
get: (name) =>
String(name).toLowerCase() === 'content-type' ? 'video/mp4' : null,
},
body,
arrayBuffer: async () => Buffer.alloc(0),
};
return state;
}
test('an upstream that goes silent after its headers is released at the idle deadline', async (t) => {
const feed = mediaUpstream();
const res = pipeTarget();
t.after(() => res.destroy());
await proxyMediaResponse(res, feed.upstream, { idleTimeoutMs: 40 });
await new Promise((resolve) => setTimeout(resolve, 20));
assert.equal(feed.released, false, 'released before the deadline was due');
await new Promise((resolve) => setTimeout(resolve, 120));
// Without the deadline this stream is held open against the camera host for
// as long as that host will keep the socket, with no bytes ever arriving.
assert.equal(feed.released, true, 'the silent upstream was never released');
assert.equal(res.writableEnded, true, 'the client was left waiting');
});
test('a live feed that keeps producing is not cut off by the idle deadline', async (t) => {
const feed = mediaUpstream({ everyMs: 10 });
const res = pipeTarget();
t.after(() => res.destroy());
await proxyMediaResponse(res, feed.upstream, { idleTimeoutMs: 40 });
await new Promise((resolve) => setTimeout(resolve, 250));
assert.equal(feed.released, false, 'a healthy feed was torn down');
assert.equal(res.writableEnded, false, 'a healthy feed was ended early');
assert.ok(
feed.produced > 5,
`the feed stopped producing (${feed.produced} chunks)`,
);
});
test('a client that cannot keep up is not mistaken for a stalled upstream', async (t) => {
// Nothing arrives from upstream while the pipe is paused, which looks exactly
// like silence unless the response is asked whether it is still draining.
const feed = mediaUpstream({ everyMs: 10 });
const res = pipeTarget({ drains: false });
t.after(() => res.destroy());
await proxyMediaResponse(res, feed.upstream, { idleTimeoutMs: 30 });
await new Promise((resolve) => setTimeout(resolve, 200));
assert.equal(
res.writableNeedDrain,
true,
'the client was not the bottleneck',
);
assert.equal(feed.released, false, 'a slow viewer tore down a healthy feed');
assert.equal(res.writableEnded, false, 'a slow viewer ended the response');
});
+15
View File
@@ -55,6 +55,21 @@ test('macOS ACL removal failure stops before chmod and fails closed', () => {
assert.deepEqual(fileSystem.calls, [], 'mode bits must not disguise an ACL-removal failure');
});
test('macOS ACL removal spawns Apple /bin/chmod by absolute path, not via PATH', () => {
const spawned = [];
hardenCredentialFile(FILE, {
platform: 'darwin',
fileSystem: fileSystemWithMode(0o100600),
spawn(command, args) {
spawned.push([command, args]);
return { status: 0, signal: null };
},
});
// A coreutils-first PATH (Nix home-manager, Homebrew) resolves bare `chmod`
// to GNU chmod, which has no `-N` and would fail every save.
assert.deepEqual(spawned, [['/bin/chmod', ['-N', FILE]]]);
});
test('POSIX hardening verifies the resulting 0600 mode', () => {
const goodFs = fileSystemWithMode(0o100600);
assert.equal(hardenCredentialFile(FILE, {
+47
View File
@@ -0,0 +1,47 @@
/**
* @module headingConfidence
*
* Consumers for the catalog's per-camera `headingConfidence` flag (#639).
*
* Packs whose feed publishes no facing get a synthetic bearing from
* `fallbackHeadingFromId` — a hash of the id string, with no geographic
* input — and are marked `headingConfidence: 'low'`. Until now nothing read
* the flag, so a guessed bearing rendered exactly like a surveyed one. These
* helpers make the guess visible: the HUD heading token gains an
* `(ESTIMATED)` tag and the coverage wireframe draws dashed instead of solid.
*
* A human-vouched pose always wins over the pack flag: a manually saved
* calibration (`calSource:'manual'`) or a hand-authored catalog entry
* (`poseSource:'curated'`) is never presented as estimated, mirroring the
* CAL badge states in calibration.js.
*/
/**
* Whether a camera's bearing is a synthetic guess that should present as
* provisional.
* @param {{headingConfidence?: string|null, calSource?: string|null,
* poseSource?: string|null}} camera - Catalog camera (or public state).
* @returns {boolean}
*/
export function isHeadingEstimated(camera) {
if (!camera) return false;
if (camera.calSource === 'manual') return false;
if (camera.poseSource === 'curated') return false;
return (
String(camera.headingConfidence || '')
.trim()
.toLowerCase() === 'low'
);
}
/**
* HUD heading token: `HDG 194°`, tagged `(ESTIMATED)` when the bearing is
* synthetic — so a hash can never read as a surveyed facing.
* @param {{headingDeg?: number, headingConfidence?: string|null,
* calSource?: string|null, poseSource?: string|null}} camera
* @returns {string}
*/
export function headingHudToken(camera) {
const hdg = Math.round(Number(camera?.headingDeg) || 0);
return `HDG ${hdg}°${isHeadingEstimated(camera) ? ' (ESTIMATED)' : ''}`;
}
@@ -0,0 +1,60 @@
// src/layers/cctv/headingConfidence.test.mjs
// Bearing-provenance consumers (#639): a pack's headingConfidence:'low'
// (a hashed synthetic bearing) must present as estimated — unless a human
// vouched for the pose via manual calibration or a curated catalog entry.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { isHeadingEstimated, headingHudToken } from './headingConfidence.js';
test('low confidence marks the bearing estimated', () => {
assert.equal(isHeadingEstimated({ headingConfidence: 'low' }), true);
});
test('confidence matching is trimmed and case-insensitive', () => {
assert.equal(isHeadingEstimated({ headingConfidence: ' LOW ' }), true);
assert.equal(isHeadingEstimated({ headingConfidence: 'Low' }), true);
});
test('medium, high, missing, and absent cameras are not estimated', () => {
assert.equal(isHeadingEstimated({ headingConfidence: 'medium' }), false);
assert.equal(isHeadingEstimated({ headingConfidence: 'high' }), false);
assert.equal(isHeadingEstimated({}), false);
assert.equal(isHeadingEstimated(null), false);
assert.equal(isHeadingEstimated(undefined), false);
});
test('a manually saved calibration overrides a low pack flag', () => {
assert.equal(
isHeadingEstimated({ headingConfidence: 'low', calSource: 'manual' }),
false,
);
});
test('a curated catalog entry overrides a low pack flag', () => {
assert.equal(
isHeadingEstimated({ headingConfidence: 'low', poseSource: 'curated' }),
false,
);
});
test('HUD token tags a synthetic bearing and rounds the degrees', () => {
assert.equal(
headingHudToken({ headingDeg: 194.4, headingConfidence: 'low' }),
'HDG 194° (ESTIMATED)',
);
});
test('HUD token stays untagged for surveyed and calibrated bearings', () => {
assert.equal(
headingHudToken({ headingDeg: 67.5, headingConfidence: 'high' }),
'HDG 68°',
);
assert.equal(
headingHudToken({
headingDeg: 225,
headingConfidence: 'low',
calSource: 'manual',
}),
'HDG 225°',
);
});
+9 -1
View File
@@ -1,5 +1,6 @@
import { CCTV_AMBIENT_CARD_MAX } from '../../data/cctvLod.js';
import { ACTIVE_FRAME_REFRESH_MS, IDLE_FRAME_REFRESH_MS } from './policy.js';
import { headingHudToken, isHeadingEstimated } from './headingConfidence.js';
export function createPresentation({
state: layerState,
@@ -35,7 +36,9 @@ export function createPresentation({
return [
`${active.camera.city.toUpperCase()} CCTV`,
`${active.camera.name.toUpperCase()}`,
`HDG ${Math.round(active.camera.headingDeg)}°`,
// A synthetic bearing (headingConfidence 'low', no human calibration)
// is tagged so a hashed guess never reads as a surveyed facing (#639).
headingHudToken(active.camera),
`FOV ${Math.round(active.camera.fovDeg)}°`,
`COVERAGE ${area.toFixed(2)}km²`,
overlapCount > 0 ? `OVERLAP ${overlapCount} cams` : 'ISOLATED VIEW',
@@ -76,6 +79,11 @@ export function createPresentation({
lat: camera.lat,
lon: camera.lon,
headingDeg: camera.headingDeg,
// Bearing provenance (#639): the pack's confidence flag plus the derived
// "is this a synthetic guess" bit (calibration-aware), so UI consumers
// never have to re-derive it.
headingConfidence: camera.headingConfidence || null,
headingEstimated: isHeadingEstimated(camera),
pitchDeg: camera.pitchDeg,
fovDeg: camera.fovDeg,
rangeM: camera.rangeM,
+28 -14
View File
@@ -1,5 +1,6 @@
import * as Cesium from 'cesium';
import { horizonOccluder } from '../../data/iconOrientation.js';
import { isHeadingEstimated } from './headingConfidence.js';
import {
ACTIVE_CAMERA_COLOR,
IDLE_CAMERA_COLOR,
@@ -233,6 +234,15 @@ export function createRendering({
);
const inVisibleSet = coverageVisible.has(record.camera.id);
// A synthetic bearing (headingConfidence 'low', no human calibration)
// draws its wireframe dashed so a guessed facing is visibly provisional
// rather than rendering identically to a surveyed one (#639). Colors,
// widths, and the active/idle emphasis are unchanged.
const bearingEstimated = isHeadingEstimated(record.camera);
const lineMaterial = (color) =>
bearingEstimated
? new Cesium.PolylineDashMaterialProperty({ color })
: color;
for (const entity of record.coverageEntities || []) {
// The frustum wireframe is part of the projection representation —
// force it on for the active camera and let it read through geometry
@@ -247,13 +257,15 @@ export function createRendering({
// active camera keeps its width/alpha emphasis in both schemes.
const hue = viewshedOn ? record.viewshedColors : null;
if (entity._coverageRole === 'cap') {
entity.polyline.material = hue
? isActive
? hue.lineActive
: hue.line
: isActive
? ACTIVE_COVERAGE_CENTER
: IDLE_COVERAGE_CENTER_MUTED;
entity.polyline.material = lineMaterial(
hue
? isActive
? hue.lineActive
: hue.line
: isActive
? ACTIVE_COVERAGE_CENTER
: IDLE_COVERAGE_CENTER_MUTED,
);
entity.polyline.width = isActive ? 2.2 : 1.0;
entity.polyline.depthFailMaterial = planeShowing
? hue
@@ -261,13 +273,15 @@ export function createRendering({
: ACTIVE_COVERAGE_CENTER_DEPTHFAIL
: undefined;
} else {
entity.polyline.material = hue
? isActive
? hue.lineActive
: hue.line.withAlpha(0.6)
: isActive
? ACTIVE_COVERAGE_EDGE
: IDLE_COVERAGE_EDGE_MUTED;
entity.polyline.material = lineMaterial(
hue
? isActive
? hue.lineActive
: hue.line.withAlpha(0.6)
: isActive
? ACTIVE_COVERAGE_EDGE
: IDLE_COVERAGE_EDGE_MUTED,
);
entity.polyline.width = isActive ? 1.8 : 0.9;
entity.polyline.depthFailMaterial = planeShowing
? hue
+12 -3
View File
@@ -4,6 +4,7 @@ import {
TILE_CACHE_MAX_ENTRIES,
FAST_FETCH_ALTITUDE,
} from './policy.js';
import { RoadRequestError, roadRequestError } from './source.js';
export function createIngestion({
state: layerState,
@@ -67,7 +68,7 @@ export function createIngestion({
);
if (!response.ok) {
throw new Error(`Overpass API returned ${response.status}`);
throw roadRequestError(response.status);
}
if (!state) {
@@ -307,8 +308,16 @@ export function createIngestion({
renderedSomething = true;
} catch (e) {
if (e?.name === 'AbortError') return;
if (generation === layerState._loadGeneration && !renderedSomething)
layerState._roadError = 'Road data temporarily unavailable';
if (generation === layerState._loadGeneration && !renderedSomething) {
// A refusal we classified carries its own line; anything else — a
// dropped connection, a malformed snapshot — keeps the general one,
// because `e.message` from the platform is not a sentence anyone
// should have to read off a panel.
layerState._roadError =
e instanceof RoadRequestError
? e.message
: 'Road data temporarily unavailable';
}
console.warn('[Data:Traffic] Fetch error:', e);
} finally {
if (generation === layerState._loadGeneration) {
+36
View File
@@ -216,3 +216,39 @@ test('parked failures back off and disabling cancels the scheduled retry', async
assert.equal(calls, 2);
assert.equal(layer.getStats().error, null);
});
test('a declined road request reaches the row as the reason, not as a shrug', async (t) => {
// Issue #661: with every public Overpass mirror refusing a network, the row
// read "Road data temporarily unavailable" while the console already held
// "Overpass API returned 406". The reporter went looking for a bad TomTom
// key — the other half of this layer, and never the problem. The status the
// code already has must survive the trip to the panel.
let status = 406;
const { layer, viewer, tick } = setup(t, async () => ({ ok: false, status }));
layer.enable(viewer);
await tick(400);
assert.equal(
layer.getStats().error,
'Overpass refused the road query (HTTP 406)',
);
status = 429;
for (let i = 0; i < 20; i++) await tick(1500);
assert.equal(
layer.getStats().error,
'Overpass rate-limited',
'a rate limit is a different instruction to the reader than a refusal',
);
});
test('an unclassified road failure keeps the general line', async (t) => {
// The complement of the test above, and the reason this is not just
// `e.message`: a dropped socket throws whatever the platform felt like
// saying, and "Failed to fetch" on a panel row helps nobody.
const { layer, viewer, tick } = setup(t, async () => {
throw new Error('socket hang up');
});
layer.enable(viewer);
await tick(400);
assert.equal(layer.getStats().error, 'Road data temporarily unavailable');
});
+62
View File
@@ -15,6 +15,68 @@ function buildOverpassQuery(
return `[out:json][timeout:${timeoutSec}];(way["highway"~"${regex}"](${south},${west},${north},${east}););out geom qt;`;
}
/**
* A road request upstream declined, carrying a line fit to show an operator.
*
* The status travels beside the message instead of only inside it, so a caller
* can branch on the code without parsing English — the same shape
* `LiveSourceError` uses in `src/sources/live/contract.js`.
*/
export class RoadRequestError extends Error {
constructor(message, { status = null } = {}) {
super(message);
this.name = 'RoadRequestError';
this.status = status;
}
}
/**
* Name an Overpass refusal in the words the traffic row will print.
*
* This layer has two upstreams — OpenStreetMap for road geometry and TomTom
* for flow — and only one of them can be down at a time. A row that says
* nothing more than "road data unavailable" sends the reader to check their
* TomTom key, which is the wrong half of the layer and costs them the
* afternoon. So the status upstream actually returned is reported: 406 from a
* public mirror is not a configuration problem the reader can fix, and saying
* so is the difference between a dead end and a next step.
*
* The first two lines match `alpr`'s Overpass source word for word; this is
* the same vocabulary, not a new one. What the status adds is *which* upstream
* and *how* it declined — the part a single-source layer never needs to say.
*
* Only what the browser can see is claimed. The proxy rotates mirrors behind
* `/api/overpass`, so a refusal arriving here means the proxy had nothing
* better to offer; how many mirrors it tried is not something this side knows,
* and the message does not pretend otherwise.
* @param {number} status - HTTP status the proxy returned; a code that is
* not a finite number is treated as absent rather than printed.
* @returns {RoadRequestError}
*/
export function roadRequestError(status) {
const code = Number.isFinite(status) ? status : null;
const message =
code === 429
? 'Overpass rate-limited'
: code === 504
? 'Overpass timed out'
: // Our own proxy answers these, not a mirror: 502 when every mirror
// failed at the network level, 503 from its local concurrency
// limiter before any mirror was asked. Neither is a refusal.
code === 502
? 'Overpass mirrors unreachable'
: code === 503
? 'Overpass temporarily unavailable'
: // A source is injected, so a caller's adapter may hand back a
// refusal with no status on it. "HTTP undefined" on a panel row is
// worse than not naming a number, so an unreadable code falls back
// to what `alpr` says when it cannot be more specific either.
code === null
? 'Overpass temporarily unavailable'
: `Overpass refused the road query (HTTP ${code})`;
return new RoadRequestError(message, { status: code });
}
/** Supply road responses, flow availability and one decoded flow cache. */
export function createTrafficSource({
fetchImpl = (...args) => globalThis.fetch(...args),
+47 -1
View File
@@ -1,7 +1,11 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { readFileSync } from 'node:fs';
import { createTrafficSource } from './source.js';
import {
RoadRequestError,
createTrafficSource,
roadRequestError,
} from './source.js';
import { clampBoundsAroundCenter } from '../../data/trafficBounds.js';
const bounds = { south: 30.267, west: -97.744, north: 30.268, east: -97.743 };
const fixture = readFileSync(
@@ -188,3 +192,45 @@ test('road sources decode direction and coordinates before scene construction',
],
});
});
test('an Overpass refusal is named by status, and keeps the status beside the words', () => {
// The panel prints this string verbatim, so the mapping is part of the
// layer's contract with the reader, not an implementation detail.
assert.equal(roadRequestError(429).message, 'Overpass rate-limited');
assert.equal(roadRequestError(504).message, 'Overpass timed out');
assert.equal(
roadRequestError(406).message,
'Overpass refused the road query (HTTP 406)',
);
assert.equal(
roadRequestError(500).message,
'Overpass refused the road query (HTTP 500)',
);
// 502 and 503 come from the local proxy (mirrors unreachable, local
// limiter busy), so they must not read as a mirror refusing the query.
assert.equal(roadRequestError(502).message, 'Overpass mirrors unreachable');
assert.equal(
roadRequestError(503).message,
'Overpass temporarily unavailable',
);
assert.equal(roadRequestError(503).status, 503);
// An injected source may refuse without a readable code; a row must never
// print "HTTP undefined" at a reader.
for (const missing of [undefined, null, NaN, 'four-oh-six']) {
const vague = roadRequestError(missing);
assert.equal(vague.message, 'Overpass temporarily unavailable');
assert.equal(vague.status, null, 'an unreadable code is absent, not zero');
}
const refused = roadRequestError(406);
assert.ok(refused instanceof RoadRequestError);
assert.ok(refused instanceof Error);
assert.equal(refused.name, 'RoadRequestError');
assert.equal(
refused.status,
406,
'a caller must be able to branch on the code without parsing English',
);
});
+87 -9
View File
@@ -1,7 +1,19 @@
import { ensureGeoidReady, geoidHeight } from '../data/geoid.js';
/** Construct an instance-owned terrainHeights service with explicit dependencies. */
export function createTerrainHeights({ source, signal }) {
/** Default live-entry ceiling for the in-memory cache (see CACHE_MAX_ENTRIES). */
export const DEFAULT_MAX_CACHE_ENTRIES = 20_000;
/**
* Construct an instance-owned terrainHeights service with explicit dependencies.
* `maxCacheEntries` overrides the cache bound; it exists so the bound is
* testable without minting twenty thousand coordinates, mirroring the injectable
* `maxBytes` on the GBFS reader.
*/
export function createTerrainHeights({
source,
signal,
maxCacheEntries = DEFAULT_MAX_CACHE_ENTRIES,
}) {
if (typeof source?.getHeights !== 'function')
throw new TypeError('Terrain heights require a source');
signal?.throwIfAborted();
@@ -45,15 +57,71 @@ export function createTerrainHeights({ source, signal }) {
/** Avoid repeatedly hitting a known-failing proxy from warm fallback reads. */
const GEOID_FALLBACK_COOLDOWN_MS = 60_000;
/**
* Live entry ceiling for the cache below. The rounding in point 2 of the file
* header makes repeated visits to one place share a key; it does nothing to
* bound the number of DISTINCT places a session visits, and this cache had no
* eviction, so a long session kept every coordinate it ever resolved for the
* lifetime of the page.
*
* This is a bound, not a budget. At the ~111 m floor grid a city-scale
* session stays well under it, so eviction is the pathological case rather
* than the normal one and a well-behaved session sees no extra network. Sized
* above the sibling caches (adsb.lol points 80, track history 200, TomTom
* tiles 256) because a miss here costs a proxy round trip, not just RAM.
*/
const CACHE_MAX_ENTRIES =
Number.isFinite(maxCacheEntries) && maxCacheEntries > 0
? Math.floor(maxCacheEntries)
: DEFAULT_MAX_CACHE_ENTRIES;
/**
* In-memory cache: `"lat.toFixed(5),lon.toFixed(5)"` -> `{ellipsoid, source}`.
* Module-scoped (not exported) — the only reads are through
* `cachedEllipsoidalGround` and the internal lookup in
* `resolveEllipsoidalGround`.
* `resolveEllipsoidalGround`. Bounded at CACHE_MAX_ENTRIES; write through
* `cacheStore` and read a consumer-facing hit through `cacheTouch` so the
* eviction order stays least-recently-used.
* @type {Map<string, {ellipsoid: number, source: 'reearth'|'geoid-fallback', retryAt?: number}>}
*/
const cache = new Map();
/**
* Store an entry and drop the coldest ones once past the cap. `Map` iterates
* in insertion order, so deleting the first key evicts the least recently
* used provided every write and every consumer read re-inserts its key.
*
* Recency rather than plain insertion order because revisits here are
* spatial: a session comes back to cells near where it already was, so the
* cell it is parked on must not be evicted merely for having been resolved
* early.
* @param {string} key
* @param {{ellipsoid: number, source: 'reearth'|'geoid-fallback', retryAt?: number}} entry
*/
function cacheStore(key, entry) {
cache.delete(key);
cache.set(key, entry);
while (cache.size > CACHE_MAX_ENTRIES) {
const coldest = cache.keys().next().value;
if (coldest === undefined) break;
cache.delete(coldest);
}
}
/**
* Read an entry and promote it to newest, so a coordinate a consumer keeps
* asking about survives eviction. Returns undefined on a miss.
* @param {string} key
* @returns {{ellipsoid: number, source: 'reearth'|'geoid-fallback', retryAt?: number}|undefined}
*/
function cacheTouch(key) {
const entry = cache.get(key);
if (entry === undefined) return undefined;
cache.delete(key);
cache.set(key, entry);
return entry;
}
/**
* Builds the rounded cache key shared between the in-memory cache and the
* `points` string sent to the proxy (see file header, point 2).
@@ -74,7 +142,7 @@ export function createTerrainHeights({ source, signal }) {
* @returns {number|null}
*/
function cachedEllipsoidalGround(lat, lon) {
const entry = cache.get(cacheKey(lat, lon));
const entry = cacheTouch(cacheKey(lat, lon));
return entry ? entry.ellipsoid : null;
}
@@ -91,7 +159,7 @@ export function createTerrainHeights({ source, signal }) {
* @returns {number|null}
*/
function cachedRealEllipsoidalGround(lat, lon) {
const entry = cache.get(cacheKey(lat, lon));
const entry = cacheTouch(cacheKey(lat, lon));
return entry && entry.source === 'reearth' ? entry.ellipsoid : null;
}
@@ -213,6 +281,12 @@ export function createTerrainHeights({ source, signal }) {
uncached.push(item);
}
// What this call resolved, so the assembly below can report a point even
// if a later chunk's writes pushed it past CACHE_MAX_ENTRIES. Without it a
// batch big enough to evict its own earlier chunks would report points as
// unresolved that it had in fact just resolved.
const resolvedThisCall = new Map();
// Resolve the network path in sequential <=CHUNK_SIZE chunks. Each chunk's
// failure is isolated to that chunk's points (geoid fallback), so a single
// bad chunk doesn't lose results for the rest of a large batch.
@@ -230,7 +304,9 @@ export function createTerrainHeights({ source, signal }) {
// one contact frozen at the geoid while its neighbors resolved).
// An omitted point now caches nothing and retries on the next warm.
if (Number.isFinite(ellipsoid)) {
cache.set(item.key, { ellipsoid, source: 'reearth' });
const entry = { ellipsoid, source: 'reearth' };
resolvedThisCall.set(item.key, entry);
cacheStore(item.key, entry);
}
}
} catch {
@@ -247,11 +323,13 @@ export function createTerrainHeights({ source, signal }) {
item.lon,
item.sourceOrthometricM,
);
cache.set(item.key, {
const entry = {
ellipsoid,
source: 'geoid-fallback',
retryAt: Date.now() + GEOID_FALLBACK_COOLDOWN_MS,
});
};
resolvedThisCall.set(item.key, entry);
cacheStore(item.key, entry);
}
}
}
@@ -260,7 +338,7 @@ export function createTerrainHeights({ source, signal }) {
// the upstream omitted has NO entry (round 6 — deliberately uncached so it
// retries later): report it unresolved instead of throwing.
return work.map((item) => {
const entry = cache.get(item.key);
const entry = cache.get(item.key) || resolvedThisCall.get(item.key);
return entry
? { ellipsoid: entry.ellipsoid, source: entry.source }
: { ellipsoid: null, source: 'unresolved' };
+130
View File
@@ -0,0 +1,130 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import {
createTerrainHeights,
DEFAULT_MAX_CACHE_ENTRIES,
} from './terrainHeights.js';
/**
* A terrain source that always resolves, so these tests exercise the cache and
* never the geoid-fallback path. Heights are derived from the coordinate so an
* assertion can name the value it expects.
*/
function stubSource() {
const batches = [];
return {
batches,
async getHeights(chunk) {
batches.push(chunk.map((item) => item.key));
return chunk.map((item) => ({ ellipsoid: heightAt(item.lon) }));
},
};
}
/** Distinct 5-decimal coordinates along the equator: 0.001, 0.002, … */
const lonAt = (i) => i / 1000;
const heightAt = (lon) => 1000 + Math.round(lon * 1000);
const pointAt = (i) => ({ lat: 0, lon: lonAt(i) });
test('the default cache bound is exported so callers can reason about it', () => {
assert.equal(DEFAULT_MAX_CACHE_ENTRIES, 20_000);
});
test('the in-memory cache stops growing once it reaches its entry bound', async () => {
const terrain = createTerrainHeights({
source: stubSource(),
maxCacheEntries: 4,
});
// Six distinct coordinates through a cache that holds four.
for (let i = 0; i < 6; i += 1) {
await terrain.resolveEllipsoidalGround([pointAt(i)]);
}
// The two coldest are gone. Before the bound existed the cache kept every
// coordinate a session ever resolved, so both of these still answered.
assert.equal(terrain.cachedEllipsoidalGround(0, lonAt(0)), null);
assert.equal(terrain.cachedEllipsoidalGround(0, lonAt(1)), null);
// The four most recent are untouched — a bound, not a flush.
for (let i = 2; i < 6; i += 1) {
assert.equal(
terrain.cachedEllipsoidalGround(0, lonAt(i)),
heightAt(lonAt(i)),
`coordinate ${i} stayed warm`,
);
}
});
test('a coordinate a consumer keeps reading outlives an equally old one it ignores', async () => {
const terrain = createTerrainHeights({
source: stubSource(),
maxCacheEntries: 4,
});
for (let i = 0; i < 4; i += 1) {
await terrain.resolveEllipsoidalGround([pointAt(i)]);
}
// Read coordinate 0 and ignore coordinate 1. They are otherwise the same age,
// so insertion order alone could not tell them apart.
assert.equal(
terrain.cachedEllipsoidalGround(0, lonAt(0)),
heightAt(lonAt(0)),
);
// Two fresh coordinates push the cache two entries past its bound.
for (let i = 4; i < 6; i += 1) {
await terrain.resolveEllipsoidalGround([pointAt(i)]);
}
assert.equal(
terrain.cachedEllipsoidalGround(0, lonAt(0)),
heightAt(lonAt(0)),
'the cell being read survived eviction',
);
assert.equal(
terrain.cachedEllipsoidalGround(0, lonAt(1)),
null,
'the same-age cell nobody read was evicted instead',
);
});
test('a batch larger than the cache bound still reports every point it resolved', async () => {
const terrain = createTerrainHeights({
source: stubSource(),
maxCacheEntries: 4,
});
const batch = Array.from({ length: 10 }, (_, i) => pointAt(i));
const results = await terrain.resolveEllipsoidalGround(batch);
// Eviction inside the call must not turn a point this batch resolved into an
// unresolved one: the results are assembled from what the call produced, not
// from whatever survived in the cache by the time it finished.
assert.equal(results.length, batch.length);
results.forEach((result, i) => {
assert.equal(result.source, 'reearth', `point ${i} reported resolved`);
assert.equal(result.ellipsoid, heightAt(lonAt(i)));
});
// The cache still honours the bound afterwards.
const warm = batch.filter(
(point) => terrain.cachedEllipsoidalGround(point.lat, point.lon) !== null,
);
assert.equal(warm.length, 4);
});
test('an out-of-range cache bound falls back to the default instead of disabling the cache', async () => {
for (const maxCacheEntries of [0, -1, Number.NaN, undefined]) {
const terrain = createTerrainHeights({
source: stubSource(),
maxCacheEntries,
});
await terrain.resolveEllipsoidalGround([pointAt(1)]);
assert.equal(
terrain.cachedEllipsoidalGround(0, lonAt(1)),
heightAt(lonAt(1)),
`maxCacheEntries=${String(maxCacheEntries)} kept a working cache`,
);
}
});
+28
View File
@@ -353,3 +353,31 @@ test('military cooldown bounds untrusted Retry-After and defaults server errors'
assert.equal(Number(result.headers['retry-after']), seconds);
}
});
test('track backfill proxy returns 502 on an oversized upstream body and caches it as an error', async (t) => {
const tracks = install(providers.trackBackfillProxies(), true);
let callCount = 0;
t.mock.method(globalThis, 'fetch', async () => {
callCount++;
return {
ok: true,
status: 200,
headers: new Map(),
body: (async function* () {
yield Buffer.alloc(6 * 1024 * 1024, 'x');
})(),
};
});
const res1 = await tracks('/api/opensky-track', '?icao24=def456');
assert.equal(res1.statusCode, 502);
assert.deepEqual(JSON.parse(res1.body), {
error: 'Upstream track response too large',
});
// Cached as a 502 (never a 200): a retry inside the window neither
// reads as an empty track nor spends OpenSky credits on another download.
const res2 = await tracks('/api/opensky-track', '?icao24=def456');
assert.equal(res2.statusCode, 502);
assert.equal(callCount, 1);
});