Merge pull request #1308 from bilawalsidhu/codex/overpass-admission-review-fixes

fix(overpass): keyless gate behind a reverse proxy, loopback admission headroom, per-phase arrival budgets
This commit is contained in:
Said Turksever authored and GitHub committed 2026-11-03 15:27:19 +00:00
commit 3ad3cc2ab7
15 files changed
+784 -161

No files matched your search

+1 -1
View File
@@ -212,7 +212,7 @@ Keep the `env`: bash treats `UID` as read-only, so a plain `UID=1026` in front o
On SELinux hosts (Fedora, RHEL), compose mounts the checkout with `:z` so the container may read it; Docker may relabel the checkout for container access, which can take a moment on a large checkout. Without that, the start fails with `Cannot find module '/app/scripts/docker-start.mjs'`.
Behind a reverse proxy, put your keys in `.env` or pass them as environment variables - Provider Settings refuses proxied requests on purpose - and add the hostname you serve it on, for example `GEV_ALLOWED_HOSTS=globe.example.lan` (see [Sharing an instance](#-sharing-an-instance)).
Behind a reverse proxy, put your keys in `.env` or pass them as environment variables - Provider Settings refuses proxied requests on purpose - and add the hostname you serve it on, for example `GEV_ALLOWED_HOSTS=globe.example.lan` (see [Sharing an instance](#-sharing-an-instance)). The map routes compare the browser's `Origin` with the `Host` the server sees, so the proxy must pass the original `Host` and, when it terminates TLS, `X-Forwarded-Proto`. Caddy does both by default; for nginx add `proxy_set_header Host $http_host;`, `proxy_set_header X-Forwarded-Proto $scheme;` and `proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;`. On an install outside Docker, that forwarding header also rate-limits the proxy's users apart from your own browser. Under the default Docker bridge network every client on the host, your browser included, reaches the container from its gateway address and shares one Overpass bucket.
</details>
+1 -1
View File
@@ -80,7 +80,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur
- **Sanitized errors** — internal error details are not echoed back to clients.
- **Coalesced OAuth refresh** and cached successful responses only (OpenSky).
- **Redacted debug logging.** The voice debug log (`.gev-logs/`, gitignored) strips API keys, bearer tokens, client secrets, and image data URLs before writing.
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`), quota-bearing (`/api/mapillary/*`, with a per-IP limit on tiles, and `/api/tomtom/*`, backed by its persistent daily tile budget) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control. The keyless terrain route (`/api/terrain/heights`) uses the same cross-site checks, minus the forwarding-header refusal: it guards only the per-client quota and point cache, not a key or a bill, and the map needs it to render behind a reverse proxy. The CCTV routes (`/api/cctv/*`) use the same keyless checks; their one key-bearing step, the Street View fallback, is skipped for any request with forwarding headers.
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`), quota-bearing (`/api/mapillary/*`, with a per-IP limit on tiles, and `/api/tomtom/*`, backed by its persistent daily tile budget) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control. The keyless map routes (`/api/terrain/heights` and `/api/overpass`) use the same cross-site checks, minus the forwarding-header refusal: they guard only per-client quotas and caches, not a key or a bill, and the map needs both to render behind a reverse proxy. `/api/overpass` also admits at most 300 POSTs a minute per peer (IPv6 peers keyed by /64) and 900 a minute across all remote peers before it reads a body. A direct loopback request — a loopback socket with none of the forwarding headers `hasProxySignals` checks (`Forwarded`, `Via`, `X-Forwarded-For`, `X-Forwarded-Host`, `X-Forwarded-Port`, `X-Forwarded-Proto`, `X-Real-IP`, `CF-Connecting-IP`, `CF-Ray`) — draws on its own per-peer admission window, so remote peers cannot exhaust the owner's admission. A reverse proxy on the same machine must add at least one of those headers (Caddy and Cloudflare do by default; nginx needs `proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;`, plus `proxy_set_header Host $http_host;` and `proxy_set_header X-Forwarded-Proto $scheme;` so the `Origin` check matches), or its traffic is indistinguishable from the owner's and shares the owner's window. With one, its requests are then counted as remote, in one bucket per proxy socket — the forwarded client address is never trusted as identity — so all users behind one proxy share 300/min. The isolation needs a loopback socket: under the default Docker bridge network every host-side client, the owner included, arrives from the container gateway and is one remote peer. **Limit:** this isolates admission only. The upstream budget (90 requests/min per client key, 300/min across all clients) is shared, so once remote traffic has used the global upstream budget, an owner request that misses the cache can still get `429`, or `503` when remote misses hold every upstream concurrency slot; cache hits are unaffected. The CCTV routes (`/api/cctv/*`) use the same keyless checks; their one key-bearing step, the Street View fallback, is skipped for any request with forwarding headers.
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads and the Maps JavaScript API that Street View runs on, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
Allowing `maps.googleapis.com` in `script-src` also trusts its JSONP endpoints, which can bypass the script allowlist if an HTML-injection bug exists; this is an accepted limitation of loading the Maps JavaScript API, alongside the existing `'unsafe-eval'` allowance.
@@ -0,0 +1,13 @@
- Keep Overpass-backed layers working behind a reverse proxy: `/api/overpass`
now uses the keyless same-site gate, admits forwarding headers and still
refuses cross-site requests. Direct loopback requests keep their own
admission window; proxied requests, including those a local reverse proxy
relays over loopback, are admitted as remote traffic keyed by their socket,
and IPv6 peers are keyed by /64, so neither LAN peers nor proxied users can
exhaust the owner's admission. The upstream budget stays shared, so an owner
cache miss can still be rate-limited (asturksever, #1308).
- Give the tracking arrival check a wall-clock budget per observed frame and
retry only an inconclusive timeout: a timeout keeps every angle it
measured, including a late frame's, and a wrong one fails without retry, so slow software-rendered
frames no longer fail the check and a real violation is never retried into a
pass (asturksever, #1308).
+10 -4
View File
@@ -3268,9 +3268,15 @@ down | auth-failed` (plus the unchanged `missing-key`/`unsupported`) with
> first later successful fetch. Smaller chunks reduce timeout risk; complete
> camera batches still wait for their sequential requests, and unresolved
> placement continues to use the existing prior until real heights arrive.
> - **Overpass cache admission:** `/api/overpass` refuses cross-site and forwarded
> POSTs through the shared local-request gate, then limits requests to 300/min
> per socket peer and 900/min globally before reading or validating bodies.
> - **Overpass cache admission:** `/api/overpass` refuses cross-site POSTs
> through the keyless same-site gate (reverse-proxy forwarding headers are
> admitted, as for `/api/terrain/heights`), then limits requests to 300/min
> per peer — IPv6 peers keyed by /64 — and 900/min across all remote peers
> before reading or validating bodies. Direct loopback (no forwarding
> headers) has its own 300/min window outside that global; a local reverse
> proxy's requests carry forwarding headers and are counted as remote, keyed
> `proxied:<socket peer>`, so they cannot spend the owner's window. Upstream
> quota stays shared, so owner cache misses can still hit 429.
> Tag-filter brackets are scanned once; nested or unmatched brackets return 400.
> It then checks fresh memory, identical in-flight work, and fresh disk entries
> before invoking its separate local 90/min upstream limiter. Cache and
@@ -5704,7 +5710,7 @@ pins and share links already supply complete boxes.
- Fleet model eligibility is distance-based with on-screen priority and hard caps (`MODEL_MAX`, `MODEL_MAX_ALL`) to avoid draw-call explosions. Each model owns its own `modelMatrix`; shared scratch matrices are forbidden because they caused stacking/flicker.
- Tracked aircraft use standalone model primitives driven from the already-settled dead-reckoned display position, while the tracked Cesium entity remains billboard-backed so `viewer.trackedEntity` always has a ready bounding sphere.
- Flights and military layers mirror the same tracking invariants: no warm-up freeze/jump, altitude-scaled framing, trail head glued to the displayed plane, no pull-out when switching targets, and no cross-layer orphan when switching between commercial and military tracks.
- Regression surface: `npm run test:track` drives the real app headless with synthetic aircraft feeds and asserts the tracking invariants without depending on live OpenSky/adsb data. Arrival-rotation checks observe the first reveal within a bounded wall-clock window before the one-second idle fallback, rather than assuming the 80 ms fleet cadence fits inside four rendered frames; an incorrect first orientation fails immediately. `src/data/trackedModelRegime.test.mjs` pins the tracked contact's threshold math, the enter/exit asymmetry, and the default-on / cockpit / TR-3B / deselect wiring in both layers.
- Regression surface: `npm run test:track` drives the real app headless with synthetic aircraft feeds and asserts the tracking invariants without depending on live OpenSky/adsb data. Arrival-rotation checks observe the settle frame and then the first reveal frame, each within a 500 ms wall-clock budget that sits inside the one-second idle fallback, rather than assuming the 80 ms fleet cadence fits inside four rendered frames; an incorrect first orientation fails immediately. Only an inconclusive timeout is retried (once); a timeout keeps every angle it measured (the settle frame before a reveal timeout, and the late frame itself, which is read before the deadline check), so a measured wrong angle fails without retry. `src/data/trackedModelRegime.test.mjs` pins the tracked contact's threshold math, the enter/exit asymmetry, and the default-on / cockpit / TR-3B / deselect wiring in both layers.
### TR-3B conversion Easter egg (August 2026)
+2 -97
View File
@@ -1102,6 +1102,7 @@
"scripts/qa-anime-sky.mjs",
"scripts/qa-sky-model-lighting.mjs",
"server/providers/overpass/requestAdmission.test.mjs",
"src/tooling/trackArrivalObservation.mjs",
"src/tooling/trackArrivalObservation.test.mjs",
"src/ui/styles/status-badge.css",
"src/ui/cctvBadges.test.mjs",
@@ -1110,101 +1111,5 @@
"src/cameraGlobeRotation.test.mjs",
"src/voice/globeResetConfirmation.test.mjs",
"src/hudDms.test.mjs",
"src/hudSummaryGate.test.mjs",
"src/hudSummaryLine.test.mjs",
"src/testSupport/stubMgrs.mjs",
"src/tools/queries/hud-caption-feed-state.test.mjs",
"src/voice/hudPlaceLabels.test.mjs",
"src/voice/panel/preferences.test.mjs",
"src/voice/panel/preferencesView.test.mjs",
"src/voice/voiceControlView.test.mjs",
"src/voice/panel/host.test.mjs",
"src/voice/panel/manualPanel.test.mjs",
"src/voice/panel/nativeActivity.test.mjs",
"src/voice/panel/inactivity.test.mjs",
"src/voice/panel/actionObserver.test.mjs",
"src/voice/realtimeCostPanel.test.mjs",
"docs/SHARE-LINK-FORMAT.md",
"src/share/wire.test.mjs",
"src/share/viewCodec.test.mjs",
"src/share/legacyShareState.test.mjs",
"src/testSupport/shareViewFixtures.mjs",
"src/ui/shareLinks.test.mjs",
"src/ui/shareButton.test.mjs",
"src/share/shareLinkPath.test.mjs",
"src/ui/kit/UiRail.test.mjs",
"src/ui/kit/UiSegmented.vue",
"src/ui/kit/UiSegmented.test.mjs",
"src/ui/kit/UiSlider.js",
"src/ui/kit/UiSlider.test.mjs",
"src/ui/kit/UiButton.vue",
"src/ui/kit/UiButton.test.mjs",
"src/ui/kit/UiChip.vue",
"src/ui/kit/UiChip.test.mjs",
"src/ui/globeActions.test.mjs",
"src/ui/safeFrameView.test.mjs",
"src/ui/kit/UiSelect.js",
"src/ui/kit/UiSelect.test.mjs",
"src/ui/kit/UiBadge.vue",
"src/ui/kit/UiBadge.test.mjs",
"src/ui/kit/UiEmpty.vue",
"src/ui/rail/RailChips.vue",
"src/ui/displayPanelActions.test.mjs",
"src/testSupport/vueDom.mjs",
"build/vueLoader.js",
"build/registerVue.js",
"src/ui/panelNavigation.test.mjs",
"src/ui/stores/owners.js",
"src/ui/stores/owners.test.mjs",
"src/ui/stores/panels.test.mjs",
"src/ui/stores/status.js",
"src/ui/statusChipsView.test.mjs",
"src/ui/statusChipOwners.test.mjs",
"src/ui/toast.js",
"src/ui/toastView.test.mjs",
"src/ui/stores/sceneInteractions.js",
"src/scenes/dataPacks/presentationView.test.mjs",
"src/ui/stores/dataPacks.js",
"src/scenes/interactionsView.test.mjs",
"src/layers/wind/inspectionMarker.js",
"src/layers/wind/inspectionMarker.test.mjs",
"src/ui/stores/windInspection.js",
"src/ui/stores/missionReplay.js",
"src/layers/launches/replayOverlay.test.mjs",
"src/ui/stores/bhoteKoshiPanel.js",
"src/data/bhoteKoshiPanelView.test.mjs",
"src/ui/stores/celestialRing.js",
"src/celestialRingView.test.mjs",
"src/ui/stores/screenAnnotations.js",
"src/ui/stores/worldOverlayActions.js",
"src/overlays/worldOverlayActionsView.test.mjs",
"src/ui/stores/awarenessDirection.js",
"src/layers/awareness/directionOverlay.test.mjs",
"src/hudReadouts.js",
"src/ui/stores/hud.js",
"src/ui/views/intelHudView.test.mjs",
"src/testSupport/hudReadouts.mjs",
"src/ui/stores/firstRun.js",
"src/firstRunLauncherView.test.mjs",
"src/firstRunCopy.js",
"src/data/headingDegrees.test.mjs",
"src/data/distanceFormat.test.mjs",
"src/sources/adsbFields.test.mjs",
"src/sources/phaseTiming.test.mjs",
"scripts/build-census-places.mjs",
"src/data/placeBoundaries.test.mjs",
"src/data/boundaryLabels.test.mjs",
"src/data/packGeometry.test.mjs",
"src/annotations/placeOutline.test.mjs",
"src/annotations/outlineIdentity.test.mjs",
"src/annotations/outlineRungs.test.mjs",
"src/annotations/outlineRungsFallback.test.mjs",
"src/search/photonSetting.test.mjs",
"src/sources/nominatimOutlines.test.mjs",
"src/sources/openFreeMapOutlines.test.mjs",
"src/sources/openFreeMapOutlinesEdges.test.mjs",
"src/standalone/layerSources.test.mjs",
"src/tooling/nominatimGuardLifecycle.test.mjs",
"src/tooling/nominatimLockOwnership.test.mjs",
"src/tooling/nominatimOutlineRoute.test.mjs"
"server/providers/common/rateLimitKeys.test.mjs"
]
+26 -9
View File
@@ -94,7 +94,7 @@
import fs from 'node:fs';
import puppeteer from 'puppeteer';
import { arrivalRotationPassed } from '../src/tooling/trackArrivalObservation.mjs';
import { arrivalAngleViolated, arrivalRotationPassed } from '../src/tooling/trackArrivalObservation.mjs';
import { classifyAircraft, CLASS_SCALE_3D, CLASS_MODEL_REAL } from '../src/data/aircraftClass.js';
import { ensureGeoidReady, geoidHeight } from '../src/data/geoid.js';
@@ -2698,16 +2698,18 @@ async function main() {
// Staged deterministically: park the camera nadir over the synthetic
// cluster, let rotations settle, TAMPER a billboard's rotation, then
// trigger each path with the camera idle. The fleet runs every 80 ms,
// independently of rendered FPS. Observe the first completed settle and
// reveal frames within 500 ms of forcing moveEnd, before the 1 s idle
// rotation refresh can repair an incorrect first reveal.
// independently of rendered FPS. Observe the first completed settle frame
// within 500 ms of forcing moveEnd and the first reveal frame within
// 500 ms of that settle frame: each budget sits inside the 1 s idle
// rotation refresh that could otherwise repair an incorrect first reveal,
// and a ~300 ms headless SwiftShader frame fits inside each of them.
// ============================================================
console.log('\nChange 4 — arrival rotation: moveEnd settle pass + horizon-reveal pass');
const arrival = await evalPage(async () => {
const v = window.__godsEyeView.viewer;
const dm = window.__godsEyeView.dataManager;
const { screenProjectedRotation } = await import(`${window.__gevQaSourceBase || '/src'}/data/iconOrientation.js`);
const { observeArrivalRotation } = await import(`${window.__gevQaSourceBase || '/src'}/tooling/trackArrivalObservation.mjs`);
const { observeArrivalRotationWithRetry } = await import(`${window.__gevQaSourceBase || '/src'}/tooling/trackArrivalObservation.mjs`);
// 3D models OFF for this phase: a model-handed-off billboard is hidden
// and skips rotation updates entirely — the probes need live billboards
// (this is also the app's default state the field report came from).
@@ -2769,7 +2771,23 @@ async function main() {
const bb = findBB(id);
if (!bb || !bb.show) return { error: `${id} missing/hidden` };
const r0 = bb.rotation; // settled tamper origin; correctness uses a fresh projection below
const observed = await observeArrivalRotation({
const hasModel = !!(window.__g3dFindModel && window.__g3dFindModel(id));
// A wall-clock timeout is inconclusive, not a wrong angle: the frame
// landed where the 1 s idle refresh could already have repaired it
// (seen once on headless SwiftShader: a 926 ms first frame). Such an
// observation gets one more attempt; one that measured a wrong angle —
// a wrong first reveal, or a wrong settle frame before a reveal
// timeout — never does.
const observed = await observeArrivalRotationWithRetry({
async prepare() {
// Freshen both layers' rotation passes right before the probe, so
// the settle frame is measured against a pass that just ran rather
// than one left over from the previous probe — with the camera
// idle, a layer otherwise runs a pass only every
// ROTATION_REFRESH_MS.
v.camera.moveEnd.raiseEvent();
await nextFrames(1);
},
scene: v.scene,
startMoveEnd() {
// Only moveEnd can repair the tampered angle at the idle pose.
@@ -2790,7 +2808,6 @@ async function main() {
};
},
});
const hasModel = !!(window.__g3dFindModel && window.__g3dFindModel(id));
return { r0, ...observed, hasModel };
};
const flightCourse = window.__SYNTH.flights.find((f) => f.icao === 'aaa002')?.track ?? 0;
@@ -2800,8 +2817,8 @@ async function main() {
military: await probe('bbb101', militaryCourse),
};
});
const fmtArr = (a) => (!a ? `no result (${arrival?.error || 'phase error'})` : a.error ? `${a.error}; elapsed=${Number(a.elapsedMs).toFixed(1)} ms`
: `settle-err=${(a.dMoveEnd * 180 / Math.PI).toFixed(1)}° reveal-err=${(a.dReveal * 180 / Math.PI).toFixed(1)}° shown=${a.shown} flipFrame=${a.flipFrame} settle=${a.moveEndMs.toFixed(1)} ms reveal=${a.revealMs.toFixed(1)} ms total=${a.elapsedMs.toFixed(1)} ms hasModel=${a.hasModel}`);
const fmtArr = (a) => (!a ? `no result (${arrival?.error || 'phase error'})` : a.error ? `${a.error}; ${arrivalAngleViolated(a) ? 'angle violation (not retried); ' : ''}${Number.isFinite(a.dMoveEnd) ? `settle-err=${(a.dMoveEnd * 180 / Math.PI).toFixed(1)}° ` : ''}${Number.isFinite(a.dReveal) ? `reveal-err=${(a.dReveal * 180 / Math.PI).toFixed(1)}° ` : ''}elapsed=${Number(a.elapsedMs).toFixed(1)} ms attempts=${a.attempts}`
: `settle-err=${(a.dMoveEnd * 180 / Math.PI).toFixed(1)}° reveal-err=${(a.dReveal * 180 / Math.PI).toFixed(1)}° shown=${a.shown} flipFrame=${a.flipFrame} settle=${a.moveEndMs.toFixed(1)} ms reveal=${a.revealMs.toFixed(1)} ms total=${a.elapsedMs.toFixed(1)} ms attempts=${a.attempts} hasModel=${a.hasModel}`);
const TAMPER_DEG = 1.5 * 180 / Math.PI; // ≈ 86° — an uncorrected nose stays this far off
record(`arrival: moveEnd + first reveal both re-aim before idle refresh, flights (tamper was ${TAMPER_DEG.toFixed(0)}°)`,
arrivalRotationPassed(arrival.flights), fmtArr(arrival.flights));
+44
View File
@@ -1,3 +1,4 @@
import { isIPv6 } from 'node:net';
import { makeRateLimiter } from '../../../src/sources/rateLimit.js';
export { makeRateLimiter } from '../../../src/sources/rateLimit.js';
@@ -60,3 +61,46 @@ export function makeCostRateLimiter(envValue, defaultPerMin) {
export function clientKey(req) {
return String(req.socket?.remoteAddress || 'local');
}
const IPV4_MAPPED_RE = /^::ffff:(\d{1,3}(?:\.\d{1,3}){3})$/i;
/**
* Whether a socket peer address is this machine: IPv4 loopback in any of its
* forms (`127.0.0.0/8`, including the IPv4-mapped `::ffff:127.x` a dual-stack
* listener reports) or the IPv6 loopback. Non-address keys are not loopback.
* @param {string} address - A `clientKey` value.
* @returns {boolean}
*/
export function isLoopbackPeer(address) {
const raw = String(address || '');
const v4 = IPV4_MAPPED_RE.exec(raw)?.[1] ?? raw;
return v4.startsWith('127.') || raw === '::1';
}
/**
* Rate-limit key that survives address rotation, for limiters whose global
* backstop everyone shares. IPv6 privacy extensions hand one host a fresh
* address at will inside its /64, so keying by the full address lets a single
* host mint per-peer quota until the global window closes for everyone else.
* IPv6 peers are keyed by their /64 prefix instead, and an IPv4-mapped
* address folds to its IPv4 form so one host cannot hold two buckets. Any
* other value (IPv4, or the `local` fallback) passes through unchanged.
* @param {import('node:http').IncomingMessage} req
* @returns {string}
*/
export function clientNetworkKey(req) {
const address = clientKey(req);
const mapped = IPV4_MAPPED_RE.exec(address);
if (mapped) return mapped[1];
const bare = address.replace(/%.*$/, ''); // link-local zone id
if (!isIPv6(bare)) return address;
const [head, tail = ''] = bare.split('::');
const headParts = head ? head.split(':') : [];
const tailParts = tail ? tail.split(':') : [];
const missing = Math.max(0, 8 - headParts.length - tailParts.length);
const hextets = [...headParts, ...Array(missing).fill('0'), ...tailParts];
return `${hextets
.slice(0, 4)
.map((h) => h.toLowerCase().replace(/^0+(?=.)/, ''))
.join(':')}::/64`;
}
@@ -0,0 +1,37 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { clientNetworkKey, isLoopbackPeer } from './rate-limit.js';
const req = (remoteAddress) => ({ socket: { remoteAddress } });
test('clientNetworkKey keys IPv6 peers by /64 and folds IPv4-mapped peers', () => {
assert.equal(clientNetworkKey(req('2001:db8:1:2::1')), '2001:db8:1:2::/64');
assert.equal(
clientNetworkKey(req('2001:0db8:0001:0002:abcd:ef01:2345:6789')),
'2001:db8:1:2::/64',
);
assert.equal(clientNetworkKey(req('2001:DB8::1')), '2001:db8:0:0::/64');
assert.equal(clientNetworkKey(req('fe80::1%en0')), 'fe80:0:0:0::/64');
assert.equal(clientNetworkKey(req('::1')), '0:0:0:0::/64');
assert.equal(clientNetworkKey(req('::ffff:10.0.0.5')), '10.0.0.5');
assert.equal(clientNetworkKey(req('10.0.0.5')), '10.0.0.5');
assert.equal(clientNetworkKey(req(undefined)), 'local');
assert.equal(clientNetworkKey(req('not-an-address')), 'not-an-address');
});
test('isLoopbackPeer recognizes every loopback form and nothing else', () => {
for (const address of ['127.0.0.1', '127.9.9.9', '::1', '::ffff:127.0.0.1']) {
assert.equal(isLoopbackPeer(address), true, address);
}
for (const address of [
'10.0.0.1',
'::ffff:10.0.0.1',
'2001:db8::1',
'::2',
'local',
'',
undefined,
]) {
assert.equal(isLoopbackPeer(address), false, String(address));
}
});
+1 -1
View File
@@ -58,7 +58,7 @@ function forwardedProtocol(req) {
/**
* Cross-site gate for keyless routes the map needs in order to render
* (`/api/terrain/heights`). It refuses the same cross-site browser shapes as
* (`/api/terrain/heights`, `/api/overpass`). It refuses the same cross-site browser shapes as
* `admitSameSite` (a `Sec-Fetch-Site` other than same-origin/none, a foreign
* or opaque `Origin`) but does not refuse reverse-proxy forwarding headers:
* those protect credentials and spend, which a keyless route has neither of,
+58 -9
View File
@@ -1,6 +1,12 @@
import { makeRateLimiter, clientKey } from './common/rate-limit.js';
import {
makeRateLimiter,
clientKey,
clientNetworkKey,
isLoopbackPeer,
} from './common/rate-limit.js';
import { hasProxySignals } from '../../src/localRequestGate.mjs';
import { readRequestBodyCapped } from './common/request.js';
import { admitSameSite } from './common/same-site.js';
import { admitSameSiteKeyless } from './common/same-site.js';
import {
OVERPASS_MAX_BODY_BYTES,
OVERPASS_MAX_CONCURRENT,
@@ -29,10 +35,21 @@ const _overpassInFlight = new Map();
let _overpassConcurrent = 0;
// Bound body-reading and validation work, including malformed requests and
// cache hits. Keep the tighter upstream budget below independent of this gate.
const _overpassRequestLimiter = makeRateLimiter({
windowMs: 60_000,
max: 300,
// cache hits, before a byte is read. Remote peers share one global window so
// address rotation cannot mint admission without bound (IPv6 peers are keyed
// by /64 for the same reason). A direct loopback request — the owner's own
// browser and the repo QA harnesses, with no forwarding headers — has its own
// per-peer window outside that global, so busy LAN peers cannot lock the owner
// out of the map. A request relayed by a reverse proxy on this machine also
// arrives from loopback, but it is remote traffic: it is admitted from the
// remote pool under its own socket-keyed bucket, never from the owner's. The
// forwarded client address is not trusted as identity. Only admission is
// isolated: the upstream budget below (90/min per key, 300/min globally) is
// shared, so an owner request that misses cache can still be refused there.
const OVERPASS_ADMISSION_WINDOW = { windowMs: 60_000, max: 300 };
const _overpassLoopbackAdmission = makeRateLimiter(OVERPASS_ADMISSION_WINDOW);
const _overpassPeerAdmission = makeRateLimiter({
...OVERPASS_ADMISSION_WINDOW,
globalMax: 900,
});
@@ -42,6 +59,33 @@ const _overpassRateLimiter = makeRateLimiter({
globalMax: 300,
});
/**
* Classify a request for admission. Direct loopback — a loopback socket with
* no forwarding header — is the owner's own pool; everything else, including a
* loopback socket that carries forwarding headers (a local reverse proxy), is
* remote. A proxied request keeps its socket identity under a `proxied:`
* prefix, so it never shares a bucket with the owner's direct requests.
* @param {import('http').IncomingMessage} req
* @returns {{direct: boolean, key: string}}
*/
function overpassAdmissionClass(req) {
const peer = clientNetworkKey(req);
if (!isLoopbackPeer(clientKey(req))) return { direct: false, key: peer };
if (hasProxySignals(req.headers || {}))
return { direct: false, key: `proxied:${peer}` };
return { direct: true, key: peer };
}
/**
* Spend one admission slot for this request; false when its bucket is over its
* window (or, for remote traffic, when the shared global window is full).
* @param {{direct: boolean, key: string}} admission - `overpassAdmissionClass(req)`.
* @returns {boolean}
*/
function admitOverpassRequest({ direct, key }) {
return (direct ? _overpassLoopbackAdmission : _overpassPeerAdmission)(key);
}
/**
* Write a completed Overpass payload to the HTTP response.
*
@@ -103,8 +147,13 @@ function overpassProxy({ routing = {} } = {}) {
return;
}
if (admitSameSite(req, res)) return;
if (!_overpassRequestLimiter(clientKey(req))) {
// Keyless gate: refuses cross-site browser requests but, unlike the
// cost-bearing routes, admits the forwarding headers a reverse proxy
// adds — the layers that depend on Overpass must keep working for an
// instance deployed the way the README documents.
if (admitSameSiteKeyless(req, res)) return;
const admission = overpassAdmissionClass(req);
if (!admitOverpassRequest(admission)) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
@@ -163,7 +212,7 @@ function overpassProxy({ routing = {} } = {}) {
// requests; boundary-class queries keep their month-long TTL.
readDisk: () =>
readOverpassDisk(cacheKey, overpassDiskTtlMs(cacheKey)),
allowUpstream: () => _overpassRateLimiter(clientKey(req)),
allowUpstream: () => _overpassRateLimiter(admission.key),
loadUpstream: async () => {
if (_overpassConcurrent >= OVERPASS_MAX_CONCURRENT)
return {
+10 -6
View File
@@ -122,12 +122,14 @@ function sanitizeOverpassBody(rawBody) {
const boundedSets = new Set();
for (let stmt of withoutTags.split(';')) {
stmt = stmt.trim();
if (!stmt || stmt.startsWith('[') || /^out\b/.test(stmt)) continue;
// The settings block (`[out:json][timeout:25]`) was blanked together with
// the tag filters, so it arrives here as an empty statement.
if (!stmt || /^out\b/.test(stmt)) continue;
// Strip output-set assignments (NOT input bounds). Bracket tag filters were
// removed above so a tag KEY/value (e.g. `way[is_in]`, `node[around]`) cannot be
// misread as a spatial bound. Bounds live in (...) / function calls / set
// refs, never inside [...], so the probe loses nothing real.
// Strip output-set assignments (NOT input bounds). With the tag filters
// already gone, a tag KEY/value (e.g. `way[is_in]`, `node[around]`) cannot
// be misread as a spatial bound; bounds live in (...) / function calls /
// set refs, never inside [...], so nothing real was lost.
const outSets = [];
const probe = stmt.replace(/->\s*\.(\w+)/g, (_, name) => {
outSets.push(name);
@@ -166,8 +168,10 @@ function sanitizeOverpassBody(rawBody) {
if (bounded) for (const name of outSets) boundedSets.add(name);
}
// Tolerate whitespace around the directive so `[ timeout : N ]` is clamped
// rather than passed through as written.
const clamped = data.replace(
/\[timeout:\s*(\d+)\s*\]/gi,
/\[\s*timeout\s*:\s*(\d+)\s*\]/gi,
(_, n) =>
`[timeout:${Math.min(Number(n) || OVERPASS_MAX_QL_TIMEOUT, OVERPASS_MAX_QL_TIMEOUT)}]`,
);
@@ -3,10 +3,24 @@ import assert from 'node:assert/strict';
import { Readable } from 'node:stream';
import { spawnSync } from 'node:child_process';
import { overpassProxy } from '../overpass.js';
import { _overpassCache } from './cache.js';
import { sanitizeOverpassBody } from './query.js';
const form = (query) => `data=${encodeURIComponent(query)}`;
// The limiters are module-level, so every test that spends admission mocks a
// clock a full day past the previous one in execution order: no test can see
// another's window, whatever order (or randomization) they run in. A new
// test that reaches admission must call it too: earlier tests leave
// timestamps days ahead of the real clock, and those still count.
let clockBase = Date.now() + 86_400_000;
function freshClock(t) {
clockBase += 86_400_000;
const clock = { now: clockBase };
t.mock.method(Date, 'now', () => clock.now);
return clock;
}
test('bracket validation rejects malformed filters without trusting their contents as bounds', () => {
for (const query of [
'node[',
@@ -37,6 +51,15 @@ test('bounded queries preserve quoted brackets, comments, multiple tags and time
query.replace('[timeout:999]', '[timeout:30]'),
);
}
// Whitespace inside the directive is clamped too, not passed through.
const spaced = sanitizeOverpassBody(
form('[out:json][ timeout : 999 ];node(around:20,1,1);out;'),
);
assert.equal(spaced.ok, true);
assert.equal(
new URLSearchParams(spaced.body).get('data'),
'[out:json][timeout:30];node(around:20,1,1);out;',
);
});
test('a burst of maximum-size unterminated filters completes within a bounded process deadline', () => {
@@ -126,7 +149,13 @@ for (const hook of ['configureServer', 'configurePreviewServer']) {
{ 'sec-fetch-site': 'cross-site' },
{ 'sec-fetch-site': 'same-site' },
{ origin: 'null' },
{ 'x-forwarded-for': '127.0.0.1' },
// Forwarding headers do not launder a cross-site request.
{ 'x-forwarded-for': '10.0.0.9', 'sec-fetch-site': 'cross-site' },
{
'x-forwarded-proto': 'https',
origin: 'http://localhost:4173',
'sec-fetch-site': 'same-origin',
},
]) {
const response = await call(route, { headers, forbidRead: true });
assert.equal(response.statusCode, 403);
@@ -136,12 +165,24 @@ for (const hook of ['configureServer', 'configurePreviewServer']) {
});
}
test('same-origin, LAN and non-browser requests still reach validation; GET status stays available', async () => {
test('same-origin, LAN, proxied and non-browser requests still reach validation; GET status stays available', async (t) => {
freshClock(t);
const route = handler();
for (const headers of [
{},
{ origin: 'http://localhost:4173', 'sec-fetch-site': 'same-origin' },
{ host: '192.168.1.10:4173', origin: 'http://192.168.1.10:4173' },
// The README's reverse-proxy deployment: Overpass is keyless, so the
// forwarding headers the proxy adds are admitted (as for terrain heights).
{
host: 'globe.example.lan',
origin: 'https://globe.example.lan',
'sec-fetch-site': 'same-origin',
'x-forwarded-for': '203.0.113.7',
'x-forwarded-proto': 'https',
'x-forwarded-host': 'globe.example.lan',
},
{ 'x-forwarded-for': '127.0.0.1' },
]) {
const response = await call(route, { headers });
assert.equal(response.statusCode, 400);
@@ -158,8 +199,7 @@ test('same-origin, LAN and non-browser requests still reach validation; GET stat
});
test('invalid requests spend admission quota before parsing, including on an unconfigured server', async (t) => {
let now = Date.now() + 120000;
t.mock.method(Date, 'now', () => now);
const clock = freshClock(t);
t.mock.method(globalThis, 'fetch', () =>
assert.fail('unexpected upstream fetch'),
);
@@ -173,6 +213,216 @@ test('invalid requests spend admission quota before parsing, including on an unc
assert.equal(denied.reads, 0);
assert.equal(denied.headers['retry-after'], '60');
assert.equal((await call(route, { peer: 'other-peer' })).statusCode, 400);
now += 60001;
clock.now += 60_001;
assert.equal((await call(route, { peer })).statusCode, 400);
});
test('the 900/min global window spans non-loopback peers only; loopback keeps its own window', async (t) => {
const clock = freshClock(t);
t.mock.method(globalThis, 'fetch', () =>
assert.fail('unexpected upstream fetch'),
);
const route = handler();
for (const peer of ['10.0.0.1', '10.0.0.2', '10.0.0.3']) {
for (let i = 0; i < 300; i++) {
assert.equal((await call(route, { peer })).statusCode, 400, peer);
}
}
const starved = await call(route, { peer: '10.0.0.4', forbidRead: true });
assert.equal(starved.statusCode, 429, 'global window is full for LAN peers');
assert.equal(starved.reads, 0);
for (const peer of ['127.0.0.1', '::1', '::ffff:127.0.0.1']) {
const owner = await call(route, { peer });
assert.equal(owner.statusCode, 400, `${peer} is admitted past the global`);
assert.equal(owner.reads, 1);
}
clock.now += 60_001;
assert.equal((await call(route, { peer: '10.0.0.4' })).statusCode, 400);
});
test('IPv6 peers share a /64 admission bucket and IPv4-mapped peers fold to IPv4', async (t) => {
const clock = freshClock(t);
t.mock.method(globalThis, 'fetch', () =>
assert.fail('unexpected upstream fetch'),
);
const route = handler();
for (let i = 0; i < 300; i++) {
assert.equal(
(await call(route, { peer: `2001:db8:1:2::${(i % 200) + 1}` }))
.statusCode,
400,
);
}
const rotated = await call(route, {
peer: '2001:db8:1:2:ffff:ffff:ffff:9',
forbidRead: true,
});
assert.equal(rotated.statusCode, 429, 'a rotating host stays in one bucket');
assert.equal(
(await call(route, { peer: '2001:db8:1:3::1' })).statusCode,
400,
);
for (let i = 0; i < 300; i++) {
assert.equal(
(await call(route, { peer: '::ffff:10.1.1.1' })).statusCode,
400,
);
}
const unmapped = await call(route, { peer: '10.1.1.1', forbidRead: true });
assert.equal(unmapped.statusCode, 429, 'mapped and bare IPv4 share a bucket');
clock.now += 60_001;
assert.equal((await call(route, { peer: '10.1.1.1' })).statusCode, 400);
});
test('cache hits spend admission budget but not the 90/min upstream budget', async (t) => {
const clock = freshClock(t);
const prior = process.env.OVERPASS_UPSTREAMS;
process.env.OVERPASS_UPSTREAMS = 'https://overpass.example/api/interpreter';
t.after(() => {
if (prior === undefined) delete process.env.OVERPASS_UPSTREAMS;
else process.env.OVERPASS_UPSTREAMS = prior;
_overpassCache.delete(hitKey);
});
let fetches = 0;
t.mock.method(globalThis, 'fetch', async () => {
fetches++;
// Not data: nothing is cached or written to disk by this test.
return new Response('upstream unavailable', { status: 503 });
});
const hit = 'node(around:20,1,1);out;';
const hitKey = `data=${encodeURIComponent(hit)}`;
_overpassCache.set(hitKey, {
status: 200,
body: '{"elements":[]}',
contentType: 'application/json',
endpoint: 'https://overpass.example/api/interpreter',
cachedAt: clock.now,
});
const route = handler();
const peer = '10.7.7.7';
// More cache hits than the upstream limiter would allow per peer.
for (let i = 0; i < 100; i++) {
const response = await call(route, { peer, body: form(hit) });
assert.equal(response.statusCode, 200);
assert.equal(response.headers['x-overpass-cache'], 'HIT');
}
assert.equal(fetches, 0);
const miss = await call(route, {
peer,
body: form('node(around:20,2,2);out;'),
});
assert.equal(fetches, 1, 'the first miss still reaches upstream');
assert.notEqual(miss.statusCode, 429);
for (let i = 0; i < 199; i++) {
assert.equal(
(await call(route, { peer, body: form(hit) })).statusCode,
200,
);
}
const exhausted = await call(route, {
peer,
body: form(hit),
forbidRead: true,
});
assert.equal(exhausted.statusCode, 429, 'hits spent the admission window');
assert.equal(exhausted.reads, 0);
});
test('a local reverse proxy cannot spend the direct-loopback reserve; owner cache hits survive it', async (t) => {
const clock = freshClock(t);
const prior = process.env.OVERPASS_UPSTREAMS;
process.env.OVERPASS_UPSTREAMS = 'https://overpass.example/api/interpreter';
const hit = 'node(around:20,3,3);out;';
const hitKey = `data=${encodeURIComponent(hit)}`;
t.after(() => {
if (prior === undefined) delete process.env.OVERPASS_UPSTREAMS;
else process.env.OVERPASS_UPSTREAMS = prior;
_overpassCache.delete(hitKey);
});
t.mock.method(globalThis, 'fetch', () =>
assert.fail('unexpected upstream fetch'),
);
_overpassCache.set(hitKey, {
status: 200,
body: '{"elements":[]}',
contentType: 'application/json',
endpoint: 'https://overpass.example/api/interpreter',
cachedAt: clock.now,
});
const route = handler();
// The proxy connects over loopback; a rotating forwarded client address
// must not mint fresh buckets, so all 300 land in one socket-keyed bucket.
for (const proxyPeer of ['127.0.0.1', '::1', '::ffff:127.0.0.1']) {
const proxied = (i) => ({
peer: proxyPeer,
headers: { 'x-forwarded-for': `203.0.113.${i % 250}` },
body: form(hit),
});
for (let i = 0; i < 300; i++) {
assert.equal((await call(route, proxied(i))).statusCode, 200, proxyPeer);
}
const refused = await call(route, { ...proxied(300), forbidRead: true });
assert.equal(refused.statusCode, 429, `${proxyPeer} proxy bucket is full`);
assert.equal(refused.reads, 0);
for (const peer of ['127.0.0.1', '::1']) {
const owner = await call(route, { peer, body: form(hit) });
assert.equal(owner.statusCode, 200, `direct ${peer} is still admitted`);
assert.equal(owner.headers['x-overpass-cache'], 'HIT');
}
clock.now += 60_001;
assert.equal((await call(route, proxied(0))).statusCode, 200);
}
});
test('proxied loopback traffic draws on the shared remote global window', async (t) => {
const clock = freshClock(t);
t.mock.method(globalThis, 'fetch', () =>
assert.fail('unexpected upstream fetch'),
);
const route = handler();
for (const peer of ['10.0.0.1', '10.0.0.2']) {
for (let i = 0; i < 300; i++) {
assert.equal((await call(route, { peer })).statusCode, 400, peer);
}
}
for (let i = 0; i < 300; i++) {
assert.equal(
(await call(route, { headers: { via: '1.1 proxy' } })).statusCode,
400,
);
}
const starved = await call(route, { peer: '10.0.0.3', forbidRead: true });
assert.equal(starved.statusCode, 429, 'proxied requests counted globally');
assert.equal((await call(route, {})).statusCode, 400, 'owner unaffected');
});
test("a local proxy has its own upstream bucket, so its misses cannot spend the owner's", async (t) => {
freshClock(t);
const prior = process.env.OVERPASS_UPSTREAMS;
process.env.OVERPASS_UPSTREAMS = 'https://overpass.example/api/interpreter';
t.after(() => {
if (prior === undefined) delete process.env.OVERPASS_UPSTREAMS;
else process.env.OVERPASS_UPSTREAMS = prior;
});
// Not data: nothing is cached, so every request below is a miss.
t.mock.method(
globalThis,
'fetch',
async () => new Response('upstream unavailable', { status: 503 }),
);
const route = handler();
const miss = form('node(around:20,4,4);out;');
// The upstream limiter's own refusal is the only miss without this header.
const passedLimiter = (response) =>
response.headers['x-overpass-cache'] !== undefined;
const proxied = { peer: '127.0.0.1', headers: { via: '1.1 proxy' } };
for (let i = 0; i < 90; i++) {
const response = await call(route, { ...proxied, body: miss });
assert.equal(passedLimiter(response), true, `proxied miss ${i}`);
}
const spent = await call(route, { ...proxied, body: miss });
assert.equal(spent.statusCode, 429);
assert.equal(passedLimiter(spent), false, 'the proxy bucket is spent');
const owner = await call(route, { peer: '127.0.0.1', body: miss });
assert.equal(passedLimiter(owner), true, 'the owner keeps its own bucket');
});
+10 -7
View File
@@ -3,13 +3,16 @@
*
* The credential panel has its own stricter gate (`admitKeySetupRequest` in
* keySetupCore.mjs): loopback socket + local Host + exact Origin + JSON
* Content-Type. The four endpoints here — `/api/realtime/token` (mints an
* OpenAI Realtime token = spends money), `/api/openai/hud-summary` (OpenAI
* spend), `/api/google/nearby-places` (Google spend), and
* `/api/realtime/debug-log` (appends to a local JSONL) — were previously
* ungated, so a hostile web page open in the same browser could reach them
* cross-site: a `fetch` carries a foreign `Origin`, and an `<img>`/navigation
* carries NO `Origin` at all but does carry `Sec-Fetch-Site: cross-site`.
* Content-Type. The endpoints behind this gate — `/api/realtime/token` (mints
* an OpenAI Realtime token = spends money), `/api/openai/hud-summary` (OpenAI
* spend), `/api/google/nearby-places` (Google spend),
* `/api/realtime/debug-log` (appends to a local JSONL) and the quota-bearing
* routes SECURITY.md lists with them — were previously ungated, so a hostile
* web page open in the same browser could reach them cross-site: a `fetch`
* carries a foreign `Origin`, and an `<img>`/navigation carries NO `Origin`
* at all but does carry `Sec-Fetch-Site: cross-site`. The keyless map routes
* (`/api/terrain/heights`, `/api/overpass`) reuse this core through
* `admitSameSiteKeyless` with the forwarding-header refusal switched off.
*
* This module is the pure core that refuses those shapes while keeping the
* documented `HOST=0.0.0.0` LAN opt-in and non-browser loopback tools (the two
+85 -13
View File
@@ -1,11 +1,27 @@
/** A reveal must beat the one-second idle rotation refresh, regardless of FPS. */
/**
* Wall-clock budget for EACH observed frame, regardless of FPS. Both the settle
* frame (measured from the forced moveEnd) and the reveal frame (measured from
* the settle frame that hid the contact) must land inside it, so neither can be
* repaired by the one-second idle rotation refresh before it is read. The
* refresh clock starts at the fleet tick (preRender) of the frame the budget is
* measured from, so the budget must leave room for that frame's own render
* time: half the refresh interval tolerates frames up to ~500 ms, and a ~300 ms
* software-rendered frame (headless SwiftShader) fits with margin.
*/
export const ARRIVAL_OBSERVATION_MS = 500;
/** A measured rotation error at or above this fails the arrival check. */
export const ARRIVAL_MAX_ERROR_DEG = 10;
/** Attempts allowed when every earlier one timed out without a wrong angle. */
export const ARRIVAL_MAX_ATTEMPTS = 2;
/**
* Observe the first completed moveEnd frame and the first subsequent reveal.
* The deadline starts when moveEnd is forced, before its rotation baseline, so
* waiting for either frame cannot let the one-second idle refresh mask a bug.
* `read` snapshots visibility and rotation error against the current position.
* The settle deadline starts when moveEnd is forced, before its rotation
* baseline; the reveal deadline restarts at the settle frame, which is where
* the contact is hidden. `read` snapshots visibility and rotation error
* against the current position.
*/
export function observeArrivalRotation({
scene,
@@ -18,6 +34,7 @@ export function observeArrivalRotation({
}) {
return new Promise((resolve, reject) => {
const startedAt = now();
let phaseStartedAt = startedAt;
let moveEnd = null;
let moveEndMs = null;
let revealFrames = 0;
@@ -34,24 +51,42 @@ export function observeArrivalRotation({
cleanup();
resolve(result);
};
const expired = () => ({
// A timeout keeps every angle it measured: the settle angle before a
// reveal timeout, and the angle on a late frame itself (a late frame is
// read first; the idle refresh can only repair an angle, so a wrong one
// seen late was wrong on time too). `arrivalAngleViolated` then makes the
// timeout final instead of retryable.
const expired = (late = null) => ({
error: `arrival ${moveEnd ? 'reveal' : 'moveEnd'} observation exceeded ${ARRIVAL_OBSERVATION_MS} ms`,
elapsedMs: now() - startedAt,
moveEndMs,
...(moveEnd
? { dMoveEnd: moveEnd.rotationError }
: late
? { dMoveEnd: late.rotationError }
: {}),
...(moveEnd && late?.shown ? { dReveal: late.rotationError } : {}),
flipFrame: -1,
});
const armDeadline = () => {
clearTimer(timer);
timer = setTimer(() => finish(expired()), ARRIVAL_OBSERVATION_MS);
};
remove = scene.postRender.addEventListener(() => {
if (done) return;
try {
const elapsedMs = now() - startedAt;
if (elapsedMs >= ARRIVAL_OBSERVATION_MS) {
finish(expired());
const at = now();
const elapsedMs = at - startedAt;
const sample = read();
if (at - phaseStartedAt >= ARRIVAL_OBSERVATION_MS) {
finish(expired(sample));
return;
}
const sample = read();
if (moveEnd === null) {
moveEnd = sample;
moveEndMs = elapsedMs;
phaseStartedAt = at;
armDeadline();
startReveal();
} else {
revealFrames++;
@@ -76,7 +111,7 @@ export function observeArrivalRotation({
reject(error);
}
});
timer = setTimer(() => finish(expired()), ARRIVAL_OBSERVATION_MS);
armDeadline();
try {
startMoveEnd();
scene.requestRender();
@@ -87,14 +122,51 @@ export function observeArrivalRotation({
});
}
/**
* Whether an observation measured a rotation error over the threshold on any
* frame it captured, including the settle frame of a reveal that then timed
* out. Such an observation is a demonstrated failure, never inconclusive.
*/
export function arrivalAngleViolated(observation) {
const limit = (ARRIVAL_MAX_ERROR_DEG * Math.PI) / 180;
return [observation?.dMoveEnd, observation?.dReveal].some(
(error) => Number.isFinite(error) && error >= limit,
);
}
/**
* Observe arrival rotation, retrying only an inconclusive timeout. A wall-clock
* timeout means the frame landed where the idle refresh could already have
* repaired it, so it earns another attempt; an observation that measured a
* wrong angle (or succeeded) is final, so a later clean attempt can never
* replace it. `prepare` runs before every attempt; the result carries
* `attempts`.
*/
export async function observeArrivalRotationWithRetry({
prepare = async () => {},
observe = observeArrivalRotation,
maxAttempts = ARRIVAL_MAX_ATTEMPTS,
...options
}) {
let observed;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
await prepare();
observed = { ...(await observe(options)), attempts: attempt };
if (!observed.error || arrivalAngleViolated(observed)) break;
}
return observed;
}
/** Test the captured frames without retrying a bad first observation. */
export function arrivalRotationPassed(observation) {
return (
!!observation &&
!observation.error &&
observation.shown === true &&
observation.elapsedMs < ARRIVAL_OBSERVATION_MS &&
(observation.dMoveEnd * 180) / Math.PI < 10 &&
(observation.dReveal * 180) / Math.PI < 10
observation.moveEndMs < ARRIVAL_OBSERVATION_MS &&
observation.revealMs < ARRIVAL_OBSERVATION_MS &&
Number.isFinite(observation.dMoveEnd) &&
Number.isFinite(observation.dReveal) &&
!arrivalAngleViolated(observation)
);
}
+231 -8
View File
@@ -1,19 +1,23 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import * as arrivalModule from './trackArrivalObservation.mjs';
import {
ARRIVAL_OBSERVATION_MS,
arrivalAngleViolated,
arrivalRotationPassed,
observeArrivalRotation,
observeArrivalRotationWithRetry,
} from './trackArrivalObservation.mjs';
function fixture() {
function fixture(start = observeArrivalRotation) {
let time = 0;
let state = { shown: true, rotationError: 0 };
const listeners = new Set();
const timers = new Map();
let renders = 0;
let moveEnds = 0;
const observation = observeArrivalRotation({
const observation = start({
scene: {
postRender: {
addEventListener(listener) {
@@ -110,7 +114,8 @@ test('no reveal fails at a deadline before the one-second idle catch-up', async
assert.equal(ARRIVAL_OBSERVATION_MS, 500);
f.frame(8, { shown: true, rotationError: 0 });
f.frame(480);
f.expire(500);
// The reveal budget is measured from the 8 ms settle frame.
f.expire(508);
const result = await f.observation;
assert.match(result.error, /reveal.*500 ms/);
assert.equal(arrivalRotationPassed(result), false);
@@ -123,20 +128,46 @@ test('no reveal fails at a deadline before the one-second idle catch-up', async
test('a late visible frame fails even if the timeout callback is delayed', async () => {
const f = fixture();
f.frame(8, { shown: true, rotationError: 0 });
f.frame(501, { shown: true, rotationError: 0 });
f.frame(509, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.match(result.error, /reveal.*500 ms/);
assert.equal(result.elapsedMs, 501);
assert.equal(result.elapsedMs, 509);
assert.equal(arrivalRotationPassed(result), false);
});
test('the deadline includes moveEnd setup rather than restarting when hidden', async () => {
test('a slow software renderer passes when each frame stays inside its own budget', async () => {
// Headless SwiftShader frames run ~300 ms: settle lands at 300 ms and the
// reveal at 600 ms from moveEnd. Both stay inside the 1 s idle refresh.
const f = fixture();
f.frame(400, { shown: true, rotationError: 0 });
f.frame(300, { shown: true, rotationError: 0 });
f.expire(500); // the settle frame re-armed the deadline to 800 ms
f.frame(600, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.equal(result.moveEndMs, 400);
assert.equal(result.moveEndMs, 300);
assert.equal(result.revealMs, 300);
assert.equal(result.elapsedMs, 600);
assert.equal(result.flipFrame, 1);
assert.equal(arrivalRotationPassed(result), true);
assert.equal(f.resources.timers, 0);
});
test('a settle frame later than its budget fails as a moveEnd timeout', async () => {
const f = fixture();
f.frame(500, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.match(result.error, /moveEnd.*500 ms/);
assert.equal(result.moveEndMs, null);
assert.equal(arrivalRotationPassed(result), false);
});
test('a reveal later than its own budget fails even after a fast settle frame', async () => {
const f = fixture();
f.frame(400, { shown: true, rotationError: 0 });
f.frame(901, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.match(result.error, /reveal.*500 ms/);
assert.equal(result.moveEndMs, 400);
assert.equal(result.elapsedMs, 901);
assert.equal(arrivalRotationPassed(result), false);
});
@@ -149,3 +180,195 @@ test('missing moveEnd frames fail without waiting for a render callback', async
assert.equal(f.resources.listeners, 0);
assert.equal(f.resources.timers, 0);
});
// Let a retry's next attempt register its postRender listener.
const settle = () => new Promise((resolve) => setImmediate(resolve));
function retryFixture(extra = {}) {
const counts = { prepares: 0 };
const f = fixture((options) =>
observeArrivalRotationWithRetry({
...options,
...extra,
async prepare() {
counts.prepares++;
},
}),
);
f.counts = counts;
return f;
}
test('a wrong settle angle followed by a reveal timeout is final, not retried into a pass', async () => {
const f = retryFixture();
await settle();
// 0.5 rad ≈ 29°: a real threshold violation on the settle frame.
f.frame(8, { shown: true, rotationError: 0.5 });
f.expire(508); // the reveal never comes inside its budget
await settle();
// An otherwise clean second attempt, which must never be started.
f.frame(516, { shown: true, rotationError: 0 });
f.frame(524, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.match(result.error, /reveal.*500 ms/);
assert.equal(result.dMoveEnd, 0.5, 'the measured settle angle is kept');
assert.equal(arrivalAngleViolated(result), true);
assert.equal(result.attempts, 1);
assert.equal(f.counts.prepares, 1);
assert.equal(f.resources.moveEnds, 1);
assert.equal(arrivalRotationPassed(result), false);
assert.equal(f.resources.listeners, 0);
assert.equal(f.resources.timers, 0);
});
test('an inconclusive reveal timeout after a correct settle earns one more attempt', async () => {
const f = retryFixture();
await settle();
f.frame(8, { shown: true, rotationError: 0 });
f.expire(508);
await settle();
f.frame(516, { shown: true, rotationError: 0 });
f.frame(596, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.equal(result.attempts, 2);
assert.equal(f.counts.prepares, 2);
assert.equal(f.resources.moveEnds, 2);
assert.equal(arrivalRotationPassed(result), true);
});
test('timeouts stop after the attempt limit and still fail', async () => {
const f = retryFixture();
await settle();
f.expire(500);
await settle();
f.expire(1000);
const result = await f.observation;
assert.match(result.error, /moveEnd.*500 ms/);
assert.equal(result.attempts, 2);
assert.equal(arrivalRotationPassed(result), false);
assert.equal(f.resources.listeners, 0);
assert.equal(f.resources.timers, 0);
});
test('a missing or non-finite angle never passes', () => {
const clean = {
shown: true,
moveEndMs: 8,
revealMs: 80,
dMoveEnd: 0,
dReveal: 0,
};
assert.equal(arrivalRotationPassed(clean), true);
for (const field of ['dMoveEnd', 'dReveal']) {
assert.equal(arrivalRotationPassed({ ...clean, [field]: NaN }), false);
assert.equal(
arrivalRotationPassed({ ...clean, [field]: undefined }),
false,
);
}
});
test('a wrong angle on a late reveal frame is final, not retried into a pass', async () => {
const f = retryFixture();
await settle();
f.frame(100, { shown: true, rotationError: 0 });
// Shown at 600 ms (500 ms after settle) still wearing the tampered nose.
f.frame(600, { shown: true, rotationError: 1.5 });
await settle();
f.frame(608, { shown: true, rotationError: 0 });
f.frame(688, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.match(result.error, /reveal.*500 ms/);
assert.equal(result.dReveal, 1.5);
assert.equal(result.attempts, 1);
assert.equal(f.resources.moveEnds, 1);
assert.equal(arrivalRotationPassed(result), false);
});
test('a wrong angle on a late settle frame is final, not retried into a pass', async () => {
const f = retryFixture();
await settle();
f.frame(926, { shown: true, rotationError: 1.5 });
await settle();
f.frame(934, { shown: true, rotationError: 0 });
f.frame(1014, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.match(result.error, /moveEnd.*500 ms/);
assert.equal(result.dMoveEnd, 1.5);
assert.equal(result.attempts, 1);
assert.equal(arrivalRotationPassed(result), false);
});
test('a late hidden reveal frame records no angle and stays retryable', async () => {
const f = retryFixture();
await settle();
f.frame(8, { shown: true, rotationError: 0 });
// Still hidden, still tampered: not a measured reveal angle.
f.frame(600, { shown: false });
const first = f.observation;
await settle();
f.frame(608, { shown: true, rotationError: 0 });
f.frame(688, { shown: true, rotationError: 0 });
const result = await first;
assert.equal(result.attempts, 2);
assert.equal(result.dReveal, 0);
assert.equal(arrivalRotationPassed(result), true);
});
test('a late frame with correct angles stays an inconclusive, retryable timeout', async () => {
const f = retryFixture();
await settle();
f.frame(926, { shown: true, rotationError: 0 });
await settle();
f.frame(934, { shown: true, rotationError: 0 });
f.frame(1014, { shown: true, rotationError: 0 });
const result = await f.observation;
assert.equal(result.attempts, 2);
assert.equal(arrivalRotationPassed(result), true);
});
test('a wrong first reveal on time is final: no retry', async () => {
const f = retryFixture();
await settle();
f.frame(8, { shown: true, rotationError: 0 });
f.frame(88, { shown: true, rotationError: 1.5 });
const result = await f.observation;
assert.equal(result.error, undefined);
assert.equal(result.attempts, 1);
assert.equal(f.resources.moveEnds, 1);
assert.equal(arrivalRotationPassed(result), false);
});
test('maxAttempts bounds inconclusive retries', async () => {
const f = retryFixture({ maxAttempts: 3 });
for (let attempt = 1; attempt <= 3; attempt++) {
await settle();
f.expire(attempt * 500);
}
const result = await f.observation;
assert.equal(result.attempts, 3);
assert.equal(f.counts.prepares, 3);
assert.equal(arrivalRotationPassed(result), false);
});
test('the 10° limit is inclusive', () => {
const limit = Math.PI / 18;
assert.equal(arrivalAngleViolated({ dMoveEnd: limit }), true);
assert.equal(arrivalAngleViolated({ dReveal: limit * 0.999 }), false);
});
test('every name the track harness imports from this module is exported', () => {
const harness = readFileSync(
new URL('../../scripts/track-regression.mjs', import.meta.url),
'utf8',
);
const names = [
...harness.matchAll(
/(?:import|const)\s*\{([^}]+)\}\s*(?:from\s*|=\s*await import\()[^;]*trackArrivalObservation\.mjs/g,
),
].flatMap((match) => match[1].split(',').map((name) => name.trim()));
assert.ok(names.includes('arrivalRotationPassed'));
assert.ok(names.includes('observeArrivalRotationWithRetry'));
for (const name of names)
assert.equal(typeof arrivalModule[name], 'function', name);
});