mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 10:24:02 +00:00
fix(osm): key place-search clients by IPv6 /64
/api/osm/features keyed its rate limit and its one-search-per-client slot by the full peer address, so an IPv6 host could rotate addresses inside its /64 to take both global slots and slip the 12/min limit. It now uses clientNetworkKey, as /api/overpass does.
This commit is contained in:
1 parent
b531dd545b
commit
6b92602589
3 files changed
+58
-4
No files matched your search
+1
-1
@@ -74,7 +74,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur
|
||||
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
|
||||
- **Transit fetches registered feeds only.** `/api/transit/vehicles/<id>` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator.
|
||||
- **Local receiver feeds are operator-configured and local-only.** `/api/local-receivers/aircraft` reads only the `aircraft.json` URLs in the server's `LOCAL_RECEIVER_FEEDS`; the browser never supplies an address. Each host must pass the shared tap address rule (`src/data/tapAddress.js`: loopback, RFC1918, `localhost`, `*.local`; no link-local, IPv6 or other names), the scheme must be http(s), the path must end in `aircraft.json`, and credentials, queries and fragments are refused. Invalid entries are logged at startup and never fetched. Reads use a 2 s timeout, refuse redirects, cap bodies at 2 MB and share one read per second; responses name each feed by band only and never carry upstream error text.
|
||||
- **Voice place search runs only on the operator's Overpass.** `/api/osm/features` answers only when `OVERPASS_UPSTREAMS` is set and never falls back to a public instance. The client sends a curated preset id, a box, a mode and a limit; the server writes and quotes the Overpass query itself, so no client or model text reaches it. Boxes are capped (36 deg² for a count, 4 deg² for a listing), as are the feature limit, the Overpass timeout and memory, and the response bytes read. Each client runs one search at a time and two run at once in all, under a per-client rate limit (12/min) and a global one (30/min); a search gets 22 s across every configured endpoint in turn (Overpass itself is told 20 s), which ends before the voice answer stops waiting at 25 s. The route sits behind the cross-site gate below. Answers are cached in memory and on disk (`.gev-cache/osm-features/`, at most 400 entries and 256 MB, kept a day), keyed to the configured instance so a repointed `OVERPASS_UPSTREAMS` never serves the old one's answers; a malformed answer is never cached.
|
||||
- **Voice place search runs only on the operator's Overpass.** `/api/osm/features` answers only when `OVERPASS_UPSTREAMS` is set and never falls back to a public instance. The client sends a curated preset id, a box, a mode and a limit; the server writes and quotes the Overpass query itself, so no client or model text reaches it. Boxes are capped (36 deg² for a count, 4 deg² for a listing), as are the feature limit, the Overpass timeout and memory, and the response bytes read. Each client (IPv6 clients keyed by /64, as on `/api/overpass`) runs one search at a time and two run at once in all, under a per-client rate limit (12/min) and a global one (30/min); a search gets 22 s across every configured endpoint in turn (Overpass itself is told 20 s), which ends before the voice answer stops waiting at 25 s. The route sits behind the cross-site gate below. Answers are cached in memory and on disk (`.gev-cache/osm-features/`, at most 400 entries and 256 MB, kept a day), keyed to the configured instance so a repointed `OVERPASS_UPSTREAMS` never serves the old one's answers; a malformed answer is never cached.
|
||||
- **Validated response formats.** Proxied bodies are relayed only in the format the route serves. CCTV frames and the Street View fallback accept JPEG, PNG, GIF, WebP and AVIF only, sent under their canonical type; any other type falls through to the next fallback. Live CCTV media must match its feed kind (`video/mp4`, `video/webm`, or `multipart/x-mixed-replace` with a boundary) before it is relayed or the camera counts as healthy, and its redirects stay on the registered origin. GBFS bodies must parse as JSON and are re-serialized as `application/json; charset=utf-8`. Camera catalog URLs are pinned by parsed URL — scheme, exact host, no credentials, default port, and a normalized path inside the agency's image directory — and a redirect on a pinned host must stay inside the same directory. Every `/api/cctv` route uses the keyless cross-site checks described below, so cameras stay reachable behind a reverse proxy; a request carrying forwarding headers never reaches the key-bearing Street View fallback and gets the placeholder frame instead, as when no key is configured.
|
||||
- **Consistent API response headers.** Every `/api` response, including errors, partial content, Host refusals and unknown routes, carries `Content-Security-Policy: default-src 'none'; sandbox` and `X-Content-Type-Options: nosniff` (`apiResponseHeadersPlugin` in `build/vite.js`, installed ahead of the Host check). A route that sets its own policy adds it as a further policy; it never replaces this one.
|
||||
- **Response-size caps and timeouts** on proxied responses.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import path from 'node:path';
|
||||
import { makeRateLimiter, clientKey } from '../common/rate-limit.js';
|
||||
import { makeRateLimiter, clientNetworkKey } from '../common/rate-limit.js';
|
||||
import { admitSameSiteKeyless } from '../common/same-site.js';
|
||||
import { createBoundedCache } from '../common/boundedCache.js';
|
||||
import {
|
||||
@@ -220,7 +220,10 @@ export function installOsmFeaturesRoute(
|
||||
});
|
||||
if (!compiled.ok)
|
||||
return send(jsonError(400, compiled.error, { code: compiled.code }));
|
||||
if (!limiter(clientKey(req)))
|
||||
// IPv6 peers by /64, as /api/overpass does: rotating addresses inside
|
||||
// one prefix earns neither a second rate bucket nor a second slot.
|
||||
const client = clientNetworkKey(req);
|
||||
if (!limiter(client))
|
||||
return send({
|
||||
...jsonError(429, 'Rate limit exceeded'),
|
||||
retryAfter: '10',
|
||||
@@ -235,7 +238,7 @@ export function installOsmFeaturesRoute(
|
||||
compiled.ql,
|
||||
(parsed) => summarizeOsmFeatures(parsed, compiled),
|
||||
controller.signal,
|
||||
clientKey(req),
|
||||
client,
|
||||
);
|
||||
if (!controller.signal.aborted) send(answer);
|
||||
} catch {
|
||||
|
||||
@@ -278,6 +278,57 @@ test('a search without a limit lists up to 500 features', async () => {
|
||||
);
|
||||
});
|
||||
|
||||
test('one IPv6 /64 is one client for slots and the rate limit', async () => {
|
||||
const release = [];
|
||||
const { request } = mount(async () => {
|
||||
await new Promise((r) => release.push(r));
|
||||
return { status: 200, body: JSON.stringify({ elements: [] }) };
|
||||
});
|
||||
const first = request(
|
||||
'preset=hospital&bbox=0,0,1,1',
|
||||
undefined,
|
||||
{},
|
||||
'2001:db8:1:2::a',
|
||||
);
|
||||
await new Promise((r) => setTimeout(r, 5));
|
||||
// A rotated address that wrongly got its own slot would wait upstream
|
||||
// too: answer every held search before asserting, so a failure never hangs.
|
||||
const rotated = request(
|
||||
'preset=school&bbox=0,0,1,1',
|
||||
undefined,
|
||||
{},
|
||||
'2001:db8:1:2:ffff::b',
|
||||
);
|
||||
await new Promise((r) => setTimeout(r, 5));
|
||||
release.splice(0).forEach((r) => r());
|
||||
assert.equal(
|
||||
(await rotated).status,
|
||||
503,
|
||||
'a rotated address waits for its first search',
|
||||
);
|
||||
assert.equal((await first).status, 200);
|
||||
|
||||
// Twelve a minute per /64, wherever in it the requests come from.
|
||||
const { request: limited } = mount(async () => ({
|
||||
status: 200,
|
||||
body: JSON.stringify({ elements: [] }),
|
||||
}));
|
||||
const statuses = [];
|
||||
for (let i = 0; i < 13; i += 1)
|
||||
statuses.push(
|
||||
(
|
||||
await limited(
|
||||
`preset=hospital&bbox=0,0,1,${(i + 1) / 20}`,
|
||||
undefined,
|
||||
{},
|
||||
`2001:db8:5:6::${(i + 1).toString(16)}`,
|
||||
)
|
||||
).status,
|
||||
);
|
||||
assert.deepEqual(statuses.slice(0, 12), Array(12).fill(200));
|
||||
assert.equal(statuses[12], 429);
|
||||
});
|
||||
|
||||
test('the route answers, caches, and shares upstream slots fairly', async () => {
|
||||
const release = [];
|
||||
let upstream = 0;
|
||||
|
||||
Reference in new issue
Block a user