mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 02:14:01 +00:00
fix(osm): end the server's place search before the voice answer gives up
The voice answer waited 25 s for /api/osm/features, while the server let Overpass run for 25 s and itself waited up to 40 s per endpoint and 45 s in all. When the browser gave up first it closed the request, the route cancelled the upstream query, and nothing was cached, so the OSM_TIMEOUT retry started the same query from scratch and timed out again. The server now tells Overpass 20 s, waits 22 s per endpoint and 22 s in all, which ends inside the client's 25 s. A test pins that ordering.
This commit is contained in:
1 parent
51637a5e96
commit
4efba8ccca
5 files changed
+37
-10
No files matched your search
+1
-1
@@ -74,7 +74,7 @@ The data proxies under `server/providers/` are written so the browser cannot tur
|
||||
- **No verbatim client headers upstream.** The CCTV media route is the one route that relays a request header (`Range`, for video seeking). It is parsed and canonicalized before it is forwarded: one `bytes=` range only, with every accepted form — explicit span, open-ended and suffix — bounded to the same 64 MiB ceiling the relay applies to a declared response body. Multi-range, malformed, inverted and non-`bytes` values are dropped and the request proceeds without a `Range`, as RFC 7233 §3.1 prescribes. Bounding the request bounds what is asked for: a response that declares no length — live streamed media, or a chunked body from an upstream that ignores the `Range` — has no ceiling. An upstream request the browser has stopped waiting for is cancelled rather than left running, whether the viewer leaves before the headers arrive or during the body.
|
||||
- **Transit fetches registered feeds only.** `/api/transit/vehicles/<id>` resolves the id against `src/data/transitFeeds.js`; the browser never supplies a URL, and a feed that is registered but disabled does not resolve at all. Redirects are followed manually and each hop is validated against the feed's own https origin before it is requested, so an off-origin or downgraded hop is refused rather than contacted. Bodies are capped at 8 MB, the protobuf is decoded server-side under entity-count and string-length ceilings, a differential feed is refused, and snapshots live 15 s in memory with no disk cache. A per-feed admission limiter and a failure cooldown ladder bound what this process can ask of any operator.
|
||||
- **Local receiver feeds are operator-configured and local-only.** `/api/local-receivers/aircraft` reads only the `aircraft.json` URLs in the server's `LOCAL_RECEIVER_FEEDS`; the browser never supplies an address. Each host must pass the shared tap address rule (`src/data/tapAddress.js`: loopback, RFC1918, `localhost`, `*.local`; no link-local, IPv6 or other names), the scheme must be http(s), the path must end in `aircraft.json`, and credentials, queries and fragments are refused. Invalid entries are logged at startup and never fetched. Reads use a 2 s timeout, refuse redirects, cap bodies at 2 MB and share one read per second; responses name each feed by band only and never carry upstream error text.
|
||||
- **Voice place search runs only on the operator's Overpass.** `/api/osm/features` answers only when `OVERPASS_UPSTREAMS` is set and never falls back to a public instance. The client sends a curated preset id, a box, a mode and a limit; the server writes and quotes the Overpass query itself, so no client or model text reaches it. Boxes are capped (36 deg² for a count, 4 deg² for a listing), as are the feature limit, the Overpass timeout and memory, and the response bytes read. Each client runs one search at a time and two run at once in all, under a per-client rate limit (12/min) and a global one (30/min); a search gets 45 s across every configured endpoint in turn. The route sits behind the cross-site gate below. Answers are cached in memory and on disk (`.gev-cache/osm-features/`, at most 400 entries and 256 MB, kept a day), keyed to the configured instance so a repointed `OVERPASS_UPSTREAMS` never serves the old one's answers; a malformed answer is never cached.
|
||||
- **Voice place search runs only on the operator's Overpass.** `/api/osm/features` answers only when `OVERPASS_UPSTREAMS` is set and never falls back to a public instance. The client sends a curated preset id, a box, a mode and a limit; the server writes and quotes the Overpass query itself, so no client or model text reaches it. Boxes are capped (36 deg² for a count, 4 deg² for a listing), as are the feature limit, the Overpass timeout and memory, and the response bytes read. Each client runs one search at a time and two run at once in all, under a per-client rate limit (12/min) and a global one (30/min); a search gets 22 s across every configured endpoint in turn (Overpass itself is told 20 s), which ends before the voice answer stops waiting at 25 s. The route sits behind the cross-site gate below. Answers are cached in memory and on disk (`.gev-cache/osm-features/`, at most 400 entries and 256 MB, kept a day), keyed to the configured instance so a repointed `OVERPASS_UPSTREAMS` never serves the old one's answers; a malformed answer is never cached.
|
||||
- **Validated response formats.** Proxied bodies are relayed only in the format the route serves. CCTV frames and the Street View fallback accept JPEG, PNG, GIF, WebP and AVIF only, sent under their canonical type; any other type falls through to the next fallback. Live CCTV media must match its feed kind (`video/mp4`, `video/webm`, or `multipart/x-mixed-replace` with a boundary) before it is relayed or the camera counts as healthy, and its redirects stay on the registered origin. GBFS bodies must parse as JSON and are re-serialized as `application/json; charset=utf-8`. Camera catalog URLs are pinned by parsed URL — scheme, exact host, no credentials, default port, and a normalized path inside the agency's image directory — and a redirect on a pinned host must stay inside the same directory. Every `/api/cctv` route uses the keyless cross-site checks described below, so cameras stay reachable behind a reverse proxy; a request carrying forwarding headers never reaches the key-bearing Street View fallback and gets the placeholder frame instead, as when no key is configured.
|
||||
- **Consistent API response headers.** Every `/api` response, including errors, partial content, Host refusals and unknown routes, carries `Content-Security-Policy: default-src 'none'; sandbox` and `X-Content-Type-Options: nosniff` (`apiResponseHeadersPlugin` in `build/vite.js`, installed ahead of the Host check). A route that sets its own policy adds it as a further policy; it never replaces this one.
|
||||
- **Response-size caps and timeouts** on proxied responses.
|
||||
|
||||
@@ -831,7 +831,7 @@ The standalone app offers OpenAI and Gemini Live through an explicit provider ch
|
||||
|
||||
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 33 tools using separate description-only metadata; a session is offered 32 of them unless `OVERPASS_UPSTREAMS` is set, which adds `osm_query` and its one prompt sentence (`server/providers/openai/tools.js`). Description customization cannot replace argument types, enum values or required fields.
|
||||
|
||||
Voice areas live in a store owned by the action runner for the page (`src/data/areaStore.js`); the model gets an `areaId`, a rounded bounding box and a summary, never geometry. `resolve_area` (`src/voice/areaActions.js`, `src/data/areaResolver.js`) checks the store, bundled Natural Earth regions, then bundled countries and states by name — a name several prominent units share returns up to four candidates unless the view is inside exactly one — and then the annotation outline ladder (`resolveAnnotationTarget`: bundled counties, Census places and neighborhoods, the guarded `/api/geocode/outline`, and Overpass only when configured). Administrative geometry is stored only when its source identity agrees with the requested unit; stale, mismatched, synthesized or otherwise unverified administrative boundaries are refused rather than relabelled. Clarification IDs use the bounded `ne:<kind>:<slug>` grammar and are accepted only from the current resolver's candidate memory. A lookup past 8 s answers `AREA_TIMEOUT` and keeps filling the caches. `around=true` asks the same ladder for the grounds enclosing a landmark (Nominatim, then the OpenFreeMap area holding it) and otherwise stores a 250 m buffer marked approximate. `analyst_query` scopes add `area`, `drawn` (newest draw-tool outline) and `annotation` (newest or named outline); region scopes use the same resolver within 3 s. Drawn outlines are a simplified copy (≤1,200 vertices); counting reads the stored geometry (`src/data/areaGeometry.js`). `find_imagery` (`src/voice/imageryActions.js`) sets the Recent Imagery box from an area, the view or the turn's pointer snapshot and pins the best acquisition; any hand change to the panel or explicit layer OFF while it searches ends the request (`DISPLACED`) without pinning or success narration. `osm_query` asks `/api/osm/features` for one curated preset in a bounded box; the route answers `OVERPASS_NOT_CONFIGURED` without an operator Overpass. Results go to the session-only `osm-places` layer and become the analyst's last answer. An explicit OSM Places OFF while the provider is pending aborts the work and prevents re-enable, result, credit and follow-up-memory commits. The voice answer deadline also aborts the client provider request, so the caller does not leave a timed-out request without a cancellation owner.
|
||||
Voice areas live in a store owned by the action runner for the page (`src/data/areaStore.js`); the model gets an `areaId`, a rounded bounding box and a summary, never geometry. `resolve_area` (`src/voice/areaActions.js`, `src/data/areaResolver.js`) checks the store, bundled Natural Earth regions, then bundled countries and states by name — a name several prominent units share returns up to four candidates unless the view is inside exactly one — and then the annotation outline ladder (`resolveAnnotationTarget`: bundled counties, Census places and neighborhoods, the guarded `/api/geocode/outline`, and Overpass only when configured). Administrative geometry is stored only when its source identity agrees with the requested unit; stale, mismatched, synthesized or otherwise unverified administrative boundaries are refused rather than relabelled. Clarification IDs use the bounded `ne:<kind>:<slug>` grammar and are accepted only from the current resolver's candidate memory. A lookup past 8 s answers `AREA_TIMEOUT` and keeps filling the caches. `around=true` asks the same ladder for the grounds enclosing a landmark (Nominatim, then the OpenFreeMap area holding it) and otherwise stores a 250 m buffer marked approximate. `analyst_query` scopes add `area`, `drawn` (newest draw-tool outline) and `annotation` (newest or named outline); region scopes use the same resolver within 3 s. Drawn outlines are a simplified copy (≤1,200 vertices); counting reads the stored geometry (`src/data/areaGeometry.js`). `find_imagery` (`src/voice/imageryActions.js`) sets the Recent Imagery box from an area, the view or the turn's pointer snapshot and pins the best acquisition; any hand change to the panel or explicit layer OFF while it searches ends the request (`DISPLACED`) without pinning or success narration. `osm_query` asks `/api/osm/features` for one curated preset in a bounded box; the route answers `OVERPASS_NOT_CONFIGURED` without an operator Overpass. Results go to the session-only `osm-places` layer and become the analyst's last answer. An explicit OSM Places OFF while the provider is pending aborts the work and prevents re-enable, result, credit and follow-up-memory commits. The voice answer deadline also aborts the client provider request, so the caller does not leave a timed-out request without a cancellation owner. The server's whole search (22 s across every configured endpoint, with Overpass told 20 s) ends before that 25 s deadline, because a browser that stops waiting cancels the upstream query and leaves nothing cached for a retry.
|
||||
|
||||
`createGevActionRunner` routes `control_cctv` enable/disable through the shared
|
||||
layer visibility lifecycle. Results distinguish successful changes from
|
||||
|
||||
@@ -24,16 +24,22 @@ export const OSM_FEATURE_LIMITS = Object.freeze({
|
||||
maxCountDeg2: 36,
|
||||
/** Most features one fetch returns. */
|
||||
maxLimit: 1000,
|
||||
/** Overpass server-side timeout, seconds. */
|
||||
timeoutS: 25,
|
||||
/** Overpass server-side timeout, seconds: it gives up before we stop waiting. */
|
||||
timeoutS: 20,
|
||||
/** Overpass server-side memory cap, bytes. */
|
||||
maxsize: 64 * 1024 * 1024,
|
||||
/** Response bytes read from the upstream. */
|
||||
maxResponseBytes: 16 * 1024 * 1024,
|
||||
/** Per-upstream wait: a busy Overpass answers these in 10–30 s. */
|
||||
upstreamTimeoutMs: 40_000,
|
||||
/** The whole search, across every configured endpoint in turn. */
|
||||
upstreamBudgetMs: 45_000,
|
||||
/** Per-upstream wait: the Overpass timeout plus transfer time. */
|
||||
upstreamTimeoutMs: 22_000,
|
||||
/**
|
||||
* The whole search, across every configured endpoint in turn. It ends
|
||||
* before the voice answer's own deadline (`OSM_QUERY_LIMITS.budgetMs` in
|
||||
* `src/voice/osmActions.js`, 25 s): a browser that gives up first closes
|
||||
* the request, which cancels the upstream query, so nothing would be
|
||||
* cached for the retry.
|
||||
*/
|
||||
upstreamBudgetMs: 22_000,
|
||||
});
|
||||
|
||||
/** Tags a returned feature keeps, besides the preset's own. */
|
||||
|
||||
@@ -18,6 +18,23 @@ import {
|
||||
} from '../../server/providers/overpass/osmFeatures.js';
|
||||
import { installOsmFeaturesRoute } from '../../server/providers/overpass/osmFeaturesRoute.js';
|
||||
import { sanitizeOverpassBody } from '../../server/providers/overpass/query.js';
|
||||
import { OSM_QUERY_LIMITS } from '../voice/osmActions.js';
|
||||
|
||||
test('the server finishes its whole search before the voice answer gives up', () => {
|
||||
// When the browser stops waiting, the route aborts the upstream query, so
|
||||
// an OSM_TIMEOUT retry could never be served from the cache.
|
||||
assert.ok(
|
||||
OSM_FEATURE_LIMITS.timeoutS * 1000 < OSM_FEATURE_LIMITS.upstreamTimeoutMs,
|
||||
'Overpass gives up before the server stops waiting for it',
|
||||
);
|
||||
assert.ok(
|
||||
OSM_FEATURE_LIMITS.upstreamTimeoutMs <= OSM_FEATURE_LIMITS.upstreamBudgetMs,
|
||||
);
|
||||
assert.ok(
|
||||
OSM_FEATURE_LIMITS.upstreamBudgetMs + 2_000 <= OSM_QUERY_LIMITS.budgetMs,
|
||||
`server budget ${OSM_FEATURE_LIMITS.upstreamBudgetMs} ms must end before the client's ${OSM_QUERY_LIMITS.budgetMs} ms`,
|
||||
);
|
||||
});
|
||||
|
||||
test('a preset and a small box compile to a bounded query', () => {
|
||||
const compiled = compileOsmFeatureQuery({
|
||||
@@ -28,7 +45,7 @@ test('a preset and a small box compile to a bounded query', () => {
|
||||
assert.equal(compiled.ok, true);
|
||||
assert.equal(
|
||||
compiled.ql,
|
||||
'[out:json][timeout:25][maxsize:67108864];(nwr["amenity"="hospital"](27.6,85.2,27.8,85.4););out center tags 50;',
|
||||
'[out:json][timeout:20][maxsize:67108864];(nwr["amenity"="hospital"](27.6,85.2,27.8,85.4););out center tags 50;',
|
||||
);
|
||||
assert.doesNotMatch(compiled.ql, /area/, 'no boundary-area scan');
|
||||
const count = compileOsmFeatureQuery({
|
||||
|
||||
@@ -34,7 +34,11 @@ export const OSM_QUERY_LIMITS = Object.freeze({
|
||||
maxCountDeg2: 36,
|
||||
defaultLimit: 500,
|
||||
maxLimit: 1000,
|
||||
/** The answer waits this long before cancelling its provider request. */
|
||||
/**
|
||||
* The answer waits this long before cancelling its provider request. It
|
||||
* must outlast the server's whole search (`OSM_FEATURE_LIMITS.upstreamBudgetMs`,
|
||||
* 22 s), or the cancelled search caches nothing for the retry.
|
||||
*/
|
||||
budgetMs: 25_000,
|
||||
});
|
||||
|
||||
|
||||
Reference in new issue
Block a user