Once the cache was full, every write sorted all 50,000 entries to find the
oldest. A Map keeps insertion order, so keep the map in fetch-time order
and delete from the front: the resolver re-inserts each point it refetches
(an overwrite would leave it in its old place), and the disk load sorts
once, since a file written before this change, or a migrated raw-batch
cache, is not in age order.
Suggested by asturksever on #975.
A stale point that is refetched becomes the newest entry, and the cache
file's order is not its age order. The ceiling must still evict the
oldest fetch. Passes on the current sort-based eviction.
The card's fire pattern is written by hand against the producers' template,
so a format change upstream would silently show English. Both locale tests
now pass the label the layer actually produced through shownEntityLabel()
under ru, and set the locale inside the try that resets it.
Main now takes pending entries as changelog.d/<pr>-<slug>.md
files (#787), so the entry leaves CHANGELOG.md for changelog.d/686-terrain-heights-bounds.md.
The fire and camera labels in the context store, getStrongestFire() and
getSelectedInfo() were translated at the source, so the voice model got
"Пожар · FRP 12 МВт" or "Cámara ALPR" in track_entity,
get_entity_context and pointer_context while every other entity name in
those payloads is English. Keep them English, as before the interface
was translated; the voice card translates them where it shows them.
The pointer chip, its live-region line and the get_entity_context card
title now translate the English entity labels the FIRMS and ALPR layers
register ("Fire · FRP {frp} MW", "ALPR camera"). Other names, such as
callsigns and place names, show as they came.
main now rejects out-of-range coordinates at the proxy itself (19fcc00b), the
same check this branch added. main's version is kept, and the changelog entry
no longer claims that part; the cache cap, file ceiling and rate limit remain.
Both disk caches fed by OVERPASS_UPSTREAMS key only on the query — the Overpass
proxy on the sanitized request body, the installations proxy on the bbox — so an
entry written against one instance is a hit for the same query against another.
After an operator repoints OVERPASS_UPSTREAMS, the previous upstream's answers
keep being served for 7 days, 30 for boundary-class queries, and indefinitely
through the serve-stale path. An install that predates #742 has a cache written
from the public instances that were the default then, and configuring your own
instance does not displace it.
Cached answers now carry a tag identifying the configured upstream set, and a
disk read refuses an entry that does not match it, on the stale path too. With
nothing configured any entry is still accepted: there is no instance to
contradict, and serving last-good data at any age is what the not-configured
path exists for. The tag is a hash of the resolved endpoint list, never the
URLs; it is written to disk and compared on read, and is not sent to clients or
surfaced in errors.
One line is logged once per process when a foreign entry is ignored, naming the
variable and what will happen, as local-receivers.js already does for a
rejected feed.
Cost worth naming: on upgrade every existing entry is untagged, so the first
outage after upgrading serves nothing where it previously served stale data,
until each query is fetched once from the configured instance.
`createVectorTileSource` took `tileJsonUrl` and `allowedOrigin` as independent
options, and `createOpenFreeMapSource` hardcoded both to the default host. A
caller that repointed only `tileJsonUrl` — the obvious single change — kept the
default `allowedOrigin`, so the TileJSON's absolute upstream `tiles[0]` passed
the origin check and every tile was fetched from the default host instead. No
error, no warning, and nothing reached the configured mirror.
`allowedOrigin` now defaults to the origin of `tileJsonUrl`, so repointing one
option is enough, and a mirror that serves the default host's TileJSON verbatim
fails with the existing `Invalid vector tile origin` instead of silently
bypassing itself. The origin check is unchanged: a TileJSON still cannot point
the client at a host the caller did not configure.
All three existing callers pass `allowedOrigin` explicitly
(`openFreeMap.js`, `alpr/source.js`, `traffic/flowSource.js`), and an explicit
value still overrides the derived one, so behaviour is unchanged for every
current call site.
The OpenAI endpoints (/api/realtime/token, /api/openai/hud-summary) now
allow 30 requests per minute per client IP and the Google Places endpoints
120, the caps the Pinokio build already ships. GEV_RATELIMIT_OPENAI_PER_MIN
and GEV_RATELIMIT_GOOGLE_PER_MIN still override them and exactly 0 disables
them; an unreadable value falls back to the default rather than to
unlimited.
Salvaged from #683 onto current main.
Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
One add/add conflict, in scripts/format-scope.json: upstream's twelve new
entries and this branch's two. Both kept — 866 entries on main, 868 here,
none dropped.
Gates on the result: npm test 5397 passing, build, test:track 109/109,
changelog:check, format:check, check:boundaries, layer-token:check.
npm test is the only suite CI runs, and its discovery walked src/ alone, so
server/providers/transitHistory.test.mjs has never executed since it was
added. Its 8 tests pass, so this was silent coverage loss rather than a hidden
failure: the file reads like covered code in review and never ran anywhere.
Add server/ to the searched roots, and pin the property with a test that walks
the repository itself and compares the two sets, so a tree added later without
updating the roots fails loudly instead of going quiet.
Suite goes from 5090 to 5099 tests, 0 failures: the 8 tests in that file plus
the regression test added here.
Dogfoods the convention rather than describing it: the entry for #787 arrives
as `changelog.d/787-changelog-fragments.md` and `CHANGELOG.md` is untouched.
A fragment needs the pull request number, which only exists once the pull
request is open, so it lands in a second commit. `changelog.d/README.md` says
to push first and add the fragment after, for the same reason.
Checked against the one consumer that constrains the design: assembling this
fragment moves the `DISPLAY ▸ Draw` marker in `CHANGELOG.md` by exactly the
length of the spliced entry, and the 1200-character window that
`src/annotations/drawTool.test.mjs` slices after it is byte-identical.
`CONTRIBUTING.md` asks every runtime change to add an entry at the top of
`CHANGELOG.md`, so two pull requests in flight write into the same few lines and
git has nothing to merge on. Against `main` at `81eb443`, `git merge-tree` over
the 85 open pull requests GitHub reports as conflicting that aren't mine puts
`CHANGELOG.md` in the conflict set of 41 — more than any other path — and for 6
it is the only one. `docs/CURRENT-STATE.md`, the other file rule 3 names,
conflicts for 23, and merges cleanly for 23 of the 41: edits land in the section
for the area you touched, so they only collide when the areas do.
A pull request now adds `changelog.d/<pull request>-<slug>.md` instead. Distinct
filenames never conflict, and the pull request number makes them distinct for
free. `npm run changelog` splices the directory into `CHANGELOG.md` and empties
it; `npm run changelog:check` reports the batch and changes nothing.
Fragment text is copied through unchanged apart from trailing whitespace.
`src/annotations/drawTool.test.mjs` reads `CHANGELOG.md` as prose and slices a
fixed window after a heading, so an assembler that reflowed an entry could move
text across that window. Fragments are deliberately outside
`scripts/format-scope.json` for the same reason.
The assembler refuses a batch before writing anything when a filename is not
`<pull request>-<slug>.md` or two fragments claim one pull request number, and
a unit test runs that check over the real `changelog.d/`, so a fragment the
release could not assemble fails `npm test` in the pull request that adds it
rather than at release time.
Refs #781.
`mapAnalystRecord` has always published `verticalRateMps`, but the tracked
seam — `getTrackedInfo()` / `getTrackedSubject()`, built by `_describeFlight`
— did not, so anything needing climb/descent had to reach past the public
descriptor into the raw poll record.
Both layers now map it, which matters because they store it under different
keys: the civil records write `verticalRate` (records.js), the military ones
write `verticalRateMps`. Mapping only one would leave
`resolveTrackedAircraftInfo()` handing back two different shapes, and a
consumer reading the field off a civil contact would silently read every
military contact as level flight.
Reported, never derived. `stickyNumber` seeds the field with null, so an
aircraft that has not transmitted a rate publishes null rather than a
level-flight 0 — absent is not level, and a flight-phase classifier that
cannot tell them apart calls a climbing aircraft cruise.
Tests: both layers, both halves (a reported rate arrives with its sign; an
absent one stays null). Mutation-verified — removing either field or
defaulting it to 0 turns them red.
`docs/CURRENT-STATE.md` records the descriptor field, the two record keys it
reconciles, and the reported-never-derived rule.
The proxy already restricted WHERE it would go — https, an allowlisted host, a
station_information/station_status path, no redirects, a 12-second deadline and
a 5 MB body cap — but nothing restricted HOW OFTEN, so the dev server would
relay an unbounded stream of requests at a third-party bikeshare operator on
any caller's behalf, one outbound fetch and one timeout timer each.
Admit 120 requests/minute per client address, global backstop 3x, refused with
429 and `Retry-After` before the target is even parsed. The ceiling comes from
the layer's own demand: `src/layers/bikeshare/registry.js` catalogues 15
systems, station information is cached per city after its first fetch, and
station status refreshes once per city every STATUS_POLL_MS (60s) with
in-flight requests deduplicated — about 15 requests a minute with everything
in range, so 120 is eight times a maximal session.
The method, host, path, redirect, deadline and size guards are unchanged.
Refs #264
Every distinct 5-decimal point was a permanent in-memory entry, a
permanent line in an uncapped `.gev-cache/terrain-heights.json`, and a
live upstream call. Nothing limited the rate, and coordinates were
checked for being finite rather than for being on Earth, so an
out-of-range pair was still a valid cache key.
Four bounds, each following a pattern this repo already uses:
- WGS-84 range is enforced in `parseTerrainPoints`, at the edge, so no
later stage has to. Poles and the antimeridian still parse.
- The point cache holds at most 50,000 entries and evicts oldest-first,
the shape `trackBackfillProxies` uses. Sized from measurement: this
checkout's cache file held 6,189 entries in 1.0 MB of real use, so the
ceiling is eight times that working set. Terrain does not move, so an
evicted point costs one refetch and never a wrong answer.
- A cache file above 24 MB is not read back, mirroring
`rocketLaunchesProxy`. An unmeasurable file is not treated as an
oversized one: without stat, readFile decides, exactly as before.
- The route carries an always-on limiter of 90/min per client, 270
global — the `/api/overpass` band, and an order of magnitude above
measured demand (a full test:track run drove 10 requests, peak 8/min).
The ceilings are constructor options so they can be exercised as
behaviour rather than asserted as constants; production passes none.
They are destructured in the body because `proxyErrorResponses.test.mjs`
slices these proxies to the first column-zero closing brace.
The ownership table lists every owner except the backend, though realtimeBackend.js owns the two requests, their separate transports, the abort/timeout scope around both, and client-secret expiry.
The method signatures are already documented with application construction, so link there rather than repeat them, and state only the ownership facts written nowhere: the connection is the backend's only consumer in the application, the resolved credential is opaque to it and is handed to negotiate whole, and client-secret expiry is refused inside the backend.
Street Traffic draws on OpenStreetMap for geometry and TomTom for flow,
and the row said "Road data temporarily unavailable" whichever one
failed. The reporter of #661 went and verified a TomTom key while every
public Overpass mirror was refusing their network with HTTP 406 — a
status fetchRoads already had and dropped on the floor.
Classify the refusal where the layer talks to Overpass: 429 is
"Overpass rate-limited", 504 is "Overpass timed out", anything else is
"Overpass refused the road query (HTTP <status>)". The first two are the
words alpr's Overpass source already uses; the status is what says which
upstream declined, which a single-source layer never has to.
The error carries its status as a RoadRequestError, following
LiveSourceError in src/sources/live/contract.js, so a caller can branch
on the code without parsing English. The catch promotes a classified
refusal and keeps the general line for everything else — the branch is
on the error type, not on its text, because a dropped socket throws
whatever the platform felt like saying and "Failed to fetch" on a panel
row helps nobody.
Retries, backoff, caching and the mirror rotation are untouched.
adsbdb supplies the aircraft type, model name and registration behind every
enriched flight, and the airline and origin/destination pair behind the
tracked contact's FROM -> TO strip. It appeared in neither attribution
surface: DATA_SOURCES.md and src/data/dataCredits.js each held zero
occurrences of it, against the rule both files state for themselves and
CONTRIBUTING's fourth pull-request item. adsb.lol is a different operator and
its ODbL credit does not reach this data.
The reporter of #256 left the patch off deliberately, saying the terms were
what should not be guessed. They are published, so this takes them from the
source rather than inventing them: adsbdb credits PlaneBase for the aircraft
data and Guillaume Michel for the ICAO-to-N-number conversion, and states that
the flight route data "is the work of David Taylor, Edinburgh and Jim Mason,
Glasgow, and may not be copied, published, or incorporated into other
databases without the explicit permission of David J Taylor, Edinburgh". No
licence or rate limit is published for the service itself.
That restriction is the reason the note records what the proxy does with what
it fetches: responses are cached per key for 24 hours in .gev-cache/adsbdb.json,
which is gitignored, so the data stays on the operator's machine and is never
bundled with this repo or redistributed.
A test pins the credit. The attribution is a requirement rather than a
preference, so it should fail loudly if a later edit drops the names or the
restriction, instead of surviving on luck.
The keyless fallback landed on the LOCATION search box only. Two other places
forward-geocode a name and neither had it: the annotation resolver returned
null, so an annotation silently failed to anchor, and the Radio layer THREW,
which surfaced as a failed voice turn rather than as a station it could not
place. src/keylessGeocoder.test.mjs now COUNTS the call sites instead of
checking them file by file, so a second seam added to a file that already has
a fallback fails the suite too.
The annotation path carries the feature's own name out of Photon. Footprint
scoring matches OSM on that canonical name rather than on the user's words, so
incidental locality tokens cannot win: "the monument near the Texas Capitol,
Austin" scored on the raw utterance picks a downtown building that shares only
"Texas", "Capitol" and "Austin".
Photon's country is deliberately NOT re-exported. It comes back in the
feature's own language — "Việt Nam", "日本", "Россия" — and
normalizeRadioCountryInput fails CLOSED on a name it cannot map, so forwarding
it would filter the station list to nothing for exactly the places the lookup
had just resolved. The composed label still names the country. A keyless
lookup must not apply a filter the keyed one never applies.
A not-found is now remembered only when every source consulted returned a
verdict. Photon not replying is a network blip, not "no such place": memoizing
it kept the query answering not-found for the rest of the session on a network
that had since recovered — the same conflation the Overpass disk cache had to
be repaired for. geocodeMissIsDefinitive states the rule once and both entry
points use it, and the adapter's own memo follows it as well.
Reverse geocoding stays out of scope: the street-label reader consumes
Google's ranked address_components, and Photon's /reverse cannot reproduce
that shape.
Photon scores proximity against the name, so a viewport bias could outrank
an exact name match by distance alone. Searching "Hue" from Austin returned
Hutto, Texas; "Ha Long" returned Long Branch. Measured against the live
service over twelve Vietnamese place names, three resolved to the wrong
continent.
Google's `bounds` prefers a result inside the box without changing what
qualifies as a result, and that is the semantics this adapter promised to
reproduce. So a biased answer is now accepted only when a candidate leads
with the name searched for, and the query otherwise re-runs unbiased — one
extra request, on exactly the searches that were going to be wrong.
Matching leads rather than contains, which is what separates "Hue" from
"Nguyen Hue Road", and compares unaccented so "Hue" and "Huế" are one name.
On the unbiased pass the head of a "place, region" query is matched first;
the tail only says where to look.
Nearby searches are unaffected: "Sixth Street" and "Barton Springs" over
Austin still take the biased answer in a single request.
Place search was Google-only: searchAndFlyTo threw without a key, and with a
key whose Geocoding API is not enabled it returned nothing. Both cases leave
the LOCATION box and the fly_to_location voice tool unable to resolve a name,
even though every other layer in the app works keyless.
src/keylessGeocoder.js adapts Photon (komoot, over OpenStreetMap) into the
shape the Google path already produces — lat/lng, label, Google-style types,
{southwest,northeast} bounds — so geocodeNavigationMode, regionFramingPlan and
the region-swath cap keep running on unchanged inputs. The translation lives at
the boundary and nowhere else; Google remains primary and its branch is
unchanged when it answers.
Detection is the empty result, not an exception: an unenabled Geocoding API
answers HTTP 200 with REQUEST_DENIED. Places near-view recovery is now
attempted only when a key exists, since Places is a Google service too.
Three translations are load-bearing, and each was found by querying the live
service rather than by reasoning about OSM tagging:
- Google's `bounds` PREFERS results inside the box; Photon's `bbox` FILTERS to
it. Sent as a bbox, searching "Ho Guom, Ha Noi" while looking at Austin
returns features: []. Sent as Photon's lat/lon proximity bias it finds the
lake, and "Sixth Street" still resolves to Austin's over Austin and to
England's over London.
- Photon's `extent` is [west, north, east, south]. Read as [w,s,e,n] it yields
an inverted but finite box that frames the camera at nothing.
- A specific OSM key=value rule must beat the coarse `type`. Photon tags a lake
`water=lake` with `type: 'other'` — unmapped, that framed Hoan Kiem Lake as a
precise place and put the camera 26 m over the water — and tags a town square
`type: 'locality'`, which framed Times Square as if it were a city.
searchAndFlyTo no longer throws when no key is present. Both callers already
read a null return as not-found (ui.js renders its miss branch, gevActions
reports ok: false), so the removed throw has no consumer.
Verified end to end through the real search box, with a key Google refuses:
one denied Google geocode, one Photon request, camera from Austin
(30.2700, -97.7400) to 21.0288, 105.8524 — Hoan Kiem Lake to within metres —
and "Ha Noi" framed at province scale. npm test 2604/2604; npm run build
clean; npm run test:track 97/3, the same three failures unmodified main
produces on this machine.
The proxy lists four mirrors and rotated on 5xx alone, so a 4xx ended the
fan-out. Measured against the live services: overpass-api.de and its lz4 alias
answer 406 to this proxy's User-Agent, while overpass.kumi.systems and
overpass.private.coffee answer 200 to the byte-identical request. Every
Overpass-backed feature — road geometry, annotation outlines, place lookup —
was failing on an Apache error page with two healthy mirrors untried.
The refusal was then cached. The cache guard read `status < 500`, so the error
page was written to memory and to disk, and the serve-stale guard used the same
threshold and declined to replace it. Boundary-class queries hold a month-long
TTL: four of twenty-three cached entries on this machine held that 406, dated
days after the mirror had stopped refusing.
Both decisions now go through one predicate. A payload is data only when it is
a 2xx that is neither rate-limited nor a body-level runtime error, so what may
be cached and what may be replaced by a stale entry cannot drift apart again.
A refusal every mirror agrees on is still reported with the first mirror's
status and body, so a malformed query says what upstream said — after every
mirror has had its chance, not instead of it. fetchOverpassPayload takes
injectable endpoints and fetch so the rotation is covered without a live
mirror; restoring either half of the old behaviour fails the new tests.
daikaginza