diff --git a/.env.example b/.env.example index 5bc02df9c..1949c48f4 100644 --- a/.env.example +++ b/.env.example @@ -129,18 +129,20 @@ AISSTREAM_API_KEY= # 0 disables the silence watch entirely. # AISSTREAM_SILENCE_TIMEOUT_MS=120000 -# Optional: TomTom live traffic flow (BYOK freemium — free tier is ~50,000 tile -# requests/day, get a key at https://developer.tomtom.com). SERVER-SIDE ONLY: +# Optional: TomTom live traffic flow (BYOK freemium — the free tier is 200,000 +# Traffic Flow & Incidents tile requests per MONTH, not per day; no credit card +# required. Get a key at https://developer.tomtom.com). SERVER-SIDE ONLY: # the browser fetches same-origin /api/tomtom/* and never sees the key. # Keyless fallback: the traffic layer runs its built-in simulation (white dots, # hardcoded per-road-class speeds) — no key required for the layer to work. # TOMTOM_API_KEY= -# Optional soft cap on upstream tile fetches per UTC day (default 40000) — a +# Optional soft cap on upstream tile fetches per UTC day (default 6000) — a # configurable application safety ceiling, not a guarantee of staying within # TomTom's free allowance (currently 200K tile requests/month: -# https://docs.tomtom.com/pricing/). Over the cap the proxy serves cached/stale -# tiles instead of hitting upstream. -# TOMTOM_DAILY_TILE_BUDGET=40000 +# https://docs.tomtom.com/pricing/). 6000/day keeps a 31-day month at 186K, +# inside that allowance. Over the cap the proxy serves cached/stale tiles +# instead of hitting upstream. +# TOMTOM_DAILY_TILE_BUDGET=6000 # Optional: CCTV layer tuning (advanced). Defaults are sensible — leave unset # unless you're customizing the camera source pack. diff --git a/.gitignore b/.gitignore index 491a82c15..1a64fff39 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,13 @@ node_modules dist +# Credential files. `.env` alone is not enough: Vite's loadEnv (see +# server/standalone/vite.config.js) and scripts/setup-doctor.mjs both read the +# full dotenv ladder — .env, .env.local, .env. and .env..local — +# so a key placed in any of them is live for the app. Only .env was ignored, +# leaving the other three committable. .env.example is the tracked template. .env +.env.* +!.env.example *.log screenshots/ qa-shots/ diff --git a/CHANGELOG.md b/CHANGELOG.md index f2c044acd..98ee67fdd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,14 @@ `DATA_CREDITS` entry surfaces the credit in the in-app Data attribution popover, and a test pins the wording the route data requires. +- Size the TomTom daily tile budget to the provider's real free allowance. + `TOMTOM_DAILY_TILE_BUDGET` defaulted to 40,000/day against an allowance + granted monthly (200,000 tile requests/month), exhausting a month in five + days and leaving the traffic layer dead for the rest of the period. The + default is now 6,000/day (186,000 over a 31-day month). Corrects the stale + "~50k/day" free-tier figure in the proxy, `.env.example` and + `DATA_SOURCES.md`. Still an application-side ceiling, not a billing cap. + - Distinguish PARTIAL vessel snapshots from STALE data in the layer panel, with accepted-record counts and unchanged retention, freshness and outage safeguards. diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 6e20435d8..d2472869d 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -78,7 +78,7 @@ How to read this: - **Camera frame content.** Public camera frames can contain people, vehicles and license plates, and the camera layer does nothing with them beyond putting them on the map. A successful upstream response is relayed as the provider served it: nothing in the camera pipeline blurs, redacts, enhances or restores it, and nothing there runs face, plate or object detection or otherwise recognises or classifies what is in the picture. (The ALPR layer described below maps where plate-reader cameras are, from locations contributors tagged in OpenStreetMap; it reads nothing from any camera frame.) When an upstream has no frame for a camera, what you see instead is a Google Street View still or the app's own placeholder card; the panel's source badge names which of them you are looking at, and a refresh that fails after a frame has arrived leaves the last good picture on screen rather than replacing it. The browser resamples each frame for display: onto a fixed 1920x1080 surface for the camera's projection plane in the scene, and onto a 192x108 canvas for the small ambient card. That scaling is the only change made to the picture, and the only pixels the camera layer reads back are a 64x36 downscale hashed to tell whether the picture has changed. Live streams are piped through; the still path reads the image into memory to pass it on — TxDOT wraps its JPEG in a JSON envelope, which is decoded there — and no frame is written to disk (the disk cache, `.gev-cache/`, holds feed JSON for other layers and never camera imagery). Outside that pipeline, the voice assistant is the one feature that sends imagery anywhere: when it answers a question about what is in view it may attach a downscaled screenshot of the rendered scene — including any camera plane on screen — to the request it sends to the voice model. Frames are fetched only from server-registered camera URLs — a client cannot hand the proxy a URL (see [SECURITY.md](SECURITY.md)). What the app does stays inside the line README draws: no named-person search, no face recognition, no tracking individuals. - **Transit (GTFS-Realtime).** The proxy requests only registered operator feeds, validates redirects, bounds responses, and uses conditional requests, short snapshot caching and failure backoff. All seven registered regions are live-enabled; each can be disabled independently. Vehicles use delayed playback between reports, with report age and waiting states shown separately. Session-local selected trails are available for every feed. MBTA additionally retains up to 15 minutes of recently observed positions in the running proxy, so available history can survive a browser reload; server restart clears it. History reads never fetch upstream, and other feeds have no proxy retention. Nothing is archived to disk or collected in the background. Operator text credits are included as a courtesy and do not imply endorsement. Heights are aligned to work with Google 3D tiles; subway markers are shown at street level because these feeds do not supply depth. - **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply. -- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 40,000, a configurable application safety ceiling, not a guarantee of staying within TomTom's monthly free allowance; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app). +- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 6,000 — sized so a 31-day month stays inside TomTom's 200K monthly free allowance; still a configurable application safety ceiling, not a guarantee; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app). - **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line. - **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable. - **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit. diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index b5727f353..b8928752b 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -3800,7 +3800,8 @@ easier to meet (detection is now on more often), but does not create it. - Traffic runs in `sim` mode (white dots, hardcoded speeds) unless `TOMTOM_API_KEY` is configured (env or Keychain `tomtom-api`/`api-key`), which enables `live` mode: TomTom flow vector tiles via the budget-governed `/api/tomtom` proxy - (`.gev-cache/tomtom/`, 120 s TTL, `TOMTOM_DAILY_TILE_BUDGET` default 40k/day), + (`.gev-cache/tomtom/`, 120 s TTL, `TOMTOM_DAILY_TILE_BUDGET` default 6k/day, + sized so a 31-day month stays inside TomTom's 200K/month free allowance), decoded client-side (`flowTiles.js`), matched onto Overpass roads (`flowMatch.js`), and rendered as green/amber/red dot color + speed/density scaling (`trafficFlowStyle.js`); closures spawn no dots; unmatched roads stay diff --git a/server/providers/traffic.js b/server/providers/traffic.js index fda118753..2a3c2e60a 100644 --- a/server/providers/traffic.js +++ b/server/providers/traffic.js @@ -23,9 +23,16 @@ import { * Budget governor (mirrors the OpenSky credit-governor philosophy — last-good * data beats a dead layer): a persistent counter (.gev-cache/tomtom/budget.json, * keyed by UTC date, reset on day change) counts upstream fetch attempts - * against a soft cap (TOMTOM_DAILY_TILE_BUDGET, default 40,000 of the free - * tier's ~50k/day). Over the cap the proxy serves stale tiles when available, - * else 429 {error:'budget'}. + * against a soft cap (TOMTOM_DAILY_TILE_BUDGET). Over the cap the proxy + * serves stale tiles when available, else 429 {error:'budget'}. + * + * The default is derived from TomTom's published free allowance, which is + * granted MONTHLY, not daily: 200,000 Traffic Flow & Incidents tile requests + * per month (https://docs.tomtom.com/pricing/). The previous 40,000/day + * default — and the "~50k/day" figure this comment used to cite — exhausted + * a month's entire allowance in five days, leaving the traffic layer dead + * for the rest of the billing period. 6,000/day keeps a full 31-day month + * inside the allowance (186,000) with headroom to spare. * * GET /api/tomtom/status → {hasKey, dailyCount, budget, date}. Keyless mode: * status reports hasKey:false and the tile endpoint 503s {error:'no_key'} @@ -37,7 +44,8 @@ export function tomtomProxy() { const TILE_TTL_MS = 120_000; const CACHE_DIR = path.join(process.cwd(), '.gev-cache', 'tomtom'); const BUDGET_PATH = path.join(CACHE_DIR, 'budget.json'); - const DEFAULT_DAILY_BUDGET = 40000; + // 200,000/month free tier / 31 days = 6,451. Rounded down for headroom. + const DEFAULT_DAILY_BUDGET = 6000; const MEM_MAX_ENTRIES = 256; const UPSTREAM_TIMEOUT_MS = 15000;