diff --git a/.env.example b/.env.example index 12f8a08257..75047a6705 100644 --- a/.env.example +++ b/.env.example @@ -205,6 +205,9 @@ AISSTREAM_API_KEY= # Traffic Flow & Incidents tile requests per MONTH, not per day; no credit card # required. Get a key at https://my.tomtom.com/keys). SERVER-SIDE ONLY: # the browser fetches same-origin /api/tomtom/* and never sees the key. +# The key needs a traffic product assigned at my.tomtom.com: either the +# TomTom Maps Traffic API or the Orbis Maps one works (the proxy tries the +# first and falls back to Orbis on HTTP 403). # Keyless fallback: the traffic layer runs its built-in simulation (white dots, # hardcoded per-road-class speeds) — no key required for the layer to work. # TOMTOM_API_KEY= diff --git a/DATA_SOURCES.md b/DATA_SOURCES.md index 0133c41125..0442e843d5 100644 --- a/DATA_SOURCES.md +++ b/DATA_SOURCES.md @@ -281,7 +281,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei - **Radio country coverage (optional).** Set `RADIO_PRIORITY_COUNTRIES=IN,PL` in the local `.env` and restart to include stations beyond the global click charts. The operator may choose up to eight unique ISO alpha-2 country codes; no key is needed. Each country adds one bounded query (at most 200 rows) and reserves up to 40 unique, usable stations inside the existing 750-station catalog. Country seeds precede specialist-tag seeds and global popularity fill; unused slots return to the shared pool. This is a coverage preference, not an exhaustive national directory or a guarantee of stations in every city. Countries with no eligible HTTPS MP3/AAC geolocated stations remain empty. An unset list preserves global selection. Country requests share the directory cache and three-worker pool; failed country queries are reported in coverage without blocking a healthy global refresh. Failed/degraded refreshes back off for one minute, with at most three mirrors per query. - **Radio Browser.** In every installation, including when `RADIO_PRIORITY_COUNTRIES` is unset, each request tries at most three mirror candidates (later fallback candidates are skipped), and failed or partial catalog refreshes back off for 60 seconds. A cold-start failure therefore continues returning 503 during that delay even if upstream recovers. `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; broad/specialist responses whose rows all fail the product's health policy do not count as successful queries. Optional priority-country outcomes are advisory: valid, in-limit responses with no eligible stations succeed as empty results, while failed or malformed country queries report unsuccessful coverage without degrading the global catalog. A usable partial cold catalog is explicitly `DEGRADED`, and non-array or empty broad/specialist payloads are rejected atomically. Individual structurally malformed rows are skipped rather than failing the whole query; the response carries their bounded `rejectedStationCount` (at most 10,000), which the client adds to its own rejected rows and reports as degraded coverage without discarding the accepted generation. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply. -- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 6,000 — sized so a 31-day month stays inside TomTom's 200K monthly free allowance; still a configurable application safety ceiling, not a guarantee; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app). +- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (from the TomTom Maps Traffic API, or the TomTom Orbis Maps Traffic API when the key is only assigned that product; 120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 6,000 — sized so a 31-day month stays inside TomTom's 200K monthly free allowance; still a configurable application safety ceiling, not a guarantee; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app). - **Routing (OSRM / FOSSGIS).** `/api/route` proxies the public OSRM instances FOSSGIS e.V. runs for the OpenStreetMap community (car, bike, foot profiles). They are keyless and intended for fair, interactive use — the proxy rate-limits clients (60/min, 200/min total), caps route span and response size, caches each route for 10 minutes, and identifies itself in the User-Agent. Heavy or automated use must run its own OSRM. Routes are computed from OpenStreetMap data (ODbL); the Directions layer and voice routes never redistribute them. A route the router cannot find is reported as such; the Directions layer draws nothing in that case, and voice routes fall back to a clearly labeled straight line. - **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable. - **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit. diff --git a/changelog.d/1809-tomtom-orbis-flow-fallback.md b/changelog.d/1809-tomtom-orbis-flow-fallback.md new file mode 100644 index 0000000000..fc453db440 --- /dev/null +++ b/changelog.d/1809-tomtom-orbis-flow-fallback.md @@ -0,0 +1,5 @@ +- Live traffic works with TomTom keys that are only assigned the Orbis Maps + traffic product: when TomTom refuses the key on the TomTom Maps flow tile + endpoint (HTTP 403), the proxy uses the Orbis one and keeps it. A key both + endpoints refuse now reads "TomTom key rejected" instead of "TomTom upstream + unreachable", and stops spending the daily tile budget on retries (#1809). diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 594f966b64..5ee5389f2e 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -4027,6 +4027,14 @@ The TomTom `/api/tomtom/*` routes run the shared same-site admission gate before status, cache or upstream work, refusing cross-site browser requests while preserving same-origin app requests and non-browser local callers. The browser layers and their rendering remain in their existing modules. +The proxy asks the TomTom Maps flow endpoint (`traffic/map/4/tile/flow/relative`) +first. On HTTP 403, which TomTom answers when a key is not assigned that +product, it asks the Orbis Maps one (`maps/orbis/traffic/flow/vector/tile`, +`apiVersion=2`) and keeps whichever answered until the key changes or the +process restarts; `flowDecode.js` reads both tile schemas into one segment +shape. When both refuse the key, tiles answer `403 {error:'key_rejected'}` +(shown as "TomTom key rejected"), `/api/tomtom/status` reports +`keyRejected: true`, and upstream is left alone for five minutes. Past the daily tile budget, or while TomTom fails, the proxy answers with its last copy of a tile (`x-tomtom-cache: STALE-BUDGET` / `STALE-ERROR`) however old it is, and every tile carries `x-tomtom-fetched-at`, the proxy fetch time diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 5f9943f711..d4354a267a 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -1214,6 +1214,7 @@ "src/sources/militaryTileGeometry.test.mjs", "src/layers/traffic/surface.test.mjs", "src/layers/traffic/roadModes.test.mjs", + "src/layers/traffic/flowDecode.test.mjs", "scripts/qa-traffic-motion.mjs", "src/layers/traffic/motion.test.mjs", "scripts/qa-journey-recorder.mjs", diff --git a/server/providers/traffic.js b/server/providers/traffic.js index 2069a08924..d38067de72 100644 --- a/server/providers/traffic.js +++ b/server/providers/traffic.js @@ -18,6 +18,18 @@ import { admitSameSite } from './common/same-site.js'; * "Traffic flow"). The key comes from TOMTOM_API_KEY server-side only — the * browser fetches same-origin `/api/tomtom/flow/{z}/{x}/{y}.pbf`. * + * Orbis fallback: TomTom keys are assigned products, and a key without the + * TomTom Maps traffic product gets 403 on the endpoint above while the same + * key works on the TomTom Orbis Maps one, + * https://api.tomtom.com/maps/orbis/traffic/flow/vector/tile/{z}/{x}/{y}?apiVersion=2 + * (same "Traffic flow" layer; `relative_speed` and `road_category` replace + * `traffic_level` and `road_type`, which the client decoder reads either + * way). On 403 the proxy asks the other endpoint and keeps whichever + * answered for the rest of the process. When both refuse the key, tiles + * answer 403 {error:'key_rejected'} and upstream is left alone for + * KEY_REJECTED_TTL_MS, so a rejected key does not spend the daily budget one + * tile at a time. Changing the key clears both. + * * Cache: memory + disk (.gev-cache/tomtom/), TTL 120 s (traffic is fresh * data), single-flight per tile, serve-stale-on-failure — the celestrakProxy * pattern. Cache hits never count against the budget. @@ -43,7 +55,7 @@ import { admitSameSite } from './common/same-site.js'; * at distinct tiles, spend the TomTom quota and leave the traffic layer on * 429 budget for the rest of the UTC day. * - * GET /api/tomtom/status → {hasKey, dailyCount, budget, date}. Keyless mode: + * GET /api/tomtom/status → {hasKey, keyRejected, dailyCount, budget, date}. Keyless mode: * status reports hasKey:false and the tile endpoint 503s {error:'no_key'} * without touching upstream — the traffic layer then stays in simulation mode. * @@ -59,6 +71,37 @@ export function tomtomProxy() { const UPSTREAM_TIMEOUT_MS = 15000; /** Flow tiles are well under 1 MB; this bounds a runaway body, not a budget. */ const TOMTOM_TILE_MAX_RESPONSE_BYTES = 8 * 1024 * 1024; + /** How long a key both endpoints refused is left alone before a retry. */ + const KEY_REJECTED_TTL_MS = 5 * 60_000; + + /** Upstream flow-tile endpoints, in the order a new key is tried. */ + const ENDPOINTS = { + legacy: (z, x, y, key) => ({ + url: + 'https://api.tomtom.com/traffic/map/4/tile/flow/relative/' + + `${z}/${x}/${y}.pbf?key=${encodeURIComponent(key)}`, + }), + orbis: (z, x, y, key) => ({ + url: + 'https://api.tomtom.com/maps/orbis/traffic/flow/vector/tile/' + + `${z}/${x}/${y}?apiVersion=2`, + headers: { 'TomTom-Api-Key': key }, + }), + }; + /** + * What is known about the configured key: the endpoint that last served + * it, and until when both endpoints are known to refuse it. Reset when the + * key changes (Provider Settings rewrites process.env at runtime). + * @type {{key:string, endpoint:'legacy'|'orbis', rejectedUntil:number}} + */ + let keyState = { key: '', endpoint: 'legacy', rejectedUntil: 0 }; + function currentKeyState() { + const key = process.env.TOMTOM_API_KEY || ''; + if (keyState.key !== key) + keyState = { key, endpoint: 'legacy', rejectedUntil: 0 }; + return keyState; + } + const isKeyRejected = () => Date.now() < currentKeyState().rejectedUntil; /** @type {Map} tile key `z/x/y` -> cached tile (kept past TTL for serve-stale). */ const mem = new Map(); @@ -149,14 +192,17 @@ export function tomtomProxy() { mem.set(key, entry); } - async function fetchUpstream(z, x, y) { - const url = - 'https://api.tomtom.com/traffic/map/4/tile/flow/relative/' + - `${z}/${x}/${y}.pbf?key=${encodeURIComponent(process.env.TOMTOM_API_KEY)}`; + async function fetchEndpoint(name, z, x, y, key) { + const { url, headers } = ENDPOINTS[name](z, x, y, key); recordUpstreamFetch(); // attempts count — upstream bills the request either way const signal = AbortSignal.timeout(UPSTREAM_TIMEOUT_MS); - const res = await fetch(url, { signal }); - if (!res.ok) throw new Error(`HTTP ${res.status}`); + const res = await fetch(url, { signal, ...(headers ? { headers } : {}) }); + if (!res.ok) { + await res.body?.cancel().catch(() => {}); + throw Object.assign(new Error(`HTTP ${res.status}`), { + status: res.status, + }); + } const bytes = await readResponseBytesCapped( res, TOMTOM_TILE_MAX_RESPONSE_BYTES, @@ -167,6 +213,41 @@ export function tomtomProxy() { return buf; } + /** + * Fetch one tile from the endpoint this key last worked on, asking the + * other one only when TomTom refuses the key (403). Both refusing marks + * the key rejected; any other failure is an ordinary upstream error. + */ + async function fetchUpstream(z, x, y) { + const state = currentKeyState(); + const other = state.endpoint === 'legacy' ? 'orbis' : 'legacy'; + try { + return await fetchEndpoint(state.endpoint, z, x, y, state.key); + } catch (err) { + if (err?.status !== 403) throw err; + } + try { + const buf = await fetchEndpoint(other, z, x, y, state.key); + if (keyState === state) { + state.endpoint = other; + console.warn( + `[tomtom-proxy] key refused on the ${other === 'orbis' ? 'TomTom Maps' : 'Orbis'} flow endpoint (HTTP 403); using the ${other === 'orbis' ? 'Orbis' : 'TomTom Maps'} one`, + ); + } + return buf; + } catch (err) { + if (err?.status !== 403) throw err; + if (keyState === state) + state.rejectedUntil = Date.now() + KEY_REJECTED_TTL_MS; + throw Object.assign( + new Error( + 'HTTP 403 on both flow endpoints: check the products assigned to the key and its domain whitelist at https://my.tomtom.com/keys', + ), + { status: 403 }, + ); + } + } + const installMiddleware = (server) => { server.middlewares.use('/api/tomtom', async (req, res) => { // Gate first: a cross-site /fetch must not reach the budget counter. @@ -204,6 +285,7 @@ export function tomtomProxy() { const b = currentBudget(); sendJson(200, { hasKey, + keyRejected: hasKey && isKeyRejected(), dailyCount: b.count, budget: dailyBudgetLimit(), date: b.date, @@ -242,6 +324,14 @@ export function tomtomProxy() { return; } + // TomTom refused the key on both endpoints: asking again per tile + // would only spend the budget. Last-good data still beats nothing. + if (isKeyRejected()) { + if (entry) sendTile(entry, 'STALE-ERROR'); + else sendJson(403, { error: 'key_rejected' }); + return; + } + // Budget governor: over the soft cap, last-good data beats a dead layer. if (isTomTomOverBudget(currentBudget(), dailyBudgetLimit())) { if (entry) { @@ -277,6 +367,8 @@ export function tomtomProxy() { sendTile(fresh, 'MISS'); } else if (entry) { sendTile(entry, 'STALE-ERROR'); // upstream down — stale beats empty + } else if (isKeyRejected()) { + sendJson(403, { error: 'key_rejected' }); } else { sendJson(502, { error: 'upstream' }); } diff --git a/src/data/traffic.test.mjs b/src/data/traffic.test.mjs index 0e8ada30a6..ca22850687 100644 --- a/src/data/traffic.test.mjs +++ b/src/data/traffic.test.mjs @@ -46,6 +46,9 @@ test('hosted account limits and service failures name their own reason', () => { assert.equal(failure(429, 'budget'), 'TomTom daily budget reached'); assert.equal(failure(503, 'no_key'), 'TomTom key unavailable'); assert.equal(failure(503, null), 'TomTom key unavailable'); + // TomTom refused the key itself: not the generic HTTP 403 line. + assert.equal(failure(403, 'key_rejected'), 'TomTom key rejected'); + assert.equal(failure(403, null), 'TomTom flow error (HTTP 403)'); assert.equal( failure(503, 'USAGE_UNAVAILABLE'), 'Traffic service unavailable', diff --git a/src/i18n/locales/ar.json b/src/i18n/locales/ar.json index 247bbc8a05..22c2ace2e7 100644 --- a/src/i18n/locales/ar.json +++ b/src/i18n/locales/ar.json @@ -1983,6 +1983,7 @@ "Moving dots colored by live flow": "نقاط متحركة ملوّنة حسب التدفق المباشر", "Moving dots": "نقاط متحركة", "TomTom key unavailable": "مفتاح TomTom غير متاح", + "TomTom key rejected": "تم رفض مفتاح TomTom", "TomTom daily budget reached": "بلغت الحصة اليومية لـ TomTom", "Monthly traffic allowance used": "تم استهلاك حصة حركة المرور الشهرية", "Traffic rate limit reached": "تم بلوغ حد طلبات حركة المرور", diff --git a/src/i18n/locales/de.json b/src/i18n/locales/de.json index c3a47bfff1..c87555440a 100644 --- a/src/i18n/locales/de.json +++ b/src/i18n/locales/de.json @@ -1860,6 +1860,7 @@ "Moving dots colored by live flow": "Bewegte Punkte nach Live-Verkehrsfluss gefärbt", "Moving dots": "Bewegte Punkte", "TomTom key unavailable": "TomTom-Schlüssel nicht verfügbar", + "TomTom key rejected": "TomTom-Schlüssel abgelehnt", "TomTom daily budget reached": "TomTom-Tagesbudget erreicht", "Monthly traffic allowance used": "Monatliches Verkehrskontingent aufgebraucht", "Traffic rate limit reached": "Verkehrs-Anfragelimit erreicht", diff --git a/src/i18n/locales/es.json b/src/i18n/locales/es.json index 71344d6280..cc83b125e5 100644 --- a/src/i18n/locales/es.json +++ b/src/i18n/locales/es.json @@ -2463,6 +2463,7 @@ "Military area": "Zona militar", "SIMULATED — {reason}": "SIMULADO — {reason}", "TomTom key unavailable": "Clave de TomTom no disponible", + "TomTom key rejected": "Clave de TomTom rechazada", "TomTom daily budget reached": "Se alcanzó el límite diario de TomTom", "Monthly traffic allowance used": "Cupo mensual de tráfico agotado", "Traffic rate limit reached": "Límite de peticiones de tráfico alcanzado", diff --git a/src/i18n/locales/fr.json b/src/i18n/locales/fr.json index 6167dcdf72..e86ab59a92 100644 --- a/src/i18n/locales/fr.json +++ b/src/i18n/locales/fr.json @@ -1247,6 +1247,7 @@ "Moving dots colored by live flow": "Points mobiles colorés selon le trafic en direct", "Moving dots": "Points mobiles", "TomTom key unavailable": "Clé TomTom indisponible", + "TomTom key rejected": "Clé TomTom refusée", "TomTom daily budget reached": "Quota quotidien TomTom atteint", "Monthly traffic allowance used": "Quota mensuel de trafic épuisé", "Traffic rate limit reached": "Limite de requêtes de trafic atteinte", diff --git a/src/i18n/locales/he.json b/src/i18n/locales/he.json index 6eddcd9f5c..5b6fec20b5 100644 --- a/src/i18n/locales/he.json +++ b/src/i18n/locales/he.json @@ -1092,6 +1092,7 @@ "TomTom roads first; OpenStreetMap fills the rest, simulated (needs a TomTom key)": "כבישי TomTom קודם; OpenStreetMap משלים את היתר, מדומים (דורש מפתח TomTom)", "Hybrid": "היברידי", "TomTom key unavailable": "מפתח TomTom אינו זמין", + "TomTom key rejected": "מפתח TomTom נדחה", "TomTom daily budget reached": "תקציב TomTom היומי אזל", "TomTom upstream unreachable": "שירות TomTom אינו נגיש", "TomTom flow unavailable": "זרימת TomTom אינה זמינה", diff --git a/src/i18n/locales/it.json b/src/i18n/locales/it.json index 9c263d0b01..54b51ae2d4 100644 --- a/src/i18n/locales/it.json +++ b/src/i18n/locales/it.json @@ -1247,6 +1247,7 @@ "Moving dots colored by live flow": "Punti in movimento colorati in base al flusso live", "Moving dots": "Punti in movimento", "TomTom key unavailable": "Chiave TomTom non disponibile", + "TomTom key rejected": "Chiave TomTom rifiutata", "TomTom daily budget reached": "Budget giornaliero TomTom esaurito", "Monthly traffic allowance used": "Quota mensile del traffico esaurita", "Traffic rate limit reached": "Limite di richieste del traffico raggiunto", diff --git a/src/i18n/locales/ja.json b/src/i18n/locales/ja.json index 90ac90519c..fb5b1f038e 100644 --- a/src/i18n/locales/ja.json +++ b/src/i18n/locales/ja.json @@ -1201,6 +1201,7 @@ "Moving dots colored by live flow": "ライブの交通流で色分けした移動ドット", "Moving dots": "移動ドット", "TomTom key unavailable": "TomTomキーを利用できません", + "TomTom key rejected": "TomTomキーが拒否されました", "TomTom daily budget reached": "TomTomの1日の上限に達しました", "Monthly traffic allowance used": "今月の交通情報の上限に達しました", "Traffic rate limit reached": "交通情報のリクエスト上限に達しました", diff --git a/src/i18n/locales/pt.json b/src/i18n/locales/pt.json index 20ba01856d..a335f2a8fe 100644 --- a/src/i18n/locales/pt.json +++ b/src/i18n/locales/pt.json @@ -1245,6 +1245,7 @@ "Moving dots colored by live flow": "Pontos em movimento coloridos pelo fluxo ao vivo", "Moving dots": "Pontos em movimento", "TomTom key unavailable": "Chave do TomTom indisponível", + "TomTom key rejected": "Chave do TomTom recusada", "TomTom daily budget reached": "Limite diário do TomTom atingido", "Monthly traffic allowance used": "Cota mensal de trânsito esgotada", "Traffic rate limit reached": "Trânsito: limite de requisições atingido", diff --git a/src/i18n/locales/ru.json b/src/i18n/locales/ru.json index 6ed7638d5d..b3caf68692 100644 --- a/src/i18n/locales/ru.json +++ b/src/i18n/locales/ru.json @@ -2512,6 +2512,7 @@ "Military area": "Военная зона", "SIMULATED — {reason}": "СИМУЛЯЦИЯ — {reason}", "TomTom key unavailable": "Ключ TomTom недоступен", + "TomTom key rejected": "Ключ TomTom отклонён", "TomTom daily budget reached": "Исчерпан дневной лимит TomTom", "Monthly traffic allowance used": "Месячный лимит данных о трафике исчерпан", "Traffic rate limit reached": "Достигнут лимит запросов данных о трафике", diff --git a/src/i18n/locales/sv.json b/src/i18n/locales/sv.json index 565c953286..46cc42e600 100644 --- a/src/i18n/locales/sv.json +++ b/src/i18n/locales/sv.json @@ -1776,6 +1776,7 @@ "Moving dots colored by live flow": "Rörliga punkter färgade efter trafikflödet live", "Moving dots": "Rörliga punkter", "TomTom key unavailable": "TomTom-nyckel otillgänglig", + "TomTom key rejected": "TomTom-nyckeln avvisades", "TomTom daily budget reached": "TomTom-dagsbudgeten är förbrukad", "Monthly traffic allowance used": "Månadens trafikkvot är förbrukad", "Traffic rate limit reached": "Trafiktjänsten har nått gränsen för förfrågningar", diff --git a/src/i18n/locales/uk.json b/src/i18n/locales/uk.json index f8eb54fd20..320a21c5fa 100644 --- a/src/i18n/locales/uk.json +++ b/src/i18n/locales/uk.json @@ -1269,6 +1269,7 @@ "Moving dots colored by live flow": "Рухомі точки, забарвлені за потоком онлайн", "Moving dots": "Рухомі точки", "TomTom key unavailable": "Ключ TomTom недоступний", + "TomTom key rejected": "Ключ TomTom відхилено", "TomTom daily budget reached": "Вичерпано денний ліміт TomTom", "Monthly traffic allowance used": "Місячний ліміт даних про дорожній рух вичерпано", "Traffic rate limit reached": "Перевищено ліміт запитів до сервісу руху", diff --git a/src/i18n/locales/zh-CN.json b/src/i18n/locales/zh-CN.json index 95ab6f9586..416859da7d 100644 --- a/src/i18n/locales/zh-CN.json +++ b/src/i18n/locales/zh-CN.json @@ -1178,6 +1178,7 @@ "Moving dots colored by live flow": "按实时路况着色的移动圆点", "Moving dots": "移动圆点", "TomTom key unavailable": "TomTom 密钥不可用", + "TomTom key rejected": "TomTom 密钥被拒绝", "TomTom daily budget reached": "已达 TomTom 每日配额", "Monthly traffic allowance used": "本月交通数据配额已用完", "Traffic rate limit reached": "已达到交通数据请求频率上限", diff --git a/src/i18n/locales/zh-Hant.json b/src/i18n/locales/zh-Hant.json index cee8632919..587091ca89 100644 --- a/src/i18n/locales/zh-Hant.json +++ b/src/i18n/locales/zh-Hant.json @@ -1178,6 +1178,7 @@ "Moving dots colored by live flow": "按實時路況著色的移動圓點", "Moving dots": "移動圓點", "TomTom key unavailable": "TomTom 密鑰不可用", + "TomTom key rejected": "TomTom 密鑰被拒絕", "TomTom daily budget reached": "已達 TomTom 每日配額", "Monthly traffic allowance used": "本月交通數據配額已用完", "Traffic rate limit reached": "已達到交通數據請求頻率上限", diff --git a/src/layers/traffic/flow.js b/src/layers/traffic/flow.js index c8b55a28e2..27af08764d 100644 --- a/src/layers/traffic/flow.js +++ b/src/layers/traffic/flow.js @@ -50,6 +50,8 @@ export function createFlow({ state: layerState, services, parts, source }) { if (code === 'RATE_LIMITED') return msg('Traffic rate limit reached'); if (status === 503 && code && code !== 'no_key') return msg('Traffic service unavailable'); + // TomTom refused the key on both of its flow endpoints. + if (code === 'key_rejected') return msg('TomTom key rejected'); if (status === 503) return msg('TomTom key unavailable'); if (status === 429) return msg('TomTom daily budget reached'); if (status === 502 || status === 504) diff --git a/src/layers/traffic/flowDecode.js b/src/layers/traffic/flowDecode.js index 9a0c248dda..f047afa463 100644 --- a/src/layers/traffic/flowDecode.js +++ b/src/layers/traffic/flowDecode.js @@ -15,6 +15,15 @@ import { VectorTile } from '@mapbox/vector-tile'; * ratio (1 = free flow). Features with a missing/non-finite `traffic_level` * are skipped unless `road_closure` is true (closures decode with level 0). * + * The proxy serves tiles from either TomTom flow endpoint, and they name + * things differently. TomTom Maps (`traffic/map/4`, style `relative`): + * `traffic_level`, `road_type` ("Motorway", "Major road", …) and + * `traffic_road_coverage` (`full` when one line carries both directions). + * TomTom Orbis Maps (`maps/orbis/traffic`, apiVersion 2): `relative_speed` + * (the same ratio), `road_category` (OpenStreetMap-style, with `_link` + * ramps) and one line per travel direction. Orbis features are decoded into + * the TomTom Maps vocabulary, so nothing downstream tells them apart. + * * Deps: `pbf@5` (PbfReader) + `@mapbox/vector-tile@3` — both tiny and * tree-shakeable; decoding happens client-side so the proxy stays a dumb * binary cache. @@ -23,6 +32,35 @@ import { VectorTile } from '@mapbox/vector-tile'; */ const FLOW_LAYER_NAME = 'Traffic flow'; + +/** + * Orbis `road_category` → the TomTom Maps `road_type` and `road_category` + * the traffic layer classifies roads by. A ramp takes its road's class; + * service roads and tracks stay unmapped, as TomTom Maps tiles left + * parking and private roads out of the drivable set. + */ +const ORBIS_ROAD_CATEGORIES = Object.freeze({ + motorway: ['Motorway', 'motorway'], + trunk: ['International road', 'trunk'], + primary: ['Major road', 'primary'], + secondary: ['Secondary road', 'secondary'], + tertiary: ['Connecting road', 'tertiary'], + street: ['Local road', 'street'], +}); + +/** Road class of a feature, in TomTom Maps terms, from either schema. */ +function roadClassOf(props) { + if (typeof props.road_type === 'string') + return [props.road_type, props.road_category ?? null]; + const category = + typeof props.road_category === 'string' + ? props.road_category.replace(/_link$/, '') + : null; + if (category && Object.hasOwn(ORBIS_ROAD_CATEGORIES, category)) + return ORBIS_ROAD_CATEGORIES[category]; + // Kept as sent, so the drivable filter drops it by category. + return ['', props.road_category ?? null]; +} /** Decode flow geometry; strict mode distinguishes damaged tiles from valid empty coverage. */ export function decodeFlowTile(data, z, x, y, { strict = false } = {}) { let layer; @@ -49,13 +87,15 @@ export function decodeFlowTile(data, z, x, y, { strict = false } = {}) { const props = feature.properties || {}; const closure = props.road_closure === true || props.road_closure === 'true'; - const rawLevel = props.traffic_level; + const orbis = + props.traffic_level === undefined && props.road_type === undefined; + const rawLevel = orbis ? props.relative_speed : props.traffic_level; const hasLevel = typeof rawLevel === 'number' && Number.isFinite(rawLevel); // Skip features we can't color — unless closed (closures render dot-free // regardless of level, so they stay useful without one). if (!hasLevel && !closure) continue; const trafficLevel = hasLevel ? Math.min(1, Math.max(0, rawLevel)) : 0; - const roadType = typeof props.road_type === 'string' ? props.road_type : ''; + const [roadType, roadCategory] = roadClassOf(props); const lines = geometry.type === 'LineString' @@ -69,10 +109,11 @@ export function decodeFlowTile(data, z, x, y, { strict = false } = {}) { coords, trafficLevel, roadType, - roadCategory: props.road_category ?? null, + roadCategory, roadSubcategory: props.road_subcategory ?? null, closure, - coverage: props.traffic_road_coverage ?? null, + // Orbis draws each travel direction as its own line. + coverage: props.traffic_road_coverage ?? (orbis ? 'one_side' : null), leftHandTraffic: props.left_hand_traffic === true || props.left_hand_traffic === 'true', diff --git a/src/layers/traffic/flowDecode.test.mjs b/src/layers/traffic/flowDecode.test.mjs new file mode 100644 index 0000000000..1c5aacb132 --- /dev/null +++ b/src/layers/traffic/flowDecode.test.mjs @@ -0,0 +1,136 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { PbfWriter } from 'pbf'; +import { decodeFlowTile } from './flowDecode.js'; +import { flowSegmentsToRoads, isDrivableFlowSegment } from './roadModes.js'; + +/** + * One "Traffic flow" layer; each feature is a two-point line carrying the + * given properties (string, number or boolean values). + */ +function flowTile(features) { + const keys = []; + const values = []; + const tags = features.map((props) => + Object.entries(props).flatMap(([key, value]) => { + if (!keys.includes(key)) keys.push(key); + values.push(value); + return [keys.indexOf(key), values.length - 1]; + }), + ); + const writer = new PbfWriter(); + writer.writeMessage( + 3, + (_, layer) => { + layer.writeVarintField(15, 2); + layer.writeStringField(1, 'Traffic flow'); + layer.writeVarintField(5, 4096); + for (const key of keys) layer.writeStringField(3, key); + for (const value of values) + layer.writeMessage( + 4, + (_, out) => { + if (typeof value === 'string') out.writeStringField(1, value); + else if (typeof value === 'boolean') + out.writeBooleanField(7, value); + else out.writeDoubleField(3, value); + }, + null, + ); + tags.forEach((featureTags, i) => + layer.writeMessage( + 2, + (_, feature) => { + feature.writeVarintField(3, 2); + feature.writePackedVarint(2, featureTags); + // MoveTo (100, 100 + i), LineTo +200 east. + feature.writePackedVarint(4, [9, 200, 200 + 2 * i, 10, 400, 0]); + }, + null, + ), + ); + }, + null, + ); + return writer.finish(); +} +const decode = (features) => decodeFlowTile(flowTile(features), 12, 935, 1686); + +test('TomTom Maps flow features decode as before', () => { + const [segment] = decode([ + { + traffic_level: 0.6, + road_type: 'Major road', + road_category: 'primary', + traffic_road_coverage: 'full', + }, + ]); + assert.equal(segment.trafficLevel, 0.6); + assert.equal(segment.roadType, 'Major road'); + assert.equal(segment.roadCategory, 'primary'); + assert.equal(segment.coverage, 'full'); + // A TomTom Maps line with no coverage tag still reports none. + assert.equal( + decode([{ traffic_level: 1, road_type: 'Motorway' }])[0].coverage, + null, + ); +}); + +test('Orbis flow features decode into the TomTom Maps road vocabulary', () => { + const segments = decode([ + { relative_speed: 0.42, road_category: 'motorway' }, + { relative_speed: 0.9, road_category: 'trunk_link' }, + { + relative_speed: 0.7, + road_category: 'street', + road_subcategory: 'residential', + left_hand_traffic: true, + }, + { relative_speed: 1.4, road_category: 'tertiary' }, + { road_category: 'secondary', road_closure: true }, + ]); + assert.deepEqual( + segments.map((s) => [s.roadType, s.roadCategory, s.trafficLevel]), + [ + ['Motorway', 'motorway', 0.42], + ['International road', 'trunk', 0.9], + ['Local road', 'street', 0.7], + ['Connecting road', 'tertiary', 1], // clamped like traffic_level + ['Secondary road', 'secondary', 0], // closures need no speed + ], + ); + assert.equal(segments[2].roadSubcategory, 'residential'); + assert.equal(segments[2].leftHandTraffic, true); + assert.equal(segments[4].closure, true); + // Orbis draws one line per travel direction. + assert.ok(segments.every((s) => s.coverage === 'one_side')); + assert.ok(segments.every(isDrivableFlowSegment)); + assert.deepEqual( + flowSegmentsToRoads(segments).map((road) => [road.type, road.oneway]), + [ + ['motorway', 1], + ['trunk', 1], + ['residential', 1], + ['tertiary', 1], + ['secondary', 1], + ], + ); +}); + +test('Orbis service roads, tracks and speedless lines never become traffic', () => { + const segments = decode([ + { relative_speed: 0.5, road_category: 'service' }, + { relative_speed: 0.5, road_category: 'track' }, + { relative_speed: 0.5 }, + { road_category: 'primary' }, // no speed and not closed: nothing to draw + ]); + assert.deepEqual( + segments.map((s) => [s.roadType, s.roadCategory]), + [ + ['', 'service'], + ['', 'track'], + ['', null], + ], + ); + assert.equal(segments.filter(isDrivableFlowSegment).length, 0); +}); diff --git a/src/tooling/environmentProviders.test.mjs b/src/tooling/environmentProviders.test.mjs index 22eb39ad3d..2953acfc89 100644 --- a/src/tooling/environmentProviders.test.mjs +++ b/src/tooling/environmentProviders.test.mjs @@ -262,6 +262,117 @@ test('traffic refuses cross-site requests in dev and preview before spending the } }); +test('traffic falls back to the Orbis flow endpoint when TomTom refuses the key on the TomTom Maps one', async (t) => { + isolate(t, { TOMTOM_API_KEY: 'orbis-only', TOMTOM_DAILY_TILE_BUDGET: '' }); + const calls = []; + t.mock.method(globalThis, 'fetch', async (raw, init = {}) => { + const url = new URL(raw); + const orbis = url.pathname.startsWith('/maps/orbis/'); + calls.push(orbis ? 'orbis' : 'legacy'); + if (orbis) { + assert.match( + url.pathname, + /^\/maps\/orbis\/traffic\/flow\/vector\/tile\/8\/[12]\/1$/, + 'no .pbf suffix or flow style on Orbis', + ); + assert.equal(url.searchParams.get('apiVersion'), '2'); + assert.equal(url.searchParams.get('key'), null); + assert.equal(init.headers['TomTom-Api-Key'], 'orbis-only'); + return new Response(new Uint8Array([4, 5, 6])); + } + assert.equal(url.pathname, '/traffic/map/4/tile/flow/relative/8/1/1.pbf'); + assert.equal(url.searchParams.get('key'), 'orbis-only'); + return Response.json( + { detailedError: { code: 'Forbidden' } }, + { status: 403 }, + ); + }); + const request = install(tomtomProxy()); + const first = await request('/flow/8/1/1.pbf'); + assert.equal(first.status, 200); + assert.equal(first.headers['x-tomtom-cache'], 'MISS'); + assert.deepEqual([...first.body], [4, 5, 6]); + assert.deepEqual(calls, ['legacy', 'orbis']); + assert.equal(json(await request('/status')).keyRejected, false); + // Both attempts were real upstream requests. + assert.equal(json(await request('/status')).dailyCount, 2); + // The working endpoint is remembered: no second 403 per tile. + calls.length = 0; + process.env.TOMTOM_API_KEY = 'orbis-only'; + assert.equal((await request('/flow/8/2/1.pbf')).status, 200); + assert.deepEqual(calls, ['orbis']); + // A new key starts over on the TomTom Maps endpoint. + calls.length = 0; + process.env.TOMTOM_API_KEY = 'another-key'; + t.mock.method(globalThis, 'fetch', async (raw) => { + calls.push( + new URL(raw).pathname.startsWith('/maps/orbis/') ? 'orbis' : 'legacy', + ); + return new Response(new Uint8Array([7])); + }); + assert.equal((await request('/flow/8/3/1.pbf')).status, 200); + assert.deepEqual(calls, ['legacy']); +}); + +test('traffic names a key both TomTom flow endpoints refuse and stops spending the budget on it', async (t) => { + isolate(t, { TOMTOM_API_KEY: 'refused', TOMTOM_DAILY_TILE_BUDGET: '' }); + let now = Date.UTC(2026, 10, 20, 12); + t.mock.method(Date, 'now', () => now); + let calls = 0; + let status = 403; + t.mock.method(globalThis, 'fetch', async () => { + calls++; + return status === 200 + ? new Response(new Uint8Array([1])) + : new Response('{}', { status }); + }); + const request = install(tomtomProxy()); + assert.equal(json(await request('/status')).keyRejected, false); + const refused = await request('/flow/8/1/1.pbf'); + assert.equal(refused.status, 403); + assert.deepEqual(json(refused), { error: 'key_rejected' }); + assert.equal(calls, 2, 'one attempt per endpoint'); + // Other tiles answer from the verdict, without asking TomTom again. + for (const x of [2, 3, 4]) + assert.deepEqual(json(await request(`/flow/8/${x}/1.pbf`)), { + error: 'key_rejected', + }); + assert.equal(calls, 2); + const state = json(await request('/status')); + assert.equal(state.hasKey, true); + assert.equal(state.keyRejected, true); + assert.equal(state.dailyCount, 2); + // The verdict expires, so a key fixed at TomTom recovers without a restart. + now += 5 * 60_000 + 1; + status = 200; + assert.equal((await request('/flow/8/1/1.pbf')).status, 200); + assert.equal(json(await request('/status')).keyRejected, false); + // A replaced key is tried at once. + status = 403; + now += 120_001; + assert.equal((await request('/flow/8/5/1.pbf')).status, 403); + process.env.TOMTOM_API_KEY = 'replacement'; + status = 200; + assert.equal((await request('/flow/8/6/1.pbf')).status, 200); +}); + +test('traffic keeps an ordinary upstream failure on one endpoint and reports it as upstream', async (t) => { + isolate(t, { TOMTOM_API_KEY: 'fixture-key', TOMTOM_DAILY_TILE_BUDGET: '' }); + const calls = []; + t.mock.method(globalThis, 'fetch', async (raw) => { + calls.push( + new URL(raw).pathname.startsWith('/maps/orbis/') ? 'orbis' : 'legacy', + ); + return new Response('', { status: 500 }); + }); + const request = install(tomtomProxy()); + const res = await request('/flow/8/1/1.pbf'); + assert.equal(res.status, 502); + assert.deepEqual(json(res), { error: 'upstream' }); + assert.deepEqual(calls, ['legacy']); + assert.equal(json(await request('/status')).keyRejected, false); +}); + test('traffic middleware answers 502 to an oversized tile without caching it', async (t) => { isolate(t, { TOMTOM_API_KEY: 'fixture-key', TOMTOM_DAILY_TILE_BUDGET: '' }); const bodies = [];