feat(transit): live public-transit vehicles from open GTFS-Realtime feeds (#587)

Supersedes #218 by @spcpza, ported onto the current component layout with fixes from review.

Verified before merge: format, package boundaries, unit suite, build, and the browser tracking suite; a browser harness for the change where one applies; adversarial review applied.

Contributor: @spcpza. His original commit is the base of this branch, and his own re-port onto main (2026-09-15, `5b69011`) was consulted while finishing the height work. That port reads every vehicle's surface from the app's shared ground-floor system (`cachedGroundFloor` / `warmGroundFloor` / `GROUND_FLOOR_LIFT_M`) rather than sampling the scene per layer, keeps a vehicle hidden until its cell answers, warms a bounded batch per poll on a timer instead of per frame, and renders at the ellipsoid above the altitude gate. That is the sounder primitive — one floor system for the app, with the rooftop-versus-street question already settled against a datum-correct DEM prior — and this branch's height work was rebuilt on it, replacing a bespoke cell cache, ground prior and rooftop rule written here. Credit for that direction is his.

Future work (not in this PR), both written up in docs/CURRENT-STATE.md:

- Readable route names. The realtime feeds carry only `route_id`; `route_short_name` / `route_long_name` live in each operator's static GTFS `routes.txt`, and headsigns in `trips.txt`. Doing it properly wants a server-side metadata loader beside the transit proxy with fixed registered static sources, bounded indexes built once and persisted, a slow conditional refresh, and a versioned route dictionary the browser caches. Shipped here: ids are never rewritten, and a cut one carries an ellipsis instead of reading as a typo.
- Below-ground rendering. Subway vehicles are drawn on the surface directly above themselves, because the feeds carry no depth and a train nobody can see is not worth drawing; the card says so plainly. Real treatment wants OSM tunnel geometry per city and a ghosted sprite tethered to the surface. The owner recorded it and left it.

Round four and after (2026-09-15). The owner tested round three under FLIR and NVG, and then asked for the design to be taken further: elegant in performance, beautiful on camera, trails with a cache for public transport, Cesium-native, and a core other moving layers can adopt. Codex Astra reviewed the plan (it confirmed the speed fault: a glide started from the drawn position at receipt time but spanned the gap between report times, so with MBTA reporting every ~10 s against a 15 s poll every second glide ran at 1.5x; it rejected a queued-segment patch), then wrote the design, then implemented it in commit-sized passes with adversarial review between them and real-browser harness runs by the integrator.

What this PR now contains beyond the contributor's original port:

- Delayed playback in a contact-agnostic core (`src/data/contactPlayback.js`): each vehicle keeps a typed, bounded fix history and a display clock that runs at one second per second through it, a feed-derived lag behind real time, holding at its newest fix rather than extrapolating and never sprinting to catch up. Marker, course, motion word, card and trail all read the same displayed sample. The clock has an explicit time-offset and seek API so a rewind is a later feature, not a rewrite; flights and vessels are untouched.
- A selected-vehicle trail (`src/data/contactTrailRenderer.js`, `src/layers/transit/trails.js`): the retained history drawn with batched Cesium primitives, clipped at display time so the head is exactly the sprite, faded by age, in the mode colour (white under sensor presets), on the same surface path as the marker.
- Bounded proxy-side history (`server/providers/transitHistory.js`): fifteen minutes and at most 128 fixes per vehicle with hard byte ceilings, MBTA only for now (opt-in per feed), served by a small endpoint that never fetches upstream, so a selected trail survives a browser reload while the server keeps running.
- Scene work bounded to what is visible and moving, an occluder, cached DETECT candidates, holds on the render loop only while something visible moves.
- Sensor presets: white-hot haloed sprites under NVG/FLIR/noir (black-hot inverts), saturated mode colours under CRT, per-mode bracket tiers in every theme, the selected card keeping its mode colour.
- A discriminating browser harness (`scripts/qa-transit.mjs`) that samples postRender frames, reads the real card anchor, and measures fleet budgets at 800 and 3,000 vehicles.

Not built, recorded as follow-ups: curated route display names (lowest priority per the owner), fleet-wide tails, route snapping, a rewind UI, adoption of the playback core by flights and vessels.

Verified at merge (2026-09-15 morning): unit suite 3,896 passing, build clean, format and boundary gates clean, tracking regression 109/109, browser harness keyless 200 pass / 0 fail (13 sensor checks unexercised because the live view offered fewer than six isolated sprites; every judged sprite passed), keyed on Google 3D tiles 198 pass with the per-frame allocation gate as the only failure. Follow-ups recorded: the keyed build's playback sampler still allocates per frame (CPU and frame time stay within budget; the 3,000-vehicle metro view sits at a 27.6 ms frame interval against a 25 ms budget), the shared detection overlay's per-paint allocation, friendlier route names, fleet-wide tails, a rewind UI, and adopting the playback core for vessels and flights.
Co-authored-by: spcpza <[email protected]>

src/tooling/transitProvider.test.mjs

@@ -0,0 +1,386 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { PbfWriter } from 'pbf';
+import { transitProxy } from 'gods-eye-view/server/providers/transit';
+import {
+  TRANSIT_BACKOFF_LADDER_MS,
+  TRANSIT_PROXY_TTL_MS,
+} from 'gods-eye-view/sources/transit';
+
+/** Mount the plugin and return a caller for its single route. */
+function install(plugin) {
+  const routes = new Map();
+  plugin.configureServer({
+    middlewares: {
+      use(route, handler) {
+        routes.set(route, handler);
+      },
+    },
+  });
+  assert.equal(routes.size, 1);
+  return async (url = '/', method = 'GET') => {
+    const res = {
+      headersSent: false,
+      writeHead(status, headers) {
+        Object.assign(this, { status, headers, headersSent: true });
+      },
+      end(body) {
+        this.body = body;
+      },
+    };
+    await [...routes.values()][0]({ url, method }, res);
+    return res;
+  };
+}
+
+/** One GTFS-RT FeedMessage with the given header and vehicles. */
+function feedBytes({
+  incrementality = 0,
+  timestamp = 1_700_000_000,
+  vehicles = [],
+}) {
+  const writer = new PbfWriter();
+  const header = new PbfWriter();
+  header.writeStringField(1, '2.0');
+  header.writeVarintField(2, incrementality);
+  header.writeVarintField(3, timestamp);
+  writer.writeBytesField(1, header.finish());
+  for (const item of vehicles) {
+    const entity = new PbfWriter();
+    entity.writeStringField(1, item.id);
+    const vehicle = new PbfWriter();
+    const position = new PbfWriter();
+    position.writeFloatField(1, item.lat);
+    position.writeFloatField(2, item.lon);
+    vehicle.writeBytesField(2, position.finish());
+    vehicle.writeVarintField(5, item.timestamp ?? timestamp);
+    const descriptor = new PbfWriter();
+    descriptor.writeStringField(1, item.id);
+    vehicle.writeBytesField(8, descriptor.finish());
+    entity.writeBytesField(4, vehicle.finish());
+    writer.writeBytesField(2, entity.finish());
+  }
+  return writer.finish();
+}
+
+function upstreamResponse({ status = 200, headers = {}, bytes = null }) {
+  const map = new Map(
+    Object.entries(headers).map(([name, value]) => [name.toLowerCase(), value]),
+  );
+  return {
+    status,
+    ok: status >= 200 && status < 300,
+    headers: { get: (name) => map.get(String(name).toLowerCase()) ?? null },
+    body: null,
+    async arrayBuffer() {
+      const view = bytes || new Uint8Array(0);
+      return view.buffer.slice(
+        view.byteOffset,
+        view.byteOffset + view.byteLength,
+      );
+    },
+  };
+}
+
+/** A scripted upstream that records every request it is actually given. */
+function scriptedUpstream(steps) {
+  const calls = [];
+  let index = 0;
+  const fetchImpl = async (url, init) => {
+    calls.push({ url, headers: init?.headers || {}, redirect: init?.redirect });
+    const step = steps[Math.min(index, steps.length - 1)];
+    index += 1;
+    if (typeof step === 'function') return step(url, init);
+    return step;
+  };
+  return { fetchImpl, calls };
+}
+
+const FULL = feedBytes({
+  vehicles: [{ id: 'bus-1', lat: 42.36, lon: -71.06 }],
+});
+
+test('a 304 revalidation is a success, not a redirect with a missing Location', async (t) => {
+  t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 });
+  const { fetchImpl, calls } = scriptedUpstream([
+    upstreamResponse({ headers: { etag: 'W/"v1"' }, bytes: FULL }),
+    upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }),
+    upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }),
+  ]);
+  const call = install(transitProxy({ fetchImpl }));
+
+  const first = await call('/vehicles/mbta');
+  assert.equal(first.status, 200);
+  assert.equal(first.headers['X-GEV-Cache'], 'MISS');
+  assert.equal(JSON.parse(first.body).count, 1);
+
+  // Past the freshness window the proxy asks again, conditionally.
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000);
+  const second = await call('/vehicles/mbta');
+  assert.equal(calls[1].headers['If-None-Match'], 'W/"v1"');
+  assert.equal(second.status, 200, 'an unchanged feed is not a failure');
+  // MISS, not STALE-ERROR: the refresh SUCCEEDED. Treating 304 as a redirect
+  // made this a failure that happened to be masked by the serve-stale path.
+  assert.equal(second.headers['X-GEV-Cache'], 'MISS');
+  assert.equal(JSON.parse(second.body).count, 1);
+
+  // And the third one too: a 304 must not have started a backoff ladder.
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000);
+  const third = await call('/vehicles/mbta');
+  assert.equal(third.status, 200);
+  assert.equal(third.headers['X-GEV-Cache'], 'MISS');
+  assert.equal(calls.length, 3, 'every poll past the TTL reached the operator');
+  assert.equal(third.headers['Retry-After'], undefined);
+  assert.equal(
+    third.headers['X-Transit-Backoff'],
+    undefined,
+    'no ladder was started',
+  );
+});
+
+test('a differential feed condemns the snapshot that preceded it', async (t) => {
+  t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 });
+  const differential = feedBytes({
+    incrementality: 1,
+    vehicles: [{ id: 'bus-1', lat: 42.36, lon: -71.06 }],
+  });
+  const { fetchImpl } = scriptedUpstream([
+    upstreamResponse({ bytes: FULL }),
+    upstreamResponse({ bytes: differential }),
+    upstreamResponse({ bytes: differential }),
+  ]);
+  const call = install(transitProxy({ fetchImpl }));
+
+  assert.equal((await call('/vehicles/mbta')).status, 200);
+
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000);
+  const refused = await call('/vehicles/mbta');
+  assert.equal(refused.status, 502);
+  assert.match(JSON.parse(refused.body).error, /differential/);
+
+  // The next caller arrives INSIDE the cooldown the failure just started, which
+  // is precisely the path that used to hand back the condemned snapshot as a
+  // 200 because it was still inside the serve-stale window.
+  t.mock.timers.setTime(Date.now() + 2_000);
+  const after = await call('/vehicles/mbta');
+  assert.notEqual(
+    after.status,
+    200,
+    `served ${after.status} with ${after.body}`,
+  );
+  assert.equal(after.headers['X-GEV-Cache'], 'NONE');
+  assert.equal(after.headers['X-Transit-Backoff'], 'cooldown');
+});
+
+test('a failing operator is put on a cooldown instead of being re-asked every poll', async (t) => {
+  t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 });
+  const { fetchImpl, calls } = scriptedUpstream([
+    upstreamResponse({ status: 500 }),
+    upstreamResponse({ status: 500 }),
+  ]);
+  const call = install(transitProxy({ fetchImpl }));
+
+  const failed = await call('/vehicles/mbta');
+  assert.equal(failed.status, 502);
+  assert.equal(
+    failed.headers['Retry-After'],
+    String(TRANSIT_BACKOFF_LADDER_MS[0] / 1000),
+  );
+  assert.equal(calls.length, 1);
+
+  // Inside the cooldown the operator is not contacted at all.
+  t.mock.timers.setTime(Date.now() + 1_000);
+  const held = await call('/vehicles/mbta');
+  assert.equal(held.status, 503);
+  assert.equal(held.headers['X-Transit-Backoff'], 'cooldown');
+  assert.equal(calls.length, 1, 'the operator was left alone');
+
+  // Past the first rung it is tried again, and a second failure waits longer.
+  t.mock.timers.setTime(Date.now() + TRANSIT_BACKOFF_LADDER_MS[0]);
+  const retried = await call('/vehicles/mbta');
+  assert.equal(calls.length, 2);
+  assert.equal(
+    retried.headers['Retry-After'],
+    String(TRANSIT_BACKOFF_LADDER_MS[1] / 1000),
+  );
+});
+
+test('a redirect is followed only within the feed origin, and never contacted first', async (t) => {
+  t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 });
+  const sameOrigin = scriptedUpstream([
+    upstreamResponse({
+      status: 302,
+      headers: { location: '/files/current.pb' },
+    }),
+    upstreamResponse({ bytes: FULL }),
+  ]);
+  const call = install(transitProxy({ fetchImpl: sameOrigin.fetchImpl }));
+  const ok = await call('/vehicles/capmetro-austin');
+  assert.equal(ok.status, 200);
+  assert.equal(sameOrigin.calls.length, 2);
+  assert.equal(
+    sameOrigin.calls[1].url,
+    'https://data.texas.gov/files/current.pb',
+  );
+  assert.equal(sameOrigin.calls[0].redirect, 'manual');
+
+  const offOrigin = scriptedUpstream([
+    upstreamResponse({
+      status: 302,
+      headers: { location: 'https://cdn.elsewhere.test/current.pb' },
+    }),
+    upstreamResponse({ bytes: FULL }),
+  ]);
+  const refuse = install(transitProxy({ fetchImpl: offOrigin.fetchImpl }));
+  const blocked = await refuse('/vehicles/capmetro-austin');
+  assert.equal(blocked.status, 504);
+  assert.equal(
+    offOrigin.calls.length,
+    1,
+    'the disallowed host was never contacted, only its Location read',
+  );
+});
+
+test('only registered feed ids reach an operator', async (t) => {
+  const { fetchImpl, calls } = scriptedUpstream([
+    upstreamResponse({ bytes: FULL }),
+  ]);
+  const call = install(transitProxy({ fetchImpl }));
+  for (const path of [
+    '/vehicles/not-a-feed',
+    '/vehicles/..%2F..%2Fetc',
+    '/vehicles/',
+    '/elsewhere',
+  ]) {
+    const missing = await call(path);
+    assert.equal(missing.status, 404, `${path} is not a door to an upstream`);
+  }
+  assert.equal(calls.length, 0, 'no operator was contacted for any of them');
+  const catalog = JSON.parse((await call('/feeds')).body);
+  assert.ok(catalog.feeds.length > 0);
+  for (const feed of catalog.feeds) {
+    assert.equal('url' in feed, false);
+    assert.equal('headers' in feed, false);
+  }
+});
+
+test('a 304 reports that the operator answered, not that the body is new', async (t) => {
+  // The body a revalidated feed keeps serving carries the fetch time of the
+  // payload it holds, which stops advancing the moment the operator's file
+  // stops changing. That is the right answer for "how old are these positions"
+  // and the wrong one for "when did we last hear from them" — read as the
+  // latter it aged a feed whose every request succeeded into silence.
+  t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 });
+  const firstAt = Date.now();
+  const { fetchImpl } = scriptedUpstream([
+    upstreamResponse({ headers: { etag: 'W/"v1"' }, bytes: FULL }),
+    upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }),
+    upstreamResponse({ status: 304, headers: { etag: 'W/"v1"' } }),
+  ]);
+  const call = install(transitProxy({ fetchImpl }));
+
+  const first = await call('/vehicles/mbta');
+  assert.equal(Number(first.headers['X-Transit-Contact']), firstAt);
+  assert.equal(JSON.parse(first.body).fetchedAt, firstAt);
+
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000);
+  const revalidatedAt = Date.now();
+  const second = await call('/vehicles/mbta');
+  assert.equal(
+    JSON.parse(second.body).fetchedAt,
+    firstAt,
+    'the positions are as old as they are',
+  );
+  assert.equal(
+    Number(second.headers['X-Transit-Contact']),
+    revalidatedAt,
+    'but the operator answered just now',
+  );
+
+  // Served from the fresh cache, the contact time is the last real contact —
+  // not the moment this request happened to arrive.
+  const third = await call('/vehicles/mbta');
+  assert.equal(third.headers['X-GEV-Cache'], 'HIT');
+  assert.equal(Number(third.headers['X-Transit-Contact']), revalidatedAt);
+});
+
+test('a feed served from cache during an outage does not claim fresh contact', async (t) => {
+  // The other half: STALE-ERROR must carry the LAST SUCCESSFUL contact, so the
+  // browser can tell a revalidated feed from a dead one.
+  t.mock.timers.enable({ apis: ['Date'], now: 1_700_000_000_000 });
+  const contactedAt = Date.now();
+  const { fetchImpl } = scriptedUpstream([
+    upstreamResponse({ bytes: FULL }),
+    upstreamResponse({ status: 500 }),
+    upstreamResponse({ status: 500 }),
+  ]);
+  const call = install(transitProxy({ fetchImpl }));
+  await call('/vehicles/mbta');
+
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1_000);
+  const during = await call('/vehicles/mbta');
+  assert.equal(during.headers['X-GEV-Cache'], 'STALE-ERROR');
+  assert.equal(
+    Number(during.headers['X-Transit-Contact']),
+    contactedAt,
+    'silence does not count as an answer',
+  );
+});
+
+// Include the provider-local storage contract in the repository's src test discovery.
+import '../../server/providers/transitHistory.test.mjs';
+
+test('history reads never fetch upstream, and 304/cache/failure replays never append', async (t) => {
+  t.mock.timers.enable({ apis: ['Date'], now: 1700000000000 });
+  const { calls, fetchImpl } = scriptedUpstream([
+    upstreamResponse({ bytes: FULL }),
+    upstreamResponse({ status: 304 }),
+    upstreamResponse({ status: 503 }),
+  ]);
+  const plugin = transitProxy({ fetchImpl });
+  t.after(() => plugin.closeBundle());
+  const call = install(plugin);
+  const read = () => call('/trail/mbta/bus-1');
+  assert.equal(JSON.parse((await read()).body).fixes.length, 0);
+  assert.equal(calls.length, 0);
+  await call('/vehicles/mbta');
+  const initial = JSON.parse((await read()).body);
+  assert.equal(initial.fixes.length, 1);
+  await call('/vehicles/mbta');
+  assert.equal(calls.length, 1);
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1);
+  await call('/vehicles/mbta');
+  assert.deepEqual(JSON.parse((await read()).body), initial);
+  t.mock.timers.setTime(Date.now() + TRANSIT_PROXY_TTL_MS + 1);
+  await call('/vehicles/mbta');
+  assert.deepEqual(JSON.parse((await read()).body), initial);
+  assert.equal(calls.length, 3);
+  for (const path of [
+    '/trail/mbta/%2Fescape',
+    '/trail/mbta/abc%00bad',
+    '/trail/mbta/%E0%A4%A',
+    `/trail/mbta/${'a'.repeat(257)}`,
+    '/trail/capmetro-austin/bus-1',
+  ])
+    assert.equal((await call(path)).status, 404);
+  assert.equal(
+    calls.length,
+    3,
+    'malformed and non-retained reads cannot reach an operator',
+  );
+});
+test('history read admission is separate from upstream admission and closes with the server', async (t) => {
+  t.mock.timers.enable({ apis: ['Date'], now: 1700000000000 });
+  const { calls, fetchImpl } = scriptedUpstream([
+    upstreamResponse({ bytes: FULL }),
+  ]);
+  const plugin = transitProxy({ fetchImpl });
+  t.after(() => plugin.closeBundle());
+  const call = install(plugin);
+  for (let i = 0; i < 30; i++)
+    assert.equal((await call('/trail/mbta/bus-1')).status, 200);
+  assert.equal((await call('/trail/mbta/bus-1')).status, 429);
+  assert.equal(calls.length, 0);
+  assert.equal((await call('/vehicles/mbta')).status, 200);
+  assert.equal(calls.length, 1);
+});