Bound fallback retry attempts with a deadline

An Esri or keyless terrain attempt that never settled kept the single-flight
retry open forever, so neither provider was retried again. Each attempt now
counts as failed after 60 seconds and the retry backs off as usual; its
signal is aborted, and a provider that lands after the deadline is released
rather than installed or cached.

CURRENT-STATE now says only the construction fallback to OSM is retried, not
the switch made after repeated Esri tile failures.
This commit is contained in:
Sameh Khamis committed 2026-10-16 16:47:11 -07:00
1 parent f0d43a62e8
commit 759c7964c7
3 files changed
+110 -9

No files matched your search

+5 -5
View File
@@ -385,10 +385,10 @@ plus up to 1.5 s of jitter, and `Retry-After` extending the cooldown when the
upstream exposes it. One console warning is logged per cooldown window. Other
failures keep Cesium's handling and the flat `EllipsoidTerrainProvider`
fallback; while that flat terrain is on a shown globe the map controller
retries Re:Earth in the background (30 s, doubling to a 10-minute cap) and
installs the real terrain without waiting for a map switch or an imagery
retry. Behind a 3D tileset it neither retries nor installs; the next globe
activation does.
retries Re:Earth in the background (30 s, doubling to a 10-minute cap, each
attempt abandoned after 60 s) and installs the real terrain without waiting
for a map switch or an imagery retry. Behind a 3D tileset it neither retries
nor installs; the next globe activation does.
`scripts/qa-terrain-429.mjs` reproduces the burst against a keyless
dev server, counts real throttles with their headers as evidence, synthesises
throttles with `--inject N` when the upstream is not throttling, and fails if
@@ -4776,7 +4776,7 @@ are omitted rather than framing the wrong part of the globe.
- The bottom Visual Presets tray presents a **five-tile MAP SOURCE row** (`#map-stack-chips`, `src/mapStackChips.js`): Google 3D, Esri Satellite, Bing Aerial, Bing Labels, and OSM. The duplicate left `#stack-panel` is retired. The five tiles share one row on desktop and two rows on narrow screens, carry `aria-pressed` on the active source, and remain keyboard-reachable with a visible focus outline.
- The lit tile follows controller state, not the click: a rejected switch (no ion token) or a superseded one (rapid A→B) leaves the genuinely active source lit, and the tray heading keeps its short-label status readout (`...` while switching, amber on `lastError`).
- Ion stacks remain visible and keyboard-focusable when no ion token is configured, but expose `aria-disabled="true"` and do not switch. Their accessible label and tooltip quote `getStacks().unavailableReason` — the same string `setStack()` puts in the toast. OSM works keyless. The `ION` badge is gated on the stack's own `requiresIon`, so a `photoreal` chip unavailable because the Google tileset failed says so instead of falsely demanding an ion token.
- Stack choice participates in share links (`src/sharelink.js`) and falls back to the best available stack when the requested one is unavailable (keyless boots land on Esri; OSM takes over automatically if Esri is unreachable; while that fallback is on screen Esri is retried in the background, 30 s doubling to a 10-minute cap, and replaces OSM once it loads). Share-link restore, the `set_map_stack` voice tool, and the chip row all land on the same `_setMapStack()` path.
- Stack choice participates in share links (`src/sharelink.js`) and falls back to the best available stack when the requested one is unavailable (keyless boots land on Esri; OSM takes over automatically if Esri is unreachable; while that construction fallback is on screen Esri is retried in the background, 30 s doubling to a 10-minute cap with each attempt abandoned after 60 s, and replaces OSM once it loads. The switch to OSM after repeated Esri tile failures is not retried; selecting Esri again tries it). Share-link restore, the `set_map_stack` voice tool, and the chip row all land on the same `_setMapStack()` path.
### Voice Map Whiteboard / Annotations (June 2026)
+29 -4
View File
@@ -12,6 +12,8 @@ const AUTOMATIC = Symbol('automatic switch');
/** Background retries of a fallback in use back off from 30 s to 10 min. */
const FALLBACK_RETRY_MS = 30_000;
const FALLBACK_RETRY_MAX_MS = 600_000;
/** A retry attempt still unsettled after a minute counts as failed. */
const FALLBACK_ATTEMPT_MS = 60_000;
const resolved = (provider, effectiveStackId) => ({
provider,
@@ -354,17 +356,38 @@ export class MapSourceController {
if (!id && !terrain) return;
this._retrying = true;
this._retryRequested = false;
const attempt = async (create) => {
// An attempt that outlives its deadline counts as a failure. Its signal is
// aborted, but the Esri and keyless factories ignore it and Cesium's
// requests have no timeout, so a late result is released, never used.
const attempt = async (create, release) => {
const expiry = new AbortController();
let timer;
const pending = Promise.resolve().then(() =>
create?.({
signal: AbortSignal.any([this._abort.signal, expiry.signal]),
}),
);
const deadline = new Promise((resolve) => {
timer = setTimeout(resolve, FALLBACK_ATTEMPT_MS, expiry);
});
try {
return await create?.({ signal: this._abort.signal });
const result = await Promise.race([pending, deadline]);
if (result !== expiry) return result;
expiry.abort();
pending.then(release, () => {});
return null;
} catch {
return null; // Still unavailable: keep the fallback.
} finally {
clearTimeout(timer);
}
};
// Each recovers as soon as it loads, never waiting on the other.
const retryImagery = async () => {
const cached = this._imageryProviders.get(id);
const provider = await attempt(this._sources.get(id).imagery);
const provider = await attempt(this._sources.get(id).imagery, (late) =>
this._dispose(late),
);
// Obsolete: destroyed, or the cache moved on while this was in flight.
if (this._destroyed || this._imageryProviders.get(id) !== cached) {
this._dispose(provider);
@@ -388,7 +411,9 @@ export class MapSourceController {
};
const retryTerrain = async () => {
const cached = this._terrainProviders.get(terrain.id);
const real = await attempt(terrain.create);
const real = await attempt(terrain.create, (late) =>
this._dispose(late?.provider),
);
if (
this._destroyed ||
this._terrainProviders.get(terrain.id) !== cached ||
+76
View File
@@ -1180,3 +1180,79 @@ test('a stalled imagery retry does not hold back recovered terrain', async (t) =
await settle();
env.controller.destroy();
});
test('a retry attempt that never settles is abandoned at its deadline and released if it lands late', async (t) => {
const hung = deferred();
const late = { id: 'late', destroy: () => (late.destroyed = true) };
const { env, installed, terrain } = terrainFixture(t, async (n) => {
if (n === 2) await hung.promise;
return n === 2 ? { provider: late } : flatTerrain(n);
});
const esri = failingEsri(env);
await env.controller.setStack('esri-imagery');
t.mock.timers.tick(30_000);
await settle();
assert.equal(esri.attempts, 2, 'imagery failed');
assert.equal(terrain.calls, 2, 'terrain hangs');
t.mock.timers.tick(60_000); // The hung attempt's deadline.
await settle();
t.mock.timers.tick(60_000); // The doubled backoff.
await settle();
assert.equal(esri.attempts, 3, 'imagery is retried again');
assert.equal(terrain.calls, 3, 'terrain is retried again');
hung.resolve();
await settle();
assert.equal(late.destroyed, true, 'the late provider is released');
assert.ok(!installed.includes(late), 'and never installed');
env.controller.destroy();
});
test('a failed retry after a mid-retry re-show re-arms once, after the doubled delay', async (t) => {
t.mock.timers.enable({ apis: ['setTimeout'] });
const env = publicFixture();
const esri = failingEsri(env);
await env.controller.setStack('esri-imagery');
const gate = deferred();
esri.next = gate.promise;
t.mock.timers.tick(30_000);
await settle();
await env.controller.setStack('osm');
await env.controller.setStack('esri-imagery');
gate.resolve();
await settle();
assert.equal(esri.attempts, 2);
t.mock.timers.tick(59_999);
await settle();
assert.equal(esri.attempts, 2, 'not before the doubled delay');
t.mock.timers.tick(1);
await settle();
assert.equal(esri.attempts, 3, 'one retry after it');
t.mock.timers.tick(119_999);
await settle();
assert.equal(esri.attempts, 3, 'no second timer was armed');
env.controller.destroy();
});
test('destroy releases recovered imagery and terrain that lands afterwards', async (t) => {
const stalled = deferred();
const real = { id: 'real', destroy: () => (real.destroyed = true) };
const { env, installed } = terrainFixture(t, async (n) => {
if (n === 1) return flatTerrain(n);
await stalled.promise;
return { provider: real };
});
const esri = failingEsri(env);
const imagery = env.providers.get('esri-imagery');
imagery.destroy = () => (imagery.destroyed = true);
await env.controller.setStack('esri-imagery');
esri.failing = false;
t.mock.timers.tick(30_000);
await settle();
assert.equal(env.imagery.at(-1).provider, imagery, 'imagery has landed');
env.controller.destroy();
stalled.resolve();
await settle();
assert.equal(imagery.destroyed, true);
assert.equal(real.destroyed, true);
assert.ok(!installed.includes(real));
});