Boot the whole application on the stub renderer, headless, in CI

- dev/stub-app/main.js starts the standalone application on the stub
  renderer with only the contract's required members (?capabilities=all
  gives it every capability) and steps its frames.
- npm run qa:stub-boot serves the application's own page with that entry,
  and fails when any Cesium module or asset is requested (the stylesheet
  the build's Cesium plugin links into every page is reported), on a page
  or console error, or when a data layer fails to turn on or off. A feed's
  failed response and the dev server client's own errors are reported;
  --strict also fails a layer that drew nothing.
- The browser CI job runs it.
- RENDERER-CONTRACT.md and CURRENT-STATE.md describe the application on
  the stub and the boot check.
This commit is contained in:
Sameh Khamis committed 2026-11-10 17:46:15 -08:00
1 parent ae4fbb5053
commit 8a4867b1b7
7 files changed
+524 -9

No files matched your search

+5
View File
@@ -102,6 +102,11 @@ jobs:
- name: Street Level gate with Mapillary fixtures
run: npm run qa:street-level:fixtures -- --strict --url http://localhost:4173
- name: The whole application on the stub renderer, with no Cesium loaded
run: |
mkdir -p qa-artifacts
npm run qa:stub-boot -- --json qa-artifacts/stub-boot.json
- name: Preview server log
if: failure()
run: cat preview.log
+60
View File
@@ -0,0 +1,60 @@
/**
* The whole application on the stub renderer (`src/render/stub`), with only
* the contract's required members: the proof that the application runs on
* those members alone, and needs no code that knows which renderer it is.
* `scripts/qa-stub-boot.mjs` serves the application's page with this entry
* in place of `src/main.js`, boots it headless, runs its data layers and
* checks that no Cesium module loads.
*
* The stub runs nothing on its own: each animation frame here ticks it once.
* `window.__stubBoot` exposes the application and the renderer it made.
*/
import { createStandaloneApplication } from '../../src/standalone/application.js';
import { createStubRendererPlugin } from '../../src/render/stub/index.js';
const renderers = [];
/** Tick a renderer once per animation frame until it is destroyed. */
function drive(renderer) {
const step = () => {
if (!renderers.includes(renderer)) return;
renderer.testing.tick();
requestAnimationFrame(step);
};
requestAnimationFrame(step);
}
// The required tier, no capability at all; `?capabilities=all` gives the
// stub every capability, to see what the layers draw with them.
const allCapabilities =
new URLSearchParams(location.search).get('capabilities') === 'all';
const plugin = createStubRendererPlugin(
allCapabilities ? {} : { capabilities: [] },
(renderer) => {
renderers.push(renderer);
const destroy = renderer.destroy.bind(renderer);
renderer.destroy = () => {
renderers.splice(renderers.indexOf(renderer), 1);
destroy();
};
drive(renderer);
},
);
const application = createStandaloneApplication({
renderer: plugin,
allowQaRegistration: true,
});
window.__stubBoot = {
application,
renderers,
started: application.start().then(
() => ({ ok: true }),
(error) => {
console.error('Stub application failed to start:', error);
return { ok: false, error: String(error?.stack || error) };
},
),
};
+40 -2
View File
@@ -6853,8 +6853,7 @@ easier to meet (detection is now on more often), but does not create it.
`src/render/contract/` types the renderer contract, version `0`: nine ports
(frame, globe, draw, camera, query, input, effects, credits and quality),
plus the renderer itself. `PORTS` lists every member at run time, and
`contract.test.mjs` keeps that list and the types in step. The application
does not yet draw through the contract: it still drives Cesium directly.
`contract.test.mjs` keeps that list and the types in step.
`src/render/stub/` is a renderer that draws nothing, with the default
camera's exact maths. `src/render/conformance/` is the suite: logic checks
in Node, plus camera traces recorded from Cesium's own `Camera` into
@@ -6862,6 +6861,45 @@ in Node, plus camera traces recorded from Cesium's own `Camera` into
checked against `goldens/visual/` on the `/dev/conformance/` page. See
[RENDERER-CONTRACT.md](RENDERER-CONTRACT.md).
### The application and its renderer
The application starts on whichever `RendererPlugin` it is given
(`createStandaloneApplication({ renderer })`), and on Cesium when none is.
`src/app/scene.js` loads the Cesium adapter (`src/render/cesium/application.js`)
only then, with `import()`; the standalone page starts that load as it
starts, beside its own setup. Outside `src/render/` no code reaches a
renderer's own objects: the record of files that still did,
`src/render/viewerAccessAllowList.json`, is empty.
- What every layer and tool is handed is the scene object
(`createSceneHost`), bound to the renderer: its container, and whether it
is destroyed. Everything else goes through the renderer's ports.
- What a renderer's own plugin adds beside the contract (stale draw-command
trims, inline panel drawing, the zoom-out guard, its own map engine, what
QA scripts read) it hands through `src/render/applicationHooks.js`; the
application uses each where given and does without where not.
Renderer-only startup (Cesium's free-look limit, credit keyboard access)
runs in that plugin's `create`.
- The map stacks switch through the plugin's map engine, else the globe
port (`src/maps/globeEngine.js`): Esri and OSM as `xyz` imagery, Cesium
ion imagery as `world` imagery, terrain as `world` terrain, or the
ellipsoid where no keyless terrain is offered, and photoreal as the port's
photoreal base. A photoreal base that fails to come back leaves the globe
shown.
- The renderer follows its container's size (`Renderer.resize`, from a
`ResizeObserver`).
`npm run qa:stub-boot` boots the whole application on the stub with only
the contract's required members (`dev/stub-app/main.js`, served in place of
`src/main.js` on the application's own page), headless. It fails if
Cesium's code is requested (the one stylesheet `vite-plugin-cesium` links
into every page is reported), on any page or console error, or if a data
layer (flights, transit, traffic, CCTV, AIS vessels, weather radar and
satellite, wind, satellites, FIRMS, wildlife) fails to turn on or off. A
feed's failed response (no key, no network) is reported, not failed;
`--strict` also fails a layer that drew nothing, and `--all-capabilities`
gives the stub every capability. CI runs it in the browser job.
`src/geo/` (export `./geo`) is the Cesium-free maths: WGS84, ECEF and
geodetic conversions, surface normals and east-north-up frames, as plain
`{x, y, z}` objects. It follows Cesium's formulas step for step, and
+48 -6
View File
@@ -170,13 +170,15 @@ back to Cesium. Both lists only ever shrink.
when it no longer drives Cesium at all; nothing is added. The list is
empty: no file outside the adapter reaches Cesium. Two files name
the adapter by design and are not on it: the application scene
(`src/app/scene.js`), the composition root that picks the Cesium renderer,
may import `src/render/cesium/application.js` and nothing else of the
adapter; the conformance page (`dev/conformance/main.js`) may import the
(`src/app/scene.js`), the composition root that picks the default renderer
when it is given none, may import `src/render/cesium/application.js` (only
then, so a page on the stub never loads Cesium) and nothing else of
the adapter; the conformance page (`dev/conformance/main.js`) may import the
adapter's harness and scene renderer. The test names both
(`ADAPTER_ENTRIES`), and that list is never grown.
- `src/render/viewerAccessAllowList.json` counts, per file off that list,
the Cesium-shaped accesses through the viewer it still makes
the Cesium-shaped accesses through the viewer it still makes (none: the
list is empty, so no file may make one)
(`src/render/viewerAccess.test.mjs`): the viewer's `entities`,
`dataSources`, `scene`, `camera`, `clock`, `imageryLayers`,
`terrainProvider`, `trackedEntity` and `canvas`, and the scene's
@@ -194,10 +196,50 @@ back to Cesium. Both lists only ever shrink.
`--check` to see what it would change): it lowers counts and drops
entries, and never adds or raises one.
## The application and its renderer
The application runs on a renderer with only the contract's required
members (the stub proves it below); nothing outside `src/render/<renderer>/` asks which renderer it runs
on (no `renderer.id`, no engine type, no engine object's presence).
- **The scene object.** `createApplicationScene` (`src/app/scene.js`) takes a
`RendererPlugin`, or a promise of one (default: Cesium, imported only then;
`loadDefaultRenderer` starts that import as the standalone page starts),
creates the renderer in the page's container, and hands every layer and
tool the application's own scene object (`createSceneHost` in
`src/render/rendererOf.js`): the container and `isDestroyed()`, bound to
the renderer. Code reaches the renderer's
ports through it (`findCamera`, `drawOf`, `canvasOf`...).
- **What only the Cesium renderer needs at startup** is done in its
`create`: for Cesium, the free-look height limit and keyboard access to its
credits' expand link (`src/render/cesium/appRenderer.js`).
- **What a plugin may hand the application beside the contract** goes through
`provideApplicationHooks(renderer, hooks)` (`src/render/applicationHooks.js`),
each hook optional and used only where given: a map engine of its own,
stale command trimming, inline panel rendering, whether an entity is its
own, the zoom-out guard, and what QA scripts read from
`window.__godsEyeView` (Cesium's viewer there, for the scripts that drive
it). None is a contract member, and a renderer without them runs the
application without that behaviour.
- **The map stacks** switch through the renderer's own map engine when its
plugin hands one, else through its globe port (`src/maps/globeEngine.js`);
see `docs/CURRENT-STATE.md` § Map source factories and coordination.
- **The proof.** `npm run qa:stub-boot` (`scripts/qa-stub-boot.mjs`) serves
the application's page with `dev/stub-app/main.js` as its entry: the whole
application on the stub with no capability at all. It fails when the page
requests any Cesium module or asset, on a page or console error, or when a
data layer (flights, transit, traffic, CCTV, vessels, weather, wind,
satellites, FIRMS, wildlife) fails to turn on or off; `--strict` also fails
a layer that draws nothing on the stub (it needs the live feeds).
`src/app/scene.test.mjs` runs the scene itself on the stub in Node.
## The stub renderer
`createStubRenderer({ capabilities, viewport, time })` implements every port
in memory and draws nothing. It has:
`createStubRenderer({ capabilities, viewport, time, container })` implements
every port in memory and draws nothing. Given a `container` (in a page), its
canvas is a real canvas filling it, which `resize()` follows; otherwise a
stand-in that takes event listeners. `createStubRendererPlugin(options,
onCreate)` is the stub as a `RendererPlugin`. It has:
- the default camera's exact maths: views, `lookAt`, flights with their
height arc and easing, `changed` thresholds and follows;
+1
View File
@@ -49,6 +49,7 @@
"test": "node scripts/run-unit-tests.mjs",
"test:perf": "node scripts/run-perf-tests.mjs",
"test:track": "node scripts/track-regression.mjs",
"qa:stub-boot": "node scripts/qa-stub-boot.mjs",
"qa:map-source-tray": "node scripts/qa-map-source-tray.mjs",
"qa:street-level": "node scripts/qa-street-level.mjs",
"qa:street-level:fixtures": "node scripts/qa-street-level.mjs --fixtures",
+8 -1
View File
@@ -1444,5 +1444,12 @@
"src/ui/ownerFlagSheet.js",
"src/data/placePacksLoading.test.mjs",
"src/ui/placePacksLoading.js",
"src/ui/placePacksLoading.test.mjs"
"src/ui/placePacksLoading.test.mjs",
"src/render/applicationHooks.js",
"src/maps/globeEngine.js",
"src/maps/globeEngine.test.mjs",
"src/render/cesium/viewers.js",
"src/app/scene.test.mjs",
"dev/stub-app/main.js",
"scripts/qa-stub-boot.mjs"
]
+362
View File
@@ -0,0 +1,362 @@
#!/usr/bin/env node
/**
* Boot the whole application on the stub renderer with only the contract's
* required members (`dev/stub-app/main.js`), headless, and check that:
*
* - it starts, with no Cesium module requested at all (no script, no
* worker, no asset it would fetch; the one Cesium stylesheet the build's
* `vite-plugin-cesium` links into every page is reported, not failed);
* - each data layer named below turns on, and off, without a page error or
* a console error (a feed's failed response is reported as the feed's,
* not failed: without network access or keys a feed answers with an
* error, and the layer must take that without one of its own; the dev
* server's client's own errors are reported as the tooling's);
* - with `--strict`, each of them draws on the stub: items in its draw sets,
* imagery added through the globe port, or a field (`draw.field`).
*
* The page is the application's own (`index.html`), served with the stub
* entry in place of `src/main.js`. Without `--url` it starts a Vite dev
* server of this checkout on `--port` (default 5290) and stops it after.
* Live feeds answer through the dev server's providers, so without network
* access or keys a layer turns on with nothing to draw: the run says so,
* and only `--strict` fails it.
*
* `--all-capabilities` gives the stub every capability instead, to see what
* the layers draw where a renderer has them all.
*
* npm run qa:stub-boot [-- --url http://127.0.0.1:5280] [--strict]
* [--settle-ms 20000] [--json path] [--all-capabilities]
*/
import { writeFileSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import puppeteer from 'puppeteer';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
/** The data layers the stub must run, by catalog id. */
export const STUB_BOOT_LAYERS = Object.freeze([
'flights',
'transit',
'traffic',
'cctv',
'ais-live-vessels',
'weather-radar',
'weather-satellite',
'wind',
'satellites',
'local-firms',
'wildlife-sightings',
]);
/** A request that is Cesium: the package, its pre-bundle, or its assets. */
export const isCesiumRequest = (url) =>
/(?:^|[/._-])cesium(?:[/._-]|$)/i.test(new URL(url).pathname);
/** The stylesheet `vite-plugin-cesium` links into every page it serves. */
const isPluginStylesheet = (url, type) =>
type === 'stylesheet' &&
new URL(url).pathname === '/cesium/Widgets/widgets.css';
/** A browser's line for a response that failed: the feed's, not the page's. */
const isFailedResponse = (text) => /^Failed to load resource:/.test(text);
/**
* An error the dev server's own client logs (its socket to the server, which
* a headless browser can refuse): the tooling's, not the application's.
*/
const isDevClient = (at) => {
try {
return new URL(at).pathname === '/@vite/client';
} catch {
return false;
}
};
function parseArgs(argv) {
const args = { strict: false, port: 5290, settleMs: 20_000 };
for (let i = 0; i < argv.length; i++) {
const arg = argv[i];
if (arg === '--strict') args.strict = true;
else if (arg === '--url') args.url = argv[++i];
else if (arg === '--port') args.port = Number(argv[++i]);
else if (arg === '--settle-ms') args.settleMs = Number(argv[++i]);
else if (arg === '--json') args.json = argv[++i];
else if (arg === '--all-capabilities') args.allCapabilities = true;
else throw new Error(`Unknown argument: ${arg}`);
}
return args;
}
async function startViteServer(port) {
const { createServer } = await import('vite');
const server = await createServer({
root: ROOT,
configFile: path.join(ROOT, 'vite.config.js'),
// No hot reload: the page is loaded once and nothing edits it.
server: { port, strictPort: true, host: '127.0.0.1', hmr: false },
logLevel: 'warn',
});
await server.listen();
return server;
}
/**
* In the page: what the stub renderer draws now, its draw sets with how many
* items each holds and the globe port's imagery layers, and how many calls
* it has had on each port member.
*/
function readDrawn() {
const renderer = window.__stubBoot?.renderers?.[0];
if (!renderer) return null;
const testing = renderer.testing;
const calls = {};
for (const call of testing.calls) {
const key = `${call.port}.${call.member}`;
calls[key] = (calls[key] ?? 0) + 1;
}
return {
sets: Object.fromEntries(
testing.sets().map((id) => [id, testing.setRecord(id)?.items?.size ?? 0]),
),
imagery: renderer.globe.imagery.list().length,
calls,
};
}
async function main() {
const args = parseArgs(process.argv.slice(2));
const server = args.url ? null : await startViteServer(args.port);
const base = (args.url ?? `http://127.0.0.1:${args.port}`).replace(/\/$/, '');
const browser = await puppeteer.launch({
headless: true,
executablePath:
process.env.PUPPETEER_EXECUTABLE_PATH ||
(await puppeteer.executablePath()),
defaultViewport: { width: 1280, height: 800 },
protocolTimeout: 300_000,
args: ['--no-sandbox'],
});
/** Every error the page reports, in order: `page`, `console` or `response`. */
const errors = [];
const cesiumRequests = [];
const cesiumStylesheets = [];
const report = { url: base, layers: {}, ok: false };
try {
const page = await browser.newPage();
page.on('console', (message) => {
if (message.type() !== 'error') return;
const text = message.text();
const at = message.location()?.url ?? null;
errors.push({
kind: isFailedResponse(text)
? 'response'
: isDevClient(at)
? 'tooling'
: 'console',
text,
at,
});
});
page.on('pageerror', (error) =>
errors.push({ kind: 'page', text: String(error?.stack || error) }),
);
await page.setRequestInterception(true);
page.on('request', async (request) => {
const url = request.url();
if (url.startsWith(base) && isCesiumRequest(url))
(isPluginStylesheet(url, request.resourceType())
? cesiumStylesheets
: cesiumRequests
).push(url);
// The application's page, with the stub entry in place of its own.
if (request.isNavigationRequest() && url.startsWith(base)) {
const response = await fetch(url);
const html = (await response.text()).replace(
'src="/src/main.js"',
'src="/dev/stub-app/main.js"',
);
if (!html.includes('/dev/stub-app/main.js'))
throw new Error('The page has no /src/main.js entry to replace');
await request.respond({
status: response.status,
contentType: 'text/html',
body: html,
});
return;
}
await request.continue();
});
await page.goto(
`${base}/${args.allCapabilities ? '?capabilities=all' : ''}`,
{ waitUntil: 'load', timeout: 180_000 },
);
await page.waitForFunction(() => window.__stubBoot, { timeout: 180_000 });
const started = await page.evaluate(() => window.__stubBoot.started);
report.started = started;
if (!started.ok) throw new Error(`Startup failed: ${started.error}`);
await page.waitForFunction(() => window.__godsEyeView?.dataManager, {
timeout: 60_000,
});
report.renderer = await page.evaluate(() => {
const renderer = window.__stubBoot.renderers[0];
return {
id: renderer.id,
capabilities: [...renderer.capabilities],
canvasInPage: document.contains(renderer.canvas),
};
});
report.startupErrors = errors.slice();
const known = await page.evaluate(() =>
window.__godsEyeView.dataManager.getAll().map((layer) => layer.id),
);
report.missingLayers = STUB_BOOT_LAYERS.filter((id) => !known.includes(id));
for (const id of STUB_BOOT_LAYERS) {
if (!known.includes(id)) continue;
const errorsBefore = errors.length;
const before = await page.evaluate(readDrawn);
const enabled = await page.evaluate(
(layerId) =>
window.__godsEyeView.dataManager
.setEnabled(layerId, true, { origin: 'user' })
.then(
(value) => ({ ok: true, value: value ?? null }),
(error) => ({ ok: false, error: String(error?.stack || error) }),
),
id,
);
// Feeds load on their own schedule; keep ticking until they draw.
const deadline = Date.now() + args.settleMs;
let drawn = { sets: {}, imagery: 0, calls: {}, fields: 0 };
while (Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, 1_000));
const now = await page.evaluate(readDrawn);
drawn = {
sets: Object.fromEntries(
Object.entries(now.sets).filter(
([setId, items]) => items > 0 && before.sets[setId] !== items,
),
),
imagery: Math.max(0, now.imagery - before.imagery),
calls: Object.fromEntries(
Object.entries(now.calls)
.map(([key, n]) => [key, n - (before.calls[key] ?? 0)])
.filter(([, n]) => n > 0),
),
};
drawn.fields = drawn.calls['draw.field'] ?? 0;
if (Object.keys(drawn.sets).length || drawn.imagery || drawn.fields)
break;
}
const stats = await page.evaluate((layerId) => {
const row = window.__godsEyeView.dataManager
.getAll()
.find((layer) => layer.id === layerId);
return {
enabled: row?.enabled ?? null,
count: row?.stats?.count ?? null,
error: row?.stats?.error ?? null,
source: row?.stats?.source ?? null,
};
}, id);
const disabled = await page.evaluate(
(layerId) =>
window.__godsEyeView.dataManager
.setEnabled(layerId, false, { origin: 'user' })
.then(
() => ({ ok: true }),
(error) => ({ ok: false, error: String(error?.stack || error) }),
),
id,
);
const layerErrors = errors.slice(errorsBefore);
report.layers[id] = {
enabled,
disabled,
stats,
drawn,
errors: layerErrors.filter(
(error) => error.kind === 'page' || error.kind === 'console',
),
failedResponses: layerErrors
.filter((error) => error.kind === 'response')
.map((error) => error.at),
};
const what = [
...Object.entries(drawn.sets).map(([setId, n]) => `${setId}=${n}`),
...(drawn.imagery ? [`${drawn.imagery} imagery layer(s)`] : []),
...(drawn.fields ? [`${drawn.fields} field(s)`] : []),
];
console.log(
`${id}: ${enabled.ok && disabled.ok ? 'ran' : 'FAILED'}; drew ${
what.length ? what.join(', ') : 'nothing'
}${stats.error ? `; feed: ${stats.error}` : ''}${
report.layers[id].failedResponses.length
? `; ${report.layers[id].failedResponses.length} failed response(s)`
: ''
}`,
);
}
report.ok = true;
} finally {
report.cesiumRequests = cesiumRequests;
report.cesiumStylesheets = cesiumStylesheets;
report.errors = errors;
await browser.close();
await server?.close();
}
const pageErrors = errors.filter((error) => error.kind === 'page');
const consoleErrors = errors.filter((error) => error.kind === 'console');
const failedResponses = errors.filter((error) => error.kind === 'response');
const toolingErrors = errors.filter((error) => error.kind === 'tooling');
const problems = [];
if (!report.started?.ok) problems.push('the application did not start');
if (!report.renderer?.canvasInPage)
problems.push("the stub's canvas is not in the page");
if (cesiumRequests.length)
problems.push(`Cesium was requested: ${cesiumRequests.join(', ')}`);
if (pageErrors.length) problems.push(`${pageErrors.length} page error(s)`);
if (consoleErrors.length)
problems.push(`${consoleErrors.length} console error(s)`);
if (report.missingLayers?.length)
problems.push(`layers not in the catalog: ${report.missingLayers}`);
for (const [id, layer] of Object.entries(report.layers)) {
if (!layer.enabled.ok || !layer.disabled.ok)
problems.push(`${id} failed to turn on or off`);
if (
args.strict &&
!Object.keys(layer.drawn.sets).length &&
!layer.drawn.imagery &&
!layer.drawn.fields
)
problems.push(`${id} drew nothing`);
}
report.problems = problems;
if (args.json) writeFileSync(args.json, JSON.stringify(report, null, 2));
for (const error of [...pageErrors, ...consoleErrors])
console.error(`${error.kind} error: ${error.text} (${error.at ?? '?'})`);
if (cesiumStylesheets.length)
console.log(
`note: the build's Cesium plugin linked ${cesiumStylesheets.join(', ')}`,
);
if (toolingErrors.length)
console.log(
`note: the dev server's client logged ${toolingErrors.length} error(s): ${[
...new Set(toolingErrors.map((error) => error.text)),
].join(' | ')}`,
);
if (failedResponses.length)
console.log(
`note: ${failedResponses.length} feed response(s) failed (keys or network)`,
);
if (problems.length) {
console.error(`FAIL: ${problems.join('; ')}`);
process.exitCode = 1;
} else console.log('ok: the application runs on the stub renderer');
}
if (import.meta.url === `file://${process.argv[1]}`)
main().catch((error) => {
console.error(error);
process.exitCode = 1;
});