fix(qa): align renderer evidence and attribution keyboard assertions

This commit is contained in:
Sameh Khamis committed 2026-09-11 14:50:57 -07:00
1 parent 01dcecf6e9
commit 67504cd871
5 files changed
+40 -14

No files matched your search

+11
View File
@@ -13,6 +13,17 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
## [Unreleased] ## [Unreleased]
### Fixed
- Complete the first-run, view-target prewarm, cockpit-plates and floor-hold
browser harness renderer portability fixes contributed by Tom-Neverwinter.
macOS retains Metal; other platforms default to SwiftShader. Cockpit renderer
assertions and evidence labels follow the actual selected mode. Floor-hold
keeps its mesh and terrain assertions; software runs are not real-GPU evidence.
First-run QA now checks the existing attribution Escape-close/focus-return
behavior while preserving the launcher-underneath regression checks.
- Datacenter and dam factories are available through scoped package exports with - Datacenter and dam factories are available through scoped package exports with
explicit context, overlay and render callbacks. The standalone app uses the explicit context, overlay and render callbacks. The standalone app uses the
same implementation and bundled datasets. same implementation and bundled datasets.
+11
View File
@@ -218,3 +218,14 @@ While recording, call out anything in these areas — this is the feedback I mos
- **No planes:** OpenSky data may be momentarily sparse; scroll out or wait a poll cycle. - **No planes:** OpenSky data may be momentarily sparse; scroll out or wait a poll cycle.
- **No GEV MIC button / voice errors:** `OPENAI_API_KEY` didn't load — use the console API for - **No GEV MIC button / voice errors:** `OPENAI_API_KEY` didn't load — use the console API for
the annotation tests and skip the voice-only ones (§2). the annotation tests and skip the voice-only ones (§2).
## Browser harness renderers
The first-run, view-target prewarm, cockpit-plates and floor-hold harnesses
select Metal on macOS and SwiftShader on other platforms. Cockpit-plates also
accepts `--swiftshader` on macOS; floor-hold retains `--angle=<backend>`.
Software runs validate their assertions but do not establish real-GPU visual
correctness. Floor-hold retains both mesh and DEM checks: an unavailable mesh
oracle fails the run even when the DEM check passes. Record the backend with
any screenshots and run GPU visual checks separately when needed.
+4 -4
View File
@@ -46,7 +46,7 @@
* node scripts/qa-cockpit-plates.mjs --url http://localhost:4268 --tag before * node scripts/qa-cockpit-plates.mjs --url http://localhost:4268 --tag before
* node scripts/qa-cockpit-plates.mjs --url http://localhost:4268 --teeth * node scripts/qa-cockpit-plates.mjs --url http://localhost:4268 --teeth
* *
* Defaults to the real GPU (ANGLE/Metal). `--swiftshader` selects software GL, * Defaults to ANGLE/Metal on macOS and software GL elsewhere. `--swiftshader` selects software GL,
* which is deterministic but is NOT real-GPU evidence; the banner says which * which is deterministic but is NOT real-GPU evidence; the banner says which
* backend actually ran and the run records it alongside the results. * backend actually ran and the run records it alongside the results.
*/ */
@@ -64,7 +64,7 @@ const APP_ORIGIN = new URL(APP_URL).origin;
const TAG = getOpt('--tag', 'after'); const TAG = getOpt('--tag', 'after');
const HEADFUL = argv.includes('--headful'); const HEADFUL = argv.includes('--headful');
const TEETH = argv.includes('--teeth'); const TEETH = argv.includes('--teeth');
const SWIFTSHADER = argv.includes('--swiftshader'); const SWIFTSHADER = argv.includes('--swiftshader') || process.platform !== 'darwin';
const SHOT_DIR = path.resolve('qa-shots/cockpitplates'); const SHOT_DIR = path.resolve('qa-shots/cockpitplates');
/** Mirrors `SKY_PLATE_SCALE` in src/overlays/worldOverlayTokens.js. */ /** Mirrors `SKY_PLATE_SCALE` in src/overlays/worldOverlayTokens.js. */
@@ -224,7 +224,7 @@ async function main() {
'--disable-setuid-sandbox', '--disable-setuid-sandbox',
// Metal is a macOS-only ANGLE backend; off-Mac the GPU branch has to fall // Metal is a macOS-only ANGLE backend; off-Mac the GPU branch has to fall
// back to the software rasterizer or WebGL init fails outright. // back to the software rasterizer or WebGL init fails outright.
...(SWIFTSHADER || process.platform !== 'darwin' ...(SWIFTSHADER
? ['--use-gl=angle', '--use-angle=swiftshader'] ? ['--use-gl=angle', '--use-angle=swiftshader']
: ['--use-angle=metal', '--enable-gpu', '--ignore-gpu-blocklist']), : ['--use-angle=metal', '--enable-gpu', '--ignore-gpu-blocklist']),
'--disable-dev-shm-usage', '--disable-dev-shm-usage',
@@ -294,7 +294,7 @@ async function main() {
console.log(` GL : ${renderer}\n`); console.log(` GL : ${renderer}\n`);
record( record(
SWIFTSHADER SWIFTSHADER
? 'running on software GL as requested (structural checks, not GPU evidence)' ? 'running on software GL (structural checks, not GPU evidence)'
: 'running on the real GPU, so the screenshots are real-GPU evidence', : 'running on the real GPU, so the screenshots are real-GPU evidence',
SWIFTSHADER ? softwareRenderer : !softwareRenderer, SWIFTSHADER ? softwareRenderer : !softwareRenderer,
renderer, renderer,
+8 -5
View File
@@ -332,14 +332,17 @@ async function runArbitrationSection(page, { shots, consoleErrors }) {
`session=${escUnderLightbox.session}`, `session=${escUnderLightbox.session}`,
); );
record( record(
'the lightbox keeps its OWN ESC semantics (Cesium binds none, so it stays)', 'ESC closes attribution without dismissing the launcher underneath',
lightboxAfterEsc.shown, !lightboxAfterEsc.shown,
`overlay shown=${lightboxAfterEsc.shown}`, `overlay shown=${lightboxAfterEsc.shown}`,
); );
// The guard disarms the launcher; it must never break it. Close the overlay // The attribution keyboard handler closes the overlay and restores focus.
// the way a visitor does and the key comes straight back. // A second Escape belongs to the now-uncovered launcher.
await page.evaluate(() => document.querySelector('.cesium-credit-lightbox-close')?.click()); const attributionFocusRestored = await page.evaluate(() => (
document.activeElement === document.querySelector('#cesium-credits .cesium-credit-expand-link')
));
record('closing attribution restores its disclosure focus', attributionFocusRestored);
await sleep(300); await sleep(300);
const uncovered = await launcherState(); const uncovered = await launcherState();
record('closing the lightbox hands the card back the key', record('closing the lightbox hands the card back the key',
+6 -5
View File
@@ -15,7 +15,7 @@
* The contact must stay ON the mesh through phase B. Without the hold it falls * The contact must stay ON the mesh through phase B. Without the hold it falls
* to the geoid, which at an inland field is ~150 m of burial. * to the geoid, which at an inland field is ~150 m of burial.
* *
* node scripts/qa-floor-hold.mjs # real GPU (ANGLE/Metal) * node scripts/qa-floor-hold.mjs # Metal on macOS, SwiftShader elsewhere
* node scripts/qa-floor-hold.mjs --headful # watch it * node scripts/qa-floor-hold.mjs --headful # watch it
* QA_BASE_URL=http://localhost:4257 node scripts/qa-floor-hold.mjs * QA_BASE_URL=http://localhost:4257 node scripts/qa-floor-hold.mjs
* *
@@ -25,10 +25,11 @@
* under Metal — a backend comparison at one point, not to be confused with the * under Metal — a backend comparison at one point, not to be confused with the
* same-backend run-to-run spread (122.1 m vs 20.6 m on a cell centre) that * same-backend run-to-run spread (122.1 m vs 20.6 m on a cell centre) that
* retired the hold chain's unvalidated tier. The * retired the hold chain's unvalidated tier. The
* default is the real GPU. Under `--angle=swiftshader` the rendered-mesh oracle * default is Metal on macOS and SwiftShader elsewhere; `--angle` overrides it.
* is reported as unavailable rather than trusted, and the run falls back to the * Mesh samples describe the selected backend, not another GPU's rendered LOD.
* bare-earth DEM (fetched server-side, so the simulated outage cannot reach * An unavailable rendered-mesh oracle remains a failed check, even if the
* it) — a weaker but still decisive check, since the geoid sits ~150 m below it. * separate bare-earth DEM check passes. Software evidence does not establish
* real-GPU visual correctness.
*/ */
import puppeteer from 'puppeteer'; import puppeteer from 'puppeteer';
import fs from 'node:fs'; import fs from 'node:fs';