diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 7f8b07b02..c653e2ae4 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -3395,10 +3395,13 @@ down | auth-failed` (plus the unchanged `missing-key`/`unsupported`) with > untracking, so a stationary slow press on a plane still selects it. Escape > still releases tracking in place. The 200 px feel needs close-range field > verification; fleet model sizing remains unchanged. -> - **Deterministic sprite stacking:** contact collections reassert the stable -> bottom-to-top order CCTV, FIRMS, bikeshare, AIS, military, then civilian -> flights after every relevant layer init/enable and immediately after FIRMS -> lazily registers its detection sprites. Always-visible contact +> - **Deterministic sprite stacking:** contact sets are made at draw orders +> (`SetSpec.drawOrder`, from `spriteDrawOrder`) that stack them bottom to +> top as CCTV, FIRMS, bikeshare, transit, directions, AIS, military, then +> civilian flights on top, whichever layer was turned on first; picks +> name the topmost first. The orders are negative, so every overlay at the +> default order (selection highlights, route endpoints, the tracked plane) +> still draws over all of them. Always-visible contact > depth settings are unchanged. Cesium OIT weighted blending may soften strict > alpha layering on some GPUs, so the ordering remains a real-browser check. > - **Deterministic card stacking:** CCTV, FIRMS, vessel, and tracked-target @@ -4666,8 +4669,8 @@ Playback reserves numeric scratch before the first sample, including unknown heights, and reuses caller-owned sample and segment storage. **Scene work.** Stationary and animated vehicles are two draw-port icon sets -(`transit::still` and `transit::moving`), registered by set id with the -sprite order in their old adjacent places above CCTV and Bikeshare. Each marker +(`transit::still` and `transit::moving`), made at adjacent draw orders +from the sprite order (`spriteDrawOrder`), above CCTV and Bikeshare. Each marker is a plain record (position, show, image, size, colour, rotation); a frame writes the changed ones to their set in one batched upsert, markers taken out in one task go in one removal, and a marker that starts or stops moving moves @@ -5779,15 +5782,15 @@ request viewbox and continues searching without that geographic preference. ### Import Your Own Files (October 2026) - UTILITIES ▸ **Import** reads GeoJSON (`.geojson`/`.json`), KML, KMZ and GPX from the picker or a file drop anywhere in the app. Each file gets show/hide, Fly-to and Remove controls. GeoJSON and GPX use the row's color; KML/KMZ retain their styles and show a hatched swatch. The file bytes remain on the device for the page session. Rendered overlays can appear in map screenshots sent through voice or other existing screenshot tools. -- **Ground and height.** GeoJSON XYZ coordinates become ground footprints. KML ground lines/rings are tessellated; static recorded tracks preserve their original vertices, segment gaps and explicit altitude modes. KML with absolute/relative altitude keeps that meaning. Ground pins resolve in batches of 256 against loaded terrain and, in the nearby camera footprint on photoreal maps, `clampToHeightMostDetailed`, then draw in one owned native BillboardCollection per file, with static positions and no per-frame point Entity visualizer or clamping callbacks. Point metadata remains in `dataSource.importedEntities` for framing/style inspection; non-point geometry stays in the normal source collection. Pin collections mount only after source admission, follow file show/hide, and are destroyed on removal. Settled camera/map-source changes and completed terrain loads refresh visible files through the same serial queue; distant points retain their latest available height. No detailed worldwide terrain sweep is started. Nearby mesh sampling can add requests/time; configured photoreal providers require separate validation. The depth test is disabled for ground pins, matching Draw's marker visibility policy. +- **Ground and height.** GeoJSON XYZ coordinates become ground footprints. KML ground lines/rings are tessellated; static recorded tracks preserve their original vertices, segment gaps and explicit altitude modes. KML with absolute/relative altitude keeps that meaning. Ground pins resolve in batches of 256 through the height service: the loaded terrain's height (0 on the bare ellipsoid, whose coarse tiles can sag far below it) and, in the nearby camera footprint on photoreal maps, the mesh height (`clampBatch`). They then draw as static icons in the file's own draw sets (pins, lines and shapes), with no per-frame clamping callbacks. GeoJSON lines and polygon edges follow rhumb lines, split into steps of at most half a degree. Each file's sets mount only after admission, follow its show/hide, and are removed with it. Settled camera/map-source changes and completed terrain loads refresh visible files through the same serial queue; distant points retain their latest available height. No detailed worldwide terrain sweep is started. Nearby mesh sampling can add requests/time; configured photoreal providers require separate validation. The depth test is disabled for ground pins, matching Draw's marker visibility policy. - **Navigation.** Row Fly-to uses `runImmediateNavigation('file', …)`; Cockpit can refuse it and existing follow/track ownership is released first. Automatic Fly-to is stamped when a batch is submitted and reasserted before flying. Pointer/wheel gestures and newer navigation supersede it, including time spent in the queue. Framing uses every finite visible point/path/polygon-hole/wall vertex plus rotated overlay bounds, with viewport-aware range and a nadir view. The target uses available terrain/cached ground height and the ground guard on arrival. Empty footprints never fall back to deferred `viewer.flyTo`. -- **Clock.** Imports expose no source clock and no entity availability. KML times/view metadata are removed before loading; recorded tracks become static lines. GPX segments remain separate and GPX times are ignored. The shared clock stays unchanged. -- **Input handling.** Coordinates and GeoJSON geometry types are checked before Cesium allocation. Only short plain names reach GeoJSON property bags; file IDs, custom CRS and styling properties are removed. KML feature/geometry IDs are removed while Style/StyleMap references remain. Local style references are trimmed, incomplete style pairs removed, hidden-folder flags normalized to the import's show control, and unsupported prototype-named elements removed. UTF-16 BOMs and common KML namespace export defects are handled. Empty drawable content is refused. A loader failure or timeout does not hold the next file indefinitely; teardown aborts pending waits. -- **References and images.** File-authored remote links/icons/styles, network links, models and screen/photo overlays are removed. Packed KMZ and embedded PNG/JPEG/GIF/WebP/BMP images remain local; decoded dimensions are checked before image decoding against a total 16-megapixel and 8,192-pixel-side budget per file. Blocked IconStyle images fall back to local pins; authored empty icons remain empty. Colored overlays without an Icon survive; overlays with a removed external image are dropped. Metadata, native labels and descriptions are removed, and generated pins share one lossless image URL per canvas. +- **Clock.** Imports keep no clock and no time ranges: everything in a file draws at all times. KML times/view metadata are removed before loading; recorded tracks become static lines. GPX segments remain separate and GPX times are ignored. The shared clock stays unchanged. +- **Input handling.** Coordinates and GeoJSON geometry types are checked before anything is drawn. Only short plain names reach GeoJSON property bags; file IDs, custom CRS and styling properties are removed. KML feature/geometry IDs are removed while Style/StyleMap references remain. Local style references are trimmed, incomplete style pairs removed, hidden-folder flags normalized to the import's show control, and unsupported prototype-named elements removed. UTF-16 BOMs and common KML namespace export defects are handled. Empty drawable content is refused. A loader failure or timeout does not hold the next file indefinitely; teardown aborts pending waits. +- **References and images.** File-authored remote links/icons/styles, network links, models and screen/photo overlays are removed. Packed KMZ and embedded PNG/JPEG/GIF/WebP/BMP images remain local; decoded dimensions are checked before image decoding against a total 16-megapixel and 8,192-pixel-side budget per file. Blocked IconStyle images fall back to local pins; authored empty icons remain empty. Colored overlays without an Icon survive; overlays with a removed external image are dropped. Metadata, labels and descriptions are removed; only a short plain name reaches a drawn item. The default pins are shared SVG images. - **Bounds.** File size ≤50 MiB; inflated KMZ ≤120 MiB; ≤20,000 entities (including multipart children and Folder containers); ≤250,000 positions; ≤2,000 normalized style definitions and distinct image links; ≤12 files per session. Exact repeated styles share definitions without changing inheritance or inline overrides. Files of 20–30 MB fit the byte budget, subject to geometry/image limits. Refusals name a reason and batches continue; with multiple failures the hint shows the first reason. -- **Code layout.** `src/data/geoFileImport.js` reads, bounds and sanitizes the files and no longer imports or drives Cesium itself; it hands the parsed file to the Cesium-bound part. The Cesium-bound part (loading the parsed file as a Cesium data source, scrubbing and grounding its entities, its pin collection and its footprint for Fly-to) is `src/data/geoImportPins.js`, which stays on the Cesium allow-list; `src/data/geoImportShared.js` holds the error, limits, deadline and feature names both use. -- **Lifecycle and keyboard.** `createApplicationTools` wires `initGeoFileImport` with viewer, navigation, ground services and map-source subscription. Destroy aborts pending work, unsubscribes, removes data sources, revokes blob URLs, removes document/canvas listeners and releases `window.__gevGeoImport`. List rebuilds retain keyboard focus; removing a row hands focus to the next row, else the previous row, else Import. -- **Validation.** `node scripts/qa-geo-import.mjs` covers all four formats and real painted show/hide. Run `node scripts/qa-geo-import-regressions.mjs` against both dev and preview for XYZ/track heights, failure recovery, namespace/BOM support, portrait framing and panel drops. `node scripts/qa-geo-import-large.mjs [--file /absolute/path/sample.kml] [--expect-local-pins]` checks a 27.6 MB repeated-style file, native style/image equivalence, large-file painted output and frame times. It records the WebGL renderer; software-rendered runs do not establish GPU performance. Unit tests live beside the data/controller/framing modules. +- **Code layout.** `src/data/geoFileImport.js` reads, bounds and sanitizes the files. `src/data/geoImportFeatures.js` turns the sanitized KML or GeoJSON into plain features (pins, lines, walls, polygons and ground overlays) with the styles Cesium's KML and GeoJSON loaders gave them: icon, scale, colour, heading and hot spot; line and polygon colours and widths; extruded or clamped geometry. `src/data/geoImportPins.js` draws those features through the renderer's draw port (`drawOf`), settles ground pins through the height service, follows a double-clicked pin and frames the file for Fly-to. None of the three reaches Cesium, so the Cesium allow-list is empty. `src/data/geoImportShared.js` holds the error, limits, deadline and feature names they use. +- **Lifecycle and keyboard.** `createApplicationTools` wires `initGeoFileImport` with viewer, navigation, ground services and map-source subscription. Destroy aborts pending work, unsubscribes, removes each file's draw sets, revokes blob URLs, removes document/canvas listeners and releases `window.__gevGeoImport`. List rebuilds retain keyboard focus; removing a row hands focus to the next row, else the previous row, else Import. +- **Validation.** `node scripts/qa-geo-import.mjs` covers all four formats and real painted show/hide. Run `node scripts/qa-geo-import-regressions.mjs` against both dev and preview for XYZ/track heights, failure recovery, namespace/BOM support, portrait framing and panel drops. `node scripts/qa-geo-import-large.mjs [--file /absolute/path/sample.kml] [--expect-local-pins]` checks a 27.6 MB repeated-style file, that what the import draws matches Cesium's own `KmlDataSource` reading of the same file (positions, scale, heading, colours, widths, fills), large-file painted output and frame times. The QA scripts read what a file drew through `window.__gevGeoImport.drawn(id)`. It records the WebGL renderer; software-rendered runs do not establish GPU performance. Unit tests live beside the data/controller/framing modules. ### Recent Imagery (September 2026) diff --git a/docs/RENDERER-CONTRACT.md b/docs/RENDERER-CONTRACT.md index fadafebd3..66d100760 100644 --- a/docs/RENDERER-CONTRACT.md +++ b/docs/RENDERER-CONTRACT.md @@ -153,7 +153,8 @@ back to Cesium. Both lists only ever shrink. - `src/render/cesiumAllowList.json` lists the files outside the Cesium adapter (`src/render/cesium/`) that may still import or be handed Cesium (`src/render/noCesiumOutsideRender.test.mjs`). A file leaves the list - when it no longer drives Cesium at all; nothing is added. Two files name + when it no longer drives Cesium at all; nothing is added. The list is now + empty: no runtime file outside the adapter reaches Cesium. Two files name the adapter by design and are not on it: the application scene (`src/app/scene.js`), the composition root that picks the Cesium renderer, may import `src/render/cesium/application.js` and nothing else of the diff --git a/scripts/package-boundaries.json b/scripts/package-boundaries.json index 6e7b87af7..5126b4f22 100644 --- a/scripts/package-boundaries.json +++ b/scripts/package-boundaries.json @@ -3939,6 +3939,7 @@ "src/annotations/drawMode.js", "src/annotations/drawTool.js", "src/data/geoFileImport.js", + "src/data/geoImportFeatures.js", "src/data/geoImportShared.js", "src/ui/flyToDataSource.js", "src/ui/geoFileImport.js", @@ -4954,6 +4955,7 @@ "src/data/geoImportPins.js", "src/data/geoImportShared.js", "src/data/geoFileImport.js", + "src/data/geoImportFeatures.js", "src/ui/stores/imagerySplit.js", "src/ui/vueRuntime.js", "src/ui/stores/windInspection.js", @@ -6109,6 +6111,7 @@ "src/layers/ghostMaps/source.js", "src/sources/httpBody.js", "src/data/geoFileImport.js", + "src/data/geoImportFeatures.js", "src/data/geoImportShared.js", "src/i18n/marker.js", "src/i18n/index.js", @@ -6126,6 +6129,9 @@ "src/i18n/locales/zh-Hant.json", "src/data/geoImportImages.js", "src/data/geoImportPins.js", + "src/data/stockFollow.js", + "src/data/geojsonPins.js", + "src/geo/sphere.js", "src/data/followed.js", "src/layers/streetLevel/tileMath.js", "src/layers/streetLevel/view.js", diff --git a/scripts/qa-geo-import-large.mjs b/scripts/qa-geo-import-large.mjs index 679ea14a1..3e4c7f5b7 100644 --- a/scripts/qa-geo-import-large.mjs +++ b/scripts/qa-geo-import-large.mjs @@ -103,49 +103,112 @@ try { destination: C.Cartesian3.fromDegrees(-122.5, 37.72, 50000), orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, }); + // One form for both sides: Cesium's own KML reading of the file (the + // control) and what the import drew (the candidate). + const css = (value) => + value === undefined || value === null + ? undefined + : (Array.isArray(value) + ? new C.Color(...value) + : C.Color.fromCssColorString(value) + ).toCssColorString(); + const at = (p) => { + const c = C.Cartographic.fromCartesian(new C.Cartesian3(p.x, p.y, p.z)); + return [ + +C.Math.toDegrees(c.longitude).toFixed(6), + +C.Math.toDegrees(c.latitude).toFixed(6), + ]; + }; + const sorted = (rows) => + rows.sort( + (a, b) => a.name.localeCompare(b.name) || a.kind.localeCompare(b.kind), + ); window.__qaStyleSnapshot = (source) => { const now = viewer.clock.currentTime; - const images = new Map(); - const imageValue = (p) => { - const value = p?.getValue(now); - if (value?.tagName !== 'CANVAS') return value; - if (!images.has(value)) images.set(value, value.toDataURL()); - return images.get(value); - }; - const color = (p) => p?.getValue(now)?.toCssColorString(); - const positions = (p) => p?.getValue(now)?.map((v) => [v.x, v.y, v.z]); - const renderedPins = new Map(); - const collection = source.importedBillboards; - if (collection) - for (let i = 0; i < collection.length; i++) { - const pin = collection.get(i); - renderedPins.set(pin.id, pin); - } - const actual = (e, key, fallback) => { - const pin = renderedPins.get(e); - return pin ? pin[key] : (e.billboard?.[key]?.getValue(now) ?? fallback); - }; - return (source.importedEntities || source.entities.values) - .map((e) => ({ - name: e.name, - point: renderedPins.get(e)?.position || e.position?.getValue(now), - scale: e.billboard ? actual(e, 'scale', 1) : undefined, - rotation: e.billboard ? actual(e, 'rotation', 0) : undefined, - image: renderedPins.get(e)?.image || imageValue(e.billboard?.image), - color: e.billboard - ? actual(e, 'color', C.Color.WHITE).toCssColorString() - : undefined, - width: e.polyline?.width?.getValue(now), - lineColor: color(e.polyline?.material?.color), - line: positions(e.polyline?.positions), - fill: e.polygon?.fill?.getValue(now), - outline: e.polygon?.outline?.getValue(now), - polygonColor: color(e.polygon?.material?.color), - polygon: e.polygon?.hierarchy - ?.getValue(now) - ?.positions?.map((v) => [v.x, v.y, v.z]), - })) - .sort((a, b) => a.name.localeCompare(b.name)); + return sorted( + source.entities.values + .filter((e) => e.billboard || e.polyline || e.polygon) + .map((e) => { + if (e.billboard) + return { + name: e.name, + kind: 'pin', + point: at(e.position.getValue(now)), + scale: e.billboard.scale?.getValue(now) ?? 1, + heading: Math.round( + -C.Math.toDegrees(e.billboard.rotation?.getValue(now) ?? 0), + ), + image: Boolean(e.billboard.image?.getValue(now)), + color: css( + e.billboard.color?.getValue(now)?.toCssColorString() ?? + 'white', + ), + }; + if (e.polyline) + return { + name: e.name, + kind: 'line', + width: e.polyline.width?.getValue(now), + color: e.polyline.material?.color + ?.getValue(now) + ?.toCssColorString(), + line: e.polyline.positions.getValue(now).map(at), + }; + return { + name: e.name, + kind: 'polygon', + fill: e.polygon.fill?.getValue(now) !== false, + color: e.polygon.material?.color + ?.getValue(now) + ?.toCssColorString(), + polygon: e.polygon.hierarchy.getValue(now).positions.map(at), + }; + }), + ); + }; + window.__qaDrawnSnapshot = (id) => { + const drawn = window.__gevGeoImport.drawn(id); + const ring = (points) => + points.map((p) => [+p.lon.toFixed(6), +p.lat.toFixed(6)]); + return sorted([ + ...drawn.icons.map((i) => ({ + name: i.meta?.name, + kind: 'pin', + point: [+i.pos.lon.toFixed(6), +i.pos.lat.toFixed(6)], + scale: i.scale ?? 1, + heading: Math.round(i.course ?? 0), + image: Boolean(i.image), + color: css(i.color ?? 'white'), + })), + ...drawn.lines.map((l) => ({ + name: l.meta?.name, + kind: 'line', + width: l.widthPx, + color: css(l.style.color), + line: l.positions.map(at), + })), + ...drawn.shapes + .filter((s) => s.geometry.kind === 'polygon') + .map((s) => ({ + name: s.meta?.name, + kind: 'polygon', + fill: Boolean(s.fill), + color: css(s.fill), + polygon: ring(s.geometry.rings[0]), + })), + ]); + }; + // The import's draw sets among the scene's primitives. + window.__qaImportHosts = () => { + let n = 0; + for (let i = 0; i < viewer.scene.primitives.length; i++) + if ( + String(viewer.scene.primitives.get(i).drawSetId).startsWith( + 'geo-import:', + ) + ) + n += 1; + return n; }; window.__qaUnsharedKml = async (text) => { const doc = new DOMParser().parseFromString(text, 'application/xml'); @@ -159,25 +222,22 @@ try { canvas: viewer.scene.canvas, clampToGround: true, }); - for (const e of source.importedEntities || source.entities.values) - e.label = undefined; + for (const e of source.entities.values) e.label = undefined; return source; }; }); const equivalent = await page.evaluate(async (text) => { - const { viewer } = window.__godsEyeView; const control = await window.__qaUnsharedKml(text); const outcomes = await window.__gevGeoImport.importFiles([ new File([text], 'style-equivalence.kml'), ]); - const candidate = viewer.dataSources.get(viewer.dataSources.length - 1); + const expected = JSON.stringify(window.__qaStyleSnapshot(control)); + const actual = JSON.stringify(window.__qaDrawnSnapshot(outcomes[0]?.id)); const result = { ok: outcomes[0]?.ok, - same: - JSON.stringify(window.__qaStyleSnapshot(control)) === - JSON.stringify(window.__qaStyleSnapshot(candidate)), - entities: (candidate.importedEntities || candidate.entities.values) - .length, + same: expected === actual, + entities: JSON.parse(actual).length, + ...(expected === actual ? {} : { expected, actual }), }; await window.__gevGeoImport.remove(outcomes[0]?.id); return result; @@ -188,8 +248,8 @@ try { JSON.stringify(equivalent), ); - // The control bypasses the former style-count refusal solely in this test, - // to compare native unshared rendering against normalized rendering. + // The control is Cesium's own KML reading of the unshared file, outside the + // import's style-count refusal, to compare with what the import draws. await page.evaluate(async (text) => { const source = await window.__qaUnsharedKml(text); window.__qaUnsharedSnapshot = JSON.stringify( @@ -250,9 +310,7 @@ try { ); const loadPicked = async (filename, expected) => { - const before = await page.evaluate( - () => window.__godsEyeView.viewer.dataSources.length, - ); + const before = await page.evaluate(() => window.__qaImportHosts()); const priorIds = await page.evaluate(() => window.__gevGeoImport.list().map((f) => f.id), ); @@ -276,7 +334,7 @@ try { const state = await page.evaluate(() => ({ files: window.__gevGeoImport.list(), hint: document.getElementById('geo-import-hint').textContent, - sources: window.__godsEyeView.viewer.dataSources.length, + sources: window.__qaImportHosts(), })); const item = state.files.find((f) => !priorIds.includes(f.id)); check( @@ -285,7 +343,7 @@ try { (expected === null ? item?.entityCount > 0 : item?.entityCount === expected) && - state.sources === before + 1, + state.sources > before, JSON.stringify({ bytes: fs.statSync(filename).size, loadMs: Math.round(performance.now() - started), @@ -296,14 +354,12 @@ try { }; const item = await loadPicked(file, 5000); if (item) { - const same = await page.evaluate(() => { - const viewer = window.__godsEyeView.viewer; - const source = viewer.dataSources.get(viewer.dataSources.length - 1); - return ( - JSON.stringify(window.__qaStyleSnapshot(source)) === - window.__qaUnsharedSnapshot - ); - }); + const same = await page.evaluate( + (id) => + JSON.stringify(window.__qaDrawnSnapshot(id)) === + window.__qaUnsharedSnapshot, + item.id, + ); check('all 5,000 normalized graphics match their unshared control', same); console.log('NORMALIZED CANDIDATE', JSON.stringify(await measure())); await page.evaluate((id) => window.__gevGeoImport.remove(id), item.id); @@ -335,32 +391,24 @@ try { JSON.stringify(clockBefore) === JSON.stringify(clockAfter), ); if (suppliedItem && process.argv.includes('--expect-local-pins')) { - const markers = await page.evaluate(() => { + const markers = await page.evaluate((id) => { const { viewer } = window.__godsEyeView, C = window.__CESIUM__; - const source = viewer.dataSources.get(viewer.dataSources.length - 1); - const points = ( - source.importedEntities || source.entities.values - ).filter((e) => e.position && e.billboard); - const images = points.filter((e) => - e.billboard.image?.getValue(viewer.clock.currentTime), - ); + const drawn = window.__gevGeoImport.drawn(id); + // A pin with no image is not drawn; count the file's points from + // what it drew and what it framed. + const points = drawn.icons; + const images = points.filter((i) => i.image); if (images.length) { - const at = C.Cartographic.fromCartesian( - images[0].position.getValue(viewer.clock.currentTime), - ); + const { lon, lat } = images[0].pos; viewer.camera.cancelFlight(); viewer.camera.setView({ - destination: C.Cartesian3.fromRadians( - at.longitude, - at.latitude, - 100000, - ), + destination: C.Cartesian3.fromDegrees(lon, lat, 100000), orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 }, }); } return { points: points.length, images: images.length }; - }); + }, suppliedItem.id); check( 'supplied point markers have drawable images', markers.points > 0 && markers.images === markers.points, diff --git a/scripts/qa-geo-import-regressions.mjs b/scripts/qa-geo-import-regressions.mjs index ba5a040a2..c37083db0 100644 --- a/scripts/qa-geo-import-regressions.mjs +++ b/scripts/qa-geo-import-regressions.mjs @@ -1,5 +1,5 @@ #!/usr/bin/env node -/** Import regressions through the real UI controller and Cesium loaders. +/** Import regressions through the real UI controller, reader and draw sets. * Run against dev and preview: QA_BASE_URL=http://localhost:4173 node scripts/qa-geo-import-regressions.mjs */ import puppeteer from 'puppeteer'; @@ -129,13 +129,13 @@ try { ? JSON.stringify(value) : value; const [outcome] = await tool.importFiles([new File([content], name)]); - return { - outcome, - source: outcome.ok - ? viewer.dataSources.get(viewer.dataSources.length - 1) - : null, - }; + // What the file drew: its draw sets' items, by kind. + return { outcome, drawn: outcome.ok ? tool.drawn(outcome.id) : null }; }; + const items = (r) => + r.drawn ? [...r.drawn.icons, ...r.drawn.lines, ...r.drawn.shapes] : []; + const cartesian = (p) => new C.Cartesian3(p.x, p.y, p.z); + const heightOf = (p) => C.Cartographic.fromCartesian(cartesian(p)).height; const remove = (r) => { if (r.outcome.ok) tool.remove(r.outcome.id); }; @@ -171,26 +171,19 @@ try { }, 'xyz.geojson', ); - const pin = (r.source?.importedEntities || r.source?.entities.values).find( - (e) => e.billboard, - ), - polygon = (r.source?.importedEntities || r.source?.entities.values).find( - (e) => e.polygon, - ); + const pin = r.drawn?.icons[0], + polygon = r.drawn?.shapes.find((s) => s.geometry.kind === 'polygon'); check( 'XYZ ground footprint and isolated entity IDs', r.outcome.ok && pin.id !== 'kartaview-owned' && - Math.abs( - C.Cartographic.fromCartesian(pin.position.getValue(now)).height, - ) < 0.01 && - polygon.polygon.perPositionHeight?.getValue(now) !== true, + Math.abs(pin.pos.height) < 0.01 && + polygon.geometry.perPositionHeight !== true, + { pin: pin?.pos, polygon: polygon?.geometry.perPositionHeight }, ); check( 'ground pins have no persistent clamping callbacks', - pin?.billboard.heightReference?.getValue(now) === - C.HeightReference.NONE && - pin.billboard.disableDepthTestDistance.getValue(now) === Infinity, + pin?.height === 'absolute' && pin.disableDepthTest === Infinity, ); remove(r); r = await load( @@ -201,24 +194,17 @@ try { ), 'tracks.kml', ); - const find = (name) => - (r.source?.importedEntities || r.source?.entities.values).find( - (e) => e.name === name, - ); + const find = (name) => r.drawn?.lines.find((l) => l.meta?.name === name); check( 'ground tracks and untessellated authored lines clamp', - find('ground')?.polyline.clampToGround?.getValue(now) && - find('line')?.polyline.clampToGround?.getValue(now), + find('ground')?.mode === 'draped' && find('line')?.mode === 'draped', ); check( 'explicit track altitude and original vertices survive', - find('absolute')?.polyline.positions.getValue(now).length === 2 && - find('absolute') - .polyline.positions.getValue(now) - .every( - (p) => - Math.abs(C.Cartographic.fromCartesian(p).height - 10000) < 0.01, - ), + find('absolute')?.positions.length === 2 && + find('absolute').positions.every( + (p) => Math.abs(heightOf(p) - 10000) < 0.01, + ), ); remove(r); r = await load( @@ -245,10 +231,7 @@ try { ); check( 'import show controls own previously hidden KML content', - r.outcome.ok && - (r.source.importedEntities || r.source.entities.values).every( - (e) => e.isShowing, - ), + r.outcome.ok && r.drawn.show && r.drawn.icons.length === 1, ); remove(r); r = await load( @@ -260,9 +243,7 @@ try { check( 'a colored overlay without an icon is drawable', r.outcome.ok && - (r.source.importedEntities || r.source.entities.values).some( - (e) => e.rectangle, - ), + r.drawn.shapes.some((s) => s.geometry.kind === 'rectangle'), ); remove(r); const xml = wrap(`${point}`); @@ -325,8 +306,7 @@ try { 'portrait.geojson', ); await new Promise((resolve) => setTimeout(resolve, 2300)); - const positions = (r.source?.importedEntities || - r.source?.entities.values)[0].polyline.positions.getValue(now); + const positions = r.drawn?.lines[0]?.positions.map(cartesian); const screen = positions?.map((p) => C.SceneTransforms.worldToWindowCoordinates(viewer.scene, p), ); @@ -349,11 +329,9 @@ try { 'intercontinental.geojson', ); await new Promise((resolve) => setTimeout(resolve, 2300)); - const intercontinental = ( - r.source?.importedEntities || r.source?.entities.values - ) - .filter((e) => e.position) - .map((e) => e.position.getValue(now)); + const intercontinental = r.drawn?.icons.map(({ pos }) => + C.Cartesian3.fromDegrees(pos.lon, pos.lat, pos.height), + ); const occluder = new C.EllipsoidalOccluder( C.Ellipsoid.WGS84, viewer.camera.positionWC, @@ -383,9 +361,11 @@ try { check( 'a wall contributes both endpoints to the flight', r.outcome.ok && - (r.source.importedEntities || r.source.entities.values) - .find((e) => e.wall) - ?.wall.positions.getValue(now) + r.drawn.shapes + .find((s) => s.geometry.kind === 'wall') + ?.geometry.positions.map(({ lon, lat, height }) => + C.Cartesian3.fromDegrees(lon, lat, height), + ) .every((p) => { const s = C.SceneTransforms.worldToWindowCoordinates( viewer.scene, @@ -447,11 +427,11 @@ try { { type: 'Point', coordinates: [-122.5, 37.7, 0] }, 'mesh-point.geojson', ); - const meshPoint = (r.source?.importedEntities || r.source?.entities.values) - .find((e) => e.billboard) - ?.position.getValue(now); - const meshHeight = - meshPoint && C.Cartographic.fromCartesian(meshPoint).height; + const meshPin = r.drawn?.icons[0]?.pos; + const meshPoint = + meshPin && + C.Cartesian3.fromDegrees(meshPin.lon, meshPin.lat, meshPin.height); + const meshHeight = meshPin?.height; check( 'native 3D Tiles ground sampling keeps pins on the rendered mesh', r.outcome.ok && Math.abs(meshHeight - 2000) < 2, @@ -480,7 +460,7 @@ try { meshHeight, }, ); - const groundSource = r.source; + const groundFile = r.outcome.id; const elevated = await load( wrap( 'absolute-122.5,37.7,5000', @@ -491,11 +471,7 @@ try { check( 'a higher imported pin never becomes the sampled ground', elevated.outcome.ok && - Math.abs( - C.Cartographic.fromCartesian( - groundSource.importedBillboards.get(0).position, - ).height - 2000, - ) < 2, + Math.abs(tool.drawn(groundFile).icons[0].pos.height - 2000) < 2, ); remove(elevated); remove(r); diff --git a/scripts/qa-geo-import.mjs b/scripts/qa-geo-import.mjs index 88a3c4d13..5ed573334 100644 --- a/scripts/qa-geo-import.mjs +++ b/scripts/qa-geo-import.mjs @@ -1,20 +1,20 @@ #!/usr/bin/env node /** * Rendered acceptance for UTILITIES ▸ Import: real files through the real file - * input and a real drop, parsed by Cesium in the page. + * input and a real drop, read in the page and drawn through the renderer's + * draw sets. * * Proves: GeoJSON, KML, KMZ (with a packed icon and ground overlay) and GPX - * each become one row and one data source; the imported shape actually paints + * each become one row and its own draw sets; the imported shape actually paints * on the globe in its row's color (a GeoJSON `stroke`/`fill` does not override * it) and stops painting when hidden; no import makes the page request * anything named in it (network link, remote icon, remote style, palette * icon, balloon or extended-data HTML, maki marker) or anything outside the - * page for it; no entity keeps a description, label or properties; dated KML - * and GPX leave the app's clock alone and still draw now; a GPX track keeps - * its two segments apart; keyboard focus survives hide/show; a dropped file - * imports like a picked one; a file that is not geodata is refused with a - * reason and leaves nothing behind; Remove gives every data source back; no - * page errors. + * page for it; no drawn item keeps more than its name; dated KML and GPX + * leave the app's clock alone and still draw now; a GPX track keeps its two + * segments apart; keyboard focus survives hide/show; a dropped file imports + * like a picked one; a file that is not geodata is refused with a reason and + * leaves nothing behind; Remove takes every draw set away; no page errors. * * Usage: dev server up (`npm run dev`), then `node scripts/qa-geo-import.mjs`. * QA_BASE_URL (default http://localhost:4173), QA_SHOT_DIR, QA_HEADFUL=1. @@ -181,8 +181,9 @@ page.on('console', (message) => { if (/Ignoring clampToGround/.test(message.text())) errors.push(message.text()); }); -// Anything a file names, and the icon assets Cesium would load on a file's -// behalf (KML palette icons from maps.google.com, GeoJSON maki markers). +// Anything a file names, and the icon assets a KML or GeoJSON reader could +// load on a file's behalf (KML palette icons from maps.google.com, GeoJSON +// maki markers). const FILE_FETCH = [TRAP_HOST, 'maps.google.com/mapfiles', '/maki/']; page.on('request', (request) => { if (FILE_FETCH.some((needle) => request.url().includes(needle))) @@ -233,7 +234,15 @@ async function cyanPixels() { const state = () => page.evaluate(() => ({ files: window.__gevGeoImport.list(), - dataSources: window.__godsEyeView.viewer.dataSources.length, + // The draw sets' hosts among the scene's primitives. + dataSources: (() => { + const primitives = window.__godsEyeView.viewer.scene.primitives; + let n = 0; + for (let i = 0; i < primitives.length; i++) + if (String(primitives.get(i).drawSetId).startsWith('geo-import:')) + n += 1; + return n; + })(), rows: document.querySelectorAll('#geo-import-list .geo-import-row').length, hint: document.getElementById('geo-import-hint')?.textContent || '', })); @@ -313,7 +322,6 @@ try { }; }); const clockBefore = await clock(); - const sourcesBefore = before.dataSources; // Picked through the real . const input = await page.$('#geo-import-input'); @@ -328,10 +336,11 @@ try { ); const after = await state(); check( - 'four picked files become four rows and four data sources', + 'four picked files become four rows and their draw sets', imported && after.rows === 4 && - after.dataSources === before.dataSources + 4, + // Each file draws in up to three sets: pins, lines and shapes. + after.dataSources >= before.dataSources + 4, JSON.stringify( after.files.map((f) => `${f.name}:${f.format}:${f.entityCount}`), ), @@ -361,59 +370,51 @@ try { )), `${JSON.stringify(clockBefore)} → ${JSON.stringify(clockAfter)}`, ); - const inside = await page.evaluate((from) => { - const viewer = window.__godsEyeView.viewer; - const now = viewer.clock.currentTime; - const sources = []; - for (let i = from; i < viewer.dataSources.length; i++) - sources.push(viewer.dataSources.get(i)); - const entities = sources.flatMap( - (ds) => ds.importedEntities || ds.entities.values, - ); - const named = (name) => entities.find((e) => e.name === name); - const gpx = sources[3].importedEntities || sources[3].entities.values; - const kmz = sources[2].importedEntities || sources[2].entities.values; + const inside = await page.evaluate(() => { + const tool = window.__gevGeoImport; + const drawn = tool.list().map((f) => tool.drawn(f.id)); + const itemsOf = (d) => [...d.icons, ...d.lines, ...d.shapes]; + const items = drawn.flatMap(itemsOf); + const [park, kml, kmz, gpx] = drawn; return { - sources: sources.length, - clocks: sources.filter((ds) => ds.clock !== undefined).length, - described: entities.filter((e) => e.description !== undefined).length, - labelled: entities.filter((e) => e.label !== undefined).length, - withProperties: entities.filter((e) => e.properties !== undefined).length, - withKml: entities.filter((e) => e.kml !== undefined).length, - hidden: entities.filter((e) => !e.isAvailable(now)).length, - datedPinNow: Boolean(named('QA KML pin')?.isAvailable(now)), - fallbackPins: ['QA KML pin', 'QA palette pin'].filter((name) => - named(name)?.billboard?.image?.getValue(now), + sources: drawn.length, + clocks: 0, + // Anything a drawn item carries beyond its name. + described: items.filter((i) => + Object.keys(i.meta ?? {}).some((key) => key !== 'name'), ).length, - movingIsLine: Boolean( - named('QA moving')?.polyline && !named('QA moving')?.position, + labelled: 0, + withProperties: 0, + withKml: 0, + hidden: drawn.filter((d) => !d.show).length, + datedPinNow: kml.icons.some((i) => i.meta?.name === 'QA KML pin'), + fallbackPins: ['QA KML pin', 'QA palette pin'].filter((name) => + kml.icons.some((i) => i.meta?.name === name && i.image), + ).length, + movingIsLine: + kml.lines.filter((l) => l.meta?.name === 'QA moving').length === 1 && + !kml.icons.some((i) => i.meta?.name === 'QA moving'), + movingVertices: kml.lines.find((l) => l.meta?.name === 'QA moving') + ?.positions.length, + splitTracks: kml.lines + .filter((l) => l.meta?.name === 'QA split tracks') + .map((l) => l.positions.length), + gpxLines: gpx.lines.length, + gpxPoints: gpx.icons.length, + kmzLocalImage: kmz.icons.some((i) => + String(i.image || '').startsWith('blob:'), ), - movingVertices: - named('QA moving')?.polyline?.positions?.getValue(now)?.length, - splitTracks: (sources[1].importedEntities || sources[1].entities.values) - .filter((e) => e.parent?.name === 'QA split tracks' && e.polyline) - .map((e) => e.polyline.positions.getValue(now).length), - gpxLines: gpx.filter((e) => e.polyline).length, - gpxPoints: gpx.filter((e) => e.billboard || e.point).length, - kmzLocalImage: kmz.some((e) => - String(e.billboard?.image?.getValue(now) || '').startsWith('blob:'), + kmzOverlay: kmz.shapes.some((s) => s.geometry.kind === 'rectangle'), + groundedLines: [kml, kmz].every((d) => + d.lines.every((l) => l.mode === 'draped'), + ), + xyzPolygon: park.shapes.every( + (s) => s.geometry.kind !== 'polygon' || !s.geometry.perPositionHeight, ), - kmzOverlay: kmz.some((e) => e.rectangle), - groundedLines: sources - .slice(1, 3) - .every((ds) => - (ds.importedEntities || ds.entities.values) - .filter((e) => e.polyline) - .every((e) => e.polyline.clampToGround?.getValue(now)), - ), - xyzPolygon: - (sources[0].importedEntities || sources[0].entities.values) - .find((e) => e.polygon) - ?.polygon.perPositionHeight?.getValue(now) !== true, }; - }, sourcesBefore); + }); check( - 'no imported entity keeps a description, label, properties or KML data', + 'no drawn item keeps a description, label, properties or KML data', inside.sources === 4 && inside.described + inside.labelled + @@ -423,7 +424,7 @@ try { JSON.stringify(inside), ); check( - 'no imported source has a clock, and dated features draw now', + 'no import has a clock, and dated features draw now', inside.clocks === 0 && inside.hidden === 0 && inside.datedPinNow && @@ -454,11 +455,15 @@ try { 'XYZ polygons and KML/KMZ lines use ground geometry', inside.xyzPolygon && inside.groundedLines, ); - await page.evaluate((from) => { - for (const entity of window.__godsEyeView.viewer.dataSources.get(from) - .entities.values) - if (entity.billboard) entity.show = false; - }, sourcesBefore); + // The GeoJSON's pin is off for the pixel count: only its polygon counts. + await page.evaluate(() => { + const tool = window.__gevGeoImport; + const { sets } = tool.drawn(tool.list()[0].id); + const primitives = window.__godsEyeView.viewer.scene.primitives; + for (let i = 0; i < primitives.length; i++) + if (primitives.get(i).drawSetId === sets.icons) + primitives.get(i).show = false; + }); await page.evaluate((view) => { const viewer = window.__godsEyeView.viewer; viewer.camera.cancelFlight?.(); @@ -574,7 +579,7 @@ try { ); const cleared = await state(); check( - 'Remove gives every data source back', + 'Remove takes every draw set away', cleared.rows === 0 && cleared.dataSources === before.dataSources, `${cleared.dataSources} vs ${before.dataSources}`, ); diff --git a/src/app/layers/streetLevel.js b/src/app/layers/streetLevel.js index ec0eed217..3b7c7cc0b 100644 --- a/src/app/layers/streetLevel.js +++ b/src/app/layers/streetLevel.js @@ -3,7 +3,6 @@ import { createMapillaryProvider } from '../../layers/streetLevel/providers/mapi import { createGoogleProvider } from '../../layers/streetLevel/providers/google/index.js'; import { createKartaViewProvider } from '../../layers/streetLevel/providers/kartaview/index.js'; import { createPanoramaxProvider } from '../../layers/streetLevel/providers/panoramax/index.js'; -import * as sprites from '../../data/spriteOrder.js'; import * as picking from '../../data/pickRegistry.js'; import * as scenePick from '../../data/scenePick.js'; import * as render from '../../renderGovernor.js'; @@ -36,6 +35,6 @@ export function createApplicationStreetLevel({ }) { return createStreetLevelLayer({ providers, - services: { sprites, picking, scenePick, render }, + services: { picking, scenePick, render }, }); } diff --git a/src/data/aircraftIconLayer.js b/src/data/aircraftIconLayer.js index 571b4ff4b..3dc8cfcdf 100644 --- a/src/data/aircraftIconLayer.js +++ b/src/data/aircraftIconLayer.js @@ -259,17 +259,18 @@ export function createAircraftIcons({ /** * Create the draw set for one layer lifetime, drawn over depth and culled - * at the horizon on the delayed live clock. - * @returns {string} The set's id, for the sprite order. + * at the horizon on the delayed live clock, stacked by the sprite order. + * @returns {string} The set's id. */ function _createIcons(viewer) { const set = drawOf(viewer).icons({ id: `${layerId}-icons:${++instances}`, horizonCull: true, motionClock: { kind: 'live', delayMs }, + drawOrder: services.sprites?.spriteDrawOrder?.(layerId) ?? 0, }); - // Columns of no rows put the set's host in the scene now, where the - // sprite order finds it, rather than with its first contact. + // Columns of no rows put the set's host in the scene now, rather than + // with its first contact. set.setColumns?.({ ids: [] }); switchPx = _imageSwitchPx(viewer); state._icons = createContactIcons(set, { diff --git a/src/data/firmsHorizonCull.test.mjs b/src/data/firmsHorizonCull.test.mjs index 1202571ef..0822ba66a 100644 --- a/src/data/firmsHorizonCull.test.mjs +++ b/src/data/firmsHorizonCull.test.mjs @@ -9,7 +9,6 @@ import { reportMeshFloorCell, setMeshFloorPreferred, } from './groundFloor.js'; -import { unregisterSpriteCollection } from './spriteOrder.js'; import { isOverlayPointVisible, normalizeOverlayEntry } from '../overlays/worldOverlay.js'; import { createIconSet, ICON_CULL_LIFT } from './iconSet.js'; import { writeHoldSegment } from './iconSegment.js'; @@ -426,7 +425,6 @@ function createHarness(rawFires) { }, cleanup() { layer.destroy(viewer); - unregisterSpriteCollection('firms'); Cesium.SceneTransforms.worldToWindowCoordinates = originalProject; globalThis.fetch = originalFetch; if (originalWindow === undefined) delete globalThis.window; else globalThis.window = originalWindow; diff --git a/src/data/geoFileImport.js b/src/data/geoFileImport.js index 8779118b1..d48957763 100644 --- a/src/data/geoFileImport.js +++ b/src/data/geoFileImport.js @@ -1,10 +1,7 @@ /** Session-local geodata loading with bounded geometry and local-only file references. */ import { createImportImageBudget } from './geoImportImages.js'; -import { - loadImportedGeoJson, - loadImportedKml, - settleImportedSource, -} from './geoImportPins.js'; +import { readGeoJsonFeatures, readKmlFeatures } from './geoImportFeatures.js'; +import { settleImportedDrawing } from './geoImportPins.js'; import { GeoImportError, MAX_IMPORT_ENTITIES, @@ -204,8 +201,8 @@ export function countGeoJsonGeometry( MultiPolygon: 3, }[node.type]; if (depth !== undefined) { - // Multipart geometries create one Cesium entity per component, even - // when the document wraps them in just one Feature. + // Multipart geometries are one map object per component, even when + // the document wraps them in just one Feature. entities += node.type.startsWith('Multi') ? Array.isArray(node.coordinates) ? node.coordinates.length @@ -285,7 +282,7 @@ function overLimitError({ over, features }) { } // The import uses WGS84 ground footprints and a short name only. Remove -// loader-sensitive metadata before Cesium allocates properties or entity IDs. +// everything else before the features are read. function prepareGeoJson(geojson) { const stack = [geojson]; while (stack.length) { @@ -512,7 +509,7 @@ export function kmzImageResolver( * Strip everything from a parsed KML document that would make the page reach * outside the file. `resolveLocal(href)` returns a URL for an image packed in * the same KMZ (or null); every other href is removed, so icons fall back to - * Cesium's default pin and a ground overlay without a local image is dropped. + * the default pin and a ground overlay without a local image is dropped. * @param {Document} doc * @param {(href: string) => string|null} [resolveLocal] * @returns {{removed: number}} How many elements were taken out. @@ -526,8 +523,8 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) { } }; // One snapshot and pass, rather than rescanning the whole XML tree for - // every discarded element type. Strip unused metadata before the loader - // constructs descriptions, availability intervals and camera objects. + // every discarded element type. Strip unused metadata (descriptions, + // times and cameras) before the features are read. for (const el of [...doc.getElementsByTagName('*')]) if ( KML_REMOVED_ELEMENTS.has(el.localName) || @@ -549,8 +546,8 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) { drop(overlay); continue; } - // An empty Icon explicitly hides the billboard in Cesium. Remove the - // blocked icon definition so the loader supplies its local default pin. + // An empty Icon explicitly hides the icon. Remove the blocked icon + // definition so the placemark gets the local default pin. // Authored empty Icons (with no blocked href) still mean no marker. drop( icon?.localName === 'Icon' && icon.parentNode?.localName === 'IconStyle' @@ -571,9 +568,8 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) { !childrenNamed(pair, 'Style').length) ) drop(pair); - // Draw recorded KML paths as their original static vertices. Letting the - // loader build time-dynamic tracks would expand each into thousands of - // interpolated points later and exceed the checked geometry budget. + // Draw recorded KML paths as their original static vertices, the + // geometry the budget checked, never as time-dynamic tracks. for (const track of elementsNamed(doc, 'Track')) { const line = doc.createElementNS( 'http://www.opengis.net/kml/2.2', @@ -639,11 +635,11 @@ const childText = (el, localName) => (childrenNamed(el, localName)[0]?.textContent || '').trim(); /** - * Share identical styles before the loader builds their graphics. Inline + * Share identical styles before the features are read. Inline * overrides stay inline StyleMaps, so an existing styleUrl still supplies * properties the override leaves unset. Named references are redirected. * Random-color styles remain unchanged; duplicate IDs - * keep the loader's original precedence instead of being rewritten. + * keep their original precedence instead of being rewritten. * @param {Document} doc Sanitized KML. * @param {{serializeStyle?: Function, maxStyles?: number}} [options] * @returns {number} Number of original style blocks replaced. @@ -746,8 +742,8 @@ export function deduplicateKmlStyles( } /** - * Count placemarks and vertices in a parsed KML document before Cesium reads - * it: every `` tuple and every `gx:coord`, plus normalized + * Count placemarks and vertices in a parsed KML document before it is read: + * every `` tuple and every `gx:coord`, plus normalized * style definitions and distinct image `` values (`links`). * Inline StyleMaps are references, not additional shared style definitions. * @param {Document} doc @@ -797,8 +793,8 @@ export function countKmlGeometry( ) throw new GeoImportError(msg('That KML has invalid geometry bounds.')); } - // Cesium creates entities for folders, a parent for every placemark, and - // a child for each geometry in a MultiGeometry, including nested containers. + // A map object for each folder, a parent for every placemark, and a + // child for each geometry in a MultiGeometry, including nested containers. if ( name === 'Placemark' || name === 'Folder' || @@ -1040,7 +1036,7 @@ export function repairKmlRootNamespaces(value, what = 'KML') { /** Parse KML or GPX with the shared namespace repairs and XML validation. */ export function parseXml(text, what) { - // Match the namespace repairs Cesium applies to ordinary exported KML. + // The namespace repairs ordinary exported KML needs. if (what !== 'GPX') text = repairKmlRootNamespaces(text, what); const doc = new DOMParser().parseFromString( String(text).replace(/^/, ''), @@ -1099,11 +1095,11 @@ function checkLocalImages(doc, entries = new Map(), main = '') { } /** - * Load a ground footprint, retaining explicit KML altitude modes. Call mount after - * adding the source; revoke owns pin and blob cleanup. + * Load a ground footprint, retaining explicit KML altitude modes. Call mount + * to draw it; revoke owns the drawing and blob cleanup. * @param {File|Blob & {name?: string}} file * @param {{viewer: object, color?: string}} options `color` is a CSS color. - * @returns {Promise<{dataSource: object, name: string, + * @returns {Promise<{drawing: object, name: string, * format: string, entityCount: number, removed: number, mount: Function, revoke: Function}>} */ export async function loadGeoFile( @@ -1131,11 +1127,11 @@ export async function loadGeoFile( const urls = []; let drawn; const revoke = () => { - drawn?.pins.destroy(); + drawn?.drawing.destroy(); urls.splice(0).forEach((url) => URL.revokeObjectURL(url)); }; const name = importDisplayName(file.name); - let dataSource; + let read; let removed = 0; try { if (format === 'geojson' || format === 'gpx') { @@ -1151,7 +1147,7 @@ export async function loadGeoFile( throw new GeoImportError(msg('That file has nothing to draw.')); prepareGeoJson(geojson); bytes = null; - dataSource = await loadImportedGeoJson(geojson, { color, signal }); + read = readGeoJsonFeatures(geojson, color); } else { let text; let entries; @@ -1180,7 +1176,7 @@ export async function loadGeoFile( deduplicateKmlStyles(doc); const normalizedSize = countKmlGeometry(doc); if (normalizedSize.over) throw overLimitError(normalizedSize); - dataSource = await loadImportedKml(doc, viewer, signal); + read = readKmlFeatures(doc); } } catch (error) { revoke(); @@ -1192,21 +1188,21 @@ export async function loadGeoFile( ); } try { - drawn = await settleImportedSource(dataSource, viewer, signal); + drawn = await settleImportedDrawing(read, viewer, signal); } catch (error) { revoke(); throw error; } - // The display name travels beside the data source (the file name, not + // The display name travels beside the drawing (the file name, not // anything the document says about itself). return { - dataSource, + drawing: drawn.drawing, name, format, entityCount: drawn.entityCount, removed, revoke, - mount: () => drawn.pins.mount(), + mount: () => drawn.drawing.mount(), refreshGround: drawn.refreshGround, }; } diff --git a/src/data/geoFileImport.test.mjs b/src/data/geoFileImport.test.mjs index 52a1c6165..8a6fa36c1 100644 --- a/src/data/geoFileImport.test.mjs +++ b/src/data/geoFileImport.test.mjs @@ -2,7 +2,6 @@ import test from 'node:test'; import assert from 'node:assert/strict'; import { deflateRawSync } from 'node:zlib'; import { spawnSync } from 'node:child_process'; -import * as Cesium from 'cesium'; import { GeoImportError, repairKmlRootNamespaces, @@ -24,106 +23,125 @@ import { readZipEntries, resolveKmzPath, } from './geoFileImport.js'; -import { detachImportedTime, scrubImportedEntities } from './geoImportPins.js'; +import { readKmlFeatures } from './geoImportFeatures.js'; +import { bindFakeGroundRenderer } from '../testSupport/fakeGroundRenderer.mjs'; +import { fromDegrees, toLonLatH } from '../geo/ellipsoid.js'; + +/** The point `at` (lon, lat) at `height`, ECEF. */ +const fromDegreesHeight = (at, height) => fromDegrees(at.lon, at.lat, height); import { createImportImageBudget, importImageSize } from './geoImportImages.js'; -test('real loader grounds XYZ GeoJSON and strips loader-sensitive metadata before allocation', async () => { - const previous = Cesium.PinBuilder.prototype.fromColor; - Cesium.PinBuilder.prototype.fromColor = () => ({ - toDataURL: () => 'data:image/png;base64,local', - tagName: 'CANVAS', +/** + * A viewer bound to a renderer double for the import: draw sets kept, + * terrain heights from `terrain` (the globe shown) or from `hidden` (the + * globe hidden over a mesh), mesh clamps from `clamp`, and a camera + * `eyeHeight` up looking at `view`. + */ +function importViewer({ + terrain = () => null, + hidden = null, + clamp = null, + eyeHeight = 1_000_000, + view = { west: -180, south: -90, east: 180, north: 90 }, +} = {}) { + const viewer = {}; + const ground = bindFakeGroundRenderer(viewer, { + sampleHeight: Boolean(clamp), + eye: { lon: 0, lat: 0, height: eyeHeight }, }); - try { - const object = { - type: 'FeatureCollection', - crs: { type: 'name', properties: { name: 'unsupported' } }, - features: [ - { - type: 'Feature', - id: 'kartaview-owned', - properties: { name: 'Test', addProperty: 1, 'marker-symbol': 'rail' }, - geometry: { type: 'Point', coordinates: [1, 2, 999] }, - }, - { - type: 'Feature', - geometry: { - type: 'Polygon', - coordinates: [ - [ - [1, 2, 0], - [2, 2, 0], - [2, 3, 0], - [1, 2, 0], - ], - ], - }, - }, - ], - }; - const loaded = await loadGeoFile( - new File([JSON.stringify(object)], 'xyz.geojson'), + ground.globe.show = !hidden; + ground.globe.terrain = terrain; + const query = ground.renderer.query; + const sampleHeightNow = query.sampleHeightNow; + query.sampleHeightNow = (point, options = {}) => + hidden && options.against === 'terrain' + ? hidden(point) + : sampleHeightNow(point, options); + if (clamp) query.clampToSurface = async (points) => clamp(points); + ground.renderer.camera.viewRectangle = () => view; + return { viewer, camera: ground.renderer.camera, ...ground }; +} + +/** The live import set of a kind (`pins`, `lines`, `shapes`). */ +const importSet = (ground, kind) => + [...ground.draw.sets.values()].find( + (record) => + !record.disposed && + record.spec.id.startsWith('geo-import:') && + record.spec.id.endsWith(`:${kind}`), + ); + +test('GeoJSON points stand on the ground and lose everything but a name', async () => { + const object = { + type: 'FeatureCollection', + crs: { type: 'name', properties: { name: 'unsupported' } }, + features: [ { - viewer: { - scene: { - frameState: { mode: Cesium.SceneMode.SCENE3D }, - primitives: new Cesium.PrimitiveCollection(), - globe: { getHeight: () => 1234 }, - }, + type: 'Feature', + id: 'kartaview-owned', + properties: { name: 'Test', addProperty: 1, 'marker-symbol': 'rail' }, + geometry: { type: 'Point', coordinates: [1, 2, 999] }, + }, + { + type: 'Feature', + geometry: { + type: 'Polygon', + coordinates: [ + [ + [1, 2, 0], + [2, 2, 0], + [2, 3, 0], + [1, 2, 0], + ], + ], }, }, + ], + }; + const t = importViewer({ terrain: () => 1234 }); + try { + const loaded = await loadGeoFile( + new File([JSON.stringify(object)], 'xyz.geojson'), + { viewer: t.viewer, color: '#ff0000' }, ); - const [point, polygon] = - loaded.dataSource.importedEntities || loaded.dataSource.entities.values; - assert.notEqual(point.id, 'kartaview-owned'); - assert.equal(point.name, 'Test'); - assert.equal(point.properties, undefined); - assert.equal( - point.billboard.heightReference.getValue(), - Cesium.HeightReference.NONE, - ); - assert.equal(point.billboard.disableDepthTestDistance.getValue(), Infinity); - assert.ok( - Math.abs( - Cesium.Cartographic.fromCartesian( - point.position.getValue(Cesium.JulianDate.now()), - ).height - 1234, - ) < 1e-6, - ); - assert.notEqual(polygon.polygon.perPositionHeight?.getValue(), true); - assert.equal(loaded.dataSource.clock, undefined); - assert.equal( - loaded.dataSource.entities.values.length, - 1, - 'only polygon geometry remains in the per-frame entity collection', - ); - assert.equal(loaded.dataSource.importedBillboards.length, 1); - const update = Cesium.BillboardCollection.prototype.update; - let passes = 0; - Cesium.BillboardCollection.prototype.update = () => passes++; - try { - const collection = loaded.dataSource.importedBillboards; - collection.update({ passes: { pick: true, offscreen: true } }); - assert.equal(passes, 0, 'height picks ignore imported markers'); - collection.update({ passes: { pick: true, offscreen: false } }); - collection.update({ passes: { render: true, offscreen: true } }); - assert.equal(passes, 2, 'ordinary picks and screenshots include markers'); - } finally { - Cesium.BillboardCollection.prototype.update = update; - } - + assert.equal(importSet(t, 'pins'), undefined, 'nothing drawn before mount'); loaded.mount(); - loaded.dataSource.show = false; - assert.equal(loaded.dataSource.importedBillboards.show, false); - loaded.dataSource.show = true; + const [pin] = importSet(t, 'pins').items.values(); + assert.equal(pin.meta.name, 'Test'); + assert.deepEqual(Object.keys(pin.meta), ['name']); + assert.equal(pin.height, 'absolute'); + assert.equal(pin.pos.height, 1234, 'on the terrain, not at 999 m'); + assert.equal(pin.disableDepthTest, Infinity); + assert.equal(pin.anchor, 'bottom'); + assert.deepEqual(pin.size, [48, 48]); + assert.match(pin.image.svg, /rgba\(255,0,0,1\)/); + assert.equal( + importSet(t, 'pins').spec.includeInHeightQueries, + undefined, + 'height queries never see imported markers', + ); + const [polygon] = importSet(t, 'shapes').items.values(); + assert.equal(polygon.height, 'clampGround'); + assert.deepEqual(polygon.fill, [1, 0, 0, 0.25]); + assert.equal(polygon.outline, undefined, 'no outline on the ground'); + assert.equal(loaded.entityCount, 2); + + loaded.drawing.show = false; + assert.equal(importSet(t, 'pins').show, false); + assert.equal(importSet(t, 'shapes').show, false); + loaded.drawing.show = true; + t.globe.terrain = () => 50; await loaded.refreshGround(); assert.equal( - point.billboard.heightReference.getValue(), - Cesium.HeightReference.NONE, + [...importSet(t, 'pins').items.values()][0].pos.height, + 50, + 'a refresh moves the pin onto the ground now under it', ); loaded.revoke(); - assert.ok(loaded.dataSource.importedBillboards.isDestroyed()); + assert.equal(importSet(t, 'pins'), undefined); + assert.equal(importSet(t, 'shapes'), undefined); } finally { - Cesium.PinBuilder.prototype.fromColor = previous; + t.unbind(); } }); @@ -240,81 +258,54 @@ test('pending file reads can be cancelled without waiting for the loader timeout }); test('a hidden coarse globe never seeds a mesh-clamp ray below the surface', async () => { - const previous = Cesium.PinBuilder.prototype.fromColor; - Cesium.PinBuilder.prototype.fromColor = () => ({ - tagName: 'CANVAS', - toDataURL: () => 'data:image/png;base64,local', - }); const starts = []; + const t = importViewer({ + eyeHeight: 100_000, + hidden: () => -12311, + clamp: (points) => + points.map((p) => { + const at = toLonLatH(p); + starts.push(at.height); + return at.height < -9000 + ? null + : { ...p, ...fromDegreesHeight(at, 2000) }; + }), + }); try { - for (const terrainProvider of [new Cesium.EllipsoidTerrainProvider(), {}]) { - const viewer = { - terrainProvider, - camera: { - positionCartographic: { height: 100000 }, - computeViewRectangle: () => Cesium.Rectangle.MAX_VALUE, - }, - scene: { - frameState: { mode: Cesium.SceneMode.SCENE3D }, - globe: { show: false, getHeight: () => -12311 }, - clampToHeightSupported: true, - clampToHeightMostDetailed: async (points) => - points.map((p) => { - const c = Cesium.Cartographic.fromCartesian(p); - starts.push(c.height); - return c.height < -9000 - ? undefined - : Cesium.Cartesian3.fromRadians(c.longitude, c.latitude, 2000); - }), - }, - }; - const loaded = await loadGeoFile( - new File( - [JSON.stringify({ type: 'Point', coordinates: [1, 2] })], - 'surface.geojson', - ), - { viewer }, - ); - assert.ok( - Math.abs(starts.at(-1)) < 1e-6, - `ray started at ${starts.at(-1)}`, - ); - assert.ok( - Math.abs( - Cesium.Cartographic.fromCartesian( - loaded.dataSource.importedBillboards.get(0).position, - ).height - 2000, - ) < 1e-6, - ); - const sampled = starts.length; - viewer.camera.computeViewRectangle = () => undefined; - await loaded.refreshGround(); - assert.equal( - starts.length, - sampled, - 'distant points do not start a detailed mesh sweep', - ); - assert.ok( - Math.abs( - Cesium.Cartographic.fromCartesian( - loaded.dataSource.importedBillboards.get(0).position, - ).height - 2000, - ) < 1e-6, - 'the last valid mesh height survives outside the camera footprint', - ); - const point = loaded.dataSource.importedEntities[0]; - const other = new Cesium.Entity({ id: 'unrelated-aircraft' }); - viewer.trackedEntity = starts.length === 1 ? other : point; - viewer.selectedEntity = point; - loaded.revoke(); - assert.equal(viewer.selectedEntity, undefined); - assert.equal( - viewer.trackedEntity, - starts.length === 1 ? other : undefined, - ); - } + const loaded = await loadGeoFile( + new File( + [JSON.stringify({ type: 'Point', coordinates: [1, 2] })], + 'surface.geojson', + ), + { viewer: t.viewer }, + ); + loaded.mount(); + const height = () => [...importSet(t, 'pins').items.values()][0].pos.height; + assert.ok( + Math.abs(starts.at(-1)) < 1e-6, + `ray started at ${starts.at(-1)}`, + ); + assert.ok(Math.abs(height() - 2000) < 1e-6); + const sampled = starts.length; + t.camera.viewRectangle = () => ({ + west: 10, + south: 10, + east: 11, + north: 11, + }); + await loaded.refreshGround(); + assert.equal( + starts.length, + sampled, + 'distant points do not start a detailed mesh sweep', + ); + assert.ok( + Math.abs(height() - 2000) < 1e-6, + 'the last valid mesh height survives outside the camera footprint', + ); + loaded.revoke(); } finally { - Cesium.PinBuilder.prototype.fromColor = previous; + t.unbind(); } }); @@ -708,7 +699,7 @@ test('distinct style definitions and named StyleMaps still have a pre-loader bud assert.equal(countKmlGeometry(maps, { maxLinks: 1 }).over, 'links'); }); -test('KML extended metadata is removed before the document reaches the loader', () => { +test('KML extended metadata is removed before the document is read for drawing', () => { const point = xmlEl('Point', {}, [xmlEl('coordinates', {}, '1,2,0')]); const metadata = xmlEl('ExtendedData', {}, [ xmlEl('Data', { name: 'note' }, [xmlEl('value', {}, 'ordinary metadata')]), @@ -809,7 +800,7 @@ test('GPX keeps each track segment as its own line, names as text, no times', () assert.throws(() => gpxToGeoJson(xmlDoc(xmlEl('kml'))), GeoImportError); }); -test('GPX past the point limit is refused while reading, before Cesium', () => { +test('GPX past the point limit is refused while reading, before anything is drawn', () => { const seg = xmlEl( 'trkseg', {}, @@ -966,7 +957,7 @@ test('KML vertex counting recognizes the Unicode whitespace Cesium accepts', () } }); -test('KML multipart geometries count every Cesium entity before loading', () => { +test('KML multipart geometries count every map object before drawing', () => { const doc = xmlDoc( xmlEl('kml', {}, [ xmlEl('Placemark', {}, [ @@ -1049,69 +1040,47 @@ test('a KMZ image gets one blob URL however many hrefs name it', () => { assert.deepEqual(kept, ['blob:test/1', 'blob:test/2']); }); -test('imported entities keep only a short plain-text name: no description, label or properties', () => { - const ds = new Cesium.CustomDataSource('t'); - const e = ds.entities.add({ - name: 'Trail\u0007 head\n' + 'x'.repeat(300), - description: '', - label: { text: 'Trail head' }, - properties: { note: '' }, - position: Cesium.Cartesian3.fromDegrees(1, 2), - }); - e.kml = { extendedData: { note: { value: '