fix(import): keep KML points visible when remote icons are blocked

This commit is contained in:
Mazeyar Moeini Feizabadi committed 2026-10-14 13:16:10 +02:00
1 parent ae51f9aee9
commit c58ff5c215
6 files changed
+146 -4

No files matched your search

+3 -1
View File
@@ -174,7 +174,9 @@
apply before loading, recorded KML tracks retain their original vertices,
and queued imports respect navigation made while waiting. Repeated KML styles
are shared before the style limit is applied, allowing large files with many
copies of the same style to load without changing their graphics.
copies of the same style to load without changing their graphics. Blocked
remote icons now fall back to local pins instead of leaving point markers
invisible, including USGS earthquake exports.
## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features)
+2 -2
View File
@@ -4328,12 +4328,12 @@ are omitted rather than framing the wrong part of the globe.
- **Navigation.** The row's Fly-to is an immediate navigation through the application shell (`runImmediateNavigation('file', …)`): Cockpit refuses it, and the current follow/track owner is released first. The fly after an import is a deferred navigation: `beginDeferredLocationNavigation()` stamps it when the pick or drop is submitted, before queued batches wait and `reassertDeferredLocationNavigation()` re-checks it just before flying, so a camera gesture, search, Cockpit or any newer navigation while files load wins and the import does not fly. Fly-to frames the features' longitudes and latitudes at the surface (thinned to 5,000 points by index, never spread into a call) and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`); `viewer.flyTo` is not used, because before terrain streams in it frames ground-clamped points kilometers below the surface.
- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time, and KML `gx:Track` / `gx:MultiTrack` geometry is converted to static lines before Cesium loads it, preserving the recorded vertices and segment gaps without expanding them through time sampling. GPX times are not read at all.
- Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer, navigation: styleManager })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`.
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, drawn locally, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML and `ExtendedData` before metadata processing. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (a blocked `IconStyle` icon definition is removed so the marker falls back to Cesium's default pin, drawn locally; an intentionally empty icon remains empty, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML and `ExtendedData` before metadata processing. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only.
- **KMZ** is read by a small zip reader (`readZipEntries`: stored and deflated entries through the platform `DecompressionStream`; no encryption, no ZIP64). The main document is `doc.kml`, else the shallowest `.kml`; relative hrefs resolve only to entries inside the archive.
- **Bounds.** A file may be at most 50 MiB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MiB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may create at most 20,000 Cesium entities, including multipart geometries and KML parent containers (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), and a KML/KMZ at most 2,000 normalized style definitions (including named StyleMaps) and distinct surviving image links (`MAX_KML_LINKS`). Identical KML styles are shared after metadata and remote-reference cleanup, preserving inline overrides and local style references; repeated copies do not exhaust the style budget. Folder containers count toward the entity budget. Files of 20–30 MB fit within the byte limit; drawing complexity is bounded separately. Geometry is counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
- **Keyboard.** Hide, show and remove rebuild the list without losing focus: the same control on the same row keeps it, a removed row hands it to the next row (else the Import button).
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Large-file gate: `node scripts/qa-geo-import-large.mjs [--file /absolute/path/to/sample.kml]` tests a 27.6 MB repeated-style fixture, compares its graphics with an unshared Cesium control, and records matched frame times on the actual WebGL renderer. Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Large-file gate: `node scripts/qa-geo-import-large.mjs [--file /absolute/path/to/sample.kml]` tests a 27.6 MB repeated-style fixture, compares its graphics with an unshared Cesium control, and records matched frame times on the actual WebGL renderer. Add `--expect-local-pins` for a USGS fixture to verify marker images and painted show/hide output. Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
### Recent Imagery (September 2026)
+85
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env node
/** Large KML acceptance: style equivalence, 25 MB import and optional real file.
* QA_BASE_URL defaults to http://localhost:4173. Optional --file /path/file.kml.
* For a USGS fixture with blocked icons, add --expect-local-pins to check paint.
*/
import fs from 'node:fs';
import os from 'node:os';
@@ -303,6 +304,90 @@ try {
'real dated KML leaves the shared clock unchanged',
JSON.stringify(clockBefore) === JSON.stringify(clockAfter),
);
if (suppliedItem && process.argv.includes('--expect-local-pins')) {
const markers = await page.evaluate(() => {
const { viewer } = window.__godsEyeView,
C = window.__CESIUM__;
const source = viewer.dataSources.get(viewer.dataSources.length - 1);
const points = source.entities.values.filter(
(e) => e.position && e.billboard,
);
const images = points.filter((e) =>
e.billboard.image?.getValue(viewer.clock.currentTime),
);
if (images.length) {
const at = C.Cartographic.fromCartesian(
images[0].position.getValue(viewer.clock.currentTime),
);
viewer.camera.cancelFlight();
viewer.camera.setView({
destination: C.Cartesian3.fromRadians(
at.longitude,
at.latitude,
100000,
),
orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 },
});
}
return { points: points.length, images: images.length };
});
check(
'supplied point markers have drawable images',
markers.points > 0 && markers.images === markers.points,
JSON.stringify(markers),
);
if (markers.points) {
const yellowPixels = async () => {
const shot = await page.screenshot({
clip: { x: 620, y: 350, width: 200, height: 200 },
type: 'png',
encoding: 'base64',
});
return page.evaluate(async (base64) => {
const bitmap = await createImageBitmap(
await (await fetch(`data:image/png;base64,${base64}`)).blob(),
);
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height),
context = canvas.getContext('2d');
context.drawImage(bitmap, 0, 0);
const data = context.getImageData(
0,
0,
bitmap.width,
bitmap.height,
).data;
let count = 0;
for (let i = 0; i < data.length; i += 4)
if (data[i] > 150 && data[i + 1] > 150 && data[i + 2] < 100)
count++;
return count;
}, shot);
};
const setVisible = (visible) =>
page.evaluate(
({ id, visible }) => window.__gevGeoImport.setVisible(id, visible),
{ id: suppliedItem.id, visible },
);
await setVisible(false);
await new Promise((resolve) => setTimeout(resolve, 1000));
const hidden = await yellowPixels();
await setVisible(true);
let shown = hidden;
const deadline = performance.now() + 10000;
while (performance.now() < deadline && shown <= hidden + 10) {
await new Promise((resolve) => setTimeout(resolve, 500));
shown = await yellowPixels();
}
check(
'real USGS markers paint when shown and disappear when hidden',
shown > hidden + 10,
JSON.stringify({ hidden, shown }),
);
await page.screenshot({
path: path.join(directory, 'real-kml-visible.png'),
});
}
}
if (suppliedItem)
await page.evaluate(
(id) => window.__gevGeoImport.remove(id),
+8
View File
@@ -362,6 +362,9 @@ try {
withKml: entities.filter((e) => e.kml !== undefined).length,
hidden: entities.filter((e) => !e.isAvailable(now)).length,
datedPinNow: Boolean(named('QA KML pin')?.isAvailable(now)),
fallbackPins: ['QA KML pin', 'QA palette pin'].filter((name) =>
named(name)?.billboard?.image?.getValue(now),
).length,
movingIsLine: Boolean(
named('QA moving')?.polyline && !named('QA moving')?.position,
),
@@ -393,6 +396,11 @@ try {
inside.movingVertices === 2,
JSON.stringify(inside),
);
check(
'blocked KML icon sources use drawable local fallback pins',
inside.fallbackPins === 2,
JSON.stringify({ fallbackPins: inside.fallbackPins }),
);
check(
'KML MultiTrack keeps its segments and original vertex count',
inside.splitTracks.join() === '2,2',
+11 -1
View File
@@ -493,7 +493,17 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) {
const local = resolveLocal(value);
if (local) href.textContent = local;
else if (/^data:image\/(png|jpeg|gif|webp|bmp);/i.test(value)) continue;
else drop(href);
else {
const icon = href.parentNode;
// An empty Icon explicitly hides the billboard in Cesium. Remove the
// blocked icon definition so the loader supplies its local default pin.
// Authored empty Icons (with no blocked href) still mean no marker.
drop(
icon?.localName === 'Icon' && icon.parentNode?.localName === 'IconStyle'
? icon
: href,
);
}
}
for (const overlay of elementsNamed(doc, 'GroundOverlay')) {
if (!elementsNamed(overlay, 'href').length) drop(overlay);
+37
View File
@@ -426,6 +426,43 @@ test('KML extended metadata is removed before the document reaches the loader',
assert.deepEqual(placemark.childNodes, [point]);
assert.equal(metadata.parentNode, null);
});
test('blocked icon sources fall back to local pins without changing intentional empty icons', () => {
const blocked = xmlEl('Icon', {}, [
xmlEl('href', {}, 'https://example.invalid/icon.png'),
]);
const blank = xmlEl('Icon');
const embedded = xmlEl('Icon', {}, [
xmlEl('href', {}, 'data:image/png;base64,AAAA'),
]);
const packed = xmlEl('Icon', {}, [xmlEl('href', {}, 'packed.png')]);
const iconStyle = xmlEl('IconStyle', {}, [
xmlEl('scale', {}, '0.3'),
blocked,
]);
const doc = xmlDoc(
xmlEl('kml', {}, [
xmlEl('Style', {}, [iconStyle]),
...[blank, embedded, packed].map((icon) =>
xmlEl('Style', {}, [xmlEl('IconStyle', {}, [icon])]),
),
]),
);
assert.equal(
sanitizeKmlDocument(doc, (href) =>
href === 'packed.png' ? 'blob:local-image' : null,
).removed,
1,
);
assert.equal(blocked.parentNode, null);
assert.equal(iconStyle.childNodes[0].textContent, '0.3');
assert.equal(blank.parentNode.localName, 'IconStyle');
assert.equal(
embedded.childNodes[0].textContent,
'data:image/png;base64,AAAA',
);
assert.equal(packed.childNodes[0].textContent, 'blob:local-image');
});
const trkpt = (lat, lon, time) =>
xmlEl('trkpt', { lat, lon }, time ? [xmlEl('time', {}, time)] : []);