mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 02:14:01 +00:00
fix(import): keep KML points visible when remote icons are blocked
This commit is contained in:
1 parent
ae51f9aee9
commit
c58ff5c215
6 files changed
+146
-4
No files matched your search
+3
-1
@@ -174,7 +174,9 @@
|
||||
apply before loading, recorded KML tracks retain their original vertices,
|
||||
and queued imports respect navigation made while waiting. Repeated KML styles
|
||||
are shared before the style limit is applied, allowing large files with many
|
||||
copies of the same style to load without changing their graphics.
|
||||
copies of the same style to load without changing their graphics. Blocked
|
||||
remote icons now fall back to local pins instead of leaving point markers
|
||||
invisible, including USGS earthquake exports.
|
||||
|
||||
## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features)
|
||||
|
||||
|
||||
@@ -4328,12 +4328,12 @@ are omitted rather than framing the wrong part of the globe.
|
||||
- **Navigation.** The row's Fly-to is an immediate navigation through the application shell (`runImmediateNavigation('file', …)`): Cockpit refuses it, and the current follow/track owner is released first. The fly after an import is a deferred navigation: `beginDeferredLocationNavigation()` stamps it when the pick or drop is submitted, before queued batches wait and `reassertDeferredLocationNavigation()` re-checks it just before flying, so a camera gesture, search, Cockpit or any newer navigation while files load wins and the import does not fly. Fly-to frames the features' longitudes and latitudes at the surface (thinned to 5,000 points by index, never spread into a call) and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`); `viewer.flyTo` is not used, because before terrain streams in it frames ground-clamped points kilometers below the surface.
|
||||
- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time, and KML `gx:Track` / `gx:MultiTrack` geometry is converted to static lines before Cesium loads it, preserving the recorded vertices and segment gaps without expanding them through time sampling. GPX times are not read at all.
|
||||
- Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer, navigation: styleManager })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`.
|
||||
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, drawn locally, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML and `ExtendedData` before metadata processing. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
|
||||
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (a blocked `IconStyle` icon definition is removed so the marker falls back to Cesium's default pin, drawn locally; an intentionally empty icon remains empty, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML and `ExtendedData` before metadata processing. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
|
||||
- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only.
|
||||
- **KMZ** is read by a small zip reader (`readZipEntries`: stored and deflated entries through the platform `DecompressionStream`; no encryption, no ZIP64). The main document is `doc.kml`, else the shallowest `.kml`; relative hrefs resolve only to entries inside the archive.
|
||||
- **Bounds.** A file may be at most 50 MiB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MiB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may create at most 20,000 Cesium entities, including multipart geometries and KML parent containers (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), and a KML/KMZ at most 2,000 normalized style definitions (including named StyleMaps) and distinct surviving image links (`MAX_KML_LINKS`). Identical KML styles are shared after metadata and remote-reference cleanup, preserving inline overrides and local style references; repeated copies do not exhaust the style budget. Folder containers count toward the entity budget. Files of 20–30 MB fit within the byte limit; drawing complexity is bounded separately. Geometry is counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
|
||||
- **Keyboard.** Hide, show and remove rebuild the list without losing focus: the same control on the same row keeps it, a removed row hands it to the next row (else the Import button).
|
||||
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Large-file gate: `node scripts/qa-geo-import-large.mjs [--file /absolute/path/to/sample.kml]` tests a 27.6 MB repeated-style fixture, compares its graphics with an unshared Cesium control, and records matched frame times on the actual WebGL renderer. Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
|
||||
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Large-file gate: `node scripts/qa-geo-import-large.mjs [--file /absolute/path/to/sample.kml]` tests a 27.6 MB repeated-style fixture, compares its graphics with an unshared Cesium control, and records matched frame times on the actual WebGL renderer. Add `--expect-local-pins` for a USGS fixture to verify marker images and painted show/hide output. Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
|
||||
|
||||
### Recent Imagery (September 2026)
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
/** Large KML acceptance: style equivalence, 25 MB import and optional real file.
|
||||
* QA_BASE_URL defaults to http://localhost:4173. Optional --file /path/file.kml.
|
||||
* For a USGS fixture with blocked icons, add --expect-local-pins to check paint.
|
||||
*/
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
@@ -303,6 +304,90 @@ try {
|
||||
'real dated KML leaves the shared clock unchanged',
|
||||
JSON.stringify(clockBefore) === JSON.stringify(clockAfter),
|
||||
);
|
||||
if (suppliedItem && process.argv.includes('--expect-local-pins')) {
|
||||
const markers = await page.evaluate(() => {
|
||||
const { viewer } = window.__godsEyeView,
|
||||
C = window.__CESIUM__;
|
||||
const source = viewer.dataSources.get(viewer.dataSources.length - 1);
|
||||
const points = source.entities.values.filter(
|
||||
(e) => e.position && e.billboard,
|
||||
);
|
||||
const images = points.filter((e) =>
|
||||
e.billboard.image?.getValue(viewer.clock.currentTime),
|
||||
);
|
||||
if (images.length) {
|
||||
const at = C.Cartographic.fromCartesian(
|
||||
images[0].position.getValue(viewer.clock.currentTime),
|
||||
);
|
||||
viewer.camera.cancelFlight();
|
||||
viewer.camera.setView({
|
||||
destination: C.Cartesian3.fromRadians(
|
||||
at.longitude,
|
||||
at.latitude,
|
||||
100000,
|
||||
),
|
||||
orientation: { heading: 0, pitch: -Math.PI / 2, roll: 0 },
|
||||
});
|
||||
}
|
||||
return { points: points.length, images: images.length };
|
||||
});
|
||||
check(
|
||||
'supplied point markers have drawable images',
|
||||
markers.points > 0 && markers.images === markers.points,
|
||||
JSON.stringify(markers),
|
||||
);
|
||||
if (markers.points) {
|
||||
const yellowPixels = async () => {
|
||||
const shot = await page.screenshot({
|
||||
clip: { x: 620, y: 350, width: 200, height: 200 },
|
||||
type: 'png',
|
||||
encoding: 'base64',
|
||||
});
|
||||
return page.evaluate(async (base64) => {
|
||||
const bitmap = await createImageBitmap(
|
||||
await (await fetch(`data:image/png;base64,${base64}`)).blob(),
|
||||
);
|
||||
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height),
|
||||
context = canvas.getContext('2d');
|
||||
context.drawImage(bitmap, 0, 0);
|
||||
const data = context.getImageData(
|
||||
0,
|
||||
0,
|
||||
bitmap.width,
|
||||
bitmap.height,
|
||||
).data;
|
||||
let count = 0;
|
||||
for (let i = 0; i < data.length; i += 4)
|
||||
if (data[i] > 150 && data[i + 1] > 150 && data[i + 2] < 100)
|
||||
count++;
|
||||
return count;
|
||||
}, shot);
|
||||
};
|
||||
const setVisible = (visible) =>
|
||||
page.evaluate(
|
||||
({ id, visible }) => window.__gevGeoImport.setVisible(id, visible),
|
||||
{ id: suppliedItem.id, visible },
|
||||
);
|
||||
await setVisible(false);
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
const hidden = await yellowPixels();
|
||||
await setVisible(true);
|
||||
let shown = hidden;
|
||||
const deadline = performance.now() + 10000;
|
||||
while (performance.now() < deadline && shown <= hidden + 10) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
shown = await yellowPixels();
|
||||
}
|
||||
check(
|
||||
'real USGS markers paint when shown and disappear when hidden',
|
||||
shown > hidden + 10,
|
||||
JSON.stringify({ hidden, shown }),
|
||||
);
|
||||
await page.screenshot({
|
||||
path: path.join(directory, 'real-kml-visible.png'),
|
||||
});
|
||||
}
|
||||
}
|
||||
if (suppliedItem)
|
||||
await page.evaluate(
|
||||
(id) => window.__gevGeoImport.remove(id),
|
||||
|
||||
@@ -362,6 +362,9 @@ try {
|
||||
withKml: entities.filter((e) => e.kml !== undefined).length,
|
||||
hidden: entities.filter((e) => !e.isAvailable(now)).length,
|
||||
datedPinNow: Boolean(named('QA KML pin')?.isAvailable(now)),
|
||||
fallbackPins: ['QA KML pin', 'QA palette pin'].filter((name) =>
|
||||
named(name)?.billboard?.image?.getValue(now),
|
||||
).length,
|
||||
movingIsLine: Boolean(
|
||||
named('QA moving')?.polyline && !named('QA moving')?.position,
|
||||
),
|
||||
@@ -393,6 +396,11 @@ try {
|
||||
inside.movingVertices === 2,
|
||||
JSON.stringify(inside),
|
||||
);
|
||||
check(
|
||||
'blocked KML icon sources use drawable local fallback pins',
|
||||
inside.fallbackPins === 2,
|
||||
JSON.stringify({ fallbackPins: inside.fallbackPins }),
|
||||
);
|
||||
check(
|
||||
'KML MultiTrack keeps its segments and original vertex count',
|
||||
inside.splitTracks.join() === '2,2',
|
||||
|
||||
@@ -493,7 +493,17 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) {
|
||||
const local = resolveLocal(value);
|
||||
if (local) href.textContent = local;
|
||||
else if (/^data:image\/(png|jpeg|gif|webp|bmp);/i.test(value)) continue;
|
||||
else drop(href);
|
||||
else {
|
||||
const icon = href.parentNode;
|
||||
// An empty Icon explicitly hides the billboard in Cesium. Remove the
|
||||
// blocked icon definition so the loader supplies its local default pin.
|
||||
// Authored empty Icons (with no blocked href) still mean no marker.
|
||||
drop(
|
||||
icon?.localName === 'Icon' && icon.parentNode?.localName === 'IconStyle'
|
||||
? icon
|
||||
: href,
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const overlay of elementsNamed(doc, 'GroundOverlay')) {
|
||||
if (!elementsNamed(overlay, 'href').length) drop(overlay);
|
||||
|
||||
@@ -426,6 +426,43 @@ test('KML extended metadata is removed before the document reaches the loader',
|
||||
assert.deepEqual(placemark.childNodes, [point]);
|
||||
assert.equal(metadata.parentNode, null);
|
||||
});
|
||||
test('blocked icon sources fall back to local pins without changing intentional empty icons', () => {
|
||||
const blocked = xmlEl('Icon', {}, [
|
||||
xmlEl('href', {}, 'https://example.invalid/icon.png'),
|
||||
]);
|
||||
const blank = xmlEl('Icon');
|
||||
const embedded = xmlEl('Icon', {}, [
|
||||
xmlEl('href', {}, 'data:image/png;base64,AAAA'),
|
||||
]);
|
||||
const packed = xmlEl('Icon', {}, [xmlEl('href', {}, 'packed.png')]);
|
||||
const iconStyle = xmlEl('IconStyle', {}, [
|
||||
xmlEl('scale', {}, '0.3'),
|
||||
blocked,
|
||||
]);
|
||||
const doc = xmlDoc(
|
||||
xmlEl('kml', {}, [
|
||||
xmlEl('Style', {}, [iconStyle]),
|
||||
...[blank, embedded, packed].map((icon) =>
|
||||
xmlEl('Style', {}, [xmlEl('IconStyle', {}, [icon])]),
|
||||
),
|
||||
]),
|
||||
);
|
||||
assert.equal(
|
||||
sanitizeKmlDocument(doc, (href) =>
|
||||
href === 'packed.png' ? 'blob:local-image' : null,
|
||||
).removed,
|
||||
1,
|
||||
);
|
||||
assert.equal(blocked.parentNode, null);
|
||||
assert.equal(iconStyle.childNodes[0].textContent, '0.3');
|
||||
assert.equal(blank.parentNode.localName, 'IconStyle');
|
||||
assert.equal(
|
||||
embedded.childNodes[0].textContent,
|
||||
'data:image/png;base64,AAAA',
|
||||
);
|
||||
assert.equal(packed.childNodes[0].textContent, 'blob:local-image');
|
||||
});
|
||||
|
||||
const trkpt = (lat, lon, time) =>
|
||||
xmlEl('trkpt', { lat, lon }, time ? [xmlEl('time', {}, time)] : []);
|
||||
|
||||
|
||||
Reference in new issue
Block a user