diff --git a/CHANGELOG.md b/CHANGELOG.md index 646fdc751..a5526434a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -176,7 +176,10 @@ are shared before the style limit is applied, allowing large files with many copies of the same style to load without changing their graphics. Blocked remote icons now fall back to local pins instead of leaving point markers - invisible, including USGS earthquake exports. + invisible, including USGS earthquake exports. Generated pins share one image + in the GPU atlas; unused time/view metadata is pruned before loading, and + obsolete track resampling code is removed. KML vertex counting follows + the loader's whitespace rules. ## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features) diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index b99bed581..b0524ee20 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -4326,10 +4326,10 @@ are omitted rather than framing the wrong part of the globe. - DISPLAY ▸ **Import** puts GeoJSON (`.geojson`/`.json`), KML, KMZ and GPX files on the globe. Files come in through the Import button (a hidden ``) or by dropping them onto the globe; the format is taken from the extension, then from the first bytes. Each file becomes its own Cesium data source, clamped to the ground, with a row to show/hide it, fly to it, or remove it. GeoJSON and GPX are drawn in the row's color from the whiteboard palette (simplestyle properties in GeoJSON are dropped); KML and KMZ keep the styles written in them, and their row shows a hatched swatch instead of a color. GPX is read by `gpxToGeoJson` and loaded as GeoJSON: waypoints are points, each route a line, and each track one `MultiLineString` with a line per ``, so a pause in a recording stays a gap. Imported files last for the page session: a reload starts empty, and nothing is uploaded or stored. - **Navigation.** The row's Fly-to is an immediate navigation through the application shell (`runImmediateNavigation('file', …)`): Cockpit refuses it, and the current follow/track owner is released first. The fly after an import is a deferred navigation: `beginDeferredLocationNavigation()` stamps it when the pick or drop is submitted, before queued batches wait and `reassertDeferredLocationNavigation()` re-checks it just before flying, so a camera gesture, search, Cockpit or any newer navigation while files load wins and the import does not fly. Fly-to frames the features' longitudes and latitudes at the surface (thinned to 5,000 points by index, never spread into a call) and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`); `viewer.flyTo` is not used, because before terrain streams in it frames ground-clamped points kilometers below the surface. -- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time, and KML `gx:Track` / `gx:MultiTrack` geometry is converted to static lines before Cesium loads it, preserving the recorded vertices and segment gaps without expanding them through time sampling. GPX times are not read at all. +- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time. Unused KML `TimeSpan`/`TimeStamp` and `LookAt`/`Camera` metadata is removed before loader construction, and KML `gx:Track` / `gx:MultiTrack` geometry is converted to static lines before Cesium loads it, preserving the recorded vertices and segment gaps without expanding them through time sampling. GPX times are not read at all. - Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer, navigation: styleManager })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`. - **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (a blocked `IconStyle` icon definition is removed so the marker falls back to Cesium's default pin, drawn locally; an intentionally empty icon remains empty, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML and `ExtendedData` before metadata processing. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin. -- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only. +- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only. Entity normalization batches collection notifications and converts constant generated pin canvases to one shared data URL per image, avoiding a duplicate GPU texture entry per marker while preserving authored image URLs and dynamic image properties. - **KMZ** is read by a small zip reader (`readZipEntries`: stored and deflated entries through the platform `DecompressionStream`; no encryption, no ZIP64). The main document is `doc.kml`, else the shallowest `.kml`; relative hrefs resolve only to entries inside the archive. - **Bounds.** A file may be at most 50 MiB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MiB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may create at most 20,000 Cesium entities, including multipart geometries and KML parent containers (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), and a KML/KMZ at most 2,000 normalized style definitions (including named StyleMaps) and distinct surviving image links (`MAX_KML_LINKS`). Identical KML styles are shared after metadata and remote-reference cleanup, preserving inline overrides and local style references; repeated copies do not exhaust the style budget. Folder containers count toward the entity budget. Files of 20–30 MB fit within the byte limit; drawing complexity is bounded separately. Geometry is counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused. - **Keyboard.** Hide, show and remove rebuild the list without losing focus: the same control on the same row keeps it, a removed row hands it to the next row (else the Import button). diff --git a/scripts/qa-geo-import-large.mjs b/scripts/qa-geo-import-large.mjs index 93b1be262..bdebdeb70 100644 --- a/scripts/qa-geo-import-large.mjs +++ b/scripts/qa-geo-import-large.mjs @@ -102,6 +102,13 @@ try { }); window.__qaStyleSnapshot = (source) => { const now = viewer.clock.currentTime; + const images = new Map(); + const imageValue = (p) => { + const value = p?.getValue(now); + if (value?.tagName !== 'CANVAS') return value; + if (!images.has(value)) images.set(value, value.toDataURL()); + return images.get(value); + }; const color = (p) => p?.getValue(now)?.toCssColorString(); const positions = (p) => p?.getValue(now)?.map((v) => [v.x, v.y, v.z]); return source.entities.values @@ -110,7 +117,7 @@ try { point: e.position?.getValue(now), scale: e.billboard?.scale?.getValue(now), rotation: e.billboard?.rotation?.getValue(now), - image: e.billboard?.image?.getValue(now), + image: imageValue(e.billboard?.image), color: color(e.billboard?.color), width: e.polyline?.width?.getValue(now), lineColor: color(e.polyline?.material?.color), diff --git a/src/data/geoFileImport.js b/src/data/geoFileImport.js index 5ba50241e..9c0e3688b 100644 --- a/src/data/geoFileImport.js +++ b/src/data/geoFileImport.js @@ -68,14 +68,22 @@ const KMZ_IMAGE_TYPES = Object.freeze({ webp: 'image/webp', bmp: 'image/bmp', }); -// KML elements that fetch, embed or execute something outside the file. -const KML_REMOVED_ELEMENTS = Object.freeze([ +// External content and metadata the session-local import does not use. +const KML_REMOVED_ELEMENTS = new Set([ 'NetworkLink', 'NetworkLinkControl', 'ScreenOverlay', 'PhotoOverlay', 'Model', 'Tour', + 'description', + 'BalloonStyle', + 'Snippet', + 'ExtendedData', + 'TimeSpan', + 'TimeStamp', + 'LookAt', + 'Camera', ]); /** An error whose message is meant for the person who chose the file. */ @@ -222,7 +230,7 @@ export function countGeoJsonGeometry( } /** The refusal for a file over a size limit, in the person's terms. */ -function overLimitError({ over, features, positions }) { +function overLimitError({ over, features }) { if (over === 'entities') return new GeoImportError( `That file creates more than ${MAX_IMPORT_ENTITIES.toLocaleString()} map objects (the limit).`, @@ -486,8 +494,11 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) { removed += 1; } }; - for (const name of KML_REMOVED_ELEMENTS) - elementsNamed(doc, name).forEach(drop); + // One snapshot and pass, rather than rescanning the whole XML tree for + // every discarded element type. Strip unused metadata before the loader + // constructs descriptions, availability intervals and camera objects. + for (const el of [...doc.getElementsByTagName('*')]) + if (KML_REMOVED_ELEMENTS.has(el.localName)) drop(el); for (const href of elementsNamed(doc, 'href')) { const value = (href.textContent || '').trim(); const local = resolveLocal(value); @@ -511,10 +522,6 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) { for (const styleUrl of elementsNamed(doc, 'styleUrl')) { if (!(styleUrl.textContent || '').trim().startsWith('#')) drop(styleUrl); } - // Placemark balloons are HTML; the app shows no info box, and keeping their - // /