mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-05 19:04:02 +00:00
fix(import): one blob URL per packed KMZ image, cap KML styles and links
A KMZ could name one large packed image in thousands of <href>s, and each one made a fresh Blob copy for Cesium to decode. kmzImageResolver now keeps one blob URL per archive entry, and countKmlGeometry refuses a document with more than 2,000 <href> and <Style> elements (MAX_KML_LINKS) before Cesium reads it. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
d8b4b0d80f
commit
90b783c532
3 files changed
+94
-22
No files matched your search
@@ -4259,10 +4259,10 @@ are omitted rather than framing the wrong part of the globe.
|
||||
- **Navigation.** The row's Fly-to is an immediate navigation through the application shell (`runImmediateNavigation('file', …)`): Cockpit refuses it, and the current follow/track owner is released first. The fly after an import is a deferred navigation: `beginDeferredLocationNavigation()` stamps it when the batch starts and `reassertDeferredLocationNavigation()` re-checks it just before flying, so a camera gesture, search, Cockpit or any newer navigation while files load wins and the import does not fly. Fly-to frames the features' longitudes and latitudes at the surface (thinned to 5,000 points by index, never spread into a call) and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`); `viewer.flyTo` is not used, because before terrain streams in it frames ground-clamped points kilometers below the surface.
|
||||
- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time, and a time-dynamic KML position (`gx:Track`) is drawn as the line it travels. GPX times are not read at all.
|
||||
- Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer, navigation: styleManager })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`.
|
||||
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, drawn locally, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs. The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
|
||||
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, drawn locally, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
|
||||
- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only.
|
||||
- **KMZ** is read by a small zip reader (`readZipEntries`: stored and deflated entries through the platform `DecompressionStream`; no encryption, no ZIP64). The main document is `doc.kml`, else the shallowest `.kml`; relative hrefs resolve only to entries inside the archive.
|
||||
- **Bounds.** A file may be at most 50 MB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may hold at most 20,000 features (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
|
||||
- **Bounds.** A file may be at most 50 MB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may hold at most 20,000 features (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), and a KML/KMZ at most 2,000 `<href>` and `<Style>` elements (`MAX_KML_LINKS`), counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
|
||||
- **Keyboard.** Hide, show and remove rebuild the list without losing focus: the same control on the same row keeps it, a removed row hands it to the next row (else the Import button).
|
||||
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
|
||||
|
||||
|
||||
+54
-20
@@ -42,6 +42,8 @@ export const MAX_IMPORTED_FILES = 12;
|
||||
export const MAX_IMPORT_ENTITIES = 20_000;
|
||||
/** Most vertices (points of every line, ring and marker) in one file. */
|
||||
export const MAX_IMPORT_POSITIONS = 250_000;
|
||||
/** Most `<href>` and `<Style>` elements in one KML/KMZ document. */
|
||||
export const MAX_KML_LINKS = 2_000;
|
||||
/** Longest plain-text name kept from a file's features. */
|
||||
export const MAX_IMPORT_NAME_LENGTH = 120;
|
||||
/** What the file picker offers. */
|
||||
@@ -207,6 +209,10 @@ export function countGeoJsonGeometry(
|
||||
|
||||
/** The refusal for a file over a size limit, in the person's terms. */
|
||||
function overLimitError({ over, features, positions }) {
|
||||
if (over === 'links')
|
||||
return new GeoImportError(
|
||||
`That file has more than ${MAX_KML_LINKS.toLocaleString()} styles and icon links (the limit).`,
|
||||
);
|
||||
if (over === 'features')
|
||||
return new GeoImportError(
|
||||
`That file has more than ${MAX_IMPORT_ENTITIES.toLocaleString()} features (the limit).`,
|
||||
@@ -414,6 +420,37 @@ export function kmzImageType(path) {
|
||||
const elementsNamed = (doc, localName) =>
|
||||
[...doc.getElementsByTagName('*')].filter((el) => el.localName === localName);
|
||||
|
||||
/**
|
||||
* A `resolveLocal` for `sanitizeKmlDocument`: the blob URL of an image packed
|
||||
* in the KMZ, or null. Each entry gets one blob and one URL however many
|
||||
* hrefs name it, so repeated icons don't copy (and decode) the image again.
|
||||
* @param {Map<string, Uint8Array>} entries
|
||||
* @param {string} main Path of the KMZ's KML document.
|
||||
* @param {(url: string) => void} onUrl Called once per new URL (to revoke later).
|
||||
* @param {(blob: Blob) => string} [createUrl]
|
||||
* @returns {(href: string) => string|null}
|
||||
*/
|
||||
export function kmzImageResolver(
|
||||
entries,
|
||||
main,
|
||||
onUrl,
|
||||
createUrl = (blob) => URL.createObjectURL(blob),
|
||||
) {
|
||||
const urls = new Map();
|
||||
return (href) => {
|
||||
const path = resolveKmzPath(href, main, entries);
|
||||
const type = path && kmzImageType(path);
|
||||
if (!type) return null;
|
||||
let url = urls.get(path);
|
||||
if (!url) {
|
||||
url = createUrl(new Blob([entries.get(path)], { type }));
|
||||
urls.set(path, url);
|
||||
onUrl(url);
|
||||
}
|
||||
return url;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip everything from a parsed KML document that would make the page reach
|
||||
* outside the file. `resolveLocal(href)` returns a URL for an image packed in
|
||||
@@ -467,25 +504,33 @@ const childText = (el, localName) =>
|
||||
|
||||
/**
|
||||
* Count placemarks and vertices in a parsed KML document before Cesium reads
|
||||
* it: every `<coordinates>` tuple and every `gx:coord`.
|
||||
* it: every `<coordinates>` tuple and every `gx:coord`, plus the `<href>`
|
||||
* and `<Style>` elements (`links`), each of which can make Cesium load and
|
||||
* decode an image.
|
||||
* @param {Document} doc
|
||||
* @returns {{features: number, positions: number, over: ''|'features'|'positions'}}
|
||||
* @returns {{features: number, positions: number, links: number,
|
||||
* over: ''|'features'|'positions'|'links'}}
|
||||
*/
|
||||
export function countKmlGeometry(
|
||||
doc,
|
||||
{
|
||||
maxFeatures = MAX_IMPORT_ENTITIES,
|
||||
maxPositions = MAX_IMPORT_POSITIONS,
|
||||
maxLinks = MAX_KML_LINKS,
|
||||
} = {},
|
||||
) {
|
||||
let features = 0;
|
||||
let positions = 0;
|
||||
let links = 0;
|
||||
const result = (over) => ({ features, positions, links, over });
|
||||
for (const el of doc.getElementsByTagName('*')) {
|
||||
const name = el.localName;
|
||||
if (name === 'Placemark') {
|
||||
if (name === 'href' || name === 'Style') {
|
||||
links += 1;
|
||||
if (links > maxLinks) return result('links');
|
||||
} else if (name === 'Placemark') {
|
||||
features += 1;
|
||||
if (features > maxFeatures)
|
||||
return { features, positions, over: 'features' };
|
||||
if (features > maxFeatures) return result('features');
|
||||
} else if (name === 'coordinates') {
|
||||
const text = el.textContent || '';
|
||||
// Tuples are separated by whitespace; count runs without splitting
|
||||
@@ -495,18 +540,16 @@ export function countKmlGeometry(
|
||||
const space = text.charCodeAt(i) <= 32;
|
||||
if (!space && !inTuple) {
|
||||
positions += 1;
|
||||
if (positions > maxPositions)
|
||||
return { features, positions, over: 'positions' };
|
||||
if (positions > maxPositions) return result('positions');
|
||||
}
|
||||
inTuple = !space;
|
||||
}
|
||||
} else if (name === 'coord') {
|
||||
positions += 1;
|
||||
if (positions > maxPositions)
|
||||
return { features, positions, over: 'positions' };
|
||||
if (positions > maxPositions) return result('positions');
|
||||
}
|
||||
}
|
||||
return { features, positions, over: '' };
|
||||
return result('');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -767,16 +810,7 @@ export async function loadGeoFile(file, { viewer, color = '#39d0ff' }) {
|
||||
const main = kmzMainDocument(entries);
|
||||
if (!main) throw new GeoImportError('That KMZ holds no KML document.');
|
||||
text = new TextDecoder().decode(entries.get(main));
|
||||
resolveLocal = (href) => {
|
||||
const path = resolveKmzPath(href, main, entries);
|
||||
const type = path && kmzImageType(path);
|
||||
if (!type) return null;
|
||||
const url = URL.createObjectURL(
|
||||
new Blob([entries.get(path)], { type }),
|
||||
);
|
||||
urls.push(url);
|
||||
return url;
|
||||
};
|
||||
resolveLocal = kmzImageResolver(entries, main, (url) => urls.push(url));
|
||||
} else {
|
||||
text = new TextDecoder().decode(bytes);
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
stripGeoJsonStyleProperties,
|
||||
detectGeoFileFormat,
|
||||
importDisplayName,
|
||||
kmzImageResolver,
|
||||
kmzImageType,
|
||||
kmzMainDocument,
|
||||
parseGeoJsonText,
|
||||
@@ -406,12 +407,49 @@ test('KML placemarks and coordinate tuples are counted before loading', () => {
|
||||
assert.deepEqual(countKmlGeometry(doc), {
|
||||
features: 2,
|
||||
positions: 42,
|
||||
links: 0,
|
||||
over: '',
|
||||
});
|
||||
assert.equal(countKmlGeometry(doc, { maxPositions: 41 }).over, 'positions');
|
||||
assert.equal(countKmlGeometry(doc, { maxFeatures: 1 }).over, 'features');
|
||||
});
|
||||
|
||||
test('KML styles and icon links are counted against their own limit', () => {
|
||||
const style = () =>
|
||||
xmlEl('Style', {}, [
|
||||
xmlEl('IconStyle', {}, [xmlEl('Icon', {}, [xmlEl('href', {}, 'a.png')])]),
|
||||
]);
|
||||
const doc = xmlDoc(xmlEl('kml', {}, [style(), style(), style()]));
|
||||
assert.equal(countKmlGeometry(doc).links, 6);
|
||||
assert.equal(countKmlGeometry(doc, { maxLinks: 5 }).over, 'links');
|
||||
});
|
||||
|
||||
test('a KMZ image gets one blob URL however many hrefs name it', () => {
|
||||
const entries = new Map([
|
||||
['doc.kml', new Uint8Array(0)],
|
||||
['a.png', new Uint8Array(8)],
|
||||
['b.png', new Uint8Array(8)],
|
||||
]);
|
||||
const made = [];
|
||||
const kept = [];
|
||||
const resolve = kmzImageResolver(
|
||||
entries,
|
||||
'doc.kml',
|
||||
(url) => kept.push(url),
|
||||
(blob) => {
|
||||
made.push(blob);
|
||||
return `blob:test/${made.length}`;
|
||||
},
|
||||
);
|
||||
for (let i = 0; i < 1000; i++) assert.equal(resolve('a.png'), 'blob:test/1');
|
||||
assert.equal(resolve('./b.png'), 'blob:test/2');
|
||||
assert.equal(resolve('a.png'), 'blob:test/1');
|
||||
assert.equal(resolve('https://example.com/a.png'), null);
|
||||
assert.equal(resolve('doc.kml'), null);
|
||||
assert.equal(made.length, 2);
|
||||
assert.deepEqual(kept, ['blob:test/1', 'blob:test/2']);
|
||||
});
|
||||
|
||||
test('simplestyle is dropped: no icon fetched, the row color is what draws', () => {
|
||||
const fc = {
|
||||
type: 'FeatureCollection',
|
||||
|
||||
Reference in new issue
Block a user