fix(import): address review on metadata, bounds, navigation, clock and GPX

- Imported metadata stays inert: descriptions, native labels, properties and
  KML extended data are dropped from every entity; names are kept as short
  plain text. GeoJSON loads with a describe that builds no HTML, and
  simplestyle (including marker-symbol) is dropped, so no icon asset is
  fetched for a file and GeoJSON draws in its row's color.
- Bound geometry before loading: features and vertices are counted in the
  parsed GeoJSON, the KML document and while reading GPX, and a file over
  20,000 features or 250,000 vertices is refused before Cesium builds it.
- Imports run one batch at a time, so concurrent picks or drops cannot pass
  the 12-file cap.
- Fly-to framing walks long tracks by index instead of spreading them into a
  call.
- Fly-to is an immediate navigation through the application shell; the fly
  after an import is a deferred navigation stamped when the batch starts and
  re-asserted just before flying.
- Dated imports never change the shared clock: imported sources expose no
  clock, entities are available for all time, and a KML gx:Track is drawn as
  its line.
- GPX is read into GeoJSON with one line per track segment, without times.
- KML/KMZ rows show a hatched swatch (they keep their own styles); keyboard
  focus is kept on the same control after hide/show and moves to the next
  row after remove.
- Unit regressions for each, and the rendered QA now imports hostile
  metadata, dated KML/GPX and a two-segment track, and fails on any request
  for a file's contents or icon assets, a clock change, or surviving
  metadata.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
kapturnerandClaude Opus 5.5 committed 2026-10-07 08:50:35 -05:00
1 parent 2fec1091d4
commit 644a341b4c
11 files changed
+1199 -75

No files matched your search

+9 -5
View File
@@ -30,11 +30,15 @@
under NLOD 2.0.
- Import your own GeoJSON, KML, KMZ and GPX files (DISPLAY ▸ Import, or drop
them onto the globe). Each file is drawn on the ground in its own color with
a row to show, hide, fly to or remove it. Files stay in the browser for the
session, and KML/KMZ network links, remote icons, remote styles and balloon
HTML are left out so an imported file cannot make the page fetch anything.
The import half of #890.
them onto the globe). GeoJSON and GPX files are drawn on the ground in the
color shown on their row; KML and KMZ keep the styles written in them. Each
file gets a row to show, hide, fly to or remove it. Files stay in the
browser for the session. An import cannot make the page fetch anything (KML
network links, remote icons and styles are left out), keeps only plain-text
names from the file (no descriptions, balloons or native labels), never
moves the app's clock (dated tracks draw for all time), and keeps GPX track
segments apart. Size limits are checked before anything is built. The import
half of #890.
## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features)
+9 -6
View File
@@ -4108,13 +4108,16 @@ are omitted rather than framing the wrong part of the globe.
### Import Your Own Files (October 2026)
- DISPLAY ▸ **Import** puts GeoJSON (`.geojson`/`.json`), KML, KMZ and GPX files on the globe. Files come in through the Import button (a hidden `<input type=file multiple>`) or by dropping them onto the globe; the format is taken from the extension, then from the first bytes. Each file becomes its own Cesium data source (`GeoJsonDataSource`, `KmlDataSource`, `GpxDataSource`), clamped to the ground, in its own color from the whiteboard palette, with a row to show/hide it, fly to it, or remove it. The first file of a batch that loads is flown to. Fly-to frames the features' longitudes and latitudes at the surface and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`). `viewer.flyTo` is not used here: before the destination's terrain streams in, it frames ground-clamped points kilometers below the surface. Imported files last for the page session: a reload starts empty, and nothing is uploaded or stored.
- Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`.
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML. KMZ images are served from blob URLs. The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request to it.
- DISPLAY ▸ **Import** puts GeoJSON (`.geojson`/`.json`), KML, KMZ and GPX files on the globe. Files come in through the Import button (a hidden `<input type=file multiple>`) or by dropping them onto the globe; the format is taken from the extension, then from the first bytes. Each file becomes its own Cesium data source, clamped to the ground, with a row to show/hide it, fly to it, or remove it. GeoJSON and GPX are drawn in the row's color from the whiteboard palette (simplestyle properties in GeoJSON are dropped); KML and KMZ keep the styles written in them, and their row shows a hatched swatch instead of a color. GPX is read by `gpxToGeoJson` and loaded as GeoJSON: waypoints are points, each route a line, and each track one `MultiLineString` with a line per `<trkseg>`, so a pause in a recording stays a gap. Imported files last for the page session: a reload starts empty, and nothing is uploaded or stored.
- **Navigation.** The row's Fly-to is an immediate navigation through the application shell (`runImmediateNavigation('file', …)`): Cockpit refuses it, and the current follow/track owner is released first. The fly after an import is a deferred navigation: `beginDeferredLocationNavigation()` stamps it when the batch starts and `reassertDeferredLocationNavigation()` re-checks it just before flying, so a camera gesture, search, Cockpit or any newer navigation while files load wins and the import does not fly. Fly-to frames the features' longitudes and latitudes at the surface (thinned to 5,000 points by index, never spread into a call) and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`); `viewer.flyTo` is not used, because before terrain streams in it frames ground-clamped points kilometers below the surface.
- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time, and a time-dynamic KML position (`gx:Track`) is drawn as the line it travels. GPX times are not read at all.
- Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer, navigation: styleManager })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`.
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, drawn locally, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs. The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only.
- **KMZ** is read by a small zip reader (`readZipEntries`: stored and deflated entries through the platform `DecompressionStream`; no encryption, no ZIP64). The main document is `doc.kml`, else the shallowest `.kml`; relative hrefs resolve only to entries inside the archive.
- **Bounds.** A file may be at most 50 MB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may create at most 20,000 entities (`MAX_IMPORT_ENTITIES`), and at most 12 files are on the globe at once (`MAX_IMPORTED_FILES`; files over that are not read). A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
- File names are rendered with `textContent` only.
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`.
- **Bounds.** A file may be at most 50 MB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may hold at most 20,000 features (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
- **Keyboard.** Hide, show and remove rebuild the list without losing focus: the same control on the same row keeps it, a removed row hands it to the next row (else the Import button).
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
### Recent Imagery (September 2026)
+134 -12
View File
@@ -5,11 +5,16 @@
*
* Proves: GeoJSON, KML, KMZ (with a packed icon and ground overlay) and GPX
* each become one row and one data source; the imported shape actually paints
* on the globe and stops painting when hidden; a KML/KMZ never makes the page
* request anything named in it (network link, remote icon, remote style); a
* dropped file imports like a picked one; a file that is not geodata is refused
* with a reason and leaves nothing behind; Remove gives every data source back;
* no page errors.
* on the globe in its row's color (a GeoJSON `stroke`/`fill` does not override
* it) and stops painting when hidden; no import makes the page request
* anything named in it (network link, remote icon, remote style, palette
* icon, balloon or extended-data HTML, maki marker) or anything outside the
* page for it; no entity keeps a description, label or properties; dated KML
* and GPX leave the app's clock alone and still draw now; a GPX track keeps
* its two segments apart; keyboard focus survives hide/show; a dropped file
* imports like a picked one; a file that is not geodata is refused with a
* reason and leaves nothing behind; Remove gives every data source back; no
* page errors.
*
* Usage: dev server up (`npm run dev`), then `node scripts/qa-geo-import.mjs`.
* QA_BASE_URL (default http://localhost:4173), QA_SHOT_DIR, QA_HEADFUL=1.
@@ -40,24 +45,39 @@ const geojson = JSON.stringify({
features: [
{
type: 'Feature',
properties: { name: 'QA park' },
// A file's own simplestyle and HTML must not survive the import.
properties: {
name: 'QA park',
stroke: '#ff0000',
fill: '#ff0000',
'fill-opacity': 1,
description: `<img src="https://${TRAP_HOST}/props.png">`,
},
geometry: { type: 'Polygon', coordinates: [ring] },
},
{
type: 'Feature',
properties: { name: 'QA pin' },
properties: { name: 'QA pin', 'marker-symbol': 'rail' },
geometry: { type: 'Point', coordinates: [-122.4862, 37.7694] },
},
],
});
const kml = (iconHref) => `<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2">
<Document>
<Style id="s"><IconStyle><Icon><href>${iconHref}</href></Icon></IconStyle></Style>
<Style id="palette"><IconStyle><Icon><href>root://icons/palette-4.png</href><gx:x>32</gx:x><gx:y>64</gx:y></Icon></IconStyle></Style>
<NetworkLink><Link><href>https://${TRAP_HOST}/link.kml</href></Link></NetworkLink>
<Placemark><name>QA KML pin</name><styleUrl>#s</styleUrl>
<description><![CDATA[<img src="https://${TRAP_HOST}/balloon.png">]]></description>
<TimeSpan><begin>2019-01-01T00:00:00Z</begin><end>2019-01-02T00:00:00Z</end></TimeSpan>
<ExtendedData><Data name="note"><value><![CDATA[<iframe src="https://${TRAP_HOST}/extended.html"></iframe>]]></value></Data></ExtendedData>
<Point><coordinates>-122.47,37.77,0</coordinates></Point></Placemark>
<Placemark><name>QA palette pin</name><styleUrl>#palette</styleUrl>
<Point><coordinates>-122.48,37.771,0</coordinates></Point></Placemark>
<Placemark><name>QA moving</name>
<gx:Track><when>2019-01-01T00:00:00Z</when><when>2019-01-01T01:00:00Z</when>
<gx:coord>-122.49 37.767 0</gx:coord><gx:coord>-122.475 37.772 0</gx:coord></gx:Track></Placemark>
<Placemark><name>QA remote style</name><styleUrl>https://${TRAP_HOST}/styles.kml#x</styleUrl>
<LineString><coordinates>-122.50,37.766 -122.46,37.772</coordinates></LineString></Placemark>
<GroundOverlay><Icon><href>https://${TRAP_HOST}/overlay.png</href></Icon>
@@ -67,8 +87,10 @@ const kml = (iconHref) => `<?xml version="1.0" encoding="UTF-8"?>
const gpx = `<?xml version="1.0" encoding="UTF-8"?>
<gpx version="1.1" creator="qa" xmlns="http://www.topografix.com/GPX/1/1">
<wpt lat="37.769" lon="-122.49"><name>QA waypoint</name></wpt>
<trk><name>QA track</name><trkseg>
<trkpt lat="37.766" lon="-122.505"/><trkpt lat="37.768" lon="-122.49"/><trkpt lat="37.772" lon="-122.47"/>
<trk><name>QA track</name><desc>&lt;img src="https://${TRAP_HOST}/gpx.png"&gt;</desc><trkseg>
<trkpt lat="37.766" lon="-122.505"><time>2018-06-01T10:00:00Z</time></trkpt><trkpt lat="37.768" lon="-122.49"><time>2018-06-01T10:05:00Z</time></trkpt>
</trkseg><trkseg>
<trkpt lat="37.770" lon="-122.48"><time>2018-06-01T11:00:00Z</time></trkpt><trkpt lat="37.772" lon="-122.47"><time>2018-06-01T11:05:00Z</time></trkpt>
</trkseg></trk>
</gpx>`;
// 1×1 PNG.
@@ -149,8 +171,12 @@ const page = await browser.newPage();
const errors = [];
const trapped = [];
page.on('pageerror', (error) => errors.push(`pageerror: ${error.message}`));
// Anything a file names, and the icon assets Cesium would load on a file's
// behalf (KML palette icons from maps.google.com, GeoJSON maki markers).
const FILE_FETCH = [TRAP_HOST, 'maps.google.com/mapfiles', '/maki/'];
page.on('request', (request) => {
if (request.url().includes(TRAP_HOST)) trapped.push(request.url());
if (FILE_FETCH.some((needle) => request.url().includes(needle)))
trapped.push(request.url());
});
let failures = 0;
function check(name, passed, detail = '') {
@@ -250,6 +276,23 @@ try {
)),
);
const cyanBefore = await cyanPixels();
const clock = () =>
page.evaluate(() => {
const c = window.__godsEyeView.viewer.clock;
const iso = (t) => window.__CESIUM__.JulianDate.toIso8601(t, 0);
return {
start: iso(c.startTime),
stop: iso(c.stopTime),
range: c.clockRange,
step: c.clockStep,
multiplier: c.multiplier,
animate: c.shouldAnimate,
// Wall-clock drift aside, the current time must not jump to 2018/2019.
year: Number(iso(c.currentTime).slice(0, 4)),
};
});
const clockBefore = await clock();
const sourcesBefore = before.dataSources;
// Picked through the real <input type=file>.
const input = await page.$('#geo-import-input');
@@ -286,6 +329,65 @@ try {
/Added 4 files\..*left out/.test(after.hint),
after.hint,
);
const clockAfter = await clock();
check(
"dated KML and GPX leave the app's clock alone",
JSON.stringify(clockAfter) === JSON.stringify(clockBefore) &&
(await page.evaluate(
() =>
window.__godsEyeView.viewer.clockTrackedDataSource?.clock ===
undefined,
)),
`${JSON.stringify(clockBefore)} → ${JSON.stringify(clockAfter)}`,
);
const inside = await page.evaluate((from) => {
const viewer = window.__godsEyeView.viewer;
const now = viewer.clock.currentTime;
const sources = [];
for (let i = from; i < viewer.dataSources.length; i++)
sources.push(viewer.dataSources.get(i));
const entities = sources.flatMap((ds) => ds.entities.values);
const named = (name) => entities.find((e) => e.name === name);
const gpx = sources[3].entities.values;
return {
sources: sources.length,
clocks: sources.filter((ds) => ds.clock !== undefined).length,
described: entities.filter((e) => e.description !== undefined).length,
labelled: entities.filter((e) => e.label !== undefined).length,
withProperties: entities.filter((e) => e.properties !== undefined).length,
withKml: entities.filter((e) => e.kml !== undefined).length,
hidden: entities.filter((e) => !e.isAvailable(now)).length,
datedPinNow: Boolean(named('QA KML pin')?.isAvailable(now)),
movingIsLine: Boolean(
named('QA moving')?.polyline && !named('QA moving')?.position,
),
gpxLines: gpx.filter((e) => e.polyline).length,
gpxPoints: gpx.filter((e) => e.billboard || e.point).length,
};
}, sourcesBefore);
check(
'no imported entity keeps a description, label, properties or KML data',
inside.sources === 4 &&
inside.described +
inside.labelled +
inside.withProperties +
inside.withKml ===
0,
JSON.stringify(inside),
);
check(
'no imported source has a clock, and dated features draw now',
inside.clocks === 0 &&
inside.hidden === 0 &&
inside.datedPinNow &&
inside.movingIsLine,
JSON.stringify(inside),
);
check(
'the GPX track keeps its two segments apart',
inside.gpxLines === 2 && inside.gpxPoints === 1,
JSON.stringify(inside),
);
await page.evaluate((view) => {
const viewer = window.__godsEyeView.viewer;
viewer.camera.cancelFlight?.();
@@ -325,6 +427,26 @@ try {
(await cyanPixels()) > cyanHidden * 4,
);
// Keyboard: focus stays on the same control when the list is rebuilt.
const toggleSelector = `#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`;
await page.focus(toggleSelector);
await page.keyboard.press('Enter');
const focusAfterHide = await page.evaluate(
(sel) => document.activeElement === document.querySelector(sel),
toggleSelector,
);
await page.keyboard.press('Enter');
const focusAfterShow = await page.evaluate(
(sel) =>
document.activeElement === document.querySelector(sel) &&
document.activeElement.getAttribute('aria-pressed') === 'true',
toggleSelector,
);
check(
'keyboard focus stays on the toggle through hide and show',
focusAfterHide && focusAfterShow,
);
// Dropped onto the globe, as a person drags from their file manager.
await page.evaluate((text) => {
const container = window.__godsEyeView.viewer.container;
@@ -387,7 +509,7 @@ try {
);
await wait(1500);
check(
'nothing named inside the KML/KMZ was ever requested',
'nothing named inside a file, and no icon asset for one, was ever requested',
trapped.length === 0,
trapped.join(' '),
);
+4 -1
View File
@@ -60,7 +60,10 @@ export function createApplicationTools({
// DISPLAY ▸ Import: the person's own GeoJSON/KML/KMZ/GPX files, each its
// own data source. They live for the page session, so the application
// lifetime removes them and the drop listeners on the viewer container.
const geoFileImport = initGeoFileImport({ viewer });
const geoFileImport = initGeoFileImport({
viewer,
navigation: styleManager,
});
defer(() => geoFileImport?.destroy());
// DATA ▸ Recent Imagery: the box tool claims the pointer like Draw and the
// panel lives on the right rail, so both belong to the application
+403 -17
View File
@@ -8,6 +8,23 @@
* before Cesium sees the document (`sanitizeKmlDocument`). Images packed
* inside a KMZ are served to Cesium from blob URLs the caller revokes.
*
* What the file says about itself stays inert: descriptions, balloons,
* extended data and GeoJSON properties are dropped from every entity, names
* are kept only as short plain text, and no native Cesium label is drawn
* (`scrubImportedEntities`). GeoJSON and GPX draw in the row's color:
* simplestyle properties are dropped, including `marker-symbol`, so no icon
* asset is loaded on the file's behalf. KML and KMZ keep their own styles.
*
* An import never moves the app's shared clock: dated KML (TimeSpan,
* TimeStamp, gx:Track) is drawn for all time, and the data source exposes no
* clock for the viewer to adopt (`detachImportedTime`). GPX is read here
* rather than by Cesium (`gpxToGeoJson`), keeping each track segment as its
* own line and leaving times out.
*
* Size is checked before Cesium builds anything (`countGeoJsonGeometry`,
* `countKmlGeometry`): a file with too many features or vertices is refused
* up front instead of after the entities exist.
*
* The bounded pieces — format detection, the KMZ (zip) reader, the size and
* count limits — are plain functions so they run under `node --test`; the
* Cesium loaders need a browser DOM and are exercised by
@@ -23,6 +40,10 @@ export const MAX_KMZ_INFLATED_BYTES = 120 * 1024 * 1024;
export const MAX_IMPORTED_FILES = 12;
/** Most entities one file may create before it is refused. */
export const MAX_IMPORT_ENTITIES = 20_000;
/** Most vertices (points of every line, ring and marker) in one file. */
export const MAX_IMPORT_POSITIONS = 250_000;
/** Longest plain-text name kept from a file's features. */
export const MAX_IMPORT_NAME_LENGTH = 120;
/** What the file picker offers. */
export const GEO_FILE_ACCEPT = '.geojson,.json,.kml,.kmz,.gpx';
@@ -127,6 +148,112 @@ export function parseGeoJsonText(text) {
return value;
}
/**
* Count features and vertices in a parsed GeoJSON object without recursion
* limits or array spreading, stopping as soon as a limit is passed.
* @param {object} geojson
* @param {{maxFeatures?: number, maxPositions?: number}} [limits]
* @returns {{features: number, positions: number, over: ''|'features'|'positions'}}
*/
export function countGeoJsonGeometry(
geojson,
{
maxFeatures = MAX_IMPORT_ENTITIES,
maxPositions = MAX_IMPORT_POSITIONS,
} = {},
) {
let features = 0;
let positions = 0;
const stack = [geojson];
while (stack.length) {
const node = stack.pop();
if (!node || typeof node !== 'object') continue;
if (node.type === 'FeatureCollection') {
const list = Array.isArray(node.features) ? node.features : [];
for (let i = 0; i < list.length; i++) stack.push(list[i]);
continue;
}
if (node.type === 'Feature') {
features += 1;
if (features > maxFeatures)
return { features, positions, over: 'features' };
stack.push(node.geometry);
continue;
}
if (node.type === 'GeometryCollection') {
const list = Array.isArray(node.geometries) ? node.geometries : [];
for (let i = 0; i < list.length; i++) stack.push(list[i]);
continue;
}
if (GEOJSON_TYPES.has(node.type)) {
// A bare geometry is one feature: Cesium makes one entity for it.
if (node === geojson) features += 1;
// Walk the coordinate arrays iteratively: a position is an array whose
// first item is a number.
const coords = [node.coordinates];
while (coords.length) {
const c = coords.pop();
if (!Array.isArray(c)) continue;
if (typeof c[0] === 'number') {
positions += 1;
if (positions > maxPositions)
return { features, positions, over: 'positions' };
} else for (let i = 0; i < c.length; i++) coords.push(c[i]);
}
}
}
return { features, positions, over: '' };
}
/** The refusal for a file over a size limit, in the person's terms. */
function overLimitError({ over, features, positions }) {
if (over === 'features')
return new GeoImportError(
`That file has more than ${MAX_IMPORT_ENTITIES.toLocaleString()} features (the limit).`,
);
return new GeoImportError(
`That file has more than ${MAX_IMPORT_POSITIONS.toLocaleString()} points (the limit)${features ? ` across ${features.toLocaleString()} features` : ''}.`,
);
}
// simplestyle-spec keys Cesium honors. `marker-symbol` would load a maki icon
// asset on the file's behalf; the colors would override the row's color.
const GEOJSON_STYLE_KEYS = Object.freeze([
'marker-symbol',
'marker-color',
'marker-size',
'stroke',
'stroke-opacity',
'stroke-width',
'fill',
'fill-opacity',
]);
/**
* Drop GeoJSON simplestyle properties, so a GeoJSON file draws in its row's
* color like a GPX file and no icon is fetched for it. Mutates and returns
* the object.
* @param {object} geojson
* @returns {object}
*/
export function stripGeoJsonStyleProperties(geojson) {
const stack = [geojson];
while (stack.length) {
const node = stack.pop();
if (!node || typeof node !== 'object') continue;
if (node.type === 'FeatureCollection' && Array.isArray(node.features))
for (let i = 0; i < node.features.length; i++)
stack.push(node.features[i]);
else if (
node.type === 'Feature' &&
node.properties &&
typeof node.properties === 'object'
)
for (const key of GEOJSON_STYLE_KEYS) delete node.properties[key];
}
return geojson;
}
/** Inflate raw-deflate bytes with the platform's stream, stopping at `maxBytes`. */
async function inflateRaw(bytes, maxBytes) {
const stream = new Blob([bytes])
@@ -326,6 +453,253 @@ export function sanitizeKmlDocument(doc, resolveLocal = () => null) {
return { removed };
}
/** Child elements of `el` with this local name (any namespace). */
const childrenNamed = (el, localName) => {
const out = [];
const kids = el?.childNodes || [];
for (let i = 0; i < kids.length; i++)
if (kids[i].nodeType === 1 && kids[i].localName === localName)
out.push(kids[i]);
return out;
};
const childText = (el, localName) =>
(childrenNamed(el, localName)[0]?.textContent || '').trim();
/**
* Count placemarks and vertices in a parsed KML document before Cesium reads
* it: every `<coordinates>` tuple and every `gx:coord`.
* @param {Document} doc
* @returns {{features: number, positions: number, over: ''|'features'|'positions'}}
*/
export function countKmlGeometry(
doc,
{
maxFeatures = MAX_IMPORT_ENTITIES,
maxPositions = MAX_IMPORT_POSITIONS,
} = {},
) {
let features = 0;
let positions = 0;
for (const el of doc.getElementsByTagName('*')) {
const name = el.localName;
if (name === 'Placemark') {
features += 1;
if (features > maxFeatures)
return { features, positions, over: 'features' };
} else if (name === 'coordinates') {
const text = el.textContent || '';
// Tuples are separated by whitespace; count runs without splitting
// a large string into a large array.
let inTuple = false;
for (let i = 0; i < text.length; i++) {
const space = text.charCodeAt(i) <= 32;
if (!space && !inTuple) {
positions += 1;
if (positions > maxPositions)
return { features, positions, over: 'positions' };
}
inTuple = !space;
}
} else if (name === 'coord') {
positions += 1;
if (positions > maxPositions)
return { features, positions, over: 'positions' };
}
}
return { features, positions, over: '' };
}
/**
* Read a parsed GPX document into GeoJSON: waypoints become points, each
* route a line and each track one MultiLineString with a line per `<trkseg>`
* (segment breaks are gaps in the recording and stay gaps). Only names are
* kept, and no times, so a dated track draws without a clock. Throws
* `GeoImportError` past the size limits.
* @param {Document} doc
* @returns {object} A FeatureCollection.
*/
export function gpxToGeoJson(
doc,
{
maxFeatures = MAX_IMPORT_ENTITIES,
maxPositions = MAX_IMPORT_POSITIONS,
} = {},
) {
const root = doc.documentElement;
if (root?.localName !== 'gpx')
throw new GeoImportError('That GPX file has no <gpx> document.');
const features = [];
let positions = 0;
const point = (el) => {
const lat = Number(el.getAttribute('lat'));
const lon = Number(el.getAttribute('lon'));
if (
!Number.isFinite(lat) ||
!Number.isFinite(lon) ||
Math.abs(lat) > 90 ||
Math.abs(lon) > 180
)
return null;
positions += 1;
if (positions > maxPositions)
throw overLimitError({
over: 'positions',
features: features.length,
positions,
});
return [lon, lat];
};
const add = (name, geometry) => {
if (features.length >= maxFeatures)
throw overLimitError({
over: 'features',
features: features.length + 1,
positions,
});
features.push({
type: 'Feature',
properties: name ? { name } : {},
geometry,
});
};
for (const wpt of childrenNamed(root, 'wpt')) {
const at = point(wpt);
if (at) add(childText(wpt, 'name'), { type: 'Point', coordinates: at });
}
for (const rte of childrenNamed(root, 'rte')) {
const line = [];
for (const pt of childrenNamed(rte, 'rtept')) {
const at = point(pt);
if (at) line.push(at);
}
if (line.length >= 2)
add(childText(rte, 'name'), { type: 'LineString', coordinates: line });
}
for (const trk of childrenNamed(root, 'trk')) {
const lines = [];
for (const seg of childrenNamed(trk, 'trkseg')) {
const line = [];
for (const pt of childrenNamed(seg, 'trkpt')) {
const at = point(pt);
if (at) line.push(at);
}
if (line.length >= 2) lines.push(line);
}
if (lines.length)
add(childText(trk, 'name'), {
type: 'MultiLineString',
coordinates: lines,
});
}
return { type: 'FeatureCollection', features };
}
/** A feature name as short plain text (no control characters), or undefined. */
export function importFeatureName(value) {
const text = String(value ?? '')
.replace(/[\u0000-\u001f\u007f]/g, ' ')
.replace(/\s+/g, ' ')
.trim();
if (!text) return undefined;
return text.length > MAX_IMPORT_NAME_LENGTH
? `${text.slice(0, MAX_IMPORT_NAME_LENGTH - 1)}…`
: text;
}
/**
* Make every entity of an imported data source inert: no description,
* balloon, extended data or properties, no native label, and a name that is
* short plain text. Returns how many entities were touched.
* @param {{entities: {values: object[]}}} dataSource
* @returns {number}
*/
export function scrubImportedEntities(dataSource) {
const list = dataSource?.entities?.values || [];
for (let i = 0; i < list.length; i++) {
const entity = list[i];
let name;
try {
name = importFeatureName(
typeof entity.name === 'string' ? entity.name : undefined,
);
} catch {
name = undefined;
}
entity.name = name;
entity.description = undefined;
entity.label = undefined;
entity.properties = undefined;
if (entity.kml && typeof entity.kml === 'object') entity.kml = undefined;
}
return list.length;
}
const TRACK_SAMPLES = 2000;
/**
* Take time out of an imported data source so adding it cannot move the app's
* shared clock: the data source reports no clock (the viewer adopts the clock
* of every added source that has one), every entity is available for all
* time, and a time-dynamic position (a KML gx:Track) is drawn as the line it
* travels instead of a moving marker.
* @param {object} dataSource
* @param {{Cesium?: object, color?: object}} [options]
* @returns {object} The same data source.
*/
export function detachImportedTime(
dataSource,
{ Cesium: C = Cesium, color } = {},
) {
Object.defineProperty(dataSource, 'clock', {
configurable: true,
enumerable: false,
get: () => undefined,
set: () => {},
});
const list = dataSource?.entities?.values || [];
for (let i = 0; i < list.length; i++) {
const entity = list[i];
const span = entity.availability;
const position = entity.position;
if (position && position.isConstant === false) {
const start = span?.start;
const stop = span?.stop;
const line = [];
if (start && stop && C.JulianDate.lessThan(start, stop)) {
const seconds = C.JulianDate.secondsDifference(stop, start);
for (let n = 0; n <= TRACK_SAMPLES; n++) {
const t = C.JulianDate.addSeconds(
start,
(seconds * n) / TRACK_SAMPLES,
new C.JulianDate(),
);
const at = position.getValue(t);
if (at) line.push(at);
}
}
entity.path = undefined;
entity.model = undefined;
if (line.length >= 2 && !entity.polyline) {
entity.billboard = undefined;
entity.point = undefined;
entity.position = undefined;
entity.polyline = new C.PolylineGraphics({
positions: line,
clampToGround: true,
width: 3,
material: color || C.Color.WHITE,
});
} else {
entity.position = line.length
? new C.ConstantPositionProperty(line[0])
: undefined;
}
}
entity.availability = undefined;
}
return dataSource;
}
function parseXml(text, what) {
const doc = new DOMParser().parseFromString(
String(text).replace(/^/, ''),
@@ -364,22 +738,27 @@ export async function loadGeoFile(file, { viewer, color = '#39d0ff' }) {
let dataSource;
let removed = 0;
try {
if (format === 'geojson') {
dataSource = await Cesium.GeoJsonDataSource.load(
parseGeoJsonText(new TextDecoder().decode(bytes)),
{
clampToGround: true,
stroke,
fill: stroke.withAlpha(0.25),
strokeWidth: 3,
markerColor: stroke,
},
);
} else if (format === 'gpx') {
dataSource = await Cesium.GpxDataSource.load(
parseXml(new TextDecoder().decode(bytes), 'GPX'),
{ clampToGround: true, trackColor: stroke, routeColor: stroke },
);
if (format === 'geojson' || format === 'gpx') {
let geojson;
if (format === 'gpx') {
geojson = gpxToGeoJson(
parseXml(new TextDecoder().decode(bytes), 'GPX'),
);
} else {
geojson = parseGeoJsonText(new TextDecoder().decode(bytes));
const size = countGeoJsonGeometry(geojson);
if (size.over) throw overLimitError(size);
stripGeoJsonStyleProperties(geojson);
}
dataSource = await Cesium.GeoJsonDataSource.load(geojson, {
clampToGround: true,
stroke,
fill: stroke.withAlpha(0.25),
strokeWidth: 3,
markerColor: stroke,
// No HTML description table is built from the file's properties.
describe: () => undefined,
});
} else {
let text;
let resolveLocal = () => null;
@@ -402,6 +781,8 @@ export async function loadGeoFile(file, { viewer, color = '#39d0ff' }) {
text = new TextDecoder().decode(bytes);
}
const doc = parseXml(text, format.toUpperCase());
const size = countKmlGeometry(doc);
if (size.over) throw overLimitError(size);
removed = sanitizeKmlDocument(doc, resolveLocal).removed;
dataSource = await Cesium.KmlDataSource.load(doc, {
camera: viewer.scene.camera,
@@ -427,6 +808,11 @@ export async function loadGeoFile(file, { viewer, color = '#39d0ff' }) {
revoke();
throw new GeoImportError('That file has nothing to draw.');
}
// GpxDataSource's `name` is read-only, so the display name travels beside it.
scrubImportedEntities(dataSource);
// Only KML has time-dynamic positions (GPX is read without times); its
// track line is drawn in KML's default white, like its other unstyled lines.
detachImportedTime(dataSource);
// The display name travels beside the data source (the file name, not
// anything the document says about itself).
return { dataSource, name, format, entityCount, removed, revoke };
}
+289
View File
@@ -1,8 +1,17 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { deflateRawSync } from 'node:zlib';
import * as Cesium from 'cesium';
import {
GeoImportError,
MAX_IMPORT_POSITIONS,
countGeoJsonGeometry,
countKmlGeometry,
detachImportedTime,
gpxToGeoJson,
importFeatureName,
scrubImportedEntities,
stripGeoJsonStyleProperties,
detectGeoFileFormat,
importDisplayName,
kmzImageType,
@@ -213,3 +222,283 @@ test('KMZ hrefs resolve only to entries inside the archive', () => {
assert.equal(kmzImageType('files/a.PNG'), 'image/png');
assert.equal(kmzImageType('files/a.svg'), null);
});
/* ── a tiny element tree with the DOM surface the readers use ─────────── */
function xmlEl(localName, attrs = {}, kids = []) {
const el = {
nodeType: 1,
localName,
childNodes: [],
getAttribute: (n) => (n in attrs ? String(attrs[n]) : null),
};
for (const k of typeof kids === 'string' ? [] : kids) el.childNodes.push(k);
const text = typeof kids === 'string' ? kids : null;
Object.defineProperty(el, 'textContent', {
get: () => text ?? el.childNodes.map((c) => c.textContent).join(''),
});
return el;
}
const xmlDoc = (root) => ({
documentElement: root,
getElementsByTagName(name) {
assert.equal(name, '*');
const out = [];
const walk = (el) => {
out.push(el);
el.childNodes.forEach(walk);
};
walk(root);
return out;
},
});
const trkpt = (lat, lon, time) =>
xmlEl('trkpt', { lat, lon }, time ? [xmlEl('time', {}, time)] : []);
test('GPX keeps each track segment as its own line, names as text, no times', () => {
const doc = xmlDoc(
xmlEl('gpx', {}, [
xmlEl('wpt', { lat: 37.77, lon: -122.49 }, [xmlEl('name', {}, 'Camp')]),
xmlEl('wpt', { lat: 99, lon: 0 }), // out of range: skipped
xmlEl('rte', {}, [
xmlEl('name', {}, 'Route'),
xmlEl('rtept', { lat: 1, lon: 2 }),
xmlEl('rtept', { lat: 3, lon: 4 }),
]),
xmlEl('trk', {}, [
xmlEl('name', {}, 'Morning walk'),
xmlEl('trkseg', {}, [
trkpt(10, 20, '2024-05-01T10:00:00Z'),
trkpt(10.1, 20.1, '2024-05-01T10:01:00Z'),
]),
// A pause in the recording: a new segment, not a straight jump.
xmlEl('trkseg', {}, [
trkpt(11, 21, '2024-05-01T11:00:00Z'),
trkpt(11.1, 21.1, '2024-05-01T11:01:00Z'),
trkpt(11.2, 21.2, '2024-05-01T11:02:00Z'),
]),
xmlEl('trkseg', {}, [trkpt(12, 22)]), // one point: not a line
]),
]),
);
const fc = gpxToGeoJson(doc);
assert.deepEqual(
fc.features.map((f) => [f.properties.name, f.geometry.type]),
[
['Camp', 'Point'],
['Route', 'LineString'],
['Morning walk', 'MultiLineString'],
],
);
assert.deepEqual(fc.features[2].geometry.coordinates, [
[
[20, 10],
[20.1, 10.1],
],
[
[21, 11],
[21.1, 11.1],
[21.2, 11.2],
],
]);
assert.doesNotMatch(JSON.stringify(fc), /2024|time/);
assert.throws(() => gpxToGeoJson(xmlDoc(xmlEl('kml'))), GeoImportError);
});
test('GPX past the point limit is refused while reading, before Cesium', () => {
const seg = xmlEl(
'trkseg',
{},
Array.from({ length: 30 }, (_, i) => trkpt(1 + i / 100, 2)),
);
const doc = xmlDoc(xmlEl('gpx', {}, [xmlEl('trk', {}, [seg])]));
assert.throws(
() => gpxToGeoJson(doc, { maxPositions: 20 }),
(e) => e instanceof GeoImportError && /more than .* points/.test(e.message),
);
assert.throws(
() =>
gpxToGeoJson(
xmlDoc(
xmlEl('gpx', {}, [
xmlEl('wpt', { lat: 1, lon: 1 }),
xmlEl('wpt', { lat: 2, lon: 2 }),
]),
),
{ maxFeatures: 1 },
),
/more than .* features/,
);
});
test('GeoJSON features and vertices are counted before loading, without recursion', () => {
const line = Array.from({ length: 50 }, (_, i) => [i / 10, 1]);
const fc = {
type: 'FeatureCollection',
features: [
{ type: 'Feature', geometry: { type: 'LineString', coordinates: line } },
{
type: 'Feature',
geometry: {
type: 'GeometryCollection',
geometries: [
{ type: 'Point', coordinates: [1, 2] },
{
type: 'Polygon',
coordinates: [
[
[0, 0],
[1, 0],
[1, 1],
[0, 0],
],
],
},
],
},
},
{ type: 'Feature', geometry: null },
],
};
assert.deepEqual(countGeoJsonGeometry(fc), {
features: 3,
positions: 55,
over: '',
});
assert.equal(
countGeoJsonGeometry(fc, { maxPositions: 54 }).over,
'positions',
);
assert.equal(countGeoJsonGeometry(fc, { maxFeatures: 2 }).over, 'features');
assert.deepEqual(
countGeoJsonGeometry({ type: 'Point', coordinates: [1, 2] }),
{
features: 1,
positions: 1,
over: '',
},
);
// A very long line is counted iteratively and stops at the limit.
const huge = {
type: 'LineString',
coordinates: Array.from({ length: MAX_IMPORT_POSITIONS + 5 }, () => [0, 0]),
};
assert.equal(countGeoJsonGeometry(huge).over, 'positions');
});
test('KML placemarks and coordinate tuples are counted before loading', () => {
const coords = (n) =>
xmlEl(
'coordinates',
{},
Array.from({ length: n }, (_, i) => `${i},1,0`).join('\n '),
);
const doc = xmlDoc(
xmlEl('kml', {}, [
xmlEl('Placemark', {}, [xmlEl('LineString', {}, [coords(40)])]),
xmlEl('Placemark', {}, [
xmlEl('Track', {}, [
xmlEl('coord', {}, '1 2 0'),
xmlEl('coord', {}, '1 3 0'),
]),
]),
]),
);
assert.deepEqual(countKmlGeometry(doc), {
features: 2,
positions: 42,
over: '',
});
assert.equal(countKmlGeometry(doc, { maxPositions: 41 }).over, 'positions');
assert.equal(countKmlGeometry(doc, { maxFeatures: 1 }).over, 'features');
});
test('simplestyle is dropped: no icon fetched, the row color is what draws', () => {
const fc = {
type: 'FeatureCollection',
features: [
{
type: 'Feature',
properties: {
'marker-symbol': 'rail',
'marker-color': '#f00',
stroke: '#0f0',
'stroke-width': 9,
fill: '#00f',
'fill-opacity': 1,
name: 'x',
},
geometry: null,
},
{ type: 'Feature', properties: null, geometry: null },
],
};
stripGeoJsonStyleProperties(fc);
assert.deepEqual(fc.features[0].properties, { name: 'x' });
});
test('imported entities keep only a short plain-text name: no description, label or properties', () => {
const ds = new Cesium.CustomDataSource('t');
const e = ds.entities.add({
name: 'Trail\u0007 <b>head</b>\n' + 'x'.repeat(300),
description: '<img src="https://example.invalid/x.png" onerror="alert(1)">',
label: { text: 'Trail head' },
properties: { note: '<script>alert(1)</script>' },
position: Cesium.Cartesian3.fromDegrees(1, 2),
});
e.kml = { extendedData: { note: { value: '<iframe src=//x>' } } };
ds.entities.add({ name: 42, position: Cesium.Cartesian3.fromDegrees(1, 2) });
assert.equal(scrubImportedEntities(ds), 2);
assert.equal(e.description, undefined);
assert.equal(e.label, undefined);
assert.equal(e.properties, undefined);
assert.equal(e.kml, undefined);
assert.ok(e.name.startsWith('Trail <b>head</b> x'));
assert.ok(e.name.length <= 120 && !/[\u0000-\u001f]/.test(e.name));
assert.equal(ds.entities.values[1].name, undefined);
assert.equal(importFeatureName(' '), undefined);
});
test('a dated import exposes no clock and draws for all time', () => {
// A data source with its own clock, as KmlDataSource has for a dated file.
class DatedSource extends Cesium.CustomDataSource {
get clock() {
return this._testClock;
}
}
const ds = new DatedSource('dated');
ds._testClock = new Cesium.DataSourceClock();
const start = Cesium.JulianDate.fromIso8601('2020-01-01T00:00:00Z');
const stop = Cesium.JulianDate.fromIso8601('2020-01-01T01:00:00Z');
const span = new Cesium.TimeIntervalCollection([
new Cesium.TimeInterval({ start, stop }),
]);
const track = new Cesium.SampledPositionProperty();
track.addSample(start, Cesium.Cartesian3.fromDegrees(10, 10));
track.addSample(stop, Cesium.Cartesian3.fromDegrees(10.1, 10.1));
const mover = ds.entities.add({
availability: span,
position: track,
path: {},
billboard: {},
});
const pin = ds.entities.add({
availability: span,
position: Cesium.Cartesian3.fromDegrees(1, 1),
});
assert.ok(ds.clock);
detachImportedTime(ds);
assert.equal(ds.clock, undefined);
// The viewer reads `clock` from every added source; setting it is ignored.
ds.clock = new Cesium.DataSourceClock();
assert.equal(ds.clock, undefined);
assert.equal(pin.availability, undefined);
assert.ok(pin.isAvailable(Cesium.JulianDate.now()));
// The moving marker became the line it travels, visible now.
assert.equal(mover.availability, undefined);
assert.equal(mover.position, undefined);
assert.equal(mover.path, undefined);
assert.equal(mover.billboard, undefined);
const line = mover.polyline.positions.getValue(Cesium.JulianDate.now());
assert.ok(line.length > 100);
assert.ok(mover.polyline.clampToGround.getValue());
});
+35 -15
View File
@@ -12,27 +12,47 @@ import { guardCameraAboveGround } from '../cameraGroundGuard.js';
const MAX_POINTS = 5000;
/** Longitude/latitude pairs of every entity's position, line and polygon. */
/**
* Longitude/latitude pairs of every entity's position, line and polygon,
* thinned to at most `MAX_POINTS`. Long tracks are walked by index, never
* spread into a call, so a file with hundreds of thousands of vertices cannot
* overflow the stack; only the kept points are converted.
*/
export function dataSourcePoints(dataSource, time = Cesium.JulianDate.now()) {
const cartesians = [];
const parts = [];
let total = 0;
for (const entity of dataSource?.entities?.values ?? []) {
const at = entity.position?.getValue?.(time);
if (at) cartesians.push(at);
if (at) {
parts.push([at]);
total += 1;
}
const line = entity.polyline?.positions?.getValue?.(time);
if (Array.isArray(line)) cartesians.push(...line);
const hierarchy = entity.polygon?.hierarchy?.getValue?.(time);
if (Array.isArray(hierarchy?.positions))
cartesians.push(...hierarchy.positions);
if (Array.isArray(line) && line.length) {
parts.push(line);
total += line.length;
}
const ring = entity.polygon?.hierarchy?.getValue?.(time)?.positions;
if (Array.isArray(ring) && ring.length) {
parts.push(ring);
total += ring.length;
}
}
const stride = Math.max(1, Math.ceil(cartesians.length / MAX_POINTS));
const stride = Math.max(1, Math.ceil(total / MAX_POINTS));
const points = [];
for (let i = 0; i < cartesians.length; i += stride) {
const carto = Cesium.Cartographic.fromCartesian(cartesians[i]);
if (!carto) continue;
points.push([
Cesium.Math.toDegrees(carto.longitude),
Cesium.Math.toDegrees(carto.latitude),
]);
let index = 0;
for (const part of parts) {
// Keep every stride-th vertex across all parts, plus each part's ends so
// a short line is never skipped entirely.
for (let i = 0; i < part.length; i++, index++) {
if (index % stride !== 0 && i !== 0 && i !== part.length - 1) continue;
const carto = Cesium.Cartographic.fromCartesian(part[i]);
if (!carto) continue;
points.push([
Cesium.Math.toDegrees(carto.longitude),
Cesium.Math.toDegrees(carto.latitude),
]);
}
}
return points;
}
+17
View File
@@ -57,3 +57,20 @@ test('the camera frames the footprint at the surface, never a buried center', ()
);
assert.equal(calls.at(-1), 'viewer.flyTo');
});
test('a very long track is framed without spreading it into a call', () => {
// 400k vertices: `push(...line)` throws RangeError (too many arguments)
// well below this; the walk keeps a bounded sample and both ends.
const n = 400_000;
const line = new Array(n);
for (let i = 0; i < n; i++)
line[i] = Cesium.Cartesian3.fromDegrees(-120 + (i / n) * 2, 40);
const ds = new Cesium.CustomDataSource('long');
ds.entities.add({ polyline: { positions: line } });
ds.entities.add({ position: Cesium.Cartesian3.fromDegrees(-110, 41) });
const points = dataSourcePoints(ds);
assert.ok(points.length <= 5002, `${points.length} points kept`);
const lons = points.map(([lon]) => lon);
assert.ok(Math.min(...lons) < -119.99 && Math.max(...lons) > -110.01);
assert.ok(lons.some((lon) => Math.abs(lon - -118.000005) < 0.01));
});
+95 -15
View File
@@ -8,6 +8,14 @@
* is accepted and what is stripped from KML before Cesium reads it.
*
* Imported files last for the page session: a reload starts empty.
*
* Imports run one batch at a time (a second pick or drop waits for the
* first), so the file cap is checked against what is really on the globe.
* Fly-to goes through the app's navigation policy (`navigation`, the
* application shell): the button is an immediate navigation, and the fly
* after an import is a deferred one, stamped when the import starts and
* re-asserted just before flying, so a camera move, Cockpit or any newer
* navigation in between wins over a slow file.
* `destroy()` removes every imported data source, revokes every blob URL and
* gives back every listener and the `window.__gevGeoImport` handle.
*/
@@ -35,6 +43,7 @@ export function initGeoFileImport({
load = loadGeoFile,
document: doc = globalThis.document,
flyToSource = flyToDataSource,
navigation = null,
}) {
const button = doc?.getElementById('geo-import-button');
const input = doc?.getElementById('geo-import-input');
@@ -44,6 +53,7 @@ export function initGeoFileImport({
const imports = [];
const listeners = [];
let queue = Promise.resolve();
let seq = 0;
let colorAt = 0;
let destroyed = false;
@@ -59,20 +69,46 @@ export function initGeoFileImport({
};
const render = () => viewer.scene?.requestRender?.();
function renderList() {
/** Which row button has focus, so a re-render can give it back. */
function focusedAction() {
const active = doc.activeElement;
if (!active || !listEl.contains?.(active)) return null;
const row = active.closest?.('[data-import-id]');
return row
? { id: row.dataset.importId, action: active.dataset?.action }
: null;
}
function restoreFocus(was) {
if (!was) return;
const rows = [...listEl.querySelectorAll('[data-import-id]')];
let row = rows.find((r) => r.dataset.importId === was.id);
// The row is gone (removed): the next row's same control, else the
// previous row's, else the Import button.
if (!row && was.index != null)
row = rows[Math.min(was.index, rows.length - 1)];
const target =
row?.querySelector?.(`button[data-action="${was.action}"]`) ||
(row ? row.querySelector?.('button') : button);
target?.focus?.();
}
function renderList(keep = focusedAction()) {
listEl.replaceChildren(
...imports.map((item) => {
const row = doc.createElement('li');
row.className = 'geo-import-row';
row.dataset.importId = item.id;
const swatch = doc.createElement('span');
swatch.className = 'geo-import-swatch';
swatch.style.background = item.color;
// KML and KMZ draw with their own styles; their swatch says so
// instead of claiming a color the globe does not show.
swatch.className = `geo-import-swatch${item.ownStyles ? ' own-styles' : ''}`;
if (!item.ownStyles) swatch.style.background = item.color;
const name = doc.createElement('span');
name.className = 'geo-import-name';
// textContent, never markup: the name comes from the file system.
name.textContent = item.name;
name.title = `${item.name} — ${item.format.toUpperCase()}, ${item.entityCount.toLocaleString()} feature${item.entityCount === 1 ? '' : 's'}`;
name.title = `${item.name} — ${item.format.toUpperCase()}, ${item.entityCount.toLocaleString()} feature${item.entityCount === 1 ? '' : 's'}${item.ownStyles ? ', drawn with its own styles' : ''}`;
const actions = [
[
item.visible ? 'visibility' : 'visibility_off',
@@ -102,6 +138,7 @@ export function initGeoFileImport({
}),
);
listEl.classList.toggle('visible', imports.length > 0);
restoreFocus(keep);
}
const find = (id) => imports.find((item) => item.id === id) || null;
@@ -117,25 +154,52 @@ export function initGeoFileImport({
return true;
}
/** Fly the camera to one imported file. */
// Not viewer.flyTo: it frames ground-clamped points underground until the
// destination's terrain has loaded (see ./flyToDataSource.js).
const fly = (item) => {
if (!item.visible) setVisible(item.id, true);
flyToSource(viewer, item.dataSource);
return true;
};
/**
* Fly the camera to one imported file: an immediate navigation, so Cockpit
* can refuse it and the current follow/track owner is released first.
*/
function flyTo(id) {
const item = find(id);
if (!item) return false;
if (!item.visible) setVisible(id, true);
// Not viewer.flyTo: it frames ground-clamped points underground until
// the destination's terrain has loaded (see ./flyToDataSource.js).
flyToSource(viewer, item.dataSource);
return true;
if (typeof navigation?.runImmediateNavigation === 'function')
return (
navigation.runImmediateNavigation('file', () => fly(item)) !== false
);
return fly(item);
}
/** Stamp a deferred fly for an import batch; false when refused. */
const beginAutoFly = () =>
typeof navigation?.beginDeferredLocationNavigation === 'function'
? navigation.beginDeferredLocationNavigation()
: null;
/** Re-assert the stamp right before flying; false when superseded. */
const autoFlyStillOwns = (generation) => {
if (generation === false) return false;
if (generation === null) return true;
return typeof navigation?.reassertDeferredLocationNavigation === 'function'
? navigation.reassertDeferredLocationNavigation(generation) === true
: true;
};
/** Remove one imported file from the globe and the list. */
function remove(id) {
const at = imports.findIndex((item) => item.id === id);
if (at < 0) return false;
const keep = focusedAction();
if (keep) keep.index = at;
const [item] = imports.splice(at, 1);
viewer.dataSources?.remove?.(item.dataSource, true);
item.revoke?.();
renderList();
renderList(keep);
render();
return true;
}
@@ -146,9 +210,19 @@ export function initGeoFileImport({
* @returns {Promise<Array<{name: string, ok: boolean, id?: string,
* removed?: number, error?: string}>>}
*/
async function importFiles(files) {
const outcomes = [];
function importFiles(files) {
const list = [...(files || [])];
const run = queue.then(() => importBatch(list));
queue = run.catch(() => {});
return run;
}
async function importBatch(list) {
const outcomes = [];
if (destroyed || !list.length) return outcomes;
// Stamped now, re-asserted before the fly: whatever navigation happens
// while the files load (a drag, a search, Cockpit) supersedes this one.
const generation = beginAutoFly();
for (const file of list) {
if (destroyed) break;
const fileName = String(file?.name || 'file');
@@ -186,6 +260,7 @@ export function initGeoFileImport({
id: `import-${seq}`,
name: loaded.name || fileName,
format: loaded.format,
ownStyles: loaded.format === 'kml' || loaded.format === 'kmz',
entityCount: loaded.entityCount,
removed: loaded.removed || 0,
color,
@@ -208,8 +283,13 @@ export function initGeoFileImport({
id: item.id,
removed: item.removed,
});
// Fly to the first file of a batch that lands, so a drop shows its result.
if (outcomes.filter((o) => o.ok).length === 1) flyTo(item.id);
// Fly to the first file of a batch that lands, so a drop shows its
// result, unless something else took the camera meanwhile.
if (
outcomes.filter((o) => o.ok).length === 1 &&
autoFlyStillOwns(generation)
)
fly(item);
}
if (!destroyed) setHint(summarize(outcomes));
return outcomes;
+194 -4
View File
@@ -30,6 +30,9 @@ class FakeClassList {
}
}
// The one focused element across every fake document (tests run one at a time).
const focus = { active: null };
class FakeElement {
constructor(tagName = 'div', id = '') {
this.tagName = tagName.toUpperCase();
@@ -91,6 +94,32 @@ class FakeElement {
this.clicks += 1;
this.emit('click');
}
focus() {
focus.active = this;
}
contains(el) {
for (let at = el; at; at = at.parent) if (at === this) return true;
return false;
}
*walk() {
for (const kid of this.children) {
yield kid;
yield* kid.walk();
}
}
querySelectorAll(selector) {
if (selector !== '[data-import-id]') throw new Error(`fake: ${selector}`);
return [...this.walk()].filter((el) => el.dataset.importId);
}
querySelector(selector) {
const exact = /^button\[data-action="([a-z]+)"\]$/.exec(selector);
for (const el of this.walk()) {
if (el.tagName !== 'BUTTON') continue;
if (selector === 'button' || (exact && el.dataset.action === exact[1]))
return el;
}
return null;
}
closest(selector) {
for (let el = this; el; el = el.parent) {
if (
@@ -105,7 +134,8 @@ class FakeElement {
}
}
function setup({ load } = {}) {
function setup({ load, navigation, flyToSource } = {}) {
focus.active = null;
const els = {
'geo-import-button': new FakeElement('button', 'geo-import-button'),
'geo-import-input': new FakeElement('input', 'geo-import-input'),
@@ -116,6 +146,9 @@ function setup({ load } = {}) {
body: new FakeElement('body'),
getElementById: (id) => els[id] ?? null,
createElement: (tag) => new FakeElement(tag),
get activeElement() {
return focus.active;
},
};
const added = [];
const removed = [];
@@ -155,6 +188,8 @@ function setup({ load } = {}) {
viewer,
load: load || defaultLoad,
document,
...(navigation ? { navigation } : {}),
...(flyToSource ? { flyToSource } : {}),
});
return {
tool,
@@ -362,6 +397,8 @@ test('a file that finishes loading after destroy is released, not added', async
},
});
const pending = t.tool.importFiles([file('late.kml')]);
// Imports are queued: let the batch start loading before destroy lands.
await new Promise((r) => setImmediate(r));
t.tool.destroy();
release();
await pending;
@@ -377,10 +414,13 @@ test('the application composition wires the import and owns its teardown', () =>
const drawAt = tools.indexOf(
'const drawTool = initDrawTool({ viewer, annotations });',
);
const importAt = tools.indexOf(
'const geoFileImport = initGeoFileImport({ viewer });',
);
const importAt = tools.indexOf('const geoFileImport = initGeoFileImport({');
assert.ok(importAt > drawAt && drawAt >= 0);
// Fly-to goes through the application shell's navigation policy.
assert.match(
tools.slice(importAt, importAt + 200),
/navigation: styleManager/,
);
assert.ok(tools.indexOf('defer(() => geoFileImport?.destroy());') > importAt);
const markup = fs.readFileSync(
new URL('./templates/display-controls.html', import.meta.url),
@@ -397,3 +437,153 @@ test('the application composition wires the import and owns its teardown', () =>
`${id} missing from the DISPLAY template`,
);
});
test('two imports at once never pass the file cap: batches run one after another', async () => {
const t = setup();
const first = t.tool.importFiles(
Array.from({ length: 8 }, (_, i) => file(`a${i}.geojson`)),
);
const second = t.tool.importFiles(
Array.from({ length: 8 }, (_, i) => file(`b${i}.geojson`)),
);
const [a, b] = await Promise.all([first, second]);
assert.equal(t.tool.list().length, MAX_IMPORTED_FILES);
assert.equal(a.filter((o) => o.ok).length, 8);
assert.equal(b.filter((o) => o.ok).length, MAX_IMPORTED_FILES - 8);
assert.equal(b.filter((o) => !o.ok).length, 16 - MAX_IMPORTED_FILES);
// The second batch's loads started only after the first batch finished.
assert.deepEqual(
t.loads.map((l) => l.file.name.slice(0, 1)).join(''),
'a'.repeat(8) + 'b'.repeat(MAX_IMPORTED_FILES - 8),
);
});
function navigationDouble({ immediate = true, reassert = true } = {}) {
const calls = [];
let generation = 0;
return {
calls,
runImmediateNavigation(noun, navigate) {
calls.push(['immediate', noun]);
return immediate ? navigate(++generation) : false;
},
beginDeferredLocationNavigation() {
calls.push(['begin']);
return ++generation;
},
reassertDeferredLocationNavigation(g) {
calls.push(['reassert', g]);
return reassert;
},
};
}
test('Fly to is an immediate navigation that Cockpit can refuse', async () => {
const flown = [];
const nav = navigationDouble({ immediate: false });
const t = setup({ navigation: nav, flyToSource: (v, ds) => flown.push(ds) });
// Batch auto-fly is deferred: stamped at the start, re-asserted before flying.
await t.tool.importFiles([file('a.geojson')]);
assert.deepEqual(nav.calls, [['begin'], ['reassert', 1]]);
assert.equal(flown.length, 1);
// The button goes through the immediate route; refused, nothing flies.
const id = t.tool.list()[0].id;
assert.equal(t.tool.flyTo(id), false);
assert.deepEqual(nav.calls.at(-1), ['immediate', 'file']);
assert.equal(flown.length, 1);
const nav2 = navigationDouble();
const t2 = setup({
navigation: nav2,
flyToSource: (v, ds) => flown.push(ds),
});
await t2.tool.importFiles([file('b.geojson')]);
assert.equal(t2.tool.flyTo(t2.tool.list()[0].id), true);
assert.equal(flown.length, 3);
});
test('a slow import does not fly once something newer took the camera', async () => {
let release;
const gate = new Promise((r) => (release = r));
const flown = [];
const nav = navigationDouble({ reassert: false });
const t = setup({
navigation: nav,
flyToSource: (v, ds) => flown.push(ds),
load: async (f) => {
await gate;
return {
dataSource: {},
name: f.name,
format: 'geojson',
entityCount: 1,
removed: 0,
revoke() {},
};
},
});
const pending = t.tool.importFiles([file('slow.geojson')]);
release();
const [out] = await pending;
assert.equal(out.ok, true);
assert.deepEqual(nav.calls, [['begin'], ['reassert', 1]]);
assert.equal(flown.length, 0);
});
test('a refused deferred stamp (Cockpit) imports without flying', async () => {
const flown = [];
const nav = navigationDouble();
nav.beginDeferredLocationNavigation = () => false;
nav.reassertDeferredLocationNavigation = () => {
throw new Error('must not re-assert a refused stamp');
};
const t = setup({ navigation: nav, flyToSource: (v, ds) => flown.push(ds) });
const [out] = await t.tool.importFiles([file('a.geojson')]);
assert.equal(out.ok, true);
assert.equal(flown.length, 0);
});
test('keyboard focus stays on the control after hide, show and remove', async () => {
const t = setup();
await t.tool.importFiles([file('a.geojson'), file('b.geojson')]);
// Clicks reach the list's delegated listener, as they bubble in a page.
const press = (btn) =>
t.els['geo-import-list'].emit('click', { target: btn });
const [first] = rows(t.els);
action(first, 'toggle').focus();
press(action(first, 'toggle'));
// The list was rebuilt; focus is on the NEW toggle of the same row.
const again = rows(t.els)[0];
assert.notEqual(again, first);
assert.equal(t.document.activeElement, action(again, 'toggle'));
assert.equal(action(again, 'toggle').getAttribute('aria-pressed'), 'false');
press(action(again, 'toggle'));
assert.equal(t.document.activeElement, action(rows(t.els)[0], 'toggle'));
// Remove the first row: focus moves to the same control on the next row.
action(rows(t.els)[0], 'remove').focus();
press(action(rows(t.els)[0], 'remove'));
assert.equal(rows(t.els).length, 1);
assert.equal(t.document.activeElement, action(rows(t.els)[0], 'remove'));
// Remove the last row: focus goes back to the Import button.
press(action(rows(t.els)[0], 'remove'));
assert.equal(t.document.activeElement, t.els['geo-import-button']);
});
test('KML and KMZ rows say they keep their own styles instead of showing a color', async () => {
const t = setup({
load: async (f) => ({
dataSource: {},
name: f.name,
format: f.name.endsWith('.kml') ? 'kml' : 'gpx',
entityCount: 2,
removed: 0,
revoke() {},
}),
});
await t.tool.importFiles([file('a.kml'), file('b.gpx')]);
const [kml, gpx] = rows(t.els);
assert.ok(kml.children[0].classList.contains('own-styles'));
assert.equal(kml.children[0].style.background, undefined);
assert.match(kml.children[1].title, /drawn with its own styles/);
assert.ok(!gpx.children[0].classList.contains('own-styles'));
assert.ok(gpx.children[0].style.background);
});
+10
View File
@@ -1031,6 +1031,16 @@ body.gev-drawing #cesiumContainer canvas {
height: 8px;
border-radius: 2px;
}
/* KML/KMZ draw with the styles written in the file, so the swatch shows a
hatch instead of a color the globe does not use. */
.geo-import-swatch.own-styles {
background: repeating-linear-gradient(
45deg,
var(--text-dim) 0 1px,
transparent 1px 3px
);
box-shadow: inset 0 0 0 1px var(--text-dim);
}
.geo-import-name {
flex: 1 1 auto;
min-width: 0;