Integrate file-import contribution onto current main for HAL-526 review

This commit is contained in:
Mazeyar Moeini Feizabadi committed 2026-10-14 00:50:18 +02:00
commit 7694c02b91
15 files changed
+3212 -2

No files matched your search

+11
View File
@@ -161,6 +161,17 @@
largest cities) and `CCTV_VEGVESEN_VIDEO=0` keeps stills only. Attributed
under NLOD 2.0.
- Import your own GeoJSON, KML, KMZ and GPX files (DISPLAY ▸ Import, or drop
them onto the globe). GeoJSON and GPX files are drawn on the ground in the
color shown on their row; KML and KMZ keep the styles written in them. Each
file gets a row to show, hide, fly to or remove it. Files stay in the
browser for the session. An import cannot make the page fetch anything (KML
network links, remote icons and styles are left out), keeps only plain-text
names from the file (no descriptions, balloons or native labels), never
moves the app's clock (dated tracks draw for all time), and keeps GPX track
segments apart. Size limits are checked before anything is built. The import
half of #890.
## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features)
- Ship each bundled data pack once. The region, marine, admin-boundary,
+13
View File
@@ -4322,6 +4322,19 @@ are omitted rather than framing the wrong part of the globe.
- Pure half: `src/annotations/drawMode.js` (session, minimum vertices, double-click de-dup at 0.5 m, vertex ceiling, degenerate-geometry refusal, ring closing, dateline-safe length/area/centroid, hint text). Regression surface: `src/annotations/drawMode.test.mjs` and `src/data/inputOwnership.test.mjs` (pure), `src/annotations/drawToolBehaviour.test.mjs` (the tool driven against DOM and viewer doubles — leases, borrowed input actions, teardown, the Enter key, preview heights), `src/annotations/drawTool.test.mjs` (structural: where files live, which ids the markup carries, what the docs may claim), and the rendered harness `scripts/qa-draw-tool.mjs`.
- The annotation tree is a declared ownership group: `./annotations` → `src/annotations/index.js` in `package.json`, with its module graph listed in `scripts/package-boundaries.json`, so an annotation module reaching outside that graph is a gate failure rather than a surprise in a layer's bundle. `drawMode.js`, `drawTool.js` and their tests are in `scripts/format-scope.json`.
### Import Your Own Files (October 2026)
- DISPLAY ▸ **Import** puts GeoJSON (`.geojson`/`.json`), KML, KMZ and GPX files on the globe. Files come in through the Import button (a hidden `<input type=file multiple>`) or by dropping them onto the globe; the format is taken from the extension, then from the first bytes. Each file becomes its own Cesium data source, clamped to the ground, with a row to show/hide it, fly to it, or remove it. GeoJSON and GPX are drawn in the row's color from the whiteboard palette (simplestyle properties in GeoJSON are dropped); KML and KMZ keep the styles written in them, and their row shows a hatched swatch instead of a color. GPX is read by `gpxToGeoJson` and loaded as GeoJSON: waypoints are points, each route a line, and each track one `MultiLineString` with a line per `<trkseg>`, so a pause in a recording stays a gap. Imported files last for the page session: a reload starts empty, and nothing is uploaded or stored.
- **Navigation.** The row's Fly-to is an immediate navigation through the application shell (`runImmediateNavigation('file', …)`): Cockpit refuses it, and the current follow/track owner is released first. The fly after an import is a deferred navigation: `beginDeferredLocationNavigation()` stamps it when the batch starts and `reassertDeferredLocationNavigation()` re-checks it just before flying, so a camera gesture, search, Cockpit or any newer navigation while files load wins and the import does not fly. Fly-to frames the features' longitudes and latitudes at the surface (thinned to 5,000 points by index, never spread into a call) and lets the camera ground guard lift the eye once the terrain has loaded (`src/ui/flyToDataSource.js`); `viewer.flyTo` is not used, because before terrain streams in it frames ground-clamped points kilometers below the surface.
- **Clock.** An import never changes the app's shared clock. The viewer adopts the clock of every data source added to it, so each imported data source reports no clock (`detachImportedTime`); every entity is available for all time, and a time-dynamic KML position (`gx:Track`) is drawn as the line it travels. GPX times are not read at all.
- Runtime entry: `createApplicationTools` (`src/app/tools.js`) calls `initGeoFileImport({ viewer, navigation: styleManager })` (`src/ui/geoFileImport.js`) after the draw tool and defers its `destroy()`, which removes every imported data source, revokes every blob URL, and gives back the drop listeners on the viewer container and the `window.__gevGeoImport` handle (`importFiles`, `list`, `setVisible`, `flyTo`, `remove`, `diagnostics`). Parsing lives in `src/data/geoFileImport.js`.
- **An imported file cannot make the page fetch anything.** Before `KmlDataSource` reads a KML/KMZ document, `sanitizeKmlDocument` removes `NetworkLink`, `NetworkLinkControl`, `ScreenOverlay`, `PhotoOverlay`, `Model` and `Tour`, every `href` that is not an image packed in the same KMZ or a `data:image/` URL (an icon then falls back to Cesium's default pin, drawn locally, and a `GroundOverlay` left without an image is dropped), every `styleUrl` that is not a local `#id`, and placemark `description`/`BalloonStyle`/`Snippet` HTML. GeoJSON `marker-symbol` (a maki icon asset) is dropped. KMZ images are served from blob URLs, one per packed image however many hrefs name it (`kmzImageResolver`). The hint says when something was left out. `scripts/qa-geo-import.mjs` imports files that name a trap host in each of those places and fails on any request that leaves the app's own origin.
- **Imported metadata stays inert.** After loading, `scrubImportedEntities` removes every entity's description, native label, properties and KML extended data, and keeps the name only as plain text without control characters, at most 120 characters. GeoJSON is loaded with a `describe` that builds no HTML. The file name is shown with `textContent` only.
- **KMZ** is read by a small zip reader (`readZipEntries`: stored and deflated entries through the platform `DecompressionStream`; no encryption, no ZIP64). The main document is `doc.kml`, else the shallowest `.kml`; relative hrefs resolve only to entries inside the archive.
- **Bounds.** A file may be at most 50 MB (`MAX_IMPORT_BYTES`), a KMZ may inflate to at most 120 MB in total (`MAX_KMZ_INFLATED_BYTES`, the zip-bomb stop), one file may hold at most 20,000 features (`MAX_IMPORT_ENTITIES`) and 250,000 vertices (`MAX_IMPORT_POSITIONS`), and a KML/KMZ at most 2,000 `<href>` and `<Style>` elements (`MAX_KML_LINKS`), counted before Cesium builds anything (`countGeoJsonGeometry`, `countKmlGeometry`, and while reading GPX), and the entity count is checked again after loading. At most 12 files are on the globe at once (`MAX_IMPORTED_FILES`): imports run one batch at a time, so two picks or drops in quick succession cannot pass the cap; files over it are not read. A refused file names its reason in the hint and does not stop the rest of the batch; a file with nothing to draw is refused.
- **Keyboard.** Hide, show and remove rebuild the list without losing focus: the same control on the same row keeps it, a removed row hands it to the next row (else the Import button).
- Gate: `node scripts/qa-geo-import.mjs` (dev server up). Unit: `src/data/geoFileImport.test.mjs`, `src/ui/geoFileImport.test.mjs`, `src/ui/flyToDataSource.test.mjs`.
### Recent Imagery (September 2026)
- The layer sits in the Data Layers panel's Cameras group beside Cameras (CCTV); Mapped ALPR Cameras moved to the Infrastructure group to make room.
+1 -1
View File
@@ -13,7 +13,7 @@
with the sources: src/materialSymbolsSubset.test.mjs fails when a source
names a glyph this list is missing, because a missing glyph does not draw
a box — the ligature never forms and the element renders the literal word. -->
<link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20,400,0,0&icon_names=adjust,arrow_drop_down,arrow_forward,arrow_left,arrow_right,bolt,chevron_left,chevron_right,close,close_fullscreen,dark_mode,draw,east,flare,flight,layers_clear,light_mode,local_fire_department,my_location,navigation,normal,on,open_in_full,public,radar,radio,right_panel_close,right_panel_open,rocket_launch,skip_next,skip_previous,view_in_ar" rel="stylesheet" />
<link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20,400,0,0&icon_names=adjust,arrow_drop_down,arrow_forward,arrow_left,arrow_right,bolt,chevron_left,chevron_right,close,close_fullscreen,dark_mode,draw,east,flare,flight,layers_clear,light_mode,local_fire_department,my_location,navigation,normal,on,open_in_full,public,radar,radio,right_panel_close,right_panel_open,rocket_launch,skip_next,skip_previous,upload_file,view_in_ar,visibility,visibility_off" rel="stylesheet" />
</head>
<body>
<!-- gev:template scene-chrome -->
+6 -1
View File
@@ -26,6 +26,8 @@
"src/annotations/drawMode.test.mjs",
"src/annotations/drawTool.js",
"src/annotations/drawTool.test.mjs",
"src/data/geoFileImport.test.mjs",
"src/ui/geoFileImport.test.mjs",
"src/annotations/adminOutline.test.mjs",
"src/annotations/regionRing.test.mjs",
"src/data/adminBoundaries.test.mjs",
@@ -199,6 +201,7 @@
"src/styles/snow.js",
"scripts/qa-visual-effects.mjs",
"scripts/qa-draw-tool.mjs",
"scripts/qa-geo-import.mjs",
"src/ui/mapSource.js",
"src/ui/mapSourceControls.js",
"src/ui/mapSourceControls.test.mjs",
@@ -981,5 +984,7 @@
"src/tools/globePanel.test.mjs",
"src/costEndpointThrottle.test.mjs",
"src/sources/gbfsStations.test.mjs",
"scripts/qa-gbfs-names.mjs"
"scripts/qa-gbfs-names.mjs",
"src/ui/flyToDataSource.js",
"src/ui/flyToDataSource.test.mjs"
]
+3
View File
@@ -2006,6 +2006,9 @@
"src/data/inputOwnership.js",
"src/annotations/drawMode.js",
"src/annotations/drawTool.js",
"src/data/geoFileImport.js",
"src/ui/flyToDataSource.js",
"src/ui/geoFileImport.js",
"src/data/routeSteps.js",
"src/layers/directions/index.js",
"src/layers/directions/source.js",
+524
View File
@@ -0,0 +1,524 @@
#!/usr/bin/env node
/**
* Rendered acceptance for DISPLAY ▸ Import: real files through the real file
* input and a real drop, parsed by Cesium in the page.
*
* Proves: GeoJSON, KML, KMZ (with a packed icon and ground overlay) and GPX
* each become one row and one data source; the imported shape actually paints
* on the globe in its row's color (a GeoJSON `stroke`/`fill` does not override
* it) and stops painting when hidden; no import makes the page request
* anything named in it (network link, remote icon, remote style, palette
* icon, balloon or extended-data HTML, maki marker) or anything outside the
* page for it; no entity keeps a description, label or properties; dated KML
* and GPX leave the app's clock alone and still draw now; a GPX track keeps
* its two segments apart; keyboard focus survives hide/show; a dropped file
* imports like a picked one; a file that is not geodata is refused with a
* reason and leaves nothing behind; Remove gives every data source back; no
* page errors.
*
* Usage: dev server up (`npm run dev`), then `node scripts/qa-geo-import.mjs`.
* QA_BASE_URL (default http://localhost:4173), QA_SHOT_DIR, QA_HEADFUL=1.
*/
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import puppeteer from 'puppeteer';
const BASE_URL = process.env.QA_BASE_URL || 'http://localhost:4173';
const SHOT_DIR = process.env.QA_SHOT_DIR || 'qa-shots/geo-import';
const HEADFUL = process.env.QA_HEADFUL === '1';
// Golden Gate Park: a big, flat, green rectangle the test polygon can cover.
const VIEW = { lon: -122.4862, lat: 37.7694, height: 4200, pitch: -88 };
const TRAP_HOST = 'gev-qa-trap.invalid';
/* ── fixtures ──────────────────────────────────────────────────────────── */
const ring = [
[-122.511, 37.7645],
[-122.4545, 37.7645],
[-122.4545, 37.7745],
[-122.511, 37.7745],
[-122.511, 37.7645],
];
const geojson = JSON.stringify({
type: 'FeatureCollection',
features: [
{
type: 'Feature',
// A file's own simplestyle and HTML must not survive the import.
properties: {
name: 'QA park',
stroke: '#ff0000',
fill: '#ff0000',
'fill-opacity': 1,
description: `<img src="https://${TRAP_HOST}/props.png">`,
},
geometry: { type: 'Polygon', coordinates: [ring] },
},
{
type: 'Feature',
properties: { name: 'QA pin', 'marker-symbol': 'rail' },
geometry: { type: 'Point', coordinates: [-122.4862, 37.7694] },
},
],
});
const kml = (iconHref) => `<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2">
<Document>
<Style id="s"><IconStyle><Icon><href>${iconHref}</href></Icon></IconStyle></Style>
<Style id="palette"><IconStyle><Icon><href>root://icons/palette-4.png</href><gx:x>32</gx:x><gx:y>64</gx:y></Icon></IconStyle></Style>
<NetworkLink><Link><href>https://${TRAP_HOST}/link.kml</href></Link></NetworkLink>
<Placemark><name>QA KML pin</name><styleUrl>#s</styleUrl>
<description><![CDATA[<img src="https://${TRAP_HOST}/balloon.png">]]></description>
<TimeSpan><begin>2019-01-01T00:00:00Z</begin><end>2019-01-02T00:00:00Z</end></TimeSpan>
<ExtendedData><Data name="note"><value><![CDATA[<iframe src="https://${TRAP_HOST}/extended.html"></iframe>]]></value></Data></ExtendedData>
<Point><coordinates>-122.47,37.77,0</coordinates></Point></Placemark>
<Placemark><name>QA palette pin</name><styleUrl>#palette</styleUrl>
<Point><coordinates>-122.48,37.771,0</coordinates></Point></Placemark>
<Placemark><name>QA moving</name>
<gx:Track><when>2019-01-01T00:00:00Z</when><when>2019-01-01T01:00:00Z</when>
<gx:coord>-122.49 37.767 0</gx:coord><gx:coord>-122.475 37.772 0</gx:coord></gx:Track></Placemark>
<Placemark><name>QA remote style</name><styleUrl>https://${TRAP_HOST}/styles.kml#x</styleUrl>
<LineString><coordinates>-122.50,37.766 -122.46,37.772</coordinates></LineString></Placemark>
<GroundOverlay><Icon><href>https://${TRAP_HOST}/overlay.png</href></Icon>
<LatLonBox><north>37.77</north><south>37.76</south><east>-122.46</east><west>-122.47</west></LatLonBox></GroundOverlay>
</Document>
</kml>`;
const gpx = `<?xml version="1.0" encoding="UTF-8"?>
<gpx version="1.1" creator="qa" xmlns="http://www.topografix.com/GPX/1/1">
<wpt lat="37.769" lon="-122.49"><name>QA waypoint</name></wpt>
<trk><name>QA track</name><desc>&lt;img src="https://${TRAP_HOST}/gpx.png"&gt;</desc><trkseg>
<trkpt lat="37.766" lon="-122.505"><time>2018-06-01T10:00:00Z</time></trkpt><trkpt lat="37.768" lon="-122.49"><time>2018-06-01T10:05:00Z</time></trkpt>
</trkseg><trkseg>
<trkpt lat="37.770" lon="-122.48"><time>2018-06-01T11:00:00Z</time></trkpt><trkpt lat="37.772" lon="-122.47"><time>2018-06-01T11:05:00Z</time></trkpt>
</trkseg></trk>
</gpx>`;
// 1×1 PNG.
const PNG = Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFBQIAX8jx0gAAAABJRU5ErkJggg==',
'base64',
);
/** Stored (uncompressed) zip — enough for a KMZ fixture. */
function storedZip(files) {
const parts = [];
const central = [];
let offset = 0;
for (const [name, data] of files) {
const n = Buffer.from(name);
const local = Buffer.alloc(30);
local.writeUInt32LE(0x04034b50, 0);
local.writeUInt32LE(data.length, 18);
local.writeUInt32LE(data.length, 22);
local.writeUInt16LE(n.length, 26);
const c = Buffer.alloc(46);
c.writeUInt32LE(0x02014b50, 0);
c.writeUInt32LE(data.length, 20);
c.writeUInt32LE(data.length, 24);
c.writeUInt16LE(n.length, 28);
c.writeUInt32LE(offset, 42);
parts.push(local, n, data);
central.push(c, n);
offset += 30 + n.length + data.length;
}
const dir = Buffer.concat(central);
const end = Buffer.alloc(22);
end.writeUInt32LE(0x06054b50, 0);
end.writeUInt16LE(files.length, 8);
end.writeUInt16LE(files.length, 10);
end.writeUInt32LE(dir.length, 12);
end.writeUInt32LE(offset, 16);
return Buffer.concat([...parts, dir, end]);
}
const kmzDoc = kml('files/pin.png').replace(
`<href>https://${TRAP_HOST}/overlay.png</href>`,
'<href>files/overlay.png</href>',
);
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gev-qa-geo-import-'));
const fixture = (name, data) => {
const file = path.join(dir, name);
fs.writeFileSync(file, data);
return file;
};
const FILES = [
fixture('park.geojson', geojson),
fixture('remote-bits.kml', kml(`https://${TRAP_HOST}/icon.png`)),
fixture(
'packed.kmz',
storedZip([
['doc.kml', Buffer.from(kmzDoc)],
['files/pin.png', PNG],
['files/overlay.png', PNG],
]),
),
fixture('walk.gpx', gpx),
];
/* ── harness ───────────────────────────────────────────────────────────── */
const browser = await puppeteer.launch({
headless: !HEADFUL,
args: [
'--no-sandbox',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
...(process.platform === 'darwin'
? ['--use-angle=metal', '--enable-gpu']
: ['--use-gl=angle', '--use-angle=swiftshader']),
],
});
const page = await browser.newPage();
const errors = [];
const trapped = [];
page.on('pageerror', (error) => errors.push(`pageerror: ${error.message}`));
// Anything a file names, and the icon assets Cesium would load on a file's
// behalf (KML palette icons from maps.google.com, GeoJSON maki markers).
const FILE_FETCH = [TRAP_HOST, 'maps.google.com/mapfiles', '/maki/'];
page.on('request', (request) => {
if (FILE_FETCH.some((needle) => request.url().includes(needle)))
trapped.push(request.url());
});
let failures = 0;
function check(name, passed, detail = '') {
console.log(
`[${passed ? 'PASS' : 'FAIL'}] ${name}${detail ? ` — ${detail}` : ''}`,
);
if (!passed) failures++;
}
fs.mkdirSync(SHOT_DIR, { recursive: true });
const shot = (name) =>
page.screenshot({
path: path.join(SHOT_DIR, `${name}.jpg`),
type: 'jpeg',
quality: 82,
});
const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const CLIP = { x: 380, y: 240, width: 480, height: 320 };
/** Pixels in the clip close to the first import color (#39d0ff). */
async function cyanPixels() {
await page.evaluate(() => window.__godsEyeView.viewer.scene.requestRender());
await wait(600);
const base64 = await page.screenshot({
clip: CLIP,
type: 'png',
encoding: 'base64',
});
return page.evaluate(async (b64) => {
const bitmap = await createImageBitmap(
await (await fetch(`data:image/png;base64,${b64}`)).blob(),
);
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
const context = canvas.getContext('2d');
context.drawImage(bitmap, 0, 0);
const { data } = context.getImageData(0, 0, bitmap.width, bitmap.height);
let n = 0;
for (let i = 0; i < data.length; i += 4) {
const [r, g, b] = [data[i], data[i + 1], data[i + 2]];
if (b > 150 && g > 120 && r < 140 && b - r > 60) n += 1;
}
return n;
}, base64);
}
const state = () =>
page.evaluate(() => ({
files: window.__gevGeoImport.list(),
dataSources: window.__godsEyeView.viewer.dataSources.length,
rows: document.querySelectorAll('#geo-import-list .geo-import-row').length,
hint: document.getElementById('geo-import-hint')?.textContent || '',
}));
try {
await page.setViewport({ width: 1280, height: 860 });
await page.goto(`${BASE_URL}/?welcome=0`, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(
() =>
window.__gevGeoImport &&
window.__godsEyeView?.viewer &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 90_000 },
);
await page.evaluate((view) => {
const viewer = window.__godsEyeView.viewer;
viewer.camera.cancelFlight?.();
viewer.camera.setView({
destination: window.__CESIUM__.Cartesian3.fromDegrees(
view.lon,
view.lat,
view.height,
),
orientation: { heading: 0, pitch: (view.pitch * Math.PI) / 180, roll: 0 },
});
viewer.scene.requestRender();
}, VIEW);
await wait(4000);
await page.evaluate(() => {
const rail = document.getElementById('pp-toggles');
if (rail?.classList.contains('collapsed'))
document
.querySelector('.panel-collapse-btn[data-collapse-target="pp-toggles"]')
?.click();
});
await page.waitForFunction(
() =>
document.getElementById('geo-import-button')?.getBoundingClientRect()
.width > 0,
{ timeout: 10_000 },
);
const before = await state();
check(
'Import is on the DISPLAY rail with an empty, hidden list',
before.rows === 0 &&
!(await page.evaluate(() =>
document
.getElementById('geo-import-list')
.classList.contains('visible'),
)),
);
const cyanBefore = await cyanPixels();
const clock = () =>
page.evaluate(() => {
const c = window.__godsEyeView.viewer.clock;
const iso = (t) => window.__CESIUM__.JulianDate.toIso8601(t, 0);
return {
start: iso(c.startTime),
stop: iso(c.stopTime),
range: c.clockRange,
step: c.clockStep,
multiplier: c.multiplier,
animate: c.shouldAnimate,
// Wall-clock drift aside, the current time must not jump to 2018/2019.
year: Number(iso(c.currentTime).slice(0, 4)),
};
});
const clockBefore = await clock();
const sourcesBefore = before.dataSources;
// Picked through the real <input type=file>.
const input = await page.$('#geo-import-input');
await input.uploadFile(...FILES);
const imported = await page
.waitForFunction(() => window.__gevGeoImport.list().length === 4, {
timeout: 60_000,
})
.then(
() => true,
() => false,
);
const after = await state();
check(
'four picked files become four rows and four data sources',
imported &&
after.rows === 4 &&
after.dataSources === before.dataSources + 4,
JSON.stringify(
after.files.map((f) => `${f.name}:${f.format}:${f.entityCount}`),
),
);
check(
'formats are recognized',
after.files.map((f) => f.format).join() === 'geojson,kml,kmz,gpx',
);
check(
'the KML and KMZ had their outside references removed',
after.files[1].removed > 0 && after.files[2].removed > 0,
`removed ${after.files[1].removed}/${after.files[2].removed}`,
);
check(
'the hint says so',
/Added 4 files\..*left out/.test(after.hint),
after.hint,
);
const clockAfter = await clock();
check(
"dated KML and GPX leave the app's clock alone",
JSON.stringify(clockAfter) === JSON.stringify(clockBefore) &&
(await page.evaluate(
() =>
window.__godsEyeView.viewer.clockTrackedDataSource?.clock ===
undefined,
)),
`${JSON.stringify(clockBefore)} → ${JSON.stringify(clockAfter)}`,
);
const inside = await page.evaluate((from) => {
const viewer = window.__godsEyeView.viewer;
const now = viewer.clock.currentTime;
const sources = [];
for (let i = from; i < viewer.dataSources.length; i++)
sources.push(viewer.dataSources.get(i));
const entities = sources.flatMap((ds) => ds.entities.values);
const named = (name) => entities.find((e) => e.name === name);
const gpx = sources[3].entities.values;
return {
sources: sources.length,
clocks: sources.filter((ds) => ds.clock !== undefined).length,
described: entities.filter((e) => e.description !== undefined).length,
labelled: entities.filter((e) => e.label !== undefined).length,
withProperties: entities.filter((e) => e.properties !== undefined).length,
withKml: entities.filter((e) => e.kml !== undefined).length,
hidden: entities.filter((e) => !e.isAvailable(now)).length,
datedPinNow: Boolean(named('QA KML pin')?.isAvailable(now)),
movingIsLine: Boolean(
named('QA moving')?.polyline && !named('QA moving')?.position,
),
gpxLines: gpx.filter((e) => e.polyline).length,
gpxPoints: gpx.filter((e) => e.billboard || e.point).length,
};
}, sourcesBefore);
check(
'no imported entity keeps a description, label, properties or KML data',
inside.sources === 4 &&
inside.described +
inside.labelled +
inside.withProperties +
inside.withKml ===
0,
JSON.stringify(inside),
);
check(
'no imported source has a clock, and dated features draw now',
inside.clocks === 0 &&
inside.hidden === 0 &&
inside.datedPinNow &&
inside.movingIsLine,
JSON.stringify(inside),
);
check(
'the GPX track keeps its two segments apart',
inside.gpxLines === 2 && inside.gpxPoints === 1,
JSON.stringify(inside),
);
await page.evaluate((view) => {
const viewer = window.__godsEyeView.viewer;
viewer.camera.cancelFlight?.();
viewer.camera.setView({
destination: window.__CESIUM__.Cartesian3.fromDegrees(
view.lon,
view.lat,
view.height,
),
orientation: { heading: 0, pitch: (view.pitch * Math.PI) / 180, roll: 0 },
});
}, VIEW);
await wait(2500);
const cyanShown = await cyanPixels();
await shot('imported');
check(
'the imported polygon paints on the globe',
cyanShown > cyanBefore * 4,
`${cyanBefore} → ${cyanShown}`,
);
const parkId = after.files[0].id;
await page.click(
`#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`,
);
const cyanHidden = await cyanPixels();
check(
'hiding it stops it painting',
cyanHidden < cyanShown / 3,
`${cyanShown} → ${cyanHidden}`,
);
await page.click(
`#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`,
);
check(
'showing it again paints it again',
(await cyanPixels()) > cyanHidden * 4,
);
// Keyboard: focus stays on the same control when the list is rebuilt.
const toggleSelector = `#geo-import-list [data-import-id="${parkId}"] button[data-action="toggle"]`;
await page.focus(toggleSelector);
await page.keyboard.press('Enter');
const focusAfterHide = await page.evaluate(
(sel) => document.activeElement === document.querySelector(sel),
toggleSelector,
);
await page.keyboard.press('Enter');
const focusAfterShow = await page.evaluate(
(sel) =>
document.activeElement === document.querySelector(sel) &&
document.activeElement.getAttribute('aria-pressed') === 'true',
toggleSelector,
);
check(
'keyboard focus stays on the toggle through hide and show',
focusAfterHide && focusAfterShow,
);
// Dropped onto the globe, as a person drags from their file manager.
await page.evaluate((text) => {
const container = window.__godsEyeView.viewer.container;
const transfer = new DataTransfer();
transfer.items.add(
new File([text], 'dropped.geojson', { type: 'application/geo+json' }),
);
for (const type of ['dragenter', 'dragover', 'drop'])
container.dispatchEvent(
new DragEvent(type, {
dataTransfer: transfer,
bubbles: true,
cancelable: true,
}),
);
}, geojson);
const dropped = await page
.waitForFunction(() => window.__gevGeoImport.list().length === 5, {
timeout: 20_000,
})
.then(
() => true,
() => false,
);
check('a file dropped on the globe imports', dropped);
// Not geodata: refused with a reason, nothing added.
await page.evaluate(() => {
const transfer = new DataTransfer();
transfer.items.add(
new File(['just some notes'], 'notes.txt', { type: 'text/plain' }),
);
window.__godsEyeView.viewer.container.dispatchEvent(
new DragEvent('drop', {
dataTransfer: transfer,
bubbles: true,
cancelable: true,
}),
);
});
await wait(800);
const refused = await state();
check(
'a file that is not geodata is refused with a reason',
refused.files.length === 5 &&
/notes\.txt: Only GeoJSON, KML, KMZ and GPX/.test(refused.hint),
refused.hint,
);
// Remove every file through its row button.
for (const { id } of refused.files)
await page.click(
`#geo-import-list [data-import-id="${id}"] button[data-action="remove"]`,
);
const cleared = await state();
check(
'Remove gives every data source back',
cleared.rows === 0 && cleared.dataSources === before.dataSources,
`${cleared.dataSources} vs ${before.dataSources}`,
);
await wait(1500);
check(
'nothing named inside a file, and no icon asset for one, was ever requested',
trapped.length === 0,
trapped.join(' '),
);
check('no page errors', errors.length === 0, errors.join(' | '));
} catch (error) {
check('harness ran to completion', false, error?.stack || String(error));
} finally {
await browser.close();
fs.rmSync(dir, { recursive: true, force: true });
}
console.log(failures ? `\n${failures} check(s) failed` : '\nall checks passed');
process.exit(failures ? 1 : 0);
+9
View File
@@ -1,6 +1,7 @@
import { SceneDirector } from '../scenes/director.js';
import { initAnnotations } from '../annotations/index.js';
import { initDrawTool } from '../annotations/drawTool.js';
import { initGeoFileImport } from '../ui/geoFileImport.js';
import { initImageryBoxTool } from '../ui/imageryBoxTool.js';
import { createRecentImageryPanel } from '../ui/recentImagery.js';
import { initGevVoiceCommands } from '../voice/gevRealtime.js';
@@ -56,6 +57,14 @@ export function createApplicationTools({
// lifetime rather than to whoever last pressed the button.
const drawTool = initDrawTool({ viewer, annotations });
defer(() => drawTool?.destroy());
// DISPLAY ▸ Import: the person's own GeoJSON/KML/KMZ/GPX files, each its
// own data source. They live for the page session, so the application
// lifetime removes them and the drop listeners on the viewer container.
const geoFileImport = initGeoFileImport({
viewer,
navigation: styleManager,
});
defer(() => geoFileImport?.destroy());
// DATA ▸ Recent Imagery: the box tool claims the pointer like Draw and the
// panel lives on the right rail, so both belong to the application
// lifetime. The tileset lets the layer drape while the globe is hidden.
+852
View File
@@ -0,0 +1,852 @@
/**
* Your own geodata on the globe: GeoJSON, KML, KMZ and GPX files read in the
* browser and handed to Cesium's own data sources.
*
* Nothing here talks to the network. The file never leaves the page, and a
* KML/KMZ file cannot make the page fetch anything either: network links,
* remote icons, remote styles, models and photo/screen overlays are removed
* before Cesium sees the document (`sanitizeKmlDocument`). Images packed
* inside a KMZ are served to Cesium from blob URLs the caller revokes.
*
* What the file says about itself stays inert: descriptions, balloons,
* extended data and GeoJSON properties are dropped from every entity, names
* are kept only as short plain text, and no native Cesium label is drawn
* (`scrubImportedEntities`). GeoJSON and GPX draw in the row's color:
* simplestyle properties are dropped, including `marker-symbol`, so no icon
* asset is loaded on the file's behalf. KML and KMZ keep their own styles.
*
* An import never moves the app's shared clock: dated KML (TimeSpan,
* TimeStamp, gx:Track) is drawn for all time, and the data source exposes no
* clock for the viewer to adopt (`detachImportedTime`). GPX is read here
* rather than by Cesium (`gpxToGeoJson`), keeping each track segment as its
* own line and leaving times out.
*
* Size is checked before Cesium builds anything (`countGeoJsonGeometry`,
* `countKmlGeometry`): a file with too many features or vertices is refused
* up front instead of after the entities exist.
*
* The bounded pieces — format detection, the KMZ (zip) reader, the size and
* count limits — are plain functions so they run under `node --test`; the
* Cesium loaders need a browser DOM and are exercised by
* `scripts/qa-geo-import.mjs`.
*/
import * as Cesium from 'cesium';
/** Largest file accepted, in bytes. */
export const MAX_IMPORT_BYTES = 50 * 1024 * 1024;
/** Largest total a KMZ may inflate to — a zip bomb stops here. */
export const MAX_KMZ_INFLATED_BYTES = 120 * 1024 * 1024;
/** Most files on the globe at once. */
export const MAX_IMPORTED_FILES = 12;
/** Most entities one file may create before it is refused. */
export const MAX_IMPORT_ENTITIES = 20_000;
/** Most vertices (points of every line, ring and marker) in one file. */
export const MAX_IMPORT_POSITIONS = 250_000;
/** Most `<href>` and `<Style>` elements in one KML/KMZ document. */
export const MAX_KML_LINKS = 2_000;
/** Longest plain-text name kept from a file's features. */
export const MAX_IMPORT_NAME_LENGTH = 120;
/** What the file picker offers. */
export const GEO_FILE_ACCEPT = '.geojson,.json,.kml,.kmz,.gpx';
const GEOJSON_TYPES = new Set([
'FeatureCollection',
'Feature',
'Point',
'MultiPoint',
'LineString',
'MultiLineString',
'Polygon',
'MultiPolygon',
'GeometryCollection',
]);
const KMZ_IMAGE_TYPES = Object.freeze({
png: 'image/png',
jpg: 'image/jpeg',
jpeg: 'image/jpeg',
gif: 'image/gif',
webp: 'image/webp',
bmp: 'image/bmp',
});
// KML elements that fetch, embed or execute something outside the file.
const KML_REMOVED_ELEMENTS = Object.freeze([
'NetworkLink',
'NetworkLinkControl',
'ScreenOverlay',
'PhotoOverlay',
'Model',
'Tour',
]);
/** An error whose message is meant for the person who chose the file. */
export class GeoImportError extends Error {
constructor(message) {
super(message);
this.name = 'GeoImportError';
}
}
/**
* The format of a file, from its name first and its first bytes second.
* @param {string} name File name.
* @param {Uint8Array} [head] The first bytes of the file.
* @returns {'geojson'|'kml'|'kmz'|'gpx'|null}
*/
export function detectGeoFileFormat(name, head) {
const ext = /\.([a-z0-9]+)$/i.exec(String(name || ''))?.[1]?.toLowerCase();
if (ext === 'geojson') return 'geojson';
if (ext === 'kml' || ext === 'kmz' || ext === 'gpx') return ext;
if (!head?.length) return ext === 'json' ? 'geojson' : null;
if (
head[0] === 0x50 &&
head[1] === 0x4b &&
head[2] === 0x03 &&
head[3] === 0x04
)
return 'kmz';
const text = new TextDecoder()
.decode(head.subarray(0, 1024))
.replace(/^/, '')
.trimStart();
if (text.startsWith('{')) return 'geojson';
if (text.startsWith('<')) {
if (/<kml[\s>]/i.test(text)) return 'kml';
if (/<gpx[\s>]/i.test(text)) return 'gpx';
}
return ext === 'json' ? 'geojson' : null;
}
/**
* A short, printable name for the list: the file name without control
* characters, cut to 60 characters.
* @param {string} name
* @returns {string}
*/
export function importDisplayName(name) {
const clean = String(name || '')
.replace(/[\u0000-\u001f\u007f]/g, '')
.trim();
if (!clean) return 'Imported file';
return clean.length > 60 ? `${clean.slice(0, 57)}…` : clean;
}
/**
* Parse and check GeoJSON text. Throws `GeoImportError` for anything that is
* not a GeoJSON object.
* @param {string} text
* @returns {object}
*/
export function parseGeoJsonText(text) {
let value;
try {
value = JSON.parse(String(text).replace(/^/, ''));
} catch {
throw new GeoImportError('That file is not valid JSON.');
}
if (!value || typeof value !== 'object' || !GEOJSON_TYPES.has(value.type))
throw new GeoImportError('That JSON file is not GeoJSON.');
if (value.type === 'FeatureCollection' && !Array.isArray(value.features))
throw new GeoImportError('That FeatureCollection has no features list.');
return value;
}
/**
* Count features and vertices in a parsed GeoJSON object without recursion
* limits or array spreading, stopping as soon as a limit is passed.
* @param {object} geojson
* @param {{maxFeatures?: number, maxPositions?: number}} [limits]
* @returns {{features: number, positions: number, over: ''|'features'|'positions'}}
*/
export function countGeoJsonGeometry(
geojson,
{
maxFeatures = MAX_IMPORT_ENTITIES,
maxPositions = MAX_IMPORT_POSITIONS,
} = {},
) {
let features = 0;
let positions = 0;
const stack = [geojson];
while (stack.length) {
const node = stack.pop();
if (!node || typeof node !== 'object') continue;
if (node.type === 'FeatureCollection') {
const list = Array.isArray(node.features) ? node.features : [];
for (let i = 0; i < list.length; i++) stack.push(list[i]);
continue;
}
if (node.type === 'Feature') {
features += 1;
if (features > maxFeatures)
return { features, positions, over: 'features' };
stack.push(node.geometry);
continue;
}
if (node.type === 'GeometryCollection') {
const list = Array.isArray(node.geometries) ? node.geometries : [];
for (let i = 0; i < list.length; i++) stack.push(list[i]);
continue;
}
if (GEOJSON_TYPES.has(node.type)) {
// A bare geometry is one feature: Cesium makes one entity for it.
if (node === geojson) features += 1;
// Walk the coordinate arrays iteratively: a position is an array whose
// first item is a number.
const coords = [node.coordinates];
while (coords.length) {
const c = coords.pop();
if (!Array.isArray(c)) continue;
if (typeof c[0] === 'number') {
positions += 1;
if (positions > maxPositions)
return { features, positions, over: 'positions' };
} else for (let i = 0; i < c.length; i++) coords.push(c[i]);
}
}
}
return { features, positions, over: '' };
}
/** The refusal for a file over a size limit, in the person's terms. */
function overLimitError({ over, features, positions }) {
if (over === 'links')
return new GeoImportError(
`That file has more than ${MAX_KML_LINKS.toLocaleString()} styles and icon links (the limit).`,
);
if (over === 'features')
return new GeoImportError(
`That file has more than ${MAX_IMPORT_ENTITIES.toLocaleString()} features (the limit).`,
);
return new GeoImportError(
`That file has more than ${MAX_IMPORT_POSITIONS.toLocaleString()} points (the limit)${features ? ` across ${features.toLocaleString()} features` : ''}.`,
);
}
// simplestyle-spec keys Cesium honors. `marker-symbol` would load a maki icon
// asset on the file's behalf; the colors would override the row's color.
const GEOJSON_STYLE_KEYS = Object.freeze([
'marker-symbol',
'marker-color',
'marker-size',
'stroke',
'stroke-opacity',
'stroke-width',
'fill',
'fill-opacity',
]);
/**
* Drop GeoJSON simplestyle properties, so a GeoJSON file draws in its row's
* color like a GPX file and no icon is fetched for it. Mutates and returns
* the object.
* @param {object} geojson
* @returns {object}
*/
export function stripGeoJsonStyleProperties(geojson) {
const stack = [geojson];
while (stack.length) {
const node = stack.pop();
if (!node || typeof node !== 'object') continue;
if (node.type === 'FeatureCollection' && Array.isArray(node.features))
for (let i = 0; i < node.features.length; i++)
stack.push(node.features[i]);
else if (
node.type === 'Feature' &&
node.properties &&
typeof node.properties === 'object'
)
for (const key of GEOJSON_STYLE_KEYS) delete node.properties[key];
}
return geojson;
}
/** Inflate raw-deflate bytes with the platform's stream, stopping at `maxBytes`. */
async function inflateRaw(bytes, maxBytes) {
const stream = new Blob([bytes])
.stream()
.pipeThrough(new DecompressionStream('deflate-raw'));
const reader = stream.getReader();
const chunks = [];
let total = 0;
try {
for (;;) {
const { done, value } = await reader.read();
if (done) break;
total += value.length;
if (total > maxBytes)
throw new GeoImportError(
'That KMZ unpacks to more than the import limit.',
);
chunks.push(value);
}
} catch (error) {
await reader.cancel().catch(() => {});
if (error instanceof GeoImportError) throw error;
throw new GeoImportError('That KMZ is damaged and could not be unpacked.');
}
const out = new Uint8Array(total);
let at = 0;
for (const chunk of chunks) {
out.set(chunk, at);
at += chunk.length;
}
return out;
}
/**
* Read the entries of a zip archive (stored or deflated; no encryption, no
* ZIP64). Directory entries are skipped.
* @param {Uint8Array} bytes The whole archive.
* @param {{maxInflatedBytes?: number}} [options]
* @returns {Promise<Map<string, Uint8Array>>} Entry path → contents.
*/
export async function readZipEntries(
bytes,
{ maxInflatedBytes = MAX_KMZ_INFLATED_BYTES } = {},
) {
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
const fail = (message = 'That KMZ is not a readable zip archive.') => {
throw new GeoImportError(message);
};
// End of central directory: 22 bytes plus a comment of up to 64 KiB.
let eocd = -1;
for (
let i = bytes.length - 22;
i >= Math.max(0, bytes.length - 22 - 0xffff);
i--
) {
if (view.getUint32(i, true) === 0x06054b50) {
eocd = i;
break;
}
}
if (eocd < 0) fail();
const count = view.getUint16(eocd + 10, true);
const dirSize = view.getUint32(eocd + 12, true);
let at = view.getUint32(eocd + 16, true);
if (count === 0xffff || at === 0xffffffff || at + dirSize > eocd)
fail('That KMZ uses a zip format this import does not read.');
const entries = new Map();
const names = new TextDecoder();
let inflated = 0;
for (let n = 0; n < count; n++) {
if (at + 46 > bytes.length || view.getUint32(at, true) !== 0x02014b50)
fail();
const flags = view.getUint16(at + 8, true);
const method = view.getUint16(at + 10, true);
const compressed = view.getUint32(at + 20, true);
const size = view.getUint32(at + 24, true);
const nameLength = view.getUint16(at + 28, true);
const extraLength = view.getUint16(at + 30, true);
const commentLength = view.getUint16(at + 32, true);
const local = view.getUint32(at + 42, true);
const name = names.decode(bytes.subarray(at + 46, at + 46 + nameLength));
at += 46 + nameLength + extraLength + commentLength;
if (name.endsWith('/')) continue;
if (flags & 0x1) fail('That KMZ is encrypted.');
if (compressed === 0xffffffff || size === 0xffffffff)
fail('That KMZ uses a zip format this import does not read.');
if (local + 30 > bytes.length || view.getUint32(local, true) !== 0x04034b50)
fail();
const start =
local +
30 +
view.getUint16(local + 26, true) +
view.getUint16(local + 28, true);
if (start + compressed > bytes.length) fail();
const data = bytes.subarray(start, start + compressed);
let contents;
if (method === 0) contents = data;
else if (method === 8)
contents = await inflateRaw(data, maxInflatedBytes - inflated);
else fail('That KMZ uses a compression method this import does not read.');
inflated += contents.length;
if (inflated > maxInflatedBytes)
fail('That KMZ unpacks to more than the import limit.');
entries.set(name, contents);
}
return entries;
}
/**
* The KML document inside a KMZ: `doc.kml` when present, else the first
* `.kml` entry nearest the archive root.
* @param {Map<string, Uint8Array>} entries
* @returns {string|null} Entry path.
*/
export function kmzMainDocument(entries) {
if (entries.has('doc.kml')) return 'doc.kml';
const kml = [...entries.keys()]
.filter((name) => /\.kml$/i.test(name))
.sort(
(a, b) => a.split('/').length - b.split('/').length || a.localeCompare(b),
);
return kml[0] ?? null;
}
/**
* Resolve a KMZ-relative href to an entry path, or null.
* @param {string} href As written in the KML.
* @param {string} base The KML entry's path.
* @param {Map<string, Uint8Array>} entries
* @returns {string|null}
*/
export function resolveKmzPath(href, base, entries) {
const raw = String(href || '').trim();
if (!raw || /^[a-z][a-z0-9+.-]*:/i.test(raw) || raw.startsWith('//'))
return null;
let decoded;
try {
decoded = decodeURIComponent(raw.split(/[?#]/)[0]);
} catch {
return null;
}
const parts = base.split('/').slice(0, -1);
for (const part of decoded.replace(/\\/g, '/').split('/')) {
if (!part || part === '.') continue;
if (part === '..') parts.pop();
else parts.push(part);
}
const path = parts.join('/');
return entries.has(path) ? path : null;
}
/** MIME type for a KMZ image entry, or null when it is not an image we serve. */
export function kmzImageType(path) {
const ext = /\.([a-z0-9]+)$/i.exec(path)?.[1]?.toLowerCase();
return KMZ_IMAGE_TYPES[ext] ?? null;
}
const elementsNamed = (doc, localName) =>
[...doc.getElementsByTagName('*')].filter((el) => el.localName === localName);
/**
* A `resolveLocal` for `sanitizeKmlDocument`: the blob URL of an image packed
* in the KMZ, or null. Each entry gets one blob and one URL however many
* hrefs name it, so repeated icons don't copy (and decode) the image again.
* @param {Map<string, Uint8Array>} entries
* @param {string} main Path of the KMZ's KML document.
* @param {(url: string) => void} onUrl Called once per new URL (to revoke later).
* @param {(blob: Blob) => string} [createUrl]
* @returns {(href: string) => string|null}
*/
export function kmzImageResolver(
entries,
main,
onUrl,
createUrl = (blob) => URL.createObjectURL(blob),
) {
const urls = new Map();
return (href) => {
const path = resolveKmzPath(href, main, entries);
const type = path && kmzImageType(path);
if (!type) return null;
let url = urls.get(path);
if (!url) {
url = createUrl(new Blob([entries.get(path)], { type }));
urls.set(path, url);
onUrl(url);
}
return url;
};
}
/**
* Strip everything from a parsed KML document that would make the page reach
* outside the file. `resolveLocal(href)` returns a URL for an image packed in
* the same KMZ (or null); every other href is removed, so icons fall back to
* Cesium's default pin and a ground overlay without a local image is dropped.
* @param {Document} doc
* @param {(href: string) => string|null} [resolveLocal]
* @returns {{removed: number}} How many elements were taken out.
*/
export function sanitizeKmlDocument(doc, resolveLocal = () => null) {
let removed = 0;
const drop = (el) => {
if (el.parentNode) {
el.parentNode.removeChild(el);
removed += 1;
}
};
for (const name of KML_REMOVED_ELEMENTS)
elementsNamed(doc, name).forEach(drop);
for (const href of elementsNamed(doc, 'href')) {
const value = (href.textContent || '').trim();
const local = resolveLocal(value);
if (local) href.textContent = local;
else if (/^data:image\/(png|jpeg|gif|webp|bmp);/i.test(value)) continue;
else drop(href);
}
for (const overlay of elementsNamed(doc, 'GroundOverlay')) {
if (!elementsNamed(overlay, 'href').length) drop(overlay);
}
for (const styleUrl of elementsNamed(doc, 'styleUrl')) {
if (!(styleUrl.textContent || '').trim().startsWith('#')) drop(styleUrl);
}
// Placemark balloons are HTML; the app shows no info box, and keeping their
// <img>/<iframe> sources out means no future surface can load them either.
for (const name of ['description', 'BalloonStyle', 'Snippet'])
elementsNamed(doc, name).forEach(drop);
return { removed };
}
/** Child elements of `el` with this local name (any namespace). */
const childrenNamed = (el, localName) => {
const out = [];
const kids = el?.childNodes || [];
for (let i = 0; i < kids.length; i++)
if (kids[i].nodeType === 1 && kids[i].localName === localName)
out.push(kids[i]);
return out;
};
const childText = (el, localName) =>
(childrenNamed(el, localName)[0]?.textContent || '').trim();
/**
* Count placemarks and vertices in a parsed KML document before Cesium reads
* it: every `<coordinates>` tuple and every `gx:coord`, plus the `<href>`
* and `<Style>` elements (`links`), each of which can make Cesium load and
* decode an image.
* @param {Document} doc
* @returns {{features: number, positions: number, links: number,
* over: ''|'features'|'positions'|'links'}}
*/
export function countKmlGeometry(
doc,
{
maxFeatures = MAX_IMPORT_ENTITIES,
maxPositions = MAX_IMPORT_POSITIONS,
maxLinks = MAX_KML_LINKS,
} = {},
) {
let features = 0;
let positions = 0;
let links = 0;
const result = (over) => ({ features, positions, links, over });
for (const el of doc.getElementsByTagName('*')) {
const name = el.localName;
if (name === 'href' || name === 'Style') {
links += 1;
if (links > maxLinks) return result('links');
} else if (name === 'Placemark') {
features += 1;
if (features > maxFeatures) return result('features');
} else if (name === 'coordinates') {
const text = el.textContent || '';
// Tuples are separated by whitespace; count runs without splitting
// a large string into a large array.
let inTuple = false;
for (let i = 0; i < text.length; i++) {
const space = text.charCodeAt(i) <= 32;
if (!space && !inTuple) {
positions += 1;
if (positions > maxPositions) return result('positions');
}
inTuple = !space;
}
} else if (name === 'coord') {
positions += 1;
if (positions > maxPositions) return result('positions');
}
}
return result('');
}
/**
* Read a parsed GPX document into GeoJSON: waypoints become points, each
* route a line and each track one MultiLineString with a line per `<trkseg>`
* (segment breaks are gaps in the recording and stay gaps). Only names are
* kept, and no times, so a dated track draws without a clock. Throws
* `GeoImportError` past the size limits.
* @param {Document} doc
* @returns {object} A FeatureCollection.
*/
export function gpxToGeoJson(
doc,
{
maxFeatures = MAX_IMPORT_ENTITIES,
maxPositions = MAX_IMPORT_POSITIONS,
} = {},
) {
const root = doc.documentElement;
if (root?.localName !== 'gpx')
throw new GeoImportError('That GPX file has no <gpx> document.');
const features = [];
let positions = 0;
const point = (el) => {
const lat = Number(el.getAttribute('lat'));
const lon = Number(el.getAttribute('lon'));
if (
!Number.isFinite(lat) ||
!Number.isFinite(lon) ||
Math.abs(lat) > 90 ||
Math.abs(lon) > 180
)
return null;
positions += 1;
if (positions > maxPositions)
throw overLimitError({
over: 'positions',
features: features.length,
positions,
});
return [lon, lat];
};
const add = (name, geometry) => {
if (features.length >= maxFeatures)
throw overLimitError({
over: 'features',
features: features.length + 1,
positions,
});
features.push({
type: 'Feature',
properties: name ? { name } : {},
geometry,
});
};
for (const wpt of childrenNamed(root, 'wpt')) {
const at = point(wpt);
if (at) add(childText(wpt, 'name'), { type: 'Point', coordinates: at });
}
for (const rte of childrenNamed(root, 'rte')) {
const line = [];
for (const pt of childrenNamed(rte, 'rtept')) {
const at = point(pt);
if (at) line.push(at);
}
if (line.length >= 2)
add(childText(rte, 'name'), { type: 'LineString', coordinates: line });
}
for (const trk of childrenNamed(root, 'trk')) {
const lines = [];
for (const seg of childrenNamed(trk, 'trkseg')) {
const line = [];
for (const pt of childrenNamed(seg, 'trkpt')) {
const at = point(pt);
if (at) line.push(at);
}
if (line.length >= 2) lines.push(line);
}
if (lines.length)
add(childText(trk, 'name'), {
type: 'MultiLineString',
coordinates: lines,
});
}
return { type: 'FeatureCollection', features };
}
/** A feature name as short plain text (no control characters), or undefined. */
export function importFeatureName(value) {
const text = String(value ?? '')
.replace(/[\u0000-\u001f\u007f]/g, ' ')
.replace(/\s+/g, ' ')
.trim();
if (!text) return undefined;
return text.length > MAX_IMPORT_NAME_LENGTH
? `${text.slice(0, MAX_IMPORT_NAME_LENGTH - 1)}…`
: text;
}
/**
* Make every entity of an imported data source inert: no description,
* balloon, extended data or properties, no native label, and a name that is
* short plain text. Returns how many entities were touched.
* @param {{entities: {values: object[]}}} dataSource
* @returns {number}
*/
export function scrubImportedEntities(dataSource) {
const list = dataSource?.entities?.values || [];
for (let i = 0; i < list.length; i++) {
const entity = list[i];
let name;
try {
name = importFeatureName(
typeof entity.name === 'string' ? entity.name : undefined,
);
} catch {
name = undefined;
}
entity.name = name;
entity.description = undefined;
entity.label = undefined;
entity.properties = undefined;
if (entity.kml && typeof entity.kml === 'object') entity.kml = undefined;
}
return list.length;
}
const TRACK_SAMPLES = 2000;
/**
* Take time out of an imported data source so adding it cannot move the app's
* shared clock: the data source reports no clock (the viewer adopts the clock
* of every added source that has one), every entity is available for all
* time, and a time-dynamic position (a KML gx:Track) is drawn as the line it
* travels instead of a moving marker.
* @param {object} dataSource
* @param {{Cesium?: object, color?: object}} [options]
* @returns {object} The same data source.
*/
export function detachImportedTime(
dataSource,
{ Cesium: C = Cesium, color } = {},
) {
Object.defineProperty(dataSource, 'clock', {
configurable: true,
enumerable: false,
get: () => undefined,
set: () => {},
});
const list = dataSource?.entities?.values || [];
for (let i = 0; i < list.length; i++) {
const entity = list[i];
const span = entity.availability;
const position = entity.position;
if (position && position.isConstant === false) {
const start = span?.start;
const stop = span?.stop;
const line = [];
if (start && stop && C.JulianDate.lessThan(start, stop)) {
const seconds = C.JulianDate.secondsDifference(stop, start);
for (let n = 0; n <= TRACK_SAMPLES; n++) {
const t = C.JulianDate.addSeconds(
start,
(seconds * n) / TRACK_SAMPLES,
new C.JulianDate(),
);
const at = position.getValue(t);
if (at) line.push(at);
}
}
entity.path = undefined;
entity.model = undefined;
if (line.length >= 2 && !entity.polyline) {
entity.billboard = undefined;
entity.point = undefined;
entity.position = undefined;
entity.polyline = new C.PolylineGraphics({
positions: line,
clampToGround: true,
width: 3,
material: color || C.Color.WHITE,
});
} else {
entity.position = line.length
? new C.ConstantPositionProperty(line[0])
: undefined;
}
}
entity.availability = undefined;
}
return dataSource;
}
function parseXml(text, what) {
const doc = new DOMParser().parseFromString(
String(text).replace(/^/, ''),
'application/xml',
);
if (doc.getElementsByTagName('parsererror').length)
throw new GeoImportError(`That ${what} file is not valid XML.`);
return doc;
}
/**
* Load one file into a Cesium data source, clamped to the ground.
* @param {File|Blob & {name?: string}} file
* @param {{viewer: Cesium.Viewer, color?: string}} options `color` is a CSS color.
* @returns {Promise<{dataSource: Cesium.DataSource, name: string,
* format: string, entityCount: number, removed: number, revoke: Function}>}
*/
export async function loadGeoFile(file, { viewer, color = '#39d0ff' }) {
if (!file || !Number.isFinite(file.size))
throw new GeoImportError('Nothing to import.');
if (file.size > MAX_IMPORT_BYTES)
throw new GeoImportError(
`That file is larger than ${MAX_IMPORT_BYTES / 1024 / 1024} MB.`,
);
const bytes = new Uint8Array(await file.arrayBuffer());
const format = detectGeoFileFormat(file.name, bytes.subarray(0, 1024));
if (!format)
throw new GeoImportError(
'Only GeoJSON, KML, KMZ and GPX files can be imported.',
);
const stroke = Cesium.Color.fromCssColorString(color);
const urls = [];
const revoke = () =>
urls.splice(0).forEach((url) => URL.revokeObjectURL(url));
const name = importDisplayName(file.name);
let dataSource;
let removed = 0;
try {
if (format === 'geojson' || format === 'gpx') {
let geojson;
if (format === 'gpx') {
geojson = gpxToGeoJson(
parseXml(new TextDecoder().decode(bytes), 'GPX'),
);
} else {
geojson = parseGeoJsonText(new TextDecoder().decode(bytes));
const size = countGeoJsonGeometry(geojson);
if (size.over) throw overLimitError(size);
stripGeoJsonStyleProperties(geojson);
}
dataSource = await Cesium.GeoJsonDataSource.load(geojson, {
clampToGround: true,
stroke,
fill: stroke.withAlpha(0.25),
strokeWidth: 3,
markerColor: stroke,
// No HTML description table is built from the file's properties.
describe: () => undefined,
});
} else {
let text;
let resolveLocal = () => null;
if (format === 'kmz') {
const entries = await readZipEntries(bytes);
const main = kmzMainDocument(entries);
if (!main) throw new GeoImportError('That KMZ holds no KML document.');
text = new TextDecoder().decode(entries.get(main));
resolveLocal = kmzImageResolver(entries, main, (url) => urls.push(url));
} else {
text = new TextDecoder().decode(bytes);
}
const doc = parseXml(text, format.toUpperCase());
const size = countKmlGeometry(doc);
if (size.over) throw overLimitError(size);
removed = sanitizeKmlDocument(doc, resolveLocal).removed;
dataSource = await Cesium.KmlDataSource.load(doc, {
camera: viewer.scene.camera,
canvas: viewer.scene.canvas,
clampToGround: true,
});
}
} catch (error) {
revoke();
if (error instanceof GeoImportError) throw error;
throw new GeoImportError(
`That ${format.toUpperCase()} file could not be read${error?.message ? `: ${error.message}` : '.'}`,
);
}
const entityCount = dataSource.entities.values.length;
if (entityCount > MAX_IMPORT_ENTITIES) {
revoke();
throw new GeoImportError(
`That file has ${entityCount.toLocaleString()} features; the limit is ${MAX_IMPORT_ENTITIES.toLocaleString()}.`,
);
}
if (entityCount === 0) {
revoke();
throw new GeoImportError('That file has nothing to draw.');
}
scrubImportedEntities(dataSource);
// Only KML has time-dynamic positions (GPX is read without times); its
// track line is drawn in KML's default white, like its other unstyled lines.
detachImportedTime(dataSource);
// The display name travels beside the data source (the file name, not
// anything the document says about itself).
return { dataSource, name, format, entityCount, removed, revoke };
}
+542
View File
@@ -0,0 +1,542 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { deflateRawSync } from 'node:zlib';
import * as Cesium from 'cesium';
import {
GeoImportError,
MAX_IMPORT_POSITIONS,
countGeoJsonGeometry,
countKmlGeometry,
detachImportedTime,
gpxToGeoJson,
importFeatureName,
scrubImportedEntities,
stripGeoJsonStyleProperties,
detectGeoFileFormat,
importDisplayName,
kmzImageResolver,
kmzImageType,
kmzMainDocument,
parseGeoJsonText,
readZipEntries,
resolveKmzPath,
} from './geoFileImport.js';
const bytes = (text) => new TextEncoder().encode(text);
/** A minimal zip writer: stored (0) or deflated (8) entries, optional flags. */
function zip(files) {
const locals = [];
const centrals = [];
let offset = 0;
for (const { name, data, method = 8, flags = 0, size } of files) {
const nameBytes = bytes(name);
const body = method === 8 ? new Uint8Array(deflateRawSync(data)) : data;
const local = new Uint8Array(30 + nameBytes.length);
const lv = new DataView(local.buffer);
lv.setUint32(0, 0x04034b50, true);
lv.setUint16(6, flags, true);
lv.setUint16(8, method, true);
lv.setUint32(18, body.length, true);
lv.setUint32(22, size ?? data.length, true);
lv.setUint16(26, nameBytes.length, true);
local.set(nameBytes, 30);
const central = new Uint8Array(46 + nameBytes.length);
const cv = new DataView(central.buffer);
cv.setUint32(0, 0x02014b50, true);
cv.setUint16(8, flags, true);
cv.setUint16(10, method, true);
cv.setUint32(20, body.length, true);
cv.setUint32(24, size ?? data.length, true);
cv.setUint16(28, nameBytes.length, true);
cv.setUint32(42, offset, true);
central.set(nameBytes, 46);
locals.push(local, body);
centrals.push(central);
offset += local.length + body.length;
}
const dirSize = centrals.reduce((n, c) => n + c.length, 0);
const eocd = new Uint8Array(22);
const ev = new DataView(eocd.buffer);
ev.setUint32(0, 0x06054b50, true);
ev.setUint16(8, files.length, true);
ev.setUint16(10, files.length, true);
ev.setUint32(12, dirSize, true);
ev.setUint32(16, offset, true);
const parts = [...locals, ...centrals, eocd];
const out = new Uint8Array(parts.reduce((n, p) => n + p.length, 0));
let at = 0;
for (const p of parts) {
out.set(p, at);
at += p.length;
}
return out;
}
test('format comes from the extension, then from the first bytes', () => {
assert.equal(detectGeoFileFormat('trail.GPX'), 'gpx');
assert.equal(detectGeoFileFormat('farm.kml'), 'kml');
assert.equal(detectGeoFileFormat('farm.kmz'), 'kmz');
assert.equal(detectGeoFileFormat('parcels.geojson'), 'geojson');
assert.equal(detectGeoFileFormat('parcels.json'), 'geojson');
assert.equal(
detectGeoFileFormat('export', bytes(' {"type":"Feature"}')),
'geojson',
);
assert.equal(
detectGeoFileFormat(
'export',
bytes('<?xml version="1.0"?><kml xmlns="x">'),
),
'kml',
);
assert.equal(
detectGeoFileFormat(
'export',
bytes('<?xml version="1.0"?>\n<gpx version="1.1">'),
),
'gpx',
);
assert.equal(
detectGeoFileFormat('export', new Uint8Array([0x50, 0x4b, 0x03, 0x04, 0])),
'kmz',
);
assert.equal(detectGeoFileFormat('notes.txt', bytes('hello')), null);
assert.equal(
detectGeoFileFormat('image.png', new Uint8Array([0x89, 0x50, 0x4e, 0x47])),
null,
);
});
test('display names are printable and bounded', () => {
assert.equal(importDisplayName(' hike\u0000.gpx '), 'hike.gpx');
assert.equal(importDisplayName(''), 'Imported file');
const long = importDisplayName(`${'x'.repeat(80)}.kml`);
assert.equal(long.length, 58);
assert.ok(long.endsWith('…'));
});
test('GeoJSON text must be a GeoJSON object', () => {
assert.equal(
parseGeoJsonText('{"type":"FeatureCollection","features":[]}').type,
'FeatureCollection',
);
assert.equal(
parseGeoJsonText('{"type":"Point","coordinates":[0,0]}').type,
'Point',
);
assert.throws(() => parseGeoJsonText('{nope'), GeoImportError);
assert.throws(() => parseGeoJsonText('{"name":"not geo"}'), /not GeoJSON/);
assert.throws(() => parseGeoJsonText('[1,2]'), /not GeoJSON/);
assert.throws(
() => parseGeoJsonText('{"type":"FeatureCollection"}'),
/no features list/,
);
});
test('a KMZ is read with stored and deflated entries, directories skipped', async () => {
const kml = bytes('<kml><Document/></kml>');
const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 1, 2, 3]);
const archive = zip([
{ name: 'files/', data: new Uint8Array(0), method: 0 },
{ name: 'doc.kml', data: kml },
{ name: 'files/icon.png', data: png, method: 0 },
]);
const entries = await readZipEntries(archive);
assert.deepEqual([...entries.keys()], ['doc.kml', 'files/icon.png']);
assert.deepEqual(entries.get('doc.kml'), kml);
assert.deepEqual(entries.get('files/icon.png'), png);
});
test('a KMZ that inflates past the limit is refused', async () => {
const archive = zip([{ name: 'doc.kml', data: new Uint8Array(200_000) }]);
await assert.rejects(
readZipEntries(archive, { maxInflatedBytes: 100_000 }),
/more than the import limit/,
);
});
test('encrypted, damaged and non-zip archives are refused with a reason', async () => {
await assert.rejects(
readZipEntries(
zip([{ name: 'doc.kml', data: bytes('<kml/>'), flags: 1, method: 0 }]),
),
/encrypted/,
);
await assert.rejects(
readZipEntries(bytes('not a zip at all, just text')),
/not a readable zip/,
);
const truncated = zip([
{ name: 'doc.kml', data: bytes('<kml/>'), method: 0 },
]);
const view = new DataView(truncated.buffer);
view.setUint32(truncated.length - 6, 9999, true); // directory offset past the end
await assert.rejects(readZipEntries(truncated), GeoImportError);
await assert.rejects(
readZipEntries(
zip([{ name: 'doc.kml', data: bytes('<kml/>'), method: 12 }]),
),
/compression method/,
);
});
test('the main KML is doc.kml, else the shallowest .kml', () => {
const e = (...names) => new Map(names.map((n) => [n, new Uint8Array(0)]));
assert.equal(kmzMainDocument(e('a/b.kml', 'doc.kml')), 'doc.kml');
assert.equal(
kmzMainDocument(e('deep/x/z.kml', 'top.kml', 'images/a.png')),
'top.kml',
);
assert.equal(kmzMainDocument(e('images/a.png')), null);
});
test('KMZ hrefs resolve only to entries inside the archive', () => {
const entries = new Map([
['doc.kml', new Uint8Array(0)],
['files/icon one.png', new Uint8Array(0)],
['overlay.jpg', new Uint8Array(0)],
]);
assert.equal(
resolveKmzPath('files/icon%20one.png', 'doc.kml', entries),
'files/icon one.png',
);
assert.equal(
resolveKmzPath('./overlay.jpg', 'doc.kml', entries),
'overlay.jpg',
);
assert.equal(
resolveKmzPath('../overlay.jpg', 'files/sub.kml', entries),
'overlay.jpg',
);
assert.equal(
resolveKmzPath('https://example.com/overlay.jpg', 'doc.kml', entries),
null,
);
assert.equal(
resolveKmzPath('//example.com/overlay.jpg', 'doc.kml', entries),
null,
);
assert.equal(resolveKmzPath('file:///etc/passwd', 'doc.kml', entries), null);
assert.equal(resolveKmzPath('missing.png', 'doc.kml', entries), null);
assert.equal(resolveKmzPath('%E0%A4%A', 'doc.kml', entries), null);
assert.equal(kmzImageType('files/a.PNG'), 'image/png');
assert.equal(kmzImageType('files/a.svg'), null);
});
/* ── a tiny element tree with the DOM surface the readers use ─────────── */
function xmlEl(localName, attrs = {}, kids = []) {
const el = {
nodeType: 1,
localName,
childNodes: [],
getAttribute: (n) => (n in attrs ? String(attrs[n]) : null),
};
for (const k of typeof kids === 'string' ? [] : kids) el.childNodes.push(k);
const text = typeof kids === 'string' ? kids : null;
Object.defineProperty(el, 'textContent', {
get: () => text ?? el.childNodes.map((c) => c.textContent).join(''),
});
return el;
}
const xmlDoc = (root) => ({
documentElement: root,
getElementsByTagName(name) {
assert.equal(name, '*');
const out = [];
const walk = (el) => {
out.push(el);
el.childNodes.forEach(walk);
};
walk(root);
return out;
},
});
const trkpt = (lat, lon, time) =>
xmlEl('trkpt', { lat, lon }, time ? [xmlEl('time', {}, time)] : []);
test('GPX keeps each track segment as its own line, names as text, no times', () => {
const doc = xmlDoc(
xmlEl('gpx', {}, [
xmlEl('wpt', { lat: 37.77, lon: -122.49 }, [xmlEl('name', {}, 'Camp')]),
xmlEl('wpt', { lat: 99, lon: 0 }), // out of range: skipped
xmlEl('rte', {}, [
xmlEl('name', {}, 'Route'),
xmlEl('rtept', { lat: 1, lon: 2 }),
xmlEl('rtept', { lat: 3, lon: 4 }),
]),
xmlEl('trk', {}, [
xmlEl('name', {}, 'Morning walk'),
xmlEl('trkseg', {}, [
trkpt(10, 20, '2024-05-01T10:00:00Z'),
trkpt(10.1, 20.1, '2024-05-01T10:01:00Z'),
]),
// A pause in the recording: a new segment, not a straight jump.
xmlEl('trkseg', {}, [
trkpt(11, 21, '2024-05-01T11:00:00Z'),
trkpt(11.1, 21.1, '2024-05-01T11:01:00Z'),
trkpt(11.2, 21.2, '2024-05-01T11:02:00Z'),
]),
xmlEl('trkseg', {}, [trkpt(12, 22)]), // one point: not a line
]),
]),
);
const fc = gpxToGeoJson(doc);
assert.deepEqual(
fc.features.map((f) => [f.properties.name, f.geometry.type]),
[
['Camp', 'Point'],
['Route', 'LineString'],
['Morning walk', 'MultiLineString'],
],
);
assert.deepEqual(fc.features[2].geometry.coordinates, [
[
[20, 10],
[20.1, 10.1],
],
[
[21, 11],
[21.1, 11.1],
[21.2, 11.2],
],
]);
assert.doesNotMatch(JSON.stringify(fc), /2024|time/);
assert.throws(() => gpxToGeoJson(xmlDoc(xmlEl('kml'))), GeoImportError);
});
test('GPX past the point limit is refused while reading, before Cesium', () => {
const seg = xmlEl(
'trkseg',
{},
Array.from({ length: 30 }, (_, i) => trkpt(1 + i / 100, 2)),
);
const doc = xmlDoc(xmlEl('gpx', {}, [xmlEl('trk', {}, [seg])]));
assert.throws(
() => gpxToGeoJson(doc, { maxPositions: 20 }),
(e) => e instanceof GeoImportError && /more than .* points/.test(e.message),
);
assert.throws(
() =>
gpxToGeoJson(
xmlDoc(
xmlEl('gpx', {}, [
xmlEl('wpt', { lat: 1, lon: 1 }),
xmlEl('wpt', { lat: 2, lon: 2 }),
]),
),
{ maxFeatures: 1 },
),
/more than .* features/,
);
});
test('GeoJSON features and vertices are counted before loading, without recursion', () => {
const line = Array.from({ length: 50 }, (_, i) => [i / 10, 1]);
const fc = {
type: 'FeatureCollection',
features: [
{ type: 'Feature', geometry: { type: 'LineString', coordinates: line } },
{
type: 'Feature',
geometry: {
type: 'GeometryCollection',
geometries: [
{ type: 'Point', coordinates: [1, 2] },
{
type: 'Polygon',
coordinates: [
[
[0, 0],
[1, 0],
[1, 1],
[0, 0],
],
],
},
],
},
},
{ type: 'Feature', geometry: null },
],
};
assert.deepEqual(countGeoJsonGeometry(fc), {
features: 3,
positions: 55,
over: '',
});
assert.equal(
countGeoJsonGeometry(fc, { maxPositions: 54 }).over,
'positions',
);
assert.equal(countGeoJsonGeometry(fc, { maxFeatures: 2 }).over, 'features');
assert.deepEqual(
countGeoJsonGeometry({ type: 'Point', coordinates: [1, 2] }),
{
features: 1,
positions: 1,
over: '',
},
);
// A very long line is counted iteratively and stops at the limit.
const huge = {
type: 'LineString',
coordinates: Array.from({ length: MAX_IMPORT_POSITIONS + 5 }, () => [0, 0]),
};
assert.equal(countGeoJsonGeometry(huge).over, 'positions');
});
test('KML placemarks and coordinate tuples are counted before loading', () => {
const coords = (n) =>
xmlEl(
'coordinates',
{},
Array.from({ length: n }, (_, i) => `${i},1,0`).join('\n '),
);
const doc = xmlDoc(
xmlEl('kml', {}, [
xmlEl('Placemark', {}, [xmlEl('LineString', {}, [coords(40)])]),
xmlEl('Placemark', {}, [
xmlEl('Track', {}, [
xmlEl('coord', {}, '1 2 0'),
xmlEl('coord', {}, '1 3 0'),
]),
]),
]),
);
assert.deepEqual(countKmlGeometry(doc), {
features: 2,
positions: 42,
links: 0,
over: '',
});
assert.equal(countKmlGeometry(doc, { maxPositions: 41 }).over, 'positions');
assert.equal(countKmlGeometry(doc, { maxFeatures: 1 }).over, 'features');
});
test('KML styles and icon links are counted against their own limit', () => {
const style = () =>
xmlEl('Style', {}, [
xmlEl('IconStyle', {}, [xmlEl('Icon', {}, [xmlEl('href', {}, 'a.png')])]),
]);
const doc = xmlDoc(xmlEl('kml', {}, [style(), style(), style()]));
assert.equal(countKmlGeometry(doc).links, 6);
assert.equal(countKmlGeometry(doc, { maxLinks: 5 }).over, 'links');
});
test('a KMZ image gets one blob URL however many hrefs name it', () => {
const entries = new Map([
['doc.kml', new Uint8Array(0)],
['a.png', new Uint8Array(8)],
['b.png', new Uint8Array(8)],
]);
const made = [];
const kept = [];
const resolve = kmzImageResolver(
entries,
'doc.kml',
(url) => kept.push(url),
(blob) => {
made.push(blob);
return `blob:test/${made.length}`;
},
);
for (let i = 0; i < 1000; i++) assert.equal(resolve('a.png'), 'blob:test/1');
assert.equal(resolve('./b.png'), 'blob:test/2');
assert.equal(resolve('a.png'), 'blob:test/1');
assert.equal(resolve('https://example.com/a.png'), null);
assert.equal(resolve('doc.kml'), null);
assert.equal(made.length, 2);
assert.deepEqual(kept, ['blob:test/1', 'blob:test/2']);
});
test('simplestyle is dropped: no icon fetched, the row color is what draws', () => {
const fc = {
type: 'FeatureCollection',
features: [
{
type: 'Feature',
properties: {
'marker-symbol': 'rail',
'marker-color': '#f00',
stroke: '#0f0',
'stroke-width': 9,
fill: '#00f',
'fill-opacity': 1,
name: 'x',
},
geometry: null,
},
{ type: 'Feature', properties: null, geometry: null },
],
};
stripGeoJsonStyleProperties(fc);
assert.deepEqual(fc.features[0].properties, { name: 'x' });
});
test('imported entities keep only a short plain-text name: no description, label or properties', () => {
const ds = new Cesium.CustomDataSource('t');
const e = ds.entities.add({
name: 'Trail\u0007 <b>head</b>\n' + 'x'.repeat(300),
description: '<img src="https://example.invalid/x.png" onerror="alert(1)">',
label: { text: 'Trail head' },
properties: { note: '<script>alert(1)</script>' },
position: Cesium.Cartesian3.fromDegrees(1, 2),
});
e.kml = { extendedData: { note: { value: '<iframe src=//x>' } } };
ds.entities.add({ name: 42, position: Cesium.Cartesian3.fromDegrees(1, 2) });
assert.equal(scrubImportedEntities(ds), 2);
assert.equal(e.description, undefined);
assert.equal(e.label, undefined);
assert.equal(e.properties, undefined);
assert.equal(e.kml, undefined);
assert.ok(e.name.startsWith('Trail <b>head</b> x'));
assert.ok(e.name.length <= 120 && !/[\u0000-\u001f]/.test(e.name));
assert.equal(ds.entities.values[1].name, undefined);
assert.equal(importFeatureName(' '), undefined);
});
test('a dated import exposes no clock and draws for all time', () => {
// A data source with its own clock, as KmlDataSource has for a dated file.
class DatedSource extends Cesium.CustomDataSource {
get clock() {
return this._testClock;
}
}
const ds = new DatedSource('dated');
ds._testClock = new Cesium.DataSourceClock();
const start = Cesium.JulianDate.fromIso8601('2020-01-01T00:00:00Z');
const stop = Cesium.JulianDate.fromIso8601('2020-01-01T01:00:00Z');
const span = new Cesium.TimeIntervalCollection([
new Cesium.TimeInterval({ start, stop }),
]);
const track = new Cesium.SampledPositionProperty();
track.addSample(start, Cesium.Cartesian3.fromDegrees(10, 10));
track.addSample(stop, Cesium.Cartesian3.fromDegrees(10.1, 10.1));
const mover = ds.entities.add({
availability: span,
position: track,
path: {},
billboard: {},
});
const pin = ds.entities.add({
availability: span,
position: Cesium.Cartesian3.fromDegrees(1, 1),
});
assert.ok(ds.clock);
detachImportedTime(ds);
assert.equal(ds.clock, undefined);
// The viewer reads `clock` from every added source; setting it is ignored.
ds.clock = new Cesium.DataSourceClock();
assert.equal(ds.clock, undefined);
assert.equal(pin.availability, undefined);
assert.ok(pin.isAvailable(Cesium.JulianDate.now()));
// The moving marker became the line it travels, visible now.
assert.equal(mover.availability, undefined);
assert.equal(mover.position, undefined);
assert.equal(mover.path, undefined);
assert.equal(mover.billboard, undefined);
const line = mover.polyline.positions.getValue(Cesium.JulianDate.now());
assert.ok(line.length > 100);
assert.ok(mover.polyline.clampToGround.getValue());
});
+90
View File
@@ -0,0 +1,90 @@
/**
* Fly to everything in a data source (an imported file, a live feed) without
* trusting Cesium's own framing of ground-clamped points: before the
* destination's terrain has streamed in, a clamped billboard reports a
* bounding sphere kilometers below the surface, and `viewer.flyTo` follows it
* underground. Instead the footprint is framed from the features' longitudes
* and latitudes, and the camera ground guard lifts the eye once the real
* surface can be measured.
*/
import * as Cesium from 'cesium';
import { guardCameraAboveGround } from '../cameraGroundGuard.js';
const MAX_POINTS = 5000;
/**
* Longitude/latitude pairs of every entity's position, line and polygon,
* thinned to at most `MAX_POINTS`. Long tracks are walked by index, never
* spread into a call, so a file with hundreds of thousands of vertices cannot
* overflow the stack; only the kept points are converted.
*/
export function dataSourcePoints(dataSource, time = Cesium.JulianDate.now()) {
const parts = [];
let total = 0;
for (const entity of dataSource?.entities?.values ?? []) {
const at = entity.position?.getValue?.(time);
if (at) {
parts.push([at]);
total += 1;
}
const line = entity.polyline?.positions?.getValue?.(time);
if (Array.isArray(line) && line.length) {
parts.push(line);
total += line.length;
}
const ring = entity.polygon?.hierarchy?.getValue?.(time)?.positions;
if (Array.isArray(ring) && ring.length) {
parts.push(ring);
total += ring.length;
}
}
const stride = Math.max(1, Math.ceil(total / MAX_POINTS));
const points = [];
let index = 0;
for (const part of parts) {
// Keep every stride-th vertex across all parts, plus each part's ends so
// a short line is never skipped entirely.
for (let i = 0; i < part.length; i++, index++) {
if (index % stride !== 0 && i !== 0 && i !== part.length - 1) continue;
const carto = Cesium.Cartographic.fromCartesian(part[i]);
if (!carto) continue;
points.push([
Cesium.Math.toDegrees(carto.longitude),
Cesium.Math.toDegrees(carto.latitude),
]);
}
}
return points;
}
/**
* @param {Cesium.Viewer} viewer
* @param {Cesium.DataSource} dataSource
* @param {{duration?: number}} [options]
* @returns {boolean} Whether there was anything to fly to.
*/
export function flyToDataSource(viewer, dataSource, { duration = 1.5 } = {}) {
const points = dataSourcePoints(dataSource);
if (!points.length) {
viewer.flyTo?.(dataSource, { duration });
return false;
}
const sphere = Cesium.BoundingSphere.fromPoints(
points.map(([lon, lat]) => Cesium.Cartesian3.fromDegrees(lon, lat, 0)),
);
const center = Cesium.Cartographic.fromCartesian(sphere.center);
viewer.camera?.flyToBoundingSphere?.(sphere, {
offset: new Cesium.HeadingPitchRange(
viewer.camera.heading || 0,
Cesium.Math.toRadians(-45),
Math.max(800, sphere.radius * 2.8),
),
duration,
complete: () =>
guardCameraAboveGround(viewer, {
lat: Cesium.Math.toDegrees(center.latitude),
lon: Cesium.Math.toDegrees(center.longitude),
}),
});
return true;
}
+76
View File
@@ -0,0 +1,76 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import * as Cesium from 'cesium';
import { dataSourcePoints, flyToDataSource } from './flyToDataSource.js';
function source() {
const ds = new Cesium.CustomDataSource('t');
ds.entities.add({
position: Cesium.Cartesian3.fromDegrees(-122.4862, 37.7694, -2667),
billboard: { heightReference: Cesium.HeightReference.CLAMP_TO_GROUND },
});
ds.entities.add({
polyline: {
positions: Cesium.Cartesian3.fromDegreesArray([
-122.49, 37.77, -122.48, 37.771,
]),
},
});
ds.entities.add({
polygon: {
hierarchy: Cesium.Cartesian3.fromDegreesArray([
-122.5, 37.76, -122.49, 37.76, -122.49, 37.765,
]),
},
});
return ds;
}
test('points come from positions, lines and polygons, as lon/lat only', () => {
const points = dataSourcePoints(source());
assert.equal(points.length, 6);
assert.ok(Math.abs(points[0][0] + 122.4862) < 1e-6);
assert.ok(Math.abs(points[0][1] - 37.7694) < 1e-6);
});
test('the camera frames the footprint at the surface, never a buried center', () => {
const calls = [];
const viewer = {
camera: {
heading: 0,
flyToBoundingSphere: (sphere, options) => calls.push({ sphere, options }),
},
flyTo: () => calls.push('viewer.flyTo'),
};
assert.equal(flyToDataSource(viewer, source()), true);
const center = Cesium.Cartographic.fromCartesian(calls[0].sphere.center);
assert.ok(Math.abs(center.height) < 50, `center height ${center.height}`);
assert.ok(calls[0].options.offset.range >= 800);
assert.equal(
typeof calls[0].options.complete,
'function',
'the ground guard runs on landing',
);
assert.equal(
flyToDataSource(viewer, new Cesium.CustomDataSource('empty')),
false,
);
assert.equal(calls.at(-1), 'viewer.flyTo');
});
test('a very long track is framed without spreading it into a call', () => {
// 400k vertices: `push(...line)` throws RangeError (too many arguments)
// well below this; the walk keeps a bounded sample and both ends.
const n = 400_000;
const line = new Array(n);
for (let i = 0; i < n; i++)
line[i] = Cesium.Cartesian3.fromDegrees(-120 + (i / n) * 2, 40);
const ds = new Cesium.CustomDataSource('long');
ds.entities.add({ polyline: { positions: line } });
ds.entities.add({ position: Cesium.Cartesian3.fromDegrees(-110, 41) });
const points = dataSourcePoints(ds);
assert.ok(points.length <= 5002, `${points.length} points kept`);
const lons = points.map(([lon]) => lon);
assert.ok(Math.min(...lons) < -119.99 && Math.max(...lons) > -110.01);
assert.ok(lons.some((lon) => Math.abs(lon - -118.000005) < 0.01));
});
+413
View File
@@ -0,0 +1,413 @@
/**
* DISPLAY ▸ Import: put your own GeoJSON, KML, KMZ or GPX files on the globe.
*
* Files come in through the Import button or by dropping them onto the globe.
* Each one becomes its own Cesium data source, clamped to the ground, with a
* row in the panel to show or hide it, fly to it, or remove it. Files are read
* in the browser and never uploaded; see `src/data/geoFileImport.js` for what
* is accepted and what is stripped from KML before Cesium reads it.
*
* Imported files last for the page session: a reload starts empty.
*
* Imports run one batch at a time (a second pick or drop waits for the
* first), so the file cap is checked against what is really on the globe.
* Fly-to goes through the app's navigation policy (`navigation`, the
* application shell): the button is an immediate navigation, and the fly
* after an import is a deferred one, stamped when the import starts and
* re-asserted just before flying, so a camera move, Cockpit or any newer
* navigation in between wins over a slow file.
* `destroy()` removes every imported data source, revokes every blob URL and
* gives back every listener and the `window.__gevGeoImport` handle.
*/
import {
GEO_FILE_ACCEPT,
GeoImportError,
MAX_IMPORTED_FILES,
loadGeoFile,
} from '../data/geoFileImport.js';
import { flyToDataSource } from './flyToDataSource.js';
// One color per file, in import order, from the whiteboard palette.
const COLORS = ['#39d0ff', '#ffb547', '#5dff9f', '#ff6b6b', '#8be9ff'];
/**
* Wire the Import control.
* @param {{viewer: object, load?: Function, document?: Document}} deps
* `load(file, {viewer, color})` defaults to `loadGeoFile`; tests pass a double.
* @returns {{destroy: Function, importFiles: Function, list: Function,
* setVisible: Function, flyTo: Function, remove: Function,
* diagnostics: Function}|null}
*/
export function initGeoFileImport({
viewer,
load = loadGeoFile,
document: doc = globalThis.document,
flyToSource = flyToDataSource,
navigation = null,
}) {
const button = doc?.getElementById('geo-import-button');
const input = doc?.getElementById('geo-import-input');
const listEl = doc?.getElementById('geo-import-list');
const hint = doc?.getElementById('geo-import-hint');
if (!viewer || !button || !input || !listEl) return null;
const imports = [];
const listeners = [];
let queue = Promise.resolve();
let seq = 0;
let colorAt = 0;
let destroyed = false;
let dragDepth = 0;
const listen = (target, type, listener, options) => {
if (!target?.addEventListener) return;
target.addEventListener(type, listener, options);
listeners.push([target, type, listener, options]);
};
const setHint = (text) => {
if (hint) hint.textContent = text || '';
};
const render = () => viewer.scene?.requestRender?.();
/** Which row button has focus, so a re-render can give it back. */
function focusedAction() {
const active = doc.activeElement;
if (!active || !listEl.contains?.(active)) return null;
const row = active.closest?.('[data-import-id]');
return row
? { id: row.dataset.importId, action: active.dataset?.action }
: null;
}
function restoreFocus(was) {
if (!was) return;
const rows = [...listEl.querySelectorAll('[data-import-id]')];
let row = rows.find((r) => r.dataset.importId === was.id);
// The row is gone (removed): the next row's same control, else the
// previous row's, else the Import button.
if (!row && was.index != null)
row = rows[Math.min(was.index, rows.length - 1)];
const target =
row?.querySelector?.(`button[data-action="${was.action}"]`) ||
(row ? row.querySelector?.('button') : button);
target?.focus?.();
}
function renderList(keep = focusedAction()) {
listEl.replaceChildren(
...imports.map((item) => {
const row = doc.createElement('li');
row.className = 'geo-import-row';
row.dataset.importId = item.id;
const swatch = doc.createElement('span');
// KML and KMZ draw with their own styles; their swatch says so
// instead of claiming a color the globe does not show.
swatch.className = `geo-import-swatch${item.ownStyles ? ' own-styles' : ''}`;
if (!item.ownStyles) swatch.style.background = item.color;
const name = doc.createElement('span');
name.className = 'geo-import-name';
// textContent, never markup: the name comes from the file system.
name.textContent = item.name;
name.title = `${item.name} — ${item.format.toUpperCase()}, ${item.entityCount.toLocaleString()} feature${item.entityCount === 1 ? '' : 's'}${item.ownStyles ? ', drawn with its own styles' : ''}`;
const actions = [
[
item.visible ? 'visibility' : 'visibility_off',
item.visible ? `Hide ${item.name}` : `Show ${item.name}`,
'toggle',
],
['my_location', `Fly to ${item.name}`, 'fly'],
['close', `Remove ${item.name}`, 'remove'],
].map(([icon, label, action]) => {
const btn = doc.createElement('button');
btn.type = 'button';
btn.className = 'geo-import-action';
btn.dataset.action = action;
btn.title = label;
btn.setAttribute('aria-label', label);
if (action === 'toggle')
btn.setAttribute('aria-pressed', String(item.visible));
const glyph = doc.createElement('span');
glyph.className = 'material-symbols-outlined';
glyph.setAttribute('aria-hidden', 'true');
glyph.textContent = icon;
btn.append(glyph);
return btn;
});
row.append(swatch, name, ...actions);
return row;
}),
);
listEl.classList.toggle('visible', imports.length > 0);
restoreFocus(keep);
}
const find = (id) => imports.find((item) => item.id === id) || null;
/** Show or hide one imported file. */
function setVisible(id, visible) {
const item = find(id);
if (!item) return false;
item.visible = Boolean(visible);
item.dataSource.show = item.visible;
renderList();
render();
return true;
}
// Not viewer.flyTo: it frames ground-clamped points underground until the
// destination's terrain has loaded (see ./flyToDataSource.js).
const fly = (item) => {
if (!item.visible) setVisible(item.id, true);
flyToSource(viewer, item.dataSource);
return true;
};
/**
* Fly the camera to one imported file: an immediate navigation, so Cockpit
* can refuse it and the current follow/track owner is released first.
*/
function flyTo(id) {
const item = find(id);
if (!item) return false;
if (typeof navigation?.runImmediateNavigation === 'function')
return (
navigation.runImmediateNavigation('file', () => fly(item)) !== false
);
return fly(item);
}
/** Stamp a deferred fly for an import batch; false when refused. */
const beginAutoFly = () =>
typeof navigation?.beginDeferredLocationNavigation === 'function'
? navigation.beginDeferredLocationNavigation()
: null;
/** Re-assert the stamp right before flying; false when superseded. */
const autoFlyStillOwns = (generation) => {
if (generation === false) return false;
if (generation === null) return true;
return typeof navigation?.reassertDeferredLocationNavigation === 'function'
? navigation.reassertDeferredLocationNavigation(generation) === true
: true;
};
/** Remove one imported file from the globe and the list. */
function remove(id) {
const at = imports.findIndex((item) => item.id === id);
if (at < 0) return false;
const keep = focusedAction();
if (keep) keep.index = at;
const [item] = imports.splice(at, 1);
viewer.dataSources?.remove?.(item.dataSource, true);
item.revoke?.();
renderList(keep);
render();
return true;
}
/**
* Import files, one after another. Returns one outcome per file.
* @param {Iterable<File>} files
* @returns {Promise<Array<{name: string, ok: boolean, id?: string,
* removed?: number, error?: string}>>}
*/
function importFiles(files) {
const list = [...(files || [])];
const run = queue.then(() => importBatch(list));
queue = run.catch(() => {});
return run;
}
async function importBatch(list) {
const outcomes = [];
if (destroyed || !list.length) return outcomes;
// Stamped now, re-asserted before the fly: whatever navigation happens
// while the files load (a drag, a search, Cockpit) supersedes this one.
const generation = beginAutoFly();
for (const file of list) {
if (destroyed) break;
const fileName = String(file?.name || 'file');
if (imports.length >= MAX_IMPORTED_FILES) {
outcomes.push({
name: fileName,
ok: false,
error: `At most ${MAX_IMPORTED_FILES} files at once — remove one first.`,
});
continue;
}
setHint(`Reading ${fileName}…`);
const color = COLORS[colorAt % COLORS.length];
let loaded;
try {
loaded = await load(file, { viewer, color });
} catch (error) {
outcomes.push({
name: fileName,
ok: false,
error:
error instanceof GeoImportError || error?.name === 'GeoImportError'
? error.message
: `That file could not be read${error?.message ? `: ${error.message}` : '.'}`,
});
continue;
}
if (destroyed) {
loaded.revoke?.();
break;
}
colorAt += 1;
seq += 1;
const item = {
id: `import-${seq}`,
name: loaded.name || fileName,
format: loaded.format,
ownStyles: loaded.format === 'kml' || loaded.format === 'kmz',
entityCount: loaded.entityCount,
removed: loaded.removed || 0,
color,
visible: true,
dataSource: loaded.dataSource,
revoke: loaded.revoke,
};
await viewer.dataSources.add(item.dataSource);
if (destroyed) {
viewer.dataSources.remove?.(item.dataSource, true);
item.revoke?.();
break;
}
imports.push(item);
renderList();
render();
outcomes.push({
name: fileName,
ok: true,
id: item.id,
removed: item.removed,
});
// Fly to the first file of a batch that lands, so a drop shows its
// result, unless something else took the camera meanwhile.
if (
outcomes.filter((o) => o.ok).length === 1 &&
autoFlyStillOwns(generation)
)
fly(item);
}
if (!destroyed) setHint(summarize(outcomes));
return outcomes;
}
// ---- controls ----------------------------------------------------------
input.accept = GEO_FILE_ACCEPT;
listen(button, 'click', () => input.click());
listen(input, 'change', () => {
const files = [...(input.files || [])];
// Clear the selection so choosing the same file again fires `change`.
input.value = '';
if (files.length) void importFiles(files);
});
listen(listEl, 'click', (event) => {
const btn = event.target?.closest?.('button[data-action]');
const id = btn?.closest?.('[data-import-id]')?.dataset.importId;
if (!btn || !id) return;
const action = btn.dataset.action;
if (action === 'toggle') setVisible(id, !find(id)?.visible);
else if (action === 'fly') flyTo(id);
else if (action === 'remove') remove(id);
});
// ---- drop onto the globe ----------------------------------------------
const container = viewer.container;
const hasFiles = (event) =>
[...(event.dataTransfer?.types || [])].includes('Files');
const setDragging = (on) =>
doc.body?.classList?.toggle('gev-geo-import-drag', on);
listen(container, 'dragenter', (event) => {
if (!hasFiles(event)) return;
event.preventDefault();
dragDepth += 1;
setDragging(true);
});
listen(container, 'dragover', (event) => {
if (!hasFiles(event)) return;
event.preventDefault();
if (event.dataTransfer) event.dataTransfer.dropEffect = 'copy';
});
listen(container, 'dragleave', (event) => {
if (!hasFiles(event)) return;
dragDepth = Math.max(0, dragDepth - 1);
if (!dragDepth) setDragging(false);
});
listen(container, 'drop', (event) => {
if (!hasFiles(event)) return;
event.preventDefault();
dragDepth = 0;
setDragging(false);
const files = [...(event.dataTransfer?.files || [])];
if (files.length) void importFiles(files);
});
renderList();
const handle = {
importFiles,
list: () =>
imports.map(({ id, name, format, entityCount, removed, visible }) => ({
id,
name,
format,
entityCount,
removed,
visible,
})),
setVisible,
flyTo,
remove,
diagnostics: () => ({
files: imports.length,
entities: imports.reduce((n, item) => n + item.entityCount, 0),
listeners: listeners.length,
destroyed,
}),
destroy() {
if (destroyed) return;
destroyed = true;
for (const item of imports.splice(0)) {
viewer.dataSources?.remove?.(item.dataSource, true);
item.revoke?.();
}
for (const [target, type, listener, options] of listeners.splice(0))
target.removeEventListener(type, listener, options);
setDragging(false);
listEl.replaceChildren();
listEl.classList.remove('visible');
setHint('');
if (globalThis.window?.__gevGeoImport === handle)
delete globalThis.window.__gevGeoImport;
},
};
if (globalThis.window) globalThis.window.__gevGeoImport = handle;
return handle;
}
/**
* One line for the hint after an import batch.
* @param {Array<{name: string, ok: boolean, error?: string, removed?: number}>} outcomes
* @returns {string}
*/
export function summarize(outcomes) {
const failed = outcomes.filter((o) => !o.ok);
const added = outcomes.length - failed.length;
const parts = [];
if (added) parts.push(`Added ${added} file${added === 1 ? '' : 's'}.`);
if (failed.length === 1) parts.push(`${failed[0].name}: ${failed[0].error}`);
else if (failed.length > 1)
parts.push(
`${failed.length} files could not be added (${failed[0].name}: ${failed[0].error})`,
);
const stripped = outcomes.reduce(
(n, o) => n + (o.ok ? o.removed || 0 : 0),
0,
);
if (added && stripped)
parts.push('Network links and remote images in KML were left out.');
return parts.join(' ');
}
+589
View File
@@ -0,0 +1,589 @@
// Behavioral tests for DISPLAY ▸ Import: the controller is driven against
// fake DOM and viewer doubles with an injected loader, and its observable
// effects are asserted — data sources added and removed, list rows, the hint,
// drop handling, the file limit, and what destroy() gives back. Real parsing
// and rendering run in `scripts/qa-geo-import.mjs`.
import test from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import { initGeoFileImport, summarize } from './geoFileImport.js';
import { GeoImportError, MAX_IMPORTED_FILES } from '../data/geoFileImport.js';
class FakeClassList {
constructor() {
this.names = new Set();
}
add(...n) {
n.forEach((x) => this.names.add(x));
}
remove(...n) {
n.forEach((x) => this.names.delete(x));
}
contains(n) {
return this.names.has(n);
}
toggle(n, force) {
const on = force === undefined ? !this.names.has(n) : Boolean(force);
if (on) this.names.add(n);
else this.names.delete(n);
return on;
}
}
// The one focused element across every fake document (tests run one at a time).
const focus = { active: null };
class FakeElement {
constructor(tagName = 'div', id = '') {
this.tagName = tagName.toUpperCase();
this.id = id;
this.classList = new FakeClassList();
this.dataset = {};
this.style = {};
this.attributes = new Map();
this.children = [];
this.parent = null;
this.listeners = [];
this.textContent = '';
this.value = '';
this.clicks = 0;
}
set className(v) {
this.classList = new FakeClassList();
String(v)
.split(/\s+/)
.filter(Boolean)
.forEach((n) => this.classList.add(n));
}
setAttribute(n, v) {
this.attributes.set(n, String(v));
}
getAttribute(n) {
return this.attributes.get(n) ?? null;
}
append(...kids) {
for (const k of kids) {
k.parent = this;
this.children.push(k);
}
}
replaceChildren(...kids) {
this.children = [];
this.append(...kids);
}
addEventListener(type, listener, options) {
this.listeners.push({ type, listener, options });
}
removeEventListener(type, listener) {
const at = this.listeners.findIndex(
(l) => l.type === type && l.listener === listener,
);
if (at >= 0) this.listeners.splice(at, 1);
}
emit(type, event = {}) {
for (const { listener } of this.listeners.filter((l) => l.type === type))
listener({
target: this,
preventDefault() {
this.prevented = true;
},
...event,
});
}
click() {
this.clicks += 1;
this.emit('click');
}
focus() {
focus.active = this;
}
contains(el) {
for (let at = el; at; at = at.parent) if (at === this) return true;
return false;
}
*walk() {
for (const kid of this.children) {
yield kid;
yield* kid.walk();
}
}
querySelectorAll(selector) {
if (selector !== '[data-import-id]') throw new Error(`fake: ${selector}`);
return [...this.walk()].filter((el) => el.dataset.importId);
}
querySelector(selector) {
const exact = /^button\[data-action="([a-z]+)"\]$/.exec(selector);
for (const el of this.walk()) {
if (el.tagName !== 'BUTTON') continue;
if (selector === 'button' || (exact && el.dataset.action === exact[1]))
return el;
}
return null;
}
closest(selector) {
for (let el = this; el; el = el.parent) {
if (
selector === 'button[data-action]' &&
el.tagName === 'BUTTON' &&
el.dataset.action
)
return el;
if (selector === '[data-import-id]' && el.dataset.importId) return el;
}
return null;
}
}
function setup({ load, navigation, flyToSource } = {}) {
focus.active = null;
const els = {
'geo-import-button': new FakeElement('button', 'geo-import-button'),
'geo-import-input': new FakeElement('input', 'geo-import-input'),
'geo-import-list': new FakeElement('ul', 'geo-import-list'),
'geo-import-hint': new FakeElement('span', 'geo-import-hint'),
};
const document = {
body: new FakeElement('body'),
getElementById: (id) => els[id] ?? null,
createElement: (tag) => new FakeElement(tag),
get activeElement() {
return focus.active;
},
};
const added = [];
const removed = [];
const flights = [];
let renders = 0;
const viewer = {
container: new FakeElement('div', 'cesiumContainer'),
scene: { requestRender: () => (renders += 1) },
dataSources: {
add: async (ds) => {
added.push(ds);
return ds;
},
remove: (ds, destroy) => {
removed.push({ ds, destroy });
return true;
},
},
flyTo: (target) => flights.push(target),
};
const revoked = [];
const loads = [];
const defaultLoad = async (file, opts) => {
loads.push({ file, opts });
if (file.fail) throw new GeoImportError(file.fail);
if (file.crash) throw new Error('boom');
return {
dataSource: { show: true },
name: file.name,
format: 'geojson',
entityCount: 3,
removed: file.removed || 0,
revoke: () => revoked.push(file.name),
};
};
const tool = initGeoFileImport({
viewer,
load: load || defaultLoad,
document,
...(navigation ? { navigation } : {}),
...(flyToSource ? { flyToSource } : {}),
});
return {
tool,
els,
document,
viewer,
added,
removed,
flights,
revoked,
loads,
renders: () => renders,
};
}
const file = (name, extra = {}) => ({ name, size: 10, ...extra });
const rows = (els) => els['geo-import-list'].children;
const action = (row, name) =>
row.children.find((c) => c.dataset.action === name);
test('returns null without its markup or viewer', () => {
const document = { getElementById: () => null };
assert.equal(initGeoFileImport({ viewer: {}, document }), null);
assert.equal(
initGeoFileImport({
viewer: null,
document: { getElementById: () => new FakeElement() },
}),
null,
);
});
test('the Import button opens the picker and a pick imports each file', async () => {
const t = setup();
t.els['geo-import-button'].click();
assert.equal(t.els['geo-import-input'].clicks, 1);
assert.equal(
t.els['geo-import-input'].accept,
'.geojson,.json,.kml,.kmz,.gpx',
);
const out = await t.tool.importFiles([file('a.geojson'), file('b.kml')]);
assert.deepEqual(
out.map((o) => o.ok),
[true, true],
);
assert.equal(t.added.length, 2);
assert.notEqual(
t.loads[0].opts.color,
t.loads[1].opts.color,
'each file gets its own color',
);
assert.equal(rows(t.els).length, 2);
assert.ok(t.els['geo-import-list'].classList.contains('visible'));
assert.equal(t.flights.length, 1, 'the first file of a batch is flown to');
assert.equal(t.els['geo-import-hint'].textContent, 'Added 2 files.');
assert.deepEqual(
t.tool.list().map((f) => f.name),
['a.geojson', 'b.kml'],
);
});
test('file names are text, never markup', async () => {
const t = setup();
await t.tool.importFiles([file('<img src=x onerror=alert(1)>.kml')]);
const name = rows(t.els)[0].children.find((c) =>
c.classList.contains('geo-import-name'),
);
assert.equal(name.textContent, '<img src=x onerror=alert(1)>.kml');
assert.equal(name.innerHTML, undefined);
});
test('row buttons hide, show, fly to and remove a file', async () => {
const t = setup();
await t.tool.importFiles([file('a.gpx')]);
const [{ id }] = t.tool.list();
const ds = t.added[0];
// the click lands on the glyph inside the button, as in a browser
const glyphClick = (name) =>
t.els['geo-import-list'].emit('click', {
target: action(rows(t.els)[0], name).children[0],
});
assert.equal(ds.show, true);
glyphClick('toggle');
assert.equal(ds.show, false);
assert.equal(t.tool.list()[0].visible, false);
assert.equal(
action(rows(t.els)[0], 'toggle').getAttribute('aria-pressed'),
'false',
);
glyphClick('fly');
assert.equal(ds.show, true, 'flying to a hidden file shows it');
assert.equal(t.flights.at(-1), ds);
glyphClick('remove');
assert.deepEqual(t.removed, [{ ds, destroy: true }]);
assert.deepEqual(t.revoked, ['a.gpx']);
assert.equal(t.tool.list().length, 0);
assert.ok(!t.els['geo-import-list'].classList.contains('visible'));
assert.equal(t.tool.remove(id), false);
});
test('a refused file says why and does not stop the rest of the batch', async () => {
const t = setup();
const out = await t.tool.importFiles([
file('bad.kml', { fail: 'That KML file is not valid XML.' }),
file('odd.gpx', { crash: true }),
file('good.geojson'),
]);
assert.deepEqual(
out.map((o) => o.ok),
[false, false, true],
);
assert.equal(out[0].error, 'That KML file is not valid XML.');
assert.match(out[1].error, /could not be read: boom/);
assert.match(
t.els['geo-import-hint'].textContent,
/^Added 1 file\. 2 files could not be added \(bad\.kml: That KML file is not valid XML\.\)/,
);
assert.equal(t.tool.list().length, 1);
});
test('the number of files on the globe is bounded', async () => {
const t = setup();
const many = Array.from({ length: MAX_IMPORTED_FILES + 2 }, (_, i) =>
file(`f${i}.geojson`),
);
const out = await t.tool.importFiles(many);
assert.equal(out.filter((o) => o.ok).length, MAX_IMPORTED_FILES);
assert.match(out.at(-1).error, /At most 12 files/);
assert.equal(
t.loads.length,
MAX_IMPORTED_FILES,
'files over the limit are not even read',
);
});
test('dropping files on the globe imports them; other drags are left alone', async () => {
const t = setup();
const c = t.viewer.container;
const files = { types: ['Files'], files: [file('drop.kmz')] };
let ev = { dataTransfer: { types: ['text/plain'] } };
c.emit('dragover', ev);
c.emit('dragenter', { dataTransfer: files });
assert.ok(t.document.body.classList.contains('gev-geo-import-drag'));
c.emit('dragleave', { dataTransfer: files });
assert.ok(!t.document.body.classList.contains('gev-geo-import-drag'));
c.emit('dragenter', { dataTransfer: files });
c.emit('drop', { dataTransfer: files });
assert.ok(!t.document.body.classList.contains('gev-geo-import-drag'));
await new Promise((r) => setImmediate(r));
assert.deepEqual(
t.tool.list().map((f) => f.name),
['drop.kmz'],
);
});
test('stripped KML content is mentioned once, for the batch that had it', () => {
assert.equal(
summarize([{ name: 'a.kml', ok: true, removed: 3 }]),
'Added 1 file. Network links and remote images in KML were left out.',
);
assert.equal(
summarize([{ name: 'b.geojson', ok: true, removed: 0 }]),
'Added 1 file.',
);
});
test('destroy removes every file and listener and the window handle', async () => {
globalThis.window = {};
try {
const t = setup();
assert.equal(globalThis.window.__gevGeoImport, t.tool);
await t.tool.importFiles([file('a.geojson'), file('b.geojson')]);
t.tool.destroy();
assert.equal(t.removed.length, 2);
assert.deepEqual(t.revoked.sort(), ['a.geojson', 'b.geojson']);
for (const el of [
t.viewer.container,
t.els['geo-import-button'],
t.els['geo-import-input'],
t.els['geo-import-list'],
])
assert.equal(el.listeners.length, 0);
assert.equal(rows(t.els).length, 0);
assert.equal(globalThis.window.__gevGeoImport, undefined);
assert.equal(t.tool.diagnostics().destroyed, true);
} finally {
delete globalThis.window;
}
});
test('a file that finishes loading after destroy is released, not added', async () => {
let release;
const gate = new Promise((r) => (release = r));
const revoked = [];
const t = setup({
load: async () => {
await gate;
return {
dataSource: {},
name: 'late',
format: 'kml',
entityCount: 1,
removed: 0,
revoke: () => revoked.push('late'),
};
},
});
const pending = t.tool.importFiles([file('late.kml')]);
// Imports are queued: let the batch start loading before destroy lands.
await new Promise((r) => setImmediate(r));
t.tool.destroy();
release();
await pending;
assert.equal(t.added.length, 0);
assert.deepEqual(revoked, ['late']);
});
test('the application composition wires the import and owns its teardown', () => {
const tools = fs.readFileSync(
new URL('../app/tools.js', import.meta.url),
'utf8',
);
const drawAt = tools.indexOf(
'const drawTool = initDrawTool({ viewer, annotations });',
);
const importAt = tools.indexOf('const geoFileImport = initGeoFileImport({');
assert.ok(importAt > drawAt && drawAt >= 0);
// Fly-to goes through the application shell's navigation policy.
assert.match(
tools.slice(importAt, importAt + 200),
/navigation: styleManager/,
);
assert.ok(tools.indexOf('defer(() => geoFileImport?.destroy());') > importAt);
const markup = fs.readFileSync(
new URL('./templates/display-controls.html', import.meta.url),
'utf8',
);
for (const id of [
'geo-import-button',
'geo-import-input',
'geo-import-list',
'geo-import-hint',
])
assert.ok(
markup.includes(`id="${id}"`),
`${id} missing from the DISPLAY template`,
);
});
test('two imports at once never pass the file cap: batches run one after another', async () => {
const t = setup();
const first = t.tool.importFiles(
Array.from({ length: 8 }, (_, i) => file(`a${i}.geojson`)),
);
const second = t.tool.importFiles(
Array.from({ length: 8 }, (_, i) => file(`b${i}.geojson`)),
);
const [a, b] = await Promise.all([first, second]);
assert.equal(t.tool.list().length, MAX_IMPORTED_FILES);
assert.equal(a.filter((o) => o.ok).length, 8);
assert.equal(b.filter((o) => o.ok).length, MAX_IMPORTED_FILES - 8);
assert.equal(b.filter((o) => !o.ok).length, 16 - MAX_IMPORTED_FILES);
// The second batch's loads started only after the first batch finished.
assert.deepEqual(
t.loads.map((l) => l.file.name.slice(0, 1)).join(''),
'a'.repeat(8) + 'b'.repeat(MAX_IMPORTED_FILES - 8),
);
});
function navigationDouble({ immediate = true, reassert = true } = {}) {
const calls = [];
let generation = 0;
return {
calls,
runImmediateNavigation(noun, navigate) {
calls.push(['immediate', noun]);
return immediate ? navigate(++generation) : false;
},
beginDeferredLocationNavigation() {
calls.push(['begin']);
return ++generation;
},
reassertDeferredLocationNavigation(g) {
calls.push(['reassert', g]);
return reassert;
},
};
}
test('Fly to is an immediate navigation that Cockpit can refuse', async () => {
const flown = [];
const nav = navigationDouble({ immediate: false });
const t = setup({ navigation: nav, flyToSource: (v, ds) => flown.push(ds) });
// Batch auto-fly is deferred: stamped at the start, re-asserted before flying.
await t.tool.importFiles([file('a.geojson')]);
assert.deepEqual(nav.calls, [['begin'], ['reassert', 1]]);
assert.equal(flown.length, 1);
// The button goes through the immediate route; refused, nothing flies.
const id = t.tool.list()[0].id;
assert.equal(t.tool.flyTo(id), false);
assert.deepEqual(nav.calls.at(-1), ['immediate', 'file']);
assert.equal(flown.length, 1);
const nav2 = navigationDouble();
const t2 = setup({
navigation: nav2,
flyToSource: (v, ds) => flown.push(ds),
});
await t2.tool.importFiles([file('b.geojson')]);
assert.equal(t2.tool.flyTo(t2.tool.list()[0].id), true);
assert.equal(flown.length, 3);
});
test('a slow import does not fly once something newer took the camera', async () => {
let release;
const gate = new Promise((r) => (release = r));
const flown = [];
const nav = navigationDouble({ reassert: false });
const t = setup({
navigation: nav,
flyToSource: (v, ds) => flown.push(ds),
load: async (f) => {
await gate;
return {
dataSource: {},
name: f.name,
format: 'geojson',
entityCount: 1,
removed: 0,
revoke() {},
};
},
});
const pending = t.tool.importFiles([file('slow.geojson')]);
release();
const [out] = await pending;
assert.equal(out.ok, true);
assert.deepEqual(nav.calls, [['begin'], ['reassert', 1]]);
assert.equal(flown.length, 0);
});
test('a refused deferred stamp (Cockpit) imports without flying', async () => {
const flown = [];
const nav = navigationDouble();
nav.beginDeferredLocationNavigation = () => false;
nav.reassertDeferredLocationNavigation = () => {
throw new Error('must not re-assert a refused stamp');
};
const t = setup({ navigation: nav, flyToSource: (v, ds) => flown.push(ds) });
const [out] = await t.tool.importFiles([file('a.geojson')]);
assert.equal(out.ok, true);
assert.equal(flown.length, 0);
});
test('keyboard focus stays on the control after hide, show and remove', async () => {
const t = setup();
await t.tool.importFiles([file('a.geojson'), file('b.geojson')]);
// Clicks reach the list's delegated listener, as they bubble in a page.
const press = (btn) =>
t.els['geo-import-list'].emit('click', { target: btn });
const [first] = rows(t.els);
action(first, 'toggle').focus();
press(action(first, 'toggle'));
// The list was rebuilt; focus is on the NEW toggle of the same row.
const again = rows(t.els)[0];
assert.notEqual(again, first);
assert.equal(t.document.activeElement, action(again, 'toggle'));
assert.equal(action(again, 'toggle').getAttribute('aria-pressed'), 'false');
press(action(again, 'toggle'));
assert.equal(t.document.activeElement, action(rows(t.els)[0], 'toggle'));
// Remove the first row: focus moves to the same control on the next row.
action(rows(t.els)[0], 'remove').focus();
press(action(rows(t.els)[0], 'remove'));
assert.equal(rows(t.els).length, 1);
assert.equal(t.document.activeElement, action(rows(t.els)[0], 'remove'));
// Remove the last row: focus goes back to the Import button.
press(action(rows(t.els)[0], 'remove'));
assert.equal(t.document.activeElement, t.els['geo-import-button']);
});
test('KML and KMZ rows say they keep their own styles instead of showing a color', async () => {
const t = setup({
load: async (f) => ({
dataSource: {},
name: f.name,
format: f.name.endsWith('.kml') ? 'kml' : 'gpx',
entityCount: 2,
removed: 0,
revoke() {},
}),
});
await t.tool.importFiles([file('a.kml'), file('b.gpx')]);
const [kml, gpx] = rows(t.els);
assert.ok(kml.children[0].classList.contains('own-styles'));
assert.equal(kml.children[0].style.background, undefined);
assert.match(kml.children[1].title, /drawn with its own styles/);
assert.ok(!gpx.children[0].classList.contains('own-styles'));
assert.ok(gpx.children[0].style.background);
});
+74
View File
@@ -1008,6 +1008,80 @@ body.gev-drawing #cesiumContainer canvas {
cursor: crosshair;
}
/* DISPLAY ▸ Import: one row per imported file, and the drop-target outline */
.geo-import-list {
display: none;
flex-direction: column;
gap: 4px;
margin: 6px 0 0;
padding: 0;
list-style: none;
}
.geo-import-list.visible {
display: flex;
}
.geo-import-row {
display: flex;
align-items: center;
gap: 6px;
min-width: 0;
}
.geo-import-swatch {
flex: 0 0 8px;
height: 8px;
border-radius: 2px;
}
/* KML/KMZ draw with the styles written in the file, so the swatch shows a
hatch instead of a color the globe does not use. */
.geo-import-swatch.own-styles {
background: repeating-linear-gradient(
45deg,
var(--text-dim) 0 1px,
transparent 1px 3px
);
box-shadow: inset 0 0 0 1px var(--text-dim);
}
.geo-import-name {
flex: 1 1 auto;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-family: var(--font-mono);
font-size: 10px;
letter-spacing: 0.6px;
color: var(--text-primary);
}
.geo-import-action {
flex: 0 0 auto;
display: inline-flex;
align-items: center;
justify-content: center;
width: 22px;
height: 22px;
padding: 0;
background: transparent;
border: 1px solid rgba(255, 255, 255, 0.14);
border-radius: 6px;
color: var(--text-dim);
cursor: pointer;
}
.geo-import-action:hover,
.geo-import-action:focus-visible {
color: var(--text-primary);
border-color: var(--accent);
}
.geo-import-action .material-symbols-outlined {
font-size: 14px;
}
.geo-import-hint:empty {
display: none;
}
body.gev-geo-import-drag #cesiumContainer {
outline: 2px dashed var(--accent);
outline-offset: -6px;
}
/* Proximity / All segmented control under the 3D toggle */
#models3d-mode-row {
min-width: 184px;
+9
View File
@@ -147,6 +147,15 @@
<span class="draw-hint" id="draw-hint" aria-live="polite"></span>
</div>
</div>
<div class="pp-toggle-group" id="geo-import-group">
<button class="pp-toggle-btn" id="geo-import-button" type="button" title="Show your own GeoJSON, KML, KMZ or GPX files on the globe (or drop them onto it). Files stay in this browser.">
<span class="pp-icon material-symbols-outlined" aria-hidden="true">upload_file</span>
<span class="pp-label">Import</span>
</button>
<input type="file" id="geo-import-input" accept=".geojson,.json,.kml,.kmz,.gpx" multiple hidden aria-label="Choose GeoJSON, KML, KMZ or GPX files">
<ul class="geo-import-list" id="geo-import-list" aria-label="Imported files"></ul>
<span class="draw-hint geo-import-hint" id="geo-import-hint" aria-live="polite"></span>
</div>
<div class="pp-toggle-group">
<button class="pp-toggle-btn" id="celestial-toggle" title="Celestial ring — reveal the full globe" aria-pressed="false">
<span class="pp-icon material-symbols-outlined" aria-hidden="true">flare</span>