diff --git a/changelog.d/1460-blender-camera-interchange.md b/changelog.d/1460-blender-camera-interchange.md new file mode 100644 index 0000000000..eb8f77773c --- /dev/null +++ b/changelog.d/1460-blender-camera-interchange.md @@ -0,0 +1,3 @@ +- Export saved Director scenes as camera-only USD (`.usda`) stages that Blender + imports natively, with ENU meter coordinates, lens in USD units, rational fps + and the same runtime shot timing as Director seeking (UBN606, #1460). diff --git a/docs/BLENDER-CAMERAS.md b/docs/BLENDER-CAMERAS.md new file mode 100644 index 0000000000..8231cacb81 --- /dev/null +++ b/docs/BLENDER-CAMERAS.md @@ -0,0 +1,138 @@ +# GEV cameras in Blender + +Export the cameras of one saved Director scene as a camera-only USD stage +(`.usda`, plain text) and open it with Blender's built-in importer. The export +needs no API key, network, browser or Blender add-on and contains no imagery or +geometry, so use independently licensed assets for your set. + +## Export a saved scene + +Save a Director project as JSON, then run from the repository root: + +```sh +mkdir -p output +node tools/export-blender-camera.mjs --input scene.json --output output/cameras.usda --scene my-scene-id --origin 39.7,-105,1600 --fps 30000/1001 --width 1920 --height 1080 --horizontal-fov-deg 70 +``` + +Paths are relative to the current directory; `output/` is ignored by git. +Omit `--scene` only when the document contains exactly one scene. `--output` +is required and must be a new file ending in `.usda`; the tool refuses to +overwrite it. No environment files are read. + +`--origin` is latitude, longitude and **WGS84 ellipsoid height in meters**. +Positions and sampled paths must remain within 100 km of that origin. Heights +are never implicitly converted from mean sea level or above ground level. +Positions must be complete; legacy documents without saved IDs or positions +need to be saved again in GEV before export. + +Projection and fps are required because Director documents do not record a +viewport lens, aspect ratio or video frame rate. `--horizontal-fov-deg` is the +horizontal perspective FOV in degrees, even for portrait output. If deriving it +from a Cesium `PerspectiveFrustum`, use +`2 * atan(tan(fovy / 2) * aspectRatio) * 180 / π` (`fovy` is in radians). +Do not assume `frustum.fov` is horizontal: it is the wider-dimension FOV. +The optional `--sensor-width-mm` defaults to 36; focal length is +`sensorWidthMm / (2 * tan(horizontalFovRadians / 2))`. + +The exporter supports integer fps or an integer ratio such as `30000/1001`, +with an effective rate from 1 to 240. Image dimensions are integers from 4 to +16384, pixels are square, horizontal FOV is 1 to 179 degrees and sensor width +is 1 to 100 mm. A lens shorter than Blender's 1 mm minimum is rejected. +Input must be a regular file within the existing 5 MiB scene-document limit; +output is limited to 100,000 frames and 64 MiB. Every shot needs at least one +sampled frame. + +## Timing and camera semantics + +Timing is the Director's **runtime** seek timing. The exporter loads the +document as Director does, so ordinary shots fly for at least 0.2 s, and takes +each shot's length and hold from `shotRuntimeTiming` in +`src/scenes/shotPresentation.js` with the default scene packs. Registered hold +overrides, minimum holds and evidence-sequence durations therefore move later +shots as they do when seeking in the app. Live playback can run longer when it +waits for a layer's media. Saved shots are never rewritten; each camera +records both the authored and the runtime hold. + +Each frame's camera is the Director's `sceneSeekState` pose at that time. +Named anchors and explicit `move.from` endpoints resolve through the existing +camera owner, and explicit linear/cubic moves use its sampler. Ordinary shots +interpolate cubically, by the shortest angle, from the previous shot's camera; +a shot that opens the scene starts at its own camera. A cut can be authored as +an explicit move with identical start/end poses. + +The bridge does not reproduce Cesium's implicit live `flyTo` trajectory, +provider readiness, layer-owned camera motion, lens changes, visual effects, +media or geometry. Export authored camera moves when repeatable playback +framing is required. This feature does not depend on GEV video export. + +Frame 1 samples time zero. Frame `n` samples `(n - 1) * denominator / numerator` +seconds. The final sample precedes the exclusive scene end; frame count is the +ceiling of duration times fps, with rounding noise at integer boundaries removed. +Shot ranges are half open: `frameStart <= frame < frameEndExclusive`. A shot +boundary switches at the first sampled time at or after the boundary. + +## What the USD stage contains + +- Stage metadata `upAxis = "Z"` and `metersPerUnit = 1`. Scene X/Y/Z are + east/north/up in meters around `--origin`, in the geodetic ENU frame (axes + from the WGS84 ellipsoid normal at the origin's latitude and longitude), so + Blender applies no axis or unit conversion. +- USD time codes equal frame numbers: `startTimeCode = 1`, `endTimeCode` is the + last frame, and `timeCodesPerSecond`/`framesPerSecond` hold the effective + rate. The exact fps ratio is kept as text. +- A root `GEV` transform at identity, with one perspective `Camera` per shot + named `Shot_001_`, `Shot_002_` and so on. Each camera has an + `xformOp:transform` matrix time sample on every frame of its own shot range, + and holds its first or last pose outside that range. Camera local X points + right, local Y up and local -Z forward, which is both the USD and the Blender + camera convention. +- Lens attributes in USD's units: `focalLength`, `horizontalAperture` and + `verticalAperture` are in **tenths of a scene unit**, so with + `metersPerUnit = 1` one unit of these attributes is 100 mm. A 25 mm lens on + a 36 mm sensor is written as `focalLength = 0.25` and + `horizontalAperture = 0.36`. The vertical aperture is the sensor width times + `height / width`, so it matches the output aspect. +- Clipping range 0.01 m to 200 km; adjust it for your set if needed. +- Source scene and shot IDs, titles, pack identifiers, authored and runtime + timing, frame ranges, origin, fps ratio, image size and data-pack + attribution text, as `userProperties:` attributes. No referenced assets are + fetched. + +## Import into Blender + +1. In Blender, choose **File > Import > Universal Scene Description**, select + the `.usda` file and keep **Set Frame Range** enabled. +2. Set **Output Properties > Format** to the exported `--width`, `--height` and + frame rate, with **Aspect X:Y** at 1:1. USD carries the time codes but not + render resolution, and Blender does not take its scene frame rate from the + file. +3. Pick a shot camera as the active camera, or bind cameras to timeline markers + at each shot's `gevFrameStart` (shown under **Object Data Properties > + Custom Properties**). + +How Blender maps the lens: its USD importer multiplies `focalLength` and the +apertures by `100 * metersPerUnit` to get millimeters. With this exporter's +`metersPerUnit = 1` that is ×100, so the imported **Focal Length** and +**Sensor Width** equal the `lensMm` and `--sensor-width-mm` values. Blender +chooses horizontal sensor fit when the horizontal aperture is at least as large +as the vertical one and vertical fit otherwise, so portrait exports import with +vertical fit and the same field of view. This mapping is in Blender's +`usd_reader_camera.cc`. + +Blender imports animated USD transforms as a **Transform Cache** constraint +that keeps the absolute path of the imported `.usda`. Put the file in its +permanent place before importing, or update the path in the constraint after +moving it. For a self-contained file, select the cameras and use **Object > +Animation > Bake Action…** with **Visual Keying** and **Clear Constraints**. +Blender stores object transforms in single precision: up to about 4 mm of +rounding at 100 km, less closer to the origin. + +## Checks + +```sh +node --test src/tooling/blender*.test.mjs +``` + +The tests compare the export with an independent Cesium camera and the +Director's runtime timing, and parse the stage with OpenUSD's `usdcat` when it +is installed. diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index ce83af7497..55ed117912 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1577,6 +1577,10 @@ seek and keeps the last settled position. Undeclared operator layers remain untouched, and the shot is not rebuilt. See [camera directions](DIRECTOR-CAMERA.md). +An opt-in offline [Blender camera bridge](BLENDER-CAMERAS.md) exports a saved +Director scene as a camera-only USD stage (`.usda`) with Director seek +interpolation and runtime shot timing; format, units and limits are in that guide. + Realtime voice composes separate connection, response/tool, Radio handoff, input and audio-meter, cost, viewport-context and diagnostic owners. The existing controller exports and session/backend contracts remain available. Tool protocol, diff --git a/scripts/format-scope.json b/scripts/format-scope.json index 9ad4bc0e33..5f9943f711 100644 --- a/scripts/format-scope.json +++ b/scripts/format-scope.json @@ -1,4 +1,13 @@ [ + "src/tooling/blender-cut-time.test.mjs", + "src/tooling/blender-import-timing.test.mjs", + "src/tooling/blender-legacy-numbers.test.mjs", + "src/tooling/blenderCamera.test.mjs", + "src/tooling/blenderCameraBom.test.mjs", + "src/tooling/blenderCameraFifoInput.test.mjs", + "src/tooling/blenderCameraPartialWrite.test.mjs", + "src/tooling/blenderCameraStderr.test.mjs", + "tools/export-blender-camera.mjs", ".prettierrc.json", "package.json", "scripts/format-scope.json", diff --git a/src/scenes/shotPresentation.js b/src/scenes/shotPresentation.js index 04759fc3fb..3c80428936 100644 --- a/src/scenes/shotPresentation.js +++ b/src/scenes/shotPresentation.js @@ -153,8 +153,8 @@ export function effectiveShotHoldSec( ); } -/** Estimate flight plus effective hold without mutating saved layer parameters. */ -export function shotRuntimeDurationSec(scene, shot, packs) { +/** Resolve a shot's effective hold and flight-plus-hold seconds without mutating saved layer parameters. */ +export function shotRuntimeTiming(scene, shot, packs) { const states = deepClone(shot?.layers || {}); const sourceRecipe = packs.recipeForShot(shot); for (const state of Object.values(states)) { @@ -168,8 +168,14 @@ export function shotRuntimeDurationSec(scene, shot, packs) { }; } } - return ( - (shot?.durationSec || DEFAULT_SHOT_DURATION_SEC) + - effectiveShotHoldSec(scene, shot, packs, states) - ); + const holdSec = effectiveShotHoldSec(scene, shot, packs, states); + return { + holdSec, + durationSec: (shot?.durationSec || DEFAULT_SHOT_DURATION_SEC) + holdSec, + }; +} + +/** Estimate flight plus effective hold without mutating saved layer parameters. */ +export function shotRuntimeDurationSec(scene, shot, packs) { + return shotRuntimeTiming(scene, shot, packs).durationSec; } diff --git a/src/tooling/blender-cut-time.test.mjs b/src/tooling/blender-cut-time.test.mjs new file mode 100644 index 0000000000..73c4277a7c --- /dev/null +++ b/src/tooling/blender-cut-time.test.mjs @@ -0,0 +1,100 @@ +// Reproduction: shot start/end times and timeline duration come from a different sum than frame ownership. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { sceneSeekState } from '../director/timeline.js'; +import { + effectiveShotHoldSec, + shotRuntimeDurationSec, +} from '../scenes/shotPresentation.js'; +import { createDefaultScenePacks } from '../scenes/packs/defaults.js'; +import { exportCameraUsda } from './blenderCamera.js'; + +const packs = createDefaultScenePacks(); +const pose = (heading) => ({ + lat: 0, + lon: 0, + alt: 0, + heading, + pitch: 0, + roll: 0, + altitudeReference: 'ellipsoid', +}); +const shot = (id, heading, durationSec) => ({ + id, + durationSec, + holdSec: 0, + camera: pose(heading), + move: { from: pose(heading), easing: 'linear' }, +}); +const options = { + origin: { lat: 0, lon: 0, height: 0 }, + fps: { numerator: 30, denominator: 1 }, + width: 1920, + height: 1080, + horizontalFovDegrees: 90, +}; +const dur = (s, sh) => shotRuntimeDurationSec(s, sh, packs); +const hold = (s, sh) => effectiveShotHoldSec(s, sh, packs); + +for (const [name, count, seconds] of [ + ['nine shot cut', 9, 0.7], + ['six shot seek total', 6, 0.2], +]) { + test(name, () => { + const scene = { + id: 'scene', + title: 'S', + shots: Array.from({ length: count }, (_, i) => + shot(`s${i + 1}`, i * 10, seconds), + ), + }; + const { samples, usda } = exportCameraUsda( + { version: 6, scenes: [scene] }, + options, + ); + for (const [attribute, field] of [ + ['gevStartTimeSec', 'startTimeSec'], + ['gevEndTimeSec', 'endTimeSec'], + ]) { + const times = [ + ...usda.matchAll(new RegExp(`userProperties:${attribute} = (.+)`, 'g')), + ]; + assert.deepEqual( + times.map(([, value]) => Number(value)), + samples.shots.map((shot) => shot[field]), + attribute, + ); + } + // Each frame's camera still follows Director seeking. + const total = scene.shots.reduce((sum, sh) => sum + dur(scene, sh), 0); + for (const frame of samples.frames) + assert.equal( + frame.shotId, + sceneSeekState(scene, frame.timeSec / total, dur, hold).shot.id, + ); + // The published duration seeks to the same owner. + for (const frame of samples.frames) + assert.equal( + frame.shotId, + sceneSeekState( + scene, + frame.timeSec / samples.timeline.durationSec, + dur, + hold, + ).shot.id, + `published total, frame ${frame.frame}`, + ); + // Documented rule: a shot starts at the first sampled time at or after its start time. + for (const s of samples.shots) { + assert.ok( + samples.frames[s.frameStart - 1].timeSec >= s.startTimeSec, + `${s.id} starts before startTimeSec`, + ); + if (s.frameStart > 1) + assert.ok( + samples.frames[s.frameStart - 2].timeSec < s.startTimeSec, + `${s.id}: frame ${s.frameStart - 1} at ${samples.frames[s.frameStart - 2].timeSec} s is at/after startTimeSec ${s.startTimeSec} but belongs to the previous shot`, + ); + } + }); +} diff --git a/src/tooling/blender-import-timing.test.mjs b/src/tooling/blender-import-timing.test.mjs new file mode 100644 index 0000000000..622a43cc1f --- /dev/null +++ b/src/tooling/blender-import-timing.test.mjs @@ -0,0 +1,64 @@ +// Reproduction: the exporter samples the raw document, Director seeks the normalized one. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { sceneSeekState } from '../director/timeline.js'; +import { parseSceneDocument } from '../director/document.js'; +import { normalizeProject } from '../scenes/project.js'; +import { + effectiveShotHoldSec, + shotRuntimeDurationSec, +} from '../scenes/shotPresentation.js'; +import { createDefaultScenePacks } from '../scenes/packs/defaults.js'; +import { sampleSceneCameras } from './blenderCamera.js'; + +const packs = createDefaultScenePacks(); +const pose = (heading) => ({ + lat: 0, + lon: 0, + alt: 0, + heading, + pitch: 0, + roll: 0, +}); +const options = { + origin: { lat: 0, lon: 0, height: 0 }, + fps: { numerator: 30, denominator: 1 }, + width: 1920, + height: 1080, + horizontalFovDegrees: 90, +}; + +test('Director import timing', () => { + const text = JSON.stringify({ + version: 6, + scenes: [ + { + id: 'scene', + title: 'S', + shots: [ + { id: 'A', durationSec: 0.1, holdSec: 0, camera: pose(0) }, + { id: 'B', durationSec: 1, holdSec: 0, camera: pose(90) }, + ], + }, + ], + }); + const loaded = normalizeProject(parseSceneDocument(text)).scenes[0]; + assert.equal(loaded.shots[0].durationSec, 0.2, 'Director loads A as 0.2 s'); + let samples; + try { + samples = sampleSceneCameras(parseSceneDocument(text), options); + } catch (error) { + // Rejecting a timing-changing document is an acceptable fix. + assert.match(error.message, /duration/i); + return; + } + const dur = (s, shot) => shotRuntimeDurationSec(s, shot, packs); + const hold = (s, shot) => effectiveShotHoldSec(s, shot, packs); + const total = loaded.shots.reduce((sum, shot) => sum + dur(loaded, shot), 0); + for (const frame of samples.frames) { + const reference = sceneSeekState(loaded, frame.timeSec / total, dur, hold) + .shot.id; + assert.equal(frame.shotId, reference, `frame ${frame.frame}`); + } + assert.equal(samples.shots[1].frameStart, 7); +}); diff --git a/src/tooling/blender-legacy-numbers.test.mjs b/src/tooling/blender-legacy-numbers.test.mjs new file mode 100644 index 0000000000..3d56505248 --- /dev/null +++ b/src/tooling/blender-legacy-numbers.test.mjs @@ -0,0 +1,52 @@ +// Reproduction: version 1-2 documents with numeric strings parse but do not export. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { parseSceneDocument } from '../director/document.js'; +import { sampleSceneCameras } from './blenderCamera.js'; + +const options = { + origin: { lat: 45, lon: 10, height: 0 }, + fps: { numerator: 30, denominator: 1 }, + width: 1920, + height: 1080, + horizontalFovDegrees: 90, +}; +const doc = (shotChanges, cameraChanges) => + JSON.stringify({ + version: 2, + scenes: [ + { + id: 'scene', + title: 'S', + shots: [ + { + id: 'A', + durationSec: 4, + holdSec: 0, + ...shotChanges, + camera: { + lat: 45, + lon: 10, + alt: 100, + heading: 0, + pitch: -30, + roll: 0, + ...cameraChanges, + }, + }, + ], + }, + ], + }); + +test('legacy numeric fields', () => { + const numeric = sampleSceneCameras(parseSceneDocument(doc({}, {})), options); + for (const [label, text] of [ + ['string coordinates', doc({}, { lat: '45', lon: '10', alt: '100' })], + ['string duration', doc({ durationSec: '4' }, {})], + ]) { + const parsed = parseSceneDocument(text); // the Director validator accepts it + const samples = sampleSceneCameras(parsed, options); + assert.deepEqual(samples.frames, numeric.frames, label); + } +}); diff --git a/src/tooling/blenderCamera.js b/src/tooling/blenderCamera.js new file mode 100644 index 0000000000..59218cb95f --- /dev/null +++ b/src/tooling/blenderCamera.js @@ -0,0 +1,403 @@ +/** Camera-only USD interchange. No viewer, providers, environment files or terrain access. */ +import * as C from 'cesium'; +import { validateSceneDocument } from '../director/document.js'; +import { resolveCameraPose } from '../director/camera.js'; +import { sceneSeekState } from '../director/timeline.js'; +import { headingPitchRollFrame } from '../geo/quaternion.js'; +import { normalizeProject } from '../scenes/project.js'; +import { shotRuntimeTiming } from '../scenes/shotPresentation.js'; +import { createDefaultScenePacks } from '../scenes/packs/defaults.js'; + +export const CAMERA_USD_SCHEMA = 'gev.camera-usd.v1'; +export const MAX_CAMERA_FRAMES = 100000; +export const MAX_CAMERA_USDA_BYTES = 64 * 1024 * 1024; +// USD camera lens attributes are tenths of a scene unit. With metersPerUnit = 1 +// one tenth is 100 mm; Blender's importer multiplies by 100 * metersPerUnit. +const MM_PER_USD_LENS_UNIT = 100; + +function number(value, label, min, max) { + if ( + typeof value !== 'number' || + !Number.isFinite(value) || + value < min || + value > max + ) { + throw new RangeError( + `${label} must be a finite number in [${min}, ${max}]`, + ); + } + return value; +} + +function integer(value, label, min, max) { + if (!Number.isInteger(number(value, label, min, max))) + throw new RangeError(`${label} must be an integer`); + return value; +} + +function position(pose, heightKey = 'alt') { + if (!pose) throw new TypeError('A complete camera position is required'); + number(pose.lat, 'lat', -90, 90); + number(pose.lon, 'lon', -180, 180); + number(pose[heightKey], heightKey, -12000, 1e9); +} + +/** Convert Cesium's WGS84/heading-pitch-roll pose to a fixed Blender ENU frame. + * Angles are degrees; alt and origin.height are WGS84 ellipsoid meters. + * The returned columns are the camera's local X (right), Y (up), Z (back), + * which is the USD and Blender camera convention (looking down local -Z). + */ +export function cameraPoseToEnu(pose, origin, label = 'Camera') { + position(pose); + position(origin, 'height'); + // Short-arc sampling can pass 360 degrees even when both saved endpoints do not. + for (const key of ['heading', 'roll']) number(pose[key], key, -720, 720); + number(pose.pitch, 'pitch', -90, 90); + const xyz = C.Cartesian3.fromDegrees( + pose.lon, + pose.lat, + pose.alt, + C.Ellipsoid.WGS84, + ); + const anchor = C.Cartesian3.fromDegrees( + origin.lon, + origin.lat, + origin.height, + C.Ellipsoid.WGS84, + ); + const distance = C.Cartesian3.distance(xyz, anchor); + if (distance > 100000) + throw new RangeError( + `${label} is ${(distance / 1000).toFixed(1)} km from the origin (limit 100 km)`, + ); + // Geodetic ENU (as in PROJ): axes from the ellipsoid normal at the origin's + // lat/lon, origin raised along it. Cesium's frame at a raised point tilts. + const frame = C.Transforms.eastNorthUpToFixedFrame( + C.Cartesian3.fromDegrees(origin.lon, origin.lat, 0, C.Ellipsoid.WGS84), + ); + const inverse = C.Matrix4.inverseTransformation( + C.Matrix4.setTranslation(frame, anchor, frame), + new C.Matrix4(), + ); + const local = C.Matrix4.multiplyByPoint(inverse, xyz, new C.Cartesian3()); + // Same basis as Cesium Camera.setView3D, including its heading - PI/2 offset. + const body = headingPitchRollFrame(xyz, { + heading: C.Math.toRadians(pose.heading) - Math.PI / 2, + pitch: C.Math.toRadians(pose.pitch), + roll: C.Math.toRadians(pose.roll), + }); + const toLocal = (axis) => + C.Matrix4.multiplyByPointAsVector(inverse, axis, new C.Cartesian3()); + const forward = toLocal(body.x); + const up = toLocal(body.z); + const right = C.Cartesian3.cross(forward, up, new C.Cartesian3()); + const array = (v) => [v.x, v.y, v.z]; + return { + position: array(local), + basisColumns: [ + array(right), + array(up), + array(C.Cartesian3.negate(forward, new C.Cartesian3())), + ], + }; +} + +function identity(value, label) { + if (typeof value !== 'string' || !value.trim()) + throw new TypeError(`${label} requires a saved ID`); + return value; +} + +/** Resolve each shot's runtime seconds and hold once, as Director seeking does. */ +export function sceneRuntimeTiming( + scene, + scenePacks = createDefaultScenePacks(), +) { + const byShot = new Map(); + for (const shot of scene.shots) { + const timing = shotRuntimeTiming(scene, shot, scenePacks); + number(timing.durationSec, `Shot ${shot.id} runtime duration`, 0, 1e7); + number(timing.holdSec, `Shot ${shot.id} runtime hold`, 0, 1e7); + byShot.set(shot, timing); + } + return { + durationFor: (_scene, shot) => byShot.get(shot).durationSec, + holdFor: (_scene, shot) => byShot.get(shot).holdSec, + }; +} + +/** Sample one saved scene with Director seek interpolation and runtime seek timing. + * Projection is explicit because scene documents do not store viewport lenses. + * Frame 1 samples time zero; the final sample precedes the exclusive end time. + */ +export function sampleSceneCameras( + project, + { + sceneId, + origin, + fps, + width, + height, + horizontalFovDegrees, + sensorWidthMm = 36, + scenePacks = createDefaultScenePacks(), + } = {}, +) { + validateSceneDocument(project); + for (const key of ['numerator', 'denominator']) + integer(fps?.[key], `fps.${key}`, 1, 1000000); + const rate = number(fps.numerator / fps.denominator, 'frame rate', 1, 240); + integer(width, 'width', 4, 16384); + integer(height, 'height', 4, 16384); + number(horizontalFovDegrees, 'horizontal FOV degrees', 1, 179); + number(sensorWidthMm, 'sensor width mm', 1, 100); + const candidates = project.scenes.filter( + (scene) => sceneId === undefined || scene.id === sceneId, + ); + if (candidates.length !== 1) { + const ids = project.scenes + .slice(0, 10) + .map(({ id }) => id) + .join(', '); + throw new Error( + sceneId === undefined + ? `Choose one scene with --scene: ${ids}` + : `No scene has ID ${sceneId}; scenes: ${ids}`, + ); + } + const saved = candidates[0]; + identity(saved.id, 'Scene'); + if (!saved.shots.length) throw new Error('Scene has no camera shots'); + for (const shot of saved.shots) { + identity(shot.id, 'Shot'); + const pose = resolveCameraPose(saved, shot.camera); + if (!pose || [pose.lat, pose.lon, pose.alt].includes(undefined)) + throw new TypeError( + `Shot ${shot.id} requires a saved position; save the scene again in GEV`, + ); + } + // Director seeks the normalized project: legacy numeric text becomes + // numbers and ordinary shots fly for at least 0.2 s. + const scene = normalizeProject({ version: project.version, scenes: [saved] }) + .scenes[0]; + const timing = sceneRuntimeTiming(scene, scenePacks); + let elapsed = 0; + const shots = scene.shots.map((shot) => { + const label = `Shot ${shot.id}`; + cameraPoseToEnu(resolveCameraPose(scene, shot.camera), origin, label); + if (shot.move) + cameraPoseToEnu(resolveCameraPose(scene, shot.move.from), origin, label); + // Running sums in sceneSeekState's order, so published seconds agree + // with the frame each shot owns. + const startTimeSec = elapsed; + elapsed += timing.durationFor(scene, shot); + return { + id: shot.id, + title: shot.title, + ...(shot.sourcePackId + ? { + sourcePackId: shot.sourcePackId, + sourcePackVersion: shot.sourcePackVersion ?? null, + } + : {}), + startTimeSec, + endTimeSec: elapsed, + flightDurationSec: shot.durationSec, + authoredHoldSec: shot.holdSec, + runtimeHoldSec: timing.holdFor(scene, shot), + frameStart: 0, + frameEndExclusive: 0, + }; + }); + // Ceiling of duration times fps. The tolerance bounds the rounding error of + // decimal flight and hold seconds and their sums, so only that noise at a + // frame boundary is dropped, never a real fraction of a frame. + const exactFrames = elapsed * rate; + const frameCount = + Math.abs(exactFrames - Math.round(exactFrames)) < + Number.EPSILON * (2 * shots.length + 1) * exactFrames + ? Math.round(exactFrames) + : Math.ceil(exactFrames); + if (frameCount > MAX_CAMERA_FRAMES) + throw new RangeError(`Export exceeds ${MAX_CAMERA_FRAMES} camera frames`); + if (frameCount < 1) throw new Error('Scene has no sampled frame at this fps'); + const frames = []; + for (let frame = 1; frame <= frameCount; frame += 1) { + const timeSec = ((frame - 1) * fps.denominator) / fps.numerator; + const state = sceneSeekState( + scene, + timeSec / elapsed, + timing.durationFor, + timing.holdFor, + ); + const shot = shots[state.shotIndex]; + if (!shot.frameStart) shot.frameStart = frame; + shot.frameEndExclusive = frame + 1; + frames.push({ + frame, + timeSec, + shotId: shot.id, + ...cameraPoseToEnu(state.camera, origin, `Shot ${shot.id}`), + }); + } + for (const shot of shots) { + if (!shot.frameStart) + throw new Error(`Shot ${shot.id} has no sampled frame at this fps`); + } + const lensMm = number( + sensorWidthMm / (2 * Math.tan(C.Math.toRadians(horizontalFovDegrees) / 2)), + 'Blender lens mm', + 1, + 10000, + ); + return { + source: { + sceneId: scene.id, + sceneTitle: saved.title || '', + attributions: (scene.dataPacks || []).map(({ attribution }) => + structuredClone(attribution), + ), + }, + coordinates: { + origin: { lat: origin.lat, lon: origin.lon, height: origin.height }, + }, + camera: { + sensorWidthMm, + sensorHeightMm: (sensorWidthMm * height) / width, + lensMm, + }, + timeline: { + fps: { numerator: fps.numerator, denominator: fps.denominator }, + width, + height, + frameStart: 1, + frameEnd: frameCount, + durationSec: elapsed, + }, + shots, + frames, + }; +} + +/** Round a finite double to 15 significant digits and normalize signed zero. */ +function usdNumber(value) { + const rounded = Number(value.toPrecision(15)); + return Object.is(rounded, -0) ? '0' : String(rounded); +} + +function usdString(value) { + return `"${String(value) + .replace(/\\/g, '\\\\') + .replace(/"/g, '\\"') + .replace(/\n/g, '\\n') + .replace(/\r/g, '\\r') + .replace(/\t/g, '\\t') + .replace( + /[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/g, + (char) => `\\x${char.charCodeAt(0).toString(16).padStart(2, '0')}`, + )}"`; +} + +/** USD prim names are identifiers; saved IDs stay in custom properties. */ +function shotPrimName(shot, index) { + const safe = shot.id.replace(/[^A-Za-z0-9_]/g, '_').slice(0, 40); + return `Shot_${String(index + 1).padStart(3, '0')}_${safe}`; +} + +/** USD matrices are row-vector: rows are camera X, Y, Z axes, then translation. */ +function usdMatrix({ position: p, basisColumns: [right, up, back] }) { + const row = (values, w) => `(${[...values, w].map(usdNumber).join(', ')})`; + return `( ${row(right, 0)}, ${row(up, 0)}, ${row(back, 0)}, ${row(p, 1)} )`; +} + +/** Write sampled cameras as a plain-text USD stage that Blender imports natively. + * Z-up meters, one perspective camera per shot, time codes equal to frame numbers. + */ +export function cameraSamplesToUsda(samples) { + const { source, coordinates, camera, timeline, shots, frames } = samples; + integer(timeline.width, 'width', 4, 16384); + integer(timeline.height, 'height', 4, 16384); + integer(timeline.frameStart, 'frameStart', 1, MAX_CAMERA_FRAMES); + integer(timeline.frameEnd, 'frameEnd', 1, MAX_CAMERA_FRAMES); + for (const shot of shots) { + integer(shot.frameStart, 'frameStart', 1, MAX_CAMERA_FRAMES); + integer( + shot.frameEndExclusive, + 'frameEndExclusive', + 2, + MAX_CAMERA_FRAMES + 1, + ); + } + const rate = timeline.fps.numerator / timeline.fps.denominator; + const json = (value) => usdString(JSON.stringify(value)); + const out = [ + '#usda 1.0', + '(', + ' defaultPrim = "GEV"', + ` doc = ${usdString("God's Eye View Director cameras: ENU meters around a WGS84 ellipsoid origin; time codes are frame numbers.")}`, + ` endTimeCode = ${timeline.frameEnd}`, + ` framesPerSecond = ${rate}`, + ' metersPerUnit = 1', + ` startTimeCode = ${timeline.frameStart}`, + ` timeCodesPerSecond = ${rate}`, + ' upAxis = "Z"', + ')', + '', + 'def Xform "GEV"', + '{', + ` custom string userProperties:gevSchema = ${usdString(CAMERA_USD_SCHEMA)}`, + ` custom string userProperties:gevSceneId = ${usdString(source.sceneId)}`, + ` custom string userProperties:gevSceneTitle = ${usdString(source.sceneTitle)}`, + ` custom string userProperties:gevOrigin = ${json(coordinates.origin)}`, + ` custom string userProperties:gevFps = ${usdString(`${timeline.fps.numerator}/${timeline.fps.denominator}`)}`, + ` custom int userProperties:gevWidth = ${timeline.width}`, + ` custom int userProperties:gevHeight = ${timeline.height}`, + ` custom string userProperties:gevAttributions = ${json(source.attributions)}`, + ]; + shots.forEach((shot, index) => { + out.push( + '', + ` def Camera ${usdString(shotPrimName(shot, index))}`, + ' {', + ' float2 clippingRange = (0.01, 200000)', + ` float focalLength = ${usdNumber(camera.lensMm / MM_PER_USD_LENS_UNIT)}`, + ` float horizontalAperture = ${usdNumber(camera.sensorWidthMm / MM_PER_USD_LENS_UNIT)}`, + ` float verticalAperture = ${usdNumber(camera.sensorHeightMm / MM_PER_USD_LENS_UNIT)}`, + ' token projection = "perspective"', + ` custom string userProperties:gevShotId = ${usdString(shot.id)}`, + ` custom string userProperties:gevShotTitle = ${usdString(shot.title)}`, + ...(shot.sourcePackId + ? [ + ` custom string userProperties:gevSourcePackId = ${usdString(shot.sourcePackId)}`, + ` custom string userProperties:gevSourcePackVersion = ${usdString(shot.sourcePackVersion ?? '')}`, + ] + : []), + ` custom int userProperties:gevFrameStart = ${shot.frameStart}`, + ` custom int userProperties:gevFrameEndExclusive = ${shot.frameEndExclusive}`, + ` custom double userProperties:gevStartTimeSec = ${number(shot.startTimeSec, 'startTimeSec', 0, Infinity)}`, + ` custom double userProperties:gevEndTimeSec = ${number(shot.endTimeSec, 'endTimeSec', 0, Infinity)}`, + ` custom double userProperties:gevFlightDurationSec = ${usdNumber(shot.flightDurationSec)}`, + ` custom double userProperties:gevAuthoredHoldSec = ${usdNumber(shot.authoredHoldSec)}`, + ` custom double userProperties:gevRuntimeHoldSec = ${usdNumber(shot.runtimeHoldSec)}`, + ' matrix4d xformOp:transform.timeSamples = {', + ); + for (let f = shot.frameStart; f < shot.frameEndExclusive; f += 1) { + out.push(` ${f}: ${usdMatrix(frames[f - 1])},`); + } + out.push( + ' }', + ' uniform token[] xformOpOrder = ["xformOp:transform"]', + ' }', + ); + }); + out.push('}', ''); + return out.join('\n'); +} + +export function exportCameraUsda(project, options) { + const samples = sampleSceneCameras(project, options); + const usda = cameraSamplesToUsda(samples); + if (Buffer.byteLength(usda) > MAX_CAMERA_USDA_BYTES) + throw new RangeError('Camera USD exceeds 64 MiB'); + return { usda, samples }; +} diff --git a/src/tooling/blenderCamera.test.mjs b/src/tooling/blenderCamera.test.mjs new file mode 100644 index 0000000000..9888782ae1 --- /dev/null +++ b/src/tooling/blenderCamera.test.mjs @@ -0,0 +1,635 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import * as Cesium from 'cesium'; +import { + mkdtempSync, + writeFileSync, + readFileSync, + rmSync, + existsSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { spawnSync } from 'node:child_process'; +import { sceneSeekState } from '../director/timeline.js'; +import { + effectiveShotHoldSec, + shotRuntimeDurationSec, +} from '../scenes/shotPresentation.js'; +import { createDefaultScenePacks } from '../scenes/packs/defaults.js'; +import { createScenePackRegistry } from '../scenes/packs/registry.js'; + +const adapter = await import('./blenderCamera.js'); + +const origin = { lat: 0, lon: 0, height: 0 }; +const pose = (changes = {}) => ({ + lat: 0, + lon: 0, + alt: 0, + heading: 0, + pitch: 0, + roll: 0, + ...changes, +}); +const options = (changes = {}) => ({ + origin, + fps: { numerator: 30, denominator: 1 }, + width: 1920, + height: 1080, + horizontalFovDegrees: 90, + ...changes, +}); +const project = ( + shots = [{ id: 'north', durationSec: 1, holdSec: 0, camera: pose() }], +) => ({ + version: 6, + scenes: [{ id: 'scene', title: 'Synthetic cameras', shots }], +}); +const near = (a, b, eps = 1e-8) => + a.forEach((v, i) => assert.ok(Math.abs(v - b[i]) < eps, `${a} != ${b}`)); +const camera = (changes = {}, opts = {}) => + adapter.sampleSceneCameras( + project([{ id: 'camera', durationSec: 1, camera: pose(changes) }]), + options(opts), + ).frames[0]; + +test('cardinal cameras use Blender X right, Y up, negative Z forward', () => { + for (const [heading, right, back] of [ + [0, [1, 0, 0], [0, -1, 0]], + [90, [0, -1, 0], [-1, 0, 0]], + [180, [-1, 0, 0], [0, 1, 0]], + [270, [0, 1, 0], [1, 0, 0]], + ]) { + const result = camera({ heading }); + near(result.basisColumns[0], right); + near(result.basisColumns[1], [0, 0, 1]); + near(result.basisColumns[2], back); + } + near(camera({ pitch: -90 }).basisColumns[2], [0, 0, 1]); + near(camera({ roll: 90 }).basisColumns[1], [1, 0, 0]); +}); + +test('ENU translation uses WGS84 meters and ellipsoid heights', () => { + near(camera({ alt: 100 }).position, [0, 0, 100]); + near( + camera( + { lat: 39.7, lon: -105, alt: 2600 }, + { origin: { lat: 39.7, lon: -105, height: 1600 } }, + ).position, + [0, 0, 1000], + 1e-6, + ); + const east = camera({ lon: 0.001 }).position; + assert.ok(Math.abs(east[0] - 111.3194907876) < 1e-6); + assert.ok(Math.abs(east[2] + 0.0009714458) < 1e-6); +}); + +test('changed tangent frames and nonzero roll match an independent Cesium Camera', () => { + const anchor = { lat: 39.7, lon: -105, height: 1600 }; + const p = pose({ + lat: 39.73, + lon: -104.97, + alt: 2400, + heading: 47, + pitch: -29, + roll: 37, + }); + const reference = new Cesium.Camera({ + drawingBufferWidth: 1920, + drawingBufferHeight: 1080, + mapProjection: new Cesium.GeographicProjection(), + mode: Cesium.SceneMode.SCENE3D, + }); + reference.setView({ + destination: Cesium.Cartesian3.fromDegrees(p.lon, p.lat, p.alt), + orientation: { + heading: Cesium.Math.toRadians(p.heading), + pitch: Cesium.Math.toRadians(p.pitch), + roll: Cesium.Math.toRadians(p.roll), + }, + }); + const inverse = Cesium.Matrix4.inverseTransformation( + Cesium.Transforms.eastNorthUpToFixedFrame( + Cesium.Cartesian3.fromDegrees(anchor.lon, anchor.lat, 0), + ), + new Cesium.Matrix4(), + ); + const arr = (v) => [v.x, v.y, v.z]; + const local = (v) => + arr( + Cesium.Matrix4.multiplyByPointAsVector( + inverse, + v, + new Cesium.Cartesian3(), + ), + ); + const actual = camera(p, { origin: anchor }); + near(actual.basisColumns[0], local(reference.rightWC)); + near(actual.basisColumns[1], local(reference.upWC)); + near( + actual.basisColumns[2], + local( + Cesium.Cartesian3.negate(reference.directionWC, new Cesium.Cartesian3()), + ), + ); +}); + +test('explicit horizontal lens works for landscape and portrait output', () => { + for (const [width, height] of [ + [1920, 1080], + [1080, 1920], + ]) { + const m = adapter.sampleSceneCameras(project(), options({ width, height })); + assert.ok(Math.abs(m.camera.lensMm - 18) < 1e-10); + assert.ok( + Math.abs(m.camera.sensorHeightMm - (36 * height) / width) < 1e-10, + ); + } +}); + +test('anchors, cubic moves, cuts, holds and rational frame timing are sampled', () => { + const p = project([ + { + id: 'move', + durationSec: 0.4, + holdSec: 0.2, + camera: { anchorId: 'end', heading: 10, pitch: 0, roll: 20 }, + move: { + from: { + ...pose({ lon: 179.999, heading: 350 }), + altitudeReference: 'ellipsoid', + }, + easing: 'cubic-in-out', + }, + }, + { + id: 'cut', + durationSec: 0.2, + holdSec: 0, + camera: pose({ lon: -179.999, heading: 90 }), + }, + ]); + p.scenes[0].anchors = [ + { + id: 'end', + lat: 0, + lon: -179.999, + alt: 20, + altitudeReference: 'ellipsoid', + }, + ]; + const m = adapter.sampleSceneCameras( + p, + options({ + origin: { lat: 0, lon: 180, height: 0 }, + fps: { numerator: 30000, denominator: 1001 }, + }), + ); + assert.equal(m.frames.length, 24); + assert.equal(m.source.sceneId, 'scene'); + assert.deepEqual( + m.shots.map((s) => s.id), + ['move', 'cut'], + ); + assert.equal(m.shots[0].endTimeSec, 0.6000000000000001); + assert.equal(m.frames[18].shotId, 'cut'); + assert.equal(m.frames[18].timeSec, (18 * 1001) / 30000); + near(m.frames[12].position, m.frames[17].position); + assert.equal(p.scenes[0].shots[0].camera.anchorId, 'end'); +}); + +test('invalid or ambiguous input fails before export', () => { + for (const change of [ + { fps: { numerator: 0, denominator: 1 } }, + { horizontalFovDegrees: undefined }, + { width: 0 }, + { origin: { lat: 0, lon: NaN, height: 0 } }, + { horizontalFovDegrees: 180 }, + ]) { + assert.throws(() => adapter.sampleSceneCameras(project(), options(change))); + } + assert.throws(() => camera({ lon: 20 }), /100 km/); + const multiple = project(); + multiple.scenes.push({ ...multiple.scenes[0], id: 'other' }); + assert.throws( + () => adapter.sampleSceneCameras(multiple, options()), + /scene/i, + ); + assert.throws( + () => + adapter.sampleSceneCameras( + project([{ id: 'missing', camera: { heading: 0 } }]), + options(), + ), + /lat|position/i, + ); + assert.throws( + () => + adapter.sampleSceneCameras( + project([{ id: 'long', camera: pose(), durationSec: 86400 }]), + options(), + ), + /frame/i, + ); +}); + +test('projection settings cannot be silently clamped by Blender', () => { + assert.throws( + () => adapter.sampleSceneCameras(project(), options({ width: 1 })), + /width/, + ); + assert.throws( + () => + adapter.sampleSceneCameras( + project(), + options({ horizontalFovDegrees: 179, sensorWidthMm: 1 }), + ), + /lens/i, + ); +}); + +test('CLI works in paths with spaces and refuses overwrites or invalid projection', () => { + const directory = mkdtempSync(join(tmpdir(), 'gev camera ')); + try { + const input = join(directory, 'saved scene.json'); + const output = join(directory, 'camera stage.usda'); + const scenes = project(); + scenes.scenes.push({ ...scenes.scenes[0], id: 'other' }); + writeFileSync(input, JSON.stringify(scenes)); + const script = fileURLToPath( + new URL('../../tools/export-blender-camera.mjs', import.meta.url), + ); + const args = [ + script, + '--input', + input, + '--output', + output, + '--scene', + 'other', + // A leading minus must not read as a missing --origin value. + '--origin', + '-0,0,0', + '--fps', + '30000/1001', + '--width', + '1920', + '--height', + '1080', + '--horizontal-fov-deg', + '60', + ]; + const first = spawnSync(process.execPath, args, { encoding: 'utf8' }); + assert.equal(first.status, 0, first.stderr); + assert.match(first.stdout, /CAMERA_EXPORT_OK/); + const contents = readFileSync(output, 'utf8'); + assert.match(contents, /^#usda 1\.0\n/); + assert.match(contents, /userProperties:gevFps = "30000\/1001"/); + assert.match(contents, /userProperties:gevSceneId = "other"/); + const unknownOutput = join(directory, 'unknown.usda'); + const unknown = [...args]; + unknown[unknown.indexOf('other')] = 'missing'; + unknown[unknown.indexOf(output)] = unknownOutput; + const missingScene = spawnSync(process.execPath, unknown, { + encoding: 'utf8', + }); + assert.equal(missingScene.status, 1); + assert.match( + missingScene.stderr, + /No scene has ID missing; scenes: scene, other/, + ); + assert.equal(existsSync(unknownOutput), false); + const repeat = spawnSync(process.execPath, args, { encoding: 'utf8' }); + assert.equal(repeat.status, 1); + assert.equal(readFileSync(output, 'utf8'), contents); + const jsonOutput = join(directory, 'cameras.json'); + const wrongExtension = [...args]; + wrongExtension[wrongExtension.indexOf(output)] = jsonOutput; + const rejected = spawnSync(process.execPath, wrongExtension, { + encoding: 'utf8', + }); + assert.equal(rejected.status, 1); + assert.match(rejected.stderr, /\.usda/); + assert.equal(existsSync(jsonOutput), false); + const invalidOutput = join(directory, 'invalid.usda'); + const invalid = [...args]; + invalid[invalid.indexOf(output)] = invalidOutput; + invalid[invalid.length - 1] = '180'; + assert.equal( + spawnSync(process.execPath, invalid, { encoding: 'utf8' }).status, + 1, + ); + assert.equal(existsSync(invalidOutput), false); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); + +test('long decimal timelines do not gain a rounding-only frame', () => { + // At 30 fps the accumulated duration is 63900.00000000164 frames. + const p = project( + Array.from({ length: 1000 }, (_, i) => ({ + id: `shot-${i}`, + durationSec: 2.13, + holdSec: 0, + camera: pose(), + })), + ); + const m = adapter.sampleSceneCameras(p, options()); + assert.equal(m.frames.length, 63900); + assert.equal(m.shots[999].frameEndExclusive, 63901); + // 1.06 + 2.99 s is 243.00000000000006 frames at 60 fps. + const held = adapter.sampleSceneCameras( + project([{ id: 'held', durationSec: 1.06, holdSec: 2.99, camera: pose() }]), + options({ fps: { numerator: 60, denominator: 1 } }), + ); + assert.equal(held.frames.length, 243); +}); + +test('a frame before the exclusive scene end is never dropped', () => { + for (const [durationSec, numerator, denominator, frames] of [ + [666.333, 23976, 1000, 15977], + [366.333, 2997, 100, 10980], + [2000.000001, 30, 1, 60001], + ]) { + const m = adapter.sampleSceneCameras( + project([{ id: 'shot', durationSec, holdSec: 0, camera: pose() }]), + options({ fps: { numerator, denominator } }), + ); + assert.equal(m.frames.length, frames, `${durationSec} s`); + assert.ok(m.frames.at(-1).timeSec < durationSec); + } + // The last shot owns only the final frame at t = 120 s. + const m = adapter.sampleSceneCameras( + project([ + { id: 'a', durationSec: 119.80000003, holdSec: 0, camera: pose() }, + { id: 'b', durationSec: 0.2, holdSec: 0, camera: pose({ heading: 10 }) }, + ]), + options({ fps: { numerator: 1, denominator: 1 } }), + ); + assert.deepEqual( + [m.shots[1].frameStart, m.shots[1].frameEndExclusive], + [121, 122], + ); +}); + +const runtimeSeek = (scene, timeSec, durationSec, packs) => + sceneSeekState( + scene, + timeSec / durationSec, + (s, shot) => shotRuntimeDurationSec(s, shot, packs), + (s, shot) => effectiveShotHoldSec(s, shot, packs), + ); + +test('shot timing follows Director runtime holds, not only the saved holdSec', () => { + // The Director's Mailung trim plays a 7.65 s hold although 15 s was saved. + const p = project([ + { + id: 'mailung', + sourcePackId: 'bhote-koshi-nepal-evidence-pack', + durationSec: 4.2, + holdSec: 15, + camera: pose(), + layers: { + 'bhote-koshi-2026': { + enabled: true, + params: { presentation: 'scene-beat', beatId: 'mailung-bazzar' }, + }, + }, + }, + { id: 'next', durationSec: 1, holdSec: 0, camera: pose({ heading: 90 }) }, + ]); + const before = structuredClone(p); + const m = adapter.sampleSceneCameras(p, options()); + const [first, second] = m.shots; + assert.equal(first.authoredHoldSec, 15); + assert.equal(first.runtimeHoldSec, 7.65); + assert.equal(second.startTimeSec, 4.2 + 7.65); + // Summing authored holds would start the next shot 7.35 s (220 frames) later. + assert.equal(second.frameStart, Math.ceil((4.2 + 7.65) * 30) + 1); + assert.equal(m.timeline.durationSec, 4.2 + 7.65 + 1); + const packs = createDefaultScenePacks(); + p.scenes[0].shots.forEach((shot, i) => + assert.equal( + m.shots[i].runtimeHoldSec, + effectiveShotHoldSec(p.scenes[0], shot, packs), + ), + ); + for (const sample of m.frames) { + const state = runtimeSeek( + p.scenes[0], + sample.timeSec, + m.timeline.durationSec, + packs, + ); + assert.equal(sample.shotId, state.shot.id); + const expected = adapter.cameraPoseToEnu(state.camera, origin); + near(sample.position, expected.position); + sample.basisColumns.forEach((axis, i) => + near(axis, expected.basisColumns[i]), + ); + } + assert.deepEqual(p, before, 'export never rewrites saved shots'); +}); + +test('registered minimum holds and evidence-sequence durations shift later shots', () => { + const packs = createScenePackRegistry({ + recipes: [ + { + id: 'sample', + runtimeControlsByBeat: { + arrival: { evidenceSequence: true, evidenceSequenceDurationSec: 2.5 }, + }, + }, + ], + adapters: [ + { minimumHoldSec: (states) => (states.overview?.enabled ? 3 : 0) }, + ], + }); + const p = project([ + { + id: 'evidence', + sourcePackId: 'sample', + durationSec: 1, + holdSec: 0.5, + camera: pose(), + layers: { + sample: { + enabled: true, + params: { presentation: 'scene-beat', beatId: 'arrival' }, + }, + }, + }, + { + id: 'overview', + durationSec: 1, + holdSec: 0, + camera: pose({ heading: 45 }), + layers: { overview: { enabled: true, params: {} } }, + }, + { id: 'last', durationSec: 1, holdSec: 0, camera: pose({ heading: 90 }) }, + ]); + const m = adapter.sampleSceneCameras(p, options({ scenePacks: packs })); + assert.deepEqual( + m.shots.map((s) => [s.startTimeSec, s.runtimeHoldSec, s.frameStart]), + [ + [0, 2.5, 1], + [3.5, 3, 106], + [7.5, 0, 226], + ], + ); + assert.equal(m.frames.length, 255); + // The holds match the Director's own seek path, which resolves layer states itself. + p.scenes[0].shots.forEach((shot, i) => + assert.equal( + m.shots[i].runtimeHoldSec, + effectiveShotHoldSec(p.scenes[0], shot, packs), + ), + ); +}); + +const usdNumbers = (text) => + text.match(/-?\d+(?:\.\d+)?(?:e[-+]?\d+)?/g).map(Number); + +test('USD stage is Z-up meters with one camera per shot and per-frame matrices', () => { + const p = project([ + { + id: 'first shot', + title: 'He said "hi"\n', + durationSec: 0.2, + holdSec: 0, + camera: pose({ heading: 30, roll: 10, alt: 50 }), + }, + { + id: 'second', + durationSec: 0.1, + holdSec: 0, + camera: pose({ heading: 200, lon: 0.0005, pitch: -30 }), + }, + ]); + const { usda, samples } = adapter.exportCameraUsda(p, options()); + assert.match(usda, /^#usda 1\.0\n/); + for (const line of [ + 'upAxis = "Z"', + 'metersPerUnit = 1', + 'startTimeCode = 1', + `endTimeCode = ${samples.timeline.frameEnd}`, + 'timeCodesPerSecond = 30', + 'framesPerSecond = 30', + 'defaultPrim = "GEV"', + ]) + assert.ok(usda.includes(`\n ${line}\n`), line); + assert.match(usda, /userProperties:gevSchema = "gev\.camera-usd\.v1"/); + const cameras = usda.split('\n def Camera ').slice(1); + assert.equal(cameras.length, 2); + assert.match(cameras[0], /^"Shot_001_first_shot"/); + assert.match(cameras[1], /^"Shot_002_second"/); + // USD lens values are tenths of a scene unit (100 mm at metersPerUnit 1): + // a 90 degree horizontal FOV on a 36 mm sensor is an 18 mm lens. + assert.match(cameras[0], /\n {8}float focalLength = 0\.18\n/); + assert.match(cameras[0], /\n {8}float horizontalAperture = 0\.36\n/); + assert.match(cameras[0], /\n {8}float verticalAperture = 0\.2025\n/); + assert.match(cameras[0], /\n {8}token projection = "perspective"\n/); + assert.ok( + cameras[0].includes( + 'custom string userProperties:gevShotTitle = "He said \\"hi\\"\\n"', + ), + ); + assert.ok( + cameras[0].includes( + 'custom string userProperties:gevShotId = "first shot"', + ), + ); + cameras.forEach((text, index) => { + const shot = samples.shots[index]; + const keys = [...text.matchAll(/\n {12}(\d+): \((.*)\),/g)]; + assert.deepEqual( + keys.map(([, key]) => Number(key)), + Array.from( + { length: shot.frameEndExclusive - shot.frameStart }, + (_, i) => shot.frameStart + i, + ), + ); + for (const [, key, matrix] of keys) { + const values = usdNumbers(matrix); + const frame = samples.frames[Number(key) - 1]; + assert.equal(frame.shotId, shot.id); + const [right, up, back] = frame.basisColumns; + near(values, [...right, 0, ...up, 0, ...back, 0, ...frame.position, 1]); + } + assert.match( + text, + /uniform token\[\] xformOpOrder = \["xformOp:transform"\]/, + ); + }); +}); + +test('USD stage records rational fps and portrait apertures', () => { + const { usda, samples } = adapter.exportCameraUsda( + project(), + options({ + width: 1080, + height: 1920, + fps: { numerator: 30000, denominator: 1001 }, + }), + ); + assert.ok(usda.includes(`\n timeCodesPerSecond = ${30000 / 1001}\n`)); + assert.ok(usda.includes('userProperties:gevFps = "30000/1001"')); + assert.match(usda, /float verticalAperture = 0\.64\n/); + assert.equal(samples.timeline.frameEnd, 30); +}); + +test('a scene is selected by sceneId', () => { + const p = project(); + p.scenes.push({ + id: 'other', + title: 'Other', + shots: [ + { id: 'east', durationSec: 1, holdSec: 0, camera: pose({ lon: 0.001 }) }, + ], + }); + const m = adapter.sampleSceneCameras(p, options({ sceneId: 'other' })); + assert.equal(m.source.sceneId, 'other'); + assert.deepEqual( + m.shots.map((s) => s.id), + ['east'], + ); + near( + m.frames[0].position, + adapter.cameraPoseToEnu(pose({ lon: 0.001 }), origin).position, + ); + assert.throws( + () => adapter.sampleSceneCameras(p, options()), + /--scene: scene, other/, + ); +}); + +const usdcat = spawnSync('usdcat', ['--help']).status === 0; +test( + 'OpenUSD parses the stage', + { skip: !usdcat && 'usdcat is not installed' }, + () => { + const directory = mkdtempSync(join(tmpdir(), 'gev usd ')); + try { + const file = join(directory, 'cameras.usda'); + const p = project([ + { id: 'a', durationSec: 0.2, holdSec: 0, camera: pose() }, + { + id: 'b', + durationSec: 0.2, + holdSec: 0, + camera: pose({ heading: 90 }), + }, + ]); + writeFileSync(file, adapter.exportCameraUsda(p, options()).usda); + const result = spawnSync('usdcat', [file], { encoding: 'utf8' }); + assert.equal(result.status, 0, result.stderr); + assert.match(result.stdout, /def Camera "Shot_002_b"/); + assert.match(result.stdout, /float focalLength = 0\.18/); + assert.match(result.stdout, /\b7: \( \(/); + } finally { + rmSync(directory, { recursive: true, force: true }); + } + }, +); diff --git a/src/tooling/blenderCameraBom.test.mjs b/src/tooling/blenderCameraBom.test.mjs new file mode 100644 index 0000000000..c716cd50bb --- /dev/null +++ b/src/tooling/blenderCameraBom.test.mjs @@ -0,0 +1,96 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + mkdtempSync, + writeFileSync, + readFileSync, + rmSync, + existsSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { spawnSync, execFileSync } from 'node:child_process'; +import { SCENE_DOCUMENT_LIMITS } from '../director/document.js'; + +const cli = fileURLToPath( + new URL('../../tools/export-blender-camera.mjs', import.meta.url), +); +const pose = (c = {}) => ({ + lat: 0, + lon: 0, + alt: 10, + heading: 0, + pitch: 0, + roll: 0, + ...c, +}); +const doc = (shots, extra = {}) => ({ + version: 6, + scenes: [{ id: 'scene', title: 'Synthetic', shots, ...extra }], +}); +const one = [{ id: 'a', durationSec: 1, holdSec: 0, camera: pose() }]; +const argsFor = (input, output, fps = '30') => [ + cli, + '--input', + input, + '--output', + output, + '--origin', + '0,0,0', + '--fps', + fps, + '--width', + '1920', + '--height', + '1080', + '--horizontal-fov-deg', + '60', +]; +const dir = () => mkdtempSync(join(tmpdir(), 'gev-camera-repro-')); + +test('a UTF-8 BOM scene exports like the same scene without BOM (Director import accepts both)', () => { + const d = dir(); + try { + const body = Buffer.from(JSON.stringify(doc(one))); + const plain = join(d, 'plain.json'); + const bom = join(d, 'bom.json'); + writeFileSync(plain, body); + writeFileSync(bom, Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), body])); + const a = spawnSync(process.execPath, argsFor(plain, join(d, 'a.usda')), { + encoding: 'utf8', + }); + const b = spawnSync(process.execPath, argsFor(bom, join(d, 'b.usda')), { + encoding: 'utf8', + }); + assert.equal(a.status, 0, a.stderr); + assert.equal(b.status, 0, b.stderr); + assert.deepEqual( + readFileSync(join(d, 'b.usda')), + readFileSync(join(d, 'a.usda')), + ); + } finally { + rmSync(d, { recursive: true, force: true }); + } +}); + +test('the input file size limit includes the UTF-8 BOM', () => { + const d = dir(); + try { + const input = join(d, 'scene.json'); + const output = join(d, 'camera.usda'); + const scene = JSON.stringify(doc(one)).padEnd( + SCENE_DOCUMENT_LIMITS.bytes, + ' ', + ); + writeFileSync(input, '\uFEFF' + scene); + const result = spawnSync(process.execPath, argsFor(input, output), { + encoding: 'utf8', + }); + assert.equal(result.status, 1); + assert.match(result.stderr, /exceeds 5 MiB/); + assert.equal(existsSync(output), false); + } finally { + rmSync(d, { recursive: true, force: true }); + } +}); diff --git a/src/tooling/blenderCameraFifoInput.test.mjs b/src/tooling/blenderCameraFifoInput.test.mjs new file mode 100644 index 0000000000..183e92f61a --- /dev/null +++ b/src/tooling/blenderCameraFifoInput.test.mjs @@ -0,0 +1,76 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + mkdtempSync, + writeFileSync, + readFileSync, + rmSync, + existsSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { spawnSync, execFileSync } from 'node:child_process'; + +const cli = fileURLToPath( + new URL('../../tools/export-blender-camera.mjs', import.meta.url), +); +const pose = (c = {}) => ({ + lat: 0, + lon: 0, + alt: 10, + heading: 0, + pitch: 0, + roll: 0, + ...c, +}); +const doc = (shots, extra = {}) => ({ + version: 6, + scenes: [{ id: 'scene', title: 'Synthetic', shots, ...extra }], +}); +const one = [{ id: 'a', durationSec: 1, holdSec: 0, camera: pose() }]; +const argsFor = (input, output, fps = '30') => [ + cli, + '--input', + input, + '--output', + output, + '--origin', + '0,0,0', + '--fps', + fps, + '--width', + '1920', + '--height', + '1080', + '--horizontal-fov-deg', + '60', +]; +const dir = () => mkdtempSync(join(tmpdir(), 'gev-camera-repro-')); + +test( + 'a FIFO --input without a writer is rejected as non-regular instead of hanging', + { skip: process.platform === 'win32' }, + () => { + const d = dir(); + try { + const fifo = join(d, 'scene.json'); + execFileSync('mkfifo', [fifo]); + const out = join(d, 'out.usda'); + const r = spawnSync(process.execPath, argsFor(fifo, out), { + encoding: 'utf8', + timeout: 30000, + }); + assert.equal( + r.signal, + null, + 'CLI hung in open() and was killed by the timeout', + ); + assert.equal(r.status, 1); + assert.match(r.stderr, /regular file/); + assert.equal(existsSync(out), false); + } finally { + rmSync(d, { recursive: true, force: true }); + } + }, +); diff --git a/src/tooling/blenderCameraPartialWrite.test.mjs b/src/tooling/blenderCameraPartialWrite.test.mjs new file mode 100644 index 0000000000..d29dda6b27 --- /dev/null +++ b/src/tooling/blenderCameraPartialWrite.test.mjs @@ -0,0 +1,91 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + mkdtempSync, + writeFileSync, + readFileSync, + rmSync, + existsSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { spawnSync, execFileSync } from 'node:child_process'; + +const cli = fileURLToPath( + new URL('../../tools/export-blender-camera.mjs', import.meta.url), +); +const pose = (c = {}) => ({ + lat: 0, + lon: 0, + alt: 10, + heading: 0, + pitch: 0, + roll: 0, + ...c, +}); +const doc = (shots, extra = {}) => ({ + version: 6, + scenes: [{ id: 'scene', title: 'Synthetic', shots, ...extra }], +}); +const one = [{ id: 'a', durationSec: 1, holdSec: 0, camera: pose() }]; +const argsFor = (input, output, fps = '30') => [ + cli, + '--input', + input, + '--output', + output, + '--origin', + '0,0,0', + '--fps', + fps, + '--width', + '1920', + '--height', + '1080', + '--horizontal-fov-deg', + '60', +]; +const dir = () => mkdtempSync(join(tmpdir(), 'gev-camera-repro-')); + +test( + 'a failed write leaves no partial stage and a retry succeeds', + { skip: process.platform === 'win32' }, + () => { + const d = dir(); + try { + const input = join(d, 'scene.json'); + writeFileSync( + input, + JSON.stringify( + doc([{ id: 'a', durationSec: 60, holdSec: 0, camera: pose() }]), + ), + ); + const out = join(d, 'o.usda'); + // RLIMIT_FSIZE (32 KiB) stands in for a full disk: write() fails with EFBIG after a prefix. + const r = spawnSync( + '/bin/sh', + [ + '-c', + 'ulimit -f 64; exec "$0" "$@"', + process.execPath, + ...argsFor(input, out), + ], + { encoding: 'utf8' }, + ); + assert.equal(r.status, 1, r.stderr); + assert.equal( + existsSync(out), + false, + 'truncated stage left at the final path', + ); + const retry = spawnSync(process.execPath, argsFor(input, out), { + encoding: 'utf8', + }); + assert.equal(retry.status, 0, retry.stderr); + assert.ok(readFileSync(out, 'utf8').trimEnd().endsWith('}')); + } finally { + rmSync(d, { recursive: true, force: true }); + } + }, +); diff --git a/src/tooling/blenderCameraStderr.test.mjs b/src/tooling/blenderCameraStderr.test.mjs new file mode 100644 index 0000000000..28b1d7d026 --- /dev/null +++ b/src/tooling/blenderCameraStderr.test.mjs @@ -0,0 +1,106 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + mkdtempSync, + writeFileSync, + readFileSync, + rmSync, + existsSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { spawnSync, execFileSync } from 'node:child_process'; + +const cli = fileURLToPath( + new URL('../../tools/export-blender-camera.mjs', import.meta.url), +); +const pose = (c = {}) => ({ + lat: 0, + lon: 0, + alt: 10, + heading: 0, + pitch: 0, + roll: 0, + ...c, +}); +const doc = (shots, extra = {}) => ({ + version: 6, + scenes: [{ id: 'scene', title: 'Synthetic', shots, ...extra }], +}); +const one = [{ id: 'a', durationSec: 1, holdSec: 0, camera: pose() }]; +const argsFor = (input, output, fps = '30') => [ + cli, + '--input', + input, + '--output', + output, + '--origin', + '0,0,0', + '--fps', + fps, + '--width', + '1920', + '--height', + '1080', + '--horizontal-fov-deg', + '60', +]; +const dir = () => mkdtempSync(join(tmpdir(), 'gev-camera-repro-')); + +const oneLineNoControl = (stderr) => { + assert.equal(stderr.at(-1), 0x0a, 'ends with one newline'); + const body = stderr.subarray(0, stderr.length - 1); + assert.equal( + body.includes(0x0a), + false, + 'diagnostic split over several lines', + ); + assert.equal( + /[\x00-\x1f\x7f-\x9f]/.test(body.toString('utf8')), + false, + 'raw control byte in stderr', + ); +}; + +test('validator diagnostics from document field names are one line without raw control bytes', () => { + const d = dir(); + try { + const input = join(d, 'scene.json'); + writeFileSync(input, JSON.stringify(doc(one, { 'x\ty\nz': 1 }))); + const r = spawnSync(process.execPath, argsFor(input, join(d, 'o.usda'))); + assert.equal(r.status, 1); + assert.match(String(r.stderr), /unsupported field/); + assert.ok(String(r.stderr).includes('x\\x09y\\x0az')); + oneLineNoControl(r.stderr); + } finally { + rmSync(d, { recursive: true, force: true }); + } +}); + +test('exporter diagnostics with a shot ID are one line without raw control bytes', () => { + const d = dir(); + try { + const input = join(d, 'scene.json'); + writeFileSync( + input, + JSON.stringify( + doc([ + { id: 'a', durationSec: 0.25, holdSec: 0, camera: pose() }, + { id: 'b\tc', durationSec: 0.25, holdSec: 0, camera: pose() }, + { id: 'd', durationSec: 1.5, holdSec: 0, camera: pose() }, + ]), + ), + ); + const r = spawnSync( + process.execPath, + argsFor(input, join(d, 'o.usda'), '1'), + ); + assert.equal(r.status, 1); + assert.match(String(r.stderr), /has no sampled frame/); + assert.ok(String(r.stderr).includes('b\\x09c')); + oneLineNoControl(r.stderr); + } finally { + rmSync(d, { recursive: true, force: true }); + } +}); diff --git a/tools/README.md b/tools/README.md index 5fd2521873..f32d6aed66 100644 --- a/tools/README.md +++ b/tools/README.md @@ -1,10 +1,12 @@ # Tools -CLI scripts for fetching, rendering, and reprojecting geospatial imagery. All tools read the Google Maps API key from `.env` automatically. +CLI scripts for camera interchange and for fetching, rendering, and reprojecting +geospatial imagery. The imagery tools read the Google Maps API key from `.env` +automatically. -Output files go to `output/` by default (gitignored). +Imagery tool output files go to `output/` by default (gitignored). -When invoking these tools from a separate project, set `GEV_PROJECT_ROOT` to its +When invoking the imagery tools from a separate project, set `GEV_PROJECT_ROOT` to its absolute directory. Environment files, dependencies and relative output paths resolve there; bundled rendering HTML stays with the tool. The same setting is supported by `dev-fresh.sh`, `dev-secure.sh`, `opensky-import-client.sh` and the @@ -186,3 +188,14 @@ node tools/streetview-headings.mjs --lat 30.266476 --lon -97.73719 # 5. Render a 3D view of the same location from above node tools/cesium-render.mjs --lookat-lat 30.266476 --lookat-lon -97.73719 --pitch -90 --height 50 --fov 90 --width 2560 --height-px 1440 ``` + +--- + +## export-blender-camera.mjs + +Exports the cameras of a saved Director scene as a plain-text USD stage +(`.usda`) that Blender opens with File > Import > Universal Scene Description. +It needs no key, environment file, browser, network access or Blender add-on, +and exports no imagery or geometry. `--output` is required; paths are relative +to the current directory and `GEV_PROJECT_ROOT` is not read. Usage, format and +lens units: [Blender cameras](../docs/BLENDER-CAMERAS.md). diff --git a/tools/export-blender-camera.mjs b/tools/export-blender-camera.mjs new file mode 100644 index 0000000000..567c8cda57 --- /dev/null +++ b/tools/export-blender-camera.mjs @@ -0,0 +1,122 @@ +#!/usr/bin/env node +/** Offline scene-to-USD camera CLI. Reads JSON only; never loads .env or a viewer. */ +import { constants } from 'node:fs'; +import { open, unlink } from 'node:fs/promises'; +import { parseArgs } from 'node:util'; +import { + parseSceneDocument, + SCENE_DOCUMENT_LIMITS, +} from '../src/director/document.js'; +import { exportCameraUsda } from '../src/tooling/blenderCamera.js'; + +/** Read a regular file through one descriptor, at most one byte past the scene limit. */ +async function readSceneText(path) { + // Non-blocking, so a FIFO without a writer reaches the isFile check instead + // of hanging in open(). Windows has no O_NONBLOCK. + const file = await open( + path, + constants.O_RDONLY | (constants.O_NONBLOCK ?? 0), + ); + try { + if (!(await file.stat()).isFile()) + throw new Error('input must be a regular file'); + const buffer = Buffer.alloc(SCENE_DOCUMENT_LIMITS.bytes + 1); + let length = 0; + while (length < buffer.length) { + const { bytesRead } = await file.read( + buffer, + length, + buffer.length - length, + ); + // Decode at EOF, dropping a UTF-8 BOM as Director's File.text() does. + if (!bytesRead) + return new TextDecoder().decode(buffer.subarray(0, length)); + length += bytesRead; + } + throw new Error('Scene exceeds 5 MiB'); + } finally { + await file.close(); + } +} + +try { + // parseArgs reads a value starting with "-" as a missing argument, so join a + // southern-hemisphere `--origin -33.86,151.21,0` into `--origin=...` first. + const args = process.argv.slice(2); + for (let i = args.length - 2; i >= 0; i -= 1) { + if (args[i] === '--origin' && /^-[\d.]/.test(args[i + 1])) + args.splice(i, 2, `--origin=${args[i + 1]}`); + } + const { values } = parseArgs({ + args, + options: { + input: { type: 'string' }, + output: { type: 'string' }, + scene: { type: 'string' }, + origin: { type: 'string' }, + fps: { type: 'string' }, + width: { type: 'string' }, + height: { type: 'string' }, + 'horizontal-fov-deg': { type: 'string' }, + 'sensor-width-mm': { type: 'string', default: '36' }, + help: { type: 'boolean', default: false }, + }, + }); + if (values.help) { + console.log( + 'Usage: node tools/export-blender-camera.mjs --input scene.json --output cameras.usda --origin lat,lon,ellipsoidHeight --fps 30 (or 30000/1001) --width 1920 --height 1080 --horizontal-fov-deg 60 [--scene id] [--sensor-width-mm 36]', + ); + } else { + for (const key of [ + 'input', + 'output', + 'origin', + 'fps', + 'width', + 'height', + 'horizontal-fov-deg', + ]) { + if (!values[key]?.trim()) throw new Error(`Missing --${key}; use --help`); + } + if (!values.output.endsWith('.usda')) + throw new Error('output must be a new file ending in .usda'); + if (!/^\d+(?:\/\d+)?$/.test(values.fps)) + throw new Error('fps must be an integer or integer ratio'); + const originParts = values.origin.split(','); + if (originParts.length !== 3 || originParts.some((part) => !part.trim())) + throw new Error('origin must be lat,lon,ellipsoidHeight'); + const document = parseSceneDocument(await readSceneText(values.input)); + const [numerator, denominator = 1] = values.fps.split('/').map(Number); + const [lat, lon, height] = originParts.map(Number); + const { usda, samples } = exportCameraUsda(document, { + sceneId: values.scene, + origin: { lat, lon, height }, + fps: { numerator, denominator }, + width: Number(values.width), + height: Number(values.height), + horizontalFovDegrees: Number(values['horizontal-fov-deg']), + sensorWidthMm: Number(values['sensor-width-mm']), + }); + const file = await open(values.output, 'wx', 0o600); + try { + await file.writeFile(usda); + await file.close(); + } catch (error) { + // Remove the partial stage this run created, so a retry can succeed. + await file.close().catch(() => {}); + await unlink(values.output); + throw error; + } + console.log( + `CAMERA_EXPORT_OK: ${samples.shots.length} shots, ${samples.frames.length} frames`, + ); + } +} catch (error) { + // Field names and IDs come from the document: escape terminal control bytes. + const message = String(error.message).replace( + /[\x00-\x1f\x7f-\x9f]/g, + (char) => `\\x${char.charCodeAt(0).toString(16).padStart(2, '0')}`, + ); + console.error(`Camera export failed: ${message}`); + process.exitCode = 1; +}