refactor: separate voice controls and configurable Realtime connections

This commit is contained in:
Sameh Khamis committed 2026-09-13 21:54:51 -07:00
1 parent 0ba00f4662
commit 1a1cbadf5b
21 files changed
+3126 -2793

No files matched your search

+23
View File
@@ -149,3 +149,26 @@ selects compatible OSRM base URLs while preserving coordinate/span limits.
`createRegionalPlaceProvider({ endpoint, requestJson })` supplies the existing
serialized Nominatim implementation. Configuration is per instance, including
routing and regional caches. Request parameters cannot override these endpoints.
### Voice connections and controls
Voice controls accept an action runner and a controller factory through
`createVoiceCommands` (`./voice/commands`). The default Realtime controller owns
microphone tracks, playback, push-to-talk, tool cancellation and radio handoff.
Its backend supplies `requestToken({ tier, signal })` and
`negotiate({ offerSdp, credential, signal })`. The Realtime-compatible adapter
(`./voice/realtime-backend`) accepts separate token and connection transports
and endpoints. Only the short-lived client secret reaches the SDP endpoint.
Stopping or ending the application lifetime aborts connection requests and
rejects delayed responses; a new start requests a fresh secret. Secret expiry
limits connection creation and does not describe the connected session lifetime.
`createStandaloneApplication({ voice })` passes these construction options to
the controls. An incompatible protocol needs a separate controller adapter;
changing an endpoint alone does not translate protocol messages. No alternate
model is bundled by this extraction. The Node Realtime provider accepts
`realtime: { endpoint, models: { standard, mini }, resolveApiKey, fetchImpl }`;
existing environment variables remain the default configuration. Model choices
come from server configuration. Keep secret keys in the server adapter.
Tool schemas, model defaults and cost estimates are unchanged. Unknown model
IDs retain the existing conservative estimate until their rates are registered.
+7
View File
@@ -1,5 +1,12 @@
# God's Eye View Current State
Voice controls compose a supplied action runner and connection controller.
Realtime token and SDP requests live in a configurable backend, with independent
transports and cancellation through response parsing. Stop and application
teardown abort pending connections; reconnect requests a new client secret.
Microphone, radio handoff, tool schemas and default model behavior are preserved.
Geospatial lookups are composed through `src/search`: forward/reverse geocoding,
text/nearby search and routing use configurable providers/endpoints. Annotation,
HUD and voice consumers share the configured service. Existing Google/Photon