benmichael/gods-eye-view
1
0 Fork 0
mirror of https://github.com/bilawalsidhu/gods-eye-view.git synced 2026-11-12 06:12:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(voice): isolate gemini live provider core

Browse Source
This commit is contained in:
manjunath committed 2026-10-19 21:40:40 +05:30
2 parents 8abc99aa0a 0f51eb21ba
commit 10916227cb
655 files changed
+45946 -26483
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+10 -15
View File
@@ -73,7 +73,7 @@ Start with the included data sources, then add your own. Each layer is a separat
- **🔭 Satellite passes:** Ask by voice when any loaded satellite next rises over you: rise, peak and set times, and whether you can see it.
- **🔎 Analyst answers:** Count, filter and rank satellites, datacenters and dams by voice, alongside flights, ships, fires and quakes; answers say when a feed is stale or on a fallback.
- **🧭 Tilt and North Up:** Switch between straight-down and a 35° oblique, or put north at the top, while a tracked target stays centered.
- **🔍 Keyless search:** Type coordinates or a bundled city or landmark to fly there with no network request; other names use Google when configured, then Photon and Nominatim.
- **🔍 Keyless search:** Type decimal coordinates, a two-dimensional WGS84 `geo:` URI (for example, `geo:39.7392,-104.9903`), or a bundled city or landmark to fly there with no geocoder request; other names use Google when configured, then Photon and Nominatim.
- **📦 Shareable scenes:** Import, preview and share Director scenes as files or bundles, with camera anchors, authored moves and data packs.
- **🌊 Nepal flood scene:** Replay the Bhote Koshi flood: flood path, witness sources and before-and-after imagery (bundled data is non-commercial; see [DATA_SOURCES.md](DATA_SOURCES.md)).
@@ -309,6 +309,10 @@ _Why cockpit mode exists: you're riding a real aircraft over real terrain — an
## 🎙️ Talk to It
The standalone app offers **OpenAI** and **Gemini** in the GEV MIC provider chooser. OpenAI is the initial default; your choice is saved in this browser. To use Gemini, add `GEMINI_API_KEY` in Provider Settings or your local `.env` file, then select Gemini. `GEMINI_LIVE_MODEL` defaults to `gemini-3.8-live`. The permanent key stays on the local server, which issues a constrained, single-use session credential.
Changing provider ends the current conversation. If voice is on, the new provider starts with the same open-mic or push-to-talk mode; a held Space key is not carried over. Gemini uses the same map actions and captions, but has no OpenAI cost meter or MINI tier. When Gemini reports that its session has expired, start voice again for a new conversation. The provider chooser does not change the OpenAI-backed AI HUD summary.
> Voice can use an **OpenAI API key** or, on the same machine, an opt-in signed-in **ChatGPT/Codex OAuth** session. The API-key path remains the default. Selecting ChatGPT OAuth in Provider Settings reuses an existing local sign-in or starts the Codex browser sign-in flow and selects OAuth when it completes. The OpenAI key still drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move.
The OAuth option is experimental. OpenAI documents ChatGPT plan usage for the [Responses API](https://developers.openai.com/siwc/token-sharing-open-source/models-and-inference); a supported third-party Realtime contract for reusing Codex tokens has not been confirmed. A successful token mint does not establish billing or support terms.
@@ -377,7 +381,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi
| 🌍 **Earthquakes** | Global seismic activity, last 24h | USGS | 🟢 |
| ◍ **Air quality** | Live AQHI on the published 1–10+ health-risk scale, with a Weather-panel card showing the band legend and the highest readings in view. **Canada only** — ECCC is the first registered provider | ECCC MSC GeoMet | 🟢 |
| 🚗 **Traffic** | Simulated vehicles on OSM roads. With TomTom, live flow speeds drive the simulation and congestion colors below ~8 km; individual vehicle positions are not live observations | TomTom + OSM | 🟢 simulation · 🟡 live flow speeds |
| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement) and opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
| 📹 **CCTV Mesh** | ~4,000 public cameras (shared cap) projected _into_ the 3D space — Austin · Texas (TxDOT) · California (Caltrans) · London (TfL) · Ontario (511) · Finland (Fintraffic) · British Columbia (DriveBC) · Estonia (Tallinn, Tarktee) · Delaware (DelDOT live video) · New South Wales (Live Traffic NSW) · Calgary · Norway (Statens vegvesen) · Toronto · King County (WA) · Washington (WSDOT) · Catalonia (Servei Català de Trànsit), plus opt-in New York City (NYC DOT TMC, needs your own NYC DOT agreement), opt-in Lithuania (Via Lietuva / Eismoinfo, photo licence unconfirmed) and opt-in Florida (FL511, individual personal use only; other use needs FDOT's written consent). Positions are published; poses are estimated priors **you calibrate by dragging a gizmo on the camera itself** | City APIs | 🟢 |
| 📷 **Mapped ALPR Cameras** | License-plate-reader camera locations tagged by OpenStreetMap contributors, loaded one city-sized view at a time, with **SHOW NEAREST**. Locations and tags only: no plate data, no video | OpenStreetMap (incl. DeFlock mapping) | 🟢 |
| 📷 **Street Level** | Street-level photos behind one panel, one chip per provider: coverage on the streets, image cones, an embedded viewer and **FOLLOW**. Mapillary, Google Street View, KartaView and Panoramax | Mapillary · Google · KartaView · Panoramax | 🟢 KartaView, Panoramax · 🟡 Mapillary token · 🔴 Google Maps key |
| 📻 **Radio** | Geolocated world radio with an **analog tuner** — drag the needle across up to 750 stations and the globe flies to each broadcaster | Radio Browser / broadcasters | 🟢 |
@@ -385,7 +389,7 @@ Twenty layers and map sources. **Eighteen have a keyless path.** Some offer addi
| 🚲 **Bikeshare** | Live station availability | GBFS | 🟢 |
| 🧭 **Directions** | Click A and B on the globe for a street-following drive, walk or cycle route draped on the terrain with turn-by-turn steps — then FLY the camera along it. No key, no geocoder, no mic | OSRM on FOSSGIS servers (OpenStreetMap) | 🟢 |
| 🔥 **Active Fires** | Live NASA FIRMS detections, trailing 24h | NASA FIRMS | 🟡 |
| 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🟡 optional token raises the allowance) |
| 🚀 **Space Missions** | Rolling 30-day launches with payload, stage, and recovery detail | Launch Library 2 | 🟢 (🔴 optional paid token raises the allowance) |
| 🎖️ **Mapped Installations** | Viewport-bounded military-site context from community mapping — incomplete by nature, and labeled that way | OpenStreetMap | 🟢 |
| 🌬️ **Wind** | Animated 10 m forecast wind with an optional speed, temperature or pressure color field and a reading at map center | NOAA GFS / ECMWF IFS | 🟢 |
| 🌦️ **Observed Weather** | Rain radar (contiguous US), Satellite clouds (GOES regional or global infrared, Clouds only or Full image) and Lightning density (Americas and Pacific) on one history timeline | NOAA nowCOAST | 🟢 |
@@ -527,9 +531,11 @@ _What the TomTom key buys you: rush-hour density painted on the city — then di
| | Key | Why | Get it |
| --- | -------------------- | ------------------------------------------------------------- | -------------------------------------------------- |
| 🟡 | **OpenSky** | ✈️ More flight-polling credits (🟢 anonymous works without) | [opensky-network.org](https://opensky-network.org) |
| 🟡 | **Launch Library 2** | 🚀 Higher space-missions request allowance (🟢 works without) | [thespacedevs.com](https://thespacedevs.com) |
| 🔴 | **Launch Library 2** | 🚀 Optional paid token for higher limits (🟢 no key needed for public access) | [thespacedevs.com/supportus](https://thespacedevs.com/supportus) |
Add these if you need higher polling allowances.
Leave `LL2_API_TOKEN` blank for standard public access. LL2 tokens are optional
paid Patreon upgrades for higher limits; there is no free API-key signup.
`npm run doctor` reports Node/npm readiness, the primary provider routes, and
where each configured provider was found without printing credential values.
@@ -662,14 +668,3 @@ One heads-up from the inside: build in this space for a week and you learn that
**🌐 God's Eye View. No place left behind.**
</div>
### Gemini voice provider
For Gemini, create a separate API key in [Google AI Studio](https://aistudio.google.com/apikey) and add it as `GEMINI_API_KEY` in **Provider Settings**. The Google Maps key does not configure Gemini voice. See Google's [API key setup and security guidance](https://ai.google.dev/gemini-api/docs/api-key). The rest of the app works without either voice key.
Gemini uses the Developer API's Live API with `gemini-3.8-live` by default (`GEMINI_LIVE_MODEL` overrides it on the server). AI Studio is where you create its key. Free-tier availability depends on the model and project quotas; see the [Gemini pricing](https://ai.google.dev/gemini-api/docs/pricing) and [rate limits](https://ai.google.dev/gemini-api/docs/rate-limits). A connection or quota failure appears in the voice error tray. The [ephemeral token](https://ai.google.dev/gemini-api/docs/live-api/ephemeral-tokens) must start its one new session within 60 seconds and is configured to expire after 30 minutes, but that token lifetime does not promise an uninterrupted 30-minute conversation: Google limits a Live connection to [around 10 minutes](https://ai.google.dev/gemini-api/docs/live-api/session-management). This version intentionally does not resume the session. When Gemini sends `goAway`, GEV stops admitting new turns and drains the current turn until its reply finishes or the supplied deadline expires, then releases local resources. Start voice again to request a fresh token and a new conversation. The same explicit restart is required before a session exceeds 256 distinct tool-call identities; old identities remain blocked for that connection.
For a clear standalone spoken or typed request to switch voice providers, the active model says in its normal voice, “I can’t switch voice providers directly. Hover over the Voice panel and use the provider dropdown.” No tool can change the selected provider. Authoritative typed requests and recognized OpenAI speech block app actions; Gemini native speech uses current socket, session, local-turn and call ownership, with captions treated as advisory. An accepted fresh `set_voice_provider` call refuses not-yet-started tools in that Gemini turn without undoing completed work. These refusals do not write the saved provider preference, reconnect voice or remount a session. Ambiguous, quoted, negated, conditional and compound text remains ordinary conversation. The manual dropdown is the only provider-switching path. Gemini captures at the device rate, resamples input to 16 kHz and plays output through a separate 24 kHz context. Microphone permission is obtained before minting its short-admission token. Congestion may drop obsolete microphone or image frames; reliable control and tool results have bounded queue and recovery limits.
Gemini Live is available alongside OpenAI in the standalone app. Choose a supported provider, then use the existing microphone button or hold Space. The API-key provider setup remains server-owned. Gemini has no OpenAI cost estimate or spend cap; OpenAI retains its configured cost controls.