mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 10:24:02 +00:00
fix(hud): ask the model only with a place label, catch flush English echoes
- Skip the model when the view has layers but no place, street or nearby label: with layers alone it invented a place. - The English-echo check treats only Latin letters as part of a word, so ids flush against kana or Han and hyphenated names are caught; it skips nominal, state words a locale keeps in English and names inside place labels. - Outside English, per-layer state ids are left out of the request. - The ja/zh cap steps back to the start of a Latin word, not the last space. - Fold the CJK cap cases into hudSummaryResponse.test.mjs.
This commit is contained in:
1 parent
7713fb8ae8
commit
dbfbfcceb9
7 files changed
+150
-107
No files matched your search
@@ -5255,10 +5255,10 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
### AI HUD Summary (June 2026)
|
||||
|
||||
- HUD `SUMMARY` readout requests a five-word intelligence-style summary from `/api/openai/hud-summary` (model `OPENAI_HUD_SUMMARY_MODEL`, default `gpt-5-nano`, minimal reasoning).
|
||||
- Input is the live basemap label context (place/street/nearby-place labels + enabled layers) and `feedProvenance`, with every layer name as the user sees it; outside English the narration note is left out, since a small model copies its English words. The model is instructed not to infer from coordinates. A view with no labels and no enabled layer is not sent: the deterministic line is shown.
|
||||
- Input is the live basemap label context (place/street/nearby-place labels + enabled layers) and `feedProvenance`, with every layer name as the user sees it; outside English the narration note and the per-layer state ids are left out (`overall` still carries the state), since a small model copies English words. The model is instructed not to infer from coordinates. A view with no place, street or nearby-place label is not sent, since with layers alone the model invents a place: the deterministic line is shown.
|
||||
- Language (October 2026): the HUD sends `locale`, which is part of the dedupe signature. The proxy keeps a shipped locale (English otherwise) and names the language in the instructions and at the start of the input; Japanese and Chinese get a 20-character phrase instead of five words.
|
||||
- The reply is structured (`text.format` JSON schema): `{summary, feedState}`, `feedState` being the snapshot state or `none`. For a view that is not nominal, a missing or different `feedState` falls back to the deterministic line. An accepted summary and the deterministic line carry the HUD's own translated state: the state word leads and the layer names trail (`STALE | … | LIVE FLIGHTS`), so the one-line readout never clips the warning; the text and then the layer names may be cut off on narrow layouts. Outside English a reply that echoes a feed-state id or an English layer name the UI translates falls back too. `get_hud_caption` builds its caption the same way and returns `feedState`.
|
||||
- Output is sanitized to five words, or for a Japanese or Chinese request locale to 20 characters (stepping back to a space rather than splitting a Latin word); falls back to the deterministic telemetry summary on error/timeout (5s abort); typewriter animation on update; a camera move during typing shows the deterministic line instead of the fragment.
|
||||
- The reply is structured (`text.format` JSON schema): `{summary, feedState}`, `feedState` being the snapshot state or `none`. For a view that is not nominal, a missing or different `feedState` falls back to the deterministic line. An accepted summary and the deterministic line carry the HUD's own translated state: the state word leads and the layer names trail (`STALE | … | LIVE FLIGHTS`), so the one-line readout never clips the warning; the text and then the layer names may be cut off on narrow layouts. Outside English a reply that echoes a feed-state id or an English layer name the UI translates falls back too, even written flush against kana or Han; a state word the locale itself keeps in English (de `FALLBACK`) and a name inside a place label do not count. `get_hud_caption` builds its caption the same way and returns `feedState`.
|
||||
- Output is sanitized to five words, or for a Japanese or Chinese request locale to 20 characters (stepping back to the start of a Latin word rather than splitting it); falls back to the deterministic telemetry summary on error/timeout (5s abort); typewriter animation on update; a camera move during typing shows the deterministic line instead of the fragment.
|
||||
- `partial` is a feed state of its own, ranked between `fallback` and `nominal`, so one partial layer no longer reads as off and partial plus nominal reads partial (HUD, voice view state, `analyst_query`, `brief`, the agent world packet). Its narration note says coverage is incomplete rather than not live.
|
||||
- A summary asked for before the camera starts moving never paints over the next view: `moveStart` marks the summary dirty and bumps its revision, as `moveEnd` does, aborts the request in flight and stops a reply still typing; neither a reply nor a fallback line (on an error, the 5 s timeout, the abort, or a failed context build) paints once the revision has moved. A dropped request forgets its signature and a timeout counts as a failure, so the next 15 s tick asks again even over the same place. An interrupted typewriter reply also forgets its signature; fully displayed answers remain reusable.
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ function toFiveWordHudSummary(value, unspaced) {
|
||||
// Other languages keep five words, even around a Han place name.
|
||||
if (!unspaced) return words.slice(0, 5).join(' ');
|
||||
// Japanese and Chinese have no spaces to count words by: cap characters,
|
||||
// stepping back to a space rather than cutting a Latin word in two.
|
||||
// stepping back to the start of a Latin word rather than cutting it in two.
|
||||
const text = words.join(' ');
|
||||
const graphemes = [
|
||||
...new Intl.Segmenter(undefined, { granularity: 'grapheme' }).segment(text),
|
||||
@@ -59,8 +59,10 @@ function toFiveWordHudSummary(value, unspaced) {
|
||||
const next = graphemes[HUD_SUMMARY_MAX_UNSPACED_CHARS];
|
||||
const inWord = (g) => /^[\p{Script=Latin}\p{N}]/u.test(g ?? '');
|
||||
if (inWord(head.at(-1)) && inWord(next)) {
|
||||
const space = head.lastIndexOf(' ');
|
||||
if (space > 0) return head.slice(0, space).join('').trim();
|
||||
let start = head.length - 1;
|
||||
while (start > 0 && inWord(head[start - 1])) start -= 1;
|
||||
// A Latin word longer than the cap is cut where the cap falls.
|
||||
if (start > 0) return head.slice(0, start).join('').trim();
|
||||
}
|
||||
return head.join('').trim();
|
||||
}
|
||||
|
||||
+40
-14
@@ -122,6 +122,29 @@ function withProvenance(text, provenance) {
|
||||
return [state, text, layers].filter(Boolean).join(' | ');
|
||||
}
|
||||
|
||||
/**
|
||||
* The English a summary outside English must not carry: feed-state ids and
|
||||
* translated layers' English names. A state word the locale keeps in English
|
||||
* (de FALLBACK) and a layer name inside a place label (Radio City) are fine.
|
||||
*/
|
||||
function summaryEnglishWords(context) {
|
||||
const places = [
|
||||
...(context.placeLabels || []),
|
||||
...(context.streetLabels || []),
|
||||
...(context.nearbyPlaceLabels || []),
|
||||
]
|
||||
.join(' ')
|
||||
.toLowerCase();
|
||||
return [
|
||||
...Object.keys(FEED_STATE_LABELS).filter(
|
||||
(state) => t(FEED_STATE_LABELS[state]).toLowerCase() !== state,
|
||||
),
|
||||
...(context.translatedLayerNames || []).filter(
|
||||
(name) => !places.includes(name.toLowerCase()),
|
||||
),
|
||||
];
|
||||
}
|
||||
|
||||
/** The name shown for a _viewBand() id. */
|
||||
function viewBandLabel(band) {
|
||||
switch (band) {
|
||||
@@ -862,13 +885,12 @@ export class IntelHUD {
|
||||
return;
|
||||
}
|
||||
if (revision !== this._summaryRevision) return;
|
||||
// Nothing to describe (no place labels, no enabled layer): the model only
|
||||
// answers about the empty input, so the plain line is shown instead.
|
||||
// No place to name: with layers alone the model invents one, so the plain
|
||||
// line (state and layers) is shown instead.
|
||||
const described = [
|
||||
context.placeLabels,
|
||||
context.streetLabels,
|
||||
context.nearbyPlaceLabels,
|
||||
context.enabledLayerLabels,
|
||||
].some((list) => list?.length);
|
||||
if (!described) {
|
||||
this._summaryDirty = false;
|
||||
@@ -916,14 +938,11 @@ export class IntelHUD {
|
||||
!hudSummaryHasEnglish(
|
||||
data.summary,
|
||||
context.locale,
|
||||
context.translatedLayerNames,
|
||||
summaryEnglishWords(context),
|
||||
)
|
||||
) {
|
||||
// The warning is the HUD's own translated state, not the model's words.
|
||||
this._lastSummaryText = withProvenance(
|
||||
data.summary,
|
||||
provenanceTag(context.feedProvenance),
|
||||
);
|
||||
this._lastSummaryText = withProvenance(data.summary, context.warning);
|
||||
}
|
||||
this._setSummaryText(this._lastSummaryText || fallbackText, animate);
|
||||
} catch (error) {
|
||||
@@ -988,9 +1007,12 @@ export class IntelHUD {
|
||||
const layerContext = hudSummaryLayerContext(
|
||||
this._dataManager?.getAll?.() || [],
|
||||
);
|
||||
const named = (layer) => ({
|
||||
// Outside English a layer's state id is left out: the model echoes it,
|
||||
// and `overall` already carries the state.
|
||||
const named = ({ feedState, ...layer }) => ({
|
||||
...layer,
|
||||
name: localizeLayerName(layer.name),
|
||||
...(locale === 'en' && { feedState }),
|
||||
});
|
||||
const { feedProvenance } = layerContext;
|
||||
const context = {
|
||||
@@ -1010,12 +1032,16 @@ export class IntelHUD {
|
||||
note: locale === 'en' ? feedProvenance.note : undefined,
|
||||
},
|
||||
};
|
||||
// Not sent and not part of the signature: the English names a reply in
|
||||
// Not sent and not part of the signature: the line's own warning, taken
|
||||
// before the layer states are left out, and the English names a reply in
|
||||
// another language must not echo.
|
||||
Object.defineProperty(context, 'translatedLayerNames', {
|
||||
value: layerContext.enabledLayerLabels.filter(
|
||||
(name) => localizeLayerName(name) !== name,
|
||||
),
|
||||
Object.defineProperties(context, {
|
||||
warning: { value: provenanceTag(feedProvenance) },
|
||||
translatedLayerNames: {
|
||||
value: layerContext.enabledLayerLabels.filter(
|
||||
(name) => localizeLayerName(name) !== name,
|
||||
),
|
||||
},
|
||||
});
|
||||
return context;
|
||||
}
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
// revmain-astra-1 / revmain-sol-1: a ja/zh reply within the 20-character
|
||||
// limit is still cut to its first five space-separated tokens.
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { Readable } from 'node:stream';
|
||||
import { HUD_SUMMARY_MAX_UNSPACED_CHARS } from './hudSummaryResponse.js';
|
||||
|
||||
let client = 200;
|
||||
async function postHudSummary(context, reply) {
|
||||
client += 1;
|
||||
const previousKey = process.env.OPENAI_API_KEY;
|
||||
const previousFetch = globalThis.fetch;
|
||||
process.env.OPENAI_API_KEY = 'test-key';
|
||||
globalThis.fetch = async () => ({
|
||||
ok: true,
|
||||
status: 200,
|
||||
json: async () => ({ output_text: reply }),
|
||||
});
|
||||
try {
|
||||
const { handleHudSummary } = await import(
|
||||
'../server/providers/openai/hud-summary.js'
|
||||
);
|
||||
const req = Object.assign(
|
||||
Readable.from([Buffer.from(JSON.stringify(context))]),
|
||||
{
|
||||
method: 'POST',
|
||||
url: '/',
|
||||
headers: {},
|
||||
socket: { remoteAddress: `10.0.1.${client}` },
|
||||
},
|
||||
);
|
||||
return await new Promise((resolve, reject) => {
|
||||
const res = {
|
||||
statusCode: 200,
|
||||
setHeader() {},
|
||||
end(body = '') {
|
||||
resolve(JSON.parse(body));
|
||||
},
|
||||
};
|
||||
Promise.resolve(handleHudSummary(req, res)).catch(reject);
|
||||
});
|
||||
} finally {
|
||||
globalThis.fetch = previousFetch;
|
||||
if (previousKey === undefined) delete process.env.OPENAI_API_KEY;
|
||||
else process.env.OPENAI_API_KEY = previousKey;
|
||||
}
|
||||
}
|
||||
|
||||
for (const [locale, summary] of [
|
||||
['ja', '東京 上空 の 航空 情報 は 古い'],
|
||||
['zh-CN', 'SFO 附近 4 架 航班 数据 过期'],
|
||||
['zh-Hant', 'SFO 附近 4 架 航班 數據 過期'],
|
||||
]) {
|
||||
test(`a ${locale} phrase within ${HUD_SUMMARY_MAX_UNSPACED_CHARS} characters is kept whole`, async () => {
|
||||
assert.ok([...summary].length <= HUD_SUMMARY_MAX_UNSPACED_CHARS);
|
||||
const body = await postHudSummary(
|
||||
{ locale, feedProvenance: { overall: 'stale' } },
|
||||
JSON.stringify({ summary, feedState: 'stale' }),
|
||||
);
|
||||
assert.equal(body.summary, summary);
|
||||
});
|
||||
}
|
||||
+57
-10
@@ -27,7 +27,8 @@ registerHooks({
|
||||
|
||||
const { IntelHUD } = await import('./hud.js');
|
||||
const { setLocale } = await import('./i18n/index.js');
|
||||
const { hudSummaryInput } = await import('./hudSummaryResponse.js');
|
||||
const { hudSummaryHasEnglish, hudSummaryInput } =
|
||||
await import('./hudSummaryResponse.js');
|
||||
|
||||
const PARTIAL_FLIGHTS = {
|
||||
id: 'flights',
|
||||
@@ -36,8 +37,11 @@ const PARTIAL_FLIGHTS = {
|
||||
stats: { count: 4, partial: true, source: 'opensky' },
|
||||
};
|
||||
|
||||
/** A ru HUD over one partial Live Flights layer, answering with `reply`. */
|
||||
function createHud(t, reply) {
|
||||
/**
|
||||
* A HUD over one partial Live Flights layer and `places`, answering with
|
||||
* `reply`; ru unless `locale` says otherwise.
|
||||
*/
|
||||
function createHud(t, reply, { places = ['Остин'], locale = 'ru' } = {}) {
|
||||
const summary = { textContent: '' };
|
||||
const previous = { document: globalThis.document, window: globalThis.window };
|
||||
globalThis.document = {
|
||||
@@ -84,9 +88,13 @@ function createHud(t, reply) {
|
||||
else globalThis[key] = value;
|
||||
}
|
||||
});
|
||||
// The test viewer has no view target, so no place lookup runs.
|
||||
const buildContext = hud._summaryContext.bind(hud);
|
||||
hud._summaryContext = async () =>
|
||||
Object.assign(await buildContext(), { placeLabels: places });
|
||||
hud._dataManager = { getAll: () => [PARTIAL_FLIGHTS] };
|
||||
hud._latestMetrics = { altM: 600 };
|
||||
setLocale('ru');
|
||||
setLocale(locale);
|
||||
return { hud, sent, shown: () => summary.textContent };
|
||||
}
|
||||
|
||||
@@ -133,13 +141,52 @@ test('a camera move during typing shows the plain line, not a fragment', (t) =>
|
||||
assert.equal(shown(), hud._composeSummary());
|
||||
});
|
||||
|
||||
test('a view with no labels and no layers does not ask the model', async (t) => {
|
||||
const { hud, sent, shown } = createHud(t, {
|
||||
summary: 'пусто',
|
||||
feedState: null,
|
||||
});
|
||||
hud._dataManager = { getAll: () => [] };
|
||||
test('a view with layers but no place label does not ask the model', async (t) => {
|
||||
const { hud, sent, shown } = createHud(
|
||||
t,
|
||||
{ summary: 'Москва Рейсы онлайн частично', feedState: 'partial' },
|
||||
{ places: [] },
|
||||
);
|
||||
await hud._updateSummary(false, true);
|
||||
assert.equal(sent.length, 0);
|
||||
assert.equal(shown(), hud._composeSummary());
|
||||
});
|
||||
|
||||
test('a ja request leaves the layer state ids to `overall`', async (t) => {
|
||||
const { hud } = createHud(t, {}, { locale: 'ja' });
|
||||
const context = await hud._summaryContext();
|
||||
assert.equal(context.feedProvenance.overall, 'partial');
|
||||
assert.ok(!('feedState' in context.enabledLayers[0]));
|
||||
assert.ok(!('feedState' in context.feedProvenance.layers[0]));
|
||||
});
|
||||
|
||||
for (const [locale, summary] of [
|
||||
['ja', '東京上空のstale航空情報'],
|
||||
['ja', '東京上空のLive Flights'],
|
||||
['zh-CN', '北京航班stale数据'],
|
||||
['ru', 'Остин Live-Flights частично'],
|
||||
]) {
|
||||
test(`English flush against other text is an echo: ${summary}`, () => {
|
||||
assert.ok(hudSummaryHasEnglish(summary, locale, ['stale', 'Live Flights']));
|
||||
});
|
||||
}
|
||||
|
||||
test('English inside a longer Latin word is not an echo', () => {
|
||||
assert.ok(!hudSummaryHasEnglish('Datos stalesque', 'es', ['stale']));
|
||||
});
|
||||
|
||||
for (const [locale, summary, places] of [
|
||||
['es', 'Tráfico aéreo nominal sobre Madrid', ['Madrid']],
|
||||
['de', 'Fallback Daten über Berlin', ['Berlin']],
|
||||
['ja', 'Live Flights Plaza周辺', ['Live Flights Plaza']],
|
||||
]) {
|
||||
test(`a ${locale} word that is not an echo is shown: ${summary}`, async (t) => {
|
||||
const { hud, shown } = createHud(
|
||||
t,
|
||||
{ summary, feedState: 'partial' },
|
||||
{ places, locale },
|
||||
);
|
||||
await hud._updateSummary(false, true);
|
||||
assert.ok(shown().includes(summary), shown());
|
||||
});
|
||||
}
|
||||
+20
-15
@@ -108,26 +108,31 @@ const SUMMARY_LANGUAGES = Object.freeze({
|
||||
const UNSPACED_LOCALES = /^(?:ja|zh)(?:-|$)/;
|
||||
|
||||
/**
|
||||
* Whether a summary for a non-English locale still carries English the HUD
|
||||
* sent only as data: a feed-state id (`stale`) or an English layer name that
|
||||
* has a translation (`Live Flights` in Russian).
|
||||
* Whether a non-English summary echoes English the HUD does not show: a
|
||||
* feed-state id or a translated layer's English name. Only Latin letters
|
||||
* extend a word, so `stale` flush against kana or Han still counts.
|
||||
* @param {string} summary The model's words.
|
||||
* @param {unknown} locale The request locale.
|
||||
* @param {string[]} [englishNames] English layer names that the UI translates.
|
||||
* @param {string[]} [words] The English words the summary must not carry.
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function hudSummaryHasEnglish(summary, locale, englishNames = []) {
|
||||
export function hudSummaryHasEnglish(summary, locale, words = []) {
|
||||
if (hudSummaryLocale(locale) === DEFAULT_LOCALE) return false;
|
||||
const text = String(summary || '').toLowerCase();
|
||||
return [...HUD_SUMMARY_FEED_STATES, ...englishNames].some((word) => {
|
||||
const needle = String(word).toLowerCase();
|
||||
let at = text.indexOf(needle);
|
||||
while (at >= 0) {
|
||||
const before = text[at - 1];
|
||||
const after = text[at + needle.length];
|
||||
if (!/\p{L}/u.test(before ?? '') && !/\p{L}/u.test(after ?? ''))
|
||||
return true;
|
||||
at = text.indexOf(needle, at + 1);
|
||||
const flat = (value) =>
|
||||
String(value || '')
|
||||
.toLowerCase()
|
||||
.replace(/-/g, ' ');
|
||||
const text = flat(summary);
|
||||
const latin = (char) => /\p{Script=Latin}/u.test(char ?? '');
|
||||
return words.some((word) => {
|
||||
const needle = flat(word);
|
||||
if (!needle) return false;
|
||||
for (
|
||||
let at = text.indexOf(needle);
|
||||
at >= 0;
|
||||
at = text.indexOf(needle, at + 1)
|
||||
) {
|
||||
if (!latin(text[at - 1]) && !latin(text[at + needle.length])) return true;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
|
||||
@@ -381,6 +381,31 @@ test('a ja cap steps back to a space instead of cutting a Latin word', async ()
|
||||
assert.equal(response.body.summary, 'San Francisco Bay');
|
||||
});
|
||||
|
||||
test('a ja cap keeps the text before a Latin word it would split', async () => {
|
||||
const { response } = await postHudSummary(
|
||||
{ locale: 'ja', feedProvenance: { overall: 'stale' } },
|
||||
JSON.stringify({
|
||||
summary: '東京 羽田空港周辺のFlightAware航空情報',
|
||||
feedState: 'stale',
|
||||
}),
|
||||
);
|
||||
assert.equal(response.body.summary, '東京 羽田空港周辺の');
|
||||
});
|
||||
|
||||
for (const [locale, summary] of [
|
||||
['ja', '東京 上空 の 航空 情報 は 古い'],
|
||||
['zh-CN', 'SFO 附近 4 架 航班 数据 过期'],
|
||||
['zh-Hant', 'SFO 附近 4 架 航班 數據 過期'],
|
||||
]) {
|
||||
test(`a ${locale} phrase within the character cap is kept whole`, async () => {
|
||||
const { response } = await postHudSummary(
|
||||
{ locale, feedProvenance: { overall: 'stale' } },
|
||||
JSON.stringify({ summary, feedState: 'stale' }),
|
||||
);
|
||||
assert.equal(response.body.summary, summary);
|
||||
});
|
||||
}
|
||||
|
||||
test('a reply that skips the format carries no summary and no state', async () => {
|
||||
const { response } = await postHudSummary(
|
||||
{ locale: 'ru', feedProvenance: { overall: 'stale' } },
|
||||
|
||||
Reference in new issue
Block a user