fix(hud): ask the model only with a place label, catch flush English echoes

- Skip the model when the view has layers but no place, street or nearby
  label: with layers alone it invented a place.
- The English-echo check treats only Latin letters as part of a word, so
  ids flush against kana or Han and hyphenated names are caught; it skips
  nominal, state words a locale keeps in English and names inside place
  labels.
- Outside English, per-layer state ids are left out of the request.
- The ja/zh cap steps back to the start of a Latin word, not the last space.
- Fold the CJK cap cases into hudSummaryResponse.test.mjs.
This commit is contained in:
German Khokhlov committed 2026-10-30 20:53:31 +04:00
1 parent 7713fb8ae8
commit dbfbfcceb9
7 files changed
+150 -107

No files matched your search

+3 -3
View File
@@ -5255,10 +5255,10 @@ and unreachable upstream (502/504) separately from road geometry.
### AI HUD Summary (June 2026) ### AI HUD Summary (June 2026)
- HUD `SUMMARY` readout requests a five-word intelligence-style summary from `/api/openai/hud-summary` (model `OPENAI_HUD_SUMMARY_MODEL`, default `gpt-5-nano`, minimal reasoning). - HUD `SUMMARY` readout requests a five-word intelligence-style summary from `/api/openai/hud-summary` (model `OPENAI_HUD_SUMMARY_MODEL`, default `gpt-5-nano`, minimal reasoning).
- Input is the live basemap label context (place/street/nearby-place labels + enabled layers) and `feedProvenance`, with every layer name as the user sees it; outside English the narration note is left out, since a small model copies its English words. The model is instructed not to infer from coordinates. A view with no labels and no enabled layer is not sent: the deterministic line is shown. - Input is the live basemap label context (place/street/nearby-place labels + enabled layers) and `feedProvenance`, with every layer name as the user sees it; outside English the narration note and the per-layer state ids are left out (`overall` still carries the state), since a small model copies English words. The model is instructed not to infer from coordinates. A view with no place, street or nearby-place label is not sent, since with layers alone the model invents a place: the deterministic line is shown.
- Language (October 2026): the HUD sends `locale`, which is part of the dedupe signature. The proxy keeps a shipped locale (English otherwise) and names the language in the instructions and at the start of the input; Japanese and Chinese get a 20-character phrase instead of five words. - Language (October 2026): the HUD sends `locale`, which is part of the dedupe signature. The proxy keeps a shipped locale (English otherwise) and names the language in the instructions and at the start of the input; Japanese and Chinese get a 20-character phrase instead of five words.
- The reply is structured (`text.format` JSON schema): `{summary, feedState}`, `feedState` being the snapshot state or `none`. For a view that is not nominal, a missing or different `feedState` falls back to the deterministic line. An accepted summary and the deterministic line carry the HUD's own translated state: the state word leads and the layer names trail (`STALE | … | LIVE FLIGHTS`), so the one-line readout never clips the warning; the text and then the layer names may be cut off on narrow layouts. Outside English a reply that echoes a feed-state id or an English layer name the UI translates falls back too. `get_hud_caption` builds its caption the same way and returns `feedState`. - The reply is structured (`text.format` JSON schema): `{summary, feedState}`, `feedState` being the snapshot state or `none`. For a view that is not nominal, a missing or different `feedState` falls back to the deterministic line. An accepted summary and the deterministic line carry the HUD's own translated state: the state word leads and the layer names trail (`STALE | … | LIVE FLIGHTS`), so the one-line readout never clips the warning; the text and then the layer names may be cut off on narrow layouts. Outside English a reply that echoes a feed-state id or an English layer name the UI translates falls back too, even written flush against kana or Han; a state word the locale itself keeps in English (de `FALLBACK`) and a name inside a place label do not count. `get_hud_caption` builds its caption the same way and returns `feedState`.
- Output is sanitized to five words, or for a Japanese or Chinese request locale to 20 characters (stepping back to a space rather than splitting a Latin word); falls back to the deterministic telemetry summary on error/timeout (5s abort); typewriter animation on update; a camera move during typing shows the deterministic line instead of the fragment. - Output is sanitized to five words, or for a Japanese or Chinese request locale to 20 characters (stepping back to the start of a Latin word rather than splitting it); falls back to the deterministic telemetry summary on error/timeout (5s abort); typewriter animation on update; a camera move during typing shows the deterministic line instead of the fragment.
- `partial` is a feed state of its own, ranked between `fallback` and `nominal`, so one partial layer no longer reads as off and partial plus nominal reads partial (HUD, voice view state, `analyst_query`, `brief`, the agent world packet). Its narration note says coverage is incomplete rather than not live. - `partial` is a feed state of its own, ranked between `fallback` and `nominal`, so one partial layer no longer reads as off and partial plus nominal reads partial (HUD, voice view state, `analyst_query`, `brief`, the agent world packet). Its narration note says coverage is incomplete rather than not live.
- A summary asked for before the camera starts moving never paints over the next view: `moveStart` marks the summary dirty and bumps its revision, as `moveEnd` does, aborts the request in flight and stops a reply still typing; neither a reply nor a fallback line (on an error, the 5 s timeout, the abort, or a failed context build) paints once the revision has moved. A dropped request forgets its signature and a timeout counts as a failure, so the next 15 s tick asks again even over the same place. An interrupted typewriter reply also forgets its signature; fully displayed answers remain reusable. - A summary asked for before the camera starts moving never paints over the next view: `moveStart` marks the summary dirty and bumps its revision, as `moveEnd` does, aborts the request in flight and stops a reply still typing; neither a reply nor a fallback line (on an error, the 5 s timeout, the abort, or a failed context build) paints once the revision has moved. A dropped request forgets its signature and a timeout counts as a failure, so the next 15 s tick asks again even over the same place. An interrupted typewriter reply also forgets its signature; fully displayed answers remain reusable.
+5 -3
View File
@@ -49,7 +49,7 @@ function toFiveWordHudSummary(value, unspaced) {
// Other languages keep five words, even around a Han place name. // Other languages keep five words, even around a Han place name.
if (!unspaced) return words.slice(0, 5).join(' '); if (!unspaced) return words.slice(0, 5).join(' ');
// Japanese and Chinese have no spaces to count words by: cap characters, // Japanese and Chinese have no spaces to count words by: cap characters,
// stepping back to a space rather than cutting a Latin word in two. // stepping back to the start of a Latin word rather than cutting it in two.
const text = words.join(' '); const text = words.join(' ');
const graphemes = [ const graphemes = [
...new Intl.Segmenter(undefined, { granularity: 'grapheme' }).segment(text), ...new Intl.Segmenter(undefined, { granularity: 'grapheme' }).segment(text),
@@ -59,8 +59,10 @@ function toFiveWordHudSummary(value, unspaced) {
const next = graphemes[HUD_SUMMARY_MAX_UNSPACED_CHARS]; const next = graphemes[HUD_SUMMARY_MAX_UNSPACED_CHARS];
const inWord = (g) => /^[\p{Script=Latin}\p{N}]/u.test(g ?? ''); const inWord = (g) => /^[\p{Script=Latin}\p{N}]/u.test(g ?? '');
if (inWord(head.at(-1)) && inWord(next)) { if (inWord(head.at(-1)) && inWord(next)) {
const space = head.lastIndexOf(' '); let start = head.length - 1;
if (space > 0) return head.slice(0, space).join('').trim(); while (start > 0 && inWord(head[start - 1])) start -= 1;
// A Latin word longer than the cap is cut where the cap falls.
if (start > 0) return head.slice(0, start).join('').trim();
} }
return head.join('').trim(); return head.join('').trim();
} }
+40 -14
View File
@@ -122,6 +122,29 @@ function withProvenance(text, provenance) {
return [state, text, layers].filter(Boolean).join(' | '); return [state, text, layers].filter(Boolean).join(' | ');
} }
/**
* The English a summary outside English must not carry: feed-state ids and
* translated layers' English names. A state word the locale keeps in English
* (de FALLBACK) and a layer name inside a place label (Radio City) are fine.
*/
function summaryEnglishWords(context) {
const places = [
...(context.placeLabels || []),
...(context.streetLabels || []),
...(context.nearbyPlaceLabels || []),
]
.join(' ')
.toLowerCase();
return [
...Object.keys(FEED_STATE_LABELS).filter(
(state) => t(FEED_STATE_LABELS[state]).toLowerCase() !== state,
),
...(context.translatedLayerNames || []).filter(
(name) => !places.includes(name.toLowerCase()),
),
];
}
/** The name shown for a _viewBand() id. */ /** The name shown for a _viewBand() id. */
function viewBandLabel(band) { function viewBandLabel(band) {
switch (band) { switch (band) {
@@ -862,13 +885,12 @@ export class IntelHUD {
return; return;
} }
if (revision !== this._summaryRevision) return; if (revision !== this._summaryRevision) return;
// Nothing to describe (no place labels, no enabled layer): the model only // No place to name: with layers alone the model invents one, so the plain
// answers about the empty input, so the plain line is shown instead. // line (state and layers) is shown instead.
const described = [ const described = [
context.placeLabels, context.placeLabels,
context.streetLabels, context.streetLabels,
context.nearbyPlaceLabels, context.nearbyPlaceLabels,
context.enabledLayerLabels,
].some((list) => list?.length); ].some((list) => list?.length);
if (!described) { if (!described) {
this._summaryDirty = false; this._summaryDirty = false;
@@ -916,14 +938,11 @@ export class IntelHUD {
!hudSummaryHasEnglish( !hudSummaryHasEnglish(
data.summary, data.summary,
context.locale, context.locale,
context.translatedLayerNames, summaryEnglishWords(context),
) )
) { ) {
// The warning is the HUD's own translated state, not the model's words. // The warning is the HUD's own translated state, not the model's words.
this._lastSummaryText = withProvenance( this._lastSummaryText = withProvenance(data.summary, context.warning);
data.summary,
provenanceTag(context.feedProvenance),
);
} }
this._setSummaryText(this._lastSummaryText || fallbackText, animate); this._setSummaryText(this._lastSummaryText || fallbackText, animate);
} catch (error) { } catch (error) {
@@ -988,9 +1007,12 @@ export class IntelHUD {
const layerContext = hudSummaryLayerContext( const layerContext = hudSummaryLayerContext(
this._dataManager?.getAll?.() || [], this._dataManager?.getAll?.() || [],
); );
const named = (layer) => ({ // Outside English a layer's state id is left out: the model echoes it,
// and `overall` already carries the state.
const named = ({ feedState, ...layer }) => ({
...layer, ...layer,
name: localizeLayerName(layer.name), name: localizeLayerName(layer.name),
...(locale === 'en' && { feedState }),
}); });
const { feedProvenance } = layerContext; const { feedProvenance } = layerContext;
const context = { const context = {
@@ -1010,12 +1032,16 @@ export class IntelHUD {
note: locale === 'en' ? feedProvenance.note : undefined, note: locale === 'en' ? feedProvenance.note : undefined,
}, },
}; };
// Not sent and not part of the signature: the English names a reply in // Not sent and not part of the signature: the line's own warning, taken
// before the layer states are left out, and the English names a reply in
// another language must not echo. // another language must not echo.
Object.defineProperty(context, 'translatedLayerNames', { Object.defineProperties(context, {
value: layerContext.enabledLayerLabels.filter( warning: { value: provenanceTag(feedProvenance) },
(name) => localizeLayerName(name) !== name, translatedLayerNames: {
), value: layerContext.enabledLayerLabels.filter(
(name) => localizeLayerName(name) !== name,
),
},
}); });
return context; return context;
} }
-62
View File
@@ -1,62 +0,0 @@
// revmain-astra-1 / revmain-sol-1: a ja/zh reply within the 20-character
// limit is still cut to its first five space-separated tokens.
import assert from 'node:assert/strict';
import test from 'node:test';
import { Readable } from 'node:stream';
import { HUD_SUMMARY_MAX_UNSPACED_CHARS } from './hudSummaryResponse.js';
let client = 200;
async function postHudSummary(context, reply) {
client += 1;
const previousKey = process.env.OPENAI_API_KEY;
const previousFetch = globalThis.fetch;
process.env.OPENAI_API_KEY = 'test-key';
globalThis.fetch = async () => ({
ok: true,
status: 200,
json: async () => ({ output_text: reply }),
});
try {
const { handleHudSummary } = await import(
'../server/providers/openai/hud-summary.js'
);
const req = Object.assign(
Readable.from([Buffer.from(JSON.stringify(context))]),
{
method: 'POST',
url: '/',
headers: {},
socket: { remoteAddress: `10.0.1.${client}` },
},
);
return await new Promise((resolve, reject) => {
const res = {
statusCode: 200,
setHeader() {},
end(body = '') {
resolve(JSON.parse(body));
},
};
Promise.resolve(handleHudSummary(req, res)).catch(reject);
});
} finally {
globalThis.fetch = previousFetch;
if (previousKey === undefined) delete process.env.OPENAI_API_KEY;
else process.env.OPENAI_API_KEY = previousKey;
}
}
for (const [locale, summary] of [
['ja', '東京 上空 の 航空 情報 は 古い'],
['zh-CN', 'SFO 附近 4 架 航班 数据 过期'],
['zh-Hant', 'SFO 附近 4 架 航班 數據 過期'],
]) {
test(`a ${locale} phrase within ${HUD_SUMMARY_MAX_UNSPACED_CHARS} characters is kept whole`, async () => {
assert.ok([...summary].length <= HUD_SUMMARY_MAX_UNSPACED_CHARS);
const body = await postHudSummary(
{ locale, feedProvenance: { overall: 'stale' } },
JSON.stringify({ summary, feedState: 'stale' }),
);
assert.equal(body.summary, summary);
});
}
+57 -10
View File
@@ -27,7 +27,8 @@ registerHooks({
const { IntelHUD } = await import('./hud.js'); const { IntelHUD } = await import('./hud.js');
const { setLocale } = await import('./i18n/index.js'); const { setLocale } = await import('./i18n/index.js');
const { hudSummaryInput } = await import('./hudSummaryResponse.js'); const { hudSummaryHasEnglish, hudSummaryInput } =
await import('./hudSummaryResponse.js');
const PARTIAL_FLIGHTS = { const PARTIAL_FLIGHTS = {
id: 'flights', id: 'flights',
@@ -36,8 +37,11 @@ const PARTIAL_FLIGHTS = {
stats: { count: 4, partial: true, source: 'opensky' }, stats: { count: 4, partial: true, source: 'opensky' },
}; };
/** A ru HUD over one partial Live Flights layer, answering with `reply`. */ /**
function createHud(t, reply) { * A HUD over one partial Live Flights layer and `places`, answering with
* `reply`; ru unless `locale` says otherwise.
*/
function createHud(t, reply, { places = ['Остин'], locale = 'ru' } = {}) {
const summary = { textContent: '' }; const summary = { textContent: '' };
const previous = { document: globalThis.document, window: globalThis.window }; const previous = { document: globalThis.document, window: globalThis.window };
globalThis.document = { globalThis.document = {
@@ -84,9 +88,13 @@ function createHud(t, reply) {
else globalThis[key] = value; else globalThis[key] = value;
} }
}); });
// The test viewer has no view target, so no place lookup runs.
const buildContext = hud._summaryContext.bind(hud);
hud._summaryContext = async () =>
Object.assign(await buildContext(), { placeLabels: places });
hud._dataManager = { getAll: () => [PARTIAL_FLIGHTS] }; hud._dataManager = { getAll: () => [PARTIAL_FLIGHTS] };
hud._latestMetrics = { altM: 600 }; hud._latestMetrics = { altM: 600 };
setLocale('ru'); setLocale(locale);
return { hud, sent, shown: () => summary.textContent }; return { hud, sent, shown: () => summary.textContent };
} }
@@ -133,13 +141,52 @@ test('a camera move during typing shows the plain line, not a fragment', (t) =>
assert.equal(shown(), hud._composeSummary()); assert.equal(shown(), hud._composeSummary());
}); });
test('a view with no labels and no layers does not ask the model', async (t) => { test('a view with layers but no place label does not ask the model', async (t) => {
const { hud, sent, shown } = createHud(t, { const { hud, sent, shown } = createHud(
summary: 'пусто', t,
feedState: null, { summary: 'Москва Рейсы онлайн частично', feedState: 'partial' },
}); { places: [] },
hud._dataManager = { getAll: () => [] }; );
await hud._updateSummary(false, true); await hud._updateSummary(false, true);
assert.equal(sent.length, 0); assert.equal(sent.length, 0);
assert.equal(shown(), hud._composeSummary()); assert.equal(shown(), hud._composeSummary());
}); });
test('a ja request leaves the layer state ids to `overall`', async (t) => {
const { hud } = createHud(t, {}, { locale: 'ja' });
const context = await hud._summaryContext();
assert.equal(context.feedProvenance.overall, 'partial');
assert.ok(!('feedState' in context.enabledLayers[0]));
assert.ok(!('feedState' in context.feedProvenance.layers[0]));
});
for (const [locale, summary] of [
['ja', '東京上空のstale航空情報'],
['ja', '東京上空のLive Flights'],
['zh-CN', '北京航班stale数据'],
['ru', 'Остин Live-Flights частично'],
]) {
test(`English flush against other text is an echo: ${summary}`, () => {
assert.ok(hudSummaryHasEnglish(summary, locale, ['stale', 'Live Flights']));
});
}
test('English inside a longer Latin word is not an echo', () => {
assert.ok(!hudSummaryHasEnglish('Datos stalesque', 'es', ['stale']));
});
for (const [locale, summary, places] of [
['es', 'Tráfico aéreo nominal sobre Madrid', ['Madrid']],
['de', 'Fallback Daten über Berlin', ['Berlin']],
['ja', 'Live Flights Plaza周辺', ['Live Flights Plaza']],
]) {
test(`a ${locale} word that is not an echo is shown: ${summary}`, async (t) => {
const { hud, shown } = createHud(
t,
{ summary, feedState: 'partial' },
{ places, locale },
);
await hud._updateSummary(false, true);
assert.ok(shown().includes(summary), shown());
});
}
+20 -15
View File
@@ -108,26 +108,31 @@ const SUMMARY_LANGUAGES = Object.freeze({
const UNSPACED_LOCALES = /^(?:ja|zh)(?:-|$)/; const UNSPACED_LOCALES = /^(?:ja|zh)(?:-|$)/;
/** /**
* Whether a summary for a non-English locale still carries English the HUD * Whether a non-English summary echoes English the HUD does not show: a
* sent only as data: a feed-state id (`stale`) or an English layer name that * feed-state id or a translated layer's English name. Only Latin letters
* has a translation (`Live Flights` in Russian). * extend a word, so `stale` flush against kana or Han still counts.
* @param {string} summary The model's words. * @param {string} summary The model's words.
* @param {unknown} locale The request locale. * @param {unknown} locale The request locale.
* @param {string[]} [englishNames] English layer names that the UI translates. * @param {string[]} [words] The English words the summary must not carry.
* @returns {boolean} * @returns {boolean}
*/ */
export function hudSummaryHasEnglish(summary, locale, englishNames = []) { export function hudSummaryHasEnglish(summary, locale, words = []) {
if (hudSummaryLocale(locale) === DEFAULT_LOCALE) return false; if (hudSummaryLocale(locale) === DEFAULT_LOCALE) return false;
const text = String(summary || '').toLowerCase(); const flat = (value) =>
return [...HUD_SUMMARY_FEED_STATES, ...englishNames].some((word) => { String(value || '')
const needle = String(word).toLowerCase(); .toLowerCase()
let at = text.indexOf(needle); .replace(/-/g, ' ');
while (at >= 0) { const text = flat(summary);
const before = text[at - 1]; const latin = (char) => /\p{Script=Latin}/u.test(char ?? '');
const after = text[at + needle.length]; return words.some((word) => {
if (!/\p{L}/u.test(before ?? '') && !/\p{L}/u.test(after ?? '')) const needle = flat(word);
return true; if (!needle) return false;
at = text.indexOf(needle, at + 1); for (
let at = text.indexOf(needle);
at >= 0;
at = text.indexOf(needle, at + 1)
) {
if (!latin(text[at - 1]) && !latin(text[at + needle.length])) return true;
} }
return false; return false;
}); });
+25
View File
@@ -381,6 +381,31 @@ test('a ja cap steps back to a space instead of cutting a Latin word', async ()
assert.equal(response.body.summary, 'San Francisco Bay'); assert.equal(response.body.summary, 'San Francisco Bay');
}); });
test('a ja cap keeps the text before a Latin word it would split', async () => {
const { response } = await postHudSummary(
{ locale: 'ja', feedProvenance: { overall: 'stale' } },
JSON.stringify({
summary: '東京 羽田空港周辺のFlightAware航空情報',
feedState: 'stale',
}),
);
assert.equal(response.body.summary, '東京 羽田空港周辺の');
});
for (const [locale, summary] of [
['ja', '東京 上空 の 航空 情報 は 古い'],
['zh-CN', 'SFO 附近 4 架 航班 数据 过期'],
['zh-Hant', 'SFO 附近 4 架 航班 數據 過期'],
]) {
test(`a ${locale} phrase within the character cap is kept whole`, async () => {
const { response } = await postHudSummary(
{ locale, feedProvenance: { overall: 'stale' } },
JSON.stringify({ summary, feedState: 'stale' }),
);
assert.equal(response.body.summary, summary);
});
}
test('a reply that skips the format carries no summary and no state', async () => { test('a reply that skips the format carries no summary and no state', async () => {
const { response } = await postHudSummary( const { response } = await postHudSummary(
{ locale: 'ru', feedProvenance: { overall: 'stale' } }, { locale: 'ru', feedProvenance: { overall: 'stale' } },