fix(firms): stop requesting the retired Suomi NPP source

NASA stopped delivering Suomi NPP data on 2026-11-02 (FIRMS news page).
With a live MAP_KEY on 2026-11-04, VIIRS_SNPP_NRT came back ok with zero
rows while NOAA-20, NOAA-21 and MODIS returned 57k, 49k and 11k, so the
source only spent one transaction per refresh. The proxy now requests
NOAA-20, NOAA-21 and MODIS NRT. The "N" satellite label stays, so cached
or archived Suomi NPP rows still read SNPP.

Tests count sources from the list instead of assuming four, and a
regression test keeps the retired source out of the production list.

Refs #849

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Itallo BrandãoandClaude Opus 5.5 committed 2026-11-04 15:15:34 -03:00
1 parent ac28a11e41
commit 84c1fef8d3
5 files changed
+31 -17

No files matched your search

+4 -3
View File
@@ -273,11 +273,12 @@ Bundled OSM-derived data is distributed under [ODbL 1.0](https://opendatacommons
> We acknowledge the use of data and/or imagery from NASA's Fire Information for Resource Management System (FIRMS) (https://earthdata.nasa.gov/firms), part of NASA's Earth Observing System Data and Information System (EOSDIS). > We acknowledge the use of data and/or imagery from NASA's Fire Information for Resource Management System (FIRMS) (https://earthdata.nasa.gov/firms), part of NASA's Earth Observing System Data and Information System (EOSDIS).
FIRMS active fires are **fetched live at runtime** (CC0 / U.S. public domain data): the FIRMS active fires are **fetched live at runtime** (CC0 / U.S. public domain data): the
`/api/firms` server-side proxy merges the three VIIRS NRT sources (NOAA-20, NOAA-21, `/api/firms` server-side proxy merges the two VIIRS NRT sources (NOAA-20, NOAA-21)
Suomi-NPP) and MODIS NRT (Terra + Aqua), clamped to the trailing 24 h, cached 30 min to respect the shared MAP_KEY and MODIS NRT (Terra + Aqua), clamped to the trailing 24 h, cached 30 min to respect the shared MAP_KEY
transaction quota. Requires a free `FIRMS_MAP_KEY` transaction quota. Requires a free `FIRMS_MAP_KEY`
(https://firms.modaps.eosdis.nasa.gov/api/map_key/); the layer is empty without it. (https://firms.modaps.eosdis.nasa.gov/api/map_key/); the layer is empty without it.
The former bundled 2026-05-25 snapshot was removed 2026-07-16. The former bundled 2026-05-25 snapshot was removed 2026-07-16. Suomi NPP (`VIIRS_SNPP_NRT`)
is no longer requested: NASA stopped delivering its data on 2026-11-02.
### Natural Earth physical regions (`natural_earth/`) ### Natural Earth physical regions (`natural_earth/`)
+5
View File
@@ -2325,6 +2325,11 @@ accessible name remains "Don't show this again". Control behavior is unchanged.
## FIRMS source status ## FIRMS source status
The FIRMS proxy requests VIIRS NOAA-20, VIIRS NOAA-21 and MODIS NRT (Terra and
Aqua). It no longer requests VIIRS Suomi NPP: NASA stopped delivering Suomi NPP
data on 2026-11-02, and the source returned no rows while still costing one
MAP_KEY transaction per refresh.
The FIRMS CSV parser rejects impossible acquisition calendar dates before the proxy filters records to the trailing 24 hours, while retaining valid UTC acquisition times. The FIRMS CSV parser rejects impossible acquisition calendar dates before the proxy filters records to the trailing 24 hours, while retaining valid UTC acquisition times.
The `get_active_fires` tool reports acquisition times as UTC ISO timestamps, The `get_active_fires` tool reports acquisition times as UTC ISO timestamps,
including zero-padded hours and minutes; missing or invalid dates and times including zero-padded hours and minutes; missing or invalid dates and times
+6 -8
View File
@@ -131,8 +131,8 @@ export async function fetchFirms(
* NASA FIRMS live active-fire proxy with a memory + disk cache. * NASA FIRMS live active-fire proxy with a memory + disk cache.
* Upstream: https://firms.modaps.eosdis.nasa.gov/api/area/csv/{KEY}/{SOURCE}/world/2 * Upstream: https://firms.modaps.eosdis.nasa.gov/api/area/csv/{KEY}/{SOURCE}/world/2
* *
* Merges three VIIRS NRT sources (NOAA-20, NOAA-21, Suomi-NPP — independent * Merges two VIIRS NRT sources (NOAA-20, NOAA-21 — independent satellites, no
* satellites, no cross-source dedup) plus MODIS_NRT (combined Terra+Aqua, * cross-source dedup) plus MODIS_NRT (combined Terra+Aqua,
* ~1 km, numeric 0-100 confidence), fetched sequentially (quota courtesy) * ~1 km, numeric 0-100 confidence), fetched sequentially (quota courtesy)
* with `days=2` (`days=1` means "current UTC day", nearly empty just after * with `days=2` (`days=1` means "current UTC day", nearly empty just after
* 00:00Z) and clamps to the trailing 24 h via src/data/firmsCsv.js. FIRMS * 00:00Z) and clamps to the trailing 24 h via src/data/firmsCsv.js. FIRMS
@@ -141,6 +141,9 @@ export async function fetchFirms(
* fresh-enough disk cache (.gev-cache/firms.json) prevents ANY upstream * fresh-enough disk cache (.gev-cache/firms.json) prevents ANY upstream
* fetch across dev-server restarts. Pattern mirrors celestrakProxy. * fetch across dev-server restarts. Pattern mirrors celestrakProxy.
* *
* VIIRS_SNPP_NRT is not requested: NASA stopped delivering Suomi NPP data on
* 2026-11-02, so that source only spent a transaction per refresh (#849).
*
* Routes: * Routes:
* GET /api/firms → {fetchedAt, stale, ttlMs, sources, count, fires} * GET /api/firms → {fetchedAt, stale, ttlMs, sources, count, fires}
* GET /api/firms/status → {hasKey, lastFetch, count, stale, ttlMs, transactions} * GET /api/firms/status → {hasKey, lastFetch, count, stale, ttlMs, transactions}
@@ -153,12 +156,7 @@ export async function fetchFirms(
export function firmsProxy() { export function firmsProxy() {
const TTL_MS = 30 * 60_000; const TTL_MS = 30 * 60_000;
const STATUS_TTL_MS = 5 * 60_000; const STATUS_TTL_MS = 5 * 60_000;
const SOURCES = [ const SOURCES = ['VIIRS_NOAA20_NRT', 'VIIRS_NOAA21_NRT', 'MODIS_NRT'];
'VIIRS_NOAA20_NRT',
'VIIRS_NOAA21_NRT',
'VIIRS_SNPP_NRT',
'MODIS_NRT',
];
/** /**
* Per-source body cap. A VIIRS world/2 pull is ~130k rows x ~100 B, about * Per-source body cap. A VIIRS world/2 pull is ~130k rows x ~100 B, about
* 13 MB per satellite in heavy fire season (MODIS runs smaller), so this is a * 13 MB per satellite in heavy fire season (MODIS runs smaller), so this is a
+14 -4
View File
@@ -21,7 +21,7 @@ assert.notEqual(start, -1, 'FIRMS refresh function exists');
const end = config.indexOf('\n }', start); const end = config.indexOf('\n }', start);
assert.notEqual(end, -1, 'FIRMS refresh function closes'); assert.notEqual(end, -1, 'FIRMS refresh function closes');
const refreshSource = config.slice(start, end + 4); const refreshSource = config.slice(start, end + 4);
const SOURCES = ['VIIRS_NOAA20_NRT', 'VIIRS_NOAA21_NRT', 'VIIRS_SNPP_NRT', 'MODIS_NRT']; const SOURCES = ['VIIRS_NOAA20_NRT', 'VIIRS_NOAA21_NRT', 'MODIS_NRT'];
const NOW = Date.UTC(2026, 8, 11, 12); const NOW = Date.UTC(2026, 8, 11, 12);
const recent = { acqDate: '2026-09-11', acqTime: '1100' }; const recent = { acqDate: '2026-09-11', acqTime: '1100' };
@@ -63,7 +63,7 @@ test('FIRMS keeps successful sources when another upstream fails', async () => {
if (source === SOURCES[1]) throw new Error('upstream unavailable'); if (source === SOURCES[1]) throw new Error('upstream unavailable');
return [recent]; return [recent];
})('fixture'); })('fixture');
assert.equal(result.fires.length, 3); assert.equal(result.fires.length, SOURCES.length - 1);
assert.deepEqual(result.sources, SOURCES.map((source, index) => ({ assert.deepEqual(result.sources, SOURCES.map((source, index) => ({
source, count: index === 1 ? 0 : 1, ok: index !== 1, source, count: index === 1 ? 0 : 1, ok: index !== 1,
}))); })));
@@ -77,7 +77,7 @@ test('FIRMS reports one failure if consuming a source throws before append', asy
// Fault injection for aggregation; ordinary parsed CSV returns an array. // Fault injection for aggregation; ordinary parsed CSV returns an array.
return { length: 1, [Symbol.iterator]() { throw new Error('aggregation failed'); } }; return { length: 1, [Symbol.iterator]() { throw new Error('aggregation failed'); } };
})('fixture'); })('fixture');
assert.equal(result.fires.length, 3); assert.equal(result.fires.length, SOURCES.length - 1);
assert.deepEqual(result.sources, SOURCES.map((source, index) => ({ assert.deepEqual(result.sources, SOURCES.map((source, index) => ({
source, count: index === 0 ? 0 : 1, ok: index !== 0, source, count: index === 0 ? 0 : 1, ok: index !== 0,
}))); })));
@@ -107,7 +107,7 @@ function wrappedAddressSelectionFailure() {
{ name: 'ConnectTimeoutError', code: 'UND_ERR_CONNECT_TIMEOUT', cause: inner }); { name: 'ConnectTimeoutError', code: 'UND_ERR_CONNECT_TIMEOUT', cause: inner });
return new TypeError('fetch failed', { cause }); return new TypeError('fetch failed', { cause });
} }
const URL_WITH_KEY = 'https://firms.modaps.eosdis.nasa.gov/api/area/csv/SECRET/VIIRS_SNPP_NRT/world/2'; const URL_WITH_KEY = 'https://firms.modaps.eosdis.nasa.gov/api/area/csv/SECRET/VIIRS_NOAA20_NRT/world/2';
test('FIRMS failures log the socket cause, never the keyed URL', () => { test('FIRMS failures log the socket cause, never the keyed URL', () => {
const described = describeFetchError(addressSelectionFailure()); const described = describeFetchError(addressSelectionFailure());
@@ -307,3 +307,13 @@ test('every FIRMS upstream request goes through the IPv4 fallback', async () =>
assert.ok(calls[0][2] instanceof AbortSignal); assert.ok(calls[0][2] instanceof AbortSignal);
assert.deepEqual(calls[1], ['read', response, calls[0][2]], 'the body read shares the request deadline'); assert.deepEqual(calls[1], ['read', response, calls[0][2]], 'the body read shares the request deadline');
}); });
test('FIRMS requests only the satellites NASA still delivers', () => {
const listed = config.match(/const SOURCES = \[([^\]]*)\];/);
assert.ok(listed, 'FIRMS source list exists');
const production = [...listed[1].matchAll(/'([A-Z0-9_]+)'/g)].map((m) => m[1]);
assert.deepEqual(production, SOURCES, 'the test mirrors the production source list');
// NASA stopped delivering Suomi NPP data on 2026-11-02 (#849): requesting it
// returns no rows and still spends a MAP_KEY transaction on every refresh.
assert.ok(!production.includes('VIIRS_SNPP_NRT'));
});
+2 -2
View File
@@ -313,9 +313,9 @@ test('FIRMS retains a large successful source during partial failure and filters
const first = json(await request()); const first = json(await request());
assert.equal(first.count, 130001); assert.equal(first.count, 130001);
assert.equal(first.sources.filter((s) => s.ok).length, 1); assert.equal(first.sources.filter((s) => s.ok).length, 1);
assert.equal(calls, 4); assert.equal(calls, first.sources.length);
assert.equal(json(await request()).count, 130001); assert.equal(json(await request()).count, 130001);
assert.equal(calls, 4); assert.equal(calls, first.sources.length);
assert.deepEqual(json(await request('/status')).transactions, { assert.deepEqual(json(await request('/status')).transactions, {
used: 3, used: 3,
limit: 5000, limit: 5000,