- Default backoff is 30 min, longer than the 15-minute catalog refresh,
so a failing Kartverket costs one attempt per backoff, not per refresh.
- A point a 200 answer leaves out is cached as no height (6 h TTL)
instead of being asked again, and waited on, every refresh.
- Only 408, 429 and 5xx stop the lookup; any other 4xx marks that
request's points as no height and the lookup carries on.
- Each request has its own 10 s timeout inside the shared 30 s deadline;
a lookup cut short only by the deadline resumes without a backoff.
- The lane waits only on a cold cache (>10% of cameras missing); a few
new cameras or expired misses are looked up in the background.
- applyVegvesenGroundHeights skips cameras without finite coordinates,
so it really never throws.
- clearKartverketHeightCache bumps a generation, so a lookup started
before a clear cannot write to the cache, backoff or pending slot.
- Credit and DATA_SOURCES note the modification (water set to 0 m,
rounded to 0.1 m) as CC BY 4.0 asks; docs match the stop rules.
- Tests for the deadline being shared, per-request timeouts, 503,
oversized and non-JSON bodies, 4xx, left-out points, Retry-After end
to end, overlapping refreshes, the warm-cache wait, bad coordinates,
the cache-clear generation and the loader's wait budget.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- State FL511's terms the same way everywhere: individual, personal use
only, and any other use (a public deployment included) needs FDOT's
express written consent. .env.example, README, dataCredits, the source
licence string, the catalog comment and the changelog fragment said only
"individual, non-commercial use".
- Page the ArcGIS catalog on exceededTransferLimit alone and advance the
offset by the rows each page returned, so a lower server-side max
record count no longer cuts the catalog short at the first short page.
- Order pages by OBJECTID_1 (this layer's object id; a plain OBJECTID is
rejected with HTTP 400) so offset paging is stable.
- Raise the page ceiling to 40 and warn when it is hit.
- Tests: short pages with exceededTransferLimit keep paging, orderByFields
is sent, and a service that ignores resultOffset stops at the ceiling.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- dataCredits `gbfs`: names the seven PBSC operators, with BiciMAD's
required "Powered by EMT de Madrid" linking to emtmadrid.es.
- dataCredits `jcdecaux-gbfs`: adds Valenbisi and Sevici.
- DATA_SOURCES.md: a BiciMAD row (Madrid's open-data catalogue publishes
the feed under EMT's terms, attribution required), one row for the other
six PBSC feeds (no licence published; attribution to the operator), and
the JCDecaux row widened to Valencia and Seville.
- CURRENT-STATE.md: the Bikeshare row, and the underscore and
geofenced-capacity rules in "GBFS upstream bounds".
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Bicing (543 stations, PBSC) and AMBici (237 e-bike stations across the
metropolitan area, nextbike) join the GBFS registry from the operators'
keyless GBFS 2.3 feeds, both listed in MobilityData's systems catalogue.
- gbfs.nextbike.net joins the proxy allowlist as an exact host.
- Credits, DATA_SOURCES, CURRENT-STATE and CHANGELOG updated; a test pins
each system's proxy reachability and the range check for its outermost
stations.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codec, product and resolution words never identify a model; generation suffixes such as Mk II no longer fall back to the base model; a mismatched manufacturer never matches, and brand-only numeric names need the brand. The datasheet FOV only fills when the source and seed give none and the pose is not curated, and DATASHEET is shown only while no calibration offset applies. Adds the CC0 extract to DATA_SOURCES.
The seven feeds publish no licence, operator or attribution fields, so DATA_SOURCES records that and credits the operator. Adds a registry test per city: allowed host and path, provider, and activation over its outermost live station.
Two gaps. The Hamilton entry never said its route ids are internal keys,
which the Guelph, Burlington and YRT entries all say of theirs; the
route-table sentence in the transit notes named those three and not
Hamilton, though it now carries the largest table of the four. Both were
stale the moment Hamilton got a table.
The exclusions were half-recorded: GO Transit and Grand River had their
reasons written down, but Ottawa, London, Thunder Bay and the static-only
majority did not, so a later reader would have re-investigated feeds that
were already surveyed and rejected. All of them are now named with the
rule each one falls to, and the survey is stated as complete against the
Mobility Database's Ontario catalogue.
Also records that a route label is never invented, which is why YRT's
unpublished 9101 renders as the feed sends it.
Burlington's route ids are internal keys with a trap in them: `351` is
route 1 while `3510` is route 10, so no prefix trim is correct and the
table is a real join against the City's own published GTFS. Its licence is
the same 2011 municipal template as Mississauga — revocable, and the terms
must travel with the data, so licenseUrl is the terms document itself.
Plain http:// has no listener at all, so https is mandatory, not preferred.
Barrie and YRT ship on. Both operators invite acceptance of their licence
through a form, and both licences also state that use is itself acceptance
("By using Barrie Transit Data, you agree to be bound by these terms";
"Your Use of Information indicates Your acceptance"). This deployment
relies on that clause, and both entries record it so the reasoning does not
have to be re-derived.
Nine Ontario feeds now, all verified live against their upstreams.
York Region's feed is first-party infrastructure under an open licence that
permits commercial use and does not even require attribution. It is the one
feed that fills the gap north of Toronto, and it is switched off anyway.
The operator asks that its licence be accepted through a form collecting a
name, company, phone and email. No credential is issued and nothing gates
the endpoint — the licence body says use is itself acceptance — but a form
asking for contact details is close enough to the registration this registry
refuses that a person should decide it, not a poller. Barrie is off for the
same reason. Both are one line from on.
Viva route ids get labels: YRT publishes `601` where every station sign says
Viva Blue. The numeric local routes already read correctly and carry no
entry, so the table is six lines, not 125.
No vehicle in this feed carries a bearing; heading comes from observed
movement, as with Durham.
Guelph's feed publishes the internal database key of a route (`2991`)
where the rider sees the number on the bus (`1`), which would have put
"Route 2991" on the label a person reads. So a feed may now carry its own
`routeLabel`, the same way it may carry its own `modeWords`. Guelph's table
is its 33 routes, taken from the City's own published GTFS; an id missing
from it renders unchanged, because a raw id you can compare against the bus
beats a blank. Every other feed is identity and carries no table.
Kingston is catalogued by the City against this exact URL. Its licence PDF
link is pinned to the canonical location rather than the legacy path that
currently redirects there — Durham's entry records what happens when such a
redirect is eventually dropped.
Barrie ships REGISTERED AND OFF. The City offers its licence for acceptance
behind a click-through gate, and nobody on this project has accepted it.
Whether to accept is a person's decision, not an assumption a poller should
make, so the feed is visible, documented, and one line from enabled.
Three keyless GTFS-Realtime feeds around Toronto, each verified live: all
vehicles carry real coordinates, and each is polled only near its own
region rather than across the Golden Horseshoe.
The licences differ in ways the registry now records rather than flattens:
Hamilton PRESCRIBES its attribution wording, so the credit is that exact
sentence; Durham makes credit optional and its catalogued licence PDF is
dead after a site relaunch, so licenseUrl points at the live page where the
v.1.0 text was actually read; Mississauga's licence is revocable and asks
that the terms travel with the data, so licenseUrl is the terms themselves.
All three are bus-only and set no route hint, so their modes default rather
than resolve. Durham publishes no bearing on any vehicle — heading comes
from observed movement, as with any feed that omits it.
GO Transit / Metrolinx stays out: its feed requires an API key.
TTC BusTime publishes keyless GTFS-Realtime VehiclePositions with real
coordinates for the surface network: ~1,180 buses and streetcars with
bearing and per-vehicle timestamps. Registering it is one registry row.
The subway is deliberately absent. BusTime covers surface vehicles only
and the subway publishes no positions, so Lines 1/2/4 are left out rather
than inferred from schedules — the same rule that keeps NYCT out.
Route numbers establish the mode: the three-digit 500-series is streetcar,
as are the four Blue Night routes on streetcar trackage. A leading 5 alone
would misclassify bus routes 50-59, so the rule is anchored and tested.
Licensing is the portal-wide Open Government Licence - Toronto; the CKAN
record for the BusTime NVAS dataset leaves its own licence field blank,
and the terms note says so rather than overstating the grant.
Refs #70
Ids were built from the feed's fid, which embeds an upstream server-start stamp,
so every upstream restart renamed all cameras (it already had since the PR
fixture). Ids now come from the validated frame URL (cat-sct-<stem>, or host
and path for partner cameras), which also collapses the five duplicate rows.
Frame links must sit directly in each host's frame directory with no explicit
port, and sctidcam must be a bare image filename. Numeric entity decoding
handles hex references and leaves invalid code points as written, so one bad
name no longer empties the pack. Counts in the docs are now ~130.
The catalog fetch now refuses redirects, cancels rejected bodies and reads
through the shared capped reader before the windows-1252 decode. The feed sends
cardinal codes plus B (both), O and null, not BW or diagonals; tests, comments
and docs now say so, with unchanged id-hash fallbacks. Documents the 8-1,200
per-pack range, drops the unverified low-volume compliance claim for facts and
a link to WSDOT's terms, and stops claiming the per-pack defaults fit under
the 4,000 cap: it is filled round-robin.
The camera endpoints are the eismoinfo.lt website backend, which the Via
Lietuva open-data statement does not list, so the pack now follows the NYC
opt-in pattern (CCTV_LITHUANIA_ENABLED=1) and the docs say why. A failed
catalog request now aborts its sibling instead of leaving it open until the
timeout, and a redirect is logged as not followed.
The catalog fetch now refuses redirects, cancels rejected bodies and reads
through the shared capped JSON reader. Duplicate ids are dropped before
prioritization so they cannot use up the per-pack cap. Vendor-generic model
strings (PTZ, PTZ IP, BOSCH/PTZ) are treated as no model. Comments and credits
say why WSDOT rows are skipped (their frames are WSDOT-hosted, outside the
pinned origin) and name Newcastle as the partner city that appears.
Keeps main's 4,000 catalog-wide camera cap, which main pins in tests; WSDOT
shares it round-robin with the other packs. The branch's earlier cap raise and
its removal of main's cap tests are reverted in this merge.
A failure now ends when its city leaves range or the camera rises above the
activation gate, so an empty map no longer reads LOAD FAILED. Late activation
and status rejections are ignored like late successes. Partial failure reads
DEGRADED, every in-range city failing over cached points reads STALE, and
freshness advances only on a refresh that succeeded. Behavioural tests drive
the real proximity, update and enable paths. Docs say no licence is published
for Toronto and list Social Bicycles and PBSC. The merge also restored the #61
GBFS v3 changelog entry that an earlier merge on main dropped.
Port to current main: main reserved layer-state token 0 for street-level, so
air-quality takes the next free token, 4 (registry, ledger, and the allocation
tests shift by one). Main moved source contracts into sourceComposition.js, so
air-quality registers its getSnapshot contract there. Package boundaries keep
main's lists plus this branch's air-quality modules and packages.
CHANGELOG: keep both new [Unreleased] entries (Panoramax, and the
emergency-transponder highlight from #856).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Keep both: Catalonia CCTV alongside the NYC DOT TMC pack in the source
imports, README layer table and CURRENT-STATE data table, and both
CHANGELOG entries.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
