mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-04 02:14:01 +00:00
feat(bikeshare): add Oslo, Bergen and Trondheim Bysykkel
Add Norway's three Urban Sharing city-bike systems to the Bike Share layer and get_bike_share. The feeds are keyless GBFS on gbfs.urbansharing.com, which joins the proxy allowlist. The proxy now sends the Client-Identifier header the operators ask clients to send. Attributed under NLOD 2.0 in DATA_SOURCES.md and the data credits. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
a08a53cf94
commit
42543c1fa1
10 files changed
+154
-1
No files matched your search
@@ -2,6 +2,12 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- Add Norway's city bikes to the Bike Share layer: Oslo, Bergen and
|
||||
Trondheim Bysykkel (Urban Sharing, ~440 stations). The feeds are keyless
|
||||
GBFS on `gbfs.urbansharing.com`, which joins the proxy allowlist; the proxy
|
||||
now sends the `Client-Identifier: gods-eye-view` header the operators ask
|
||||
for. Attributed under NLOD 2.0. `get_bike_share` covers the three cities too.
|
||||
|
||||
- Add Norway's road cameras from Statens vegvesen as a CCTV pack. The camera
|
||||
list is one keyless GeoJSON request to the agency's OGC API view of its
|
||||
DATEX 3.1 CCTV table, so no DATEX account is needed. Every working camera
|
||||
|
||||
@@ -56,6 +56,7 @@ How to read this:
|
||||
| **Open Calgary** (`data.calgary.ca`) | CCTV camera catalog + frames, Calgary | [Open Government Licence – City of Calgary](https://data.calgary.ca/stories/s/Open-Calgary-Terms-of-Use/u45n-7awa) — attribution REQUIRED | "Contains information licensed under the Open Government Licence – City of Calgary" |
|
||||
| **Statens vegvesen** (catalog `ogckart-sn1.atlas.vegvesen.no`, frames `kamera.atlas.vegvesen.no`, HLS `kamera.vegvesen.no`) | CCTV camera catalog + frames + live HLS video, Norway road cameras | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED | "Contains data under the Norwegian licence for Open Government data (NLOD) distributed by Statens vegvesen" |
|
||||
| **GBFS (Lyft / BCycle)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` |
|
||||
| **Oslo / Bergen / Trondheim Bysykkel** (Urban Sharing, `gbfs.urbansharing.com`) | Bikeshare availability, Oslo, Bergen and Trondheim city bikes. Keyless GBFS; the proxy sends the `Client-Identifier` header the operators ask for | [NLOD 2.0](https://data.norge.no/nlod/en/2.0) — attribution REQUIRED ([Oslo](https://oslobysykkel.no/en/open-data/realtime), [Bergen](https://bergenbysykkel.no/en/apne-data/sanntid), [Trondheim](https://trondheimbysykkel.no/en/open-data/realtime)) | "Contains data under the Norwegian licence for Open Government data (NLOD) made available by Oslo Bysykkel, Bergen Bysykkel and Trondheim Bysykkel" |
|
||||
| **MBTA / MassDOT** (cdn.mbta.com) | Transit layer live vehicles and up to 15 minutes of recently observed positions, Greater Boston | MBTA / MassDOT open realtime vehicle data, used for the live view and up to fifteen minutes of recently observed positions; published for developer use and not against their use. | Attribution to MBTA / MassDOT (courtesy text) |
|
||||
| **CapMetro** (data.texas.gov) | Transit layer live vehicles, Austin TX | CapMetro open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to CapMetro (courtesy text) |
|
||||
| **Metro Transit** (Metropolitan Council) | Transit layer live vehicles, Minneapolis–St Paul | Metro Transit / Metropolitan Council open realtime vehicle data, used for the live view; published for developer use and not against their use. | Attribution to Metro Transit / Metropolitan Council (courtesy text) |
|
||||
|
||||
@@ -961,6 +961,9 @@ The station reader addresses the proxy as `/api/gbfs/<encoded feed URL>`; the
|
||||
feed address is read from the path, and a query-string form is refused with a 400. GBFS refuses upstream redirects and enforces its 5 MiB response cap while
|
||||
streaming. The 12-second deadline includes reading the body, and rejected or
|
||||
stalled downloads are cancelled. Development and preview use the same handler.
|
||||
Every upstream request carries `Client-Identifier: gods-eye-view`, which Urban
|
||||
Sharing asks of clients of the Oslo, Bergen and Trondheim Bysykkel feeds
|
||||
(`gbfs.urbansharing.com`); other operators ignore it.
|
||||
|
||||
## Remaining local service modules
|
||||
|
||||
@@ -2922,7 +2925,7 @@ its criteria cannot be silently ignored.
|
||||
| CCTV | Austin + Caltrans (CA) + TfL London + Ontario 511 + Fintraffic (FI) + DriveBC (BC) + TxDOT (TX) + Estonia (Tallinn, Tarktee) + Live Traffic NSW + Open Calgary Open Data + Statens vegvesen (NO, stills + live HLS) + Street View fallback | `src/data/cctv.js` | `/api/cctv` | 10s (active) |
|
||||
| Radio | Radio Browser (public-domain station directory) | `src/data/radio.js` | `/api/radio/stations`, `/api/radio/click/:uuid` | 45 min directory refresh |
|
||||
| Transit 🚌 | Operator GTFS-Realtime VehiclePositions (7 keyless regions, `src/data/transitFeeds.js`) | `src/layers/transit/` via `src/app/layers/transit.js` | `/api/transit` | 15s (poll + delayed playback) |
|
||||
| Bikeshare 🚲 | GBFS (Lyft + BCycle) | `src/data/bikeshare.js` | `/api/gbfs` | 60s |
|
||||
| Bikeshare 🚲 | GBFS (Lyft + BCycle + Urban Sharing NO) | `src/data/bikeshare.js` | `/api/gbfs` | 60s |
|
||||
| Directions 🧭 | OSRM on FOSSGIS servers (OpenStreetMap) | `src/data/directions.js` | `/api/route` (`steps=1`) | on placement / mode change |
|
||||
| Datacenters ▣ | OSM extract (bundled) | `src/data/localLayers.js` | — | static |
|
||||
| Dams ▰ | OpenInfraMap/OSM extract (bundled) | `src/data/localLayers.js` | — | static |
|
||||
|
||||
@@ -11,6 +11,9 @@ import {
|
||||
/** Upstream fetch timeout for GBFS requests (ms). */
|
||||
const GBFS_PROXY_TIMEOUT_MS = 12000;
|
||||
|
||||
/** Value of the Client-Identifier header sent with every upstream request. */
|
||||
export const GBFS_CLIENT_IDENTIFIER = 'gods-eye-view';
|
||||
|
||||
export const GBFS_MAX_BODY_BYTES = 5 * 1024 * 1024; // 5 MB
|
||||
|
||||
function gbfsRedirectHost(location, requestUrl) {
|
||||
@@ -60,6 +63,9 @@ export async function fetchGbfsUpstream(
|
||||
headers: {
|
||||
Accept: 'application/json',
|
||||
'User-Agent': 'gods-eye-view-gbfs-proxy/1.0',
|
||||
// Urban Sharing (the Norwegian city-bike feeds) asks every client to
|
||||
// name itself in this header; other operators ignore it.
|
||||
'Client-Identifier': GBFS_CLIENT_IDENTIFIER,
|
||||
},
|
||||
redirect: 'manual',
|
||||
signal: controller.signal,
|
||||
|
||||
@@ -251,6 +251,13 @@ export const DATA_CREDITS = [
|
||||
key: 'gbfs',
|
||||
html: 'Bikeshare availability: GBFS operator feeds (e.g. Austin BCycle)',
|
||||
},
|
||||
{
|
||||
key: 'urbansharing-gbfs',
|
||||
html:
|
||||
'City bikes (Norway): contains data under the ' +
|
||||
'<a href="https://data.norge.no/nlod/en/2.0" target="_blank" rel="noopener">Norwegian licence for Open Government data (NLOD)</a> ' +
|
||||
'made available by Oslo Bysykkel, Bergen Bysykkel and Trondheim Bysykkel',
|
||||
},
|
||||
{
|
||||
key: 'osrm-routing',
|
||||
// The service asks for its attribution to carry a "fix the map" link, so
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
GBFS_CLIENT_IDENTIFIER,
|
||||
GBFS_MAX_BODY_BYTES,
|
||||
fetchGbfsUpstream,
|
||||
} from '../../server/providers/gbfs.js';
|
||||
@@ -56,6 +57,21 @@ test('GBFS upstream fetch passes a direct 200 through unchanged', async () => {
|
||||
});
|
||||
});
|
||||
|
||||
test('GBFS upstream fetch names the app in the Client-Identifier header', async () => {
|
||||
let observedHeaders = null;
|
||||
await fetchGbfsUpstream(
|
||||
'https://gbfs.urbansharing.com/oslobysykkel.no/station_status.json',
|
||||
{
|
||||
fetchImpl: async (_url, options) => {
|
||||
observedHeaders = options.headers;
|
||||
return new Response(STATION_BODY, { status: 200 });
|
||||
},
|
||||
},
|
||||
);
|
||||
assert.equal(GBFS_CLIENT_IDENTIFIER, 'gods-eye-view');
|
||||
assert.equal(observedHeaders['Client-Identifier'], GBFS_CLIENT_IDENTIFIER);
|
||||
});
|
||||
|
||||
test('GBFS upstream fetch forwards a non-redirect error status for the middleware to relay', async () => {
|
||||
const result = await fetchGbfsUpstream(STATION_URL, {
|
||||
fetchImpl: async () => new Response(null, { status: 404 }),
|
||||
|
||||
@@ -8,6 +8,7 @@ const GBFS_ALLOWED_HOSTS = new Set([
|
||||
'austin.publicbikesystem.net',
|
||||
'hon.publicbikesystem.net',
|
||||
'chat.publicbikesystem.net',
|
||||
'gbfs.urbansharing.com',
|
||||
]);
|
||||
|
||||
/**
|
||||
|
||||
@@ -17,6 +17,7 @@ import { createModel } from './model.js';
|
||||
import { CITY_BY_ID } from './registry.js';
|
||||
|
||||
const AUSTIN = CITY_BY_ID.get('austin-capmetro');
|
||||
const OSLO = CITY_BY_ID.get('oslo-bysykkel');
|
||||
|
||||
const STATION_INFORMATION = {
|
||||
data: {
|
||||
@@ -204,6 +205,49 @@ test('the proxy relays its cache policy and upstream host through the mounted ro
|
||||
);
|
||||
});
|
||||
|
||||
test('the Norwegian city-bike systems are served through the mounted route', async () => {
|
||||
for (const [id, systemId] of [
|
||||
['oslo-bysykkel', 'oslobysykkel.no'],
|
||||
['bergen-bysykkel', 'bergenbysykkel.no'],
|
||||
['trondheim-bysykkel', 'trondheimbysykkel.no'],
|
||||
]) {
|
||||
const system = CITY_BY_ID.get(id);
|
||||
assert.ok(system, `${id} is registered`);
|
||||
assert.deepEqual(system.hosts, ['gbfs.urbansharing.com']);
|
||||
assert.equal(
|
||||
system.stationStatusUrl,
|
||||
`https://gbfs.urbansharing.com/${systemId}/station_status.json`,
|
||||
);
|
||||
}
|
||||
|
||||
await withUpstream(
|
||||
{
|
||||
[OSLO.stationInformationUrl]: STATION_INFORMATION,
|
||||
[OSLO.stationStatusUrl]: STATION_STATUS,
|
||||
},
|
||||
async (requested) => {
|
||||
const source = createBikeshareSource({ fetchImpl: mountGbfsProxy() });
|
||||
const info = await source.getStations(OSLO.stationInformationUrl);
|
||||
const status = await source.getStations(OSLO.stationStatusUrl);
|
||||
assert.deepEqual(requested, [
|
||||
OSLO.stationInformationUrl,
|
||||
OSLO.stationStatusUrl,
|
||||
]);
|
||||
assert.equal(info.data.stations.length, 1);
|
||||
assert.equal(status.data.stations.length, 1);
|
||||
|
||||
const response = await mountGbfsProxy()(
|
||||
'/api/gbfs/' + encodeURIComponent(OSLO.stationStatusUrl),
|
||||
);
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(
|
||||
response.headers.get('x-gbfs-upstream'),
|
||||
'gbfs.urbansharing.com',
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('the query-string shape the client used to send is refused by the mounted route', async () => {
|
||||
await withUpstream(
|
||||
{ [AUSTIN.stationStatusUrl]: STATION_STATUS },
|
||||
|
||||
@@ -23,6 +23,26 @@ function bcycleEntry({
|
||||
...buildBcycleUrls(systemId),
|
||||
};
|
||||
}
|
||||
function urbanSharingEntry({
|
||||
id,
|
||||
city,
|
||||
centerLat,
|
||||
centerLon,
|
||||
systemId,
|
||||
provider,
|
||||
loadRadiusKm = 100,
|
||||
}) {
|
||||
return {
|
||||
id,
|
||||
city,
|
||||
centerLat,
|
||||
centerLon,
|
||||
loadRadiusKm,
|
||||
provider,
|
||||
stationInformationUrl: `https://gbfs.urbansharing.com/${systemId}/station_information.json`,
|
||||
stationStatusUrl: `https://gbfs.urbansharing.com/${systemId}/station_status.json`,
|
||||
};
|
||||
}
|
||||
const RAW_GBFS_CITY_REGISTRY = [
|
||||
{
|
||||
id: 'nyc-citibike',
|
||||
@@ -318,6 +338,30 @@ const RAW_GBFS_CITY_REGISTRY = [
|
||||
centerLon: -119.6982,
|
||||
systemId: 'bcycle_santabarbara',
|
||||
}),
|
||||
urbanSharingEntry({
|
||||
id: 'oslo-bysykkel',
|
||||
city: 'Oslo, Norway',
|
||||
centerLat: 59.9139,
|
||||
centerLon: 10.7522,
|
||||
systemId: 'oslobysykkel.no',
|
||||
provider: 'Oslo Bysykkel',
|
||||
}),
|
||||
urbanSharingEntry({
|
||||
id: 'bergen-bysykkel',
|
||||
city: 'Bergen, Norway',
|
||||
centerLat: 60.3913,
|
||||
centerLon: 5.3221,
|
||||
systemId: 'bergenbysykkel.no',
|
||||
provider: 'Bergen Bysykkel',
|
||||
}),
|
||||
urbanSharingEntry({
|
||||
id: 'trondheim-bysykkel',
|
||||
city: 'Trondheim, Norway',
|
||||
centerLat: 63.4305,
|
||||
centerLon: 10.3951,
|
||||
systemId: 'trondheimbysykkel.no',
|
||||
provider: 'Trondheim Bysykkel',
|
||||
}),
|
||||
];
|
||||
function normalizeRegistryEntry(entry) {
|
||||
const id = String(entry?.id || '')
|
||||
|
||||
@@ -429,6 +429,31 @@ test('bike share names the systems it did not search', async () => {
|
||||
);
|
||||
});
|
||||
|
||||
test('bike share over Oslo searches Oslo Bysykkel only', async () => {
|
||||
const requested = [];
|
||||
const result = await composeCatalog({
|
||||
tools: coreTools,
|
||||
services: {
|
||||
bikeshare: {
|
||||
systems: GBFS_CITY_REGISTRY,
|
||||
getStations: async (url) => {
|
||||
requested.push(url);
|
||||
return { data: { stations: [] } };
|
||||
},
|
||||
},
|
||||
},
|
||||
}).call('get_bike_share', {
|
||||
area: { lat: 59.9139, lon: 10.7522, radius_km: 5 },
|
||||
});
|
||||
assert.deepEqual(result.data.systems, ['Oslo, Norway']);
|
||||
assert.deepEqual(result.data.systems_not_searched, []);
|
||||
assert.ok(
|
||||
requested.every((url) =>
|
||||
url.startsWith('https://gbfs.urbansharing.com/oslobysykkel.no/'),
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
test('traffic counts only roads inside a radius area and reports missing tiles', async () => {
|
||||
const flow = (segments, partial = false) => ({
|
||||
getStatus: async () => ({ hasKey: true }),
|
||||
|
||||
Reference in new issue
Block a user