Exploring the Exploring the Potential of Potential of

OpenStreetMap DataOpenStreetMap Data

JerryJerry CloughCloughSK53 on OpenStreetMap

@SK53onOSMSK53.osm@gmail.com

Maps Matter Blog : www.sk53-osm.blogspot.com

OSM (still) in data collection phase

Sep-12 Mar-13 Sep-13 Mar-14 Sep-14 Mar-150%

20%

40%

60%

80%

100%

120% Post Boxes Mapped by City on OSM

percentage of current total Pittsburgh

Zurich

Karlsruhe

Krakow

Tallinn

Nantes

Salamanca

Nottingham

Aug-12 Aug-140

200

400

600

800

1000

1200

1400

1600

0

20

40

60

80

100

120

140

160

180

200

bookies

NG restaurants

Sep-12 Feb-13 Jul-13 Dec-13 May-14 Oct-14 Mar-150%

20%

40%

60%

80%

100%

120%Restaurants by City on OSM

San Francisco

Pittsburgh

St Louis

Seattle

Oklahoma City

Denver

Zurich

Karlsruhe

Munich

Krakow

Tallinn

Nantes

Salamanca

Nottingham

Themes

● How complete is X on OSM?● Where are the gaps in X?● Is OSM data usable?● If so, what's involved?● Can further information be derived?

ToolkitToolkit● OSM Extracts from Geofabrik● Overpass● taginfo & umap● osmconvert & osmfilter● osm2pgsql & osmosis● PostgreSQL● QGIS● R● Possibles

– Routing Packages: graphhopper, OpenTripPlanner

– RapidMiner

DataData● Data in PostGIS:

– Static data for GB using osm2pgsql

● Lose interconnectedness● Avoid issues in creating

polygons

– Pre-filtered topic-based data sets

– Limited use of (osmosis) snapshot schema

● Common processes:– Geodata » Graph » Geodata

– Gridding of data

Looking at the DataLooking at the DataCounts, Visualisations, ComparisonCounts, Visualisations, Comparison

Simple Visualisations: Simple Visualisations: Pub Density in Great BritainPub Density in Great Britain

Basic CountsBasic CountsSome 'Interesting' Stats for Great Britain

(with apologies to Ordnance Survey)

● Pylons: 75,446 (OSGB: 80,517)● Post Boxes: 51,023 (93.728) ● Camp sites: 4,347 (8,908) ● Buildings: 3,394,208 (35,397,754)● Bus Stops: 222,505 (354,099)

● Petrol Stations: 6,575 (7,702)● Addresses: 892,938 (27,341,262)● Electricity Poles: 150,209 (183, 987)

● Road length: 522,627 km (407,532 km)

● 11 post boxes with Edward VIII cypher

● Only 420 War Memorials

● 1,353 Fire Hydrants

● 2,000 Real Ale pubs

– 109 with Real Fires

● 7,845 Cycle Parking

● 300 Wildlife Hides

● 10,968 Stiles

● 2,033 Canal Locks

● 8 Knitting Shops

Ordnance Survey figures: /www.ordnancesurvey.co.uk/blog/2013/04/10-fascinating-facts-from-ordnance-survey/OSM figures (May '15): /taginfo.openstreetmap.org.uk/

Comparisons:Comparisons:Data CompletenessData Completeness

Food Hygiene Open Data© 2015OSM contributors

Comparisons:Comparisons:Data AccuracyData Accuracy

Land Registry Inspire parcelsOSM gardens etc, Sutton Coldfield (B72)© 2013 OSM Contributors@2013 Crown Copyright & Database right

Comparisons:Comparisons:Missing DataMissing Data

© 2015 OpenStreetMap contributorsSurrey Rights of Way from rowmaps.org.uk

Comparisons:Comparisons:Internal to OSMInternal to OSM

Building a Global Data Building a Global Data SetSet

Retail in OSMRetail in OSM

Long-tail of shop tagsLong-tail of shop tags

Pubs on OpenStreetMapPubs on OpenStreetMap

© 2015 OpenStreetMap contributors; Country Polygons also OSM

01

02

03

10

11

12

2021252628

29

30

31

3132

33

41

51

5253

61

979899X1

01

02

03

10

11

12

20

2526282930

31

3233

41

51

5253

61

979899

01

02

03

10

11

12

2021252628293031313233

41

51

5253

61

979899X1

01

02

03

10

11

12

2025262829

3031313233

41

51

5253

61

9899X1

01

02

03

10

11

12

20

2526282930

31313233

41

51

5253

61

979899X1

01

02

03

10

11

12

20

252628

29

30

31313233

41

51

5253

61

9899X1

02

10

11

12

61

01

02

03

10

11

1220252628

29

3031

32

33

41

51

5253

61

01

02

03

10

11

12

20252628293031323341

51

5253

61

979899

01

02

03

10

11

12

202526282930313233

41

51

5253

61

9899

01

02

03

10

11

12

20

25

2629303133

41

51

61

01

02

03

10

11

12

20

21252628

29

30

31313233

41

51

5253

61

979899

01

02

03

10

11

12

2021252628

29

303132

33

41

51

5253

61

979899

0.00

0.25

0.50

0.75

1.00

0.00 0.25 0.50 0.75 1.00x

y

ret_grp_seq_cd

01

02

03

10

11

12

20

21

25

26

28

29

30

31

32

33

41

51

52

53

61

97

98

99

X1

Cartograms from Retail POIsCartograms from Retail POIs

© 2015 OpenStreetMap contributors; Country Polygons Natural EarthCreated using Scapetoad

Derived & Emergent Derived & Emergent DataData

Landuse PolygonsLanduse Polygons

Deriving Data : Landuse PolygonsDeriving Data : Landuse Polygons

●

●

Gridding PolygonsGridding Polygons• Intersection of all

features on 1km grid– Reduce polygon size

– Performance

– Avoid joining on geometries (use key for grid cell)

PostGIS ProcessingPostGIS ProcessingOSM

Polygons

OSMLines

Painter'sAlgorithm

Rules

ClippedPolygons

ClippedLines

Cleaned &Clipped

Polygons

UA ShapePolygons

Clean GeometryGridded UA

ClassesFilter on Tags & Grid

Gridded &Buffered

UA ClassesTag Filter, Grid & Buffer

Clip to Area

Clip to Area

Piecewise Union Union Step 1

Un

ion

Union Step 2

Me

rge

Class GriddedPolygons

Merge

GridGridded UAPolygons

UnionClipping areasby UA Class

Clip

pin

g R

eg

ion

FinalPolygons

CompareUA/OSM

Union/Intersect/Difference

Emergent Data : Retail land use

Classifying Retail Areas

SummarySummary

● Potential future use of OSM data can be tested by judicious choice of data sets.

● Evaluation of suitability of OSM data for a given purpose solely using internal criteria is still hard.

● Deriving rather than consuming data is often (technically) involved.

● It can be fun in its own right!

I'm not alone!I'm not alone!