The Opportunities and Challenges of Open Data Professor...
Transcript of The Opportunities and Challenges of Open Data Professor...
![Page 1: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/1.jpg)
The Opportunities and Challenges of Open Data
Professor Sir @Nigel_Shadbolt FREng
CILIP 2016, Brighton
![Page 2: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/2.jpg)
scarcity to abundance…
![Page 3: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/3.jpg)
scarcity to abundance…
![Page 4: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/4.jpg)
Kryder’s Law
Kryder’s Law
powers of 10
![Page 5: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/5.jpg)
![Page 6: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/6.jpg)
the data spectrum
![Page 7: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/7.jpg)
the data spectrum
![Page 8: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/8.jpg)
the data spectrum
![Page 9: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/9.jpg)
the data spectrum
![Page 10: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/10.jpg)
the data spectrum
![Page 11: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/11.jpg)
open data: nothing new…
![Page 12: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/12.jpg)
the power of open…
Open Licences
Open Standards
Open DataOpen Participation
Open Source
![Page 13: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/13.jpg)
![Page 14: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/14.jpg)
![Page 15: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/15.jpg)
![Page 16: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/16.jpg)
![Page 17: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/17.jpg)
![Page 18: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/18.jpg)
![Page 19: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/19.jpg)
![Page 20: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/20.jpg)
open data and insurance…
![Page 21: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/21.jpg)
open data and insurance…
![Page 22: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/22.jpg)
open data and spending
![Page 23: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/23.jpg)
open data and crime
![Page 24: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/24.jpg)
open data and transport
![Page 25: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/25.jpg)
open data and the environment
![Page 26: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/26.jpg)
open data and cultural heritage
![Page 27: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/27.jpg)
open data and research
![Page 28: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/28.jpg)
open data and research
![Page 30: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/30.jpg)
http://theodi.org
![Page 31: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/31.jpg)
Use evidencefor investment
Share knowledge
Create ideas together
Build demonstrator
Communicateimpact
![Page 32: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/32.jpg)
![Page 34: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/34.jpg)
http://opencorporates.com/viz/financial/index.html#
![Page 35: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/35.jpg)
open data benefits…
Political
• Transparency• Accountability• Participation
Economic
• Efficiency• Innovation• Growth
Social
• Inclusion• Poverty• Diversity
Research
• Dissemination• Innovation
Media
• Data Journalism
• Data Literacy
Data
• Engagement• Improvement• Acquisition
![Page 36: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/36.jpg)
The four micro principles of the Semantic Web
1. All entities of interest, such as information resources, real-world objects, and vocabulary terms should be identified by URIs.
2. URI references should be dereferenceable, meaning that an application can look up a URI over the HTTP protocol and retrieve RDF data about the identified resource.
3. Data should be provided using the RDF/XML syntax4. Data should be interlinked with other data.
linked data
![Page 37: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/37.jpg)
data on the web - the LOD
![Page 38: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/38.jpg)
data on the web - the LOD
![Page 39: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/39.jpg)
Linked 5★ Gov Data – towards an National Data Infrastructure
•With data.gov.uk a national digital infrastructure being built
•URIs for schools, roads, bus stops, post codes, properties, companies, ...
•Some of the data links across and connects other data together
•Key data link points exist
![Page 40: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/40.jpg)
Open Linked Data and Local Government -http://opendatacommunities.org/
![Page 41: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/41.jpg)
Open Linked Data and Propertyhttp://landregistry.data.gov.uk/
![Page 42: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/42.jpg)
Open Linked Data and Companieshttp://www.companieshouse.gov.uk/toolsToHelp/data
Products.shtml
![Page 43: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/43.jpg)
open data infrastructures: local, national, international
Administrative Geography
Companies
Properties
Energy Consumption
Providers
TariffsContracts
Suppliers
Spending
Timetables
Performance
Fares
![Page 44: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/44.jpg)
“data hugging” excuses…
• We know the data is wrong• We know the data is wrong, and people will tell us where it is wrong• We know the data is wrong, and we will waste valuable resources
inputting the corrections people send us• People will draw superficial conclusions from the data without
understanding the wider picture• People will construct league tables from it• It will generate more Freedom of Information requests• It might be combined with other data to identify individuals/sensitive
information• It will cost too much to put it into a standard format• Our IT suppliers will charge us a fortune to do an ad hoc extract
![Page 45: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/45.jpg)
“data hugging” excuses…
• We know the data is wrong• We know the data is wrong, and people will tell us where it is wrong• We know the data is wrong, and we will waste valuable resources
inputting the corrections people send us• People will draw superficial conclusions from the data without
understanding the wider picture• People will construct league tables from it• It will generate more Freedom of Information requests• It might be combined with other data to identify individuals/sensitive
information• It will cost too much to put it into a standard format• Our IT suppliers will charge us a fortune to do an ad hoc extract
• All these sometimes have some truth in them• Often they rationalise the official fear of the unknown• Data owners need to be helped through • Example & precedent are your friends• Data curators and data scientists on the up…
![Page 46: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/46.jpg)
challenges: boundaries…
![Page 47: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/47.jpg)
• There will be a very great deal of data….
• Who bears the cost of supporting it?
• Can we really treat the Web as a large decentraliseddatabase?
• Dotsam and netsam?
Shut
ters
tock
challenges: computation and storage…
![Page 48: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/48.jpg)
– Interoperability– Endurance– Formats– Proprietary– Migration
challenges: curation…
![Page 49: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/49.jpg)
• A typical example• NaPTAN (public
access transport points)
• Includes 360,00 bus stops
• Around 18,000 errors
challenges: quality…
![Page 50: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/50.jpg)
• A typical example• NaPTAN (public
access transport points)
• Includes 360,00 bus stops
• Around 18,000 errors• Which can be crowd
source improved
challenges: coverage…
![Page 51: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/51.jpg)
• Can’t trust people with the data
• They might interpret it incorrectly
• Do they have the skills• New levels of data
literacy
challenges: data literacy…
![Page 52: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/52.jpg)
![Page 53: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/53.jpg)
• Can’t trust people with the data
• They might interpret it incorrectly
• Do they have the skills• New levels of data
literacy
challenges: data literacy…
![Page 54: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/54.jpg)
opportunities for you
• Everyone depends on data/information/knowledge….
• Data literacy will be a new essential skill…• Your knowledge and insights, methods and
techniques more important, more central more required than ever…– Indexing, organisation, context, sampling, anonymisation…
• You are essential to making data capture, publication, analysis and interpretation business as normal…
• Everyone is a data/information professional now…
![Page 55: The Opportunities and Challenges of Open Data Professor ...cilipconference.org.uk/wp-content/uploads/Sir-Nigel-Shadbolt-reduce… · PowerPoint Presentation Author: Nigel Shadbolt](https://reader033.fdocuments.us/reader033/viewer/2022050114/5f4ae7611ed97844592ecb30/html5/thumbnails/55.jpg)
“by making the demand for openness a paramount issue, quite new possibilities would be created, which, if purposefully followed up, might bring humanity a long way forward towards …co-operation on the progress of civilization”
an open world…