Open Data Trentino presented at the European Commission (JRC)
-
Upload
lorenzino-vaccari -
Category
Technology
-
view
108 -
download
1
description
Transcript of Open Data Trentino presented at the European Commission (JRC)
10/04/231 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://dati.trentino.it/
Open Government Data*
*Part of this presentation is taken from the “Open Government Data Tutorial” gave at CLEI2013 Conference by Lorenzino Vaccari and Juan Pane (Universidad Nacional de Asuncion, Paraguay)
Lorenzino Vaccari
Autonomous Province of Trento, Trento, Italy [email protected]
10/04/232 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
In this presentation…• Introduce Open Government Data
• Intro (Part 1)• Issues (Part 2)
• If you need it, how can you organize it?• Real experience (Part 3)
• Reusing open data• Applications (Part 4)• Semantic layer (Part 5)
10/04/233 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected] 15/10/2013Juan Pane, Lorenzino Vaccari3http://www.point-fort.com/index.php?2012/01/25/805-why-how-what
http://www.point-fort.com/index.php?2012/01/25/805-why-how-what
10/04/234 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
What?
“is data that can be freely used, reused and redistributed by anyone – subject only, at most, to the requirement to attribute and
sharealike.” *
*(Source: )
http://www.opendefinition.org
10/04/235 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
usereuse
“open” = redistributioncommercial reusederivative works
BUT, may require:- attribution- share alike
http://myfbcovers.com/uploads/covers/2012/06/09/16628a1094aa012f7c6e0025902480d2/watermarked_cover.jpg
J. Gray (OKF): http://www.slideshare.net/jwyg/open-government-data-what-why-how
10/04/236 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
The value is in its use
10/04/237 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Maurizio Napolitano: http://www.youtube.com/watch?v=YlkjrVAW43Q
10/04/238 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
New visualizations
J. Gray (OKF): http://www.slideshare.net/jwyg/open-government-data-what-why-how
http://wheredoesmymoneygo.org/
10/04/239 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
New visualizations
13/11/20139J. Gray (OKF): http://www.slideshare.net/jwyg/open-government-data-what-why-how
http://openspending.org
10/04/2310 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Why The Open data are the knowledge base to:
Improve the economic grow and the entrepreneurship based on the development of digital services reusing Public Sector Information
Answer to social needs through the publication of innovative services and applications
Aims at reducing the cost of the public administrative activities within Public – Private Partnerships (PPP)
Improve the transparency of the activities of the public institutions and the participation of the citizens to these activities
J. Gray (OKF): http://www.slideshare.net/jwyg/open-government-data-what-why-how
10/04/2311 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
How - PrinciplesTim Berners-Lee (5-Stars of Linked Open Data)vs.Tim Davis (5-Stars of Open Data Engagement)vs.OGD: Ten principles for opening up government information…
http://sunlightfoundation.com/policy/documents/ten-open-data-principles/
http://5stardata.info/
http://www.timdavies.org.uk/2012/01/21/5-stars-of-open-data-engagement/
10/04/2312 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
5 Stars Linked Open DataTim Berners-Lee
http://5stardata.info
10/04/2313 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Create Communityhttp://msnbcmedia.msn.com/j/MSNBC/Components/Photo/_new/pb-121007-spain-tarragona-pyramid-nj-02.photoblog900.jpg
5-Stars of Open Data Engagement
Tim Davis
10/04/2314 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Open Government Data: Ten principles for opening up government information1. Completeness
2. Primacy (primary source)
3. Timeliness
4. Ease of Physical and Electronic
Access
5. Machine readability
6. Non-discrimination
7. Use of Commonly Owned
Standards
8. Licensing
9. Permanence
10. Usage Costs
10/04/2315 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
State of the ArtWhat is happening around us?• Globally• Europe• Italy
10/04/2316 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Open Data Charter - G8The principles are: Open Data by Default Quality and Quantity Useable by All Releasing Data for Improved Governance Releasing Data for Innovation
http://opensource.com/government/13/7/open-data-charter-g8
https://www.gov.uk/government/publications/open-data-charter/g8-open-data-charter-and-technical-annex
10/04/2317 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://opensource.com/government/13/7/open-data-charter-g8
http://census.okfn.org/
Open Data Census (OKF)
10/04/2318 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://opensource.com/government/13/7/open-data-charter-g8
http://census.okfn.org/country/
10/04/2319 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://opensource.com/government/13/7/open-data-charter-g8
http://census.okfn.org/
Open Data Barometer (ODI)
10/04/2320 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OGD in Europe
screenshots
http://epsiplatform.eu/content/european-psi-scoreboard
10/04/2321 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OGD in EuropeInsert table
http://epsiplatform.eu/content/european-psi-scoreboard http://epsiplatform.eu/content/psi-scoreboard-indicator-list
10/04/2322 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://open-data.europa.eu/
10/04/2323 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OGD in Italy
http://www.dati.gov.it/content/infografica
10/04/2324 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OGD: Part 2 - Issues
10/04/2325 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected] 08/10/2013Juan Pane, Lorenzino Vaccari25http://evian-thesource.com/kids-having-fun/http://evian-thesource.com/kids-having-fun/
10/04/2326 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Open Data. Oh ohh
08/10/2013Juan Pane, Lorenzino Vaccari26
LegalLegalOrganizationalOrganizational TechnicalTechnicalAdoptionAdoptionBarriersBarriers
ContextualContextual
http://www.wallpapermania.eu/wallpaper/trick-or-treat-cute-pumpkins-lanterns-halloween-wallpaper
10/04/2327 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://de.straba.us/wp-content/uploads/2012/08/barrieres_for_implementation_of_ogd.png
10/04/2328 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Organizational Barriers
• Not ready• Lack of resources
• IT• Human
• Don’t want to be ready
http://montcomediation.org/images/MCMC_MyWayYourWay.jpg
10/04/2329 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Legal barriersOpen the Data
All the data that was produced using public money has to be made publicly available (with exceptions)
vs PrivacyYou cannot open data that could allow
correlation of private personal data
Or the complete lack of legislation!
10/04/2330 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Adoption barriersData is not contextualizedPeople are not informedOpening data is a complex task, opening cleaned
data is even more complex.Unclear licenses
10/04/2331 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Technical BarriersAccess to data:
OrganizationalTechnical, Downtimes, logins, Payment fees
Fragmentation, incomplete data, scattered
FormatCataloging, indexing, searchLack of explicit semantics, metadataData is not reliableConflicting standards, models,
ontologies
10/04/2332 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
BarriersZuiderwijk et al 2010
Listed 118 socio-technical impediments for opening data in the literature.FindabilityUsabilityUnderstandablityQualityLinkingComparability and compatibilityMetadata….
http://www.ejeg.com/issue/download.html?idArticle=255
10/04/2333 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Context Barriers
Privileged access to dataOther companies what to avoid legislation of
privacy.Transparency is bad for fraudulent business
http://img.gawkerassets.com/img/182n8vzdlg1iojpg/original.jpg
10/04/2334 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://netdna.webdesignerdepot.com/uploads/photo_manipulation/manipulation-9.jpg
10/04/2335 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Part 3 - Real Experience
10/04/2336 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Our story started with GeoData…
http://www.territorio.provincia.tn.it
10/04/2337 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
5 Stars Linked Geo Data Catalog
DBpedia TrentinoGeoData Freebase
10/04/2338 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
The “Open Data in Trentino” project
• The “Open Data in Trentino” project is a 3 years initiative finalized to develop an open data infrastructure to enhance Service Innovation for Trentino following the PAT strategy for services innovation enabled by ICT. The project will be developed within a partnership between Trento RISE and the Autonomous Province of Trento (PAT) according to the innovation PAT model
• Goals• Improved quality of life for citizens• Open Data and local businesses• Transparency• Improved efficiency and productivity
10/04/2339 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Workplan – Best practices Not only a Project, but also a “Change management process”
Best Practices:- Guidelines (metadata, formats, licences)- Point of contact (domain, operator)- ONE dataset each provider- Community Building- Distributed catalog- Clear Licences- Enterprises- Courses- Contest
10/04/2340 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Nome (Acronimo) Descrizione
Tipo di Dato Estensione del file
Comma Separated Value (CSV) Formato testuale per l'interscambio testuale di tabelle, le cui righe corrispondono a linee e i cui valori delle singole colonne sono separati da una virgola (o punto e virgola)
Dato tabellare .csv
Geographic Markup Language (GML) Formato XML utile allo scambio di dati territoriali di tipo vettoriale
Dato geografico vettoriale
.gml
Keyhole Markup Language (KML) Formato basato su XML creato per gestire dati territoriali in tre dimensioni nei programmi Google Earth, Google Maps
Dato geografico vettoriale
.kml
Open Document Format (ODF) Formato per l'archiviazione e lo scambio di documenti di testo, fogli di calcolo, diagrammi e presentazioni
Dato tabellare .odc
Resource Description Framework (RDF) Basato su XML, e' lo strumento base proposto da World Wide Web Consortium (W3C) per la codifica, lo scambio e il riutilizzo di metadati strutturati e consente l'interoperabilità tra applicazioni che si scambiano informazioni sul Web
Dato strutturato .rdf
ESRI Shapefile (SHP) Lo Shapefile ESRI è un popolare formato vettoriale per sistemi informativi geografici. Il dato geografico viene distribuito normalmente attraverso tre o quattro files (se indicato il sistema di riferimento delle coordinate). Il formato è stato rilasciato da ESRI come formato (quasi) aperto
Dato geografico vettoriale
.shp, .shx, .dbf,
.prj
Extensible Markup Language (XML) E' un formato di markup, ovvero basato su un meccanismo che consente di definire e controllare il significato degli elementi contenuti in un documento o in un testo attraverso delle etichette (markup)
Dato strutturato .xml
Guidelines
10/04/2341 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
…MeteoMeteo GeoDatiGeoDati StatisticaStatistica Comune
TrentoComuneTrento TrasportiTrasporti Etc…Etc……
Tecnological platform
10/04/2342 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Data Sources
10/04/2343 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Data Sources Plan
Novembre
Dicembre Gennaio
Catasto # 18
SGC CSW #9
10 20 30 302010 10 20 30
Attività Culturalii #59
Servizio Istruzione #57Attività Form #58
Dati Energia #30
Dati Progettone #63
Dati Motorizzazione #72
Elettorali #35
Gestione Strade #16
Bilancio PAT #37, 38
PersonalePAT #41
Turismo STU #53Idrometr
ici#26
Trentino Cultura #32
Ufficio Rifiuti #34
Servzio Europa #56
Aff FinanziariConsuienze #36
Min. Linguistiche #48
Pub. Esercizi #49
Imp Funivie #50
Immigrazione #52
Sovr. Beni Arch #60Dati Scuola #61
Agenzie Forestali #64Incendi #65
Cinformi Stranieri #66
Imp Depurazione #68Opere Civili #69
Dati Traffico Stra #70
Gestioni Patrimonialii #71
Dati SAT #28
Dati Cons. Prov #3
Trasporti 2.0 #6
OsservatorioLavori Pubb #17
Comune Trento update
Dati Cons. Prov #3
Dati SAT #28
Dati Cons. Prov #3
Dati SAT #28
Dati Cons. Prov #3
10/04/2344 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Catalog
The Open Knowledge Foundation (OKF) is a non-profit organisation founded in 2004 and dedicated to promoting open data and open content in all their forms – including government data, publicly funded research and public domain cultural content.
(2004)
http://okfn.org
10/04/2345 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://dati.trentino.it*
Analysis: http://dati.trentino.it/stats Admin: http://dati.trentino.it/admin Harvesting: http://dati.trentino.it/harvest
* Available for all the data providers of Trentino
10/04/2346 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Services
10/04/2347 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Legal Issues
Permissions: share, create, adapt
Actual interoperability!
Constraints: nothing!
http://www.hoax-slayer.com/images/privacy.jpghttp://www.destateparks.com/images/general_info/privacy_policy.jpg
Permissions: share, create, adapt
Actual interoperability!
Constraints: nothing!
10/04/2348 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Organizational Issues - Macro
10/04/2349 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Organizational Issues - Micro
10/04/2350 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Community buildingMunicipalities“Consorzio dei
Comuni”
Municipalities“Consorzio dei
Comuni”
“Comunità di Valle”
of Trentino
“Comunità di Valle”
of Trentino
Private Companies
Private Companies CitizensCitizens
Educational Institutes
Educational Institutes
Research InstitutesResearch Institutes
10/04/2351 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
International Community
10/04/2352 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Also Trentino is going to launch a challenge to build software applications and creative products (multimedia, audiovisual products, posters, illustrations) based on the datasets published on the http://dati.trentino.it open data catalog.
#ODTChallenge will be the official hashtag for our first open data challenge in Trentino!
10/04/2353 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
10/04/2354 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
8 months until now68.555 visits 7.988 unique visits2.516 downloads
37,36% returning visitors
62,64% new visitors
NOW- ALL the departmnets demand to be involved- Plus other local actors
AgricultureCultureGeographical DataWelfareWeather ForecastSocial policiesStatisticsTransports…MUNICIPALITY OF TRENTO, and
INFORMATICA TRENTINA
580 datasetsprovided by 10 departments of PAT…
20 reporting errors15 asking for new data10 new suggestions6 OD Applications
100% ENTHUSIASTIC REACTIONS
10/04/2355 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Want to Know more? A couple of links
10/04/2356 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://www.theodi.org/
10/04/2357 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://schoolofdata.org/
10/04/2358 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://schoolofdata.org/online-resources/
10/04/2359 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OGD: Part 4 - Applications
10/04/2360 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Apps4Italy
10/04/2361 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Best Application: http://parlamento17.openpolis.it/
10/04/2362 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Open Bilancio
Best Idea: http://opendata.comune.fi.it/open_bilancio/
10/04/2363 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://limaio.innovacion.pe/ http://www.limaio.com/demo
Open Source, Open Data, Open Hardware
10/04/2364 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://www.mysociety.org/2007/more-travel-maps/morehousing
10/04/2365 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Johann MITTHEISZ (CIO der Stadt Wien)
http://www.slideshare.net/BrigitteLutz/keynote-mittheisz-cio-stadt-wien/16
Total hours to develop 38 applications:around 2.600
City of Wien saved around 208.000 Euro
10/04/2366 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Beyond Data (The OpenStreetMap Case)
10/04/2367 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OpenStreetMap
~
OpenStreetMap project creates and provides geographical data, such as road maps, freely available to anyone. Behind the establishment and growth of the project have been restrictions on use or availability of map information across much of the world and the advent of inexpensive portable satellite navigation devices.
OpenStreetMap is a free map of theworld, created by someone like you
10/04/2368 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://tools.geofabrik.de/mc/?mt0=mapnik&mt1=googlemap&lon=11.12042&lat=46.07224&zoom=18
10/04/2369 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Watercolor maps
http://content.stamen.com/files/cartography/index_watercolor.html#18.00/46.07204/11.12097
10/04/2370 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
From maps to blankets…
http://softcities.net
10/04/2371 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Sharing Data Globally(the eHabitat example)
10/04/2372 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
The Group of Earth Observation
Source: http://www.slideshare.net/angeled/geoss © GEO secretariat84 GEO members and 61 Participating organizations
10/04/2373 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
GEOSS Data Sharing Principles • Full and Open
Exchange of Data, recognizing Relevant International Instruments and National Policies
• Data and Products at Minimum Time delay and Minimum Cost
• Free of Charge or minimal Cost for Research and Education
http://www.geoportal.org/web/guest/geo_home
10/04/2374 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
GEOSS for biodiversity
http://www.eurogeoss-broker.eu/
10/04/2375 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
The eHabitat Model
http://ehabitat-wps.jrc.ec.europa.eu/ehabitat/
10/04/2376 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
OGD: Part 5 – Semantic Layer
10/04/2377 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Available
Structured
Open formats
Redefenceable
Linked
Linked Open Data
The best data is an open data
Vs.
All data must be perfect
10/04/2378 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Lack of explicit semanticsThe real meaning of the data was kept in the developers mind when creating the data
78http://goo.gl/npEHKr
10/04/2379 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Lack of explicit semanticsCan lead to things like:
10/04/2380 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Semantic heterogeneityDifference in the meaning of local data
10/04/2381 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Available
Structured
Open formats
Redefenceable
Linked
Data Catalog
Data Catalog
Entity centric
Importing tool
Entity centric
Importing tool
10/04/2382 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Entity centric: Added valueAggregated dataAccurate data, manually curatedUnique identifiers, distributed perspectives
Re-think identifiersSemantified values
E1
name Juan Pane
nationality italian
lives in Trento
affiliation Univ. Trento
E2
name Ignacio P. F.
born in Paraguay
date of birth 1980
affiliation PF-UNA
10/04/2383 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
EntitiesReal world: is something that has a distinct,
separate existence, although it need not be a material (physical) existence. Has a set of properties, which evolve over time. Example:
Mental: personal (local) model created and maintained by a person that references and describes a real world entity.
Digital: capture the semantics of real world entities, provided by people.
10/04/2384 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Entity based Semantic Layer:• Address the integration problems due to
semantic heterogeneity:• Different formats• Different identifiers• Implicit semantics• Homonyms, synonyms, aliases• Partial knowledge• Knowledge evolution
http://www.webfoundation.org/2011/11/5-star-open-data-initiatives/
10/04/2385 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
ImportingTool
ImportingTool
The semantic Layer: why?
ImportingTool
ImportingTool
ImportingTool
ImportingTool
REST/HTTPREST/HTTP
i i+1v0
Applications use entities instead of raw data
10/04/2386 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Importing steps
Selection
Schema Matching
Data Validation
Semantic Enrichment
Reconciliation
Exporting
Publishing
Visualization
1.
2.
3.
4.
5.
6.
7.
8.
Take raw data from dati.trentino.it
Cleanse data
Map to an EntityType
Link data to entities/concepts
Update/insert entities
Export to Entitypedia
Publish to dati.trentino.it
Get insights about entities
10/04/2387 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
1. Source SelectionImport one data file at a time
10/04/2388 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
2. Schema MatchingSelect a target type of entity -> correspondences between the input columns and the output attributes
nome provincia descrizione funivie lat long
Andalo (1047) Provincia di Trento
Sorge su un'ampia sella prativa al centro...
3 654463 712857
Canazei (1450) Trento Prov. Situato all'estremità settentrionale della...
2 511504 147444
10/04/2389 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
3. Data ValidationApplies format and structure validation and possible automatic transformations needed to have the input data in the expected format.
10/04/2390 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
4. Semantic Enrichment (1/2)Entity disambiguation: Transform text references into links to existing entities.
10/04/2391 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
4. Semantic Enrichment (2/2)Natural Language Processing: Extract concepts and entity references from free-text.
10/04/2392 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
5. ReconciliationRun Identity Management Algorithms to identify each row as a new or existing entity.
Result•No Match•Match•Multiple Matches
Action:•Use ID•New ID•Ignore Row
10/04/2393 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
6. ExportingAt this point:We know what to export.All values for target attributes conform to the expected format.All text has been semantified (NLP).All textual references to entities are converted to linksEach row has an identifier
i i+1v0
10/04/2394 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
7. PublishingPut back the semantified entities into CKAN so that
the entities can be Open Data and can be found in the same catalog as the original data.
Developers and find the data files of the cleaned, aggregated entities
But can also interact with the entities via the Entitypedia APIs
8. VisualizationSearch and Navigation
10/04/2395 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Our Goal
TN
UK
BEES
10/04/2396 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
http://www.youtube.com/watch?v=Bq_ZWl1ZXA0
BEYOND
10/04/2397 Lorenzino Vaccari - Autonomous Province of Trento, Trento, Italy - [email protected]
Thanks to all the Open Data in Trentino Team and in particular to:Juan Pane, Maurizio Napolitano, Marco Combetto, Moaz Reyad and Luca Paolazzi