Post on 26-Dec-2015
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
1
Overview
• Humboldt: a GMES and INSPIRE project
• Data harmonisation (WP7)
• Suggestions?
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
2
HUMBOLDT – Integrate knowledge
• A FP6 EU project
• GMES and INSPIRE
• An INSPIRE test case
• Project Title
Development of a framework for Data
Harmonisation and Service Integration
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
3
Spatial Information Infrastructure(s)
BusinessessConsumers
s
Governments
Users
Information
DemographicsHealth Transportation
Crime
Real Estate
……infrastructures rely on a variety of technology infrastructures rely on a variety of technology “standards” and network connections.“standards” and network connections.
Network Connections
Finance
Environment
ShoppingPolitics
Leasure
Economic
Defense
PublicSafety
Internet, World Wide Web, and other standards
Source: Reed 2002
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
4
HUMBOLDT – Integrate knowledge
• A FP6 EU project
• GMES and INSPIRE
• An INSPIRE test case
• Project Title
Development of a framework for Data
Harmonisation and Service Integration
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
5
Two kinds of end products
• The Humboldt Framework
An open source software components toolkit to build an SDI
Can be Web services, local components
The ‘glue’ to connect, configure and administer these components
Tutorials, best practives
• Data models for a number of application domains (the scenarios)
The common models (in UML, in GML/XML schemas, in thesauri, in metadata registries)
A data harmonisation module in the Framework (run time)
Guidelines, tools (design time)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
6
HUMBOLDT - Leitmotif
• Project philosophy
Re-use existing –
Extend by need –
Arrive at the ESDI –
Prosper by application
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
7Geo-services
Topo= Map Server
Clients
BaseMap = Map Server
ImageryImagery= Map= MapServerServer
Raster= Map Server
Network = Map Server
Distributed Mapping or geo-enabled services to present and analyze information from “Geo-Servers” using different vendors technology and rendering methods
RDBMS / AEC / CAD / GIS = Feature Servers Stored data ‘Export’ Rendering
From servers to clients
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
8
Integrate, re-use, share
HUMBOLDT framework Development
HU
MB
OLD
T S
cenario
- 1
HU
MB
OLD
T S
cenario
- 2
HU
MB
OLD
T S
cenario
- 3
HU
MB
OLD
T S
cenario
- 4
HU
MB
OLD
T S
cenario
- 5
HU
MB
OLD
T S
cenario
- 6
HU
MB
OLD
T S
cenario
- 7
HU
MB
OLD
T S
cenario
- 8
HU
MB
OLD
T S
cenario
-9+
+
Rela
ted S
cenario
s
Use
r N
eed
s
Data
S
ets
GM
ES
Pro
j.
…
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
9
Scenario applications
• 1. Border Security (cross-border)
• 2. Urban Planning
• 3. Forest (cross-border, number of central European countries)
• 4. Protected Areas (follow-up of Nature-GIS)
• 5. European Risk Atlas (cross-border natural disaster information)
• 6. Water (extension of SDIGER project)
• 7. Ocean (oil pollution, integration of ocean and land data)
• 8. Galileo (atmospheric information, mobile devices, sensors)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
10
Our (=TUD) role in the project
• Lead of WP 7: data harmonisation
• We are also part of other work packages:
WP 3: state of the art
WP 4: software development methodology
WP 5: design phase of the Framework
WP 9: scenarios
WP 11 and 12: dissemination and training
plus small role in WP 10 and 2
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
11
Information flow between WPs
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
WP 7 Data harmonisation (TU Delft WP Lead)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
13
ETRA Spain
HSRS Czech Republic
INGR Czech Republic Profile for security
TUD Netherlands WP Lead, Concept of application-specific data
models
ULSoR Italy
FOMI Hungary
MARIS Netherlands Profile for ocean and marine environment
RT-GIS Germany Profile for risk management
IFREMER France
NERC/BODC United Kingdom
HCMR Greece
SMHI Sweden Profile for atmosphere
TPZ Italy
CNR-IREA Italy
FMI Czech Republic Profile for Land cover and vegetation
IGP Portugal
CLS France
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
14
Role of WP7
•2.4.3. Providing Common Spatial Data Models …
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
15
2.4.3. Providing Common Spatial Data Models …
•Conceptual = implementation-neutral
•Vector data and raster data
•Data content: geographic and ‘thematic’ attributes
•Also the metadata (fuzzy boundary between data and metadata)
•Also look at classification schemas, visualization part
(cartographic consistency)
•Common models = find the optimal level of harmonisation
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
16
What belongs to the common model, what is an organisation-specific extension?
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
17
Also cross-border cartography
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
18
Cross border: missing info / gaps
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
19
Same area: conflation
Inconsistencies between datasets: which one is the ‘best’ representation
(c.f. Uitermark research)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
20
3102 Weg
Befestigter oder unbefestigter Geländestreifen, der zum Befahrenund/oder Begehen vorgesehen ist.
Paved or unpaved stretch of landscape that is suited for driving vehicles or walking.
Roads suitable for vehicle that are not open to the general public e.g. forest roads, agricultural roads
Road
A way maintained for vehicular use.
Roads for bicycleand tractor might be included
Footpath,Cart track,Bridle path etc.
EuroRoadS
3101 StrasseBefestigter, dem allgemeinen Verkehr dienender Verkehrsweg
all-weather (paved) way that can be used by the general public
ATKIS (Germany)
Data Content Definition (Semantics)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
21
3102 Weg
Befestigter oder unbefestigter Geländestreifen, der zum Befahrenund/oder Begehen vorgesehen ist.
Paved or unpaved stretch of landscape that is suited for driving vehicles or walking.
Roads suitable for vehicle that are not open to the general public e.g. forest roads, agricultural roads
Road
A way maintained for vehicular use.
Roads for bicycleand tractor might be included
Footpath,Cart track,etc.
3101 StrasseBefestigter, dem allgemeinen Verkehr dienender Verkehrsweg
all-weather (paved) way that can be used by the general public
Data Content Definition (Semantics)
Common model: Data Dictionary
Language: natural language, multilingual, Ontologies
EuroRoadS ATKIS (Germany)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
22
The “core” model: a link between sectors
RoadsWater
Core
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
23
Water Roads
Themes and sectors overlap
Transport
Environment
Core
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
24
Water Roads
As sectoral interoperability develops, the common core grows …
Transport
Environment
CoreCore
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
25
Water Roads
Common and other Reference Data
Transport
Environment
CoreCoreCore
CommonReference
Data
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
26
Role of WP7
•2.4.3. Providing Common Spatial Data Models …
•2.4.3. …. and Allowing an Efficient Transition to Harmonised
Data (WP5/WP7)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
27
2.4.3. …. and Allowing an Efficient Transition to Harmonised Data (WP5/WP7)• ESDI: transition to common / “shared” data models
• Without data loss
• With minimum costs
• Humboldt objective: provide tools and best practices
Procedures / workflow how to create common models and organization-specific
extensions
Software components for mapping of legacy data models to the harmonised data model
…
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
28
“Mapping” between data models: role of ontology tools and languages
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
29One method: ‘create view’ (SQL)
CREATE VIEW CultHistObject ASSELECT concat('archis.',id#) AS fid, decode(WAARDE, 'archeologische betekenis', 'not assessed' ... 'hoge archeologische waarde', 'high' 'zeer hoge archeologische waarde', 'very high' 'zeer hoge archeologische waarde, beschermd', 'very high' 'unknown') AS cultHistValue, decode(WAARDE, 'archeologische betekenis', 'not assessed yet' 'zeer hoge archeologische waarde, beschermd', 'monument' 'none') AS juridStatus, to_char(null) AS partOfCultHistObj#, ...FROM archisUNIONSELECT concat('monreg.',id#) AS fid, to_char(null) AS cultHistValue, 'national monument' AS juridStatus, concat('monreg.',Complex-id#) AS partOfCultHistObj#, ...FROM monregUNION ...
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
30
Another language: OWL<owl:Class rdf:about="MO_SUBJECT_MF"> <owl:equivalentClass rdf:resource="ccdm.owl#NaturalPerson"/></owl:Class>
<owl:Class rdf:ID="MO_SUBJECT_MF"> <owl:intersectionOf rdf:parseType="Collection"> <owl:Class rdf:about="#MO_SUBJECT"/> <owl:Restriction> <owl:onProperty> <owl:DatatypeProperty rdf:about="#MO_SUBJECT_NAT_PERS_CODE"/> </owl:onProperty> <owl:allValuesFrom> <owl:Class> <owl:unionOf rdf:parseType="Collection"> <owl:Class rdf:about="#M"/> <owl:Class rdf:about="#V"/> </owl:unionOf> ...
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
31
Tools, techniques, standards, …
• Tools, software components
Creation + maintenance of data models and relations between data models
• Languages to describe data models
UML (and OCL)
OWL (Web Ontology Language)
XML Schema (already towards physical implementation)
• Feature catalogues, metadata
ISO 19110 (Methodology for feature cataloguing)
ISO 19115 (Metadata)
ISO 19139 (Geographic Metadata Implementation Model)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
32
Role of WP7
•2.4.3. Providing Common Spatial Data Models …
•2.4.3. …. and Allowing an Efficient Transition to Harmonised
Data (WP5/WP7)
•2.4.8. Crossing technical, administrative, cultural and other
borders (WP7/WP11)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
33
Role of WP7
•2.4.3. Providing Common Spatial Data Models …
•2.4.3. …. and Allowing an Efficient Transition to Harmonised
Data (WP5/WP7)
•2.4.8. Crossing technical, administrative, cultural and other
borders (WP7/WP11)
•2.4.4. Maximizing Spatial Data Usefulness (WP5/WP8)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
34
2.4.4. Maximizing Spatial Data Usefulness (WP5/WP8)•Re-use of geo-information outside original context
•Efficient in terms of data collection
•Also effective in terms of user needs?
•How to clarify meaning of geo-information to outsiders
•Necessary: rich metadata, explicit semantics, intelligent search
engines
•-> fitness for use assessment by end-users in other contexts
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
35
WP7: input for WP9 Scenarios
•Application-specific data models (scenario profiles)
•Tools for “mapping” from source model to target model
From existing data sets to application-specific profile
Between application-specific profiles (re-use in another application domain)
•Best practices, techniques
•Test data (harmonized and not yet harmonized)
•Multilingual aspect
Which WP9 scenarios as test case?
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
36
.. and input from WP9 Scenarios
• Implementability of application-specific profiles
• Experiences in scenarios may result in
Adaptation of profiles
Other harmonisation strategies
• Flexibility, maintainabiltiy of the profiles
• EASY TRANSITION, USABLE ??
• (remember the motto yesterday): the day-to-day work of end-users is
central focal point
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
37
Scenario applications
• 1. Border Security (cross-border)
• 2. Urban Planning
• 3. Forest (cross-border, number of central European countries)
• 4. Protected Areas (follow-up of Nature-GIS)
• 5. European Risk Atlas (cross-border natural disaster information)
• 6. Water (extension of SDIGER project)
• 7. Ocean (oil pollution, integration of ocean and land data)
• 8. Galileo (atmospheric information, mobile devices, sensors)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
38
Data (model) harmonisation
•Several approaches to data integration and model harmonisation
•Harmonised core data models with organisation-specific
extensions
•… semantic translation between models
•… peer-to-peer data integration (c.f. OpenKnowledge EU project)
•… with mediator software, e.g. ontology reasoners?
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
39
How to implement harmonized, common models?•Again different approaches
•Adaptation of the actual data storage, OR
•On-the-fly, virtual integration:
by mediator software in the middle layer?
by database views?
in an intelligent client?
•Different requirements -> different architectures
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
40
Requirements -> technologyType of use Quality of result Mapping to
common model
How and where
Exploratory,
ad hoc
Best guess, fuzzy - -
Projects or
structural
cooperation
Medium or high On-the-fly, during
data retrieval and
use
By application
software (selection
filters, mediators,
ontology reasoners)
Time-critical
applications
High Pre-defined At data source (views
on tables, automated
replication)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
41
Query translation (via core model)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
42
Multi-source ‘fusion’ -> integrated view Integrated View
Data -> Maps
Topography
Transport
Elevation
Hydro
Soils, Vegetation
Weather
RiversRoadsImagesSoils, Vegetation ...
CatalogView
Framework(Clearinghouse)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
43
What can go wrong• General
Scale: high precision combined with low precision
Dimension: when 3D models are ‘flattened’ to 2D
Time: versions, up-to-dateness, different moments in time
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
44
What can go wrong #2• Cross-border
Projection, coordinate system
Cartographic symbols and classification
• Cross-domain/sector
Semantic ambiguity, unclear content, different definitions
Re-use of data for another purpose (for what it was not intended)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
45
Data integration on-the-fly: what can go wrong?• Combining ‘live’ (sensor) data with existing ‘static’ data:
GPS positional accuracy (when tracking persons, vehicles, other moving objects)
Inconsistent data formats, projection systems
• General:
Performance: too slow?
System robustness + security
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
46
Interoperability issues
• Combination of software components of many vendors
• No monopoly of one AEC or GIS or CAD or geo-database product, but …
• Combine the best products (for editing, for viewing, for analysis, for
management and storage)
• Government agencies (in Europe) ask for ‘open’ standards (‘open’ = not
vendor specific)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
47
Needed for re-use and sharing of data• Catalog = to know what data is available
• Intermediary data transfer formats
• Standardization of interfaces between distributed software components =
(Web) services
• Common information models
• Agreement on access policy (including security, authorization, privacy)
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
48
Towards an (E)SDI
• Discover what usefull spatial data is available
• Agree on access policy and security between organizations
• Create system architecture for accessing this distributed data
• Combine with ‘live’ data from the field
• Test functionality, adapt in iterations
• Keep data up-2-date, continuous process
• Extend, be flexible, …
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
49
Example
<owl:Class rdf:ID="Church"> <owl:intersectionOf rdf:parseType="Collection"> <owl:Class rdf:about="#Building"/> <owl:Restriction> <owl:onProperty rdf:resource="#hasFunction" /> <owl:allValuesFrom rdf:resource="#Religion" /> </owl:Restriction> </owl:intersectionOf> </owl:Class>
<owl:Class rdf:ID="Cathedral"> <rdfs:subClassOf rdf:resource="#Church"/> </owl:Class>
© 2006 Humboldt Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.org
50
Example 2 <owl:Class rdf:ID="ArcheologicalMonument"> <rdfs:subClassOf rdf:resource="#CultHistObject"/> </owl:Class>
<owl:Class rdf:ID="ProtectedArcheologicalMonument"> <owl:intersectionOf rdf:parseType="Collection"> <owl:Class rdf:about="#ArcheologicalMonument"/> <owl:Restriction> <owl:onProperty rdf:resource="#legalStatus"/> <owl:allValuesFrom rdf:resource="#officialMonument" /> </owl:Restriction> </owl:intersectionOf> </owl:Class>