Post on 11-Jan-2016
Lesley Wyborn and Stuart Girvan
Geoscience Australia, pmd*CRC
ACHIEVING PRACTICAL CROSS-JURISDICTIONAL INTEROPERABILITY
FOR THE AUSTRALIAN MINING INDUSTRY
Australian Government
Geoscience Australia
PPDM Australian Data Management Conference 2004 Geoscience Australia
Outline of Presentation• This presentation is about the Australian AUSindustry
‘Interoperability for Geospatial Data Project’
• Our presentation will be based on the OpenGIS® Reference Model
• We will discuss: o what were the drivers for this project
o which pieces of the standard service orientated architecture were developed
o whether other communities and domains can use our results
• We will elaborate on what were the most important lessons learnt
PPDM Australian Data Management Conference 2004 Geoscience Australia
Interoperability for Geospatial Data Project:AcknowledgementsPeople
o Simon Cox, Rob Woodcock, Joan Esterle – CSIROo Stuart Girvan – Predictive Mineral Discovery CRCo Tim Mackey, Aaron Sedgmen – Geoscience Australiao Rob Atkinson, Peter Barrs – Social Change Online
Funding Organisations (~$250K cash)o AUSIndustryo pmd*CRCo Minerals Council of Australiao Geoscience Australiao Every State and Territory Geoscience Agency
PPDM Australian Data Management Conference 2004 Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
Engineering Viewpoint
Information Viewpoint
Computational Viewpoint
Technology Viewpoint
Business drivers, policies Who else is doing it?
What technology do we use?
Whatlanguage are wespeaking?
What components & Interfaces?
Source http://www.opengeospatial.org/specs
Who does what?
PPDM Australian Data Management Conference 2004 Geoscience Australia
Key Driver: Minerals Exploration Action Agenda –
June 2004Industry input highlighted• problems in gaining access to pre-competitive geoscience
information.
• described existing information as commonly incomplete and
fragmented across eight government agencies, each with its own
information management systems and structures.
• noted that the disparate systems lead to inefficiencies causing
higher costs, reduced effectiveness and increased risk incurred by
the industry and its service providers
PPDM Australian Data Management Conference 2004 Geoscience Australia
Key Driver: Minerals Exploration Action Agenda –
June 2004Key Initiative
o Australian Government, State and Territory geoscience
agencies, professional associations and industry to
cooperatively develop and implement nation-wide protocols,
standards and systems that provide internet-based access to,
and effective storage and archiving of, industry and government
exploration-related DATA
Recommended actionso Implement web-based services for on-line access
o Develop and endorse a plan for implementation of an Australian
Earth Science Grid
Client has to download
and thenlocallymergeeach fileinto
single data set
The Geoscience Portal is an index of websites
To merge common data types from all surveys either 1. All data sets are downloaded from each source and then combined or
2. All data are combined into a single data set and hosted by one source: currency of data is a major issue
NSWdatabases
Victorian Databases
Queensland Databases
Tasmanian databases
NTDatabases
GADatabases
WADatabases
SADatabases
PPDM Australian Data Management Conference 2004 Geoscience Australia
Global Driver: the exponentially increasing digital world
Therefore:• No long do data,
programs and computer grunt have to be local
• We need to switch to a distributed culture, ie an interoperable culture where data, programs and computer grunt are ‘operable’ regardless of where they are housed
Moore’s Law vs. storage improvements vs. optical improvements. Graph from Scientific American (Jan-2001) by Cleo Vilett, source Vined Khoslan, Kleiner, Caufield and Perkins.
PPDM Australian Data Management Conference 2004 Geoscience Australia
Evolution of the Internet
Source: http://www.dstc.edu.au/Tech_Transfer/Events/Canberra/web_services_cnb02.pdf
People
People
People
Content
Application
Application
PPDM Australian Data Management Conference 2004 Geoscience Australia
Multiple
overlaid maps
One GetMap request:
Web Map Service (WMS) can access multiple maps
Borders
Elevation
Cloud cover
Cities
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004 Geoscience Australia
WMS GetMap returns a server’s “dumb” JPEG, GIF or PNG representation of the data on the server. It does NOT return the actual data, only a bitmap of the data.
WMS can’t “give data away.”
Roma
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004 Geoscience Australia
Web FeatureServer
GetFeaturerequest:
Feature &attributedata
Web Feature Service (WFS) returns data.
I-95
I-295
I-87
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004 Geoscience Australia
Multiplethematic
data layers
GetFeaturerequest:
Web Feature Service (WFS) gets operable feature data from multiple servers
Cities
BordersElevation
Each layer is data, not merely a view:Country is:• Name: Italy• Population: 57,500,000• Area: 301,325 sq km. . .
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004 Geoscience Australia
Why seek funding from the AUSIndustry Innovation Access
Program (IAccP)• AUSIndustry is part of the Department of
Industry, Tourism and Resources
• The goal of the IAccP is to promote innovation and competitiveness by improving Australian access to global, leading-edge research and technologies and facilitate their uptake by Australian firms, particularly by SME’s and researchers.
• The IAccP is a technology diffusion program
PPDM Australian Data Management Conference 2004 Geoscience Australia
External links – who else is doing it?
This project provides a mechanism to understand and share practice with other domainsAustralian:
o ASIBA Spatial Interoperability Demonstration Project (SIDP)o Australian Spatial Data Infrastructure (ASDI)o Australian Earth and Ocean Network (AEON)o Australia Partnership for Advanced Computing (APAC)o National Oceans Office (NOO)o Australian Antarctic Division (AAD)
Internationalo GEON (Geoscience Network) - USo NERC Datagrid (UK National Environmental Research Council)o IUGS CGI (International Union of Geological Sciences)
PPDM Australian Data Management Conference 2004 Geoscience Australia
Policy Issues for our Project
• Pricing and Chargingo For both Commonwealth, State and Territory
Geoscience Agencies, all data are free over the interneto Thus we did not have to build charging into our project
• Managing Intellectual Propertyo Easy to put data source and request for client to
acknowledge into data set o Acknowledgement of the data source
at the destination is difficult to enforce
• Copyright and licensingo Source Agency still maintains copyrighto They still need to know who is using ito How this is done is a global issueo Geoscience Australia is participating in the OGC project
on Digital Rights Management (DRM)
PPDM Australian Data Management Conference 2004 Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
Engineering Viewpoint
Information Viewpoint
Computational Viewpoint
Technology Viewpoint
Who does what?
Business drivers, policies Who else is doing it?
What technology do we use?
Whatlanguage are wespeaking?
What components & Interfaces?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004 Geoscience Australia
Computational Viewpoint: Standard Web Services
Architecture
Services
Registry
ClientApplications
publish
bind
find
Data Access Services
ModelManagement
ServicesProcessing
Services
Computational Viewpoint: Notional Architecture
Services
Data Access Services
Registry
ClientApplications
ModelManagement
Services
FeaturesCoverages Models
Real-timedata
publish
bind
find
OntologyService
Metadata
PersistentBindings
FeatureType
Catalog
DatasetMetadata
ProcessingServices
PPDM Australian Data Management Conference 2004 Geoscience Australia
Computational Viewpoint: our simplified architecture of
components and interfaces
Services
Registry
ClientApplications
publish
bind
find
Data Access Services
For our demonstrator we are1. not using registries to find or publish services2. focusing on delivering data as operable ‘features’3. hard wiring the applications to the data services
ModelManagement
Services
ProcessingServices
PPDM Australian Data Management Conference 2004 Geoscience Australia
Computational Viewpoint: registries & catalogues
Services
Registry
ClientApplications
publish
bind
find
Data Access Services
NOO, QLD NRE, WA DLI, SEEGRID These people are developing/thinking about developing registries and catalogues in Australia
PPDM Australian Data Management Conference 2004 Geoscience Australia
Services
WMS
Coord.Transf.Service
CoveragePortrayalService
Data Services
Portrayal ServicesProcessing Services
ArchivesArchives
WFSWFS WCSWCS SCS Geocoding Chaining
Gazetteer
This Project/ SEEGrid
ASIBA CANRI WA DLI
QLD NRE GA etc
WA DLI GA
Data Access Services
Who else is doing what in OZ in the Computational Viewpoint
PortrayalServices Processing
Services
PPDM Australian Data Management Conference 2004 Geoscience Australia
Our Computational Viewpoint of Components and Interfaces
ServicesClient
Applications
bind Data Access Services
This project is focusing on Data Access Services
and on binding these to client applications
PPDM Australian Data Management Conference 2004 Geoscience Australia
Main International Standards used in Project
• OGC Services o Web Feature Service (WFS)o Web Map Service (WMS)
• OGC Languages o Geography Markup Language (GML)o eXploration and Mining Markup Language (XMML)
• W3C Languageso eXtensible Markup Language (XML)
• W3C Web serviceso Simple Object Access Protocol (SOAP)
PPDM Australian Data Management Conference 2004 Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
Engineering Viewpoint
Information Viewpoint
Computational Viewpoint
Technology Viewpoint
Business drivers, policies Who else is doing it?
What technology do we use?
Whatlanguage are wespeaking?
What components & Interfaces?
Source http://www.opengeospatial.org/specs
Who does what?
PPDM Australian Data Management Conference 2004 Geoscience Australia
Who is doing what in Australia in language standards to enable
data binding
Encodings
GML XIMA SLD
ServiceCapabilities
SensorML Obs&Meas
Image Metadata
CSIRO SEEGRID OGC
ServicesClient
Applicationsbind
Data Access Services
PPDM Australian Data Management Conference 2004 Geoscience Australia
Traditional GIS data models for Geosceince do not allow the same data to be viewed seamlessly in
2D, 3D, 4D• Data are in points, lines and polygons
• Geometry-centric abstraction relates to the implementation, not the business object
• One shape per feature does not allow multiple spatial properties,
• scale-dependent versions …
Tenement
PPDM Australian Data Management Conference 2004 Geoscience Australia
Information Viewpoint in the OZ Geosciences
Borehole collar location shape collar diameter length operator logs related observations …
Fault shape surface trace displacement age …
Ore-body commodity deposit type host formation shape (point, polygon, 3D
shell) resource estimate …
Observation location subject/specimen/station property/theme/measurand method operator date/time result (+ type/reference
system/scale/classification) …
• We use ‘feature’ models which:o are based on geological conceptso allow for multiple geometries in
2D, 3D, and 4D
Basin formations shape – time
dependent resource estimate …
PPDM Australian Data Management Conference 2004 Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
Engineering Viewpoint
Information Viewpoint
Computational Viewpoint
Technology Viewpoint
Who does what?
Business drivers, policies Who else is doing it?
What technology do we use?
Whatlanguage are wespeaking?
What components & Interfaces?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004 Geoscience Australia
SEEGrid Community Engineering Viewpoint
Our Project’s Focus
SEEGrid is helping to coordinate the building of an interconnected distributed modelling and computational services, whereby the models as well as the input data and modelling programs are preserved in a model library
PPDM Australian Data Management Conference 2004 Geoscience Australia
Our project’s Engineering Viewpoint
ServicesClient
Applications
bind Data Access Services
PPDM Australian Data Management Conference 2004 Geoscience Australia
Bind
Our projects Engineering Viewpoint in Detail
(who engineered what?)
WA GeochemistryFeature
Data Source
SAWeb Feature Service (WFS)
ClientApplications
GA GeochemistryFeature
Data Source
SA GeochemistryFeature
Data Source
WAWeb Feature Service (WFS)
GAWeb Feature Service (WFS)
South Australia
Western Australia
Gescience Australia
pmd*CRC, GA, CSIRO, Social Change On-line
PPDM Australian Data Management Conference 2004 Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
Engineering Viewpoint
Information Viewpoint
Computational Viewpoint
Technology Viewpoint
Business drivers, policies Who else is doing it?
What technology do we use?
Whatlanguage are wespeaking?
What components & Interfaces?
Source http://www.opengeospatial.org/specs
Who does what?
PPDM Australian Data Management Conference 2004 Geoscience Australia
Our Project’s Technology Viewpoint:
what technology do we use?
ServicesClientApplications
bind Data Access Services
PPDM Australian Data Management Conference 2004 Geoscience Australia
Bind
XMML, GML
Our projects Technology Viewpoint in Detail
ClientApplications
Geoserver (Open Source) PostGIS (OpenSource)
Web Map Composer(Proprietary)
GA ReportingApplication
GA PLOT-ITApplication
South Australia
Western Australia
Geoscience Australia
WAWeb Feature Service (WFS)
SAWeb Feature Service (WFS)
GAWeb Feature Service (WFS)
GA GeochemistryFeature
Data Source
Oracle (Proprietary)
PostGIS (Open Source)
SA GeochemistryFeature
Data Source
WA GeochemistryFeature
Data Source
PPDM Australian Data Management Conference 2004 Geoscience Australia
Our Technology for distributing data
• We used Geoserver – open source
• Needed modification to be able too Source data from complex database (many tables of data) o Produce complex application schemas (ie more complex
than simple GML such as XMML)
• Other WFS software*:o Degree (open source), Cadcorp SIS Map Server, CARIS
Spatial Fusion Enterprise, RedSpider Web 3.0, Map Manager, GenaWare, SclFeatureServer, JCarnacGIS, GeoMedia WebMap, MapXtreme (MapInfo), MapServer(UMN) 4.2 + more
o We do not know if any of these can do complex mapping
* Source: OGC Registered Products
PPDM Australian Data Management Conference 2004 Geoscience Australia
Clienthas todown load andthenlocallymergeeachfileinto asingledata set
The Geoscience Portal is an index of websites
To merge common data types from all surveys either 1. All data sets are downloaded from each source and then combined or
2. All data are combined into a single data set and hosted by one source: currency of data is a major issue
NSWdatabases
Victorian Databases
Queensland Databases
Tasmania databases
NTDatabases
GADatabases
WADatabases
SADatabases
GADatabases
WADatabases
SADatabases
Many other data types besides geochem!!!
Project results: the Geoscience Portal will now have these wfs
components to enable• access of SA, WA, GA geochem data in real time via common interfaces• each participant to develop their database to suit their business needs
Chem
Chem
ChemWA
GeochemInterface
SAGeochemInterface
GAGeochemInterface
NSWdatabases
Victorian Databases
Queensland Databases
Tasmania databases
Northern TerritoryDatabases
wfs interfaces still needed for these Surveys
PPDM Australian Data Management Conference 2004 Geoscience Australia
Information Viewpoint requires the most work
Enterprise Viewpoint
Engineering Viewpoint
Information Viewpoint
Computational Viewpoint
Technology Viewpoint
Business drivers, policies Who else is doing it?
What technology do we use?
Whatlanguage are wespeaking?
What components & Interfaces?
Source http://www.opengeospatial.org/specs
Who does what?
PPDM Australian Data Management Conference 2004 Geoscience Australia
XMML schema’s developed by CSIRO
• Borehole• Observation (OGC SensorWeb) • Gravity measurement• Geochemistry/Assay result • Geological material • Geological timescale • Mineral occurrence • Procedure, Project, Station, Specimen, Tenement• Point, Curve, Surface, Solid volume with properties• Structural geology• Time-series • Finite element model (FLAC, FastFlo)• Simulation/model state
implemented but only for WA, SA and GA
PPDM Australian Data Management Conference 2004 Geoscience Australia
Who is the XMML Community?
• The XMML community led by CSIRO is one of the most advanced globally – why?
• The XMML community has been going since 2000• Data types within each domain have different
communities • For XMML
o For locations community is global – this led to GML3o For Geochemistry metadata, community is the Australian
Mining industry• This illustrates the complexity of developing
community schema• There is no national or international coordinating
agency
Introducing the Solid Earth and
Environmental Grid
(SEEGrid)
Community languages
http:www.seegrid.csiro.au
PPDM Australian Data Management Conference 2004 Geoscience Australia
Community pages
PPDM Australian Data Management Conference 2004 Geoscience Australia
Lesson’s learnt: Business Issues
• Interoperability represents perhaps the most significant paradigm shift in how data and information are managed and utilised since the emergence of the Internet
• The Interoperable world is a Markedly Different Business Paradigm: change is from ‘supply’ to ‘demand’o In the supply or ‘push’ paradigm, suppliers push their content to
clients, how they think their clients want to utilise that contento In the demand or user ‘pull’ paradigm clients access and use
what data and information they require, from any data supplier that can supply authoritative data in the appropriate standardised formats.
• For the Interoperable paradigm, data suppliers must be able to map their content word for word to international standard interfaces.
Source: Williams, Neil, 2004. Interoperability – responding to National Drivers. http://www.osdm.gov.au/osdm/docs/resources/osdm_interoperability_con_03112004/neilwilliams1.pdf
PPDM Australian Data Management Conference 2004 Geoscience Australia
Lesson’s learnt: Impacts on Business
• Business processo Workflow and versioning control are criticalo Archiving dynamically changing data is an issue
• As clients get very large data sets, statistically valid data mining is a realityo Much more detailed metadata will be required for
machine to machine transfer (SensorML in particular)o Errors are more likely to be picked upo Need for, and administration of, organisation-wide
QA/QC will be a major impact
PPDM Australian Data Management Conference 2004 Geoscience Australia
Key Take Home Messages• Cross jurisdictional interoperability has been achieved in the
geoscience community (almost) by standards based interfaces
• The final project release will be in March at the SEEGrid 2 workshop in Canberra – please come
• The greatest amount of work still needs to be done on the XML data models (application schemas) used in any data system within and between communities
• The ability of technology and specifications to deal with data models that realistically reflect the complexities of real world geoscience from the technical perspective
• Effective global governance for dividing the complexities of the real world into manageable pieces for effective data modelling is required and will help mitigate the above
• Track our progress (or better still join us) on http://www.seegrid.csiro.auhttp://www.pmdcrc.com.au/http://geoserver.sourceforge.net