Grid-enabling OGC Web Services

Andrew Woolf, Arif ShaonSTFC e-Science CentreRutherford Appleton Lab

Outline

• OGC background• Profiling approach• Refactoring approach• Grid-enabling WPS

OGC background

• Key architectural frameworks– Reference Model for Open Distributed

Processing (RM-ODP, ISO 10746-{1,2,3,4})

– ISO TC211 Domain Reference Model (ISO 19101)

• Enterprise viewpoint– roles, activities, policies (incl. VO)

• Information viewpoint– semantics of information and information

processing (static, invariant, dynamic schema)

• Computational viewpoint– interfaces and computational objects (cf. CORBA

IDL, WSDL portTypes)

• Engineering viewpoint– distribution infrastructure (e.g. web services,

WSRF vs OGSI)

• Technology viewpoint– choices of technology (e.g. app servers, DBMS)

RM-ODP

ISO 19101 ‘Domain Reference Model’

A geospatial dataset…

…consists of features and related objects…

…in a defined logical

structure…

…delivered through

services…

…and described by metadata.

servicesservices

SOA

WSRF

OGCWSDL

metadatametadata

resource discovery

catalogue

datadataaccess

integration

information

XML

registries

‘Spatial Data Infrastructure’

Summarymetadata data services

Enterprise •discovery •download, view •process

Information •ISO 19115•ISO 19139

•OGC Geography Markup Language (ISO 19136)

•OWS Common

Computational

•OGC Catalogue Service for the Web (CSW)

•OGC Web Feature Service (ISO 19142)•OGC Web Coverage Service•OGC Web Map Service (ISO 19128)

•OGC Web Processing Service (WPS)

Engineering

Technology

Grid-enabling OGC: profiling

• Use existing specifications• Incorporate ‘Grid’ functionality

through convention, e.g.– specific parameters for security

tokens

• Implementation uses Grid middleware at backend

Grid-enabling OGC: refactoring

• Problems:– tight coupling: service ↔ data

• data providers have to be service providers

– latency• general approach for asynchronous

– service coherence• data → (view, download, process)

– workflow abstraction• abstraction: data, operations

– service consistency• data resource: WPS vs. W{M,F,C}S

Grid-enabling OGC: refactoring

• WSRF:– separates ‘service’ and ‘stateful

resource’ upon which service acts– WS-Resource (‘implied resource pattern’):

• identity (WS-Addressing)• lifetime (WS-ResourceLifetime)• state/properties (WS-ResourceProperties)

– publish-subscribe mechanism for state changes (WS-Notification)

Grid-enabling OGC: refactoring

Existing OGC WSRF-refactored

Grid-enabling WPS• WPS provides standardised means of

publishing, discovery and use of geospatial processes on the web

• Fundamentally, both WPS and Grid provide a remote interface for invoking computational processes

Grid-enabling WPS• JSDL provides a standardised description of a

computational job and its required resources• Similarities with WPS:

– process description information (Identifier, JobIdentification/JobName)

– process outputs and inputs (DataInputs, POSIXApplication/Input,Argument,Output, DataStaging/Source)

– sufficient to produce a valid JSDL document

• Differences with WPS:– resource requirements (filesystem parameters, disk space,

operating system, CPU, etc.)– data staging instructions for input and output

Grid-enabling WPS• WPS+JSDL as interface to encapsulated Grid

resource

• JSDL consumed by compliant implementation, e.g. GridSAM

• Two broad options:– conceptually distinct data input parameters (through a

specific ‘JSDL’ parameter)– “ordinary” data input parameters

Grid-enabling WPS• Three approaches to distinct JSDL data input parameters:

– URL to a complete pre-created JSDL document, or valid snippethttp://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;JSDL=http://www.foo.com/myfoo.jsdl@Format=text/xml&storeExecuteResponse=true&status=true

– URL-encoded JSDL document or snippethttp://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;JSDL=%3CJobDefinition%3E%3CJobDescription%3E%3CResources%3E%3CTotalDiskSpace%3E%3CExact%3E5%3C%2FExact%3E%3C%2FTotalDiskSpace%3E%3CTotalCPUCount%3E%3CExact%3E1%3C%2FExact%3E%3C%2FTotalCPUCount%3E%3C%2FResources%3E%3C%2FJobDescription%3E%3C%2FJobDefinition%3E@Format=text/xml@Schema=http://server/jsdl.xsd&storeExecuteResponse=true&status=true

– Microformathttp://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;JSDL=[TotalDiskSpace=5;TotalCPUCount=1]@Format=text/kvp&storeExecuteResponse=true&status=true

Grid-enabling WPS

• JSDL parameters encoded as individual WPS DataInput parameters

http://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGenerator&DataInput=other_inputs=xxx;TotalDiskSpace=5;TotalCPUCount=1&storeExecuteResponse=true&status=true

• BUT....– JSDL parameters are not inputs to the WPS process,

but rather specify required resources

Grid-enabling WPS

• Security– OGC security is subject of

considerable ongoing work– for now, used lightweight (and

dodgey) solution:• MyProxy parameters embedded in

request• secure transport layer

Grid-enabling WPS

• Implementation:– UK National Grid Service– BADC Trajectory Service– part of OWS-6 testbed– WPS 1.0.0-compliant

Grid-enabling WPS

Grid-enabled

WPS

Pylons Framework

GridSAM Client

UK NGS

GridSAM Service

Grid

WPS SOAP/Proxy Layer

MyProxy Server

Execute Process (SOAP)

Verify Cert.

Job Id

Query/Response

Response

WPS Response

Grid-enabling WPS

Conclusions• OGC adopts a sophisticated architectural

approach to distributed geospatial data/processing– NB: OGC-OGF MoU, OGF-28 (Munich Mar.

2010)

• Grid-enabling OGC:– ‘Profiling’ approach– ‘Refactoring’ approach

• Example: grid-enabled WPS profile

More info...

• OWS-6 WPS Grid Processing Profile Engineering Report– http://portal.opengeospatial.org/files/?artifact_id=3497

7

• OWS-6 interactive demonstration– http://www.opengeospatial.org/pub/www/ows6/index.h

tml

• WPS Change Request for controlling & checking asynch processes– https://portal.opengeospatial.org/files/?artifact_id=367

32