Generalization Web ServicesMoritz Neun

University of Zurich

Motivation

Generalization Web Services – Moritz Neun – 19. November 2007

3

Motivation 1

Much progress of web cartography through OGC standards. WMS and WFS allow automated access and presentation of cartographic data Pre-calculation and usage of MRDB can support these services only partially

New requirements on delivering and generating on-demand and on-the-fly maps, containing more specific and tailor-made information

OGC (2002) has proposed Feature Generalisation Services, but no further developments yet

Generalization Web Services – Moritz Neun – 19. November 2007

4

Motivation 2

A large number of generalisation algorithms, auxiliary data structures, cartographic constraints and measures is developed and published

Isolated prototypes implemented with different programming languages

Integrated within closed monolithic systems(Clarity, ArcGIS, Genesys, ….)

Generalization Web Services – Moritz Neun – 19. November 2007

5

Motivation 2

Consequences: Benchmarks and comparison of algorithms difficult No reuse of available generalisation functionality

and support data structures - researchers have to start from scratch

Following that, rarely addressed “advanced questions” on the generalisation process such as combination of several operators, orchestration, …

Interoperable (research) platform for sharing of (generalization) functionalities needed

History

Generalization Web Services – Moritz Neun – 19. November 2007

7

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004),

Sarjakoski et. al (2005)Burghardt et. al (2005),

Neun et. al (2006, 2007)Regnauld (2006, 2007)Harrower and Bloch (2006)Lemmens et al. (2006),

Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

•XSLT techniques are combined with Java programming for real-time generalisation

•Examples show two results of different XSLT processes with different generalisation functionality (selection and simplification)

Generalization Web Services – Moritz Neun – 19. November 2007

8

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2006, 2007)Regnauld (2006, 2007)Harrower and Bloch (2006)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• Need for a common research platform in the domain of map generalisation was expressed

• Requirements for an open generalisation system• Related OGC developments were presented

(WMS, SLD, SVG, WFS, GML, FES, SOAP, WSDL, …)

Generalization Web Services – Moritz Neun – 19. November 2007

9

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2006, 2007)Regnauld (2006, 2007)Harrower and Bloch (2006)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• OXYGENE platform developed at the COGIT laboratory of IGN

• Based on Java technology and different open source components

• Open source release in 2005

Generalization Web Services – Moritz Neun – 19. November 2007

10

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2006, 2007)Regnauld (2006, 2007)Harrower and Bloch (2006)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• GiMoDig services based on layer architecture • Layers encapsulating data integration, data

transformation into GML, data processing (generalisation) and device dependent styling (portal layer)

• Further development of the approach from Lehto and Kilpeläinen with XSLT processing

Generalization Web Services – Moritz Neun – 19. November 2007

11

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2005, 2007)Regnauld (2006, 2007)Harrower and Bloch (2006)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• WebGen platform • Accessible from different clients

(Web Browser, JUMP, …)• DEMO

Generalization Web Services – Moritz Neun – 19. November 2007

12

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2005, 2007)Harrower and Bloch (2006)Regnauld (2006, 2007)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• MapShaper for browser based generalisation• Aim on interactive user controlled generalisation, no platform-independent

coupling of generalisation services• Strong relation of user interface and generalisation functionality

(limited flexibility - new generalisation functions requires new client versions)

Generalization Web Services – Moritz Neun – 19. November 2007

13

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2005, 2007)Harrower and Bloch (2006)Regnauld (2006, 2007)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• Concept of system architecture for on-demand derivation systems• Usage of Geo-Ontologies to formalise input data, user requirements,

cartographic knowledge and service descriptions

Generalization Web Services – Moritz Neun – 19. November 2007

14

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2005, 2007)Harrower and Bloch (2006)Regnauld (2006, 2007)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

• Implementation based on OGC Web Processing Services

• Usage of profiles to describe syntax and the semantics of the operation

Generalization Web Services – Moritz Neun – 19. November 2007

15

History

Lehto and Kilpeläinen (2000, 2001)Edwardes et. al (2003)Badard and Braun (2003)Harrie and Johansson (2003)Sester et. al (2004), Sarjakoski et. al (2005)Burghardt et. al (2005), Neun et. al (2005, 2007)Harrower and Bloch (2006)Regnauld (2006, 2007)Lemmens et al. (2006), Forster and Stoter (2006, 2007)

Open architectures based on distributed platforms have attracted significant interest in the generalisation community

Generalization Web Services – Moritz Neun – 19. November 2007

16

History

M.F. Goodchild (2005). GIS and modeling overview. In D.J. Maguire, M. Batty, and M.F. Goodchild, editors, GIS, Spatial Analysis, and Modeling. Redlands, CA: ESRI Press, pp. 1–18.

“There is also increasing interest in providing basic GIS services, such as geocoding, as remotely invokable methods implemented on the Web. In the next few years, dramatic improvements are expected in the availability of techniques for sharing methods and models.”

Web Services

Generalization Web Services – Moritz Neun – 19. November 2007

18

Web Service Technology

Generalization Services are Web Services: Client-Server structure Component architecture:

- encapsulation of functionalities and resources - generic interfaces (interface description) - loosely coupled by contract Platform independence: usage of standard web protocols (HTTP and XML) Interoperable Remote Procedure Calls (RPC)

(remote processing instead of data delivery)

Web Services are enabling interoperability

Generalization Web Services – Moritz Neun – 19. November 2007

19

Middleware Generalization ServicesTypical on-demand or on-the-fly generalization service for web map services: Used in combination with data services (e.g. WFS) Translator or compiler for converting and

combining datasets in order to be displayed Usually bound to one or multiple

specific and predefined data delivery services

Generalization Web Services – Moritz Neun – 19. November 2007

20

Toolbox Generalization Services

Various processing services provide their functionalities as distributed toolbox Data to be processed is provided by the user Everybody can present own services Platform

independentservice access

Coupling of different generalizationsystems

research platform WebGen framework

State-of-the-Art Toolbox Services

Generalization Web Services – Moritz Neun – 19. November 2007

22

State-of-the-Art

• WebGen research platform• Web Processing Services

• ArcGIS Server (ESRI)• SerAx (Axes Systems)

Toolbox-like service approachesfor remote processing

open commercial

Generalization Web Services – Moritz Neun – 19. November 2007

23

WebGen

Our prototype of a generalisation toolbox service framework Different plug-ins Generic interface

descriptions Registry for

service discovery Servers for

hosting differentalgorithms

Generalization Web Services – Moritz Neun – 19. November 2007

24

WPS

WPS Concepts from Foerster (2006)

Web Processing Service (OGC Draft)

Intended for many purposes not only generalisation

Raster and vector data Very open (and vague)

definitions of interface descriptions and data formats

Currently rather aproposal than areal standard

More concrete profileneeded for generalization

Generalization Web Services – Moritz Neun – 19. November 2007

25

WPS

Foerster (2006) shows WPS implementation for generalisation(partially adopted from WebGen)

Intended for working together e.g. with a web map server like geoserver but not as middleware (geoserver calls the WPS)

Provides ready generalised maps to the requesting client

Generalization Web Services – Moritz Neun – 19. November 2007

26

axpand

Generalsation functionsas internal services

Adopts WebGen concepts(registry & interface descriptions)

possible use of external services

Management of service workflows

Data is not sent in the service calls (remains in central database, only references are passed)

Uses service concept but rather closed and proprietary

Generalization Web Services – Moritz Neun – 19. November 2007

27

ArcGIS Geoprocessing Services

Geoprocessing services for the ESRI ArcGIS Server

All types of geoprocessing tools can be served

Works only with ArcGIS products and algorithms

Rather closed and proprietary commercial system

The WebGen Framework

Generalization Web Services – Moritz Neun – 19. November 2007

29

The WebGen Framework

Implementation of the Toolbox Services Model: Various different distributed servers can provide

generalization services Generic interface

descriptions Registry for global

service discovery Services can use

other services(workflow)

Generalization Server

ServiceService

ServiceService

Generalization Server

ServiceService

ServiceService

Generalization Server

Registry Server

exec

ute

know

s

Generalization Web Services – Moritz Neun – 19. November 2007

30

The WebGen Framework

Client plug-ins for different platforms Remote processing (execute)

Generalization Server

ServiceService

ServiceService

Generalization Server

ServiceService

ServiceService

Generalization Server

JUMP GIS ClientPlug-in

Local data

AJAX Web ClientBrowser

Local data

Clarity Client

Plug-inLocal data

ArcGIS Client

Registry Server

query services

service descriptions

execute

execute

Generalization Web Services – Moritz Neun – 19. November 2007

31

WebGen Clients, Registry & Server

client server

Generalization Web Services – Moritz Neun – 19. November 2007

32

WebGen Registry

Generalization Web Services – Moritz Neun – 19. November 2007

33

WebGen Services

Large number of services available!

Generalization Web Services – Moritz Neun – 19. November 2007

34

WebGen Interface Descriptions

Generic XMLinterface descriptions: Nothing predefined

in the clients Dynamic adding of

new services Powerful also for

complex interfaces

Generalization Web Services – Moritz Neun – 19. November 2007

35

WebGen Client-Server Transfer

Generalization Web Services – Moritz Neun – 19. November 2007

36

Possible WebGen Scenario

LAN / InternetLAN / Internet

ServerJUMP

Server Plug-In

JUMP/JTSAlgorithms

ServerArcGIS

Server Plug-In

ArcGIS Toolbox

Workstation JUMP

local datasets

ToolboxPlug-In

Workstation ArcGIS

local datasets

ToolboxPlug-In

Workstation Clarity

local datasets

ToolboxPlug-In

ServerClarity

Server Plug-In

AGENTAlgorithms

Registry

coupling of systems for research or production

Generalization Service Categories

Generalization Web Services – Moritz Neun – 19. November 2007

38

MRDB

Generalization Service Categories

Gen

era

liza

tion

Serv

ices

Su

pp

ort

Serv

ices

Operator Services(simplification, aggregation, …)

service interface

interactive interface

Process Services(workflow control, evaluation)

service interface

interactive interface

Service Consumer(research, map production, …)

Support Services(attributes, triangulations …)

service interface

interactive interface

Generalization Web Services – Moritz Neun – 19. November 2007

39

Support Services

Provision of auxiliaryenriching information(data enrichment)

make structural knowledge explicit can be exploited by other services

differentiation by the output data type simple entities (geometries / attributes) complex relations (hierarchical / non-hierarchical)

Support Services

Entities(geometries, attributes)

Relations(hierarchical, non-

hierarchical)

Generalization Web Services – Moritz Neun – 19. November 2007

40

Operator Services

Operator services are implementing concrete generalization algorithms Perform geometrical and semantical transformations of map objects Can be context dependent by using structural knowledge (from support services)

Examples from building generalization:

RemovalSimplification Displacement

Generalization Web Services – Moritz Neun – 19. November 2007

41

Process Services

Workflow / orchestration of different services predefined batch workflows, adaptive workflows and fully automated systems: Selection of the appropriate algorithms

(operator services) for a situation Use of the right parameter values Application of operators in the proper sequence

simplifyinitial typify displace

Discussion

Generalization Web Services – Moritz Neun – 19. November 2007

43

Discussion

Strengths of the WebGen approach Provision of various functionalities as interoperable toolbox (research platform) Functional subdivision of generalization in

support, operator and process services Sharing, coupling and reuse of functionalities

at very different levels of granularity Provision of complex spatial data structures Central registry for service discovery Generic interfaces allow real interoperability Parallelization possible (clustering)

Generalization Web Services – Moritz Neun – 19. November 2007

44

Service Usage Scenarios

Keep in mind that different user types need different functionalities and levels of complexity: Novice user (planner wants to simplify a map) Expert user (NMA wants to couple systems) Researcher (wants to evaluate an algorithm) Data display (convert data from WFS on-the-fly)

Generalization Web Services – Moritz Neun – 19. November 2007

45

Where to start?

Challenges: Meet user scenarios Common generic (XML) service data model Generic syntactic and semantic interface descriptions needed (service capabilities and requirements) Granularity (especially of support services, stateless or stateful services)

Some work required (WebGen is still a prototype) Advance WebGen? Extend WPS with concrete profile?

Thank you!

Generalization Web Services – Moritz Neun – 19. November 2007

47

WebGen Registry