A RESTful Encounter -...

WHITEPAPER

This paper will review how, using the implementation of a RESTful web service, geospatial data can be accessed through FDO and displayed online in a variety of formats.

REST (REpresentational State Transfer) is an architecture that describes distributed computing within a mixed environment, such as we see across the Internet. REST is a lightweight framework that relies on HTTP protocol. With the evolution of open standards and, particularly in our case, of geospatial standards as evidenced through the work of bodies like the Open Geospatial Foundation (OSGeo), and Autodesk’s release of the Feature Data Object (FDO) API and the MapGuide Open Source platform, RESTful architecture provides a method of increasing the interoperability of geospatial web-based applications and access to data stores without high overheads in development or difficulties in access across distributed systems.

Background

REST first was introduced as an architecture definition by Roy Fielding in his PhD dissertation at the University of California, Irvine in 2000. This definition, or style, as it is often called, was used to explain the construction of large scale hypermedia systems, notably the Internet, although it is not restricted to that environment. While an abstraction, the principles of REST have been used with success to explain the scalability of the HTTP protocol.

As the Internet has evolved, it has graduated from a system where information is linked and presented to one where we participate in both consuming information and contributing to it, constantly changing the depth and purposes for which the information is used. This evolution constantly faces the challenge of interoperability among the technologies and data stores that make up the system as a whole. There have been several attempts to create frameworks to address and standardize this distributed computing, notably CORBA and DCOM, but these are complex and require interaction with any firewall through which they must pass information. Web applications using server side controls and javascript clients can be, and often are, developed using PHP, ASP or .NET, but vary for each application. Simple Object Access Protocol (SOAP), which has been supported by IBM and Microsoft and REST, which is supported by Amazon and Google, are more recent attempts to create easily managed web interfaces.

REST in More Detail

Since Fielding’s initial work, the current REST style has relied largely on the work of the W3C Technical Architecture Group (TAG) in conjunction with the development of HTTP 1.1, based on the existing design of HTTP 1.0. Although there are many implementations of the REST architectural style, the World Wide Web is the largest.

The properties of the REST architectural style are well documented in a number of sources, but I’ll list them here for convenience.

Performance

Scalability of component interactions

Simplicity of interfaces

Modifiability of components to meet changing needs (even while the application is running

Background

REST in More Detail

Geospatial open source

GeoREST

Some Examples

In This Issue

A RESTful Encounter

Visibility of communication between components by service agents

Portability of component deployment

Reliability

By applying specific interaction constraints to components, connectors, and data elements, you can realize the properties of the style. The formal REST constraints are listed here.

Client and Server Separation

By separating the concerns of the clients from those of the servers, interaction between the two components can be improved significantly. Clients are not burdened with understanding functions such as data storage or other local processes on the server and servers are not concerned with any user state on the client. The two are separated by some form of a uniform interface. Development on both sides of the interface is improved as is portability. Client or server side components can be replaced or upgraded with no entanglement, as long as the interface between them does not change. Servers become much more scalable without worry about client issues.

Uniform Interface

As mentioned previously, a uniform interface constraint is required to properly maintain the independence and manage the interactions in the separated client – server architecture.

Statelessness

In addition to the separation of clients and servers, there is a constraint that no client-related state or context is stored on the server. When a client makes a request to a server, all the information required by the server to process that request forms part of the request. The server need not maintain any information about the state of the client or the nature and content of the request. Any information about the state of the session is contained in the client. However, this does not limit the server’s ability to send information about the state to some other service (e.g. a database) to provide for authentication or maintain some period of persistence for that particular session state. When the client wants to move to a new state, it sends a request to the server. As an example, think of a web browser sending a URL for a web page to a server. The server processes the request and the page is displayed in the browser. The browser then changes state by requesting a new page.

Cacheability

As we can see on the Web, a client can cache a response. Because of that ability, a response must contain information about its ability to be or not be cached. A response which cannot be cached would have the effect of disallowing the client to use data which may be out of date or not appropriate as a response to some other request. Caching can speed up significantly, or even eliminate certain client – server dialogs.

Layering and Transparency

Usually a client cannot, based on the response, discern whether it is communicating directly with the final server initiating the response or whether it is connected to some intermediary server which is passing the request and response. Because this is all transparent, using intermediary servers may be beneficial for scalability, security, and stability within the system.

Code on Demand

Code on demand is really the only optional constraint within a REST architecture. Code on demand allows a server to pass additional functionality to a client through executable code sent to the client. A common example of this is javascript code executed on the client as part of the response.

• A RESTful Encounter

When a system complies with the above constraints, it conforms to the REST architectural-style, and is said to be a RESTful system.

While REST usually is thought of in terms of HTTP, it is not limited to that protocol, and may be based on any application layer protocol provided the criteria are met for a meaningful representational state transfer. As we have noted, a RESTful system relies on the use of an existing, well-defined interface and other built-in capabilities provided by the chosen network protocol, and it will minimize any additional application-specific features on top of it.

In contrast, a protocol such as the SOAP RPC over HTTP, encourages each application designer to define new, application-specific operations that supplant HTTP operations. By doing this, many of HTTP’s existing capabilities are disregarded. The advantage SOAP has over REST is that SOAP, by not following HTTP conventions, works equally well over TCP, named pipes and message queues, to name a few.

An important concept in REST is the existence of resources which are sources of specific information, each of which is referenced with a global identifier. In HTTP, this is the URI (Uniform Resource Identifier, aka URL). In order to take advantage of these resources, the components of the network communicate via a standardized interface (e.g., HTTP) and exchange representations of these resources. These representations are the actual documents which convey the information.

Later in this paper we will look at how geospatial data (the resources) will be exposed through a web service to supply the information we request in the format we want (the representations). This RESTful web service is a web API implemented using HTTP and REST principles. The resources will have four defined aspects.

The base URI for the web API, such as http://geodatasite.com/resources/.

The Internet media type of the data supported by the web API. This often is JSON or XML, but can be any other valid Internet media type such as an image or plain text, provided that it is a valid hypertext standard.

The set of operations supported by the web API using HTTP methods (e.g., GET, PUT, POST, or DELETE).

The API must be hypertext driven.

In addition to the obvious components of the system, such as URIs, Internet media types, and request and response codes, HTTP has operations called request methods. These are listed below.

GET – Return a list of the results of the collection

POST – add a new member to the collection

PUT – replace the entire collection

DELETE – delete either the entire collection or selected members

The PUT and DELETE methods are called idempotent methods, since they have consequences for the collection and multiple requests of these methods on the collection should have the same effect as a single request (replacement or deletion). The GET method is a safe or nullipotent method, meaning that calling it produces no consequences for the collection. The POST method, although it does have consequences, generally is not considered idempotent, as multiple POST requests can have varying effects on the collection.


Because of the layering constraint in REST, there may be any number of connectors used to mediate a request, but each connection does so without any knowledge beyond its own request. Within a RESTful system an application can interact with a resource by knowing the URI (the identifier) and the required action (POST, GET, etc.). The requesting application does not need to know anything about the path the request takes or what technical components are involved along the route. In other words, handling things like proxies, gateways, firewalls or anything else between the requesting application and the server actually holding the information are not the concern of the request application or part of the request structure. In order to present the information returned from the request, however, the requesting application does need to understand the format of the information (its representation), which typically is a document (HTML, XML, JSON), although it may be an image, plain text or any other content, provided that content follows a valid standard.

Because REST is an architectural style, there is no official standard for RESTful web APIs, unlike SOAP-based web services, because SOAP is a protocol, not an architecture. Even though REST itself is not a standard, a RESTful implementation can use standards like HTTP, URI, and XML.

Geospatial open source

There are, within organizations dealing with geospatial data, a number of challenges. These include, but are not limited to:

Changes in the workforce as longer term experienced workers within infrastructure industries retire and are replaced by newer workers without an immediate in-depth knowledge of the infrastructure in place. These industries include water, waste water, power, gas, telecommunications and roads and highways. The existing knowledge base has to be preserved and made easily accessible to those requiring it.

The volume and complexity of data is increasing rapidly. Larger volumes of data need to be processed, cataloged and distributed. Data arrives from many sources, both collected internally and from the public domain. A growing amount of this data is transient and needs to be addressed in a format that is useful for the current applications consuming it.

As the data volume grows, interoperability becomes an issue. Most organizations with an engineering focus have a large degree of balkanization among their various divisions, including CAD, mobile systems, GIS and spatial data stores and tabular financial and business systems. Many of these systems are proprietary, often legacy, applications from different vendors and are incompatible with each other. Productivity and efficiency are the business forces which are forcing IT teams to look for ways to break down interoperability barriers.

Open Standards is one of the most important technical advances to provide a foundation for interoperability. The Internet, and specifically the Web is an example of the application of open standards. Within the geospatial world, standards proposed and developed by the Open Geospatial Consortium (OGC) allow the exchange of spatial data. While web applications from many of the major geospatial vendors are, for the most part, still proprietary, there are open source projects that are moving in the direction of an open standards model.

Shortly after the formation of the Open Source Geospatial Foundation (OSGeo), Autodesk released the source of the Feature Data Object (FDO) application programming interface and the MapGuide Open Source platform to the open source community. FDO is different from other programming interfaces in that it was designed specifically to support the editing and versioning of spatial data. FDO provides consistent access to a large number of spatial data stores including:

Oracle Spatial

ArcSDE

SHP

SDF

GDAL/OGR


PostGIS

MySQL

SQL Server Spatial

SQLite

ODBC

WMS

WFS

The C++ code that makes up the FDO source, which is available from OSGeo, has been compiled to run on Windows and Linux. Similarly, the source code for MapGuide Open Source is available from OSGeo.

Having open standards addresses one part of the problem. However, there is still the issue of how to expose these applications in a general way on the Web. For example, you can wrap a PHP, JSP, or ASP programming interface around an application with a Javascript client, but this approach will be different for each Web application. A more general approach would be to wrap the C++ code with standards-based Web services. This allows client applications to access geospatial data and services in a standard way.

GeoREST

The GeoREST extension originally was built by SL-King, an Autodesk third party developer. It was built for Autodesk Infrastructure Map Server (AIMS) and MapGuide Open Source (MGOS). It allows you to connect to spatial data through AIMS or MGOS running on a web server such as IIS (Windows) or Apache (Windows, Linux) via FDO or directly through FDO without the need for the AIMS or MGOS servers on a web server.

A number of Autodesk applications, including Autodesk AutoCAD Civil 3D, Autodesk AutoCAD Map 3D, AIMS, AutoCAD Utility Design and Autodesk Infraworks connect to spatial data using FDO. In addition, non-Autodesk applications such as MGOS, FDO Toolbox, and FME from Safe Software also communicate with spatial data through FDO.

You can see some of the providers listed below (image from Autodesk Infrastructure Studio 2014).


As we can see from our look at the REST architectural style, GeoREST should allow us to access spatial content through a URL using a standard web request without the need for any additional components such as ASP.NET or PHP to process the request and deliver the content. GeoREST will expose each FDO connection as a web service and will return the data representation either directly or through a web server running AIMS or MGOS.

To see this in action, let’s take a look at the GeoRESTHTTPServer. This application sets up a web service on port 99 (its default). It’s found here.

This is the executable.

When we run it, we get a command window. The server will run as long as the window remains open. By default, the server will run on port 99.


Using this URL will connect you to the REST HTTP server. Remember, this is not a web server like IIS, but a web service just for REST. You may notice desktop shortcuts were installed to both start the server and request this URL when AIMS was installed.

You can see what’s happening when you look at the REST server command window.

The URL returns an index page that has several examples of how GeoREST is used and the types of representations it will return from the various sources.


Taking a closer look at the GeoREST folder, you will see that it contains a number of other folders. The \bin folder contains the executable content, while the \conf folder contains the configuration settings for each of the resources that are exposed.

In the \conf folder you will see the \Sheboygan\Property folder. This folder contains a number of TPL files, as well as an XML file called restcfg.xml. The TPL files are templates, the property_html_* files are templates for publishing the HTML data representations and the property_kml_* files are templates for publishing KML (Google Earth) data representations.

The key to explaining how all this comes together is found in the restcfg.xml file (shown below). Each resource will have its own configuration file. This file lists the details of the resource type, how to connect to it and what representations are available to publish the data returned from it. This restcfg.xml file publishes directly from an FDO connection; it does not use AIMS or MGOS.

<RestConfig> <Data> <Resource uripart=”property”>

Here’s where we define the source of the data.

<Source type=”FDO”> <Provider>OSGeo.SDF</Provider> <ConnectionString>File=C:\Program Files\Autodesk\ Autodesk Infrastructure Web Server Extension 2013\www\ GeoREST\data\Sheboygan\Parcels.sdf;ReadOnly=FALSE;</ConnectionString> <FeatureClass>SHP_Schema:Parcels</FeatureClass </Source>

The source is using an SDF provider with a connection string pointing to its location and the feature class being used within the Parcels.sdf file.

Next are the representations (JSON, XML, OData, HTML and KML). Each of the representation sections describes how the data is to be published, the method used to obtain the data (GET), the default number of records per page and the total number of records to be returned in the record set.


The JSON and XML representations are quite straightforward.

<Representation renderer=”JSON” pattern=”.json”/> <Representation renderer=”XML” pattern=”.xml”/>

Both HTML and KML use templates to render their output. As you can see, each has a template for different results; a single record, multiple records, no records found and an error condition.

<Representation renderer=”Template” pattern=”.html” mimetype=”text/html” > <Templates section=”PROPERTY” prefix=”PROPERTY_”> <Many file=”property_html_many.tpl” /> <Single file=”property_html_single.tpl” /> <None file=”property_html_none.tpl” /> <Error file=”property_html_error.tpl” /> </Templates> <Method name=”GET” defaultcount=”5” maxcount=”500” /> </Representation>

<Representation renderer=”Template” pattern=”.kml” mimetype=”application/vnd.google-earth.kml+xml” > <Templates section=”PROPERTY” prefix=”PROPERTY_”> <Many file=”property_kml_many.tpl” /> <Single file=”property_kml_single.tpl” /> <None file=”property_kml_none.tpl” /> <Error file=”property_kml_error.tpl” /> </Templates> <Method name=”GET” maxcount=”500” maxbboxwidth=”0.009” maxbboxheight=”0.009” /> </Representation>

<Representation renderer=”Template” pattern=”.kmz” mimetype=”application/vnd.google-earth.kmz” > <Templates section=”PROPERTY” prefix=”PROPERTY_”> <Many file=”property_kml_many.tpl” /> <Single file=”property_kml_single.tpl” /> <None file=”property_kml_none.tpl” /> <Error file=”property_kml_error.tpl” /> </Templates> <Method name=”GET” maxcount=”500” maxbboxwidth=”0.009” maxbboxheight=”0.009” /> </Representation>

The OData representation also is straightforward.

<Representation renderer=”OData” />

</Resource> </Data> </RestConfig>


Some Examples

Let’s look at some sample output using the examples from the index.html page. This is the single property record page rendered in HTML.

You will notice in the page URL that the feature ID forms part of the address (11516). The 11516.html representation uses the single record HTML template referenced in the restcfg.xml file. The record is retrieved and formatted as an HTML page. This page may be bookmarked, as its location does not change.

In the lower right of the image above, just above the red arrows, you will see an icon representing a broken link to an image. What is missing is a PNG image of the property. In order for this to display we would need an AIMS or MGOS source, not just a straight FDO connection source.

The red arrows point to other representations. Clicking on the 11516.xml link publishes this.


Note that it is the same resource but represented as XML rather than HTML. Since the feature geometry is part of the data you will see it in the output.

<Property> <Name>SHPGEOM</Name> <Value> POLYGON ((-87.712237415474647 43.736351669336358, -87.712237037423108 43.735967587352981, -87.712350547717662 43.735967528749221, -87.712350736741215 43.736159569656138, -87.712350925767808 43.736351610556582, -87.712237415474647 43.736351669336358)) </Value> </Property>


Here’s the KML (Google Earth) representation of the same parcel using the 11516.kml link. The KML templates launch Google Earth, then zoom into the parcel area.

So far, all these examples have used the GET method to return a record. You can use other methods such as POST to alter record content. More on that in a future paper, perhaps.

About the Author: Jeff Morrow

Jeff, a senior civil and geospatial consultant with IMAGINIT, has over thirty years’ experience in the mining and engineering sectors. He has been involved with AutoCAD and the Autodesk civil software applications since their earliest days, and has consulted on the development of standards and procedures, CAD management, database and technical applications, training and implementation, and project management. Jeff is recognized for his work in mineral exploration, minesite reclamation and environmental issues. He is a graduate of McGill University, and is an AutoCAD Civil 3D Implementation Certified Expert. You can reach him at [email protected].

Page 13 • A RESTful Encounter

161 WORCESTER ROAD, SUITE 401, FRAMINGHAM, MA 01701

imaginit.rand.com 800-356-9050 508-6631401

A RESTful Encounter -...

Documents

Transcript of A RESTful Encounter -...