Dynamic Tiled Map Services: Supporting Query-Based

Visualization of Large-Scale Raster Geospatial Data

Jianting Zhang12, Simin You2

City College1 & Graduate Center2 of

The City University of New York

Outline•Motivation and Introduction•Background and Related Work •The Proposed Solution

•System Architecture

•Query Processing Server

•Tile Image Generation and Caching

•Visualization Client

•Experiments and Evaluation•Conclusion and Future Work

Introduction/Motivation

3

If you load your own data in Google Earth,

Wouldn’t it be nicer if you can query your data and highlight the query results?

In addition to simple display, zoom in/out, pan

Global 30s Precipitation Data from WorldClim (Interpolated 1950-2000)

Coloring Schema:Green: 0 mmRed: 100 mmLinear Interpolation

Undergraduate Project: Generate Dynamic KML Files for Interactive Visualization in Google Earth (C. Dasrat/CCNY)

Jan July

Introduction/Motivation• Task: Show the regions where precipitation amount in January is

between [p1,p2). • Intuitive Solution 1: pre-generate all query results for all possible

p1 and p2 combinations and then publish them as Tiled WMS services – virtually impossible

• Intuitive Solution 2: output the query results and then publish them as Tiled WMS services – better, but still suffer from slow start– Reading/examining/outputting a raster of 43200*21600 cells takes minutes– Generating image tiles take tens of minutes or even longer (TileCache)– This is for each of every query

• Our Solution: Index raster data, perform the query in main memory, dynamically generate tiled images on-demand based on user’s current view and cache the tiled images as necessary

Background & Related Work

• Spectral, spatial and temporal resolutions of geospatial raster data are getting increasingly finer larger data volumes– The next generation GOES-R satellite will provide

global coverage at the 0.5-2 km resolution every 5 minutes (16 bands)

– Large-scale model simulation (e.g. WRF)

Background & Related Work

• Manually examine all the data through visual display is not possible anymore– Human eyes can only effectively distinguish a

limited number of colors at a time – Studies show that screen resolution beyond 4000 by

4000 pixels is not effective

• Query data and highlight results (Region of Interests) for further analysis become more preferable

Background & Related Work

• Tiled map service techniques: – Google Map/Earth, Microsoft Bing Map

– ArcGIS, MapServer/TileCache

– They are used for static images, not dynamic query results

• Query Driven Visual Exploration for scientific data • Spatial Databases

– Vector data (R-Tree/Quadtree indexing)

– Raster data

System Architecture

Indexing for Query Processing

BMMQ-Tree (SSDBM’10)

Tile Based Query

• Step 1: locate the quadtree node representing the tile being requested (x,y,L) that satisfy query criteria [V1,V2). – Divide x and y by 2 L times– Use the reverse order of the reminders to travel the tree – Stop the process if any node along the travel path does not

satisfy the query criteria

• Step2: starting from the located quadtree node– Traversal k levels of the sub-tree at the node (tile size s=2k )– Discard tree nodes that do not satisfy the query criteria– Return all the qualified nodes with top-left coordinates and level

Querying with value range [1,3] under tile (0,1,1)

Tile Image Generation and Caching

• Assuming the tile being requested is (x,y,L), the tile size is s=2k and the quadtree nodes are returned in the form of (r,c,l) triples, we can simply draw a square starting at (r,c) with length 2k-(l-L)

• Since l<=k+L, the length is guaranteed to be equal or larger than 1 pixel.

• When k+L is larger than the maximum level of the quadtree, i.e., k + L >max_quad_lev, digital zoom-in will be applied (does not convey additional information)

Tile Image Generation and Caching

• Java ImageIO package– Easier to use than GD package that MapServer relies

on

• Part of TDS source tree is extracted and assembled for image caching– TDS: THREDDS Data Server (UCAR Unidata)– Designed for OPeNDAP and OGC WCS– We reuse its caching sub-system through code

reorganization– Significant development saving on coding and testing

Visualization Client

• ArcGIS Flex API– Rich Internet Application (RIA) framework– Built-in APIs to visualize statically cached titled map – Allows to control rendering the canvas in a Web

browser at the pixels level– Poor rendering performance when the number of

squares goes beyond thousands

Visualization Client

private var _baseURL:String = "http://134.74.112.202:8080/pngsvr/PNGServlet?";

override protected function getTileURL(level:Number, row:Number, col:Number): URLRequest{

var url:String=_baseURL+"minB="+_minB+"&maxB="+_maxB+"&level="+level+"&row="+row+"&col="+col;return new URLRequest(url);

}

• Extending ArcGIS Flex API TiledMapServiceLayer class to visualize dynamically tiled images in ArcGIS Flex applications

Experiments and Evaluation

• Data: WorldClim January Precipitation Data at 30s resolution (43200*21600)

• Value range [0,1003]• Number of bins=32 • Quadtree level=16• Tile image size =256*256 (k=8)• Query processing server: Dell T5400• Tile image/Web server: Lenovo T400s

Experiments and Evaluation

Estimating End-to-End time

• Assume available network bandwidth=300k Bps TT=10ms

• Assume client display area 1024*102416 tiles

• Assume no server/client side caching (cold start)

• Estimated time: (QT+GT+TT)*16 = (50+10+10)*16=1120 ms

Summary

•Online demo: http://134.74.112.202/comgeo/testoverlay.html

•We have designed algorithms to efficiently perform tile-based queries on quadtrees and to convert quadrant-based query results into tiled images.

•An end-to-end prototype has been developed to demonstrate the feasibility of the proposed dynamic tiled map services approach

•Experiments results have showed that the prototype system achieves an end-to-end performance in the order of sub-second for 1024*1024 pixels display area using 16 tiles.

Additional Information

• GPU-based indexing– Nvidia Quadro FX3700 GPU card with

112 cores and 512M device memory

– Raster size is limited to 4096*4096 due to device memory constraints 11*5 blocks

– 20X speedup (8.7s vs. 0.4s)

– We expect to index the same global data on SGI Octane III 2-node mini-cluster with 4 GPU cards in about 1-5 seconds after fine-tuning our current codebase real time indexing