Small-Scale Raster Map Projection Transformation Using a Virtual System

to Interactively Share Computing Resources and Data

U.S. Department of the InteriorU.S. Geological Survey

Michael P. Finn

Briefing to CyberGIS Technical Points of ContactBest Practices Series19 July 2012

Collaborators/ Co-Authors• Yan Liu

– University of Illinois at Urbana-Champaign (UIUC), CyberInfrastructure and Geospatial Information (CIGI) Laboratory

• David M. Mattli– USGS, Center of Excellence for Geospatial Information Science (CEGIS)

• Babak Behzad– UIUC, Department of Computer Science

• Kristina H. Yamamoto– USGS, CEGIS

• Anand Padmanabhan– UIUC, CIGI Laboratory

• Qingfeng (Gene) Guan– University of Nebraska -- Lincoln, School of Natural Resources

• Eric Shook– UIUC, CIGI Laboratory

Outline

• Overview/ Objectives• Motivation for HPC-Research• pRasterBlaster – a primer and the early days• Current Implementation/ Program Components• Performance• Status

Overview

• HPC Return to the Mainframe?• Parallel computing for raster (map projection,

other)– Raster was born to be parallelized

• CEGIS parallel computing as part of National Science Foundation CyberGIS Grant

Objectives

• Develop a rapid raster projection transformation and Web service using HPC technology -- pRasterBlaster

• Provide a software package that better handles known problems for wide use in the modeling community

Motivation for HPC-Research

• Raster processing too slow for large datasets• mapIMG Example: Re-projecting a 1GB raster

dataset can take 45-60 minutes• Solution:• Solve problems using multiple processors• Create user-centric Web 2.0 interface that hides non-

important, complex HPC implementation details

pRasterBlaster Primer

• mapIMG in HPC Environment• Map Projection/ Reprojection for Raster

Datasets• Rigorous Geometry Handling and Novel

Resampling– Resampling Options for categorical data and

population counts (also standard continuous data resampling methods)

pRasterBlaster (the Early Days)

• Very Similar to mapIMG (V. 03)• Row-Wise Decomposition• I/O Occurred Directly in Program Inner Loop• Disadvantages

– Resampling Requires a lot of I/O– Changing I/O Implementation Difficult – Difficulty with Multiple Processors Writing to a

Shared file

Parallel Program

• Use Multiple Processors to Decrease Run Time• Evaluation

– Wall Time Vs. Speedup• Amdahl’s Law: Calculate Maximum Speedup

– Communications Overhead• Communications Between Processors is SLOW

– See Last Week’s Briefing by Babak Behzad

Current Implementation (1.0)

• I/O and Reprojection Separated into Stages• I/O Can Be Batched and Reordered• Partitioning More Flexible

– Partition can be less than a row

Version 1.0 Components

• Prologue– Open input files, calculate size of output, calculate

partitions, create output and temp files

• Reprojection – for each partition in parallel– Read input to memory, iterate through output

pixels and reproject, write partition to temp files

• Epilogue– Write final output

Performance Observations

• Evaluations ongoing – Working closely with CIGI

• The Overhead of Copying the Output File is Very High

• Likely to Adopt CIGI Suggestion/ Work– Related to creating a new, high performance I/O

based on netCDF-4/ HDF-5 parallel functions

• Need More Testing In Other Environments

Current Status

• pRasterBlaster development continuing• pRasterBlaster Version 1.0 released 01 April• pRasterBlaster Version 1.5 soon (fix identified

bottlenecks)• Iterative testing• dRasterBlaster

– d forDesktop; essentially mapIMG 4.0

• Library of core functions shared by both dRasterBlaster & pRasterBlaster (libRasterBlaster)

References• Atkins, D. E., K. K. Droegemeier, et al. (2003). Revolutionizing Science and Engineering

Through Cyberinfrastructure: Report of the National Science Foundation Blue-Ribbon Advisory Panel on Cyberinfrastructure. Arlington, VA, National Science Foundation.

• Behzad, Babak, Yan Liu, Eric Shook, Michael P. Finn, David M. Mattli, and Shaowen Wang (2012). A Performance Profiling Strategy for High-Performance Map Re-Projection of Coarse-Scale Spatial Raster Data. Abstract accepted for presentation at the Auto-Carto 2012, A Cartography and Geographic Information Society Research Symposium, Columbus, OH.

• Wang, Shaowen and Yan Liu (2009) TeraGrid GIScience Gateway: Bridging cyberinfrastructure and GIScience. International Journal of Geographical Information Science, Volume 23, Number 5, May, pages 631-656.

• Wang, Shaowen, Yan Liu, Nancy Wilkins-Diehr, and Stuart Martin (2009) SimpleGrid toolkit: Enabling geosciences gateways to cyberinfrastructure. Computers and Geosciences, Volume 35, Number 12, December, pages 2283-2294.

• Williams, Michael S., Michael P. Finn, and Robert A. Buehler (2006). An Open Source, Object-Oriented General Cartographic Transformation Program (GCTP). Abstract presented at the International Society for Photogrammetry and Remote Sensing Commission IV Symposium on Geospatial Databases for Sustainable Development, Goa, India.

Background Presentations and Publications

• Multi-Resolution Raster Project– Map Projections:

http://cegis.usgs.gov/projection/index.html – Older MRR:

http://cegis.usgs.gov/watershed/index.html and http://cegis.usgs.gov/soil_moisture/index.html

Plans for Near-Term

• Release pRasterBlaster 1.5• Finish libRasterBlaster• Release dRasterBlaster

Small-Scale Raster Map Projection Transformation Using a Virtual System

to Interactively Share Computing Resources and Data

U.S. Department of the InteriorU.S. Geological Survey

QUESTIONS??!?

Briefing to CyberGIS Technical Points of ContactBest Practices Series19 July 2012