Global CyberBridges: Hurricane Mitigation enabled

by ICTHeidi Alvarez, Florida International

University9th Annual Global LambdaGrid Workshop

October 27-28, 2009

Daejeon Convention Center, Deajeon, Korea

Presentation Outline

International Hurricane Research Center (IRHC) Hurricane Mitigation*A Research Agenda Aimed at Mitigating Hurricane

Hazards

Global CyberBridges Hurricane MitigationBackground and MotivationRole of Cyberinfrastructure and Global CyberBridgesHurricane Mitigation Project OverviewProject StatusCyberinfrastructure Contributions

Conclusion

*Slides for IRHC courtesy of Dr. Stephen Letterman, Director

Wind Damage

Storm Surge Inundation

Freshwater Flooding

Beach Erosion

IHRC Laboratories Insurance, Financial and

Economic Research

Dedicated to defining the hurricane threat to the economy

Developed the first public catastrophe model to predict damage and insured losses

Provides technical assistance to hurricane vulnerable stakeholders

Quantitatively assesses vulnerability of coastal areas to storm-induced beach erosion and hurricane surges

Utilizes advanced airborne laser mapping and computer animation (LIDAR) Coastal Research

IHRC Laboratories

Social Science Research Studies how individuals

and groups respond to hurricanes

Formulates methods to improve the resilience of communities

Wind Engineering Research Investigating solutions to

making homes and buildings more hurricane resistant

Measuring hurricane surface winds with instrumented towers in actual storm landfalls

Conducting wind, pressure and impact testing

Mitigation Research ToolsLIDAR Mapping

Miami-Dade LIDAR Collect

Areas of LIDAR Data Acquisition

Case in Point: Hurricane Katrina

Satellite view, Katrina

Wind Towers Team and Portable Doppler Radar Unit Coordinated Data Collection

IHRC Storm Surge Prediction

John and Rita Kennedy are shown, Tuesday, outside the collapsed second floor of a friend's house after it was destroyed by Hurricane Katrina on the beach in Biloxi, Miss (AFP photo by Robert Sullivan) Posted Aug.30, 2005http://www.chicagotribune.com/news/nationworld/chi-0508310183aug31,1,5787808.story?coll=chi-newsnationworld-hed

Storm Surge Prediction

Current Wind Engineering Research

IHRC Mitigation Research is taking on the problem of how to keep homes and businesses safer from damage caused by punishing hurricane winds

THE STORM SURGE

Wall of Water Set a Record

Hurricane Katrina's storm surge - the wall of water it pushed ashore when

it struck the Gulf Coast on Monday - was the highest ever measured in the United States, scien tists said yesterday.

Stephen P. Leatherman, director of the International Hurricane Re search Center at Florida Interna tional University, said the surge at Bay St. Louis, Miss., was 29 feet. Sci entists from Louisiana State Univer sity, using different mathematical models, said their estimate was low er - 25 feet.

Either way, this hurricane easily surpassed the previous record, the 22-foot storm surge of Hurricane Camille, which struck in 1969 near Pass Christian, Miss., a few miles east of Bay St. Louis.

Dr. Leatherman said scientists from Florida International and the University of Florida gathered wind data from towers they set up along the hurricane's projected path just before it struck. They used this data and previous measurements of the topography of the ocean floor and the nearby land to calculate the height of the surge.

Wall of Wind: Phase IFabricated by

Diamondback Airboats

Delivered in January 2005

Presently developing the active control system that will be duplicated in Phase II

What is Global CyberBridges?Cyberinfrastructure Training, Education,

Advancement, and Mentoring for Our 21st Century Workforce (CI-TEAM)

Three year award (Oct. 2006 - Dec. 2009) for $765,000 total to CIARA at FIU

The program expands on CyberBridges, which was initiated in 2005 to help FIU scientists and engineers advance their research through cyberinfrastructure (CI).

Global CyberBridges; Hurricane Mitigation Project Team

Advisors Students

Dr. Heidi Alvarez, Director FIU Center for Internet Augmented Research and Assessment (CIARA), PI for GCB heidi@fiu.edu

Javier Delgado, FIU Global CyberBridges (GCB) Ph.D. Fellow Project Lead javier.delgado@fiu.edu

Javier Figueroa, FIU

Dr. S. Masoud Sadjadi, FIU School of Computer and Information Science (SCIS), Co-PI for GCB sadjadi@cs.fiu.edu

Zhao “Wendy” Juan, Computer Network Information Center, Chinese Academy of Sciences (CNIC of CAS) GCB Master’s Student Lead

Dr. Hugh Willoughby, FIU Earth Sciences Distinguished Research Professor

Bi Shuren, CNIC of CAS

Dr. Kai Nan, CNIC of CAS Silvio Luiz Stanzani, UniSantos, Brazil

Dr. Esteban Walter Gonzalez Clua, Federal University Fluminense (UFF) Informatics Department

Mark Eirik Scortegagne Joselli, UFF, Brazil

Participants Distribution 2009

• Weather Research and Forecasting “WRF” (only GCB students)• FIU (Miami): 3 students

• 1 meteorology and 2 computer science

• UFF (Brazil): 2 students

• Visualization platform• FIU: 4 students• CNIC: 2 students

Image Source: http://mls.jpl.nasa.gov

Hurricane Mitigation Background

Computationally Intensive

Improvement requires cross-disciplinary expertise

High Performance Computing

Meta-scheduling

Resource Allocation

Work flow Management

Weather Modeling

Weather Research and Forecasting (WRF)

Hurricane Andrew, Florida 1992 Katrina, New Orleans 2005 Ike, Cuba 2008

Research Motivation Hurricanes cost coastal

regions financial and personal damage

Damage can be mitigated, but: Impact area prediction is

inaccurate Simulation using commodity

computers is not precise

Alarming Statistics 40% of (small-medium sized)

companies shut down within 36 months, if forced closed for 3 or more days after a hurricane

Local communities lose jobs and hundreds of millions of dollars to their economy

If 5% of businesses in South Florida recover one week earlier, then we can prevent $219,300,000 in non-property economic losses

Why Apply Cyberinfrastructure to Research & Learning?Preparation for a globalized workforce

Innovation is now driven by global collaboration

Diverse (and complementary) expertise

Enable transparent cyberinfrastructure

In Global CyberBridges, students are the bridges

Javier Delgado, FIU Zhao “Wendy” Juan, CNIC

Hurricane Mitigation Project Overview

Goals High-resolution forecasts with guaranteed simulation

execution times Human-friendly portal High-resolution visualization modality High Resolution Hurricane Forecasting

We create: A distributed software model that can run on heterogeneous

computing nodes at multiple sites simultaneously to improve Speed of results Resolution of the numerical model Scalability of requests by interested parties In other words, we need to grid-enable Weather Research and

Forecasting (WRF) software system

WRF Information: http://wrf-model.org/index.php

10-km WRF4-km WRF

Dashed magenta indicates approximate area of rainfall

Produced by convective parameterization Parameterized convection (on the 10 km grid) cannot differentiate different mode of convection

Why So Many Processors?

Source: NCAR(www.ncep.noaa.gov/nwp50/Presentations/Thu_06_17_04/Session_9/Kuo_50th_NWP/Kuo_50th_NWP.ppt)

FIUFIU

Job-Flow Manager

Job-Flow Manager

Peer-to-peerProtocols

Web-Base Portal

Web-Base Portal

Meteorologist Meteorologist

Local Resources

Local Resources

Local Resources

Local Resources

Meta-Scheduler

Meta-Scheduler

Local scheduler

Local scheduler

Local scheduler

Local scheduler

1

2 3 4

5 6

7

1

4

6

1

2 3

5

7

12357 1 4 67

Resource Policies

Resource Policies

CNICCNIC

Interaction of the Components

WRF Data

3-Layer Nested Domain that covers Florida

Distributing WRF over a WAN slows performance due to high input/output

Communication across the WAN occurs before and after the job runo Before: Send domain input.

There are three stages documented at http://www.mmm.ucar.edu/wrf/users/docs/user_guide/users_guide_chap3.htm

Domain Resolutions:o 1.7km for the inner domaino 5km for the middle domain o 15km for the outer domain

For the input data:o Static Geographical Data for the

domain + Other geographical data: About 250 MBs.

o MET Data: 35MB/time step (of simulation). We use a time step of 6hours, so for

a 3 day forecast the total size is 210MBs. Real Data: 101MB (for a 3 day forecast)

For the output data: o About 215MB per time step (of

simulation) is generated. o Time step of 1 hour. o 3-day forecast, 215*24*3 = 15.4 GB of

data without compression

WRF Web Portal

WRF Portal Hi-Res Visualization

Modeling WRF Behavior

Paradox of computationally-intensive jobs: Underestimated execution time = killed job Overestimated execution time = prohibitive queue time

Grid computing drawbacks Less reliable than cluster computing No built in quality assurance mechanism Hurricane prediction is time-sensitive, so it needs to work around

this

Meta-scheduler addresses the quality assurance issue

To predict execution time, model the software Pick a representative simulation domain Execute it on various platforms with various configurations Devise a model for execution time prediction and implement it in software Test model Adjust until prediction accuracy is within 10 percent

Mathematical Modeling

ProfilingCode Inspection & Modeling

An Incremental

Process

Parameter Estimation

Modeling WRF Behavior

Current Execution Prediction Accuracy

Adequate accuracy on multiple platforms

Cross-cluster:8-node, 32-bit Intel Cluster16-node, 64-bit Intel ClusterDifferent (simulated) CPU speed and number-of-node

executions

Inter-cluster on MareNostrum Supercomputer of Barcelona Supercomputing CenterUp to 128-nodes

MareNostrum Info: http://www.top500.org/system/8242

Visualization Platform for Hurricane MitigationScalable Adaptive Graphics Environment (SAGE)

Scalable Hundreds of Screens can

be used Built with high-

performance applications in mind

Extensible Provides API for creating

custom SAGE applications

Porting an application is not trivial SAGE is developed by UIC Electronic Visualization Laboratory.

NSF SCI-0225642 & ANI-0225642

4 by 5 SAGE Display Wall at CNIC

Enhancements to SAGERemote Desktop

Enhancement

A responsive remote desktop modality is essential for collaboration and e-Learning

Users can share their display for all collaborators to see

Non-portable applications can also be displayed

Wii Remote input interface

A traditional mouse makes it difficult to work with a large display

Global CyberBridges Overall Contributions

Weather Forecasting Students in different scientific fields from 3 different

continents exposed to the problem through a remote class Grid-computing related methodologies for addressing

these problems have been presented Collaborative publications in progress

Visualization Based on the difficulties we had in the class, we are trying

to implement a cutting-edge e-Learning environment based on SAGE

Publication: Javier Delgado, Mark Joselli, Silvio Stanzani, S. Masoud Sadjadi, Esteban Clua, and Heidi Alvarez. A learning and collaboration platform based on SAGE. In Proceedings of the 14th Western Canadian Conference on Computing Education (WCCCE 2009), Simon Fraser University, Vancouver, Canada, May 2009. (Accepted for publication.)

Acknowledgments

Global CyberBridges NSF CI-TEAM OCI-0636031

MareNostrum Supercomputer support NSF-PIRE OISE-0730065

Scalable Adaptive Graphics Environment (SAGE) NSF SCI-0225642, ANI-0225642

NSF research assistance grants: HRD-0833093, CNS-0426125, CNS-052081, CNS-0540592, IIS-0308155

For more information: www.cyberbridges.net and heidi@fiu.edu

Thank You!