Grid-based System for Flood Forecasting

Cracow Grid Workshop ’03, 27-29.10.2003

Ladislav Hluchy

Institute of Informatics SAS

in co-operation with Water Research Institute, Vah River Authority and Slovak Hydrometeorological Institute

Slovakia

hluchy.ui@savba.sk

Outline

• Introduction

• Flood Forecasting

• Grid infrastructure for Flood Forecasting

• Use cases

• Grid-based Implementation

• Results

• Conclusion

Flood Forecasting one of the Geospatial Applications

• Applications that use data from Geographic Information System (GIS)

• Typical applications: flood forecasting, fire simulations, environmental risk management etc.

Flood Forecasting

• Topical problem: floods have caused widespread damages in the recent years

• Common interest: many countries threatened• Many potential users: governments, flood crisis

teams, insurance companies, public,• Requires Grid technology

ANFAS ArchitectureModelling ServerDatabase Server

ANFAS corecomponent

HTTP ProtocolHTTP Protocol

Data Preparation

ANFAS exchange bus (Middleware : RMI, CORBA, Http)

ANFAS bus extension (CORBA / RMI)

Model AccessComponent

Model ManagerModel ManagerDataManager

DataManager

FileExchange

Component

FileExchange

ComponentAdministration

& user manager

Administration & user manager

ApplicationServer

Map ServerMap Server

GIS Accesscomponent

GIS ArcView

ANFAS bus extension (CORBA / RMI)

Co-operative, ExplanatoryComponent

ANFAS Client

Modellingpreparation

Modelling activation

& follow-up

Modelling activation

& follow-upResults

exploitation

Resultsexploitation

NumericalModel

Integration to ANFAS core

Machine 1ANFAS Core Server

currently hosted by EADS-MS&I

Machine 2 hostingRPS Controller

i-cluster

Use of the i-clusterin the ANFAS system

SMS/FESWMS • FESWMS has been developed under funding by the U.S. Federal

Highways Administration (FHWA) • FESWMS is specifically suited for modeling regions involving flow

control structures, such as are encountered at the intersection of roadways and waterways. Specifically, the FESWMS model allows the user to include weirs, culverts, drop inlets, and bridge piers into a standard 2D finite element model.

• As there is highway planned at the Vah River pilot site in Slovakia, the choice of FESWMS model is important

• SMS provides graphical tools for defining these structures and controlling analysis using the FESWMS model. Both pre- and post-processing capabilities are included in the interface.

Detailed FESWMS structures

Input files

Finite element

Nonlinearsolver

write solutionto the file

Solution file

Updatesolution

Generating matrix

Linear solver

Nonlinear solverSolution schema

Parallel matrix generation

Generating partial matrix

PARALLEL LINEAR SOLVER

Updating solution

Implementation issuesReal flood modeling software is much more complicated than its mathematical model:

– Mathematical model of flood is well-known (partial differential equations finite elements nonlinear solver linear solver)

– Real software has to deal with• Input processing: different types input data, different variations of each type,

different formats of each variation• Special cases: wetting/drying, raining/evaporation, special constructions

(bridges, dams, culverts), wind effect, …• Calibration of results• Graphical user interface (GUI), visualisation• Error checking, documentation

As the result, source code of real software may be hundreds times longer than source code of mathematical model

Remote processing

Pre-processing

Post-processing

Processing input data

Save solutions

Parallelcomputational

kernel R

emote processing

Planned highway in the Váh pilot site

Main part affected by highway

LIDAR+highway position

Bytca city

Predmiervillage

Predmier village in orthophotomap

Predmier village in LIDARANFAS

TIN network at PredmierANFAS

Scenario: Water level for current terrain situation (Q-100-year)

Water depth

Scenario: Water level for highway without bridges (Q-100-year)

Water depth

Water level is about 70cm

higher than for situation without

highway

Scenario: Water level for highway with 2 bridges (Q-100-year)

Water depth

Water level is about 30cm

higher than for situation without

highway

Why Grid?

• Cooperation: requires cooperation between many organizations (meteorological institutes, river authorities) from many countries

• Data management: needs large amount of data of different sources, different owners, different countries, different access right

• Computation power: forecasting require large computational power for modeling and simulation

Virtual Organization

• Purpose– Shared data and computational power for flood

forecasting– Cooperation between users for flood forecasting

• Requirements– Identify and define clear relationships between users– Authentication: certificate authorities– Authorization: access right for each data/resources– Collaborative tools– Security

Flood Forecasting VO

Storage systems

databases

surface automatic meteorological and hydrological stations

systems for acquisition and processing of satellite information

meteorological radars

External sources of informationGlobal and regional centers GTSEUMETSAT and NOAAHydrological services of other countries

Data sources

meteorological models

hydrological models

hydraulic models

High performance computers

Grid infrastructure

Flood crisis teams meteorologistshydrologistshydraulic engineers

river authoritiesenergyinsurance companiesnavigation

mediapublic

Virtual Organization for Fire Simulation

Storage systems

databases

GISFuel type- vegetation- canopy cover

GISTopography-elevation-slope

Meteorological data source

Weather

- wind direction, speed

- temperature, rel. humidity

FIRE MODELDescriptive Numerical Parameters

Data sources

Grid infrastructure

Fire ManagementCreation Decision Support system, prevention-Terrain, resources, - capacities

Fire suppression authorities- training- operation mode

Ecosystem authorities Universities,Insurance companies

Fire Modelling SystemFARSITE

Data management

• Typical data: satellite images, radar images, measured data from hydrological stations, topographical data, historical data, simulation results

• Different formats, different quality, different owners, different access right

• Metadata server:data description, security, replication

FloodVO data transfer

Data sources

Storage systems

Databases

surface automatic meteorological and

hydrological stations

systems for acquisition and processing of

satellite information

meteorological radars

External sources of informationGlobal and regional centers GTSEUMETSAT and NOAAHydrological services of other countries

meteorological models

hydrological models

hydraulic models

Grid infrastructure

Flood crisis teams meteorologistshydrologistshydraulic engineers

river authoritiesenergyinsurance companiesnavigation

mediapublic

DataGrid

• EDG Replica Manager• EDG Local Replica Catalogue• EDG Replication Metadata Catalogue• EDG Replica Optimization Service

Storage Element

Storage control

Metadata

EDG RM

EDG RMC

EDG ROS

EDG LRC EDG LRC EDG LRC

DataGrid (cont.)

Grid computing

• Many multidisciplinary simulations are needed for flood forecasting

• For critical situations, short response times are very important

• Numerical simulations are computationally intensive

• Grid can offer the necessary computational power

Visualization

• Data are stored in many different formats• Unified visualization tools may simplify the user-

interface• Many data for flood forecasting has spatial

character

=> GIS software may be used as the unified visualization tool

Portal

• The unified user-interface• Allow users access to the VO remotely• Simple requirements on clients - based on

standard Web technologies

3 current portals

1. Based on GridPort using Globus grid toolkit

2. Based on Jetspeed portal framework using DataGrid/CrossGrid services

3. Migrating Desktop - java fat client using DataGrid/CrossGrid services

GridPort

• A set of Perl scripts that enable Perl based portal (its CGI scripts) to use grid services of underlying Globus toolkit

• Wraps Globus’ command line tools

• Provides session management

• Provides no additional portal infrastructure

Architecture of GridPort based portal

Portal(Apache

web server)

StorageG

P, …

Resource 1

Resource 2

Resource n

Storage & Portal Machine

User’s web browser

GridPort screenshot

Job submission in Flood-VO

XML file (parameter description)

Config. file (default values of parameters)

New config file

Job script file

globus-job-submit machine job_script config_file

Flood-VO: Job list

Flood-VO: Field data

II SAS

Time Value

00:00:00 102.00000 cm

01:00:00 126.00000 cm

02:00:00 103.00000 cm

03:00:00 80.00000 cm

04:00:00 70.00000 cm

05:00:00 65.00000 cm

Jetspeed

• Portal framework

• Server-side Java based engine (application server)

• Client services are plugged using software components called portlets.

• User can arrange portlets – position, size, visibility

Jetspeed - architecture

Application portal screenshot (Jetspeed)

Application Portal

Migrating Desktop (MD)

• Java application (applet) running at the client computer

• Provides interface to all basic grid services (authentication, job

management, file management)

• Application specific job parameter input and job submission is

supported via application plug-ins

• has built-in viewer for common picture formats (jpeg, gif, png) and

text files, advanced visualization of results via application specific

visualization plug-in

• Being developed in the context of the CrossGrid project

Migrating Desktop

Screenshot of MD with Job submission wizard dialog

Use case: Cascade simulationData sources

Hydrological simulation

Hydraulic simulation

Portal

Meteorological simulation

Model characteristics

• ALADIN (meteorological model)– Limited area model– Operated by 13 Euro-Mediterranean countries– ALADIN/SLOVAKIA operated by SHMI– More than 1M lines of source code (mainly F90)– Developed for 64 bit big-endian architecture– Proprietary - requires nondisclosure agreement

• ALADIN (meteorological model)– Type: MPI parallel task, possible parameter studies –

multiple executions– CPU time: approximately one hour on 8 processors– I/O size: 33/180 MB per run– Scalability: on fast Ethernet up to 8 processors– Input data: boundary conditions– Output data: quantitative precipitation forecast,

temperature

• HSPF (hydrological model)– Type: sequential task, multiple executions (high

throughput computing)– CPU time: very small (seconds - minute)– I/O size: 1-10 MB– Scalability: HTC– Input data: quantitative precipitation, temperature,

topographical data– Output data: hydrograph

• FESWMS (hydraulic model)– Funded by US Federal Highway Administration– Distributed in commercial package SMS by EMS-I– Source code available (direct cooperation with

developer)– Optimized and parallelized by II SAS

• FESWMS (hydraulic model)– Type: MPI parallel task, multiple executions with

different input data– CPU time: 10min to several hours per a task– I/O size: 10-100 MB– Scalability: good for smaller number of processor (to

16). – Input data: inflow, topographical data– Output data: water levels and velocities

DaveF model

• A time-explicit finite-volume model from the same developers as FESWMS. It is considered as the complement of FESWMS and it is best suitable for unsteady state with critical or super-critical flow (dam-breaking, flash flood, flood with wetting/drying in large expanses)

• DaveF uses the same graphical environment like FESWMS (SMS) and similar input/output format =>can be easily added into ANFAS system

• Parallel version of DaveF has been developed for clusters by II-SAS and shows good results

Architecture

PortalComputing

element

Resource broker

Storage element

Login to FloodVO

PortalComputing

element

Resource broker

Storage element

Choose simulation

PortalComputing

element

Resource broker

Storage element

Enter input parameters

PortalComputing

element

Resource broker

Storage element

Visualization

PortalComputing

element

Resource broker

Storage element

Visualization

PortalComputing

element

Resource broker

Storage element

Download simulation results

PortalComputing

element

Resource broker

Storage element

Cascade simulation

PortalComputing

element

Resource broker

Storage element

Call meteorology master scriptRun meteorological simulationExtract hydrological input from resultsCall hydrology master script Run hydrological simulationCheck results if (inflow > critical flow)If yes: call hydraulics master script run hydraulic simulation

Cascade simulation

PortalPortalComputing

element

Resource broker

Storage element

Resource broker

Computing element

Storage element

Portal

Resource broker

Storage element

Cascade simulation

PortalComputing

element

Resource broker

Storage element

Cascade simulation

PortalComputing

element

Resource broker

Storage element

Cascade simulation

PortalPortalComputing

element

Resource broker

Storage element

Resource broker

Computing element

Storage element

Portal

Resource broker

Storage element

Cascade simulation

PortalComputing

element

Resource broker

Storage element

Cascade simulation

PortalComputing

element

Resource broker

Storage element

Dynamic flood simulation for t=315 minutes

Dynamic flood simulation for t=1905 minutesthe maximum water level

Dynamic flood simulation step 1 time 0

Dynamic flood simulation step 3 time 0:30

Future work

• Adding more models • Performance analysis and optimization• Improving data management (repositories)• Adding more information about data (metadata)• Adding collaborative tools

Use case: Simulation sequence

Hydrology

HydrologyHydrology

Hydraulics

MeteorologyCZ Meteorology

MeteorologyA

MeteorologyD

MeteorologyH

MeteorologyCH

Danube River basin

Experts

Data providers

Navigation

Computingcenters

Flood crisisteams

Riverauthorities

MediaPublic

Virtual Organizationfor Flood Forecasting

FloodGridFloodGridEnergy

Insurancecompanies

simulations

information

Knowledge based Flood forecasting

Data Sources

Visualization/Output Processing

MeteorologicalSimulations

HydrologicalSimulations

HydraulicSimulations

Metadata & Knowledge Repository

Automatic or Semiautomatic

ReasoningExternalResources(Web services)

Grid infrastructure

Simulation Metadata (parameters, area, sim. method, etc)

User interaction(Semiautomatic)

Job Submission (based on evaluation of previous jobs outputs)

Information aboutjob run (relevance)

Actual data fromobservation stations(relevance)

Grid-based System for Flood Forecasting

Documents

Transcript of Grid-based System for Flood Forecasting

Local flood forecasting for local flood risk areas

Flood forecasting methodology in Alberta - Evan Friesenhan

Flood Forecasting - Rainfall Measurement and Forecasting · 2021. 2. 15. · Flood Forecasting Rainfall Measurement and Forecasting R&D Technical Report W5C-013/4/TR K A Tilford,

FLOOD FORECASTING AND WARNING NETWORK PERFORMANCE ...

Community-Based Flood Forecasting and Warning System …

FLOOD FORECASTING IN THE REPUBLIC OF MACEDONIA TRAINING WORKSHOP ON FLOOD RISK ASSESSMENT September 27 – October 1 2010, Istanbul, TURKEY FLOOD FORECASTING.

Flood forecasting and warning for Muar River: Non-structural measures for flood … · 2020. 4. 2. · Flood forecasting and warning for Muar River: Non-structural measures for flood

Development of Flood Forecasting System Using …for upstream catchment, forecasting the inflow to reservoir, whereas the downstream catchment is devoid of a sound flood forecasting

Flash Flood Forecasting: An Ingredients Based Methodology

Using Flood Modeller for real-time Flood Forecasting in Wales · Using Flood Modeller for real-time Flood Forecasting in Wales •NRW •Flood Forecasting in Wales •Mapping vs Forecasting

FLOOD FORECASTING AND WARNING …

1 Hydroinformatics Support to Flood Forecasting and Flood Management ADRI VERWEY Abstract

Updating the Flood Forecasting and Warning …...Updating the Flood Forecasting and Warning Implementation Guidelines Tom Hogenbirk, P.Eng. Ontario Flood Risk Management Workshop September

Surface Water Flood Forecasting · roads flood forecasting showed that there is no precedent for large scale pluvial flood forecasting due to the localised nature of flood events.

USE OF ESTIMATION TECHNIQUES FOR FLOOD FORECASTING

Real-Time Flood Forecasting · 16 | Real-Time Flood Forecasting • In many cases, existing models can be refitted for forecasting • Tools are readily available for transforming

Flash Flood Forecasting Subsystems - MetEd · Chapter 5: Flash Flood Forecasting Subsystems. 5-4. Flash Flood Early Warning System Reference Guide. 4. The Flash Flood Monitoring Program

Real time flood risk forecasting presentation

Improvements in flood forecasting

Flood Forecasting and Warning Issuanceflood forecasting and warning system, the problem and obstacle dealing with the flood forecasting and warning issuance during 2011 inundation