Project: Funded by: Supervised by:Chair for Hydrology and River Basin

Management (Prof. Dr. Markus Disse)

Presented by: Qing Lin, M.Sc.

Technical University of Munich

Arcisstraße 21, 80333 Munich

tsching.lin@tum.de

Departement of Geography

(Prof. Dr. Ralf Ludwig)

Presented by: Florian Willkofer, M.Sc.

Ludwig-Maximilians-University Munich

Luisenstr. 37, 80333 Munich

florian.willkofer@lmu.de

Flash Flood Modeling on HPC SystemsQing Lin1, Karl Broich1, Thomas Pflugbeil1, Johannes Mitterer1, Maria Kaiser1, Hai Nguyen3, Fabian von

Trentini2, Florian Willkofer2, Ralf Ludwig2, Markus Disse1

Introduction

City

Catchment

Transfers

Figure 7: Schematic of the discharge transfers at the transition from catchment to

city and the spatially distributed surface runoff (points)

Figure 8: Resolution and gauges of the hydrological model WaSiM (25m) and the

hydrodynamic model (TELEMAC, appr. 5m) at two transfer gauges

Table 1: Applied models in the HiOS project.

Figure 3: Comparisons of speed-up curves between pure MPI and hybrid

MPI/OpenMP for TELEMAC (Shang 2014).

Figure 4: Scaling of WaSiM for one year of simulation on SuperMUC Phase 2

The Cooperation Project HiOS

Figure 1: Flash flood in Simbach a. Inn, 1.6.2016. © pa/dpa/Walter Geiring.

1) Technical University of Munich (TUM), Chair of Hydrology and River Basin Management, 80333 München, Germany2) Ludwig Maximilian University of Munich (LMU), Department of Geography, 80333 München, Germany 3) Leibniz Supercomputing Centre (LRZ) of the Bavarian Academy of Sciences and Humanities, 85748 Garching b. München, Germany

What is a Flash Flood?

Flood of short duration with a relatively

high peak discharge (IHP/OHP, 1998).

Why are Flash Floods so dangerous?

• occurrence within minutes to hours and

not necessarily bound to watercourses.

• cascade effects: landslides, mudflow,

sediment transport, etc.

• Excessive hydrodynamic forces with

high destructive potential for buildings

and other assets.

• heavy rain events are small-scale and

highly variable

• conventional measuring systems can

not measure flash floods efficiently

HPC Topology

All models simulate processes in a spatially

2-dimensional and time-depended space.

Output: n x m-datasets for the

catchment/town region for each time level.

Models suited for HPC-cluster split the

computational problem in several parts:

• Hydrology: WaSiM, catchment split.

• Hydrodyn.: TELEMAC, town region split.

• Input: One catchment/town with n-

parameter sets and m-load cases.

Models suited for cloud calculate the

domain en bloc:

• GIS-Template: Parallelism using virtual

machines VM (embarrassingly parallel

use).

• Hydrological models: Parallelism using

VMs.

• Input: One catchment with n-parameter

sets and m-load cases.

Summary and Outlook

High performance computing is an essential part of the HiOS-project. The benefit

results from the combined use of cloud and cluster systems. The need for HPC is

caused by

• Great amount of measured and observed data.

• Discretization of catchment areas covering all of Bavaria.

• High discretization of 80 town regions.

• Long time series (2 days) to be calculated at a low time resolution (dt =1 sec).

• Need for analysis of hundreds of variations.

The compute cloud is already used for GIS-analysis (ArcGIS-server) and hydrodynamic

calculations. The parallelized programs WaSiM and TELEMAC are carried out on the

cluster CoolMUC2. It is planned to expand to the more powerful SuperMUC-cluster.

First runs proved the performance gain on this system.

The benefits of HPC for the time consuming simulations of the 80 Bavarian towns and

municipalities will be analyzed and optimized in the next work package.

Model Coupling

Two types of offline, one-way coupling of

hydrological and hydrodynamic models:

Advantages

• Better representation of hydrological

processes (e.g. infiltration and runoff)

• Smaller domain (city area) for the high-

resolution hydrodynamic model saves

computation time and resources.

• Possibility to compare inflows from the

two model types and surface runoff

from the hydrological model

against SCS-CN method (NRCS,

2010).

Applied Models

Three different kinds of models are used in

the HiOS project:

• GIS-Tools (GIS)

• Hydrological Models

• Hydrodynamic Models

The models WaSiM and TELEMAC are

parallelized and scaled on HPC cluster

systems. All other models can be used on

cloud systems.

Project Goal

• Development of a method to evaluate and classify the risk due to surface

runoff and flash floods using a GIS application.

• The GIS application will be used to query, link and evaluate the factors

promoting and triggering the generation of surface runoff and flash floods.

• Detailed study on 80 towns and municipalities in Bavaria using coupled

hydrological and hydrodynamic simulation.

• Generation of a flash flood indicator map for Bavaria indicating different

hazard zones for each of the more than 2000 Bavarian municipalities.

• Project ‚Hinweiskarte Oberflächenabfluss und Sturzflut‘ (Indicator Map for

Surface Runoff and Flash Floods) abbreviated HiOS.

• Funded by the Bavarian State Ministry of the Environment and Consumer

Protection (StMUV).

• Supervised by the Bavarian Environment Agency.

• 3 years (August 2017 until August 2020).

Literature

Hörmansperger 2018 : Entwicklung eines GIS-Tools für die Gefahren- und Risikoanalyse von Sturzfluten in Siedlungsgebieten, master thesis, TUMIHP/OHP, 1998 : IHP/OHP, Int. Glossary of HydrologyNRCS, 2010 : Natural Resources Conservation Service, Chapter 15: Time of concentration. Washington, USA : United States Department of Agruculture, Part 630 National Engineering Handbook, 2010.Shang 2014 : High performance computing for flood simulation using Telemac based on hybrid MPI/OpenMP parallel programming; Shang, Zhi; Int. Journal of Modeling, Simulation and Scientific Computing; Vol. 5, No. 4 (2014)

Figure 5: Topology for HPC cluster, each box represents a node/part consisting

of n-cores/ subdomains (n=28 for SuperMUC at LRZ), N=number of timelevels.

Figure 6: Topology for cloud system, VM=virtual machine.

VM 1 VM i VM nTemplate

Input

Output Output Output

Input Cluster Results

Domain A with

parameter set

Domain B with

parameter set

Domain C with

parameter set

Domain D with

parameter setDomain E with

parameter set

1

iN

1

iN

1

iN

1

iN

mxnxN Results A

mxnxN Results B

mxnxN Results C+E

mxnxN Results D

Figure 2: Flash flood hazard map (Hörmansperger 2018).

prediction and warning of flash floods

is difficult.

flash floods are still little understood

and documented.

GIS

ArcGIS Pro

ArcMap

QGIS

Hydrology

LARSIM

WaSiM

Lumped

models

Hydraulics

TELEMAC

Hydro_AS-2D

FloodArea

P-DWave

WaSiM 10.00.03 – strong

scaling @ SuperMUC

Haswell

Intel MPI / Walltime

• 3h temporal resolution

• 1 model year

2920 time steps

Coupling of WaSiM 25m

and TELEMAC 5m

legend

gauge WaSiM (25m)

gauge hydrodynamics

urban area coupling points

scalar flow rate [m³/s]

flow network WaSiM (25m)

0.0 – 0.5

0.6 – 1.5

1.6 – 3.0

3.1 – 4 5

4.6 – 6.5

6.6 – 10.0

10.1 – 17.0

hydrology

inflow from catchment

areas to the city area

spatially distributed

surface runoff for city

hydrodynamics

boundary input at

transfer points

effective

precipitation input

typ

e 1

typ

e 2