The Quest for Consistently Incorrect Models

Nick Brown

21-08-2012

A Insert presentation title and date here. (A = slide number)

1st

3rd

1st

3rd

1st

4th

3rd

1st5th

4th

3rd

5th1st

4th

5th

4th

1st5th

4th

3rd

1st5th

4th

1.5 Million People

6000 km2

1.5 Million People

6000 km2

7500 Staff

7500 Staff

6 modellers

AC Stormwater Unit

• 140 people

• Projects

• Operations

• Consents

• Planning

– Predict flooding

– $70M/annum capital works to address flooding

b1

Slide 14

b1 brownn2, 19/08/2012

3rd July 2012

• 100 year event in Auckland

ARI

10 Minute

Duration

ARI

20 Minute

Duration

ARI

30 Minute

Duration

ARI

60 Minute

Duration

ARI

120 Minute

Duration

ARI

180 Minute

Duration

3 July 2012

Recent 100 Year Events

3 July 2012

28 Jan 2011

Recent 100 Year Events

3 July 2012

28 June 2000

28 Jan 2011

Recent 100 Year Events

3 July 2012

28 June 2000

28 Jan 2011

30 Dec 1988

Recent 100 Year Events

3 July 201222 May 1985

28 June 2000

28 Jan 2011

22 May 1985

30 Dec 1988

Recent 100 Year Events

3 July 201222 May 1985

30 June 1979

28 June 2000

28 Jan 2011

22 May 1985

30 Dec 1988

Recent 100 Year Events

Expect 100yr Event every

3 Years

3 July 201222 May 1985

30 June 1979

28 June 2000

28 Jan 2011

22 May 1985

30 Dec 1988

Recent 100 Year Events

Expect 100yr Event every

3 Years

Expect significant Event >20 times per year

The Modelling

Model Reviews

• ~50 Models reviewed

• Many many problems

• <1 in 3 useable

Model Reviews

3 Categories

• Less than Ideal

• Theoretically Correct

• Good model – wrong scope

Model Reviews

3 Categories

• Less than Ideal

• Theoretically Correct

• Good model – wrong scope

Less than Ideal

• Pervious & Impervious - same runoff

• Catchment Boundary wrong

• Model extents wrong – starts below area of interest

• Upstream storage not considered

• No reticulation included

• Schematisation poor

• Survey data not used

Pond outlets

1m1

05

0 m

m

1m1

05

0 m

m

D=1050 mm

C=3300 mm

D=1050 mm

C=3300 mm

Reality = 1050mm Orifice Modelled as 3300mm Weir

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Weir Discharge (l/s)

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

90% more flow

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

190% more flow

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

280% more flow

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

380% more flow

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

470% more flow

0

5,000

10,000

15,000

20,000

25,000

30,000

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Water Level (m)

Dis

cha

rge

(l/

s)

Horizontal Orifice Weir Discharge (l/s)

2m less head

2D Rapid Flood Hazard

200 m3/s too low

Calibration

Gauge

Gauge

Calibrated travel time 2 hrs

Gauge

Calibrated travel time 2 hrs

Gauge

Calibrated travel time 2 hrs

Gauge

Calibrated travel time 2 hrs

Calibrated travel time 0.5 hrs

Gauge

Gauge

Calibrated travel time 0.2 hrs

Gauge

Calibrated travel time 0.2 hrs

10 m/s

Gauge

Calibrated travel time 0.2 hrs

10 m/s

10 m/s for 9.58 Seconds

Not for 600 Seconds

Hydrographs - Hydrology

Model Cross Sections

25.0

25.5

26.0

26.5

27.0

27.5

28.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0Chainage

Ele

va

tio

n

Model Cross Section

25.0

25.5

26.0

26.5

27.0

27.5

28.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0Chainage

Ele

va

tio

n

Model Cross Section

Surveyed

25.0

25.5

26.0

26.5

27.0

27.5

28.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0Chainage

Ele

va

tio

n

Model Cross Section

Surveyed

Water Level

Storage

0

1000

2000

3000

4000

5000

6000

7000

23.4 23.6 23.8 24 24.2 24.4 24.6 24.8 25 25.2 25.4

elevation m

sto

rag

e v

olu

me m

3

0

1000

2000

3000

4000

5000

6000

7000

23.4 23.6 23.8 24 24.2 24.4 24.6 24.8 25 25.2 25.4

elevation m

sto

rag

e v

olu

me m

3

0

1000

2000

3000

4000

5000

6000

7000

23.4 23.6 23.8 24 24.2 24.4 24.6 24.8 25 25.2 25.4

elevation m

sto

rag

e v

olu

me m

3

75% more

volume

Culverts

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0

Flow (m3/s)

He

ad

Wa

ter

Dep

th (

m)

MOUSE ModelModel

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0

Flow (m3/s)

Hea

d W

ate

r D

ep

th (

m)

MOUSE Model Culvert ManualModel

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0

Flow (m3/s)

Hea

d W

ate

r D

ep

th (

m)

MOUSE Model Culvert Manual

Flow 11.5 m3/s

MOUSE model 3.5 m

ModelModel

Model

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0

Flow (m3/s)

Hea

d W

ate

r D

ep

th (

m)

MOUSE Model Culvert Manual

Flow 11.5 m3/s

MOUSE model 3.5 m

Culvert Manual 6.0 m

Difference 2.5 m

2.5m

Model

Model

10 m/s velocity

MOUSEModel

4 m/s velocity

Culvert Manual

Velocity & Inlet Losses

HERE

HERE

HERE

HERE

Normal Manhole

Wrong Node Type

• 2D outfall

• outlet

Theoretically Correct

10

11

12

13

14

15

16

17

18

19

0 10 20 30 40 50 60 70

Without Bank Marker, n = 0.08

10

11

12

13

14

15

16

17

18

19

0 10 20 30 40 50 60 70

Without Bank Marker, n = 0.08

With Bank Marker, n = 0.08

10

11

12

13

14

15

16

17

18

19

0 10 20 30 40 50 60 70

Without Bank Marker, n = 0.08

With Bank Marker, n = 0.08

With Bank Marker and

Distributed Roughness

Channel n = 0.04 &

Floodplain n = 0.12

Stream Conveyance Estimation

11

12

13

14

15

16

17

18

19

0 1000 2000 3000 4000 5000 6000 7000

Stream Conveyance

Ele

va

tio

n

Without Marker, n = 0.08

Stream Conveyance Estimation

11

12

13

14

15

16

17

18

19

0 1000 2000 3000 4000 5000 6000 7000

Stream Conveyance

Ele

va

tio

n

Without Marker, n = 0.08 With Marker, n = 0.08

Stream Conveyance Estimation

11

12

13

14

15

16

17

18

19

0 1000 2000 3000 4000 5000 6000 7000

Stream Conveyance

Ele

va

tio

n

Without Marker, n = 0.08With Marker, n = 0.08With Marker and Distributed Roughness, Channel n = 0.04 & Floodplain n = 0.12

Stream Conveyance Estimation

11

12

13

14

15

16

17

18

19

0 1000 2000 3000 4000 5000 6000 7000

Stream Conveyance

Ele

va

tio

n

Without Marker, n = 0.08With Marker, n = 0.08With Marker and Distributed Roughness, Channel n = 0.04 & Floodplain n = 0.12

2 ¼ > Flow @ same Elevation

10

11

12

13

14

15

16

17

18

19

0 10 20 30 40 50 60 70

Without Bank Marker, n = 0.08

27 Habitable

Floors Flooded

10

11

12

13

14

15

16

17

18

19

0 10 20 30 40 50 60 70

Without Bank Marker, n = 0.08

With Bank Marker, n = 0.08

9 Habitable

Floors Flooded

10

11

12

13

14

15

16

17

18

19

0 10 20 30 40 50 60 70

Without Bank Marker, n = 0.08

With Bank Marker, n = 0.08

With Bank Marker and

Distributed Roughness

Channel n = 0.04 &

Floodplain n = 0.12

0 Habitable

Floors Flooded

Good Model Wrong Scope

• Grid to large for ground slope

126 m

116 m

10m Elevation change across grid

126 m

116 m

10m Elevation change across grid

121.2 m Predicted Flood Level

Review Summary

• Easily double or halve flow

• Results precise but not accurate

• Floodplains wrong by +/- 3m

• Results cannot be used to set building floor levels

• Results cannot be used to scope projects

The Plan

• Outsource commodity work

• Specifications are for commodity work

• Keep high risk work in-house

• Review all models (external & internal)

• Create Specification & standard review method

• Rolling 10-15 year programme to model all catchments

Model Quality

Best PracticePoor

Model Quality

Best Practice

a

Poor

Consultants

Hydrology

Model Quality

Best Practice

a

Poor

Consultants

HydrologyHydraulics

Model Quality

Best Practice

a

bbb

Poor

Consultants

Model Quality

Best Practice

a

bcbb

Poor

Consultants

Model Quality

Best Practice

a

d

bcbb

Poor

Consultants

Model Quality

Best Practice

a

d

e

bcbb

Poor

Consultants

Model Quality

Best Practice

a

d

e

Specifications

bcbb

Poor

Consultants

Model Quality

Best Practice

a

d

e

Specifications

bcbb

Poor

Consultants

Model Quality

Best Practice

a

d

e

Specifications

bcbb

Poor

Consultants

Model Quality

Best Practice

a

d

e

Specifications

bcbb

Poor

Consultants

Loss

Model Quality

Best Practice

a

d

e

Specifications

bcbb

Poor

Consultants

Loss

GAIN

Model Quality

Best Practice

a

d

e

Specifications

bcbb

Poor

Consultants

Loss

GAIN

Minimum NOT Maximum Standard

Specifications

• Mix of Specification and Guideline

– Instruction and Advice

• Basis for modelling contracts

• Details modelling process

• Updated as needed

• Basis for commodity work

• Allows best practice to be captured so you know which model was produced using which understanding of best practice

What do they Cover

• Flood Modelling

• Rainfall/Runoff method

• Hydraulics

• Report structure

• Deliverables

• Modelling Process

• Content and look of maps

• Includes standard council model review form

Summary

• We don’t know everything

• We do know something

• We don’t know necessarily what is right

• Specification sets a minimum standard

• To manage a modelling and capital works programme need consistency

• Provide consistently incorrect models

Questions

What they don’t cover

• Continuous simulation

• Modelling of non-extreme events

• LID modelling

• Stream erosion

• Geomorpholigical change assessment

• Blockage scenarios

What they are weak on

• Options analysis

• Flood Damage assessment

• Flood Hazard Assessment

• Report mapping format