January 2005 1

Harmonising Quality Assurance in model based catchment and river basin

management: The HarmoniQuA approach

Dr Gareth Old

CEH Wallingford, UK

January 2005 2

HarmoniQuA partners

WU, The Netherlands GEUS, Denmark NTUA, Athens, Greece CEH-Wallingford, UK WL | Delft Hydraulics,The Netherlands Cemagref, France BfG, Germany SMHI, Sweden VITUKI Plc, Hungary Uni Dortmund, Germany LNEC, Portugal DHI Hydroinform a.s.,

Czech Republic

January 2005 3

Structure

Modelling in the UK Modelling problems and existing Quality Assurance The HarmoniQuA approach: Modelling Support Tool (MoST) Evaluation of the benefits of using MoST

January 2005 4

Modelling in the UK

Modelling is widely used in UK river management Commissioned by EA/SEPA or the WSCs/SW Conducted by both large ‘framework consultants’ and small

companies Domains: Groundwater; precipitation-runoff; hydrodynamics;

flood forecasting; surface water quality; ecology (socio-economics)

Quality Assurance: Technical audit; Problems often identified

January 2005 5

Modelling problems

Ambiguous Terminology: lack of mutual understanding between key-players. Multidisciplinary projects!

Malpractice: input data; model set-up; calibration and validation; uncertainty; scope

Data: poor documentation of quality and sourceProcess knowledge: Insufficient; users and clients unaware of

assumptions and conceptualisationCommunication: Miscommunication between modelling team

and stakeholders

January 2005 6

Modelling Problems

Using results for decision making: Discarding uncertainty, credibility of modelling

Transparency of the modelling process: Lack of documentation

Socio-economic and political issues: Fail to integrate domains, socio-economic issues and stakeholders

January 2005 7

Existing Quality Assurance Existing guidance:

• WaPUG Code of Practice – UK• Groundwater Resources Modelling Guidance Note - UK• Good Modelling Practice Handbook - Netherlands• Murray-Darling groundwater flow modelling - Australia• Bay-Delta modelling protocol - USA

Contract specificationMany UK modelling study contracts have associated detailed specifications of tasks, procedures and standards.

Quality Assurance ‘Tools’None identified. However, some modelling packages provide internal audit trailsExisting Guidance :Paper based, Regional/National and domain specific

January 2005 8

The HarmoniQuA Approach

Developed the Modelling Support Tool (MoST) to provide

a user friendly Quality Assurance framework for modelling

teams

• Guide: to ensure a model has been properly applied;• Monitor: to record decisions, methods and data used in

these tasks;• Report: to provide reports suitable for managers/clients,

modellers, auditors, stakeholder and the general public.

January 2005 9

The HarmoniQuA Approach Support for modelling 7 domains (groundwater, precipitation-

runoff, river hydrodynamics, flood forecasting, water quality, ecology and socio-economics). Also, multi-domain studies

For several user types (water managers, modellers, auditors, stakeholders and members of the public)

Modelling with different levels of complexity

Modelling by teams

Glossary

Step 1

Model Study Plan

Step 2

Data and Conceptualisation

Step 3

Model Setup

Step 4

Calibration and Validation

Step 5

Simulation and Evaluation

January 2005 11

Model StudyPlan

Data andConceptualisation

ModelSet-up

Calibrationand

Validation

Simulation andEvaluation

Model TeamSingle/multiple domain

MoST

Reporting and client review take place in each step

User

Knowledge BaseGuidelinesSoftware capabilitiesGlossaryDomains:

GroundwaterPrecipitation-runoffHydrodynamicsFlood forecastingWater qualityBiota (ecology)Socio-economics

GuidanceGeneric + specific for:- model domain- user- job complexity

ReportingSpecific for typesof users

MonitoringGeneric + specific for:- model domain- user- job complexity

AdviseFrom previousmodel projects

ModelArchive Model journal, Project A

Model journal, Project B

Model journal, Project C

Model journal, Project D

HarmoniQuA MoST overview

January 2005 12

The HarmoniQuA WorkPackages

Knowledge on modellinggeneric + subdomain specificglossary

Knowledge Base devlopmentModelling Support Tool

Testing tool of WP 2Testing knowledge of WP1

WP 1

WP 2 WP 3

DisseminationTraining MaterialExploitation

WP 4Students

Public

Existingknowledge

Experts

Professionals

January 2005 13

Using HarmoniQuA MoST

Project Initiation: Options available Conducting and monitoring tasks, activities and methods Consulting guidance The interactive glossary Reporting

January 2005 34

Evaluation of the benefits of using MoST

MoST supports key user types through the modelling process Standardised european guidance for 7 domains (plus generic) Supports integrated multi-domain studies of various levels of

complexity Supports modelling in teams Flexible system: • List of tasks and activities can be customised• Can de-select strictly follow guidance

January 2005 35

Evaluation of the benefits of using MoST

MoST provides users with guidance: Synthesised from published documents and expert consultations

MoST monitors and records the actions of users: Providing an auditable trail that is reproducible

MoST provides standard reports customised to specific requirements

January 2005 36

Evaluation of the benefits of using MoST

Remaining problems:

• Model code selection (see BMW project: http://www.ymparisto.fi/)• Coupling of models (see HarmonIT project: http://www.harmonit.org/)

January 2005 37

Conclusion

MoST enhances the credibility of modelling studies by making user friendly Quality Assurance procedures readily available to its users

Modelling Support Tool (MoST) and integrated ‘Knowledge Base’Available from public website

First release of MoST: Expect it to be developed Training material is available on the public website: It includes

presentations, computer screen movies and case studies Please Download, Experiment and Comment!

January 2005 38

HarmoniQuA MoST

Public Website: www.HarmoniQuA.org

Email: HarmoniQuA@ceh.ac.uk