Just say NO to equifinality
2
HYDROLOGICAL PROCESSES Hydrol. Process. 21, 1979 – 1980 (2007) Published online 4 June 2007 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/hyp.6800 Just say NO to equifinality Stuart Hamilton* Environment Canada–Meteorological Service, 401 Burrard Street, Vancouver, British Columbia v6c3s5, Canada *Correspondence to: Stuart Hamilton, Environment Canada–Meteorological Service, 401 Burrard Street, Vancouver, British Columbia v6c3s5, Canada. E-mail: [email protected] Received 19 April 2007 Accepted 24 April 2007 Beven’s (2006) commentary on the uncertainty in hydrological modelling seems to have touched a bit of a nerve in the hydrological community. However, while there may be different definitions for uncertainty (Montanari, 2007), different approaches to quantifying it (Todini and Mantovan, in press), and calls for more effort in evaluating it (Hall et al., 2007), there is no denial that uncertainty is an inherent and unwelcome attribute of all predictions. This conversation about adapting to the rude reality of uncertainty could be misconstrued as an admission that we have reached a limit in hydrology, where improvements in the value of our predictions are less a function of theoretical advances or of further field observations and experimentation and more a function of advances in numeric methods or computing horsepower. Furthermore, young hydrologists overhearing this discussion may be excused if they mistook the reaction to Beven’s call for strategically advancing the science and ethics of hydrology as an argument for devising ever more elegant and sophisticated apologies for shortcomings in our hydrological predictions. This current focus on how to maximize the usefulness of models in the context of uncertainty is pragmatic. However, optimizing the present value of model output does not preclude a vision of the future where models are not only useful but are also truthful. Uncertainty is frustrating hydrologists in a way that is similar to the way that microbes frustrated medical science before the discovery of antibiotics. We need to find ways to move beyond mere treatment of symptoms to resolution of underlying pathologies. Like most hydrologists, I believed that I used parameters responsibly. I discovered my addiction to poorly defined parameter ranges almost by accident. A colleague wrote an R script, which made it easy to produce dotty-plots of my calibrations showing parameter identifiability. My first reaction was denial, that there must be a problem with the objective function, but gradually I have come to the realization that I have a history of parameter-abuse. Accepting that I suffer from equifinality was the first step on the long road to recovery. The signs of addiction are subtle but recognizable. Hydrographs that are just a little too good, where the uncertainty in model predictions is less than the uncertainty in the driving force data. Axis-scaling that is a little too compressed, hiding inconsistencies in the model fit. You may recognize these symptoms in your colleagues but deny that you too, keep a flask of parameter ranges in a desk drawer that you sip generously from when faced with a difficult hydrograph. The wrack and ruin of parameter abuse is wide-spread. We need to be particularly concerned about the effect our example has on young hydrologists who are just now trying to learn how to calibrate responsibly. We need to provide alternatives to parameter abuse. A priori parameter identification (e.g. Wagener and Wheater, 2006) and development of observation systems that will support increased under- standing and the development and testing of hydrologic theories and organizing principles (e.g. McDonnell et al., 2007) are the basis for the first of the four-pillars approach of Prevention, Treatment, Harm Copyright 2007 Crown in the right of Canada. Published by John Wiley & Sons, Ltd. 1979
-
Upload
stuart-hamilton -
Category
Documents
-
view
218 -
download
2
Transcript of Just say NO to equifinality
HYDROLOGICAL PROCESSESHydrol. Process. 21, 1979–1980 (2007)Published online 4 June 2007 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/hyp.6800
Just say NO to equifinality
Stuart Hamilton*Environment Canada–MeteorologicalService, 401 Burrard Street, Vancouver,British Columbia v6c3s5, Canada
*Correspondence to:Stuart Hamilton, EnvironmentCanada–Meteorological Service,401 Burrard Street, Vancouver,British Columbia v6c3s5, Canada.E-mail: [email protected]
Received 19 April 2007Accepted 24 April 2007
Beven’s (2006) commentary on the uncertainty in hydrological modellingseems to have touched a bit of a nerve in the hydrological community.However, while there may be different definitions for uncertainty(Montanari, 2007), different approaches to quantifying it (Todini andMantovan, in press), and calls for more effort in evaluating it (Hall et al.,2007), there is no denial that uncertainty is an inherent and unwelcomeattribute of all predictions.
This conversation about adapting to the rude reality of uncertaintycould be misconstrued as an admission that we have reached a limit inhydrology, where improvements in the value of our predictions are lessa function of theoretical advances or of further field observations andexperimentation and more a function of advances in numeric methodsor computing horsepower. Furthermore, young hydrologists overhearingthis discussion may be excused if they mistook the reaction to Beven’scall for strategically advancing the science and ethics of hydrology as anargument for devising ever more elegant and sophisticated apologies forshortcomings in our hydrological predictions.
This current focus on how to maximize the usefulness of modelsin the context of uncertainty is pragmatic. However, optimizing thepresent value of model output does not preclude a vision of the futurewhere models are not only useful but are also truthful. Uncertainty isfrustrating hydrologists in a way that is similar to the way that microbesfrustrated medical science before the discovery of antibiotics. We needto find ways to move beyond mere treatment of symptoms to resolutionof underlying pathologies.
Like most hydrologists, I believed that I used parameters responsibly.I discovered my addiction to poorly defined parameter ranges almost byaccident. A colleague wrote an R script, which made it easy to producedotty-plots of my calibrations showing parameter identifiability. My firstreaction was denial, that there must be a problem with the objectivefunction, but gradually I have come to the realization that I have ahistory of parameter-abuse. Accepting that I suffer from equifinalitywas the first step on the long road to recovery.
The signs of addiction are subtle but recognizable. Hydrographs thatare just a little too good, where the uncertainty in model predictions isless than the uncertainty in the driving force data. Axis-scaling that isa little too compressed, hiding inconsistencies in the model fit. You mayrecognize these symptoms in your colleagues but deny that you too, keepa flask of parameter ranges in a desk drawer that you sip generouslyfrom when faced with a difficult hydrograph.
The wrack and ruin of parameter abuse is wide-spread. We needto be particularly concerned about the effect our example has onyoung hydrologists who are just now trying to learn how to calibrateresponsibly. We need to provide alternatives to parameter abuse. Apriori parameter identification (e.g. Wagener and Wheater, 2006) anddevelopment of observation systems that will support increased under-standing and the development and testing of hydrologic theories andorganizing principles (e.g. McDonnell et al., 2007) are the basis forthe first of the four-pillars approach of Prevention, Treatment, Harm
Copyright 2007 Crown in the right of Canada.Published by John Wiley & Sons, Ltd. 1979
S. HAMILTON
Reduction, and Enforcement needed for a sustainablesolution to a global problem.
Treatment by means of group therapy in the form ofModeler’s Anonymous meetings is needed to providea non-judgmental environment for purging modellers’angst. Within this supportive environment we canexplore questions like: Have we crossed over fromModern Science, governed by the Laws of Nature,to Post-Modern Science, governed by Moore’s Law(Moore, 1965)? If so, what is the ethical basis forpost-modern science? Whereas modern science isbased on empirical evidence provided by repeatable,independent, observations, in post-modern sciencerepeating results proves nothing until we develop theability to make independent observations at a scalerelevant to our models.
Harm reduction can be achieved by promoting theuse of multi-criteria or multi-objective calibrationtechniques, or by qualifying output with methodssuch as GLUE (Beven and Binley, 1992). However,it would be naıve to think that such techniques are asolution to the underlying problem. Frequently, multi-criteria techniques rely on response function data thatare even more tenuous than discharge data and wecannot even agree on whether uncertainty estimationis helpful or not. Harm reduction is, by definition, achoice of the lesser of two evils.
Finally, the fourth pillar is crucial but at the sametime is also seen as the most harsh or cruel ofall of the pillars. The goal of enforcement is notto criminalize hydrologic modellers but rather toensure accountability. We enable the persistence ofparameter abuse by being soft in the peer-review
process. We need to send a strong message that resultsbased on parameter abuse will not be published.
Parameters that map watershed function from ascale that is relevant to any given drop of waterto a scale relevant to human uses of water areintegral to our work. Given that we cannot measureparameters explicitly, we must estimate them byinference. Research that assists in defining what anygiven parameter value should be (as a subset of whatcould be), gives hope to those of us caught in thedespair of having too many degrees of freedom.
References
Beven K. 2006. On undermining the science? Hydrological Processes20: 3141–3146.
Beven KJ, Binley AM. 1992. The future of distributed models: modelcalibration and uncertainty prediction. Hydrological Processes 6:279–298.
Hall J, O’Connell E, Ewen J. 2007. On not undermining the science:coherence, validation and expertise. Discussion of Invited Commen-tary by Keith Beven. Hydrological Processes 21: 985–988.
McDonnell JJ, Sivapalan M, Vache K, Dunn S, Grant G, Haggerty R,Hinz C, Hooper R, Kirchner J, Roderick ML, Selker J, Weiler M.2007. Moving beyond heterogeneity and process complexity: a newvision for watershed hydrology. Water Resources Research(in press).
Montanari A. 2007. What do we mean by uncertainty? HydrologicalProcesses 21: 841–845.
Moore G. 1965. Cramming more components onto integrated cir-cuits. Electronics 38(8): 114–116.
Todini E, Mantovan P. 2006. Comment on: ‘On undermining thescience?’ by Keith Beven. Hydrological Processes 20: 3141–3146.
Wagener T, Wheater HS. 2006. Parameter estimation and regional-ization for continuous rainfall-runoff models including uncertainty.Journal of Hydrology 320: 132–154.
Copyright 2007 Crown in the right of Canada.Published by John Wiley & Sons, Ltd. 1980 Hydrol. Process. 21, 1979–1980 (2007)
DOI: 10.1002/hyp
