EIANZ October 2008EIANZ October 2008

U i t j d t dUsing expert judgement and uncertainty analysis for y y

biodiversity management

Mark Burgman

OutlineOutline

A taxonomy of uncertaintyMaking decisions under uncertainty: practicalMaking decisions under uncertainty: practical

applications of uncertainty analysis‘T ki ’ Si dh Ib• ‘Taking’ Sindh Ibex

• Habitat models • How many samples?• IUCN conservation assessmentsIUCN conservation assessments

‘Classical’ taxonomy of uncertaintyuncertainty

Variability is naturally occurring, unpredictable change, differences in parameters attributable to ‘true’ or ‘inherent’ variation. i.e. natural variation

Incertitude is lack of knowledge about parameters or models. i.e, measurement error, systematic error, model uncertainty, subjective judgement

Quantitative methodsQuantitative methods

Natural variation

• depends on unjustifiable assumptions• can’t deal with non-statistical uncertainty

RiskLack of knowledge• deals poorly with dependencies• creates an illusion of false precision

t t+1Time

Probability arithmetic, ‘classical’ decision theory, Monte Carlo

Linguistic uncertaintyg y

• Ambiguity – words have two or more meanings, and it is not clear which is meant.

V b d li• Vagueness – borderline cases.

• Underspecificity unwanted generality(Regan et al 2002)

• Underspecificity – unwanted generality.

• Context dependence a failure to specify• Context dependence – a failure to specify context.

UnderspecificityUnderspecificity

There is a 70% chanceThere is a 70% chance of rain.

Possible interpretations• rain during 70% of the day (time)• rain over 70% of the area (area) • on 70% of days such as this one, there will be at least some rain at a particular point (the weather station) (days such as this one)

UnderspecificityUnderspecificity

There’s a 70% chance of rain

Gi H t i d B k F l & K t ik l Ri kGigerenzer, Hertwig, van den Broek, Fasolo, & Katsikopoulos, Risk Analysis (in press)

Sindh IbexSindh Ibex

“We don’t know enough gto build a model”

How many can we take?

1

How many can we take?

0.75

0.5ob

abili

ty

0.25

Pro

00 2500 5000 7500 10000 12500

Population size

GAM

Model and parameter uncertainty

GLM

GAM

GARP

parameter uncertainty

GLMGARP

Florida scrub jayFlorida scrub jay

How do you ydecide that a species is absent from an area?

Overconfidence

Settlement depth Height to failure

0 8

1of a clay layer of an earth embankment

0.6

0.8

udgm

ents

0.2

0.4Ju

01 2 3 4 5 6 7 1 2 3 4 5 6 7

ExpertsExperts

(Hynes and Vanmarche 1977)

Insensitivity to sample sizeInsensitivity to sample size

Tversky and Kahneman (1974)

‘the law of small numbers’: most people (including experienced scientists) expect small samples to represent the pop lation from hich the ererepresent the population from which they were drawn to a degree that can only be assumed with much larger samplesmuch larger samples.

Challenge problemg p

H 0 1 h d d d b• How many 0.1 ha quadrats do you need to be 90% certain of encountering 95% of the

d l i 40 h ?woody plants in a 40 ha area?

(Baran, 2000)

Testing Risk Classificationsg

Mountain pygmy possum(B )

Desert tortoise(G h i ii)(Burramys parvus)

IUCN: Endangered Florida scrub jay(Aphelocoma

(Gopherus agassizii)Heritage: Apparently Secure (G4)

coerulescens coerulescen)Millsap: Threatened(Regan et al 2005)

IUCN Critically endangeredIUCN Critically endangered

IF Decline of >= 80% in 10 years or 3 generationsOR Range <100 km2 or occupied habitat <10 km2

ANDat least 2 of the following 3 conditions are met:1) severely fragmented or in 1 subpopulation1) severely fragmented or in 1 subpopulation2) continuing to decline3) fluctuations > 1 order of magnitude

OR b f t i di id l < 250OR number of mature individuals < 250AND

at least 1 of the following 2 conditions are met:1) >=25% decline in 3 years / 1 generation1) 25% decline in 3 years / 1 generation2) continuing decline and 1 subpopulation or <=50 per subpopulation

OR < 50 mature individualsOR >= 50% risk of extinction in 10 years / 3

generations.

IUCN

1

IUCN

12345678ec

ies

89

1011

Spe

121314

LR VU EN CRIUCN 2000

LR VU EN CR

(Regan et al 2005)

IUCN

1

IUCN

12345678ec

ies

89

1011

Spe

121314

LR VU EN CRIUCN 2000

LR VU EN CR

(Regan et al 2005)

IUCN

1

IUCN

Why?1234

• Unwillingness to make inferences

5678ec

ies • Parameter estimates spanning thresholds

V i i i li i j d (l )89

1011

Spe • Variation in qualitative judgements (language)

• Inconsistent logic and data entry mistakes121314

LR VU EN CR

Inconsistent logic and data entry mistakes (relatively rare)

IUCN 2000LR VU EN CR

(Regan et al 2005)

Expert judgement?Expert judgement?

A specialist knows... ‘a lot of things, … details … not covered in the general texts...’

‘I t thi ki f l t f li b t k l d d‘I am not thinking of general gut feeling but knowledge and experiences … in the light of which [the expert] can judge’

Appeals to authority

Irrefutable, unspecified wisdom and unassailable expert status generates a culture of technical controlculture of technical control

‘There is a naked assertion that the identity of the expert warrants acceptance of theof the expert warrants acceptance of the proposal’ (Walton).

Appeal to authority is legitimate only ifAppeal to authority is legitimate only if it can be challenged.