Fundamentals of Risk Fundamentals of Risk ManagementManagement

The Ubiquity of Risks in The Ubiquity of Risks in Today’s Socio-technical Today’s Socio-technical

SystemsSystems Automobile and plane crashesAutomobile and plane crashes Toxic chemical spills and explosionsToxic chemical spills and explosions Nuclear accidentsNuclear accidents Water and food security (contamination, shortage)Water and food security (contamination, shortage) Genetic manipulationGenetic manipulation Infectious diseases (AIDS, bird flu)Infectious diseases (AIDS, bird flu) Global climatic changeGlobal climatic change Ozone depletionOzone depletion Extinction of speciesExtinction of species Oil dependenceOil dependence War and terrorismWar and terrorism On and on and on………On and on and on………

The Rational Actor StandardThe Rational Actor Standard

Choice of actionsChoice of actions Range of possible outcomesRange of possible outcomes Assign probabilities to outcomesAssign probabilities to outcomes Order outcomes according to Order outcomes according to

preferencespreferences Select optimal actionSelect optimal action Expected value calculationsExpected value calculations

The Rational Actor “acts”The Rational Actor “acts”

Action is “…. purposeful behavior. Or we Action is “…. purposeful behavior. Or we may say, action is will put into operation and may say, action is will put into operation and transformed into an agency, is aiming at transformed into an agency, is aiming at ends and goals, is the ego’s meaningful ends and goals, is the ego’s meaningful response to stimuli and to the conditions of response to stimuli and to the conditions of its environment, is a person’s conscious its environment, is a person’s conscious adjustment to the state of the universe that adjustment to the state of the universe that determines his life.” determines his life.”

Ludwig Von Mises, Ludwig Von Mises, Human ActionHuman Action (1949) : p. (1949) : p. 1111

Human action vs. Human Human action vs. Human behaviorbehavior

Action has Action has reasonsreasons, behavior has , behavior has causescauses Action: “institution,” “will,” “duty,” Action: “institution,” “will,” “duty,”

“responsibility”, “obligation”, “right”, “responsibility”, “obligation”, “right”, “constitution”, “contract”, “custom”, “constitution”, “contract”, “custom”, “traditions,” and “collective choices.” “traditions,” and “collective choices.”

Behavior: “social organization,” “behavioral Behavior: “social organization,” “behavioral measurement,” “human relations,” measurement,” “human relations,” “reliability,” “validity,” “organizational “reliability,” “validity,” “organizational design,” “attitude,” “human performance,” design,” “attitude,” “human performance,” “power,” “performance evaluation,” and “power,” “performance evaluation,” and “leadership.” “leadership.”

Risk Evaluation vs. Risk Evaluation vs. Management: A Proposed Management: A Proposed

DistinctionDistinction Risk Evaluation: the cognitive Risk Evaluation: the cognitive

process of determining or specifying process of determining or specifying a risk prior to making a behavioral a risk prior to making a behavioral commitmentcommitment

Risk Management: the cognitive and Risk Management: the cognitive and behavioral process of designing and behavioral process of designing and implementing controls on risky implementing controls on risky systemssystems

Evolution of Risk Evolution of Risk ManagementManagement

Early humans: risks small scale, local; Early humans: risks small scale, local; management relatively straightforwardmanagement relatively straightforward

Technology led to evolution of systems Technology led to evolution of systems with larger scale, greater complexitywith larger scale, greater complexity

Risk evaluation and management, today Risk evaluation and management, today define the key analytical lens for define the key analytical lens for attempting to anticipate the consequences attempting to anticipate the consequences of human action in social and of human action in social and environmental systemsenvironmental systems

Definition of a SystemDefinition of a System

““A section of the universe which an A section of the universe which an observer chooses to separate in thought observer chooses to separate in thought from the rest of the universe for the from the rest of the universe for the purpose of considering and discussing the purpose of considering and discussing the various changes which may occur within it various changes which may occur within it under various conditions” (J. W. Gibbs)under various conditions” (J. W. Gibbs)

Usual distinction made between (1) an Usual distinction made between (1) an observed object, (2) a perception of the observed object, (2) a perception of the object, and (3) a model of the object object, and (3) a model of the object

A Generalized Structural Graphic of a System

Process

Outputi

Outputj

Inputs

Inputt

Inputu

Feedback

Environmental boundary

Environment

The Concept of a StructureThe Concept of a Structure

A generic notion: the “structure” of a A generic notion: the “structure” of a building, a brain, an organization, a building, a brain, an organization, a society, a research design, a decision society, a research design, a decision process, a “system”process, a “system”

A discernable (persistent) pattern or set of A discernable (persistent) pattern or set of relations that connect a set of terms, relations that connect a set of terms, nodes, or objects nodes, or objects

Formal bodies of scientific knowledge have Formal bodies of scientific knowledge have a “structure”a “structure”

Theory

Assumptions

Methodology

Hypothesis

Applications

Generic Structure of a Body of Scientific Knowledge:(Modified from Warfield, 1990)

denotes a “steering” relationship

Systems concepts provide a Systems concepts provide a means of integrating basic means of integrating basic

tools of thinkingtools of thinking MemoryMemory AssociationAssociation Pattern discernment and recognitionPattern discernment and recognition ReasonReason ExperienceExperience InventionInvention ExperimentationExperimentation IntuitionIntuition

Risk Management through Risk Management through Systems “Control”Systems “Control”

Choosing the inputs to a system so as to Choosing the inputs to a system so as to make the state or outputs change in (or make the state or outputs change in (or close to) some desired way. (Arbib) close to) some desired way. (Arbib)

A constraining effect on a variableA constraining effect on a variable A directing influence on the behavior of a A directing influence on the behavior of a

system, or the setting of the parameters of system, or the setting of the parameters of a system. a system.

Any one-way communication, which by Any one-way communication, which by definition conditions a receiver's behavior definition conditions a receiver's behavior in some respects, involves control in some respects, involves control (Principia Cybernetica Web) (Principia Cybernetica Web)

The Concept of a ConstraintThe Concept of a Constraint

Limitations that preclude certain Limitations that preclude certain actionsactions

May preclude achievement of certain May preclude achievement of certain objectivesobjectives

Actions not precluded are “feasible”Actions not precluded are “feasible” Elastic (removable) vs. inelastic (not Elastic (removable) vs. inelastic (not

removable) constraintsremovable) constraints ““Binding” vs. “nonbinding” constraintsBinding” vs. “nonbinding” constraints

                                                                                                                                    

Leibig’s Law of the MinimumLeibig’s Law of the Minimum

Large Scale SystemsLarge Scale Systems

Costs Costs Numbers of people Numbers of people Geographical or other extent of influenceGeographical or other extent of influence Volume of information required to describe Volume of information required to describe

what is happening within the systemwhat is happening within the system Number of interaction between componentsNumber of interaction between components Size of the potential for disasterSize of the potential for disaster Extent of consequences of systems failure.Extent of consequences of systems failure.

The Structure of a ConceptThe Structure of a Concept

Underconceptualization: a problem in Underconceptualization: a problem in managing risk in large scale, complex managing risk in large scale, complex systemssystems

Concept: a cognitively significant word or Concept: a cognitively significant word or phrase used in a proposition to represent phrase used in a proposition to represent some aspect(s) of a systemsome aspect(s) of a system

The division of a concept: C is a concept, The division of a concept: C is a concept, D(C) is a division of C if D(C) partitions C D(C) is a division of C if D(C) partitions C into components (Warfield 1990) into components (Warfield 1990)

Complexity mayComplexity may bebe conceived conceived as:as:

“…“…..the predicament presented to ..the predicament presented to the human mind when it encounters the human mind when it encounters conceptualizations that exceed its conceptualizations that exceed its unaided power to assess or unaided power to assess or evaluate.” evaluate.”

John N. WarfieldJohn N. Warfield, , Understanding Understanding Complexity: Thought and BehaviorComplexity: Thought and Behavior

Structural Structural UnderconceptualizationUnderconceptualization

When managing risk in complex systems, When managing risk in complex systems, the formal, logical structure of the system the formal, logical structure of the system is usually not specified. Why?is usually not specified. Why?

IgnoranceIgnorance Decision-makers have diverse beliefs Decision-makers have diverse beliefs

about the situation (often do not even about the situation (often do not even share same linguistic domain)share same linguistic domain)

Organizational linguisticsOrganizational linguistics Differing valuesDiffering values

The Genesis of ValueThe Genesis of Value

What are values?What are values? Are values controlled by hereditary Are values controlled by hereditary

instincts, genes, and personalities?instincts, genes, and personalities? Are individuals’ values determined by Are individuals’ values determined by

society?society? Are any values autonomously chosen?Are any values autonomously chosen? Can a unified theory of risk evaluation ever Can a unified theory of risk evaluation ever

be obtained without answering these be obtained without answering these questions?questions?

A Key to Coherence in A Key to Coherence in Managing Risk in Complex Managing Risk in Complex

SystemsSystems Recognition that there is an unavoidable trade Recognition that there is an unavoidable trade

off between simple assumptions and off between simple assumptions and sophisticated models, and sophisticated sophisticated models, and sophisticated assumptions and simple modelsassumptions and simple models

The former is more the norm (e.g. neoclassical The former is more the norm (e.g. neoclassical and price theoretic models of economic and price theoretic models of economic systems all assume homo economicus; Marx’s systems all assume homo economicus; Marx’s model assumed communist man)model assumed communist man)

The latter is generally superior when dealing The latter is generally superior when dealing with large scale complex systems with large scale complex systems

Disparate Risks and Disparate Risks and CatastrophesCatastrophes

Disparate risks: a serious imbalance among Disparate risks: a serious imbalance among outcomes (no matter probabilities)outcomes (no matter probabilities)

A potential “catastrophe” exists when risks A potential “catastrophe” exists when risks are so disparate that one outcome is are so disparate that one outcome is “incomparably” greater than the alternatives“incomparably” greater than the alternatives

Rescher’s principle: All else equal, an action Rescher’s principle: All else equal, an action that might lead to a catastrophe makes that that might lead to a catastrophe makes that action ineligible (avoid any real catastrophe action ineligible (avoid any real catastrophe at any ordinary cost).at any ordinary cost).

Insurance Against Insurance Against CatastropheCatastrophe

Insurance as a device for reducing Insurance as a device for reducing the prospect of substantial loss (a the prospect of substantial loss (a trade of hazards)trade of hazards)

The value of (price the rationale The value of (price the rationale actor is willing to pay for) an actor is willing to pay for) an insurance policy depends upon the insurance policy depends upon the alternative actions in the choice set.alternative actions in the choice set.

Risk DilemmasRisk Dilemmas

Sometimes, all of the alternative Sometimes, all of the alternative feasible actions entail catastrophefeasible actions entail catastrophe

In this case, the relative size of the In this case, the relative size of the catastrophe is the paramount factorcatastrophe is the paramount factor

The distinction between catastrophe The distinction between catastrophe and acceptable risk is a judgmental and acceptable risk is a judgmental issue (example of Japanese attack on issue (example of Japanese attack on Pearl Harbor, page 90, Rescher)Pearl Harbor, page 90, Rescher)

A “Fundamental Principle” of A “Fundamental Principle” of the Theory of Riskthe Theory of Risk

Risk acceptability is always Risk acceptability is always comparative, relative to the options, comparative, relative to the options, and thus unacceptability is never and thus unacceptability is never absolute, only relativeabsolute, only relative

Implicit: the risk-free option is, at Implicit: the risk-free option is, at times, not an element in the choice times, not an element in the choice setset

Uncertainty Uncertainty

Uncertainty: indetermination through Uncertainty: indetermination through ignorance of otherwise, of some of the ignorance of otherwise, of some of the characterizing elements of a risk situationcharacterizing elements of a risk situation

Rescher’s three modal taxonomy:Rescher’s three modal taxonomy:

(a) Probability uncertainty (P-uncertainty)(a) Probability uncertainty (P-uncertainty)

(b) Result uncertainty (R-uncertainty)(b) Result uncertainty (R-uncertainty)

(c) Outcome uncertainty (O-uncertainty)(c) Outcome uncertainty (O-uncertainty)

Plato: lack of trust in sense experience Plato: lack of trust in sense experience as a basis for knowledgeas a basis for knowledge

Rene Descartes: systematic doubt, Rene Descartes: systematic doubt, and the search for a priori certainty and the search for a priori certainty

David Hume: the problem of inductionDavid Hume: the problem of induction Wittgenstein: language gamesWittgenstein: language games Karl Popper: InductionKarl Popper: Induction

A brief history of A brief history of uncertainty uncertainty

P-uncertaintyP-uncertainty

Probabilities cannot be reliably Probabilities cannot be reliably determineddetermined

Insufficient data to establish relative Insufficient data to establish relative frequenciesfrequencies

Insufficient theories, models, or data to Insufficient theories, models, or data to establish likelihoods (e.g. nuclear casks)establish likelihoods (e.g. nuclear casks)

Subjective probabilities Subjective probabilities What probabilities does one use when What probabilities does one use when

evaluating the risk of nuclear war?evaluating the risk of nuclear war?

R-uncertaintyR-uncertainty

One faces outcomes with One faces outcomes with imponderable valuesimponderable values

Uncertainty about how to evaluate Uncertainty about how to evaluate an outcomean outcome

How did Truman value the loss of life How did Truman value the loss of life when deciding to bomb Japan?when deciding to bomb Japan?

O-UncertaintyO-Uncertainty

Scientific knowledge is inadequate to Scientific knowledge is inadequate to describe the nature of the hazarddescribe the nature of the hazard

Cannot fully specify range of Cannot fully specify range of outcomesoutcomes

What outcomes will follow from the US What outcomes will follow from the US military invasion of Iraq?military invasion of Iraq?

What is the national security risk What is the national security risk attributable to US dependence upon attributable to US dependence upon Mid-Eastern oil?Mid-Eastern oil?

The Ecology of Uncertainty: Sources, The Ecology of Uncertainty: Sources, Indicators, and Strategies for Indicators, and Strategies for

Informational UncertaintyInformational Uncertainty Schunn, Kirschenbaum, and Trafton: effort Schunn, Kirschenbaum, and Trafton: effort

to develop one taxonomy each for to develop one taxonomy each for sources, indicators and strategiessources, indicators and strategies

Focus upon evaluation and management Focus upon evaluation and management of uncertainty in three domainsof uncertainty in three domainsa) meteorological forecasting, a) meteorological forecasting, b) fMRI data analysisb) fMRI data analysis

c) submarine operationsc) submarine operations Interest in informational uncertaintyInterest in informational uncertainty

InformationInformation

Shannon, “that which reduces uncertainty”Shannon, “that which reduces uncertainty” Bateson, “a difference that makes a Bateson, “a difference that makes a

difference” difference” “…“…. equivalent of or the capacity of . equivalent of or the capacity of

something to perform organizational work, something to perform organizational work, the difference between two forms of the difference between two forms of organization or between two states of organization or between two states of uncertainty before and after a message has uncertainty before and after a message has been received” Principia Cyberneticabeen received” Principia Cybernetica

Elementary unit in information theory: “bit,” Elementary unit in information theory: “bit,” a difference (e.g. between a 0 and a 1)a difference (e.g. between a 0 and a 1)

Sources of Informational Sources of Informational UncertaintyUncertainty

Physics uncertaintyPhysics uncertainty Computational uncertainty Computational uncertainty Visualization uncertaintyVisualization uncertainty Cognitive uncertaintyCognitive uncertainty

““Physics Uncertainty” is Physics Uncertainty” is uncertainty attributable to:uncertainty attributable to:

Absence of a signalAbsence of a signal Noise/bias in the signalNoise/bias in the signal Noise/bias in the way the signal is Noise/bias in the way the signal is

transducedtransduced

““Computational Uncertainty” is Computational Uncertainty” is uncertainty attributable to:uncertainty attributable to:

Unpredicted [perhaps unpredictable?] Unpredicted [perhaps unpredictable?] future changes in the systemfuture changes in the system

Statistical artifacts (e.g. the imposition of Statistical artifacts (e.g. the imposition of linearity, statistical assumptions, linearity, statistical assumptions, smoothing operations on data)smoothing operations on data)

Fast and cheap: elaborate algorithms that Fast and cheap: elaborate algorithms that require lots of time to run, even on high require lots of time to run, even on high powered computers (more time than is powered computers (more time than is available to the decision-maker) available to the decision-maker)

““Visualization Uncertainty” is Visualization Uncertainty” is uncertainty attributable to:uncertainty attributable to:

Limits on the information that can be Limits on the information that can be logically derived from a structural logically derived from a structural graphicgraphic

Sometimes information is missing from Sometimes information is missing from the structural graphicthe structural graphic

Sometimes multiple dimensions are Sometimes multiple dimensions are represented by composite indicatorsrepresented by composite indicators

Sometimes there are multiple graphics Sometimes there are multiple graphics that contain conflicting informationthat contain conflicting information

““Cognitive Uncertainty” is Cognitive Uncertainty” is uncertainty attributable to:uncertainty attributable to:

perceptual error (e.g. in encoding, perceptual error (e.g. in encoding, storing, retrieving, or processing visual storing, retrieving, or processing visual information from the measurement information from the measurement devices)devices)

memory encoding error memory encoding error information overloadinformation overload retrieval errorretrieval error background knowledge errorbackground knowledge error skill errorskill error

Uncertainty TradeoffsUncertainty Tradeoffs

Tradeoffs evidently exist in which Tradeoffs evidently exist in which reductions in one source of uncertainty reductions in one source of uncertainty come at the cost of increases in other come at the cost of increases in other sourcessources

Physical and computational uncertaintyPhysical and computational uncertainty Visualization and cognitive uncertaintyVisualization and cognitive uncertainty Memory and perception within cognitive Memory and perception within cognitive

uncertaintyuncertainty

Indicators of UncertaintyIndicators of Uncertainty

See no or unusually weak patternsSee no or unusually weak patterns See an impossible patternSee an impossible pattern See a pattern that mismatches a See a pattern that mismatches a

known state of the worldknown state of the world See a pattern that is inconsistent See a pattern that is inconsistent

across data sourcesacross data sources

Strategies for Dealing with Strategies for Dealing with UncertaintyUncertainty

Check likely errorsCheck likely errors Focus on reliable sourcesFocus on reliable sources Adjust for known deviations from truthAdjust for known deviations from truth Average across sourcesAverage across sources Acquire more dataAcquire more data Gather better dataGather better data Bound uncertainty (satisfice)Bound uncertainty (satisfice)

Implications of “Deep Implications of “Deep Similarity” Between DomainsSimilarity” Between Domains

Taxonomies may be generalizable to other Taxonomies may be generalizable to other domainsdomains

Dissatisfaction with conventional Dissatisfaction with conventional distinction between “internal” and distinction between “internal” and “external” uncertainty“external” uncertainty

Dissatisfaction with distinction between Dissatisfaction with distinction between model errors and inherent stochastic model errors and inherent stochastic variabilityvariability

Minor role of linguistic uncertaintyMinor role of linguistic uncertainty

Risk Management Strategies: Risk Management Strategies: The Interrelation of RulesThe Interrelation of Rules

Three “cardinal rules” of risk Three “cardinal rules” of risk management:management:

1)1) Maximize expected valuesMaximize expected values2)2) Avoid CatastrophesAvoid Catastrophes3)3) Dismiss Dismiss ExtremelyExtremely Remote Possibilities Remote Possibilities

The order of preference between The order of preference between them is: (3), (2), (1)them is: (3), (2), (1)

Risk Management DecisionsRisk Management Decisions

May be oriented toward outcomes or toward May be oriented toward outcomes or toward processesprocesses

Outcome orientation tends to be normativeOutcome orientation tends to be normative Process orientation tends to be descriptiveProcess orientation tends to be descriptive Process orientation considers three stages:Process orientation considers three stages:

1) Pre-decision stage1) Pre-decision stage2) Decision stage2) Decision stage3) Post-decision post-decision3) Post-decision post-decision

*** Each stage is itself composed of a series of *** Each stage is itself composed of a series of partial decisions ***partial decisions ***

The Pre-decision StageThe Pre-decision Stage

Motivated by a sense of discontent or Motivated by a sense of discontent or conflict attributable to difference conflict attributable to difference between an ideal and the real world between an ideal and the real world (a problem)(a problem)

The decision-maker recognizes that The decision-maker recognizes that the ideal alternative is infeasible (not the ideal alternative is infeasible (not in the choice set)in the choice set)

The Decision Stage: Sequences The Decision Stage: Sequences of Partial Decisions of Partial Decisions

Choice: a matter of imposing constraints Choice: a matter of imposing constraints upon the decision situation in such a way as upon the decision situation in such a way as to rule out alternatives to rule out alternatives

Alternative ways of binding the decision Alternative ways of binding the decision problem may be triedproblem may be tried

The choice is made when constraints are The choice is made when constraints are imposed and allowed to bind upon decision imposed and allowed to bind upon decision situations until only one alternative remains situations until only one alternative remains

In Zeleny’s model, the operative rule is to thus In Zeleny’s model, the operative rule is to thus select the alternative that comes closest to the select the alternative that comes closest to the idealideal

Question IQuestion I

Does the lack of an explanation of Does the lack of an explanation of the genesis of value pose a serious the genesis of value pose a serious threat to the orthodox theory of risk threat to the orthodox theory of risk evaluation? If not, explain. If so, evaluation? If not, explain. If so, what in your view is the appropriate what in your view is the appropriate intellectual response?intellectual response?

Question IIQuestion II

Is there in your view a sense in which Is there in your view a sense in which might one consider global security to might one consider global security to be a matter of risk management? If be a matter of risk management? If so, in what way or ways do the basic so, in what way or ways do the basic concepts discussed today help to concepts discussed today help to elucidate and examine it? If not, what elucidate and examine it? If not, what alternative approach would you alternative approach would you suggest other than risk management?suggest other than risk management?

Question IIIQuestion III

In your view, how realistic is the In your view, how realistic is the reasonable person standard at reasonable person standard at describing the psychological and social describing the psychological and social foundations of risk evaluation and foundations of risk evaluation and management? Do you believe that risk management? Do you believe that risk managers really make autonomous managers really make autonomous decisions, or is all of the psychology decisions, or is all of the psychology involved in risk evaluation and involved in risk evaluation and management really social psychology?management really social psychology?