8/7/2019 Marrin--Understanding What Intelligence Analysts Do--1997

1/3

http://lists101.his.com/pipermail/intelforum/2005-July/008358.html

Date: Thu, 30 Jun 2005 16:51:16 -0700 (PDT)From: Stephen Marrin

Subject: Understanding What Intell Analysts Do

To: intelforum at lists101.his.com

All:

For entertainment value on a long weekend, the short essay copied below is a slightly adapted

version of some thoughts describing what intelligence analysts do when they do analysis that Iposted to an internal CIA bulletin board--the DI Discussion Database--on 6 May 1997. I hope it

has relevance for those of you who do or have done intelligence analysis....

Since I am now an academic, I also believe the discussion has relevance to those who studyintelligence analysis from the academic side. Specifically, the underlying issue of the essay has

to do with the use of heuristics in foreign policy decisionmaking.

In terms of international relations, these cognitive shortcuts can be applied at 2 different levels ofanalysis.

First, these cognitive shortcuts are used in the intelligence analysis process by analysts who play

a role in the decisionmaking process by providing their understandings of events (traditionallythose that occurred overseas, but now--for homeland security/law enforcement analysts--

domestically as well) to relevant decisionmakers. The focus on cognitive processes at this level(ie. within bureaucracies) can be folded into an assessment of the influence that variations in

information flow within organizations have on decisionmaking...in other words, GrahamAllison's organizational process model.

The second level of analysis is that of the individual decisionmaker; focusing on how these kinds

of heuristics *as applied by individual decisionmakers rather than intelligence analysts* mightaffect the final decision or policy. In other words, this approach would be akin to Robert Jervis'

(and others') focus on the influence of factors at the individual level of analysis (psychology,history, perceptions, etc) and how they influence decisionmaking and ultimately state behavior.

When you get right down to it, I am addressing epistemology in this essay....epistemology of

international relations and the understandingsalbeit flawed--of those who interpret(intelligence analysts) and act (decisionmakers) based on their understandings.

Hope it is of interest, and all comments welcome...

Regards,Steve Marrin

# # #

8/7/2019 Marrin--Understanding What Intelligence Analysts Do--1997

2/3

Modified Posting to DI Discussion Database

May 6, 1997

Complexity is in the Eye of the Beholder

The question of what is simple and what is complicated has been floating in the back of my mindfor a while, ever since I was told that the extremely complicated job that I was asked to do was in

reality exceedingly simple. Based on this thought--in combination with an article on complexityin the New York Times todaylet me throw out a hypothesis which may clarify the debate of

the above postings.

Reality is complicated. The human mind cannot comprehend randomness, so we look forpatterns. One kind of a pattern is a cause/effect relationship. When the causal linkages can be

established we become able to understand and predict outcomes. But nothing in the real worldconsists of one cause/effect relationship. Instead we have to deal with outcomes that are the

result of the interactions of many causes. In trying to determine the impact of the independentvariables on the outcome we make frequent mistakes. We can overestimate, underestimate, or

miss altogether the influence of a particular independent variable on the end result. But as longas we are close, its good enough. Humans are lazy, and do not want to do all this work all the

time. Once we find a causal relationship that approximates reality, it becomes a rule.

And to make things even easier, we will adopt the rules of others--conventional wisdom. Theserules are then applied the next time we think the cause shows up. The application of these rules is

assumptions at work. Some are accurate, some are partially accurate, and some are wrong. Thosewho build correct frameworks of assumptions--those who apply accurate hypotheses--become

known as experts. But even experts can be wrong because their expertise is based on rules whichare at best blunt approximations of reality.

In addition, we can aggregate the rules by which most people act, and upon these rules are built

the social sciences. But individuals themselves remain unpredictable, hence error will alwaysremain a part of the enterprise.

Now to apply this to the work of intelligence.

International relations is an interdisciplinary social science. This institution uses many different

kinds of analysts (political, military, economic, imagery, transnational, etc) to derive and identifythe operative variables, their interactions, and subsequent impacts.

Understanding is nice, but prediction (or more accurately, estimating) is the name of the

intelligence game. Prediction in international relations is possible, but very difficult to doaccurately. Many times we are right because of all the time and effort to ensure that the

information at our disposal is such that it helps us to create more accurate interpretations ofreality. But we will also be wrong for any of innumerable reasons--including the use of

inappropriate assumptions--regardless of whether it is derivation of broad societal trends or spotcrisis prediction that we are tasked with. In the end any prediction will be an approximation of

8/7/2019 Marrin--Understanding What Intelligence Analysts Do--1997

3/3

the real world and therefore subject to some amount of error. The whole idea of this enterprise isto reduce the margin of error so that when a judgement is asked for we can confidently give it.

The amount of work we are required to do all depends on what is needed. If all that is needed is

for us to identify threats--to say 'there could be a problem over there....maybe y'all should do

something about it' then crisis prediction may not be that difficult after all. But if you thinkintelligence should be able to predict a crisis by identifying its cause exactly--and thereby givingthe policy folks a better idea of how to solve the problem--prediction becomes more difficult in a

hurry.

One possible way to rigorously approach the issue of prediction and reduce the possibility oferror is to use analytical methodologies to self-consciously break down the prediction into

variables and develop future scenarios based on our perceptions of their relative influence. In thisway we can productively use assumptions to weed through complexity.

My concluding thought is that the determination of whether something is easy or complicated

will depend upon the number of assumptions in play. If you assume a lot it will be easy to createpredictions. However, reality constantly reminds us that many of our assumptions are

inaccurately applied. But if you leave assumptions as open questions the project becomes morecomplicated as it has to deal with a greater number of variables and outcomes. Different people

studying the same issue could find the task exceedingly easy or incredibly complicateddepending upon their own personal frameworks of assumptions.

It all hinges on assumptions and how you use them.

Stephen Marrin