Marrin Historical Overview of All Source Analysis 2008

13
Stephen Marrin 1 July 2008 Remarks for State/INR conference on Learning the Lessons of All-Source Intelligence Analysis Topic: Historical Overview of All-source Analysis: What are the lessons of the past that we have failed to incorporat e into our current practice of intelligence analysis? How effectively has the literature of intelligence analysis been at building k nowledge about the craft and profession? What efforts are afoot to help scholars and pr actitioners alike to learn more about intelligence analysis? XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Hi; my name is Steve Marrin, and I am a professor of intelligence studies at Mercyhurst College in Erie, PA. I started teaching fullt ime about 2 years ago, which makes me sort of a newbie academic, but I’ve been studying intelligence—mostly intelligence analysis  —for over 15 years. As an undergraduate, I studied the causes of intelligence failure, and read the writings of  people like Robert Jervis, Richard Betts, Michael Handel, Art Hulnick, and Loch Johnson. And then I went to work at CIA as an analyst, and quickly discovered that theory did not match practice. For example, as a junior analyst, I onc e started a paper with the sentence “We do not know…” and then provided three explanations for the particular situation, with no evidence indicating that any one was any more plausible than another. I was subsequently told by my manager that the first line had to go, because CIA an alysts never admit when they don’t know something. Interesting…. How does that square with the Powell framework: “Tell ‘em what you know; tell ‘em what you don’t know; and tell them what you think?” Apparently, that framework did not have widespread acceptance at CIA in the mid-1990s, even though the literature on intelligence clearly established the interest that decisionmakers had in knowing not only what analysts knew, but also what they didn’t know. Anyway, going back to that analytic report I had drafted, because the CIA office that I was in at that time put forth a single analytic line rather than multiple interpretations, this manager chose one of the three explanations I had put together essentially at random, and said that was going to be the CIA analysis on the subject. “We believe….” And off it went. Hmm. So much for alternative analysis…. And there was no room for dissenting opinions in that office either. During the time I was there, I witnessed a dispute over an analytic judgment that led to a winner’s analysis 1

Transcript of Marrin Historical Overview of All Source Analysis 2008

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 1/13

Stephen Marrin

1 July 2008

Remarks for State/INR conference on Learning the Lessons of All-Source

Intelligence Analysis

Topic: Historical Overview of All-source Analysis: What are the lessons of the pastthat we have failed to incorporate into our current practice of intelligence analysis? How

effectively has the literature of intelligence analysis been at building knowledge about the

craft and profession? What efforts are afoot to help scholars and practitioners alike tolearn more about intelligence analysis?

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Hi; my name is Steve Marrin, and I am a professor of intelligence studies at MercyhurstCollege in Erie, PA. I started teaching fulltime about 2 years ago, which makes me sort

of a newbie academic, but I’ve been studying intelligence—mostly intelligence analysis

 —for over 15 years.

As an undergraduate, I studied the causes of intelligence failure, and read the writings of 

 people like Robert Jervis, Richard Betts, Michael Handel, Art Hulnick, and LochJohnson. And then I went to work at CIA as an analyst, and quickly discovered that

theory did not match practice.

For example, as a junior analyst, I once started a paper with the sentence “We do notknow…” and then provided three explanations for the particular situation, with no

evidence indicating that any one was any more plausible than another. I was subsequently

told by my manager that the first line had to go, because CIA analysts never admit whenthey don’t know something. Interesting….

How does that square with the Powell framework: “Tell ‘em what you know; tell ‘emwhat you don’t know; and tell them what you think?”

Apparently, that framework did not have widespread acceptance at CIA in the mid-1990s,even though the literature on intelligence clearly established the interest that

decisionmakers had in knowing not only what analysts knew, but also what they didn’t

know.

Anyway, going back to that analytic report I had drafted, because the CIA office that I

was in at that time put forth a single analytic line rather than multiple interpretations, this

manager chose one of the three explanations I had put together essentially at random, andsaid that was going to be the CIA analysis on the subject. “We believe….” And off it

went.

Hmm. So much for alternative analysis….

And there was no room for dissenting opinions in that office either. During the time I was

there, I witnessed a dispute over an analytic judgment that led to a winner’s analysis

1

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 2/13

 being put forth as the only analysis, and the loser being described as not a team player for 

fighting for his interpretation, even though he was a subject matter expert and knew more

about the country than the winner in the bureaucratic struggle. And then I saw the loser’sforecast actually occur—surprising decisionmakers in the process, and being

subsequently described in the newspapers as a high-profile intelligence failure.

Because of these incidents—and a series of others, related to apparent disregard for 

analytic expertise, disinterest in trying to identify ‘ground truth’ rather than just report

what was in the raw intelligence, and so on--I became a disillusioned analyst in relativelyshort order.

 Not because I didn’t see value in the enterprise, but rather because I thought that the

function of intelligence analysis was too important to allow these kinds of flawed practices to be embedded within analytic processes.

The literature I had read clearly indicated that the value to decisionmakers came in

 providing them not just with alternative interpretations for a situation, but also anexplanation for why those interpretations differed. Yet here I was, in the world of the

 practitioner, witnessing an intelligence failure because best practices identified in theliterature were not implemented, and in-depth subject matter expertise was ignored.

This wasn’t the dark ages of intelligence analysis, but we weren’t yet at enlightenment.

During the 4 years I was at CIA, not once was I taught a structured analytic method or technique. Heuer’s Psychology of Intelligence Analysis book was not yet released, and

nobody knew what ACH was. As Randy Pherson has said, the model for doing

intelligence analysis at that time was to “read as much as you have time to read that day,think about it and suck an answer out of your thumb, and write it down in as crisp a

manner as possible.” And that’s a pretty good description of what intelligence analysis

involved when I was at CIA.

But a lot was going on behind the scenes that built the foundation for the kind of 

conference we have here today. The T-2000 course and the development of analytictradecraft; the promotion of alternative analysis and other structured analytic techniques,

the creation of CIA’s Kent School and its Career Analyst Program, and so on.

This was very much a time of growth and change; of discovery and re-discovery, with myown personal emphasis more on the organization, management, training, and processes of 

intelligence analysis rather than its practice. But all of these lessons helped shape

 practice, by establishing training course content, human resource and development policies, standard operating procedures in terms of analytic production, and the

development of new best practices in the field. What I was interested in was everything

that shaped what an analyst did when he or she was on the job doing analysis.

So I left the Agency to go study intelligence analysis in academia. Along the way, I have

written almost 20 articles or book chapters addressing various aspects of intelligence

analysis, each involving research into a different segment of the intelligence literature. So

2

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 3/13

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 4/13

For example, when I wrote a paper comparing and contrasting intelligence analysis to

medical diagnosis, I referenced the 1983 article by Walter Laqueur on “Intelligence and

Medicine,” providing an overview of his article and going on to explain why mine addedto it. In this way, the intent was to try to layer new knowledge on top of old; not pretend

that I was the first to write on a subject, but to use what had been written previously as

the foundation to my paper; to try to ensure that knowledge was cumulative.

I also tried to do the same thing with a paper on intelligence analysis professionalization,

 by using George Allen’s 1985 CIA Studies in Intelligence article on the“Professionalization of Intelligence” as its conceptual starting point.

But this kind of practice is the exception rather than the rule in intelligence studies, and

as a result the literature has a Groundhog Day-like flavor to it if you go back over thedecades to review the concepts.

For example, in the literature review for my PhD dissertation, I evaluated what I’m

calling the proximity hypothesis, or the effect that proximity between analysts anddecisionmakers may have on the accuracy or utility of the finished product. In doing so, I

went through the literature back to the 1940s, and discovered that many people had‘invented’ the same concept time after time after time. And they even ended up giving it

different names.

Two scholars writing at about the same time in the 1980s gave the relevant schools of thought two different names—both of which have since been forgotten—and later writers

have identified up to 7 or 8 primary representatives of those schools. But almost none of 

the writers reference each other or those who wrote about it previously. So the same‘school of thought’ can be identified with up to 5 different names, depending on who was

doing the writing at the time.

If you read each contribution in isolation, it says something interesting about the issue.

But when you read the different contributions in succession, the repetition and conceptual

overlaps become obvious. Knowledge on this particular subject has not been cumulative.In some ways, the writers in the 1940s and 1950s had a better grasp of the key issues at

 play than those who write about it today.

Why does this matter for practitioners? Is this as academic as it sounds?

The answer is no; while it may be academic, it does have direct relevance for 

 practitioners…

The chair of my dissertation committee is Philip Zelikow, who was Counselor of the

State Department, and before that was executive director of the 9/11 Commission.

Zelikow has a particular interest in the intersection of intelligence and policy, with a

 particular approach that he and Ernest May developed for Harvard’s Intelligence and

Policy Program. May was also a consultant on the 9/11 Commission as well.

4

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 5/13

I believe—but do not know for sure—that part of the structure of NCTC that was written

into the 9/11 Commission Report was partly intended to bridge the gap betweenintelligence analysts and decisionmakers. This is an effort to try to take our 

understandings of organizational best practice, and implement them. And where does that

understanding come from? From the accumulated wisdom in the literature. But as I just pointed out, our accumulated wisdom in terms of how to structure the interface between

analysts and decisionmakers isn’t very solid. And because our knowledge is not very

solid, neither is the implementation of the recommendations.

As I was researching the dissertation, I found an example of an interface between

analysts and decisionmakers that it appears has been completely forgotten by both

academia and government. It used to exist. It actually received a fair amount of praise for  being an effective way to communicate. But it hasn’t been referenced in the literature for 

almost 30 years. I asked a couple of longtime former senior intelligence officers if they

had ever heard of it, and the answer is “no.” I also asked a member of the Principals

committee during the Clinton administration if he had ever heard of it. Again, the answer was no.

However, when I wrote it up for the dissertation and sent it to Zelikow, he said the ODNI

was implementing something similar. Which is interesting. My guess is that what we

have here is another re-invention of the wheel, that could have been prevented if the

intelligence literature had been more cumulative.

The point that I’m trying to make is that literature is the embodiment of knowledge, and

the state of knowledge in terms of our understanding of intelligence analysis is not nearlywhat it could or should be. And this can have a direct impact on practitioners.

And this is only one concept, one research question. I believe the same kind of Groundhog Day-like effect exists along a number of dimensions in the intelligence

literature…

Its really sort of sad; even though the purpose of academia is to develop and grow

knowledge over time, we have failed to do so in the study of intelligence analysis….

SHERMAN KENT

The only real progress that has been made on this front over the past few years is the

establishment of Sherman Kent as one of the giants in the field. But I don’t think that was

an accident. I think Jack Davis spent about 15 years trying to raise the profile of Kent,and succeeded. I’m not saying that Kent was unknown prior to Jack Davis; only that he

wasn’t referenced to the extent that he is today. And this higher profile is a good thing,

 because it provides a touchstone in the literature that later scholars and practitioners canrefer back to, and use as a jumping off point to make additional observations and

contributions to the body of knowledge…

5

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 6/13

But there were other people writing on intelligence at the time who were just as

important….and perhaps more important….whose writings are worth studying by those

who both write about—and practice—intelligence.

HILSMAN

Another mostly forgotten writer on intelligence analysis was Roger Hilsman, a former director of State/INR. In 1953, only 4 years after Kent’s book was published, Hilsman

made a strong case for a closer relationship between intelligence analysis and

decisionmaking. Hilsman is actually as good or better than Kendall as a counter to someof Kent’s ideas.

Hilsman argued that “a more effective integration of knowledge and action”—or 

intelligence analysis and decisionmaking—will require intelligence analysts to becomemore policy-oriented.1 Hilsman directly questioned the separation of intelligence from

decisionmaking by asking “whether this division of labor is a wise or even a valid one”2

and he ended up concluding that it was “both arbitrary and awkward.”

Hilsman goes on to say that in order for intelligence to be “useful and significant” it

“should be frankly and consciously concerned with policy” and that its practitionersshould have “a frame of mind which is… instrumental, action-conscious, policy-oriented.

The major task before the researchers is one of recasting their thought to the context of 

action, and adapting their tools to the needs of policy.”3 

In other words, Hilsman believed that intelligence analysts should work much closer with

decisionmakers than the Kent model that has become adopted as the ‘traditional’

approach in the intelligence community.

People know what Kent said about the intersection between intelligence analysis and

decisionmaking, but they seem to have forgotten Hilsman, even though his ideas have asmuch relevance as anything else written about the subject over the past 50 years...

In terms of practitioners, what would the value be of re-discovering Hilsman? The answer is: A much better, more substantive debate about the respective roles and functions of 

intelligence analysis vis a vis decisionmaker assessment. It might even lead to a re-

defining of the concept of politicization, which I am beginning to conclude is--in and of 

itself--badly conceptualized….

PLATT and KNORR 

Another example of forgotten scholarship that would be helpful for those trying tounderstand intelligence analysis is Washington Platt’s 1957 book “Strategic Intelligence

Production: Basic Principles.” Just listen to the titles of his chapters and relate the content

to the work that you are doing:

• Principles of Intelligence Production;

1 Hilsman. “Intelligence and Policy-Making in Foreign Affairs.” 45.2 Hilsman. “Intelligence and Policy-Making in Foreign Affairs.” 25.3 Hilsman. “Intelligence and Policy-Making in Foreign Affairs.” 44.

6

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 7/13

• From Information to Intelligence;

• Intelligence Production: An Act of Creative Thinking;

• Help From the Social Sciences;

• Probability and Certainty;

• Forecasting;

• Characteristics of the Intelligence Profession

All of these chapters could make contributions to the ongoing discussions regarding

analytic process, utility of the social sciences, increasing imagination, futures work andforecasting, and professionalization. But Platt’s work falls into the same category as

Hilsman….its been forgotten over time, even though it contains some important ideas

that current scholars and practitioners would find quite interesting.

Finally, another forgotten piece of scholarship is Klaus Knorr’s 1964 monograph

“Foreign Intelligence and the Social Sciences.” I found this little book—written 45 years

ago—to be the best evaluation of how analysts use social science methodology in the

entire intelligence literature including the recent literature.

Yet all of this knowledge has been forgotten. Current academics are rebuilding thewheels that were built decades ago. And current practitioners do not have as solid a grasp

of what it is that they do than these scholars did over a half century ago.

And there has been just as much scholarship forgotten over the most recent few decades

as well.

Thomas Hughes—perhaps not coincidentally, another former director of State/INR— 

wrote a short monograph in 1976 on the relationship between intelligence and policy that

noted British intelligence scholar (and former longtime GCHQ officer) Michael Hermanthinks is the best writing that’s ever been done on the subject. But I challenge you to findmore than a handful of references to it in the literature.

Bottom line: you have to know who the giants are to stand on their shoulders. Right nowI’m not sure that (with a few exceptions) there is agreement on the part of intelligence

studies scholars or practitioners regarding who those giants are.

FAILING TO BE COMPREHENSIVE

Another problem with the intelligence literature is its failure to be comprehensive

Intelligence FailureThe most well-developed part of the analytic literature deals with intelligence failure,

which grew out of the post-Pearl Harbor emphasis on the study of strategic surprise. But

after Michael Handel and Richard Betts covered this ground in the 1970s and 1980s,there did not appear to be much else that could be said. Intelligence failures will be

inevitable.

7

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 8/13

There are inherent pathologies—in Betts’ terminology, enemies of intelligence—that will

ensure that perfection is unattainable. So the research agenda that had previously been

established --identify causes of failure in the hopes that these causes could be addressedand failure prevented--came to a dead-end. Conceptually, there hasn’t been much

 progress in the intelligence failure literature since the late 1970s.

The only real potentially productive research agenda that scholars could point to was to

study successes rather than failures. But the academic community decided that was a

 bridge too far because of classification issues. But now--with the creation of the CIA andODNI lessons learned centers--this research agenda can finally be implemented. So the

study of intelligence failures….which hasn’t really gone anywhere for almost 30

years….can finally be replaced with the study of intelligence successes.

But that doesn’t mean that there is no more room at the inn for people who want to study

intelligence failure. Just that their contributions will be marginal. Essentially, this is the

Betts fallback position; intelligence failures may be inevitable, but you might be able to

 prevent failure on the margins. To use the baseball analogy that Robert Jervis and Dick Betts have periodically used, perfection is unattainable so a fielding percentage standard

is inappropriate for intelligence analysis. However, a batting average standard mightwork better. So how might the batting average of the individual batter….or baseball team

in the aggregate…be raised?

There are a lot of different ways to answer that question; all of which relate torecommendations and suggestions for improving intelligence analysis. In my “Preventing

Intelligence Failure” article, I suggested a two-prong approach; first using “more rigorous

tradecraft to minimize faulty or incomplete analysis,” and then implementing a moreseamless integration of that better product into decisionmaking processes.

Structured Methods

The first part of this approach—producing a better analytic product—is also addressed in

the part of the literature that talks about structured methods.

For example, in 1978 Dick Heuer edited a book titled “Quantitative Approaches to

Political Intelligence: The CIA Experience.” It is perhaps the best scholarly effort to

rigorously document the value of quantitative methods to intelligence analysis. It did

exactly what was necessary to create a literature; it captured in the form of a publicationnew thinking and research on topics of interest.

But its been 30 years since that book was published, and there isn’t anything else like itout there. There are a smattering of other voices in this discussion over the value of 

structured methods—for example, Rob Folker’s Masters thesis and JMIC Occasional

 paper; a couple of articles by Stan Feder; a couple of follow-on articles by Heuer andPherson; a contribution or two by Steve Rieber; a short article I wrote last year on

structured analytic techniques versus intuition; and a handful of others…..but the

literature is just not that large.

8

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 9/13

The literatures regarding the value of structured methods outside the intelligence analysis

domain is much larger, but the research agenda necessary to reach out to other domains

and literatures and exploit them for the value they have for understanding and improvingintelligence analysis is still in its infancy….

Analogies and Other FieldsAs I have argued elsewhere, the exploitation of other domains and literatures is best

accomplished through the rigorous evaluation of analogous fields such as medicine. The

 paper that was published in 2006 under the title “Improving Intelligence Analysis byLooking to the Medical Profession” had its beginnings in 2000, while I was working at

CIA.

At the time, I was also involved in a difficult medical situation where someone I knewwas having health problems that the doctors were having difficulty diagnosing. The

 process that the doctors were going through to diagnose the problem appeared to me to be

very similar to the processes that an analyst goes through when tackling a difficult

question.

At the time, I was a participant on Jack Davis’ internal CIA discussion group onAlternative Analysis, and in that forum raised some of the issues regarding the analogy

 between medicine and intelligence. Some of the other participants explained to me that

the similarities I was seeing in the practice of both fields were related more to the

application of the scientific method than anything else. I was too busy at the time to studythis aspect further, but had more time in 2003 and 2004 when I wrote the draft of the

medical analogy paper that was eventually published.

I don’t have time to lay out the content of that paper here, but the general idea was to

look at medical practice and see what we can learn from it; both similarities and

differences. But I also didn’t think that the utility of studying analogies stopped atmedicine. The conclusion of that paper included a suggestion that intelligence

 professionals “look to analogous professions for ideas that can be adapted to an

intelligence context.”

I went on to say that “Many of the challenges intelligence analysts face are not as unique

as its practitioners believe, but the insularity of the field prevents them from being able to

identify the lessons from other professions that could be useful as models to follow. As aresult, the first task is to identify analogous professions, and examine them for the lessons

they might provide. Any profession that encounters similar problems—such as medicine,

 journalism, law, or law enforcement— may provide fertile ground for deriving ideas toimprove existing practices.”

And let me add to that list, academia and its emphasis on knowledge creation andaggregation, which is more or less what I’m outlining in my talk here today….

Others have also explored different aspects of other fields for their utility in

understanding intelligence analysis:

9

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 10/13

• Heuer has looked at the psychology literature

• Zegart has looked at the organizational and institutional literatures

• R. Scott Rodgers has compared clinical psychology to intelligence analysis

And so on and so forth, with additional authors like Steve Rieber, Rob Johnston, Charles

Weiss and others exploring different domains of knowledge for their relevance tointelligence analysis.

So there is the beginning of an effort to increase the breadth of our understandings of intelligence analysis, although other fields of knowledge have yet to be fully explored.

Other concepts or methods of similar value can come from sociology, anthropology,organizational theory, and so on. More writings need to be encouraged--and different

angles need to be explored --so that we can maximize our learning opportunities along

the way….

At this point, our understandings of intelligence analysis are growing in leaps and boundsdue to these kinds of evaluations, although in the aggregate the literature in its entirety is

not anywhere close to comprehensive. This may happen in time, but it won’t be any timesoon…

RECOMMENDATIONS/SOLUTIONS

In this last part of my talk today, I want to lay out a series of suggestions or 

recommendations that—if implemented—might bolster the intelligence analysis literatureand make it both more cumulative and more comprehensive.

For those of you who have read my professionalization paper, what we are talking abouthere is the segment of the professionalization process related to knowledgeadvancement…

Document the literature:

The first step would be to document what you know. This has to be a dynamic rather than

static process; perhaps an annualized bibliographic book series. The last comprehensive

intelligence bibliography that I am aware of was Neal Petersen’s bibliography publishedin 1990. That book—which is a great resource for students of intelligence—has not been

updated since. The closest approximation that I am aware of is the online Muskingum

College intelligence bibliography compiled by J. Ransom Clark, with the appendix to the

2006 University of Maryland’s Project on the Future of Intelligence Analysis coming in adistant second.

If the goal is to be able to learn from the past, then documenting the knowledge that has been developed thus far is important. Will scholars or academics do this on their own?

Probably not. Maybe the best way to go would be to partner up with J. Ransom Clark and

see if he’d be willing to use his online bibliography as the starting point for a much morecomprehensive bibliography. And that may require governmental funding. But if you

10

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 11/13

don’t fund this kind of project, you will never have any idea of what knowledge has

already been created, and will continue to reinvent wheels ad infinitum….

Review and evaluate the Literature:

The second step would be to evaluate what you know; the literature that has already been

developed. This would involve a variety of literature reviews oriented towards identifyinga variety of research questions that have been explored sufficiently, others that still

require some work, and yet still others that have not yet been answered.

The Intelligence Studies Section at ISA is participating in a project that is heading in this

direction, called the Compendium project. The section broke the intelligence studies

literature down into 20 different topics, and lined up authors to write literature reviews on

each of those topics. When published by Blackwell next year, this project should providecurrent and future scholars with a starting point for understanding the current state of that

segment of the literature. Not all the topics address analysis, but enough do to make the

 project worth highlighting for purposes of today’s conference. The authors of the essays

on intelligence analysis, intelligence failure, and the intersection of intelligence anddecisionmaking spoke on a panel at the ISA conference earlier this year, and all

acknowledged that their segments of the literature failed to be cumulative.

So the Compendium Project will begin the process of evaluating what we know, by

reviewing, evaluating and critiquing the current parameters of intelligence scholarship,

and highlighting what we don’t know.

Add to the literature

The next step in the process, after evaluating what we know and highlighting what wedon’t know, is to begin the process of filling in the gaps in the literature. There are a

variety of ways to do this, including by developing a dedicated (and funded) research

agenda akin to the Army War College’s Key Strategic Issues List to back-fill gaps inknowledge. Another way to fill in gaps in knowledge is through themed conferences and

symposia…perhaps made annual events….like the Godson events at Georgetown that led

to the Intelligence Requirements book series, or the recent joint ODNI/RANDsymposium on intelligence theory.

But both of those kinds of events are by their nature invitation only. There should also be

a mechanism for getting new ideas into the literature, and that means opening up thevenue to public participation. A model for this kind of activity could be the 2005

International Conference on Intelligence Analysis that Mitre coordinated on behalf of the

then-ADCI for analysis and production (Lowenthal). This conference was modeled on anacademic conference, with a paper requirement and proposals open to the public.

Rather than do this kind of work unilaterally, maybe you should partner up with existingassociations that have the infrastructure, contacts, and knowledge necessary to pull it off.

ODNI partnered up with INSA for the ODNI conference on analytic transformation last

year, but other associations exist such as ISA’s Intelligence Studies Section--which

would be a good partner for those wanting to engage academia--as would the 400+

11

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 12/13

membership of the International Association for Intelligence Education. In both cases, the

associations could provide a focal point for acquiring and coordinating the kinds of 

knowledge that would be helpful in terms of outreach to the academic community.

Institutionalize these efforts

The next step would be to institutionalize these efforts, through Centers for IntelligenceAnalysis Research and Scholarship. Not Centers of Academic Excellence; that’s a

different program with a different purpose. Instead, the goal should be to rely on

academia for its strengths, including knowledge of scholarship and the ability toaggregate knowledge so that the end product is more than the sum of its parts. Frankly,

this is the kind of project that I’d be interested in getting involved in, but I also think its

establishment would pay dividends back to the Community in terms of providing it with a

one-stop shop for knowledge and expertise about intelligence analysis.

Create a Learning Feedback Loop

Finally, an absolutely mandatory aspect of building knowledge is to then exploit that

knowledge by disseminating it to those who can use it. That could involve synching upthose who research and write about intelligence and those who teach it, so that you have a

feedback loop established to maximize learning. The students could be those in academiain intelligence studies or intelligence school programs; or those in governmental training

courses. The Harvard Intelligence and Policy Program also provides a potential model for 

those who want to establish a form of continuing education in the field devoted not to

 practitioner proficiency per se, but rather broader understandings of purpose and how best to manage the enterprise.

So when you summarize each of these suggestions, the steps involve:

• Document what you know

Evaluate it for gaps or holes• Work to fill those gaps in knowledge

• Distribute this knowledge to those who need it (create a dynamic feedback loop)

 Now here’s my takeaway for those of you in government who are trying to figure outhow to establish a dynamic learning process regarding intelligence analysis:

All you have to do is follow those same 4 steps….

Let me leave you with a few questions geared to get you thinking along the lines of an

academic who builds knowledge for a living:

Here’s my first question for you: do you have any idea what the intelligence community

knows about how to do intelligence analysis, and how to do it better? If the answer is

“no”—and I assume the answer is “no”—then what kind of documentation or repositorywould be of greatest value? Which entity should be the one doing the documenting? Or 

should it be a collaborative enterprise, with a focal point per analytic agency or 

organization?

12

8/6/2019 Marrin Historical Overview of All Source Analysis 2008

http://slidepdf.com/reader/full/marrin-historical-overview-of-all-source-analysis-2008 13/13

In terms evaluating existing knowledge, do you have any idea what you know, and— 

more importantly, what you don’t know—about the practice of intelligence analysis? If 

not, then what kind of evaluative process are you going to establish to meet thisrequirement?

As for filling gaps in knowledge, what projects are currently underway? What gap or need to they fill? Are they prioritized in any way, or just ad hoc research efforts? Being

able to establish a strategic approach to filling gaps in the literature will be an important

 part of this kind of knowledge accumulation effort…

And finally, how are you planning on synchronizing each of these functions with the

dissemination of this knowledge? Are you planning to incorporate the content into

training courses? Or maybe you want to use this knowledge to shape professionaldevelopment programs.

However you use the knowledge, somehow there must be some kind of interface between

those who acquire, store, develop, and disseminate knowledge about intelligenceanalysis. What kind of architecture will you build to meet these purposes?

Thanks for your time and attention.

13