Improving Intelligence Analysis

Bridging the Gap between Scholarship and Practice

Stephen Marrin

• Verlag: Routledge
• Jahr: 2012
• ISBN:
  9781136680472
• Auflage: 1
• Thema: Politics & International Relations
• Thema: Intelligence & Espionage

This book on intelligence analysis written by intelligence expert Dr. Stephen Marrin argues that scholarship can play a valuable role in improving intelligence analysis.

Improving intelligence analysis requires bridging the gap between scholarship and practice. Compared to the more established academic disciplines of political science and international relations, intelligence studies scholarship is generally quite relevant to practice. Yet a substantial gap exists nonetheless. Even though there are many intelligence analysts, very few of them are aware of the various writings on intelligence analysis which could help them improve their own processes and products. If the gap between scholarship and practice were to be bridged, practitioners would be able to access and exploit the literature in order to acquire new ways to think about, frame, conceptualize, and improve the analytic process and the resulting product. This volume contributes to the broader discussion regarding mechanisms and methods for improving intelligence analysis processes and products. It synthesizes these articles into a coherent whole, linking them together through common themes, and emphasizes the broader vision of intelligence analysis in the introduction and conclusion chapters.

The book will be of great interest to students of intelligence studies, strategic studies, US national security, US foreign policy, security studies and political science in general, as well as professional intelligence analysts and managers.

1 Bridging the Gap Between Scholarship and Practice

Improving intelligence analysis requires bridging the gap between scholarship and practice. Compared to the more established academic disciplines of political science and international relations, the intelligence studies scholarship is generally quite relevant to practice. Yet a substantial gap exists nonetheless. Even though there are many intelligence analysts, very few of them are aware of the various writings on intelligence analysis which could help them improve their own processes and products. If the gap between scholarship and practice is to be bridged, practitioners would be able to access and exploit the literature in order to acquire new ways to think about, frame, conceptualize, and improve the analytic process and the resulting product.

What Knowledge do Intelligence Analysts Need?

Intelligence analysis involves the interpretation of information about the adversary or environment for purposes of assisting decisionmaking. There are many different kinds of intelligence analysts in many different disciplines, from civilian national security to military to law enforcement to business. As a result, intelligence analysis as a professional discipline has practitioners all across the world, some of whom join professional intelligence analysis-related associations and explore the nature of the discipline in conferences and workshops, contribute to the growing dedicated literature on intelligence analysis, and even take college and university courses dedicated exclusively to exploring the nature and practice of intelligence analysis.

In order to successfully achieve their purpose, intelligence analysts need to possess both subject matter knowledge related to their specific analytic focus— the kind of knowledge necessary to describe, explain, evaluate, and forecast the actions of the adversary or the environment—as well as process knowledge related to exactly how to do the work of analysis. To acquire subject matter knowledge useful for improving intelligence analysis, one might look to area studies, comparative politics, international relations, and other subject matter disciplines. But if one wants to acquire knowledge on the processes, concepts, and context for understanding and improving intelligence analysis, one would look to the intelligence studies literature.

Intelligence studies as a field of knowledge is about intelligence itself, and the portion of it that overlaps analysis specializes in building up the kind of process knowledge that intelligence analysts require to do their jobs successfully. This knowledge includes how intelligence is collected, analyzed, processed, and distributed, all within group, organizational, cultural, and national contexts.

This knowledge can then be used by practitioners to improve their performance. Doing this well requires being self-conscious about what the analytic process involves: the management, organization, and processes related to the performance of intelligence analysis. It also requires the creation, interpretation, and exploitation of knowledge about intelligence analysis. While practitioners can and do develop this kind of knowledge,¹ they do so primarily on an ad hoc, opportunistic basis and do not have the same kind of infrastructure for growing knowledge about process that exists in academia. As a result, the emphasis on process knowledge related to intelligence analysis should lead one automatically to the world of scholarly literatures.

Scholarship can Provide That Knowledge

Despite the significant potential benefit that scholarship on intelligence analysis can provide the practitioner, it has been portrayed as scarce and therefore difficult to find and exploit. Roger George and James Bruce, two former CIA officers and the editors of a recent book on intelligence analysis, report that “as of 2007, the body of scholarly writing on intelligence analysis remains . . . surprisingly thin.”² In addition, Amy Zegart, a professor at the University of California at Los Angeles, has argued that intelligence studies has been ignored by academia, citing its failure to teach courses and publish works on intelligence.³ But is this really the case? Is the literature as scarce as these experts portray?

Fortunately, the state of the intelligence studies literature in general and the intelligence analysis literature specifically is not as bleak as these authors describe. When one surveys the extensive intelligence studies literature in all its variety over the 60 plus years of its existence, the literature can appear to be quite large indeed.⁴ Through books, journal articles, conference papers, and the like, it is difficult for any scholar to master the large literature on intelligence analysis, which is growing larger every day. This ever-expanding literature makes up the body of knowledge in the field, and is the knowledge base that practitioners can exploit to answer the questions necessary to improve analytic practice.

This literature has significant potential value for practice because, unlike other kinds of scholarship, the intelligence studies scholarship is much closer to practice than it is to theory. In other words, the literature itself is generally applied in nature. The questions asked generally address real-world, practitioner-oriented problems and the answers generally provide real-world, practitioner-oriented solutions.

Levels of abstraction employed in the intelligence literature are minimal, and even what little intelligence and intelligence analysis theory exists is primarily derivative of an effort on the part of intelligence practitioners to understand and explain the intelligence function. Specifically, “in 2005, the Office of the Director of National Intelligence (ODNI) in partnership with the RAND Corporation convened a one-day workshop . . . to discuss how theories underlie our intelligence work and might lead to a better understanding of intelligence.”⁵ A byproduct of this workshop included four subsequent panels at International Studies Association conferences all addressing intelligence theory,⁶ as well as the book Intelligence Theory: Key Questions and Debates.⁷ In other words, intelligence theory has not been developed separate from practice in an academic ivory tower, but rather hand-in-hand with practice in a form of theory/practice partnership.

As for how all of this relates to intelligence analysis, this means that embedded in the scholarship are answers to questions that relate to the kinds of process knowledge that would be most effective in improving intelligence analysis. An example might be the effort to develop a general theory of intelligence, and explain how it could help shape a more specific theory of intelligence analysis.⁸

The scholarly literature on the methods and processes of intelligence analysis begins in the 1940s with George Pettee, Sherman Kent, and Willmoore Kendall, is continued in the 1950s by Roger Hilsman and Washington Platt, is deepened in the 1960s by Klaus Knorr and Sherman Kent, then in the 1970s is extended directly into analytic methods by Richards Heuer and others. By the 1980s, the scholarly literature on intelligence analysis had become well-established, and even though coverage was relatively spotty the kinds of questions asked were increasing in variety. In the 1990s and into the 2000s, particularly due to increasing openness after the end of the Cold War, the literature on intelligence analysis exploded in terms of variety and quality of contributions to scholarship.

A careful reading of the accumulated literature on intelligence analysis would provide a practitioner with a substantial amount of useful knowledge, including criteria and metrics for evaluating analytic quality, best practices in terms of employing analytic methods to support inferences and judgments, guidance for how to develop core competencies necessary for the production of quality intelligence analysis, an understanding of the significance of organizational structures and processes in the development and aggregation of different kinds of analytic expertise in a team or unit context, and so on.

Unfortunately, despite the potential benefit for the practitioner inherent in this literature, a significant gap between scholarship and practice prevents the potential value of the former for the latter from being realized.

Gap Between Scholarship and Practice

Intelligence analysts should be able to benefit from the scholarship on intelligence analysis that is produced, but many—perhaps most—do not. While both scholars and practitioners evaluate information, there is a significant functional differentiation that separates the two fields, as well as a resulting personality differentiation of the people who choose to go into them. Scholars are contemplative and conceptual because their primary mission is to understand and explain: to maximize the growth of knowledge. Relative to scholars, intelligence practitioners, even intelligence analysts, are focused on getting the job done.

As a result of the differences between the worlds of academia and practice, intelligence analysts have a tendency to roll their eyes at the word “literature” as the touchstone of the irrelevant academic. In some cases they have a point. Sometimes theory is overly removed from the day-to-day issues of the practitioner, and while it can help in the process of understanding and explaining big issues it may not have much relevance for the working level practitioner immersed in the highly detailed minutiae involved in interpreting raw intelligence.

In addition, the culture of the intelligence field is infused with a distinct orientation that the best way to learn is not by reading but by doing; that somehow academia and its emphasis on developing, documenting, and disseminating knowledge is a lesser form of learning and that the only real knowledge worth having is that gained by experience. But what is experience other than a trial and error process of learning from your own mistakes? In order to gain a lot of experience, you have to make a lot of mistakes. On its face, that seems to be a pretty ineffective way to learn.

Another way to learn is from the experiences and mistakes of others. And where do you find those lessons? By reading what others have written, in the literature. While some literatures—such as parts of the international relations literatures–are deeply theoretical and derivative of an ivory tower completely separated from the average intelligence practitioner, others have direct applicability to practitioners and the intelligence studies literature is one of those.

In the intelligence field, much of the literature consists of lessons derived from the experiences of practitioners who then became academicians, or academicians who became practitioners, and do what academicians are good at: developing, documenting, and disseminating knowledge consisting of their understandings and explanations for intelligence practice and its improvement. Their goal is not to create abstractions for the purposes of navel-gazing, but rather to conceptualize the function of intelligence in such a way as to make its practice easier to manage and improve. The literature thus provides the mechanism for knowledge transmission to change from verbal lore to a more structured process of formalized education that enables current and future practitioners to learn from the mistakes of those who came before them.⁹

For the practitioner, the literature on intelligence analysis should be viewed as a knowledge repository, where they can go to find out what people have learned about intelligence analysis thus far. Yet the pernicious effects of functional differentiation—even when complementary and geared to achieve the same goals—frequently leads to competition, rivalry, and “us versus them” thinking that progresses to a form of stereotyping.¹⁰

When scholars and practitioners get together, sometimes there is collaboration and sometimes there is competition. In terms of the latter, on multiple occasions it has been suggested that those scholars who do not have previous experience as intelligence professionals should not teach intelligence studies because they bring very little useful knowledge to the table, with the obvious implication that experience in the field of intelligence trumps knowledge about intelligence.

To use one of these discussions as an illustration, in February 2008 a contributor to the electronic mailing list of the International Association for Intelligence Education (IAFIE) suggested that practical experience in intelligence was the key criterion for being considered an intelligence expert and that all others who did not have practical experience in the field—to include scholars since “studying and critiquing alone do not count”—were the equivalent to “back seat drivers.”¹¹ In response, Geneva College professor Thomas Copeland suggested that such a definition was overly limiting, and that while experience in the practice of intelligence is “vital,” he believes that “at least a few academics have had some useful things to say” about the practice of intelligence.¹² He also pointed out that if scholars cannot be considered experts on intelligence issues, then intelligence analysts cannot be considered experts on international issues or situations that they have not experienced first hand, such as terrorism or drug trafficking.¹³

Interestingly, Copeland’s defense of the scholar in the face of practitioner preference for “real world” experience addresses exactly the same challenges that intelligence analysts have faced vis-à-vis policymakers’ questions about their value. For much of their history, intelligence analysts have been perceived by policy officials as irrelevant ivory tower academics with book knowledge but no real sense of how policy was made and no real world experience. As described by Roger Hilsman in the early 1950s, these policy officials “distrust” the intelligence analyst, seeing him or her

as a long-haired academic, poring over musty books in dusty libraries far from the realities of practical life. On the other hand, they think that “practical” experience is the true—in fact, the only—path to knowledge and judgment.¹⁴

If the value of knowledge produced by intelligence analysts has value to the policymaker, then the value of knowledge produced by the scholar may have the same kind of value for the intelligence analyst. The failure of practitioners to listen to the knowledge available to them regardless of source may prevent them from performing at peak levels of efficiency and effectiveness.

But how many practitioners are aware of the accumulated knowledge and wisdom available to them in the scholarship? The answer appears to be “not many” because of the presence of the gap that exists between scholars and practitioners and the general failure on the part of both to bridge that gap.

Negative Consequences Resulting From the Gap

The following illustrations of the negative consequences that can result from the gap between scholarship and practice are drawn from personal experiences as an analyst at CIA. Before arriving at CIA, I studied the causes of intelligence failure and read the writings of scholars like Robert Jervis, Richard Betts, Michael Handel, Art Hulnick, and Loch Johnson. Then I went to work at CIA as an analyst, and quickly discovered that the best practices described in the scholarship were not being followed by the practitioners.

For example, as a junior analyst I was asked to explain why a particular situation was occurring overseas. Because there was no actual evidence about that particular situation, I started the paper with the phrase “We do not know why” the situation is occurring, and then proceeded to supply three plausible explanations given the intelligence available. I was subsequently told by my manager that the first line had to be deleted, because CIA analysts never admit when they do not know something. Yet the literature on intelligence analysis that I was aware of clearly indicated that decisionmakers had an interest in knowing not only what analysts know, but also what they do not know. This is the core idea behind the Colin Powell-derived statement “Tell ’em what you know; tell ’em what you don’t know; and tell ’em what you think,”¹⁵ but is also embedded in longtime CIA analytic methodologist Jack Davis’ writings and interviews from the 1990s. If the manager had known that the scholarship indicated it is important to keep decisionmakers informed of gaps in knowledge or information, then he might not have requested that the opening sentence be deleted.

In addition, that same manager then asked me to look at the three explanations I provided, and choose the one that had the strongest support behind it and say that “we (as in CIA) believe” that this one explanation was the reason for what was happening overseas. Unfortunately, we really had no reason to believe that explanation was any better or more accurate than the others; it was just the one explanation with the most solid (or perhaps more accurately, least flimsy) evidenti...

Inhaltsverzeichnis

• Cover
• Half Title
• Title Page
• Copyright
• Contents
• Preface
• 1. Bridging the Gap Between Scholarship and Practice
• 2. Describing Intelligence Analysis
• 3. Improving the Science of Intelligence Analysis
• 4. Improving the Art of Intelligence Analysis
• 5. Improving Intelligence Analysis With Analytic Teams
• 6. Improving Intelligence Analysis Through Training and Education
• 7. Using Analogies to Improve Intelligence Analysis
• 8. Improving Intelligence Analysis as a Profession
• 9. The Importance of Scholarship to Practice
• Notes
• Select bibiliography
• Index