On Estimating Reactions
APPROVED FOR RELEASE 1994
CIA HISTORICAL REVIEW PROGRAM
18 SEPT 95
SECRET
Shortcomings and net usefulness of a sharply policy-pointed class of estimative exercise.
ON ESTIMATING REACTIONS
John Whitman
The most fascinating and frustrating of the National Intelligence Estimates which an estimates officer writes begin as follows:
THE PROBLEM
To estimate Communist reactions to a U.S. course of action involving ...
These estimates form a quite distinct category. They originate in a unique way; they pose special problems of organization; their coordination with the representatives of the USIB member agencies is exceptionally difficult; and final USIB approval almost always requires more than one meeting, often more than two. Herein reside the frustrations, to which I shall devote the greater part of what follows. The fascination lies in the assurance that the drafter is involved in major and immediate decisions of U.S. policy. No other estimates can generate in his breast quite such a sharp sense of relevance to action.
These papers are often miscalled "contingency estimates." Contingencies figure in almost all NIEs. Sometimes they concern what one foreign country may do if a neighbor takes certain steps, e.g., what Pakistan will do if India embarks on a nuclear weapons program. Sometimes a contingency lying in possible U.S. action is examined as part of a wider study, e.g., in the course of a general estimate on South Korea, ROK reactions to a reduction of U.S. military aid may be explored. To avoid confusion with these, it will be useful to reserve the term "reaction estimates" for those NIEs which are addressed exclusively to the question of other countries' -- usually Communist powers' -- responses to a postulated U.S. course of action in a crisis situation.
Top of Page
Origination
Reaction estimates are never self-initiated. They are commissioned by policy-making departments which are considering taking some specific course of action and want an appraisal of how the enemy will probably respond. They are invariably written against short deadlines and deal with immediately critical problems. Those of the last few years have dealt principally with three situationsBerlin, Laos, and Vietnam. The first were requested by the Berlin Task Force in the State Department; the father of the other two sets is an alumnus of the Office of National Estimates who migrated to policy-making posts and established a practice -- now sustained by the White House, the joint Chiefs, and others -- of subjecting a great variety of Indochina policy proposals to the estimative test.
After writing quite a number of reaction estimates, I'm still not entirely sure why requesters keep on asking for them. The results, as we shall see, are often of dubious value. Sometimes I suspect that the commissions come from opponents of the policy proposal who hope that the estimators will help them kill it. But the process does reflect a fundamental principle of intelligence: that when early enemy reactions are the critical test of a policy proposal, these reactions should be estimated in advance, not by proponents or opponents, but by someone uninvolved in the heat of policy contention. Full objectivity is of course a counsel of perfection, but I think it correct, not merely charitable, to say that the policy makers should and do feel better -- feel protected against the full force of bias -- when they have an outside opinion. And since these matters are too sensitive to be submitted to public opinion, they turn to intelligence as an inside outsider.
Top of Page
Terms of Reference
And intelligence always bucks. We are never satisfied with the way the questions are put. They are far too general; we need a clearer idea of what the United States proposes to do; in particular we need sharper distinctions among the various steps to be taken in a sequence. Very well, responds the policy maker, and lists for us four major steps and a dozen specific actions within each, including inter alia, say, the exact inventory of implements to be used in each of three probes on the Berlin autobahn. Now we are really outraged. Perhaps, we say, we can provide some general guidance, but how do you expect us to distinguish between reactions on the one hand to ten air sorties against troop concentrations in Laos with high explosives and on the other to fifteen sorties against lines of communication with napalm?
It would be nice to think that we eventually sort out with the requester the proper level of detail and can proceed to answer questions which are governed by the limits of professional intelligence and human judgment. Unfortunately, this is often not the case; the question of proper terms of reference dogs us to the very end of the process. The reason, I am sorry to say, is that we are not "outside" after all. Each of the intelligence agencies works for a particular policy maker. Even the Director of Central Intelligence is, under one of his hats, a senior policy advisor to the President. And it is uncanny how the choice of a level of detail will influence the estimated enemy reaction, and therefore the seeming wisdom of the proposed policy. A proposal may appear to bloom with fair prospects when viewed in a general way, yet prove to be studded with thorns when examined in detail. Surely everyone can understand this; how many bright ideas have we all had which might survive one or at most two levels of detailed criticism but fell apart at the third? And when that third level is reached, do we not insist that it's a good idea "in principle" and plead for a reconsideration at the higher, more favorable level of generalization?
Top of Page
Other Kinds of Bias
That was a fairly subtle point. A more obvious one is that the participating agencies may already, at their policy-making summits, have decided what they think about the proposed U.S. course. Their intelligence arms are then under pressure, of course, to bend the estimate toward these conclusions. There are two barriers against this: the fortitude of the drafter and the chairman, and the collective conscience -- a sense of mutual responsibility, really -- which has grown up over the years in the estimative community. The latter works surprisingly well most of the time.
Another source of bias, again on the subtle side, lies in the fact that the estimators are American citizens, rooting for their country. If the policy proposal is not outrageously unreasonable, it is well-nigh impossible for us to bring ourselves to a firm estimate that the United States is bound to lose. We can make differential judgments in which some parts of the policy look more likely than others to produce the desired results. But at some point the course of action will usually culminate in a sheer test of will, and how can we bring ourselves to estimate that we will be the first to falter?1
But having bared all these misgivings, I remain persuaded that the policy maker is better off for having solicited an estimate of enemy reactions from intelligence agencies which, bureaucratically tied though they are to policy departments, are by training and inclination and conscience freer from commitments to policy than he and his colleagues are. And so we proceed with the drafting, knowing that we will have to continue solving and re-solving the terms-of-reference question as honestly as we can.
Top of Page
The Drafting
(Though the precepts which follow may all be golden truths, they are not likely to be of much help to the next estimator who has to draft a reaction estimate. This poor fellow will have to read the request, negotiate its unclarities with some ill-informed representative of the requester, exchange confusions with the newly appointed chairman of the estimate, and produce a first draft -- all within 24 or ten or even six hours. Theory is gray, Lenin remarked, but the tree of life is ever green. Or, as Stalin put it, cadres decide everything.)
Through the bitter experience of many redrafts I have learned that it is absolutely indispensable to begin a reaction estimate with an analysis of the situation preceding the U.S. action proposed. Usually, in fact, it is necessary to back up two steps: In many cases the U.S. policy presupposes an enemy initiative which then sets us into motion along the hypothesized line. What, then, did he mean by this action? Did he expect our reaction, in which case he presumably has a pre-planned counter-reaction? Or would he be taken aback by what we did and discover himself in the midst of some major miscalculation, unready with a next move and wholly uncertain about further U.S. intentions? These questions make a great deal of difference. Often no single answer can be given, and instead there emerge alternative analysis which must then be run out in parallel through the remainder of the estimate. Well why not, you say, but I promise you that this plays hell with drafting a paper simple and intelligible enough to be useful.
An ordinary NIE -- on Soviet military policy, say, or the outlook for Brazil -- has a theme, a tone. A competent drafter will marshal his facts and his ideas and construct an argument which leads to a single or a few major conclusions. If he does not, there's no use writing the estimate. If he does, and if he constructs well, then his betters2 may tug and pull at his paragraphs, alter his adjectives, and qualify his estimative passages, but his message still comes through.
It is fatal to approach a reaction estimate in this fashion. The drafter will encounter a long succession of close judgments as he works his way through the paper. Most of them will be near the 50-50 mark; if they were not, the estimate would not have been requested. He will make some of them in one direction, others in the opposite. He will estimate "desirable" reactions to some U.S. moves, "undesirable" ones to others. Out of the sum total of these, some general theme may in the end emerge, but he had better let this happen rather than aim at it. For his paper consists essentially of nothing but this succession of judgments, and many of them will be changed before the USIB finally signs off. If his draft is built around a theme, he will have to restructure, probably sooner rather than later.
But he can make his contribution. In thinking through the questions, he can try to find the turning points, the stage or stages which constitute, in Alsopian language, the "crunch." This, I think, is a real service. It tells the policy maker, not what will happen, but what to worry most or pray hardest about. It tells him about the moment of truth -- what its content will be and where, as he gropes along an uncertain path, he may expect to encounter it. To do this well is a triumph.
If he is lucky enough to find a turning point, the wise drafter will stop and point in both directions. He will give a scrupulously complete list of arguments why the enemy might do what we want him to do. Then he will give an exhaustive set of reasons why the enemy might do just the opposite. This is another service. It gets the policy maker to think about all the factors, the unpleasant as well as the pleasant ones. And it insures that he cannot dismiss the conclusion which follows on the grounds that the intelligence people forgot something important.
Top of Page
The Result
Once this is done, the drafter can be rather casual about which direction he chooses. It doesn't much matter; the Director will make up his mind, some USIB members will join him in the text, and others will take footnotes of dissent. But if the text has not laid the proper groundwork for these decisions, the drafter will have to endure an hour of confused argument at the USIB table and then start over again.
Nor, I would maintain, should it make very much difference what the USIB decides. The President surely would be silly to let his decision be determined by whether intelligence said the chances, were "slightly better than even" or "slightly worse than even." In the first place, the policy proposal has probably been changed in two or three minor ways, just enough to render the estimate slightly inapplicable, when he gets it. In the second place, the policy would assuredly be modified in the course of its implementation, enough to render the estimate more than slightly inapplicable. In the third and resounding place, everyone from drafter to President knows that the future is plain unknowable.
Well then, why write a reaction estimate? Because it is always a help to have the issues defined. Because the estimate may serve to highlight a forgotten or glossed-over problem. Because it may dispose of some wild, far-out ideas which heretofore had not been adequately confronted. Because a sober and at least partially disinterested accounting of risks and chances may not be available from any other source. Because subsequent policy argument can perhaps be more realistic.
You will notice that I have been very sparing of examples. This is because all reaction estimates are classified Top Secret and distributed to a small readership. In fact, they are not even accorded a permanent printing. Garden-variety NIEs get their conclusions distributed as rapidly as possible, after USIB approval, in an informal offset version. Subsequently the reader receives a handsome printed version of the full text. Reaction estimates get the first treatment but not the second. Thus their covers bear two of the finest, most lucid sentences ever written in the U.S. Government:
"NOTE: This is the estimate. No further versions will be published."
1 This irreducible element of bias probably saved us (the estimators, not the United States) in a series of Berlin estimates. The USSR's local advantages seemed overwhelming, and it was very hard to see how various U.S. courses of action could surmount the crisis. Gritting our teeth, we estimated some even chances. More important, the President gritted his teeth and made us right.
2 First the Board of National Estimates, then the representatives of the USIB agencies, finally the USIB itself.
Top of Page
SECRET
