One position I mean to argue appears deceptively obvious: it is simply that there are important differences between men and machines as thinkers. I would argue that, however intelligent machines may be made to be, there are some acts that ought to be attempted only by humans. One socially significant question I thus intend to raise is over the proper place of computers in the social order. But, as we shall see, the issue transcends computers in that it must ultimately deal with logicality itself - quite apart from whether logicality is encoded in computer programs or not.
The lay reader may be forgiven for being more than slightly incredulous that anyone should maintain that human thought is entirely computable. But this very incredulity may itself be a sign of how marvelously subtly and seductively modern science has come to influence man's imaginative construction of reality.
Surely, much of what we today regard as good and useful, as well as much of what we call knowledge and wisdom, we owe to science. But science may also be seen as an addictive drug. Not only has our unbounded feeding on science caused us to become dependent on it, but, as happens with may other drugs taken in increasing dosages, science has been gradually converted into a slow-acting poison. Beginning perhaps with Francis Bacon's misreading of the genuine promise of science, man has been seduced into wishing and working for the establishment of an age of rationality, but with his vision of rationality tragically twisted so as to equate it with logicality. Thus have we very nearly come to the point where almost every genuine human dilemma is seen as a mere paradox, as a merely apparent contradiction that could be untangled by judicious applications of cold logic derived from a higher standpoint. Even murderous wars have come to be perceived as mere problems to be solved by hordes of professional problem-solvers. As Hannah Arendt said about recent makers and executors of policy in the Pentagon:
"They were not just intelligent, but prided themselves on being "rational" . . . They were eager to find formulas, preferably expressed in a pseudo-mathematical language, that would unify the most disparate phenomena with which reality presented them; that is, they were eager to discover laws by which to explain and predict political and historical facts as though those were as necessary, and thus as reliable, as the physicists once believed natural phenomena to be . . . [They] did not judge; they calculated . . . an utterly irrational confidence in the calculability of reality [became] the leitmotiv of decision making."
And so too have nearly all political confrontations, such as those between races and those between the governed and their governors, come to be perceived as mere failures of communication. Such rips in the social fabric can then be systematically repaired by the expert application of the latest information-handling techniques - at least so it is believed. And so the rationality-is-logicality equation, which the very success of science has drugged us into adopting as virtually an axiom, has led us to deny the very existence of human conflict, hence the very possibility of the collision of genuinely incommensurable human interests and of disparate human values, hence the existence of human values themselves.
I am greatly tempted to comment on this because I think in it Joseph Weizenbaum lays out some of the most intrinsically dangerous consequences of the adoption of that invention of Francis Bacon and Rene Descartes, the modern scientistic-materialistic conception of reality. I am deeply tempted to comment at length on that except that the philosopher he quoted, Hannah Arendt, did a far better job in that essay, Lying in Politics which he takes those too brief excerpts from.
The first explanation that comes to mind to answer the question “How could they?” is likely to point to the interconnectedness of deception and self-deception. In the contest between public statements, always over-optimistic, and the truthful reports of the intelligence community, persistently bleak and ominous, the public statements were likely to win simply because they were public. The great advantage of publicly established and accepted propositions over whatever an individual may secretly know or believe to be the truth is neatly illustrated by a medieval anecdote, according to which as entry, on duty to watch and warn the townspeople of the approach of the enemy, jokingly sounded a false alarm, and was the last to rush to the walls to defend the town against his imagined enemies. From this, one may conclude that the moresuccessful a liar is, the more people he has convinced, the more likely it is that he will end by believing his own lies.
In the Pentagon Papers, we deal with people who did their utmost to win the minds of the people, that is, to manipulate but since they labored in a free country where all kinds of information were available, they never really succeeded. Because of their relatively high station and their position in government, they were better shielded—in spite of their privileged knowledge of “top secrets”—against this public information, which also more or less told the factual truth, than those whom they tried to convince and of whom they were likely to think in terms of mere audiences, “silent majorities,”who were supposed to watch the scenarists’ productions. The fact that the Pentagon Papers revealed hardly any spectacular news testifies to the liars’ failure to create the convinced audience which they then could join themselves.
Still, the presence of what Ellsberg has called the process of “internal self-deception” is beyond doubt, but it is as though the normal process of self-deceiving were reversed; it was not as though deception ended with self-deception. The deceivers started with self-deception. Probably because of their high station and their astounding self-assurance, they were so convinced of overwhelming success, not on the battlefield but on the grounds of public relations, and so certain of thesoundness of their psychological premises about the unlimited possibilities in manipulating people, that they anticipatedgeneral belief and victory in the battle for people’s minds. And since they lived anyhow in a defactualized world, they did not find it difficult to pay no more attention to the fact that their audience refused to be convinced than to other facts.
The internal world of government, with its bureaucracy on one hand, its social life on the other, made self-deception relatively easy. It seems that no ivory tower of the scholars has ever better prepared the mind for wholly ignoring the facts of life than the various think tanks did for the problem-solvers and the reputation of the White House for the President’s advisers. It was in this atmosphere, where defeat was less feared than admitting defeat, that the misleading statements about the disasters of the Têt offensive and the Cambodian invasion were concocted. But what is even more important is that the truth about such decisive matters could be successfully covered up only in these internal circles by worries about how to avoid becoming “the first American President to lose a war” and by the always present preoccupations with the next election.
So far as problem solving, in contrast to public relations managing, is concerned, self-deception, even “internal self-deception,”is no satisfactory answer to the question “How could they?” Self-deception still pre-supposes a distinction between truth and falsehood, between fact and fantasy, which disappears in an entirely defactualized mind. In the realm of politics, where secrecy and deliberate deception have always played a significant role, self-deception is the danger par excellence; the self-deceived deceiver loses all contact, not only with his audience but with the real world which will catchup with him, as he can remove only his mind from it and not his body.
The problem-solvers who knew all the facts presented regularly to them in the reports of the intelligence community had only to rely on their techniques, that is, on the various ways of translating qualities and contents into quantities and numbers with which to calculate outcomes, which then, unaccountably, never came true, in order to eliminate, day in and day out, what they knew to be real. The reason why this could work for so many years is precisely that “the goals pursued by the United States government were almost exclusively psychological,”that is, matters of the mind.
Reading the memos, the options, the scenarios, the way percentages are ascribed to the potential risks and returns—“too many risks with too little return”—of contemplated actions, one sometimes has the impression that a computer ratherthan “decision makers” had been let loose in Southeast Asia. The problem-solvers did not judge, they calculated; their self-confidence did not even need self-deception to be sustained in the midst of so many misjudgments, for it relied on the evidence of mathematical, purely rational truth. Except, of course, that this “truth” was entirely irrelevant for the“problem” at hand. If, for instance, it can be calculated that the outcome of a certain action is “less likely to be a general war than more likely,” it does not follow that we can choose it even if the proportion were eighty to twenty, because of the enormity and incalculable quality of the risk; and the same is true when the odds of reform in the Saigon government versus the “chance that we would wind up like the French in 1954” are 70 percent to 30 percent.
That is a nice outlook for a gambler, not for a statesman, and even the gambler would be better advised to take into account what gain or loss would actually mean for him in daily life. Loss may mean utter ruin and gain no more than some welcome but nonessential improvement of his financial affairs. Only if nothing real is at stake for the gambler—a bit more or less money is not likely to make any difference in his standard of life—can he safely rely on the percentage game. The trouble with our conduct of the war in South Vietnam was that no such control, given by reality itself, ever existed in the minds of either the decision makers or the problem-solvers.
"They were not just intelligent, but prided themselves on being "rational" . . . They were eager to find formulas, preferably expressed in a pseudo-mathematical language, that would unify the most disparate phenomena with which reality presented them; that is, they were eager to discover laws by which to explain and predict political and historical facts as though those were as necessary, and thus as reliable, as the physicists once believed natural phenomena to be . . . [They] did not judge; they calculated . . . an utterly irrational confidence in the calculability of reality [became] the leitmotiv of decision making."
ReplyDeleteI think first of Halberstam's accounting of LBJ in Vietnam, "The Best and the Brightest." Of or Isaac Asimov's Foundation trilogy,which centered itself precisely on the ability of Hari Seldon to accurately calculate human "history" far, far into the future.
How he actually did that was a bit of narrative and rhetorical sleight of hand; it was the idea that it was possible which Asimov embraced. The funny thing is, except for a few movies here and there stealing mostly the titles of his books, Asimov is completely forgotten now. He was a Big Deal when he was alive; at least in science fiction circles. More and more I look back on his work and think "What a clown."
The determination, for example, that humanity would leave Earth and live happily elsewhere. Left out of that bold prediction was the reality of how gravity and biomes (not even a concept when Asimov was alive) create a space in which we can live and grow food, and those conditions can't be replicated elsewhere in the universe. Except, of course, through the magic of "terraforming." I guess we can alter Mars enough to make it grow food; if we can find a water source. And get the bacteria to adapt first, and then plant life to thrive long enough to create topsoil. That should take only a billion years or so. Then there's the problem of radiation, temperature, breathable atmosphere...
These things that pass for knowledge I can't understand.
"The trouble with our conduct of the war in South Vietnam was that no such control, given by reality itself, ever existed in the minds of either the decision makers or the problem-solvers."
Yup; although all of that, I'd contend, was simply "respectable" cover for the fact they couldn't imagine American ingenuity and technology handily defeating "the little brown men in black pajamas." That was the real root of the reason why defeat was unimaginable.
IMHO, anyway.
That essay is fifty years old, it is amazing that we have learned less than nothing, if anything we've gone backward.
ReplyDeleteYeah, nothing's reall changed.
DeleteIncluding my skills at not typiing what I mean to say. Obviously in my penultimate paragraph I should have said "not handily defeating."
Ah, well....