14-Aug-85 16:43:57-PDT,35455;000000000011
Return-Path: <[email protected]>
Received: from SRI-CSL.ARPA by SRI-CSLA.ARPA with TCP; Wed 14 Aug 85 16:39:30-PDT
Received:  from WISC-RSCH by SRI-CSL via DDN;  14 Aug 85 16:33:18-PDT
Received: from uwmacc.UUCP by wisc-rsch.arpa; Wed, 14 Aug 85 18:33:00 cdt
Received: by maccunix.UUCP (4.12/4.7)
       id AA12046; Wed, 14 Aug 85 18:08:57 cdt
Date: Wed, 14 Aug 85 18:08:57 cdt
From: [email protected] (Latitudinarian Lobster)
Message-Id: <[email protected]>
To: [email protected]
Subject: CPSR-Madison paper for an issue of risks?


The following may be reproduced in any form, as long as the text and credits
remain unmodified.  It is a paper especially suited to those who don't already
know alot about computing.  Please mail comments or corrections to:

Jeff Myers
University of Wisconsin-Madison         reflect the views of any other
Madison Academic Computing Center       person or group at UW-Madison.
1210 West Dayton Street
Madison, WI  53706
ARPA: [email protected]
UUCP: ..!{harvard,ucbvax,allegra,heurikon,ihnp4,seismo}!uwvax!uwmacc!myers
BitNet: MYERS at MACCWISC

-------------------------------------------------------------------------------

                C O M P U T E R    U N R E L I A B I L I T Y

                                   A N D

                           N U C L E A R    W A R

    Larry Travis, Ph.D., Professor of Computer Sciences, UW-Madison
             Daniel Stock, M.S., Computer Sciences, UW-Madison
            Michael Scott, Ph.D., Computer Sciences, UW-Madison
           Jeffrey D. Myers, M.S., Computer Sciences, UW-Madison
             James Greuel, M.S., Computer Sciences, UW-Madison
James Goodman, Ph.D., Assistant Professor of Computer Sciences, UW-Madison
    Robin Cooper, Ph.D., Associate Professor of Linguistics, UW-Madison
            Greg Brewster, M.S., Computer Sciences, UW-Madison

                              Madison Chapter
              Computer Professionals for Social Responsibility

                                 June 1984

          Originally prepared for a workshop at a symposium on the
                    Medical Consequences of Nuclear War
                        Madison, WI, 15 October 1983


1.  Computer Use in the Military Today
James Greuel
Greg Brewster

    Computer use in the military, as in other areas, is widespread and  grow-
ing at a dramatic rate.  It is estimated that in 1982 approximately 20% of the
programmers in the United States were writing software for  military  applica-
tions.   This Section describes the extent to which current defense department
policies are dependent upon computer technology.

1.1.  Computerized Weapons

    We first consider the direct use of computers in U.S. weapon systems.   A
prime  example  is a category of weapons called Precision Guided Munitions, or
PGMs.  These devices depend on computerized guidance systems  to  home  in  on
their  targets.   The  cruise  missile,  for example, can be programmed with a
digitized map of the surface over which it is to fly.  During  flight,  micro-
electronic  sensors  scan  the  terrain,  comparing it to the map.  If the two
views disagree, appropriate flight adjustments are  made.   The  sensors  also
scan  ahead  for  obstructions, allowing the missile to fly very low and avoid
radar detection.  The cruise missile  has  been  tested  successfully,  though
small changes in terrain appearance do cause problems for the guidance system.
For example, the amount of change in light and dark  terrain  features  intro-
duced  by  a snowfall can require a change in the digitized map.  In the event
of a war, any craters from previous explosions in the missile path could  cer-
tainly cause navigational difficulties.

    Currently under development are missiles that will use on-board computers
to "recognize" their targets.  These missiles can be fired in a general direc-
tion and allowed to search for an appropriate  target.   Prototypes  have  had
difficulties,  however,  in  finding  camouflaged targets.  They have also had
difficulty in distinguishing between objects with similar visual features: for
example, between tanks and large boulders.

    Computers are often used to assist  human  beings  in  the  operation  of
sophisticated weapons systems.  The F-15 fighter jet has 45 micro-computers on
board.  These are designed to improve maneuverability, locate targets at up to
100 miles, and aim missiles at those targets.  The Air Force has had consider-
able difficulty keeping the F-15  airborne.   Field  repairs  require  skilled
technicians  and computerized diagnostic equipment.  There is currently a pro-
posal under consideration to build large maintenance centers on the  East  and
West coasts solely for the purpose of maintaining F-15s.

    Computerized weapons are all susceptible to tactics known  as  Electronic
Counter-Measures--ECMs.   These  are  steps  taken to confuse or disrupt enemy
computer and radar systems.  They can be as simple as a pilot dropping  strips
of  aluminum (called chaff) to "foil" enemy radar, or they can be as sophisti-
cated as the electronic equipment  one  finds  aboard  such  aircraft  as  the
stealth  bomber.   This equipment attempts to fool enemy radar into that indi-
cating the plane is somewhere other than where it really is.

1.2.  Weapons Design

    A second use of computers in the military is in the  design  of  weapons.
An example is the development of self-forging fragments--disks of metal inside
a bomb that are shaped into conical, armor-piercing projectiles by  the  force
of  the explosion.  The idea for such weapons grew out of computer analysis of
the kinetic processes that take place inside a  detonating  warhead.   Similar
work  led  to  the development of area munitions, including so-called fuel-air
explosives, which precisely distribute and  ignite  enough  explosive  gas  to
level a full city block.

1.3.  Simulations

    A third use of military computers is in the area of simulation.   Comput-
ers  have  been used not only to model conventional warfare, such as dogfights
between jet fighters, but to simulate nuclear combat as  well.   For  example,
computers  have  been  used to predict the effects of a nuclear exchange under
various scenarios and to calculate the remaining  war-making  capabilities  of
each  side.   The  results  of these programs, which are highly susceptible to
programmer error and  user  bias,  have  in  part  formed  the  basis  of  the
government's claims of Soviet nuclear superiority.

1.4.  Command, Control, Communications, and Intelligence

    The fourth use of computer technology in the military is more  vulnerable
to  computer error and ECMs than any other.  An intricate system of computers,
satellites, telephone lines, radar facilities, surveillance planes,  and  air-
borne  and  underground  control  rooms  constitutes the eyes, ears, and, in a
frightening sense, the very brains of our national defense.   This  system  is
called Command, Control, Communication, and Intelligence--C3I.

    In the context of nuclear war, the purpose of C3I is two-fold:

(1)  To provide information to  the  right  people  quickly  enough  during  a
    surprise nuclear attack for the "right" decisions to be made.

(2)  To maintain a system of command, control, and communication once  nuclear
    war begins.

    Among other things, C3I monitors the world's air space;  observes  Soviet
tests  and  launchings; provides navigational assistance to our missiles, air-
craft, and submarines; controls  the  position  and  orientation  of  military
satellites; and collects and analyzes the results of the military's electronic
intelligence-gathering  operations.   There  are  currently  proposals  before
Congress  to add artificial intelligence capabilities to many computers in the
C3I system, allowing it to make recommendations and, perhaps, eventually deci-
sions  on  the  firing  of  missiles  when  there is too little time for human
decision-making.

    The heart of C3I is a collection of Honeywell H6000 computers located  at
various  sites  across  the  country.  These computers were built in the early
'70s on the basis of designs created in the '60s.  As many  people  know,  any
computer  that  old  may be likened in a sense to an early horseless carriage.
It would be easy  to  spend  several  billions  of  dollars  to  update  those
machines.  The result might be able to accomplish more tasks more quickly than
before, but it would not be more reliable:  the  sources  of  computer  errors
described in the remainder of this paper apply not only to obsolete computers,
but to "state-of-the-art" machines as well.   What  we  discuss  are  inherent
sources  of  unreliability  in  all computer systems.  No amount of money will
change the fact that computers make mistakes.  Only  sensible  human  policies
will keep those mistakes from starting World War III.

2.  Causes of Unreliability
Daniel Stock
Michael Scott

    Anyone who has done battle with a computerized  billing  system  realizes
that while computers are useful, they are by no means perfect.  Computers have
problems, and we can classify those problems into three general categories:

           problems with data
           problems with hardware
           problems with software

We discuss each of these categories in turn, and illustrate them with histori-
cal examples.

2.1.  Data Errors

    Data is the information fed into a computer for use in its  calculations.
Section 1 of this paper has already described the two most important causes of
data  unreliability  in  computerized  weapons  systems:  electronic  counter-
measures and the general havoc of battlefield conditions.

    Even a minor data error can have drastic consequences.  A town  in  Rhode
Island decided to computerize its tax records in 1972.  A misplaced letter 'P'
on a single punched card led the town to believe that its tax base  was  seven
million  dollars higher than it actually was.  As a result of the mistake, the
tax rate was set far too low, and the town found itself in a  nasty  financial
bind.

    A scarier mistake made the papers on November 9th, 1979.  By accident,  a
war  game  simulation tape was fed into the computers monitoring American air-
space at North American Air Defense (NORAD).  Strategic Air  Command  went  on
immediate  alert.   B-52  bomber  crews were sent to their planes, ten missile
intercepting fighter planes were scrambled, and U.S. missiles were readied for
launch.   The mistake was discovered in only 6 minutes, but it took 20 minutes
to return to normal status, and the Soviets had plenty of time to  notice  our
alert and take action of their own.

2.2.  Hardware Errors

    By "hardware" we mean the actual physical components of a  computer  sys-
tem.   These components are the least important source of computer unreliabil-
ity, but even so they cannot be ignored.  Physical components  wear  out,  and
even  brand  new  parts  can be confused by small amounts of natural radiation
that destroy the information they contain.  Careful quality  controls,  backup
components, and the storage of redundant information can reduce the likelihood
of hardware errors, but they cannot prevent them.

    Military computers are built to exacting specifications.  They break down
less  often  than their civilian counterparts, but they are not foolproof.  On
June 3rd, 1980, and again on June 6th, a faulty integrated circuit in a  Chey-
enne  Mt. computer announced a Soviet attack.  Again Strategic Air Command was
placed on alert.  Again human beings caught the mistakes in time.

    The Air Force does not publicize such incidents.  The three major  alerts
in  1979  and  1980  were  leaked to the press.  They prompted a congressional
investigation.  The investigation revealed that in the 18 month period  ending
June  30,  1980,  there were 151 false alarms at NORAD.  Most were prompted by
missile tests in Russia.  Five were serious enough to put us on alert  status.
The two not mentioned above were caused by a Soviet submarine test near Japan,
and by an old rocket body falling out of orbit.  Air Force officials quoted in
the  congressional study revealed that equipment failures produce two or three
false alarms each year.

2.3.  Software Errors

    The most serious source of unreliability in computer systems  is  neither
data  nor  hardware.  It is software--the programs that tell computers what to
do.  Typical military programs amount to thousands of pages  of  code.   Human
beings  are  simply  not  capable of constructing anything that large and that
complex without making mistakes.  Many errors are detected in simulated tests,
but  the  only practical way to find all mistakes is to put a program into use
and wait until it misbehaves.

    Programming a computer amounts to providing the machine, in advance, with
instructions  for every possible situation it may encounter.  That isn't easy.
Consider some examples:

    In one of the simulated flights of  the  space  shuttle,  the  astronauts
decided  to  abort their flight, then changed their minds, then tried to abort
the flight again.  The program running in their on-board computer went into an
"infinite  loop," rendering itself useless.  It had never occurred to the pro-
grammers that anyone would try to abort the same shuttle flight twice.

    Many readers will remember the terrible floods on the Colorado  River  in
June  of  1983.   According  to  the governor of Nevada, those floods were the
direct result of miscalculations by the Federal Bureau of  Reclamation's  com-
puters.   Programmers  did  not  anticipate  the bizarre weather caused by the
tropical storm El Nino.  Their programs kept too much water  behind  the  dams
all  spring,  and when the snow runoff came there wasn't enough room remaining
to hold it all.

    Even if programmers were smart enough to foresee all contingencies,  they
would still be left with the incredible task of explaining those contingencies
to a computer, in excruciating detail.  Slip-ups are inevitable.  In March  of
1979  the  Nuclear Regulatory Commission discovered a bug in the programs that
had been used to design five atomic power plants on the East  Coast.   Because
of  the bug, the plants would have been unable to survive earthquakes in their
area.

    There is an endless supply of  these  examples.   Most  readers  probably
remember  the computer communications problem that delayed the first flight of
the space shuttle.  They may not remember that two Mariner space flights  were
lost completely because of programming errors.  One program had a period where
there should have been a comma.  Another was missing the word 'NOT.'

    A more amusing mistake was discovered in  simulated  tests  of  the  F-16
fighter.  Left uncorrected, it would not have been at all amusing to the first
pilot to take his craft across the  equator.   On-board  navigation  computers
would promptly have turned the plane upside down.

2.4.  Electromagnetic Pulse

    One final source of unreliability is a phenomenon encountered only in the
presence  of nuclear explosions.  It is a problem so severe it threatens every
piece of electronic equipment in North America and dashes any hope of waging a
"limited"  nuclear war.  The phenomenon is know as EMP--ElectroMagnetic Pulse.
When an atomic bomb is exploded above the earth's atmosphere, gamma  rays  are
released.  These gamma rays collide with air molecules in the upper reaches of
the atmosphere to produce so-called Compton electrons that are captured by the
earth's  magnetic  field  and  that  lead to a massive jolt of electromagnetic
energy blanketing thousands of square miles.  A single  large  bomb  detonated
300  miles above Omaha, Nebraska would produce an effect roughly equivalent to
striking every medium-sized metal object in the continental United States with
a bolt of lighting, all at once.  Electric fields of between 25,000 and 50,000
volts per meter would wipe out the commercial communications and power  grids,
and  cripple  nearly all computers.  Military C3I would be devastated.  In the
aftermath of EMP, there would be no hope of coordinating the  strategy  neces-
sary  to  fight a "protracted," "limited" nuclear war.  EMP, combined with our
dependence on computerized controls, pushes military planners into a situation
where they must fire all their bombs at once, or lose the ability to fire them
at all.

3.  Artificial Intelligence and the Military
Robin Cooper

    A good idea of  the  kind  of  technological  research  the  military  is
involved  in  can be obtained by looking at the research program of DARPA, the
Defense Advanced Research Projects Agency.  This agency fulfills  the  central
research  function of the Department of Defense with an appropriation of 729.6
million dollars in 1983 (around 18% of DoD's total investment in  science  and
technology).  DARPA currently has five main focuses of interest:

(1)  Strategic technology.  This is mainly devoted to space-based  strategies,
    working  on  the assumption that "existing technologies have already made
    space a potential battlefield."

(2)  Tactical technology.  This involves the development of  weapons  used  in
    the air and on land and sea.  For example, it includes the development of
    cruise missiles which rely on computers for their ability to find targets
    and avoid hazards, such as enemy radar.

(3)  Directed Energy.  This involves laser and particle  beam  technology  for
    defense and communication.

(4)  Defense Sciences.  This involves basic research that will  be  useful  in
    other  projects,  such  as  the development of new kinds of materials and
    electronic devices.  It includes Systems Sciences, a project  to  improve
    man-machine systems and monitoring technology.  It focuses on such things
    as software for command and control and computer-based training  technol-
    ogy.

(5)  Information processing.  This involves developing technologies  that  can
    gather, receive and communicate information to human beings or other com-
    puters.  A large component of  the  research  in  this  program  involves
    Artificial Intelligence.

    The term "Artificial Intelligence" (AI) refers to  techniques  for  using
the  representation  of knowledge in computers in a sophisticated way in order
to perform tasks which involve fairly complex reasoning--reasoning of  such  a
kind that the machine is successfully imitating some aspect of the behavior of
a human being.  The imitated behavior may be very  limited.   Much  successful
current  research  is devoted to the writing of programs that the non-computer
scientist would not consider intelligent at all--not in the  normal  sense  of
the word as applied to human beings.

    AI has applications in a number of areas that are of direct  interest  to
the military.  For example:

(1)  Natural language  --  This  would  enable  military  personnel  (or  even
    presidents) to consult the computer directly without any training in com-
    puter techniques, perhaps even by voice so that they would  not  have  to
    type.   Pilots  would  be  able  to give oral commands to weapons systems
    while using both hands to fly their aircraft.

(2)  Vision -- An intelligent vision system would be  able  to  interpret  the
    light or radar signals it receives and make inferences (like perspective)
    based on this information.  Vision  systems  could  be  used  for  robots
    traversing  strange terrain (such as the moon) or for surveillance satel-
    lites or missiles.

    The reliance on AI is one of the scarier implications of current military
research.   "The  goal  is  to make it possible for computers to assist and/or
relieve military personnel in complex as well as routine decision-making tasks
which are information or personnel intensive, tedious, dangerous, or in situa-
tions which are rapidly changing."  This goal is scary because of the particu-
lar  vulnerability of AI programs to the sorts of software errors described in
Section 2.

    One common AI  technique  for  handling  inference  involves  programming
scripts  into  the  computer.   An expert system for restaurants, for example,
would have in its knowledge base an account of the kinds of things  that  nor-
mally happen in restaurants--a script for restaurants.  Imagine the same tech-
nique applied to nuclear war.  Is it possible to imagine a script that antici-
pates  every  eventuality?   No matter how inventive the programmer, is it not
likely that something will fail to go according to plan?  The effects  of  EMP
were overlooked for more than 20 years.  What else have we failed to consider?

    Many researchers are deeply convinced that machines will never be able to
make really intelligent decisions in human terms unless we can build a machine
that is exactly like a human and will go through all the learning  experiences
that  humans  go  through.  In the event of a nuclear attack a large number of
entirely novel situations will be arising in rapid succession.  It is meaning-
less  to  expect that AI systems will ever be able to provide security in such
an event.

    Even if it were possible to design complete and accurate  scripts,  there
would still be the problem of implementing the design in the form of a working
program.  AI systems are very large and complex.  Moreover, many of  the  sys-
tems  the  military  wants to build will be distributed, that is they will use
several different computers located in several different places  and  communi-
cating  with  each other.  No one programmer can ever have a complete overview
of such a system.  The programming task must be shared by a  large  number  of
people,  each  of  whom writes a single module.  Changes to improve one module
may prevent it from interacting properly with other modules, despite the  fact
that  every module does exactly what its programmer thought it should do.  The
resulting errors may require a large amount of practical use  of  the  program
before they surface and can be corrected.

    Military planners are faced with a dilemma: either they keep a system  at
the  state  of the art, which means constantly making changes, or they keep it
as it is for several years, only fixing bugs.  The first approach leads to  an
unreliable system; the second approach leads to an outdated system.

    With civilian computers, one can balance these considerations  in  a  way
that leads to sufficiently up-to-date systems with a tolerable level of relia-
bility.  It may be unpleasant to be stranded at O'Hare because  the  airline's
computers  lost  a  reservation,  but it is not a disaster.  It is a disaster,
perhaps, if the computer on board a 747 malfunctions and causes the  plane  to
crash, but even then it is not the end of the world.  It may indeed be the end
of the world if a computer error causes nuclear war.

4.  Implications
Larry Travis
James Goodman

    For many years, our stated national policy for deterrence  was  known  as
MAD--Mutual  Assured  Destruction--the  promise  that  an attack by the Soviet
Union would be answered by an all-out retaliatory strike by the United States.
This policy was modified under the Carter Administration to include the possi-
bility of a "limited nuclear war," and the idea was subsequently  endorsed  by
the National Republican Platform in 1980.  In this Section, we argue

(1)  that limited nuclear war is not feasible, and

(2)  that technical developments and policy decisions are  rapidly  increasing
    the likelihood of an accidental nuclear war.

    It is tempting to pursue a policy of Mutual Assured Destruction  for  the
indefinite  future.   After all, there have been no nuclear wars since MAD was
adopted.  By the same line of reasoning, however, a resident of New York  City
might  decide  that  Central Park was safe at night after walking through it a
couple of times without getting mugged.  We're not sure we'd try  it.   It  is
not at all clear that MAD is actually responsible for the nuclear truce of the
last 30 years.  We may have just been lucky.  Either way, there  is  consider-
able  evidence  that  MAD will not prove to be a viable policy for the next 30
years.

    Two important trends are undermining MAD.  The first is  an  increase  in
the effectiveness of modern weapons systems.  New and more powerful armaments,
coupled with delivery systems of  remarkable  precision,  have  made  a  first
strike  more  effective  than ever before.  The theory of deterrence says that
threatened retaliation can prevent such  an  attack.   But  to  be  effective,
either  the  retaliation must be initiated before the first warheads have been
detonated, or else the missiles and the communication systems must retain suf-
ficient  capacity  to  coordinate  a  massive  attack  afterwards.  The latter
option, "survivability," is becoming less and less likely.  Powerful and accu-
rate  weapons  can  destroy  all  but  the  most  thoroughly protected command
centers.  Moreover, as described in Section 2, electromagnetic pulse is almost
certain  to cripple most of what remains.  The first option, known as "launch-
on-warning," or "launch-under-attack," is emerging more and more as the  stra-
tegy of choice.

    Unfortunately, launch-on-warning is complicated  by  a  second  important
trend:  a decrease in decision-making time.  In the '50s, when nuclear weapons
were delivered by bombers, it would have taken ten hours or more to deliver  a
weapon  across  the  vast  distance required.  Either side could have detected
such an attack by radar and had hours to establish  readiness,  possibly  open
communications  with the other side, and select an appropriate response before
the first bombs exploded.  With the development of ICBMs  in  the  late  '50s,
that  warning  time was reduced to about 25 or 30 minutes.  With the installa-
tion of Pershing II missiles in Europe, the time available for the Soviets  to
respond  to  a perceived attack on their capital has been reduced to somewhere
in the range of 6 to 12 minutes.  An EMP burst can be triggered with even less
warning,  since the warheads need not be directly on target, and need not fall
to earth.  Recent substantial improvements in the Soviet submarine fleet  have
put the United States in much the same situation as its adversary.

    What are the consequences of these trends?  Certainly  the  effectiveness
of  a  first strike argues against the possibility of a "limited" nuclear war.
The arms race in recent years has focused not so much on numbers of weapons or
the  size  of  warheads,  but  rather  on the precision with which they can be
delivered.  Leaders can be annihilated quickly, or at least  their  leadership
role  can  be  destroyed  by the lack of communication links.  They need to be
able to assume that their policies will  be  carried  out  in  their  absence.
Under these conditions, the decision to launch missiles clearly resides with a
very large number of  people.   Almost  certainly,  every  submarine  carrying
nuclear  weapons  has  on  board a group of people who could collectively make
such a decision.  This delegation of authority alone would  seem  to  preclude
any  notion  of  a "limited response."  Combined with the fact that C3I can be
incapacitated easily and quickly, it makes a "protracted" war impossible.

    A even more serious consequence of recent trends is increased reliance on
inherently  unreliably  computer systems.  The effectiveness of a first strike
appears to make  launch-on-warning  more  or  less  essential.   The  lack  of
decision-making  time  makes  it  more  or less impossible, at least for human
beings.  Six or twelve minutes is hardly enough time to confirm an attack, let
alone alert leaders and allow them to make intelligent decisions.  The incred-
ibly short time from first warning until  the  first  incoming  weapons  might
explode makes it highly unlikely that a launch-on-warning policy could include
human intervention in the decision-making process.  Certainly  not  humans  at
the  highest  levels  of  command.  The temptation to place much or all of the
decision-making authority in the "hands" of automatic computer systems is  all
but  irresistable.   Computers  have  none  of  the  time limitations of human
beings.  They also have no common sense.

    We hope that the information in Section 2 has made it clear that  comput-
ers cannot be trusted with the future of the planet.  The theory of deterrence
depends totally and unconditionally on the rationality and  good  judgment  of
the  superpowers.   That rationality is suspect enough with humans in command.
With computers in command, the theory falls apart.

    Launch-on-warning, presumably under computer control, has been  advocated
by many people in policy-making positions.  Harold Brown, Secretary of Defense
during the Carter administration, is one example.  Warnings  he  made  to  the
Soviet  Union  during that administration suggested that the United States had
actually adopted such a strategy.  That suggestion appears to have been  inac-
curate.   Many  defense experts believe, however, that the Soviets have indeed
adopted launch-on-warning, certainly  since  the  Pershing  II  missiles  were
deployed,  if not before.  Whether or not the Russians currently employ such a
strategy, there are a number of objective pressures which appear to be pushing
both sides toward a policy of launch-on-warning.

(1)  The increased accuracy of delivery systems makes a  first  strike  poten-
    tially  more  effective,  particularly  against  land-based missiles, and
    makes rapid response imperative.  This is  necessary  because  the  land-
    based  missiles  may  be destroyed before they are launched.  The Soviets
    have greater concern in this regard than does the United States,  because
    the Soviets place greater reliance on land-based missiles.  However, both
    sides feel compelled to launch a retaliatory strike before  the  incoming
    missiles explode because the attack will also severely cripple communica-
    tions, making it difficult or impossible to coordinate retaliation.

(2)  The decrease in decision-making time means that  if  weapons  are  to  be
    launched  before they are destroyed, they must be launched almost immedi-
    ately upon detection of an attack.

(3)  The increased complexity of weapons and the greater number  of  resulting
    options means that much more time is necessary for human decisions.  More
    strategies must be considered, and there is a greater likelihood of human
    miscalculation.

(4)  The President's proposal for "Star Wars"  missile  defense  will  require
    decisions  to  be  made  with only seconds of warning, not minutes.  Com-
    pletely autonomous  computer  control  is  a  foregone  conclusion.   The
    installation  of  such  a  defense  system would amount to an adoption of
    launch-on-warning, since its use would be an act of war.

    Whenever a new application of computer technology is proposed, and a pol-
itical decision must be made about its development, it is necessary to compare
its benefits to its costs.  In order to evaluate a cost which may or  may  not
happen,  the normal technique is to multiply the probability of its occurrence
by its cost if it does occur.  But what cost can we assign to the  destruction
of  human  civilization,  or  even the complete loss of the human species?  If
this cost is infinite, then any probability at all of such an event makes  the
cost of deploying such a system also infinite.  Since 100 per cent reliability
is unattainable, it is only reasonable to limit the use of computers to appli-
cations in which we can tolerate an occasional mistake.

References

1.  Computer Use in the Military Today

C. Simpson, "Computers in Combat," Science Digest, October 1982.

R. R. Everett, "Yesterday, Today, and Tomorrow in Command, Control, and Com-
munications," Technology Review, January 1982.

"Command, Control Capability Upgraded," Aviation Week and Space Technology, 3
January 1983.

"An Upheaval in U.S. Strategic Thought," Science, 2 April 1982.

R. T. Smith, "They Have More EMT Than We," Science, 2 April 1982.

A. D. Frank, "Hello, Central, Give Me Bomb Control," Forbes, 23 November 1981.

W. Arkin and P. Pringle, "C3I : Command Post for Armageddon," Nation, 9 April
1983.

"The Conventional Weapons Fallacy," Nation, 9 April 1983.

L. Siegel, "Space Wars," The Progressive, June 1982.

2.  Causes of Unreliability

"Computers and the U.S. Military Don't Mix," Science, 14 March 1980.

"False Alerts and Faulty Computers," Science, 5 June 1981.

    Two articles by William J. Broad examining problems the military has had
    with computers.

"Electromagnetic Pulses: Potential Crippler," IEEE Spectrum, May 1981.

D. L. Stein, "Electromagnetic Pulse -- The Uncertain Certainty," Bulletin of
the Atomic Scientists, March 1983.

    Two good articles on the causes, effects, and implications for military
    strategy of EMP.

IEEE Spectrum, Volume 18, number 10 (October 1981).

IEEE Spectrum, Volume 19, number 10 (October 1982).

    Special issues of a fairly easy-to-read journal for electrical engineers.
    The topics are "Technology in War and Peace" and "Reliability."

Letters from the Editor, ACM SIGSOFT Software Engineering Notes, Volumes 4-8
(1979-1983).

    The editor of this journal, Peter G. Neumann, collects and publishes
    reports of intesting computer "bugs."  Particularly recommended are
    Volume 4, number 2; Volume 5, numbers 2 and 3; and Volume 7, number 1.

G. J. Myers, Software Reliability, John Wiley and Sons, 1976.

    An old but good textbook, peppered with suggestions on how to approach
    reliability and examples of software that didn't.

"Friendly Fire," Science 83, May 1983, page 10.

    Exocets and the Sheffield.

"Nevada Governor Says Errors Led to Flooding," The New York Times, July 5,
1983, page I-10, column 6.

    Brief note on computers and the flood.

3.  Artificial Intelligence and the Military

Defense Advanced Research Projects Agency, "Fiscal Year 1984 Research and
Development Program (U): A Summary Description," April 1983.

4.  Implications

R. Thaxton, "Nuclear War by Computer Chip," The Progressive, August 1980.

R. Thaxton, "The Logic of Nuclear Escalation," The Progressive, February 1982.

J. Steinbruner, "Launch under Attack," Scientific American, January 1984.

M. Bundy, "Ending War Before It Starts," Review of two books in the New York
Times Book Review, 9 October 1983.