~Subject: RISKS DIGEST 13.55
REPLY-TO:
[email protected]
RISKS-LIST: RISKS-FORUM Digest Friday 5 June 1992 Volume 13 : Issue 55
FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
Contents:
The sinking of the trawler "Antares" (Brian Randell)
Another "But I'm Not Dead" story (Bill Winn)
*67 TOGGLES calling-number-id blocking (Bob Frankston)
One-Armed Bandits? (Bob Frankston, Roland Ouellette)
Girl Kidnaped by her Computer! (Misinformation About Computers) (Ellen Spertus)
Re: Girl killed in automatic car window (David Parnas)
Barry's Bug (Eric Haines)
German Unification Breaks Ohio Bell's Billing System (Adnan C. Yaqub)
Human namespace collisions (Frederick G. M. Roeber)
A name is a name is a name (Rick Simkin)
"Benevolent" Viruses (A. Padgett Peterson)
Software in the Air Scares: CAA and article authors respond (Simon Marshall)
The RISKS Forum is moderated. Contributions should be relevant, sound, in
good taste, objective, coherent, concise, and nonrepetitious. Diversity is
welcome. CONTRIBUTIONS to
[email protected], with relevant, substantive
"Subject:" line. Others may be ignored! Contributions will not be ACKed.
The load is too great. **PLEASE** INCLUDE YOUR NAME & INTERNET FROM: ADDRESS,
especially .UUCP folks. REQUESTS please to
[email protected].
Vol i issue j, type "FTP CRVAX.SRI.COM<CR>login anonymous<CR>AnyNonNullPW<CR>
CD RISKS:<CR>GET RISKS-i.j<CR>" (where i=1 to 13, j always TWO digits). Vol i
summaries in j=00; "dir risks-*.*<CR>" gives directory; "bye<CR>" logs out.
The COLON in "CD RISKS:" is essential. "CRVAX.SRI.COM" = "128.18.10.1".
<CR>=CarriageReturn; FTPs may differ; UNIX prompts for username, password.
ALL CONTRIBUTIONS CONSIDERED AS PERSONAL COMMENTS; USUAL DISCLAIMERS APPLY.
Relevant contributions may appear in the RISKS section of regular issues
of ACM SIGSOFT's SOFTWARE ENGINEERING NOTES, unless you state otherwise.
**************************************************************
*** If you cannot read RISKS on-line, try FAX! For info, ***
*** please telephone 310-455-9300 (or send FAX to RISKS at ***
*** 310-455-2364). EMail to
[email protected] . ***
**************************************************************
----------------------------------------------------------------------
Date: Fri, 5 Jun 1992 09:44:56 +0100
From:
[email protected]
Subject: The sinking of the trawler "Antares"
[Here is an article about an on-going court martial in the UK. The sort of
situation and allegations discussed are well-known to RISKs readers, so I
have provided the quote essentially just for the record.
Brian Randell
Computing Laboratory, The University, Newcastle upon Tyne, NE1 7RU, UK
EMAIL =
[email protected] PHONE = +44 91 222 7923 ]
COMPUTER BLAMED FOR SEA COLLISION (The Independent, 5 Jun 1992)
A Trainee submarine commander yesterday blamed a computer error for an accident
which sank a trawler and killed four Scottish fishermen. Lieutenant Commander
Peter McDonnell told a court martial at HMS Drake in Plymouth that he trusted
HMS Trenchant's computer system when it told him he was at least three miles
away from a possible collision with the Scottish trawler Antares. He said his
generation of submariners preferred to rely on the computer rather than a
manual plotting system which a senior submarine captain earlier told the
hearing was a more trustworthy method in busy waters.
Four men died in November 1990 when the Antares was dragged to the bottom of
the Firth of Clyde by HMS Trenchant. Lt Cdr McDonnell, 33, from Glossop,
Derbyshire, had just completed the last exercise of a six-month command course
known as the Perisher when the accident occurred at 2.18am. He denies six
charges of negligence. Yesterday he told the hearing that he had not even
known that Trenchant had passed close to the Antares and another fishing boat
five minutes before he ordered the submarine to turn around and head back
towards them.
The hearing continues today.
[
[email protected] found most of that in The London Times as well.]
------------------------------
Date: Wed, 3 JUN 92 15:27:41 EST
From:
[email protected]
Subject: Another "But I'm Not Dead" story
SORRY, BUDDY - IT SAYS RIGHT HERE THAT YOU'RE DEAD
(Indianapolis Star, June 3, 1992)
And you think you've had trouble dealing with apathetic bureaucrats?
Meet Eugene Smith of Doylestown, PA. The healthy 33-year-old has spent the
past 2.5 years convincing authorities he's not dead. The frustrating error
cost him his driver's license and his job. He still can't get a license, and
he's still fighting nine traffic violations that he says aren't his. Smith
traces the trouble to the theft of his wallet in 1988. He believes the thief
used his driver's license, racked up violations that led to the license
suspension, then died in a traffic accident.
In February 1990, a police officer stopped Smith and told him his car regis-
tration was expired and that state computer records showed he was dead. "He
said I was dead, and because of that I was not allowed to drive," said Smith.
"I agreed that it would be [a] hazard for a dead person to be driving."
Life isn't easy for an officially dead man. Without a license, Smith lost his
job as a driver for a warehouse. Without that job, he had to find a cheaper
place to live and take a job nearby, at a deli. Being an officially dead
taxpayer, no one in the state capital took him seriously. "I would call and I
could hear them say, `Oh, this is that guy again,' and I could hear them laugh
and they would say nobody there could help me," Smith said.
Finally, Susan Rakus, an aide to Democratic U.S. Rep. Peter Kostmayer, took his
case and persuaded the state motor vehicle agency to resurrect Smith [isn't
this against separation of church and state?]. But Smith still can't get a
license -- he's still accused of a string of years-old traffic violations.
"Obviously we dropped the ball on this," Rick Schoen, state transportation
department spokesman, said Tuesday.
William Joseph Winn
[email protected]
------------------------------
Date: Thu 4 Jun 1992 00:13 -0400
From:
[email protected]
Subject: *67 TOGGLES caller-id blocking
There has been a discussion going on in the Telecom forum about *67 which
TOGGLES(!!!!!) the caller-id blocking state of a phone line -- at least in
those areas with caller-id blocking. The rationale for requiring caller-id
blocking in some states is that there are situations where disclosing one's
location might be life-threatening as in the case of a shelter for battered
women or maybe a protected witness. Of course, there are also normal privacy
considerations.
If one always was sure of the default state of the line one was using a toggle
might work. But there is no way to determine the state beyond faith that the
telco's computer is exactly synchronized with one's expectations and that one
has is using the assumed CO lines on multi-line systems. If one is a visitor,
all bets are off. As from plain errors made in the business office or at the
CO, one reader pointed out that one some switches reloading the software loses
the settings. Another reader pointed out that *67 isn't an accident but the
specified behavior.
The stupidity (the word risk doesn't do justice to the situation) is obvious.
I'm more puzzled about how it came about. I generally lean towards incompetence
as an explanation rather than conspiracy but since some of the rationale for
requiring caller-id comes from public safety considerations, I'm surprised that
no one has challenged this approach as failing to satisfy this requirement and,
by providing the illusion of caller-id blocking, might increase the risk.
While on this subject, there is also the issue of access control over
information passed via signalling protocols. Telcos are assumed to have full
access and subscribers none. But some organizations can act as their own
telcos. The MIT ISDN switch comes to mind. Which side of the protection
barrier are they on? ANI is similar to caller-id but is nonblocked and
delivered when calling an 800 #. This means that if I give out my personal
800#, I will eventually (on the next bill) get their #.
------------------------------
Date: Thu 4 Jun 1992 09:31 -0400
From:
[email protected]
Subject: One-Armed Bandits?
In today's Wall Street Journal, there was a feature piece on a slot machine
tournament in Atlantic City. The problem was that the machines were returning
a 70.6% payoff rather than the 96.4% planned. "After the tournament ended and
the prizes were awarded, the manufacturer called back to report that the two
kinds of chips it shipped were incompatible with each other". Aside from all
the issues of how this might have happened, the real danger is soft failure
that are hard to detect. The only reason someone even looked for a problem was
the unique circumstances of a tournament which provided an environment to
notice the statistical anomalies Apparently there is no constant checking to
see that the statistical results match the predicted results.
The *67 (above) and this story both illustrate a risk of not understanding the
philosophical (as well as engineering) concept of closed-loop systems, i.e.,
those with feedback so that one can determine the result of an action. This is
a lesson that should feedback to nontechnology systems also.
[Chuck Weinstock <
[email protected]> also noted the slot machine
saga, as did Roland Ouellette, who added the note that follows. PGN
------------------------------
Date: Fri, 5 Jun 92 09:59:08 EDT
From: Roland Ouellette <
[email protected]>
Subject: One-armed bandits too efficient
This makes me wonder if anyone actually tests these machines: people at the
factory or regulators at the casinos. Also would this sort of error be noticed
only with an event like this and ordinarily go undetected?
Roland Ouellette
------------------------------
Date: Thu, 4 Jun 92 15:21:46 EDT
From:
[email protected] (Ellen Spertus)
Subject: Girl Kidnaped by her Computer! (Misinformation About Computers)
I've had up on my door an article from the 4/14/92 Weekly World News an
American tabloid) with a headline: "Girl, 13, kidnaped by her computer!" Here
is an excerpt:
A desperate plea for help on a computer screen and a
girl vanishing into thin air has everyone baffled ---
and a high-tech computer game is the prime suspect.
Game creator and computer expert Christian Lambert
believes a glitch in his game Mindbender might have
caused a computer to swallow 13-year-old Patrice
Toussaint into her computer.
"Mindbender is only supposed to have eight levels,"
Lambert said. "But this one version somehow has an
extra level. A level that is not supposed to be there!
The only thing I can figure out now is that she's
playing the ninth level --- inside the machine!"....
Lambert speculates that if she is in the computer, the
only way out for her is if she wins the game. But
it's difficult to know for sure how long it will take,
Lambert said.
"As long as her parents don't turn off the machine
Patrice will be safe," he said. "The rest is up to her."
Why am I posting this to comp.risks? Do I really think there is a risk of
people being kidnaped by computers? No (although at times, when I work on my
thesis, I wonder.) The risk is the misinformation people receive about
computers. I don't worry too much about the WWN, but I was concerned about an
educational show I watched last night, Mathnet, based on a segment of the PBS
educational television show, Square One. Mathnet is a spoof of the detective
show Dragnet, and the detectives use math to solve crimes. So far, so good,
but on last night's episode, the crime they solved was the kidnaping of a
baseball player whose disappearance had been unnoticed because he had been
replaced by an android which had been able to talk and play baseball. An
educational show would not show space aliens or magic, so the implication of
including human-like robots is that they are technically feasible.
Similarly, when I recently visited Epcot, an amusement park that is supposed to
be educational, the computer exhibit featured an electronic character that was
able to understand and even physically transport its human companion.
I expect (and enjoy) such unrealism in tabloids and in science fiction, but it
should not appear in educational settings. I suspect that a large percentage
of people, if asked, would say that a robot could currently be built that could
pass as human, based on all the misinformation they receive.
Ellen Spertus
------------------------------
Date: Wed, 3 Jun 1992 16:46:21 -0400
From: David Parnas <
[email protected]>
Subject: Re: Girl killed in automatic car window (Ian Spalding)
Isn't it just like our technocratic society to react to such an accident,
caused by a completely unnecessary luxury becoming too complex, by making it
even more complex? Wouldn't the simpler solution be to ban automatic windows or
even power windows instead of requiring another safety interlock? Nobody needs
such things but, unfortunately, there are car models in which you can't get an
ABS (good thing) without buying power windows (artificially induced desire). I
told my dealer that I was willing to pay extra for manual windows, but could
not get them.
------------------------------
Date: Thu, 4 Jun 92 09:34:57 -0400
From: Eric Haines <
[email protected]>
Subject: Barry's Bug
Viruses are a dime a dozen nowadays, but I thought this one was of particular
interest (though I do have to wonder if the issue of "Computing" magazine was
from April 1st...).
>From Communications of the ACM, June 1992 (vol.35, no.6), page 10:
Barry's Bug...
Viruses, as we all know, can play strange and frightening games with
computer-based data. Now, "Computing" magazine has reported a new strain that
plays some strange, and yes, frightening music. It's called the Barry Manilow
Virus - a phantom bug that's infiltrating a growing number of computer systems,
scaring users with such tunes as "Mandy" and "Copacabana." The virus is a
collection from the singer's "Greatest Hits" album. Once detonated, the virus
spins out a continuous stream of Manilow's million sellers. Experts are
working feverishly on an antidote for this plague.
-- Eric Haines
------------------------------
Date: Fri, 5 Jun 1992 21:44:51 GMT
From:
[email protected] (Adnan C. Yaqub)
Subject: German Unification Breaks Ohio Bell's Billing System
My family is enrolled in AT&T's World Reach-out plan. This plan provides
discounted calls to many countries throughout the world during designated
times, including what used to be West Germany. However there are no discounts
to what used to be East Germany (GDR). At our house, we call Germany (the
western part) a lot.
Yesterday we received our May phone bill from Ohio Bell. I noticed that after
around May 5 our calls to Germany did not have the Reach-out discount. Also,
the designation of the location called was changed from "Ger Fed Rep" to
"Germany".
I called AT&T, and a rate adjuster told me that the problem was with Ohio
Bell's billing software. It seems that their software was keying off the "Ger
Fed Rep" to apply the Reach-out discount, not the country code (49). Thus, in
May, when AT&T decided to change the designation "Ger Fed Rep" to "Germany",
the software broke.
AT&T credited me the difference, which was $21.00. I wonder how many other
phone companies will have the same problem and how many other people will be
affected.
Adnan Yaqub (
[email protected]) Allen-Bradley Company, Inc., 747 Alpha Drive,
Highland Hts., OH 44143, USA Phone: +1 216 646 4670 FAX: +1 216 646 4484
------------------------------
Date: Fri, 5 Jun 1992 21:46:29 GMT
From:
[email protected]
Subject: Human namespace collisions (Re: Earnest, RISKS-13.54)
With the increasing amount of casual communication these computer networks
(like usenet) are encouraging, this namespace collision situation is
likely to increase. I recently experienced this.
A few months ago, I posted an article to comp.realtime which quoted the US GAO
report on the Patriot missile failure. Somebody read it there, and reposted it
to the widely-read comp.risks forum. Shortly thereafter, I received an e-mail
message from another person named Fred Roeber. He works for Raytheon, the
makers of the Patriot system! His father, also named Fred Roeber, also works
for Raytheon. He saw my article, and immediately fired off letters to his
superiors, alerting them that the posting was *not* inside information from
either one of them, but public information from someone with the same name.
Luckily, it seems that no harm has come from this. In fact, two branches of a
family that hadn't known about each other can now fill in some gaps in the
family tree. But if one of his superiors had seen the article first, and acted
prematurely; or if the GAO or I had made a mistake that Raytheon might have
considered slanderous, the results could have been much worse for him.
The RISK seems to me to be that if we do not realize just how large this
increasingly popular global community is, we may mis-estimate the probability
of such a collision, and make mistaken assumptions about identity.
Frederick G. M. Roeber | CERN -- European Center for Nuclear Research
e-mail:
[email protected] or
[email protected] | work: +41 22 767 31 80
r-mail: CERN/PPE, 1211 Geneva 23, Switzerland | home: +33 50 42 19 44
------------------------------
Date: Fri, 5 Jun 92 10:05:06 CDT
From:
[email protected] (Rick Simkin)
Subject: A name is a name is a name
A little over a year ago, I was hounded by a collection agency for debts owed
by Richard Simkin, a car dealer in northern Illinois. It took about a month
(and a letter to the Better Business Bureau) to convince the agency that I
wasn't their man.
Late last fall, I applied for and received a Discover Card. About 4 months
later, Discover Merchant Services decided that my name matched that of Richard
Simkin of Roselle Motors and tried to collect his debts from me.
The pattern was to leave a phone message, or send a letter, telling me to call
Ranee. Phone messages (especially the first time, when all this was news to
me) never said why I should call. When I would call, Ranee was never in the
office, so I'd end up talking to someone else. I'd explain that I wasn't a car
dealer, and that they'd mixed me up with somebody else. They'd promise to take
care of the problem; once a supervisor told me that I shouldn't have gotten a
letter at all--he couldn't even figure out how it got to me, since my address
wasn't on the record of the delinquent merchant--and I should ignore it.
I've cancelled my account now, hoping that if there's no customer record, they
won't match it to their merchant record. I'm told that Discover policy
requires more than a matching name to claim that two records represent the same
person; and that by that policy, my record does not match that of the car
dealer's.
Computer Risks:
- Computer programs don't always reflect company policy.
- Flexible tools (such as a database query language and mail merge)
provide an easy means to act on wrong assumptions, and don't
always leave audit trails the way tailored applications can.
Rick Simkin UUCP: uunet!dlogics!rsimkin
Datalogics, Inc. INTERNET:
[email protected]
441 W. Huron St. PHONE: +1 312 2664437
Chicago, Illinois 60610-3498 USA FAX: +1 312 2664473
------------------------------
Date: Thu, 4 Jun 92 08:24:59 -0400
From:
[email protected] (A. Padgett Peterson)
Subject: "Benevolent" Viruses (Ts'o, RISKS-13.54)
>It all boils down to what your definition of "virus". My definition of "virus"
>is a piece of software which transmits itself from machine to machine without
>the knowledge or permission of either a user on the system or the system
>administrator of the machine.
While I agree with the first part, I must disagree with the second. A virus
is nothing more than a propagating program. "Knowledge or permission" has
nothing to do with the purpose of a virus. The only factor that is necessary
is some sort of rules base to maximise the probability of viable propagation.
Personally, I deplore the common use of viruses primarily because it is
inherently destructive whether or not the programmer was intentionally
malicious. The current crop of PC viruses (what most people know as viruses is
a function of personal computers - single tasking unprotected architectures) is
obviously only a subset of Dr. Cohen's envelope.
The incredible diversity of what the world considers a "PC" is what makes even
the most innocuous virus destructive in some cases. Take STONED for example. It
has only two functions: 1) To propagate 2) To occasionally display a message.
The fact that it (and its close variants) are statistically the most common
virus in the world today indicates that it is very good at (1).
However, in some cases, probably not understood by its creator, STONED is
destructive. Hard disks created without any hidden sectors (early FDISK),
floppy disks with nearly full root directories, and UNIX systems may become
unusable.
This type of problem also occurs with professional software and any reader can
name major products that would not run on a particular machine. (Years ago the
true test of a "100% compatible" PC was whether or not it could run "Flight
Simulator" properly. The interesting thing about FS was that the early versions
ran without any operating system, you just booted the PC with the FS disk in
"A:").
The point that I am trying to make is that very few people really understand PC
architectures at the BIOS/Microcode level and this is necessary to be able to
write "safe" low-level code. Most viruses are not intentionally destructive,
however their mistakes often have the same effect. Consequently, while I can
conceive of a "benevolent" virus, I would not necessarily trust one on my
systems.
Having said that, consider the following case: a LAN server that as part of the
logon script checks the client for the presence of resident security software,
verifies its integrity, and automatically updates the software on the client if
missing or an older version. This would meet the test of software that is
self-propagating and rules based. Even if user intervention is required to
continue, given the alternative of being denied access to the LAN, few will
refuse. Is this a "benevolent" virus ? (can give commercial examples).
Padgett
------------------------------
Date: Thu, 4 Jun 1992 22:01:09 +0000
From: Simon Marshall <
[email protected]>S
Subject: Software in the Air Scares: CAA and article authors respond
In RISKS-13.50, I reported an article concerning software errors in auto-pilots
of Boeings flown by British Airways, which appeared on the front page of the
``Sunday Telegraph'', May 17. My reason was to bring attention to the
article's content, which was that there were ``10 serious incidents involving
computer errors in January'' with BA.
I then made a number of comments, principally that this appeared to be a high
incidence rate; that the errors occurred in auto-pilots which I assumed to be
relatively simple systems (as compared to fly-by-wire) in which there is much
experience of design; that a comment made by a British Airways spokesman that
the software was CAA approved and tested for 100 hours before entering service
hardly reassuring.
Imagine my surprise when I received a phone call a week later from an
exasperated Dan Hawkes of the CAA. I am reporting this more than a week after
the fact, largely from memory. His main complaint was that the article had
been quoted without question, and that so often (as we know from newspaper
reporting of our own fields) these articles are of dubious reliability and
sensational. He made a further comment that he felt that academic input to the
issue of software reliability in aircraft was largely negative.
He reported to me that the software problems in the auto-pilots arose as a
result of a modification to software; the cause had been rapidly located and
fixed. Recovering from the initial shock of his call, I attempted to don a
journalistic hat and ask a number of questions.
I suggested that the MTBF of 10^-9 for software is unverifiable. This he was
happy to agree with, but stated that auditing and monitoring of all stages of
the software design and development gave a high level of confidence in its
performance. Overall design meant that no single possible on-board failure (be
it software of mechanical) could result in loss of aircraft integrity. He
stated that as all of these involved auto-pilots, there was never any danger to
the aircraft as pilots are always there to take remedial action when necessary.
In effect, that these were not serious errors at all. I think Nancy Leveson
(a name he was familiar with - ``an academic'') has pointed out the dangers of
making highly trained pilots into computer monitors.
I then raised the point that this certainly cannot apply to fly-by-wire
software, as in this situation pilots are not monitors but dependent users.
His answer was that the auditing and monitoring is more rigorous in the design
and development of fly-by-wire, and that (to paraphrase) ``there have not been
any failures yet''. Again his message was re-assurance; there is no serious
risk. I could not get a real answer as to where the 10^-9 figure came from.
I then decided to attempt to get in contact with the authors of the original
article, Robert Matthews and Christopher Elliot. Robert Matthews (Science
Correspondent) told me that the basis of the article had come from Flywise (as
pointed out by Martyn Thomas, RISKS-13.51), and had been checked out with BALPA
(union), BA and CAA (who were ``not all that helpful'') before publication. He
stood by the article, and added that the airline companies and authorities were
a closed world, and getting any information from them near impossible. Sounds
familiar? He had not received any satisfactory explanation of the software
reliability figure of 10^-9.
I swapped sources; a few issues of RISKS for a few tidbits from him. The issue
of Flywise states that the software incidents were due to ``software design
defect[s]''. An interesting titbit was a paper from Boeing on structural
airworthiness. According to their figures, in terms of hull loss rates per
departures, to 1988 the A320 was worse than any other commercial jet since the
Comet. Though none due to software; that hasn't happened yet.
Simon Marshall, Dept. of Computer Science, University of Hull, Hull HU6 7RX, UK
Email:
[email protected] Phone: +44 482 465181 Fax: 466666
------------------------------
End of RISKS-FORUM Digest 13.55
************************
