Atucc.1671
net.micro
utzoo!decvax!duke!tucc!dgary
Fri Apr 9 15:04:43 1982
Re: whuxlb.206: IBM PC divide bug revisited
My experimentation with a PC confirms earlier
reports that the arithmetic bug is indeed in the output
routines.
For latecomers, the problem is this: Dividing .1 by 10
yields .001 if and only iff double precision
arithmetic is used. PC BASIC uses a #
to indicate double precision, so
PRINT .1#/10# prints .001.
This is also the case with double precision
variables.
What's going on? Well, .1#/10# is
stored as something like 9.9999999d-3.
This is within the fuzz-range of 10d-3,
but the normal output routine messes up
the normalization and decimal placement
and gives us .001 (1d-3) instead of the
corect .01 (10d-3).
For obvious reasonns, this only happens
when rounding up, so .1# * .1# is okay.
The long-term solution is to badger IBM
into giving us new ROMs. In the meantime,
PRINT USING will eliminate the problem.
-Gary Grady,
Duke User Services (dgary)
-----------------------------------------------------------------
gopher://quux.org/ conversion by John Goerzen <
[email protected]>
of
http://communication.ucsd.edu/A-News/
This Usenet Oldnews Archive
article may be copied and distributed freely, provided:
1. There is no money collected for the text(s) of the articles.
2. The following notice remains appended to each copy:
The Usenet Oldnews Archive: Compilation Copyright (C) 1981, 1996
Bruce Jones, Henry Spencer, David Wiseman.
