```Date: Fri, 3 Jul 1998 02:51:30 GMT Reply-To: Melvin Klassen Sender: "SAS(r) Discussion" From: Melvin Klassen Organization: University of Victoria Subject: Re: Precision Content-Type: text/plain; charset="us-ascii" On Wed, 1 Jul 1998 20:06:04, Myra.Oltsik@responseinsurance.com wrote: > I got a \$0.01 difference in a value calculated in MVS > and a value calculated on the PC (NT). The calculation is: > > (\$269.90 / 124) * 31 > > The PC gives the results as \$67.475, rounding to \$67.48. > With MVS, it rounds to \$67.47. I want to document the reason for the difference > as this data is used for Accounting purposes. The 1 penny difference isn't a big deal, > but if an auditor finds it in the future, I want the explanation ready. Using REXX and OS/2 on a desktop system with an Intel Pentium processor, I can get many "answers": say (269.90/124)*31 67.4749999 ................................................ REXXTRY.CMD on OS/2 say 269.90*(31/124) 67.4750 ................................................ REXXTRY.CMD on OS/2 say (269.90*31)/124 67.475 The first result rounds "down" to 67.470000, but the others do not. The "real" problem is that '269.90' cannot be stored without "round-off" error, and multiplying by 1/4 (suprisingly!) affects the answer, on some platforms. Why? The IBM System/390 stores the number as "some fraction multiplied by some power of 16", while other platforms store the number as "some fraction multiplied by some power of 2". On the System/390, dividing by 4 causes both the "fraction" and the "exponent" to be altered, and one or two bits of the fraction are being truncated. On the power-of-two platform, only the exponent needs to be altered, while the fractional-part remains unchanged. ```

Back to: Top of message | Previous page | Main SAS-L page