Apache OpenOffice (AOO) Bugzilla – Issue 1182
TRUNC() makes weired calculation error
Last modified: 2012-08-29 09:54:55 UTC
Hi, I recognised a strange calculation error in OOo when playing with the function trunc in the spreadsheet-application: When entering '=TRUNC(9.63/3;14)' OOo returns a value of 3.21000000000001000000 (display of decimals set to 20) This *only* happens when telling the function to trunc at the 14th decimal. And this doesnt occur when letting OOo calculate the value of 9.63, eg enter '=TRUNC((2*3.4+2.83)/3;14)' and the results will in any case be 3.21000... (if you don't tell the function to trunc before the 2nd decimal of course) ciao Christian (running Caldera OpenLinux 3.1, kernel 2.4.2) PS: I found this because I tested if a bug in SO5.2 that was reported in the staroffice.de.support.starcalc <HINT> newsgroup <\HINT> still occours in OOo (SO5.2 returns a value of 3.209999... for '=TRUNC((2*3.4+2.83)/3;14)')
.
Reproduceble. But in my opinion this isn't a bug. I think this has something to do with handling of numerical mathematics.
Of course this is related to binary floating point representation of decimal values. The old SO52 behavior of getting 3.209999 as a result is fixed. Since our specs say that we calculate with an accuracy of 12 significant digits and the disturbing 1 occurs at the 15th digit (14th decimal place) I wouldn't consider this a real bug, though uneloquent. I'll keep this issue with lowest priority and have a look at it if time permits.
Hi, I can live with this one set to lowest priority - no question. But I think you didn't quite get my point: How would you explain, that when you set the decimals to a value greater than 14 you'll get the correct results? So I don't think this is related to the overall handling of the floating point numerics, but a special problem of the Trunc-function. (if OOo calculates =Trunc((2*3.4+2.83)/3;14) you won't get the '1' at the 14th decimal - you also wont't get the one when you just let OOo calculte =9.83/3) So again: why would the number of (displayed) decimals affect the (internally) calculated result? This, IMHO, is a bug. (But as I stated before, I myself wouldn't spend too much time on it, as long as there are more critical bugs that are waiting to be fixed - and as Eric wrote, the specs (where?) only cover 12 significant digits) ciao Christian
Well, as IEEEFP has ~ 15 digits of precision truncating to 14 can be problematic, I think...
The number of displayed decimals always affects the internal value. To cope with all the oddities of binary floating point representation there is always some internal rounding necessary for every visible representation, no matter if you, as the user, call a TRUNC() or ROUND() or similar or nothing thereof. This internal display rounding differs on the number of decimals to be displayed. If you're interested in how that works you may take a look at tools/source/generic/solmath.cxx method SolarMath::DoubleToString(). As for why there is a difference between =Trunc((2*3.4+2.83)/3;14) and =Trunc(9.63/3;14): it's just that =(2*3.4+2.83) and =9.63 produce a different binary pattern, because no number is exactly representable if it's not factorable by powers of 2 (0.125, 0.25, 0.5, 0, 1, 2, 4, and so on). Already adding two of those inaccurate numbers results in an accumulated error.
*** Issue 84626 has been marked as a duplicate of this issue. ***
Close it as invalid.