Apache OpenOffice (AOO) Bugzilla – Issue 80122
Miscalculations in floating point operations
Last modified: 2007-07-31 16:38:32 UTC
I have spotted a strange issue that occurs when I try to subtract three float figures like: 0,71 - 0,7 - 0,01 The result should be 0, but isn't. Maxing out the number of digits, I get the following: 0,71000000000000000000 - 0,70000000000000000000 - 0,01000000000000000000 = -0,00000000000000010235 It appears to be the same with several other figures, too, yet not with all. I have put together a few examples in the file available through the URL. My guess would be a problem with periodic binary representations of certain values in combination with the internal rounding algorithm. Maybe you can spot it? Thanks for the great project, and keep up the good work! Thomas
Created attachment 47153 [details] Examples for good and bad calculations
Hi Eike, a double to Issue 18154 ? If so please close as double. Frank
The problem is not necessarily that I need more than 12 digits precision, but that the test for a-b-c == 0 is completely useless in these cases, which takes away a lot of reliability of the tool. So it's rather more than a nice to have. best, Thomas
dup *** This issue has been marked as a duplicate of 18154 ***
Closing.