Issue 96872

Summary: Bug in calculation, formula "=(n+1)-n" (where n >= 2^48) gives 0 as result
Product: Calc Reporter: lhotari <lari.hotari>
Component: codeAssignee: AOO issues mailing list <issues>
Status: CONFIRMED --- QA Contact:
Severity: Trivial    
Priority: P3 CC: discoleo, issues, maho.nakata, rb.henschel, shanzhu33
Version: OOo 2.4.1Keywords: oooqa
Target Milestone: ---   
Hardware: PC   
OS: Linux, all   
Issue Type: DEFECT Latest Confirmation in: ---
Developer Difficulty: ---
Attachments:
Description Flags
Document to compare Calc and intern calculation. The document contains Macros. none

Description lhotari 2008-12-04 05:12:45 UTC
I noticed that there is some kind of bug in OOCalc 2.4.1 (Ubuntu 8.04 Hardy,
32bit kernel: 2.6.24-22-386 #1 Mon Nov 24 17:51:53 UTC 2008 i686 GNU/Linux):

=500000000000009-500000000000008
gives zero as the result

I made some tests and I found out that the limit is 2^48. 

formula "=(n+1)-n" (where n >= 2^48) always gives 0 as the result.

=281474976710657-281474976710656
0 (incorrect)

=281474976710656-281474976710655
1 (correct)

=(POWER(2;48)-1)-(POWER(2;48)-2)
1 (correct)

=(POWER(2;48)+2)-(POWER(2;48)+1)
0 (incorrect)

=(POWER(2;48)+123)-(POWER(2;48)+122)
0 (incorrect)
Comment 1 lhotari 2008-12-04 05:18:31 UTC
I found this in google:
http://www.oooforum.org/forum/viewtopic.phtml?t=44241
Comment 2 Regina Henschel 2008-12-04 15:42:44 UTC
Having 52 bit for mantissa, I would expect correct results up to 2^51. The error
seems to be somewhere in minus-operation in Calc. If you do all the calculation
inside macros, it is correct till 2^52. See attached document.
Comment 3 Regina Henschel 2008-12-04 15:44:19 UTC
Created attachment 58515 [details]
Document to compare Calc and intern calculation. The document contains Macros.
Comment 4 discoleo 2008-12-04 21:48:25 UTC
I proposed some time ago, as far as I remember, a feature to warn the user IF an
overflow or underflow happens. While underflow can be safely ignored most of the
time, overflow ca NOT be ignored.

So, this is basically independent of the precision. There will be always an
overflow occurring, as long as OOo will use the current floating point
implementation, it is only a matter of when it is occurring. Warning the user is
in my opinion a wise decision. At least he might be aware that the result is wrong.
Comment 5 maho.nakata 2008-12-05 11:34:38 UTC
discoleo:
we need exception handling when overflow underflow occurs.

Comment 6 maho.nakata 2008-12-05 11:35:41 UTC
discoleo:
we need exception handling when overflow underflow occurs.
very interesting issue...
Comment 7 Regina Henschel 2008-12-07 23:42:05 UTC
The problem has been told earlier in issue 69798, which was closed as duplicate
of issue 69749. But I think the problem in issue 69749 is different. There is
the problem, that the decimal number has no exact binary representation. But
here the binary representation is exact, and it does not exceed the range of 52
bits. So it is neither a "overflow" problem nor a problem of converting decimal
numbers.

Internally, when using double type, you can calculate this example without
error, as you can see when executing the macros. But if the marco uses the cell
content it fails too. Therefore I think, that the error must be somewhere on the
way from the decimal cell content to its binary representation.

Comment 8 Regina Henschel 2012-08-29 16:30:04 UTC
*** Issue 120099 has been marked as a duplicate of this issue. ***