Issue 69798 - wrong computation of 2^48 + 1 - 2^48
Summary: wrong computation of 2^48 + 1 - 2^48
Status: CLOSED DUPLICATE of issue 69749
Alias: None
Product: Calc
Classification: Application
Component: code (show other issues)
Version: OOo 2.0.3
Hardware: All All
: P3 Trivial (vote)
Target Milestone: ---
Assignee: ooo
QA Contact: issues@sc
URL:
Keywords: oooqa
Depends on:
Blocks:
 
Reported: 2006-09-23 17:44 UTC by scoazera
Modified: 2010-11-12 10:34 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description scoazera 2006-09-23 17:44:08 UTC
Very very very severe!

--------------------------------

=2^48 + 1 - 2^48

the result is 0 instead of 1

--------------------------------

2^48 - 2^48 + 1

is OK

--------------------------------

=2^48 + 2 - 2^48

is OK

--------------------------------


=2^48 + 3 - 2^48

is OK

--------------------------------

etc.
Comment 1 ace_dent 2006-09-24 01:47:14 UTC
Tested with OOo 2.0.3, WinXP.
Confirmed. Entered the following cells:
A1 = 2^48
A2 = A1 + 1
A3 = A2 - 2^48

Result in A3 is 0 not 1. For other values (ie. A2=A1+2), the result is correct.
Would suggest this is some rounding issue due to the large numbers involved
(2^48 = 281,474,976,710,656).

Moved down to P2, since I suspect this bug may have been around for a while(?).

Regards,
Andrew
Comment 2 frank 2006-09-26 11:12:17 UTC
Hi Eike,

as discussed on the phone assigned to you.

Frank
Comment 3 ooo 2006-09-28 13:58:41 UTC
This is related to the "pull to zero" rounding of IEEE 754 double values
depending on magnitudes, which btw is also done in Excel, just that it shows up
there with 2^50: =2^50+1-2^50 results in 0.0

Fits into the "consolidate precision and rounding" topic.

*** This issue has been marked as a duplicate of 69749 ***
Comment 4 ooo 2006-09-28 13:59:06 UTC
Closing dup.
Comment 5 Unknown 2010-11-12 10:34:16 UTC
Created attachment 74713