Apache OpenOffice (AOO) Bugzilla – Full Text Issue Listing
|Summary:||Formatting accuracy problem of numbers exceeding decimal digits' representation precision.|
|Component:||formatting||Assignee:||AOO issues mailing list <issues>|
|Status:||CONFIRMED ---||QA Contact:|
|Issue Type:||DEFECT||Latest Confirmation in:||---|
Description kacubuntu 2011-02-10 16:59:53 UTC
i have found that when a list of values is summed , then the result is manually typed in to another cell, that there can be a discrepancy of enormous magnitude. this is regardless of the cell used, the sheet used, the file used (even if copied from a csv file.) the error is not replicated in OOffice 3.2 in Vista, only Ubuntu 10.10 . I am using the OOffice packaged with Ubuntu. I have tried to use an alternative, in Ubuntu, but TNA (i cant seem to find a downgrade , and i have failed to install OO 3.3). It is most odd , as the calculation works for smaller values and larger values, and appears to be restricted to a certain subset of values. I can email a summary file. You might like to try the following calc on you versions to see if it works for you: ROW1:76 73 72 78 74 71 75 77 79 675(sum) 675(user defined) 0(difference calc) ROW2:=2^A1 …... (sum) (manual entry of sum value) (I.e.1,206,564,636,373,190,000,000,000) (difference value reported as 4429185024) all other columns give correct answer of zero. Formulae are entered in first column or row respectively then dragged across. Manual entry checked several times and re-entered several times. If first value of A2 is changed, a similar error occurs. I think OOfice is amazing apart from this . Thanks very much keep up the good work.
Comment 2 ooo 2011-02-10 18:32:10 UTC
This is a formatting accuracy problem. The numbers added actually give not the same result as what is displayed, can be easily verified by manually calculating the sum. If the SUM(...) result is copied to clipboard and then pasted special (as number without formula) to the manual entry the difference is displayed as zero even if the visual representation of the copied value doesn't change.
Comment 3 kacubuntu 2011-02-11 11:22:23 UTC
The summation is in fact quite accurate, yet is rounded by the spreadsheets requirement for 15 significant figures, which i understand is due to floating point numbers. The error is not remedied by cutting and pasting without formulas as advised. If the following test is run, one will find that similar calculations on either side (i.e 2^60 to 69 and sam for 80-89 ) are not concerned by such 'formatting' errors. Further checks indicate that this problem occurs in Vista as well as Ubuntu. If one checks the value of the figure, it suggests as you point out that this may be the correct figure which indicates for this value (and maybe others) restrictions of FPU normally seen with such equipment is excelled.