Issue 116916 - Formatting accuracy problem of numbers exceeding decimal digits' representation precision.
Summary: Formatting accuracy problem of numbers exceeding decimal digits' representati...
Alias: None
Product: Calc
Classification: Application
Component: formatting (show other issues)
Version: OOo 3.2
Hardware: PC Linux, all
: P3 Trivial (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
Depends on:
Reported: 2011-02-10 16:59 UTC by kacubuntu
Modified: 2017-05-20 11:35 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---

strqange calc bug (9.28 KB, application/vnd.oasis.opendocument.spreadsheet)
2011-02-10 17:02 UTC, kacubuntu
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description kacubuntu 2011-02-10 16:59:53 UTC
i have found that when a list of values is summed , then the result is manually
typed in to another cell, that there can be a discrepancy of enormous magnitude.

this is regardless of the cell used, the sheet used, the file used (even if
copied from a csv file.)

the error is not replicated in OOffice 3.2 in Vista, only Ubuntu 10.10 . I am
using the OOffice packaged with Ubuntu.  I have tried to use an alternative, in
Ubuntu, but TNA (i cant seem to find a downgrade , and i have failed to install
OO 3.3).  It is most odd , as the calculation works for smaller values and
larger values, and appears to be restricted to a certain subset of values.  I
can email a summary file.

You might like to try the following calc on you versions to see if it works for
ROW1:76	73	72	78	74	71	75	77	79	675(sum)		675(user defined)		0(difference calc)
ROW2:=2^A1 …...		(sum)	(manual entry of sum value)
(I.e.1,206,564,636,373,190,000,000,000)		(difference value reported as 4429185024)

all other columns give correct answer of zero.  Formulae are entered in first
column or row respectively then dragged across.  Manual entry checked several
times and re-entered several times.  If first value of A2 is changed, a similar
error occurs.

I think OOfice is amazing apart from this .  Thanks very much keep up the good
Comment 1 kacubuntu 2011-02-10 17:02:28 UTC
Created attachment 75810 [details]
strqange calc bug
Comment 2 ooo 2011-02-10 18:32:10 UTC
This is a formatting accuracy problem. The numbers added actually give not the
same result as what is displayed, can be easily verified by manually calculating
the sum. If the SUM(...) result is copied to clipboard and then pasted special
(as number without formula) to the manual entry the difference is displayed as
zero even if the visual representation of the copied value doesn't change.
Comment 3 kacubuntu 2011-02-11 11:22:23 UTC
The summation is in fact quite accurate, yet is rounded by the spreadsheets
requirement for 15 significant figures, which i understand is due to floating
point numbers.  The error is not remedied by cutting and pasting without
formulas as advised.  If the following test is run, one will find that similar
calculations on either side (i.e 2^60 to 69 and sam for 80-89 ) are not
concerned by such 'formatting' errors. Further checks indicate that this problem
occurs in Vista as well as Ubuntu.  
If one checks the value of the figure, it suggests as you point out that this
may be the correct figure which indicates for this value (and maybe others)
restrictions of FPU normally seen with such equipment is excelled.   
Comment 4 Marcus 2017-05-20 11:35:11 UTC
Reset assigne to the default "".