Issue 107509 - Wrong rank computation of sum values
Summary: Wrong rank computation of sum values
Alias: None
Product: Calc
Classification: Application
Component: code (show other issues)
Version: OOO310m19
Hardware: PC Windows XP
: P3 Normal (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
Depends on:
Reported: 2009-12-07 21:10 UTC by twonkeys
Modified: 2013-02-09 04:02 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: 3.4.1
Developer Difficulty: ---

Wrong computation of rank (9.72 KB, application/vnd.oasis.opendocument.spreadsheet)
2009-12-07 21:11 UTC, twonkeys
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description twonkeys 2009-12-07 21:10:15 UTC
In two rows, I computed the simple sum of four numbers.  Then I computed the
rank of these two sum values.  Although the sums are identical, the ranks come
out as 1 and 2.

When comparing the two sums with the "if" function, they are regarded as equal.

When copying both rows and subtracting 10 from both of the first values, the
ranks come out as 1 and 1 (as expected).

I attached a simple sample document, where the sums are marked blue, the ranks
yellow and the "if" formulas green.

I don't know if it matters, but I'm using OO in its German version.
Comment 1 twonkeys 2009-12-07 21:11:22 UTC
Created attachment 66544 [details]
Wrong computation of rank
Comment 2 twonkeys 2009-12-07 21:22:38 UTC
I just double checked this file on a x86_64 Linux system (Ubuntu 9.10) with the
exact same OO build number (OOO310m19).  The behaviour is equally wrong here.
Comment 3 discoleo 2009-12-09 16:31:08 UTC
Weird issue, probably because of rounding in one case, and no rounding in the
other case.

The 2 sums are probably different using floating point arithmetic, though Clac
is likely to round them to the nearest "shortest number" when displaying the
result. This result may however behave oddly in subsequent operations: the
rounded value might be used by the IF() function, while the extended result is
used by the RANK() function.

The handling of floating point results in Calc needs to be confirmed by a
someone familiar with the code, though (either Eike or Niklas).
Comment 4 Rob Weir 2013-02-09 04:02:49 UTC
At first glance this does seem like a numeric precision limitation, but when I display the numbers to fill display precision, the first set is the same:


According to the ODF specification, "If a number in Data occurs more than once it is given the same rank, but increments the rank for subsequent different numbers".

So I'm going to mark this as confirmed, and recommend that a developer investigate to see what is going on here.  It might be just a precision issue in the end, but it might be something else.