Issue 25368 - mis-calculation in autofill for numbers with one or two decimal places
Summary: mis-calculation in autofill for numbers with one or two decimal places
Alias: None
Product: Calc
Classification: Application
Component: code (show other issues)
Version: OOo 1.1
Hardware: All All
: P3 Trivial (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
: 103784 (view as issue list)
Depends on:
Reported: 2004-02-11 14:13 UTC by kyoshida
Modified: 2017-05-20 11:13 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Note You need to log in before you can comment on or make changes to this issue.
Description kyoshida 2004-02-11 14:13:35 UTC
When a column contains a series of numbers ranging from 1.1 through 4.5
incremented by 0.1, Calc's autofill mis-calculates the next number down as 2.1
where it should logically be 4.6.  Steps to reproduce this is as follows:

1. Enter 1.1 into A1, and 1.2 into A2.
2. Highlight the two cells, and drag it down to A35 so that A35 contains 4.5.
3. While cells A1 through A35 are still highlighted, drag the highlighted area
farther down.  Now you'll see the series of 2.1, 2.2, 2.3,...

I've tested it using a variety of types of numbers.  It seems that this
miscalculation occurs only when the numbers have one or two decimal places. 
I've tested it using whole numbers, and numbers with more than two decimal
places, but this problem did not occur.

Seems like the above steps are not the only way to reproduce this problem, but I
haven't discovered any pattern that's worth reporting yet.

To give proper credit, this bug was first reported by a member of the Japanese
localization team.

Comment 1 frank 2004-02-11 14:44:38 UTC
Hi Niklas,

ifr you drag the autofil handle in steps, no problem arises except you reach
4.5, after that the described things will happen.

Changed the Version to 1.1 because it's there also.

Comment 2 frank 2004-05-05 10:35:16 UTC
Due to time problems this is re-targeted to OOo2.0
Comment 3 kyoshida 2004-06-30 15:18:30 UTC
The problem may be due to the fact that only the values of the first 10 cells
are used to calculate the next number in the series.  As a demonstration, follow
Steps 1 and 2 in my original post, then

3. Instead of highlighting A1 - A35, highlight only A19 - A35.
4. Drag the selection by using the autofill handle.  This time you will get 3.9.

I speculate this is because the value of the 10th cell in the selection has 3.8,
and Calc mis-calculates the next number as 3.9.

If you change your selection in Step 3 above, you'll get a different result. 
But the number calculated is always one increment from the value of the 10th
cell in selection.


Comment 4 kyoshida 2004-07-03 16:40:11 UTC
Actually my previous comment is not entirely correct.  The calculated cell value
in Step 3 is exactly the value in the first cell in selection + 1.  This is
because the second fill action is interpreted as a "simple fill", instead of a
"linear fill" internally.

I've checked the code, and identified the area of the problem to be in
ScTable::FillAnalyse, where the evaluation for linearity of the selected cells
fails due to an issue with the precisions of the cell values involved in the

Don't know how to fix this issue, though.

Comment 5 kyoshida 2009-07-27 14:14:16 UTC
*** Issue 103784 has been marked as a duplicate of this issue. ***
Comment 6 tomwb 2009-07-28 01:58:51 UTC

Thank you for clarifying it as dupe.  I did no know if my example was anything
like what you had experimented with.

Comment 7 Marcus 2017-05-20 11:13:46 UTC
Reset assigne to the default "".