Bug 65792 - Using the INT function, POI and Micro Excel Calculation results do not match(ver:3.17)
Summary: Using the INT function, POI and Micro Excel Calculation results do not match(...
Status: RESOLVED FIXED
Alias: None
Product: POI
Classification: Unclassified
Component: XSSF (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal (vote)
Target Milestone: ---
Assignee: POI Developers List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-07 07:50 UTC by caoxj
Modified: 2022-06-24 00:00 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description caoxj 2022-01-07 07:50:55 UTC
I used the INT function in cell formula, POI(Ver3.17) and Micro Excel calculatng results do not match.
e.g.
Formulas : INT(880000000*0.84900%/3)
POI Calculation results:   2490399
Micro Excel Calculation results: 2490400
Comment 1 PJ Fanning 2022-01-07 07:52:15 UTC
v3.17 is not maintained - please try v5.1.0
Comment 2 PJ Fanning 2022-01-07 07:53:27 UTC
Use DataFormatter to format the cell - the result is stored in double format and that can lead to rounding issues (but this is also required to match how Excel works)
Comment 3 caoxj 2022-01-07 08:12:16 UTC
(In reply to PJ Fanning from comment #1)
> v3.17 is not maintained - please try v5.1.0

Thanks,
but my system can't use more than 3.17.
Comment 4 caoxj 2022-01-07 13:01:24 UTC
(In reply to PJ Fanning from comment #2)
> Use DataFormatter to format the cell - the result is stored in double format
> and that can lead to rounding issues (but this is also required to match how
> Excel works)

The problem does not occur when getting the value, but after calculating the sheet and saving the Excel file, when opening Excel file with Micro Excel, the invalid value calculated by POI is displayed.
1. calculating all cell of sheet(POI)
2. saving the Excel file(POI)
3. opening Excel file with Micro Excel(Micro Excel)
4. invalid value of a cell(value:2490399)
5. Double-click on the cell and press enter and the value is displayed correctly(value:2490400) 
So the formatting method is not available.
Comment 5 PJ Fanning 2022-01-07 13:27:14 UTC
One workaround is to call `workbook.setForceFormulaRecalculation(true);` - this will get excel to recalculate all the formulas when it loads the excel file.
Comment 6 PJ Fanning 2022-01-07 19:13:56 UTC
We are in the middle of a release so I don't want to merge any changes until that is done.

I have reproduced this issue and it actually happens in the `880000000*0.84900%` part. The multiplication is done using 2 java doubles and the result is not exact. The correct result is 7471200.0 but the POI java code produces something more like 7471199.999999.

I hacked the multiply code as below but it is a hack. The idea is to try to round doubles that are very close to integers to those integers.


     public static final Function MultiplyEval = new TwoOperandNumericOperation() {
         @Override
         protected double evaluate(double d0, double d1) {
-            return d0*d1;
+            BigDecimal bd = new BigDecimal(d0).multiply(new BigDecimal(d1));
+            BigDecimal possibleInt = bd.setScale(8, RoundingMode.HALF_UP);
+            if (possibleInt.doubleValue() == possibleInt.setScale(0, RoundingMode.HALF_UP).doubleValue()) {
+                return possibleInt.doubleValue();
+            } else {
+                return bd.doubleValue();
+            }
         }
     };
Comment 7 PJ Fanning 2022-01-07 20:18:17 UTC
https://github.com/apache/poi/pull/293 is an attempt to fix this
Comment 8 PJ Fanning 2022-01-09 13:11:02 UTC
While this issue does show some issues with how POI multiplies numbers and small rounding issues that occur - I think this would be a lot less of a problem if the user used the ROUND function instead of INT.

INT(1.99999) comes out as 1 but ROUND(1.99999, 0) is 2 - ROUND is more forgiving of rounding issues.
Comment 9 caoxj 2022-01-10 01:49:25 UTC
(In reply to PJ Fanning from comment #8)
> While this issue does show some issues with how POI multiplies numbers and
> small rounding issues that occur - I think this would be a lot less of a
> problem if the user used the ROUND function instead of INT.
> 
> INT(1.99999) comes out as 1 but ROUND(1.99999, 0) is 2 - ROUND is more
> forgiving of rounding issues.

I am soory!
The ROUNDDOWN function is known to solve this issue, but the Excel file is used as a contract and cannot be modified.

I want a fixed version of this issue of ver3.17, is it possible?
Comment 10 Andreas Beeker 2022-01-10 06:20:43 UTC
(In reply to caoxj from comment #9)
> The ROUNDDOWN function is known to solve this issue, but the Excel file is
> used as a contract and cannot be modified.

if the contract is based on Excel formulas, which includes truncations by design, it might be worth to communicate this error possibility with the parties involved ...

> I want a fixed version of this issue of ver3.17, is it possible?

only in the very rare case of security patches we would consider this, so not this time. but you could try to apply or adapt the patch yourself - just download the 3.17 src zip: https://archive.apache.org/dist/poi/release/src/
Comment 11 caoxj 2022-01-13 06:46:19 UTC
Please tell me how to fix V3.17, I will try to build
Comment 12 Andreas Beeker 2022-01-13 07:44:36 UTC
(In reply to caoxj from comment #11)
> Please tell me how to fix V3.17, I will try to build

get the source from comment #10 and apply/adapt the patch from comment #7.
if you face problems building it, it might be easier for you to upgrade to 5.1.0 ... which we recommend anyway
Comment 13 caoxj 2022-01-13 09:04:44 UTC
(In reply to Andreas Beeker from comment #12)
> (In reply to caoxj from comment #11)
> > Please tell me how to fix V3.17, I will try to build
> 
> get the source from comment #10 and apply/adapt the patch from comment #7.
> if you face problems building it, it might be easier for you to upgrade to
> 5.1.0 ... which we recommend anyway

Thank you.
After applying / applying the patch from comment # 7, this formula was OK, but many other formulas became NG.
Can this patch support both 5.x and 3.17?
Comment 14 PJ Fanning 2022-01-13 09:59:40 UTC
that comment 7 patch is work in progress - I need to debug issues - if you have any specific examples that fail, report them - I am not working on this at the moment because I have a lot of other stuff in progress

Did you ever try the solution in comment 5?
Comment 15 PJ Fanning 2022-01-13 10:01:43 UTC
And I have no intention of trying to patch POI 3.17 - the patch I am working on is for the latest POI release and that alone. If anyone wants to try it out on a separate code base, they are free to do so - but I'm not very interested in helping them to do that.
Comment 16 PJ Fanning 2022-01-15 00:36:45 UTC
added r1897066
Comment 17 caoxj 2022-02-03 04:45:44 UTC
Thank you very much.
Since the impact is small in a hurry, only INT is supported.

NumericFunction.java

	public static final Function INT = new OneArg() {
		protected double evaluate(double d) {
-			return Math.round(d-0.5);

+			if(d > 0){
+				return MathX.roundDown(d, 0);
+			}else{
+				return MathX.roundUp(d, 0);
+			}
		}
	};
Comment 18 PJ Fanning 2022-02-03 10:07:27 UTC
the current INT implementation is correct - see https://support.microsoft.com/en-us/office/int-function-a6c4af9e-356d-4369-ab6a-cb1fd9d343ef - I have added a test to prove it - and verified against my copy of Excel
Comment 19 caoxj 2022-06-24 00:00:37 UTC
(In reply to PJ Fanning from comment #18)
> the current INT implementation is correct - see
> https://support.microsoft.com/en-us/office/int-function-a6c4af9e-356d-4369-
> ab6a-cb1fd9d343ef - I have added a test to prove it - and verified against
> my copy of Excel

Thank you very much.