Apache OpenOffice (AOO) Bugzilla – Issue 30309
on basic module, return wrong result from mathematical operation
Last modified: 2013-02-24 21:07:31 UTC
Hello i use a basic program, runing on spreadsheet. i recognize that a wrong result from division operation. here is the operation -1*ty(ll)/ubound(kolon_nokta()) in this operation ty() is an array as single when ll=1 ty(1)=-0.11 ubound(kolon_nokta())=2 result must be 0.055 but programme calculates 0.0549999997019768 please help me
It sounds like a rounding error which sometimes cant be avoided. Can I ask, if you put =-1*ty(11) into one cell and =ubound(kolon_nokta()) into another cell, do they individually give you the correct values? (make sure you display all the decical places) If so, does =-0.11/2 in a cell on it's own give you the correct result? Are you certain it's the division operator that is hte problem? Can you please attach the sheet that is giving you the problem? Doesn't seem Priority one, this is for "issues that make it impossible to work with the version", im switching it to P3.
erdurmaz emailed me the following script REM ***** BASIC ***** option explicit Sub Main dim a as single dim b as integer a=-1*.55 b=2 msgbox a/b End Sub I was able to duplicate the behaviour however, it seems to just be the fact a single was used. A double would have provided the necisary precicion so if the code is changed to: REM ***** BASIC ***** option explicit Sub Main dim a as double dim b as integer a=-1*.55 b=2 msgbox a/b End Sub it produces the correct result. Belive this is not an issue.
Invalid, not an issue with OOo
closed as invalid