Issue 108228 - table: bool has negative sign when used in formula expression
 Summary: table: bool has negative sign when used in formula expression
 Status: RESOLVED FIXED Writer Application ui 4.0.0-dev All All P3 Trivial (vote) 4.0.0 Pedro Giffuni issues@sw Show dependency tree / graph

 Reported: 2010-01-11 18:13 UTC by Joe Smith 2013-07-13 14:24 UTC (History) 1 user (show) issues DEFECT --- ---

Attachments
Sample document showing problem (10.18 KB, application/vnd.oasis.opendocument.text)
2010-01-11 18:14 UTC, Joe Smith
no flags Details

 Note You need to log in before you can comment on or make changes to this issue.
 Joe Smith 2010-01-11 18:13:52 UTC Formulas in Writer may use conditional expressions, which produce a boolean result. By convention, the true/false result is represented by the numeric values 1 and 0. This is also the case in a Writer if a formula contains a bare conditional expression: 1 eq 1 -> 1 1 eq 0 -> 0 However, if the conditional expression is used as part of a larger formula, and the conditional produces a true result, the conditional result has a negative sign in the larger expression, and causes incorrect results: 1*(1 eq 1) -> -1 (should be 1) 1-(1 eq 1) -> 2 (should be 0) Tested in OOo 3.2rc1, 3.1.1 and 2.4.2 on Fedora Linux 11. Joe Smith 2010-01-11 18:14:50 UTC Created Sample document showing problem michael.ruess 2010-01-12 12:13:36 UTC MRU->OS: see attached document; a formula like 1*(1 eq 1) will give -1 as result. Oliver Specht 2010-01-12 14:37:22 UTC ->jes: What makes you think a boolean TRUE has to be +1? Joe Smith 2010-01-12 16:34:38 UTC @os: Of course the _test_ for T/F may check for zero/non-zero and the sign does not matter, but when the result of the expression is used for doing math, the sign matters a great deal. First, it is not internally consistent: the expression returns +1 when used alone, but -1 when used in a larger expression. Second, it is not consistent with common use, including Calc, where boolean expressions produce 0/1 when used in an expression. Third, the status quo is less useful than the consistent production of 0/1: since Writer provides no explicit conditional expression the value of the conditional itself can be used as a workaround in some cases. Having a consistent and familiar behavior would surely improve any such use. Of course, fixing this would be a low priority, but at least the unexpected behavior is on record. Pedro Giffuni 2013-01-23 16:26:52 UTC The issue is very well explained in the LibreOffice list: http://lists.freedesktop.org/archives/libreoffice/2010-December/004303.html "The problem behaviour occurs because the boolean expression "true" evaluates to -1 rather than +1 in formulas in Writer. So if, for example, you create a table formula in Writer, (Table->Formula) and enter something like 1+(2==2), it evaluates to 0 rather than 2. It turns out the reason for this is buried in sbxdef.hxx, where we have // The numeric values of TRUE and FALSE enum SbxBOOL { SbxFALSE = 0, SbxTRUE = -1 }; ..." The proposed fix is in file sw/source/core/bastyp/calc.cxx function SwSbxValue::MakeDouble change the comparison in 1763 so that the action happens when GetType() is equal to either SbxSTRING or SbxBOOL Pedro Giffuni 2013-01-23 23:02:46 UTC (In reply to comment #5) > The issue is very well explained in the LibreOffice list: > http://lists.freedesktop.org/archives/libreoffice/2010-December/004303.html > ... > > The proposed fix is in file > sw/source/core/bastyp/calc.cxx function SwSbxValue::MakeDouble > > change the comparison in 1763 so that the action happens > when GetType() is equal to either SbxSTRING or SbxBOOL Mattias Johnnson has given permission to adopt his patch under ALv2. I have a small variant that I will be committing after I confirm it works. SVN Robot 2013-01-24 02:43:26 UTC "pfg" committed SVN revision 1437842 into trunk: i108228 - table: bool has negative sign when used in formula expression. Pedro Giffuni 2013-01-24 02:45:25 UTC Verified to work with the test document. Thanks to Mattias! hdu@apache.org 2013-07-11 06:47:35 UTC Adjusting the target to the release that will contain the fix. Joe Smith 2013-07-13 14:24:37 UTC Testing AOO400m3(Build:9702) - Rev. 1502185 2013-07-11 08:22:42 (Thu, 11 Jul 2013) - Linux i686 on Fedora 17 Looks great: all tests in sample document give expected results now, although it requires Tools > Update > Update All (or Fields) to refresh the calculations. Many thanks to everyone who helped fix this!