If I call CTRow#isSetCustomHeight() on a row tag which has customHeight="false" the method call actually returns true. ECMA 376 specifications (on 18.3.1.73) say that the possible values for attribute "customHeight" of "row" node are defined by the W3C XML Schema boolean datatype so both "0" and "false" values should be accepted. Please note that Microsoft Excel writes "1" for "true" and removes the attribute for "false": in both cases CTRow#isSetCustomHeight() works fine, whereas CTRow#setCustomHeight(false) actually writes "false", which is not properly recognized by CTRow#isSetCustomHeight().
Thr CTRow class is generated from the OOXML XSD schemas. So the bug is XSSFRow or other XSSF class not comparing CTRow.isSetCustomHeight against all valid boolean values. If you have the time to find the offending XSSF classes and write a patch, it'd be greatly appreciated!
I did take a look, the generated isSetCustomHeight() actually does the following: return this.get_store().find_attribute_user(CUSTOMHEIGHT$16) != null; so it returns true if the attribute appears in the XML. The only place where POI itself uses isSetCustomHeight() is during clearing the manually set height, where a general isSet-call makes sense to use the default that is applied when the flag is not set at all. So I currently don't see much that can be done inside Apache POI here, please reopen the bug with more suggestions if you disagree.
I probably have misunanderstood the meaning of isSetCustomHeight(). It seems that the method I looked for was getCustomHeight(). Since in Java boolean getters do not usually starts with "get" I have got confused by "isSet...". Unfortunately, since these classes are autogenerated, patching with better naming or clear javadoc is not possible. Thanks for your answer and sorry again.