Bug 18947

Summary: [PATCH]Some String values garble on different platforms
Product: POI Reporter: Toshiaki Kamoshida <kamoshida.toshiaki>
Component: HSSFAssignee: POI Developers List <dev>
Status: RESOLVED FIXED    
Severity: major CC: elvira_gurevich
Priority: P3    
Version: 2.0-dev   
Target Milestone: ---   
Hardware: All   
OS: other   
Attachments: Testcases to find problems easily. \testcases\org\apache\poi\hssf\usermodel\TestSBCSOnDefaultEncoding.java
PATCH to fix FooterRecord,HeaderRecord,BoundSheetRecord,and NameRecord
PATCH to fix ALL problems.
Sorry,my private text editor seems to be dealing about the RETURN CODE at StringUtil incorrectly.If this patch is incorrect,please you insert StringUtil#putUncompressedUnicode() at the right position anywhere...

Description Toshiaki Kamoshida 2003-04-11 10:17:21 UTC
Because Record#fillFields() in some Concrete Record classes,use Constractor of 
java.lang.String with no Encoding Indicator to decode 8bit compressed 
charcter's byte array,so some string values will garble on different 
platforms,depends on the effect of "DEFAULT ENCODING OF RUNTIME".
Comment 1 Toshiaki Kamoshida 2003-04-11 10:19:28 UTC
Created attachment 5795 [details]
Testcases to find problems easily.  \testcases\org\apache\poi\hssf\usermodel\TestSBCSOnDefaultEncoding.java
Comment 2 Toshiaki Kamoshida 2003-04-11 10:28:24 UTC
Sorry,I can't find ways to realize other bugs with testcases outside 
hssf.usermodel...
Please add testcases to find and fix others.
The places using constractor of String are;
record\FontRecord.java(167)
record\FormatRecord.java(135)
record\LabelRecord.java(160)
record\StringRecord.java(137)
record\StyleRecord.java(137)
record\UnicodeString.java(154)
record\WriteAccessRecord.java(118)
record\formula\StringPtg.java(82)
Comment 3 Toshiaki Kamoshida 2003-04-11 10:43:38 UTC
Created attachment 5797 [details]
PATCH to fix FooterRecord,HeaderRecord,BoundSheetRecord,and NameRecord
Comment 4 Toshiaki Kamoshida 2003-04-11 10:46:36 UTC
The testcase is using sun.io.Converters to emulate the situation easily,
So it may works on Sun JRE.Please attention.
Comment 5 Toshiaki Kamoshida 2003-04-21 10:49:42 UTC
I found to realize the bug fully in your env easily.
Please tested by BiffViewer (I didn't know there is BiffViewer XP)with the 
bootstrap like this;

public class Test {
  public static void main(String[] args){
   System.setProperty("file.encoding","UTF-16");
   sun.io.Converters.resetDefaultEncodingName();
   org.apache.poi.hssf.dev.BiffViewer.main(args);
  }
}

and you can emulate executing POI at another platforms.
Please check the sheets Elvira Gurevich submitted at Bug 17791 with this.
Comment 6 Toshiaki Kamoshida 2003-04-21 10:51:55 UTC
Created attachment 5923 [details]
PATCH to fix ALL problems.
Comment 7 Toshiaki Kamoshida 2003-04-21 11:08:19 UTC
Of course,-Dfile.encoding=NATIVE_ENCODING_NAME may be able to test it,but in my 
env,this way cause;

Error occurred during initialization of VM
java.lang.Error: Can't find java.home ??

It means,when solving System property java.home,because of the Decoder related 
to NATIVE_ENCODING_NAME,VM can't solve the property.

So it is more safety that we change defaut encoding after main thread is 
started.
Comment 8 Andy Oliver 2003-04-21 21:18:50 UTC
Here is what I got: 

D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\NameRecord.java:759: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_12_name_text = StringUtil.getFromCompressedUnicode(data, 15 +
offset,
                                           ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\NameRecord.java:767: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_14_custom_menu_text       = StringUtil.getFromCompressedUnicod
e(data, start_of_custom_menu_text + offset,
                                                        ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\NameRecord.java:771: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_15_description_text       = StringUtil.getFromCompressedUnicod
e(data, start_of_description_text + offset,
                                                        ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\NameRecord.java:775: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_16_help_topic_text        = StringUtil.getFromCompressedUnicod
e(data, start_of_help_topic_text + offset,
                                                        ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\NameRecord.java:779: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_17_status_bar_text        = StringUtil.getFromCompressedUnicod
e(data, start_of_status_bar_text +  offset,
                                                        ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\FontRecord.java:167: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
                field_11_font_name = StringUtil.getFromCompressedUnicode(data, 1
6,
                                               ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\HeaderRecord.java:122: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_2_header     = StringUtil.getFromCompressedUnicode(data, 3 + o
ffset, // [Shawn] Changed 1 to 3 for offset of string
                                           ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\FooterRecord.java:122: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_2_footer     = StringUtil.getFromCompressedUnicode(data, 3 + o
ffset, // [Shawn] Changed 1 to 3 for offset of string
                                           ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\StringRecord.java:137: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_3_string = StringUtil.getFromCompressedUnicode(data, 3 + offse
t, field_1_string_length);
                                       ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\BoundSheetRecord.java:145: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_5_sheetname = StringUtil.getFromCompressedUnicode( data, 8 + o
ffset, nameLength );
                                          ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\FormatRecord.java:135: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,short)
location: class org.apache.poi.util.StringUtil
          field_4_formatstring = StringUtil.getFromCompressedUnicode(data, 5 + o
ffset, field_3_unicode_len );
                                           ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\LabelRecord.java:160: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,short)
location: class org.apache.poi.util.StringUtil
            field_6_value = StringUtil.getFromCompressedUnicode(data, 9 + offset
, getStringLength());
                                      ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\StyleRecord.java:137: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
            field_3_name        = StringUtil.getFromCompressedUnicode(data, 3 +
offset,
                                            ^
D:\cygwin\home\androliv\deleteme\jakarta-poi\build\java\org\apache\poi\hssf\reco
rd\WriteAccessRecord.java:118: cannot resolve symbol
symbol  : method getFromCompressedUnicode (byte[],int,int)
location: class org.apache.poi.util.StringUtil
        field_1_username = StringUtil.getFromCompressedUnicode(data, 3 + offset,
 data.length - 4);
                                     ^
14 errors

BUILD FAILED
D:\cygwin\home\androliv\deleteme\jakarta-poi\tools\cents\centipede.cent\xbuild.x
ml:28: Compile failed; see the compiler error output for details.

Total time: 1 minute 2 seconds

androliv@ANDROLIV-W2K01 ~/deleteme/jakarta-poi
$

Please fix and reopen.
Comment 9 Toshiaki Kamoshida 2003-04-22 03:50:07 UTC
Created attachment 5939 [details]
Sorry,my private text editor seems to be dealing about the RETURN CODE at StringUtil incorrectly.If this patch is incorrect,please you insert StringUtil#putUncompressedUnicode() at the right position anywhere...
Comment 10 Toshiaki Kamoshida 2003-04-22 03:54:43 UTC
I resubmitted a PATCH.
I changed the position of StringUtil#putUncomporessedUnicode().
If this patch is incorrect,please you insert StringUtil#putUncompressedUnicode
() at the right position anywhere...
Comment 11 Andy Oliver 2003-04-22 16:41:25 UTC
applied.  Thanks!
Comment 12 Andy Oliver 2003-07-24 15:55:17 UTC
*** Bug 17791 has been marked as a duplicate of this bug. ***