Bug 31749 - {Need help urgently}[This is critical] workbook.write() corrupts the file......?
Summary: {Need help urgently}[This is critical] workbook.write() corrupts the file......?
Status: RESOLVED FIXED
Alias: None
Product: POI
Classification: Unclassified
Component: HSSF (show other bugs)
Version: 2.5-FINAL
Hardware: Other Windows XP
: P3 critical (vote)
Target Milestone: ---
Assignee: POI Developers List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-10-16 22:24 UTC by as
Modified: 2007-01-16 01:40 UTC (History)
0 users



Attachments
this is one sheet from the orig file.The complete file is very big but this sheet has also the same problem. (403.00 KB, application/octet-stream)
2004-10-19 02:39 UTC, as
Details
test (13.50 KB, application/octet-stream)
2004-10-19 02:53 UTC, as
Details

Note You need to log in before you can comment on or make changes to this bug.
Description as 2004-10-16 22:24:18 UTC
I am running into series of problems.
Here is the sample code which i m executing.

********
import java.io.*;

import org.apache.poi.hssf.usermodel.HSSFWorkbook;

public class TestPoi {
	
	public static void main(String[] args) throws Exception{
		
		HSSFWorkbook workbook = new HSSFWorkbook(new FileInputStream
(new File("c://test//uw.xls")),true);
		FileOutputStream fos = new FileOutputStream(new File
("c://test//uw_output.xls"));
		workbook.write(fos);
		fos.close();
	}

}
******************

The output file uw_output.xls is totally unusable as i am not able to open 
this file.
Here are series of steps which i performed.

1. When i executed this programe for the first time i got  warnings  like 
below one.

[WARNING] Unknown Ptg 3c (60) at cell (36,2)

Then i deleted all the rows having the junk characters and  i executed the 
programe again.This time it did not generate any warnings but the excel file 
which it created was unreadable.
Comment 1 as 2004-10-17 01:42:32 UTC
********uw_output.xls file did not get corrputed when i used 
POIFSFileSystem.write() method and i could open this(uw_output.xls) file 
successfully.

		FileOutputStream fos = new FileOutputStream(new File
("c://test//uw_output.xls"));		
		POIFSFileSystem pfs =  new POIFSFileSystem(new FileInputStream
(new File("c://test//uw.xls")));
		pfs.writeFilesystem(fos);

I am not sure why workbook.write() corrupts the file.

But i am running into some other issue now.
When i make the change in this xls file andclick save in the file menu , it 
give me excel error saying 'document not saved'??
any clues..

Comment 2 Jason Height 2004-10-17 22:24:13 UTC
Please attach your template file uw.xls to this bug using the "Create a new
attachment" hyperlink.

Without this it makes it impossible to debug.
Comment 3 as 2004-10-19 02:39:47 UTC
Created attachment 13135 [details]
this is one sheet from the orig file.The complete file is very big but this sheet has also the same problem.
Comment 4 as 2004-10-19 02:53:24 UTC
Created attachment 13136 [details]
test
Comment 5 as 2004-10-20 19:08:41 UTC
Hi there, If you are too busy to look into the defect right now ,Please give me 
some pointers where you think poi library can fail and fix so that i can debug 
it myself ..i will really be thankful if someone can look into it..but one 
thing is sure that this my excel sheet contains VB code it and i m not sure how 
much u support that.
Comment 6 Jason Height 2004-10-20 23:07:15 UTC
Easiest way to debug this stuff is to use the biff viewer to get a text dump of
the original file, and then do the same with the one re-written by poi. 

Then do a diff on the files to check for differences. Aside for some records
that are marked as unknown, and not written by poi (they are not required) there
should be very few differences.

I could do this but not for another week.

Jason
Comment 7 as 2004-10-21 02:54:23 UTC
What tool i should use as i downloaded the ExcelFile viewer v1.2 it does show 
excel records but i am not able to take dump to text file so that i can  
compare the files.
Comment 8 Jason Height 2004-10-21 03:13:31 UTC
http://jakarta.apache.org/poi/hssf/how-to.html

search for biffview

here is the .bat file that i use in windows to run it:

java -classpath f:/home/jmheight/poi/clean/jakarta-poi/build/dist/poi-2.1-
20041013.jar org.apache.poi.hssf.dev.BiffViewer %1
Comment 9 as 2004-10-21 03:21:09 UTC
thanks a lot Jason! really appreciate this!! :)
Comment 10 as 2004-10-21 04:47:47 UTC
hi there, i was trying to biffview the input and output files but i could get 
partial dump of the file as i got the exception

Here is the exception while i was trying to biffview input file

*********
java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at org.apache.poi.ddf.EscherArrayProperty.getElement
(EscherArrayProperty
.java:110)
        at org.apache.poi.ddf.EscherArrayProperty.toString
(EscherArrayProperty.j
ava:131)
        at org.apache.poi.ddf.EscherOptRecord.toString
(EscherOptRecord.java:138)

        at org.apache.poi.ddf.EscherContainerRecord.toString
(EscherContainerReco
rd.java:158)
        at org.apache.poi.hssf.record.AbstractEscherHolderRecord.toString
(Abstra
ctEscherHolderRecord.java:126)
        at org.apache.poi.hssf.dev.BiffViewer.dumpNormal(BiffViewer.java:148)
        at org.apache.poi.hssf.dev.BiffViewer.access$1(BiffViewer.java:144)
        at org.apache.poi.hssf.dev.BiffViewer$RecordDetails.dump
(BiffViewer.java
:700)
        at org.apache.poi.hssf.dev.BiffViewer.createRecords
(BiffViewer.java:120)

        at org.apache.poi.hssf.dev.BiffViewer.run(BiffViewer.java:73)
        at org.apache.poi.hssf.dev.BiffViewer.main(BiffViewer.java:652)
***************

And here is the exception while i was trying to biff view output file 
(generated by poi library)

**********
java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at org.apache.poi.ddf.EscherArrayProperty.getElement
(EscherArrayProperty
.java:110)
        at org.apache.poi.ddf.EscherArrayProperty.toString
(EscherArrayProperty.j
ava:131)
        at org.apache.poi.ddf.EscherOptRecord.toString
(EscherOptRecord.java:138)

        at org.apache.poi.ddf.EscherContainerRecord.toString
(EscherContainerReco
rd.java:158)
        at org.apache.poi.hssf.record.AbstractEscherHolderRecord.toString
(Abstra
ctEscherHolderRecord.java:126)
        at org.apache.poi.hssf.dev.BiffViewer.dumpNormal(BiffViewer.java:148)
        at org.apache.poi.hssf.dev.BiffViewer.access$1(BiffViewer.java:144)
        at org.apache.poi.hssf.dev.BiffViewer$RecordDetails.dump
(BiffViewer.java
:700)
        at org.apache.poi.hssf.dev.BiffViewer.createRecords
(BiffViewer.java:120)

        at org.apache.poi.hssf.dev.BiffViewer.run(BiffViewer.java:73)
        at org.apache.poi.hssf.dev.BiffViewer.main(BiffViewer.java:652)
**************

Can this be the reason , why the output file is getting corrupt while writing?

Comment 11 as 2004-10-21 20:21:34 UTC
i think i have reached the dead end in this debugging.
Need urgent help in this....
Comment 12 Glen Stampoultzis 2004-10-25 11:35:08 UTC
Done some tracing through.  The problem seems to be related to the little up and
down arrow's you have in the sheet.  I removed them and the sheet started
rewriting without corruption.
Comment 13 Glen Stampoultzis 2004-10-25 12:09:43 UTC
Okay... I think I've narrowed down the issue.  It's actually an instance of the
really wierd problem I encountered in the way Excel sometimes writes it's
continue records.  See
http://www.jroller.com/page/gstamp/20040817#excel_and_the_escher_format for more
information.

Basically what's happening at the moment is the system is (incorrectly) trying
to call processContinueRecord() for the preceeding OBJ record.  Probably what
should happen in this case is that the continue record is applied last record
capable of handling a continue (or left along otherwise).

You can hack around this problem by altering the line:

179 in RecordFactory.java (CVS HEAD) to:

                            if (rectype == ContinueRecord.sid &&
                                ! (last_record instanceof ContinueRecord) && //
include continuation records after
                                ! (last_record instanceof UnknownRecord) && //
unknown records or previous continuation records
                                ! (last_record instanceof ObjRecord) &&
                                ! (last_record instanceof TextObjectBaseRecord) 
                            )


Unfortunately this doesn't completely cure the problem.  There still seems to be
messages that come up when you open the sheet.
Comment 14 as 2004-10-25 13:54:26 UTC
Yes, the situation is little better when i made the code change suggested by 
you.Now I could open the output file and it seems the structure of excel is 
intact..but now the other problem is that excel crashes when i try to save the 
file and i am not able to open the output file after i set the cell values 
progammatically, excel gives me nasty error "Excel.exe has generated error".
Comment 15 as 2004-11-01 01:01:49 UTC
Glen, thanks for the fix..that has helped me as i can open the output  excel 
sheet  but i am not able to save the excel document.It gives  excel 
message 'document not saved' and if i try to close the document excel 
application crashes.

Comment 16 Glen Stampoultzis 2004-11-01 10:17:58 UTC
When I get some more free time I'll look into it some more.  Can't make any
promises though.  
Comment 17 as 2004-11-08 05:56:04 UTC
Glen.. the solution u suggested works for me..and the only small thing left is 
the problem which i mentioned in bug#31979 and i have been trying to debug 
this for long time and i m stumped on this issue..need some help or pointers 
in this issue.
Comment 18 Avik Sengupta 2007-01-16 01:40:54 UTC
Works as of 12Jan2007, Testcase added. 
http://issues.apache.org/bugzilla/show_bug.cgi?id=41366