Bug 44921

Summary: Coding Error: This method should never be called. This ptg should be converted
Product: POI Reporter: Gary Golub <ggolub>
Component: HSSFAssignee: POI Developers List <dev>
Severity: normal    
Priority: P2    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Attachments: Spreadsheet generated with poi-3.0-rc4-20070503.jar
Original spreadsheet
Regenerated spreadsheet
Utility functions used by the main script
Beanshell script that causes the error.

Description Gary Golub 2008-05-02 06:12:05 UTC
Created attachment 21901 [details]
Spreadsheet generated with poi-3.0-rc4-20070503.jar

I used POI to generate a spreadsheet with data from Oracle.  The end users add data, including formulas, to the initial version which is intended for use as part of a data migration.  Prior to the migration I regenerate the spreadsheet and use another script to parse out differences between the original/modified one and the newly generated one.  The script worked reasonably well with 3.0rc4 but fails with 3.0.2 and 3.1beta.  The errors are as follows:

Script threw exception: Sourced file: compareDocReports.bsh : Method Invocation oldBook.write : at Line: 241 : in file: compareDocReports.bsh : oldBook .write ( fos )
Called from method: modSpreadsheet : at Line: 250 : in file: compareDocReports.bsh : modSpreadsheet ( oldBook , newBook )
Called from method: run : at Line: 253 : in file: compareDocReports.bsh : run ( bsh .args [ 0 ] , bsh .args [ 1 ] )
Target exception: java.lang.RuntimeException: Coding Error: This method should never be called. This ptg should be converted

My guess is that the spreadsheet contains an invalid formula (not evident when I open it in Excel) but I don't know anything about ptg.  Furthermore, the unchanged script worked fine with the older POI release which indicates that this problem must be with the newer software.

Can you provide any guidance?
Comment 1 Gary Golub 2008-05-02 06:13:16 UTC
Created attachment 21902 [details]
Original spreadsheet

This is the original spreadsheet, stripped down to a few lines to demonstrate the issue.
Comment 2 Gary Golub 2008-05-02 06:13:54 UTC
Created attachment 21903 [details]
Regenerated spreadsheet

This is the second generation of the spreadsheet, stripped down to a few lines to demonstrate the issue.
Comment 3 Gary Golub 2008-05-02 06:14:59 UTC
Created attachment 21904 [details]
Utility functions used by the main script
Comment 4 Gary Golub 2008-05-02 06:15:38 UTC
Created attachment 21905 [details]
Beanshell script that causes the error.

Script runs fine with 3.0rc4 but not with 3.0.2 or 3.1beta
Comment 5 Josh Micich 2008-05-02 15:35:54 UTC
Fix applied in svn r652934

The fix involves allowing all relative ref Ptgs (RefN*.java and AreaN*.java) to be written out (/serialized).

This bug was probably not visible before, because conditional formats and data validation are recent additions to POI.  These use relative-reference Ptgs (similar to those of shared formulas).  In earlier versions of POI a strategy was adopted to convert these Ptgs to absolute references (I assume) to make the formula evaluation stuff easier.  Hence relative reference Ptgs were never expected to be serialized.  For the new conditional format and data validation stuff, the conversion does not take place (and probably shouldn't) so that's why the error occurred.

Seems like Andy hit a very similar problem before (see svn r513604).  I can't find a bugzilla record though. 

There are significant problems with how relative references are handled by POI (see bug 44636).  This bug was also fixed with a small change, but a more correct solution will probably involve skipping the shared formula conversion stuff.

For interest, here are the two stack trace fragments showing how the conditional format and data validation stuff is relevant to this bug:

RefNVPtg.writeBytes(byte[], int) line: 51	
Ptg.serializePtgStack(Stack, byte[], int) line: 394	
CFRuleRecord.serialize(int, byte[]) line: 567	
CFRecordsAggregate.serialize(int, byte[]) line: 168	
Sheet.serialize(int, byte[]) line: 830	

RefNVPtg.writeBytes(byte[], int) line: 51	
DVRecord.serialize(int, byte[]) line: 462	
Sheet.serialize(int, byte[]) line: 830	

BTW - when supplying sample code, please try to use plain java (1.4 or 1.5).  Also try to cut out as much unneeded stuff as possible.  This bug can actually be re-produced with the following code:

HSSFWorkbook wb = new HSSFWorkbook(new FileInputStream("ex44921-21902.xls"));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
Comment 6 Josh Micich 2008-05-02 15:51:51 UTC
(In reply to comment #5)
> Fix applied in svn r652934

One more file (RefNVPtg.java). The most important file too. Having problems with my TortoiseSVN.

Applied in svn r652936