Bug 46280

Summary: Unexpected ContinueRecord in RowRecordsAggregate when PivotTables are present
Product: POI Reporter: Josh Micich <josh>
Component: HSSFAssignee: POI Developers List <dev>
Status: RESOLVED FIXED    
Severity: normal    
Priority: P2    
Version: 3.5-dev   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Attachments: BiffViewer analysis
summary of previous attachment

Description Josh Micich 2008-11-24 14:12:21 UTC
This bugzilla was opened in response to the mailing list posting made by Axel Roeslein:

http://mail-archives.apache.org/mod_mbox/poi-user/200811.mbox/%3C29bf25ca0811200724y620db645p1783ac1c0bd5085d@mail.gmail.com%3E

The following stack trace occurred:

Exception in thread "main" java.lang.RuntimeException: Unexpected record type (org.apache.poi.hssf.record.ContinueRecord)
       at org.apache.poi.hssf.record.aggregates.RowRecordsAggregate.<init>(RowRecordsAggregate.java:87)
       at org.apache.poi.hssf.model.Sheet.<init>(Sheet.java:197)
       at org.apache.poi.hssf.model.Sheet.createSheet(Sheet.java:159)
       at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:286)
       at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:200)
       at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:182)

Unfortunately the workbook had sensitive data in it, so it couldn't be uploaded.  Re-saving in Excel makes the problem go away, and the problem has not been reproduced with another workbook yet.

Further investigation revealed that the offending ContinueRecord was an extension to a "SXLI - Line Item Array" (0x00B5) record, which is a PivotTable record.  Indeed, the workbook does contain a large pivot table.
Comment 1 Josh Micich 2008-11-24 15:05:42 UTC
Fix applied in svn r720318

The logic inside RowRecordsAggregate was changed to allow for ContinueRecords after any UnknownRecord.

The arrangement of PivotTable records in the worksheet stream is not discussed in the OOO documentation.  Brief investigation suggests that they come after all of the row-block records of the current sheet (regardless of the actually positioning of the pivot table(s) on the sheet).

However, Excel 2007 seems to put MSODRAWING(0x00EC) before the PivotTable records, and this should have prevented this bug, since RowBlocksReader stops at DrawingRecord.sid(0x00EC).  I am assuming that the offending file does not have an MSODRAWING record before the PivotTable records, so I added a fix in RecordOrderer.isEndOfRowBlock(int) to make sure PivotTable records are not included in the 'row blocks'.

My original assumption was that re-saving with Excel removed the offending ContinueRecord.  Now I am guessing that re-saving with Excel put the MSODRAWING record in the right place.

Comment 2 Axel Rose 2008-11-25 03:17:37 UTC
Created attachment 22935 [details]
BiffViewer analysis

I updated my sources from svn and rechecked. Reading works as expcected.

    I would like to know if MSODRAWING (0x00EC) is
    missing before the PivotTable records (0x00B* etc).

No, as far as I can see.

    Is the first PivotTable record 0x00B0?

No.

    Does re-saving with Excel insert MSODRAWING
    in the expected place?

What is the expected place?
 

Please see the attached file. It was produces outputting the BiffViewer result
to "note23.1" and then doing "grep sid=0x note23.1 > note23.sid.txt"
Comment 3 Josh Micich 2008-11-25 11:00:45 UTC
Created attachment 22947 [details]
summary of previous attachment

I have summarized the attached 'note23.sid.txt' to just the following records:
 - All BOFs and EOFs to see where the sheets start and stop
 - For each sheet containing SXVIEW, all Records from DBCELL to SXVIEW
 - All continued records (only 2 in the whole workbook)

In the summary, it can be seen that only in the fifth sheet is SXVIEW not preceeded with MSODRAWING. This also happens to be the only sheet with a continued PivotTable record (recno=7750). Both conditions had to occur together to make this bug occur.