Bug 46918

Summary: RecordFormatException with Pivot table
Product: POI Reporter: Axel Rose <axel.roeslein>
Component: HSSFAssignee: POI Developers List <dev>
Severity: major    
Priority: P2    
Version: 3.5-dev   
Target Milestone: ---   
Hardware: All   
OS: All   

Description Axel Rose 2009-03-26 04:07:29 UTC
With a certain type of XLS data I get this exception:

Exception in thread "main" org.apache.poi.hssf.record.RecordFormatException: Unable to construct record instance
	at org.apache.poi.hssf.record.RecordFactory$ReflectionRecordCreator.create(RecordFactory.java:71)
	at org.apache.poi.hssf.record.RecordFactory.createSingleRecord(RecordFactory.java:254)
	at org.apache.poi.hssf.record.RecordFactory.createRecords(RecordFactory.java:373)
	at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:277)
	at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:202)
	at org.apache.poi.hssf.usermodel.HSSFWorkbook.<init>(HSSFWorkbook.java:184)
	at ExcelExtractor.openWorkbook(ExcelExtractor.java:668)
	at ExtractorTest.main(ExtractorTest.java:70)
Caused by: org.apache.poi.hssf.record.RecordFormatException: Not enough data (0) to read requested (2) bytes
	at org.apache.poi.hssf.record.RecordInputStream.checkRecordPosition(RecordInputStream.java:185)
	at org.apache.poi.hssf.record.RecordInputStream.readUShort(RecordInputStream.java:230)
	at org.apache.poi.hssf.record.pivottable.ExtendedPivotTableViewFieldsRecord.<init>(ExtendedPivotTableViewFieldsRecord.java:53)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
	at org.apache.poi.hssf.record.RecordFactory$ReflectionRecordCreator.create(RecordFactory.java:63)
	... 7 more

Excel has no problem opening this file, POI 3.1 works fine on the input.
I checked with POI 3.5 latest svn March 26.
Comment 1 Nick Burch 2009-03-26 05:05:31 UTC
Please upload a sample problem file, so we can take a look at why some records are the wrong length
Comment 2 Axel Rose 2009-03-26 05:50:01 UTC
When checking the file in question BiffViewer throws an Exception

java -cp build/classes org.apache.poi.hssf.dev.BiffViewer /tmp/t1.xls > /tmp/t1.txt
        at java.lang.System.arraycopy(Native Method)
        at org.apache.poi.ddf.UnknownEscherRecord.fillFields(UnknownEscherRecord.java:77)
        at org.apache.poi.ddf.EscherContainerRecord.fillFields(EscherContainerRecord.java:53)
        at org.apache.poi.ddf.EscherContainerRecord.fillFields(EscherContainerRecord.java:53)
        at org.apache.poi.hssf.record.AbstractEscherHolderRecord.convertToEscherRecords(AbstractEscherHolderRecord.java:82)
        at org.apache.poi.hssf.record.AbstractEscherHolderRecord.<init>(AbstractEscherHolderRecord.java:68)
        at org.apache.poi.hssf.record.DrawingGroupRecord.<init>(DrawingGroupRecord.java:41)
        at org.apache.poi.hssf.dev.BiffViewer.createRecord(BiffViewer.java:149)
        at org.apache.poi.hssf.dev.BiffViewer.createRecords(BiffViewer.java:84)
        at org.apache.poi.hssf.dev.BiffViewer.main(BiffViewer.java:398)

The output looks ok to me.
What should I check within the BiffViewer output?
(content cannot be made public)
Comment 3 Nick Burch 2009-03-26 08:29:11 UTC
Looking at your stacktrace, your exception is apparently being triggered by some invalid escher record headers. However, without the file we can't really do much, so alas you'll need to dive into the poi sourcecode in your debugger and investigate it yourself (well, or find a file you can share that also triggers the issue!)
Comment 4 Axel Rose 2009-03-26 09:03:43 UTC
I wish I could debug.

Using Eclipse 3.5, creating a new project, importing from existing source, using defaults

106 errors, e.g.
NumericRanges cannot be resolved	ColumnHelper.java	POI35/ooxml/java/org/apache/poi/xssf/usermodel/helpers	line 176	Java Problem
NumericRanges cannot be resolved	TestNumericRanges.java	POI35/ooxml/testcases/org/apache/poi/xssf/util	line 55	Java Problem

It might of course be caused by my Eclipse settings.
Is there any adaptable .classpath available for a quick start with Eclipse?

Comment 5 Josh Micich 2009-03-31 14:18:42 UTC
Axel supplied me with the hex of the offending ExtendedPivotTableViewFieldsRecord:
00 01 0A 00 1E 14 00 0A FF FF FF FF 00 00

This is 10 bytes shorter than POI was expecting (existing test cases have 20 bytes of data in this record).  The 10 byte format is probably from earlier Excel versions (perhaps 97) POI now reads both formats of this record.

Fixed in svn r760464

junit added