Bug 45676 - poi-3.5-beta1 XSSF event usermodel occasionally truncates cell values
Summary: poi-3.5-beta1 XSSF event usermodel occasionally truncates cell values
Status: RESOLVED FIXED
Alias: None
Product: POI
Classification: Unclassified
Component: POI Overall (show other bugs)
Version: unspecified
Hardware: PC Windows XP
: P1 blocker (vote)
Target Milestone: ---
Assignee: POI Developers List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-08-22 09:34 UTC by Paul Dobson
Modified: 2008-09-13 07:01 UTC (History)
1 user (show)



Attachments
Large excel file for testing. (555.08 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2008-09-10 11:26 UTC, Paul Dobson
Details
Java program to test using SampleBigWorkbook.xlsx (3.77 KB, text/plain)
2008-09-10 11:36 UTC, Paul Dobson
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Dobson 2008-08-22 09:34:29 UTC
When parsing large excel files using XSSF and the event usermodel, when the needed data referred to by start and length variables refer to the end of ch[], only part of the cell value is returned.  This will either give an incorrect numeric value of a cell, or refer an incorrect position in the sharedStrings file.
This can be verified by using the poi.xssf.eventusermodel.examples to parse a very large sheet (several thousand rows with a couple dozen columns).  Put a conditional breakpoint on the value of the variable start when start + length is 2048 in the method "public void characters(char[] ch, int start, int length) throws SAXException".  This will eventually give you a value that is truncated.
The value of ch needs to go to 2048 characters, then back up to the last complete set of cell data rather than always cutting at exactly 2048 characters and possibly truncating cell data being referred to.
This is a showstopper because the results of using the XSSF eventusermodel can not be trusted when parsing large excel files.

Thanks.

Paul Dobson
Comment 1 Nick Burch 2008-09-07 13:04:59 UTC
Any chance you could upload a suitably large file that triggers this bug?

I have a feeling it's actually a bug in the example, rather than in xssf itself, but I'll need a sample file to test this hunch
Comment 2 Paul Dobson 2008-09-10 11:26:24 UTC
Created attachment 22550 [details]
Large excel file for testing.

Use this file with the attached sample code to duplicate the problem. The file has values in each cell corresponding the location of the cell.  The sample code will only work with this file or a similarly formatted excel file.
Comment 3 Paul Dobson 2008-09-10 11:36:48 UTC
Created attachment 22551 [details]
Java program to test using SampleBigWorkbook.xlsx

This program is based on the sample code.  It outputs lines when it finds that the cell value from the shared strings table does not match what it should be.  It also outputs the value of the cell that was clipped to return the incorrect string from the shared strings table.  

Let me know if you need anything else.

Thanks,

Paul
Comment 4 Paul Dobson 2008-09-10 11:40:55 UTC
I have attached a sample .xlsx file as well as a java program to test the sample file.  The program will only work on the attached .xlsx file or one with cell content laid out in a similar way.

Let me know if this is what you needed.

Paul
Comment 5 Paul Dobson 2008-09-11 15:01:27 UTC
I did some more testing.  You are right.  It seems to be a problem in the example code.  Consider the following change to endElement, and characters methods.  I don't know if this is best way to fix the problem but it seems to work in my case.

public void endElement(String uri, String localName, String name)
                        throws SAXException {
    // v => contents of a cell
    // Output after we've seen the string contents
                    
    if(nextIsString) {
        idx = Integer.parseInt(lastContents);
        lastContents = sst.getSharedStringAt(idx);
    }
    if(name.equals("v")) {
    System.out.println(lastContents);
    }
     lastContents = "";
}

public void characters(char[] ch, int start, int length)
				throws SAXException {
    lastContents += new String(ch, start, length);
			
}
Comment 6 Nick Burch 2008-09-13 07:01:26 UTC
Yup, your fix looks like it's the right one, thanks for figuring that out.

I've fixed the documentation and example in svn, so they'll be included in the next beta