Bug 57699

Summary: Support Strict OOXML files
Product: POI Reporter: Nick Burch <apache>
Component: XSSFAssignee: POI Developers List <dev>
Status: NEW ---    
Severity: enhancement CC: matafagafo, miguel.penetra, sergeymalafeev, sm462x, thtjf2424, tmasocha06, twilcock
Priority: P2    
Version: 3.12-dev   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Attachments: spam

Description Nick Burch 2015-03-13 11:43:49 UTC
Office 2013 has added the option to save as "strict" ooxml files, which as reported in http://stackoverflow.com/questions/29023542/how-to-parse-strict-xlsx-file-in-java have a different core type

In r1666410 some sample strict xlsx files have been added, support is needed to support them (for reading at least)
Comment 1 Nick Burch 2015-03-13 18:01:59 UTC
It looks like some namespace munging is going to be required to properly support this. After making changes to ExtractorFactory and POIXMLDocumentPart to handle the differing core relationship type, it now fails at the xmlbeans level:

org.apache.xmlbeans.XmlException: error: The document is not a workbook@http://schemas.openxmlformats.org/spreadsheetml/2006/main: document element namespace mismatch expected "http://schemas.openxmlformats.org/spreadsheetml/2006/main" got "http://purl.oclc.org/ooxml/spreadsheetml/main"
	at org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:399)

Caused by: org.apache.xmlbeans.XmlException: error: The document is not a workbook@http://schemas.openxmlformats.org/spreadsheetml/2006/main: document element namespace mismatch expected "http://schemas.openxmlformats.org/spreadsheetml/2006/main" got "http://purl.oclc.org/ooxml/spreadsheetml/main"
	at org.apache.xmlbeans.impl.store.Locale.verifyDocumentType(Locale.java:459)
	at org.apache.xmlbeans.impl.store.Locale.autoTypeDocument(Locale.java:364)
	at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1280)
	at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:1264)
	at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:345)
	at org.openxmlformats.schemas.spreadsheetml.x2006.main.WorkbookDocument$Factory.parse(Unknown Source)

The purl namespace crops up in most of the xml files at least somewhere, so a general mapping solution is probably required if we want to take this further
Comment 2 Dominik Stadler 2016-02-27 10:24:56 UTC
*** Bug 57914 has been marked as a duplicate of this bug. ***
Comment 3 PJ Fanning 2016-07-17 12:58:18 UTC
http://pyxb.sourceforge.net/PyXB-1.2.2/bundles.html has a list of namespace URLs that could be used in a mapping class.
Comment 4 PJ Fanning 2016-07-17 13:28:16 UTC
Without spending much time on this, I have been unable to track down the XSDs with the purl namespaces (OOXML Strict). From accounts, they should be very similar to the OOXML Transitional schemas other than the namespaces.
2 approaches pop to mind.
1. In poi-ooxml-schemas, we could create XmlBeans for the OOXML Strict namespaces by using modified versions of the OOXML Transitional schemas.
2. support a transformation of the XML in input docs so that the OOXML Strict namespaces are replaced by OOXML transitional equivalents.
Comment 5 PJ Fanning 2016-07-17 15:51:17 UTC
I have added some basic prototype code to convert Strict OOXML files to https://github.com/pjfanning/ooxml-strict-converter - there is still a lot of work to do but I'm just posting it here if anyone wants to review what I'm doing.
Comment 6 Javen O'Neal 2016-07-18 00:34:47 UTC
Looks good so far.

In the interest of wanting to start committing this early so that we can update our unit tests to handle XSSF Strict:
* Are we planning on having XSSFWorkbook transparently handle strict workbooks or will be have a different class for that?
Will this be in the o.a.p.xssf.usermodel package or are we going to package it in o.a.p.xssf.extractor or create o.a.p.xssf.strict?

In the long term, I would like for POI to be able to read and write strict files without having to downconvert to non-strict. This probably affects how we go about packaging this--making it more than a distant examples or static utility converter class.
Comment 7 Dominik Stadler 2016-07-18 08:19:07 UTC
FYI, there is also a converter provided by Microsoft: https://www.microsoft.com/en-us/download/details.aspx?id=38828, could come in handy when doing development work on this topic.
Comment 8 PJ Fanning 2016-07-18 22:46:44 UTC
Hi Javen,
I can understand that we will want to be able to save POI documents using Strict OOXML but my focus for now is just on the down-porting to Transitional OOXML to allow parsing.
For now, I'm looking at a standalone utility to down-port but this could be plugged into XSSFWorkbook and XSSF extractor under the hood. They could either do some pre-processing of the input doc to determine if it is Strict OOXML and the down-port to a temp file and then read from the temp file.
My prototype code is working now for the SimpleStrict.xlsx in the POI test data folder.
I'll see about testing with more input files.
Comment 9 tmasocha06 2018-06-26 06:48:32 UTC
Created attachment 35988 [details]
Comment 10 Piotr Wilkin 2019-05-20 14:26:49 UTC
Over two years have passed - has there been any work done on this / any milestone?
Comment 11 Dominik Stadler 2019-05-20 14:56:34 UTC
No, it seems none of the contributors needs it urgently enough to warrant spending time on it.

As this is a purely community supported project without commercial backing, your best bet to get progress on this will be to provide patches/time yourself if you can contribute in any way.
Comment 12 Piotr Wilkin 2019-05-20 15:25:04 UTC
Yeah, which is why I was asking :> there were some partial results done by some people, I'll see if something can be done.
Comment 13 sm462x 2019-06-30 18:44:56 UTC
I am interested in working on this issue. Will be willing to work with some if somebody is already is working on it otherwise I can take it independently.
Comment 14 Andreas Beeker 2019-10-15 06:05:32 UTC
*** Bug 63847 has been marked as a duplicate of this bug. ***
Comment 15 PJ Fanning 2021-02-13 21:48:39 UTC
There is some basic support in excel-streaming-reader, documented at https://github.com/pjfanning/excel-streaming-reader#ooxml-strict-format
Comment 16 Sebastian Böhm 2021-03-20 06:11:24 UTC
Good Morning, I like to contribute to the support of strict OOXML files (as I want to get rid of my workaround scripts). What is the current state? Are there already some changes/progress in any branch? If not, I would try to use the information from the comments here and start working on that.
Comment 17 PJ Fanning 2021-03-20 15:30:07 UTC
Hi Sebastian - supporting Strict OOXML format is a big undertaking and my impression is that this format has not really taken off (there aren't too many people asking about it). If you read my comment from 2021-02-13, you'll see that there is some support already but it is limited.

I would encourage you to spell out your plan because we may not necessarily want to support the strict format in poi-ooxml lib and might want to add a new jar dedicated to strict format - because we could end up bloating poi-ooxml by trying to support both formats in one jar.