Bug 62151

Summary: Illegal reflective access by org.apache.poi.util.DocumentHelper and org.apache.poi.util is accessible from more than one module
Product: POI Reporter: Bart E <bvdhel>
Component: POI OverallAssignee: POI Developers List <dev>
Status: RESOLVED FIXED    
Severity: normal CC: kalakrishnan
Priority: P2    
Version: 3.17-FINAL   
Target Milestone: ---   
Hardware: All   
OS: All   
Bug Depends on: 62355    
Bug Blocks:    

Description Bart E 2018-03-02 05:55:09 UTC
Dear, 

Got two problems working with Apache Poi 3.17 first is in the jar-files

The package org.apache.poi.util is accessible from more than one module: poi, poi.ooxml

And this next one, during running the application making the reports

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.poi.util.DocumentHelper (file:/C:/Java/poi/poi-bin-3.17-20170915/poi-3.17/poi-ooxml-3.17.jar) to method com.sun.org.apache.xerces.internal.util.SecurityManager.setEntityExpansionLimit(int)
WARNING: Please consider reporting this to the maintainers of org.apache.poi.util.DocumentHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Regards,

Bart E.
Comment 1 Nick Burch 2018-03-02 11:18:08 UTC
The former we're unlikely to fix - Apache POI has multiple jars which make up the project and allow you to exclude the parts of the project you don't want. Because of design decisions taken by the Java 9 jigsaw team, our only fix is to provide a single uber-jar with everything in. We're discussing that now, but you could easily build your own if you wanted in the mean time

The latter I don't know how we can fix. For security reasons, we need to tell the built-in Java XML parser to turn on non-default settings to increase the parsing security. I don't believe that there's a public API for doing that, only the private ones.

Are you able to find any Java 9 advice on how to turn on the XML security features with the built-in xml parser using only public calls in Java 9?
Comment 2 Dominik Stadler 2018-04-02 21:01:14 UTC
I think a first step will be Automatic-Module-Name in the MANIFEST.MF of the jar-files, see http://branchandbound.net/blog/java/2017/12/automatic-module-name/, need to read up some more on this before starting on adding this, though.
Comment 3 James Pether Sörling 2018-04-06 08:11:38 UTC
(In reply to Nick Burch from comment #1)
> The former we're unlikely to fix - Apache POI has multiple jars which make
> up the project and allow you to exclude the parts of the project you don't
> want. Because of design decisions taken by the Java 9 jigsaw team, our only
> fix is to provide a single uber-jar with everything in. We're discussing
> that now, but you could easily build your own if you wanted in the mean time

Also have the same issue "The package org.apache.poi.X is accessible from more than one module: poi, poi.ooxml"

Maybe it is time with the 4.0.0 release to make sure each jar do not contain any overlapping packages. Understand it is bad to break compatibility for existing 3.x releases.
Comment 4 Axel Howind 2018-04-06 11:47:03 UTC
@Dominic: It would be nice if 4.0 would not have automatic module names but instead "real" modules (aka with a real module-info.java). Is there any chance this could be achieved? I think I could easily do it in Gradle, but I am no expert in Maven or Ant. If there's a chance to get this into 4.0, I'd try to find some time to look into it anyway.

I am aware of Stephen Colebourne's blog post concerning JPMS (http://blog.joda.org/2018/03/jpms-negative-benefits.html), but I think it could be done in a way that solves most if not all of the issues he mentions (i.e. compatibility with Java 8 and Android).
Comment 5 Dominik Stadler 2018-04-07 04:36:31 UTC
It will require large code-refactorings to move everything into a separate package and none of the third-party libs that Apache POI uses is currently available as module, so as far as I see (don't know too much about Java 9 to be honest) we cannot release a full Java 9 module until these are available as modules, or?
Comment 6 Alan Bateman 2018-05-01 13:03:33 UTC
The JAXP tutorial has a page on the processing limits that might be useful:
https://docs.oracle.com/javase/tutorial/jaxp/limits/limits.html
Comment 7 Dominik Stadler 2018-05-24 08:04:20 UTC
*** Bug 62406 has been marked as a duplicate of this bug. ***
Comment 8 Dominik Stadler 2018-05-24 08:05:21 UTC
From duplicated bug:


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.poi.poifs.nio.FileBackedDataSource$1 (file:/name_of_Jar_file) to method java.nio.DirectByteBuffer.cleaner()
WARNING: Please consider reporting this to the maintainers of org.apache.poi.poifs.nio.FileBackedDataSource$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Comment 9 Andreas Beeker 2018-05-24 22:11:54 UTC
For 4.x I'm quite sure that we don't go for full Jigsaw compatibility - but automatic modules aren't that bad in my point of view.

Regarding the first issue of the original op, this will be covered in #62355
and see also my sketchy plan for Jigsaw compatibility there.
Comment 10 Dominik Stadler 2019-03-02 09:18:36 UTC
FYI, jaxb in current versions also causes some warnings about reflective access, we cannot fix those until jaxb >= 2.4.0 is available, see https://stackoverflow.com/a/50251510/411846 for details, you can set a system property "com.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize" to avoid this warning
Comment 11 Dominik Stadler 2019-03-02 09:21:39 UTC
In r1854630 we now try to avoid this illegal reflective access via an approach taken from Apache Hadoop, it should now work for Java <=8 and Java >=9 via two different approaches.

As this bug is mostly about the illegal reflective access, I think it can be closed. Having conflicting packages was already solved in a different issue. 
Providing full modules will need to be handled elsewhere.